2024-05-12T03:36:27.724 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-12T03:36:27.724 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --name teuthology-2024-05-01_22:24:03-fs-reef-distro-default-smithi --owner scheduled_teuthology@teuthology --description fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} --archive /home/teuthworker/archive/teuthology-2024-05-01_22:24:03-fs-reef-distro-default-smithi/7685017 -- /home/teuthworker/archive/teuthology-2024-05-01_22:24:03-fs-reef-distro-default-smithi/7685017/orig.config.yaml 2024-05-12T03:36:27.777 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-12T03:36:27.870 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-05-01_22:24:03-fs-reef-distro-default-smithi/7685017 branch: reef description: fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7685017' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi meta: - desc: 'setup ceph/quincy ' name: teuthology-2024-05-01_22:24:03-fs-reef-distro-default-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 8.stream overrides: admin_socket: branch: reef ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - OSD_DOWN - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: 58249552a22f437b8cb49ed6b1068e1158bed605 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 58249552a22f437b8cb49ed6b1068e1158bed605 kclient: syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: reef sha1: fccfb453906af8847786c471919d347112ea0360 owner: scheduled_teuthology@teuthology priority: 930 repo: https://git.ceph.com/ceph.git roles: - - host.a - client.0 - osd.0 - osd.1 - osd.2 - - host.b - client.1 - osd.3 - osd.4 - osd.5 seed: 7998 sha1: 58249552a22f437b8cb49ed6b1068e1158bed605 sleep_before_teardown: 0 subset: 407/512 suite: fs suite_branch: reef suite_path: /home/teuthworker/src/git.ceph.com_ceph_fccfb453906af8847786c471919d347112ea0360/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: fccfb453906af8847786c471919d347112ea0360 targets: smithi032.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM6+E1ZOfltaZUZhW75ipR9UF/xLND7F/dzURXMhLIg1efwt/pRqs9P4GPtOvzZ/Yz00BHfZ6Yqbhv+gU8oiDLY= smithi033.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLXj6EiPmflYQsMVB04Ib6euDtM77z2Z1HSEetzuh7uSO7ixFy8H90H1oj6fJRGaXnj4+bn5Rt0OJ5WW6HK9RNQ= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: branch: quincy exclude_packages: - ceph-volume - print: '**** done install task...' - cephadm: cephadm_branch: quincy cephadm_git_url: https://github.com/ceph/ceph conf: osd: osd_class_default_list: '*' osd_class_load_list: '*' image: quay.ceph.io/ceph-ci/ceph:quincy roleless: true - print: '**** done end installing quincy cephadm ...' - cephadm.shell: host.a: - ceph config set mgr mgr/cephadm/use_repo_digest true --force - print: '**** done cephadm.shell ceph config set mgr...' - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: host.a: - ceph fs volume create cephfs --placement=4 - ceph fs dump - cephadm.shell: host.a: - ceph fs set cephfs max_mds 1 - cephadm.shell: host.a: - ceph fs set cephfs allow_standby_replay true - cephadm.shell: host.a: - ceph fs set cephfs inline_data true --yes-i-really-really-mean-it - cephadm.shell: host.a: - ceph fs dump - ceph --format=json fs dump | jq -e ".filesystems | length == 1" - while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done - fs.pre_upgrade_save: null - ceph-fuse: null - print: '**** done client' - parallel: - upgrade-tasks - workload-tasks - cephadm.shell: host.a: - ceph fs dump - fs.post_upgrade_checks: null teuthology: fragments_dropped: - /home/teuthology/src/git.ceph.com_ceph_reef/qa/suites/fs/upgrade/mds_upgrade_sequence/tasks/3-upgrade-mgr-staggered.yaml meta: {} postmerge: [] variables: fail_fs: false teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-01_22:24:03 tube: smithi upgrade-tasks: sequential: - cephadm.shell: env: - sha1 host.a: - ceph config set mgr mgr/orchestrator/fail_fs false || true - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 host.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done - ceph orch ps - ceph orch upgrade status - ceph health detail - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 workload-tasks: sequential: - workunit: clients: all: - suites/fsstress.sh 2024-05-12T03:36:27.870 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_fccfb453906af8847786c471919d347112ea0360/qa; will attempt to use it 2024-05-12T03:36:27.871 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_fccfb453906af8847786c471919d347112ea0360/qa/tasks 2024-05-12T03:36:27.872 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-12T03:36:27.873 INFO:teuthology.task.internal:Checking packages... 2024-05-12T03:36:27.897 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '58249552a22f437b8cb49ed6b1068e1158bed605' 2024-05-12T03:36:27.897 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-12T03:36:27.897 INFO:teuthology.packaging:ref: None 2024-05-12T03:36:27.897 INFO:teuthology.packaging:tag: None 2024-05-12T03:36:27.897 INFO:teuthology.packaging:branch: reef 2024-05-12T03:36:27.897 INFO:teuthology.packaging:sha1: 58249552a22f437b8cb49ed6b1068e1158bed605 2024-05-12T03:36:27.897 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=reef 2024-05-12T03:36:28.115 INFO:teuthology.task.internal:Found packages for ceph version 18.2.2-1335.gb806bdbd 2024-05-12T03:36:28.117 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-12T03:36:28.124 INFO:teuthology.task.internal:no buildpackages task found 2024-05-12T03:36:28.124 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-12T03:36:28.159 INFO:teuthology.task.internal:Saving configuration 2024-05-12T03:36:28.183 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-12T03:36:28.189 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-12T03:36:28.215 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi032.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-05-01_22:24:03-fs-reef-distro-default-smithi/7685017', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-12 03:31:48.055764', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM6+E1ZOfltaZUZhW75ipR9UF/xLND7F/dzURXMhLIg1efwt/pRqs9P4GPtOvzZ/Yz00BHfZ6Yqbhv+gU8oiDLY='} 2024-05-12T03:36:28.240 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi033.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-05-01_22:24:03-fs-reef-distro-default-smithi/7685017', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-12 03:31:48.053853', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLXj6EiPmflYQsMVB04Ib6euDtM77z2Z1HSEetzuh7uSO7ixFy8H90H1oj6fJRGaXnj4+bn5Rt0OJ5WW6HK9RNQ='} 2024-05-12T03:36:28.240 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-12T03:36:28.246 INFO:teuthology.task.internal:roles: ubuntu@smithi032.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2024-05-12T03:36:28.246 INFO:teuthology.task.internal:roles: ubuntu@smithi033.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2024-05-12T03:36:28.246 INFO:teuthology.run_tasks:Running task console_log... 2024-05-12T03:36:28.332 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f323876d160>, signals=[15]) 2024-05-12T03:36:28.333 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-12T03:36:28.340 INFO:teuthology.task.internal:Opening connections... 2024-05-12T03:36:28.340 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi032.front.sepia.ceph.com 2024-05-12T03:36:28.342 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-12T03:36:28.419 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi033.front.sepia.ceph.com 2024-05-12T03:36:28.420 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-12T03:36:28.506 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-12T03:36:28.513 DEBUG:teuthology.orchestra.run.smithi032:> uname -m 2024-05-12T03:36:28.536 INFO:teuthology.orchestra.run.smithi032.stdout:x86_64 2024-05-12T03:36:28.537 DEBUG:teuthology.orchestra.run.smithi032:> cat /etc/os-release 2024-05-12T03:36:28.597 INFO:teuthology.orchestra.run.smithi032.stdout:NAME="CentOS Stream" 2024-05-12T03:36:28.597 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION="8" 2024-05-12T03:36:28.597 INFO:teuthology.orchestra.run.smithi032.stdout:ID="centos" 2024-05-12T03:36:28.598 INFO:teuthology.orchestra.run.smithi032.stdout:ID_LIKE="rhel fedora" 2024-05-12T03:36:28.598 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION_ID="8" 2024-05-12T03:36:28.598 INFO:teuthology.orchestra.run.smithi032.stdout:PLATFORM_ID="platform:el8" 2024-05-12T03:36:28.598 INFO:teuthology.orchestra.run.smithi032.stdout:PRETTY_NAME="CentOS Stream 8" 2024-05-12T03:36:28.598 INFO:teuthology.orchestra.run.smithi032.stdout:ANSI_COLOR="0;31" 2024-05-12T03:36:28.598 INFO:teuthology.orchestra.run.smithi032.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-05-12T03:36:28.598 INFO:teuthology.orchestra.run.smithi032.stdout:HOME_URL="https://centos.org/" 2024-05-12T03:36:28.598 INFO:teuthology.orchestra.run.smithi032.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-12T03:36:28.598 INFO:teuthology.orchestra.run.smithi032.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-05-12T03:36:28.598 INFO:teuthology.orchestra.run.smithi032.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-12T03:36:28.599 INFO:teuthology.lock.ops:Updating smithi032.front.sepia.ceph.com on lock server 2024-05-12T03:36:28.621 DEBUG:teuthology.orchestra.run.smithi033:> uname -m 2024-05-12T03:36:28.645 INFO:teuthology.orchestra.run.smithi033.stdout:x86_64 2024-05-12T03:36:28.645 DEBUG:teuthology.orchestra.run.smithi033:> cat /etc/os-release 2024-05-12T03:36:28.707 INFO:teuthology.orchestra.run.smithi033.stdout:NAME="CentOS Stream" 2024-05-12T03:36:28.707 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION="8" 2024-05-12T03:36:28.707 INFO:teuthology.orchestra.run.smithi033.stdout:ID="centos" 2024-05-12T03:36:28.707 INFO:teuthology.orchestra.run.smithi033.stdout:ID_LIKE="rhel fedora" 2024-05-12T03:36:28.707 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION_ID="8" 2024-05-12T03:36:28.708 INFO:teuthology.orchestra.run.smithi033.stdout:PLATFORM_ID="platform:el8" 2024-05-12T03:36:28.708 INFO:teuthology.orchestra.run.smithi033.stdout:PRETTY_NAME="CentOS Stream 8" 2024-05-12T03:36:28.708 INFO:teuthology.orchestra.run.smithi033.stdout:ANSI_COLOR="0;31" 2024-05-12T03:36:28.708 INFO:teuthology.orchestra.run.smithi033.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-05-12T03:36:28.708 INFO:teuthology.orchestra.run.smithi033.stdout:HOME_URL="https://centos.org/" 2024-05-12T03:36:28.708 INFO:teuthology.orchestra.run.smithi033.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-12T03:36:28.708 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-05-12T03:36:28.708 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-12T03:36:28.708 INFO:teuthology.lock.ops:Updating smithi033.front.sepia.ceph.com on lock server 2024-05-12T03:36:28.729 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-12T03:36:28.739 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-12T03:36:28.759 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-12T03:36:28.759 DEBUG:teuthology.orchestra.run.smithi032:> test '!' -e /home/ubuntu/cephtest 2024-05-12T03:36:28.762 DEBUG:teuthology.orchestra.run.smithi033:> test '!' -e /home/ubuntu/cephtest 2024-05-12T03:36:28.783 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-12T03:36:28.788 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-12T03:36:28.789 DEBUG:teuthology.orchestra.run.smithi032:> test -z $(ls -A /var/lib/ceph) 2024-05-12T03:36:28.824 DEBUG:teuthology.orchestra.run.smithi033:> test -z $(ls -A /var/lib/ceph) 2024-05-12T03:36:28.855 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-12T03:36:28.915 INFO:teuthology.run_tasks:Running task kernel... 2024-05-12T03:36:28.930 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-05-12T03:36:28.931 INFO:teuthology.task.kernel:config {'host.a': {'kdb': 1, 'sha1': 'distro'}, 'host.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-05-12T03:36:28.931 DEBUG:teuthology.orchestra.run.smithi032:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-12T03:36:28.932 DEBUG:teuthology.orchestra.run.smithi033:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-12T03:36:28.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-12T03:36:28.953 DEBUG:teuthology.orchestra.run.smithi032:> uname -r 2024-05-12T03:36:28.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-12T03:36:28.954 DEBUG:teuthology.orchestra.run.smithi033:> uname -r 2024-05-12T03:36:29.015 INFO:teuthology.orchestra.run.smithi032.stdout:4.18.0-489.el8.x86_64 2024-05-12T03:36:29.015 INFO:teuthology.task.kernel:Running kernel on smithi032: 4.18.0-489.el8.x86_64 2024-05-12T03:36:29.016 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum install -y kernel 2024-05-12T03:36:29.016 INFO:teuthology.orchestra.run.smithi033.stdout:4.18.0-489.el8.x86_64 2024-05-12T03:36:29.017 INFO:teuthology.task.kernel:Running kernel on smithi033: 4.18.0-489.el8.x86_64 2024-05-12T03:36:29.017 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum install -y kernel 2024-05-12T03:36:33.259 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 8 - AppStream 17 MB/s | 28 MB 00:01 2024-05-12T03:36:33.378 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 8 - AppStream 16 MB/s | 28 MB 00:01 2024-05-12T03:36:40.061 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 8 - BaseOS 6.8 MB/s | 10 MB 00:01 2024-05-12T03:36:40.362 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 8 - BaseOS 6.4 MB/s | 10 MB 00:01 2024-05-12T03:36:42.402 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 8 - Extras 32 kB/s | 18 kB 00:00 2024-05-12T03:36:42.479 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 8 - Extras 47 kB/s | 18 kB 00:00 2024-05-12T03:36:42.827 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 8 - Extras common packages 25 kB/s | 7.7 kB 00:00 2024-05-12T03:36:42.904 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 8 - Extras common packages 25 kB/s | 7.7 kB 00:00 2024-05-12T03:36:43.062 INFO:teuthology.orchestra.run.smithi032.stdout:Copr repo for python3-asyncssh owned by ceph 25 kB/s | 3.5 kB 00:00 2024-05-12T03:36:43.159 INFO:teuthology.orchestra.run.smithi033.stdout:Copr repo for python3-asyncssh owned by ceph 22 kB/s | 3.5 kB 00:00 2024-05-12T03:36:43.975 INFO:teuthology.orchestra.run.smithi033.stdout:Extra Packages for Enterprise Linux 19 MB/s | 14 MB 00:00 2024-05-12T03:36:44.690 INFO:teuthology.orchestra.run.smithi032.stdout:Extra Packages for Enterprise Linux 8.8 MB/s | 14 MB 00:01 2024-05-12T03:36:48.473 INFO:teuthology.orchestra.run.smithi033.stdout:lab-extras 437 kB/s | 24 kB 00:00 2024-05-12T03:36:49.138 INFO:teuthology.orchestra.run.smithi032.stdout:lab-extras 323 kB/s | 24 kB 00:00 2024-05-12T03:36:51.803 INFO:teuthology.orchestra.run.smithi033.stdout:Package kernel-4.18.0-348.el8.x86_64 is already installed. 2024-05-12T03:36:51.803 INFO:teuthology.orchestra.run.smithi033.stdout:Package kernel-4.18.0-489.el8.x86_64 is already installed. 2024-05-12T03:36:51.923 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-05-12T03:36:51.924 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-05-12T03:36:51.924 INFO:teuthology.orchestra.run.smithi033.stdout: Package Architecture Version Repository Size 2024-05-12T03:36:51.924 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-05-12T03:36:51.924 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2024-05-12T03:36:51.924 INFO:teuthology.orchestra.run.smithi033.stdout: kernel x86_64 4.18.0-552.1.1.el8 baseos 10 M 2024-05-12T03:36:51.924 INFO:teuthology.orchestra.run.smithi033.stdout:Installing dependencies: 2024-05-12T03:36:51.924 INFO:teuthology.orchestra.run.smithi033.stdout: kernel-core x86_64 4.18.0-552.1.1.el8 baseos 44 M 2024-05-12T03:36:51.925 INFO:teuthology.orchestra.run.smithi033.stdout: kernel-modules x86_64 4.18.0-552.1.1.el8 baseos 36 M 2024-05-12T03:36:51.925 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:36:51.925 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2024-05-12T03:36:51.925 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-05-12T03:36:51.925 INFO:teuthology.orchestra.run.smithi033.stdout:Install 3 Packages 2024-05-12T03:36:51.925 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:36:51.925 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 90 M 2024-05-12T03:36:51.926 INFO:teuthology.orchestra.run.smithi033.stdout:Installed size: 96 M 2024-05-12T03:36:51.926 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading Packages: 2024-05-12T03:36:52.478 INFO:teuthology.orchestra.run.smithi032.stdout:Package kernel-4.18.0-348.el8.x86_64 is already installed. 2024-05-12T03:36:52.478 INFO:teuthology.orchestra.run.smithi032.stdout:Package kernel-4.18.0-489.el8.x86_64 is already installed. 2024-05-12T03:36:52.583 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-05-12T03:36:52.584 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-05-12T03:36:52.584 INFO:teuthology.orchestra.run.smithi032.stdout: Package Architecture Version Repository Size 2024-05-12T03:36:52.584 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-05-12T03:36:52.584 INFO:teuthology.orchestra.run.smithi032.stdout:Installing: 2024-05-12T03:36:52.585 INFO:teuthology.orchestra.run.smithi032.stdout: kernel x86_64 4.18.0-552.1.1.el8 baseos 10 M 2024-05-12T03:36:52.585 INFO:teuthology.orchestra.run.smithi032.stdout:Installing dependencies: 2024-05-12T03:36:52.585 INFO:teuthology.orchestra.run.smithi032.stdout: kernel-core x86_64 4.18.0-552.1.1.el8 baseos 44 M 2024-05-12T03:36:52.585 INFO:teuthology.orchestra.run.smithi032.stdout: kernel-modules x86_64 4.18.0-552.1.1.el8 baseos 36 M 2024-05-12T03:36:52.585 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:36:52.585 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction Summary 2024-05-12T03:36:52.585 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-05-12T03:36:52.585 INFO:teuthology.orchestra.run.smithi032.stdout:Install 3 Packages 2024-05-12T03:36:52.585 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:36:52.586 INFO:teuthology.orchestra.run.smithi032.stdout:Total download size: 90 M 2024-05-12T03:36:52.586 INFO:teuthology.orchestra.run.smithi032.stdout:Installed size: 96 M 2024-05-12T03:36:52.586 INFO:teuthology.orchestra.run.smithi032.stdout:Downloading Packages: 2024-05-12T03:36:53.399 INFO:teuthology.orchestra.run.smithi033.stdout:(1/3): kernel-4.18.0-552.1.1.el8.x86_64.rpm 8.1 MB/s | 10 MB 00:01 2024-05-12T03:36:54.983 INFO:teuthology.orchestra.run.smithi033.stdout:(2/3): kernel-core-4.18.0-552.1.1.el8.x86_64.rp 15 MB/s | 44 MB 00:02 2024-05-12T03:36:55.775 INFO:teuthology.orchestra.run.smithi033.stdout:(3/3): kernel-modules-4.18.0-552.1.1.el8.x86_64 10 MB/s | 36 MB 00:03 2024-05-12T03:36:55.775 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2024-05-12T03:36:55.775 INFO:teuthology.orchestra.run.smithi033.stdout:Total 24 MB/s | 90 MB 00:03 2024-05-12T03:36:56.494 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2024-05-12T03:36:56.954 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction check succeeded. 2024-05-12T03:36:56.954 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2024-05-12T03:36:58.257 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded. 2024-05-12T03:36:58.284 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2024-05-12T03:37:01.264 INFO:teuthology.orchestra.run.smithi033.stdout: Preparing : 1/1 2024-05-12T03:37:01.825 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : kernel-core-4.18.0-552.1.1.el8.x86_64 1/3 2024-05-12T03:37:02.056 INFO:teuthology.orchestra.run.smithi032.stdout:(1/3): kernel-4.18.0-552.1.1.el8.x86_64.rpm 1.1 MB/s | 10 MB 00:09 2024-05-12T03:37:05.535 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: kernel-core-4.18.0-552.1.1.el8.x86_64 1/3 2024-05-12T03:37:05.924 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : kernel-modules-4.18.0-552.1.1.el8.x86_64 2/3 2024-05-12T03:37:07.039 INFO:teuthology.orchestra.run.smithi032.stdout:(2/3): kernel-modules-4.18.0-552.1.1.el8.x86_64 2.6 MB/s | 36 MB 00:14 2024-05-12T03:37:07.939 INFO:teuthology.orchestra.run.smithi032.stdout:(3/3): kernel-core-4.18.0-552.1.1.el8.x86_64.rp 2.9 MB/s | 44 MB 00:15 2024-05-12T03:37:07.939 INFO:teuthology.orchestra.run.smithi032.stdout:-------------------------------------------------------------------------------- 2024-05-12T03:37:07.940 INFO:teuthology.orchestra.run.smithi032.stdout:Total 5.9 MB/s | 90 MB 00:15 2024-05-12T03:37:08.659 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction check 2024-05-12T03:37:09.124 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction check succeeded. 2024-05-12T03:37:09.124 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction test 2024-05-12T03:37:09.608 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: kernel-modules-4.18.0-552.1.1.el8.x86_64 2/3 2024-05-12T03:37:09.885 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : kernel-4.18.0-552.1.1.el8.x86_64 3/3 2024-05-12T03:37:10.457 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction test succeeded. 2024-05-12T03:37:10.490 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction 2024-05-12T03:37:13.463 INFO:teuthology.orchestra.run.smithi032.stdout: Preparing : 1/1 2024-05-12T03:37:14.226 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : kernel-core-4.18.0-552.1.1.el8.x86_64 1/3 2024-05-12T03:37:17.983 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: kernel-core-4.18.0-552.1.1.el8.x86_64 1/3 2024-05-12T03:37:18.367 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : kernel-modules-4.18.0-552.1.1.el8.x86_64 2/3 2024-05-12T03:37:22.056 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: kernel-modules-4.18.0-552.1.1.el8.x86_64 2/3 2024-05-12T03:37:22.378 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : kernel-4.18.0-552.1.1.el8.x86_64 3/3 2024-05-12T03:37:33.369 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: kernel-core-4.18.0-552.1.1.el8.x86_64 3/3 2024-05-12T03:37:33.395 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: kernel-modules-4.18.0-552.1.1.el8.x86_64 3/3 2024-05-12T03:37:33.763 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: kernel-4.18.0-552.1.1.el8.x86_64 3/3 2024-05-12T03:37:33.763 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : kernel-4.18.0-552.1.1.el8.x86_64 1/3 2024-05-12T03:37:33.763 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : kernel-core-4.18.0-552.1.1.el8.x86_64 2/3 2024-05-12T03:37:34.304 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : kernel-modules-4.18.0-552.1.1.el8.x86_64 3/3 2024-05-12T03:37:34.305 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:37:34.305 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2024-05-12T03:37:34.305 INFO:teuthology.orchestra.run.smithi033.stdout: kernel-4.18.0-552.1.1.el8.x86_64 2024-05-12T03:37:34.305 INFO:teuthology.orchestra.run.smithi033.stdout: kernel-core-4.18.0-552.1.1.el8.x86_64 2024-05-12T03:37:34.305 INFO:teuthology.orchestra.run.smithi033.stdout: kernel-modules-4.18.0-552.1.1.el8.x86_64 2024-05-12T03:37:34.305 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:37:34.305 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-05-12T03:37:34.474 DEBUG:teuthology.orchestra.run.smithi033:> rpm -q kernel | sort -rV | head -n 1 2024-05-12T03:37:34.743 INFO:teuthology.orchestra.run.smithi033.stdout:kernel-4.18.0-552.1.1.el8.x86_64 2024-05-12T03:37:34.743 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-552.1.1.el8.x86_64 2024-05-12T03:37:34.743 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-489.el8.x86_64 Expected: 4.18.0-552.1.1.el8.x86_64 2024-05-12T03:37:34.743 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-12T03:37:34.743 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-12T03:37:34.743 INFO:teuthology.task.kernel:Installing distro kernel on host.b... 2024-05-12T03:37:34.744 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi033.front.sepia.ceph.com, path=None, version=distro) 2024-05-12T03:37:34.744 DEBUG:teuthology.orchestra.run.smithi033:> rpm -q kernel | sort -rV | head -n 1 2024-05-12T03:37:34.966 INFO:teuthology.orchestra.run.smithi033.stdout:kernel-4.18.0-552.1.1.el8.x86_64 2024-05-12T03:37:34.966 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-552.1.1.el8.x86_64 2024-05-12T03:37:34.966 DEBUG:teuthology.orchestra.run.smithi033:> sudo rpm -qi grub2-tools 2024-05-12T03:37:35.000 INFO:teuthology.orchestra.run.smithi033.stdout:Name : grub2-tools 2024-05-12T03:37:35.001 INFO:teuthology.orchestra.run.smithi033.stdout:Epoch : 1 2024-05-12T03:37:35.001 INFO:teuthology.orchestra.run.smithi033.stdout:Version : 2.02 2024-05-12T03:37:35.001 INFO:teuthology.orchestra.run.smithi033.stdout:Release : 129.el8 2024-05-12T03:37:35.001 INFO:teuthology.orchestra.run.smithi033.stdout:Architecture: x86_64 2024-05-12T03:37:35.001 INFO:teuthology.orchestra.run.smithi033.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2024-05-12T03:37:35.001 INFO:teuthology.orchestra.run.smithi033.stdout:Group : System Environment/Base 2024-05-12T03:37:35.001 INFO:teuthology.orchestra.run.smithi033.stdout:Size : 9265586 2024-05-12T03:37:35.001 INFO:teuthology.orchestra.run.smithi033.stdout:License : GPLv3+ 2024-05-12T03:37:35.001 INFO:teuthology.orchestra.run.smithi033.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2024-05-12T03:37:35.001 INFO:teuthology.orchestra.run.smithi033.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2024-05-12T03:37:35.001 INFO:teuthology.orchestra.run.smithi033.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2024-05-12T03:37:35.001 INFO:teuthology.orchestra.run.smithi033.stdout:Build Host : kbuilder.bsys.centos.org 2024-05-12T03:37:35.001 INFO:teuthology.orchestra.run.smithi033.stdout:Relocations : (not relocatable) 2024-05-12T03:37:35.001 INFO:teuthology.orchestra.run.smithi033.stdout:Packager : CentOS BuildSystem 2024-05-12T03:37:35.002 INFO:teuthology.orchestra.run.smithi033.stdout:Vendor : CentOS 2024-05-12T03:37:35.002 INFO:teuthology.orchestra.run.smithi033.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-12T03:37:35.002 INFO:teuthology.orchestra.run.smithi033.stdout:Summary : Support tools for GRUB. 2024-05-12T03:37:35.002 INFO:teuthology.orchestra.run.smithi033.stdout:Description : 2024-05-12T03:37:35.002 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:37:35.002 INFO:teuthology.orchestra.run.smithi033.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-12T03:37:35.002 INFO:teuthology.orchestra.run.smithi033.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-12T03:37:35.002 INFO:teuthology.orchestra.run.smithi033.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-12T03:37:35.002 INFO:teuthology.orchestra.run.smithi033.stdout:hardware devices. 2024-05-12T03:37:35.002 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:37:35.002 INFO:teuthology.orchestra.run.smithi033.stdout:This subpackage provides tools for support of all platforms. 2024-05-12T03:37:35.003 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-12T03:37:35.003 INFO:teuthology.task.kernel:Updating grub on smithi033 to boot 4.18.0-552.1.1.el8.x86_64 2024-05-12T03:37:35.003 DEBUG:teuthology.orchestra.run.smithi033:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-12T03:37:36.339 INFO:teuthology.orchestra.run.smithi033.stderr:Generating grub configuration file ... 2024-05-12T03:37:38.525 INFO:teuthology.orchestra.run.smithi033.stderr:done 2024-05-12T03:37:38.527 DEBUG:teuthology.orchestra.run.smithi033:> mktemp 2024-05-12T03:37:38.559 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/tmp.2cLQbEvfv1 2024-05-12T03:37:38.560 DEBUG:teuthology.orchestra.run.smithi033:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.2cLQbEvfv1 2024-05-12T03:37:38.633 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 0666 /tmp/tmp.2cLQbEvfv1 2024-05-12T03:37:38.791 DEBUG:teuthology.orchestra.remote:smithi033:/tmp/tmp.2cLQbEvfv1 is 6KB 2024-05-12T03:37:38.840 DEBUG:teuthology.orchestra.run.smithi033:> rm -fr /tmp/tmp.2cLQbEvfv1 2024-05-12T03:37:38.857 DEBUG:teuthology.orchestra.run.smithi033:> sudo /bin/ls /boot/loader/entries || true 2024-05-12T03:37:38.924 INFO:teuthology.orchestra.run.smithi033.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2024-05-12T03:37:38.924 INFO:teuthology.orchestra.run.smithi033.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2024-05-12T03:37:38.924 INFO:teuthology.orchestra.run.smithi033.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2024-05-12T03:37:38.924 INFO:teuthology.orchestra.run.smithi033.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-552.1.1.el8.x86_64.conf 2024-05-12T03:37:38.925 DEBUG:teuthology.orchestra.run.smithi033:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-4.18.0-552.1.1.el8.x86_64 2024-05-12T03:37:39.074 DEBUG:teuthology.orchestra.run.smithi033:> sudo shutdown -r now 2024-05-12T03:37:45.972 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: kernel-core-4.18.0-552.1.1.el8.x86_64 3/3 2024-05-12T03:37:45.997 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: kernel-modules-4.18.0-552.1.1.el8.x86_64 3/3 2024-05-12T03:37:46.743 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: kernel-4.18.0-552.1.1.el8.x86_64 3/3 2024-05-12T03:37:46.743 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : kernel-4.18.0-552.1.1.el8.x86_64 1/3 2024-05-12T03:37:46.743 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : kernel-core-4.18.0-552.1.1.el8.x86_64 2/3 2024-05-12T03:37:47.292 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : kernel-modules-4.18.0-552.1.1.el8.x86_64 3/3 2024-05-12T03:37:47.292 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:37:47.292 INFO:teuthology.orchestra.run.smithi032.stdout:Installed: 2024-05-12T03:37:47.292 INFO:teuthology.orchestra.run.smithi032.stdout: kernel-4.18.0-552.1.1.el8.x86_64 2024-05-12T03:37:47.292 INFO:teuthology.orchestra.run.smithi032.stdout: kernel-core-4.18.0-552.1.1.el8.x86_64 2024-05-12T03:37:47.292 INFO:teuthology.orchestra.run.smithi032.stdout: kernel-modules-4.18.0-552.1.1.el8.x86_64 2024-05-12T03:37:47.293 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:37:47.293 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2024-05-12T03:37:47.437 DEBUG:teuthology.orchestra.run.smithi032:> rpm -q kernel | sort -rV | head -n 1 2024-05-12T03:37:47.663 INFO:teuthology.orchestra.run.smithi032.stdout:kernel-4.18.0-552.1.1.el8.x86_64 2024-05-12T03:37:47.663 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-552.1.1.el8.x86_64 2024-05-12T03:37:47.663 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-489.el8.x86_64 Expected: 4.18.0-552.1.1.el8.x86_64 2024-05-12T03:37:47.663 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-12T03:37:47.663 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-12T03:37:47.664 INFO:teuthology.task.kernel:Installing distro kernel on host.a... 2024-05-12T03:37:47.664 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi032.front.sepia.ceph.com, path=None, version=distro) 2024-05-12T03:37:47.664 DEBUG:teuthology.orchestra.run.smithi032:> rpm -q kernel | sort -rV | head -n 1 2024-05-12T03:37:47.884 INFO:teuthology.orchestra.run.smithi032.stdout:kernel-4.18.0-552.1.1.el8.x86_64 2024-05-12T03:37:47.885 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-552.1.1.el8.x86_64 2024-05-12T03:37:47.885 DEBUG:teuthology.orchestra.run.smithi032:> sudo rpm -qi grub2-tools 2024-05-12T03:37:47.919 INFO:teuthology.orchestra.run.smithi032.stdout:Name : grub2-tools 2024-05-12T03:37:47.920 INFO:teuthology.orchestra.run.smithi032.stdout:Epoch : 1 2024-05-12T03:37:47.920 INFO:teuthology.orchestra.run.smithi032.stdout:Version : 2.02 2024-05-12T03:37:47.920 INFO:teuthology.orchestra.run.smithi032.stdout:Release : 129.el8 2024-05-12T03:37:47.920 INFO:teuthology.orchestra.run.smithi032.stdout:Architecture: x86_64 2024-05-12T03:37:47.920 INFO:teuthology.orchestra.run.smithi032.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2024-05-12T03:37:47.920 INFO:teuthology.orchestra.run.smithi032.stdout:Group : System Environment/Base 2024-05-12T03:37:47.920 INFO:teuthology.orchestra.run.smithi032.stdout:Size : 9265586 2024-05-12T03:37:47.920 INFO:teuthology.orchestra.run.smithi032.stdout:License : GPLv3+ 2024-05-12T03:37:47.920 INFO:teuthology.orchestra.run.smithi032.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2024-05-12T03:37:47.920 INFO:teuthology.orchestra.run.smithi032.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2024-05-12T03:37:47.920 INFO:teuthology.orchestra.run.smithi032.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2024-05-12T03:37:47.920 INFO:teuthology.orchestra.run.smithi032.stdout:Build Host : kbuilder.bsys.centos.org 2024-05-12T03:37:47.920 INFO:teuthology.orchestra.run.smithi032.stdout:Relocations : (not relocatable) 2024-05-12T03:37:47.921 INFO:teuthology.orchestra.run.smithi032.stdout:Packager : CentOS BuildSystem 2024-05-12T03:37:47.921 INFO:teuthology.orchestra.run.smithi032.stdout:Vendor : CentOS 2024-05-12T03:37:47.921 INFO:teuthology.orchestra.run.smithi032.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-12T03:37:47.921 INFO:teuthology.orchestra.run.smithi032.stdout:Summary : Support tools for GRUB. 2024-05-12T03:37:47.921 INFO:teuthology.orchestra.run.smithi032.stdout:Description : 2024-05-12T03:37:47.921 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:37:47.921 INFO:teuthology.orchestra.run.smithi032.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-12T03:37:47.921 INFO:teuthology.orchestra.run.smithi032.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-12T03:37:47.921 INFO:teuthology.orchestra.run.smithi032.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-12T03:37:47.921 INFO:teuthology.orchestra.run.smithi032.stdout:hardware devices. 2024-05-12T03:37:47.921 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:37:47.921 INFO:teuthology.orchestra.run.smithi032.stdout:This subpackage provides tools for support of all platforms. 2024-05-12T03:37:47.922 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-12T03:37:47.922 INFO:teuthology.task.kernel:Updating grub on smithi032 to boot 4.18.0-552.1.1.el8.x86_64 2024-05-12T03:37:47.922 DEBUG:teuthology.orchestra.run.smithi032:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-12T03:37:49.352 INFO:teuthology.orchestra.run.smithi032.stderr:Generating grub configuration file ... 2024-05-12T03:37:51.489 INFO:teuthology.orchestra.run.smithi032.stderr:done 2024-05-12T03:37:51.491 DEBUG:teuthology.orchestra.run.smithi032:> mktemp 2024-05-12T03:37:51.523 INFO:teuthology.orchestra.run.smithi032.stdout:/tmp/tmp.6LDcDODtmw 2024-05-12T03:37:51.523 DEBUG:teuthology.orchestra.run.smithi032:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.6LDcDODtmw 2024-05-12T03:37:51.595 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0666 /tmp/tmp.6LDcDODtmw 2024-05-12T03:37:51.738 DEBUG:teuthology.orchestra.remote:smithi032:/tmp/tmp.6LDcDODtmw is 6KB 2024-05-12T03:37:51.786 DEBUG:teuthology.orchestra.run.smithi032:> rm -fr /tmp/tmp.6LDcDODtmw 2024-05-12T03:37:51.804 DEBUG:teuthology.orchestra.run.smithi032:> sudo /bin/ls /boot/loader/entries || true 2024-05-12T03:37:51.880 INFO:teuthology.orchestra.run.smithi032.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2024-05-12T03:37:51.881 INFO:teuthology.orchestra.run.smithi032.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2024-05-12T03:37:51.881 INFO:teuthology.orchestra.run.smithi032.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2024-05-12T03:37:51.881 INFO:teuthology.orchestra.run.smithi032.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-552.1.1.el8.x86_64.conf 2024-05-12T03:37:51.882 DEBUG:teuthology.orchestra.run.smithi032:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-4.18.0-552.1.1.el8.x86_64 2024-05-12T03:37:52.066 DEBUG:teuthology.orchestra.run.smithi032:> sudo shutdown -r now 2024-05-12T03:38:09.093 INFO:teuthology.task.kernel:Checking client host.b for new kernel version... 2024-05-12T03:38:09.093 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-05-12T03:38:09.094 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-12T03:38:22.082 INFO:teuthology.task.kernel:Checking client host.a for new kernel version... 2024-05-12T03:38:22.083 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2024-05-12T03:38:22.083 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-12T03:39:09.138 DEBUG:teuthology.orchestra.remote:timed out 2024-05-12T03:39:18.147 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-05-12T03:39:18.148 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-12T03:39:18.485 DEBUG:teuthology.orchestra.run.smithi033:> true 2024-05-12T03:39:18.795 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-05-12T03:39:18.795 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "4.18.0-552.1.1.el8.x86_64"... 2024-05-12T03:39:18.795 DEBUG:teuthology.orchestra.run.smithi033:> uname -r 2024-05-12T03:39:18.816 INFO:teuthology.orchestra.run.smithi033.stdout:4.18.0-552.1.1.el8.x86_64 2024-05-12T03:39:18.816 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-552.1.1.el8.x86_64 vs 4.18.0-552.1.1.el8.x86_64 2024-05-12T03:39:18.816 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-12T03:39:18.816 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-12T03:39:19.817 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2024-05-12T03:39:19.818 DEBUG:teuthology.orchestra.run.smithi033:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-12T03:39:19.884 INFO:teuthology.orchestra.run.smithi033.stdout:ttyS1 2024-05-12T03:39:19.901 DEBUG:teuthology.parallel:result is None 2024-05-12T03:39:22.086 DEBUG:teuthology.orchestra.remote:timed out 2024-05-12T03:39:31.087 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2024-05-12T03:39:31.088 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-12T03:39:31.359 DEBUG:teuthology.orchestra.run.smithi032:> true 2024-05-12T03:39:31.643 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi032.front.sepia.ceph.com' 2024-05-12T03:39:31.643 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "4.18.0-552.1.1.el8.x86_64"... 2024-05-12T03:39:31.643 DEBUG:teuthology.orchestra.run.smithi032:> uname -r 2024-05-12T03:39:31.664 INFO:teuthology.orchestra.run.smithi032.stdout:4.18.0-552.1.1.el8.x86_64 2024-05-12T03:39:31.664 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-552.1.1.el8.x86_64 vs 4.18.0-552.1.1.el8.x86_64 2024-05-12T03:39:31.665 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-12T03:39:31.665 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-12T03:39:32.666 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2024-05-12T03:39:32.666 DEBUG:teuthology.orchestra.run.smithi032:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-12T03:39:32.757 INFO:teuthology.orchestra.run.smithi032.stdout:ttyS1 2024-05-12T03:39:32.770 DEBUG:teuthology.parallel:result is None 2024-05-12T03:39:32.771 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-12T03:39:32.778 INFO:teuthology.task.internal:Creating test directory... 2024-05-12T03:39:32.778 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-12T03:39:32.780 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-12T03:39:32.803 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-12T03:39:32.825 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-12T03:39:32.859 INFO:teuthology.task.internal:Creating archive directory... 2024-05-12T03:39:32.860 DEBUG:teuthology.orchestra.run.smithi032:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-12T03:39:32.862 DEBUG:teuthology.orchestra.run.smithi033:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-12T03:39:32.905 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-12T03:39:32.915 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-12T03:39:32.915 DEBUG:teuthology.orchestra.run.smithi032:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-12T03:39:32.943 DEBUG:teuthology.orchestra.run.smithi033:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-12T03:39:32.982 INFO:teuthology.orchestra.run.smithi033.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-12T03:39:32.983 INFO:teuthology.orchestra.run.smithi032.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-12T03:39:32.993 INFO:teuthology.orchestra.run.smithi033.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-12T03:39:32.995 INFO:teuthology.orchestra.run.smithi032.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-12T03:39:32.997 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-12T03:39:33.004 INFO:teuthology.task.internal:Configuring sudo... 2024-05-12T03:39:33.004 DEBUG:teuthology.orchestra.run.smithi032:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-12T03:39:33.040 DEBUG:teuthology.orchestra.run.smithi033:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-12T03:39:33.072 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-12T03:39:33.085 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-12T03:39:33.085 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-12T03:39:33.113 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-12T03:39:33.136 DEBUG:teuthology.orchestra.run.smithi032:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-12T03:39:33.220 DEBUG:teuthology.orchestra.run.smithi032:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-12T03:39:33.275 DEBUG:teuthology.orchestra.run.smithi032:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-12T03:39:33.363 DEBUG:teuthology.orchestra.run.smithi032:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-12T03:39:33.404 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-05-12T03:39:33.404 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-12T03:39:33.475 DEBUG:teuthology.orchestra.run.smithi033:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-12T03:39:33.524 DEBUG:teuthology.orchestra.run.smithi033:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-12T03:39:33.581 DEBUG:teuthology.orchestra.run.smithi033:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-12T03:39:33.670 DEBUG:teuthology.orchestra.run.smithi033:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-12T03:39:33.715 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-12T03:39:33.715 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-12T03:39:33.785 DEBUG:teuthology.orchestra.run.smithi032:> sudo service rsyslog restart 2024-05-12T03:39:33.787 DEBUG:teuthology.orchestra.run.smithi033:> sudo service rsyslog restart 2024-05-12T03:39:33.853 INFO:teuthology.orchestra.run.smithi032.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-12T03:39:33.906 INFO:teuthology.orchestra.run.smithi033.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-12T03:39:34.326 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-12T03:39:34.334 INFO:teuthology.task.internal:Starting timer... 2024-05-12T03:39:34.334 INFO:teuthology.run_tasks:Running task pcp... 2024-05-12T03:39:34.387 INFO:teuthology.run_tasks:Running task selinux... 2024-05-12T03:39:34.428 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-05-12T03:39:34.428 DEBUG:teuthology.orchestra.run.smithi032:> sudo service auditd rotate 2024-05-12T03:39:34.497 INFO:teuthology.orchestra.run.smithi032.stdout:Rotating logs: 2024-05-12T03:39:34.499 DEBUG:teuthology.orchestra.run.smithi033:> sudo service auditd rotate 2024-05-12T03:39:34.599 INFO:teuthology.orchestra.run.smithi033.stdout:Rotating logs: 2024-05-12T03:39:34.600 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-12T03:39:34.601 DEBUG:teuthology.orchestra.run.smithi032:> /usr/sbin/getenforce 2024-05-12T03:39:34.679 INFO:teuthology.orchestra.run.smithi032.stdout:Permissive 2024-05-12T03:39:34.679 DEBUG:teuthology.orchestra.run.smithi033:> /usr/sbin/getenforce 2024-05-12T03:39:34.754 INFO:teuthology.orchestra.run.smithi033.stdout:Permissive 2024-05-12T03:39:34.755 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi032.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi033.front.sepia.ceph.com': 'permissive'} 2024-05-12T03:39:34.755 DEBUG:teuthology.orchestra.run.smithi032:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-12T03:39:34.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-12T03:39:34.786 DEBUG:teuthology.orchestra.run.smithi033:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-12T03:39:34.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-12T03:39:34.815 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-12T03:39:34.815 DEBUG:teuthology.orchestra.run.smithi032:> sudo /usr/sbin/setenforce permissive 2024-05-12T03:39:34.854 DEBUG:teuthology.orchestra.run.smithi033:> sudo /usr/sbin/setenforce permissive 2024-05-12T03:39:34.881 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-12T03:39:34.891 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-12T03:39:34.908 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-12T03:39:34.909 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi032.front.sepia.ceph.com,smithi033.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-12T03:45:50.947 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi032.front.sepia.ceph.com'), Remote(name='ubuntu@smithi033.front.sepia.ceph.com')] 2024-05-12T03:45:50.948 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2024-05-12T03:45:50.949 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-12T03:45:51.027 DEBUG:teuthology.orchestra.run.smithi032:> true 2024-05-12T03:45:51.055 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi032.front.sepia.ceph.com' 2024-05-12T03:45:51.055 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-05-12T03:45:51.056 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-12T03:45:51.130 DEBUG:teuthology.orchestra.run.smithi033:> true 2024-05-12T03:45:51.204 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-05-12T03:45:51.204 INFO:teuthology.run_tasks:Running task clock... 2024-05-12T03:45:51.214 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-12T03:45:51.215 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-12T03:45:51.215 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-12T03:45:51.217 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-12T03:45:51.218 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-12T03:45:51.249 INFO:teuthology.orchestra.run.smithi032.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-12T03:45:51.264 INFO:teuthology.orchestra.run.smithi032.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-12T03:45:51.282 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-12T03:45:51.298 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-12T03:45:51.332 INFO:teuthology.orchestra.run.smithi032.stderr:sudo: ntpd: command not found 2024-05-12T03:45:51.344 INFO:teuthology.orchestra.run.smithi032.stdout:506 Cannot talk to daemon 2024-05-12T03:45:51.358 INFO:teuthology.orchestra.run.smithi032.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-12T03:45:51.371 INFO:teuthology.orchestra.run.smithi032.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-12T03:45:51.373 INFO:teuthology.orchestra.run.smithi033.stderr:sudo: ntpd: command not found 2024-05-12T03:45:51.385 INFO:teuthology.orchestra.run.smithi033.stdout:506 Cannot talk to daemon 2024-05-12T03:45:51.401 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-12T03:45:51.415 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-12T03:45:51.435 INFO:teuthology.orchestra.run.smithi032.stderr:bash: ntpq: command not found 2024-05-12T03:45:51.438 INFO:teuthology.orchestra.run.smithi032.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-12T03:45:51.439 INFO:teuthology.orchestra.run.smithi032.stdout:=============================================================================== 2024-05-12T03:45:51.439 INFO:teuthology.orchestra.run.smithi032.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-12T03:45:51.439 INFO:teuthology.orchestra.run.smithi032.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-12T03:45:51.439 INFO:teuthology.orchestra.run.smithi032.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-12T03:45:51.439 INFO:teuthology.orchestra.run.smithi032.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-12T03:45:51.492 INFO:teuthology.orchestra.run.smithi033.stderr:bash: ntpq: command not found 2024-05-12T03:45:51.495 INFO:teuthology.orchestra.run.smithi033.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-12T03:45:51.495 INFO:teuthology.orchestra.run.smithi033.stdout:=============================================================================== 2024-05-12T03:45:51.495 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-12T03:45:51.495 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-12T03:45:51.495 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-12T03:45:51.495 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-12T03:45:51.496 INFO:teuthology.run_tasks:Running task pexec... 2024-05-12T03:45:51.505 INFO:teuthology.task.pexec:Executing custom commands... 2024-05-12T03:45:51.506 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi032.front.sepia.ceph.com 2024-05-12T03:45:51.506 DEBUG:teuthology.orchestra.run.smithi032:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-05-12T03:45:51.506 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi033.front.sepia.ceph.com 2024-05-12T03:45:51.506 DEBUG:teuthology.orchestra.run.smithi033:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-05-12T03:45:52.287 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:01:32 ago on Sun 12 May 2024 03:44:20 AM UTC. 2024-05-12T03:45:52.319 INFO:teuthology.orchestra.run.smithi033.stdout:Last metadata expiration check: 0:01:31 ago on Sun 12 May 2024 03:44:21 AM UTC. 2024-05-12T03:45:54.607 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-05-12T03:45:54.608 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-05-12T03:45:54.608 INFO:teuthology.orchestra.run.smithi032.stdout: Package Architecture Version Repository Size 2024-05-12T03:45:54.608 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-05-12T03:45:54.608 INFO:teuthology.orchestra.run.smithi032.stdout:Resetting modules: 2024-05-12T03:45:54.608 INFO:teuthology.orchestra.run.smithi032.stdout: container-tools 2024-05-12T03:45:54.608 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:45:54.609 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction Summary 2024-05-12T03:45:54.609 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-05-12T03:45:54.609 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:45:54.620 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-05-12T03:45:54.621 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-05-12T03:45:54.621 INFO:teuthology.orchestra.run.smithi033.stdout: Package Architecture Version Repository Size 2024-05-12T03:45:54.621 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-05-12T03:45:54.621 INFO:teuthology.orchestra.run.smithi033.stdout:Resetting modules: 2024-05-12T03:45:54.622 INFO:teuthology.orchestra.run.smithi033.stdout: container-tools 2024-05-12T03:45:54.622 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:45:54.622 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2024-05-12T03:45:54.622 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-05-12T03:45:54.622 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:45:54.862 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2024-05-12T03:45:54.895 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-05-12T03:45:55.676 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:01:35 ago on Sun 12 May 2024 03:44:20 AM UTC. 2024-05-12T03:45:55.724 INFO:teuthology.orchestra.run.smithi033.stdout:Last metadata expiration check: 0:01:34 ago on Sun 12 May 2024 03:44:21 AM UTC. 2024-05-12T03:45:57.997 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-05-12T03:45:58.039 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-05-12T03:45:58.039 INFO:teuthology.orchestra.run.smithi032.stdout: Package Arch Version Repository Size 2024-05-12T03:45:58.039 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-05-12T03:45:58.039 INFO:teuthology.orchestra.run.smithi032.stdout:Installing group/module packages: 2024-05-12T03:45:58.039 INFO:teuthology.orchestra.run.smithi032.stdout: aardvark-dns x86_64 2:1.10.0-1.module_el8+871+552be958 appstream 1.0 M 2024-05-12T03:45:58.039 INFO:teuthology.orchestra.run.smithi032.stdout: buildah x86_64 2:1.33.6-2.module_el8+976+11360731 appstream 9.6 M 2024-05-12T03:45:58.039 INFO:teuthology.orchestra.run.smithi032.stdout: cockpit-podman noarch 84.1-1.module_el8+977+8ab54275 appstream 689 k 2024-05-12T03:45:58.039 INFO:teuthology.orchestra.run.smithi032.stdout: crun x86_64 1.14.3-2.module_el8+968+fbb249c7 appstream 258 k 2024-05-12T03:45:58.040 INFO:teuthology.orchestra.run.smithi032.stdout: netavark x86_64 2:1.10.3-1.module_el8+951+32019cde appstream 4.1 M 2024-05-12T03:45:58.040 INFO:teuthology.orchestra.run.smithi032.stdout: python3-podman noarch 4.9.0-1.module_el8+847+7863d4e6 appstream 180 k 2024-05-12T03:45:58.040 INFO:teuthology.orchestra.run.smithi032.stdout: skopeo x86_64 2:1.14.3-0.1.module_el8+885+7da147f3 appstream 8.8 M 2024-05-12T03:45:58.040 INFO:teuthology.orchestra.run.smithi032.stdout: toolbox x86_64 0.0.99.5-2.module_el8+971+3d3df00d appstream 2.5 M 2024-05-12T03:45:58.040 INFO:teuthology.orchestra.run.smithi032.stdout: udica noarch 0.2.6-20.module_el8+804+f131391c appstream 53 k 2024-05-12T03:45:58.040 INFO:teuthology.orchestra.run.smithi032.stdout:Installing dependencies: 2024-05-12T03:45:58.040 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 appstream 25 k 2024-05-12T03:45:58.040 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyxdg noarch 0.25-16.el8 appstream 94 k 2024-05-12T03:45:58.040 INFO:teuthology.orchestra.run.smithi032.stdout: yajl x86_64 2.1.0-12.el8 appstream 41 k 2024-05-12T03:45:58.040 INFO:teuthology.orchestra.run.smithi032.stdout:Installing weak dependencies: 2024-05-12T03:45:58.041 INFO:teuthology.orchestra.run.smithi032.stdout: criu-libs x86_64 3.15-3.module_el8.7.0+1216+b022c01d appstream 38 k 2024-05-12T03:45:58.041 INFO:teuthology.orchestra.run.smithi032.stdout:Installing module profiles: 2024-05-12T03:45:58.041 INFO:teuthology.orchestra.run.smithi032.stdout: container-tools/common 2024-05-12T03:45:58.041 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:45:58.041 INFO:teuthology.orchestra.run.smithi032.stdout:Enabling module streams: 2024-05-12T03:45:58.041 INFO:teuthology.orchestra.run.smithi032.stdout: container-tools rhel8 2024-05-12T03:45:58.041 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:45:58.041 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction Summary 2024-05-12T03:45:58.041 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-05-12T03:45:58.041 INFO:teuthology.orchestra.run.smithi032.stdout:Install 13 Packages 2024-05-12T03:45:58.041 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:45:58.042 INFO:teuthology.orchestra.run.smithi032.stdout:Total download size: 27 M 2024-05-12T03:45:58.042 INFO:teuthology.orchestra.run.smithi032.stdout:Installed size: 91 M 2024-05-12T03:45:58.042 INFO:teuthology.orchestra.run.smithi032.stdout:Downloading Packages: 2024-05-12T03:45:58.044 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-05-12T03:45:58.086 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-05-12T03:45:58.086 INFO:teuthology.orchestra.run.smithi033.stdout: Package Arch Version Repository Size 2024-05-12T03:45:58.086 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-05-12T03:45:58.086 INFO:teuthology.orchestra.run.smithi033.stdout:Installing group/module packages: 2024-05-12T03:45:58.086 INFO:teuthology.orchestra.run.smithi033.stdout: aardvark-dns x86_64 2:1.10.0-1.module_el8+871+552be958 appstream 1.0 M 2024-05-12T03:45:58.086 INFO:teuthology.orchestra.run.smithi033.stdout: buildah x86_64 2:1.33.6-2.module_el8+976+11360731 appstream 9.6 M 2024-05-12T03:45:58.086 INFO:teuthology.orchestra.run.smithi033.stdout: cockpit-podman noarch 84.1-1.module_el8+977+8ab54275 appstream 689 k 2024-05-12T03:45:58.086 INFO:teuthology.orchestra.run.smithi033.stdout: crun x86_64 1.14.3-2.module_el8+968+fbb249c7 appstream 258 k 2024-05-12T03:45:58.086 INFO:teuthology.orchestra.run.smithi033.stdout: netavark x86_64 2:1.10.3-1.module_el8+951+32019cde appstream 4.1 M 2024-05-12T03:45:58.086 INFO:teuthology.orchestra.run.smithi033.stdout: python3-podman noarch 4.9.0-1.module_el8+847+7863d4e6 appstream 180 k 2024-05-12T03:45:58.086 INFO:teuthology.orchestra.run.smithi033.stdout: skopeo x86_64 2:1.14.3-0.1.module_el8+885+7da147f3 appstream 8.8 M 2024-05-12T03:45:58.087 INFO:teuthology.orchestra.run.smithi033.stdout: toolbox x86_64 0.0.99.5-2.module_el8+971+3d3df00d appstream 2.5 M 2024-05-12T03:45:58.087 INFO:teuthology.orchestra.run.smithi033.stdout: udica noarch 0.2.6-20.module_el8+804+f131391c appstream 53 k 2024-05-12T03:45:58.087 INFO:teuthology.orchestra.run.smithi033.stdout:Installing dependencies: 2024-05-12T03:45:58.087 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 appstream 25 k 2024-05-12T03:45:58.087 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyxdg noarch 0.25-16.el8 appstream 94 k 2024-05-12T03:45:58.087 INFO:teuthology.orchestra.run.smithi033.stdout: yajl x86_64 2.1.0-12.el8 appstream 41 k 2024-05-12T03:45:58.087 INFO:teuthology.orchestra.run.smithi033.stdout:Installing weak dependencies: 2024-05-12T03:45:58.087 INFO:teuthology.orchestra.run.smithi033.stdout: criu-libs x86_64 3.15-3.module_el8.7.0+1216+b022c01d appstream 38 k 2024-05-12T03:45:58.087 INFO:teuthology.orchestra.run.smithi033.stdout:Installing module profiles: 2024-05-12T03:45:58.087 INFO:teuthology.orchestra.run.smithi033.stdout: container-tools/common 2024-05-12T03:45:58.087 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:45:58.088 INFO:teuthology.orchestra.run.smithi033.stdout:Enabling module streams: 2024-05-12T03:45:58.088 INFO:teuthology.orchestra.run.smithi033.stdout: container-tools rhel8 2024-05-12T03:45:58.088 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:45:58.088 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2024-05-12T03:45:58.088 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-05-12T03:45:58.088 INFO:teuthology.orchestra.run.smithi033.stdout:Install 13 Packages 2024-05-12T03:45:58.088 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:45:58.089 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 27 M 2024-05-12T03:45:58.089 INFO:teuthology.orchestra.run.smithi033.stdout:Installed size: 91 M 2024-05-12T03:45:58.089 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading Packages: 2024-05-12T03:45:58.361 INFO:teuthology.orchestra.run.smithi032.stdout:(1/13): cockpit-podman-84.1-1.module_el8+977+8a 2.5 MB/s | 689 kB 00:00 2024-05-12T03:45:58.395 INFO:teuthology.orchestra.run.smithi032.stdout:(2/13): aardvark-dns-1.10.0-1.module_el8+871+55 3.4 MB/s | 1.0 MB 00:00 2024-05-12T03:45:58.420 INFO:teuthology.orchestra.run.smithi032.stdout:(3/13): criu-libs-3.15-3.module_el8.7.0+1216+b0 645 kB/s | 38 kB 00:00 2024-05-12T03:45:58.453 INFO:teuthology.orchestra.run.smithi032.stdout:(4/13): crun-1.14.3-2.module_el8+968+fbb249c7.x 4.3 MB/s | 258 kB 00:00 2024-05-12T03:45:58.525 INFO:teuthology.orchestra.run.smithi033.stdout:(1/13): cockpit-podman-84.1-1.module_el8+977+8a 2.7 MB/s | 689 kB 00:00 2024-05-12T03:45:58.529 INFO:teuthology.orchestra.run.smithi032.stdout:(5/13): python3-podman-4.9.0-1.module_el8+847+7 2.3 MB/s | 180 kB 00:00 2024-05-12T03:45:58.558 INFO:teuthology.orchestra.run.smithi033.stdout:(2/13): aardvark-dns-1.10.0-1.module_el8+871+55 3.5 MB/s | 1.0 MB 00:00 2024-05-12T03:45:58.592 INFO:teuthology.orchestra.run.smithi033.stdout:(3/13): criu-libs-3.15-3.module_el8.7.0+1216+b0 566 kB/s | 38 kB 00:00 2024-05-12T03:45:58.596 INFO:teuthology.orchestra.run.smithi032.stdout:(6/13): python3-pytoml-0.1.14-5.git7dea353.el8. 371 kB/s | 25 kB 00:00 2024-05-12T03:45:58.642 INFO:teuthology.orchestra.run.smithi033.stdout:(4/13): crun-1.14.3-2.module_el8+968+fbb249c7.x 3.0 MB/s | 258 kB 00:00 2024-05-12T03:45:58.679 INFO:teuthology.orchestra.run.smithi032.stdout:(7/13): netavark-1.10.3-1.module_el8+951+32019c 16 MB/s | 4.1 MB 00:00 2024-05-12T03:45:58.705 INFO:teuthology.orchestra.run.smithi032.stdout:(8/13): python3-pyxdg-0.25-16.el8.noarch.rpm 864 kB/s | 94 kB 00:00 2024-05-12T03:45:58.718 INFO:teuthology.orchestra.run.smithi033.stdout:(5/13): python3-podman-4.9.0-1.module_el8+847+7 2.3 MB/s | 180 kB 00:00 2024-05-12T03:45:58.776 INFO:teuthology.orchestra.run.smithi033.stdout:(6/13): python3-pytoml-0.1.14-5.git7dea353.el8. 426 kB/s | 25 kB 00:00 2024-05-12T03:45:58.838 INFO:teuthology.orchestra.run.smithi032.stdout:(9/13): buildah-1.33.6-2.module_el8+976+1136073 13 MB/s | 9.6 MB 00:00 2024-05-12T03:45:58.877 INFO:teuthology.orchestra.run.smithi033.stdout:(7/13): python3-pyxdg-0.25-16.el8.noarch.rpm 936 kB/s | 94 kB 00:00 2024-05-12T03:45:58.930 INFO:teuthology.orchestra.run.smithi032.stdout:(10/13): toolbox-0.0.99.5-2.module_el8+971+3d3d 11 MB/s | 2.5 MB 00:00 2024-05-12T03:45:58.947 INFO:teuthology.orchestra.run.smithi032.stdout:(11/13): udica-0.2.6-20.module_el8+804+f131391c 486 kB/s | 53 kB 00:00 2024-05-12T03:45:58.952 INFO:teuthology.orchestra.run.smithi033.stdout:(8/13): netavark-1.10.3-1.module_el8+951+32019c 11 MB/s | 4.1 MB 00:00 2024-05-12T03:45:58.981 INFO:teuthology.orchestra.run.smithi032.stdout:(12/13): yajl-2.1.0-12.el8.x86_64.rpm 818 kB/s | 41 kB 00:00 2024-05-12T03:45:59.131 INFO:teuthology.orchestra.run.smithi032.stdout:(13/13): skopeo-1.14.3-0.1.module_el8+885+7da14 20 MB/s | 8.8 MB 00:00 2024-05-12T03:45:59.131 INFO:teuthology.orchestra.run.smithi032.stdout:-------------------------------------------------------------------------------- 2024-05-12T03:45:59.132 INFO:teuthology.orchestra.run.smithi032.stdout:Total 25 MB/s | 27 MB 00:01 2024-05-12T03:45:59.161 INFO:teuthology.orchestra.run.smithi033.stdout:(9/13): buildah-1.33.6-2.module_el8+976+1136073 11 MB/s | 9.6 MB 00:00 2024-05-12T03:45:59.219 INFO:teuthology.orchestra.run.smithi033.stdout:(10/13): toolbox-0.0.99.5-2.module_el8+971+3d3d 9.5 MB/s | 2.5 MB 00:00 2024-05-12T03:45:59.245 INFO:teuthology.orchestra.run.smithi033.stdout:(11/13): udica-0.2.6-20.module_el8+804+f131391c 630 kB/s | 53 kB 00:00 2024-05-12T03:45:59.278 INFO:teuthology.orchestra.run.smithi033.stdout:(12/13): yajl-2.1.0-12.el8.x86_64.rpm 701 kB/s | 41 kB 00:00 2024-05-12T03:45:59.408 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction check 2024-05-12T03:45:59.431 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction check succeeded. 2024-05-12T03:45:59.431 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction test 2024-05-12T03:45:59.537 INFO:teuthology.orchestra.run.smithi033.stdout:(13/13): skopeo-1.14.3-0.1.module_el8+885+7da14 13 MB/s | 8.8 MB 00:00 2024-05-12T03:45:59.537 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2024-05-12T03:45:59.537 INFO:teuthology.orchestra.run.smithi033.stdout:Total 19 MB/s | 27 MB 00:01 2024-05-12T03:45:59.706 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction test succeeded. 2024-05-12T03:45:59.709 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction 2024-05-12T03:45:59.816 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2024-05-12T03:45:59.839 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction check succeeded. 2024-05-12T03:45:59.839 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2024-05-12T03:46:00.119 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded. 2024-05-12T03:46:00.123 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2024-05-12T03:46:00.350 INFO:teuthology.orchestra.run.smithi032.stdout: Preparing : 1/1 2024-05-12T03:46:00.813 INFO:teuthology.orchestra.run.smithi033.stdout: Preparing : 1/1 2024-05-12T03:46:01.335 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : yajl-2.1.0-12.el8.x86_64 1/13 2024-05-12T03:46:01.623 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 2/13 2024-05-12T03:46:01.783 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/13 2024-05-12T03:46:01.827 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : yajl-2.1.0-12.el8.x86_64 1/13 2024-05-12T03:46:01.996 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 4/13 2024-05-12T03:46:02.066 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 2/13 2024-05-12T03:46:02.250 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_ 5/13 2024-05-12T03:46:02.251 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/13 2024-05-12T03:46:02.468 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 4/13 2024-05-12T03:46:02.693 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_ 5/13 2024-05-12T03:46:02.809 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x8 6/13 2024-05-12T03:46:03.218 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x8 6/13 2024-05-12T03:46:03.855 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : netavark-2:1.10.3-1.module_el8+951+32019cde.x86_64 7/13 2024-05-12T03:46:04.123 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : buildah-2:1.33.6-2.module_el8+976+11360731.x86_64 8/13 2024-05-12T03:46:04.259 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : netavark-2:1.10.3-1.module_el8+951+32019cde.x86_64 7/13 2024-05-12T03:46:04.329 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : crun-1.14.3-2.module_el8+968+fbb249c7.x86_64 9/13 2024-05-12T03:46:04.535 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : buildah-2:1.33.6-2.module_el8+976+11360731.x86_64 8/13 2024-05-12T03:46:04.751 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : crun-1.14.3-2.module_el8+968+fbb249c7.x86_64 9/13 2024-05-12T03:46:04.778 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-podman-4.9.0-1.module_el8+847+7863d4e6.noa 10/13 2024-05-12T03:46:04.998 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : toolbox-0.0.99.5-2.module_el8+971+3d3df00d.x86_64 11/13 2024-05-12T03:46:05.206 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : udica-0.2.6-20.module_el8+804+f131391c.noarch 12/13 2024-05-12T03:46:05.225 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-podman-4.9.0-1.module_el8+847+7863d4e6.noa 10/13 2024-05-12T03:46:05.443 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : toolbox-0.0.99.5-2.module_el8+971+3d3df00d.x86_64 11/13 2024-05-12T03:46:05.462 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : cockpit-podman-84.1-1.module_el8+977+8ab54275.noar 13/13 2024-05-12T03:46:05.660 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : udica-0.2.6-20.module_el8+804+f131391c.noarch 12/13 2024-05-12T03:46:05.884 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : cockpit-podman-84.1-1.module_el8+977+8ab54275.noar 13/13 2024-05-12T03:46:05.903 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: cockpit-podman-84.1-1.module_el8+977+8ab54275.noar 13/13 2024-05-12T03:46:05.903 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x8 1/13 2024-05-12T03:46:05.903 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : buildah-2:1.33.6-2.module_el8+976+11360731.x86_64 2/13 2024-05-12T03:46:05.903 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : cockpit-podman-84.1-1.module_el8+977+8ab54275.noar 3/13 2024-05-12T03:46:05.903 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_ 4/13 2024-05-12T03:46:05.903 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : crun-1.14.3-2.module_el8+968+fbb249c7.x86_64 5/13 2024-05-12T03:46:05.903 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : netavark-2:1.10.3-1.module_el8+951+32019cde.x86_64 6/13 2024-05-12T03:46:05.905 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-podman-4.9.0-1.module_el8+847+7863d4e6.noa 7/13 2024-05-12T03:46:05.905 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 8/13 2024-05-12T03:46:05.905 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 9/13 2024-05-12T03:46:05.905 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 10/13 2024-05-12T03:46:05.905 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : toolbox-0.0.99.5-2.module_el8+971+3d3df00d.x86_64 11/13 2024-05-12T03:46:05.905 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : udica-0.2.6-20.module_el8+804+f131391c.noarch 12/13 2024-05-12T03:46:06.362 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: cockpit-podman-84.1-1.module_el8+977+8ab54275.noar 13/13 2024-05-12T03:46:06.362 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x8 1/13 2024-05-12T03:46:06.362 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : buildah-2:1.33.6-2.module_el8+976+11360731.x86_64 2/13 2024-05-12T03:46:06.362 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : cockpit-podman-84.1-1.module_el8+977+8ab54275.noar 3/13 2024-05-12T03:46:06.362 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_ 4/13 2024-05-12T03:46:06.362 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : crun-1.14.3-2.module_el8+968+fbb249c7.x86_64 5/13 2024-05-12T03:46:06.362 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : netavark-2:1.10.3-1.module_el8+951+32019cde.x86_64 6/13 2024-05-12T03:46:06.362 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-podman-4.9.0-1.module_el8+847+7863d4e6.noa 7/13 2024-05-12T03:46:06.363 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 8/13 2024-05-12T03:46:06.363 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 9/13 2024-05-12T03:46:06.363 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 10/13 2024-05-12T03:46:06.363 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : toolbox-0.0.99.5-2.module_el8+971+3d3df00d.x86_64 11/13 2024-05-12T03:46:06.363 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : udica-0.2.6-20.module_el8+804+f131391c.noarch 12/13 2024-05-12T03:46:06.373 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 13/13 2024-05-12T03:46:06.373 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:46:06.373 INFO:teuthology.orchestra.run.smithi032.stdout:Installed: 2024-05-12T03:46:06.373 INFO:teuthology.orchestra.run.smithi032.stdout: aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x86_64 2024-05-12T03:46:06.373 INFO:teuthology.orchestra.run.smithi032.stdout: buildah-2:1.33.6-2.module_el8+976+11360731.x86_64 2024-05-12T03:46:06.373 INFO:teuthology.orchestra.run.smithi032.stdout: cockpit-podman-84.1-1.module_el8+977+8ab54275.noarch 2024-05-12T03:46:06.373 INFO:teuthology.orchestra.run.smithi032.stdout: criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_64 2024-05-12T03:46:06.373 INFO:teuthology.orchestra.run.smithi032.stdout: crun-1.14.3-2.module_el8+968+fbb249c7.x86_64 2024-05-12T03:46:06.373 INFO:teuthology.orchestra.run.smithi032.stdout: netavark-2:1.10.3-1.module_el8+951+32019cde.x86_64 2024-05-12T03:46:06.373 INFO:teuthology.orchestra.run.smithi032.stdout: python3-podman-4.9.0-1.module_el8+847+7863d4e6.noarch 2024-05-12T03:46:06.374 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-05-12T03:46:06.374 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-05-12T03:46:06.374 INFO:teuthology.orchestra.run.smithi032.stdout: skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 2024-05-12T03:46:06.374 INFO:teuthology.orchestra.run.smithi032.stdout: toolbox-0.0.99.5-2.module_el8+971+3d3df00d.x86_64 2024-05-12T03:46:06.374 INFO:teuthology.orchestra.run.smithi032.stdout: udica-0.2.6-20.module_el8+804+f131391c.noarch 2024-05-12T03:46:06.374 INFO:teuthology.orchestra.run.smithi032.stdout: yajl-2.1.0-12.el8.x86_64 2024-05-12T03:46:06.374 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:46:06.374 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2024-05-12T03:46:06.629 DEBUG:teuthology.parallel:result is None 2024-05-12T03:46:06.890 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 13/13 2024-05-12T03:46:06.890 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:46:06.890 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2024-05-12T03:46:06.890 INFO:teuthology.orchestra.run.smithi033.stdout: aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x86_64 2024-05-12T03:46:06.890 INFO:teuthology.orchestra.run.smithi033.stdout: buildah-2:1.33.6-2.module_el8+976+11360731.x86_64 2024-05-12T03:46:06.890 INFO:teuthology.orchestra.run.smithi033.stdout: cockpit-podman-84.1-1.module_el8+977+8ab54275.noarch 2024-05-12T03:46:06.890 INFO:teuthology.orchestra.run.smithi033.stdout: criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_64 2024-05-12T03:46:06.890 INFO:teuthology.orchestra.run.smithi033.stdout: crun-1.14.3-2.module_el8+968+fbb249c7.x86_64 2024-05-12T03:46:06.890 INFO:teuthology.orchestra.run.smithi033.stdout: netavark-2:1.10.3-1.module_el8+951+32019cde.x86_64 2024-05-12T03:46:06.890 INFO:teuthology.orchestra.run.smithi033.stdout: python3-podman-4.9.0-1.module_el8+847+7863d4e6.noarch 2024-05-12T03:46:06.891 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-05-12T03:46:06.891 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-05-12T03:46:06.891 INFO:teuthology.orchestra.run.smithi033.stdout: skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 2024-05-12T03:46:06.891 INFO:teuthology.orchestra.run.smithi033.stdout: toolbox-0.0.99.5-2.module_el8+971+3d3df00d.x86_64 2024-05-12T03:46:06.891 INFO:teuthology.orchestra.run.smithi033.stdout: udica-0.2.6-20.module_el8+804+f131391c.noarch 2024-05-12T03:46:06.891 INFO:teuthology.orchestra.run.smithi033.stdout: yajl-2.1.0-12.el8.x86_64 2024-05-12T03:46:06.891 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:46:06.891 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-05-12T03:46:07.196 DEBUG:teuthology.parallel:result is None 2024-05-12T03:46:07.196 INFO:teuthology.run_tasks:Running task install... 2024-05-12T03:46:07.206 DEBUG:teuthology.task.install:project ceph 2024-05-12T03:46:07.206 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '58249552a22f437b8cb49ed6b1068e1158bed605'}} 2024-05-12T03:46:07.206 DEBUG:teuthology.task.install:config {'branch': 'quincy', 'exclude_packages': ['ceph-volume'], 'flavor': 'default', 'sha1': '58249552a22f437b8cb49ed6b1068e1158bed605'} 2024-05-12T03:46:07.206 INFO:teuthology.task.install:Using flavor: default 2024-05-12T03:46:07.214 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-12T03:46:07.214 INFO:teuthology.task.install:extra packages: [] 2024-05-12T03:46:07.215 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'quincy', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '58249552a22f437b8cb49ed6b1068e1158bed605', 'tag': None, 'wait_for_package': False} 2024-05-12T03:46:07.215 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-12T03:46:07.215 INFO:teuthology.packaging:ref: None 2024-05-12T03:46:07.215 INFO:teuthology.packaging:tag: None 2024-05-12T03:46:07.215 INFO:teuthology.packaging:branch: quincy 2024-05-12T03:46:07.216 INFO:teuthology.packaging:sha1: 58249552a22f437b8cb49ed6b1068e1158bed605 2024-05-12T03:46:07.216 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=quincy 2024-05-12T03:46:07.217 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'quincy', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '58249552a22f437b8cb49ed6b1068e1158bed605', 'tag': None, 'wait_for_package': False} 2024-05-12T03:46:07.218 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-12T03:46:07.218 INFO:teuthology.packaging:ref: None 2024-05-12T03:46:07.218 INFO:teuthology.packaging:tag: None 2024-05-12T03:46:07.218 INFO:teuthology.packaging:branch: quincy 2024-05-12T03:46:07.218 INFO:teuthology.packaging:sha1: 58249552a22f437b8cb49ed6b1068e1158bed605 2024-05-12T03:46:07.218 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=quincy 2024-05-12T03:46:07.438 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/quincy/c190bdeec0cb2c5a9760aa085c602e6994249982/centos/8/flavors/default/ 2024-05-12T03:46:07.439 INFO:teuthology.task.install.rpm:Package version is 17.2.7-939.gc190bdee 2024-05-12T03:46:07.478 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/quincy/c190bdeec0cb2c5a9760aa085c602e6994249982/centos/8/flavors/default/ 2024-05-12T03:46:07.479 INFO:teuthology.task.install.rpm:Package version is 17.2.7-939.gc190bdee 2024-05-12T03:46:07.614 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/quincy/c190bdeec0cb2c5a9760aa085c602e6994249982/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/quincy/c190bdeec0cb2c5a9760aa085c602e6994249982/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/quincy/c190bdeec0cb2c5a9760aa085c602e6994249982/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-12T03:46:07.615 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-05-12T03:46:07.615 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-12T03:46:07.648 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/quincy/c190bdeec0cb2c5a9760aa085c602e6994249982/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/quincy/c190bdeec0cb2c5a9760aa085c602e6994249982/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/quincy/c190bdeec0cb2c5a9760aa085c602e6994249982/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-12T03:46:07.649 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-12T03:46:07.649 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-12T03:46:07.650 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-05-12T03:46:07.650 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-12T03:46:07.650 INFO:teuthology.packaging:ref: None 2024-05-12T03:46:07.650 INFO:teuthology.packaging:tag: None 2024-05-12T03:46:07.650 INFO:teuthology.packaging:branch: quincy 2024-05-12T03:46:07.650 INFO:teuthology.packaging:sha1: 58249552a22f437b8cb49ed6b1068e1158bed605 2024-05-12T03:46:07.650 DEBUG:teuthology.orchestra.run.smithi032:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/quincy/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-12T03:46:07.682 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-05-12T03:46:07.682 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-12T03:46:07.682 INFO:teuthology.packaging:ref: None 2024-05-12T03:46:07.682 INFO:teuthology.packaging:tag: None 2024-05-12T03:46:07.682 INFO:teuthology.packaging:branch: quincy 2024-05-12T03:46:07.682 INFO:teuthology.packaging:sha1: 58249552a22f437b8cb49ed6b1068e1158bed605 2024-05-12T03:46:07.682 DEBUG:teuthology.orchestra.run.smithi033:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/quincy/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-12T03:46:07.725 DEBUG:teuthology.orchestra.run.smithi032:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-12T03:46:07.753 DEBUG:teuthology.orchestra.run.smithi033:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-12T03:46:07.808 DEBUG:teuthology.orchestra.run.smithi032:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-12T03:46:07.833 DEBUG:teuthology.orchestra.run.smithi033:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-12T03:46:07.877 INFO:teuthology.orchestra.run.smithi032.stdout:check_obsoletes = 1 2024-05-12T03:46:07.879 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum clean all 2024-05-12T03:46:07.901 INFO:teuthology.orchestra.run.smithi033.stdout:check_obsoletes = 1 2024-05-12T03:46:07.903 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum clean all 2024-05-12T03:46:08.166 INFO:teuthology.orchestra.run.smithi032.stdout:78 files removed 2024-05-12T03:46:08.190 INFO:teuthology.orchestra.run.smithi033.stdout:78 files removed 2024-05-12T03:46:08.195 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum -y install ceph-radosgw 2024-05-12T03:46:08.220 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum -y install ceph-radosgw 2024-05-12T03:46:08.923 INFO:teuthology.orchestra.run.smithi032.stdout:ceph packages for x86_64 259 kB/s | 78 kB 00:00 2024-05-12T03:46:09.233 INFO:teuthology.orchestra.run.smithi033.stdout:ceph packages for x86_64 135 kB/s | 78 kB 00:00 2024-05-12T03:46:09.291 INFO:teuthology.orchestra.run.smithi032.stdout:ceph noarch packages 33 kB/s | 11 kB 00:00 2024-05-12T03:46:09.666 INFO:teuthology.orchestra.run.smithi032.stdout:ceph source packages 4.2 kB/s | 1.4 kB 00:00 2024-05-12T03:46:09.739 INFO:teuthology.orchestra.run.smithi033.stdout:ceph noarch packages 24 kB/s | 11 kB 00:00 2024-05-12T03:46:10.007 INFO:teuthology.orchestra.run.smithi033.stdout:ceph source packages 6.2 kB/s | 1.4 kB 00:00 2024-05-12T03:46:11.000 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 8 - AppStream 22 MB/s | 28 MB 00:01 2024-05-12T03:46:11.165 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 8 - AppStream 25 MB/s | 28 MB 00:01 2024-05-12T03:46:17.876 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 8 - BaseOS 6.5 MB/s | 10 MB 00:01 2024-05-12T03:46:18.185 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 8 - BaseOS 6.4 MB/s | 10 MB 00:01 2024-05-12T03:46:19.993 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 8 - Extras 44 kB/s | 18 kB 00:00 2024-05-12T03:46:20.436 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 8 - Extras common packages 24 kB/s | 7.7 kB 00:00 2024-05-12T03:46:20.475 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 8 - Extras 34 kB/s | 18 kB 00:00 2024-05-12T03:46:20.803 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 8 - Extras common packages 37 kB/s | 7.7 kB 00:00 2024-05-12T03:46:20.845 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 8 - PowerTools 16 MB/s | 5.0 MB 00:00 2024-05-12T03:46:21.346 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 8 - PowerTools 11 MB/s | 5.0 MB 00:00 2024-05-12T03:46:21.955 INFO:teuthology.orchestra.run.smithi032.stdout:Copr repo for python3-asyncssh owned by ceph 28 kB/s | 3.5 kB 00:00 2024-05-12T03:46:22.614 INFO:teuthology.orchestra.run.smithi033.stdout:Copr repo for python3-asyncssh owned by ceph 13 kB/s | 3.5 kB 00:00 2024-05-12T03:46:23.417 INFO:teuthology.orchestra.run.smithi033.stdout:Extra Packages for Enterprise Linux 19 MB/s | 14 MB 00:00 2024-05-12T03:46:27.906 INFO:teuthology.orchestra.run.smithi033.stdout:lab-extras 379 kB/s | 24 kB 00:00 2024-05-12T03:46:28.314 INFO:teuthology.orchestra.run.smithi032.stdout:Extra Packages for Enterprise Linux 2.2 MB/s | 14 MB 00:06 2024-05-12T03:46:31.826 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-05-12T03:46:31.829 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-05-12T03:46:31.829 INFO:teuthology.orchestra.run.smithi033.stdout: Package Arch Version Repository Size 2024-05-12T03:46:31.829 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-05-12T03:46:31.829 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2024-05-12T03:46:31.829 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-radosgw x86_64 2:17.2.7-939.gc190bdee.el8 ceph 17 M 2024-05-12T03:46:31.829 INFO:teuthology.orchestra.run.smithi033.stdout:Upgrading: 2024-05-12T03:46:31.829 INFO:teuthology.orchestra.run.smithi033.stdout: librados2 x86_64 2:17.2.7-939.gc190bdee.el8 ceph 3.5 M 2024-05-12T03:46:31.829 INFO:teuthology.orchestra.run.smithi033.stdout: librbd1 x86_64 2:17.2.7-939.gc190bdee.el8 ceph 3.6 M 2024-05-12T03:46:31.829 INFO:teuthology.orchestra.run.smithi033.stdout:Installing dependencies: 2024-05-12T03:46:31.829 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-base x86_64 2:17.2.7-939.gc190bdee.el8 ceph 5.3 M 2024-05-12T03:46:31.830 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-common x86_64 2:17.2.7-939.gc190bdee.el8 ceph 24 M 2024-05-12T03:46:31.830 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-selinux x86_64 2:17.2.7-939.gc190bdee.el8 ceph 24 k 2024-05-12T03:46:31.830 INFO:teuthology.orchestra.run.smithi033.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-05-12T03:46:31.830 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs2 x86_64 2:17.2.7-939.gc190bdee.el8 ceph 800 k 2024-05-12T03:46:31.830 INFO:teuthology.orchestra.run.smithi033.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-05-12T03:46:31.830 INFO:teuthology.orchestra.run.smithi033.stdout: librabbitmq x86_64 0.9.0-5.el8 baseos 47 k 2024-05-12T03:46:31.830 INFO:teuthology.orchestra.run.smithi033.stdout: libradosstriper1 x86_64 2:17.2.7-939.gc190bdee.el8 ceph 506 k 2024-05-12T03:46:31.830 INFO:teuthology.orchestra.run.smithi033.stdout: librdkafka x86_64 1.6.1-1.el8 appstream 677 k 2024-05-12T03:46:31.830 INFO:teuthology.orchestra.run.smithi033.stdout: librgw2 x86_64 2:17.2.7-939.gc190bdee.el8 ceph 5.6 M 2024-05-12T03:46:31.830 INFO:teuthology.orchestra.run.smithi033.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-05-12T03:46:31.830 INFO:teuthology.orchestra.run.smithi033.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2024-05-12T03:46:31.830 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-argparse x86_64 2:17.2.7-939.gc190bdee.el8 ceph 45 k 2024-05-12T03:46:31.830 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-common x86_64 2:17.2.7-939.gc190bdee.el8 ceph 117 k 2024-05-12T03:46:31.831 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cephfs x86_64 2:17.2.7-939.gc190bdee.el8 ceph 190 k 2024-05-12T03:46:31.831 INFO:teuthology.orchestra.run.smithi033.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2024-05-12T03:46:31.831 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rados x86_64 2:17.2.7-939.gc190bdee.el8 ceph 371 k 2024-05-12T03:46:31.831 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rbd x86_64 2:17.2.7-939.gc190bdee.el8 ceph 359 k 2024-05-12T03:46:31.831 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rgw x86_64 2:17.2.7-939.gc190bdee.el8 ceph 106 k 2024-05-12T03:46:31.831 INFO:teuthology.orchestra.run.smithi033.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-05-12T03:46:31.831 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:46:31.831 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2024-05-12T03:46:31.831 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-05-12T03:46:31.831 INFO:teuthology.orchestra.run.smithi033.stdout:Install 21 Packages 2024-05-12T03:46:31.831 INFO:teuthology.orchestra.run.smithi033.stdout:Upgrade 2 Packages 2024-05-12T03:46:31.831 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:46:31.832 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 64 M 2024-05-12T03:46:31.832 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading Packages: 2024-05-12T03:46:32.806 INFO:teuthology.orchestra.run.smithi032.stdout:lab-extras 416 kB/s | 24 kB 00:00 2024-05-12T03:46:32.915 INFO:teuthology.orchestra.run.smithi033.stdout:(1/23): ceph-base-17.2.7-939.gc190bdee.el8.x86_ 7.9 MB/s | 5.3 MB 00:00 2024-05-12T03:46:32.991 INFO:teuthology.orchestra.run.smithi033.stdout:(2/23): ceph-selinux-17.2.7-939.gc190bdee.el8.x 325 kB/s | 24 kB 00:00 2024-05-12T03:46:33.108 INFO:teuthology.orchestra.run.smithi033.stdout:(3/23): libcephfs2-17.2.7-939.gc190bdee.el8.x86 6.7 MB/s | 800 kB 00:00 2024-05-12T03:46:33.225 INFO:teuthology.orchestra.run.smithi033.stdout:(4/23): libradosstriper1-17.2.7-939.gc190bdee.e 4.2 MB/s | 506 kB 00:00 2024-05-12T03:46:33.800 INFO:teuthology.orchestra.run.smithi033.stdout:(5/23): ceph-radosgw-17.2.7-939.gc190bdee.el8.x 11 MB/s | 17 MB 00:01 2024-05-12T03:46:33.858 INFO:teuthology.orchestra.run.smithi033.stdout:(6/23): python3-ceph-argparse-17.2.7-939.gc190b 770 kB/s | 45 kB 00:00 2024-05-12T03:46:33.942 INFO:teuthology.orchestra.run.smithi033.stdout:(7/23): librgw2-17.2.7-939.gc190bdee.el8.x86_64 7.8 MB/s | 5.6 MB 00:00 2024-05-12T03:46:33.967 INFO:teuthology.orchestra.run.smithi033.stdout:(8/23): python3-ceph-common-17.2.7-939.gc190bde 1.1 MB/s | 117 kB 00:00 2024-05-12T03:46:34.001 INFO:teuthology.orchestra.run.smithi033.stdout:(9/23): python3-cephfs-17.2.7-939.gc190bdee.el8 3.2 MB/s | 190 kB 00:00 2024-05-12T03:46:34.055 INFO:teuthology.orchestra.run.smithi033.stdout:(10/23): python3-rados-17.2.7-939.gc190bdee.el8 4.1 MB/s | 371 kB 00:00 2024-05-12T03:46:34.093 INFO:teuthology.orchestra.run.smithi033.stdout:(11/23): python3-rbd-17.2.7-939.gc190bdee.el8.x 3.8 MB/s | 359 kB 00:00 2024-05-12T03:46:34.119 INFO:teuthology.orchestra.run.smithi033.stdout:(12/23): python3-rgw-17.2.7-939.gc190bdee.el8.x 1.7 MB/s | 106 kB 00:00 2024-05-12T03:46:34.311 INFO:teuthology.orchestra.run.smithi033.stdout:(13/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.3 MB/s | 259 kB 00:00 2024-05-12T03:46:34.344 INFO:teuthology.orchestra.run.smithi033.stdout:(14/23): librdkafka-1.6.1-1.el8.x86_64.rpm 2.6 MB/s | 677 kB 00:00 2024-05-12T03:46:34.370 INFO:teuthology.orchestra.run.smithi033.stdout:(15/23): python3-prettytable-0.7.2-14.el8.noarc 750 kB/s | 44 kB 00:00 2024-05-12T03:46:34.778 INFO:teuthology.orchestra.run.smithi033.stdout:(16/23): ceph-common-17.2.7-939.gc190bdee.el8.x 9.3 MB/s | 24 MB 00:02 2024-05-12T03:46:34.803 INFO:teuthology.orchestra.run.smithi033.stdout:(17/23): librabbitmq-0.9.0-5.el8.x86_64.rpm 103 kB/s | 47 kB 00:00 2024-05-12T03:46:34.937 INFO:teuthology.orchestra.run.smithi033.stdout:(18/23): gperftools-libs-2.7-9.el8.x86_64.rpm 540 kB/s | 306 kB 00:00 2024-05-12T03:46:35.079 INFO:teuthology.orchestra.run.smithi033.stdout:(19/23): liboath-2.6.2-3.el8.x86_64.rpm 196 kB/s | 59 kB 00:00 2024-05-12T03:46:35.113 INFO:teuthology.orchestra.run.smithi033.stdout:(20/23): libunwind-1.3.1-3.el8.x86_64.rpm 243 kB/s | 75 kB 00:00 2024-05-12T03:46:35.470 INFO:teuthology.orchestra.run.smithi033.stdout:(21/23): thrift-0.13.0-2.el8.x86_64.rpm 3.3 MB/s | 1.7 MB 00:00 2024-05-12T03:46:35.831 INFO:teuthology.orchestra.run.smithi033.stdout:(22/23): librados2-17.2.7-939.gc190bdee.el8.x86 4.6 MB/s | 3.5 MB 00:00 2024-05-12T03:46:36.102 INFO:teuthology.orchestra.run.smithi033.stdout:(23/23): librbd1-17.2.7-939.gc190bdee.el8.x86_6 3.6 MB/s | 3.6 MB 00:00 2024-05-12T03:46:36.104 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2024-05-12T03:46:36.104 INFO:teuthology.orchestra.run.smithi033.stdout:Total 15 MB/s | 64 MB 00:04 2024-05-12T03:46:36.149 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2024-05-12T03:46:36.203 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction check succeeded. 2024-05-12T03:46:36.204 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2024-05-12T03:46:36.597 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded. 2024-05-12T03:46:36.601 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2024-05-12T03:46:36.725 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-05-12T03:46:36.728 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-05-12T03:46:36.728 INFO:teuthology.orchestra.run.smithi032.stdout: Package Arch Version Repository Size 2024-05-12T03:46:36.728 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-05-12T03:46:36.728 INFO:teuthology.orchestra.run.smithi032.stdout:Installing: 2024-05-12T03:46:36.728 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-radosgw x86_64 2:17.2.7-939.gc190bdee.el8 ceph 17 M 2024-05-12T03:46:36.728 INFO:teuthology.orchestra.run.smithi032.stdout:Upgrading: 2024-05-12T03:46:36.728 INFO:teuthology.orchestra.run.smithi032.stdout: librados2 x86_64 2:17.2.7-939.gc190bdee.el8 ceph 3.5 M 2024-05-12T03:46:36.728 INFO:teuthology.orchestra.run.smithi032.stdout: librbd1 x86_64 2:17.2.7-939.gc190bdee.el8 ceph 3.6 M 2024-05-12T03:46:36.729 INFO:teuthology.orchestra.run.smithi032.stdout:Installing dependencies: 2024-05-12T03:46:36.729 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-base x86_64 2:17.2.7-939.gc190bdee.el8 ceph 5.3 M 2024-05-12T03:46:36.729 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-common x86_64 2:17.2.7-939.gc190bdee.el8 ceph 24 M 2024-05-12T03:46:36.729 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-selinux x86_64 2:17.2.7-939.gc190bdee.el8 ceph 24 k 2024-05-12T03:46:36.729 INFO:teuthology.orchestra.run.smithi032.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-05-12T03:46:36.729 INFO:teuthology.orchestra.run.smithi032.stdout: libcephfs2 x86_64 2:17.2.7-939.gc190bdee.el8 ceph 800 k 2024-05-12T03:46:36.729 INFO:teuthology.orchestra.run.smithi032.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-05-12T03:46:36.729 INFO:teuthology.orchestra.run.smithi032.stdout: librabbitmq x86_64 0.9.0-5.el8 baseos 47 k 2024-05-12T03:46:36.729 INFO:teuthology.orchestra.run.smithi032.stdout: libradosstriper1 x86_64 2:17.2.7-939.gc190bdee.el8 ceph 506 k 2024-05-12T03:46:36.729 INFO:teuthology.orchestra.run.smithi032.stdout: librdkafka x86_64 1.6.1-1.el8 appstream 677 k 2024-05-12T03:46:36.729 INFO:teuthology.orchestra.run.smithi032.stdout: librgw2 x86_64 2:17.2.7-939.gc190bdee.el8 ceph 5.6 M 2024-05-12T03:46:36.729 INFO:teuthology.orchestra.run.smithi032.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-05-12T03:46:36.729 INFO:teuthology.orchestra.run.smithi032.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2024-05-12T03:46:36.729 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ceph-argparse x86_64 2:17.2.7-939.gc190bdee.el8 ceph 45 k 2024-05-12T03:46:36.730 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ceph-common x86_64 2:17.2.7-939.gc190bdee.el8 ceph 117 k 2024-05-12T03:46:36.730 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cephfs x86_64 2:17.2.7-939.gc190bdee.el8 ceph 190 k 2024-05-12T03:46:36.730 INFO:teuthology.orchestra.run.smithi032.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2024-05-12T03:46:36.730 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rados x86_64 2:17.2.7-939.gc190bdee.el8 ceph 371 k 2024-05-12T03:46:36.730 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rbd x86_64 2:17.2.7-939.gc190bdee.el8 ceph 359 k 2024-05-12T03:46:36.730 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rgw x86_64 2:17.2.7-939.gc190bdee.el8 ceph 106 k 2024-05-12T03:46:36.730 INFO:teuthology.orchestra.run.smithi032.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-05-12T03:46:36.730 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:46:36.730 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction Summary 2024-05-12T03:46:36.731 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-05-12T03:46:36.731 INFO:teuthology.orchestra.run.smithi032.stdout:Install 21 Packages 2024-05-12T03:46:36.731 INFO:teuthology.orchestra.run.smithi032.stdout:Upgrade 2 Packages 2024-05-12T03:46:36.731 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:46:36.732 INFO:teuthology.orchestra.run.smithi032.stdout:Total download size: 64 M 2024-05-12T03:46:36.732 INFO:teuthology.orchestra.run.smithi032.stdout:Downloading Packages: 2024-05-12T03:46:37.726 INFO:teuthology.orchestra.run.smithi033.stdout: Preparing : 1/1 2024-05-12T03:46:37.944 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/25 2024-05-12T03:46:38.137 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 2/25 2024-05-12T03:46:38.224 INFO:teuthology.orchestra.run.smithi032.stdout:(1/23): ceph-radosgw-17.2.7-939.gc190bdee.el8.x 16 MB/s | 17 MB 00:01 2024-05-12T03:46:38.274 INFO:teuthology.orchestra.run.smithi032.stdout:(2/23): ceph-selinux-17.2.7-939.gc190bdee.el8.x 489 kB/s | 24 kB 00:00 2024-05-12T03:46:38.350 INFO:teuthology.orchestra.run.smithi032.stdout:(3/23): libcephfs2-17.2.7-939.gc190bdee.el8.x86 10 MB/s | 800 kB 00:00 2024-05-12T03:46:38.408 INFO:teuthology.orchestra.run.smithi032.stdout:(4/23): libradosstriper1-17.2.7-939.gc190bdee.e 8.5 MB/s | 506 kB 00:00 2024-05-12T03:46:38.540 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 2/25 2024-05-12T03:46:38.742 INFO:teuthology.orchestra.run.smithi032.stdout:(5/23): librgw2-17.2.7-939.gc190bdee.el8.x86_64 17 MB/s | 5.6 MB 00:00 2024-05-12T03:46:38.745 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : librados2-2:17.2.7-939.gc190bdee.el8.x86_64 3/25 2024-05-12T03:46:38.792 INFO:teuthology.orchestra.run.smithi032.stdout:(6/23): python3-ceph-argparse-17.2.7-939.gc190b 901 kB/s | 45 kB 00:00 2024-05-12T03:46:38.840 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librados2-2:17.2.7-939.gc190bdee.el8.x86_64 3/25 2024-05-12T03:46:38.842 INFO:teuthology.orchestra.run.smithi032.stdout:(7/23): python3-ceph-common-17.2.7-939.gc190bde 2.3 MB/s | 117 kB 00:00 2024-05-12T03:46:38.901 INFO:teuthology.orchestra.run.smithi032.stdout:(8/23): python3-cephfs-17.2.7-939.gc190bdee.el8 3.2 MB/s | 190 kB 00:00 2024-05-12T03:46:38.959 INFO:teuthology.orchestra.run.smithi032.stdout:(9/23): python3-rados-17.2.7-939.gc190bdee.el8. 6.2 MB/s | 371 kB 00:00 2024-05-12T03:46:39.051 INFO:teuthology.orchestra.run.smithi032.stdout:(10/23): ceph-base-17.2.7-939.gc190bdee.el8.x86 2.8 MB/s | 5.3 MB 00:01 2024-05-12T03:46:39.077 INFO:teuthology.orchestra.run.smithi032.stdout:(11/23): python3-rbd-17.2.7-939.gc190bdee.el8.x 3.0 MB/s | 359 kB 00:00 2024-05-12T03:46:39.100 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rados-2:17.2.7-939.gc190bdee.el8.x86_64 4/25 2024-05-12T03:46:39.110 INFO:teuthology.orchestra.run.smithi032.stdout:(12/23): python3-rgw-17.2.7-939.gc190bdee.el8.x 1.8 MB/s | 106 kB 00:00 2024-05-12T03:46:39.287 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libcephfs2-2:17.2.7-939.gc190bdee.el8.x86_64 5/25 2024-05-12T03:46:39.302 INFO:teuthology.orchestra.run.smithi032.stdout:(13/23): librdkafka-1.6.1-1.el8.x86_64.rpm 2.9 MB/s | 677 kB 00:00 2024-05-12T03:46:39.328 INFO:teuthology.orchestra.run.smithi032.stdout:(14/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.2 MB/s | 259 kB 00:00 2024-05-12T03:46:39.353 INFO:teuthology.orchestra.run.smithi032.stdout:(15/23): python3-prettytable-0.7.2-14.el8.noarc 874 kB/s | 44 kB 00:00 2024-05-12T03:46:39.629 INFO:teuthology.orchestra.run.smithi032.stdout:(16/23): librabbitmq-0.9.0-5.el8.x86_64.rpm 157 kB/s | 47 kB 00:00 2024-05-12T03:46:39.714 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libcephfs2-2:17.2.7-939.gc190bdee.el8.x86_64 5/25 2024-05-12T03:46:39.887 INFO:teuthology.orchestra.run.smithi032.stdout:(17/23): gperftools-libs-2.7-9.el8.x86_64.rpm 573 kB/s | 306 kB 00:00 2024-05-12T03:46:39.920 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : librbd1-2:17.2.7-939.gc190bdee.el8.x86_64 6/25 2024-05-12T03:46:40.017 INFO:teuthology.orchestra.run.smithi032.stdout:(18/23): libunwind-1.3.1-3.el8.x86_64.rpm 581 kB/s | 75 kB 00:00 2024-05-12T03:46:40.046 INFO:teuthology.orchestra.run.smithi032.stdout:(19/23): liboath-2.6.2-3.el8.x86_64.rpm 141 kB/s | 59 kB 00:00 2024-05-12T03:46:40.144 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librbd1-2:17.2.7-939.gc190bdee.el8.x86_64 6/25 2024-05-12T03:46:40.296 INFO:teuthology.orchestra.run.smithi032.stdout:(20/23): thrift-0.13.0-2.el8.x86_64.rpm 6.2 MB/s | 1.7 MB 00:00 2024-05-12T03:46:40.367 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : thrift-0.13.0-2.el8.x86_64 7/25 2024-05-12T03:46:40.583 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librabbitmq-0.9.0-5.el8.x86_64 8/25 2024-05-12T03:46:41.283 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librdkafka-1.6.1-1.el8.x86_64 9/25 2024-05-12T03:46:41.447 INFO:teuthology.orchestra.run.smithi032.stdout:(21/23): librbd1-17.2.7-939.gc190bdee.el8.x86_6 3.1 MB/s | 3.6 MB 00:01 2024-05-12T03:46:41.513 INFO:teuthology.orchestra.run.smithi032.stdout:(22/23): librados2-17.2.7-939.gc190bdee.el8.x86 2.4 MB/s | 3.5 MB 00:01 2024-05-12T03:46:41.653 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librgw2-2:17.2.7-939.gc190bdee.el8.x86_64 10/25 2024-05-12T03:46:41.725 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librgw2-2:17.2.7-939.gc190bdee.el8.x86_64 10/25 2024-05-12T03:46:41.882 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-ceph-argparse-2:17.2.7-939.gc190bdee.el8.x 11/25 2024-05-12T03:46:42.091 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cephfs-2:17.2.7-939.gc190bdee.el8.x86_64 12/25 2024-05-12T03:46:42.333 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rgw-2:17.2.7-939.gc190bdee.el8.x86_64 13/25 2024-05-12T03:46:42.535 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rbd-2:17.2.7-939.gc190bdee.el8.x86_64 14/25 2024-05-12T03:46:42.678 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libradosstriper1-2:17.2.7-939.gc190bdee.el8.x86_64 15/25 2024-05-12T03:46:42.751 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libradosstriper1-2:17.2.7-939.gc190bdee.el8.x86_64 15/25 2024-05-12T03:46:42.966 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 16/25 2024-05-12T03:46:43.114 INFO:teuthology.orchestra.run.smithi032.stdout:(23/23): ceph-common-17.2.7-939.gc190bdee.el8.x 4.0 MB/s | 24 MB 00:05 2024-05-12T03:46:43.118 INFO:teuthology.orchestra.run.smithi032.stdout:-------------------------------------------------------------------------------- 2024-05-12T03:46:43.118 INFO:teuthology.orchestra.run.smithi032.stdout:Total 10 MB/s | 64 MB 00:06 2024-05-12T03:46:43.154 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 17/25 2024-05-12T03:46:43.161 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction check 2024-05-12T03:46:43.211 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction check succeeded. 2024-05-12T03:46:43.211 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction test 2024-05-12T03:46:43.369 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 18/25 2024-05-12T03:46:43.606 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-ceph-common-2:17.2.7-939.gc190bdee.el8.x86 19/25 2024-05-12T03:46:43.619 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction test succeeded. 2024-05-12T03:46:43.622 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction 2024-05-12T03:46:44.714 INFO:teuthology.orchestra.run.smithi032.stdout: Preparing : 1/1 2024-05-12T03:46:44.916 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/25 2024-05-12T03:46:45.119 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 2/25 2024-05-12T03:46:45.527 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 2/25 2024-05-12T03:46:45.744 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : librados2-2:17.2.7-939.gc190bdee.el8.x86_64 3/25 2024-05-12T03:46:45.840 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librados2-2:17.2.7-939.gc190bdee.el8.x86_64 3/25 2024-05-12T03:46:46.057 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-rados-2:17.2.7-939.gc190bdee.el8.x86_64 4/25 2024-05-12T03:46:46.202 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libcephfs2-2:17.2.7-939.gc190bdee.el8.x86_64 5/25 2024-05-12T03:46:46.493 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-common-2:17.2.7-939.gc190bdee.el8.x86_64 20/25 2024-05-12T03:46:46.632 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libcephfs2-2:17.2.7-939.gc190bdee.el8.x86_64 5/25 2024-05-12T03:46:47.263 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-common-2:17.2.7-939.gc190bdee.el8.x86_64 20/25 2024-05-12T03:46:47.264 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : librbd1-2:17.2.7-939.gc190bdee.el8.x86_64 6/25 2024-05-12T03:46:47.264 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librbd1-2:17.2.7-939.gc190bdee.el8.x86_64 6/25 2024-05-12T03:46:47.264 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : thrift-0.13.0-2.el8.x86_64 7/25 2024-05-12T03:46:47.399 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : librabbitmq-0.9.0-5.el8.x86_64 8/25 2024-05-12T03:46:47.400 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-common-2:17.2.7-939.gc190bdee.el8.x86_64 20/25 2024-05-12T03:46:47.690 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-base-2:17.2.7-939.gc190bdee.el8.x86_64 21/25 2024-05-12T03:46:47.789 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-base-2:17.2.7-939.gc190bdee.el8.x86_64 21/25 2024-05-12T03:46:48.018 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-selinux-2:17.2.7-939.gc190bdee.el8.x86_64 22/25 2024-05-12T03:46:48.073 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : librdkafka-1.6.1-1.el8.x86_64 9/25 2024-05-12T03:46:48.202 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : librgw2-2:17.2.7-939.gc190bdee.el8.x86_64 10/25 2024-05-12T03:46:48.274 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librgw2-2:17.2.7-939.gc190bdee.el8.x86_64 10/25 2024-05-12T03:46:48.448 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-ceph-argparse-2:17.2.7-939.gc190bdee.el8.x 11/25 2024-05-12T03:46:48.614 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cephfs-2:17.2.7-939.gc190bdee.el8.x86_64 12/25 2024-05-12T03:46:48.882 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-rgw-2:17.2.7-939.gc190bdee.el8.x86_64 13/25 2024-05-12T03:46:49.085 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-rbd-2:17.2.7-939.gc190bdee.el8.x86_64 14/25 2024-05-12T03:46:49.219 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libradosstriper1-2:17.2.7-939.gc190bdee.el8.x86_64 15/25 2024-05-12T03:46:49.321 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libradosstriper1-2:17.2.7-939.gc190bdee.el8.x86_64 15/25 2024-05-12T03:46:49.532 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 16/25 2024-05-12T03:46:49.812 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 17/25 2024-05-12T03:46:50.359 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 18/25 2024-05-12T03:46:50.547 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-ceph-common-2:17.2.7-939.gc190bdee.el8.x86 19/25 2024-05-12T03:46:53.360 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-common-2:17.2.7-939.gc190bdee.el8.x86_64 20/25 2024-05-12T03:46:53.619 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-common-2:17.2.7-939.gc190bdee.el8.x86_64 20/25 2024-05-12T03:46:54.245 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-common-2:17.2.7-939.gc190bdee.el8.x86_64 20/25 2024-05-12T03:46:54.520 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-base-2:17.2.7-939.gc190bdee.el8.x86_64 21/25 2024-05-12T03:46:54.634 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-base-2:17.2.7-939.gc190bdee.el8.x86_64 21/25 2024-05-12T03:46:54.847 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-selinux-2:17.2.7-939.gc190bdee.el8.x86_64 22/25 2024-05-12T03:47:08.455 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-selinux-2:17.2.7-939.gc190bdee.el8.x86_64 22/25 2024-05-12T03:47:08.455 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /sys 2024-05-12T03:47:08.455 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /proc 2024-05-12T03:47:08.455 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /dev 2024-05-12T03:47:08.455 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /run 2024-05-12T03:47:08.455 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /mnt 2024-05-12T03:47:08.455 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /var/tmp 2024-05-12T03:47:08.456 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /home 2024-05-12T03:47:08.456 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /tmp 2024-05-12T03:47:08.456 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /dev 2024-05-12T03:47:08.456 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:47:10.286 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-radosgw-2:17.2.7-939.gc190bdee.el8.x86_64 23/25 2024-05-12T03:47:10.397 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-939.gc190bdee.el8.x86_64 23/25 2024-05-12T03:47:10.397 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-05-12T03:47:10.651 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-05-12T03:47:10.651 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2024-05-12T03:47:11.398 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2024-05-12T03:47:11.398 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-base-2:17.2.7-939.gc190bdee.el8.x86_64 1/25 2024-05-12T03:47:11.398 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-common-2:17.2.7-939.gc190bdee.el8.x86_64 2/25 2024-05-12T03:47:11.398 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-radosgw-2:17.2.7-939.gc190bdee.el8.x86_64 3/25 2024-05-12T03:47:11.398 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-selinux-2:17.2.7-939.gc190bdee.el8.x86_64 4/25 2024-05-12T03:47:11.398 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libcephfs2-2:17.2.7-939.gc190bdee.el8.x86_64 5/25 2024-05-12T03:47:11.398 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libradosstriper1-2:17.2.7-939.gc190bdee.el8.x86_64 6/25 2024-05-12T03:47:11.398 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librgw2-2:17.2.7-939.gc190bdee.el8.x86_64 7/25 2024-05-12T03:47:11.399 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-ceph-argparse-2:17.2.7-939.gc190bdee.el8.x 8/25 2024-05-12T03:47:11.399 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-ceph-common-2:17.2.7-939.gc190bdee.el8.x86 9/25 2024-05-12T03:47:11.399 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cephfs-2:17.2.7-939.gc190bdee.el8.x86_64 10/25 2024-05-12T03:47:11.399 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rados-2:17.2.7-939.gc190bdee.el8.x86_64 11/25 2024-05-12T03:47:11.399 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rbd-2:17.2.7-939.gc190bdee.el8.x86_64 12/25 2024-05-12T03:47:11.399 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rgw-2:17.2.7-939.gc190bdee.el8.x86_64 13/25 2024-05-12T03:47:11.399 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librdkafka-1.6.1-1.el8.x86_64 14/25 2024-05-12T03:47:11.399 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 15/25 2024-05-12T03:47:11.399 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 16/25 2024-05-12T03:47:11.399 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librabbitmq-0.9.0-5.el8.x86_64 17/25 2024-05-12T03:47:11.399 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 18/25 2024-05-12T03:47:11.400 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 19/25 2024-05-12T03:47:11.400 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 20/25 2024-05-12T03:47:11.400 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 21/25 2024-05-12T03:47:11.400 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librados2-2:17.2.7-939.gc190bdee.el8.x86_64 22/25 2024-05-12T03:47:11.400 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2024-05-12T03:47:11.400 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librbd1-2:17.2.7-939.gc190bdee.el8.x86_64 24/25 2024-05-12T03:47:12.322 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2024-05-12T03:47:12.323 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:47:12.323 INFO:teuthology.orchestra.run.smithi033.stdout:Upgraded: 2024-05-12T03:47:12.323 INFO:teuthology.orchestra.run.smithi033.stdout: librados2-2:17.2.7-939.gc190bdee.el8.x86_64 2024-05-12T03:47:12.323 INFO:teuthology.orchestra.run.smithi033.stdout: librbd1-2:17.2.7-939.gc190bdee.el8.x86_64 2024-05-12T03:47:12.323 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2024-05-12T03:47:12.323 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-base-2:17.2.7-939.gc190bdee.el8.x86_64 2024-05-12T03:47:12.323 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-common-2:17.2.7-939.gc190bdee.el8.x86_64 2024-05-12T03:47:12.323 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-radosgw-2:17.2.7-939.gc190bdee.el8.x86_64 2024-05-12T03:47:12.323 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-selinux-2:17.2.7-939.gc190bdee.el8.x86_64 2024-05-12T03:47:12.323 INFO:teuthology.orchestra.run.smithi033.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-05-12T03:47:12.323 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs2-2:17.2.7-939.gc190bdee.el8.x86_64 2024-05-12T03:47:12.324 INFO:teuthology.orchestra.run.smithi033.stdout: liboath-2.6.2-3.el8.x86_64 2024-05-12T03:47:12.324 INFO:teuthology.orchestra.run.smithi033.stdout: librabbitmq-0.9.0-5.el8.x86_64 2024-05-12T03:47:12.324 INFO:teuthology.orchestra.run.smithi033.stdout: libradosstriper1-2:17.2.7-939.gc190bdee.el8.x86_64 2024-05-12T03:47:12.324 INFO:teuthology.orchestra.run.smithi033.stdout: librdkafka-1.6.1-1.el8.x86_64 2024-05-12T03:47:12.324 INFO:teuthology.orchestra.run.smithi033.stdout: librgw2-2:17.2.7-939.gc190bdee.el8.x86_64 2024-05-12T03:47:12.324 INFO:teuthology.orchestra.run.smithi033.stdout: libunwind-1.3.1-3.el8.x86_64 2024-05-12T03:47:12.324 INFO:teuthology.orchestra.run.smithi033.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-05-12T03:47:12.324 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-argparse-2:17.2.7-939.gc190bdee.el8.x86_64 2024-05-12T03:47:12.324 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-common-2:17.2.7-939.gc190bdee.el8.x86_64 2024-05-12T03:47:12.324 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cephfs-2:17.2.7-939.gc190bdee.el8.x86_64 2024-05-12T03:47:12.324 INFO:teuthology.orchestra.run.smithi033.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-05-12T03:47:12.324 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rados-2:17.2.7-939.gc190bdee.el8.x86_64 2024-05-12T03:47:12.325 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rbd-2:17.2.7-939.gc190bdee.el8.x86_64 2024-05-12T03:47:12.325 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rgw-2:17.2.7-939.gc190bdee.el8.x86_64 2024-05-12T03:47:12.325 INFO:teuthology.orchestra.run.smithi033.stdout: thrift-0.13.0-2.el8.x86_64 2024-05-12T03:47:12.325 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:47:12.325 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-05-12T03:47:12.890 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum -y install ceph-test 2024-05-12T03:47:13.647 INFO:teuthology.orchestra.run.smithi033.stdout:Last metadata expiration check: 0:00:46 ago on Sun 12 May 2024 03:46:27 AM UTC. 2024-05-12T03:47:15.143 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-selinux-2:17.2.7-939.gc190bdee.el8.x86_64 22/25 2024-05-12T03:47:15.143 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /sys 2024-05-12T03:47:15.143 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /proc 2024-05-12T03:47:15.143 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /dev 2024-05-12T03:47:15.143 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /run 2024-05-12T03:47:15.143 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /mnt 2024-05-12T03:47:15.143 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /var/tmp 2024-05-12T03:47:15.143 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /home 2024-05-12T03:47:15.143 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /tmp 2024-05-12T03:47:15.143 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /dev 2024-05-12T03:47:15.144 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:47:15.421 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-05-12T03:47:15.422 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-05-12T03:47:15.422 INFO:teuthology.orchestra.run.smithi033.stdout: Package Arch Version Repository Size 2024-05-12T03:47:15.422 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-05-12T03:47:15.422 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2024-05-12T03:47:15.422 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-test x86_64 2:17.2.7-939.gc190bdee.el8 ceph 53 M 2024-05-12T03:47:15.422 INFO:teuthology.orchestra.run.smithi033.stdout:Installing dependencies: 2024-05-12T03:47:15.423 INFO:teuthology.orchestra.run.smithi033.stdout: jq x86_64 1.6-8.el8 appstream 203 k 2024-05-12T03:47:15.423 INFO:teuthology.orchestra.run.smithi033.stdout: libcephsqlite x86_64 2:17.2.7-939.gc190bdee.el8 ceph 178 k 2024-05-12T03:47:15.423 INFO:teuthology.orchestra.run.smithi033.stdout: oniguruma x86_64 6.8.2-3.el8 appstream 188 k 2024-05-12T03:47:15.423 INFO:teuthology.orchestra.run.smithi033.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2024-05-12T03:47:15.423 INFO:teuthology.orchestra.run.smithi033.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2024-05-12T03:47:15.423 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:47:15.423 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2024-05-12T03:47:15.423 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-05-12T03:47:15.423 INFO:teuthology.orchestra.run.smithi033.stdout:Install 6 Packages 2024-05-12T03:47:15.423 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:47:15.424 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 54 M 2024-05-12T03:47:15.424 INFO:teuthology.orchestra.run.smithi033.stdout:Installed size: 236 M 2024-05-12T03:47:15.424 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading Packages: 2024-05-12T03:47:15.651 INFO:teuthology.orchestra.run.smithi033.stdout:(1/6): jq-1.6-8.el8.x86_64.rpm 1.1 MB/s | 203 kB 00:00 2024-05-12T03:47:15.677 INFO:teuthology.orchestra.run.smithi033.stdout:(2/6): libcephsqlite-17.2.7-939.gc190bdee.el8.x 895 kB/s | 178 kB 00:00 2024-05-12T03:47:15.735 INFO:teuthology.orchestra.run.smithi033.stdout:(3/6): oniguruma-6.8.2-3.el8.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-05-12T03:47:15.786 INFO:teuthology.orchestra.run.smithi033.stdout:(4/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 71 kB 00:00 2024-05-12T03:47:15.894 INFO:teuthology.orchestra.run.smithi033.stdout:(5/6): socat-1.7.4.1-1.el8.x86_64.rpm 1.5 MB/s | 323 kB 00:00 2024-05-12T03:47:16.927 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-radosgw-2:17.2.7-939.gc190bdee.el8.x86_64 23/25 2024-05-12T03:47:17.004 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-939.gc190bdee.el8.x86_64 23/25 2024-05-12T03:47:17.004 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-05-12T03:47:17.251 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-05-12T03:47:17.251 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2024-05-12T03:47:17.974 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2024-05-12T03:47:17.974 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-base-2:17.2.7-939.gc190bdee.el8.x86_64 1/25 2024-05-12T03:47:17.974 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-common-2:17.2.7-939.gc190bdee.el8.x86_64 2/25 2024-05-12T03:47:17.974 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-radosgw-2:17.2.7-939.gc190bdee.el8.x86_64 3/25 2024-05-12T03:47:17.974 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-selinux-2:17.2.7-939.gc190bdee.el8.x86_64 4/25 2024-05-12T03:47:17.975 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libcephfs2-2:17.2.7-939.gc190bdee.el8.x86_64 5/25 2024-05-12T03:47:17.975 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libradosstriper1-2:17.2.7-939.gc190bdee.el8.x86_64 6/25 2024-05-12T03:47:17.975 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librgw2-2:17.2.7-939.gc190bdee.el8.x86_64 7/25 2024-05-12T03:47:17.975 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-ceph-argparse-2:17.2.7-939.gc190bdee.el8.x 8/25 2024-05-12T03:47:17.975 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-ceph-common-2:17.2.7-939.gc190bdee.el8.x86 9/25 2024-05-12T03:47:17.975 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cephfs-2:17.2.7-939.gc190bdee.el8.x86_64 10/25 2024-05-12T03:47:17.975 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-rados-2:17.2.7-939.gc190bdee.el8.x86_64 11/25 2024-05-12T03:47:17.975 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-rbd-2:17.2.7-939.gc190bdee.el8.x86_64 12/25 2024-05-12T03:47:17.975 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-rgw-2:17.2.7-939.gc190bdee.el8.x86_64 13/25 2024-05-12T03:47:17.975 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librdkafka-1.6.1-1.el8.x86_64 14/25 2024-05-12T03:47:17.975 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 15/25 2024-05-12T03:47:17.975 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 16/25 2024-05-12T03:47:17.975 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librabbitmq-0.9.0-5.el8.x86_64 17/25 2024-05-12T03:47:17.975 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 18/25 2024-05-12T03:47:17.976 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 19/25 2024-05-12T03:47:17.976 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 20/25 2024-05-12T03:47:17.976 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 21/25 2024-05-12T03:47:17.976 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librados2-2:17.2.7-939.gc190bdee.el8.x86_64 22/25 2024-05-12T03:47:17.976 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2024-05-12T03:47:17.976 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librbd1-2:17.2.7-939.gc190bdee.el8.x86_64 24/25 2024-05-12T03:47:18.169 INFO:teuthology.orchestra.run.smithi033.stdout:(6/6): ceph-test-17.2.7-939.gc190bdee.el8.x86_6 20 MB/s | 53 MB 00:02 2024-05-12T03:47:18.171 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2024-05-12T03:47:18.171 INFO:teuthology.orchestra.run.smithi033.stdout:Total 19 MB/s | 54 MB 00:02 2024-05-12T03:47:18.214 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2024-05-12T03:47:18.237 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction check succeeded. 2024-05-12T03:47:18.237 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2024-05-12T03:47:18.542 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded. 2024-05-12T03:47:18.545 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2024-05-12T03:47:18.931 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2024-05-12T03:47:18.931 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:47:18.931 INFO:teuthology.orchestra.run.smithi032.stdout:Upgraded: 2024-05-12T03:47:18.931 INFO:teuthology.orchestra.run.smithi032.stdout: librados2-2:17.2.7-939.gc190bdee.el8.x86_64 2024-05-12T03:47:18.931 INFO:teuthology.orchestra.run.smithi032.stdout: librbd1-2:17.2.7-939.gc190bdee.el8.x86_64 2024-05-12T03:47:18.931 INFO:teuthology.orchestra.run.smithi032.stdout:Installed: 2024-05-12T03:47:18.932 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-base-2:17.2.7-939.gc190bdee.el8.x86_64 2024-05-12T03:47:18.932 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-common-2:17.2.7-939.gc190bdee.el8.x86_64 2024-05-12T03:47:18.932 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-radosgw-2:17.2.7-939.gc190bdee.el8.x86_64 2024-05-12T03:47:18.932 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-selinux-2:17.2.7-939.gc190bdee.el8.x86_64 2024-05-12T03:47:18.932 INFO:teuthology.orchestra.run.smithi032.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-05-12T03:47:18.932 INFO:teuthology.orchestra.run.smithi032.stdout: libcephfs2-2:17.2.7-939.gc190bdee.el8.x86_64 2024-05-12T03:47:18.932 INFO:teuthology.orchestra.run.smithi032.stdout: liboath-2.6.2-3.el8.x86_64 2024-05-12T03:47:18.932 INFO:teuthology.orchestra.run.smithi032.stdout: librabbitmq-0.9.0-5.el8.x86_64 2024-05-12T03:47:18.932 INFO:teuthology.orchestra.run.smithi032.stdout: libradosstriper1-2:17.2.7-939.gc190bdee.el8.x86_64 2024-05-12T03:47:18.932 INFO:teuthology.orchestra.run.smithi032.stdout: librdkafka-1.6.1-1.el8.x86_64 2024-05-12T03:47:18.932 INFO:teuthology.orchestra.run.smithi032.stdout: librgw2-2:17.2.7-939.gc190bdee.el8.x86_64 2024-05-12T03:47:18.932 INFO:teuthology.orchestra.run.smithi032.stdout: libunwind-1.3.1-3.el8.x86_64 2024-05-12T03:47:18.932 INFO:teuthology.orchestra.run.smithi032.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-05-12T03:47:18.932 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ceph-argparse-2:17.2.7-939.gc190bdee.el8.x86_64 2024-05-12T03:47:18.933 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ceph-common-2:17.2.7-939.gc190bdee.el8.x86_64 2024-05-12T03:47:18.933 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cephfs-2:17.2.7-939.gc190bdee.el8.x86_64 2024-05-12T03:47:18.933 INFO:teuthology.orchestra.run.smithi032.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-05-12T03:47:18.933 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rados-2:17.2.7-939.gc190bdee.el8.x86_64 2024-05-12T03:47:18.933 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rbd-2:17.2.7-939.gc190bdee.el8.x86_64 2024-05-12T03:47:18.933 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rgw-2:17.2.7-939.gc190bdee.el8.x86_64 2024-05-12T03:47:18.933 INFO:teuthology.orchestra.run.smithi032.stdout: thrift-0.13.0-2.el8.x86_64 2024-05-12T03:47:18.933 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:47:18.933 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2024-05-12T03:47:19.092 INFO:teuthology.orchestra.run.smithi033.stdout: Preparing : 1/1 2024-05-12T03:47:19.278 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum -y install ceph-test 2024-05-12T03:47:19.294 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 1/6 2024-05-12T03:47:19.459 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 2/6 2024-05-12T03:47:19.630 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : oniguruma-6.8.2-3.el8.x86_64 3/6 2024-05-12T03:47:19.708 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: oniguruma-6.8.2-3.el8.x86_64 3/6 2024-05-12T03:47:19.922 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : jq-1.6-8.el8.x86_64 4/6 2024-05-12T03:47:20.034 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:00:48 ago on Sun 12 May 2024 03:46:32 AM UTC. 2024-05-12T03:47:20.114 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libcephsqlite-2:17.2.7-939.gc190bdee.el8.x86_64 5/6 2024-05-12T03:47:21.802 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-05-12T03:47:21.803 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-05-12T03:47:21.803 INFO:teuthology.orchestra.run.smithi032.stdout: Package Arch Version Repository Size 2024-05-12T03:47:21.803 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-05-12T03:47:21.803 INFO:teuthology.orchestra.run.smithi032.stdout:Installing: 2024-05-12T03:47:21.803 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-test x86_64 2:17.2.7-939.gc190bdee.el8 ceph 53 M 2024-05-12T03:47:21.803 INFO:teuthology.orchestra.run.smithi032.stdout:Installing dependencies: 2024-05-12T03:47:21.803 INFO:teuthology.orchestra.run.smithi032.stdout: jq x86_64 1.6-8.el8 appstream 203 k 2024-05-12T03:47:21.803 INFO:teuthology.orchestra.run.smithi032.stdout: libcephsqlite x86_64 2:17.2.7-939.gc190bdee.el8 ceph 178 k 2024-05-12T03:47:21.804 INFO:teuthology.orchestra.run.smithi032.stdout: oniguruma x86_64 6.8.2-3.el8 appstream 188 k 2024-05-12T03:47:21.804 INFO:teuthology.orchestra.run.smithi032.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2024-05-12T03:47:21.804 INFO:teuthology.orchestra.run.smithi032.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2024-05-12T03:47:21.804 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:47:21.804 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction Summary 2024-05-12T03:47:21.804 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-05-12T03:47:21.804 INFO:teuthology.orchestra.run.smithi032.stdout:Install 6 Packages 2024-05-12T03:47:21.804 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:47:21.805 INFO:teuthology.orchestra.run.smithi032.stdout:Total download size: 54 M 2024-05-12T03:47:21.805 INFO:teuthology.orchestra.run.smithi032.stdout:Installed size: 236 M 2024-05-12T03:47:21.805 INFO:teuthology.orchestra.run.smithi032.stdout:Downloading Packages: 2024-05-12T03:47:22.027 INFO:teuthology.orchestra.run.smithi032.stdout:(1/6): jq-1.6-8.el8.x86_64.rpm 1.2 MB/s | 203 kB 00:00 2024-05-12T03:47:22.053 INFO:teuthology.orchestra.run.smithi032.stdout:(2/6): libcephsqlite-17.2.7-939.gc190bdee.el8.x 916 kB/s | 178 kB 00:00 2024-05-12T03:47:22.078 INFO:teuthology.orchestra.run.smithi032.stdout:(3/6): oniguruma-6.8.2-3.el8.x86_64.rpm 3.6 MB/s | 188 kB 00:00 2024-05-12T03:47:22.122 INFO:teuthology.orchestra.run.smithi032.stdout:(4/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.6 MB/s | 71 kB 00:00 2024-05-12T03:47:22.239 INFO:teuthology.orchestra.run.smithi032.stdout:(5/6): socat-1.7.4.1-1.el8.x86_64.rpm 1.7 MB/s | 323 kB 00:00 2024-05-12T03:47:24.568 INFO:teuthology.orchestra.run.smithi032.stdout:(6/6): ceph-test-17.2.7-939.gc190bdee.el8.x86_6 19 MB/s | 53 MB 00:02 2024-05-12T03:47:24.571 INFO:teuthology.orchestra.run.smithi032.stdout:-------------------------------------------------------------------------------- 2024-05-12T03:47:24.571 INFO:teuthology.orchestra.run.smithi032.stdout:Total 19 MB/s | 54 MB 00:02 2024-05-12T03:47:24.614 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction check 2024-05-12T03:47:24.637 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction check succeeded. 2024-05-12T03:47:24.637 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction test 2024-05-12T03:47:24.943 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction test succeeded. 2024-05-12T03:47:24.947 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction 2024-05-12T03:47:25.441 INFO:teuthology.orchestra.run.smithi032.stdout: Preparing : 1/1 2024-05-12T03:47:25.556 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libcephsqlite-2:17.2.7-939.gc190bdee.el8.x86_64 5/6 2024-05-12T03:47:25.664 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 1/6 2024-05-12T03:47:25.853 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-test-2:17.2.7-939.gc190bdee.el8.x86_64 6/6 2024-05-12T03:47:25.867 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 2/6 2024-05-12T03:47:26.014 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : oniguruma-6.8.2-3.el8.x86_64 3/6 2024-05-12T03:47:26.095 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: oniguruma-6.8.2-3.el8.x86_64 3/6 2024-05-12T03:47:26.286 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-test-2:17.2.7-939.gc190bdee.el8.x86_64 6/6 2024-05-12T03:47:26.286 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-test-2:17.2.7-939.gc190bdee.el8.x86_64 1/6 2024-05-12T03:47:26.286 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libcephsqlite-2:17.2.7-939.gc190bdee.el8.x86_64 2/6 2024-05-12T03:47:26.287 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : jq-1.6-8.el8.x86_64 3/6 2024-05-12T03:47:26.287 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : oniguruma-6.8.2-3.el8.x86_64 4/6 2024-05-12T03:47:26.287 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 5/6 2024-05-12T03:47:26.322 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : jq-1.6-8.el8.x86_64 4/6 2024-05-12T03:47:26.589 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libcephsqlite-2:17.2.7-939.gc190bdee.el8.x86_64 5/6 2024-05-12T03:47:26.794 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 6/6 2024-05-12T03:47:26.794 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:47:26.795 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2024-05-12T03:47:26.795 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-test-2:17.2.7-939.gc190bdee.el8.x86_64 2024-05-12T03:47:26.795 INFO:teuthology.orchestra.run.smithi033.stdout: jq-1.6-8.el8.x86_64 2024-05-12T03:47:26.795 INFO:teuthology.orchestra.run.smithi033.stdout: libcephsqlite-2:17.2.7-939.gc190bdee.el8.x86_64 2024-05-12T03:47:26.795 INFO:teuthology.orchestra.run.smithi033.stdout: oniguruma-6.8.2-3.el8.x86_64 2024-05-12T03:47:26.795 INFO:teuthology.orchestra.run.smithi033.stdout: socat-1.7.4.1-1.el8.x86_64 2024-05-12T03:47:26.795 INFO:teuthology.orchestra.run.smithi033.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-05-12T03:47:26.795 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:47:26.795 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-05-12T03:47:27.057 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum -y install ceph 2024-05-12T03:47:27.824 INFO:teuthology.orchestra.run.smithi033.stdout:Last metadata expiration check: 0:01:00 ago on Sun 12 May 2024 03:46:27 AM UTC. 2024-05-12T03:47:29.615 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-05-12T03:47:29.619 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-05-12T03:47:29.619 INFO:teuthology.orchestra.run.smithi033.stdout: Package Arch Version Repository Size 2024-05-12T03:47:29.619 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-05-12T03:47:29.619 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2024-05-12T03:47:29.619 INFO:teuthology.orchestra.run.smithi033.stdout: ceph x86_64 2:17.2.7-939.gc190bdee.el8 ceph 6.4 k 2024-05-12T03:47:29.619 INFO:teuthology.orchestra.run.smithi033.stdout:Installing dependencies: 2024-05-12T03:47:29.619 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mds x86_64 2:17.2.7-939.gc190bdee.el8 ceph 2.3 M 2024-05-12T03:47:29.619 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr x86_64 2:17.2.7-939.gc190bdee.el8 ceph 1.6 M 2024-05-12T03:47:29.619 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-modules-core noarch 2:17.2.7-939.gc190bdee.el8 ceph-noarch 240 k 2024-05-12T03:47:29.619 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mon x86_64 2:17.2.7-939.gc190bdee.el8 ceph 4.0 M 2024-05-12T03:47:29.620 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-osd x86_64 2:17.2.7-939.gc190bdee.el8 ceph 17 M 2024-05-12T03:47:29.620 INFO:teuthology.orchestra.run.smithi033.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-05-12T03:47:29.620 INFO:teuthology.orchestra.run.smithi033.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-05-12T03:47:29.620 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2024-05-12T03:47:29.620 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-05-12T03:47:29.620 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-05-12T03:47:29.620 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cryptography x86_64 3.2.1-7.el8 baseos 641 k 2024-05-12T03:47:29.620 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-05-12T03:47:29.620 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-05-12T03:47:29.620 INFO:teuthology.orchestra.run.smithi033.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-05-12T03:47:29.620 INFO:teuthology.orchestra.run.smithi033.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2024-05-12T03:47:29.620 INFO:teuthology.orchestra.run.smithi033.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2024-05-12T03:47:29.621 INFO:teuthology.orchestra.run.smithi033.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-05-12T03:47:29.621 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-05-12T03:47:29.621 INFO:teuthology.orchestra.run.smithi033.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-05-12T03:47:29.621 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2024-05-12T03:47:29.621 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2024-05-12T03:47:29.621 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pytz noarch 2017.2-11.el8 appstream 57 k 2024-05-12T03:47:29.621 INFO:teuthology.orchestra.run.smithi033.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-05-12T03:47:29.621 INFO:teuthology.orchestra.run.smithi033.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-05-12T03:47:29.621 INFO:teuthology.orchestra.run.smithi033.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-05-12T03:47:29.621 INFO:teuthology.orchestra.run.smithi033.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-05-12T03:47:29.621 INFO:teuthology.orchestra.run.smithi033.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-05-12T03:47:29.622 INFO:teuthology.orchestra.run.smithi033.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-05-12T03:47:29.622 INFO:teuthology.orchestra.run.smithi033.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-05-12T03:47:29.622 INFO:teuthology.orchestra.run.smithi033.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2024-05-12T03:47:29.622 INFO:teuthology.orchestra.run.smithi033.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-05-12T03:47:29.622 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:47:29.622 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2024-05-12T03:47:29.622 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-05-12T03:47:29.622 INFO:teuthology.orchestra.run.smithi033.stdout:Install 32 Packages 2024-05-12T03:47:29.622 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:47:29.623 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 29 M 2024-05-12T03:47:29.623 INFO:teuthology.orchestra.run.smithi033.stdout:Installed size: 113 M 2024-05-12T03:47:29.623 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading Packages: 2024-05-12T03:47:29.888 INFO:teuthology.orchestra.run.smithi033.stdout:(1/32): ceph-17.2.7-939.gc190bdee.el8.x86_64.rp 49 kB/s | 6.4 kB 00:00 2024-05-12T03:47:30.230 INFO:teuthology.orchestra.run.smithi033.stdout:(2/32): ceph-mon-17.2.7-939.gc190bdee.el8.x86_6 12 MB/s | 4.0 MB 00:00 2024-05-12T03:47:30.631 INFO:teuthology.orchestra.run.smithi033.stdout:(3/32): ceph-mds-17.2.7-939.gc190bdee.el8.x86_6 2.6 MB/s | 2.3 MB 00:00 2024-05-12T03:47:30.689 INFO:teuthology.orchestra.run.smithi033.stdout:(4/32): ceph-mgr-17.2.7-939.gc190bdee.el8.x86_6 1.7 MB/s | 1.6 MB 00:00 2024-05-12T03:47:30.739 INFO:teuthology.orchestra.run.smithi033.stdout:(5/32): ceph-mgr-modules-core-17.2.7-939.gc190b 2.2 MB/s | 240 kB 00:00 2024-05-12T03:47:30.848 INFO:teuthology.orchestra.run.smithi033.stdout:(6/32): python3-markupsafe-0.23-19.el8.x86_64.r 359 kB/s | 39 kB 00:00 2024-05-12T03:47:30.873 INFO:teuthology.orchestra.run.smithi033.stdout:(7/32): python3-mako-1.0.6-14.el8.noarch.rpm 855 kB/s | 157 kB 00:00 2024-05-12T03:47:30.915 INFO:teuthology.orchestra.run.smithi033.stdout:(8/32): python3-pytz-2017.2-11.el8.noarch.rpm 1.3 MB/s | 57 kB 00:00 2024-05-12T03:47:30.940 INFO:teuthology.orchestra.run.smithi033.stdout:(9/32): python3-pyOpenSSL-19.0.0-1.el8.noarch.r 1.1 MB/s | 103 kB 00:00 2024-05-12T03:47:31.141 INFO:teuthology.orchestra.run.smithi033.stdout:(10/32): ceph-osd-17.2.7-939.gc190bdee.el8.x86_ 19 MB/s | 17 MB 00:00 2024-05-12T03:47:31.191 INFO:teuthology.orchestra.run.smithi033.stdout:(11/32): python3-werkzeug-0.12.2-4.el8.noarch.r 1.6 MB/s | 457 kB 00:00 2024-05-12T03:47:31.541 INFO:teuthology.orchestra.run.smithi033.stdout:(12/32): python3-pycparser-2.14-14.el8.noarch.r 312 kB/s | 109 kB 00:00 2024-05-12T03:47:31.566 INFO:teuthology.orchestra.run.smithi033.stdout:(13/32): python3-cffi-1.11.5-6.el8.x86_64.rpm 397 kB/s | 249 kB 00:00 2024-05-12T03:47:31.625 INFO:teuthology.orchestra.run.smithi033.stdout:(14/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 521 kB/s | 44 kB 00:00 2024-05-12T03:47:31.650 INFO:teuthology.orchestra.run.smithi033.stdout:(15/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 2.2 MB/s | 185 kB 00:00 2024-05-12T03:47:31.718 INFO:teuthology.orchestra.run.smithi033.stdout:(16/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 1.8 MB/s | 173 kB 00:00 2024-05-12T03:47:31.751 INFO:teuthology.orchestra.run.smithi033.stdout:(17/32): python3-cherrypy-18.4.0-1.el8.noarch.r 3.7 MB/s | 384 kB 00:00 2024-05-12T03:47:31.776 INFO:teuthology.orchestra.run.smithi033.stdout:(18/32): python3-jaraco-6.2-6.el8.noarch.rpm 191 kB/s | 11 kB 00:00 2024-05-12T03:47:31.802 INFO:teuthology.orchestra.run.smithi033.stdout:(19/32): python3-jaraco-functools-2.0-4.el8.noa 350 kB/s | 18 kB 00:00 2024-05-12T03:47:31.827 INFO:teuthology.orchestra.run.smithi033.stdout:(20/32): python3-logutils-0.3.5-11.el8.noarch.r 972 kB/s | 49 kB 00:00 2024-05-12T03:47:31.852 INFO:teuthology.orchestra.run.smithi033.stdout:(21/32): python3-more-itertools-7.2.0-3.el8.noa 1.1 MB/s | 59 kB 00:00 2024-05-12T03:47:31.877 INFO:teuthology.orchestra.run.smithi033.stdout:(22/32): python3-cryptography-3.2.1-7.el8.x86_6 871 kB/s | 641 kB 00:00 2024-05-12T03:47:31.939 INFO:teuthology.orchestra.run.smithi033.stdout:(23/32): python3-portend-2.6-1.el8.noarch.rpm 180 kB/s | 16 kB 00:00 2024-05-12T03:47:31.970 INFO:teuthology.orchestra.run.smithi033.stdout:(24/32): python3-simplegeneric-0.8.1-17.el8.noa 204 kB/s | 19 kB 00:00 2024-05-12T03:47:31.985 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libcephsqlite-2:17.2.7-939.gc190bdee.el8.x86_64 5/6 2024-05-12T03:47:31.995 INFO:teuthology.orchestra.run.smithi033.stdout:(25/32): python3-pecan-1.3.2-9.el8.noarch.rpm 1.6 MB/s | 283 kB 00:00 2024-05-12T03:47:32.021 INFO:teuthology.orchestra.run.smithi033.stdout:(26/32): python3-singledispatch-3.4.0.3-18.el8. 294 kB/s | 24 kB 00:00 2024-05-12T03:47:32.046 INFO:teuthology.orchestra.run.smithi033.stdout:(27/32): python3-tempora-1.14.1-5.el8.noarch.rp 386 kB/s | 29 kB 00:00 2024-05-12T03:47:32.071 INFO:teuthology.orchestra.run.smithi033.stdout:(28/32): python3-trustme-0.6.0-4.el8.noarch.rpm 365 kB/s | 27 kB 00:00 2024-05-12T03:47:32.096 INFO:teuthology.orchestra.run.smithi033.stdout:(29/32): python3-waitress-1.4.3-1.el8.noarch.rp 5.0 MB/s | 255 kB 00:00 2024-05-12T03:47:32.151 INFO:teuthology.orchestra.run.smithi033.stdout:(30/32): python3-webtest-2.0.33-1.el8.noarch.rp 1.0 MB/s | 85 kB 00:00 2024-05-12T03:47:32.180 INFO:teuthology.orchestra.run.smithi033.stdout:(31/32): python3-zc-lockfile-2.0-2.el8.noarch.r 275 kB/s | 23 kB 00:00 2024-05-12T03:47:32.205 INFO:teuthology.orchestra.run.smithi033.stdout:(32/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 1.5 MB/s | 251 kB 00:00 2024-05-12T03:47:32.207 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2024-05-12T03:47:32.208 INFO:teuthology.orchestra.run.smithi033.stdout:Total 11 MB/s | 29 MB 00:02 2024-05-12T03:47:32.254 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-test-2:17.2.7-939.gc190bdee.el8.x86_64 6/6 2024-05-12T03:47:32.293 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2024-05-12T03:47:32.333 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction check succeeded. 2024-05-12T03:47:32.333 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2024-05-12T03:47:32.549 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-test-2:17.2.7-939.gc190bdee.el8.x86_64 6/6 2024-05-12T03:47:32.549 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-test-2:17.2.7-939.gc190bdee.el8.x86_64 1/6 2024-05-12T03:47:32.549 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libcephsqlite-2:17.2.7-939.gc190bdee.el8.x86_64 2/6 2024-05-12T03:47:32.549 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : jq-1.6-8.el8.x86_64 3/6 2024-05-12T03:47:32.549 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : oniguruma-6.8.2-3.el8.x86_64 4/6 2024-05-12T03:47:32.549 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 5/6 2024-05-12T03:47:32.660 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded. 2024-05-12T03:47:32.663 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2024-05-12T03:47:33.003 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 6/6 2024-05-12T03:47:33.003 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:47:33.003 INFO:teuthology.orchestra.run.smithi032.stdout:Installed: 2024-05-12T03:47:33.003 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-test-2:17.2.7-939.gc190bdee.el8.x86_64 2024-05-12T03:47:33.003 INFO:teuthology.orchestra.run.smithi032.stdout: jq-1.6-8.el8.x86_64 2024-05-12T03:47:33.003 INFO:teuthology.orchestra.run.smithi032.stdout: libcephsqlite-2:17.2.7-939.gc190bdee.el8.x86_64 2024-05-12T03:47:33.004 INFO:teuthology.orchestra.run.smithi032.stdout: oniguruma-6.8.2-3.el8.x86_64 2024-05-12T03:47:33.004 INFO:teuthology.orchestra.run.smithi032.stdout: socat-1.7.4.1-1.el8.x86_64 2024-05-12T03:47:33.004 INFO:teuthology.orchestra.run.smithi032.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-05-12T03:47:33.004 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:47:33.004 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2024-05-12T03:47:33.406 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum -y install ceph 2024-05-12T03:47:33.885 INFO:teuthology.orchestra.run.smithi033.stdout: Preparing : 1/1 2024-05-12T03:47:34.096 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2024-05-12T03:47:34.166 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:01:02 ago on Sun 12 May 2024 03:46:32 AM UTC. 2024-05-12T03:47:34.353 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2024-05-12T03:47:34.558 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/32 2024-05-12T03:47:34.810 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 4/32 2024-05-12T03:47:35.033 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 5/32 2024-05-12T03:47:35.190 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 6/32 2024-05-12T03:47:35.448 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 7/32 2024-05-12T03:47:35.601 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 8/32 2024-05-12T03:47:35.837 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 9/32 2024-05-12T03:47:35.936 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-05-12T03:47:35.940 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-05-12T03:47:35.940 INFO:teuthology.orchestra.run.smithi032.stdout: Package Arch Version Repository Size 2024-05-12T03:47:35.940 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-05-12T03:47:35.940 INFO:teuthology.orchestra.run.smithi032.stdout:Installing: 2024-05-12T03:47:35.940 INFO:teuthology.orchestra.run.smithi032.stdout: ceph x86_64 2:17.2.7-939.gc190bdee.el8 ceph 6.4 k 2024-05-12T03:47:35.940 INFO:teuthology.orchestra.run.smithi032.stdout:Installing dependencies: 2024-05-12T03:47:35.940 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mds x86_64 2:17.2.7-939.gc190bdee.el8 ceph 2.3 M 2024-05-12T03:47:35.940 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr x86_64 2:17.2.7-939.gc190bdee.el8 ceph 1.6 M 2024-05-12T03:47:35.941 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-modules-core noarch 2:17.2.7-939.gc190bdee.el8 ceph-noarch 240 k 2024-05-12T03:47:35.941 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mon x86_64 2:17.2.7-939.gc190bdee.el8 ceph 4.0 M 2024-05-12T03:47:35.941 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-osd x86_64 2:17.2.7-939.gc190bdee.el8 ceph 17 M 2024-05-12T03:47:35.941 INFO:teuthology.orchestra.run.smithi032.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-05-12T03:47:35.941 INFO:teuthology.orchestra.run.smithi032.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-05-12T03:47:35.941 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2024-05-12T03:47:35.941 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-05-12T03:47:35.941 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-05-12T03:47:35.941 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cryptography x86_64 3.2.1-7.el8 baseos 641 k 2024-05-12T03:47:35.941 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-05-12T03:47:35.941 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-05-12T03:47:35.941 INFO:teuthology.orchestra.run.smithi032.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-05-12T03:47:35.941 INFO:teuthology.orchestra.run.smithi032.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2024-05-12T03:47:35.941 INFO:teuthology.orchestra.run.smithi032.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2024-05-12T03:47:35.941 INFO:teuthology.orchestra.run.smithi032.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-05-12T03:47:35.941 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-05-12T03:47:35.942 INFO:teuthology.orchestra.run.smithi032.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-05-12T03:47:35.942 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2024-05-12T03:47:35.942 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2024-05-12T03:47:35.942 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pytz noarch 2017.2-11.el8 appstream 57 k 2024-05-12T03:47:35.942 INFO:teuthology.orchestra.run.smithi032.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-05-12T03:47:35.942 INFO:teuthology.orchestra.run.smithi032.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-05-12T03:47:35.942 INFO:teuthology.orchestra.run.smithi032.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-05-12T03:47:35.942 INFO:teuthology.orchestra.run.smithi032.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-05-12T03:47:35.942 INFO:teuthology.orchestra.run.smithi032.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-05-12T03:47:35.942 INFO:teuthology.orchestra.run.smithi032.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-05-12T03:47:35.942 INFO:teuthology.orchestra.run.smithi032.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-05-12T03:47:35.942 INFO:teuthology.orchestra.run.smithi032.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2024-05-12T03:47:35.942 INFO:teuthology.orchestra.run.smithi032.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-05-12T03:47:35.942 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:47:35.943 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction Summary 2024-05-12T03:47:35.943 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-05-12T03:47:35.943 INFO:teuthology.orchestra.run.smithi032.stdout:Install 32 Packages 2024-05-12T03:47:35.943 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:47:35.943 INFO:teuthology.orchestra.run.smithi032.stdout:Total download size: 29 M 2024-05-12T03:47:35.943 INFO:teuthology.orchestra.run.smithi032.stdout:Installed size: 113 M 2024-05-12T03:47:35.943 INFO:teuthology.orchestra.run.smithi032.stdout:Downloading Packages: 2024-05-12T03:47:36.115 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 10/32 2024-05-12T03:47:36.283 INFO:teuthology.orchestra.run.smithi032.stdout:(1/32): ceph-17.2.7-939.gc190bdee.el8.x86_64.rp 33 kB/s | 6.4 kB 00:00 2024-05-12T03:47:36.381 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 11/32 2024-05-12T03:47:36.442 INFO:teuthology.orchestra.run.smithi032.stdout:(2/32): ceph-mgr-17.2.7-939.gc190bdee.el8.x86_6 4.5 MB/s | 1.6 MB 00:00 2024-05-12T03:47:36.645 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 12/32 2024-05-12T03:47:36.709 INFO:teuthology.orchestra.run.smithi032.stdout:(3/32): ceph-mds-17.2.7-939.gc190bdee.el8.x86_6 3.7 MB/s | 2.3 MB 00:00 2024-05-12T03:47:36.809 INFO:teuthology.orchestra.run.smithi032.stdout:(4/32): ceph-mgr-modules-core-17.2.7-939.gc190b 2.4 MB/s | 240 kB 00:00 2024-05-12T03:47:36.976 INFO:teuthology.orchestra.run.smithi032.stdout:(5/32): python3-mako-1.0.6-14.el8.noarch.rpm 942 kB/s | 157 kB 00:00 2024-05-12T03:47:37.034 INFO:teuthology.orchestra.run.smithi032.stdout:(6/32): ceph-mon-17.2.7-939.gc190bdee.el8.x86_6 5.4 MB/s | 4.0 MB 00:00 2024-05-12T03:47:37.047 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 13/32 2024-05-12T03:47:37.060 INFO:teuthology.orchestra.run.smithi032.stdout:(7/32): python3-markupsafe-0.23-19.el8.x86_64.r 465 kB/s | 39 kB 00:00 2024-05-12T03:47:37.110 INFO:teuthology.orchestra.run.smithi032.stdout:(8/32): python3-pytz-2017.2-11.el8.noarch.rpm 1.1 MB/s | 57 kB 00:00 2024-05-12T03:47:37.185 INFO:teuthology.orchestra.run.smithi032.stdout:(9/32): python3-pyOpenSSL-19.0.0-1.el8.noarch.r 687 kB/s | 103 kB 00:00 2024-05-12T03:47:37.211 INFO:teuthology.orchestra.run.smithi032.stdout:(10/32): python3-werkzeug-0.12.2-4.el8.noarch.r 4.5 MB/s | 457 kB 00:00 2024-05-12T03:47:37.315 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cryptography-3.2.1-7.el8.x86_64 14/32 2024-05-12T03:47:37.511 INFO:teuthology.orchestra.run.smithi032.stdout:(11/32): ceph-osd-17.2.7-939.gc190bdee.el8.x86_ 16 MB/s | 17 MB 00:01 2024-05-12T03:47:37.519 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 15/32 2024-05-12T03:47:37.678 INFO:teuthology.orchestra.run.smithi032.stdout:(12/32): python3-cffi-1.11.5-6.el8.x86_64.rpm 505 kB/s | 249 kB 00:00 2024-05-12T03:47:37.762 INFO:teuthology.orchestra.run.smithi032.stdout:(13/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 524 kB/s | 44 kB 00:00 2024-05-12T03:47:37.825 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 16/32 2024-05-12T03:47:37.829 INFO:teuthology.orchestra.run.smithi032.stdout:(14/32): python3-cryptography-3.2.1-7.el8.x86_6 1.0 MB/s | 641 kB 00:00 2024-05-12T03:47:37.854 INFO:teuthology.orchestra.run.smithi032.stdout:(15/32): python3-pycparser-2.14-14.el8.noarch.r 318 kB/s | 109 kB 00:00 2024-05-12T03:47:37.880 INFO:teuthology.orchestra.run.smithi032.stdout:(16/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 1.5 MB/s | 185 kB 00:00 2024-05-12T03:47:37.913 INFO:teuthology.orchestra.run.smithi032.stdout:(17/32): python3-jaraco-6.2-6.el8.noarch.rpm 334 kB/s | 11 kB 00:00 2024-05-12T03:47:37.955 INFO:teuthology.orchestra.run.smithi032.stdout:(18/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 1.3 MB/s | 173 kB 00:00 2024-05-12T03:47:37.972 INFO:teuthology.orchestra.run.smithi032.stdout:(19/32): python3-jaraco-functools-2.0-4.el8.noa 300 kB/s | 18 kB 00:00 2024-05-12T03:47:37.997 INFO:teuthology.orchestra.run.smithi032.stdout:(20/32): python3-logutils-0.3.5-11.el8.noarch.r 1.1 MB/s | 49 kB 00:00 2024-05-12T03:47:38.022 INFO:teuthology.orchestra.run.smithi032.stdout:(21/32): python3-more-itertools-7.2.0-3.el8.noa 1.1 MB/s | 59 kB 00:00 2024-05-12T03:47:38.048 INFO:teuthology.orchestra.run.smithi032.stdout:(22/32): python3-cherrypy-18.4.0-1.el8.noarch.r 1.9 MB/s | 384 kB 00:00 2024-05-12T03:47:38.055 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 17/32 2024-05-12T03:47:38.073 INFO:teuthology.orchestra.run.smithi032.stdout:(23/32): python3-portend-2.6-1.el8.noarch.rpm 310 kB/s | 16 kB 00:00 2024-05-12T03:47:38.099 INFO:teuthology.orchestra.run.smithi032.stdout:(24/32): python3-simplegeneric-0.8.1-17.el8.noa 372 kB/s | 19 kB 00:00 2024-05-12T03:47:38.124 INFO:teuthology.orchestra.run.smithi032.stdout:(25/32): python3-singledispatch-3.4.0.3-18.el8. 476 kB/s | 24 kB 00:00 2024-05-12T03:47:38.149 INFO:teuthology.orchestra.run.smithi032.stdout:(26/32): python3-tempora-1.14.1-5.el8.noarch.rp 584 kB/s | 29 kB 00:00 2024-05-12T03:47:38.176 INFO:teuthology.orchestra.run.smithi032.stdout:(27/32): python3-trustme-0.6.0-4.el8.noarch.rpm 550 kB/s | 27 kB 00:00 2024-05-12T03:47:38.200 INFO:teuthology.orchestra.run.smithi032.stdout:(28/32): python3-pecan-1.3.2-9.el8.noarch.rpm 1.4 MB/s | 283 kB 00:00 2024-05-12T03:47:38.233 INFO:teuthology.orchestra.run.smithi032.stdout:(29/32): python3-waitress-1.4.3-1.el8.noarch.rp 3.0 MB/s | 255 kB 00:00 2024-05-12T03:47:38.284 INFO:teuthology.orchestra.run.smithi032.stdout:(30/32): python3-webtest-2.0.33-1.el8.noarch.rp 1.0 MB/s | 85 kB 00:00 2024-05-12T03:47:38.309 INFO:teuthology.orchestra.run.smithi032.stdout:(31/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 1.8 MB/s | 251 kB 00:00 2024-05-12T03:47:38.325 INFO:teuthology.orchestra.run.smithi032.stdout:(32/32): python3-zc-lockfile-2.0-2.el8.noarch.r 250 kB/s | 23 kB 00:00 2024-05-12T03:47:38.329 INFO:teuthology.orchestra.run.smithi032.stdout:-------------------------------------------------------------------------------- 2024-05-12T03:47:38.329 INFO:teuthology.orchestra.run.smithi032.stdout:Total 12 MB/s | 29 MB 00:02 2024-05-12T03:47:38.411 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction check 2024-05-12T03:47:38.452 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction check succeeded. 2024-05-12T03:47:38.452 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction test 2024-05-12T03:47:38.457 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/32 2024-05-12T03:47:38.783 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction test succeeded. 2024-05-12T03:47:38.786 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction 2024-05-12T03:47:38.945 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/32 2024-05-12T03:47:39.149 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 20/32 2024-05-12T03:47:39.367 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2024-05-12T03:47:39.645 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2024-05-12T03:47:39.905 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2024-05-12T03:47:40.047 INFO:teuthology.orchestra.run.smithi032.stdout: Preparing : 1/1 2024-05-12T03:47:40.173 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/32 2024-05-12T03:47:40.297 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2024-05-12T03:47:40.520 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2024-05-12T03:47:40.531 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 25/32 2024-05-12T03:47:40.783 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/32 2024-05-12T03:47:40.897 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2024-05-12T03:47:41.144 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 4/32 2024-05-12T03:47:41.312 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-939.gc190bdee.el8.n 27/32 2024-05-12T03:47:41.333 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 5/32 2024-05-12T03:47:41.508 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 6/32 2024-05-12T03:47:41.589 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-2:17.2.7-939.gc190bdee.el8.x86_64 28/32 2024-05-12T03:47:41.716 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 7/32 2024-05-12T03:47:41.869 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 8/32 2024-05-12T03:47:42.081 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 9/32 2024-05-12T03:47:42.309 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 10/32 2024-05-12T03:47:42.555 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 11/32 2024-05-12T03:47:42.789 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 12/32 2024-05-12T03:47:43.194 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 13/32 2024-05-12T03:47:43.203 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-2:17.2.7-939.gc190bdee.el8.x86_64 28/32 2024-05-12T03:47:43.430 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-osd-2:17.2.7-939.gc190bdee.el8.x86_64 29/32 2024-05-12T03:47:43.466 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cryptography-3.2.1-7.el8.x86_64 14/32 2024-05-12T03:47:43.662 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 15/32 2024-05-12T03:47:43.860 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-osd-2:17.2.7-939.gc190bdee.el8.x86_64 29/32 2024-05-12T03:47:43.969 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 16/32 2024-05-12T03:47:44.165 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 17/32 2024-05-12T03:47:44.507 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/32 2024-05-12T03:47:44.602 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mon-2:17.2.7-939.gc190bdee.el8.x86_64 30/32 2024-05-12T03:47:44.711 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/32 2024-05-12T03:47:44.837 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mon-2:17.2.7-939.gc190bdee.el8.x86_64 30/32 2024-05-12T03:47:44.908 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 20/32 2024-05-12T03:47:45.051 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mds-2:17.2.7-939.gc190bdee.el8.x86_64 31/32 2024-05-12T03:47:45.070 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mds-2:17.2.7-939.gc190bdee.el8.x86_64 31/32 2024-05-12T03:47:45.085 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2024-05-12T03:47:45.218 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-2:17.2.7-939.gc190bdee.el8.x86_64 32/32 2024-05-12T03:47:45.364 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2024-05-12T03:47:45.577 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2024-05-12T03:47:45.648 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-2:17.2.7-939.gc190bdee.el8.x86_64 32/32 2024-05-12T03:47:45.648 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-2:17.2.7-939.gc190bdee.el8.x86_64 1/32 2024-05-12T03:47:45.648 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mds-2:17.2.7-939.gc190bdee.el8.x86_64 2/32 2024-05-12T03:47:45.648 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-2:17.2.7-939.gc190bdee.el8.x86_64 3/32 2024-05-12T03:47:45.648 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mon-2:17.2.7-939.gc190bdee.el8.x86_64 4/32 2024-05-12T03:47:45.648 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-osd-2:17.2.7-939.gc190bdee.el8.x86_64 5/32 2024-05-12T03:47:45.648 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-939.gc190bdee.el8.n 6/32 2024-05-12T03:47:45.648 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 7/32 2024-05-12T03:47:45.648 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 8/32 2024-05-12T03:47:45.649 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 9/32 2024-05-12T03:47:45.649 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 10/32 2024-05-12T03:47:45.649 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 11/32 2024-05-12T03:47:45.649 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 12/32 2024-05-12T03:47:45.649 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cryptography-3.2.1-7.el8.x86_64 13/32 2024-05-12T03:47:45.649 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 14/32 2024-05-12T03:47:45.649 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 15/32 2024-05-12T03:47:45.649 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 16/32 2024-05-12T03:47:45.649 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 17/32 2024-05-12T03:47:45.649 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 18/32 2024-05-12T03:47:45.649 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 19/32 2024-05-12T03:47:45.649 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 20/32 2024-05-12T03:47:45.649 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 21/32 2024-05-12T03:47:45.649 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 22/32 2024-05-12T03:47:45.649 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 23/32 2024-05-12T03:47:45.650 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-portend-2.6-1.el8.noarch 24/32 2024-05-12T03:47:45.650 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 25/32 2024-05-12T03:47:45.650 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 26/32 2024-05-12T03:47:45.650 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 27/32 2024-05-12T03:47:45.650 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 28/32 2024-05-12T03:47:45.650 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 29/32 2024-05-12T03:47:45.651 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 30/32 2024-05-12T03:47:45.651 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 31/32 2024-05-12T03:47:45.840 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/32 2024-05-12T03:47:46.183 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 25/32 2024-05-12T03:47:46.499 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2024-05-12T03:47:46.707 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 32/32 2024-05-12T03:47:46.707 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:47:46.707 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2024-05-12T03:47:46.707 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-2:17.2.7-939.gc190bdee.el8.x86_64 2024-05-12T03:47:46.707 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mds-2:17.2.7-939.gc190bdee.el8.x86_64 2024-05-12T03:47:46.708 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-2:17.2.7-939.gc190bdee.el8.x86_64 2024-05-12T03:47:46.708 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-modules-core-2:17.2.7-939.gc190bdee.el8.noarch 2024-05-12T03:47:46.708 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mon-2:17.2.7-939.gc190bdee.el8.x86_64 2024-05-12T03:47:46.708 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-osd-2:17.2.7-939.gc190bdee.el8.x86_64 2024-05-12T03:47:46.708 INFO:teuthology.orchestra.run.smithi033.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-05-12T03:47:46.708 INFO:teuthology.orchestra.run.smithi033.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-05-12T03:47:46.708 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-05-12T03:47:46.708 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-05-12T03:47:46.708 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-05-12T03:47:46.708 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cryptography-3.2.1-7.el8.x86_64 2024-05-12T03:47:46.708 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-6.2-6.el8.noarch 2024-05-12T03:47:46.708 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-05-12T03:47:46.708 INFO:teuthology.orchestra.run.smithi033.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-05-12T03:47:46.708 INFO:teuthology.orchestra.run.smithi033.stdout: python3-mako-1.0.6-14.el8.noarch 2024-05-12T03:47:46.709 INFO:teuthology.orchestra.run.smithi033.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-05-12T03:47:46.709 INFO:teuthology.orchestra.run.smithi033.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-05-12T03:47:46.709 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-05-12T03:47:46.709 INFO:teuthology.orchestra.run.smithi033.stdout: python3-portend-2.6-1.el8.noarch 2024-05-12T03:47:46.709 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-05-12T03:47:46.709 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pycparser-2.14-14.el8.noarch 2024-05-12T03:47:46.709 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pytz-2017.2-11.el8.noarch 2024-05-12T03:47:46.709 INFO:teuthology.orchestra.run.smithi033.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-05-12T03:47:46.709 INFO:teuthology.orchestra.run.smithi033.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-05-12T03:47:46.709 INFO:teuthology.orchestra.run.smithi033.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-05-12T03:47:46.709 INFO:teuthology.orchestra.run.smithi033.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-05-12T03:47:46.709 INFO:teuthology.orchestra.run.smithi033.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-05-12T03:47:46.709 INFO:teuthology.orchestra.run.smithi033.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-05-12T03:47:46.709 INFO:teuthology.orchestra.run.smithi033.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-05-12T03:47:46.709 INFO:teuthology.orchestra.run.smithi033.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-05-12T03:47:46.710 INFO:teuthology.orchestra.run.smithi033.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-05-12T03:47:46.710 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:47:46.710 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-05-12T03:47:46.877 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-939.gc190bdee.el8.n 27/32 2024-05-12T03:47:46.948 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum -y install ceph-base 2024-05-12T03:47:47.071 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-2:17.2.7-939.gc190bdee.el8.x86_64 28/32 2024-05-12T03:47:47.737 INFO:teuthology.orchestra.run.smithi033.stdout:Last metadata expiration check: 0:01:20 ago on Sun 12 May 2024 03:46:27 AM UTC. 2024-05-12T03:47:48.680 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mgr-2:17.2.7-939.gc190bdee.el8.x86_64 28/32 2024-05-12T03:47:48.863 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-osd-2:17.2.7-939.gc190bdee.el8.x86_64 29/32 2024-05-12T03:47:49.290 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-osd-2:17.2.7-939.gc190bdee.el8.x86_64 29/32 2024-05-12T03:47:49.433 INFO:teuthology.orchestra.run.smithi033.stdout:Package ceph-base-2:17.2.7-939.gc190bdee.el8.x86_64 is already installed. 2024-05-12T03:47:49.486 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mon-2:17.2.7-939.gc190bdee.el8.x86_64 30/32 2024-05-12T03:47:49.487 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-05-12T03:47:49.488 INFO:teuthology.orchestra.run.smithi033.stdout:Nothing to do. 2024-05-12T03:47:49.488 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-05-12T03:47:49.586 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum -y install cephadm 2024-05-12T03:47:49.714 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mon-2:17.2.7-939.gc190bdee.el8.x86_64 30/32 2024-05-12T03:47:49.911 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mds-2:17.2.7-939.gc190bdee.el8.x86_64 31/32 2024-05-12T03:47:49.930 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mds-2:17.2.7-939.gc190bdee.el8.x86_64 31/32 2024-05-12T03:47:50.053 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-2:17.2.7-939.gc190bdee.el8.x86_64 32/32 2024-05-12T03:47:50.369 INFO:teuthology.orchestra.run.smithi033.stdout:Last metadata expiration check: 0:01:23 ago on Sun 12 May 2024 03:46:27 AM UTC. 2024-05-12T03:47:50.462 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-2:17.2.7-939.gc190bdee.el8.x86_64 32/32 2024-05-12T03:47:50.463 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-2:17.2.7-939.gc190bdee.el8.x86_64 1/32 2024-05-12T03:47:50.463 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mds-2:17.2.7-939.gc190bdee.el8.x86_64 2/32 2024-05-12T03:47:50.463 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-2:17.2.7-939.gc190bdee.el8.x86_64 3/32 2024-05-12T03:47:50.463 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mon-2:17.2.7-939.gc190bdee.el8.x86_64 4/32 2024-05-12T03:47:50.463 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-osd-2:17.2.7-939.gc190bdee.el8.x86_64 5/32 2024-05-12T03:47:50.463 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-939.gc190bdee.el8.n 6/32 2024-05-12T03:47:50.463 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 7/32 2024-05-12T03:47:50.463 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 8/32 2024-05-12T03:47:50.463 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 9/32 2024-05-12T03:47:50.463 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 10/32 2024-05-12T03:47:50.463 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 11/32 2024-05-12T03:47:50.463 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 12/32 2024-05-12T03:47:50.463 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cryptography-3.2.1-7.el8.x86_64 13/32 2024-05-12T03:47:50.464 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 14/32 2024-05-12T03:47:50.464 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 15/32 2024-05-12T03:47:50.464 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 16/32 2024-05-12T03:47:50.464 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 17/32 2024-05-12T03:47:50.464 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 18/32 2024-05-12T03:47:50.464 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 19/32 2024-05-12T03:47:50.464 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 20/32 2024-05-12T03:47:50.464 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 21/32 2024-05-12T03:47:50.464 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 22/32 2024-05-12T03:47:50.464 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 23/32 2024-05-12T03:47:50.464 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-portend-2.6-1.el8.noarch 24/32 2024-05-12T03:47:50.464 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 25/32 2024-05-12T03:47:50.464 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 26/32 2024-05-12T03:47:50.464 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 27/32 2024-05-12T03:47:50.465 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 28/32 2024-05-12T03:47:50.465 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 29/32 2024-05-12T03:47:50.465 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 30/32 2024-05-12T03:47:50.465 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 31/32 2024-05-12T03:47:51.484 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 32/32 2024-05-12T03:47:51.484 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:47:51.484 INFO:teuthology.orchestra.run.smithi032.stdout:Installed: 2024-05-12T03:47:51.484 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-2:17.2.7-939.gc190bdee.el8.x86_64 2024-05-12T03:47:51.485 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mds-2:17.2.7-939.gc190bdee.el8.x86_64 2024-05-12T03:47:51.485 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-2:17.2.7-939.gc190bdee.el8.x86_64 2024-05-12T03:47:51.485 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-modules-core-2:17.2.7-939.gc190bdee.el8.noarch 2024-05-12T03:47:51.485 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mon-2:17.2.7-939.gc190bdee.el8.x86_64 2024-05-12T03:47:51.485 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-osd-2:17.2.7-939.gc190bdee.el8.x86_64 2024-05-12T03:47:51.485 INFO:teuthology.orchestra.run.smithi032.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-05-12T03:47:51.485 INFO:teuthology.orchestra.run.smithi032.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-05-12T03:47:51.485 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-05-12T03:47:51.485 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-05-12T03:47:51.485 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-05-12T03:47:51.485 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cryptography-3.2.1-7.el8.x86_64 2024-05-12T03:47:51.485 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-6.2-6.el8.noarch 2024-05-12T03:47:51.485 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-05-12T03:47:51.485 INFO:teuthology.orchestra.run.smithi032.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-05-12T03:47:51.486 INFO:teuthology.orchestra.run.smithi032.stdout: python3-mako-1.0.6-14.el8.noarch 2024-05-12T03:47:51.486 INFO:teuthology.orchestra.run.smithi032.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-05-12T03:47:51.486 INFO:teuthology.orchestra.run.smithi032.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-05-12T03:47:51.486 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-05-12T03:47:51.486 INFO:teuthology.orchestra.run.smithi032.stdout: python3-portend-2.6-1.el8.noarch 2024-05-12T03:47:51.486 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-05-12T03:47:51.486 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pycparser-2.14-14.el8.noarch 2024-05-12T03:47:51.486 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pytz-2017.2-11.el8.noarch 2024-05-12T03:47:51.486 INFO:teuthology.orchestra.run.smithi032.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-05-12T03:47:51.486 INFO:teuthology.orchestra.run.smithi032.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-05-12T03:47:51.486 INFO:teuthology.orchestra.run.smithi032.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-05-12T03:47:51.486 INFO:teuthology.orchestra.run.smithi032.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-05-12T03:47:51.486 INFO:teuthology.orchestra.run.smithi032.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-05-12T03:47:51.487 INFO:teuthology.orchestra.run.smithi032.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-05-12T03:47:51.487 INFO:teuthology.orchestra.run.smithi032.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-05-12T03:47:51.487 INFO:teuthology.orchestra.run.smithi032.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-05-12T03:47:51.487 INFO:teuthology.orchestra.run.smithi032.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-05-12T03:47:51.487 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:47:51.487 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2024-05-12T03:47:51.717 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum -y install ceph-base 2024-05-12T03:47:52.132 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-05-12T03:47:52.133 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-05-12T03:47:52.133 INFO:teuthology.orchestra.run.smithi033.stdout: Package Arch Version Repository Size 2024-05-12T03:47:52.133 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-05-12T03:47:52.133 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2024-05-12T03:47:52.133 INFO:teuthology.orchestra.run.smithi033.stdout: cephadm noarch 2:17.2.7-939.gc190bdee.el8 ceph-noarch 82 k 2024-05-12T03:47:52.133 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:47:52.133 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2024-05-12T03:47:52.133 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-05-12T03:47:52.133 INFO:teuthology.orchestra.run.smithi033.stdout:Install 1 Package 2024-05-12T03:47:52.134 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:47:52.134 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 82 k 2024-05-12T03:47:52.134 INFO:teuthology.orchestra.run.smithi033.stdout:Installed size: 372 k 2024-05-12T03:47:52.134 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading Packages: 2024-05-12T03:47:52.304 INFO:teuthology.orchestra.run.smithi033.stdout:cephadm-17.2.7-939.gc190bdee.el8.noarch.rpm 484 kB/s | 82 kB 00:00 2024-05-12T03:47:52.304 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2024-05-12T03:47:52.304 INFO:teuthology.orchestra.run.smithi033.stdout:Total 480 kB/s | 82 kB 00:00 2024-05-12T03:47:52.305 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2024-05-12T03:47:52.313 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction check succeeded. 2024-05-12T03:47:52.313 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2024-05-12T03:47:52.315 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded. 2024-05-12T03:47:52.318 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2024-05-12T03:47:52.360 INFO:teuthology.orchestra.run.smithi033.stdout: Preparing : 1/1 2024-05-12T03:47:52.506 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:01:20 ago on Sun 12 May 2024 03:46:32 AM UTC. 2024-05-12T03:47:52.958 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: cephadm-2:17.2.7-939.gc190bdee.el8.noarch 1/1 2024-05-12T03:47:53.142 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : cephadm-2:17.2.7-939.gc190bdee.el8.noarch 1/1 2024-05-12T03:47:53.377 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: cephadm-2:17.2.7-939.gc190bdee.el8.noarch 1/1 2024-05-12T03:47:53.787 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : cephadm-2:17.2.7-939.gc190bdee.el8.noarch 1/1 2024-05-12T03:47:53.788 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:47:53.788 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2024-05-12T03:47:53.788 INFO:teuthology.orchestra.run.smithi033.stdout: cephadm-2:17.2.7-939.gc190bdee.el8.noarch 2024-05-12T03:47:53.788 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:47:53.788 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-05-12T03:47:54.031 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum -y install ceph-immutable-object-cache 2024-05-12T03:47:54.236 INFO:teuthology.orchestra.run.smithi032.stdout:Package ceph-base-2:17.2.7-939.gc190bdee.el8.x86_64 is already installed. 2024-05-12T03:47:54.289 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-05-12T03:47:54.291 INFO:teuthology.orchestra.run.smithi032.stdout:Nothing to do. 2024-05-12T03:47:54.291 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2024-05-12T03:47:54.393 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum -y install cephadm 2024-05-12T03:47:54.818 INFO:teuthology.orchestra.run.smithi033.stdout:Last metadata expiration check: 0:01:27 ago on Sun 12 May 2024 03:46:27 AM UTC. 2024-05-12T03:47:55.178 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:01:23 ago on Sun 12 May 2024 03:46:32 AM UTC. 2024-05-12T03:47:56.589 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-05-12T03:47:56.589 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-05-12T03:47:56.589 INFO:teuthology.orchestra.run.smithi033.stdout: Package Arch Version Repo Size 2024-05-12T03:47:56.589 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-05-12T03:47:56.590 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2024-05-12T03:47:56.590 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-939.gc190bdee.el8 ceph 162 k 2024-05-12T03:47:56.590 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:47:56.590 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2024-05-12T03:47:56.590 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-05-12T03:47:56.590 INFO:teuthology.orchestra.run.smithi033.stdout:Install 1 Package 2024-05-12T03:47:56.590 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:47:56.591 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 162 k 2024-05-12T03:47:56.591 INFO:teuthology.orchestra.run.smithi033.stdout:Installed size: 494 k 2024-05-12T03:47:56.591 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading Packages: 2024-05-12T03:47:56.955 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-05-12T03:47:56.956 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-05-12T03:47:56.956 INFO:teuthology.orchestra.run.smithi032.stdout: Package Arch Version Repository Size 2024-05-12T03:47:56.956 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-05-12T03:47:56.956 INFO:teuthology.orchestra.run.smithi032.stdout:Installing: 2024-05-12T03:47:56.957 INFO:teuthology.orchestra.run.smithi032.stdout: cephadm noarch 2:17.2.7-939.gc190bdee.el8 ceph-noarch 82 k 2024-05-12T03:47:56.957 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:47:56.957 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction Summary 2024-05-12T03:47:56.957 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-05-12T03:47:56.957 INFO:teuthology.orchestra.run.smithi032.stdout:Install 1 Package 2024-05-12T03:47:56.957 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:47:56.957 INFO:teuthology.orchestra.run.smithi032.stdout:Total download size: 82 k 2024-05-12T03:47:56.957 INFO:teuthology.orchestra.run.smithi032.stdout:Installed size: 372 k 2024-05-12T03:47:56.957 INFO:teuthology.orchestra.run.smithi032.stdout:Downloading Packages: 2024-05-12T03:47:56.975 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-immutable-object-cache-17.2.7-939.gc190bde 421 kB/s | 162 kB 00:00 2024-05-12T03:47:56.976 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2024-05-12T03:47:56.976 INFO:teuthology.orchestra.run.smithi033.stdout:Total 419 kB/s | 162 kB 00:00 2024-05-12T03:47:56.976 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2024-05-12T03:47:56.989 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction check succeeded. 2024-05-12T03:47:56.989 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2024-05-12T03:47:57.046 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded. 2024-05-12T03:47:57.049 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2024-05-12T03:47:57.162 INFO:teuthology.orchestra.run.smithi033.stdout: Preparing : 1/1 2024-05-12T03:47:57.207 INFO:teuthology.orchestra.run.smithi032.stdout:cephadm-17.2.7-939.gc190bdee.el8.noarch.rpm 328 kB/s | 82 kB 00:00 2024-05-12T03:47:57.208 INFO:teuthology.orchestra.run.smithi032.stdout:-------------------------------------------------------------------------------- 2024-05-12T03:47:57.208 INFO:teuthology.orchestra.run.smithi032.stdout:Total 326 kB/s | 82 kB 00:00 2024-05-12T03:47:57.208 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction check 2024-05-12T03:47:57.217 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction check succeeded. 2024-05-12T03:47:57.217 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction test 2024-05-12T03:47:57.219 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction test succeeded. 2024-05-12T03:47:57.222 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction 2024-05-12T03:47:57.269 INFO:teuthology.orchestra.run.smithi032.stdout: Preparing : 1/1 2024-05-12T03:47:57.383 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-939.gc190bdee.e 1/1 2024-05-12T03:47:57.812 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: cephadm-2:17.2.7-939.gc190bdee.el8.noarch 1/1 2024-05-12T03:47:57.813 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-939.gc190bdee.e 1/1 2024-05-12T03:47:58.000 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : cephadm-2:17.2.7-939.gc190bdee.el8.noarch 1/1 2024-05-12T03:47:58.200 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-939.gc190bdee.e 1/1 2024-05-12T03:47:58.200 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:47:58.200 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2024-05-12T03:47:58.200 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-immutable-object-cache-2:17.2.7-939.gc190bdee.el8.x86_64 2024-05-12T03:47:58.200 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:47:58.200 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-05-12T03:47:58.237 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: cephadm-2:17.2.7-939.gc190bdee.el8.noarch 1/1 2024-05-12T03:47:58.440 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum -y install ceph-mgr 2024-05-12T03:47:58.610 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : cephadm-2:17.2.7-939.gc190bdee.el8.noarch 1/1 2024-05-12T03:47:58.610 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:47:58.610 INFO:teuthology.orchestra.run.smithi032.stdout:Installed: 2024-05-12T03:47:58.610 INFO:teuthology.orchestra.run.smithi032.stdout: cephadm-2:17.2.7-939.gc190bdee.el8.noarch 2024-05-12T03:47:58.611 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:47:58.611 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2024-05-12T03:47:58.861 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum -y install ceph-immutable-object-cache 2024-05-12T03:47:59.226 INFO:teuthology.orchestra.run.smithi033.stdout:Last metadata expiration check: 0:01:32 ago on Sun 12 May 2024 03:46:27 AM UTC. 2024-05-12T03:47:59.647 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:01:27 ago on Sun 12 May 2024 03:46:32 AM UTC. 2024-05-12T03:48:00.938 INFO:teuthology.orchestra.run.smithi033.stdout:Package ceph-mgr-2:17.2.7-939.gc190bdee.el8.x86_64 is already installed. 2024-05-12T03:48:00.990 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-05-12T03:48:00.991 INFO:teuthology.orchestra.run.smithi033.stdout:Nothing to do. 2024-05-12T03:48:00.991 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-05-12T03:48:01.100 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum -y install ceph-mgr-dashboard 2024-05-12T03:48:01.432 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-05-12T03:48:01.433 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-05-12T03:48:01.433 INFO:teuthology.orchestra.run.smithi032.stdout: Package Arch Version Repo Size 2024-05-12T03:48:01.433 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-05-12T03:48:01.433 INFO:teuthology.orchestra.run.smithi032.stdout:Installing: 2024-05-12T03:48:01.433 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-939.gc190bdee.el8 ceph 162 k 2024-05-12T03:48:01.433 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:48:01.433 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction Summary 2024-05-12T03:48:01.433 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-05-12T03:48:01.433 INFO:teuthology.orchestra.run.smithi032.stdout:Install 1 Package 2024-05-12T03:48:01.433 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:48:01.434 INFO:teuthology.orchestra.run.smithi032.stdout:Total download size: 162 k 2024-05-12T03:48:01.434 INFO:teuthology.orchestra.run.smithi032.stdout:Installed size: 494 k 2024-05-12T03:48:01.434 INFO:teuthology.orchestra.run.smithi032.stdout:Downloading Packages: 2024-05-12T03:48:01.885 INFO:teuthology.orchestra.run.smithi033.stdout:Last metadata expiration check: 0:01:34 ago on Sun 12 May 2024 03:46:27 AM UTC. 2024-05-12T03:48:02.053 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-immutable-object-cache-17.2.7-939.gc190bde 261 kB/s | 162 kB 00:00 2024-05-12T03:48:02.054 INFO:teuthology.orchestra.run.smithi032.stdout:-------------------------------------------------------------------------------- 2024-05-12T03:48:02.054 INFO:teuthology.orchestra.run.smithi032.stdout:Total 260 kB/s | 162 kB 00:00 2024-05-12T03:48:02.055 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction check 2024-05-12T03:48:02.068 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction check succeeded. 2024-05-12T03:48:02.068 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction test 2024-05-12T03:48:02.125 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction test succeeded. 2024-05-12T03:48:02.129 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction 2024-05-12T03:48:02.620 INFO:teuthology.orchestra.run.smithi032.stdout: Preparing : 1/1 2024-05-12T03:48:03.626 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-939.gc190bdee.e 1/1 2024-05-12T03:48:03.661 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-05-12T03:48:03.662 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-05-12T03:48:03.662 INFO:teuthology.orchestra.run.smithi033.stdout: Package Arch Version Repository Size 2024-05-12T03:48:03.662 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-05-12T03:48:03.662 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2024-05-12T03:48:03.662 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-dashboard noarch 2:17.2.7-939.gc190bdee.el8 ceph-noarch 3.4 M 2024-05-12T03:48:03.662 INFO:teuthology.orchestra.run.smithi033.stdout:Installing dependencies: 2024-05-12T03:48:03.662 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-grafana-dashboards noarch 2:17.2.7-939.gc190bdee.el8 ceph-noarch 24 k 2024-05-12T03:48:03.663 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-prometheus-alerts noarch 2:17.2.7-939.gc190bdee.el8 ceph-noarch 14 k 2024-05-12T03:48:03.663 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2024-05-12T03:48:03.663 INFO:teuthology.orchestra.run.smithi033.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-05-12T03:48:03.663 INFO:teuthology.orchestra.run.smithi033.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-05-12T03:48:03.663 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:48:03.663 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2024-05-12T03:48:03.663 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-05-12T03:48:03.663 INFO:teuthology.orchestra.run.smithi033.stdout:Install 6 Packages 2024-05-12T03:48:03.663 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:48:03.664 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 3.7 M 2024-05-12T03:48:03.664 INFO:teuthology.orchestra.run.smithi033.stdout:Installed size: 80 M 2024-05-12T03:48:03.664 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading Packages: 2024-05-12T03:48:04.079 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-939.gc190bdee.e 1/1 2024-05-12T03:48:04.452 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-939.gc190bdee.e 1/1 2024-05-12T03:48:04.452 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:48:04.452 INFO:teuthology.orchestra.run.smithi032.stdout:Installed: 2024-05-12T03:48:04.452 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-immutable-object-cache-2:17.2.7-939.gc190bdee.el8.x86_64 2024-05-12T03:48:04.452 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:48:04.452 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2024-05-12T03:48:04.504 INFO:teuthology.orchestra.run.smithi033.stdout:(1/6): ceph-prometheus-alerts-17.2.7-939.gc190b 104 kB/s | 14 kB 00:00 2024-05-12T03:48:04.604 INFO:teuthology.orchestra.run.smithi033.stdout:(2/6): ceph-grafana-dashboards-17.2.7-939.gc190 101 kB/s | 24 kB 00:00 2024-05-12T03:48:04.710 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum -y install ceph-mgr 2024-05-12T03:48:04.813 INFO:teuthology.orchestra.run.smithi033.stdout:(3/6): python3-jwt-1.6.1-2.el8.noarch.rpm 138 kB/s | 43 kB 00:00 2024-05-12T03:48:04.838 INFO:teuthology.orchestra.run.smithi033.stdout:(4/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 144 kB/s | 34 kB 00:00 2024-05-12T03:48:04.972 INFO:teuthology.orchestra.run.smithi033.stdout:(5/6): ceph-mgr-dashboard-17.2.7-939.gc190bdee. 5.5 MB/s | 3.4 MB 00:00 2024-05-12T03:48:05.247 INFO:teuthology.orchestra.run.smithi033.stdout:(6/6): python3-routes-2.4.1-12.el8.noarch.rpm 453 kB/s | 196 kB 00:00 2024-05-12T03:48:05.251 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2024-05-12T03:48:05.251 INFO:teuthology.orchestra.run.smithi033.stdout:Total 2.3 MB/s | 3.7 MB 00:01 2024-05-12T03:48:05.263 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2024-05-12T03:48:05.273 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction check succeeded. 2024-05-12T03:48:05.273 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2024-05-12T03:48:05.369 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded. 2024-05-12T03:48:05.373 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2024-05-12T03:48:05.495 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:01:33 ago on Sun 12 May 2024 03:46:32 AM UTC. 2024-05-12T03:48:05.672 INFO:teuthology.orchestra.run.smithi033.stdout: Preparing : 1/1 2024-05-12T03:48:05.910 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/6 2024-05-12T03:48:06.192 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/6 2024-05-12T03:48:06.425 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 3/6 2024-05-12T03:48:06.572 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-939.gc190bdee.el8.no 4/6 2024-05-12T03:48:07.211 INFO:teuthology.orchestra.run.smithi032.stdout:Package ceph-mgr-2:17.2.7-939.gc190bdee.el8.x86_64 is already installed. 2024-05-12T03:48:07.264 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-05-12T03:48:07.265 INFO:teuthology.orchestra.run.smithi032.stdout:Nothing to do. 2024-05-12T03:48:07.266 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2024-05-12T03:48:07.367 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum -y install ceph-mgr-dashboard 2024-05-12T03:48:07.746 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-939.gc190bdee.el8.n 5/6 2024-05-12T03:48:08.034 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-939.gc190bdee.el8.noarch 6/6 2024-05-12T03:48:08.145 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:01:36 ago on Sun 12 May 2024 03:46:32 AM UTC. 2024-05-12T03:48:08.362 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-939.gc190bdee.el8.noarch 6/6 2024-05-12T03:48:08.362 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-939.gc190bdee.el8.n 1/6 2024-05-12T03:48:08.362 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-939.gc190bdee.el8.noarch 2/6 2024-05-12T03:48:08.362 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-939.gc190bdee.el8.no 3/6 2024-05-12T03:48:08.362 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 4/6 2024-05-12T03:48:08.362 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 5/6 2024-05-12T03:48:08.770 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 6/6 2024-05-12T03:48:08.770 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:48:08.771 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2024-05-12T03:48:08.771 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-grafana-dashboards-2:17.2.7-939.gc190bdee.el8.noarch 2024-05-12T03:48:08.771 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-dashboard-2:17.2.7-939.gc190bdee.el8.noarch 2024-05-12T03:48:08.771 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-prometheus-alerts-2:17.2.7-939.gc190bdee.el8.noarch 2024-05-12T03:48:08.771 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-05-12T03:48:08.771 INFO:teuthology.orchestra.run.smithi033.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-05-12T03:48:08.771 INFO:teuthology.orchestra.run.smithi033.stdout: python3-routes-2.4.1-12.el8.noarch 2024-05-12T03:48:08.771 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:48:08.771 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-05-12T03:48:08.899 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum -y install ceph-mgr-diskprediction-local 2024-05-12T03:48:09.693 INFO:teuthology.orchestra.run.smithi033.stdout:Last metadata expiration check: 0:01:42 ago on Sun 12 May 2024 03:46:27 AM UTC. 2024-05-12T03:48:09.922 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-05-12T03:48:09.923 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-05-12T03:48:09.923 INFO:teuthology.orchestra.run.smithi032.stdout: Package Arch Version Repository Size 2024-05-12T03:48:09.923 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-05-12T03:48:09.923 INFO:teuthology.orchestra.run.smithi032.stdout:Installing: 2024-05-12T03:48:09.923 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-dashboard noarch 2:17.2.7-939.gc190bdee.el8 ceph-noarch 3.4 M 2024-05-12T03:48:09.923 INFO:teuthology.orchestra.run.smithi032.stdout:Installing dependencies: 2024-05-12T03:48:09.924 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-grafana-dashboards noarch 2:17.2.7-939.gc190bdee.el8 ceph-noarch 24 k 2024-05-12T03:48:09.924 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-prometheus-alerts noarch 2:17.2.7-939.gc190bdee.el8 ceph-noarch 14 k 2024-05-12T03:48:09.924 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2024-05-12T03:48:09.924 INFO:teuthology.orchestra.run.smithi032.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-05-12T03:48:09.924 INFO:teuthology.orchestra.run.smithi032.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-05-12T03:48:09.924 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:48:09.924 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction Summary 2024-05-12T03:48:09.924 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-05-12T03:48:09.924 INFO:teuthology.orchestra.run.smithi032.stdout:Install 6 Packages 2024-05-12T03:48:09.924 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:48:09.925 INFO:teuthology.orchestra.run.smithi032.stdout:Total download size: 3.7 M 2024-05-12T03:48:09.925 INFO:teuthology.orchestra.run.smithi032.stdout:Installed size: 80 M 2024-05-12T03:48:09.925 INFO:teuthology.orchestra.run.smithi032.stdout:Downloading Packages: 2024-05-12T03:48:10.764 INFO:teuthology.orchestra.run.smithi032.stdout:(1/6): ceph-grafana-dashboards-17.2.7-939.gc190 175 kB/s | 24 kB 00:00 2024-05-12T03:48:10.789 INFO:teuthology.orchestra.run.smithi032.stdout:(2/6): ceph-prometheus-alerts-17.2.7-939.gc190b 89 kB/s | 14 kB 00:00 2024-05-12T03:48:10.873 INFO:teuthology.orchestra.run.smithi032.stdout:(3/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 403 kB/s | 34 kB 00:00 2024-05-12T03:48:10.957 INFO:teuthology.orchestra.run.smithi032.stdout:(4/6): python3-routes-2.4.1-12.el8.noarch.rpm 2.3 MB/s | 196 kB 00:00 2024-05-12T03:48:11.040 INFO:teuthology.orchestra.run.smithi032.stdout:(5/6): ceph-mgr-dashboard-17.2.7-939.gc190bdee. 8.1 MB/s | 3.4 MB 00:00 2024-05-12T03:48:11.090 INFO:teuthology.orchestra.run.smithi032.stdout:(6/6): python3-jwt-1.6.1-2.el8.noarch.rpm 131 kB/s | 43 kB 00:00 2024-05-12T03:48:11.093 INFO:teuthology.orchestra.run.smithi032.stdout:-------------------------------------------------------------------------------- 2024-05-12T03:48:11.093 INFO:teuthology.orchestra.run.smithi032.stdout:Total 3.1 MB/s | 3.7 MB 00:01 2024-05-12T03:48:11.106 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction check 2024-05-12T03:48:11.117 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction check succeeded. 2024-05-12T03:48:11.117 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction test 2024-05-12T03:48:11.208 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction test succeeded. 2024-05-12T03:48:11.212 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction 2024-05-12T03:48:11.457 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-05-12T03:48:11.459 INFO:teuthology.orchestra.run.smithi033.stdout:============================================================================================ 2024-05-12T03:48:11.459 INFO:teuthology.orchestra.run.smithi033.stdout: Package Arch Version Repository Size 2024-05-12T03:48:11.459 INFO:teuthology.orchestra.run.smithi033.stdout:============================================================================================ 2024-05-12T03:48:11.459 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2024-05-12T03:48:11.459 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-939.gc190bdee.el8 ceph-noarch 7.4 M 2024-05-12T03:48:11.459 INFO:teuthology.orchestra.run.smithi033.stdout:Installing dependencies: 2024-05-12T03:48:11.459 INFO:teuthology.orchestra.run.smithi033.stdout: libgfortran x86_64 8.5.0-22.el8 baseos 645 k 2024-05-12T03:48:11.459 INFO:teuthology.orchestra.run.smithi033.stdout: libquadmath x86_64 8.5.0-22.el8 baseos 172 k 2024-05-12T03:48:11.459 INFO:teuthology.orchestra.run.smithi033.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2024-05-12T03:48:11.459 INFO:teuthology.orchestra.run.smithi033.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2024-05-12T03:48:11.459 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2024-05-12T03:48:11.459 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2024-05-12T03:48:11.460 INFO:teuthology.orchestra.run.smithi033.stdout: python3-scipy x86_64 1.0.0-21.module_el8+762+77bd8591 appstream 14 M 2024-05-12T03:48:11.460 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:48:11.460 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2024-05-12T03:48:11.460 INFO:teuthology.orchestra.run.smithi033.stdout:============================================================================================ 2024-05-12T03:48:11.460 INFO:teuthology.orchestra.run.smithi033.stdout:Install 8 Packages 2024-05-12T03:48:11.460 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:48:11.460 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 36 M 2024-05-12T03:48:11.461 INFO:teuthology.orchestra.run.smithi033.stdout:Installed size: 210 M 2024-05-12T03:48:11.461 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading Packages: 2024-05-12T03:48:11.511 INFO:teuthology.orchestra.run.smithi032.stdout: Preparing : 1/1 2024-05-12T03:48:11.717 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/6 2024-05-12T03:48:11.987 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/6 2024-05-12T03:48:12.177 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 3/6 2024-05-12T03:48:12.179 INFO:teuthology.orchestra.run.smithi033.stdout:(1/8): openblas-0.3.15-4.el8.x86_64.rpm 13 MB/s | 4.7 MB 00:00 2024-05-12T03:48:12.280 INFO:teuthology.orchestra.run.smithi033.stdout:(2/8): openblas-threads-0.3.15-4.el8.x86_64.rpm 11 MB/s | 4.9 MB 00:00 2024-05-12T03:48:12.299 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-939.gc190bdee.el8.no 4/6 2024-05-12T03:48:12.380 INFO:teuthology.orchestra.run.smithi033.stdout:(3/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 19 MB/s | 3.7 MB 00:00 2024-05-12T03:48:12.406 INFO:teuthology.orchestra.run.smithi033.stdout:(4/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 1.8 MB/s | 225 kB 00:00 2024-05-12T03:48:13.255 INFO:teuthology.orchestra.run.smithi033.stdout:(5/8): python3-scipy-1.0.0-21.module_el8+762+77 16 MB/s | 14 MB 00:00 2024-05-12T03:48:13.503 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-939.gc190bdee.el8.n 5/6 2024-05-12T03:48:13.617 INFO:teuthology.orchestra.run.smithi033.stdout:(6/8): libgfortran-8.5.0-22.el8.x86_64.rpm 533 kB/s | 645 kB 00:01 2024-05-12T03:48:13.732 INFO:teuthology.orchestra.run.smithi033.stdout:(7/8): libquadmath-8.5.0-22.el8.x86_64.rpm 362 kB/s | 172 kB 00:00 2024-05-12T03:48:13.883 INFO:teuthology.orchestra.run.smithi033.stdout:(8/8): ceph-mgr-diskprediction-local-17.2.7-939 3.6 MB/s | 7.4 MB 00:02 2024-05-12T03:48:13.884 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2024-05-12T03:48:13.885 INFO:teuthology.orchestra.run.smithi033.stdout:Total 15 MB/s | 36 MB 00:02 2024-05-12T03:48:13.988 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-939.gc190bdee.el8.noarch 6/6 2024-05-12T03:48:14.125 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2024-05-12T03:48:14.145 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction check succeeded. 2024-05-12T03:48:14.145 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2024-05-12T03:48:14.277 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-939.gc190bdee.el8.noarch 6/6 2024-05-12T03:48:14.277 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-939.gc190bdee.el8.n 1/6 2024-05-12T03:48:14.277 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-939.gc190bdee.el8.noarch 2/6 2024-05-12T03:48:14.277 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-939.gc190bdee.el8.no 3/6 2024-05-12T03:48:14.277 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 4/6 2024-05-12T03:48:14.277 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 5/6 2024-05-12T03:48:14.552 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded. 2024-05-12T03:48:14.556 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2024-05-12T03:48:14.688 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 6/6 2024-05-12T03:48:14.688 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:48:14.688 INFO:teuthology.orchestra.run.smithi032.stdout:Installed: 2024-05-12T03:48:14.688 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-grafana-dashboards-2:17.2.7-939.gc190bdee.el8.noarch 2024-05-12T03:48:14.688 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-dashboard-2:17.2.7-939.gc190bdee.el8.noarch 2024-05-12T03:48:14.688 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-prometheus-alerts-2:17.2.7-939.gc190bdee.el8.noarch 2024-05-12T03:48:14.688 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-05-12T03:48:14.688 INFO:teuthology.orchestra.run.smithi032.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-05-12T03:48:14.689 INFO:teuthology.orchestra.run.smithi032.stdout: python3-routes-2.4.1-12.el8.noarch 2024-05-12T03:48:14.689 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:48:14.689 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2024-05-12T03:48:14.822 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum -y install ceph-mgr-diskprediction-local 2024-05-12T03:48:15.219 INFO:teuthology.orchestra.run.smithi033.stdout: Preparing : 1/1 2024-05-12T03:48:15.388 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libquadmath-8.5.0-22.el8.x86_64 1/8 2024-05-12T03:48:15.570 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libquadmath-8.5.0-22.el8.x86_64 1/8 2024-05-12T03:48:15.603 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:01:43 ago on Sun 12 May 2024 03:46:32 AM UTC. 2024-05-12T03:48:15.771 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libgfortran-8.5.0-22.el8.x86_64 2/8 2024-05-12T03:48:16.409 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libgfortran-8.5.0-22.el8.x86_64 2/8 2024-05-12T03:48:16.604 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : openblas-0.3.15-4.el8.x86_64 3/8 2024-05-12T03:48:17.260 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 3/8 2024-05-12T03:48:17.366 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-05-12T03:48:17.368 INFO:teuthology.orchestra.run.smithi032.stdout:============================================================================================ 2024-05-12T03:48:17.368 INFO:teuthology.orchestra.run.smithi032.stdout: Package Arch Version Repository Size 2024-05-12T03:48:17.368 INFO:teuthology.orchestra.run.smithi032.stdout:============================================================================================ 2024-05-12T03:48:17.368 INFO:teuthology.orchestra.run.smithi032.stdout:Installing: 2024-05-12T03:48:17.368 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-939.gc190bdee.el8 ceph-noarch 7.4 M 2024-05-12T03:48:17.368 INFO:teuthology.orchestra.run.smithi032.stdout:Installing dependencies: 2024-05-12T03:48:17.368 INFO:teuthology.orchestra.run.smithi032.stdout: libgfortran x86_64 8.5.0-22.el8 baseos 645 k 2024-05-12T03:48:17.368 INFO:teuthology.orchestra.run.smithi032.stdout: libquadmath x86_64 8.5.0-22.el8 baseos 172 k 2024-05-12T03:48:17.368 INFO:teuthology.orchestra.run.smithi032.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2024-05-12T03:48:17.368 INFO:teuthology.orchestra.run.smithi032.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2024-05-12T03:48:17.368 INFO:teuthology.orchestra.run.smithi032.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2024-05-12T03:48:17.369 INFO:teuthology.orchestra.run.smithi032.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2024-05-12T03:48:17.369 INFO:teuthology.orchestra.run.smithi032.stdout: python3-scipy x86_64 1.0.0-21.module_el8+762+77bd8591 appstream 14 M 2024-05-12T03:48:17.369 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:48:17.369 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction Summary 2024-05-12T03:48:17.369 INFO:teuthology.orchestra.run.smithi032.stdout:============================================================================================ 2024-05-12T03:48:17.369 INFO:teuthology.orchestra.run.smithi032.stdout:Install 8 Packages 2024-05-12T03:48:17.369 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:48:17.369 INFO:teuthology.orchestra.run.smithi032.stdout:Total download size: 36 M 2024-05-12T03:48:17.369 INFO:teuthology.orchestra.run.smithi032.stdout:Installed size: 210 M 2024-05-12T03:48:17.370 INFO:teuthology.orchestra.run.smithi032.stdout:Downloading Packages: 2024-05-12T03:48:17.403 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 4/8 2024-05-12T03:48:17.965 INFO:teuthology.orchestra.run.smithi032.stdout:(1/8): openblas-0.3.15-4.el8.x86_64.rpm 13 MB/s | 4.7 MB 00:00 2024-05-12T03:48:18.040 INFO:teuthology.orchestra.run.smithi032.stdout:(2/8): openblas-threads-0.3.15-4.el8.x86_64.rpm 11 MB/s | 4.9 MB 00:00 2024-05-12T03:48:18.108 INFO:teuthology.orchestra.run.smithi032.stdout:(3/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 3.3 MB/s | 225 kB 00:00 2024-05-12T03:48:18.162 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 4/8 2024-05-12T03:48:18.166 INFO:teuthology.orchestra.run.smithi032.stdout:(4/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 19 MB/s | 3.7 MB 00:00 2024-05-12T03:48:18.400 INFO:teuthology.orchestra.run.smithi032.stdout:(5/8): ceph-mgr-diskprediction-local-17.2.7-939 9.3 MB/s | 7.4 MB 00:00 2024-05-12T03:48:18.528 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2024-05-12T03:48:18.550 INFO:teuthology.orchestra.run.smithi032.stdout:(6/8): python3-scipy-1.0.0-21.module_el8+762+77 32 MB/s | 14 MB 00:00 2024-05-12T03:48:18.850 INFO:teuthology.orchestra.run.smithi032.stdout:(7/8): libgfortran-8.5.0-22.el8.x86_64.rpm 944 kB/s | 645 kB 00:00 2024-05-12T03:48:18.959 INFO:teuthology.orchestra.run.smithi032.stdout:(8/8): libquadmath-8.5.0-22.el8.x86_64.rpm 309 kB/s | 172 kB 00:00 2024-05-12T03:48:18.960 INFO:teuthology.orchestra.run.smithi032.stdout:-------------------------------------------------------------------------------- 2024-05-12T03:48:18.960 INFO:teuthology.orchestra.run.smithi032.stdout:Total 23 MB/s | 36 MB 00:01 2024-05-12T03:48:19.198 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction check 2024-05-12T03:48:19.217 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction check succeeded. 2024-05-12T03:48:19.217 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction test 2024-05-12T03:48:19.279 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-05-12T03:48:20.088 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction test succeeded. 2024-05-12T03:48:20.091 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction 2024-05-12T03:48:20.852 INFO:teuthology.orchestra.run.smithi032.stdout: Preparing : 1/1 2024-05-12T03:48:20.998 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libquadmath-8.5.0-22.el8.x86_64 1/8 2024-05-12T03:48:21.143 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libquadmath-8.5.0-22.el8.x86_64 1/8 2024-05-12T03:48:21.155 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-05-12T03:48:21.290 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libgfortran-8.5.0-22.el8.x86_64 2/8 2024-05-12T03:48:21.926 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libgfortran-8.5.0-22.el8.x86_64 2/8 2024-05-12T03:48:22.115 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : openblas-0.3.15-4.el8.x86_64 3/8 2024-05-12T03:48:22.484 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_6 7/8 2024-05-12T03:48:22.695 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-939.gc190bdee 8/8 2024-05-12T03:48:22.768 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 3/8 2024-05-12T03:48:22.914 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 4/8 2024-05-12T03:48:23.036 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-939.gc190bdee 8/8 2024-05-12T03:48:23.036 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-939.gc190bdee 1/8 2024-05-12T03:48:23.036 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 2/8 2024-05-12T03:48:23.036 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 3/8 2024-05-12T03:48:23.037 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 4/8 2024-05-12T03:48:23.037 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 5/8 2024-05-12T03:48:23.037 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_6 6/8 2024-05-12T03:48:23.037 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libgfortran-8.5.0-22.el8.x86_64 7/8 2024-05-12T03:48:23.502 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libquadmath-8.5.0-22.el8.x86_64 8/8 2024-05-12T03:48:23.502 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:48:23.502 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2024-05-12T03:48:23.502 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-diskprediction-local-2:17.2.7-939.gc190bdee.el8.noarch 2024-05-12T03:48:23.502 INFO:teuthology.orchestra.run.smithi033.stdout: libgfortran-8.5.0-22.el8.x86_64 2024-05-12T03:48:23.502 INFO:teuthology.orchestra.run.smithi033.stdout: libquadmath-8.5.0-22.el8.x86_64 2024-05-12T03:48:23.503 INFO:teuthology.orchestra.run.smithi033.stdout: openblas-0.3.15-4.el8.x86_64 2024-05-12T03:48:23.503 INFO:teuthology.orchestra.run.smithi033.stdout: openblas-threads-0.3.15-4.el8.x86_64 2024-05-12T03:48:23.503 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-05-12T03:48:23.503 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-05-12T03:48:23.503 INFO:teuthology.orchestra.run.smithi033.stdout: python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_64 2024-05-12T03:48:23.503 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:48:23.503 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-05-12T03:48:23.651 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 4/8 2024-05-12T03:48:23.771 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum -y install ceph-mgr-rook 2024-05-12T03:48:24.023 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2024-05-12T03:48:24.265 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-05-12T03:48:24.553 INFO:teuthology.orchestra.run.smithi033.stdout:Last metadata expiration check: 0:01:57 ago on Sun 12 May 2024 03:46:27 AM UTC. 2024-05-12T03:48:26.139 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-05-12T03:48:26.333 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-05-12T03:48:26.335 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-05-12T03:48:26.335 INFO:teuthology.orchestra.run.smithi033.stdout: Package Arch Version Repository Size 2024-05-12T03:48:26.335 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-05-12T03:48:26.335 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2024-05-12T03:48:26.335 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-rook noarch 2:17.2.7-939.gc190bdee.el8 ceph-noarch 47 k 2024-05-12T03:48:26.335 INFO:teuthology.orchestra.run.smithi033.stdout:Installing dependencies: 2024-05-12T03:48:26.335 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-05-12T03:48:26.336 INFO:teuthology.orchestra.run.smithi033.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-05-12T03:48:26.336 INFO:teuthology.orchestra.run.smithi033.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-05-12T03:48:26.336 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpatch noarch 1.21-2.el8 appstream 27 k 2024-05-12T03:48:26.336 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpointer noarch 1.10-11.el8 appstream 20 k 2024-05-12T03:48:26.336 INFO:teuthology.orchestra.run.smithi033.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-05-12T03:48:26.336 INFO:teuthology.orchestra.run.smithi033.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2024-05-12T03:48:26.336 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2024-05-12T03:48:26.336 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2024-05-12T03:48:26.336 INFO:teuthology.orchestra.run.smithi033.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2024-05-12T03:48:26.336 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-05-12T03:48:26.336 INFO:teuthology.orchestra.run.smithi033.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-05-12T03:48:26.336 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:48:26.336 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2024-05-12T03:48:26.337 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-05-12T03:48:26.337 INFO:teuthology.orchestra.run.smithi033.stdout:Install 13 Packages 2024-05-12T03:48:26.337 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:48:26.337 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 1.9 M 2024-05-12T03:48:26.337 INFO:teuthology.orchestra.run.smithi033.stdout:Installed size: 25 M 2024-05-12T03:48:26.338 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading Packages: 2024-05-12T03:48:26.847 INFO:teuthology.orchestra.run.smithi033.stdout:(1/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 182 kB/s | 27 kB 00:00 2024-05-12T03:48:26.872 INFO:teuthology.orchestra.run.smithi033.stdout:(2/13): python3-jsonpointer-1.10-11.el8.noarch. 117 kB/s | 20 kB 00:00 2024-05-12T03:48:26.964 INFO:teuthology.orchestra.run.smithi033.stdout:(3/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.1 MB/s | 126 kB 00:00 2024-05-12T03:48:26.990 INFO:teuthology.orchestra.run.smithi033.stdout:(4/13): ceph-mgr-rook-17.2.7-939.gc190bdee.el8. 164 kB/s | 47 kB 00:00 2024-05-12T03:48:27.015 INFO:teuthology.orchestra.run.smithi033.stdout:(5/13): python3-pyasn1-modules-0.3.7-6.el8.noar 771 kB/s | 110 kB 00:00 2024-05-12T03:48:27.249 INFO:teuthology.orchestra.run.smithi033.stdout:(6/13): python3-cachetools-3.1.1-4.el8.noarch.r 141 kB/s | 33 kB 00:00 2024-05-12T03:48:27.307 INFO:teuthology.orchestra.run.smithi033.stdout:(7/13): python3-requests-oauthlib-1.0.0-1.el8.n 134 kB/s | 43 kB 00:00 2024-05-12T03:48:27.333 INFO:teuthology.orchestra.run.smithi033.stdout:(8/13): python3-certifi-2018.10.15-7.el8.noarch 195 kB/s | 16 kB 00:00 2024-05-12T03:48:27.400 INFO:teuthology.orchestra.run.smithi033.stdout:(9/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 356 kB/s | 155 kB 00:00 2024-05-12T03:48:27.403 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_6 7/8 2024-05-12T03:48:27.542 INFO:teuthology.orchestra.run.smithi033.stdout:(10/13): python3-google-auth-1.1.1-10.el8.noarc 347 kB/s | 81 kB 00:00 2024-05-12T03:48:27.575 INFO:teuthology.orchestra.run.smithi033.stdout:(11/13): python3-kubernetes-11.0.0-6.el8.noarch 4.8 MB/s | 1.2 MB 00:00 2024-05-12T03:48:27.609 INFO:teuthology.orchestra.run.smithi033.stdout:(12/13): python3-websocket-client-0.56.0-5.el8. 918 kB/s | 61 kB 00:00 2024-05-12T03:48:27.623 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-939.gc190bdee 8/8 2024-05-12T03:48:27.634 INFO:teuthology.orchestra.run.smithi033.stdout:(13/13): python3-rsa-4.9-2.el8.noarch.rpm 275 kB/s | 64 kB 00:00 2024-05-12T03:48:27.637 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2024-05-12T03:48:27.637 INFO:teuthology.orchestra.run.smithi033.stdout:Total 1.5 MB/s | 1.9 MB 00:01 2024-05-12T03:48:27.694 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2024-05-12T03:48:27.710 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction check succeeded. 2024-05-12T03:48:27.711 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2024-05-12T03:48:27.805 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded. 2024-05-12T03:48:27.809 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2024-05-12T03:48:27.959 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-939.gc190bdee 8/8 2024-05-12T03:48:27.959 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-939.gc190bdee 1/8 2024-05-12T03:48:27.960 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 2/8 2024-05-12T03:48:27.960 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 3/8 2024-05-12T03:48:27.960 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 4/8 2024-05-12T03:48:27.960 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 5/8 2024-05-12T03:48:27.960 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_6 6/8 2024-05-12T03:48:27.960 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libgfortran-8.5.0-22.el8.x86_64 7/8 2024-05-12T03:48:28.360 INFO:teuthology.orchestra.run.smithi033.stdout: Preparing : 1/1 2024-05-12T03:48:28.375 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libquadmath-8.5.0-22.el8.x86_64 8/8 2024-05-12T03:48:28.375 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:48:28.375 INFO:teuthology.orchestra.run.smithi032.stdout:Installed: 2024-05-12T03:48:28.375 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-diskprediction-local-2:17.2.7-939.gc190bdee.el8.noarch 2024-05-12T03:48:28.375 INFO:teuthology.orchestra.run.smithi032.stdout: libgfortran-8.5.0-22.el8.x86_64 2024-05-12T03:48:28.375 INFO:teuthology.orchestra.run.smithi032.stdout: libquadmath-8.5.0-22.el8.x86_64 2024-05-12T03:48:28.376 INFO:teuthology.orchestra.run.smithi032.stdout: openblas-0.3.15-4.el8.x86_64 2024-05-12T03:48:28.376 INFO:teuthology.orchestra.run.smithi032.stdout: openblas-threads-0.3.15-4.el8.x86_64 2024-05-12T03:48:28.376 INFO:teuthology.orchestra.run.smithi032.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-05-12T03:48:28.376 INFO:teuthology.orchestra.run.smithi032.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-05-12T03:48:28.376 INFO:teuthology.orchestra.run.smithi032.stdout: python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_64 2024-05-12T03:48:28.376 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:48:28.376 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2024-05-12T03:48:28.488 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum -y install ceph-mgr-rook 2024-05-12T03:48:28.591 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2024-05-12T03:48:28.788 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 2/13 2024-05-12T03:48:29.056 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rsa-4.9-2.el8.noarch 3/13 2024-05-12T03:48:29.271 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:01:57 ago on Sun 12 May 2024 03:46:32 AM UTC. 2024-05-12T03:48:29.302 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 4/13 2024-05-12T03:48:29.553 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 5/13 2024-05-12T03:48:29.813 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 6/13 2024-05-12T03:48:30.133 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 7/13 2024-05-12T03:48:30.418 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2024-05-12T03:48:31.038 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-05-12T03:48:31.040 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-05-12T03:48:31.040 INFO:teuthology.orchestra.run.smithi032.stdout: Package Arch Version Repository Size 2024-05-12T03:48:31.040 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-05-12T03:48:31.040 INFO:teuthology.orchestra.run.smithi032.stdout:Installing: 2024-05-12T03:48:31.040 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-rook noarch 2:17.2.7-939.gc190bdee.el8 ceph-noarch 47 k 2024-05-12T03:48:31.040 INFO:teuthology.orchestra.run.smithi032.stdout:Installing dependencies: 2024-05-12T03:48:31.040 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-05-12T03:48:31.040 INFO:teuthology.orchestra.run.smithi032.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-05-12T03:48:31.040 INFO:teuthology.orchestra.run.smithi032.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-05-12T03:48:31.040 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jsonpatch noarch 1.21-2.el8 appstream 27 k 2024-05-12T03:48:31.040 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jsonpointer noarch 1.10-11.el8 appstream 20 k 2024-05-12T03:48:31.040 INFO:teuthology.orchestra.run.smithi032.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-05-12T03:48:31.040 INFO:teuthology.orchestra.run.smithi032.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2024-05-12T03:48:31.041 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2024-05-12T03:48:31.041 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2024-05-12T03:48:31.041 INFO:teuthology.orchestra.run.smithi032.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2024-05-12T03:48:31.041 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-05-12T03:48:31.041 INFO:teuthology.orchestra.run.smithi032.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-05-12T03:48:31.041 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:48:31.041 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction Summary 2024-05-12T03:48:31.041 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-05-12T03:48:31.041 INFO:teuthology.orchestra.run.smithi032.stdout:Install 13 Packages 2024-05-12T03:48:31.041 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:48:31.042 INFO:teuthology.orchestra.run.smithi032.stdout:Total download size: 1.9 M 2024-05-12T03:48:31.042 INFO:teuthology.orchestra.run.smithi032.stdout:Installed size: 25 M 2024-05-12T03:48:31.042 INFO:teuthology.orchestra.run.smithi032.stdout:Downloading Packages: 2024-05-12T03:48:31.465 INFO:teuthology.orchestra.run.smithi032.stdout:(1/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 216 kB/s | 27 kB 00:00 2024-05-12T03:48:31.491 INFO:teuthology.orchestra.run.smithi032.stdout:(2/13): python3-jsonpointer-1.10-11.el8.noarch. 134 kB/s | 20 kB 00:00 2024-05-12T03:48:31.516 INFO:teuthology.orchestra.run.smithi032.stdout:(3/13): ceph-mgr-rook-17.2.7-939.gc190bdee.el8. 272 kB/s | 47 kB 00:00 2024-05-12T03:48:31.535 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2024-05-12T03:48:31.566 INFO:teuthology.orchestra.run.smithi032.stdout:(4/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.2 MB/s | 126 kB 00:00 2024-05-12T03:48:31.591 INFO:teuthology.orchestra.run.smithi032.stdout:(5/13): python3-pyasn1-modules-0.3.7-6.el8.noar 1.1 MB/s | 110 kB 00:00 2024-05-12T03:48:31.775 INFO:teuthology.orchestra.run.smithi032.stdout:(6/13): python3-cachetools-3.1.1-4.el8.noarch.r 180 kB/s | 33 kB 00:00 2024-05-12T03:48:31.817 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 10/13 2024-05-12T03:48:31.843 INFO:teuthology.orchestra.run.smithi032.stdout:(7/13): python3-requests-oauthlib-1.0.0-1.el8.n 155 kB/s | 43 kB 00:00 2024-05-12T03:48:31.868 INFO:teuthology.orchestra.run.smithi032.stdout:(8/13): python3-certifi-2018.10.15-7.el8.noarch 177 kB/s | 16 kB 00:00 2024-05-12T03:48:31.985 INFO:teuthology.orchestra.run.smithi032.stdout:(9/13): python3-google-auth-1.1.1-10.el8.noarch 698 kB/s | 81 kB 00:00 2024-05-12T03:48:32.043 INFO:teuthology.orchestra.run.smithi032.stdout:(10/13): python3-oauthlib-2.1.0-1.el8.noarch.rp 294 kB/s | 155 kB 00:00 2024-05-12T03:48:32.068 INFO:teuthology.orchestra.run.smithi032.stdout:(11/13): python3-rsa-4.9-2.el8.noarch.rpm 770 kB/s | 64 kB 00:00 2024-05-12T03:48:32.077 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 11/13 2024-05-12T03:48:32.252 INFO:teuthology.orchestra.run.smithi032.stdout:(12/13): python3-websocket-client-0.56.0-5.el8. 294 kB/s | 61 kB 00:00 2024-05-12T03:48:32.286 INFO:teuthology.orchestra.run.smithi032.stdout:(13/13): python3-kubernetes-11.0.0-6.el8.noarch 2.8 MB/s | 1.2 MB 00:00 2024-05-12T03:48:32.286 INFO:teuthology.orchestra.run.smithi032.stdout:-------------------------------------------------------------------------------- 2024-05-12T03:48:32.286 INFO:teuthology.orchestra.run.smithi032.stdout:Total 1.6 MB/s | 1.9 MB 00:01 2024-05-12T03:48:32.314 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 12/13 2024-05-12T03:48:32.347 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction check 2024-05-12T03:48:32.364 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction check succeeded. 2024-05-12T03:48:32.364 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction test 2024-05-12T03:48:32.461 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction test succeeded. 2024-05-12T03:48:32.464 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction 2024-05-12T03:48:32.527 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-rook-2:17.2.7-939.gc190bdee.el8.noarch 13/13 2024-05-12T03:48:32.848 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-939.gc190bdee.el8.noarch 13/13 2024-05-12T03:48:32.848 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-rook-2:17.2.7-939.gc190bdee.el8.noarch 1/13 2024-05-12T03:48:32.848 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 2/13 2024-05-12T03:48:32.848 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 3/13 2024-05-12T03:48:32.848 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 4/13 2024-05-12T03:48:32.848 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 5/13 2024-05-12T03:48:32.848 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2024-05-12T03:48:32.849 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2024-05-12T03:48:32.849 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 8/13 2024-05-12T03:48:32.849 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 9/13 2024-05-12T03:48:32.849 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 10/13 2024-05-12T03:48:32.849 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 11/13 2024-05-12T03:48:32.849 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 12/13 2024-05-12T03:48:33.005 INFO:teuthology.orchestra.run.smithi032.stdout: Preparing : 1/1 2024-05-12T03:48:33.244 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2024-05-12T03:48:33.424 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 2/13 2024-05-12T03:48:33.482 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 13/13 2024-05-12T03:48:33.482 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:48:33.482 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2024-05-12T03:48:33.483 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-rook-2:17.2.7-939.gc190bdee.el8.noarch 2024-05-12T03:48:33.483 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-05-12T03:48:33.483 INFO:teuthology.orchestra.run.smithi033.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-05-12T03:48:33.483 INFO:teuthology.orchestra.run.smithi033.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-05-12T03:48:33.483 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-05-12T03:48:33.483 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-05-12T03:48:33.483 INFO:teuthology.orchestra.run.smithi033.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-05-12T03:48:33.483 INFO:teuthology.orchestra.run.smithi033.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-05-12T03:48:33.484 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-05-12T03:48:33.484 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-05-12T03:48:33.484 INFO:teuthology.orchestra.run.smithi033.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-05-12T03:48:33.484 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rsa-4.9-2.el8.noarch 2024-05-12T03:48:33.484 INFO:teuthology.orchestra.run.smithi033.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-05-12T03:48:33.484 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:48:33.484 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-05-12T03:48:33.628 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum -y install ceph-mgr-cephadm 2024-05-12T03:48:33.701 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-rsa-4.9-2.el8.noarch 3/13 2024-05-12T03:48:33.913 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 4/13 2024-05-12T03:48:34.146 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 5/13 2024-05-12T03:48:34.364 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 6/13 2024-05-12T03:48:34.414 INFO:teuthology.orchestra.run.smithi033.stdout:Last metadata expiration check: 0:02:07 ago on Sun 12 May 2024 03:46:27 AM UTC. 2024-05-12T03:48:34.675 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 7/13 2024-05-12T03:48:34.895 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2024-05-12T03:48:36.013 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2024-05-12T03:48:36.188 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-05-12T03:48:36.189 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-05-12T03:48:36.190 INFO:teuthology.orchestra.run.smithi033.stdout: Package Arch Version Repository Size 2024-05-12T03:48:36.190 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-05-12T03:48:36.190 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2024-05-12T03:48:36.190 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-cephadm noarch 2:17.2.7-939.gc190bdee.el8 ceph-noarch 123 k 2024-05-12T03:48:36.190 INFO:teuthology.orchestra.run.smithi033.stdout:Installing dependencies: 2024-05-12T03:48:36.190 INFO:teuthology.orchestra.run.smithi033.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-05-12T03:48:36.190 INFO:teuthology.orchestra.run.smithi033.stdout: python3-babel noarch 2.5.1-7.el8 appstream 4.8 M 2024-05-12T03:48:36.190 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jinja2 noarch 2.10.1-4.el8 appstream 578 k 2024-05-12T03:48:36.190 INFO:teuthology.orchestra.run.smithi033.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-05-12T03:48:36.190 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:48:36.190 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2024-05-12T03:48:36.190 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-05-12T03:48:36.190 INFO:teuthology.orchestra.run.smithi033.stdout:Install 5 Packages 2024-05-12T03:48:36.190 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:48:36.191 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 6.0 M 2024-05-12T03:48:36.191 INFO:teuthology.orchestra.run.smithi033.stdout:Installed size: 26 M 2024-05-12T03:48:36.191 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading Packages: 2024-05-12T03:48:36.254 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 10/13 2024-05-12T03:48:36.466 INFO:teuthology.orchestra.run.smithi033.stdout:(1/5): ceph-mgr-cephadm-17.2.7-939.gc190bdee.el 654 kB/s | 123 kB 00:00 2024-05-12T03:48:36.508 INFO:teuthology.orchestra.run.smithi033.stdout:(2/5): python3-jinja2-2.10.1-4.el8.noarch.rpm 2.5 MB/s | 578 kB 00:00 2024-05-12T03:48:36.513 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 11/13 2024-05-12T03:48:36.683 INFO:teuthology.orchestra.run.smithi033.stdout:(3/5): python3-babel-2.5.1-7.el8.noarch.rpm 12 MB/s | 4.8 MB 00:00 2024-05-12T03:48:36.717 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 12/13 2024-05-12T03:48:36.742 INFO:teuthology.orchestra.run.smithi033.stdout:(4/5): python3-natsort-7.1.1-2.el8.noarch.rpm 264 kB/s | 60 kB 00:00 2024-05-12T03:48:36.833 INFO:teuthology.orchestra.run.smithi033.stdout:(5/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 1.3 MB/s | 476 kB 00:00 2024-05-12T03:48:36.835 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2024-05-12T03:48:36.835 INFO:teuthology.orchestra.run.smithi033.stdout:Total 9.3 MB/s | 6.0 MB 00:00 2024-05-12T03:48:36.887 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2024-05-12T03:48:36.896 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-rook-2:17.2.7-939.gc190bdee.el8.noarch 13/13 2024-05-12T03:48:36.898 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction check succeeded. 2024-05-12T03:48:36.898 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2024-05-12T03:48:37.003 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded. 2024-05-12T03:48:37.006 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2024-05-12T03:48:37.213 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-939.gc190bdee.el8.noarch 13/13 2024-05-12T03:48:37.213 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-rook-2:17.2.7-939.gc190bdee.el8.noarch 1/13 2024-05-12T03:48:37.213 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 2/13 2024-05-12T03:48:37.213 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 3/13 2024-05-12T03:48:37.213 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 4/13 2024-05-12T03:48:37.213 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 5/13 2024-05-12T03:48:37.214 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2024-05-12T03:48:37.214 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2024-05-12T03:48:37.214 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 8/13 2024-05-12T03:48:37.214 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 9/13 2024-05-12T03:48:37.214 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 10/13 2024-05-12T03:48:37.214 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 11/13 2024-05-12T03:48:37.214 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 12/13 2024-05-12T03:48:37.291 INFO:teuthology.orchestra.run.smithi033.stdout: Preparing : 1/1 2024-05-12T03:48:37.666 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 1/5 2024-05-12T03:48:37.727 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 13/13 2024-05-12T03:48:37.727 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:48:37.727 INFO:teuthology.orchestra.run.smithi032.stdout:Installed: 2024-05-12T03:48:37.727 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-rook-2:17.2.7-939.gc190bdee.el8.noarch 2024-05-12T03:48:37.727 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-05-12T03:48:37.727 INFO:teuthology.orchestra.run.smithi032.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-05-12T03:48:37.727 INFO:teuthology.orchestra.run.smithi032.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-05-12T03:48:37.727 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-05-12T03:48:37.727 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-05-12T03:48:37.727 INFO:teuthology.orchestra.run.smithi032.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-05-12T03:48:37.727 INFO:teuthology.orchestra.run.smithi032.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-05-12T03:48:37.727 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-05-12T03:48:37.727 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-05-12T03:48:37.728 INFO:teuthology.orchestra.run.smithi032.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-05-12T03:48:37.728 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rsa-4.9-2.el8.noarch 2024-05-12T03:48:37.728 INFO:teuthology.orchestra.run.smithi032.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-05-12T03:48:37.728 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:48:37.728 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2024-05-12T03:48:37.879 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum -y install ceph-mgr-cephadm 2024-05-12T03:48:38.662 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:02:06 ago on Sun 12 May 2024 03:46:32 AM UTC. 2024-05-12T03:48:38.720 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2024-05-12T03:48:39.127 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 3/5 2024-05-12T03:48:39.405 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jinja2-2.10.1-4.el8.noarch 4/5 2024-05-12T03:48:40.433 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-05-12T03:48:40.433 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-05-12T03:48:40.434 INFO:teuthology.orchestra.run.smithi032.stdout: Package Arch Version Repository Size 2024-05-12T03:48:40.434 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-05-12T03:48:40.434 INFO:teuthology.orchestra.run.smithi032.stdout:Installing: 2024-05-12T03:48:40.434 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-cephadm noarch 2:17.2.7-939.gc190bdee.el8 ceph-noarch 123 k 2024-05-12T03:48:40.434 INFO:teuthology.orchestra.run.smithi032.stdout:Installing dependencies: 2024-05-12T03:48:40.434 INFO:teuthology.orchestra.run.smithi032.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-05-12T03:48:40.434 INFO:teuthology.orchestra.run.smithi032.stdout: python3-babel noarch 2.5.1-7.el8 appstream 4.8 M 2024-05-12T03:48:40.434 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jinja2 noarch 2.10.1-4.el8 appstream 578 k 2024-05-12T03:48:40.434 INFO:teuthology.orchestra.run.smithi032.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-05-12T03:48:40.435 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:48:40.435 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction Summary 2024-05-12T03:48:40.435 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-05-12T03:48:40.435 INFO:teuthology.orchestra.run.smithi032.stdout:Install 5 Packages 2024-05-12T03:48:40.435 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:48:40.435 INFO:teuthology.orchestra.run.smithi032.stdout:Total download size: 6.0 M 2024-05-12T03:48:40.435 INFO:teuthology.orchestra.run.smithi032.stdout:Installed size: 26 M 2024-05-12T03:48:40.436 INFO:teuthology.orchestra.run.smithi032.stdout:Downloading Packages: 2024-05-12T03:48:40.542 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-939.gc190bdee.el8.noarch 5/5 2024-05-12T03:48:40.843 INFO:teuthology.orchestra.run.smithi032.stdout:(1/5): python3-jinja2-2.10.1-4.el8.noarch.rpm 2.5 MB/s | 578 kB 00:00 2024-05-12T03:48:40.885 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-939.gc190bdee.el8.noarch 5/5 2024-05-12T03:48:40.885 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-939.gc190bdee.el8.noarch 1/5 2024-05-12T03:48:40.885 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 2/5 2024-05-12T03:48:40.885 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jinja2-2.10.1-4.el8.noarch 3/5 2024-05-12T03:48:40.885 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2024-05-12T03:48:40.901 INFO:teuthology.orchestra.run.smithi032.stdout:(2/5): ceph-mgr-cephadm-17.2.7-939.gc190bdee.el 432 kB/s | 123 kB 00:00 2024-05-12T03:48:40.968 INFO:teuthology.orchestra.run.smithi032.stdout:(3/5): python3-babel-2.5.1-7.el8.noarch.rpm 14 MB/s | 4.8 MB 00:00 2024-05-12T03:48:41.227 INFO:teuthology.orchestra.run.smithi032.stdout:(4/5): python3-natsort-7.1.1-2.el8.noarch.rpm 185 kB/s | 60 kB 00:00 2024-05-12T03:48:41.305 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 5/5 2024-05-12T03:48:41.305 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:48:41.305 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2024-05-12T03:48:41.305 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-cephadm-2:17.2.7-939.gc190bdee.el8.noarch 2024-05-12T03:48:41.306 INFO:teuthology.orchestra.run.smithi033.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-05-12T03:48:41.306 INFO:teuthology.orchestra.run.smithi033.stdout: python3-babel-2.5.1-7.el8.noarch 2024-05-12T03:48:41.306 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jinja2-2.10.1-4.el8.noarch 2024-05-12T03:48:41.306 INFO:teuthology.orchestra.run.smithi033.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-05-12T03:48:41.306 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:48:41.306 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-05-12T03:48:41.429 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum -y install ceph-fuse 2024-05-12T03:48:41.585 INFO:teuthology.orchestra.run.smithi032.stdout:(5/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 641 kB/s | 476 kB 00:00 2024-05-12T03:48:41.586 INFO:teuthology.orchestra.run.smithi032.stdout:-------------------------------------------------------------------------------- 2024-05-12T03:48:41.586 INFO:teuthology.orchestra.run.smithi032.stdout:Total 5.2 MB/s | 6.0 MB 00:01 2024-05-12T03:48:41.641 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction check 2024-05-12T03:48:41.654 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction check succeeded. 2024-05-12T03:48:41.654 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction test 2024-05-12T03:48:41.768 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction test succeeded. 2024-05-12T03:48:41.771 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction 2024-05-12T03:48:42.036 INFO:teuthology.orchestra.run.smithi032.stdout: Preparing : 1/1 2024-05-12T03:48:42.214 INFO:teuthology.orchestra.run.smithi033.stdout:Last metadata expiration check: 0:02:15 ago on Sun 12 May 2024 03:46:27 AM UTC. 2024-05-12T03:48:42.378 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 1/5 2024-05-12T03:48:43.383 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2024-05-12T03:48:43.741 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 3/5 2024-05-12T03:48:43.997 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-05-12T03:48:43.998 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-05-12T03:48:43.998 INFO:teuthology.orchestra.run.smithi033.stdout: Package Arch Version Repository Size 2024-05-12T03:48:43.998 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-05-12T03:48:43.998 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2024-05-12T03:48:43.998 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-fuse x86_64 2:17.2.7-939.gc190bdee.el8 ceph 877 k 2024-05-12T03:48:43.998 INFO:teuthology.orchestra.run.smithi033.stdout:Installing dependencies: 2024-05-12T03:48:43.998 INFO:teuthology.orchestra.run.smithi033.stdout: fuse x86_64 2.9.7-19.el8 baseos 83 k 2024-05-12T03:48:43.998 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:48:43.998 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2024-05-12T03:48:43.999 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-05-12T03:48:43.999 INFO:teuthology.orchestra.run.smithi033.stdout:Install 2 Packages 2024-05-12T03:48:43.999 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:48:43.999 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 960 k 2024-05-12T03:48:43.999 INFO:teuthology.orchestra.run.smithi033.stdout:Installed size: 3.1 M 2024-05-12T03:48:43.999 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading Packages: 2024-05-12T03:48:44.008 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jinja2-2.10.1-4.el8.noarch 4/5 2024-05-12T03:48:44.204 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-939.gc190bdee.el8.noarch 5/5 2024-05-12T03:48:44.417 INFO:teuthology.orchestra.run.smithi033.stdout:(1/2): fuse-2.9.7-19.el8.x86_64.rpm 228 kB/s | 83 kB 00:00 2024-05-12T03:48:44.451 INFO:teuthology.orchestra.run.smithi033.stdout:(2/2): ceph-fuse-17.2.7-939.gc190bdee.el8.x86_6 2.2 MB/s | 877 kB 00:00 2024-05-12T03:48:44.451 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2024-05-12T03:48:44.451 INFO:teuthology.orchestra.run.smithi033.stdout:Total 2.1 MB/s | 960 kB 00:00 2024-05-12T03:48:44.462 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2024-05-12T03:48:44.484 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction check succeeded. 2024-05-12T03:48:44.484 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2024-05-12T03:48:44.516 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-939.gc190bdee.el8.noarch 5/5 2024-05-12T03:48:44.516 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-939.gc190bdee.el8.noarch 1/5 2024-05-12T03:48:44.516 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 2/5 2024-05-12T03:48:44.516 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jinja2-2.10.1-4.el8.noarch 3/5 2024-05-12T03:48:44.517 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2024-05-12T03:48:44.578 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded. 2024-05-12T03:48:44.581 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2024-05-12T03:48:44.758 INFO:teuthology.orchestra.run.smithi033.stdout: Preparing : 1/1 2024-05-12T03:48:44.914 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 5/5 2024-05-12T03:48:44.914 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:48:44.914 INFO:teuthology.orchestra.run.smithi032.stdout:Installed: 2024-05-12T03:48:44.914 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-cephadm-2:17.2.7-939.gc190bdee.el8.noarch 2024-05-12T03:48:44.914 INFO:teuthology.orchestra.run.smithi032.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-05-12T03:48:44.914 INFO:teuthology.orchestra.run.smithi032.stdout: python3-babel-2.5.1-7.el8.noarch 2024-05-12T03:48:44.914 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jinja2-2.10.1-4.el8.noarch 2024-05-12T03:48:44.914 INFO:teuthology.orchestra.run.smithi032.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-05-12T03:48:44.914 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:48:44.915 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2024-05-12T03:48:45.040 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum -y install ceph-fuse 2024-05-12T03:48:45.055 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : fuse-2.9.7-19.el8.x86_64 1/2 2024-05-12T03:48:45.283 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-fuse-2:17.2.7-939.gc190bdee.el8.x86_64 2/2 2024-05-12T03:48:45.690 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-fuse-2:17.2.7-939.gc190bdee.el8.x86_64 2/2 2024-05-12T03:48:45.690 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-fuse-2:17.2.7-939.gc190bdee.el8.x86_64 1/2 2024-05-12T03:48:45.822 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:02:13 ago on Sun 12 May 2024 03:46:32 AM UTC. 2024-05-12T03:48:46.196 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : fuse-2.9.7-19.el8.x86_64 2/2 2024-05-12T03:48:46.197 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:48:46.197 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2024-05-12T03:48:46.197 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-fuse-2:17.2.7-939.gc190bdee.el8.x86_64 fuse-2.9.7-19.el8.x86_64 2024-05-12T03:48:46.197 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:48:46.197 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-05-12T03:48:46.413 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum -y install librados-devel 2024-05-12T03:48:47.210 INFO:teuthology.orchestra.run.smithi033.stdout:Last metadata expiration check: 0:02:20 ago on Sun 12 May 2024 03:46:27 AM UTC. 2024-05-12T03:48:47.604 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-05-12T03:48:47.604 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-05-12T03:48:47.605 INFO:teuthology.orchestra.run.smithi032.stdout: Package Arch Version Repository Size 2024-05-12T03:48:47.605 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-05-12T03:48:47.605 INFO:teuthology.orchestra.run.smithi032.stdout:Installing: 2024-05-12T03:48:47.605 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-fuse x86_64 2:17.2.7-939.gc190bdee.el8 ceph 877 k 2024-05-12T03:48:47.605 INFO:teuthology.orchestra.run.smithi032.stdout:Installing dependencies: 2024-05-12T03:48:47.605 INFO:teuthology.orchestra.run.smithi032.stdout: fuse x86_64 2.9.7-19.el8 baseos 83 k 2024-05-12T03:48:47.605 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:48:47.605 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction Summary 2024-05-12T03:48:47.605 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-05-12T03:48:47.605 INFO:teuthology.orchestra.run.smithi032.stdout:Install 2 Packages 2024-05-12T03:48:47.605 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:48:47.606 INFO:teuthology.orchestra.run.smithi032.stdout:Total download size: 960 k 2024-05-12T03:48:47.606 INFO:teuthology.orchestra.run.smithi032.stdout:Installed size: 3.1 M 2024-05-12T03:48:47.606 INFO:teuthology.orchestra.run.smithi032.stdout:Downloading Packages: 2024-05-12T03:48:48.144 INFO:teuthology.orchestra.run.smithi032.stdout:(1/2): fuse-2.9.7-19.el8.x86_64.rpm 234 kB/s | 83 kB 00:00 2024-05-12T03:48:48.285 INFO:teuthology.orchestra.run.smithi032.stdout:(2/2): ceph-fuse-17.2.7-939.gc190bdee.el8.x86_6 1.7 MB/s | 877 kB 00:00 2024-05-12T03:48:48.286 INFO:teuthology.orchestra.run.smithi032.stdout:-------------------------------------------------------------------------------- 2024-05-12T03:48:48.286 INFO:teuthology.orchestra.run.smithi032.stdout:Total 1.4 MB/s | 960 kB 00:00 2024-05-12T03:48:48.299 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction check 2024-05-12T03:48:48.322 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction check succeeded. 2024-05-12T03:48:48.322 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction test 2024-05-12T03:48:48.421 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction test succeeded. 2024-05-12T03:48:48.425 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction 2024-05-12T03:48:48.602 INFO:teuthology.orchestra.run.smithi032.stdout: Preparing : 1/1 2024-05-12T03:48:48.901 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : fuse-2.9.7-19.el8.x86_64 1/2 2024-05-12T03:48:48.983 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-05-12T03:48:48.984 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-05-12T03:48:48.984 INFO:teuthology.orchestra.run.smithi033.stdout: Package Arch Version Repo Size 2024-05-12T03:48:48.984 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-05-12T03:48:48.984 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2024-05-12T03:48:48.984 INFO:teuthology.orchestra.run.smithi033.stdout: librados-devel x86_64 2:17.2.7-939.gc190bdee.el8 ceph 130 k 2024-05-12T03:48:48.984 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:48:48.984 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2024-05-12T03:48:48.984 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-05-12T03:48:48.984 INFO:teuthology.orchestra.run.smithi033.stdout:Install 1 Package 2024-05-12T03:48:48.984 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:48:48.984 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 130 k 2024-05-12T03:48:48.984 INFO:teuthology.orchestra.run.smithi033.stdout:Installed size: 464 k 2024-05-12T03:48:48.985 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading Packages: 2024-05-12T03:48:49.137 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-fuse-2:17.2.7-939.gc190bdee.el8.x86_64 2/2 2024-05-12T03:48:49.273 INFO:teuthology.orchestra.run.smithi033.stdout:librados-devel-17.2.7-939.gc190bdee.el8.x86_64. 449 kB/s | 130 kB 00:00 2024-05-12T03:48:49.274 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2024-05-12T03:48:49.274 INFO:teuthology.orchestra.run.smithi033.stdout:Total 447 kB/s | 130 kB 00:00 2024-05-12T03:48:49.274 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2024-05-12T03:48:49.280 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction check succeeded. 2024-05-12T03:48:49.280 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2024-05-12T03:48:49.335 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded. 2024-05-12T03:48:49.338 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2024-05-12T03:48:49.464 INFO:teuthology.orchestra.run.smithi033.stdout: Preparing : 1/1 2024-05-12T03:48:49.542 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-fuse-2:17.2.7-939.gc190bdee.el8.x86_64 2/2 2024-05-12T03:48:49.543 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-fuse-2:17.2.7-939.gc190bdee.el8.x86_64 1/2 2024-05-12T03:48:49.690 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librados-devel-2:17.2.7-939.gc190bdee.el8.x86_64 1/1 2024-05-12T03:48:49.978 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : fuse-2.9.7-19.el8.x86_64 2/2 2024-05-12T03:48:49.979 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:48:49.979 INFO:teuthology.orchestra.run.smithi032.stdout:Installed: 2024-05-12T03:48:49.979 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-fuse-2:17.2.7-939.gc190bdee.el8.x86_64 fuse-2.9.7-19.el8.x86_64 2024-05-12T03:48:49.979 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:48:49.979 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2024-05-12T03:48:50.122 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librados-devel-2:17.2.7-939.gc190bdee.el8.x86_64 1/1 2024-05-12T03:48:50.213 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum -y install librados-devel 2024-05-12T03:48:50.502 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librados-devel-2:17.2.7-939.gc190bdee.el8.x86_64 1/1 2024-05-12T03:48:50.502 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:48:50.503 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2024-05-12T03:48:50.503 INFO:teuthology.orchestra.run.smithi033.stdout: librados-devel-2:17.2.7-939.gc190bdee.el8.x86_64 2024-05-12T03:48:50.503 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:48:50.503 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-05-12T03:48:50.706 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum -y install libcephfs2 2024-05-12T03:48:51.011 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:02:19 ago on Sun 12 May 2024 03:46:32 AM UTC. 2024-05-12T03:48:51.498 INFO:teuthology.orchestra.run.smithi033.stdout:Last metadata expiration check: 0:02:24 ago on Sun 12 May 2024 03:46:27 AM UTC. 2024-05-12T03:48:52.779 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-05-12T03:48:52.780 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-05-12T03:48:52.780 INFO:teuthology.orchestra.run.smithi032.stdout: Package Arch Version Repo Size 2024-05-12T03:48:52.780 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-05-12T03:48:52.780 INFO:teuthology.orchestra.run.smithi032.stdout:Installing: 2024-05-12T03:48:52.780 INFO:teuthology.orchestra.run.smithi032.stdout: librados-devel x86_64 2:17.2.7-939.gc190bdee.el8 ceph 130 k 2024-05-12T03:48:52.781 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:48:52.781 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction Summary 2024-05-12T03:48:52.781 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-05-12T03:48:52.781 INFO:teuthology.orchestra.run.smithi032.stdout:Install 1 Package 2024-05-12T03:48:52.781 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:48:52.781 INFO:teuthology.orchestra.run.smithi032.stdout:Total download size: 130 k 2024-05-12T03:48:52.781 INFO:teuthology.orchestra.run.smithi032.stdout:Installed size: 464 k 2024-05-12T03:48:52.781 INFO:teuthology.orchestra.run.smithi032.stdout:Downloading Packages: 2024-05-12T03:48:53.108 INFO:teuthology.orchestra.run.smithi032.stdout:librados-devel-17.2.7-939.gc190bdee.el8.x86_64. 396 kB/s | 130 kB 00:00 2024-05-12T03:48:53.109 INFO:teuthology.orchestra.run.smithi032.stdout:-------------------------------------------------------------------------------- 2024-05-12T03:48:53.109 INFO:teuthology.orchestra.run.smithi032.stdout:Total 394 kB/s | 130 kB 00:00 2024-05-12T03:48:53.109 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction check 2024-05-12T03:48:53.115 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction check succeeded. 2024-05-12T03:48:53.115 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction test 2024-05-12T03:48:53.170 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction test succeeded. 2024-05-12T03:48:53.172 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction 2024-05-12T03:48:53.249 INFO:teuthology.orchestra.run.smithi033.stdout:Package libcephfs2-2:17.2.7-939.gc190bdee.el8.x86_64 is already installed. 2024-05-12T03:48:53.285 INFO:teuthology.orchestra.run.smithi032.stdout: Preparing : 1/1 2024-05-12T03:48:53.301 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-05-12T03:48:53.302 INFO:teuthology.orchestra.run.smithi033.stdout:Nothing to do. 2024-05-12T03:48:53.302 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-05-12T03:48:53.469 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : librados-devel-2:17.2.7-939.gc190bdee.el8.x86_64 1/1 2024-05-12T03:48:53.691 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum -y install libcephfs-devel 2024-05-12T03:48:53.777 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librados-devel-2:17.2.7-939.gc190bdee.el8.x86_64 1/1 2024-05-12T03:48:54.200 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librados-devel-2:17.2.7-939.gc190bdee.el8.x86_64 1/1 2024-05-12T03:48:54.200 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:48:54.200 INFO:teuthology.orchestra.run.smithi032.stdout:Installed: 2024-05-12T03:48:54.200 INFO:teuthology.orchestra.run.smithi032.stdout: librados-devel-2:17.2.7-939.gc190bdee.el8.x86_64 2024-05-12T03:48:54.200 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:48:54.200 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2024-05-12T03:48:54.411 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum -y install libcephfs2 2024-05-12T03:48:54.480 INFO:teuthology.orchestra.run.smithi033.stdout:Last metadata expiration check: 0:02:27 ago on Sun 12 May 2024 03:46:27 AM UTC. 2024-05-12T03:48:55.200 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:02:23 ago on Sun 12 May 2024 03:46:32 AM UTC. 2024-05-12T03:48:56.241 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-05-12T03:48:56.242 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-05-12T03:48:56.242 INFO:teuthology.orchestra.run.smithi033.stdout: Package Arch Version Repo Size 2024-05-12T03:48:56.242 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-05-12T03:48:56.242 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2024-05-12T03:48:56.242 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs-devel x86_64 2:17.2.7-939.gc190bdee.el8 ceph 25 k 2024-05-12T03:48:56.242 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:48:56.243 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2024-05-12T03:48:56.243 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-05-12T03:48:56.243 INFO:teuthology.orchestra.run.smithi033.stdout:Install 1 Package 2024-05-12T03:48:56.243 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:48:56.243 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 25 k 2024-05-12T03:48:56.243 INFO:teuthology.orchestra.run.smithi033.stdout:Installed size: 109 k 2024-05-12T03:48:56.243 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading Packages: 2024-05-12T03:48:56.456 INFO:teuthology.orchestra.run.smithi033.stdout:libcephfs-devel-17.2.7-939.gc190bdee.el8.x86_64 117 kB/s | 25 kB 00:00 2024-05-12T03:48:56.456 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2024-05-12T03:48:56.457 INFO:teuthology.orchestra.run.smithi033.stdout:Total 116 kB/s | 25 kB 00:00 2024-05-12T03:48:56.457 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2024-05-12T03:48:56.460 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction check succeeded. 2024-05-12T03:48:56.460 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2024-05-12T03:48:56.462 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded. 2024-05-12T03:48:56.465 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2024-05-12T03:48:56.518 INFO:teuthology.orchestra.run.smithi033.stdout: Preparing : 1/1 2024-05-12T03:48:56.722 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libcephfs-devel-2:17.2.7-939.gc190bdee.el8.x86_64 1/1 2024-05-12T03:48:56.910 INFO:teuthology.orchestra.run.smithi032.stdout:Package libcephfs2-2:17.2.7-939.gc190bdee.el8.x86_64 is already installed. 2024-05-12T03:48:56.962 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-05-12T03:48:56.963 INFO:teuthology.orchestra.run.smithi032.stdout:Nothing to do. 2024-05-12T03:48:56.963 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2024-05-12T03:48:57.036 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-939.gc190bdee.el8.x86_64 1/1 2024-05-12T03:48:57.066 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum -y install libcephfs-devel 2024-05-12T03:48:57.404 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libcephfs-devel-2:17.2.7-939.gc190bdee.el8.x86_64 1/1 2024-05-12T03:48:57.404 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:48:57.404 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2024-05-12T03:48:57.404 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs-devel-2:17.2.7-939.gc190bdee.el8.x86_64 2024-05-12T03:48:57.404 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:48:57.404 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-05-12T03:48:57.519 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum -y install librados2 2024-05-12T03:48:57.857 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:02:25 ago on Sun 12 May 2024 03:46:32 AM UTC. 2024-05-12T03:48:58.301 INFO:teuthology.orchestra.run.smithi033.stdout:Last metadata expiration check: 0:02:31 ago on Sun 12 May 2024 03:46:27 AM UTC. 2024-05-12T03:48:59.630 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-05-12T03:48:59.631 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-05-12T03:48:59.631 INFO:teuthology.orchestra.run.smithi032.stdout: Package Arch Version Repo Size 2024-05-12T03:48:59.631 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-05-12T03:48:59.631 INFO:teuthology.orchestra.run.smithi032.stdout:Installing: 2024-05-12T03:48:59.631 INFO:teuthology.orchestra.run.smithi032.stdout: libcephfs-devel x86_64 2:17.2.7-939.gc190bdee.el8 ceph 25 k 2024-05-12T03:48:59.631 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:48:59.631 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction Summary 2024-05-12T03:48:59.631 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-05-12T03:48:59.631 INFO:teuthology.orchestra.run.smithi032.stdout:Install 1 Package 2024-05-12T03:48:59.631 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:48:59.631 INFO:teuthology.orchestra.run.smithi032.stdout:Total download size: 25 k 2024-05-12T03:48:59.632 INFO:teuthology.orchestra.run.smithi032.stdout:Installed size: 109 k 2024-05-12T03:48:59.632 INFO:teuthology.orchestra.run.smithi032.stdout:Downloading Packages: 2024-05-12T03:48:59.771 INFO:teuthology.orchestra.run.smithi032.stdout:libcephfs-devel-17.2.7-939.gc190bdee.el8.x86_64 178 kB/s | 25 kB 00:00 2024-05-12T03:48:59.772 INFO:teuthology.orchestra.run.smithi032.stdout:-------------------------------------------------------------------------------- 2024-05-12T03:48:59.772 INFO:teuthology.orchestra.run.smithi032.stdout:Total 176 kB/s | 25 kB 00:00 2024-05-12T03:48:59.772 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction check 2024-05-12T03:48:59.776 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction check succeeded. 2024-05-12T03:48:59.776 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction test 2024-05-12T03:48:59.778 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction test succeeded. 2024-05-12T03:48:59.781 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction 2024-05-12T03:48:59.830 INFO:teuthology.orchestra.run.smithi032.stdout: Preparing : 1/1 2024-05-12T03:49:00.018 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libcephfs-devel-2:17.2.7-939.gc190bdee.el8.x86_64 1/1 2024-05-12T03:49:00.059 INFO:teuthology.orchestra.run.smithi033.stdout:Package librados2-2:17.2.7-939.gc190bdee.el8.x86_64 is already installed. 2024-05-12T03:49:00.111 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-05-12T03:49:00.112 INFO:teuthology.orchestra.run.smithi033.stdout:Nothing to do. 2024-05-12T03:49:00.112 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-05-12T03:49:00.216 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum -y install librbd1 2024-05-12T03:49:00.332 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-939.gc190bdee.el8.x86_64 1/1 2024-05-12T03:49:00.703 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libcephfs-devel-2:17.2.7-939.gc190bdee.el8.x86_64 1/1 2024-05-12T03:49:00.703 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:49:00.703 INFO:teuthology.orchestra.run.smithi032.stdout:Installed: 2024-05-12T03:49:00.703 INFO:teuthology.orchestra.run.smithi032.stdout: libcephfs-devel-2:17.2.7-939.gc190bdee.el8.x86_64 2024-05-12T03:49:00.703 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:49:00.703 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2024-05-12T03:49:00.812 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum -y install librados2 2024-05-12T03:49:00.996 INFO:teuthology.orchestra.run.smithi033.stdout:Last metadata expiration check: 0:02:33 ago on Sun 12 May 2024 03:46:27 AM UTC. 2024-05-12T03:49:01.591 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:02:29 ago on Sun 12 May 2024 03:46:32 AM UTC. 2024-05-12T03:49:02.719 INFO:teuthology.orchestra.run.smithi033.stdout:Package librbd1-2:17.2.7-939.gc190bdee.el8.x86_64 is already installed. 2024-05-12T03:49:02.771 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-05-12T03:49:02.772 INFO:teuthology.orchestra.run.smithi033.stdout:Nothing to do. 2024-05-12T03:49:02.772 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-05-12T03:49:02.884 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum -y install python3-rados 2024-05-12T03:49:03.307 INFO:teuthology.orchestra.run.smithi032.stdout:Package librados2-2:17.2.7-939.gc190bdee.el8.x86_64 is already installed. 2024-05-12T03:49:03.362 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-05-12T03:49:03.363 INFO:teuthology.orchestra.run.smithi032.stdout:Nothing to do. 2024-05-12T03:49:03.363 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2024-05-12T03:49:03.665 INFO:teuthology.orchestra.run.smithi033.stdout:Last metadata expiration check: 0:02:36 ago on Sun 12 May 2024 03:46:27 AM UTC. 2024-05-12T03:49:03.676 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum -y install librbd1 2024-05-12T03:49:04.461 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:02:32 ago on Sun 12 May 2024 03:46:32 AM UTC. 2024-05-12T03:49:05.371 INFO:teuthology.orchestra.run.smithi033.stdout:Package python3-rados-2:17.2.7-939.gc190bdee.el8.x86_64 is already installed. 2024-05-12T03:49:05.423 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-05-12T03:49:05.424 INFO:teuthology.orchestra.run.smithi033.stdout:Nothing to do. 2024-05-12T03:49:05.425 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-05-12T03:49:05.560 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum -y install python3-rgw 2024-05-12T03:49:06.198 INFO:teuthology.orchestra.run.smithi032.stdout:Package librbd1-2:17.2.7-939.gc190bdee.el8.x86_64 is already installed. 2024-05-12T03:49:06.249 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-05-12T03:49:06.251 INFO:teuthology.orchestra.run.smithi032.stdout:Nothing to do. 2024-05-12T03:49:06.251 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2024-05-12T03:49:06.345 INFO:teuthology.orchestra.run.smithi033.stdout:Last metadata expiration check: 0:02:39 ago on Sun 12 May 2024 03:46:27 AM UTC. 2024-05-12T03:49:06.360 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum -y install python3-rados 2024-05-12T03:49:07.139 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:02:35 ago on Sun 12 May 2024 03:46:32 AM UTC. 2024-05-12T03:49:08.075 INFO:teuthology.orchestra.run.smithi033.stdout:Package python3-rgw-2:17.2.7-939.gc190bdee.el8.x86_64 is already installed. 2024-05-12T03:49:08.128 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-05-12T03:49:08.129 INFO:teuthology.orchestra.run.smithi033.stdout:Nothing to do. 2024-05-12T03:49:08.129 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-05-12T03:49:08.226 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum -y install python3-cephfs 2024-05-12T03:49:08.867 INFO:teuthology.orchestra.run.smithi032.stdout:Package python3-rados-2:17.2.7-939.gc190bdee.el8.x86_64 is already installed. 2024-05-12T03:49:08.921 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-05-12T03:49:08.922 INFO:teuthology.orchestra.run.smithi032.stdout:Nothing to do. 2024-05-12T03:49:08.922 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2024-05-12T03:49:09.011 INFO:teuthology.orchestra.run.smithi033.stdout:Last metadata expiration check: 0:02:42 ago on Sun 12 May 2024 03:46:27 AM UTC. 2024-05-12T03:49:09.035 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum -y install python3-rgw 2024-05-12T03:49:09.814 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:02:37 ago on Sun 12 May 2024 03:46:32 AM UTC. 2024-05-12T03:49:10.722 INFO:teuthology.orchestra.run.smithi033.stdout:Package python3-cephfs-2:17.2.7-939.gc190bdee.el8.x86_64 is already installed. 2024-05-12T03:49:10.773 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-05-12T03:49:10.774 INFO:teuthology.orchestra.run.smithi033.stdout:Nothing to do. 2024-05-12T03:49:10.775 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-05-12T03:49:10.883 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum -y install python3-rbd 2024-05-12T03:49:11.551 INFO:teuthology.orchestra.run.smithi032.stdout:Package python3-rgw-2:17.2.7-939.gc190bdee.el8.x86_64 is already installed. 2024-05-12T03:49:11.606 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-05-12T03:49:11.607 INFO:teuthology.orchestra.run.smithi032.stdout:Nothing to do. 2024-05-12T03:49:11.607 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2024-05-12T03:49:11.665 INFO:teuthology.orchestra.run.smithi033.stdout:Last metadata expiration check: 0:02:44 ago on Sun 12 May 2024 03:46:27 AM UTC. 2024-05-12T03:49:11.717 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum -y install python3-cephfs 2024-05-12T03:49:12.501 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:02:40 ago on Sun 12 May 2024 03:46:32 AM UTC. 2024-05-12T03:49:13.383 INFO:teuthology.orchestra.run.smithi033.stdout:Package python3-rbd-2:17.2.7-939.gc190bdee.el8.x86_64 is already installed. 2024-05-12T03:49:13.436 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-05-12T03:49:13.438 INFO:teuthology.orchestra.run.smithi033.stdout:Nothing to do. 2024-05-12T03:49:13.438 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-05-12T03:49:13.535 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum -y install rbd-fuse 2024-05-12T03:49:14.217 INFO:teuthology.orchestra.run.smithi032.stdout:Package python3-cephfs-2:17.2.7-939.gc190bdee.el8.x86_64 is already installed. 2024-05-12T03:49:14.270 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-05-12T03:49:14.270 INFO:teuthology.orchestra.run.smithi032.stdout:Nothing to do. 2024-05-12T03:49:14.271 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2024-05-12T03:49:14.317 INFO:teuthology.orchestra.run.smithi033.stdout:Last metadata expiration check: 0:02:47 ago on Sun 12 May 2024 03:46:27 AM UTC. 2024-05-12T03:49:14.375 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum -y install python3-rbd 2024-05-12T03:49:15.155 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:02:43 ago on Sun 12 May 2024 03:46:32 AM UTC. 2024-05-12T03:49:16.143 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-05-12T03:49:16.143 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-05-12T03:49:16.143 INFO:teuthology.orchestra.run.smithi033.stdout: Package Architecture Version Repository Size 2024-05-12T03:49:16.143 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-05-12T03:49:16.143 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2024-05-12T03:49:16.143 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-fuse x86_64 2:17.2.7-939.gc190bdee.el8 ceph 91 k 2024-05-12T03:49:16.144 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:49:16.144 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2024-05-12T03:49:16.144 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-05-12T03:49:16.144 INFO:teuthology.orchestra.run.smithi033.stdout:Install 1 Package 2024-05-12T03:49:16.144 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:49:16.144 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 91 k 2024-05-12T03:49:16.144 INFO:teuthology.orchestra.run.smithi033.stdout:Installed size: 268 k 2024-05-12T03:49:16.145 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading Packages: 2024-05-12T03:49:16.575 INFO:teuthology.orchestra.run.smithi033.stdout:rbd-fuse-17.2.7-939.gc190bdee.el8.x86_64.rpm 210 kB/s | 91 kB 00:00 2024-05-12T03:49:16.576 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2024-05-12T03:49:16.576 INFO:teuthology.orchestra.run.smithi033.stdout:Total 210 kB/s | 91 kB 00:00 2024-05-12T03:49:16.577 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2024-05-12T03:49:16.587 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction check succeeded. 2024-05-12T03:49:16.588 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2024-05-12T03:49:16.642 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded. 2024-05-12T03:49:16.645 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2024-05-12T03:49:16.754 INFO:teuthology.orchestra.run.smithi033.stdout: Preparing : 1/1 2024-05-12T03:49:16.879 INFO:teuthology.orchestra.run.smithi032.stdout:Package python3-rbd-2:17.2.7-939.gc190bdee.el8.x86_64 is already installed. 2024-05-12T03:49:16.932 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-05-12T03:49:16.934 INFO:teuthology.orchestra.run.smithi032.stdout:Nothing to do. 2024-05-12T03:49:16.934 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2024-05-12T03:49:16.985 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : rbd-fuse-2:17.2.7-939.gc190bdee.el8.x86_64 1/1 2024-05-12T03:49:17.042 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum -y install rbd-fuse 2024-05-12T03:49:17.308 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: rbd-fuse-2:17.2.7-939.gc190bdee.el8.x86_64 1/1 2024-05-12T03:49:17.698 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : rbd-fuse-2:17.2.7-939.gc190bdee.el8.x86_64 1/1 2024-05-12T03:49:17.698 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:49:17.698 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2024-05-12T03:49:17.699 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-fuse-2:17.2.7-939.gc190bdee.el8.x86_64 2024-05-12T03:49:17.699 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:49:17.699 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-05-12T03:49:17.834 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:02:45 ago on Sun 12 May 2024 03:46:32 AM UTC. 2024-05-12T03:49:17.903 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum -y install rbd-mirror 2024-05-12T03:49:18.694 INFO:teuthology.orchestra.run.smithi033.stdout:Last metadata expiration check: 0:02:51 ago on Sun 12 May 2024 03:46:27 AM UTC. 2024-05-12T03:49:19.598 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-05-12T03:49:19.599 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-05-12T03:49:19.599 INFO:teuthology.orchestra.run.smithi032.stdout: Package Architecture Version Repository Size 2024-05-12T03:49:19.599 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-05-12T03:49:19.599 INFO:teuthology.orchestra.run.smithi032.stdout:Installing: 2024-05-12T03:49:19.599 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-fuse x86_64 2:17.2.7-939.gc190bdee.el8 ceph 91 k 2024-05-12T03:49:19.599 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:49:19.599 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction Summary 2024-05-12T03:49:19.599 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-05-12T03:49:19.599 INFO:teuthology.orchestra.run.smithi032.stdout:Install 1 Package 2024-05-12T03:49:19.600 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:49:19.600 INFO:teuthology.orchestra.run.smithi032.stdout:Total download size: 91 k 2024-05-12T03:49:19.600 INFO:teuthology.orchestra.run.smithi032.stdout:Installed size: 268 k 2024-05-12T03:49:19.600 INFO:teuthology.orchestra.run.smithi032.stdout:Downloading Packages: 2024-05-12T03:49:19.894 INFO:teuthology.orchestra.run.smithi032.stdout:rbd-fuse-17.2.7-939.gc190bdee.el8.x86_64.rpm 308 kB/s | 91 kB 00:00 2024-05-12T03:49:19.895 INFO:teuthology.orchestra.run.smithi032.stdout:-------------------------------------------------------------------------------- 2024-05-12T03:49:19.895 INFO:teuthology.orchestra.run.smithi032.stdout:Total 306 kB/s | 91 kB 00:00 2024-05-12T03:49:19.895 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction check 2024-05-12T03:49:19.906 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction check succeeded. 2024-05-12T03:49:19.906 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction test 2024-05-12T03:49:19.959 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction test succeeded. 2024-05-12T03:49:19.962 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction 2024-05-12T03:49:20.066 INFO:teuthology.orchestra.run.smithi032.stdout: Preparing : 1/1 2024-05-12T03:49:20.232 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : rbd-fuse-2:17.2.7-939.gc190bdee.el8.x86_64 1/1 2024-05-12T03:49:20.467 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-05-12T03:49:20.467 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-05-12T03:49:20.467 INFO:teuthology.orchestra.run.smithi033.stdout: Package Arch Version Repo Size 2024-05-12T03:49:20.467 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-05-12T03:49:20.467 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2024-05-12T03:49:20.468 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-mirror x86_64 2:17.2.7-939.gc190bdee.el8 ceph 3.5 M 2024-05-12T03:49:20.468 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:49:20.468 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2024-05-12T03:49:20.468 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-05-12T03:49:20.468 INFO:teuthology.orchestra.run.smithi033.stdout:Install 1 Package 2024-05-12T03:49:20.468 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:49:20.469 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 3.5 M 2024-05-12T03:49:20.469 INFO:teuthology.orchestra.run.smithi033.stdout:Installed size: 16 M 2024-05-12T03:49:20.469 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading Packages: 2024-05-12T03:49:20.512 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: rbd-fuse-2:17.2.7-939.gc190bdee.el8.x86_64 1/1 2024-05-12T03:49:20.856 INFO:teuthology.orchestra.run.smithi033.stdout:rbd-mirror-17.2.7-939.gc190bdee.el8.x86_64.rpm 8.9 MB/s | 3.5 MB 00:00 2024-05-12T03:49:20.856 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2024-05-12T03:49:20.857 INFO:teuthology.orchestra.run.smithi033.stdout:Total 8.9 MB/s | 3.5 MB 00:00 2024-05-12T03:49:20.857 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2024-05-12T03:49:20.869 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction check succeeded. 2024-05-12T03:49:20.869 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2024-05-12T03:49:20.930 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : rbd-fuse-2:17.2.7-939.gc190bdee.el8.x86_64 1/1 2024-05-12T03:49:20.930 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:49:20.930 INFO:teuthology.orchestra.run.smithi032.stdout:Installed: 2024-05-12T03:49:20.930 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-fuse-2:17.2.7-939.gc190bdee.el8.x86_64 2024-05-12T03:49:20.930 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:49:20.930 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2024-05-12T03:49:20.935 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded. 2024-05-12T03:49:20.938 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2024-05-12T03:49:21.158 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum -y install rbd-mirror 2024-05-12T03:49:21.429 INFO:teuthology.orchestra.run.smithi033.stdout: Preparing : 1/1 2024-05-12T03:49:21.653 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : rbd-mirror-2:17.2.7-939.gc190bdee.el8.x86_64 1/1 2024-05-12T03:49:21.947 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:02:49 ago on Sun 12 May 2024 03:46:32 AM UTC. 2024-05-12T03:49:22.088 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: rbd-mirror-2:17.2.7-939.gc190bdee.el8.x86_64 1/1 2024-05-12T03:49:22.506 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : rbd-mirror-2:17.2.7-939.gc190bdee.el8.x86_64 1/1 2024-05-12T03:49:22.506 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:49:22.506 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2024-05-12T03:49:22.506 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-mirror-2:17.2.7-939.gc190bdee.el8.x86_64 2024-05-12T03:49:22.506 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:49:22.507 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-05-12T03:49:22.747 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum -y install rbd-nbd 2024-05-12T03:49:23.543 INFO:teuthology.orchestra.run.smithi033.stdout:Last metadata expiration check: 0:02:56 ago on Sun 12 May 2024 03:46:27 AM UTC. 2024-05-12T03:49:23.715 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-05-12T03:49:23.716 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-05-12T03:49:23.716 INFO:teuthology.orchestra.run.smithi032.stdout: Package Arch Version Repo Size 2024-05-12T03:49:23.716 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-05-12T03:49:23.716 INFO:teuthology.orchestra.run.smithi032.stdout:Installing: 2024-05-12T03:49:23.716 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-mirror x86_64 2:17.2.7-939.gc190bdee.el8 ceph 3.5 M 2024-05-12T03:49:23.716 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:49:23.716 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction Summary 2024-05-12T03:49:23.716 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-05-12T03:49:23.716 INFO:teuthology.orchestra.run.smithi032.stdout:Install 1 Package 2024-05-12T03:49:23.716 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:49:23.717 INFO:teuthology.orchestra.run.smithi032.stdout:Total download size: 3.5 M 2024-05-12T03:49:23.717 INFO:teuthology.orchestra.run.smithi032.stdout:Installed size: 16 M 2024-05-12T03:49:23.717 INFO:teuthology.orchestra.run.smithi032.stdout:Downloading Packages: 2024-05-12T03:49:24.106 INFO:teuthology.orchestra.run.smithi032.stdout:rbd-mirror-17.2.7-939.gc190bdee.el8.x86_64.rpm 8.9 MB/s | 3.5 MB 00:00 2024-05-12T03:49:24.106 INFO:teuthology.orchestra.run.smithi032.stdout:-------------------------------------------------------------------------------- 2024-05-12T03:49:24.107 INFO:teuthology.orchestra.run.smithi032.stdout:Total 8.9 MB/s | 3.5 MB 00:00 2024-05-12T03:49:24.107 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction check 2024-05-12T03:49:24.120 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction check succeeded. 2024-05-12T03:49:24.120 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction test 2024-05-12T03:49:24.187 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction test succeeded. 2024-05-12T03:49:24.190 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction 2024-05-12T03:49:24.664 INFO:teuthology.orchestra.run.smithi032.stdout: Preparing : 1/1 2024-05-12T03:49:24.858 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : rbd-mirror-2:17.2.7-939.gc190bdee.el8.x86_64 1/1 2024-05-12T03:49:25.282 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: rbd-mirror-2:17.2.7-939.gc190bdee.el8.x86_64 1/1 2024-05-12T03:49:25.312 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-05-12T03:49:25.312 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-05-12T03:49:25.312 INFO:teuthology.orchestra.run.smithi033.stdout: Package Architecture Version Repository Size 2024-05-12T03:49:25.312 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-05-12T03:49:25.313 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2024-05-12T03:49:25.313 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-nbd x86_64 2:17.2.7-939.gc190bdee.el8 ceph 175 k 2024-05-12T03:49:25.313 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:49:25.313 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2024-05-12T03:49:25.313 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-05-12T03:49:25.313 INFO:teuthology.orchestra.run.smithi033.stdout:Install 1 Package 2024-05-12T03:49:25.313 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:49:25.313 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 175 k 2024-05-12T03:49:25.314 INFO:teuthology.orchestra.run.smithi033.stdout:Installed size: 556 k 2024-05-12T03:49:25.314 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading Packages: 2024-05-12T03:49:25.608 INFO:teuthology.orchestra.run.smithi033.stdout:rbd-nbd-17.2.7-939.gc190bdee.el8.x86_64.rpm 593 kB/s | 175 kB 00:00 2024-05-12T03:49:25.609 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2024-05-12T03:49:25.609 INFO:teuthology.orchestra.run.smithi033.stdout:Total 590 kB/s | 175 kB 00:00 2024-05-12T03:49:25.609 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2024-05-12T03:49:25.621 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction check succeeded. 2024-05-12T03:49:25.621 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2024-05-12T03:49:25.675 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded. 2024-05-12T03:49:25.678 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2024-05-12T03:49:25.693 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : rbd-mirror-2:17.2.7-939.gc190bdee.el8.x86_64 1/1 2024-05-12T03:49:25.693 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:49:25.693 INFO:teuthology.orchestra.run.smithi032.stdout:Installed: 2024-05-12T03:49:25.693 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-mirror-2:17.2.7-939.gc190bdee.el8.x86_64 2024-05-12T03:49:25.693 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:49:25.694 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2024-05-12T03:49:25.894 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum -y install rbd-nbd 2024-05-12T03:49:25.980 INFO:teuthology.orchestra.run.smithi033.stdout: Preparing : 1/1 2024-05-12T03:49:26.200 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : rbd-nbd-2:17.2.7-939.gc190bdee.el8.x86_64 1/1 2024-05-12T03:49:26.499 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: rbd-nbd-2:17.2.7-939.gc190bdee.el8.x86_64 1/1 2024-05-12T03:49:26.685 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:02:54 ago on Sun 12 May 2024 03:46:32 AM UTC. 2024-05-12T03:49:26.913 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : rbd-nbd-2:17.2.7-939.gc190bdee.el8.x86_64 1/1 2024-05-12T03:49:26.913 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:49:26.913 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2024-05-12T03:49:26.913 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-nbd-2:17.2.7-939.gc190bdee.el8.x86_64 2024-05-12T03:49:26.913 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:49:26.913 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-05-12T03:49:27.182 DEBUG:teuthology.parallel:result is None 2024-05-12T03:49:28.465 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-05-12T03:49:28.466 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-05-12T03:49:28.466 INFO:teuthology.orchestra.run.smithi032.stdout: Package Architecture Version Repository Size 2024-05-12T03:49:28.466 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-05-12T03:49:28.466 INFO:teuthology.orchestra.run.smithi032.stdout:Installing: 2024-05-12T03:49:28.466 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-nbd x86_64 2:17.2.7-939.gc190bdee.el8 ceph 175 k 2024-05-12T03:49:28.466 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:49:28.466 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction Summary 2024-05-12T03:49:28.466 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-05-12T03:49:28.466 INFO:teuthology.orchestra.run.smithi032.stdout:Install 1 Package 2024-05-12T03:49:28.467 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:49:28.468 INFO:teuthology.orchestra.run.smithi032.stdout:Total download size: 175 k 2024-05-12T03:49:28.468 INFO:teuthology.orchestra.run.smithi032.stdout:Installed size: 556 k 2024-05-12T03:49:28.468 INFO:teuthology.orchestra.run.smithi032.stdout:Downloading Packages: 2024-05-12T03:49:28.687 INFO:teuthology.orchestra.run.smithi032.stdout:rbd-nbd-17.2.7-939.gc190bdee.el8.x86_64.rpm 794 kB/s | 175 kB 00:00 2024-05-12T03:49:28.688 INFO:teuthology.orchestra.run.smithi032.stdout:-------------------------------------------------------------------------------- 2024-05-12T03:49:28.688 INFO:teuthology.orchestra.run.smithi032.stdout:Total 790 kB/s | 175 kB 00:00 2024-05-12T03:49:28.688 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction check 2024-05-12T03:49:28.700 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction check succeeded. 2024-05-12T03:49:28.700 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction test 2024-05-12T03:49:28.752 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction test succeeded. 2024-05-12T03:49:28.756 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction 2024-05-12T03:49:28.876 INFO:teuthology.orchestra.run.smithi032.stdout: Preparing : 1/1 2024-05-12T03:49:29.056 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : rbd-nbd-2:17.2.7-939.gc190bdee.el8.x86_64 1/1 2024-05-12T03:49:29.354 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: rbd-nbd-2:17.2.7-939.gc190bdee.el8.x86_64 1/1 2024-05-12T03:49:29.774 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : rbd-nbd-2:17.2.7-939.gc190bdee.el8.x86_64 1/1 2024-05-12T03:49:29.774 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:49:29.774 INFO:teuthology.orchestra.run.smithi032.stdout:Installed: 2024-05-12T03:49:29.774 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-nbd-2:17.2.7-939.gc190bdee.el8.x86_64 2024-05-12T03:49:29.774 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:49:29.774 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2024-05-12T03:49:29.989 DEBUG:teuthology.parallel:result is None 2024-05-12T03:49:29.989 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-12T03:49:29.990 INFO:teuthology.packaging:ref: None 2024-05-12T03:49:29.990 INFO:teuthology.packaging:tag: None 2024-05-12T03:49:29.990 INFO:teuthology.packaging:branch: quincy 2024-05-12T03:49:29.990 INFO:teuthology.packaging:sha1: 58249552a22f437b8cb49ed6b1068e1158bed605 2024-05-12T03:49:29.990 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=quincy 2024-05-12T03:49:30.364 DEBUG:teuthology.orchestra.run.smithi032:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-12T03:49:30.388 INFO:teuthology.orchestra.run.smithi032.stdout:17.2.7-939.gc190bdee.el8 2024-05-12T03:49:30.389 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-939.gc190bdee.el8 2024-05-12T03:49:30.389 INFO:teuthology.task.install:The correct ceph version 17.2.7-939.gc190bdee is installed. 2024-05-12T03:49:30.391 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-12T03:49:30.391 INFO:teuthology.packaging:ref: None 2024-05-12T03:49:30.391 INFO:teuthology.packaging:tag: None 2024-05-12T03:49:30.391 INFO:teuthology.packaging:branch: quincy 2024-05-12T03:49:30.391 INFO:teuthology.packaging:sha1: 58249552a22f437b8cb49ed6b1068e1158bed605 2024-05-12T03:49:30.391 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=quincy 2024-05-12T03:49:30.705 DEBUG:teuthology.orchestra.run.smithi033:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-12T03:49:30.733 INFO:teuthology.orchestra.run.smithi033.stdout:17.2.7-939.gc190bdee.el8 2024-05-12T03:49:30.733 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-939.gc190bdee.el8 2024-05-12T03:49:30.733 INFO:teuthology.task.install:The correct ceph version 17.2.7-939.gc190bdee is installed. 2024-05-12T03:49:30.735 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-12T03:49:30.735 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-05-12T03:49:30.735 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-12T03:49:30.767 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-12T03:49:30.767 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-12T03:49:30.806 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-12T03:49:30.872 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-05-12T03:49:30.872 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/usr/bin/daemon-helper 2024-05-12T03:49:30.904 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-12T03:49:30.975 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-12T03:49:30.975 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/usr/bin/daemon-helper 2024-05-12T03:49:31.008 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-12T03:49:31.080 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-12T03:49:31.081 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-05-12T03:49:31.081 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-12T03:49:31.112 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-12T03:49:31.183 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-12T03:49:31.183 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-12T03:49:31.215 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-12T03:49:31.288 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-12T03:49:31.288 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-05-12T03:49:31.288 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/usr/bin/stdin-killer 2024-05-12T03:49:31.319 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-12T03:49:31.389 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-12T03:49:31.389 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/usr/bin/stdin-killer 2024-05-12T03:49:31.422 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-12T03:49:31.493 INFO:teuthology.run_tasks:Running task print... 2024-05-12T03:49:31.503 INFO:teuthology.task.print:**** done install task... 2024-05-12T03:49:31.503 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-12T03:49:31.668 INFO:tasks.cephadm:Config: {'cephadm_branch': 'quincy', 'cephadm_git_url': 'https://github.com/ceph/ceph', 'conf': {'osd': {'osd_class_default_list': '*', 'osd_class_load_list': '*', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}}, 'image': 'quay.ceph.io/ceph-ci/ceph:quincy', 'roleless': True, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'OSD_DOWN', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': '58249552a22f437b8cb49ed6b1068e1158bed605'} 2024-05-12T03:49:31.668 INFO:tasks.cephadm:Cluster image is quay.ceph.io/ceph-ci/ceph:quincy 2024-05-12T03:49:31.669 INFO:tasks.cephadm:Cluster fsid is a3e178f8-1012-11ef-bc98-c7b262605968 2024-05-12T03:49:31.669 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-12T03:49:31.669 INFO:tasks.cephadm:No mon roles; fabricating mons 2024-05-12T03:49:31.669 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi032': '172.21.15.32', 'mon.smithi033': '172.21.15.33'} 2024-05-12T03:49:31.669 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-12T03:49:31.670 DEBUG:teuthology.orchestra.run.smithi032:> sudo hostname $(hostname -s) 2024-05-12T03:49:31.703 DEBUG:teuthology.orchestra.run.smithi033:> sudo hostname $(hostname -s) 2024-05-12T03:49:31.737 INFO:tasks.cephadm:Downloading cephadm (repo https://github.com/ceph/ceph ref quincy)... 2024-05-12T03:49:31.737 DEBUG:teuthology.orchestra.run.smithi032:> curl --silent https://raw.githubusercontent.com/ceph/ceph/quincy/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-12T03:49:31.973 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-rw-r--. 1 ubuntu ubuntu 375153 May 12 03:49 /home/ubuntu/cephtest/cephadm 2024-05-12T03:49:31.974 DEBUG:teuthology.orchestra.run.smithi033:> curl --silent https://raw.githubusercontent.com/ceph/ceph/quincy/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-12T03:49:32.175 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-rw-r--. 1 ubuntu ubuntu 375153 May 12 03:49 /home/ubuntu/cephtest/cephadm 2024-05-12T03:49:32.176 DEBUG:teuthology.orchestra.run.smithi032:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-12T03:49:32.199 DEBUG:teuthology.orchestra.run.smithi033:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-12T03:49:32.233 INFO:tasks.cephadm:Pulling image quay.ceph.io/ceph-ci/ceph:quincy on all hosts... 2024-05-12T03:49:32.233 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy pull 2024-05-12T03:49:32.241 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy pull 2024-05-12T03:49:33.503 INFO:teuthology.orchestra.run.smithi032.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-05-12T03:49:33.632 INFO:teuthology.orchestra.run.smithi033.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-05-12T03:50:01.626 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-05-12T03:50:01.626 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_version": "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)", 2024-05-12T03:50:01.626 INFO:teuthology.orchestra.run.smithi032.stdout: "image_id": "284803f7d9cf4b212bf198be96aa63a7825e7f8afc4fa40d8c7cda867d7632da", 2024-05-12T03:50:01.626 INFO:teuthology.orchestra.run.smithi032.stdout: "repo_digests": [ 2024-05-12T03:50:01.626 INFO:teuthology.orchestra.run.smithi032.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:ee006be5fca483ef0dbd8a31d4eb01b45353777583562644120e2a9d4e2e442e" 2024-05-12T03:50:01.626 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-05-12T03:50:01.626 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-05-12T03:50:05.539 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-12T03:50:05.539 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_version": "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)", 2024-05-12T03:50:05.539 INFO:teuthology.orchestra.run.smithi033.stdout: "image_id": "284803f7d9cf4b212bf198be96aa63a7825e7f8afc4fa40d8c7cda867d7632da", 2024-05-12T03:50:05.539 INFO:teuthology.orchestra.run.smithi033.stdout: "repo_digests": [ 2024-05-12T03:50:05.539 INFO:teuthology.orchestra.run.smithi033.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:ee006be5fca483ef0dbd8a31d4eb01b45353777583562644120e2a9d4e2e442e" 2024-05-12T03:50:05.540 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-12T03:50:05.540 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-12T03:50:05.561 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /etc/ceph 2024-05-12T03:50:05.599 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /etc/ceph 2024-05-12T03:50:05.632 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 777 /etc/ceph 2024-05-12T03:50:05.670 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 777 /etc/ceph 2024-05-12T03:50:05.701 INFO:tasks.cephadm:Writing seed config... 2024-05-12T03:50:05.702 INFO:tasks.cephadm: override: [osd] osd_class_default_list = * 2024-05-12T03:50:05.702 INFO:tasks.cephadm: override: [osd] osd_class_load_list = * 2024-05-12T03:50:05.703 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-05-12T03:50:05.703 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-05-12T03:50:05.703 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-05-12T03:50:05.703 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-12T03:50:05.703 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-12T03:50:05.703 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-05-12T03:50:05.703 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-05-12T03:50:05.703 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-12T03:50:05.703 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-12T03:50:05.703 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-05-12T03:50:05.703 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-12T03:50:05.703 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-12T03:50:05.703 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-12T03:50:05.703 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-12T03:50:05.703 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-12T03:50:05.703 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-12T03:50:05.704 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-12T03:50:05.704 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-12T03:50:05.704 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-12T03:50:05.704 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-12T03:50:05.704 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-12T03:50:05.704 INFO:tasks.cephadm: override: [global] mon pg warn min per osd = 0 2024-05-12T03:50:05.704 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-12T03:50:05.704 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-12T03:50:05.704 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-12T03:50:05.704 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-12T03:50:05.704 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-12T03:50:05.704 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-12T03:50:05.704 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-12T03:50:05.704 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-12T03:50:05.704 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-12T03:50:05.704 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-12T03:50:05.705 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-12T03:50:05.705 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-12T03:50:05.705 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-12T03:50:05.705 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-12T03:50:05.705 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-12T03:50:05.705 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-12T03:50:05.705 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-05-12T03:50:05.706 DEBUG:teuthology.orchestra.run.smithi032:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-12T03:50:05.731 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = a3e178f8-1012-11ef-bc98-c7b262605968 mon pg warn min per osd = 0 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true osd_class_default_list = * osd_class_load_list = * bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-12T03:50:05.731 DEBUG:teuthology.orchestra.run.smithi032:mon.smithi032> sudo journalctl -f -n 0 -u ceph-a3e178f8-1012-11ef-bc98-c7b262605968@mon.smithi032.service 2024-05-12T03:50:05.773 INFO:tasks.cephadm:Bootstrapping... 2024-05-12T03:50:05.773 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy -v bootstrap --fsid a3e178f8-1012-11ef-bc98-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.32 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-12T03:50:05.804 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:-- Logs begin at Sun 2024-05-12 03:39:04 UTC. -- 2024-05-12T03:50:05.992 INFO:teuthology.orchestra.run.smithi032.stdout:-------------------------------------------------------------------------------- 2024-05-12T03:50:05.992 INFO:teuthology.orchestra.run.smithi032.stdout:cephadm ['--image', 'quay.ceph.io/ceph-ci/ceph:quincy', '-v', 'bootstrap', '--fsid', 'a3e178f8-1012-11ef-bc98-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-ip', '172.21.15.32', '--skip-admin-label'] 2024-05-12T03:50:06.022 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stdout 4.9.4-dev 2024-05-12T03:50:06.023 INFO:teuthology.orchestra.run.smithi032.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-12T03:50:06.023 INFO:teuthology.orchestra.run.smithi032.stdout:Verifying podman|docker is present... 2024-05-12T03:50:06.055 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stdout 4.9.4-dev 2024-05-12T03:50:06.055 INFO:teuthology.orchestra.run.smithi032.stdout:Verifying lvm2 is present... 2024-05-12T03:50:06.055 INFO:teuthology.orchestra.run.smithi032.stdout:Verifying time synchronization is in place... 2024-05-12T03:50:06.062 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-12T03:50:06.062 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-12T03:50:06.068 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-12T03:50:06.068 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout inactive 2024-05-12T03:50:06.074 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout enabled 2024-05-12T03:50:06.080 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout active 2024-05-12T03:50:06.080 INFO:teuthology.orchestra.run.smithi032.stdout:Unit chronyd.service is enabled and running 2024-05-12T03:50:06.080 INFO:teuthology.orchestra.run.smithi032.stdout:Repeating the final host check... 2024-05-12T03:50:06.112 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stdout 4.9.4-dev 2024-05-12T03:50:06.112 INFO:teuthology.orchestra.run.smithi032.stdout:podman (/bin/podman) version 4.9.4 is present 2024-05-12T03:50:06.112 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl is present 2024-05-12T03:50:06.112 INFO:teuthology.orchestra.run.smithi032.stdout:lvcreate is present 2024-05-12T03:50:06.119 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-12T03:50:06.119 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-12T03:50:06.124 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-12T03:50:06.124 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout inactive 2024-05-12T03:50:06.130 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout enabled 2024-05-12T03:50:06.137 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout active 2024-05-12T03:50:06.137 INFO:teuthology.orchestra.run.smithi032.stdout:Unit chronyd.service is enabled and running 2024-05-12T03:50:06.137 INFO:teuthology.orchestra.run.smithi032.stdout:Host looks OK 2024-05-12T03:50:06.137 INFO:teuthology.orchestra.run.smithi032.stdout:Cluster fsid: a3e178f8-1012-11ef-bc98-c7b262605968 2024-05-12T03:50:06.137 INFO:teuthology.orchestra.run.smithi032.stdout:Acquiring lock 140017074373408 on /run/cephadm/a3e178f8-1012-11ef-bc98-c7b262605968.lock 2024-05-12T03:50:06.137 INFO:teuthology.orchestra.run.smithi032.stdout:Lock 140017074373408 acquired on /run/cephadm/a3e178f8-1012-11ef-bc98-c7b262605968.lock 2024-05-12T03:50:06.138 INFO:teuthology.orchestra.run.smithi032.stdout:Verifying IP 172.21.15.32 port 3300 ... 2024-05-12T03:50:06.138 INFO:teuthology.orchestra.run.smithi032.stdout:Verifying IP 172.21.15.32 port 6789 ... 2024-05-12T03:50:06.138 INFO:teuthology.orchestra.run.smithi032.stdout:Base mon IP(s) is [172.21.15.32:3300, 172.21.15.32:6789], mon addrv is [v2:172.21.15.32:3300,v1:172.21.15.32:6789] 2024-05-12T03:50:06.142 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.32 metric 100 2024-05-12T03:50:06.142 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.32 metric 100 2024-05-12T03:50:06.145 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-12T03:50:06.145 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-05-12T03:50:06.145 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1772sec hoplimit 64 pref medium 2024-05-12T03:50:06.148 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-12T03:50:06.148 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-12T03:50:06.148 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-12T03:50:06.148 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout 3: ens1f0: mtu 1500 state UP qlen 1000 2024-05-12T03:50:06.148 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout inet6 fe80::ae1f:6bff:fef8:2602/64 scope link 2024-05-12T03:50:06.148 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-12T03:50:06.148 INFO:teuthology.orchestra.run.smithi032.stdout:Mon IP `172.21.15.32` is in CIDR network `172.21.0.0/20` 2024-05-12T03:50:06.149 INFO:teuthology.orchestra.run.smithi032.stdout:Mon IP `172.21.15.32` is in CIDR network `172.21.0.0/20` 2024-05-12T03:50:06.149 INFO:teuthology.orchestra.run.smithi032.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-12T03:50:06.149 INFO:teuthology.orchestra.run.smithi032.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-12T03:50:06.150 INFO:teuthology.orchestra.run.smithi032.stdout:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-05-12T03:50:06.749 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stdout 284803f7d9cf4b212bf198be96aa63a7825e7f8afc4fa40d8c7cda867d7632da 2024-05-12T03:50:06.749 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stderr Trying to pull quay.ceph.io/ceph-ci/ceph:quincy... 2024-05-12T03:50:06.749 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stderr Getting image source signatures 2024-05-12T03:50:06.749 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stderr Copying blob sha256:b5c516c4126c06f657ba42b13a9f342d68ac6f58835e40c7488f5b5a54c75084 2024-05-12T03:50:06.750 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stderr Copying blob sha256:bf5b95bd68251da39c6cd2eb96c6ba4138bb37a62bdc39287908b25e92047d7e 2024-05-12T03:50:06.750 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stderr Copying config sha256:284803f7d9cf4b212bf198be96aa63a7825e7f8afc4fa40d8c7cda867d7632da 2024-05-12T03:50:06.750 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-12T03:50:08.065 INFO:teuthology.orchestra.run.smithi032.stdout:ceph: stdout ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable) 2024-05-12T03:50:08.065 INFO:teuthology.orchestra.run.smithi032.stdout:Ceph version: ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable) 2024-05-12T03:50:08.065 INFO:teuthology.orchestra.run.smithi032.stdout:Extracting ceph user uid/gid from container image... 2024-05-12T03:50:09.161 INFO:teuthology.orchestra.run.smithi032.stdout:stat: stdout 167 167 2024-05-12T03:50:09.161 INFO:teuthology.orchestra.run.smithi032.stdout:Creating initial keys... 2024-05-12T03:50:12.732 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph-authtool: stdout AQByPEBm+O+bIxAAVcQeZm5nEajyNkjoFIOZQA== 2024-05-12T03:50:13.905 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph-authtool: stdout AQB1PEBm4JRvChAAB+AnF7DuLPGD4sl5N9gYAA== 2024-05-12T03:50:15.047 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph-authtool: stdout AQB2PEBmn2OWGRAAvjNRXxOXYOQn44Jo2iqsNg== 2024-05-12T03:50:15.048 INFO:teuthology.orchestra.run.smithi032.stdout:Creating initial monmap... 2024-05-12T03:50:16.150 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-12T03:50:16.151 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = octopus 2024-05-12T03:50:16.151 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to a3e178f8-1012-11ef-bc98-c7b262605968 2024-05-12T03:50:16.151 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-12T03:50:16.151 INFO:teuthology.orchestra.run.smithi032.stdout:monmaptool for smithi032 [v2:172.21.15.32:3300,v1:172.21.15.32:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-12T03:50:16.151 INFO:teuthology.orchestra.run.smithi032.stdout:setting min_mon_release = octopus 2024-05-12T03:50:16.151 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/monmaptool: set fsid to a3e178f8-1012-11ef-bc98-c7b262605968 2024-05-12T03:50:16.152 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-12T03:50:16.152 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:50:16.152 INFO:teuthology.orchestra.run.smithi032.stdout:Creating mon... 2024-05-12T03:50:17.295 INFO:teuthology.orchestra.run.smithi032.stdout:create mon.smithi032 on 2024-05-12T03:50:17.409 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2024-05-12T03:50:17.499 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-12T03:50:17.598 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-a3e178f8-1012-11ef-bc98-c7b262605968.target → /etc/systemd/system/ceph-a3e178f8-1012-11ef-bc98-c7b262605968.target. 2024-05-12T03:50:17.598 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-a3e178f8-1012-11ef-bc98-c7b262605968.target → /etc/systemd/system/ceph-a3e178f8-1012-11ef-bc98-c7b262605968.target. 2024-05-12T03:50:17.728 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-a3e178f8-1012-11ef-bc98-c7b262605968@mon.smithi032 2024-05-12T03:50:17.728 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Failed to reset failed state of unit ceph-a3e178f8-1012-11ef-bc98-c7b262605968@mon.smithi032.service: Unit ceph-a3e178f8-1012-11ef-bc98-c7b262605968@mon.smithi032.service not loaded. 2024-05-12T03:50:17.827 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-a3e178f8-1012-11ef-bc98-c7b262605968.target.wants/ceph-a3e178f8-1012-11ef-bc98-c7b262605968@mon.smithi032.service → /etc/systemd/system/ceph-a3e178f8-1012-11ef-bc98-c7b262605968@.service. 2024-05-12T03:50:17.868 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:17 smithi032 systemd[1]: Starting Ceph mon.smithi032 for a3e178f8-1012-11ef-bc98-c7b262605968... 2024-05-12T03:50:18.407 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-12T03:50:18.408 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout disabled 2024-05-12T03:50:18.416 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-12T03:50:18.416 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout inactive 2024-05-12T03:50:18.416 INFO:teuthology.orchestra.run.smithi032.stdout:firewalld.service is not enabled 2024-05-12T03:50:18.416 INFO:teuthology.orchestra.run.smithi032.stdout:Not possible to enable service . firewalld.service is not available 2024-05-12T03:50:18.416 INFO:teuthology.orchestra.run.smithi032.stdout:Waiting for mon to start... 2024-05-12T03:50:18.416 INFO:teuthology.orchestra.run.smithi032.stdout:Waiting for mon... 2024-05-12T03:50:18.640 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:18 smithi032 ceph-mon[39515]: mkfs a3e178f8-1012-11ef-bc98-c7b262605968 2024-05-12T03:50:18.640 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:18 smithi032 ceph-mon[39515]: mon.smithi032 is new leader, mons smithi032 in quorum (ranks 0) 2024-05-12T03:50:18.640 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:18 smithi032 bash[39476]: 64ef260df81478709866aeec98d7bedcd5b9a595f5e1108f4df4ee9cbca6b3e2 2024-05-12T03:50:18.640 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:18 smithi032 systemd[1]: Started Ceph mon.smithi032 for a3e178f8-1012-11ef-bc98-c7b262605968. 2024-05-12T03:50:19.583 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout cluster: 2024-05-12T03:50:19.583 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout id: a3e178f8-1012-11ef-bc98-c7b262605968 2024-05-12T03:50:19.583 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-12T03:50:19.583 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2024-05-12T03:50:19.583 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout services: 2024-05-12T03:50:19.583 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum smithi032 (age 0.649156s) 2024-05-12T03:50:19.584 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-12T03:50:19.584 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-12T03:50:19.584 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2024-05-12T03:50:19.584 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout data: 2024-05-12T03:50:19.584 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-12T03:50:19.584 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-12T03:50:19.584 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-12T03:50:19.584 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout pgs: 2024-05-12T03:50:19.584 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2024-05-12T03:50:19.584 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:18.924+0000 7f31a6698700 1 Processor -- start 2024-05-12T03:50:19.584 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:18.924+0000 7f31a6698700 1 -- start start 2024-05-12T03:50:19.584 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:18.924+0000 7f31a6698700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31a0078c60 0x7f31a0077260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:50:19.584 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:18.924+0000 7f31a6698700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a0079070 con 0x7f31a0078c60 2024-05-12T03:50:19.584 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:18.925+0000 7f319ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31a0078c60 0x7f31a0077260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:50:19.584 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:18.925+0000 7f319ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31a0078c60 0x7f31a0077260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40200/0 (socket says 172.21.15.32:40200) 2024-05-12T03:50:19.585 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:18.925+0000 7f319ffff700 1 -- 172.21.15.32:0/3386580632 learned_addr learned my addr 172.21.15.32:0/3386580632 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:50:19.585 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:18.925+0000 7f319ffff700 1 -- 172.21.15.32:0/3386580632 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a00777a0 con 0x7f31a0078c60 2024-05-12T03:50:19.585 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:18.925+0000 7f319ffff700 1 --2- 172.21.15.32:0/3386580632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31a0078c60 0x7f31a0077260 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f31900098e0 tx=0x7f3190009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=857d0c951398368a server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:50:19.585 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:18.926+0000 7f319effd700 1 -- 172.21.15.32:0/3386580632 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f319000b860 con 0x7f31a0078c60 2024-05-12T03:50:19.585 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:18.926+0000 7f319effd700 1 -- 172.21.15.32:0/3386580632 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f319000b9c0 con 0x7f31a0078c60 2024-05-12T03:50:19.585 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:18.926+0000 7f319effd700 1 -- 172.21.15.32:0/3386580632 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f319000bc90 con 0x7f31a0078c60 2024-05-12T03:50:19.585 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:18.926+0000 7f31a6698700 1 -- 172.21.15.32:0/3386580632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31a0078c60 msgr2=0x7f31a0077260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:50:19.585 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:18.926+0000 7f31a6698700 1 --2- 172.21.15.32:0/3386580632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31a0078c60 0x7f31a0077260 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f31900098e0 tx=0x7f3190009250 comp rx=0 tx=0).stop 2024-05-12T03:50:19.585 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:18.926+0000 7f31a6698700 1 -- 172.21.15.32:0/3386580632 shutdown_connections 2024-05-12T03:50:19.585 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:18.926+0000 7f31a6698700 1 --2- 172.21.15.32:0/3386580632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31a0078c60 0x7f31a0077260 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:50:19.585 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:18.926+0000 7f31a6698700 1 -- 172.21.15.32:0/3386580632 >> 172.21.15.32:0/3386580632 conn(0x7f31a00fdd30 msgr2=0x7f31a0100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:50:19.585 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:18.926+0000 7f31a6698700 1 -- 172.21.15.32:0/3386580632 shutdown_connections 2024-05-12T03:50:19.585 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:18.926+0000 7f31a6698700 1 -- 172.21.15.32:0/3386580632 wait complete. 2024-05-12T03:50:19.585 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:18.927+0000 7f31a6698700 1 Processor -- start 2024-05-12T03:50:19.585 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:18.927+0000 7f31a6698700 1 -- start start 2024-05-12T03:50:19.585 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:18.927+0000 7f31a6698700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31a0078c60 0x7f31a0112e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:50:19.586 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:18.927+0000 7f31a6698700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a0113350 con 0x7f31a0078c60 2024-05-12T03:50:19.586 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:18.927+0000 7f319ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31a0078c60 0x7f31a0112e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:50:19.586 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:18.927+0000 7f319ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31a0078c60 0x7f31a0112e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40204/0 (socket says 172.21.15.32:40204) 2024-05-12T03:50:19.586 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:18.927+0000 7f319ffff700 1 -- 172.21.15.32:0/550884767 learned_addr learned my addr 172.21.15.32:0/550884767 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:50:19.586 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:18.927+0000 7f319ffff700 1 -- 172.21.15.32:0/550884767 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31900094f0 con 0x7f31a0078c60 2024-05-12T03:50:19.586 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:18.928+0000 7f319ffff700 1 --2- 172.21.15.32:0/550884767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31a0078c60 0x7f31a0112e10 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3190009290 tx=0x7f3190016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:50:19.586 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:18.928+0000 7f319cff9700 1 -- 172.21.15.32:0/550884767 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f319001c5c0 con 0x7f31a0078c60 2024-05-12T03:50:19.586 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:18.928+0000 7f319cff9700 1 -- 172.21.15.32:0/550884767 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f319001c740 con 0x7f31a0078c60 2024-05-12T03:50:19.586 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:18.928+0000 7f31a6698700 1 -- 172.21.15.32:0/550884767 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31a0113550 con 0x7f31a0078c60 2024-05-12T03:50:19.586 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:18.928+0000 7f319cff9700 1 -- 172.21.15.32:0/550884767 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f319001ca50 con 0x7f31a0078c60 2024-05-12T03:50:19.586 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:18.928+0000 7f31a6698700 1 -- 172.21.15.32:0/550884767 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31a0113f80 con 0x7f31a0078c60 2024-05-12T03:50:19.586 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:18.928+0000 7f319cff9700 1 -- 172.21.15.32:0/550884767 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f319001cbb0 con 0x7f31a0078c60 2024-05-12T03:50:19.586 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:18.928+0000 7f319cff9700 1 -- 172.21.15.32:0/550884767 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f319001ac30 con 0x7f31a0078c60 2024-05-12T03:50:19.587 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:18.929+0000 7f31a6698700 1 -- 172.21.15.32:0/550884767 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31a005f5c0 con 0x7f31a0078c60 2024-05-12T03:50:19.587 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:18.931+0000 7f319cff9700 1 -- 172.21.15.32:0/550884767 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7f31900243e0 con 0x7f31a0078c60 2024-05-12T03:50:19.587 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:19.003+0000 7f31a6698700 1 -- 172.21.15.32:0/550884767 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f31a0114a30 con 0x7f31a0078c60 2024-05-12T03:50:19.587 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:19.003+0000 7f319cff9700 1 -- 172.21.15.32:0/550884767 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+325 (secure 0 0 0) 0x7f3190024580 con 0x7f31a0078c60 2024-05-12T03:50:19.587 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:19.004+0000 7f31a6698700 1 -- 172.21.15.32:0/550884767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31a0078c60 msgr2=0x7f31a0112e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:50:19.587 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:19.004+0000 7f31a6698700 1 --2- 172.21.15.32:0/550884767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31a0078c60 0x7f31a0112e10 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3190009290 tx=0x7f3190016000 comp rx=0 tx=0).stop 2024-05-12T03:50:19.587 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:19.004+0000 7f31a6698700 1 -- 172.21.15.32:0/550884767 shutdown_connections 2024-05-12T03:50:19.587 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:19.004+0000 7f31a6698700 1 --2- 172.21.15.32:0/550884767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31a0078c60 0x7f31a0112e10 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:50:19.587 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:19.004+0000 7f31a6698700 1 -- 172.21.15.32:0/550884767 >> 172.21.15.32:0/550884767 conn(0x7f31a00fdd30 msgr2=0x7f31a0109b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:50:19.587 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:19.004+0000 7f31a6698700 1 -- 172.21.15.32:0/550884767 shutdown_connections 2024-05-12T03:50:19.587 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:19.004+0000 7f31a6698700 1 -- 172.21.15.32:0/550884767 wait complete. 2024-05-12T03:50:19.587 INFO:teuthology.orchestra.run.smithi032.stdout:mon is available 2024-05-12T03:50:19.588 INFO:teuthology.orchestra.run.smithi032.stdout:Assimilating anything we can from ceph.conf... 2024-05-12T03:50:19.777 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:19 smithi032 ceph-mon[39515]: mon.smithi032 is new leader, mons smithi032 in quorum (ranks 0) 2024-05-12T03:50:19.777 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:19 smithi032 ceph-mon[39515]: monmap e1: 1 mons at {smithi032=[v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-12T03:50:19.777 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:19 smithi032 ceph-mon[39515]: fsmap 2024-05-12T03:50:19.777 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:19 smithi032 ceph-mon[39515]: osdmap e1: 0 total, 0 up, 0 in 2024-05-12T03:50:19.777 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:19 smithi032 ceph-mon[39515]: mgrmap e1: no daemons active 2024-05-12T03:50:19.778 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:19 smithi032 ceph-mon[39515]: from='client.? 172.21.15.32:0/550884767' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-05-12T03:50:20.521 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:20 smithi032 ceph-mon[39515]: from='client.? 172.21.15.32:0/2864996786' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-12T03:50:20.521 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:20 smithi032 ceph-mon[39515]: from='client.? 172.21.15.32:0/2864996786' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-12T03:50:20.824 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2024-05-12T03:50:20.824 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout [global] 2024-05-12T03:50:20.824 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout fsid = a3e178f8-1012-11ef-bc98-c7b262605968 2024-05-12T03:50:20.824 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.32:3300,v1:172.21.15.32:6789] 2024-05-12T03:50:20.824 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-12T03:50:20.824 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-12T03:50:20.824 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-12T03:50:20.825 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-12T03:50:20.825 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2024-05-12T03:50:20.825 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-12T03:50:20.825 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-12T03:50:20.825 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2024-05-12T03:50:20.825 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout [osd] 2024-05-12T03:50:20.825 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-12T03:50:20.825 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-12T03:50:20.825 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-12T03:50:20.825 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-12T03:50:20.825 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-12T03:50:20.825 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:20.086+0000 7fdfbe7f1700 1 Processor -- start 2024-05-12T03:50:20.825 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:20.086+0000 7fdfbe7f1700 1 -- start start 2024-05-12T03:50:20.825 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:20.086+0000 7fdfbe7f1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfb8101580 0x7fdfb8101990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:50:20.825 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:20.086+0000 7fdfbe7f1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb8101ed0 con 0x7fdfb8101580 2024-05-12T03:50:20.826 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:20.086+0000 7fdfb7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfb8101580 0x7fdfb8101990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:50:20.826 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:20.086+0000 7fdfb7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfb8101580 0x7fdfb8101990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40206/0 (socket says 172.21.15.32:40206) 2024-05-12T03:50:20.826 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:20.086+0000 7fdfb7fff700 1 -- 172.21.15.32:0/2644765459 learned_addr learned my addr 172.21.15.32:0/2644765459 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:50:20.826 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:20.086+0000 7fdfb7fff700 1 -- 172.21.15.32:0/2644765459 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfb8102010 con 0x7fdfb8101580 2024-05-12T03:50:20.826 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:20.087+0000 7fdfb7fff700 1 --2- 172.21.15.32:0/2644765459 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfb8101580 0x7fdfb8101990 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fdfa80098e0 tx=0x7fdfa8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb2f12be9236a9f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:50:20.826 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:20.087+0000 7fdfb6ffd700 1 -- 172.21.15.32:0/2644765459 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdfa800b860 con 0x7fdfb8101580 2024-05-12T03:50:20.826 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:20.087+0000 7fdfb6ffd700 1 -- 172.21.15.32:0/2644765459 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfa800b9c0 con 0x7fdfb8101580 2024-05-12T03:50:20.826 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:20.087+0000 7fdfb6ffd700 1 -- 172.21.15.32:0/2644765459 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdfa800bc90 con 0x7fdfb8101580 2024-05-12T03:50:20.826 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:20.087+0000 7fdfbe7f1700 1 -- 172.21.15.32:0/2644765459 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfb8101580 msgr2=0x7fdfb8101990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:50:20.826 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:20.087+0000 7fdfbe7f1700 1 --2- 172.21.15.32:0/2644765459 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfb8101580 0x7fdfb8101990 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fdfa80098e0 tx=0x7fdfa8009250 comp rx=0 tx=0).stop 2024-05-12T03:50:20.826 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:20.087+0000 7fdfbe7f1700 1 -- 172.21.15.32:0/2644765459 shutdown_connections 2024-05-12T03:50:20.826 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:20.087+0000 7fdfbe7f1700 1 --2- 172.21.15.32:0/2644765459 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfb8101580 0x7fdfb8101990 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:50:20.826 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:20.087+0000 7fdfbe7f1700 1 -- 172.21.15.32:0/2644765459 >> 172.21.15.32:0/2644765459 conn(0x7fdfb80fcc50 msgr2=0x7fdfb80ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:50:20.826 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:20.087+0000 7fdfbe7f1700 1 -- 172.21.15.32:0/2644765459 shutdown_connections 2024-05-12T03:50:20.827 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:20.087+0000 7fdfbe7f1700 1 -- 172.21.15.32:0/2644765459 wait complete. 2024-05-12T03:50:20.827 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:20.088+0000 7fdfbe7f1700 1 Processor -- start 2024-05-12T03:50:20.827 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:20.088+0000 7fdfbe7f1700 1 -- start start 2024-05-12T03:50:20.827 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:20.088+0000 7fdfbe7f1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfb8101580 0x7fdfb8076b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:50:20.827 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:20.088+0000 7fdfbe7f1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb81138d0 con 0x7fdfb8101580 2024-05-12T03:50:20.827 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:20.088+0000 7fdfb7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfb8101580 0x7fdfb8076b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:50:20.827 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:20.088+0000 7fdfb7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfb8101580 0x7fdfb8076b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40218/0 (socket says 172.21.15.32:40218) 2024-05-12T03:50:20.827 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:20.088+0000 7fdfb7fff700 1 -- 172.21.15.32:0/2864996786 learned_addr learned my addr 172.21.15.32:0/2864996786 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:50:20.827 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:20.089+0000 7fdfb7fff700 1 -- 172.21.15.32:0/2864996786 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfa80094f0 con 0x7fdfb8101580 2024-05-12T03:50:20.827 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:20.089+0000 7fdfb7fff700 1 --2- 172.21.15.32:0/2864996786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfb8101580 0x7fdfb8076b10 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8003080 tx=0x7fdfa8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:50:20.827 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:20.089+0000 7fdfb4ff9700 1 -- 172.21.15.32:0/2864996786 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdfa801c5c0 con 0x7fdfb8101580 2024-05-12T03:50:20.827 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:20.089+0000 7fdfb4ff9700 1 -- 172.21.15.32:0/2864996786 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfa801c720 con 0x7fdfb8101580 2024-05-12T03:50:20.827 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:20.089+0000 7fdfbe7f1700 1 -- 172.21.15.32:0/2864996786 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfb8113a10 con 0x7fdfb8101580 2024-05-12T03:50:20.827 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:20.089+0000 7fdfb4ff9700 1 -- 172.21.15.32:0/2864996786 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdfa801ca30 con 0x7fdfb8101580 2024-05-12T03:50:20.827 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:20.089+0000 7fdfbe7f1700 1 -- 172.21.15.32:0/2864996786 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfb8113ea0 con 0x7fdfb8101580 2024-05-12T03:50:20.828 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:20.090+0000 7fdfb4ff9700 1 -- 172.21.15.32:0/2864996786 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fdfa801cc60 con 0x7fdfb8101580 2024-05-12T03:50:20.828 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:20.090+0000 7fdfb4ff9700 1 -- 172.21.15.32:0/2864996786 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fdfa801ac80 con 0x7fdfb8101580 2024-05-12T03:50:20.828 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:20.090+0000 7fdfbe7f1700 1 -- 172.21.15.32:0/2864996786 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfb8109600 con 0x7fdfb8101580 2024-05-12T03:50:20.828 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:20.092+0000 7fdfb4ff9700 1 -- 172.21.15.32:0/2864996786 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7fdfa801a450 con 0x7fdfb8101580 2024-05-12T03:50:20.828 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:20.161+0000 7fdfbe7f1700 1 -- 172.21.15.32:0/2864996786 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fdfb804bed0 con 0x7fdfb8101580 2024-05-12T03:50:20.828 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:20.165+0000 7fdfb4ff9700 1 -- 172.21.15.32:0/2864996786 <== mon.0 v2:172.21.15.32:3300/0 7 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7fdfa8024850 con 0x7fdfb8101580 2024-05-12T03:50:20.828 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:20.165+0000 7fdfb4ff9700 1 -- 172.21.15.32:0/2864996786 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7fdfb804bed0 con 0x7fdfb8101580 2024-05-12T03:50:20.828 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:20.166+0000 7fdfbe7f1700 1 -- 172.21.15.32:0/2864996786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfb8101580 msgr2=0x7fdfb8076b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:50:20.828 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:20.166+0000 7fdfbe7f1700 1 --2- 172.21.15.32:0/2864996786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfb8101580 0x7fdfb8076b10 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8003080 tx=0x7fdfa8016000 comp rx=0 tx=0).stop 2024-05-12T03:50:20.828 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:20.166+0000 7fdfbe7f1700 1 -- 172.21.15.32:0/2864996786 shutdown_connections 2024-05-12T03:50:20.828 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:20.166+0000 7fdfbe7f1700 1 --2- 172.21.15.32:0/2864996786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfb8101580 0x7fdfb8076b10 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:50:20.828 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:20.166+0000 7fdfbe7f1700 1 -- 172.21.15.32:0/2864996786 >> 172.21.15.32:0/2864996786 conn(0x7fdfb80fcc50 msgr2=0x7fdfb8103fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:50:20.828 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:20.166+0000 7fdfbe7f1700 1 -- 172.21.15.32:0/2864996786 shutdown_connections 2024-05-12T03:50:20.828 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:20.166+0000 7fdfbe7f1700 1 -- 172.21.15.32:0/2864996786 wait complete. 2024-05-12T03:50:20.829 INFO:teuthology.orchestra.run.smithi032.stdout:Generating new minimal ceph.conf... 2024-05-12T03:50:21.778 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:21 smithi032 ceph-mon[39515]: from='client.? 172.21.15.32:0/600603880' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:50:21.993 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:21.328+0000 7fc305b4f700 1 Processor -- start 2024-05-12T03:50:21.993 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:21.328+0000 7fc305b4f700 1 -- start start 2024-05-12T03:50:21.993 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:21.328+0000 7fc305b4f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc300076c70 0x7fc3001051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:50:21.993 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:21.328+0000 7fc305b4f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3001056e0 con 0x7fc300076c70 2024-05-12T03:50:21.993 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:21.328+0000 7fc2ff7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc300076c70 0x7fc3001051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:50:21.993 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:21.328+0000 7fc2ff7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc300076c70 0x7fc3001051a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40234/0 (socket says 172.21.15.32:40234) 2024-05-12T03:50:21.993 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:21.328+0000 7fc2ff7fe700 1 -- 172.21.15.32:0/1110081248 learned_addr learned my addr 172.21.15.32:0/1110081248 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:50:21.994 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:21.328+0000 7fc2ff7fe700 1 -- 172.21.15.32:0/1110081248 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc300105820 con 0x7fc300076c70 2024-05-12T03:50:21.994 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:21.329+0000 7fc2ff7fe700 1 --2- 172.21.15.32:0/1110081248 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc300076c70 0x7fc3001051a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc2f00098e0 tx=0x7fc2f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69d6303432963fd6 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:50:21.994 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:21.329+0000 7fc2fe7fc700 1 -- 172.21.15.32:0/1110081248 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc2f000b860 con 0x7fc300076c70 2024-05-12T03:50:21.994 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:21.329+0000 7fc2fe7fc700 1 -- 172.21.15.32:0/1110081248 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7fc2f000be30 con 0x7fc300076c70 2024-05-12T03:50:21.994 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:21.329+0000 7fc2fe7fc700 1 -- 172.21.15.32:0/1110081248 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc2f0010380 con 0x7fc300076c70 2024-05-12T03:50:21.994 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:21.329+0000 7fc305b4f700 1 -- 172.21.15.32:0/1110081248 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc300076c70 msgr2=0x7fc3001051a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:50:21.994 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:21.329+0000 7fc305b4f700 1 --2- 172.21.15.32:0/1110081248 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc300076c70 0x7fc3001051a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc2f00098e0 tx=0x7fc2f0009250 comp rx=0 tx=0).stop 2024-05-12T03:50:21.994 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:21.329+0000 7fc305b4f700 1 -- 172.21.15.32:0/1110081248 shutdown_connections 2024-05-12T03:50:21.994 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:21.329+0000 7fc305b4f700 1 --2- 172.21.15.32:0/1110081248 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc300076c70 0x7fc3001051a0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:50:21.994 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:21.329+0000 7fc305b4f700 1 -- 172.21.15.32:0/1110081248 >> 172.21.15.32:0/1110081248 conn(0x7fc3000fcc50 msgr2=0x7fc3000ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:50:21.994 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:21.329+0000 7fc305b4f700 1 -- 172.21.15.32:0/1110081248 shutdown_connections 2024-05-12T03:50:21.994 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:21.329+0000 7fc305b4f700 1 -- 172.21.15.32:0/1110081248 wait complete. 2024-05-12T03:50:21.994 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:21.330+0000 7fc305b4f700 1 Processor -- start 2024-05-12T03:50:21.994 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:21.330+0000 7fc305b4f700 1 -- start start 2024-05-12T03:50:21.994 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:21.331+0000 7fc305b4f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc300076c70 0x7fc300102af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:50:21.995 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:21.331+0000 7fc305b4f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc300103030 con 0x7fc300076c70 2024-05-12T03:50:21.995 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:21.331+0000 7fc2ff7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc300076c70 0x7fc300102af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:50:21.995 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:21.331+0000 7fc2ff7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc300076c70 0x7fc300102af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40250/0 (socket says 172.21.15.32:40250) 2024-05-12T03:50:21.995 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:21.331+0000 7fc2ff7fe700 1 -- 172.21.15.32:0/600603880 learned_addr learned my addr 172.21.15.32:0/600603880 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:50:21.995 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:21.331+0000 7fc2ff7fe700 1 -- 172.21.15.32:0/600603880 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2f00094f0 con 0x7fc300076c70 2024-05-12T03:50:21.995 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:21.331+0000 7fc2ff7fe700 1 --2- 172.21.15.32:0/600603880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc300076c70 0x7fc300102af0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0006b00 tx=0x7fc2f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:50:21.995 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:21.331+0000 7fc2e7fff700 1 -- 172.21.15.32:0/600603880 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc2f0010a80 con 0x7fc300076c70 2024-05-12T03:50:21.995 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:21.331+0000 7fc2e7fff700 1 -- 172.21.15.32:0/600603880 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7fc2f001a430 con 0x7fc300076c70 2024-05-12T03:50:21.995 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:21.331+0000 7fc305b4f700 1 -- 172.21.15.32:0/600603880 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3001010a0 con 0x7fc300076c70 2024-05-12T03:50:21.995 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:21.331+0000 7fc2e7fff700 1 -- 172.21.15.32:0/600603880 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc2f00245b0 con 0x7fc300076c70 2024-05-12T03:50:21.995 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:21.331+0000 7fc305b4f700 1 -- 172.21.15.32:0/600603880 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc300101590 con 0x7fc300076c70 2024-05-12T03:50:21.995 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:21.332+0000 7fc2e7fff700 1 -- 172.21.15.32:0/600603880 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fc2f00247e0 con 0x7fc300076c70 2024-05-12T03:50:21.995 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:21.332+0000 7fc2e7fff700 1 -- 172.21.15.32:0/600603880 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fc2f002e830 con 0x7fc300076c70 2024-05-12T03:50:21.995 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:21.332+0000 7fc305b4f700 1 -- 172.21.15.32:0/600603880 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc30004bed0 con 0x7fc300076c70 2024-05-12T03:50:21.995 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:21.335+0000 7fc2e7fff700 1 -- 172.21.15.32:0/600603880 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7fc2f0010be0 con 0x7fc300076c70 2024-05-12T03:50:21.996 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:21.405+0000 7fc305b4f700 1 -- 172.21.15.32:0/600603880 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fc3001017d0 con 0x7fc300076c70 2024-05-12T03:50:21.996 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:21.405+0000 7fc2e7fff700 1 -- 172.21.15.32:0/600603880 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7fc2f0010e40 con 0x7fc300076c70 2024-05-12T03:50:21.996 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:21.406+0000 7fc305b4f700 1 -- 172.21.15.32:0/600603880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc300076c70 msgr2=0x7fc300102af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:50:21.996 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:21.406+0000 7fc305b4f700 1 --2- 172.21.15.32:0/600603880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc300076c70 0x7fc300102af0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0006b00 tx=0x7fc2f0009250 comp rx=0 tx=0).stop 2024-05-12T03:50:21.996 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:21.406+0000 7fc305b4f700 1 -- 172.21.15.32:0/600603880 shutdown_connections 2024-05-12T03:50:21.996 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:21.406+0000 7fc305b4f700 1 --2- 172.21.15.32:0/600603880 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc300076c70 0x7fc300102af0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:50:21.996 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:21.406+0000 7fc305b4f700 1 -- 172.21.15.32:0/600603880 >> 172.21.15.32:0/600603880 conn(0x7fc3000fcc50 msgr2=0x7fc3000fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:50:21.996 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:21.406+0000 7fc305b4f700 1 -- 172.21.15.32:0/600603880 shutdown_connections 2024-05-12T03:50:21.996 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:21.406+0000 7fc305b4f700 1 -- 172.21.15.32:0/600603880 wait complete. 2024-05-12T03:50:21.996 INFO:teuthology.orchestra.run.smithi032.stdout:Restarting the monitor... 2024-05-12T03:50:22.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:22 smithi032 systemd[1]: Stopping Ceph mon.smithi032 for a3e178f8-1012-11ef-bc98-c7b262605968... 2024-05-12T03:50:22.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:22 smithi032 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-mon-smithi032[39492]: 2024-05-12T03:50:22.270+0000 7f6fb8bb5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi032 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-12T03:50:22.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:22 smithi032 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-mon-smithi032[39492]: 2024-05-12T03:50:22.270+0000 7f6fb8bb5700 -1 mon.smithi032@0(leader) e1 *** Got Signal Terminated *** 2024-05-12T03:50:23.789 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:23 smithi032 bash[39846]: ceph-a3e178f8-1012-11ef-bc98-c7b262605968-mon-smithi032 2024-05-12T03:50:25.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:25 smithi032 systemd[1]: ceph-a3e178f8-1012-11ef-bc98-c7b262605968@mon.smithi032.service: Succeeded. 2024-05-12T03:50:25.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:25 smithi032 systemd[1]: Stopped Ceph mon.smithi032 for a3e178f8-1012-11ef-bc98-c7b262605968. 2024-05-12T03:50:25.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:25 smithi032 systemd[1]: Starting Ceph mon.smithi032 for a3e178f8-1012-11ef-bc98-c7b262605968... 2024-05-12T03:50:27.078 INFO:teuthology.orchestra.run.smithi032.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-12T03:50:27.286 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: set uid:gid to 167:167 (ceph:ceph) 2024-05-12T03:50:27.287 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable), process ceph-mon, pid 7 2024-05-12T03:50:27.287 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: pidfile_write: ignore empty --pid-file 2024-05-12T03:50:27.287 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: load: jerasure load: lrc 2024-05-12T03:50:27.287 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: RocksDB version: 6.15.5 2024-05-12T03:50:27.287 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-05-12T03:50:27.287 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Compile date May 10 2024 2024-05-12T03:50:27.287 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: DB SUMMARY 2024-05-12T03:50:27.287 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: DB Session ID: 09T8ORFK11RZY83XKCPC 2024-05-12T03:50:27.287 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: CURRENT file: CURRENT 2024-05-12T03:50:27.287 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: IDENTITY file: IDENTITY 2024-05-12T03:50:27.287 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: MANIFEST file: MANIFEST-000009 size: 131 Bytes 2024-05-12T03:50:27.287 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi032/store.db dir, Total Num: 1, files: 000008.sst 2024-05-12T03:50:27.287 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi032/store.db: 000010.log size: 96748 ; 2024-05-12T03:50:27.288 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.error_if_exists: 0 2024-05-12T03:50:27.288 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.create_if_missing: 0 2024-05-12T03:50:27.288 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.paranoid_checks: 1 2024-05-12T03:50:27.288 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-12T03:50:27.288 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.env: 0x559a61a14b00 2024-05-12T03:50:27.288 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.fs: Posix File System 2024-05-12T03:50:27.288 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.info_log: 0x559a62db7620 2024-05-12T03:50:27.288 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.max_file_opening_threads: 16 2024-05-12T03:50:27.288 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.statistics: (nil) 2024-05-12T03:50:27.288 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.use_fsync: 0 2024-05-12T03:50:27.288 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.max_log_file_size: 0 2024-05-12T03:50:27.288 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-12T03:50:27.288 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-12T03:50:27.288 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.keep_log_file_num: 1000 2024-05-12T03:50:27.288 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.recycle_log_file_num: 0 2024-05-12T03:50:27.288 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.allow_fallocate: 1 2024-05-12T03:50:27.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.allow_mmap_reads: 0 2024-05-12T03:50:27.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.allow_mmap_writes: 0 2024-05-12T03:50:27.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.use_direct_reads: 0 2024-05-12T03:50:27.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-12T03:50:27.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.create_missing_column_families: 0 2024-05-12T03:50:27.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.db_log_dir: 2024-05-12T03:50:27.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-smithi032/store.db 2024-05-12T03:50:27.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-12T03:50:27.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-12T03:50:27.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-12T03:50:27.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-12T03:50:27.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-12T03:50:27.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-12T03:50:27.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.advise_random_on_open: 1 2024-05-12T03:50:27.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.db_write_buffer_size: 0 2024-05-12T03:50:27.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.write_buffer_manager: 0x559a63aaaa20 2024-05-12T03:50:27.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-12T03:50:27.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-05-12T03:50:27.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-12T03:50:27.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-12T03:50:27.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.rate_limiter: (nil) 2024-05-12T03:50:27.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-12T03:50:27.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.wal_recovery_mode: 2 2024-05-12T03:50:27.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.enable_thread_tracking: 0 2024-05-12T03:50:27.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.enable_pipelined_write: 0 2024-05-12T03:50:27.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.unordered_write: 0 2024-05-12T03:50:27.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-12T03:50:27.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-12T03:50:27.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-12T03:50:27.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-12T03:50:27.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.row_cache: None 2024-05-12T03:50:27.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.wal_filter: None 2024-05-12T03:50:27.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-12T03:50:27.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.allow_ingest_behind: 0 2024-05-12T03:50:27.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.preserve_deletes: 0 2024-05-12T03:50:27.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.two_write_queues: 0 2024-05-12T03:50:27.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.manual_wal_flush: 0 2024-05-12T03:50:27.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.atomic_flush: 0 2024-05-12T03:50:27.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-12T03:50:27.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-12T03:50:27.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-12T03:50:27.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.log_readahead_size: 0 2024-05-12T03:50:27.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-12T03:50:27.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.best_efforts_recovery: 0 2024-05-12T03:50:27.292 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-12T03:50:27.292 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-12T03:50:27.292 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.allow_data_in_errors: 0 2024-05-12T03:50:27.292 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.db_host_id: __hostname__ 2024-05-12T03:50:27.292 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.max_background_jobs: 2 2024-05-12T03:50:27.293 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.max_background_compactions: -1 2024-05-12T03:50:27.293 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.max_subcompactions: 1 2024-05-12T03:50:27.293 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-12T03:50:27.294 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-12T03:50:27.294 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-12T03:50:27.294 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.max_total_wal_size: 0 2024-05-12T03:50:27.294 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-12T03:50:27.294 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-12T03:50:27.294 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-12T03:50:27.294 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-12T03:50:27.294 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.max_open_files: -1 2024-05-12T03:50:27.294 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.bytes_per_sync: 0 2024-05-12T03:50:27.294 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-12T03:50:27.294 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-12T03:50:27.294 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.compaction_readahead_size: 0 2024-05-12T03:50:27.294 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.max_background_flushes: -1 2024-05-12T03:50:27.294 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Compression algorithms supported: 2024-05-12T03:50:27.294 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-12T03:50:27.295 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: kZSTD supported: 0 2024-05-12T03:50:27.295 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: kXpressCompression supported: 0 2024-05-12T03:50:27.295 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: kLZ4HCCompression supported: 1 2024-05-12T03:50:27.295 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: kZlibCompression supported: 1 2024-05-12T03:50:27.295 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: kSnappyCompression supported: 1 2024-05-12T03:50:27.295 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: kLZ4Compression supported: 1 2024-05-12T03:50:27.295 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: kBZip2Compression supported: 0 2024-05-12T03:50:27.295 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-12T03:50:27.295 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi032/store.db/MANIFEST-000009 2024-05-12T03:50:27.295 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-05-12T03:50:27.295 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-12T03:50:27.295 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.merge_operator: 2024-05-12T03:50:27.295 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.compaction_filter: None 2024-05-12T03:50:27.295 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.compaction_filter_factory: None 2024-05-12T03:50:27.295 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.sst_partitioner_factory: None 2024-05-12T03:50:27.295 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-12T03:50:27.296 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-12T03:50:27.296 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x559a62d8af58) 2024-05-12T03:50:27.296 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: cache_index_and_filter_blocks: 1 2024-05-12T03:50:27.296 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-12T03:50:27.296 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-12T03:50:27.296 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: pin_top_level_index_and_filter: 1 2024-05-12T03:50:27.296 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: index_type: 0 2024-05-12T03:50:27.296 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: data_block_index_type: 0 2024-05-12T03:50:27.296 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: index_shortening: 1 2024-05-12T03:50:27.296 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-12T03:50:27.296 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: hash_index_allow_collision: 1 2024-05-12T03:50:27.296 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: checksum: 1 2024-05-12T03:50:27.296 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: no_block_cache: 0 2024-05-12T03:50:27.296 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: block_cache: 0x559a62dbd090 2024-05-12T03:50:27.296 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: block_cache_name: BinnedLRUCache 2024-05-12T03:50:27.296 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: block_cache_options: 2024-05-12T03:50:27.297 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: capacity : 536870912 2024-05-12T03:50:27.297 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: num_shard_bits : 4 2024-05-12T03:50:27.297 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: strict_capacity_limit : 0 2024-05-12T03:50:27.297 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: high_pri_pool_ratio: 0.000 2024-05-12T03:50:27.297 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: block_cache_compressed: (nil) 2024-05-12T03:50:27.297 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: persistent_cache: (nil) 2024-05-12T03:50:27.297 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: block_size: 4096 2024-05-12T03:50:27.297 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: block_size_deviation: 10 2024-05-12T03:50:27.297 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: block_restart_interval: 16 2024-05-12T03:50:27.297 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: index_block_restart_interval: 1 2024-05-12T03:50:27.297 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: metadata_block_size: 4096 2024-05-12T03:50:27.297 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: partition_filters: 0 2024-05-12T03:50:27.297 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: use_delta_encoding: 1 2024-05-12T03:50:27.297 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-05-12T03:50:27.297 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: whole_key_filtering: 1 2024-05-12T03:50:27.297 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: verify_compression: 0 2024-05-12T03:50:27.297 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: read_amp_bytes_per_bit: 0 2024-05-12T03:50:27.298 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: format_version: 4 2024-05-12T03:50:27.298 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: enable_index_compression: 1 2024-05-12T03:50:27.298 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: block_align: 0 2024-05-12T03:50:27.298 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.write_buffer_size: 33554432 2024-05-12T03:50:27.298 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.max_write_buffer_number: 2 2024-05-12T03:50:27.298 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.compression: NoCompression 2024-05-12T03:50:27.298 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.bottommost_compression: Disabled 2024-05-12T03:50:27.298 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.prefix_extractor: nullptr 2024-05-12T03:50:27.298 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-12T03:50:27.298 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.num_levels: 7 2024-05-12T03:50:27.298 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-12T03:50:27.298 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-12T03:50:27.298 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-12T03:50:27.299 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-12T03:50:27.299 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-12T03:50:27.299 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-12T03:50:27.299 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-12T03:50:27.299 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-12T03:50:27.299 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-12T03:50:27.299 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-12T03:50:27.299 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-12T03:50:27.299 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.compression_opts.level: 32767 2024-05-12T03:50:27.299 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.compression_opts.strategy: 0 2024-05-12T03:50:27.299 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-12T03:50:27.299 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-12T03:50:27.299 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-12T03:50:27.299 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.compression_opts.enabled: false 2024-05-12T03:50:27.299 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-12T03:50:27.300 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-12T03:50:27.300 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-12T03:50:27.300 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.target_file_size_base: 67108864 2024-05-12T03:50:27.300 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-12T03:50:27.300 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-12T03:50:27.300 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-12T03:50:27.300 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-12T03:50:27.300 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-12T03:50:27.300 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-12T03:50:27.300 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-12T03:50:27.300 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-12T03:50:27.300 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-12T03:50:27.300 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-12T03:50:27.300 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-12T03:50:27.300 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-12T03:50:27.300 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-12T03:50:27.301 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.arena_block_size: 4194304 2024-05-12T03:50:27.301 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-12T03:50:27.301 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-12T03:50:27.301 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-05-12T03:50:27.301 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.disable_auto_compactions: 0 2024-05-12T03:50:27.301 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-12T03:50:27.301 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-12T03:50:27.301 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-12T03:50:27.301 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-12T03:50:27.301 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-12T03:50:27.301 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-12T03:50:27.301 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-12T03:50:27.301 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-12T03:50:27.301 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-12T03:50:27.301 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-12T03:50:27.301 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.table_properties_collectors: 2024-05-12T03:50:27.302 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.inplace_update_support: 0 2024-05-12T03:50:27.302 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-12T03:50:27.302 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-12T03:50:27.302 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-12T03:50:27.302 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-12T03:50:27.302 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.bloom_locality: 0 2024-05-12T03:50:27.302 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.max_successive_merges: 0 2024-05-12T03:50:27.302 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-12T03:50:27.302 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.paranoid_file_checks: 0 2024-05-12T03:50:27.302 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.force_consistency_checks: 1 2024-05-12T03:50:27.302 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.report_bg_io_stats: 0 2024-05-12T03:50:27.302 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.ttl: 2592000 2024-05-12T03:50:27.302 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-12T03:50:27.302 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.enable_blob_files: false 2024-05-12T03:50:27.302 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.min_blob_size: 0 2024-05-12T03:50:27.302 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.blob_file_size: 268435456 2024-05-12T03:50:27.303 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-12T03:50:27.303 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-12T03:50:27.303 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-12T03:50:27.303 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi032/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 11, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-12T03:50:27.303 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 5 2024-05-12T03:50:27.303 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: [db/version_set.cc:4082] Creating manifest 13 2024-05-12T03:50:27.303 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715485827024577, "job": 1, "event": "recovery_started", "wal_files": [10]} 2024-05-12T03:50:27.303 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #10 mode 2 2024-05-12T03:50:27.303 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-05-12T03:50:27.303 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715485827025909, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 14, "file_size": 92117, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 90212, "index_size": 265, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13225, "raw_average_key_size": 49, "raw_value_size": 82951, "raw_average_value_size": 310, "num_data_blocks": 11, "num_entries": 267, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1715485827, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "6b5adf2d-d544-4e40-89cd-4cfcd89bf7d6", "db_session_id": "09T8ORFK11RZY83XKCPC"}} 2024-05-12T03:50:27.303 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: [db/version_set.cc:4082] Creating manifest 15 2024-05-12T03:50:27.303 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715485827026498, "job": 1, "event": "recovery_finished"} 2024-05-12T03:50:27.303 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-smithi032/store.db/000010.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-12T03:50:27.303 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x559a62d51180 2024-05-12T03:50:27.304 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: DB pointer 0x559a62e26000 2024-05-12T03:50:27.304 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-05-12T03:50:27.304 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-05-12T03:50:27.304 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: ** DB Stats ** 2024-05-12T03:50:27.304 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-12T03:50:27.304 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-12T03:50:27.304 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-12T03:50:27.304 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-12T03:50:27.304 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-12T03:50:27.304 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-05-12T03:50:27.304 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-12T03:50:27.304 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: 2024-05-12T03:50:27.304 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: ** Compaction Stats [default] ** 2024-05-12T03:50:27.304 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-05-12T03:50:27.304 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-12T03:50:27.304 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: L0 2/0 91.76 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 105.2 0.00 0.00 1 0.001 0 0 2024-05-12T03:50:27.305 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Sum 2/0 91.76 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 105.2 0.00 0.00 1 0.001 0 0 2024-05-12T03:50:27.305 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 105.2 0.00 0.00 1 0.001 0 0 2024-05-12T03:50:27.305 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: 2024-05-12T03:50:27.305 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: ** Compaction Stats [default] ** 2024-05-12T03:50:27.305 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-05-12T03:50:27.305 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-12T03:50:27.305 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 105.2 0.00 0.00 1 0.001 0 0 2024-05-12T03:50:27.305 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-12T03:50:27.305 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-12T03:50:27.305 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-12T03:50:27.305 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-12T03:50:27.305 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-12T03:50:27.305 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-12T03:50:27.305 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Cumulative compaction: 0.00 GB write, 16.85 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-12T03:50:27.305 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Interval compaction: 0.00 GB write, 16.85 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-12T03:50:27.305 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-12T03:50:27.306 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: 2024-05-12T03:50:27.306 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-12T03:50:27.306 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: 2024-05-12T03:50:27.306 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: ** Compaction Stats [default] ** 2024-05-12T03:50:27.306 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-05-12T03:50:27.306 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-12T03:50:27.306 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: L0 2/0 91.76 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 105.2 0.00 0.00 1 0.001 0 0 2024-05-12T03:50:27.306 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Sum 2/0 91.76 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 105.2 0.00 0.00 1 0.001 0 0 2024-05-12T03:50:27.306 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-05-12T03:50:27.306 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: 2024-05-12T03:50:27.306 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: ** Compaction Stats [default] ** 2024-05-12T03:50:27.306 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-05-12T03:50:27.306 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-12T03:50:27.306 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 105.2 0.00 0.00 1 0.001 0 0 2024-05-12T03:50:27.306 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-12T03:50:27.306 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-12T03:50:27.307 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-12T03:50:27.307 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-12T03:50:27.307 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-12T03:50:27.307 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-12T03:50:27.307 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Cumulative compaction: 0.00 GB write, 16.72 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-12T03:50:27.307 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-12T03:50:27.307 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-12T03:50:27.307 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: 2024-05-12T03:50:27.307 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-12T03:50:27.307 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: starting mon.smithi032 rank 0 at public addrs [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] at bind addrs [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi032 fsid a3e178f8-1012-11ef-bc98-c7b262605968 2024-05-12T03:50:27.307 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: mon.smithi032@-1(???) e1 preinit fsid a3e178f8-1012-11ef-bc98-c7b262605968 2024-05-12T03:50:27.307 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: mon.smithi032@-1(???).mds e1 new map 2024-05-12T03:50:27.307 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: mon.smithi032@-1(???).mds e1 print_map 2024-05-12T03:50:27.307 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: e1 2024-05-12T03:50:27.307 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-12T03:50:27.308 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-12T03:50:27.308 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: legacy client fscid: -1 2024-05-12T03:50:27.308 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: 2024-05-12T03:50:27.308 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: No filesystems configured 2024-05-12T03:50:27.308 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: mon.smithi032@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-12T03:50:27.308 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: mon.smithi032@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-12T03:50:27.308 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: mon.smithi032@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-12T03:50:27.308 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: mon.smithi032@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-12T03:50:27.308 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: mon.smithi032@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-12T03:50:27.308 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 bash[39970]: d7fea430c6e221647431b6bd59a4fc8a00a76871d049e62e09e8e392b66356f1 2024-05-12T03:50:27.308 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 systemd[1]: Started Ceph mon.smithi032 for a3e178f8-1012-11ef-bc98-c7b262605968. 2024-05-12T03:50:27.308 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: mon.smithi032 is new leader, mons smithi032 in quorum (ranks 0) 2024-05-12T03:50:27.308 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: monmap e1: 1 mons at {smithi032=[v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-12T03:50:27.308 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: fsmap 2024-05-12T03:50:27.308 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: osdmap e1: 0 total, 0 up, 0 in 2024-05-12T03:50:27.308 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:27 smithi032 ceph-mon[40008]: mgrmap e1: no daemons active 2024-05-12T03:50:28.310 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:27.588+0000 7f09c2771700 1 Processor -- start 2024-05-12T03:50:28.311 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:27.588+0000 7f09c2771700 1 -- start start 2024-05-12T03:50:28.311 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:27.588+0000 7f09c2771700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09bc102b90 0x7f09bc102fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:50:28.311 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:27.588+0000 7f09c2771700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09bc1034e0 con 0x7f09bc102b90 2024-05-12T03:50:28.311 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:27.589+0000 7f09bbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09bc102b90 0x7f09bc102fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:50:28.311 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:27.589+0000 7f09bbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09bc102b90 0x7f09bc102fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47652/0 (socket says 172.21.15.32:47652) 2024-05-12T03:50:28.311 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:27.589+0000 7f09bbfff700 1 -- 172.21.15.32:0/3821494972 learned_addr learned my addr 172.21.15.32:0/3821494972 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:50:28.312 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:27.589+0000 7f09bbfff700 1 -- 172.21.15.32:0/3821494972 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09bc103620 con 0x7f09bc102b90 2024-05-12T03:50:28.312 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:27.589+0000 7f09bbfff700 1 --2- 172.21.15.32:0/3821494972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09bc102b90 0x7f09bc102fa0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f09b00098e0 tx=0x7f09b0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11414c6c66fbf037 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:50:28.312 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:27.589+0000 7f09baffd700 1 -- 172.21.15.32:0/3821494972 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f09b000b860 con 0x7f09bc102b90 2024-05-12T03:50:28.312 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:27.589+0000 7f09baffd700 1 -- 172.21.15.32:0/3821494972 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f09b000be30 con 0x7f09bc102b90 2024-05-12T03:50:28.312 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:27.589+0000 7f09baffd700 1 -- 172.21.15.32:0/3821494972 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f09b0010380 con 0x7f09bc102b90 2024-05-12T03:50:28.312 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:27.590+0000 7f09c2771700 1 -- 172.21.15.32:0/3821494972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09bc102b90 msgr2=0x7f09bc102fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:50:28.313 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:27.590+0000 7f09c2771700 1 --2- 172.21.15.32:0/3821494972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09bc102b90 0x7f09bc102fa0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f09b00098e0 tx=0x7f09b0009250 comp rx=0 tx=0).stop 2024-05-12T03:50:28.313 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:27.590+0000 7f09c2771700 1 -- 172.21.15.32:0/3821494972 shutdown_connections 2024-05-12T03:50:28.313 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:27.590+0000 7f09c2771700 1 --2- 172.21.15.32:0/3821494972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09bc102b90 0x7f09bc102fa0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:50:28.313 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:27.590+0000 7f09c2771700 1 -- 172.21.15.32:0/3821494972 >> 172.21.15.32:0/3821494972 conn(0x7f09bc0fc380 msgr2=0x7f09bc0fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:50:28.313 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:27.590+0000 7f09c2771700 1 -- 172.21.15.32:0/3821494972 shutdown_connections 2024-05-12T03:50:28.313 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:27.590+0000 7f09c2771700 1 -- 172.21.15.32:0/3821494972 wait complete. 2024-05-12T03:50:28.314 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:27.590+0000 7f09c2771700 1 Processor -- start 2024-05-12T03:50:28.314 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:27.591+0000 7f09c2771700 1 -- start start 2024-05-12T03:50:28.314 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:27.591+0000 7f09c2771700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09bc102b90 0x7f09bc113550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:50:28.314 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:27.591+0000 7f09c2771700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09bc113a90 con 0x7f09bc102b90 2024-05-12T03:50:28.314 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:27.591+0000 7f09bbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09bc102b90 0x7f09bc113550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:50:28.315 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:27.591+0000 7f09bbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09bc102b90 0x7f09bc113550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47662/0 (socket says 172.21.15.32:47662) 2024-05-12T03:50:28.315 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:27.591+0000 7f09bbfff700 1 -- 172.21.15.32:0/3355095735 learned_addr learned my addr 172.21.15.32:0/3355095735 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:50:28.315 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:27.591+0000 7f09bbfff700 1 -- 172.21.15.32:0/3355095735 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b00094f0 con 0x7f09bc102b90 2024-05-12T03:50:28.315 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:27.591+0000 7f09bbfff700 1 --2- 172.21.15.32:0/3355095735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09bc102b90 0x7f09bc113550 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f09b0000c00 tx=0x7f09b0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:50:28.315 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:27.591+0000 7f09b8ff9700 1 -- 172.21.15.32:0/3355095735 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f09b0010a80 con 0x7f09bc102b90 2024-05-12T03:50:28.315 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:27.591+0000 7f09c2771700 1 -- 172.21.15.32:0/3355095735 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09bc113c90 con 0x7f09bc102b90 2024-05-12T03:50:28.315 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:27.591+0000 7f09b8ff9700 1 -- 172.21.15.32:0/3355095735 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f09b001a430 con 0x7f09bc102b90 2024-05-12T03:50:28.315 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:27.591+0000 7f09b8ff9700 1 -- 172.21.15.32:0/3355095735 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f09b00245b0 con 0x7f09bc102b90 2024-05-12T03:50:28.315 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:27.591+0000 7f09c2771700 1 -- 172.21.15.32:0/3355095735 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09bc114800 con 0x7f09bc102b90 2024-05-12T03:50:28.316 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:27.592+0000 7f09b8ff9700 1 -- 172.21.15.32:0/3355095735 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f09b0010be0 con 0x7f09bc102b90 2024-05-12T03:50:28.316 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:27.592+0000 7f09b8ff9700 1 -- 172.21.15.32:0/3355095735 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f09b002e790 con 0x7f09bc102b90 2024-05-12T03:50:28.316 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:27.592+0000 7f09c2771700 1 -- 172.21.15.32:0/3355095735 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09bc05f5c0 con 0x7f09bc102b90 2024-05-12T03:50:28.316 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:27.595+0000 7f09b8ff9700 1 -- 172.21.15.32:0/3355095735 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7f09b001a960 con 0x7f09bc102b90 2024-05-12T03:50:28.316 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:27.663+0000 7f09c2771700 1 -- 172.21.15.32:0/3355095735 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f09bc1087d0 con 0x7f09bc102b90 2024-05-12T03:50:28.316 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:27.666+0000 7f09b8ff9700 1 -- 172.21.15.32:0/3355095735 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f09b00247f0 con 0x7f09bc102b90 2024-05-12T03:50:28.316 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:27.666+0000 7f09c2771700 1 -- 172.21.15.32:0/3355095735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09bc102b90 msgr2=0x7f09bc113550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:50:28.317 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:27.666+0000 7f09c2771700 1 --2- 172.21.15.32:0/3355095735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09bc102b90 0x7f09bc113550 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f09b0000c00 tx=0x7f09b0009250 comp rx=0 tx=0).stop 2024-05-12T03:50:28.317 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:27.666+0000 7f09c2771700 1 -- 172.21.15.32:0/3355095735 shutdown_connections 2024-05-12T03:50:28.317 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:27.666+0000 7f09c2771700 1 --2- 172.21.15.32:0/3355095735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09bc102b90 0x7f09bc113550 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:50:28.317 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:27.666+0000 7f09c2771700 1 -- 172.21.15.32:0/3355095735 >> 172.21.15.32:0/3355095735 conn(0x7f09bc0fc380 msgr2=0x7f09bc0fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:50:28.317 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:27.667+0000 7f09c2771700 1 -- 172.21.15.32:0/3355095735 shutdown_connections 2024-05-12T03:50:28.317 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:27.667+0000 7f09c2771700 1 -- 172.21.15.32:0/3355095735 wait complete. 2024-05-12T03:50:28.317 INFO:teuthology.orchestra.run.smithi032.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-12T03:50:28.317 INFO:teuthology.orchestra.run.smithi032.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-12T03:50:28.317 INFO:teuthology.orchestra.run.smithi032.stdout:Creating mgr... 2024-05-12T03:50:28.318 INFO:teuthology.orchestra.run.smithi032.stdout:Verifying port 9283 ... 2024-05-12T03:50:28.451 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-a3e178f8-1012-11ef-bc98-c7b262605968@mgr.smithi032.ukatkq 2024-05-12T03:50:28.451 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Failed to reset failed state of unit ceph-a3e178f8-1012-11ef-bc98-c7b262605968@mgr.smithi032.ukatkq.service: Unit ceph-a3e178f8-1012-11ef-bc98-c7b262605968@mgr.smithi032.ukatkq.service not loaded. 2024-05-12T03:50:28.546 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-a3e178f8-1012-11ef-bc98-c7b262605968.target.wants/ceph-a3e178f8-1012-11ef-bc98-c7b262605968@mgr.smithi032.ukatkq.service → /etc/systemd/system/ceph-a3e178f8-1012-11ef-bc98-c7b262605968@.service. 2024-05-12T03:50:28.907 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:28 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/3355095735' entity='client.admin' 2024-05-12T03:50:29.147 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-12T03:50:29.148 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout disabled 2024-05-12T03:50:29.156 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-12T03:50:29.157 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout inactive 2024-05-12T03:50:29.157 INFO:teuthology.orchestra.run.smithi032.stdout:firewalld.service is not enabled 2024-05-12T03:50:29.157 INFO:teuthology.orchestra.run.smithi032.stdout:Not possible to enable service . firewalld.service is not available 2024-05-12T03:50:29.165 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-12T03:50:29.166 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout disabled 2024-05-12T03:50:29.175 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-12T03:50:29.175 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout inactive 2024-05-12T03:50:29.175 INFO:teuthology.orchestra.run.smithi032.stdout:firewalld.service is not enabled 2024-05-12T03:50:29.175 INFO:teuthology.orchestra.run.smithi032.stdout:Not possible to open ports <[9283]>. firewalld.service is not available 2024-05-12T03:50:29.175 INFO:teuthology.orchestra.run.smithi032.stdout:Waiting for mgr to start... 2024-05-12T03:50:29.175 INFO:teuthology.orchestra.run.smithi032.stdout:Waiting for mgr... 2024-05-12T03:50:30.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:29 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/2586096235' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-12T03:50:30.486 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2024-05-12T03:50:30.486 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout { 2024-05-12T03:50:30.487 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "fsid": "a3e178f8-1012-11ef-bc98-c7b262605968", 2024-05-12T03:50:30.487 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "health": { 2024-05-12T03:50:30.487 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-12T03:50:30.487 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-12T03:50:30.487 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-12T03:50:30.487 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-05-12T03:50:30.487 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-12T03:50:30.487 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-12T03:50:30.487 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 0 2024-05-12T03:50:30.487 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2024-05-12T03:50:30.488 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-12T03:50:30.488 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "smithi032" 2024-05-12T03:50:30.488 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2024-05-12T03:50:30.488 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-05-12T03:50:30.488 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-12T03:50:30.488 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-12T03:50:30.489 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-05-12T03:50:30.489 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-12T03:50:30.489 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-05-12T03:50:30.489 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-12T03:50:30.489 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-12T03:50:30.489 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-12T03:50:30.489 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-12T03:50:30.489 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-12T03:50:30.498 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-12T03:50:30.498 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-12T03:50:30.498 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-12T03:50:30.498 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-05-12T03:50:30.498 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-12T03:50:30.498 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-12T03:50:30.499 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-12T03:50:30.499 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-12T03:50:30.499 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-12T03:50:30.499 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-12T03:50:30.499 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-12T03:50:30.499 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-12T03:50:30.499 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-12T03:50:30.499 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-05-12T03:50:30.499 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-12T03:50:30.499 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-12T03:50:30.499 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-12T03:50:30.500 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-12T03:50:30.500 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-05-12T03:50:30.500 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-12T03:50:30.500 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-12T03:50:30.500 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-12T03:50:30.500 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-12T03:50:30.500 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-12T03:50:30.500 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-12T03:50:30.500 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "restful" 2024-05-12T03:50:30.500 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2024-05-12T03:50:30.500 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-12T03:50:30.500 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-05-12T03:50:30.501 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-12T03:50:30.501 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-12T03:50:30.501 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "modified": "2024-05-12T03:50:18.356618+0000", 2024-05-12T03:50:30.501 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-12T03:50:30.501 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-05-12T03:50:30.501 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-12T03:50:30.501 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout } 2024-05-12T03:50:30.501 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:29.760+0000 7f60ee11f700 1 Processor -- start 2024-05-12T03:50:30.501 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:29.760+0000 7f60ee11f700 1 -- start start 2024-05-12T03:50:30.501 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:29.760+0000 7f60ee11f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60e8076bd0 0x7f60e8107640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:50:30.501 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:29.760+0000 7f60ee11f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e8103030 con 0x7f60e8076bd0 2024-05-12T03:50:30.501 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:29.760+0000 7f60e77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60e8076bd0 0x7f60e8107640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:50:30.501 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:29.760+0000 7f60e77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60e8076bd0 0x7f60e8107640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47688/0 (socket says 172.21.15.32:47688) 2024-05-12T03:50:30.501 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:29.760+0000 7f60e77fe700 1 -- 172.21.15.32:0/2601188829 learned_addr learned my addr 172.21.15.32:0/2601188829 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:50:30.501 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:29.761+0000 7f60e77fe700 1 -- 172.21.15.32:0/2601188829 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60e8107b80 con 0x7f60e8076bd0 2024-05-12T03:50:30.502 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:29.761+0000 7f60e77fe700 1 --2- 172.21.15.32:0/2601188829 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60e8076bd0 0x7f60e8107640 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f60d80098e0 tx=0x7f60d8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9b39370525e9fd1 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:50:30.502 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:29.761+0000 7f60e67fc700 1 -- 172.21.15.32:0/2601188829 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f60d800b860 con 0x7f60e8076bd0 2024-05-12T03:50:30.502 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:29.761+0000 7f60e67fc700 1 -- 172.21.15.32:0/2601188829 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f60d800be30 con 0x7f60e8076bd0 2024-05-12T03:50:30.502 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:29.761+0000 7f60e67fc700 1 -- 172.21.15.32:0/2601188829 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f60d8010380 con 0x7f60e8076bd0 2024-05-12T03:50:30.502 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:29.762+0000 7f60ee11f700 1 -- 172.21.15.32:0/2601188829 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60e8076bd0 msgr2=0x7f60e8107640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:50:30.502 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:29.762+0000 7f60ee11f700 1 --2- 172.21.15.32:0/2601188829 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60e8076bd0 0x7f60e8107640 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f60d80098e0 tx=0x7f60d8009250 comp rx=0 tx=0).stop 2024-05-12T03:50:30.502 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:29.762+0000 7f60ee11f700 1 -- 172.21.15.32:0/2601188829 shutdown_connections 2024-05-12T03:50:30.502 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:29.762+0000 7f60ee11f700 1 --2- 172.21.15.32:0/2601188829 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60e8076bd0 0x7f60e8107640 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:50:30.502 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:29.762+0000 7f60ee11f700 1 -- 172.21.15.32:0/2601188829 >> 172.21.15.32:0/2601188829 conn(0x7f60e80fcc50 msgr2=0x7f60e80ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:50:30.502 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:29.762+0000 7f60ee11f700 1 -- 172.21.15.32:0/2601188829 shutdown_connections 2024-05-12T03:50:30.502 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:29.762+0000 7f60ee11f700 1 -- 172.21.15.32:0/2601188829 wait complete. 2024-05-12T03:50:30.502 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:29.762+0000 7f60ee11f700 1 Processor -- start 2024-05-12T03:50:30.502 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:29.762+0000 7f60ee11f700 1 -- start start 2024-05-12T03:50:30.502 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:29.763+0000 7f60ee11f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60e8076bd0 0x7f60e8113570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:50:30.502 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:29.763+0000 7f60ee11f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e8113ab0 con 0x7f60e8076bd0 2024-05-12T03:50:30.503 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:29.763+0000 7f60e77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60e8076bd0 0x7f60e8113570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:50:30.503 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:29.763+0000 7f60e77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60e8076bd0 0x7f60e8113570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47694/0 (socket says 172.21.15.32:47694) 2024-05-12T03:50:30.503 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:29.763+0000 7f60e77fe700 1 -- 172.21.15.32:0/2586096235 learned_addr learned my addr 172.21.15.32:0/2586096235 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:50:30.503 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:29.763+0000 7f60e77fe700 1 -- 172.21.15.32:0/2586096235 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60d80094f0 con 0x7f60e8076bd0 2024-05-12T03:50:30.503 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:29.763+0000 7f60e77fe700 1 --2- 172.21.15.32:0/2586096235 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60e8076bd0 0x7f60e8113570 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f60d8006b00 tx=0x7f60d8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:50:30.503 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:29.763+0000 7f60cffff700 1 -- 172.21.15.32:0/2586096235 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f60d8010a80 con 0x7f60e8076bd0 2024-05-12T03:50:30.503 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:29.763+0000 7f60cffff700 1 -- 172.21.15.32:0/2586096235 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f60d801a430 con 0x7f60e8076bd0 2024-05-12T03:50:30.503 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:29.763+0000 7f60ee11f700 1 -- 172.21.15.32:0/2586096235 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60e8113cb0 con 0x7f60e8076bd0 2024-05-12T03:50:30.503 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:29.763+0000 7f60cffff700 1 -- 172.21.15.32:0/2586096235 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f60d80245b0 con 0x7f60e8076bd0 2024-05-12T03:50:30.503 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:29.763+0000 7f60ee11f700 1 -- 172.21.15.32:0/2586096235 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60e8114820 con 0x7f60e8076bd0 2024-05-12T03:50:30.504 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:29.764+0000 7f60cffff700 1 -- 172.21.15.32:0/2586096235 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f60d80247e0 con 0x7f60e8076bd0 2024-05-12T03:50:30.504 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:29.764+0000 7f60cffff700 1 -- 172.21.15.32:0/2586096235 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f60d802e820 con 0x7f60e8076bd0 2024-05-12T03:50:30.504 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:29.764+0000 7f60ee11f700 1 -- 172.21.15.32:0/2586096235 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60e805f5c0 con 0x7f60e8076bd0 2024-05-12T03:50:30.504 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:29.766+0000 7f60cffff700 1 -- 172.21.15.32:0/2586096235 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7f60d8010be0 con 0x7f60e8076bd0 2024-05-12T03:50:30.504 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:29.839+0000 7f60ee11f700 1 -- 172.21.15.32:0/2586096235 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f60e81087e0 con 0x7f60e8076bd0 2024-05-12T03:50:30.504 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:29.839+0000 7f60cffff700 1 -- 172.21.15.32:0/2586096235 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7f60d8010d80 con 0x7f60e8076bd0 2024-05-12T03:50:30.504 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:29.840+0000 7f60ee11f700 1 -- 172.21.15.32:0/2586096235 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60e8076bd0 msgr2=0x7f60e8113570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:50:30.504 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:29.840+0000 7f60ee11f700 1 --2- 172.21.15.32:0/2586096235 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60e8076bd0 0x7f60e8113570 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f60d8006b00 tx=0x7f60d8009250 comp rx=0 tx=0).stop 2024-05-12T03:50:30.505 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:29.840+0000 7f60ee11f700 1 -- 172.21.15.32:0/2586096235 shutdown_connections 2024-05-12T03:50:30.505 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:29.840+0000 7f60ee11f700 1 --2- 172.21.15.32:0/2586096235 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60e8076bd0 0x7f60e8113570 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:50:30.505 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:29.840+0000 7f60ee11f700 1 -- 172.21.15.32:0/2586096235 >> 172.21.15.32:0/2586096235 conn(0x7f60e80fcc50 msgr2=0x7f60e80fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:50:30.505 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:29.840+0000 7f60ee11f700 1 -- 172.21.15.32:0/2586096235 shutdown_connections 2024-05-12T03:50:30.505 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:29.840+0000 7f60ee11f700 1 -- 172.21.15.32:0/2586096235 wait complete. 2024-05-12T03:50:30.505 INFO:teuthology.orchestra.run.smithi032.stdout:mgr not available, waiting (1/15)... 2024-05-12T03:50:35.665 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:35 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/4184221780' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-12T03:50:36.789 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:36 smithi032 ceph-mon[40008]: Activating manager daemon smithi032.ukatkq 2024-05-12T03:50:36.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:36 smithi032 ceph-mon[40008]: mgrmap e2: smithi032.ukatkq(active, starting, since 0.00303453s) 2024-05-12T03:50:36.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:36 smithi032 ceph-mon[40008]: from='mgr.14100 172.21.15.32:0/2500990795' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-12T03:50:36.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:36 smithi032 ceph-mon[40008]: from='mgr.14100 172.21.15.32:0/2500990795' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-12T03:50:36.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:36 smithi032 ceph-mon[40008]: from='mgr.14100 172.21.15.32:0/2500990795' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-12T03:50:36.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:36 smithi032 ceph-mon[40008]: from='mgr.14100 172.21.15.32:0/2500990795' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mon metadata", "id": "smithi032"}]: dispatch 2024-05-12T03:50:36.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:36 smithi032 ceph-mon[40008]: from='mgr.14100 172.21.15.32:0/2500990795' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mgr metadata", "who": "smithi032.ukatkq", "id": "smithi032.ukatkq"}]: dispatch 2024-05-12T03:50:36.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:36 smithi032 ceph-mon[40008]: Manager daemon smithi032.ukatkq is now available 2024-05-12T03:50:36.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:36 smithi032 ceph-mon[40008]: from='mgr.14100 172.21.15.32:0/2500990795' entity='mgr.smithi032.ukatkq' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi032.ukatkq/mirror_snapshot_schedule"}]: dispatch 2024-05-12T03:50:36.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:36 smithi032 ceph-mon[40008]: from='mgr.14100 172.21.15.32:0/2500990795' entity='mgr.smithi032.ukatkq' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi032.ukatkq/trash_purge_schedule"}]: dispatch 2024-05-12T03:50:36.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:36 smithi032 ceph-mon[40008]: from='mgr.14100 172.21.15.32:0/2500990795' entity='mgr.smithi032.ukatkq' 2024-05-12T03:50:36.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:36 smithi032 ceph-mon[40008]: from='mgr.14100 172.21.15.32:0/2500990795' entity='mgr.smithi032.ukatkq' 2024-05-12T03:50:36.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:36 smithi032 ceph-mon[40008]: from='mgr.14100 172.21.15.32:0/2500990795' entity='mgr.smithi032.ukatkq' 2024-05-12T03:50:36.950 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2024-05-12T03:50:36.950 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout { 2024-05-12T03:50:36.950 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "fsid": "a3e178f8-1012-11ef-bc98-c7b262605968", 2024-05-12T03:50:36.950 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "health": { 2024-05-12T03:50:36.950 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-12T03:50:36.950 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-12T03:50:36.950 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-12T03:50:36.950 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-05-12T03:50:36.950 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-12T03:50:36.951 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-12T03:50:36.951 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 0 2024-05-12T03:50:36.951 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2024-05-12T03:50:36.951 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-12T03:50:36.951 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "smithi032" 2024-05-12T03:50:36.951 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2024-05-12T03:50:36.951 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-05-12T03:50:36.951 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-12T03:50:36.951 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-12T03:50:36.951 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-05-12T03:50:36.951 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-12T03:50:36.951 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-05-12T03:50:36.951 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-12T03:50:36.951 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-12T03:50:36.951 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-12T03:50:36.961 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-12T03:50:36.961 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-12T03:50:36.962 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-12T03:50:36.962 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-12T03:50:36.962 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-12T03:50:36.962 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-05-12T03:50:36.962 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-12T03:50:36.962 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-12T03:50:36.963 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-12T03:50:36.963 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-12T03:50:36.963 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-12T03:50:36.963 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-12T03:50:36.963 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-12T03:50:36.963 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-12T03:50:36.963 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-12T03:50:36.964 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-05-12T03:50:36.964 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-12T03:50:36.964 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-12T03:50:36.964 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-12T03:50:36.964 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-12T03:50:36.964 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-05-12T03:50:36.964 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-12T03:50:36.965 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-12T03:50:36.965 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-12T03:50:36.965 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-12T03:50:36.965 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-12T03:50:36.965 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-12T03:50:36.965 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "restful" 2024-05-12T03:50:36.965 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2024-05-12T03:50:36.965 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-12T03:50:36.966 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-05-12T03:50:36.966 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-12T03:50:36.966 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-12T03:50:36.966 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "modified": "2024-05-12T03:50:18.356618+0000", 2024-05-12T03:50:36.966 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-12T03:50:36.966 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-05-12T03:50:36.966 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-12T03:50:36.966 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout } 2024-05-12T03:50:36.967 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:35.403+0000 7f5d00070700 1 Processor -- start 2024-05-12T03:50:36.967 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:35.403+0000 7f5d00070700 1 -- start start 2024-05-12T03:50:36.967 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:35.403+0000 7f5d00070700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cf8076c70 0x7f5cf81051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:50:36.967 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:35.403+0000 7f5d00070700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf81056e0 con 0x7f5cf8076c70 2024-05-12T03:50:36.967 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:35.403+0000 7f5cfde0c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cf8076c70 0x7f5cf81051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:50:36.967 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:35.403+0000 7f5cfde0c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cf8076c70 0x7f5cf81051a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36124/0 (socket says 172.21.15.32:36124) 2024-05-12T03:50:36.968 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:35.403+0000 7f5cfde0c700 1 -- 172.21.15.32:0/888863045 learned_addr learned my addr 172.21.15.32:0/888863045 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:50:36.968 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:35.404+0000 7f5cfde0c700 1 -- 172.21.15.32:0/888863045 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf8105820 con 0x7f5cf8076c70 2024-05-12T03:50:36.968 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:35.404+0000 7f5cfde0c700 1 --2- 172.21.15.32:0/888863045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cf8076c70 0x7f5cf81051a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5cf40098e0 tx=0x7f5cf4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=544a8c8703771115 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:50:36.968 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:35.404+0000 7f5cfce0a700 1 -- 172.21.15.32:0/888863045 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5cf400b860 con 0x7f5cf8076c70 2024-05-12T03:50:36.968 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:35.404+0000 7f5cfce0a700 1 -- 172.21.15.32:0/888863045 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f5cf400be30 con 0x7f5cf8076c70 2024-05-12T03:50:36.968 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:35.404+0000 7f5cfce0a700 1 -- 172.21.15.32:0/888863045 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5cf4010380 con 0x7f5cf8076c70 2024-05-12T03:50:36.968 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:35.405+0000 7f5d00070700 1 -- 172.21.15.32:0/888863045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cf8076c70 msgr2=0x7f5cf81051a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:50:36.968 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:35.405+0000 7f5d00070700 1 --2- 172.21.15.32:0/888863045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cf8076c70 0x7f5cf81051a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5cf40098e0 tx=0x7f5cf4009250 comp rx=0 tx=0).stop 2024-05-12T03:50:36.969 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:35.405+0000 7f5d00070700 1 -- 172.21.15.32:0/888863045 shutdown_connections 2024-05-12T03:50:36.969 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:35.405+0000 7f5d00070700 1 --2- 172.21.15.32:0/888863045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cf8076c70 0x7f5cf81051a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:50:36.969 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:35.405+0000 7f5d00070700 1 -- 172.21.15.32:0/888863045 >> 172.21.15.32:0/888863045 conn(0x7f5cf80fcc50 msgr2=0x7f5cf80ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:50:36.969 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:35.405+0000 7f5d00070700 1 -- 172.21.15.32:0/888863045 shutdown_connections 2024-05-12T03:50:36.969 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:35.405+0000 7f5d00070700 1 -- 172.21.15.32:0/888863045 wait complete. 2024-05-12T03:50:36.969 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:35.405+0000 7f5d00070700 1 Processor -- start 2024-05-12T03:50:36.970 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:35.405+0000 7f5d00070700 1 -- start start 2024-05-12T03:50:36.970 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:35.405+0000 7f5d00070700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cf8076c70 0x7f5cf8111350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:50:36.970 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:35.405+0000 7f5d00070700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf8111890 con 0x7f5cf8076c70 2024-05-12T03:50:36.970 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:35.406+0000 7f5cfde0c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cf8076c70 0x7f5cf8111350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:50:36.970 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:35.406+0000 7f5cfde0c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cf8076c70 0x7f5cf8111350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36138/0 (socket says 172.21.15.32:36138) 2024-05-12T03:50:36.970 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:35.406+0000 7f5cfde0c700 1 -- 172.21.15.32:0/4184221780 learned_addr learned my addr 172.21.15.32:0/4184221780 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:50:36.970 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:35.406+0000 7f5cfde0c700 1 -- 172.21.15.32:0/4184221780 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf40094f0 con 0x7f5cf8076c70 2024-05-12T03:50:36.970 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:35.406+0000 7f5cfde0c700 1 --2- 172.21.15.32:0/4184221780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cf8076c70 0x7f5cf8111350 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5cf4006d40 tx=0x7f5cf4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:50:36.970 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:35.406+0000 7f5cee7fc700 1 -- 172.21.15.32:0/4184221780 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5cf40109b0 con 0x7f5cf8076c70 2024-05-12T03:50:36.970 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:35.406+0000 7f5cee7fc700 1 -- 172.21.15.32:0/4184221780 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f5cf4024440 con 0x7f5cf8076c70 2024-05-12T03:50:36.971 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:35.406+0000 7f5d00070700 1 -- 172.21.15.32:0/4184221780 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cf8111a90 con 0x7f5cf8076c70 2024-05-12T03:50:36.971 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:35.406+0000 7f5cee7fc700 1 -- 172.21.15.32:0/4184221780 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5cf401a5f0 con 0x7f5cf8076c70 2024-05-12T03:50:36.971 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:35.406+0000 7f5d00070700 1 -- 172.21.15.32:0/4184221780 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cf8112600 con 0x7f5cf8076c70 2024-05-12T03:50:36.971 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:35.407+0000 7f5cee7fc700 1 -- 172.21.15.32:0/4184221780 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f5cf4010b10 con 0x7f5cf8076c70 2024-05-12T03:50:36.971 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:35.407+0000 7f5cee7fc700 1 -- 172.21.15.32:0/4184221780 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f5cf402d3f0 con 0x7f5cf8076c70 2024-05-12T03:50:36.971 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:35.407+0000 7f5d00070700 1 -- 172.21.15.32:0/4184221780 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cf805f5c0 con 0x7f5cf8076c70 2024-05-12T03:50:36.971 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:35.409+0000 7f5cee7fc700 1 -- 172.21.15.32:0/4184221780 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7f5cf4006d70 con 0x7f5cf8076c70 2024-05-12T03:50:36.971 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:35.483+0000 7f5d00070700 1 -- 172.21.15.32:0/4184221780 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f5cf8106760 con 0x7f5cf8076c70 2024-05-12T03:50:36.971 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:35.483+0000 7f5cee7fc700 1 -- 172.21.15.32:0/4184221780 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7f5cf8106760 con 0x7f5cf8076c70 2024-05-12T03:50:36.971 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:35.484+0000 7f5d00070700 1 -- 172.21.15.32:0/4184221780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cf8076c70 msgr2=0x7f5cf8111350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:50:36.971 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:35.484+0000 7f5d00070700 1 --2- 172.21.15.32:0/4184221780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cf8076c70 0x7f5cf8111350 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5cf4006d40 tx=0x7f5cf4009250 comp rx=0 tx=0).stop 2024-05-12T03:50:36.971 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:35.484+0000 7f5d00070700 1 -- 172.21.15.32:0/4184221780 shutdown_connections 2024-05-12T03:50:36.972 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:35.484+0000 7f5d00070700 1 --2- 172.21.15.32:0/4184221780 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cf8076c70 0x7f5cf8111350 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:50:36.972 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:35.484+0000 7f5d00070700 1 -- 172.21.15.32:0/4184221780 >> 172.21.15.32:0/4184221780 conn(0x7f5cf80fcc50 msgr2=0x7f5cf80fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:50:36.972 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:35.484+0000 7f5d00070700 1 -- 172.21.15.32:0/4184221780 shutdown_connections 2024-05-12T03:50:36.972 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:35.484+0000 7f5d00070700 1 -- 172.21.15.32:0/4184221780 wait complete. 2024-05-12T03:50:36.972 INFO:teuthology.orchestra.run.smithi032.stdout:mgr not available, waiting (2/15)... 2024-05-12T03:50:38.039 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:37 smithi032 ceph-mon[40008]: mgrmap e3: smithi032.ukatkq(active, since 1.00711s) 2024-05-12T03:50:40.016 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:39 smithi032 ceph-mon[40008]: mgrmap e4: smithi032.ukatkq(active, since 3s) 2024-05-12T03:50:40.016 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:39 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/2127278389' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-12T03:50:40.286 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2024-05-12T03:50:40.302 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout { 2024-05-12T03:50:40.302 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "fsid": "a3e178f8-1012-11ef-bc98-c7b262605968", 2024-05-12T03:50:40.302 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "health": { 2024-05-12T03:50:40.302 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-12T03:50:40.302 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-12T03:50:40.302 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-12T03:50:40.303 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-05-12T03:50:40.303 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-12T03:50:40.303 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-12T03:50:40.303 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 0 2024-05-12T03:50:40.303 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2024-05-12T03:50:40.303 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-12T03:50:40.303 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "smithi032" 2024-05-12T03:50:40.303 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2024-05-12T03:50:40.303 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-05-12T03:50:40.303 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-12T03:50:40.303 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-12T03:50:40.303 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-05-12T03:50:40.303 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-12T03:50:40.303 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-05-12T03:50:40.303 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-12T03:50:40.304 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-12T03:50:40.304 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-12T03:50:40.304 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-12T03:50:40.304 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-12T03:50:40.304 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-12T03:50:40.304 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-12T03:50:40.304 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-12T03:50:40.304 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-05-12T03:50:40.304 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-12T03:50:40.304 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-12T03:50:40.304 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-12T03:50:40.304 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-12T03:50:40.304 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-12T03:50:40.304 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-12T03:50:40.304 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-12T03:50:40.304 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-12T03:50:40.305 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-12T03:50:40.305 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-05-12T03:50:40.305 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-12T03:50:40.305 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-12T03:50:40.305 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-12T03:50:40.305 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-12T03:50:40.305 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-05-12T03:50:40.305 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-12T03:50:40.305 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-12T03:50:40.305 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-12T03:50:40.305 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-12T03:50:40.305 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-12T03:50:40.306 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-12T03:50:40.306 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "restful" 2024-05-12T03:50:40.306 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2024-05-12T03:50:40.306 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-12T03:50:40.306 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-05-12T03:50:40.306 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-12T03:50:40.306 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-12T03:50:40.306 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "modified": "2024-05-12T03:50:18.356618+0000", 2024-05-12T03:50:40.306 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-12T03:50:40.306 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-05-12T03:50:40.306 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-12T03:50:40.306 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout } 2024-05-12T03:50:40.306 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:39.445+0000 7f9520ad7700 1 Processor -- start 2024-05-12T03:50:40.306 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:39.445+0000 7f9520ad7700 1 -- start start 2024-05-12T03:50:40.307 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:39.445+0000 7f9520ad7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f951c102610 0x7f951c102a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:50:40.307 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:39.445+0000 7f9520ad7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f951c102f60 con 0x7f951c102610 2024-05-12T03:50:40.307 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:39.445+0000 7f951a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f951c102610 0x7f951c102a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:50:40.307 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:39.445+0000 7f951a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f951c102610 0x7f951c102a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36232/0 (socket says 172.21.15.32:36232) 2024-05-12T03:50:40.307 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:39.445+0000 7f951a59c700 1 -- 172.21.15.32:0/270643419 learned_addr learned my addr 172.21.15.32:0/270643419 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:50:40.307 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:39.446+0000 7f951a59c700 1 -- 172.21.15.32:0/270643419 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f951c1030a0 con 0x7f951c102610 2024-05-12T03:50:40.307 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:39.446+0000 7f951a59c700 1 --2- 172.21.15.32:0/270643419 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f951c102610 0x7f951c102a20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f950c0098e0 tx=0x7f950c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d2ea46c4d9f6c73 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:50:40.307 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:39.446+0000 7f951959a700 1 -- 172.21.15.32:0/270643419 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f950c00b860 con 0x7f951c102610 2024-05-12T03:50:40.307 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:39.446+0000 7f951959a700 1 -- 172.21.15.32:0/270643419 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f950c00be30 con 0x7f951c102610 2024-05-12T03:50:40.307 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:39.446+0000 7f951959a700 1 -- 172.21.15.32:0/270643419 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f950c010380 con 0x7f951c102610 2024-05-12T03:50:40.307 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:39.446+0000 7f9520ad7700 1 -- 172.21.15.32:0/270643419 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f951c102610 msgr2=0x7f951c102a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:50:40.307 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:39.446+0000 7f9520ad7700 1 --2- 172.21.15.32:0/270643419 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f951c102610 0x7f951c102a20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f950c0098e0 tx=0x7f950c009250 comp rx=0 tx=0).stop 2024-05-12T03:50:40.307 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:39.447+0000 7f9520ad7700 1 -- 172.21.15.32:0/270643419 shutdown_connections 2024-05-12T03:50:40.307 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:39.447+0000 7f9520ad7700 1 --2- 172.21.15.32:0/270643419 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f951c102610 0x7f951c102a20 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:50:40.307 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:39.447+0000 7f9520ad7700 1 -- 172.21.15.32:0/270643419 >> 172.21.15.32:0/270643419 conn(0x7f951c0fddd0 msgr2=0x7f951c1001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:50:40.308 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:39.447+0000 7f9520ad7700 1 -- 172.21.15.32:0/270643419 shutdown_connections 2024-05-12T03:50:40.308 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:39.447+0000 7f9520ad7700 1 -- 172.21.15.32:0/270643419 wait complete. 2024-05-12T03:50:40.308 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:39.447+0000 7f9520ad7700 1 Processor -- start 2024-05-12T03:50:40.308 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:39.447+0000 7f9520ad7700 1 -- start start 2024-05-12T03:50:40.308 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:39.447+0000 7f9520ad7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f951c102610 0x7f951c078c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:50:40.308 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:39.447+0000 7f9520ad7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f951c079170 con 0x7f951c102610 2024-05-12T03:50:40.308 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:39.447+0000 7f951a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f951c102610 0x7f951c078c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:50:40.308 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:39.447+0000 7f951a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f951c102610 0x7f951c078c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36246/0 (socket says 172.21.15.32:36246) 2024-05-12T03:50:40.308 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:39.447+0000 7f951a59c700 1 -- 172.21.15.32:0/2127278389 learned_addr learned my addr 172.21.15.32:0/2127278389 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:50:40.308 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:39.448+0000 7f951a59c700 1 -- 172.21.15.32:0/2127278389 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f950c0094f0 con 0x7f951c102610 2024-05-12T03:50:40.308 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:39.448+0000 7f951a59c700 1 --2- 172.21.15.32:0/2127278389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f951c102610 0x7f951c078c30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f950c000c00 tx=0x7f950c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:50:40.308 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:39.448+0000 7f950affd700 1 -- 172.21.15.32:0/2127278389 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f950c010a80 con 0x7f951c102610 2024-05-12T03:50:40.308 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:39.448+0000 7f950affd700 1 -- 172.21.15.32:0/2127278389 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f950c01a430 con 0x7f951c102610 2024-05-12T03:50:40.308 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:39.448+0000 7f9520ad7700 1 -- 172.21.15.32:0/2127278389 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f951c0771e0 con 0x7f951c102610 2024-05-12T03:50:40.309 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:39.448+0000 7f950affd700 1 -- 172.21.15.32:0/2127278389 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f950c0245b0 con 0x7f951c102610 2024-05-12T03:50:40.309 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:39.448+0000 7f9520ad7700 1 -- 172.21.15.32:0/2127278389 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f951c0776d0 con 0x7f951c102610 2024-05-12T03:50:40.309 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:39.449+0000 7f950affd700 1 -- 172.21.15.32:0/2127278389 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 43365+0+0 (secure 0 0 0) 0x7f950c0247a0 con 0x7f951c102610 2024-05-12T03:50:40.309 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:39.449+0000 7f9520ad7700 1 -- 172.21.15.32:0/2127278389 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f951c04bed0 con 0x7f951c102610 2024-05-12T03:50:40.309 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:39.449+0000 7f950affd700 1 --2- 172.21.15.32:0/2127278389 >> [v2:172.21.15.32:6800/1121526649,v1:172.21.15.32:6801/1121526649] conn(0x7f9504036790 0x7f9504038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:50:40.309 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:39.449+0000 7f950affd700 1 -- 172.21.15.32:0/2127278389 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f950c04ae40 con 0x7f951c102610 2024-05-12T03:50:40.309 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:39.449+0000 7f9519d9b700 1 --2- 172.21.15.32:0/2127278389 >> [v2:172.21.15.32:6800/1121526649,v1:172.21.15.32:6801/1121526649] conn(0x7f9504036790 0x7f9504038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:50:40.309 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:39.449+0000 7f9519d9b700 1 --2- 172.21.15.32:0/2127278389 >> [v2:172.21.15.32:6800/1121526649,v1:172.21.15.32:6801/1121526649] conn(0x7f9504036790 0x7f9504038c40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9510009990 tx=0x7f9510006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:50:40.309 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:39.454+0000 7f950affd700 1 -- 172.21.15.32:0/2127278389 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f950c023800 con 0x7f951c102610 2024-05-12T03:50:40.309 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:39.728+0000 7f9520ad7700 1 -- 172.21.15.32:0/2127278389 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f951c077e20 con 0x7f951c102610 2024-05-12T03:50:40.309 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:39.728+0000 7f950affd700 1 -- 172.21.15.32:0/2127278389 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7f950c023800 con 0x7f951c102610 2024-05-12T03:50:40.309 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:39.730+0000 7f9520ad7700 1 -- 172.21.15.32:0/2127278389 >> [v2:172.21.15.32:6800/1121526649,v1:172.21.15.32:6801/1121526649] conn(0x7f9504036790 msgr2=0x7f9504038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:50:40.309 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:39.730+0000 7f9520ad7700 1 --2- 172.21.15.32:0/2127278389 >> [v2:172.21.15.32:6800/1121526649,v1:172.21.15.32:6801/1121526649] conn(0x7f9504036790 0x7f9504038c40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9510009990 tx=0x7f9510006e40 comp rx=0 tx=0).stop 2024-05-12T03:50:40.309 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:39.730+0000 7f9520ad7700 1 -- 172.21.15.32:0/2127278389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f951c102610 msgr2=0x7f951c078c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:50:40.309 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:39.730+0000 7f9520ad7700 1 --2- 172.21.15.32:0/2127278389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f951c102610 0x7f951c078c30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f950c000c00 tx=0x7f950c009250 comp rx=0 tx=0).stop 2024-05-12T03:50:40.310 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:39.730+0000 7f9520ad7700 1 -- 172.21.15.32:0/2127278389 shutdown_connections 2024-05-12T03:50:40.310 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:39.730+0000 7f9520ad7700 1 --2- 172.21.15.32:0/2127278389 >> [v2:172.21.15.32:6800/1121526649,v1:172.21.15.32:6801/1121526649] conn(0x7f9504036790 0x7f9504038c40 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:50:40.310 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:39.730+0000 7f9520ad7700 1 --2- 172.21.15.32:0/2127278389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f951c102610 0x7f951c078c30 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:50:40.310 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:39.730+0000 7f9520ad7700 1 -- 172.21.15.32:0/2127278389 >> 172.21.15.32:0/2127278389 conn(0x7f951c0fddd0 msgr2=0x7f951c0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:50:40.310 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:39.730+0000 7f9520ad7700 1 -- 172.21.15.32:0/2127278389 shutdown_connections 2024-05-12T03:50:40.310 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:39.730+0000 7f9520ad7700 1 -- 172.21.15.32:0/2127278389 wait complete. 2024-05-12T03:50:40.310 INFO:teuthology.orchestra.run.smithi032.stdout:mgr is available 2024-05-12T03:50:41.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:41 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/2337043245' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-12T03:50:41.632 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2024-05-12T03:50:41.633 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout [global] 2024-05-12T03:50:41.633 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout fsid = a3e178f8-1012-11ef-bc98-c7b262605968 2024-05-12T03:50:41.633 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-12T03:50:41.633 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-12T03:50:41.633 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-12T03:50:41.633 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-12T03:50:41.633 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2024-05-12T03:50:41.633 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-12T03:50:41.633 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-12T03:50:41.633 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2024-05-12T03:50:41.633 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout [osd] 2024-05-12T03:50:41.633 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-12T03:50:41.634 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-12T03:50:41.634 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-12T03:50:41.634 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-12T03:50:41.634 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-12T03:50:41.634 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:40.793+0000 7fe1792b7700 1 Processor -- start 2024-05-12T03:50:41.634 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:40.793+0000 7fe1792b7700 1 -- start start 2024-05-12T03:50:41.634 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:40.793+0000 7fe1792b7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe174076bd0 0x7fe174105200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:50:41.634 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:40.793+0000 7fe1792b7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe174105740 con 0x7fe174076bd0 2024-05-12T03:50:41.634 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:40.794+0000 7fe172ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe174076bd0 0x7fe174105200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:50:41.634 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:40.794+0000 7fe172ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe174076bd0 0x7fe174105200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36256/0 (socket says 172.21.15.32:36256) 2024-05-12T03:50:41.634 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:40.794+0000 7fe172ffd700 1 -- 172.21.15.32:0/1470499576 learned_addr learned my addr 172.21.15.32:0/1470499576 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:50:41.634 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:40.794+0000 7fe172ffd700 1 -- 172.21.15.32:0/1470499576 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe174105880 con 0x7fe174076bd0 2024-05-12T03:50:41.634 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:40.794+0000 7fe172ffd700 1 --2- 172.21.15.32:0/1470499576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe174076bd0 0x7fe174105200 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe1680098e0 tx=0x7fe168009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5b05a09c30fec73 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:50:41.635 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:40.794+0000 7fe171ffb700 1 -- 172.21.15.32:0/1470499576 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe16800b860 con 0x7fe174076bd0 2024-05-12T03:50:41.635 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:40.794+0000 7fe171ffb700 1 -- 172.21.15.32:0/1470499576 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7fe16800be30 con 0x7fe174076bd0 2024-05-12T03:50:41.635 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:40.794+0000 7fe171ffb700 1 -- 172.21.15.32:0/1470499576 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe168010380 con 0x7fe174076bd0 2024-05-12T03:50:41.635 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:40.795+0000 7fe1792b7700 1 -- 172.21.15.32:0/1470499576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe174076bd0 msgr2=0x7fe174105200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:50:41.635 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:40.795+0000 7fe1792b7700 1 --2- 172.21.15.32:0/1470499576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe174076bd0 0x7fe174105200 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe1680098e0 tx=0x7fe168009250 comp rx=0 tx=0).stop 2024-05-12T03:50:41.635 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:40.795+0000 7fe1792b7700 1 -- 172.21.15.32:0/1470499576 shutdown_connections 2024-05-12T03:50:41.635 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:40.795+0000 7fe1792b7700 1 --2- 172.21.15.32:0/1470499576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe174076bd0 0x7fe174105200 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:50:41.635 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:40.795+0000 7fe1792b7700 1 -- 172.21.15.32:0/1470499576 >> 172.21.15.32:0/1470499576 conn(0x7fe1740fcc50 msgr2=0x7fe1740ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:50:41.635 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:40.795+0000 7fe1792b7700 1 -- 172.21.15.32:0/1470499576 shutdown_connections 2024-05-12T03:50:41.635 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:40.795+0000 7fe1792b7700 1 -- 172.21.15.32:0/1470499576 wait complete. 2024-05-12T03:50:41.635 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:40.796+0000 7fe1792b7700 1 Processor -- start 2024-05-12T03:50:41.635 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:40.796+0000 7fe1792b7700 1 -- start start 2024-05-12T03:50:41.635 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:40.796+0000 7fe1792b7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe174076bd0 0x7fe174111370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:50:41.635 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:40.796+0000 7fe1792b7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1741118b0 con 0x7fe174076bd0 2024-05-12T03:50:41.636 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:40.796+0000 7fe172ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe174076bd0 0x7fe174111370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:50:41.636 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:40.796+0000 7fe172ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe174076bd0 0x7fe174111370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36262/0 (socket says 172.21.15.32:36262) 2024-05-12T03:50:41.636 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:40.796+0000 7fe172ffd700 1 -- 172.21.15.32:0/2337043245 learned_addr learned my addr 172.21.15.32:0/2337043245 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:50:41.636 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:40.796+0000 7fe172ffd700 1 -- 172.21.15.32:0/2337043245 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1680094f0 con 0x7fe174076bd0 2024-05-12T03:50:41.636 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:40.796+0000 7fe172ffd700 1 --2- 172.21.15.32:0/2337043245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe174076bd0 0x7fe174111370 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe168006d40 tx=0x7fe168009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:50:41.636 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:40.796+0000 7fe15b7fe700 1 -- 172.21.15.32:0/2337043245 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe1680109b0 con 0x7fe174076bd0 2024-05-12T03:50:41.636 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:40.796+0000 7fe15b7fe700 1 -- 172.21.15.32:0/2337043245 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7fe168024440 con 0x7fe174076bd0 2024-05-12T03:50:41.636 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:40.796+0000 7fe1792b7700 1 -- 172.21.15.32:0/2337043245 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe174111ab0 con 0x7fe174076bd0 2024-05-12T03:50:41.636 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:40.797+0000 7fe15b7fe700 1 -- 172.21.15.32:0/2337043245 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe16801a5f0 con 0x7fe174076bd0 2024-05-12T03:50:41.636 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:40.797+0000 7fe1792b7700 1 -- 172.21.15.32:0/2337043245 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe174112620 con 0x7fe174076bd0 2024-05-12T03:50:41.636 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:40.797+0000 7fe15b7fe700 1 -- 172.21.15.32:0/2337043245 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 43365+0+0 (secure 0 0 0) 0x7fe16801a870 con 0x7fe174076bd0 2024-05-12T03:50:41.636 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:40.798+0000 7fe1792b7700 1 -- 172.21.15.32:0/2337043245 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe17405f5c0 con 0x7fe174076bd0 2024-05-12T03:50:41.636 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:40.798+0000 7fe15b7fe700 1 --2- 172.21.15.32:0/2337043245 >> [v2:172.21.15.32:6800/1121526649,v1:172.21.15.32:6801/1121526649] conn(0x7fe15c036740 0x7fe15c038bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:50:41.636 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:40.798+0000 7fe15b7fe700 1 -- 172.21.15.32:0/2337043245 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fe16804a700 con 0x7fe174076bd0 2024-05-12T03:50:41.637 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:40.798+0000 7fe1727fc700 1 --2- 172.21.15.32:0/2337043245 >> [v2:172.21.15.32:6800/1121526649,v1:172.21.15.32:6801/1121526649] conn(0x7fe15c036740 0x7fe15c038bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:50:41.637 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:40.798+0000 7fe1727fc700 1 --2- 172.21.15.32:0/2337043245 >> [v2:172.21.15.32:6800/1121526649,v1:172.21.15.32:6801/1121526649] conn(0x7fe15c036740 0x7fe15c038bf0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe164009990 tx=0x7fe164006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:50:41.637 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:40.802+0000 7fe15b7fe700 1 -- 172.21.15.32:0/2337043245 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fe1680189a0 con 0x7fe174076bd0 2024-05-12T03:50:41.637 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:40.998+0000 7fe1792b7700 1 -- 172.21.15.32:0/2337043245 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fe17404bed0 con 0x7fe174076bd0 2024-05-12T03:50:41.637 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:40.999+0000 7fe15b7fe700 1 -- 172.21.15.32:0/2337043245 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7fe168018350 con 0x7fe174076bd0 2024-05-12T03:50:41.637 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:41.001+0000 7fe1792b7700 1 -- 172.21.15.32:0/2337043245 >> [v2:172.21.15.32:6800/1121526649,v1:172.21.15.32:6801/1121526649] conn(0x7fe15c036740 msgr2=0x7fe15c038bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:50:41.637 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:41.001+0000 7fe1792b7700 1 --2- 172.21.15.32:0/2337043245 >> [v2:172.21.15.32:6800/1121526649,v1:172.21.15.32:6801/1121526649] conn(0x7fe15c036740 0x7fe15c038bf0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe164009990 tx=0x7fe164006e40 comp rx=0 tx=0).stop 2024-05-12T03:50:41.637 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:41.001+0000 7fe1792b7700 1 -- 172.21.15.32:0/2337043245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe174076bd0 msgr2=0x7fe174111370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:50:41.637 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:41.001+0000 7fe1792b7700 1 --2- 172.21.15.32:0/2337043245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe174076bd0 0x7fe174111370 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe168006d40 tx=0x7fe168009250 comp rx=0 tx=0).stop 2024-05-12T03:50:41.637 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:41.001+0000 7fe1792b7700 1 -- 172.21.15.32:0/2337043245 shutdown_connections 2024-05-12T03:50:41.637 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:41.001+0000 7fe1792b7700 1 --2- 172.21.15.32:0/2337043245 >> [v2:172.21.15.32:6800/1121526649,v1:172.21.15.32:6801/1121526649] conn(0x7fe15c036740 0x7fe15c038bf0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:50:41.637 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:41.001+0000 7fe1792b7700 1 --2- 172.21.15.32:0/2337043245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe174076bd0 0x7fe174111370 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:50:41.637 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:41.001+0000 7fe1792b7700 1 -- 172.21.15.32:0/2337043245 >> 172.21.15.32:0/2337043245 conn(0x7fe1740fcc50 msgr2=0x7fe1740fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:50:41.637 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:41.001+0000 7fe1792b7700 1 -- 172.21.15.32:0/2337043245 shutdown_connections 2024-05-12T03:50:41.637 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:41.001+0000 7fe1792b7700 1 -- 172.21.15.32:0/2337043245 wait complete. 2024-05-12T03:50:41.638 INFO:teuthology.orchestra.run.smithi032.stdout:Enabling cephadm module... 2024-05-12T03:50:42.941 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:42 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/2229275598' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-12T03:50:43.264 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:42.292+0000 7f8041856700 1 Processor -- start 2024-05-12T03:50:43.264 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:42.292+0000 7f8041856700 1 -- start start 2024-05-12T03:50:43.264 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:42.292+0000 7f8041856700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f803c102a00 0x7f803c102e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:50:43.264 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:42.292+0000 7f8041856700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f803c103350 con 0x7f803c102a00 2024-05-12T03:50:43.264 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:42.292+0000 7f803affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f803c102a00 0x7f803c102e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:50:43.264 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:42.292+0000 7f803affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f803c102a00 0x7f803c102e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54920/0 (socket says 172.21.15.32:54920) 2024-05-12T03:50:43.264 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:42.292+0000 7f803affd700 1 -- 172.21.15.32:0/3226491926 learned_addr learned my addr 172.21.15.32:0/3226491926 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:50:43.264 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:42.292+0000 7f803affd700 1 -- 172.21.15.32:0/3226491926 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f803c103490 con 0x7f803c102a00 2024-05-12T03:50:43.264 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:42.292+0000 7f803affd700 1 --2- 172.21.15.32:0/3226491926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f803c102a00 0x7f803c102e10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f802c0098e0 tx=0x7f802c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f57724e6ec5ea0d server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:50:43.265 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:42.292+0000 7f8039ffb700 1 -- 172.21.15.32:0/3226491926 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f802c00b860 con 0x7f803c102a00 2024-05-12T03:50:43.265 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:42.292+0000 7f8039ffb700 1 -- 172.21.15.32:0/3226491926 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f802c00be30 con 0x7f803c102a00 2024-05-12T03:50:43.265 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:42.292+0000 7f8039ffb700 1 -- 172.21.15.32:0/3226491926 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f802c010380 con 0x7f803c102a00 2024-05-12T03:50:43.265 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:42.292+0000 7f8041856700 1 -- 172.21.15.32:0/3226491926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f803c102a00 msgr2=0x7f803c102e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:50:43.265 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:42.292+0000 7f8041856700 1 --2- 172.21.15.32:0/3226491926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f803c102a00 0x7f803c102e10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f802c0098e0 tx=0x7f802c009250 comp rx=0 tx=0).stop 2024-05-12T03:50:43.265 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:42.293+0000 7f8041856700 1 -- 172.21.15.32:0/3226491926 shutdown_connections 2024-05-12T03:50:43.265 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:42.293+0000 7f8041856700 1 --2- 172.21.15.32:0/3226491926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f803c102a00 0x7f803c102e10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:50:43.265 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:42.293+0000 7f8041856700 1 -- 172.21.15.32:0/3226491926 >> 172.21.15.32:0/3226491926 conn(0x7f803c0fc340 msgr2=0x7f803c0fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:50:43.265 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:42.293+0000 7f8041856700 1 -- 172.21.15.32:0/3226491926 shutdown_connections 2024-05-12T03:50:43.265 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:42.293+0000 7f8041856700 1 -- 172.21.15.32:0/3226491926 wait complete. 2024-05-12T03:50:43.265 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:42.293+0000 7f8041856700 1 Processor -- start 2024-05-12T03:50:43.265 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:42.293+0000 7f8041856700 1 -- start start 2024-05-12T03:50:43.265 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:42.293+0000 7f8041856700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f803c102a00 0x7f803c1076b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:50:43.265 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:42.293+0000 7f8041856700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f803c107bf0 con 0x7f803c102a00 2024-05-12T03:50:43.265 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:42.294+0000 7f803affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f803c102a00 0x7f803c1076b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:50:43.266 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:42.294+0000 7f803affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f803c102a00 0x7f803c1076b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54930/0 (socket says 172.21.15.32:54930) 2024-05-12T03:50:43.266 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:42.294+0000 7f803affd700 1 -- 172.21.15.32:0/2229275598 learned_addr learned my addr 172.21.15.32:0/2229275598 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:50:43.266 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:42.294+0000 7f803affd700 1 -- 172.21.15.32:0/2229275598 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f802c0094f0 con 0x7f803c102a00 2024-05-12T03:50:43.266 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:42.294+0000 7f803affd700 1 --2- 172.21.15.32:0/2229275598 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f803c102a00 0x7f803c1076b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f802c000c00 tx=0x7f802c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:50:43.266 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:42.294+0000 7f80237fe700 1 -- 172.21.15.32:0/2229275598 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f802c010960 con 0x7f803c102a00 2024-05-12T03:50:43.266 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:42.294+0000 7f80237fe700 1 -- 172.21.15.32:0/2229275598 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f802c024440 con 0x7f803c102a00 2024-05-12T03:50:43.266 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:42.294+0000 7f80237fe700 1 -- 172.21.15.32:0/2229275598 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f802c01a5f0 con 0x7f803c102a00 2024-05-12T03:50:43.266 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:42.294+0000 7f8041856700 1 -- 172.21.15.32:0/2229275598 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f803c105c60 con 0x7f803c102a00 2024-05-12T03:50:43.266 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:42.294+0000 7f8041856700 1 -- 172.21.15.32:0/2229275598 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f803c106150 con 0x7f803c102a00 2024-05-12T03:50:43.266 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:42.295+0000 7f80237fe700 1 -- 172.21.15.32:0/2229275598 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 43365+0+0 (secure 0 0 0) 0x7f802c01a820 con 0x7f803c102a00 2024-05-12T03:50:43.266 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:42.295+0000 7f80237fe700 1 --2- 172.21.15.32:0/2229275598 >> [v2:172.21.15.32:6800/1121526649,v1:172.21.15.32:6801/1121526649] conn(0x7f8024036790 0x7f8024038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:50:43.266 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:42.295+0000 7f80237fe700 1 -- 172.21.15.32:0/2229275598 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f802c04a750 con 0x7f803c102a00 2024-05-12T03:50:43.266 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:42.295+0000 7f8041856700 1 -- 172.21.15.32:0/2229275598 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f803c04bed0 con 0x7f803c102a00 2024-05-12T03:50:43.266 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:42.295+0000 7f803a7fc700 1 --2- 172.21.15.32:0/2229275598 >> [v2:172.21.15.32:6800/1121526649,v1:172.21.15.32:6801/1121526649] conn(0x7f8024036790 0x7f8024038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:50:43.266 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:42.296+0000 7f803a7fc700 1 --2- 172.21.15.32:0/2229275598 >> [v2:172.21.15.32:6800/1121526649,v1:172.21.15.32:6801/1121526649] conn(0x7f8024036790 0x7f8024038c40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8030009990 tx=0x7f8030006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:50:43.267 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:42.300+0000 7f80237fe700 1 -- 172.21.15.32:0/2229275598 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f802c022340 con 0x7f803c102a00 2024-05-12T03:50:43.267 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:42.548+0000 7f8041856700 1 -- 172.21.15.32:0/2229275598 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f803c102230 con 0x7f803c102a00 2024-05-12T03:50:43.267 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:42.601+0000 7f80237fe700 1 -- 172.21.15.32:0/2229275598 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 5) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f802c024620 con 0x7f803c102a00 2024-05-12T03:50:43.267 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:42.602+0000 7f80237fe700 1 -- 172.21.15.32:0/2229275598 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f802c049d10 con 0x7f803c102a00 2024-05-12T03:50:43.267 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:42.605+0000 7f8041856700 1 -- 172.21.15.32:0/2229275598 >> [v2:172.21.15.32:6800/1121526649,v1:172.21.15.32:6801/1121526649] conn(0x7f8024036790 msgr2=0x7f8024038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:50:43.267 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:42.605+0000 7f8041856700 1 --2- 172.21.15.32:0/2229275598 >> [v2:172.21.15.32:6800/1121526649,v1:172.21.15.32:6801/1121526649] conn(0x7f8024036790 0x7f8024038c40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8030009990 tx=0x7f8030006e40 comp rx=0 tx=0).stop 2024-05-12T03:50:43.267 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:42.605+0000 7f8041856700 1 -- 172.21.15.32:0/2229275598 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f803c102a00 msgr2=0x7f803c1076b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:50:43.267 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:42.605+0000 7f8041856700 1 --2- 172.21.15.32:0/2229275598 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f803c102a00 0x7f803c1076b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f802c000c00 tx=0x7f802c009250 comp rx=0 tx=0).stop 2024-05-12T03:50:43.267 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:42.605+0000 7f8041856700 1 -- 172.21.15.32:0/2229275598 shutdown_connections 2024-05-12T03:50:43.267 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:42.605+0000 7f8041856700 1 --2- 172.21.15.32:0/2229275598 >> [v2:172.21.15.32:6800/1121526649,v1:172.21.15.32:6801/1121526649] conn(0x7f8024036790 0x7f8024038c40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:50:43.267 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:42.605+0000 7f8041856700 1 --2- 172.21.15.32:0/2229275598 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f803c102a00 0x7f803c1076b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:50:43.267 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:42.606+0000 7f8041856700 1 -- 172.21.15.32:0/2229275598 >> 172.21.15.32:0/2229275598 conn(0x7f803c0fc340 msgr2=0x7f803c0fd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:50:43.267 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:42.606+0000 7f8041856700 1 -- 172.21.15.32:0/2229275598 shutdown_connections 2024-05-12T03:50:43.267 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:42.606+0000 7f8041856700 1 -- 172.21.15.32:0/2229275598 wait complete. 2024-05-12T03:50:43.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:43 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/2229275598' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-12T03:50:43.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:43 smithi032 ceph-mon[40008]: mgrmap e5: smithi032.ukatkq(active, since 6s) 2024-05-12T03:50:44.757 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout { 2024-05-12T03:50:44.757 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-12T03:50:44.757 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-12T03:50:44.757 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "active_name": "smithi032.ukatkq", 2024-05-12T03:50:44.757 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-12T03:50:44.757 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout } 2024-05-12T03:50:44.757 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:43.835+0000 7f5362f8c700 1 Processor -- start 2024-05-12T03:50:44.757 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:43.835+0000 7f5362f8c700 1 -- start start 2024-05-12T03:50:44.757 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:43.835+0000 7f5362f8c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f535c076c30 0x7f535c1051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:50:44.757 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:43.835+0000 7f5362f8c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f535c105720 con 0x7f535c076c30 2024-05-12T03:50:44.757 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:43.835+0000 7f5360d28700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f535c076c30 0x7f535c1051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:50:44.758 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:43.835+0000 7f5360d28700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f535c076c30 0x7f535c1051e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54964/0 (socket says 172.21.15.32:54964) 2024-05-12T03:50:44.758 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:43.835+0000 7f5360d28700 1 -- 172.21.15.32:0/3670413685 learned_addr learned my addr 172.21.15.32:0/3670413685 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:50:44.758 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:43.835+0000 7f5360d28700 1 -- 172.21.15.32:0/3670413685 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f535c105860 con 0x7f535c076c30 2024-05-12T03:50:44.758 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:43.836+0000 7f5360d28700 1 --2- 172.21.15.32:0/3670413685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f535c076c30 0x7f535c1051e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f53500098e0 tx=0x7f5350009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8857a7d244395019 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:50:44.758 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:43.836+0000 7f535b7fe700 1 -- 172.21.15.32:0/3670413685 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f535000b860 con 0x7f535c076c30 2024-05-12T03:50:44.758 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:43.836+0000 7f535b7fe700 1 -- 172.21.15.32:0/3670413685 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f535000be30 con 0x7f535c076c30 2024-05-12T03:50:44.758 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:43.836+0000 7f535b7fe700 1 -- 172.21.15.32:0/3670413685 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5350010380 con 0x7f535c076c30 2024-05-12T03:50:44.758 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:43.836+0000 7f5362f8c700 1 -- 172.21.15.32:0/3670413685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f535c076c30 msgr2=0x7f535c1051e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:50:44.758 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:43.836+0000 7f5362f8c700 1 --2- 172.21.15.32:0/3670413685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f535c076c30 0x7f535c1051e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f53500098e0 tx=0x7f5350009250 comp rx=0 tx=0).stop 2024-05-12T03:50:44.758 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:43.836+0000 7f5362f8c700 1 -- 172.21.15.32:0/3670413685 shutdown_connections 2024-05-12T03:50:44.758 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:43.836+0000 7f5362f8c700 1 --2- 172.21.15.32:0/3670413685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f535c076c30 0x7f535c1051e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:50:44.758 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:43.836+0000 7f5362f8c700 1 -- 172.21.15.32:0/3670413685 >> 172.21.15.32:0/3670413685 conn(0x7f535c0fcc50 msgr2=0x7f535c0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:50:44.758 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:43.836+0000 7f5362f8c700 1 -- 172.21.15.32:0/3670413685 shutdown_connections 2024-05-12T03:50:44.759 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:43.837+0000 7f5362f8c700 1 -- 172.21.15.32:0/3670413685 wait complete. 2024-05-12T03:50:44.759 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:43.837+0000 7f5362f8c700 1 Processor -- start 2024-05-12T03:50:44.759 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:43.837+0000 7f5362f8c700 1 -- start start 2024-05-12T03:50:44.759 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:43.837+0000 7f5362f8c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f535c076c30 0x7f535c111370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:50:44.759 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:43.837+0000 7f5362f8c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f535c1118b0 con 0x7f535c076c30 2024-05-12T03:50:44.759 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:43.837+0000 7f5360d28700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f535c076c30 0x7f535c111370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:50:44.759 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:43.838+0000 7f5360d28700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f535c076c30 0x7f535c111370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54966/0 (socket says 172.21.15.32:54966) 2024-05-12T03:50:44.759 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:43.838+0000 7f5360d28700 1 -- 172.21.15.32:0/270070416 learned_addr learned my addr 172.21.15.32:0/270070416 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:50:44.759 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:43.838+0000 7f5360d28700 1 -- 172.21.15.32:0/270070416 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53500094f0 con 0x7f535c076c30 2024-05-12T03:50:44.759 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:43.838+0000 7f5360d28700 1 --2- 172.21.15.32:0/270070416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f535c076c30 0x7f535c111370 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5350006d40 tx=0x7f5350009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:50:44.759 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:43.838+0000 7f53597fa700 1 -- 172.21.15.32:0/270070416 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f53500109b0 con 0x7f535c076c30 2024-05-12T03:50:44.759 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:43.838+0000 7f53597fa700 1 -- 172.21.15.32:0/270070416 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f5350024440 con 0x7f535c076c30 2024-05-12T03:50:44.759 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:43.838+0000 7f5362f8c700 1 -- 172.21.15.32:0/270070416 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f535c111ab0 con 0x7f535c076c30 2024-05-12T03:50:44.759 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:43.838+0000 7f53597fa700 1 -- 172.21.15.32:0/270070416 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f535001a5f0 con 0x7f535c076c30 2024-05-12T03:50:44.759 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:43.838+0000 7f5362f8c700 1 -- 172.21.15.32:0/270070416 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f535c112620 con 0x7f535c076c30 2024-05-12T03:50:44.760 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:43.839+0000 7f53597fa700 1 -- 172.21.15.32:0/270070416 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f535001a870 con 0x7f535c076c30 2024-05-12T03:50:44.760 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:43.839+0000 7f53597fa700 1 --2- 172.21.15.32:0/270070416 >> [v2:172.21.15.32:6800/1121526649,v1:172.21.15.32:6801/1121526649] conn(0x7f534c036790 0x7f534c038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:50:44.760 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:43.839+0000 7f53597fa700 1 -- 172.21.15.32:0/270070416 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f535004a690 con 0x7f535c076c30 2024-05-12T03:50:44.760 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:43.839+0000 7f535bfff700 1 -- 172.21.15.32:0/270070416 >> [v2:172.21.15.32:6800/1121526649,v1:172.21.15.32:6801/1121526649] conn(0x7f534c036790 msgr2=0x7f534c038c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/1121526649 2024-05-12T03:50:44.760 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:43.839+0000 7f535bfff700 1 --2- 172.21.15.32:0/270070416 >> [v2:172.21.15.32:6800/1121526649,v1:172.21.15.32:6801/1121526649] conn(0x7f534c036790 0x7f534c038c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-12T03:50:44.760 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:43.840+0000 7f5362f8c700 1 -- 172.21.15.32:0/270070416 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f535c05f5c0 con 0x7f535c076c30 2024-05-12T03:50:44.760 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:43.848+0000 7f53597fa700 1 -- 172.21.15.32:0/270070416 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f53500189a0 con 0x7f535c076c30 2024-05-12T03:50:44.760 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:44.040+0000 7f535bfff700 1 -- 172.21.15.32:0/270070416 >> [v2:172.21.15.32:6800/1121526649,v1:172.21.15.32:6801/1121526649] conn(0x7f534c036790 msgr2=0x7f534c038c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/1121526649 2024-05-12T03:50:44.760 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:44.040+0000 7f535bfff700 1 --2- 172.21.15.32:0/270070416 >> [v2:172.21.15.32:6800/1121526649,v1:172.21.15.32:6801/1121526649] conn(0x7f534c036790 0x7f534c038c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-12T03:50:44.760 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:44.099+0000 7f5362f8c700 1 -- 172.21.15.32:0/270070416 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f535c112860 con 0x7f535c076c30 2024-05-12T03:50:44.760 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:44.100+0000 7f53597fa700 1 -- 172.21.15.32:0/270070416 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+103 (secure 0 0 0) 0x7f5350048030 con 0x7f535c076c30 2024-05-12T03:50:44.760 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:44.102+0000 7f5362f8c700 1 -- 172.21.15.32:0/270070416 >> [v2:172.21.15.32:6800/1121526649,v1:172.21.15.32:6801/1121526649] conn(0x7f534c036790 msgr2=0x7f534c038c40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-12T03:50:44.760 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:44.102+0000 7f5362f8c700 1 --2- 172.21.15.32:0/270070416 >> [v2:172.21.15.32:6800/1121526649,v1:172.21.15.32:6801/1121526649] conn(0x7f534c036790 0x7f534c038c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:50:44.760 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:44.102+0000 7f5362f8c700 1 -- 172.21.15.32:0/270070416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f535c076c30 msgr2=0x7f535c111370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:50:44.761 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:44.102+0000 7f5362f8c700 1 --2- 172.21.15.32:0/270070416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f535c076c30 0x7f535c111370 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5350006d40 tx=0x7f5350009250 comp rx=0 tx=0).stop 2024-05-12T03:50:44.761 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:44.102+0000 7f5362f8c700 1 -- 172.21.15.32:0/270070416 shutdown_connections 2024-05-12T03:50:44.761 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:44.102+0000 7f5362f8c700 1 --2- 172.21.15.32:0/270070416 >> [v2:172.21.15.32:6800/1121526649,v1:172.21.15.32:6801/1121526649] conn(0x7f534c036790 0x7f534c038c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:50:44.761 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:44 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/270070416' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-12T03:50:44.762 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:44.102+0000 7f5362f8c700 1 --2- 172.21.15.32:0/270070416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f535c076c30 0x7f535c111370 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:50:44.762 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:44.102+0000 7f5362f8c700 1 -- 172.21.15.32:0/270070416 >> 172.21.15.32:0/270070416 conn(0x7f535c0fcc50 msgr2=0x7f535c0fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:50:44.762 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:44.102+0000 7f5362f8c700 1 -- 172.21.15.32:0/270070416 shutdown_connections 2024-05-12T03:50:44.762 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:44.102+0000 7f5362f8c700 1 -- 172.21.15.32:0/270070416 wait complete. 2024-05-12T03:50:44.762 INFO:teuthology.orchestra.run.smithi032.stdout:Waiting for the mgr to restart... 2024-05-12T03:50:44.762 INFO:teuthology.orchestra.run.smithi032.stdout:Waiting for mgr epoch 5... 2024-05-12T03:50:49.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:49 smithi032 ceph-mon[40008]: Active manager daemon smithi032.ukatkq restarted 2024-05-12T03:50:49.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:49 smithi032 ceph-mon[40008]: Activating manager daemon smithi032.ukatkq 2024-05-12T03:50:49.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:49 smithi032 ceph-mon[40008]: osdmap e2: 0 total, 0 up, 0 in 2024-05-12T03:50:49.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:49 smithi032 ceph-mon[40008]: mgrmap e6: smithi032.ukatkq(active, starting, since 0.00331914s) 2024-05-12T03:50:49.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:49 smithi032 ceph-mon[40008]: from='mgr.14118 172.21.15.32:0/4199128226' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mon metadata", "id": "smithi032"}]: dispatch 2024-05-12T03:50:49.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:49 smithi032 ceph-mon[40008]: from='mgr.14118 172.21.15.32:0/4199128226' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mgr metadata", "who": "smithi032.ukatkq", "id": "smithi032.ukatkq"}]: dispatch 2024-05-12T03:50:49.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:49 smithi032 ceph-mon[40008]: from='mgr.14118 172.21.15.32:0/4199128226' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-12T03:50:49.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:49 smithi032 ceph-mon[40008]: from='mgr.14118 172.21.15.32:0/4199128226' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-12T03:50:49.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:49 smithi032 ceph-mon[40008]: from='mgr.14118 172.21.15.32:0/4199128226' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-12T03:50:49.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:49 smithi032 ceph-mon[40008]: Manager daemon smithi032.ukatkq is now available 2024-05-12T03:50:49.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:49 smithi032 ceph-mon[40008]: from='mgr.14118 172.21.15.32:0/4199128226' entity='mgr.smithi032.ukatkq' 2024-05-12T03:50:49.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:49 smithi032 ceph-mon[40008]: from='mgr.14118 172.21.15.32:0/4199128226' entity='mgr.smithi032.ukatkq' 2024-05-12T03:50:49.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:49 smithi032 ceph-mon[40008]: from='mgr.14118 172.21.15.32:0/4199128226' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T03:50:49.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:49 smithi032 ceph-mon[40008]: from='mgr.14118 172.21.15.32:0/4199128226' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T03:50:49.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:49 smithi032 ceph-mon[40008]: from='mgr.14118 172.21.15.32:0/4199128226' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T03:50:50.767 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:50 smithi032 ceph-mon[40008]: Found migration_current of "None". Setting to last migration. 2024-05-12T03:50:50.767 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:50 smithi032 ceph-mon[40008]: from='mgr.14118 172.21.15.32:0/4199128226' entity='mgr.smithi032.ukatkq' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi032.ukatkq/mirror_snapshot_schedule"}]: dispatch 2024-05-12T03:50:50.767 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:50 smithi032 ceph-mon[40008]: from='mgr.14118 172.21.15.32:0/4199128226' entity='mgr.smithi032.ukatkq' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi032.ukatkq/trash_purge_schedule"}]: dispatch 2024-05-12T03:50:50.767 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:50 smithi032 ceph-mon[40008]: mgrmap e7: smithi032.ukatkq(active, since 1.00511s) 2024-05-12T03:50:50.989 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout { 2024-05-12T03:50:50.989 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-12T03:50:50.989 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-12T03:50:50.989 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout } 2024-05-12T03:50:50.990 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:45.278+0000 7f3e9b020700 1 Processor -- start 2024-05-12T03:50:50.990 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:45.278+0000 7f3e9b020700 1 -- start start 2024-05-12T03:50:50.990 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:45.278+0000 7f3e9b020700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e94076c20 0x7f3e94105160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:50:50.990 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:45.278+0000 7f3e9b020700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e941056a0 con 0x7f3e94076c20 2024-05-12T03:50:50.990 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:45.279+0000 7f3e98dbc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e94076c20 0x7f3e94105160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:50:50.990 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:45.279+0000 7f3e98dbc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e94076c20 0x7f3e94105160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54968/0 (socket says 172.21.15.32:54968) 2024-05-12T03:50:50.990 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:45.279+0000 7f3e98dbc700 1 -- 172.21.15.32:0/3965881947 learned_addr learned my addr 172.21.15.32:0/3965881947 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:50:50.990 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:45.279+0000 7f3e98dbc700 1 -- 172.21.15.32:0/3965881947 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e941057e0 con 0x7f3e94076c20 2024-05-12T03:50:50.990 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:45.279+0000 7f3e98dbc700 1 --2- 172.21.15.32:0/3965881947 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e94076c20 0x7f3e94105160 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3e880098e0 tx=0x7f3e88009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c0fdb5579d3ad2 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:50:50.990 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:45.279+0000 7f3e937fe700 1 -- 172.21.15.32:0/3965881947 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3e8800b860 con 0x7f3e94076c20 2024-05-12T03:50:50.990 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:45.279+0000 7f3e937fe700 1 -- 172.21.15.32:0/3965881947 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f3e8800be30 con 0x7f3e94076c20 2024-05-12T03:50:50.991 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:45.279+0000 7f3e937fe700 1 -- 172.21.15.32:0/3965881947 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3e88010380 con 0x7f3e94076c20 2024-05-12T03:50:50.991 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:45.280+0000 7f3e9b020700 1 -- 172.21.15.32:0/3965881947 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e94076c20 msgr2=0x7f3e94105160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:50:50.991 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:45.280+0000 7f3e9b020700 1 --2- 172.21.15.32:0/3965881947 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e94076c20 0x7f3e94105160 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3e880098e0 tx=0x7f3e88009250 comp rx=0 tx=0).stop 2024-05-12T03:50:50.991 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:45.280+0000 7f3e9b020700 1 -- 172.21.15.32:0/3965881947 shutdown_connections 2024-05-12T03:50:50.991 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:45.280+0000 7f3e9b020700 1 --2- 172.21.15.32:0/3965881947 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e94076c20 0x7f3e94105160 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:50:50.991 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:45.280+0000 7f3e9b020700 1 -- 172.21.15.32:0/3965881947 >> 172.21.15.32:0/3965881947 conn(0x7f3e940fcbf0 msgr2=0x7f3e940ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:50:50.991 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:45.280+0000 7f3e9b020700 1 -- 172.21.15.32:0/3965881947 shutdown_connections 2024-05-12T03:50:50.991 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:45.280+0000 7f3e9b020700 1 -- 172.21.15.32:0/3965881947 wait complete. 2024-05-12T03:50:50.991 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:45.280+0000 7f3e9b020700 1 Processor -- start 2024-05-12T03:50:50.991 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:45.281+0000 7f3e9b020700 1 -- start start 2024-05-12T03:50:50.991 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:45.281+0000 7f3e9b020700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e94076c20 0x7f3e94102ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:50:50.991 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:45.281+0000 7f3e9b020700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e94102ff0 con 0x7f3e94076c20 2024-05-12T03:50:50.992 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:45.281+0000 7f3e98dbc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e94076c20 0x7f3e94102ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:50:50.992 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:45.281+0000 7f3e98dbc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e94076c20 0x7f3e94102ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54980/0 (socket says 172.21.15.32:54980) 2024-05-12T03:50:50.992 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:45.281+0000 7f3e98dbc700 1 -- 172.21.15.32:0/35048785 learned_addr learned my addr 172.21.15.32:0/35048785 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:50:50.992 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:45.281+0000 7f3e98dbc700 1 -- 172.21.15.32:0/35048785 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e880094f0 con 0x7f3e94076c20 2024-05-12T03:50:50.992 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:45.281+0000 7f3e98dbc700 1 --2- 172.21.15.32:0/35048785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e94076c20 0x7f3e94102ab0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3e88006b00 tx=0x7f3e88009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:50:50.992 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:45.282+0000 7f3e917fa700 1 -- 172.21.15.32:0/35048785 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3e88010a80 con 0x7f3e94076c20 2024-05-12T03:50:50.992 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:45.282+0000 7f3e9b020700 1 -- 172.21.15.32:0/35048785 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e94101060 con 0x7f3e94076c20 2024-05-12T03:50:50.992 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:45.282+0000 7f3e917fa700 1 -- 172.21.15.32:0/35048785 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f3e8801a430 con 0x7f3e94076c20 2024-05-12T03:50:50.992 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:45.282+0000 7f3e917fa700 1 -- 172.21.15.32:0/35048785 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3e880245b0 con 0x7f3e94076c20 2024-05-12T03:50:50.992 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:45.282+0000 7f3e9b020700 1 -- 172.21.15.32:0/35048785 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e94101550 con 0x7f3e94076c20 2024-05-12T03:50:50.992 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:45.282+0000 7f3e917fa700 1 -- 172.21.15.32:0/35048785 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f3e88024830 con 0x7f3e94076c20 2024-05-12T03:50:50.993 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:45.283+0000 7f3e917fa700 1 --2- 172.21.15.32:0/35048785 >> [v2:172.21.15.32:6800/1121526649,v1:172.21.15.32:6801/1121526649] conn(0x7f3e84036790 0x7f3e84038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:50:50.993 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:45.283+0000 7f3e9b020700 1 -- 172.21.15.32:0/35048785 --> [v2:172.21.15.32:6800/1121526649,v1:172.21.15.32:6801/1121526649] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3e9404bed0 con 0x7f3e84036790 2024-05-12T03:50:50.993 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:45.283+0000 7f3e917fa700 1 -- 172.21.15.32:0/35048785 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f3e8804ad00 con 0x7f3e94076c20 2024-05-12T03:50:50.993 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:45.283+0000 7f3e93fff700 1 -- 172.21.15.32:0/35048785 >> [v2:172.21.15.32:6800/1121526649,v1:172.21.15.32:6801/1121526649] conn(0x7f3e84036790 msgr2=0x7f3e84038c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/1121526649 2024-05-12T03:50:50.993 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:45.283+0000 7f3e93fff700 1 --2- 172.21.15.32:0/35048785 >> [v2:172.21.15.32:6800/1121526649,v1:172.21.15.32:6801/1121526649] conn(0x7f3e84036790 0x7f3e84038c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-12T03:50:50.993 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:45.484+0000 7f3e93fff700 1 -- 172.21.15.32:0/35048785 >> [v2:172.21.15.32:6800/1121526649,v1:172.21.15.32:6801/1121526649] conn(0x7f3e84036790 msgr2=0x7f3e84038c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/1121526649 2024-05-12T03:50:50.993 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:45.484+0000 7f3e93fff700 1 --2- 172.21.15.32:0/35048785 >> [v2:172.21.15.32:6800/1121526649,v1:172.21.15.32:6801/1121526649] conn(0x7f3e84036790 0x7f3e84038c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-12T03:50:50.993 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:45.885+0000 7f3e93fff700 1 -- 172.21.15.32:0/35048785 >> [v2:172.21.15.32:6800/1121526649,v1:172.21.15.32:6801/1121526649] conn(0x7f3e84036790 msgr2=0x7f3e84038c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/1121526649 2024-05-12T03:50:50.993 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:45.885+0000 7f3e93fff700 1 --2- 172.21.15.32:0/35048785 >> [v2:172.21.15.32:6800/1121526649,v1:172.21.15.32:6801/1121526649] conn(0x7f3e84036790 0x7f3e84038c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-12T03:50:50.993 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:46.686+0000 7f3e93fff700 1 -- 172.21.15.32:0/35048785 >> [v2:172.21.15.32:6800/1121526649,v1:172.21.15.32:6801/1121526649] conn(0x7f3e84036790 msgr2=0x7f3e84038c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/1121526649 2024-05-12T03:50:50.994 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:46.686+0000 7f3e93fff700 1 --2- 172.21.15.32:0/35048785 >> [v2:172.21.15.32:6800/1121526649,v1:172.21.15.32:6801/1121526649] conn(0x7f3e84036790 0x7f3e84038c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-12T03:50:50.994 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:48.287+0000 7f3e93fff700 1 -- 172.21.15.32:0/35048785 >> [v2:172.21.15.32:6800/1121526649,v1:172.21.15.32:6801/1121526649] conn(0x7f3e84036790 msgr2=0x7f3e84038c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/1121526649 2024-05-12T03:50:50.994 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:48.287+0000 7f3e93fff700 1 --2- 172.21.15.32:0/35048785 >> [v2:172.21.15.32:6800/1121526649,v1:172.21.15.32:6801/1121526649] conn(0x7f3e84036790 0x7f3e84038c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-12T03:50:50.994 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:49.380+0000 7f3e917fa700 1 -- 172.21.15.32:0/35048785 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mgrmap(e 6) v1 ==== 43143+0+0 (secure 0 0 0) 0x7f3e8802e960 con 0x7f3e94076c20 2024-05-12T03:50:50.994 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:49.380+0000 7f3e917fa700 1 -- 172.21.15.32:0/35048785 >> [v2:172.21.15.32:6800/1121526649,v1:172.21.15.32:6801/1121526649] conn(0x7f3e84036790 msgr2=0x7f3e84038c40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-12T03:50:50.994 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:49.380+0000 7f3e917fa700 1 --2- 172.21.15.32:0/35048785 >> [v2:172.21.15.32:6800/1121526649,v1:172.21.15.32:6801/1121526649] conn(0x7f3e84036790 0x7f3e84038c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:50:50.994 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:50.381+0000 7f3e917fa700 1 -- 172.21.15.32:0/35048785 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 7) v1 ==== 43270+0+0 (secure 0 0 0) 0x7f3e8802dcc0 con 0x7f3e94076c20 2024-05-12T03:50:50.994 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:50.381+0000 7f3e917fa700 1 --2- 172.21.15.32:0/35048785 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f3e84039f50 0x7f3e8403c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:50:50.994 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:50.381+0000 7f3e917fa700 1 -- 172.21.15.32:0/35048785 --> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3e9404bed0 con 0x7f3e84039f50 2024-05-12T03:50:50.995 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:50.381+0000 7f3e93fff700 1 --2- 172.21.15.32:0/35048785 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f3e84039f50 0x7f3e8403c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:50:50.995 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:50.382+0000 7f3e93fff700 1 --2- 172.21.15.32:0/35048785 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f3e84039f50 0x7f3e8403c330 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3e800039d0 tx=0x7f3e80009350 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:50:50.995 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:50.382+0000 7f3e917fa700 1 -- 172.21.15.32:0/35048785 <== mgr.14118 v2:172.21.15.32:6800/2967201790 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7f3e9404bed0 con 0x7f3e84039f50 2024-05-12T03:50:50.995 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:50.388+0000 7f3e9b020700 1 -- 172.21.15.32:0/35048785 --> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f3e94106500 con 0x7f3e84039f50 2024-05-12T03:50:50.995 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:50.388+0000 7f3e917fa700 1 -- 172.21.15.32:0/35048785 <== mgr.14118 v2:172.21.15.32:6800/2967201790 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f3e94106500 con 0x7f3e84039f50 2024-05-12T03:50:50.995 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:50.389+0000 7f3e9b020700 1 -- 172.21.15.32:0/35048785 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f3e84039f50 msgr2=0x7f3e8403c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:50:50.995 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:50.389+0000 7f3e9b020700 1 --2- 172.21.15.32:0/35048785 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f3e84039f50 0x7f3e8403c330 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3e800039d0 tx=0x7f3e80009350 comp rx=0 tx=0).stop 2024-05-12T03:50:50.995 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:50.389+0000 7f3e9b020700 1 -- 172.21.15.32:0/35048785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e94076c20 msgr2=0x7f3e94102ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:50:50.995 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:50.389+0000 7f3e9b020700 1 --2- 172.21.15.32:0/35048785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e94076c20 0x7f3e94102ab0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3e88006b00 tx=0x7f3e88009250 comp rx=0 tx=0).stop 2024-05-12T03:50:50.995 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:50.389+0000 7f3e9b020700 1 -- 172.21.15.32:0/35048785 shutdown_connections 2024-05-12T03:50:50.995 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:50.389+0000 7f3e9b020700 1 --2- 172.21.15.32:0/35048785 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f3e84039f50 0x7f3e8403c330 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:50:50.996 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:50.389+0000 7f3e9b020700 1 --2- 172.21.15.32:0/35048785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e94076c20 0x7f3e94102ab0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:50:50.996 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:50.389+0000 7f3e9b020700 1 --2- 172.21.15.32:0/35048785 >> [v2:172.21.15.32:6800/1121526649,v1:172.21.15.32:6801/1121526649] conn(0x7f3e84036790 0x7f3e84038c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:50:50.996 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:50.389+0000 7f3e9b020700 1 -- 172.21.15.32:0/35048785 >> 172.21.15.32:0/35048785 conn(0x7f3e940fcbf0 msgr2=0x7f3e940fd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:50:50.996 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:50.389+0000 7f3e9b020700 1 -- 172.21.15.32:0/35048785 shutdown_connections 2024-05-12T03:50:50.996 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:50.389+0000 7f3e9b020700 1 -- 172.21.15.32:0/35048785 wait complete. 2024-05-12T03:50:50.996 INFO:teuthology.orchestra.run.smithi032.stdout:mgr epoch 5 is available 2024-05-12T03:50:50.996 INFO:teuthology.orchestra.run.smithi032.stdout:Setting orchestrator backend to cephadm... 2024-05-12T03:50:51.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:51 smithi032 ceph-mon[40008]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-12T03:50:51.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:51 smithi032 ceph-mon[40008]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-12T03:50:51.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:51 smithi032 ceph-mon[40008]: from='mgr.14118 172.21.15.32:0/4199128226' entity='mgr.smithi032.ukatkq' 2024-05-12T03:50:51.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:51 smithi032 ceph-mon[40008]: from='mgr.14118 172.21.15.32:0/4199128226' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T03:50:52.448 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:51.584+0000 7f70779c6700 1 Processor -- start 2024-05-12T03:50:52.448 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:51.585+0000 7f70779c6700 1 -- start start 2024-05-12T03:50:52.448 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:51.585+0000 7f70779c6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7070102bb0 0x7f7070102fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:50:52.449 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:51.585+0000 7f70779c6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7070103500 con 0x7f7070102bb0 2024-05-12T03:50:52.449 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:51.585+0000 7f7075762700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7070102bb0 0x7f7070102fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:50:52.449 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:51.585+0000 7f7075762700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7070102bb0 0x7f7070102fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55054/0 (socket says 172.21.15.32:55054) 2024-05-12T03:50:52.449 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:51.585+0000 7f7075762700 1 -- 172.21.15.32:0/3605430957 learned_addr learned my addr 172.21.15.32:0/3605430957 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:50:52.449 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:51.585+0000 7f7075762700 1 -- 172.21.15.32:0/3605430957 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7070103640 con 0x7f7070102bb0 2024-05-12T03:50:52.449 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:51.585+0000 7f7075762700 1 --2- 172.21.15.32:0/3605430957 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7070102bb0 0x7f7070102fc0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f706c0098e0 tx=0x7f706c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52ddda98fafed6d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:50:52.449 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:51.586+0000 7f7067fff700 1 -- 172.21.15.32:0/3605430957 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f706c00b860 con 0x7f7070102bb0 2024-05-12T03:50:52.449 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:51.586+0000 7f7067fff700 1 -- 172.21.15.32:0/3605430957 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f706c00be30 con 0x7f7070102bb0 2024-05-12T03:50:52.449 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:51.586+0000 7f7067fff700 1 -- 172.21.15.32:0/3605430957 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f706c010380 con 0x7f7070102bb0 2024-05-12T03:50:52.449 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:51.586+0000 7f70779c6700 1 -- 172.21.15.32:0/3605430957 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7070102bb0 msgr2=0x7f7070102fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:50:52.449 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:51.586+0000 7f70779c6700 1 --2- 172.21.15.32:0/3605430957 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7070102bb0 0x7f7070102fc0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f706c0098e0 tx=0x7f706c009250 comp rx=0 tx=0).stop 2024-05-12T03:50:52.449 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:51.586+0000 7f70779c6700 1 -- 172.21.15.32:0/3605430957 shutdown_connections 2024-05-12T03:50:52.449 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:51.586+0000 7f70779c6700 1 --2- 172.21.15.32:0/3605430957 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7070102bb0 0x7f7070102fc0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:50:52.450 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:51.586+0000 7f70779c6700 1 -- 172.21.15.32:0/3605430957 >> 172.21.15.32:0/3605430957 conn(0x7f70700fc380 msgr2=0x7f70700fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:50:52.450 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:51.586+0000 7f70779c6700 1 -- 172.21.15.32:0/3605430957 shutdown_connections 2024-05-12T03:50:52.450 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:51.586+0000 7f70779c6700 1 -- 172.21.15.32:0/3605430957 wait complete. 2024-05-12T03:50:52.450 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:51.587+0000 7f70779c6700 1 Processor -- start 2024-05-12T03:50:52.450 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:51.587+0000 7f70779c6700 1 -- start start 2024-05-12T03:50:52.450 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:51.587+0000 7f70779c6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7070102bb0 0x7f7070113530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:50:52.450 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:51.587+0000 7f70779c6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7070113a70 con 0x7f7070102bb0 2024-05-12T03:50:52.450 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:51.587+0000 7f7075762700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7070102bb0 0x7f7070113530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:50:52.450 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:51.587+0000 7f7075762700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7070102bb0 0x7f7070113530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55066/0 (socket says 172.21.15.32:55066) 2024-05-12T03:50:52.450 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:51.587+0000 7f7075762700 1 -- 172.21.15.32:0/1393966050 learned_addr learned my addr 172.21.15.32:0/1393966050 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:50:52.450 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:51.588+0000 7f7075762700 1 -- 172.21.15.32:0/1393966050 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f706c0094f0 con 0x7f7070102bb0 2024-05-12T03:50:52.450 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:51.588+0000 7f7075762700 1 --2- 172.21.15.32:0/1393966050 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7070102bb0 0x7f7070113530 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f706c006b00 tx=0x7f706c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:50:52.450 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:51.588+0000 7f7065ffb700 1 -- 172.21.15.32:0/1393966050 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f706c0109b0 con 0x7f7070102bb0 2024-05-12T03:50:52.450 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:51.588+0000 7f7065ffb700 1 -- 172.21.15.32:0/1393966050 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f706c024440 con 0x7f7070102bb0 2024-05-12T03:50:52.451 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:51.588+0000 7f70779c6700 1 -- 172.21.15.32:0/1393966050 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7070113c70 con 0x7f7070102bb0 2024-05-12T03:50:52.451 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:51.588+0000 7f7065ffb700 1 -- 172.21.15.32:0/1393966050 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f706c01a5f0 con 0x7f7070102bb0 2024-05-12T03:50:52.451 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:51.588+0000 7f70779c6700 1 -- 172.21.15.32:0/1393966050 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70701147e0 con 0x7f7070102bb0 2024-05-12T03:50:52.451 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:51.589+0000 7f7065ffb700 1 -- 172.21.15.32:0/1393966050 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 7) v1 ==== 43270+0+0 (secure 0 0 0) 0x7f706c01a860 con 0x7f7070102bb0 2024-05-12T03:50:52.451 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:51.589+0000 7f70779c6700 1 -- 172.21.15.32:0/1393966050 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f707005f5c0 con 0x7f7070102bb0 2024-05-12T03:50:52.451 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:51.589+0000 7f7065ffb700 1 --2- 172.21.15.32:0/1393966050 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f705c0366b0 0x7f705c038b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:50:52.451 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:51.589+0000 7f7065ffb700 1 -- 172.21.15.32:0/1393966050 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f706c04a7a0 con 0x7f7070102bb0 2024-05-12T03:50:52.451 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:51.589+0000 7f7074f61700 1 --2- 172.21.15.32:0/1393966050 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f705c0366b0 0x7f705c038b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:50:52.451 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:51.590+0000 7f7074f61700 1 --2- 172.21.15.32:0/1393966050 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f705c0366b0 0x7f705c038b60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7060009990 tx=0x7f7060006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:50:52.451 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:51.594+0000 7f7065ffb700 1 -- 172.21.15.32:0/1393966050 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f706c018b40 con 0x7f7070102bb0 2024-05-12T03:50:52.451 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:51.802+0000 7f70779c6700 1 -- 172.21.15.32:0/1393966050 --> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f70700fd620 con 0x7f705c0366b0 2024-05-12T03:50:52.451 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:51.809+0000 7f7065ffb700 1 -- 172.21.15.32:0/1393966050 <== mgr.14118 v2:172.21.15.32:6800/2967201790 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f70700fd620 con 0x7f705c0366b0 2024-05-12T03:50:52.451 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:51.811+0000 7f70779c6700 1 -- 172.21.15.32:0/1393966050 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f705c0366b0 msgr2=0x7f705c038b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:50:52.452 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:52 smithi032 ceph-mon[40008]: [12/May/2024:03:50:51] ENGINE Bus STARTING 2024-05-12T03:50:52.452 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:52 smithi032 ceph-mon[40008]: [12/May/2024:03:50:51] ENGINE Serving on https://172.21.15.32:7150 2024-05-12T03:50:52.452 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:52 smithi032 ceph-mon[40008]: [12/May/2024:03:50:51] ENGINE Bus STARTED 2024-05-12T03:50:52.452 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:52 smithi032 ceph-mon[40008]: from='mgr.14118 172.21.15.32:0/4199128226' entity='mgr.smithi032.ukatkq' 2024-05-12T03:50:52.452 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:52 smithi032 ceph-mon[40008]: from='mgr.14118 172.21.15.32:0/4199128226' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T03:50:52.452 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:52 smithi032 ceph-mon[40008]: mgrmap e8: smithi032.ukatkq(active, since 2s) 2024-05-12T03:50:52.453 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:51.811+0000 7f70779c6700 1 --2- 172.21.15.32:0/1393966050 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f705c0366b0 0x7f705c038b60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7060009990 tx=0x7f7060006e40 comp rx=0 tx=0).stop 2024-05-12T03:50:52.453 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:51.811+0000 7f70779c6700 1 -- 172.21.15.32:0/1393966050 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7070102bb0 msgr2=0x7f7070113530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:50:52.453 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:51.811+0000 7f70779c6700 1 --2- 172.21.15.32:0/1393966050 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7070102bb0 0x7f7070113530 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f706c006b00 tx=0x7f706c009250 comp rx=0 tx=0).stop 2024-05-12T03:50:52.453 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:51.811+0000 7f70779c6700 1 -- 172.21.15.32:0/1393966050 shutdown_connections 2024-05-12T03:50:52.454 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:51.811+0000 7f70779c6700 1 --2- 172.21.15.32:0/1393966050 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f705c0366b0 0x7f705c038b60 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:50:52.454 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:51.811+0000 7f70779c6700 1 --2- 172.21.15.32:0/1393966050 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7070102bb0 0x7f7070113530 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:50:52.454 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:51.811+0000 7f70779c6700 1 -- 172.21.15.32:0/1393966050 >> 172.21.15.32:0/1393966050 conn(0x7f70700fc380 msgr2=0x7f70700fced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:50:52.454 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:51.812+0000 7f70779c6700 1 -- 172.21.15.32:0/1393966050 shutdown_connections 2024-05-12T03:50:52.454 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:51.812+0000 7f70779c6700 1 -- 172.21.15.32:0/1393966050 wait complete. 2024-05-12T03:50:53.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:53 smithi032 ceph-mon[40008]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T03:50:53.805 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-12T03:50:53.806 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:52.997+0000 7f32c1266700 1 Processor -- start 2024-05-12T03:50:53.806 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:52.997+0000 7f32c1266700 1 -- start start 2024-05-12T03:50:53.806 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:52.997+0000 7f32c1266700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32bc103170 0x7f32bc105550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:50:53.806 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:52.997+0000 7f32c1266700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32bc076d20 con 0x7f32bc103170 2024-05-12T03:50:53.806 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:52.997+0000 7f32baffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32bc103170 0x7f32bc105550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:50:53.806 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:52.997+0000 7f32baffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32bc103170 0x7f32bc105550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45836/0 (socket says 172.21.15.32:45836) 2024-05-12T03:50:53.806 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:52.997+0000 7f32baffd700 1 -- 172.21.15.32:0/4147262984 learned_addr learned my addr 172.21.15.32:0/4147262984 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:50:53.806 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:52.998+0000 7f32baffd700 1 -- 172.21.15.32:0/4147262984 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32bc076e60 con 0x7f32bc103170 2024-05-12T03:50:53.806 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:52.998+0000 7f32baffd700 1 --2- 172.21.15.32:0/4147262984 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32bc103170 0x7f32bc105550 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f32ac0098e0 tx=0x7f32ac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93c7112715544d66 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:50:53.806 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:52.998+0000 7f32b9ffb700 1 -- 172.21.15.32:0/4147262984 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f32ac00b860 con 0x7f32bc103170 2024-05-12T03:50:53.806 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:52.998+0000 7f32b9ffb700 1 -- 172.21.15.32:0/4147262984 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f32ac00be30 con 0x7f32bc103170 2024-05-12T03:50:53.806 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:52.998+0000 7f32b9ffb700 1 -- 172.21.15.32:0/4147262984 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f32ac010380 con 0x7f32bc103170 2024-05-12T03:50:53.806 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:52.998+0000 7f32c1266700 1 -- 172.21.15.32:0/4147262984 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32bc103170 msgr2=0x7f32bc105550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:50:53.806 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:52.998+0000 7f32c1266700 1 --2- 172.21.15.32:0/4147262984 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32bc103170 0x7f32bc105550 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f32ac0098e0 tx=0x7f32ac009250 comp rx=0 tx=0).stop 2024-05-12T03:50:53.807 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:52.998+0000 7f32c1266700 1 -- 172.21.15.32:0/4147262984 shutdown_connections 2024-05-12T03:50:53.807 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:52.998+0000 7f32c1266700 1 --2- 172.21.15.32:0/4147262984 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32bc103170 0x7f32bc105550 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:50:53.807 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:52.998+0000 7f32c1266700 1 -- 172.21.15.32:0/4147262984 >> 172.21.15.32:0/4147262984 conn(0x7f32bc0fcc10 msgr2=0x7f32bc0ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:50:53.807 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:52.999+0000 7f32c1266700 1 -- 172.21.15.32:0/4147262984 shutdown_connections 2024-05-12T03:50:53.807 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:52.999+0000 7f32c1266700 1 -- 172.21.15.32:0/4147262984 wait complete. 2024-05-12T03:50:53.807 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:52.999+0000 7f32c1266700 1 Processor -- start 2024-05-12T03:50:53.807 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:52.999+0000 7f32c1266700 1 -- start start 2024-05-12T03:50:53.807 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:52.999+0000 7f32c1266700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32bc103170 0x7f32bc1112e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:50:53.807 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:52.999+0000 7f32c1266700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32bc111820 con 0x7f32bc103170 2024-05-12T03:50:53.807 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:53.000+0000 7f32baffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32bc103170 0x7f32bc1112e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:50:53.807 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:53.000+0000 7f32baffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32bc103170 0x7f32bc1112e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45842/0 (socket says 172.21.15.32:45842) 2024-05-12T03:50:53.808 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:53.000+0000 7f32baffd700 1 -- 172.21.15.32:0/1973054514 learned_addr learned my addr 172.21.15.32:0/1973054514 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:50:53.808 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:53.000+0000 7f32baffd700 1 -- 172.21.15.32:0/1973054514 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32ac0094f0 con 0x7f32bc103170 2024-05-12T03:50:53.808 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:53.000+0000 7f32baffd700 1 --2- 172.21.15.32:0/1973054514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32bc103170 0x7f32bc1112e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f32ac006d40 tx=0x7f32ac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:50:53.808 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:53.000+0000 7f32a37fe700 1 -- 172.21.15.32:0/1973054514 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f32ac0109b0 con 0x7f32bc103170 2024-05-12T03:50:53.808 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:53.000+0000 7f32a37fe700 1 -- 172.21.15.32:0/1973054514 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f32ac024440 con 0x7f32bc103170 2024-05-12T03:50:53.808 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:53.000+0000 7f32a37fe700 1 -- 172.21.15.32:0/1973054514 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f32ac01a5f0 con 0x7f32bc103170 2024-05-12T03:50:53.808 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:53.000+0000 7f32c1266700 1 -- 172.21.15.32:0/1973054514 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32bc111a80 con 0x7f32bc103170 2024-05-12T03:50:53.808 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:53.000+0000 7f32c1266700 1 -- 172.21.15.32:0/1973054514 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32bc112590 con 0x7f32bc103170 2024-05-12T03:50:53.808 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:53.001+0000 7f32a37fe700 1 -- 172.21.15.32:0/1973054514 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f32ac01a870 con 0x7f32bc103170 2024-05-12T03:50:53.808 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:53.001+0000 7f32c1266700 1 -- 172.21.15.32:0/1973054514 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32bc05f5c0 con 0x7f32bc103170 2024-05-12T03:50:53.808 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:53.001+0000 7f32a37fe700 1 --2- 172.21.15.32:0/1973054514 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f32a4036790 0x7f32a4038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:50:53.808 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:53.001+0000 7f32a37fe700 1 -- 172.21.15.32:0/1973054514 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f32ac04a6e0 con 0x7f32bc103170 2024-05-12T03:50:53.809 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:53.002+0000 7f32ba7fc700 1 --2- 172.21.15.32:0/1973054514 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f32a4036790 0x7f32a4038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:50:53.809 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:53.002+0000 7f32ba7fc700 1 --2- 172.21.15.32:0/1973054514 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f32a4036790 0x7f32a4038c40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f32b0009990 tx=0x7f32b0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:50:53.809 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:53.006+0000 7f32a37fe700 1 -- 172.21.15.32:0/1973054514 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f32ac018530 con 0x7f32bc103170 2024-05-12T03:50:53.809 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:53.206+0000 7f32c1266700 1 -- 172.21.15.32:0/1973054514 --> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f32bc0fde80 con 0x7f32a4036790 2024-05-12T03:50:53.809 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:53.207+0000 7f32a37fe700 1 -- 172.21.15.32:0/1973054514 <== mgr.14118 v2:172.21.15.32:6800/2967201790 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f32bc0fde80 con 0x7f32a4036790 2024-05-12T03:50:53.809 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:53.209+0000 7f32c1266700 1 -- 172.21.15.32:0/1973054514 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f32a4036790 msgr2=0x7f32a4038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:50:53.809 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:53.209+0000 7f32c1266700 1 --2- 172.21.15.32:0/1973054514 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f32a4036790 0x7f32a4038c40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f32b0009990 tx=0x7f32b0006e40 comp rx=0 tx=0).stop 2024-05-12T03:50:53.809 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:53.209+0000 7f32c1266700 1 -- 172.21.15.32:0/1973054514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32bc103170 msgr2=0x7f32bc1112e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:50:53.809 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:53.209+0000 7f32c1266700 1 --2- 172.21.15.32:0/1973054514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32bc103170 0x7f32bc1112e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f32ac006d40 tx=0x7f32ac009250 comp rx=0 tx=0).stop 2024-05-12T03:50:53.809 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:53.209+0000 7f32c1266700 1 -- 172.21.15.32:0/1973054514 shutdown_connections 2024-05-12T03:50:53.809 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:53.209+0000 7f32c1266700 1 --2- 172.21.15.32:0/1973054514 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f32a4036790 0x7f32a4038c40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:50:53.809 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:53.209+0000 7f32c1266700 1 --2- 172.21.15.32:0/1973054514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32bc103170 0x7f32bc1112e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:50:53.809 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:53.209+0000 7f32c1266700 1 -- 172.21.15.32:0/1973054514 >> 172.21.15.32:0/1973054514 conn(0x7f32bc0fcc10 msgr2=0x7f32bc0fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:50:53.809 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:53.209+0000 7f32c1266700 1 -- 172.21.15.32:0/1973054514 shutdown_connections 2024-05-12T03:50:53.809 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:53.209+0000 7f32c1266700 1 -- 172.21.15.32:0/1973054514 wait complete. 2024-05-12T03:50:53.810 INFO:teuthology.orchestra.run.smithi032.stdout:Generating ssh key... 2024-05-12T03:50:54.789 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:54 smithi032 ceph-mon[40008]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T03:50:55.762 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:54.340+0000 7fc6c4fdd700 1 Processor -- start 2024-05-12T03:50:55.762 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:54.340+0000 7fc6c4fdd700 1 -- start start 2024-05-12T03:50:55.762 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:54.340+0000 7fc6c4fdd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6c0101580 0x7fc6c0101990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:50:55.762 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:54.340+0000 7fc6c4fdd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c0101ed0 con 0x7fc6c0101580 2024-05-12T03:50:55.762 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:54.340+0000 7fc6be59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6c0101580 0x7fc6c0101990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:50:55.762 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:54.341+0000 7fc6be59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6c0101580 0x7fc6c0101990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45852/0 (socket says 172.21.15.32:45852) 2024-05-12T03:50:55.762 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:54.341+0000 7fc6be59c700 1 -- 172.21.15.32:0/1035335514 learned_addr learned my addr 172.21.15.32:0/1035335514 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:50:55.762 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:54.341+0000 7fc6be59c700 1 -- 172.21.15.32:0/1035335514 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c0102010 con 0x7fc6c0101580 2024-05-12T03:50:55.762 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:54.341+0000 7fc6be59c700 1 --2- 172.21.15.32:0/1035335514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6c0101580 0x7fc6c0101990 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc6b00098e0 tx=0x7fc6b0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=912130a2ef7dd887 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:50:55.762 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:54.341+0000 7fc6bd59a700 1 -- 172.21.15.32:0/1035335514 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc6b000b860 con 0x7fc6c0101580 2024-05-12T03:50:55.762 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:54.341+0000 7fc6bd59a700 1 -- 172.21.15.32:0/1035335514 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7fc6b000be30 con 0x7fc6c0101580 2024-05-12T03:50:55.763 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:54.341+0000 7fc6bd59a700 1 -- 172.21.15.32:0/1035335514 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc6b0010380 con 0x7fc6c0101580 2024-05-12T03:50:55.763 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:54.342+0000 7fc6c4fdd700 1 -- 172.21.15.32:0/1035335514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6c0101580 msgr2=0x7fc6c0101990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:50:55.763 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:54.342+0000 7fc6c4fdd700 1 --2- 172.21.15.32:0/1035335514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6c0101580 0x7fc6c0101990 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc6b00098e0 tx=0x7fc6b0009250 comp rx=0 tx=0).stop 2024-05-12T03:50:55.763 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:54.342+0000 7fc6c4fdd700 1 -- 172.21.15.32:0/1035335514 shutdown_connections 2024-05-12T03:50:55.763 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:54.342+0000 7fc6c4fdd700 1 --2- 172.21.15.32:0/1035335514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6c0101580 0x7fc6c0101990 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:50:55.763 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:54.342+0000 7fc6c4fdd700 1 -- 172.21.15.32:0/1035335514 >> 172.21.15.32:0/1035335514 conn(0x7fc6c00fcc50 msgr2=0x7fc6c00ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:50:55.763 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:54.342+0000 7fc6c4fdd700 1 -- 172.21.15.32:0/1035335514 shutdown_connections 2024-05-12T03:50:55.763 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:54.342+0000 7fc6c4fdd700 1 -- 172.21.15.32:0/1035335514 wait complete. 2024-05-12T03:50:55.763 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:54.342+0000 7fc6c4fdd700 1 Processor -- start 2024-05-12T03:50:55.763 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:54.343+0000 7fc6c4fdd700 1 -- start start 2024-05-12T03:50:55.763 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:54.343+0000 7fc6c4fdd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6c0101580 0x7fc6c0076b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:50:55.763 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:54.343+0000 7fc6c4fdd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c0114000 con 0x7fc6c0101580 2024-05-12T03:50:55.763 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:54.343+0000 7fc6be59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6c0101580 0x7fc6c0076b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:50:55.763 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:54.343+0000 7fc6be59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6c0101580 0x7fc6c0076b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45864/0 (socket says 172.21.15.32:45864) 2024-05-12T03:50:55.763 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:54.343+0000 7fc6be59c700 1 -- 172.21.15.32:0/171534806 learned_addr learned my addr 172.21.15.32:0/171534806 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:50:55.764 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:54.343+0000 7fc6be59c700 1 -- 172.21.15.32:0/171534806 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6b00094f0 con 0x7fc6c0101580 2024-05-12T03:50:55.764 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:54.343+0000 7fc6be59c700 1 --2- 172.21.15.32:0/171534806 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6c0101580 0x7fc6c0076b30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0000c00 tx=0x7fc6b0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:50:55.764 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:54.343+0000 7fc6b6ffd700 1 -- 172.21.15.32:0/171534806 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc6b0010a80 con 0x7fc6c0101580 2024-05-12T03:50:55.764 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:54.343+0000 7fc6b6ffd700 1 -- 172.21.15.32:0/171534806 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7fc6b001a430 con 0x7fc6c0101580 2024-05-12T03:50:55.764 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:54.343+0000 7fc6c4fdd700 1 -- 172.21.15.32:0/171534806 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6c0114140 con 0x7fc6c0101580 2024-05-12T03:50:55.764 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:54.344+0000 7fc6b6ffd700 1 -- 172.21.15.32:0/171534806 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc6b00245b0 con 0x7fc6c0101580 2024-05-12T03:50:55.764 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:54.344+0000 7fc6c4fdd700 1 -- 172.21.15.32:0/171534806 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6c01145d0 con 0x7fc6c0101580 2024-05-12T03:50:55.764 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:54.344+0000 7fc6b6ffd700 1 -- 172.21.15.32:0/171534806 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7fc6b0010be0 con 0x7fc6c0101580 2024-05-12T03:50:55.764 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:54.344+0000 7fc6c4fdd700 1 -- 172.21.15.32:0/171534806 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6c004bed0 con 0x7fc6c0101580 2024-05-12T03:50:55.764 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:54.345+0000 7fc6b6ffd700 1 --2- 172.21.15.32:0/171534806 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7fc6ac0367e0 0x7fc6ac038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:50:55.764 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:54.345+0000 7fc6b6ffd700 1 -- 172.21.15.32:0/171534806 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fc6b004acd0 con 0x7fc6c0101580 2024-05-12T03:50:55.764 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:54.345+0000 7fc6bdd9b700 1 --2- 172.21.15.32:0/171534806 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7fc6ac0367e0 0x7fc6ac038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:50:55.764 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:54.345+0000 7fc6bdd9b700 1 --2- 172.21.15.32:0/171534806 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7fc6ac0367e0 0x7fc6ac038c90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc6a8009990 tx=0x7fc6a8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:50:55.764 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:54.349+0000 7fc6b6ffd700 1 -- 172.21.15.32:0/171534806 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fc6b0050020 con 0x7fc6c0101580 2024-05-12T03:50:55.764 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:54.545+0000 7fc6c4fdd700 1 -- 172.21.15.32:0/171534806 --> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fc6c0114810 con 0x7fc6ac0367e0 2024-05-12T03:50:55.765 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:55.152+0000 7fc6b6ffd700 1 -- 172.21.15.32:0/171534806 <== mgr.14118 v2:172.21.15.32:6800/2967201790 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fc6c0114810 con 0x7fc6ac0367e0 2024-05-12T03:50:55.765 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:55.154+0000 7fc6c4fdd700 1 -- 172.21.15.32:0/171534806 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7fc6ac0367e0 msgr2=0x7fc6ac038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:50:55.765 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:55.154+0000 7fc6c4fdd700 1 --2- 172.21.15.32:0/171534806 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7fc6ac0367e0 0x7fc6ac038c90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc6a8009990 tx=0x7fc6a8006e40 comp rx=0 tx=0).stop 2024-05-12T03:50:55.765 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:55.154+0000 7fc6c4fdd700 1 -- 172.21.15.32:0/171534806 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6c0101580 msgr2=0x7fc6c0076b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:50:55.765 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:55.154+0000 7fc6c4fdd700 1 --2- 172.21.15.32:0/171534806 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6c0101580 0x7fc6c0076b30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0000c00 tx=0x7fc6b0009250 comp rx=0 tx=0).stop 2024-05-12T03:50:55.765 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:55.154+0000 7fc6c4fdd700 1 -- 172.21.15.32:0/171534806 shutdown_connections 2024-05-12T03:50:55.765 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:55.154+0000 7fc6c4fdd700 1 --2- 172.21.15.32:0/171534806 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7fc6ac0367e0 0x7fc6ac038c90 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:50:55.765 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:55.154+0000 7fc6c4fdd700 1 --2- 172.21.15.32:0/171534806 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6c0101580 0x7fc6c0076b30 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:50:55.765 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:55.154+0000 7fc6c4fdd700 1 -- 172.21.15.32:0/171534806 >> 172.21.15.32:0/171534806 conn(0x7fc6c00fcc50 msgr2=0x7fc6c00fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:50:55.765 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:55 smithi032 ceph-mon[40008]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T03:50:55.765 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:55 smithi032 ceph-mon[40008]: Generating ssh key... 2024-05-12T03:50:55.766 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:55 smithi032 ceph-mon[40008]: from='mgr.14118 172.21.15.32:0/4199128226' entity='mgr.smithi032.ukatkq' 2024-05-12T03:50:55.766 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:55 smithi032 ceph-mon[40008]: from='mgr.14118 172.21.15.32:0/4199128226' entity='mgr.smithi032.ukatkq' 2024-05-12T03:50:55.766 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:55.154+0000 7fc6c4fdd700 1 -- 172.21.15.32:0/171534806 shutdown_connections 2024-05-12T03:50:55.766 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:55.154+0000 7fc6c4fdd700 1 -- 172.21.15.32:0/171534806 wait complete. 2024-05-12T03:50:56.969 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:56 smithi032 ceph-mon[40008]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T03:50:58.345 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHTfrwnLNrjtWpbcV83GLhyASEUjgXYkH0sFRt2L1Tvm67ejf3Xh8Yl1BWvuRL4eV7IIVigK2WexlUuXODx3cRc2PneH4kgoUA+QzI33fXd3FnRO6QKF0okku0/nfJxVxOyzAx6qYpkx9LOWjRTABljZiKoUtTlnkLJH+Zkulhxvb+9F6ad5lf8glq/IOHmoFBfc9xiQivVQg1asWwm5RiNCKOnoq+/YBofdDZVun7mgzw5EySK5lXyctXrxRYvr0hWJn0bogC8dbDEzqKy2PcNh5olEMzkHySRvj5h2qHKb30evy5UCYVFN8y8DqeiJMq/oehTqbmpSBMpw8OVpby5n4Bo3CPTI2kV/PI5Pdzx05DCc0VqSdrWloJpd1hknUdKHKiAUohyh2ck4fjMkTS6Ob2kcQ1oMv/p0WGzYybjaKPDcuNpmLikkMki3SrI1Rd5LOF2IuGghg0ODrbqdWbJ35iYOsj28IG2Rft2peRM9TSYJ5PUZrbvj1H3fLoVVc= ceph-a3e178f8-1012-11ef-bc98-c7b262605968 2024-05-12T03:50:58.345 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:56.353+0000 7f6a91c29700 1 Processor -- start 2024-05-12T03:50:58.345 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:56.353+0000 7f6a91c29700 1 -- start start 2024-05-12T03:50:58.345 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:56.353+0000 7f6a91c29700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a8c076bf0 0x7f6a8c105220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:50:58.346 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:56.353+0000 7f6a91c29700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8c105760 con 0x7f6a8c076bf0 2024-05-12T03:50:58.346 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:56.353+0000 7f6a8b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a8c076bf0 0x7f6a8c105220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:50:58.346 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:56.353+0000 7f6a8b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a8c076bf0 0x7f6a8c105220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45866/0 (socket says 172.21.15.32:45866) 2024-05-12T03:50:58.346 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:56.353+0000 7f6a8b7fe700 1 -- 172.21.15.32:0/3637590957 learned_addr learned my addr 172.21.15.32:0/3637590957 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:50:58.346 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:56.354+0000 7f6a8b7fe700 1 -- 172.21.15.32:0/3637590957 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a8c1058a0 con 0x7f6a8c076bf0 2024-05-12T03:50:58.346 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:56.354+0000 7f6a8b7fe700 1 --2- 172.21.15.32:0/3637590957 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a8c076bf0 0x7f6a8c105220 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c0098e0 tx=0x7f6a7c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c84700a3b9dd37af server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:50:58.346 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:56.354+0000 7f6a8a7fc700 1 -- 172.21.15.32:0/3637590957 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6a7c00b8f0 con 0x7f6a8c076bf0 2024-05-12T03:50:58.346 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:56.354+0000 7f6a8a7fc700 1 -- 172.21.15.32:0/3637590957 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f6a7c011420 con 0x7f6a8c076bf0 2024-05-12T03:50:58.346 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:56.354+0000 7f6a8a7fc700 1 -- 172.21.15.32:0/3637590957 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6a7c010540 con 0x7f6a8c076bf0 2024-05-12T03:50:58.346 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:56.354+0000 7f6a91c29700 1 -- 172.21.15.32:0/3637590957 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a8c076bf0 msgr2=0x7f6a8c105220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:50:58.346 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:56.354+0000 7f6a91c29700 1 --2- 172.21.15.32:0/3637590957 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a8c076bf0 0x7f6a8c105220 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c0098e0 tx=0x7f6a7c009250 comp rx=0 tx=0).stop 2024-05-12T03:50:58.346 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:56.354+0000 7f6a91c29700 1 -- 172.21.15.32:0/3637590957 shutdown_connections 2024-05-12T03:50:58.346 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:56.354+0000 7f6a91c29700 1 --2- 172.21.15.32:0/3637590957 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a8c076bf0 0x7f6a8c105220 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:50:58.347 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:56.354+0000 7f6a91c29700 1 -- 172.21.15.32:0/3637590957 >> 172.21.15.32:0/3637590957 conn(0x7f6a8c0fcc50 msgr2=0x7f6a8c0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:50:58.347 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:56.355+0000 7f6a91c29700 1 -- 172.21.15.32:0/3637590957 shutdown_connections 2024-05-12T03:50:58.347 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:56.355+0000 7f6a91c29700 1 -- 172.21.15.32:0/3637590957 wait complete. 2024-05-12T03:50:58.347 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:56.355+0000 7f6a91c29700 1 Processor -- start 2024-05-12T03:50:58.347 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:56.356+0000 7f6a91c29700 1 -- start start 2024-05-12T03:50:58.347 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:56.356+0000 7f6a91c29700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a8c076bf0 0x7f6a8c1113e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:50:58.347 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:56.356+0000 7f6a91c29700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8c111920 con 0x7f6a8c076bf0 2024-05-12T03:50:58.347 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:56.356+0000 7f6a8b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a8c076bf0 0x7f6a8c1113e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:50:58.347 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:56.356+0000 7f6a8b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a8c076bf0 0x7f6a8c1113e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45868/0 (socket says 172.21.15.32:45868) 2024-05-12T03:50:58.347 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:56.356+0000 7f6a8b7fe700 1 -- 172.21.15.32:0/512792530 learned_addr learned my addr 172.21.15.32:0/512792530 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:50:58.347 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:56.356+0000 7f6a8b7fe700 1 -- 172.21.15.32:0/512792530 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a7c0094f0 con 0x7f6a8c076bf0 2024-05-12T03:50:58.347 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:56.356+0000 7f6a8b7fe700 1 --2- 172.21.15.32:0/512792530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a8c076bf0 0x7f6a8c1113e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c006d40 tx=0x7f6a7c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:50:58.347 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:56.356+0000 7f6a73fff700 1 -- 172.21.15.32:0/512792530 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6a7c00beb0 con 0x7f6a8c076bf0 2024-05-12T03:50:58.347 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:56.356+0000 7f6a73fff700 1 -- 172.21.15.32:0/512792530 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f6a7c010e80 con 0x7f6a8c076bf0 2024-05-12T03:50:58.347 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:56.357+0000 7f6a91c29700 1 -- 172.21.15.32:0/512792530 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a8c111b20 con 0x7f6a8c076bf0 2024-05-12T03:50:58.348 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:56.357+0000 7f6a73fff700 1 -- 172.21.15.32:0/512792530 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6a7c024e00 con 0x7f6a8c076bf0 2024-05-12T03:50:58.348 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:56.357+0000 7f6a91c29700 1 -- 172.21.15.32:0/512792530 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a8c112690 con 0x7f6a8c076bf0 2024-05-12T03:50:58.348 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:56.358+0000 7f6a73fff700 1 -- 172.21.15.32:0/512792530 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f6a7c01a4c0 con 0x7f6a8c076bf0 2024-05-12T03:50:58.348 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:56.358+0000 7f6a73fff700 1 --2- 172.21.15.32:0/512792530 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f6a74036770 0x7f6a74038c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:50:58.348 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:56.358+0000 7f6a91c29700 1 -- 172.21.15.32:0/512792530 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a8c05f5c0 con 0x7f6a8c076bf0 2024-05-12T03:50:58.348 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:56.358+0000 7f6a73fff700 1 -- 172.21.15.32:0/512792530 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f6a7c04a6b0 con 0x7f6a8c076bf0 2024-05-12T03:50:58.348 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:56.358+0000 7f6a8affd700 1 --2- 172.21.15.32:0/512792530 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f6a74036770 0x7f6a74038c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:50:58.348 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:56.358+0000 7f6a8affd700 1 --2- 172.21.15.32:0/512792530 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f6a74036770 0x7f6a74038c20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6a80009990 tx=0x7f6a80006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:50:58.348 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:56.363+0000 7f6a73fff700 1 -- 172.21.15.32:0/512792530 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f6a7c02d3f0 con 0x7f6a8c076bf0 2024-05-12T03:50:58.348 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:56.559+0000 7f6a91c29700 1 -- 172.21.15.32:0/512792530 --> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f6a8c0fe050 con 0x7f6a74036770 2024-05-12T03:50:58.348 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:56.559+0000 7f6a73fff700 1 -- 172.21.15.32:0/512792530 <== mgr.14118 v2:172.21.15.32:6800/2967201790 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f6a8c0fe050 con 0x7f6a74036770 2024-05-12T03:50:58.348 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:56.561+0000 7f6a91c29700 1 -- 172.21.15.32:0/512792530 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f6a74036770 msgr2=0x7f6a74038c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:50:58.348 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:56.561+0000 7f6a91c29700 1 --2- 172.21.15.32:0/512792530 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f6a74036770 0x7f6a74038c20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6a80009990 tx=0x7f6a80006e40 comp rx=0 tx=0).stop 2024-05-12T03:50:58.348 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:56.561+0000 7f6a91c29700 1 -- 172.21.15.32:0/512792530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a8c076bf0 msgr2=0x7f6a8c1113e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:50:58.349 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:56.561+0000 7f6a91c29700 1 --2- 172.21.15.32:0/512792530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a8c076bf0 0x7f6a8c1113e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c006d40 tx=0x7f6a7c009250 comp rx=0 tx=0).stop 2024-05-12T03:50:58.349 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:56.561+0000 7f6a91c29700 1 -- 172.21.15.32:0/512792530 shutdown_connections 2024-05-12T03:50:58.349 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:56.561+0000 7f6a91c29700 1 --2- 172.21.15.32:0/512792530 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f6a74036770 0x7f6a74038c20 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:50:58.349 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:56.561+0000 7f6a91c29700 1 --2- 172.21.15.32:0/512792530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a8c076bf0 0x7f6a8c1113e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:50:58.349 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:56.562+0000 7f6a91c29700 1 -- 172.21.15.32:0/512792530 >> 172.21.15.32:0/512792530 conn(0x7f6a8c0fcc50 msgr2=0x7f6a8c0fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:50:58.349 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:56.562+0000 7f6a91c29700 1 -- 172.21.15.32:0/512792530 shutdown_connections 2024-05-12T03:50:58.349 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:56.562+0000 7f6a91c29700 1 -- 172.21.15.32:0/512792530 wait complete. 2024-05-12T03:50:58.349 INFO:teuthology.orchestra.run.smithi032.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-12T03:50:58.349 INFO:teuthology.orchestra.run.smithi032.stdout:Adding key to root@localhost authorized_keys... 2024-05-12T03:50:58.349 INFO:teuthology.orchestra.run.smithi032.stdout:Adding host smithi032... 2024-05-12T03:50:59.931 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:59 smithi032 ceph-mon[40008]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi032", "addr": "172.21.15.32", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T03:50:59.931 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:50:59 smithi032 ceph-mon[40008]: Deploying cephadm binary to smithi032 2024-05-12T03:51:01.511 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout Added host 'smithi032' with addr '172.21.15.32' 2024-05-12T03:51:01.511 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:58.897+0000 7f39e311a700 1 Processor -- start 2024-05-12T03:51:01.511 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:58.897+0000 7f39e311a700 1 -- start start 2024-05-12T03:51:01.512 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:58.897+0000 7f39e311a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39dc101580 0x7f39dc101990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:01.512 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:58.897+0000 7f39e311a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39dc101ed0 con 0x7f39dc101580 2024-05-12T03:51:01.512 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:58.897+0000 7f39e0eb6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39dc101580 0x7f39dc101990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:51:01.512 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:58.897+0000 7f39e0eb6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39dc101580 0x7f39dc101990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45876/0 (socket says 172.21.15.32:45876) 2024-05-12T03:51:01.512 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:58.897+0000 7f39e0eb6700 1 -- 172.21.15.32:0/1381776009 learned_addr learned my addr 172.21.15.32:0/1381776009 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:51:01.512 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:58.897+0000 7f39e0eb6700 1 -- 172.21.15.32:0/1381776009 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39dc102010 con 0x7f39dc101580 2024-05-12T03:51:01.512 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:58.898+0000 7f39e0eb6700 1 --2- 172.21.15.32:0/1381776009 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39dc101580 0x7f39dc101990 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f39d000bb80 tx=0x7f39d000b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f27b1a2595fa8a33 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:51:01.512 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:58.898+0000 7f39db7fe700 1 -- 172.21.15.32:0/1381776009 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f39d000dd00 con 0x7f39dc101580 2024-05-12T03:51:01.512 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:58.898+0000 7f39db7fe700 1 -- 172.21.15.32:0/1381776009 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f39d000de60 con 0x7f39dc101580 2024-05-12T03:51:01.512 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:58.898+0000 7f39db7fe700 1 -- 172.21.15.32:0/1381776009 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f39d0018840 con 0x7f39dc101580 2024-05-12T03:51:01.512 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:58.898+0000 7f39e311a700 1 -- 172.21.15.32:0/1381776009 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39dc101580 msgr2=0x7f39dc101990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:01.512 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:58.898+0000 7f39e311a700 1 --2- 172.21.15.32:0/1381776009 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39dc101580 0x7f39dc101990 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f39d000bb80 tx=0x7f39d000b4f0 comp rx=0 tx=0).stop 2024-05-12T03:51:01.512 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:58.898+0000 7f39e311a700 1 -- 172.21.15.32:0/1381776009 shutdown_connections 2024-05-12T03:51:01.512 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:58.898+0000 7f39e311a700 1 --2- 172.21.15.32:0/1381776009 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39dc101580 0x7f39dc101990 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:01.513 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:58.898+0000 7f39e311a700 1 -- 172.21.15.32:0/1381776009 >> 172.21.15.32:0/1381776009 conn(0x7f39dc0fcc50 msgr2=0x7f39dc0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:51:01.513 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:58.898+0000 7f39e311a700 1 -- 172.21.15.32:0/1381776009 shutdown_connections 2024-05-12T03:51:01.513 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:58.898+0000 7f39e311a700 1 -- 172.21.15.32:0/1381776009 wait complete. 2024-05-12T03:51:01.513 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:58.899+0000 7f39e311a700 1 Processor -- start 2024-05-12T03:51:01.513 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:58.899+0000 7f39e311a700 1 -- start start 2024-05-12T03:51:01.513 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:58.899+0000 7f39e311a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39dc101580 0x7f39dc10f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:01.513 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:58.899+0000 7f39e311a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39dc10f600 con 0x7f39dc101580 2024-05-12T03:51:01.513 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:58.899+0000 7f39e0eb6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39dc101580 0x7f39dc10f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:51:01.513 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:58.899+0000 7f39e0eb6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39dc101580 0x7f39dc10f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45890/0 (socket says 172.21.15.32:45890) 2024-05-12T03:51:01.513 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:58.899+0000 7f39e0eb6700 1 -- 172.21.15.32:0/2609184413 learned_addr learned my addr 172.21.15.32:0/2609184413 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:51:01.513 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:58.900+0000 7f39e0eb6700 1 -- 172.21.15.32:0/2609184413 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39d000b790 con 0x7f39dc101580 2024-05-12T03:51:01.513 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:58.900+0000 7f39e0eb6700 1 --2- 172.21.15.32:0/2609184413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39dc101580 0x7f39dc10f0c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f39d0006d40 tx=0x7f39d000b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:51:01.513 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:58.900+0000 7f39d97fa700 1 -- 172.21.15.32:0/2609184413 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f39d0010a00 con 0x7f39dc101580 2024-05-12T03:51:01.513 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:58.900+0000 7f39d97fa700 1 -- 172.21.15.32:0/2609184413 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7f39d001d050 con 0x7f39dc101580 2024-05-12T03:51:01.513 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:58.900+0000 7f39e311a700 1 -- 172.21.15.32:0/2609184413 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39dc10f800 con 0x7f39dc101580 2024-05-12T03:51:01.514 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:58.900+0000 7f39d97fa700 1 -- 172.21.15.32:0/2609184413 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f39d0022590 con 0x7f39dc101580 2024-05-12T03:51:01.514 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:58.900+0000 7f39e311a700 1 -- 172.21.15.32:0/2609184413 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39dc110370 con 0x7f39dc101580 2024-05-12T03:51:01.514 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:58.901+0000 7f39d97fa700 1 -- 172.21.15.32:0/2609184413 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f39d0023470 con 0x7f39dc101580 2024-05-12T03:51:01.514 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:58.901+0000 7f39d97fa700 1 --2- 172.21.15.32:0/2609184413 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f39cc036790 0x7f39cc038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:01.514 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:58.901+0000 7f39e311a700 1 -- 172.21.15.32:0/2609184413 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39dc05f5c0 con 0x7f39dc101580 2024-05-12T03:51:01.514 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:58.901+0000 7f39d97fa700 1 -- 172.21.15.32:0/2609184413 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f39d0014070 con 0x7f39dc101580 2024-05-12T03:51:01.514 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:58.901+0000 7f39dbfff700 1 --2- 172.21.15.32:0/2609184413 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f39cc036790 0x7f39cc038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:51:01.514 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:58.902+0000 7f39dbfff700 1 --2- 172.21.15.32:0/2609184413 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f39cc036790 0x7f39cc038c40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f39c8009990 tx=0x7f39c8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:51:01.514 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:58.906+0000 7f39d97fa700 1 -- 172.21.15.32:0/2609184413 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f39d0022cb0 con 0x7f39dc101580 2024-05-12T03:51:01.514 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:50:59.106+0000 7f39e311a700 1 -- 172.21.15.32:0/2609184413 --> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi032", "addr": "172.21.15.32", "target": ["mon-mgr", ""]}) v1 -- 0x7f39dc110720 con 0x7f39cc036790 2024-05-12T03:51:01.514 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:00.896+0000 7f39d97fa700 1 -- 172.21.15.32:0/2609184413 <== mgr.14118 v2:172.21.15.32:6800/2967201790 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f39dc110720 con 0x7f39cc036790 2024-05-12T03:51:01.514 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:00.899+0000 7f39e311a700 1 -- 172.21.15.32:0/2609184413 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f39cc036790 msgr2=0x7f39cc038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:01.514 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:00.899+0000 7f39e311a700 1 --2- 172.21.15.32:0/2609184413 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f39cc036790 0x7f39cc038c40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f39c8009990 tx=0x7f39c8006e40 comp rx=0 tx=0).stop 2024-05-12T03:51:01.514 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:00.899+0000 7f39e311a700 1 -- 172.21.15.32:0/2609184413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39dc101580 msgr2=0x7f39dc10f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:01.515 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:00.899+0000 7f39e311a700 1 --2- 172.21.15.32:0/2609184413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39dc101580 0x7f39dc10f0c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f39d0006d40 tx=0x7f39d000b4f0 comp rx=0 tx=0).stop 2024-05-12T03:51:01.515 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:00.899+0000 7f39e311a700 1 -- 172.21.15.32:0/2609184413 shutdown_connections 2024-05-12T03:51:01.515 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:00.899+0000 7f39e311a700 1 --2- 172.21.15.32:0/2609184413 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f39cc036790 0x7f39cc038c40 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:01.515 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:00.899+0000 7f39e311a700 1 --2- 172.21.15.32:0/2609184413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39dc101580 0x7f39dc10f0c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:01.515 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:00.899+0000 7f39e311a700 1 -- 172.21.15.32:0/2609184413 >> 172.21.15.32:0/2609184413 conn(0x7f39dc0fcc50 msgr2=0x7f39dc105970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:51:01.515 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:00.899+0000 7f39e311a700 1 -- 172.21.15.32:0/2609184413 shutdown_connections 2024-05-12T03:51:01.515 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:00.899+0000 7f39e311a700 1 -- 172.21.15.32:0/2609184413 wait complete. 2024-05-12T03:51:01.515 INFO:teuthology.orchestra.run.smithi032.stdout:Deploying mon service with default placement... 2024-05-12T03:51:02.039 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:01 smithi032 ceph-mon[40008]: from='mgr.14118 172.21.15.32:0/4199128226' entity='mgr.smithi032.ukatkq' 2024-05-12T03:51:02.039 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:01 smithi032 ceph-mon[40008]: Added host smithi032 2024-05-12T03:51:02.039 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:01 smithi032 ceph-mon[40008]: from='mgr.14118 172.21.15.32:0/4199128226' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T03:51:03.513 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:03 smithi032 ceph-mon[40008]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T03:51:03.514 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:03 smithi032 ceph-mon[40008]: Saving service mon spec with placement count:5 2024-05-12T03:51:03.514 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:03 smithi032 ceph-mon[40008]: from='mgr.14118 172.21.15.32:0/4199128226' entity='mgr.smithi032.ukatkq' 2024-05-12T03:51:03.928 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-12T03:51:03.928 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:02.174+0000 7fcca4994700 1 Processor -- start 2024-05-12T03:51:03.928 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:02.174+0000 7fcca4994700 1 -- start start 2024-05-12T03:51:03.928 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:02.174+0000 7fcca4994700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcca0076bd0 0x7fcca0105190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:03.928 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:02.174+0000 7fcca4994700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca01056d0 con 0x7fcca0076bd0 2024-05-12T03:51:03.929 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:02.174+0000 7fcc9e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcca0076bd0 0x7fcca0105190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:51:03.929 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:02.174+0000 7fcc9e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcca0076bd0 0x7fcca0105190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55452/0 (socket says 172.21.15.32:55452) 2024-05-12T03:51:03.929 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:02.174+0000 7fcc9e59c700 1 -- 172.21.15.32:0/164155152 learned_addr learned my addr 172.21.15.32:0/164155152 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:51:03.929 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:02.175+0000 7fcc9e59c700 1 -- 172.21.15.32:0/164155152 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcca0105810 con 0x7fcca0076bd0 2024-05-12T03:51:03.929 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:02.175+0000 7fcc9e59c700 1 --2- 172.21.15.32:0/164155152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcca0076bd0 0x7fcca0105190 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fcc900098e0 tx=0x7fcc90009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac82b9e8c2ee74d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:51:03.929 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:02.175+0000 7fcc9d59a700 1 -- 172.21.15.32:0/164155152 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcc9000b860 con 0x7fcca0076bd0 2024-05-12T03:51:03.929 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:02.175+0000 7fcc9d59a700 1 -- 172.21.15.32:0/164155152 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7fcc9000be30 con 0x7fcca0076bd0 2024-05-12T03:51:03.929 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:02.175+0000 7fcc9d59a700 1 -- 172.21.15.32:0/164155152 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcc90010380 con 0x7fcca0076bd0 2024-05-12T03:51:03.929 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:02.175+0000 7fcca4994700 1 -- 172.21.15.32:0/164155152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcca0076bd0 msgr2=0x7fcca0105190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:03.930 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:02.175+0000 7fcca4994700 1 --2- 172.21.15.32:0/164155152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcca0076bd0 0x7fcca0105190 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fcc900098e0 tx=0x7fcc90009250 comp rx=0 tx=0).stop 2024-05-12T03:51:03.930 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:02.176+0000 7fcca4994700 1 -- 172.21.15.32:0/164155152 shutdown_connections 2024-05-12T03:51:03.930 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:02.176+0000 7fcca4994700 1 --2- 172.21.15.32:0/164155152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcca0076bd0 0x7fcca0105190 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:03.930 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:02.176+0000 7fcca4994700 1 -- 172.21.15.32:0/164155152 >> 172.21.15.32:0/164155152 conn(0x7fcca00fcbc0 msgr2=0x7fcca00ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:51:03.930 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:02.176+0000 7fcca4994700 1 -- 172.21.15.32:0/164155152 shutdown_connections 2024-05-12T03:51:03.930 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:02.176+0000 7fcca4994700 1 -- 172.21.15.32:0/164155152 wait complete. 2024-05-12T03:51:03.930 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:02.176+0000 7fcca4994700 1 Processor -- start 2024-05-12T03:51:03.931 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:02.176+0000 7fcca4994700 1 -- start start 2024-05-12T03:51:03.931 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:02.176+0000 7fcca4994700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcca0076bd0 0x7fcca0111370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:03.931 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:02.176+0000 7fcca4994700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca01118b0 con 0x7fcca0076bd0 2024-05-12T03:51:03.931 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:02.177+0000 7fcc9e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcca0076bd0 0x7fcca0111370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:51:03.931 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:02.177+0000 7fcc9e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcca0076bd0 0x7fcca0111370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55464/0 (socket says 172.21.15.32:55464) 2024-05-12T03:51:03.931 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:02.177+0000 7fcc9e59c700 1 -- 172.21.15.32:0/3686127249 learned_addr learned my addr 172.21.15.32:0/3686127249 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:51:03.931 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:02.177+0000 7fcc9e59c700 1 -- 172.21.15.32:0/3686127249 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc900094f0 con 0x7fcca0076bd0 2024-05-12T03:51:03.931 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:02.177+0000 7fcc9e59c700 1 --2- 172.21.15.32:0/3686127249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcca0076bd0 0x7fcca0111370 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcc90006d40 tx=0x7fcc90009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:51:03.931 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:02.177+0000 7fcc8effd700 1 -- 172.21.15.32:0/3686127249 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcc900109b0 con 0x7fcca0076bd0 2024-05-12T03:51:03.931 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:02.177+0000 7fcc8effd700 1 -- 172.21.15.32:0/3686127249 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1009+0+0 (secure 0 0 0) 0x7fcc90024440 con 0x7fcca0076bd0 2024-05-12T03:51:03.931 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:02.177+0000 7fcca4994700 1 -- 172.21.15.32:0/3686127249 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcca0111ab0 con 0x7fcca0076bd0 2024-05-12T03:51:03.932 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:02.177+0000 7fcc8effd700 1 -- 172.21.15.32:0/3686127249 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcc9001a5f0 con 0x7fcca0076bd0 2024-05-12T03:51:03.932 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:02.177+0000 7fcca4994700 1 -- 172.21.15.32:0/3686127249 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcca0112620 con 0x7fcca0076bd0 2024-05-12T03:51:03.932 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:02.178+0000 7fcc8effd700 1 -- 172.21.15.32:0/3686127249 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7fcc90024970 con 0x7fcca0076bd0 2024-05-12T03:51:03.932 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:02.178+0000 7fcca4994700 1 -- 172.21.15.32:0/3686127249 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcca005f5c0 con 0x7fcca0076bd0 2024-05-12T03:51:03.932 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:02.178+0000 7fcc8effd700 1 --2- 172.21.15.32:0/3686127249 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7fcc880367e0 0x7fcc88038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:03.932 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:02.178+0000 7fcc8effd700 1 -- 172.21.15.32:0/3686127249 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fcc9004a5f0 con 0x7fcca0076bd0 2024-05-12T03:51:03.932 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:02.179+0000 7fcc9dd9b700 1 --2- 172.21.15.32:0/3686127249 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7fcc880367e0 0x7fcc88038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:51:03.932 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:02.179+0000 7fcc9dd9b700 1 --2- 172.21.15.32:0/3686127249 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7fcc880367e0 0x7fcc88038c90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fcc94009990 tx=0x7fcc94006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:51:03.932 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:02.183+0000 7fcc8effd700 1 -- 172.21.15.32:0/3686127249 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fcc9002d3f0 con 0x7fcca0076bd0 2024-05-12T03:51:03.932 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:02.408+0000 7fcca4994700 1 -- 172.21.15.32:0/3686127249 --> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}) v1 -- 0x7fcca00fdfa0 con 0x7fcc880367e0 2024-05-12T03:51:03.932 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:02.413+0000 7fcc8effd700 1 -- 172.21.15.32:0/3686127249 <== mgr.14118 v2:172.21.15.32:6800/2967201790 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fcca00fdfa0 con 0x7fcc880367e0 2024-05-12T03:51:03.933 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:02.416+0000 7fcca4994700 1 -- 172.21.15.32:0/3686127249 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7fcc880367e0 msgr2=0x7fcc88038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:03.933 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:02.416+0000 7fcca4994700 1 --2- 172.21.15.32:0/3686127249 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7fcc880367e0 0x7fcc88038c90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fcc94009990 tx=0x7fcc94006e40 comp rx=0 tx=0).stop 2024-05-12T03:51:03.933 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:02.416+0000 7fcca4994700 1 -- 172.21.15.32:0/3686127249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcca0076bd0 msgr2=0x7fcca0111370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:03.933 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:02.416+0000 7fcca4994700 1 --2- 172.21.15.32:0/3686127249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcca0076bd0 0x7fcca0111370 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcc90006d40 tx=0x7fcc90009250 comp rx=0 tx=0).stop 2024-05-12T03:51:03.933 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:02.416+0000 7fcca4994700 1 -- 172.21.15.32:0/3686127249 shutdown_connections 2024-05-12T03:51:03.933 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:02.416+0000 7fcca4994700 1 --2- 172.21.15.32:0/3686127249 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7fcc880367e0 0x7fcc88038c90 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:03.933 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:02.416+0000 7fcca4994700 1 --2- 172.21.15.32:0/3686127249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcca0076bd0 0x7fcca0111370 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:03.933 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:02.416+0000 7fcca4994700 1 -- 172.21.15.32:0/3686127249 >> 172.21.15.32:0/3686127249 conn(0x7fcca00fcbc0 msgr2=0x7fcca00fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:51:03.933 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:02.417+0000 7fcca4994700 1 -- 172.21.15.32:0/3686127249 shutdown_connections 2024-05-12T03:51:03.933 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:02.417+0000 7fcca4994700 1 -- 172.21.15.32:0/3686127249 wait complete. 2024-05-12T03:51:03.933 INFO:teuthology.orchestra.run.smithi032.stdout:Deploying mgr service with default placement... 2024-05-12T03:51:04.913 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:04 smithi032 ceph-mon[40008]: from='mgr.14118 172.21.15.32:0/4199128226' entity='mgr.smithi032.ukatkq' 2024-05-12T03:51:04.913 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:04 smithi032 ceph-mon[40008]: from='mgr.14118 172.21.15.32:0/4199128226' entity='mgr.smithi032.ukatkq' 2024-05-12T03:51:05.409 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-12T03:51:05.409 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:04.493+0000 7f2130089700 1 Processor -- start 2024-05-12T03:51:05.409 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:04.493+0000 7f2130089700 1 -- start start 2024-05-12T03:51:05.409 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:04.493+0000 7f2130089700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2128102610 0x7f2128102a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:05.409 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:04.493+0000 7f2130089700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2128102f60 con 0x7f2128102610 2024-05-12T03:51:05.409 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:04.494+0000 7f212de25700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2128102610 0x7f2128102a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:51:05.410 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:04.494+0000 7f212de25700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2128102610 0x7f2128102a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55474/0 (socket says 172.21.15.32:55474) 2024-05-12T03:51:05.410 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:04.494+0000 7f212de25700 1 -- 172.21.15.32:0/970598783 learned_addr learned my addr 172.21.15.32:0/970598783 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:51:05.410 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:04.494+0000 7f212de25700 1 -- 172.21.15.32:0/970598783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21281030a0 con 0x7f2128102610 2024-05-12T03:51:05.410 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:04.494+0000 7f212de25700 1 --2- 172.21.15.32:0/970598783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2128102610 0x7f2128102a20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f21240098e0 tx=0x7f2124009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ae22fcf83da644d server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:51:05.410 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:04.494+0000 7f212ce23700 1 -- 172.21.15.32:0/970598783 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f212400b860 con 0x7f2128102610 2024-05-12T03:51:05.410 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:04.494+0000 7f212ce23700 1 -- 172.21.15.32:0/970598783 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f212400be70 con 0x7f2128102610 2024-05-12T03:51:05.410 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:04.494+0000 7f212ce23700 1 -- 172.21.15.32:0/970598783 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2124010380 con 0x7f2128102610 2024-05-12T03:51:05.410 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:04.495+0000 7f2130089700 1 -- 172.21.15.32:0/970598783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2128102610 msgr2=0x7f2128102a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:05.410 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:04.495+0000 7f2130089700 1 --2- 172.21.15.32:0/970598783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2128102610 0x7f2128102a20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f21240098e0 tx=0x7f2124009250 comp rx=0 tx=0).stop 2024-05-12T03:51:05.410 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:04.495+0000 7f2130089700 1 -- 172.21.15.32:0/970598783 shutdown_connections 2024-05-12T03:51:05.410 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:04.495+0000 7f2130089700 1 --2- 172.21.15.32:0/970598783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2128102610 0x7f2128102a20 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:05.410 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:04.495+0000 7f2130089700 1 -- 172.21.15.32:0/970598783 >> 172.21.15.32:0/970598783 conn(0x7f21280fddd0 msgr2=0x7f21281001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:51:05.410 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:04.496+0000 7f2130089700 1 -- 172.21.15.32:0/970598783 shutdown_connections 2024-05-12T03:51:05.410 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:04.496+0000 7f2130089700 1 -- 172.21.15.32:0/970598783 wait complete. 2024-05-12T03:51:05.411 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:04.496+0000 7f2130089700 1 Processor -- start 2024-05-12T03:51:05.411 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:04.496+0000 7f2130089700 1 -- start start 2024-05-12T03:51:05.411 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:04.496+0000 7f2130089700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2128102610 0x7f2128111390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:05.411 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:04.496+0000 7f2130089700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21281118d0 con 0x7f2128102610 2024-05-12T03:51:05.411 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:04.496+0000 7f212de25700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2128102610 0x7f2128111390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:51:05.411 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:04.497+0000 7f212de25700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2128102610 0x7f2128111390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55490/0 (socket says 172.21.15.32:55490) 2024-05-12T03:51:05.411 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:04.497+0000 7f212de25700 1 -- 172.21.15.32:0/3405603428 learned_addr learned my addr 172.21.15.32:0/3405603428 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:51:05.411 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:04.497+0000 7f212de25700 1 -- 172.21.15.32:0/3405603428 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21240094f0 con 0x7f2128102610 2024-05-12T03:51:05.411 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:04.497+0000 7f212de25700 1 --2- 172.21.15.32:0/3405603428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2128102610 0x7f2128111390 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2124011ca0 tx=0x7f2124009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:51:05.411 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:04.497+0000 7f211e7fc700 1 -- 172.21.15.32:0/3405603428 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2124010850 con 0x7f2128102610 2024-05-12T03:51:05.411 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:04.497+0000 7f211e7fc700 1 -- 172.21.15.32:0/3405603428 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2124010e60 con 0x7f2128102610 2024-05-12T03:51:05.411 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:04.497+0000 7f2130089700 1 -- 172.21.15.32:0/3405603428 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2128111ad0 con 0x7f2128102610 2024-05-12T03:51:05.411 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:04.497+0000 7f211e7fc700 1 -- 172.21.15.32:0/3405603428 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f212401ae20 con 0x7f2128102610 2024-05-12T03:51:05.411 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:04.497+0000 7f2130089700 1 -- 172.21.15.32:0/3405603428 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2128112640 con 0x7f2128102610 2024-05-12T03:51:05.411 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:04.498+0000 7f211e7fc700 1 -- 172.21.15.32:0/3405603428 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f212402d3f0 con 0x7f2128102610 2024-05-12T03:51:05.411 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:04.498+0000 7f211e7fc700 1 --2- 172.21.15.32:0/3405603428 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f2114036790 0x7f2114038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:05.412 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:04.498+0000 7f211e7fc700 1 -- 172.21.15.32:0/3405603428 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f212401a6a0 con 0x7f2128102610 2024-05-12T03:51:05.412 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:04.498+0000 7f212d624700 1 --2- 172.21.15.32:0/3405603428 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f2114036790 0x7f2114038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:51:05.412 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:04.498+0000 7f2130089700 1 -- 172.21.15.32:0/3405603428 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f212805f5c0 con 0x7f2128102610 2024-05-12T03:51:05.412 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:04.499+0000 7f212d624700 1 --2- 172.21.15.32:0/3405603428 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f2114036790 0x7f2114038c40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2118009990 tx=0x7f2118006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:51:05.412 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:04.504+0000 7f211e7fc700 1 -- 172.21.15.32:0/3405603428 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f212402b8e0 con 0x7f2128102610 2024-05-12T03:51:05.412 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:04.747+0000 7f2130089700 1 -- 172.21.15.32:0/3405603428 --> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7f21280ff020 con 0x7f2114036790 2024-05-12T03:51:05.412 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:04.751+0000 7f211e7fc700 1 -- 172.21.15.32:0/3405603428 <== mgr.14118 v2:172.21.15.32:6800/2967201790 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f21280ff020 con 0x7f2114036790 2024-05-12T03:51:05.412 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:04.753+0000 7f2130089700 1 -- 172.21.15.32:0/3405603428 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f2114036790 msgr2=0x7f2114038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:05.412 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:04.753+0000 7f2130089700 1 --2- 172.21.15.32:0/3405603428 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f2114036790 0x7f2114038c40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2118009990 tx=0x7f2118006e40 comp rx=0 tx=0).stop 2024-05-12T03:51:05.412 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:04.753+0000 7f2130089700 1 -- 172.21.15.32:0/3405603428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2128102610 msgr2=0x7f2128111390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:05.412 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:04.753+0000 7f2130089700 1 --2- 172.21.15.32:0/3405603428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2128102610 0x7f2128111390 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2124011ca0 tx=0x7f2124009250 comp rx=0 tx=0).stop 2024-05-12T03:51:05.412 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:04.753+0000 7f2130089700 1 -- 172.21.15.32:0/3405603428 shutdown_connections 2024-05-12T03:51:05.412 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:04.753+0000 7f2130089700 1 --2- 172.21.15.32:0/3405603428 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f2114036790 0x7f2114038c40 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:05.412 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:04.753+0000 7f2130089700 1 --2- 172.21.15.32:0/3405603428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2128102610 0x7f2128111390 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:05.413 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:04.753+0000 7f2130089700 1 -- 172.21.15.32:0/3405603428 >> 172.21.15.32:0/3405603428 conn(0x7f21280fddd0 msgr2=0x7f21280fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:51:05.413 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:04.753+0000 7f2130089700 1 -- 172.21.15.32:0/3405603428 shutdown_connections 2024-05-12T03:51:05.413 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:04.753+0000 7f2130089700 1 -- 172.21.15.32:0/3405603428 wait complete. 2024-05-12T03:51:05.413 INFO:teuthology.orchestra.run.smithi032.stdout:Deploying crash service with default placement... 2024-05-12T03:51:06.039 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:05 smithi032 ceph-mon[40008]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T03:51:06.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:05 smithi032 ceph-mon[40008]: Saving service mgr spec with placement count:2 2024-05-12T03:51:06.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:05 smithi032 ceph-mon[40008]: from='mgr.14118 172.21.15.32:0/4199128226' entity='mgr.smithi032.ukatkq' 2024-05-12T03:51:06.833 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout Scheduled crash update... 2024-05-12T03:51:06.834 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:05.953+0000 7f89e848d700 1 Processor -- start 2024-05-12T03:51:06.834 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:05.953+0000 7f89e848d700 1 -- start start 2024-05-12T03:51:06.834 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:05.953+0000 7f89e848d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89e0103130 0x7f89e0105510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:06.834 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:05.953+0000 7f89e848d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e0076d60 con 0x7f89e0103130 2024-05-12T03:51:06.834 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:05.953+0000 7f89e6229700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89e0103130 0x7f89e0105510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:51:06.834 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:05.953+0000 7f89e6229700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89e0103130 0x7f89e0105510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55496/0 (socket says 172.21.15.32:55496) 2024-05-12T03:51:06.834 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:05.953+0000 7f89e6229700 1 -- 172.21.15.32:0/2190618223 learned_addr learned my addr 172.21.15.32:0/2190618223 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:51:06.834 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:05.953+0000 7f89e6229700 1 -- 172.21.15.32:0/2190618223 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e0076ea0 con 0x7f89e0103130 2024-05-12T03:51:06.834 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:05.954+0000 7f89e6229700 1 --2- 172.21.15.32:0/2190618223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89e0103130 0x7f89e0105510 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f89dc0098e0 tx=0x7f89dc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c4b9e01ffb76350 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:51:06.834 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:05.954+0000 7f89e5227700 1 -- 172.21.15.32:0/2190618223 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f89dc00b860 con 0x7f89e0103130 2024-05-12T03:51:06.834 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:05.954+0000 7f89e5227700 1 -- 172.21.15.32:0/2190618223 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f89dc00be70 con 0x7f89e0103130 2024-05-12T03:51:06.834 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:05.954+0000 7f89e5227700 1 -- 172.21.15.32:0/2190618223 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f89dc010380 con 0x7f89e0103130 2024-05-12T03:51:06.834 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:05.954+0000 7f89e848d700 1 -- 172.21.15.32:0/2190618223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89e0103130 msgr2=0x7f89e0105510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:06.834 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:05.954+0000 7f89e848d700 1 --2- 172.21.15.32:0/2190618223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89e0103130 0x7f89e0105510 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f89dc0098e0 tx=0x7f89dc009250 comp rx=0 tx=0).stop 2024-05-12T03:51:06.834 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:05.954+0000 7f89e848d700 1 -- 172.21.15.32:0/2190618223 shutdown_connections 2024-05-12T03:51:06.835 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:05.954+0000 7f89e848d700 1 --2- 172.21.15.32:0/2190618223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89e0103130 0x7f89e0105510 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:06.835 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:05.954+0000 7f89e848d700 1 -- 172.21.15.32:0/2190618223 >> 172.21.15.32:0/2190618223 conn(0x7f89e00fcc10 msgr2=0x7f89e00ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:51:06.835 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:05.954+0000 7f89e848d700 1 -- 172.21.15.32:0/2190618223 shutdown_connections 2024-05-12T03:51:06.835 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:05.955+0000 7f89e848d700 1 -- 172.21.15.32:0/2190618223 wait complete. 2024-05-12T03:51:06.835 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:05.955+0000 7f89e848d700 1 Processor -- start 2024-05-12T03:51:06.835 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:05.955+0000 7f89e848d700 1 -- start start 2024-05-12T03:51:06.835 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:05.955+0000 7f89e848d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89e0103130 0x7f89e01113b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:06.835 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:05.955+0000 7f89e848d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e01118f0 con 0x7f89e0103130 2024-05-12T03:51:06.835 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:05.955+0000 7f89e6229700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89e0103130 0x7f89e01113b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:51:06.835 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:05.955+0000 7f89e6229700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89e0103130 0x7f89e01113b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55508/0 (socket says 172.21.15.32:55508) 2024-05-12T03:51:06.835 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:05.955+0000 7f89e6229700 1 -- 172.21.15.32:0/3474095188 learned_addr learned my addr 172.21.15.32:0/3474095188 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:51:06.835 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:05.956+0000 7f89e6229700 1 -- 172.21.15.32:0/3474095188 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89dc0094f0 con 0x7f89e0103130 2024-05-12T03:51:06.835 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:05.956+0000 7f89e6229700 1 --2- 172.21.15.32:0/3474095188 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89e0103130 0x7f89e01113b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f89dc0094c0 tx=0x7f89dc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:51:06.835 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:05.956+0000 7f89d6ffd700 1 -- 172.21.15.32:0/3474095188 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f89dc010850 con 0x7f89e0103130 2024-05-12T03:51:06.835 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:05.956+0000 7f89d6ffd700 1 -- 172.21.15.32:0/3474095188 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f89dc010e60 con 0x7f89e0103130 2024-05-12T03:51:06.836 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:05.956+0000 7f89d6ffd700 1 -- 172.21.15.32:0/3474095188 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f89dc01a430 con 0x7f89e0103130 2024-05-12T03:51:06.836 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:05.956+0000 7f89e848d700 1 -- 172.21.15.32:0/3474095188 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89e0111af0 con 0x7f89e0103130 2024-05-12T03:51:06.836 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:05.956+0000 7f89e848d700 1 -- 172.21.15.32:0/3474095188 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89e0112660 con 0x7f89e0103130 2024-05-12T03:51:06.836 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:05.957+0000 7f89d6ffd700 1 -- 172.21.15.32:0/3474095188 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f89dc0247b0 con 0x7f89e0103130 2024-05-12T03:51:06.836 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:05.957+0000 7f89e848d700 1 -- 172.21.15.32:0/3474095188 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89e0111f80 con 0x7f89e0103130 2024-05-12T03:51:06.836 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:05.957+0000 7f89d6ffd700 1 --2- 172.21.15.32:0/3474095188 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f89cc036790 0x7f89cc038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:06.836 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:05.957+0000 7f89d6ffd700 1 -- 172.21.15.32:0/3474095188 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f89dc04a610 con 0x7f89e0103130 2024-05-12T03:51:06.836 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:05.957+0000 7f89e5a28700 1 --2- 172.21.15.32:0/3474095188 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f89cc036790 0x7f89cc038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:51:06.836 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:05.958+0000 7f89e5a28700 1 --2- 172.21.15.32:0/3474095188 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f89cc036790 0x7f89cc038c40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f89d0009990 tx=0x7f89d0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:51:06.836 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:05.962+0000 7f89d6ffd700 1 -- 172.21.15.32:0/3474095188 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f89dc022520 con 0x7f89e0103130 2024-05-12T03:51:06.836 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:06.167+0000 7f89e848d700 1 -- 172.21.15.32:0/3474095188 --> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}) v1 -- 0x7f89e00fdf40 con 0x7f89cc036790 2024-05-12T03:51:06.836 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:06.171+0000 7f89d6ffd700 1 -- 172.21.15.32:0/3474095188 <== mgr.14118 v2:172.21.15.32:6800/2967201790 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f89e00fdf40 con 0x7f89cc036790 2024-05-12T03:51:06.836 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:06.173+0000 7f89e848d700 1 -- 172.21.15.32:0/3474095188 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f89cc036790 msgr2=0x7f89cc038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:06.836 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:06.173+0000 7f89e848d700 1 --2- 172.21.15.32:0/3474095188 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f89cc036790 0x7f89cc038c40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f89d0009990 tx=0x7f89d0006e40 comp rx=0 tx=0).stop 2024-05-12T03:51:06.836 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:06.173+0000 7f89e848d700 1 -- 172.21.15.32:0/3474095188 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89e0103130 msgr2=0x7f89e01113b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:06.836 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:06.173+0000 7f89e848d700 1 --2- 172.21.15.32:0/3474095188 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89e0103130 0x7f89e01113b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f89dc0094c0 tx=0x7f89dc009250 comp rx=0 tx=0).stop 2024-05-12T03:51:06.836 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:06.173+0000 7f89e848d700 1 -- 172.21.15.32:0/3474095188 shutdown_connections 2024-05-12T03:51:06.837 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:06.173+0000 7f89e848d700 1 --2- 172.21.15.32:0/3474095188 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f89cc036790 0x7f89cc038c40 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:06.837 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:06.173+0000 7f89e848d700 1 --2- 172.21.15.32:0/3474095188 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89e0103130 0x7f89e01113b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:06.837 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:06.173+0000 7f89e848d700 1 -- 172.21.15.32:0/3474095188 >> 172.21.15.32:0/3474095188 conn(0x7f89e00fcc10 msgr2=0x7f89e00fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:51:06.837 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:06.173+0000 7f89e848d700 1 -- 172.21.15.32:0/3474095188 shutdown_connections 2024-05-12T03:51:06.837 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:06.174+0000 7f89e848d700 1 -- 172.21.15.32:0/3474095188 wait complete. 2024-05-12T03:51:06.837 INFO:teuthology.orchestra.run.smithi032.stdout:Deploying prometheus service with default placement... 2024-05-12T03:51:07.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:07 smithi032 ceph-mon[40008]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T03:51:07.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:07 smithi032 ceph-mon[40008]: Saving service crash spec with placement * 2024-05-12T03:51:07.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:07 smithi032 ceph-mon[40008]: from='mgr.14118 172.21.15.32:0/4199128226' entity='mgr.smithi032.ukatkq' 2024-05-12T03:51:07.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:07 smithi032 ceph-mon[40008]: from='mgr.14118 172.21.15.32:0/4199128226' entity='mgr.smithi032.ukatkq' 2024-05-12T03:51:08.257 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout Scheduled prometheus update... 2024-05-12T03:51:08.257 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:07.380+0000 7f90900fa700 1 Processor -- start 2024-05-12T03:51:08.257 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:07.381+0000 7f90900fa700 1 -- start start 2024-05-12T03:51:08.257 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:07.381+0000 7f90900fa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9088103a40 0x7f9088103e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:08.258 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:07.381+0000 7f90900fa700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9088104390 con 0x7f9088103a40 2024-05-12T03:51:08.258 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:07.381+0000 7f908de96700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9088103a40 0x7f9088103e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:51:08.258 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:07.381+0000 7f908de96700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9088103a40 0x7f9088103e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55520/0 (socket says 172.21.15.32:55520) 2024-05-12T03:51:08.258 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:07.381+0000 7f908de96700 1 -- 172.21.15.32:0/1018877121 learned_addr learned my addr 172.21.15.32:0/1018877121 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:51:08.258 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:07.381+0000 7f908de96700 1 -- 172.21.15.32:0/1018877121 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90881044d0 con 0x7f9088103a40 2024-05-12T03:51:08.258 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:07.382+0000 7f908de96700 1 --2- 172.21.15.32:0/1018877121 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9088103a40 0x7f9088103e50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f90840098e0 tx=0x7f9084009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c22d4bb0ced68a2f server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:51:08.258 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:07.382+0000 7f908ce94700 1 -- 172.21.15.32:0/1018877121 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f908400b860 con 0x7f9088103a40 2024-05-12T03:51:08.258 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:07.382+0000 7f908ce94700 1 -- 172.21.15.32:0/1018877121 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f908400be70 con 0x7f9088103a40 2024-05-12T03:51:08.258 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:07.382+0000 7f908ce94700 1 -- 172.21.15.32:0/1018877121 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9084010380 con 0x7f9088103a40 2024-05-12T03:51:08.258 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:07.382+0000 7f90900fa700 1 -- 172.21.15.32:0/1018877121 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9088103a40 msgr2=0x7f9088103e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:08.258 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:07.382+0000 7f90900fa700 1 --2- 172.21.15.32:0/1018877121 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9088103a40 0x7f9088103e50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f90840098e0 tx=0x7f9084009250 comp rx=0 tx=0).stop 2024-05-12T03:51:08.258 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:07.383+0000 7f90900fa700 1 -- 172.21.15.32:0/1018877121 shutdown_connections 2024-05-12T03:51:08.258 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:07.383+0000 7f90900fa700 1 --2- 172.21.15.32:0/1018877121 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9088103a40 0x7f9088103e50 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:08.259 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:07.383+0000 7f90900fa700 1 -- 172.21.15.32:0/1018877121 >> 172.21.15.32:0/1018877121 conn(0x7f90880fc390 msgr2=0x7f90880fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:51:08.259 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:07.383+0000 7f90900fa700 1 -- 172.21.15.32:0/1018877121 shutdown_connections 2024-05-12T03:51:08.259 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:07.383+0000 7f90900fa700 1 -- 172.21.15.32:0/1018877121 wait complete. 2024-05-12T03:51:08.259 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:07.383+0000 7f90900fa700 1 Processor -- start 2024-05-12T03:51:08.259 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:07.383+0000 7f90900fa700 1 -- start start 2024-05-12T03:51:08.259 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:07.384+0000 7f90900fa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9088103a40 0x7f9088111340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:08.259 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:07.384+0000 7f90900fa700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9088111880 con 0x7f9088103a40 2024-05-12T03:51:08.259 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:07.384+0000 7f908de96700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9088103a40 0x7f9088111340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:51:08.259 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:07.384+0000 7f908de96700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9088103a40 0x7f9088111340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55530/0 (socket says 172.21.15.32:55530) 2024-05-12T03:51:08.259 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:07.384+0000 7f908de96700 1 -- 172.21.15.32:0/1410294144 learned_addr learned my addr 172.21.15.32:0/1410294144 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:51:08.259 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:07.384+0000 7f908de96700 1 -- 172.21.15.32:0/1410294144 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90840094f0 con 0x7f9088103a40 2024-05-12T03:51:08.259 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:07.384+0000 7f908de96700 1 --2- 172.21.15.32:0/1410294144 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9088103a40 0x7f9088111340 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f908400c010 tx=0x7f9084009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:51:08.259 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:07.384+0000 7f907e7fc700 1 -- 172.21.15.32:0/1410294144 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9084010960 con 0x7f9088103a40 2024-05-12T03:51:08.259 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:07.384+0000 7f90900fa700 1 -- 172.21.15.32:0/1410294144 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9088111a80 con 0x7f9088103a40 2024-05-12T03:51:08.260 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:07.384+0000 7f907e7fc700 1 -- 172.21.15.32:0/1410294144 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f908401a430 con 0x7f9088103a40 2024-05-12T03:51:08.260 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:07.384+0000 7f907e7fc700 1 -- 172.21.15.32:0/1410294144 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9084019580 con 0x7f9088103a40 2024-05-12T03:51:08.260 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:07.384+0000 7f90900fa700 1 -- 172.21.15.32:0/1410294144 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90881125f0 con 0x7f9088103a40 2024-05-12T03:51:08.260 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:07.385+0000 7f90900fa700 1 -- 172.21.15.32:0/1410294144 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f908805f5c0 con 0x7f9088103a40 2024-05-12T03:51:08.260 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:07.385+0000 7f907e7fc700 1 -- 172.21.15.32:0/1410294144 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f9084019800 con 0x7f9088103a40 2024-05-12T03:51:08.260 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:07.386+0000 7f907e7fc700 1 --2- 172.21.15.32:0/1410294144 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f9074036790 0x7f9074038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:08.260 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:07.386+0000 7f907e7fc700 1 -- 172.21.15.32:0/1410294144 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f908404a5c0 con 0x7f9088103a40 2024-05-12T03:51:08.260 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:07.386+0000 7f908d695700 1 --2- 172.21.15.32:0/1410294144 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f9074036790 0x7f9074038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:51:08.260 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:07.386+0000 7f908d695700 1 --2- 172.21.15.32:0/1410294144 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f9074036790 0x7f9074038c40 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9078009990 tx=0x7f9078006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:51:08.260 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:07.391+0000 7f907e7fc700 1 -- 172.21.15.32:0/1410294144 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f908402b900 con 0x7f9088103a40 2024-05-12T03:51:08.260 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:07.604+0000 7f90900fa700 1 -- 172.21.15.32:0/1410294144 --> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}) v1 -- 0x7f9088113170 con 0x7f9074036790 2024-05-12T03:51:08.260 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:07.608+0000 7f907e7fc700 1 -- 172.21.15.32:0/1410294144 <== mgr.14118 v2:172.21.15.32:6800/2967201790 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7f9088113170 con 0x7f9074036790 2024-05-12T03:51:08.260 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:07.610+0000 7f90900fa700 1 -- 172.21.15.32:0/1410294144 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f9074036790 msgr2=0x7f9074038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:08.260 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:07.610+0000 7f90900fa700 1 --2- 172.21.15.32:0/1410294144 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f9074036790 0x7f9074038c40 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9078009990 tx=0x7f9078006e40 comp rx=0 tx=0).stop 2024-05-12T03:51:08.260 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:07.610+0000 7f90900fa700 1 -- 172.21.15.32:0/1410294144 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9088103a40 msgr2=0x7f9088111340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:08.261 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:07.610+0000 7f90900fa700 1 --2- 172.21.15.32:0/1410294144 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9088103a40 0x7f9088111340 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f908400c010 tx=0x7f9084009250 comp rx=0 tx=0).stop 2024-05-12T03:51:08.261 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:07.610+0000 7f90900fa700 1 -- 172.21.15.32:0/1410294144 shutdown_connections 2024-05-12T03:51:08.261 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:07.610+0000 7f90900fa700 1 --2- 172.21.15.32:0/1410294144 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f9074036790 0x7f9074038c40 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:08.261 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:07.610+0000 7f90900fa700 1 --2- 172.21.15.32:0/1410294144 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9088103a40 0x7f9088111340 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:08.261 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:07.610+0000 7f90900fa700 1 -- 172.21.15.32:0/1410294144 >> 172.21.15.32:0/1410294144 conn(0x7f90880fc390 msgr2=0x7f9088108130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:51:08.261 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:07.610+0000 7f90900fa700 1 -- 172.21.15.32:0/1410294144 shutdown_connections 2024-05-12T03:51:08.261 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:07.611+0000 7f90900fa700 1 -- 172.21.15.32:0/1410294144 wait complete. 2024-05-12T03:51:08.261 INFO:teuthology.orchestra.run.smithi032.stdout:Deploying grafana service with default placement... 2024-05-12T03:51:08.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:08 smithi032 ceph-mon[40008]: from='mgr.14118 172.21.15.32:0/4199128226' entity='mgr.smithi032.ukatkq' 2024-05-12T03:51:08.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:08 smithi032 ceph-mon[40008]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T03:51:08.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:08 smithi032 ceph-mon[40008]: Saving service prometheus spec with placement count:1 2024-05-12T03:51:08.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:08 smithi032 ceph-mon[40008]: from='mgr.14118 172.21.15.32:0/4199128226' entity='mgr.smithi032.ukatkq' 2024-05-12T03:51:10.789 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:10 smithi032 ceph-mon[40008]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:51:11.322 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout Scheduled grafana update... 2024-05-12T03:51:11.328 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:10.267+0000 7fb2ccfbf700 1 Processor -- start 2024-05-12T03:51:11.328 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:10.267+0000 7fb2ccfbf700 1 -- start start 2024-05-12T03:51:11.328 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:10.267+0000 7fb2ccfbf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2c8101450 0x7fb2c8101860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:11.329 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:10.267+0000 7fb2ccfbf700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c8101da0 con 0x7fb2c8101450 2024-05-12T03:51:11.329 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:10.268+0000 7fb2c659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2c8101450 0x7fb2c8101860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:51:11.329 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:10.268+0000 7fb2c659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2c8101450 0x7fb2c8101860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55544/0 (socket says 172.21.15.32:55544) 2024-05-12T03:51:11.329 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:10.268+0000 7fb2c659c700 1 -- 172.21.15.32:0/1090691277 learned_addr learned my addr 172.21.15.32:0/1090691277 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:51:11.329 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:10.268+0000 7fb2c659c700 1 -- 172.21.15.32:0/1090691277 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c8101ee0 con 0x7fb2c8101450 2024-05-12T03:51:11.329 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:10.268+0000 7fb2c659c700 1 --2- 172.21.15.32:0/1090691277 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2c8101450 0x7fb2c8101860 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb2b800bb80 tx=0x7fb2b800b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63f46659f50752ca server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:51:11.329 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:10.268+0000 7fb2c559a700 1 -- 172.21.15.32:0/1090691277 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb2b800dd00 con 0x7fb2c8101450 2024-05-12T03:51:11.329 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:10.268+0000 7fb2c559a700 1 -- 172.21.15.32:0/1090691277 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb2b800de60 con 0x7fb2c8101450 2024-05-12T03:51:11.329 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:10.268+0000 7fb2c559a700 1 -- 172.21.15.32:0/1090691277 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb2b80188d0 con 0x7fb2c8101450 2024-05-12T03:51:11.329 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:10.269+0000 7fb2ccfbf700 1 -- 172.21.15.32:0/1090691277 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2c8101450 msgr2=0x7fb2c8101860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:11.329 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:10.269+0000 7fb2ccfbf700 1 --2- 172.21.15.32:0/1090691277 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2c8101450 0x7fb2c8101860 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb2b800bb80 tx=0x7fb2b800b4f0 comp rx=0 tx=0).stop 2024-05-12T03:51:11.329 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:10.269+0000 7fb2ccfbf700 1 -- 172.21.15.32:0/1090691277 shutdown_connections 2024-05-12T03:51:11.330 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:10.269+0000 7fb2ccfbf700 1 --2- 172.21.15.32:0/1090691277 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2c8101450 0x7fb2c8101860 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:11.330 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:10.269+0000 7fb2ccfbf700 1 -- 172.21.15.32:0/1090691277 >> 172.21.15.32:0/1090691277 conn(0x7fb2c80fcbf0 msgr2=0x7fb2c80ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:51:11.330 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:10.269+0000 7fb2ccfbf700 1 -- 172.21.15.32:0/1090691277 shutdown_connections 2024-05-12T03:51:11.330 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:10.269+0000 7fb2ccfbf700 1 -- 172.21.15.32:0/1090691277 wait complete. 2024-05-12T03:51:11.330 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:10.269+0000 7fb2ccfbf700 1 Processor -- start 2024-05-12T03:51:11.330 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:10.269+0000 7fb2ccfbf700 1 -- start start 2024-05-12T03:51:11.330 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:10.270+0000 7fb2ccfbf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2c8101450 0x7fb2c810f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:11.331 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:10.270+0000 7fb2ccfbf700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c810f6e0 con 0x7fb2c8101450 2024-05-12T03:51:11.331 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:10.270+0000 7fb2c659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2c8101450 0x7fb2c810f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:51:11.331 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:10.270+0000 7fb2c659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2c8101450 0x7fb2c810f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55560/0 (socket says 172.21.15.32:55560) 2024-05-12T03:51:11.331 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:10.270+0000 7fb2c659c700 1 -- 172.21.15.32:0/3089836847 learned_addr learned my addr 172.21.15.32:0/3089836847 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:51:11.331 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:10.270+0000 7fb2c659c700 1 -- 172.21.15.32:0/3089836847 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2b800b790 con 0x7fb2c8101450 2024-05-12T03:51:11.331 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:10.270+0000 7fb2c659c700 1 --2- 172.21.15.32:0/3089836847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2c8101450 0x7fb2c810f1a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8000c00 tx=0x7fb2b800b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:51:11.331 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:10.270+0000 7fb2b6ffd700 1 -- 172.21.15.32:0/3089836847 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb2b8010c00 con 0x7fb2c8101450 2024-05-12T03:51:11.331 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:10.270+0000 7fb2b6ffd700 1 -- 172.21.15.32:0/3089836847 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb2b801d040 con 0x7fb2c8101450 2024-05-12T03:51:11.331 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:10.270+0000 7fb2ccfbf700 1 -- 172.21.15.32:0/3089836847 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2c810f8e0 con 0x7fb2c8101450 2024-05-12T03:51:11.331 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:10.270+0000 7fb2b6ffd700 1 -- 172.21.15.32:0/3089836847 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb2b80225f0 con 0x7fb2c8101450 2024-05-12T03:51:11.331 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:10.270+0000 7fb2ccfbf700 1 -- 172.21.15.32:0/3089836847 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2c8110400 con 0x7fb2c8101450 2024-05-12T03:51:11.331 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:10.271+0000 7fb2b6ffd700 1 -- 172.21.15.32:0/3089836847 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7fb2b8022750 con 0x7fb2c8101450 2024-05-12T03:51:11.331 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:10.271+0000 7fb2b6ffd700 1 --2- 172.21.15.32:0/3089836847 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7fb2b00367e0 0x7fb2b0038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:11.332 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:10.271+0000 7fb2b6ffd700 1 -- 172.21.15.32:0/3089836847 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fb2b8014070 con 0x7fb2c8101450 2024-05-12T03:51:11.332 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:10.271+0000 7fb2ccfbf700 1 -- 172.21.15.32:0/3089836847 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2c805f5c0 con 0x7fb2c8101450 2024-05-12T03:51:11.332 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:10.272+0000 7fb2c5d9b700 1 --2- 172.21.15.32:0/3089836847 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7fb2b00367e0 0x7fb2b0038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:51:11.332 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:10.272+0000 7fb2c5d9b700 1 --2- 172.21.15.32:0/3089836847 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7fb2b00367e0 0x7fb2b0038c90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc009990 tx=0x7fb2bc006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:51:11.332 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:10.276+0000 7fb2b6ffd700 1 -- 172.21.15.32:0/3089836847 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb2b8028070 con 0x7fb2c8101450 2024-05-12T03:51:11.332 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:10.492+0000 7fb2ccfbf700 1 -- 172.21.15.32:0/3089836847 --> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}) v1 -- 0x7fb2c81106e0 con 0x7fb2b00367e0 2024-05-12T03:51:11.332 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:10.496+0000 7fb2b6ffd700 1 -- 172.21.15.32:0/3089836847 <== mgr.14118 v2:172.21.15.32:6800/2967201790 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7fb2c81106e0 con 0x7fb2b00367e0 2024-05-12T03:51:11.332 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:10.498+0000 7fb2ccfbf700 1 -- 172.21.15.32:0/3089836847 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7fb2b00367e0 msgr2=0x7fb2b0038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:11.332 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:10.498+0000 7fb2ccfbf700 1 --2- 172.21.15.32:0/3089836847 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7fb2b00367e0 0x7fb2b0038c90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc009990 tx=0x7fb2bc006e40 comp rx=0 tx=0).stop 2024-05-12T03:51:11.332 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:10.498+0000 7fb2ccfbf700 1 -- 172.21.15.32:0/3089836847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2c8101450 msgr2=0x7fb2c810f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:11.332 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:10.498+0000 7fb2ccfbf700 1 --2- 172.21.15.32:0/3089836847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2c8101450 0x7fb2c810f1a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8000c00 tx=0x7fb2b800b4f0 comp rx=0 tx=0).stop 2024-05-12T03:51:11.332 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:10.499+0000 7fb2ccfbf700 1 -- 172.21.15.32:0/3089836847 shutdown_connections 2024-05-12T03:51:11.333 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:10.499+0000 7fb2ccfbf700 1 --2- 172.21.15.32:0/3089836847 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7fb2b00367e0 0x7fb2b0038c90 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:11.333 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:10.499+0000 7fb2ccfbf700 1 --2- 172.21.15.32:0/3089836847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2c8101450 0x7fb2c810f1a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:11.333 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:10.499+0000 7fb2ccfbf700 1 -- 172.21.15.32:0/3089836847 >> 172.21.15.32:0/3089836847 conn(0x7fb2c80fcbf0 msgr2=0x7fb2c8105970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:51:11.333 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:10.499+0000 7fb2ccfbf700 1 -- 172.21.15.32:0/3089836847 shutdown_connections 2024-05-12T03:51:11.333 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:10.499+0000 7fb2ccfbf700 1 -- 172.21.15.32:0/3089836847 wait complete. 2024-05-12T03:51:11.333 INFO:teuthology.orchestra.run.smithi032.stdout:Deploying node-exporter service with default placement... 2024-05-12T03:51:11.583 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:11 smithi032 ceph-mon[40008]: from='client.14148 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T03:51:11.583 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:11 smithi032 ceph-mon[40008]: Saving service grafana spec with placement count:1 2024-05-12T03:51:11.583 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:11 smithi032 ceph-mon[40008]: from='mgr.14118 172.21.15.32:0/4199128226' entity='mgr.smithi032.ukatkq' 2024-05-12T03:51:12.777 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout Scheduled node-exporter update... 2024-05-12T03:51:12.777 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:11.916+0000 7f4e9e421700 1 Processor -- start 2024-05-12T03:51:12.777 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:11.916+0000 7f4e9e421700 1 -- start start 2024-05-12T03:51:12.777 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:11.916+0000 7f4e9e421700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e98076bb0 0x7f4e98105140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:12.777 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:11.916+0000 7f4e9e421700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e98105680 con 0x7f4e98076bb0 2024-05-12T03:51:12.777 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:11.916+0000 7f4e97fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e98076bb0 0x7f4e98105140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:51:12.778 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:11.916+0000 7f4e97fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e98076bb0 0x7f4e98105140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57794/0 (socket says 172.21.15.32:57794) 2024-05-12T03:51:12.778 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:11.916+0000 7f4e97fff700 1 -- 172.21.15.32:0/3433593432 learned_addr learned my addr 172.21.15.32:0/3433593432 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:51:12.778 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:11.917+0000 7f4e97fff700 1 -- 172.21.15.32:0/3433593432 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e981057c0 con 0x7f4e98076bb0 2024-05-12T03:51:12.778 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:11.917+0000 7f4e97fff700 1 --2- 172.21.15.32:0/3433593432 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e98076bb0 0x7f4e98105140 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c0098e0 tx=0x7f4e8c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7924dee5e2b9e4fe server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:51:12.778 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:11.917+0000 7f4e96ffd700 1 -- 172.21.15.32:0/3433593432 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4e8c00b860 con 0x7f4e98076bb0 2024-05-12T03:51:12.778 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:11.917+0000 7f4e96ffd700 1 -- 172.21.15.32:0/3433593432 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4e8c00be70 con 0x7f4e98076bb0 2024-05-12T03:51:12.778 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:11.917+0000 7f4e96ffd700 1 -- 172.21.15.32:0/3433593432 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4e8c010380 con 0x7f4e98076bb0 2024-05-12T03:51:12.778 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:11.917+0000 7f4e9e421700 1 -- 172.21.15.32:0/3433593432 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e98076bb0 msgr2=0x7f4e98105140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:12.778 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:11.917+0000 7f4e9e421700 1 --2- 172.21.15.32:0/3433593432 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e98076bb0 0x7f4e98105140 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c0098e0 tx=0x7f4e8c009250 comp rx=0 tx=0).stop 2024-05-12T03:51:12.778 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:11.918+0000 7f4e9e421700 1 -- 172.21.15.32:0/3433593432 shutdown_connections 2024-05-12T03:51:12.778 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:11.918+0000 7f4e9e421700 1 --2- 172.21.15.32:0/3433593432 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e98076bb0 0x7f4e98105140 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:12.779 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:11.918+0000 7f4e9e421700 1 -- 172.21.15.32:0/3433593432 >> 172.21.15.32:0/3433593432 conn(0x7f4e980fcb50 msgr2=0x7f4e980fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:51:12.779 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:11.918+0000 7f4e9e421700 1 -- 172.21.15.32:0/3433593432 shutdown_connections 2024-05-12T03:51:12.779 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:11.918+0000 7f4e9e421700 1 -- 172.21.15.32:0/3433593432 wait complete. 2024-05-12T03:51:12.779 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:11.918+0000 7f4e9e421700 1 Processor -- start 2024-05-12T03:51:12.779 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:11.918+0000 7f4e9e421700 1 -- start start 2024-05-12T03:51:12.779 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:11.919+0000 7f4e9e421700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e98076bb0 0x7f4e98102a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:12.779 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:11.919+0000 7f4e9e421700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e98102fd0 con 0x7f4e98076bb0 2024-05-12T03:51:12.779 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:11.919+0000 7f4e97fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e98076bb0 0x7f4e98102a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:51:12.779 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:11.919+0000 7f4e97fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e98076bb0 0x7f4e98102a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57804/0 (socket says 172.21.15.32:57804) 2024-05-12T03:51:12.779 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:11.919+0000 7f4e97fff700 1 -- 172.21.15.32:0/2446048483 learned_addr learned my addr 172.21.15.32:0/2446048483 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:51:12.779 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:11.919+0000 7f4e97fff700 1 -- 172.21.15.32:0/2446048483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e8c0094f0 con 0x7f4e98076bb0 2024-05-12T03:51:12.779 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:11.919+0000 7f4e97fff700 1 --2- 172.21.15.32:0/2446048483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e98076bb0 0x7f4e98102a90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c011b60 tx=0x7f4e8c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:51:12.779 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:11.919+0000 7f4e94ff9700 1 -- 172.21.15.32:0/2446048483 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4e8c010800 con 0x7f4e98076bb0 2024-05-12T03:51:12.779 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:11.919+0000 7f4e94ff9700 1 -- 172.21.15.32:0/2446048483 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4e8c010e10 con 0x7f4e98076bb0 2024-05-12T03:51:12.779 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:11.919+0000 7f4e9e421700 1 -- 172.21.15.32:0/2446048483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e98101040 con 0x7f4e98076bb0 2024-05-12T03:51:12.780 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:11.919+0000 7f4e94ff9700 1 -- 172.21.15.32:0/2446048483 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4e8c01ae60 con 0x7f4e98076bb0 2024-05-12T03:51:12.780 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:11.919+0000 7f4e9e421700 1 -- 172.21.15.32:0/2446048483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e98101530 con 0x7f4e98076bb0 2024-05-12T03:51:12.780 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:11.920+0000 7f4e94ff9700 1 -- 172.21.15.32:0/2446048483 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f4e8c01a430 con 0x7f4e98076bb0 2024-05-12T03:51:12.780 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:11.920+0000 7f4e9e421700 1 -- 172.21.15.32:0/2446048483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e9804bed0 con 0x7f4e98076bb0 2024-05-12T03:51:12.780 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:11.921+0000 7f4e94ff9700 1 --2- 172.21.15.32:0/2446048483 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f4e880367e0 0x7f4e88038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:12.780 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:11.921+0000 7f4e94ff9700 1 -- 172.21.15.32:0/2446048483 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f4e8c04a660 con 0x7f4e98076bb0 2024-05-12T03:51:12.780 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:11.921+0000 7f4e977fe700 1 --2- 172.21.15.32:0/2446048483 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f4e880367e0 0x7f4e88038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:51:12.780 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:11.921+0000 7f4e977fe700 1 --2- 172.21.15.32:0/2446048483 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f4e880367e0 0x7f4e88038c90 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4e84009990 tx=0x7f4e84006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:51:12.780 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:11.925+0000 7f4e94ff9700 1 -- 172.21.15.32:0/2446048483 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f4e8c02b950 con 0x7f4e98076bb0 2024-05-12T03:51:12.780 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:12.130+0000 7f4e9e421700 1 -- 172.21.15.32:0/2446048483 --> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7f4e98101c40 con 0x7f4e880367e0 2024-05-12T03:51:12.780 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:12.134+0000 7f4e94ff9700 1 -- 172.21.15.32:0/2446048483 <== mgr.14118 v2:172.21.15.32:6800/2967201790 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7f4e98101c40 con 0x7f4e880367e0 2024-05-12T03:51:12.780 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:12 smithi032 ceph-mon[40008]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:51:12.781 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:12 smithi032 ceph-mon[40008]: from='mgr.14118 172.21.15.32:0/4199128226' entity='mgr.smithi032.ukatkq' 2024-05-12T03:51:12.782 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:12.136+0000 7f4e9e421700 1 -- 172.21.15.32:0/2446048483 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f4e880367e0 msgr2=0x7f4e88038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:12.782 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:12.136+0000 7f4e9e421700 1 --2- 172.21.15.32:0/2446048483 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f4e880367e0 0x7f4e88038c90 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4e84009990 tx=0x7f4e84006e40 comp rx=0 tx=0).stop 2024-05-12T03:51:12.782 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:12.136+0000 7f4e9e421700 1 -- 172.21.15.32:0/2446048483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e98076bb0 msgr2=0x7f4e98102a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:12.783 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:12.136+0000 7f4e9e421700 1 --2- 172.21.15.32:0/2446048483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e98076bb0 0x7f4e98102a90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c011b60 tx=0x7f4e8c009250 comp rx=0 tx=0).stop 2024-05-12T03:51:12.783 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:12.137+0000 7f4e9e421700 1 -- 172.21.15.32:0/2446048483 shutdown_connections 2024-05-12T03:51:12.783 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:12.137+0000 7f4e9e421700 1 --2- 172.21.15.32:0/2446048483 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f4e880367e0 0x7f4e88038c90 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:12.783 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:12.137+0000 7f4e9e421700 1 --2- 172.21.15.32:0/2446048483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e98076bb0 0x7f4e98102a90 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:12.783 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:12.137+0000 7f4e9e421700 1 -- 172.21.15.32:0/2446048483 >> 172.21.15.32:0/2446048483 conn(0x7f4e980fcb50 msgr2=0x7f4e980fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:51:12.783 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:12.137+0000 7f4e9e421700 1 -- 172.21.15.32:0/2446048483 shutdown_connections 2024-05-12T03:51:12.783 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:12.137+0000 7f4e9e421700 1 -- 172.21.15.32:0/2446048483 wait complete. 2024-05-12T03:51:12.783 INFO:teuthology.orchestra.run.smithi032.stdout:Deploying alertmanager service with default placement... 2024-05-12T03:51:13.789 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:13 smithi032 ceph-mon[40008]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T03:51:13.789 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:13 smithi032 ceph-mon[40008]: Saving service node-exporter spec with placement * 2024-05-12T03:51:14.218 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout Scheduled alertmanager update... 2024-05-12T03:51:14.218 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:13.367+0000 7f0fbb0c6700 1 Processor -- start 2024-05-12T03:51:14.218 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:13.367+0000 7f0fbb0c6700 1 -- start start 2024-05-12T03:51:14.219 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:13.367+0000 7f0fbb0c6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fb4076bd0 0x7f0fb4105190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:14.219 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:13.367+0000 7f0fbb0c6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb41056d0 con 0x7f0fb4076bd0 2024-05-12T03:51:14.219 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:13.367+0000 7f0fb8e62700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fb4076bd0 0x7f0fb4105190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:51:14.219 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:13.367+0000 7f0fb8e62700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fb4076bd0 0x7f0fb4105190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57810/0 (socket says 172.21.15.32:57810) 2024-05-12T03:51:14.219 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:13.367+0000 7f0fb8e62700 1 -- 172.21.15.32:0/3855646538 learned_addr learned my addr 172.21.15.32:0/3855646538 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:51:14.219 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:13.367+0000 7f0fb8e62700 1 -- 172.21.15.32:0/3855646538 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fb4105810 con 0x7f0fb4076bd0 2024-05-12T03:51:14.219 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:13.368+0000 7f0fb8e62700 1 --2- 172.21.15.32:0/3855646538 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fb4076bd0 0x7f0fb4105190 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0fa80098e0 tx=0x7f0fa8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1cc977176a54cd1 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:51:14.219 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:13.368+0000 7f0fb37fe700 1 -- 172.21.15.32:0/3855646538 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0fa800b860 con 0x7f0fb4076bd0 2024-05-12T03:51:14.219 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:13.368+0000 7f0fb37fe700 1 -- 172.21.15.32:0/3855646538 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0fa800be70 con 0x7f0fb4076bd0 2024-05-12T03:51:14.219 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:13.368+0000 7f0fb37fe700 1 -- 172.21.15.32:0/3855646538 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0fa8010380 con 0x7f0fb4076bd0 2024-05-12T03:51:14.219 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:13.368+0000 7f0fbb0c6700 1 -- 172.21.15.32:0/3855646538 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fb4076bd0 msgr2=0x7f0fb4105190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:14.219 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:13.368+0000 7f0fbb0c6700 1 --2- 172.21.15.32:0/3855646538 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fb4076bd0 0x7f0fb4105190 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0fa80098e0 tx=0x7f0fa8009250 comp rx=0 tx=0).stop 2024-05-12T03:51:14.219 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:13.368+0000 7f0fbb0c6700 1 -- 172.21.15.32:0/3855646538 shutdown_connections 2024-05-12T03:51:14.219 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:13.368+0000 7f0fbb0c6700 1 --2- 172.21.15.32:0/3855646538 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fb4076bd0 0x7f0fb4105190 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:14.220 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:13.368+0000 7f0fbb0c6700 1 -- 172.21.15.32:0/3855646538 >> 172.21.15.32:0/3855646538 conn(0x7f0fb40fcbc0 msgr2=0x7f0fb40ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:51:14.220 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:13.368+0000 7f0fbb0c6700 1 -- 172.21.15.32:0/3855646538 shutdown_connections 2024-05-12T03:51:14.220 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:13.368+0000 7f0fbb0c6700 1 -- 172.21.15.32:0/3855646538 wait complete. 2024-05-12T03:51:14.220 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:13.369+0000 7f0fbb0c6700 1 Processor -- start 2024-05-12T03:51:14.220 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:13.369+0000 7f0fbb0c6700 1 -- start start 2024-05-12T03:51:14.220 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:13.369+0000 7f0fbb0c6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fb4076bd0 0x7f0fb4102ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:14.220 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:13.369+0000 7f0fbb0c6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb4103020 con 0x7f0fb4076bd0 2024-05-12T03:51:14.220 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:13.369+0000 7f0fb8e62700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fb4076bd0 0x7f0fb4102ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:51:14.220 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:13.369+0000 7f0fb8e62700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fb4076bd0 0x7f0fb4102ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57812/0 (socket says 172.21.15.32:57812) 2024-05-12T03:51:14.220 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:13.369+0000 7f0fb8e62700 1 -- 172.21.15.32:0/2752160486 learned_addr learned my addr 172.21.15.32:0/2752160486 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:51:14.220 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:13.370+0000 7f0fb8e62700 1 -- 172.21.15.32:0/2752160486 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fa80094f0 con 0x7f0fb4076bd0 2024-05-12T03:51:14.220 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:13.370+0000 7f0fb8e62700 1 --2- 172.21.15.32:0/2752160486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fb4076bd0 0x7f0fb4102ae0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8011ca0 tx=0x7f0fa8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:51:14.220 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:13.370+0000 7f0fb17fa700 1 -- 172.21.15.32:0/2752160486 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0fa8010800 con 0x7f0fb4076bd0 2024-05-12T03:51:14.220 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:13.370+0000 7f0fb17fa700 1 -- 172.21.15.32:0/2752160486 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0fa8010e10 con 0x7f0fb4076bd0 2024-05-12T03:51:14.221 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:13.370+0000 7f0fbb0c6700 1 -- 172.21.15.32:0/2752160486 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fb4101090 con 0x7f0fb4076bd0 2024-05-12T03:51:14.221 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:13.370+0000 7f0fb17fa700 1 -- 172.21.15.32:0/2752160486 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0fa801aea0 con 0x7f0fb4076bd0 2024-05-12T03:51:14.221 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:13.370+0000 7f0fbb0c6700 1 -- 172.21.15.32:0/2752160486 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fb4101580 con 0x7f0fb4076bd0 2024-05-12T03:51:14.221 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:13.371+0000 7f0fb17fa700 1 -- 172.21.15.32:0/2752160486 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f0fa8019410 con 0x7f0fb4076bd0 2024-05-12T03:51:14.221 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:13.371+0000 7f0fb17fa700 1 --2- 172.21.15.32:0/2752160486 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f0fa4036830 0x7f0fa4038ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:14.221 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:13.371+0000 7f0fb17fa700 1 -- 172.21.15.32:0/2752160486 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f0fa801a780 con 0x7f0fb4076bd0 2024-05-12T03:51:14.221 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:13.371+0000 7f0fb3fff700 1 --2- 172.21.15.32:0/2752160486 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f0fa4036830 0x7f0fa4038ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:51:14.221 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:13.371+0000 7f0fbb0c6700 1 -- 172.21.15.32:0/2752160486 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fb404bed0 con 0x7f0fb4076bd0 2024-05-12T03:51:14.221 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:13.372+0000 7f0fb3fff700 1 --2- 172.21.15.32:0/2752160486 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f0fa4036830 0x7f0fa4038ce0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0009990 tx=0x7f0fa0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:51:14.221 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:13.376+0000 7f0fb17fa700 1 -- 172.21.15.32:0/2752160486 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f0fa8019710 con 0x7f0fb4076bd0 2024-05-12T03:51:14.221 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:13.581+0000 7f0fbb0c6700 1 -- 172.21.15.32:0/2752160486 --> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}) v1 -- 0x7f0fb4101c10 con 0x7f0fa4036830 2024-05-12T03:51:14.221 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:13.585+0000 7f0fb17fa700 1 -- 172.21.15.32:0/2752160486 <== mgr.14118 v2:172.21.15.32:6800/2967201790 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7f0fb4101c10 con 0x7f0fa4036830 2024-05-12T03:51:14.221 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:13.587+0000 7f0fbb0c6700 1 -- 172.21.15.32:0/2752160486 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f0fa4036830 msgr2=0x7f0fa4038ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:14.222 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:13.587+0000 7f0fbb0c6700 1 --2- 172.21.15.32:0/2752160486 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f0fa4036830 0x7f0fa4038ce0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0009990 tx=0x7f0fa0006e40 comp rx=0 tx=0).stop 2024-05-12T03:51:14.222 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:13.587+0000 7f0fbb0c6700 1 -- 172.21.15.32:0/2752160486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fb4076bd0 msgr2=0x7f0fb4102ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:14.222 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:13.587+0000 7f0fbb0c6700 1 --2- 172.21.15.32:0/2752160486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fb4076bd0 0x7f0fb4102ae0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8011ca0 tx=0x7f0fa8009250 comp rx=0 tx=0).stop 2024-05-12T03:51:14.222 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:13.587+0000 7f0fbb0c6700 1 -- 172.21.15.32:0/2752160486 shutdown_connections 2024-05-12T03:51:14.222 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:13.587+0000 7f0fbb0c6700 1 --2- 172.21.15.32:0/2752160486 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f0fa4036830 0x7f0fa4038ce0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:14.222 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:13.587+0000 7f0fbb0c6700 1 --2- 172.21.15.32:0/2752160486 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fb4076bd0 0x7f0fb4102ae0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:14.222 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:13.587+0000 7f0fbb0c6700 1 -- 172.21.15.32:0/2752160486 >> 172.21.15.32:0/2752160486 conn(0x7f0fb40fcbc0 msgr2=0x7f0fb40fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:51:14.222 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:13.587+0000 7f0fbb0c6700 1 -- 172.21.15.32:0/2752160486 shutdown_connections 2024-05-12T03:51:14.222 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:13.587+0000 7f0fbb0c6700 1 -- 172.21.15.32:0/2752160486 wait complete. 2024-05-12T03:51:14.789 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:14 smithi032 ceph-mon[40008]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:51:14.789 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:14 smithi032 ceph-mon[40008]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T03:51:14.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:14 smithi032 ceph-mon[40008]: Saving service alertmanager spec with placement count:1 2024-05-12T03:51:14.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:14 smithi032 ceph-mon[40008]: from='mgr.14118 172.21.15.32:0/4199128226' entity='mgr.smithi032.ukatkq' 2024-05-12T03:51:15.616 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:14.788+0000 7fc57654d700 1 Processor -- start 2024-05-12T03:51:15.616 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:14.788+0000 7fc57654d700 1 -- start start 2024-05-12T03:51:15.616 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:14.788+0000 7fc57654d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc570076c50 0x7fc5701051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:15.616 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:14.788+0000 7fc57654d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc570105700 con 0x7fc570076c50 2024-05-12T03:51:15.616 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:14.789+0000 7fc56ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc570076c50 0x7fc5701051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:51:15.616 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:14.789+0000 7fc56ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc570076c50 0x7fc5701051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57828/0 (socket says 172.21.15.32:57828) 2024-05-12T03:51:15.616 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:14.789+0000 7fc56ffff700 1 -- 172.21.15.32:0/4294238540 learned_addr learned my addr 172.21.15.32:0/4294238540 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:51:15.616 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:14.789+0000 7fc56ffff700 1 -- 172.21.15.32:0/4294238540 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc570105840 con 0x7fc570076c50 2024-05-12T03:51:15.616 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:14.789+0000 7fc56ffff700 1 --2- 172.21.15.32:0/4294238540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc570076c50 0x7fc5701051c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc5600098e0 tx=0x7fc560009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47f29ad5f15d945 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:51:15.616 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:14.789+0000 7fc56effd700 1 -- 172.21.15.32:0/4294238540 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc56000b860 con 0x7fc570076c50 2024-05-12T03:51:15.616 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:14.789+0000 7fc56effd700 1 -- 172.21.15.32:0/4294238540 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc56000be70 con 0x7fc570076c50 2024-05-12T03:51:15.616 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:14.789+0000 7fc56effd700 1 -- 172.21.15.32:0/4294238540 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc560010380 con 0x7fc570076c50 2024-05-12T03:51:15.616 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:14.790+0000 7fc57654d700 1 -- 172.21.15.32:0/4294238540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc570076c50 msgr2=0x7fc5701051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:15.617 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:14.790+0000 7fc57654d700 1 --2- 172.21.15.32:0/4294238540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc570076c50 0x7fc5701051c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc5600098e0 tx=0x7fc560009250 comp rx=0 tx=0).stop 2024-05-12T03:51:15.617 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:14.790+0000 7fc57654d700 1 -- 172.21.15.32:0/4294238540 shutdown_connections 2024-05-12T03:51:15.617 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:14.790+0000 7fc57654d700 1 --2- 172.21.15.32:0/4294238540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc570076c50 0x7fc5701051c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:15.617 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:14.790+0000 7fc57654d700 1 -- 172.21.15.32:0/4294238540 >> 172.21.15.32:0/4294238540 conn(0x7fc5700fcc50 msgr2=0x7fc5700ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:51:15.617 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:14.790+0000 7fc57654d700 1 -- 172.21.15.32:0/4294238540 shutdown_connections 2024-05-12T03:51:15.617 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:14.790+0000 7fc57654d700 1 -- 172.21.15.32:0/4294238540 wait complete. 2024-05-12T03:51:15.617 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:14.790+0000 7fc57654d700 1 Processor -- start 2024-05-12T03:51:15.617 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:14.791+0000 7fc57654d700 1 -- start start 2024-05-12T03:51:15.617 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:14.791+0000 7fc57654d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc570076c50 0x7fc570102b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:15.617 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:14.791+0000 7fc57654d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc570103050 con 0x7fc570076c50 2024-05-12T03:51:15.617 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:14.791+0000 7fc56ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc570076c50 0x7fc570102b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:51:15.617 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:14.791+0000 7fc56ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc570076c50 0x7fc570102b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57842/0 (socket says 172.21.15.32:57842) 2024-05-12T03:51:15.617 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:14.791+0000 7fc56ffff700 1 -- 172.21.15.32:0/665272889 learned_addr learned my addr 172.21.15.32:0/665272889 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:51:15.617 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:14.791+0000 7fc56ffff700 1 -- 172.21.15.32:0/665272889 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5600094f0 con 0x7fc570076c50 2024-05-12T03:51:15.618 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:14.791+0000 7fc56ffff700 1 --2- 172.21.15.32:0/665272889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc570076c50 0x7fc570102b10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc5600094c0 tx=0x7fc560009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:51:15.618 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:14.792+0000 7fc56cff9700 1 -- 172.21.15.32:0/665272889 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc560010850 con 0x7fc570076c50 2024-05-12T03:51:15.618 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:14.792+0000 7fc56cff9700 1 -- 172.21.15.32:0/665272889 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc560010e60 con 0x7fc570076c50 2024-05-12T03:51:15.618 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:14.792+0000 7fc57654d700 1 -- 172.21.15.32:0/665272889 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5701010c0 con 0x7fc570076c50 2024-05-12T03:51:15.618 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:14.792+0000 7fc56cff9700 1 -- 172.21.15.32:0/665272889 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc56001a430 con 0x7fc570076c50 2024-05-12T03:51:15.618 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:14.792+0000 7fc57654d700 1 -- 172.21.15.32:0/665272889 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5701015b0 con 0x7fc570076c50 2024-05-12T03:51:15.618 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:14.793+0000 7fc57654d700 1 -- 172.21.15.32:0/665272889 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc57004bed0 con 0x7fc570076c50 2024-05-12T03:51:15.618 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:14.793+0000 7fc56cff9700 1 -- 172.21.15.32:0/665272889 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7fc5600247b0 con 0x7fc570076c50 2024-05-12T03:51:15.618 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:14.793+0000 7fc56cff9700 1 --2- 172.21.15.32:0/665272889 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7fc55c036790 0x7fc55c038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:15.618 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:14.793+0000 7fc56cff9700 1 -- 172.21.15.32:0/665272889 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fc56004a830 con 0x7fc570076c50 2024-05-12T03:51:15.618 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:14.793+0000 7fc56f7fe700 1 --2- 172.21.15.32:0/665272889 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7fc55c036790 0x7fc55c038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:51:15.618 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:14.794+0000 7fc56f7fe700 1 --2- 172.21.15.32:0/665272889 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7fc55c036790 0x7fc55c038c40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc558009990 tx=0x7fc558006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:51:15.618 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:14.797+0000 7fc56cff9700 1 -- 172.21.15.32:0/665272889 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fc560022520 con 0x7fc570076c50 2024-05-12T03:51:15.619 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:14.992+0000 7fc57654d700 1 -- 172.21.15.32:0/665272889 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7fc5701065c0 con 0x7fc570076c50 2024-05-12T03:51:15.619 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:14.995+0000 7fc56cff9700 1 -- 172.21.15.32:0/665272889 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7fc560022340 con 0x7fc570076c50 2024-05-12T03:51:15.619 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:14.997+0000 7fc57654d700 1 -- 172.21.15.32:0/665272889 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7fc55c036790 msgr2=0x7fc55c038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:15.619 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:14.997+0000 7fc57654d700 1 --2- 172.21.15.32:0/665272889 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7fc55c036790 0x7fc55c038c40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc558009990 tx=0x7fc558006e40 comp rx=0 tx=0).stop 2024-05-12T03:51:15.619 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:14.997+0000 7fc57654d700 1 -- 172.21.15.32:0/665272889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc570076c50 msgr2=0x7fc570102b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:15.619 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:14.997+0000 7fc57654d700 1 --2- 172.21.15.32:0/665272889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc570076c50 0x7fc570102b10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc5600094c0 tx=0x7fc560009250 comp rx=0 tx=0).stop 2024-05-12T03:51:15.619 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:14.997+0000 7fc57654d700 1 -- 172.21.15.32:0/665272889 shutdown_connections 2024-05-12T03:51:15.619 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:14.997+0000 7fc57654d700 1 --2- 172.21.15.32:0/665272889 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7fc55c036790 0x7fc55c038c40 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:15.619 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:14.997+0000 7fc57654d700 1 --2- 172.21.15.32:0/665272889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc570076c50 0x7fc570102b10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:15.619 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:14.997+0000 7fc57654d700 1 -- 172.21.15.32:0/665272889 >> 172.21.15.32:0/665272889 conn(0x7fc5700fcc50 msgr2=0x7fc5700fd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:51:15.619 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:14.997+0000 7fc57654d700 1 -- 172.21.15.32:0/665272889 shutdown_connections 2024-05-12T03:51:15.619 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:14.997+0000 7fc57654d700 1 -- 172.21.15.32:0/665272889 wait complete. 2024-05-12T03:51:16.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:15 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/665272889' entity='client.admin' 2024-05-12T03:51:16.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:15 smithi032 ceph-mon[40008]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:51:16.999 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:16.234+0000 7f726d265700 1 Processor -- start 2024-05-12T03:51:16.999 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:16.234+0000 7f726d265700 1 -- start start 2024-05-12T03:51:16.999 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:16.234+0000 7f726d265700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7268078c40 0x7f7268077240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:16.999 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:16.234+0000 7f726d265700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7268079050 con 0x7f7268078c40 2024-05-12T03:51:16.999 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:16.234+0000 7f7266ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7268078c40 0x7f7268077240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:51:16.999 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:16.234+0000 7f7266ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7268078c40 0x7f7268077240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57858/0 (socket says 172.21.15.32:57858) 2024-05-12T03:51:16.999 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:16.234+0000 7f7266ffd700 1 -- 172.21.15.32:0/4259830015 learned_addr learned my addr 172.21.15.32:0/4259830015 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:51:16.999 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:16.235+0000 7f7266ffd700 1 -- 172.21.15.32:0/4259830015 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7268077780 con 0x7f7268078c40 2024-05-12T03:51:16.999 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:16.235+0000 7f7266ffd700 1 --2- 172.21.15.32:0/4259830015 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7268078c40 0x7f7268077240 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f72580098e0 tx=0x7f7258009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b404a41ee95613ce server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:51:16.999 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:16.235+0000 7f7265ffb700 1 -- 172.21.15.32:0/4259830015 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f725800b860 con 0x7f7268078c40 2024-05-12T03:51:16.999 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:16.235+0000 7f7265ffb700 1 -- 172.21.15.32:0/4259830015 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f725800be70 con 0x7f7268078c40 2024-05-12T03:51:17.000 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:16.235+0000 7f7265ffb700 1 -- 172.21.15.32:0/4259830015 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7258010380 con 0x7f7268078c40 2024-05-12T03:51:17.000 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:16.235+0000 7f726d265700 1 -- 172.21.15.32:0/4259830015 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7268078c40 msgr2=0x7f7268077240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:17.000 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:16.235+0000 7f726d265700 1 --2- 172.21.15.32:0/4259830015 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7268078c40 0x7f7268077240 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f72580098e0 tx=0x7f7258009250 comp rx=0 tx=0).stop 2024-05-12T03:51:17.000 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:16.235+0000 7f726d265700 1 -- 172.21.15.32:0/4259830015 shutdown_connections 2024-05-12T03:51:17.000 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:16.235+0000 7f726d265700 1 --2- 172.21.15.32:0/4259830015 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7268078c40 0x7f7268077240 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:17.000 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:16.235+0000 7f726d265700 1 -- 172.21.15.32:0/4259830015 >> 172.21.15.32:0/4259830015 conn(0x7f72680fdd70 msgr2=0x7f7268100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:51:17.000 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:16.236+0000 7f726d265700 1 -- 172.21.15.32:0/4259830015 shutdown_connections 2024-05-12T03:51:17.000 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:16.236+0000 7f726d265700 1 -- 172.21.15.32:0/4259830015 wait complete. 2024-05-12T03:51:17.000 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:16.236+0000 7f726d265700 1 Processor -- start 2024-05-12T03:51:17.000 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:16.236+0000 7f726d265700 1 -- start start 2024-05-12T03:51:17.000 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:16.236+0000 7f726d265700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7268078c40 0x7f7268113580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:17.000 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:16.236+0000 7f726d265700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7268113ac0 con 0x7f7268078c40 2024-05-12T03:51:17.000 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:16.237+0000 7f7266ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7268078c40 0x7f7268113580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:51:17.000 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:16.237+0000 7f7266ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7268078c40 0x7f7268113580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57862/0 (socket says 172.21.15.32:57862) 2024-05-12T03:51:17.000 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:16.237+0000 7f7266ffd700 1 -- 172.21.15.32:0/3583054309 learned_addr learned my addr 172.21.15.32:0/3583054309 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:51:17.001 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:16.237+0000 7f7266ffd700 1 -- 172.21.15.32:0/3583054309 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72580094f0 con 0x7f7268078c40 2024-05-12T03:51:17.001 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:16.237+0000 7f7266ffd700 1 --2- 172.21.15.32:0/3583054309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7268078c40 0x7f7268113580 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f72580094c0 tx=0x7f7258009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:51:17.001 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:16.237+0000 7f724f7fe700 1 -- 172.21.15.32:0/3583054309 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7258010850 con 0x7f7268078c40 2024-05-12T03:51:17.001 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:16.237+0000 7f726d265700 1 -- 172.21.15.32:0/3583054309 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7268113cc0 con 0x7f7268078c40 2024-05-12T03:51:17.001 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:16.237+0000 7f724f7fe700 1 -- 172.21.15.32:0/3583054309 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7258010e60 con 0x7f7268078c40 2024-05-12T03:51:17.001 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:16.237+0000 7f724f7fe700 1 -- 172.21.15.32:0/3583054309 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f725801a430 con 0x7f7268078c40 2024-05-12T03:51:17.001 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:16.237+0000 7f726d265700 1 -- 172.21.15.32:0/3583054309 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7268114830 con 0x7f7268078c40 2024-05-12T03:51:17.001 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:16.238+0000 7f724f7fe700 1 -- 172.21.15.32:0/3583054309 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f725801a6b0 con 0x7f7268078c40 2024-05-12T03:51:17.001 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:16.238+0000 7f726d265700 1 -- 172.21.15.32:0/3583054309 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f726805f5c0 con 0x7f7268078c40 2024-05-12T03:51:17.001 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:16.238+0000 7f724f7fe700 1 --2- 172.21.15.32:0/3583054309 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f7250036790 0x7f7250038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:17.001 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:16.238+0000 7f724f7fe700 1 -- 172.21.15.32:0/3583054309 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f725804a5d0 con 0x7f7268078c40 2024-05-12T03:51:17.001 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:16.239+0000 7f72667fc700 1 --2- 172.21.15.32:0/3583054309 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f7250036790 0x7f7250038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:51:17.001 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:16.239+0000 7f72667fc700 1 --2- 172.21.15.32:0/3583054309 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f7250036790 0x7f7250038c40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f725c009990 tx=0x7f725c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:51:17.001 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:16.243+0000 7f724f7fe700 1 -- 172.21.15.32:0/3583054309 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f7258022520 con 0x7f7268078c40 2024-05-12T03:51:17.002 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:16.437+0000 7f726d265700 1 -- 172.21.15.32:0/3583054309 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f726804bed0 con 0x7f7268078c40 2024-05-12T03:51:17.002 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:16.440+0000 7f724f7fe700 1 -- 172.21.15.32:0/3583054309 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f7258022340 con 0x7f7268078c40 2024-05-12T03:51:17.002 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:16.442+0000 7f726d265700 1 -- 172.21.15.32:0/3583054309 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f7250036790 msgr2=0x7f7250038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:17.002 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:16.442+0000 7f726d265700 1 --2- 172.21.15.32:0/3583054309 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f7250036790 0x7f7250038c40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f725c009990 tx=0x7f725c006e40 comp rx=0 tx=0).stop 2024-05-12T03:51:17.002 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:16.442+0000 7f726d265700 1 -- 172.21.15.32:0/3583054309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7268078c40 msgr2=0x7f7268113580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:17.002 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:16.442+0000 7f726d265700 1 --2- 172.21.15.32:0/3583054309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7268078c40 0x7f7268113580 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f72580094c0 tx=0x7f7258009250 comp rx=0 tx=0).stop 2024-05-12T03:51:17.002 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:16.442+0000 7f726d265700 1 -- 172.21.15.32:0/3583054309 shutdown_connections 2024-05-12T03:51:17.002 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:16.442+0000 7f726d265700 1 --2- 172.21.15.32:0/3583054309 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f7250036790 0x7f7250038c40 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:17.002 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:16.442+0000 7f726d265700 1 --2- 172.21.15.32:0/3583054309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7268078c40 0x7f7268113580 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:17.002 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:16.442+0000 7f726d265700 1 -- 172.21.15.32:0/3583054309 >> 172.21.15.32:0/3583054309 conn(0x7f72680fdd70 msgr2=0x7f72680fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:51:17.002 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:16.443+0000 7f726d265700 1 -- 172.21.15.32:0/3583054309 shutdown_connections 2024-05-12T03:51:17.003 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:16.443+0000 7f726d265700 1 -- 172.21.15.32:0/3583054309 wait complete. 2024-05-12T03:51:17.003 INFO:teuthology.orchestra.run.smithi032.stdout:Enabling the dashboard module... 2024-05-12T03:51:17.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:17 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/3583054309' entity='client.admin' 2024-05-12T03:51:18.789 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:18 smithi032 ceph-mon[40008]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:51:18.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:18 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/2741972314' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-12T03:51:19.131 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:17.546+0000 7ff51dfe8700 1 Processor -- start 2024-05-12T03:51:19.132 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:17.546+0000 7ff51dfe8700 1 -- start start 2024-05-12T03:51:19.132 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:17.546+0000 7ff51dfe8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff518101490 0x7ff5181018a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:19.132 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:17.546+0000 7ff51dfe8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff518101de0 con 0x7ff518101490 2024-05-12T03:51:19.132 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:17.546+0000 7ff5177fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff518101490 0x7ff5181018a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:51:19.132 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:17.546+0000 7ff5177fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff518101490 0x7ff5181018a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57870/0 (socket says 172.21.15.32:57870) 2024-05-12T03:51:19.132 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:17.546+0000 7ff5177fe700 1 -- 172.21.15.32:0/3755692814 learned_addr learned my addr 172.21.15.32:0/3755692814 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:51:19.132 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:17.546+0000 7ff5177fe700 1 -- 172.21.15.32:0/3755692814 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff518101f20 con 0x7ff518101490 2024-05-12T03:51:19.132 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:17.547+0000 7ff5177fe700 1 --2- 172.21.15.32:0/3755692814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff518101490 0x7ff5181018a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff5080098e0 tx=0x7ff508009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a585c7634db60297 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:51:19.132 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:17.547+0000 7ff5167fc700 1 -- 172.21.15.32:0/3755692814 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff50800b860 con 0x7ff518101490 2024-05-12T03:51:19.132 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:17.547+0000 7ff5167fc700 1 -- 172.21.15.32:0/3755692814 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff50800be70 con 0x7ff518101490 2024-05-12T03:51:19.132 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:17.547+0000 7ff5167fc700 1 -- 172.21.15.32:0/3755692814 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff508010380 con 0x7ff518101490 2024-05-12T03:51:19.132 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:17.547+0000 7ff51dfe8700 1 -- 172.21.15.32:0/3755692814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff518101490 msgr2=0x7ff5181018a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:19.133 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:17.547+0000 7ff51dfe8700 1 --2- 172.21.15.32:0/3755692814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff518101490 0x7ff5181018a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff5080098e0 tx=0x7ff508009250 comp rx=0 tx=0).stop 2024-05-12T03:51:19.133 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:17.547+0000 7ff51dfe8700 1 -- 172.21.15.32:0/3755692814 shutdown_connections 2024-05-12T03:51:19.133 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:17.547+0000 7ff51dfe8700 1 --2- 172.21.15.32:0/3755692814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff518101490 0x7ff5181018a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:19.133 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:17.547+0000 7ff51dfe8700 1 -- 172.21.15.32:0/3755692814 >> 172.21.15.32:0/3755692814 conn(0x7ff5180fcbf0 msgr2=0x7ff5180ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:51:19.133 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:17.547+0000 7ff51dfe8700 1 -- 172.21.15.32:0/3755692814 shutdown_connections 2024-05-12T03:51:19.133 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:17.547+0000 7ff51dfe8700 1 -- 172.21.15.32:0/3755692814 wait complete. 2024-05-12T03:51:19.133 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:17.548+0000 7ff51dfe8700 1 Processor -- start 2024-05-12T03:51:19.133 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:17.548+0000 7ff51dfe8700 1 -- start start 2024-05-12T03:51:19.133 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:17.549+0000 7ff51dfe8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff518101490 0x7ff518076b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:19.133 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:17.549+0000 7ff51dfe8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5181140a0 con 0x7ff518101490 2024-05-12T03:51:19.133 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:17.549+0000 7ff5177fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff518101490 0x7ff518076b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:51:19.133 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:17.549+0000 7ff5177fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff518101490 0x7ff518076b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57880/0 (socket says 172.21.15.32:57880) 2024-05-12T03:51:19.133 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:17.549+0000 7ff5177fe700 1 -- 172.21.15.32:0/2741972314 learned_addr learned my addr 172.21.15.32:0/2741972314 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:51:19.133 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:17.549+0000 7ff5177fe700 1 -- 172.21.15.32:0/2741972314 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5080094f0 con 0x7ff518101490 2024-05-12T03:51:19.133 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:17.549+0000 7ff5177fe700 1 --2- 172.21.15.32:0/2741972314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff518101490 0x7ff518076b10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff50800c010 tx=0x7ff508009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:51:19.134 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:17.549+0000 7ff4fffff700 1 -- 172.21.15.32:0/2741972314 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff508010960 con 0x7ff518101490 2024-05-12T03:51:19.134 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:17.549+0000 7ff4fffff700 1 -- 172.21.15.32:0/2741972314 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff50801a430 con 0x7ff518101490 2024-05-12T03:51:19.134 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:17.549+0000 7ff51dfe8700 1 -- 172.21.15.32:0/2741972314 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5181141e0 con 0x7ff518101490 2024-05-12T03:51:19.134 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:17.549+0000 7ff4fffff700 1 -- 172.21.15.32:0/2741972314 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff508019580 con 0x7ff518101490 2024-05-12T03:51:19.134 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:17.549+0000 7ff51dfe8700 1 -- 172.21.15.32:0/2741972314 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff518114670 con 0x7ff518101490 2024-05-12T03:51:19.134 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:17.550+0000 7ff4fffff700 1 -- 172.21.15.32:0/2741972314 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7ff508019800 con 0x7ff518101490 2024-05-12T03:51:19.134 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:17.550+0000 7ff4fffff700 1 --2- 172.21.15.32:0/2741972314 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7ff500036790 0x7ff500038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:19.134 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:17.550+0000 7ff51dfe8700 1 -- 172.21.15.32:0/2741972314 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff51804bed0 con 0x7ff518101490 2024-05-12T03:51:19.134 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:17.551+0000 7ff4fffff700 1 -- 172.21.15.32:0/2741972314 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7ff50804a5b0 con 0x7ff518101490 2024-05-12T03:51:19.134 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:17.551+0000 7ff516ffd700 1 --2- 172.21.15.32:0/2741972314 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7ff500036790 0x7ff500038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:51:19.134 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:17.551+0000 7ff516ffd700 1 --2- 172.21.15.32:0/2741972314 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7ff500036790 0x7ff500038c40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff50c009990 tx=0x7ff50c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:51:19.134 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:17.555+0000 7ff4fffff700 1 -- 172.21.15.32:0/2741972314 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ff50802bb30 con 0x7ff518101490 2024-05-12T03:51:19.134 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:17.806+0000 7ff51dfe8700 1 -- 172.21.15.32:0/2741972314 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7ff5181031d0 con 0x7ff518101490 2024-05-12T03:51:19.135 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:18.444+0000 7ff4fffff700 1 -- 172.21.15.32:0/2741972314 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 9) v1 ==== 43389+0+0 (secure 0 0 0) 0x7ff50801a5a0 con 0x7ff518101490 2024-05-12T03:51:19.135 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:18.445+0000 7ff4fffff700 1 -- 172.21.15.32:0/2741972314 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7ff50804a190 con 0x7ff518101490 2024-05-12T03:51:19.135 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:18.448+0000 7ff51dfe8700 1 -- 172.21.15.32:0/2741972314 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7ff500036790 msgr2=0x7ff500038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:19.135 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:18.448+0000 7ff51dfe8700 1 --2- 172.21.15.32:0/2741972314 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7ff500036790 0x7ff500038c40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff50c009990 tx=0x7ff50c006e40 comp rx=0 tx=0).stop 2024-05-12T03:51:19.135 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:18.448+0000 7ff51dfe8700 1 -- 172.21.15.32:0/2741972314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff518101490 msgr2=0x7ff518076b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:19.135 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:18.448+0000 7ff51dfe8700 1 --2- 172.21.15.32:0/2741972314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff518101490 0x7ff518076b10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff50800c010 tx=0x7ff508009250 comp rx=0 tx=0).stop 2024-05-12T03:51:19.135 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:18.448+0000 7ff51dfe8700 1 -- 172.21.15.32:0/2741972314 shutdown_connections 2024-05-12T03:51:19.135 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:18.448+0000 7ff51dfe8700 1 --2- 172.21.15.32:0/2741972314 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7ff500036790 0x7ff500038c40 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:19.135 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:18.448+0000 7ff51dfe8700 1 --2- 172.21.15.32:0/2741972314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff518101490 0x7ff518076b10 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:19.135 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:18.448+0000 7ff51dfe8700 1 -- 172.21.15.32:0/2741972314 >> 172.21.15.32:0/2741972314 conn(0x7ff5180fcbf0 msgr2=0x7ff5180fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:51:19.135 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:18.449+0000 7ff51dfe8700 1 -- 172.21.15.32:0/2741972314 shutdown_connections 2024-05-12T03:51:19.135 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:18.449+0000 7ff51dfe8700 1 -- 172.21.15.32:0/2741972314 wait complete. 2024-05-12T03:51:19.789 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:19 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/2741972314' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-12T03:51:19.789 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:19 smithi032 ceph-mon[40008]: mgrmap e9: smithi032.ukatkq(active, since 29s) 2024-05-12T03:51:20.494 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout { 2024-05-12T03:51:20.494 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-12T03:51:20.495 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-12T03:51:20.495 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "active_name": "smithi032.ukatkq", 2024-05-12T03:51:20.495 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-12T03:51:20.495 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout } 2024-05-12T03:51:20.495 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:19.644+0000 7faa9f2c7700 1 Processor -- start 2024-05-12T03:51:20.495 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:19.644+0000 7faa9f2c7700 1 -- start start 2024-05-12T03:51:20.495 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:19.644+0000 7faa9f2c7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa98102610 0x7faa98102a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:20.495 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:19.644+0000 7faa9f2c7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa98102f60 con 0x7faa98102610 2024-05-12T03:51:20.495 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:19.644+0000 7faa9d063700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa98102610 0x7faa98102a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:51:20.495 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:19.644+0000 7faa9d063700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa98102610 0x7faa98102a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57916/0 (socket says 172.21.15.32:57916) 2024-05-12T03:51:20.495 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:19.644+0000 7faa9d063700 1 -- 172.21.15.32:0/3053574249 learned_addr learned my addr 172.21.15.32:0/3053574249 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:51:20.495 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:19.645+0000 7faa9d063700 1 -- 172.21.15.32:0/3053574249 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa981030a0 con 0x7faa98102610 2024-05-12T03:51:20.495 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:19.645+0000 7faa9d063700 1 --2- 172.21.15.32:0/3053574249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa98102610 0x7faa98102a20 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7faa940098e0 tx=0x7faa94009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a8cb8118ee08338 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:51:20.496 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:19.645+0000 7faa8ffff700 1 -- 172.21.15.32:0/3053574249 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faa9400b860 con 0x7faa98102610 2024-05-12T03:51:20.496 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:19.645+0000 7faa8ffff700 1 -- 172.21.15.32:0/3053574249 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7faa9400be70 con 0x7faa98102610 2024-05-12T03:51:20.496 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:19.645+0000 7faa8ffff700 1 -- 172.21.15.32:0/3053574249 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faa94010380 con 0x7faa98102610 2024-05-12T03:51:20.496 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:19.646+0000 7faa9f2c7700 1 -- 172.21.15.32:0/3053574249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa98102610 msgr2=0x7faa98102a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:20.496 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:19.646+0000 7faa9f2c7700 1 --2- 172.21.15.32:0/3053574249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa98102610 0x7faa98102a20 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7faa940098e0 tx=0x7faa94009250 comp rx=0 tx=0).stop 2024-05-12T03:51:20.496 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:19.646+0000 7faa9f2c7700 1 -- 172.21.15.32:0/3053574249 shutdown_connections 2024-05-12T03:51:20.496 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:19.646+0000 7faa9f2c7700 1 --2- 172.21.15.32:0/3053574249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa98102610 0x7faa98102a20 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:20.496 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:19.646+0000 7faa9f2c7700 1 -- 172.21.15.32:0/3053574249 >> 172.21.15.32:0/3053574249 conn(0x7faa980fdd90 msgr2=0x7faa981001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:51:20.496 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:19.646+0000 7faa9f2c7700 1 -- 172.21.15.32:0/3053574249 shutdown_connections 2024-05-12T03:51:20.496 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:19.646+0000 7faa9f2c7700 1 -- 172.21.15.32:0/3053574249 wait complete. 2024-05-12T03:51:20.496 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:19.646+0000 7faa9f2c7700 1 Processor -- start 2024-05-12T03:51:20.496 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:19.647+0000 7faa9f2c7700 1 -- start start 2024-05-12T03:51:20.496 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:19.647+0000 7faa9f2c7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa98102610 0x7faa98111370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:20.496 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:19.647+0000 7faa9f2c7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa981118b0 con 0x7faa98102610 2024-05-12T03:51:20.496 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:19.647+0000 7faa9d063700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa98102610 0x7faa98111370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:51:20.497 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:19.647+0000 7faa9d063700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa98102610 0x7faa98111370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57932/0 (socket says 172.21.15.32:57932) 2024-05-12T03:51:20.497 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:19.647+0000 7faa9d063700 1 -- 172.21.15.32:0/1311106281 learned_addr learned my addr 172.21.15.32:0/1311106281 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:51:20.497 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:19.647+0000 7faa9d063700 1 -- 172.21.15.32:0/1311106281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa940094f0 con 0x7faa98102610 2024-05-12T03:51:20.497 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:19.647+0000 7faa9d063700 1 --2- 172.21.15.32:0/1311106281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa98102610 0x7faa98111370 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7faa94011ca0 tx=0x7faa94009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:51:20.497 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:19.647+0000 7faa8dffb700 1 -- 172.21.15.32:0/1311106281 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faa94010850 con 0x7faa98102610 2024-05-12T03:51:20.497 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:19.647+0000 7faa9f2c7700 1 -- 172.21.15.32:0/1311106281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa98111ab0 con 0x7faa98102610 2024-05-12T03:51:20.497 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:19.647+0000 7faa8dffb700 1 -- 172.21.15.32:0/1311106281 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7faa94010e60 con 0x7faa98102610 2024-05-12T03:51:20.497 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:19.647+0000 7faa8dffb700 1 -- 172.21.15.32:0/1311106281 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faa9401ae20 con 0x7faa98102610 2024-05-12T03:51:20.497 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:19.648+0000 7faa9f2c7700 1 -- 172.21.15.32:0/1311106281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa98112620 con 0x7faa98102610 2024-05-12T03:51:20.497 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:19.648+0000 7faa8dffb700 1 -- 172.21.15.32:0/1311106281 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 9) v1 ==== 43389+0+0 (secure 0 0 0) 0x7faa9402d3f0 con 0x7faa98102610 2024-05-12T03:51:20.497 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:19.649+0000 7faa8dffb700 1 --2- 172.21.15.32:0/1311106281 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7faa840367e0 0x7faa84038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:20.497 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:19.649+0000 7faa9f2c7700 1 -- 172.21.15.32:0/1311106281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa9805f5c0 con 0x7faa98102610 2024-05-12T03:51:20.497 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:19.649+0000 7faa9c862700 1 -- 172.21.15.32:0/1311106281 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7faa840367e0 msgr2=0x7faa84038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/2967201790 2024-05-12T03:51:20.497 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:19.649+0000 7faa8dffb700 1 -- 172.21.15.32:0/1311106281 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7faa9404b450 con 0x7faa98102610 2024-05-12T03:51:20.497 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:19.649+0000 7faa9c862700 1 --2- 172.21.15.32:0/1311106281 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7faa840367e0 0x7faa84038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-12T03:51:20.498 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:19.653+0000 7faa8dffb700 1 -- 172.21.15.32:0/1311106281 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7faa9402b8e0 con 0x7faa98102610 2024-05-12T03:51:20.498 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:19.850+0000 7faa9c862700 1 -- 172.21.15.32:0/1311106281 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7faa840367e0 msgr2=0x7faa84038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/2967201790 2024-05-12T03:51:20.498 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:19.850+0000 7faa9c862700 1 --2- 172.21.15.32:0/1311106281 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7faa840367e0 0x7faa84038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-12T03:51:20.498 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:19.901+0000 7faa9f2c7700 1 -- 172.21.15.32:0/1311106281 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7faa981129c0 con 0x7faa98102610 2024-05-12T03:51:20.498 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:19.901+0000 7faa8dffb700 1 -- 172.21.15.32:0/1311106281 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+103 (secure 0 0 0) 0x7faa9402a6e0 con 0x7faa98102610 2024-05-12T03:51:20.498 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:19.903+0000 7faa9f2c7700 1 -- 172.21.15.32:0/1311106281 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7faa840367e0 msgr2=0x7faa84038c90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-12T03:51:20.498 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:19.903+0000 7faa9f2c7700 1 --2- 172.21.15.32:0/1311106281 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7faa840367e0 0x7faa84038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:20.498 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:19.903+0000 7faa9f2c7700 1 -- 172.21.15.32:0/1311106281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa98102610 msgr2=0x7faa98111370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:20.498 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:19.903+0000 7faa9f2c7700 1 --2- 172.21.15.32:0/1311106281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa98102610 0x7faa98111370 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7faa94011ca0 tx=0x7faa94009250 comp rx=0 tx=0).stop 2024-05-12T03:51:20.498 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:19.904+0000 7faa9f2c7700 1 -- 172.21.15.32:0/1311106281 shutdown_connections 2024-05-12T03:51:20.498 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:19.904+0000 7faa9f2c7700 1 --2- 172.21.15.32:0/1311106281 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7faa840367e0 0x7faa84038c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:20.498 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:19.904+0000 7faa9f2c7700 1 --2- 172.21.15.32:0/1311106281 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa98102610 0x7faa98111370 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:20.498 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:19.904+0000 7faa9f2c7700 1 -- 172.21.15.32:0/1311106281 >> 172.21.15.32:0/1311106281 conn(0x7faa980fdd90 msgr2=0x7faa980fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:51:20.498 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:19.904+0000 7faa9f2c7700 1 -- 172.21.15.32:0/1311106281 shutdown_connections 2024-05-12T03:51:20.499 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:19.904+0000 7faa9f2c7700 1 -- 172.21.15.32:0/1311106281 wait complete. 2024-05-12T03:51:20.499 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:20 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/1311106281' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-12T03:51:20.499 INFO:teuthology.orchestra.run.smithi032.stdout:Waiting for the mgr to restart... 2024-05-12T03:51:20.499 INFO:teuthology.orchestra.run.smithi032.stdout:Waiting for mgr epoch 9... 2024-05-12T03:51:25.386 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:25 smithi032 ceph-mon[40008]: Active manager daemon smithi032.ukatkq restarted 2024-05-12T03:51:25.386 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:25 smithi032 ceph-mon[40008]: Activating manager daemon smithi032.ukatkq 2024-05-12T03:51:25.386 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:25 smithi032 ceph-mon[40008]: osdmap e3: 0 total, 0 up, 0 in 2024-05-12T03:51:25.386 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:25 smithi032 ceph-mon[40008]: mgrmap e10: smithi032.ukatkq(active, starting, since 0.00309726s) 2024-05-12T03:51:25.386 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:25 smithi032 ceph-mon[40008]: from='mgr.14160 172.21.15.32:0/1781291074' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mon metadata", "id": "smithi032"}]: dispatch 2024-05-12T03:51:25.386 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:25 smithi032 ceph-mon[40008]: from='mgr.14160 172.21.15.32:0/1781291074' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mgr metadata", "who": "smithi032.ukatkq", "id": "smithi032.ukatkq"}]: dispatch 2024-05-12T03:51:25.386 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:25 smithi032 ceph-mon[40008]: from='mgr.14160 172.21.15.32:0/1781291074' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-12T03:51:25.387 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:25 smithi032 ceph-mon[40008]: from='mgr.14160 172.21.15.32:0/1781291074' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-12T03:51:25.387 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:25 smithi032 ceph-mon[40008]: from='mgr.14160 172.21.15.32:0/1781291074' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-12T03:51:25.387 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:25 smithi032 ceph-mon[40008]: Manager daemon smithi032.ukatkq is now available 2024-05-12T03:51:25.387 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:25 smithi032 ceph-mon[40008]: from='mgr.14160 172.21.15.32:0/1781291074' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T03:51:25.387 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:25 smithi032 ceph-mon[40008]: from='mgr.14160 172.21.15.32:0/1781291074' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T03:51:26.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:26 smithi032 ceph-mon[40008]: from='mgr.14160 172.21.15.32:0/1781291074' entity='mgr.smithi032.ukatkq' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi032.ukatkq/mirror_snapshot_schedule"}]: dispatch 2024-05-12T03:51:26.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:26 smithi032 ceph-mon[40008]: from='mgr.14160 172.21.15.32:0/1781291074' entity='mgr.smithi032.ukatkq' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi032.ukatkq/trash_purge_schedule"}]: dispatch 2024-05-12T03:51:26.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:26 smithi032 ceph-mon[40008]: mgrmap e11: smithi032.ukatkq(active, since 1.00554s) 2024-05-12T03:51:27.030 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout { 2024-05-12T03:51:27.031 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-12T03:51:27.031 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-12T03:51:27.031 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout } 2024-05-12T03:51:27.031 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:21.077+0000 7f56a3f9d700 1 Processor -- start 2024-05-12T03:51:27.031 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:21.077+0000 7f56a3f9d700 1 -- start start 2024-05-12T03:51:27.031 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:21.077+0000 7f56a3f9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f569c076be0 0x7f569c1051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:27.031 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:21.077+0000 7f56a3f9d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f569c1056e0 con 0x7f569c076be0 2024-05-12T03:51:27.031 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:21.078+0000 7f56a1d39700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f569c076be0 0x7f569c1051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:51:27.031 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:21.078+0000 7f56a1d39700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f569c076be0 0x7f569c1051a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57934/0 (socket says 172.21.15.32:57934) 2024-05-12T03:51:27.031 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:21.078+0000 7f56a1d39700 1 -- 172.21.15.32:0/985913794 learned_addr learned my addr 172.21.15.32:0/985913794 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:51:27.031 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:21.078+0000 7f56a1d39700 1 -- 172.21.15.32:0/985913794 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f569c105820 con 0x7f569c076be0 2024-05-12T03:51:27.031 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:21.078+0000 7f56a1d39700 1 --2- 172.21.15.32:0/985913794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f569c076be0 0x7f569c1051a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f56980098e0 tx=0x7f5698009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfb94e933ec944b server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:51:27.031 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:21.078+0000 7f56a0d37700 1 -- 172.21.15.32:0/985913794 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f569800b860 con 0x7f569c076be0 2024-05-12T03:51:27.032 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:21.078+0000 7f56a0d37700 1 -- 172.21.15.32:0/985913794 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f569800be70 con 0x7f569c076be0 2024-05-12T03:51:27.032 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:21.078+0000 7f56a0d37700 1 -- 172.21.15.32:0/985913794 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5698010380 con 0x7f569c076be0 2024-05-12T03:51:27.032 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:21.079+0000 7f56a3f9d700 1 -- 172.21.15.32:0/985913794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f569c076be0 msgr2=0x7f569c1051a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:27.032 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:21.079+0000 7f56a3f9d700 1 --2- 172.21.15.32:0/985913794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f569c076be0 0x7f569c1051a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f56980098e0 tx=0x7f5698009250 comp rx=0 tx=0).stop 2024-05-12T03:51:27.032 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:21.079+0000 7f56a3f9d700 1 -- 172.21.15.32:0/985913794 shutdown_connections 2024-05-12T03:51:27.032 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:21.079+0000 7f56a3f9d700 1 --2- 172.21.15.32:0/985913794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f569c076be0 0x7f569c1051a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:27.032 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:21.079+0000 7f56a3f9d700 1 -- 172.21.15.32:0/985913794 >> 172.21.15.32:0/985913794 conn(0x7f569c0fcbf0 msgr2=0x7f569c0ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:51:27.032 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:21.079+0000 7f56a3f9d700 1 -- 172.21.15.32:0/985913794 shutdown_connections 2024-05-12T03:51:27.032 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:21.079+0000 7f56a3f9d700 1 -- 172.21.15.32:0/985913794 wait complete. 2024-05-12T03:51:27.032 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:21.080+0000 7f56a3f9d700 1 Processor -- start 2024-05-12T03:51:27.032 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:21.080+0000 7f56a3f9d700 1 -- start start 2024-05-12T03:51:27.032 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:21.080+0000 7f56a3f9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f569c076be0 0x7f569c1113b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:27.032 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:21.080+0000 7f56a3f9d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f569c1118f0 con 0x7f569c076be0 2024-05-12T03:51:27.032 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:21.080+0000 7f56a1d39700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f569c076be0 0x7f569c1113b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:51:27.032 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:21.080+0000 7f56a1d39700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f569c076be0 0x7f569c1113b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57946/0 (socket says 172.21.15.32:57946) 2024-05-12T03:51:27.033 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:21.080+0000 7f56a1d39700 1 -- 172.21.15.32:0/782590161 learned_addr learned my addr 172.21.15.32:0/782590161 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:51:27.033 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:21.080+0000 7f56a1d39700 1 -- 172.21.15.32:0/782590161 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56980094f0 con 0x7f569c076be0 2024-05-12T03:51:27.033 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:21.080+0000 7f56a1d39700 1 --2- 172.21.15.32:0/782590161 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f569c076be0 0x7f569c1113b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5698006d40 tx=0x7f5698009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:51:27.033 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:21.081+0000 7f56927fc700 1 -- 172.21.15.32:0/782590161 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5698010960 con 0x7f569c076be0 2024-05-12T03:51:27.033 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:21.081+0000 7f56927fc700 1 -- 172.21.15.32:0/782590161 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f569801a430 con 0x7f569c076be0 2024-05-12T03:51:27.033 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:21.081+0000 7f56a3f9d700 1 -- 172.21.15.32:0/782590161 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f569c111af0 con 0x7f569c076be0 2024-05-12T03:51:27.033 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:21.081+0000 7f56927fc700 1 -- 172.21.15.32:0/782590161 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f56980245b0 con 0x7f569c076be0 2024-05-12T03:51:27.033 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:21.081+0000 7f56a3f9d700 1 -- 172.21.15.32:0/782590161 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f569c112660 con 0x7f569c076be0 2024-05-12T03:51:27.033 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:21.082+0000 7f56927fc700 1 -- 172.21.15.32:0/782590161 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 9) v1 ==== 43389+0+0 (secure 0 0 0) 0x7f569802d3f0 con 0x7f569c076be0 2024-05-12T03:51:27.033 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:21.082+0000 7f56927fc700 1 --2- 172.21.15.32:0/782590161 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f5688036830 0x7f5688038ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:27.033 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:21.082+0000 7f56927fc700 1 -- 172.21.15.32:0/782590161 --> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f56880393f0 con 0x7f5688036830 2024-05-12T03:51:27.033 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:21.082+0000 7f56927fc700 1 -- 172.21.15.32:0/782590161 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f569804a400 con 0x7f569c076be0 2024-05-12T03:51:27.033 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:21.082+0000 7f56a1538700 1 -- 172.21.15.32:0/782590161 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f5688036830 msgr2=0x7f5688038ce0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/2967201790 2024-05-12T03:51:27.033 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:21.082+0000 7f56a1538700 1 --2- 172.21.15.32:0/782590161 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f5688036830 0x7f5688038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-12T03:51:27.034 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:21.283+0000 7f56a1538700 1 -- 172.21.15.32:0/782590161 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f5688036830 msgr2=0x7f5688038ce0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/2967201790 2024-05-12T03:51:27.034 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:21.283+0000 7f56a1538700 1 --2- 172.21.15.32:0/782590161 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f5688036830 0x7f5688038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-12T03:51:27.034 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:21.684+0000 7f56a1538700 1 -- 172.21.15.32:0/782590161 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f5688036830 msgr2=0x7f5688038ce0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/2967201790 2024-05-12T03:51:27.034 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:21.684+0000 7f56a1538700 1 --2- 172.21.15.32:0/782590161 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f5688036830 0x7f5688038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-12T03:51:27.034 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:22.485+0000 7f56a1538700 1 -- 172.21.15.32:0/782590161 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f5688036830 msgr2=0x7f5688038ce0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/2967201790 2024-05-12T03:51:27.034 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:22.485+0000 7f56a1538700 1 --2- 172.21.15.32:0/782590161 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f5688036830 0x7f5688038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-12T03:51:27.034 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:24.086+0000 7f56a1538700 1 -- 172.21.15.32:0/782590161 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f5688036830 msgr2=0x7f5688038ce0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/2967201790 2024-05-12T03:51:27.034 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:24.086+0000 7f56a1538700 1 --2- 172.21.15.32:0/782590161 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f5688036830 0x7f5688038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-12T03:51:27.034 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:25.269+0000 7f56927fc700 1 -- 172.21.15.32:0/782590161 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mgrmap(e 10) v1 ==== 43156+0+0 (secure 0 0 0) 0x7f5698018d20 con 0x7f569c076be0 2024-05-12T03:51:27.034 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:25.270+0000 7f56927fc700 1 -- 172.21.15.32:0/782590161 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f5688036830 msgr2=0x7f5688038ce0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-12T03:51:27.034 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:25.270+0000 7f56927fc700 1 --2- 172.21.15.32:0/782590161 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f5688036830 0x7f5688038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:27.034 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:26.271+0000 7f56927fc700 1 -- 172.21.15.32:0/782590161 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 11) v1 ==== 43283+0+0 (secure 0 0 0) 0x7f569802dac0 con 0x7f569c076be0 2024-05-12T03:51:27.034 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:26.271+0000 7f56927fc700 1 --2- 172.21.15.32:0/782590161 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f568803a070 0x7f568803c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:27.034 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:26.271+0000 7f56927fc700 1 -- 172.21.15.32:0/782590161 --> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f56880393f0 con 0x7f568803a070 2024-05-12T03:51:27.035 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:26.272+0000 7f56a1538700 1 --2- 172.21.15.32:0/782590161 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f568803a070 0x7f568803c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:51:27.035 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:26.272+0000 7f56a1538700 1 --2- 172.21.15.32:0/782590161 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f568803a070 0x7f568803c450 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f568c0039d0 tx=0x7f568c009350 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:51:27.035 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:26.273+0000 7f56927fc700 1 -- 172.21.15.32:0/782590161 <== mgr.14160 v2:172.21.15.32:6800/1988399583 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7f56880393f0 con 0x7f568803a070 2024-05-12T03:51:27.035 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:26.279+0000 7f56a3f9d700 1 -- 172.21.15.32:0/782590161 --> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f569c111f80 con 0x7f568803a070 2024-05-12T03:51:27.035 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:26.279+0000 7f56927fc700 1 -- 172.21.15.32:0/782590161 <== mgr.14160 v2:172.21.15.32:6800/1988399583 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f569c111f80 con 0x7f568803a070 2024-05-12T03:51:27.035 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:26.279+0000 7f56a3f9d700 1 -- 172.21.15.32:0/782590161 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f568803a070 msgr2=0x7f568803c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:27.035 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:26.279+0000 7f56a3f9d700 1 --2- 172.21.15.32:0/782590161 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f568803a070 0x7f568803c450 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f568c0039d0 tx=0x7f568c009350 comp rx=0 tx=0).stop 2024-05-12T03:51:27.035 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:26.279+0000 7f56a3f9d700 1 -- 172.21.15.32:0/782590161 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f569c076be0 msgr2=0x7f569c1113b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:27.035 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:26.279+0000 7f56a3f9d700 1 --2- 172.21.15.32:0/782590161 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f569c076be0 0x7f569c1113b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5698006d40 tx=0x7f5698009250 comp rx=0 tx=0).stop 2024-05-12T03:51:27.035 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:26.279+0000 7f56a3f9d700 1 -- 172.21.15.32:0/782590161 shutdown_connections 2024-05-12T03:51:27.035 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:26.279+0000 7f56a3f9d700 1 --2- 172.21.15.32:0/782590161 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f568803a070 0x7f568803c450 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:27.035 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:26.279+0000 7f56a3f9d700 1 --2- 172.21.15.32:0/782590161 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f569c076be0 0x7f569c1113b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:27.035 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:26.279+0000 7f56a3f9d700 1 --2- 172.21.15.32:0/782590161 >> [v2:172.21.15.32:6800/2967201790,v1:172.21.15.32:6801/2967201790] conn(0x7f5688036830 0x7f5688038ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:27.035 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:26.279+0000 7f56a3f9d700 1 -- 172.21.15.32:0/782590161 >> 172.21.15.32:0/782590161 conn(0x7f569c0fcbf0 msgr2=0x7f569c0fd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:51:27.035 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:26.280+0000 7f56a3f9d700 1 -- 172.21.15.32:0/782590161 shutdown_connections 2024-05-12T03:51:27.036 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:26.280+0000 7f56a3f9d700 1 -- 172.21.15.32:0/782590161 wait complete. 2024-05-12T03:51:27.036 INFO:teuthology.orchestra.run.smithi032.stdout:mgr epoch 9 is available 2024-05-12T03:51:27.036 INFO:teuthology.orchestra.run.smithi032.stdout:Generating a dashboard self-signed certificate... 2024-05-12T03:51:27.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:27 smithi032 ceph-mon[40008]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-12T03:51:27.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:27 smithi032 ceph-mon[40008]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-12T03:51:27.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:27 smithi032 ceph-mon[40008]: from='mgr.14160 172.21.15.32:0/1781291074' entity='mgr.smithi032.ukatkq' 2024-05-12T03:51:28.519 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-12T03:51:28.519 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:27.574+0000 7fe2835ef700 1 Processor -- start 2024-05-12T03:51:28.519 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:27.574+0000 7fe2835ef700 1 -- start start 2024-05-12T03:51:28.519 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:27.574+0000 7fe2835ef700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe27c076c10 0x7fe27c107600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:28.519 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:27.574+0000 7fe2835ef700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe27c102ff0 con 0x7fe27c076c10 2024-05-12T03:51:28.519 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:27.575+0000 7fe28138b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe27c076c10 0x7fe27c107600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:51:28.519 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:27.575+0000 7fe28138b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe27c076c10 0x7fe27c107600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60156/0 (socket says 172.21.15.32:60156) 2024-05-12T03:51:28.520 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:27.575+0000 7fe28138b700 1 -- 172.21.15.32:0/3712913463 learned_addr learned my addr 172.21.15.32:0/3712913463 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:51:28.520 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:27.575+0000 7fe28138b700 1 -- 172.21.15.32:0/3712913463 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe27c107b40 con 0x7fe27c076c10 2024-05-12T03:51:28.520 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:27.575+0000 7fe28138b700 1 --2- 172.21.15.32:0/3712913463 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe27c076c10 0x7fe27c107600 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe2780098e0 tx=0x7fe278009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=118d98ee88309419 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:51:28.520 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:27.575+0000 7fe273fff700 1 -- 172.21.15.32:0/3712913463 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe27800b860 con 0x7fe27c076c10 2024-05-12T03:51:28.520 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:27.576+0000 7fe273fff700 1 -- 172.21.15.32:0/3712913463 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe27800be70 con 0x7fe27c076c10 2024-05-12T03:51:28.520 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:27.576+0000 7fe273fff700 1 -- 172.21.15.32:0/3712913463 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe278010380 con 0x7fe27c076c10 2024-05-12T03:51:28.520 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:27.576+0000 7fe2835ef700 1 -- 172.21.15.32:0/3712913463 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe27c076c10 msgr2=0x7fe27c107600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:28.520 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:27.576+0000 7fe2835ef700 1 --2- 172.21.15.32:0/3712913463 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe27c076c10 0x7fe27c107600 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe2780098e0 tx=0x7fe278009250 comp rx=0 tx=0).stop 2024-05-12T03:51:28.520 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:27.576+0000 7fe2835ef700 1 -- 172.21.15.32:0/3712913463 shutdown_connections 2024-05-12T03:51:28.520 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:27.576+0000 7fe2835ef700 1 --2- 172.21.15.32:0/3712913463 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe27c076c10 0x7fe27c107600 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:28.521 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:27.576+0000 7fe2835ef700 1 -- 172.21.15.32:0/3712913463 >> 172.21.15.32:0/3712913463 conn(0x7fe27c0fcc50 msgr2=0x7fe27c0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:51:28.521 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:27.576+0000 7fe2835ef700 1 -- 172.21.15.32:0/3712913463 shutdown_connections 2024-05-12T03:51:28.521 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:27.576+0000 7fe2835ef700 1 -- 172.21.15.32:0/3712913463 wait complete. 2024-05-12T03:51:28.521 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:27.577+0000 7fe2835ef700 1 Processor -- start 2024-05-12T03:51:28.521 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:27.577+0000 7fe2835ef700 1 -- start start 2024-05-12T03:51:28.521 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:27.577+0000 7fe2835ef700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe27c076c10 0x7fe27c113590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:28.521 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:27.577+0000 7fe2835ef700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe27c113ad0 con 0x7fe27c076c10 2024-05-12T03:51:28.521 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:27.577+0000 7fe28138b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe27c076c10 0x7fe27c113590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:51:28.521 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:27.577+0000 7fe28138b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe27c076c10 0x7fe27c113590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60164/0 (socket says 172.21.15.32:60164) 2024-05-12T03:51:28.521 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:27.577+0000 7fe28138b700 1 -- 172.21.15.32:0/388121242 learned_addr learned my addr 172.21.15.32:0/388121242 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:51:28.521 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:27.578+0000 7fe28138b700 1 -- 172.21.15.32:0/388121242 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2780094f0 con 0x7fe27c076c10 2024-05-12T03:51:28.522 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:27.578+0000 7fe28138b700 1 --2- 172.21.15.32:0/388121242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe27c076c10 0x7fe27c113590 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe278006d40 tx=0x7fe278009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:51:28.522 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:27.578+0000 7fe271ffb700 1 -- 172.21.15.32:0/388121242 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe278010960 con 0x7fe27c076c10 2024-05-12T03:51:28.522 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:27.578+0000 7fe271ffb700 1 -- 172.21.15.32:0/388121242 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe27801a430 con 0x7fe27c076c10 2024-05-12T03:51:28.522 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:27.578+0000 7fe2835ef700 1 -- 172.21.15.32:0/388121242 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe27c113cd0 con 0x7fe27c076c10 2024-05-12T03:51:28.522 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:27.578+0000 7fe271ffb700 1 -- 172.21.15.32:0/388121242 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe2780245b0 con 0x7fe27c076c10 2024-05-12T03:51:28.522 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:27.578+0000 7fe2835ef700 1 -- 172.21.15.32:0/388121242 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe27c114840 con 0x7fe27c076c10 2024-05-12T03:51:28.522 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:27.579+0000 7fe271ffb700 1 -- 172.21.15.32:0/388121242 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 11) v1 ==== 43283+0+0 (secure 0 0 0) 0x7fe27802d3f0 con 0x7fe27c076c10 2024-05-12T03:51:28.522 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:27.579+0000 7fe271ffb700 1 --2- 172.21.15.32:0/388121242 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7fe2680366b0 0x7fe268038b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:28.522 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:27.579+0000 7fe2835ef700 1 -- 172.21.15.32:0/388121242 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe27c05f5c0 con 0x7fe27c076c10 2024-05-12T03:51:28.523 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:27.579+0000 7fe280b8a700 1 --2- 172.21.15.32:0/388121242 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7fe2680366b0 0x7fe268038b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:51:28.523 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:27.579+0000 7fe271ffb700 1 -- 172.21.15.32:0/388121242 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fe27804a270 con 0x7fe27c076c10 2024-05-12T03:51:28.523 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:27.580+0000 7fe280b8a700 1 --2- 172.21.15.32:0/388121242 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7fe2680366b0 0x7fe268038b60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe26c009990 tx=0x7fe26c006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:51:28.523 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:27.584+0000 7fe271ffb700 1 -- 172.21.15.32:0/388121242 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fe27802b8e0 con 0x7fe27c076c10 2024-05-12T03:51:28.523 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:27.792+0000 7fe2835ef700 1 -- 172.21.15.32:0/388121242 --> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fe27c0fdfa0 con 0x7fe2680366b0 2024-05-12T03:51:28.523 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:27.918+0000 7fe271ffb700 1 -- 172.21.15.32:0/388121242 <== mgr.14160 v2:172.21.15.32:6800/1988399583 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fe27c0fdfa0 con 0x7fe2680366b0 2024-05-12T03:51:28.523 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:27.920+0000 7fe2835ef700 1 -- 172.21.15.32:0/388121242 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7fe2680366b0 msgr2=0x7fe268038b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:28.523 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:27.920+0000 7fe2835ef700 1 --2- 172.21.15.32:0/388121242 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7fe2680366b0 0x7fe268038b60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe26c009990 tx=0x7fe26c006e40 comp rx=0 tx=0).stop 2024-05-12T03:51:28.523 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:27.920+0000 7fe2835ef700 1 -- 172.21.15.32:0/388121242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe27c076c10 msgr2=0x7fe27c113590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:28.523 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:27.920+0000 7fe2835ef700 1 --2- 172.21.15.32:0/388121242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe27c076c10 0x7fe27c113590 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe278006d40 tx=0x7fe278009250 comp rx=0 tx=0).stop 2024-05-12T03:51:28.524 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:27.920+0000 7fe2835ef700 1 -- 172.21.15.32:0/388121242 shutdown_connections 2024-05-12T03:51:28.524 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:27.920+0000 7fe2835ef700 1 --2- 172.21.15.32:0/388121242 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7fe2680366b0 0x7fe268038b60 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:28.524 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:27.920+0000 7fe2835ef700 1 --2- 172.21.15.32:0/388121242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe27c076c10 0x7fe27c113590 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:28.524 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:27.920+0000 7fe2835ef700 1 -- 172.21.15.32:0/388121242 >> 172.21.15.32:0/388121242 conn(0x7fe27c0fcc50 msgr2=0x7fe27c0fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:51:28.524 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:27.920+0000 7fe2835ef700 1 -- 172.21.15.32:0/388121242 shutdown_connections 2024-05-12T03:51:28.524 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:27.920+0000 7fe2835ef700 1 -- 172.21.15.32:0/388121242 wait complete. 2024-05-12T03:51:28.524 INFO:teuthology.orchestra.run.smithi032.stdout:Creating initial admin user... 2024-05-12T03:51:29.273 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:28 smithi032 ceph-mon[40008]: [12/May/2024:03:51:27] ENGINE Bus STARTING 2024-05-12T03:51:29.273 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:28 smithi032 ceph-mon[40008]: from='mgr.14160 172.21.15.32:0/1781291074' entity='mgr.smithi032.ukatkq' 2024-05-12T03:51:29.273 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:28 smithi032 ceph-mon[40008]: from='mgr.14160 172.21.15.32:0/1781291074' entity='mgr.smithi032.ukatkq' 2024-05-12T03:51:29.274 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:28 smithi032 ceph-mon[40008]: [12/May/2024:03:51:28] ENGINE Serving on https://172.21.15.32:7150 2024-05-12T03:51:29.274 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:28 smithi032 ceph-mon[40008]: [12/May/2024:03:51:28] ENGINE Bus STARTED 2024-05-12T03:51:29.274 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:28 smithi032 ceph-mon[40008]: from='mgr.14160 172.21.15.32:0/1781291074' entity='mgr.smithi032.ukatkq' 2024-05-12T03:51:29.274 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:28 smithi032 ceph-mon[40008]: from='mgr.14160 172.21.15.32:0/1781291074' entity='mgr.smithi032.ukatkq' 2024-05-12T03:51:29.274 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:28 smithi032 ceph-mon[40008]: mgrmap e12: smithi032.ukatkq(active, since 2s) 2024-05-12T03:51:30.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:29 smithi032 ceph-mon[40008]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T03:51:30.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:29 smithi032 ceph-mon[40008]: from='mgr.14160 172.21.15.32:0/1781291074' entity='mgr.smithi032.ukatkq' 2024-05-12T03:51:30.872 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$9hGwIXyGuXMLqq6NT83uDukIeEiByjtWJkTFueaT6Ucb/sSTGkQU2", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1715485889, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-12T03:51:30.872 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:29.064+0000 7fe5618eb700 1 Processor -- start 2024-05-12T03:51:30.872 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:29.065+0000 7fe5618eb700 1 -- start start 2024-05-12T03:51:30.873 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:29.065+0000 7fe5618eb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe55c076c00 0x7fe55c1051b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:30.873 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:29.065+0000 7fe5618eb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe55c1056f0 con 0x7fe55c076c00 2024-05-12T03:51:30.873 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:29.065+0000 7fe55affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe55c076c00 0x7fe55c1051b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:51:30.873 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:29.065+0000 7fe55affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe55c076c00 0x7fe55c1051b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60180/0 (socket says 172.21.15.32:60180) 2024-05-12T03:51:30.873 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:29.065+0000 7fe55affd700 1 -- 172.21.15.32:0/3032076856 learned_addr learned my addr 172.21.15.32:0/3032076856 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:51:30.873 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:29.065+0000 7fe55affd700 1 -- 172.21.15.32:0/3032076856 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe55c105830 con 0x7fe55c076c00 2024-05-12T03:51:30.873 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:29.065+0000 7fe55affd700 1 --2- 172.21.15.32:0/3032076856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe55c076c00 0x7fe55c1051b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe54c0098e0 tx=0x7fe54c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8487f4c96191f096 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:51:30.873 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:29.066+0000 7fe559ffb700 1 -- 172.21.15.32:0/3032076856 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe54c00b860 con 0x7fe55c076c00 2024-05-12T03:51:30.873 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:29.066+0000 7fe559ffb700 1 -- 172.21.15.32:0/3032076856 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe54c00be70 con 0x7fe55c076c00 2024-05-12T03:51:30.873 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:29.066+0000 7fe559ffb700 1 -- 172.21.15.32:0/3032076856 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe54c010380 con 0x7fe55c076c00 2024-05-12T03:51:30.873 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:29.066+0000 7fe5618eb700 1 -- 172.21.15.32:0/3032076856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe55c076c00 msgr2=0x7fe55c1051b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:30.873 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:29.066+0000 7fe5618eb700 1 --2- 172.21.15.32:0/3032076856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe55c076c00 0x7fe55c1051b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe54c0098e0 tx=0x7fe54c009250 comp rx=0 tx=0).stop 2024-05-12T03:51:30.873 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:29.066+0000 7fe5618eb700 1 -- 172.21.15.32:0/3032076856 shutdown_connections 2024-05-12T03:51:30.873 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:29.066+0000 7fe5618eb700 1 --2- 172.21.15.32:0/3032076856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe55c076c00 0x7fe55c1051b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:30.873 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:29.066+0000 7fe5618eb700 1 -- 172.21.15.32:0/3032076856 >> 172.21.15.32:0/3032076856 conn(0x7fe55c0fcc00 msgr2=0x7fe55c0ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:51:30.874 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:29.066+0000 7fe5618eb700 1 -- 172.21.15.32:0/3032076856 shutdown_connections 2024-05-12T03:51:30.874 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:29.066+0000 7fe5618eb700 1 -- 172.21.15.32:0/3032076856 wait complete. 2024-05-12T03:51:30.874 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:29.067+0000 7fe5618eb700 1 Processor -- start 2024-05-12T03:51:30.874 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:29.067+0000 7fe5618eb700 1 -- start start 2024-05-12T03:51:30.874 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:29.067+0000 7fe5618eb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe55c076c00 0x7fe55c102b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:30.874 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:29.067+0000 7fe5618eb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe55c103040 con 0x7fe55c076c00 2024-05-12T03:51:30.874 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:29.067+0000 7fe55affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe55c076c00 0x7fe55c102b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:51:30.874 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:29.067+0000 7fe55affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe55c076c00 0x7fe55c102b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60194/0 (socket says 172.21.15.32:60194) 2024-05-12T03:51:30.874 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:29.067+0000 7fe55affd700 1 -- 172.21.15.32:0/1857193798 learned_addr learned my addr 172.21.15.32:0/1857193798 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:51:30.874 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:29.068+0000 7fe55affd700 1 -- 172.21.15.32:0/1857193798 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe54c0094f0 con 0x7fe55c076c00 2024-05-12T03:51:30.874 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:29.068+0000 7fe55affd700 1 --2- 172.21.15.32:0/1857193798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe55c076c00 0x7fe55c102b00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe54c006d40 tx=0x7fe54c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:51:30.874 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:29.068+0000 7fe5437fe700 1 -- 172.21.15.32:0/1857193798 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe54c010960 con 0x7fe55c076c00 2024-05-12T03:51:30.874 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:29.068+0000 7fe5437fe700 1 -- 172.21.15.32:0/1857193798 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe54c01a430 con 0x7fe55c076c00 2024-05-12T03:51:30.874 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:29.068+0000 7fe5618eb700 1 -- 172.21.15.32:0/1857193798 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe55c1010b0 con 0x7fe55c076c00 2024-05-12T03:51:30.875 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:29.068+0000 7fe5437fe700 1 -- 172.21.15.32:0/1857193798 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe54c0245b0 con 0x7fe55c076c00 2024-05-12T03:51:30.875 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:29.068+0000 7fe5618eb700 1 -- 172.21.15.32:0/1857193798 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe55c1015a0 con 0x7fe55c076c00 2024-05-12T03:51:30.875 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:29.069+0000 7fe5437fe700 1 -- 172.21.15.32:0/1857193798 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 12) v1 ==== 43389+0+0 (secure 0 0 0) 0x7fe54c02d3f0 con 0x7fe55c076c00 2024-05-12T03:51:30.875 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:29.069+0000 7fe5437fe700 1 --2- 172.21.15.32:0/1857193798 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7fe5440367e0 0x7fe544038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:30.875 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:29.069+0000 7fe5618eb700 1 -- 172.21.15.32:0/1857193798 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe55c04bed0 con 0x7fe55c076c00 2024-05-12T03:51:30.875 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:29.069+0000 7fe5437fe700 1 -- 172.21.15.32:0/1857193798 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fe54c04b440 con 0x7fe55c076c00 2024-05-12T03:51:30.875 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:29.069+0000 7fe55a7fc700 1 --2- 172.21.15.32:0/1857193798 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7fe5440367e0 0x7fe544038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:51:30.875 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:29.070+0000 7fe55a7fc700 1 --2- 172.21.15.32:0/1857193798 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7fe5440367e0 0x7fe544038c90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe550009990 tx=0x7fe550006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:51:30.875 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:29.074+0000 7fe5437fe700 1 -- 172.21.15.32:0/1857193798 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fe54c02b8e0 con 0x7fe55c076c00 2024-05-12T03:51:30.875 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:29.286+0000 7fe5618eb700 1 -- 172.21.15.32:0/1857193798 --> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fe55c05f3b0 con 0x7fe5440367e0 2024-05-12T03:51:30.875 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:29.532+0000 7fe5437fe700 1 -- 172.21.15.32:0/1857193798 <== mgr.14160 v2:172.21.15.32:6800/1988399583 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7fe55c05f3b0 con 0x7fe5440367e0 2024-05-12T03:51:30.875 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:29.534+0000 7fe5618eb700 1 -- 172.21.15.32:0/1857193798 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7fe5440367e0 msgr2=0x7fe544038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:30.875 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:29.534+0000 7fe5618eb700 1 --2- 172.21.15.32:0/1857193798 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7fe5440367e0 0x7fe544038c90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe550009990 tx=0x7fe550006e40 comp rx=0 tx=0).stop 2024-05-12T03:51:30.875 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:29.534+0000 7fe5618eb700 1 -- 172.21.15.32:0/1857193798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe55c076c00 msgr2=0x7fe55c102b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:30.876 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:29.534+0000 7fe5618eb700 1 --2- 172.21.15.32:0/1857193798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe55c076c00 0x7fe55c102b00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe54c006d40 tx=0x7fe54c009250 comp rx=0 tx=0).stop 2024-05-12T03:51:30.876 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:29.534+0000 7fe5618eb700 1 -- 172.21.15.32:0/1857193798 shutdown_connections 2024-05-12T03:51:30.876 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:29.534+0000 7fe5618eb700 1 --2- 172.21.15.32:0/1857193798 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7fe5440367e0 0x7fe544038c90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:30.876 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:29.534+0000 7fe5618eb700 1 --2- 172.21.15.32:0/1857193798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe55c076c00 0x7fe55c102b00 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:30.876 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:29.534+0000 7fe5618eb700 1 -- 172.21.15.32:0/1857193798 >> 172.21.15.32:0/1857193798 conn(0x7fe55c0fcc00 msgr2=0x7fe55c0fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:51:30.876 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:29.534+0000 7fe5618eb700 1 -- 172.21.15.32:0/1857193798 shutdown_connections 2024-05-12T03:51:30.876 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:29.534+0000 7fe5618eb700 1 -- 172.21.15.32:0/1857193798 wait complete. 2024-05-12T03:51:30.876 INFO:teuthology.orchestra.run.smithi032.stdout:Fetching dashboard port number... 2024-05-12T03:51:32.240 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 8443 2024-05-12T03:51:32.246 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:31.411+0000 7f18b4742700 1 Processor -- start 2024-05-12T03:51:32.246 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:31.411+0000 7f18b4742700 1 -- start start 2024-05-12T03:51:32.246 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:31.411+0000 7f18b4742700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18ac076bf0 0x7f18ac107620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:32.246 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:31.411+0000 7f18b4742700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ac103010 con 0x7f18ac076bf0 2024-05-12T03:51:32.246 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:31.411+0000 7f18b24de700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18ac076bf0 0x7f18ac107620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:51:32.246 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:31.411+0000 7f18b24de700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18ac076bf0 0x7f18ac107620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60208/0 (socket says 172.21.15.32:60208) 2024-05-12T03:51:32.246 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:31.411+0000 7f18b24de700 1 -- 172.21.15.32:0/1577647442 learned_addr learned my addr 172.21.15.32:0/1577647442 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:51:32.246 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:31.411+0000 7f18b24de700 1 -- 172.21.15.32:0/1577647442 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18ac107b60 con 0x7f18ac076bf0 2024-05-12T03:51:32.246 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:31.412+0000 7f18b24de700 1 --2- 172.21.15.32:0/1577647442 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18ac076bf0 0x7f18ac107620 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f18a80098e0 tx=0x7f18a8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b27f1c8e245daf54 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:51:32.246 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:31.412+0000 7f18b14dc700 1 -- 172.21.15.32:0/1577647442 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f18a800b8f0 con 0x7f18ac076bf0 2024-05-12T03:51:32.246 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:31.412+0000 7f18b14dc700 1 -- 172.21.15.32:0/1577647442 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f18a8011420 con 0x7f18ac076bf0 2024-05-12T03:51:32.246 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:31.412+0000 7f18b14dc700 1 -- 172.21.15.32:0/1577647442 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f18a8010540 con 0x7f18ac076bf0 2024-05-12T03:51:32.247 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:31.412+0000 7f18b4742700 1 -- 172.21.15.32:0/1577647442 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18ac076bf0 msgr2=0x7f18ac107620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:32.247 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:31.412+0000 7f18b4742700 1 --2- 172.21.15.32:0/1577647442 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18ac076bf0 0x7f18ac107620 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f18a80098e0 tx=0x7f18a8009250 comp rx=0 tx=0).stop 2024-05-12T03:51:32.247 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:31.412+0000 7f18b4742700 1 -- 172.21.15.32:0/1577647442 shutdown_connections 2024-05-12T03:51:32.247 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:31.412+0000 7f18b4742700 1 --2- 172.21.15.32:0/1577647442 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18ac076bf0 0x7f18ac107620 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:32.247 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:31.412+0000 7f18b4742700 1 -- 172.21.15.32:0/1577647442 >> 172.21.15.32:0/1577647442 conn(0x7f18ac0fcc50 msgr2=0x7f18ac0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:51:32.247 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:31.412+0000 7f18b4742700 1 -- 172.21.15.32:0/1577647442 shutdown_connections 2024-05-12T03:51:32.247 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:31.412+0000 7f18b4742700 1 -- 172.21.15.32:0/1577647442 wait complete. 2024-05-12T03:51:32.247 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:31.413+0000 7f18b4742700 1 Processor -- start 2024-05-12T03:51:32.247 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:31.413+0000 7f18b4742700 1 -- start start 2024-05-12T03:51:32.247 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:31.413+0000 7f18b4742700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18ac076bf0 0x7f18ac113660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:32.247 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:31.413+0000 7f18b4742700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ac113ba0 con 0x7f18ac076bf0 2024-05-12T03:51:32.247 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:31.413+0000 7f18b24de700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18ac076bf0 0x7f18ac113660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:51:32.247 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:31.413+0000 7f18b24de700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18ac076bf0 0x7f18ac113660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60218/0 (socket says 172.21.15.32:60218) 2024-05-12T03:51:32.247 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:31.413+0000 7f18b24de700 1 -- 172.21.15.32:0/357613530 learned_addr learned my addr 172.21.15.32:0/357613530 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:51:32.247 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:31.413+0000 7f18b24de700 1 -- 172.21.15.32:0/357613530 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a80094f0 con 0x7f18ac076bf0 2024-05-12T03:51:32.248 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:31.414+0000 7f18b24de700 1 --2- 172.21.15.32:0/357613530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18ac076bf0 0x7f18ac113660 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f18a8003080 tx=0x7f18a8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:51:32.248 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:31.414+0000 7f18a2ffd700 1 -- 172.21.15.32:0/357613530 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f18a800be30 con 0x7f18ac076bf0 2024-05-12T03:51:32.248 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:31.414+0000 7f18a2ffd700 1 -- 172.21.15.32:0/357613530 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f18a8010c60 con 0x7f18ac076bf0 2024-05-12T03:51:32.248 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:31.414+0000 7f18b4742700 1 -- 172.21.15.32:0/357613530 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18ac113da0 con 0x7f18ac076bf0 2024-05-12T03:51:32.248 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:31.414+0000 7f18b4742700 1 -- 172.21.15.32:0/357613530 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18ac114910 con 0x7f18ac076bf0 2024-05-12T03:51:32.248 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:31.414+0000 7f18a2ffd700 1 -- 172.21.15.32:0/357613530 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f18a8024d20 con 0x7f18ac076bf0 2024-05-12T03:51:32.248 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:31.415+0000 7f18a2ffd700 1 -- 172.21.15.32:0/357613530 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7f18a801a430 con 0x7f18ac076bf0 2024-05-12T03:51:32.248 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:31.415+0000 7f18b4742700 1 -- 172.21.15.32:0/357613530 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18900052e0 con 0x7f18ac076bf0 2024-05-12T03:51:32.248 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:31.415+0000 7f18a2ffd700 1 --2- 172.21.15.32:0/357613530 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f18980368d0 0x7f1898038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:32.248 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:31.415+0000 7f18a2ffd700 1 -- 172.21.15.32:0/357613530 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f18a804a7b0 con 0x7f18ac076bf0 2024-05-12T03:51:32.248 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:31.415+0000 7f18b1cdd700 1 --2- 172.21.15.32:0/357613530 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f18980368d0 0x7f1898038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:51:32.248 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:31.415+0000 7f18b1cdd700 1 --2- 172.21.15.32:0/357613530 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f18980368d0 0x7f1898038d80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f189c009990 tx=0x7f189c006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:51:32.248 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:31.419+0000 7f18a2ffd700 1 -- 172.21.15.32:0/357613530 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f18a8018940 con 0x7f18ac076bf0 2024-05-12T03:51:32.248 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:31.614+0000 7f18b4742700 1 -- 172.21.15.32:0/357613530 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f1890005b10 con 0x7f18ac076bf0 2024-05-12T03:51:32.249 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:31.615+0000 7f18a2ffd700 1 -- 172.21.15.32:0/357613530 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f18a8018350 con 0x7f18ac076bf0 2024-05-12T03:51:32.249 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:31.617+0000 7f18b4742700 1 -- 172.21.15.32:0/357613530 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f18980368d0 msgr2=0x7f1898038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:32.249 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:31.617+0000 7f18b4742700 1 --2- 172.21.15.32:0/357613530 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f18980368d0 0x7f1898038d80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f189c009990 tx=0x7f189c006e40 comp rx=0 tx=0).stop 2024-05-12T03:51:32.249 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:31.617+0000 7f18b4742700 1 -- 172.21.15.32:0/357613530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18ac076bf0 msgr2=0x7f18ac113660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:32.249 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:31.617+0000 7f18b4742700 1 --2- 172.21.15.32:0/357613530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18ac076bf0 0x7f18ac113660 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f18a8003080 tx=0x7f18a8009250 comp rx=0 tx=0).stop 2024-05-12T03:51:32.249 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:31.617+0000 7f18b4742700 1 -- 172.21.15.32:0/357613530 shutdown_connections 2024-05-12T03:51:32.249 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:31.617+0000 7f18b4742700 1 --2- 172.21.15.32:0/357613530 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f18980368d0 0x7f1898038d80 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:32.249 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:31.617+0000 7f18b4742700 1 --2- 172.21.15.32:0/357613530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18ac076bf0 0x7f18ac113660 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:32.249 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:31.617+0000 7f18b4742700 1 -- 172.21.15.32:0/357613530 >> 172.21.15.32:0/357613530 conn(0x7f18ac0fcc50 msgr2=0x7f18ac0fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:51:32.249 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:31.617+0000 7f18b4742700 1 -- 172.21.15.32:0/357613530 shutdown_connections 2024-05-12T03:51:32.249 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:31.617+0000 7f18b4742700 1 -- 172.21.15.32:0/357613530 wait complete. 2024-05-12T03:51:32.253 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-12T03:51:32.253 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout disabled 2024-05-12T03:51:32.262 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-12T03:51:32.262 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout inactive 2024-05-12T03:51:32.262 INFO:teuthology.orchestra.run.smithi032.stdout:firewalld.service is not enabled 2024-05-12T03:51:32.262 INFO:teuthology.orchestra.run.smithi032.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-12T03:51:32.264 INFO:teuthology.orchestra.run.smithi032.stdout:Ceph Dashboard is now available at: 2024-05-12T03:51:32.264 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:51:32.264 INFO:teuthology.orchestra.run.smithi032.stdout: URL: https://smithi032.front.sepia.ceph.com:8443/ 2024-05-12T03:51:32.264 INFO:teuthology.orchestra.run.smithi032.stdout: User: admin 2024-05-12T03:51:32.264 INFO:teuthology.orchestra.run.smithi032.stdout: Password: bj3ucw53yy 2024-05-12T03:51:32.264 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:51:32.265 INFO:teuthology.orchestra.run.smithi032.stdout:Saving cluster configuration to /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/config directory 2024-05-12T03:51:32.265 INFO:teuthology.orchestra.run.smithi032.stdout:Enabling autotune for osd_memory_target 2024-05-12T03:51:32.515 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:32 smithi032 ceph-mon[40008]: mgrmap e13: smithi032.ukatkq(active, since 6s) 2024-05-12T03:51:32.515 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:32 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/357613530' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-12T03:51:33.711 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:32.826+0000 7f8ba4f62700 1 Processor -- start 2024-05-12T03:51:33.711 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:32.826+0000 7f8ba4f62700 1 -- start start 2024-05-12T03:51:33.711 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:32.826+0000 7f8ba4f62700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ba0076b90 0x7f8ba0105280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:33.711 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:32.826+0000 7f8ba4f62700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba01057c0 con 0x7f8ba0076b90 2024-05-12T03:51:33.711 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:32.826+0000 7f8b9e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ba0076b90 0x7f8ba0105280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:51:33.711 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:32.826+0000 7f8b9e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ba0076b90 0x7f8ba0105280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45442/0 (socket says 172.21.15.32:45442) 2024-05-12T03:51:33.711 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:32.826+0000 7f8b9e59c700 1 -- 172.21.15.32:0/1265559622 learned_addr learned my addr 172.21.15.32:0/1265559622 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:51:33.712 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:32.827+0000 7f8b9e59c700 1 -- 172.21.15.32:0/1265559622 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ba0105900 con 0x7f8ba0076b90 2024-05-12T03:51:33.712 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:32.827+0000 7f8b9e59c700 1 --2- 172.21.15.32:0/1265559622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ba0076b90 0x7f8ba0105280 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8b900098e0 tx=0x7f8b90009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36bfad41691154e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:51:33.712 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:32.827+0000 7f8b9d59a700 1 -- 172.21.15.32:0/1265559622 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8b9000b860 con 0x7f8ba0076b90 2024-05-12T03:51:33.712 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:32.827+0000 7f8b9d59a700 1 -- 172.21.15.32:0/1265559622 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8b9000be70 con 0x7f8ba0076b90 2024-05-12T03:51:33.712 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:32.827+0000 7f8b9d59a700 1 -- 172.21.15.32:0/1265559622 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8b90010380 con 0x7f8ba0076b90 2024-05-12T03:51:33.712 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:32.827+0000 7f8ba4f62700 1 -- 172.21.15.32:0/1265559622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ba0076b90 msgr2=0x7f8ba0105280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:33.712 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:32.827+0000 7f8ba4f62700 1 --2- 172.21.15.32:0/1265559622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ba0076b90 0x7f8ba0105280 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8b900098e0 tx=0x7f8b90009250 comp rx=0 tx=0).stop 2024-05-12T03:51:33.712 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:32.828+0000 7f8ba4f62700 1 -- 172.21.15.32:0/1265559622 shutdown_connections 2024-05-12T03:51:33.712 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:32.828+0000 7f8ba4f62700 1 --2- 172.21.15.32:0/1265559622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ba0076b90 0x7f8ba0105280 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:33.712 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:32.828+0000 7f8ba4f62700 1 -- 172.21.15.32:0/1265559622 >> 172.21.15.32:0/1265559622 conn(0x7f8ba00fcc50 msgr2=0x7f8ba00ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:51:33.712 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:32.828+0000 7f8ba4f62700 1 -- 172.21.15.32:0/1265559622 shutdown_connections 2024-05-12T03:51:33.713 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:32.828+0000 7f8ba4f62700 1 -- 172.21.15.32:0/1265559622 wait complete. 2024-05-12T03:51:33.713 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:32.828+0000 7f8ba4f62700 1 Processor -- start 2024-05-12T03:51:33.713 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:32.828+0000 7f8ba4f62700 1 -- start start 2024-05-12T03:51:33.713 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:32.828+0000 7f8ba4f62700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ba0076b90 0x7f8ba01113b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:33.713 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:32.828+0000 7f8ba4f62700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba01118f0 con 0x7f8ba0076b90 2024-05-12T03:51:33.713 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:32.828+0000 7f8b9e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ba0076b90 0x7f8ba01113b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:51:33.713 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:32.828+0000 7f8b9e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ba0076b90 0x7f8ba01113b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45458/0 (socket says 172.21.15.32:45458) 2024-05-12T03:51:33.713 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:32.828+0000 7f8b9e59c700 1 -- 172.21.15.32:0/3445110591 learned_addr learned my addr 172.21.15.32:0/3445110591 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:51:33.713 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:32.829+0000 7f8b9e59c700 1 -- 172.21.15.32:0/3445110591 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b900094f0 con 0x7f8ba0076b90 2024-05-12T03:51:33.713 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:32.829+0000 7f8b9e59c700 1 --2- 172.21.15.32:0/3445110591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ba0076b90 0x7f8ba01113b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8b900094c0 tx=0x7f8b90009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:51:33.713 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:32.829+0000 7f8b8effd700 1 -- 172.21.15.32:0/3445110591 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8b90010850 con 0x7f8ba0076b90 2024-05-12T03:51:33.713 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:32.829+0000 7f8b8effd700 1 -- 172.21.15.32:0/3445110591 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8b90010e60 con 0x7f8ba0076b90 2024-05-12T03:51:33.714 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:32.829+0000 7f8ba4f62700 1 -- 172.21.15.32:0/3445110591 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ba0111af0 con 0x7f8ba0076b90 2024-05-12T03:51:33.714 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:32.829+0000 7f8b8effd700 1 -- 172.21.15.32:0/3445110591 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8b9001a430 con 0x7f8ba0076b90 2024-05-12T03:51:33.714 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:32.829+0000 7f8ba4f62700 1 -- 172.21.15.32:0/3445110591 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ba0112660 con 0x7f8ba0076b90 2024-05-12T03:51:33.714 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:32.830+0000 7f8b8effd700 1 -- 172.21.15.32:0/3445110591 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7f8b9001a660 con 0x7f8ba0076b90 2024-05-12T03:51:33.714 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:32.830+0000 7f8b8effd700 1 --2- 172.21.15.32:0/3445110591 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f8b880368d0 0x7f8b88038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:33.714 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:32.830+0000 7f8b8effd700 1 -- 172.21.15.32:0/3445110591 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f8b9004a740 con 0x7f8ba0076b90 2024-05-12T03:51:33.714 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:32.830+0000 7f8b9dd9b700 1 --2- 172.21.15.32:0/3445110591 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f8b880368d0 0x7f8b88038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:51:33.714 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:32.830+0000 7f8ba4f62700 1 -- 172.21.15.32:0/3445110591 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ba005f5c0 con 0x7f8ba0076b90 2024-05-12T03:51:33.714 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:32.831+0000 7f8b9dd9b700 1 --2- 172.21.15.32:0/3445110591 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f8b880368d0 0x7f8b88038d80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8b94009990 tx=0x7f8b94006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:51:33.714 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:32.835+0000 7f8b8effd700 1 -- 172.21.15.32:0/3445110591 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f8b9001a960 con 0x7f8ba0076b90 2024-05-12T03:51:33.715 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:33.029+0000 7f8ba4f62700 1 -- 172.21.15.32:0/3445110591 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f8ba004bed0 con 0x7f8ba0076b90 2024-05-12T03:51:33.715 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:33.029+0000 7f8b8effd700 1 -- 172.21.15.32:0/3445110591 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f8b90023ed0 con 0x7f8ba0076b90 2024-05-12T03:51:33.715 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:33.031+0000 7f8ba4f62700 1 -- 172.21.15.32:0/3445110591 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f8b880368d0 msgr2=0x7f8b88038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:33.715 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:33.031+0000 7f8ba4f62700 1 --2- 172.21.15.32:0/3445110591 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f8b880368d0 0x7f8b88038d80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8b94009990 tx=0x7f8b94006e40 comp rx=0 tx=0).stop 2024-05-12T03:51:33.715 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:33.031+0000 7f8ba4f62700 1 -- 172.21.15.32:0/3445110591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ba0076b90 msgr2=0x7f8ba01113b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:33.715 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:33.031+0000 7f8ba4f62700 1 --2- 172.21.15.32:0/3445110591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ba0076b90 0x7f8ba01113b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8b900094c0 tx=0x7f8b90009250 comp rx=0 tx=0).stop 2024-05-12T03:51:33.715 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:33.031+0000 7f8ba4f62700 1 -- 172.21.15.32:0/3445110591 shutdown_connections 2024-05-12T03:51:33.715 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:33.031+0000 7f8ba4f62700 1 --2- 172.21.15.32:0/3445110591 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f8b880368d0 0x7f8b88038d80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:33.715 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:33.031+0000 7f8ba4f62700 1 --2- 172.21.15.32:0/3445110591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ba0076b90 0x7f8ba01113b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:33.715 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:33.031+0000 7f8ba4f62700 1 -- 172.21.15.32:0/3445110591 >> 172.21.15.32:0/3445110591 conn(0x7f8ba00fcc50 msgr2=0x7f8ba00fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:51:33.715 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:33.032+0000 7f8ba4f62700 1 -- 172.21.15.32:0/3445110591 shutdown_connections 2024-05-12T03:51:33.716 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:33.032+0000 7f8ba4f62700 1 -- 172.21.15.32:0/3445110591 wait complete. 2024-05-12T03:51:35.208 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:34.270+0000 7f852179f700 1 Processor -- start 2024-05-12T03:51:35.208 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:34.270+0000 7f852179f700 1 -- start start 2024-05-12T03:51:35.208 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:34.270+0000 7f852179f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f851c078c60 0x7f851c077260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:35.208 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:34.270+0000 7f852179f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f851c079070 con 0x7f851c078c60 2024-05-12T03:51:35.209 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:34.270+0000 7f851affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f851c078c60 0x7f851c077260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:51:35.209 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:34.270+0000 7f851affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f851c078c60 0x7f851c077260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45472/0 (socket says 172.21.15.32:45472) 2024-05-12T03:51:35.209 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:34.270+0000 7f851affd700 1 -- 172.21.15.32:0/760083301 learned_addr learned my addr 172.21.15.32:0/760083301 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:51:35.209 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:34.271+0000 7f851affd700 1 -- 172.21.15.32:0/760083301 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f851c0777a0 con 0x7f851c078c60 2024-05-12T03:51:35.209 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:34.271+0000 7f851affd700 1 --2- 172.21.15.32:0/760083301 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f851c078c60 0x7f851c077260 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f85100098e0 tx=0x7f8510009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f13b8bb333bcaa4 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:51:35.209 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:34.271+0000 7f8519ffb700 1 -- 172.21.15.32:0/760083301 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f851000b860 con 0x7f851c078c60 2024-05-12T03:51:35.209 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:34.271+0000 7f8519ffb700 1 -- 172.21.15.32:0/760083301 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f851000be70 con 0x7f851c078c60 2024-05-12T03:51:35.209 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:34.271+0000 7f8519ffb700 1 -- 172.21.15.32:0/760083301 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8510010380 con 0x7f851c078c60 2024-05-12T03:51:35.209 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:34.271+0000 7f852179f700 1 -- 172.21.15.32:0/760083301 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f851c078c60 msgr2=0x7f851c077260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:35.209 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:34.271+0000 7f852179f700 1 --2- 172.21.15.32:0/760083301 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f851c078c60 0x7f851c077260 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f85100098e0 tx=0x7f8510009250 comp rx=0 tx=0).stop 2024-05-12T03:51:35.209 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:34.272+0000 7f852179f700 1 -- 172.21.15.32:0/760083301 shutdown_connections 2024-05-12T03:51:35.209 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:34.272+0000 7f852179f700 1 --2- 172.21.15.32:0/760083301 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f851c078c60 0x7f851c077260 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:35.209 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:34.272+0000 7f852179f700 1 -- 172.21.15.32:0/760083301 >> 172.21.15.32:0/760083301 conn(0x7f851c0fdd70 msgr2=0x7f851c100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:51:35.209 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:34.272+0000 7f852179f700 1 -- 172.21.15.32:0/760083301 shutdown_connections 2024-05-12T03:51:35.210 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:34.272+0000 7f852179f700 1 -- 172.21.15.32:0/760083301 wait complete. 2024-05-12T03:51:35.210 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:34.272+0000 7f852179f700 1 Processor -- start 2024-05-12T03:51:35.210 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:34.272+0000 7f852179f700 1 -- start start 2024-05-12T03:51:35.210 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:34.273+0000 7f852179f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f851c078c60 0x7f851c1135a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:35.210 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:34.273+0000 7f852179f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f851c113ae0 con 0x7f851c078c60 2024-05-12T03:51:35.210 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:34.273+0000 7f851affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f851c078c60 0x7f851c1135a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:51:35.210 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:34.273+0000 7f851affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f851c078c60 0x7f851c1135a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45480/0 (socket says 172.21.15.32:45480) 2024-05-12T03:51:35.210 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:34.273+0000 7f851affd700 1 -- 172.21.15.32:0/4275889492 learned_addr learned my addr 172.21.15.32:0/4275889492 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:51:35.210 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:34.273+0000 7f851affd700 1 -- 172.21.15.32:0/4275889492 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85100094f0 con 0x7f851c078c60 2024-05-12T03:51:35.210 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:34.273+0000 7f851affd700 1 --2- 172.21.15.32:0/4275889492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f851c078c60 0x7f851c1135a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f85100094c0 tx=0x7f8510009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:51:35.210 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:34.273+0000 7f85037fe700 1 -- 172.21.15.32:0/4275889492 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8510010850 con 0x7f851c078c60 2024-05-12T03:51:35.210 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:34.273+0000 7f85037fe700 1 -- 172.21.15.32:0/4275889492 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8510010e60 con 0x7f851c078c60 2024-05-12T03:51:35.210 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:34.273+0000 7f852179f700 1 -- 172.21.15.32:0/4275889492 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f851c113ce0 con 0x7f851c078c60 2024-05-12T03:51:35.210 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:34.273+0000 7f85037fe700 1 -- 172.21.15.32:0/4275889492 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f851001a430 con 0x7f851c078c60 2024-05-12T03:51:35.211 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:34.273+0000 7f852179f700 1 -- 172.21.15.32:0/4275889492 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f851c114850 con 0x7f851c078c60 2024-05-12T03:51:35.211 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:34.274+0000 7f852179f700 1 -- 172.21.15.32:0/4275889492 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f851c05f5c0 con 0x7f851c078c60 2024-05-12T03:51:35.211 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:34.274+0000 7f85037fe700 1 -- 172.21.15.32:0/4275889492 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7f85100247b0 con 0x7f851c078c60 2024-05-12T03:51:35.211 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:34.274+0000 7f85037fe700 1 --2- 172.21.15.32:0/4275889492 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f8504036880 0x7f8504038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:35.211 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:34.275+0000 7f85037fe700 1 -- 172.21.15.32:0/4275889492 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f851004a860 con 0x7f851c078c60 2024-05-12T03:51:35.211 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:34.275+0000 7f851a7fc700 1 --2- 172.21.15.32:0/4275889492 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f8504036880 0x7f8504038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:51:35.211 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:34.275+0000 7f851a7fc700 1 --2- 172.21.15.32:0/4275889492 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f8504036880 0x7f8504038d30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f850c009990 tx=0x7f850c006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:51:35.211 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:34.279+0000 7f85037fe700 1 -- 172.21.15.32:0/4275889492 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f8510024440 con 0x7f851c078c60 2024-05-12T03:51:35.211 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:34.585+0000 7f852179f700 1 -- 172.21.15.32:0/4275889492 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f851c04bed0 con 0x7f851c078c60 2024-05-12T03:51:35.211 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:34.587+0000 7f85037fe700 1 -- 172.21.15.32:0/4275889492 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v33)=0 set mgr/dashboard/cluster/status v33) v1 ==== 153+0+0 (secure 0 0 0) 0x7f8510022340 con 0x7f851c078c60 2024-05-12T03:51:35.211 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:34.589+0000 7f852179f700 1 -- 172.21.15.32:0/4275889492 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f8504036880 msgr2=0x7f8504038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:35.211 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:34.589+0000 7f852179f700 1 --2- 172.21.15.32:0/4275889492 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f8504036880 0x7f8504038d30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f850c009990 tx=0x7f850c006e40 comp rx=0 tx=0).stop 2024-05-12T03:51:35.211 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:34.589+0000 7f852179f700 1 -- 172.21.15.32:0/4275889492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f851c078c60 msgr2=0x7f851c1135a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:35.211 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:34.589+0000 7f852179f700 1 --2- 172.21.15.32:0/4275889492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f851c078c60 0x7f851c1135a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f85100094c0 tx=0x7f8510009250 comp rx=0 tx=0).stop 2024-05-12T03:51:35.211 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:34.589+0000 7f852179f700 1 -- 172.21.15.32:0/4275889492 shutdown_connections 2024-05-12T03:51:35.212 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:34.589+0000 7f852179f700 1 --2- 172.21.15.32:0/4275889492 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f8504036880 0x7f8504038d30 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:35.212 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:34.589+0000 7f852179f700 1 --2- 172.21.15.32:0/4275889492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f851c078c60 0x7f851c1135a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:35.212 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:34.589+0000 7f852179f700 1 -- 172.21.15.32:0/4275889492 >> 172.21.15.32:0/4275889492 conn(0x7f851c0fdd70 msgr2=0x7f851c0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:51:35.212 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:34.589+0000 7f852179f700 1 -- 172.21.15.32:0/4275889492 shutdown_connections 2024-05-12T03:51:35.212 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-12T03:51:34.590+0000 7f852179f700 1 -- 172.21.15.32:0/4275889492 wait complete. 2024-05-12T03:51:35.212 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-12T03:51:35.212 INFO:teuthology.orchestra.run.smithi032.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-12T03:51:35.212 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:51:35.212 INFO:teuthology.orchestra.run.smithi032.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-12T03:51:35.212 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:51:35.212 INFO:teuthology.orchestra.run.smithi032.stdout:Or, if you are only running a single cluster on this host: 2024-05-12T03:51:35.212 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:51:35.212 INFO:teuthology.orchestra.run.smithi032.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-12T03:51:35.212 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:51:35.212 INFO:teuthology.orchestra.run.smithi032.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-12T03:51:35.212 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:51:35.213 INFO:teuthology.orchestra.run.smithi032.stdout: ceph telemetry on 2024-05-12T03:51:35.213 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:51:35.213 INFO:teuthology.orchestra.run.smithi032.stdout:For more information see: 2024-05-12T03:51:35.213 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:51:35.213 INFO:teuthology.orchestra.run.smithi032.stdout: https://docs.ceph.com/docs/master/mgr/telemetry/ 2024-05-12T03:51:35.213 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:51:35.213 INFO:teuthology.orchestra.run.smithi032.stdout:Bootstrap complete. 2024-05-12T03:51:35.246 INFO:tasks.cephadm:Fetching config... 2024-05-12T03:51:35.246 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-05-12T03:51:35.247 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-12T03:51:35.267 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-12T03:51:35.267 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-05-12T03:51:35.267 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-12T03:51:35.329 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-12T03:51:35.329 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-05-12T03:51:35.329 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/keyring of=/dev/stdout 2024-05-12T03:51:35.399 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-12T03:51:35.399 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-05-12T03:51:35.399 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-12T03:51:35.459 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-12T03:51:35.460 DEBUG:teuthology.orchestra.run.smithi032:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHTfrwnLNrjtWpbcV83GLhyASEUjgXYkH0sFRt2L1Tvm67ejf3Xh8Yl1BWvuRL4eV7IIVigK2WexlUuXODx3cRc2PneH4kgoUA+QzI33fXd3FnRO6QKF0okku0/nfJxVxOyzAx6qYpkx9LOWjRTABljZiKoUtTlnkLJH+Zkulhxvb+9F6ad5lf8glq/IOHmoFBfc9xiQivVQg1asWwm5RiNCKOnoq+/YBofdDZVun7mgzw5EySK5lXyctXrxRYvr0hWJn0bogC8dbDEzqKy2PcNh5olEMzkHySRvj5h2qHKb30evy5UCYVFN8y8DqeiJMq/oehTqbmpSBMpw8OVpby5n4Bo3CPTI2kV/PI5Pdzx05DCc0VqSdrWloJpd1hknUdKHKiAUohyh2ck4fjMkTS6Ob2kcQ1oMv/p0WGzYybjaKPDcuNpmLikkMki3SrI1Rd5LOF2IuGghg0ODrbqdWbJ35iYOsj28IG2Rft2peRM9TSYJ5PUZrbvj1H3fLoVVc= ceph-a3e178f8-1012-11ef-bc98-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-12T03:51:35.539 INFO:teuthology.orchestra.run.smithi032.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHTfrwnLNrjtWpbcV83GLhyASEUjgXYkH0sFRt2L1Tvm67ejf3Xh8Yl1BWvuRL4eV7IIVigK2WexlUuXODx3cRc2PneH4kgoUA+QzI33fXd3FnRO6QKF0okku0/nfJxVxOyzAx6qYpkx9LOWjRTABljZiKoUtTlnkLJH+Zkulhxvb+9F6ad5lf8glq/IOHmoFBfc9xiQivVQg1asWwm5RiNCKOnoq+/YBofdDZVun7mgzw5EySK5lXyctXrxRYvr0hWJn0bogC8dbDEzqKy2PcNh5olEMzkHySRvj5h2qHKb30evy5UCYVFN8y8DqeiJMq/oehTqbmpSBMpw8OVpby5n4Bo3CPTI2kV/PI5Pdzx05DCc0VqSdrWloJpd1hknUdKHKiAUohyh2ck4fjMkTS6Ob2kcQ1oMv/p0WGzYybjaKPDcuNpmLikkMki3SrI1Rd5LOF2IuGghg0ODrbqdWbJ35iYOsj28IG2Rft2peRM9TSYJ5PUZrbvj1H3fLoVVc= ceph-a3e178f8-1012-11ef-bc98-c7b262605968 2024-05-12T03:51:35.553 DEBUG:teuthology.orchestra.run.smithi033:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHTfrwnLNrjtWpbcV83GLhyASEUjgXYkH0sFRt2L1Tvm67ejf3Xh8Yl1BWvuRL4eV7IIVigK2WexlUuXODx3cRc2PneH4kgoUA+QzI33fXd3FnRO6QKF0okku0/nfJxVxOyzAx6qYpkx9LOWjRTABljZiKoUtTlnkLJH+Zkulhxvb+9F6ad5lf8glq/IOHmoFBfc9xiQivVQg1asWwm5RiNCKOnoq+/YBofdDZVun7mgzw5EySK5lXyctXrxRYvr0hWJn0bogC8dbDEzqKy2PcNh5olEMzkHySRvj5h2qHKb30evy5UCYVFN8y8DqeiJMq/oehTqbmpSBMpw8OVpby5n4Bo3CPTI2kV/PI5Pdzx05DCc0VqSdrWloJpd1hknUdKHKiAUohyh2ck4fjMkTS6Ob2kcQ1oMv/p0WGzYybjaKPDcuNpmLikkMki3SrI1Rd5LOF2IuGghg0ODrbqdWbJ35iYOsj28IG2Rft2peRM9TSYJ5PUZrbvj1H3fLoVVc= ceph-a3e178f8-1012-11ef-bc98-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-12T03:51:35.600 INFO:teuthology.orchestra.run.smithi033.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHTfrwnLNrjtWpbcV83GLhyASEUjgXYkH0sFRt2L1Tvm67ejf3Xh8Yl1BWvuRL4eV7IIVigK2WexlUuXODx3cRc2PneH4kgoUA+QzI33fXd3FnRO6QKF0okku0/nfJxVxOyzAx6qYpkx9LOWjRTABljZiKoUtTlnkLJH+Zkulhxvb+9F6ad5lf8glq/IOHmoFBfc9xiQivVQg1asWwm5RiNCKOnoq+/YBofdDZVun7mgzw5EySK5lXyctXrxRYvr0hWJn0bogC8dbDEzqKy2PcNh5olEMzkHySRvj5h2qHKb30evy5UCYVFN8y8DqeiJMq/oehTqbmpSBMpw8OVpby5n4Bo3CPTI2kV/PI5Pdzx05DCc0VqSdrWloJpd1hknUdKHKiAUohyh2ck4fjMkTS6Ob2kcQ1oMv/p0WGzYybjaKPDcuNpmLikkMki3SrI1Rd5LOF2IuGghg0ODrbqdWbJ35iYOsj28IG2Rft2peRM9TSYJ5PUZrbvj1H3fLoVVc= ceph-a3e178f8-1012-11ef-bc98-c7b262605968 2024-05-12T03:51:35.612 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-12T03:51:35.789 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:35 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/4275889492' entity='client.admin' 2024-05-12T03:51:35.884 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T03:51:37.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:37.662+0000 7fa8361a4700 1 -- 172.21.15.32:0/1762809150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8300fbed0 msgr2=0x7fa8300fc2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:37.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:37.662+0000 7fa8361a4700 1 --2- 172.21.15.32:0/1762809150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8300fbed0 0x7fa8300fc2e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa8200048c0 tx=0x7fa820009e30 comp rx=0 tx=0).stop 2024-05-12T03:51:37.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:37.662+0000 7fa8361a4700 1 -- 172.21.15.32:0/1762809150 shutdown_connections 2024-05-12T03:51:37.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:37.662+0000 7fa8361a4700 1 --2- 172.21.15.32:0/1762809150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8300fbed0 0x7fa8300fc2e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:37.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:37.662+0000 7fa8361a4700 1 -- 172.21.15.32:0/1762809150 >> 172.21.15.32:0/1762809150 conn(0x7fa8300f67c0 msgr2=0x7fa8300f8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:51:37.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:37.662+0000 7fa8361a4700 1 -- 172.21.15.32:0/1762809150 shutdown_connections 2024-05-12T03:51:37.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:37.662+0000 7fa8361a4700 1 -- 172.21.15.32:0/1762809150 wait complete. 2024-05-12T03:51:37.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:37.663+0000 7fa8361a4700 1 Processor -- start 2024-05-12T03:51:37.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:37.663+0000 7fa8361a4700 1 -- start start 2024-05-12T03:51:37.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:37.663+0000 7fa8361a4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8300fbed0 0x7fa8300f8750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:37.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:37.663+0000 7fa8361a4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8300f8c90 con 0x7fa8300fbed0 2024-05-12T03:51:37.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:37.663+0000 7fa82f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8300fbed0 0x7fa8300f8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:51:37.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:37.664+0000 7fa82f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8300fbed0 0x7fa8300f8750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45508/0 (socket says 172.21.15.32:45508) 2024-05-12T03:51:37.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:37.664+0000 7fa82f7fe700 1 -- 172.21.15.32:0/2572791578 learned_addr learned my addr 172.21.15.32:0/2572791578 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:51:37.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:37.664+0000 7fa82f7fe700 1 -- 172.21.15.32:0/2572791578 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa820005040 con 0x7fa8300fbed0 2024-05-12T03:51:37.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:37.664+0000 7fa82f7fe700 1 --2- 172.21.15.32:0/2572791578 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8300fbed0 0x7fa8300f8750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa82000b860 tx=0x7fa820009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:51:37.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:37.664+0000 7fa817fff700 1 -- 172.21.15.32:0/2572791578 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa820009580 con 0x7fa8300fbed0 2024-05-12T03:51:37.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:37.664+0000 7fa8361a4700 1 -- 172.21.15.32:0/2572791578 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8300f6d60 con 0x7fa8300fbed0 2024-05-12T03:51:37.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:37.664+0000 7fa817fff700 1 -- 172.21.15.32:0/2572791578 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa8200096e0 con 0x7fa8300fbed0 2024-05-12T03:51:37.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:37.664+0000 7fa817fff700 1 -- 172.21.15.32:0/2572791578 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa82001c5b0 con 0x7fa8300fbed0 2024-05-12T03:51:37.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:37.664+0000 7fa8361a4700 1 -- 172.21.15.32:0/2572791578 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8300f7250 con 0x7fa8300fbed0 2024-05-12T03:51:37.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:37.665+0000 7fa817fff700 1 -- 172.21.15.32:0/2572791578 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7fa82001c820 con 0x7fa8300fbed0 2024-05-12T03:51:37.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:37.665+0000 7fa8361a4700 1 -- 172.21.15.32:0/2572791578 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa83005f5c0 con 0x7fa8300fbed0 2024-05-12T03:51:37.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:37.665+0000 7fa817fff700 1 --2- 172.21.15.32:0/2572791578 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7fa818036880 0x7fa818038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:37.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:37.665+0000 7fa817fff700 1 -- 172.21.15.32:0/2572791578 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fa820049cf0 con 0x7fa8300fbed0 2024-05-12T03:51:37.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:37.665+0000 7fa82effd700 1 --2- 172.21.15.32:0/2572791578 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7fa818036880 0x7fa818038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:51:37.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:37.666+0000 7fa82effd700 1 --2- 172.21.15.32:0/2572791578 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7fa818036880 0x7fa818038d30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa824009990 tx=0x7fa824006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:51:37.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:37.670+0000 7fa817fff700 1 -- 172.21.15.32:0/2572791578 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fa820023300 con 0x7fa8300fbed0 2024-05-12T03:51:37.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:37.863+0000 7fa8361a4700 1 -- 172.21.15.32:0/2572791578 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7fa83004bed0 con 0x7fa8300fbed0 2024-05-12T03:51:37.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:37.866+0000 7fa817fff700 1 -- 172.21.15.32:0/2572791578 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7fa820011b60 con 0x7fa8300fbed0 2024-05-12T03:51:37.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:37.868+0000 7fa8361a4700 1 -- 172.21.15.32:0/2572791578 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7fa818036880 msgr2=0x7fa818038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:37.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:37.868+0000 7fa8361a4700 1 --2- 172.21.15.32:0/2572791578 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7fa818036880 0x7fa818038d30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa824009990 tx=0x7fa824006e40 comp rx=0 tx=0).stop 2024-05-12T03:51:37.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:37.868+0000 7fa8361a4700 1 -- 172.21.15.32:0/2572791578 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8300fbed0 msgr2=0x7fa8300f8750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:37.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:37.868+0000 7fa8361a4700 1 --2- 172.21.15.32:0/2572791578 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8300fbed0 0x7fa8300f8750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa82000b860 tx=0x7fa820009e30 comp rx=0 tx=0).stop 2024-05-12T03:51:37.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:37.868+0000 7fa8361a4700 1 -- 172.21.15.32:0/2572791578 shutdown_connections 2024-05-12T03:51:37.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:37.868+0000 7fa8361a4700 1 --2- 172.21.15.32:0/2572791578 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7fa818036880 0x7fa818038d30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:37.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:37.868+0000 7fa8361a4700 1 --2- 172.21.15.32:0/2572791578 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8300fbed0 0x7fa8300f8750 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:37.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:37.868+0000 7fa8361a4700 1 -- 172.21.15.32:0/2572791578 >> 172.21.15.32:0/2572791578 conn(0x7fa8300f67c0 msgr2=0x7fa8300fab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:51:37.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:37.868+0000 7fa8361a4700 1 -- 172.21.15.32:0/2572791578 shutdown_connections 2024-05-12T03:51:37.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:37.868+0000 7fa8361a4700 1 -- 172.21.15.32:0/2572791578 wait complete. 2024-05-12T03:51:39.109 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-12T03:51:39.110 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-12T03:51:39.137 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:38 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/2572791578' entity='client.admin' 2024-05-12T03:51:39.137 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:38 smithi032 ceph-mon[40008]: from='mgr.14160 172.21.15.32:0/1781291074' entity='mgr.smithi032.ukatkq' 2024-05-12T03:51:39.137 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:38 smithi032 ceph-mon[40008]: from='mgr.14160 172.21.15.32:0/1781291074' entity='mgr.smithi032.ukatkq' 2024-05-12T03:51:39.137 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:38 smithi032 ceph-mon[40008]: from='mgr.14160 172.21.15.32:0/1781291074' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "who": "osd/host:smithi032", "name": "osd_memory_target"}]: dispatch 2024-05-12T03:51:39.137 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:38 smithi032 ceph-mon[40008]: from='mgr.14160 172.21.15.32:0/1781291074' entity='mgr.smithi032.ukatkq' 2024-05-12T03:51:39.137 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:38 smithi032 ceph-mon[40008]: from='mgr.14160 172.21.15.32:0/1781291074' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi032", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-12T03:51:39.137 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:38 smithi032 ceph-mon[40008]: from='mgr.14160 172.21.15.32:0/1781291074' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi032", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-05-12T03:51:39.137 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:38 smithi032 ceph-mon[40008]: from='mgr.14160 172.21.15.32:0/1781291074' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:51:39.138 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:38 smithi032 ceph-mon[40008]: Deploying daemon crash.smithi032 on smithi032 2024-05-12T03:51:39.410 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T03:51:41.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:41.658+0000 7f63ec07e700 1 -- 172.21.15.32:0/3032599321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63e40fca40 msgr2=0x7f63e40fce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:41.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:41.658+0000 7f63ec07e700 1 --2- 172.21.15.32:0/3032599321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63e40fca40 0x7f63e40fce50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f63e00048c0 tx=0x7f63e0009e30 comp rx=0 tx=0).stop 2024-05-12T03:51:41.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:41.658+0000 7f63ec07e700 1 -- 172.21.15.32:0/3032599321 shutdown_connections 2024-05-12T03:51:41.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:41.658+0000 7f63ec07e700 1 --2- 172.21.15.32:0/3032599321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63e40fca40 0x7f63e40fce50 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:41.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:41.658+0000 7f63ec07e700 1 -- 172.21.15.32:0/3032599321 >> 172.21.15.32:0/3032599321 conn(0x7f63e40f81a0 msgr2=0x7f63e40fa5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:51:41.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:41.658+0000 7f63ec07e700 1 -- 172.21.15.32:0/3032599321 shutdown_connections 2024-05-12T03:51:41.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:41.659+0000 7f63ec07e700 1 -- 172.21.15.32:0/3032599321 wait complete. 2024-05-12T03:51:41.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:41.659+0000 7f63ec07e700 1 Processor -- start 2024-05-12T03:51:41.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:41.659+0000 7f63ec07e700 1 -- start start 2024-05-12T03:51:41.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:41.659+0000 7f63ec07e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63e40fca40 0x7f63e4071d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:41.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:41.659+0000 7f63ec07e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e4072240 con 0x7f63e40fca40 2024-05-12T03:51:41.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:41.659+0000 7f63e9e1a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63e40fca40 0x7f63e4071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:51:41.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:41.659+0000 7f63e9e1a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63e40fca40 0x7f63e4071d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45530/0 (socket says 172.21.15.32:45530) 2024-05-12T03:51:41.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:41.659+0000 7f63e9e1a700 1 -- 172.21.15.32:0/4172472349 learned_addr learned my addr 172.21.15.32:0/4172472349 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:51:41.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:41.660+0000 7f63e9e1a700 1 -- 172.21.15.32:0/4172472349 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63e0005040 con 0x7f63e40fca40 2024-05-12T03:51:41.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:41.660+0000 7f63e9e1a700 1 --2- 172.21.15.32:0/4172472349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63e40fca40 0x7f63e4071d00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f63e0000c00 tx=0x7f63e0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:51:41.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:41.660+0000 7f63da7fc700 1 -- 172.21.15.32:0/4172472349 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f63e0009620 con 0x7f63e40fca40 2024-05-12T03:51:41.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:41.660+0000 7f63da7fc700 1 -- 172.21.15.32:0/4172472349 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f63e0009780 con 0x7f63e40fca40 2024-05-12T03:51:41.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:41.660+0000 7f63ec07e700 1 -- 172.21.15.32:0/4172472349 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63e4070310 con 0x7f63e40fca40 2024-05-12T03:51:41.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:41.660+0000 7f63da7fc700 1 -- 172.21.15.32:0/4172472349 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f63e001c5b0 con 0x7f63e40fca40 2024-05-12T03:51:41.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:41.660+0000 7f63ec07e700 1 -- 172.21.15.32:0/4172472349 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63e4070800 con 0x7f63e40fca40 2024-05-12T03:51:41.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:41.661+0000 7f63da7fc700 1 -- 172.21.15.32:0/4172472349 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7f63e001c760 con 0x7f63e40fca40 2024-05-12T03:51:41.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:41.661+0000 7f63da7fc700 1 --2- 172.21.15.32:0/4172472349 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f63d0036880 0x7f63d0038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:41.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:41.661+0000 7f63da7fc700 1 -- 172.21.15.32:0/4172472349 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f63e004aa20 con 0x7f63e40fca40 2024-05-12T03:51:41.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:41.661+0000 7f63ec07e700 1 -- 172.21.15.32:0/4172472349 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63e405f5c0 con 0x7f63e40fca40 2024-05-12T03:51:41.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:41.661+0000 7f63e9619700 1 --2- 172.21.15.32:0/4172472349 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f63d0036880 0x7f63d0038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:51:41.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:41.662+0000 7f63e9619700 1 --2- 172.21.15.32:0/4172472349 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f63d0036880 0x7f63d0038d30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f63d4009990 tx=0x7f63d4006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:51:41.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:41.667+0000 7f63da7fc700 1 -- 172.21.15.32:0/4172472349 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f63e0011db0 con 0x7f63e40fca40 2024-05-12T03:51:41.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:41.872+0000 7f63ec07e700 1 -- 172.21.15.32:0/4172472349 --> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f63e4070ab0 con 0x7f63d0036880 2024-05-12T03:51:41.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:41.875+0000 7f63da7fc700 1 -- 172.21.15.32:0/4172472349 <== mgr.14160 v2:172.21.15.32:6800/1988399583 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f63e4070ab0 con 0x7f63d0036880 2024-05-12T03:51:41.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:41.877+0000 7f63ec07e700 1 -- 172.21.15.32:0/4172472349 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f63d0036880 msgr2=0x7f63d0038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:41.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:41.877+0000 7f63ec07e700 1 --2- 172.21.15.32:0/4172472349 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f63d0036880 0x7f63d0038d30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f63d4009990 tx=0x7f63d4006e40 comp rx=0 tx=0).stop 2024-05-12T03:51:41.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:41.877+0000 7f63ec07e700 1 -- 172.21.15.32:0/4172472349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63e40fca40 msgr2=0x7f63e4071d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:41.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:41.877+0000 7f63ec07e700 1 --2- 172.21.15.32:0/4172472349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63e40fca40 0x7f63e4071d00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f63e0000c00 tx=0x7f63e0009e30 comp rx=0 tx=0).stop 2024-05-12T03:51:41.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:41.877+0000 7f63ec07e700 1 -- 172.21.15.32:0/4172472349 shutdown_connections 2024-05-12T03:51:41.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:41.877+0000 7f63ec07e700 1 --2- 172.21.15.32:0/4172472349 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f63d0036880 0x7f63d0038d30 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:41.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:41.877+0000 7f63ec07e700 1 --2- 172.21.15.32:0/4172472349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63e40fca40 0x7f63e4071d00 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:41.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:41.877+0000 7f63ec07e700 1 -- 172.21.15.32:0/4172472349 >> 172.21.15.32:0/4172472349 conn(0x7f63e40f81a0 msgr2=0x7f63e41039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:51:41.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:41.877+0000 7f63ec07e700 1 -- 172.21.15.32:0/4172472349 shutdown_connections 2024-05-12T03:51:41.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:41.878+0000 7f63ec07e700 1 -- 172.21.15.32:0/4172472349 wait complete. 2024-05-12T03:51:43.149 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:42 smithi032 ceph-mon[40008]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T03:51:43.149 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:42 smithi032 ceph-mon[40008]: from='mgr.14160 172.21.15.32:0/1781291074' entity='mgr.smithi032.ukatkq' 2024-05-12T03:51:43.407 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi033 2024-05-12T03:51:43.408 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-12T03:51:43.408 DEBUG:teuthology.orchestra.run.smithi033:> dd of=/etc/ceph/ceph.conf 2024-05-12T03:51:43.430 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-12T03:51:43.430 DEBUG:teuthology.orchestra.run.smithi033:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-12T03:51:43.492 INFO:tasks.cephadm:Adding host smithi033 to orchestrator... 2024-05-12T03:51:43.493 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- ceph orch host add smithi033 2024-05-12T03:51:44.085 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T03:51:45.253 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:45 smithi032 ceph-mon[40008]: from='mgr.14160 172.21.15.32:0/1781291074' entity='mgr.smithi032.ukatkq' 2024-05-12T03:51:45.254 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:45 smithi032 ceph-mon[40008]: from='mgr.14160 172.21.15.32:0/1781291074' entity='mgr.smithi032.ukatkq' 2024-05-12T03:51:45.254 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:45 smithi032 ceph-mon[40008]: from='mgr.14160 172.21.15.32:0/1781291074' entity='mgr.smithi032.ukatkq' 2024-05-12T03:51:45.254 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:45 smithi032 ceph-mon[40008]: from='mgr.14160 172.21.15.32:0/1781291074' entity='mgr.smithi032.ukatkq' 2024-05-12T03:51:45.254 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:45 smithi032 ceph-mon[40008]: Deploying daemon node-exporter.smithi032 on smithi032 2024-05-12T03:51:45.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:45.836+0000 7f4b075fc700 1 -- 172.21.15.32:0/2673651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b000fb940 msgr2=0x7f4b000fbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:45.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:45.836+0000 7f4b075fc700 1 --2- 172.21.15.32:0/2673651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b000fb940 0x7f4b000fbd50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4afc0048c0 tx=0x7f4afc009e30 comp rx=0 tx=0).stop 2024-05-12T03:51:45.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:45.836+0000 7f4b075fc700 1 -- 172.21.15.32:0/2673651 shutdown_connections 2024-05-12T03:51:45.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:45.836+0000 7f4b075fc700 1 --2- 172.21.15.32:0/2673651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b000fb940 0x7f4b000fbd50 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:45.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:45.836+0000 7f4b075fc700 1 -- 172.21.15.32:0/2673651 >> 172.21.15.32:0/2673651 conn(0x7f4b000f70a0 msgr2=0x7f4b000f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:51:45.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:45.836+0000 7f4b075fc700 1 -- 172.21.15.32:0/2673651 shutdown_connections 2024-05-12T03:51:45.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:45.836+0000 7f4b075fc700 1 -- 172.21.15.32:0/2673651 wait complete. 2024-05-12T03:51:45.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:45.836+0000 7f4b075fc700 1 Processor -- start 2024-05-12T03:51:45.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:45.836+0000 7f4b075fc700 1 -- start start 2024-05-12T03:51:45.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:45.836+0000 7f4b075fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b000fb940 0x7f4b00065bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:45.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:45.836+0000 7f4b075fc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0010fc90 con 0x7f4b000fb940 2024-05-12T03:51:45.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:45.837+0000 7f4b05398700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b000fb940 0x7f4b00065bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:51:45.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:45.837+0000 7f4b05398700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b000fb940 0x7f4b00065bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56738/0 (socket says 172.21.15.32:56738) 2024-05-12T03:51:45.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:45.837+0000 7f4b05398700 1 -- 172.21.15.32:0/3036716750 learned_addr learned my addr 172.21.15.32:0/3036716750 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:51:45.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:45.837+0000 7f4b05398700 1 -- 172.21.15.32:0/3036716750 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4afc005040 con 0x7f4b000fb940 2024-05-12T03:51:45.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:45.837+0000 7f4b05398700 1 --2- 172.21.15.32:0/3036716750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b000fb940 0x7f4b00065bd0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4afc009300 tx=0x7f4afc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:51:45.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:45.837+0000 7f4af5ffb700 1 -- 172.21.15.32:0/3036716750 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4afc009600 con 0x7f4b000fb940 2024-05-12T03:51:45.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:45.837+0000 7f4af5ffb700 1 -- 172.21.15.32:0/3036716750 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4afc009760 con 0x7f4b000fb940 2024-05-12T03:51:45.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:45.837+0000 7f4b075fc700 1 -- 172.21.15.32:0/3036716750 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b0010fdd0 con 0x7f4b000fb940 2024-05-12T03:51:45.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:45.837+0000 7f4af5ffb700 1 -- 172.21.15.32:0/3036716750 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4afc01c5b0 con 0x7f4b000fb940 2024-05-12T03:51:45.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:45.837+0000 7f4b075fc700 1 -- 172.21.15.32:0/3036716750 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b001102c0 con 0x7f4b000fb940 2024-05-12T03:51:45.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:45.838+0000 7f4af5ffb700 1 -- 172.21.15.32:0/3036716750 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7f4afc01c870 con 0x7f4b000fb940 2024-05-12T03:51:45.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:45.838+0000 7f4b075fc700 1 -- 172.21.15.32:0/3036716750 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b00106040 con 0x7f4b000fb940 2024-05-12T03:51:45.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:45.839+0000 7f4af5ffb700 1 --2- 172.21.15.32:0/3036716750 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f4aec036880 0x7f4aec038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:45.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:45.839+0000 7f4af5ffb700 1 -- 172.21.15.32:0/3036716750 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f4afc049b50 con 0x7f4b000fb940 2024-05-12T03:51:45.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:45.839+0000 7f4b04b97700 1 --2- 172.21.15.32:0/3036716750 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f4aec036880 0x7f4aec038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:51:45.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:45.839+0000 7f4b04b97700 1 --2- 172.21.15.32:0/3036716750 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f4aec036880 0x7f4aec038d30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4af0009990 tx=0x7f4af0006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:51:45.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:45.843+0000 7f4af5ffb700 1 -- 172.21.15.32:0/3036716750 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f4afc013420 con 0x7f4b000fb940 2024-05-12T03:51:46.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:46.042+0000 7f4b075fc700 1 -- 172.21.15.32:0/3036716750 --> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi033", "target": ["mon-mgr", ""]}) v1 -- 0x7f4b0005e3a0 con 0x7f4aec036880 2024-05-12T03:51:46.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:46 smithi032 ceph-mon[40008]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:51:46.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:46 smithi032 ceph-mon[40008]: from='mgr.14160 172.21.15.32:0/1781291074' entity='mgr.smithi032.ukatkq' 2024-05-12T03:51:47.786 INFO:teuthology.orchestra.run.smithi032.stdout:Added host 'smithi033' with addr '172.21.15.33' 2024-05-12T03:51:47.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:47.786+0000 7f4af5ffb700 1 -- 172.21.15.32:0/3036716750 <== mgr.14160 v2:172.21.15.32:6800/1988399583 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f4b0005e3a0 con 0x7f4aec036880 2024-05-12T03:51:47.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:47.788+0000 7f4b075fc700 1 -- 172.21.15.32:0/3036716750 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f4aec036880 msgr2=0x7f4aec038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:47.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:47.788+0000 7f4b075fc700 1 --2- 172.21.15.32:0/3036716750 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f4aec036880 0x7f4aec038d30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4af0009990 tx=0x7f4af0006e40 comp rx=0 tx=0).stop 2024-05-12T03:51:47.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:47.788+0000 7f4b075fc700 1 -- 172.21.15.32:0/3036716750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b000fb940 msgr2=0x7f4b00065bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:47.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:47.788+0000 7f4b075fc700 1 --2- 172.21.15.32:0/3036716750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b000fb940 0x7f4b00065bd0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4afc009300 tx=0x7f4afc009e30 comp rx=0 tx=0).stop 2024-05-12T03:51:47.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:47.788+0000 7f4b075fc700 1 -- 172.21.15.32:0/3036716750 shutdown_connections 2024-05-12T03:51:47.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:47.788+0000 7f4b075fc700 1 --2- 172.21.15.32:0/3036716750 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f4aec036880 0x7f4aec038d30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:47.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:47.788+0000 7f4b075fc700 1 --2- 172.21.15.32:0/3036716750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b000fb940 0x7f4b00065bd0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:47.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:47.788+0000 7f4b075fc700 1 -- 172.21.15.32:0/3036716750 >> 172.21.15.32:0/3036716750 conn(0x7f4b000f70a0 msgr2=0x7f4b000f7940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:51:47.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:47.788+0000 7f4b075fc700 1 -- 172.21.15.32:0/3036716750 shutdown_connections 2024-05-12T03:51:47.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:47.788+0000 7f4b075fc700 1 -- 172.21.15.32:0/3036716750 wait complete. 2024-05-12T03:51:47.789 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:47 smithi032 ceph-mon[40008]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi033", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T03:51:48.425 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:48 smithi032 ceph-mon[40008]: Deploying cephadm binary to smithi033 2024-05-12T03:51:48.426 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:48 smithi032 ceph-mon[40008]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:51:48.426 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:48 smithi032 ceph-mon[40008]: from='mgr.14160 172.21.15.32:0/1781291074' entity='mgr.smithi032.ukatkq' 2024-05-12T03:51:48.791 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- ceph orch host ls --format=json 2024-05-12T03:51:49.359 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T03:51:49.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:49 smithi032 ceph-mon[40008]: Added host smithi033 2024-05-12T03:51:50.789 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:50 smithi032 ceph-mon[40008]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:51:50.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:50 smithi032 ceph-mon[40008]: from='mgr.14160 172.21.15.32:0/1781291074' entity='mgr.smithi032.ukatkq' 2024-05-12T03:51:50.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:50 smithi032 ceph-mon[40008]: from='mgr.14160 172.21.15.32:0/1781291074' entity='mgr.smithi032.ukatkq' 2024-05-12T03:51:50.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:50 smithi032 ceph-mon[40008]: from='mgr.14160 172.21.15.32:0/1781291074' entity='mgr.smithi032.ukatkq' 2024-05-12T03:51:50.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:50 smithi032 ceph-mon[40008]: from='mgr.14160 172.21.15.32:0/1781291074' entity='mgr.smithi032.ukatkq' 2024-05-12T03:51:50.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:50 smithi032 ceph-mon[40008]: Deploying daemon alertmanager.smithi032 on smithi032 2024-05-12T03:51:50.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:50 smithi032 ceph-mon[40008]: from='mgr.14160 172.21.15.32:0/1781291074' entity='mgr.smithi032.ukatkq' 2024-05-12T03:51:52.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:52.016+0000 7fc3f330c700 1 -- 172.21.15.32:0/2084335591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3ec065cb0 msgr2=0x7fc3ec0ff6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:52.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:52.016+0000 7fc3f330c700 1 --2- 172.21.15.32:0/2084335591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3ec065cb0 0x7fc3ec0ff6b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc3e80048c0 tx=0x7fc3e8009e30 comp rx=0 tx=0).stop 2024-05-12T03:51:52.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:52.016+0000 7fc3f330c700 1 -- 172.21.15.32:0/2084335591 shutdown_connections 2024-05-12T03:51:52.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:52.016+0000 7fc3f330c700 1 --2- 172.21.15.32:0/2084335591 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3ec065cb0 0x7fc3ec0ff6b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:52.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:52.016+0000 7fc3f330c700 1 -- 172.21.15.32:0/2084335591 >> 172.21.15.32:0/2084335591 conn(0x7fc3ec0f70a0 msgr2=0x7fc3ec0f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:51:52.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:52.017+0000 7fc3f330c700 1 -- 172.21.15.32:0/2084335591 shutdown_connections 2024-05-12T03:51:52.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:52.017+0000 7fc3f330c700 1 -- 172.21.15.32:0/2084335591 wait complete. 2024-05-12T03:51:52.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:52.017+0000 7fc3f330c700 1 Processor -- start 2024-05-12T03:51:52.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:52.017+0000 7fc3f330c700 1 -- start start 2024-05-12T03:51:52.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:52.017+0000 7fc3f330c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3ec065cb0 0x7fc3ec0fd000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:52.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:52.017+0000 7fc3f330c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ec0fd540 con 0x7fc3ec065cb0 2024-05-12T03:51:52.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:52.017+0000 7fc3f10a8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3ec065cb0 0x7fc3ec0fd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:51:52.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:52.017+0000 7fc3f10a8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3ec065cb0 0x7fc3ec0fd000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37976/0 (socket says 172.21.15.32:37976) 2024-05-12T03:51:52.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:52.017+0000 7fc3f10a8700 1 -- 172.21.15.32:0/476001541 learned_addr learned my addr 172.21.15.32:0/476001541 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:51:52.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:52.018+0000 7fc3f10a8700 1 -- 172.21.15.32:0/476001541 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3e8005040 con 0x7fc3ec065cb0 2024-05-12T03:51:52.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:52.018+0000 7fc3f10a8700 1 --2- 172.21.15.32:0/476001541 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3ec065cb0 0x7fc3ec0fd000 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc3e800b3b0 tx=0x7fc3e8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:51:52.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:52.018+0000 7fc3e1ffb700 1 -- 172.21.15.32:0/476001541 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc3e80095d0 con 0x7fc3ec065cb0 2024-05-12T03:51:52.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:52.018+0000 7fc3e1ffb700 1 -- 172.21.15.32:0/476001541 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc3e8009730 con 0x7fc3ec065cb0 2024-05-12T03:51:52.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:52.018+0000 7fc3f330c700 1 -- 172.21.15.32:0/476001541 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3ec0fb610 con 0x7fc3ec065cb0 2024-05-12T03:51:52.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:52.018+0000 7fc3e1ffb700 1 -- 172.21.15.32:0/476001541 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc3e801c5b0 con 0x7fc3ec065cb0 2024-05-12T03:51:52.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:52.018+0000 7fc3f330c700 1 -- 172.21.15.32:0/476001541 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3ec0fbb00 con 0x7fc3ec065cb0 2024-05-12T03:51:52.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:52.019+0000 7fc3e1ffb700 1 -- 172.21.15.32:0/476001541 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7fc3e801c830 con 0x7fc3ec065cb0 2024-05-12T03:51:52.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:52.019+0000 7fc3e1ffb700 1 --2- 172.21.15.32:0/476001541 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7fc3d8036880 0x7fc3d8038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:52.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:52.019+0000 7fc3e1ffb700 1 -- 172.21.15.32:0/476001541 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fc3e8049a50 con 0x7fc3ec065cb0 2024-05-12T03:51:52.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:52.019+0000 7fc3f08a7700 1 --2- 172.21.15.32:0/476001541 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7fc3d8036880 0x7fc3d8038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:51:52.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:52.019+0000 7fc3f330c700 1 -- 172.21.15.32:0/476001541 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3ec05f5c0 con 0x7fc3ec065cb0 2024-05-12T03:51:52.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:52.020+0000 7fc3f08a7700 1 --2- 172.21.15.32:0/476001541 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7fc3d8036880 0x7fc3d8038d30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc009990 tx=0x7fc3dc006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:51:52.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:52.024+0000 7fc3e1ffb700 1 -- 172.21.15.32:0/476001541 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fc3e8023300 con 0x7fc3ec065cb0 2024-05-12T03:51:52.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:52.239+0000 7fc3f330c700 1 -- 172.21.15.32:0/476001541 --> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc3ec0fbd40 con 0x7fc3d8036880 2024-05-12T03:51:52.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:52.240+0000 7fc3e1ffb700 1 -- 172.21.15.32:0/476001541 <== mgr.14160 v2:172.21.15.32:6800/1988399583 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+159 (secure 0 0 0) 0x7fc3ec0fbd40 con 0x7fc3d8036880 2024-05-12T03:51:52.240 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:51:52.240 INFO:teuthology.orchestra.run.smithi032.stdout:[{"addr": "172.21.15.32", "hostname": "smithi032", "labels": [], "status": ""}, {"addr": "172.21.15.33", "hostname": "smithi033", "labels": [], "status": ""}] 2024-05-12T03:51:52.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:52.242+0000 7fc3f330c700 1 -- 172.21.15.32:0/476001541 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7fc3d8036880 msgr2=0x7fc3d8038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:52.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:52.242+0000 7fc3f330c700 1 --2- 172.21.15.32:0/476001541 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7fc3d8036880 0x7fc3d8038d30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc009990 tx=0x7fc3dc006e40 comp rx=0 tx=0).stop 2024-05-12T03:51:52.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:52.243+0000 7fc3f330c700 1 -- 172.21.15.32:0/476001541 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3ec065cb0 msgr2=0x7fc3ec0fd000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:52.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:52.243+0000 7fc3f330c700 1 --2- 172.21.15.32:0/476001541 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3ec065cb0 0x7fc3ec0fd000 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc3e800b3b0 tx=0x7fc3e8009e30 comp rx=0 tx=0).stop 2024-05-12T03:51:52.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:52.243+0000 7fc3f330c700 1 -- 172.21.15.32:0/476001541 shutdown_connections 2024-05-12T03:51:52.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:52.243+0000 7fc3f330c700 1 --2- 172.21.15.32:0/476001541 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7fc3d8036880 0x7fc3d8038d30 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:52.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:52.243+0000 7fc3f330c700 1 --2- 172.21.15.32:0/476001541 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3ec065cb0 0x7fc3ec0fd000 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:52.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:52.243+0000 7fc3f330c700 1 -- 172.21.15.32:0/476001541 >> 172.21.15.32:0/476001541 conn(0x7fc3ec0f70a0 msgr2=0x7fc3ec0f7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:51:52.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:52.243+0000 7fc3f330c700 1 -- 172.21.15.32:0/476001541 shutdown_connections 2024-05-12T03:51:52.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:52.243+0000 7fc3f330c700 1 -- 172.21.15.32:0/476001541 wait complete. 2024-05-12T03:51:52.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:52 smithi032 ceph-mon[40008]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:51:53.131 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-12T03:51:53.132 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- ceph osd crush tunables default 2024-05-12T03:51:53.789 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:53 smithi032 ceph-mon[40008]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-12T03:51:54.166 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T03:51:54.768 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:54 smithi032 ceph-mon[40008]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:51:56.789 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:56 smithi032 ceph-mon[40008]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:51:58.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:58.081+0000 7fdb3f078700 1 -- 172.21.15.32:0/2624279503 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb380fb940 msgr2=0x7fdb380fbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:58.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:58.081+0000 7fdb3f078700 1 --2- 172.21.15.32:0/2624279503 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb380fb940 0x7fdb380fbd50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdb280048c0 tx=0x7fdb28009e30 comp rx=0 tx=0).stop 2024-05-12T03:51:58.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:58.081+0000 7fdb3f078700 1 -- 172.21.15.32:0/2624279503 shutdown_connections 2024-05-12T03:51:58.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:58.081+0000 7fdb3f078700 1 --2- 172.21.15.32:0/2624279503 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb380fb940 0x7fdb380fbd50 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:58.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:58.081+0000 7fdb3f078700 1 -- 172.21.15.32:0/2624279503 >> 172.21.15.32:0/2624279503 conn(0x7fdb380f70a0 msgr2=0x7fdb380f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:51:58.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:58.081+0000 7fdb3f078700 1 -- 172.21.15.32:0/2624279503 shutdown_connections 2024-05-12T03:51:58.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:58.081+0000 7fdb3f078700 1 -- 172.21.15.32:0/2624279503 wait complete. 2024-05-12T03:51:58.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:58.082+0000 7fdb3f078700 1 Processor -- start 2024-05-12T03:51:58.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:58.082+0000 7fdb3f078700 1 -- start start 2024-05-12T03:51:58.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:58.082+0000 7fdb3f078700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb380fb940 0x7fdb3810bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:58.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:58.082+0000 7fdb3f078700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3810c440 con 0x7fdb380fb940 2024-05-12T03:51:58.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:58.082+0000 7fdb3ce14700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb380fb940 0x7fdb3810bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:51:58.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:58.083+0000 7fdb3ce14700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb380fb940 0x7fdb3810bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37994/0 (socket says 172.21.15.32:37994) 2024-05-12T03:51:58.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:58.083+0000 7fdb3ce14700 1 -- 172.21.15.32:0/2276454721 learned_addr learned my addr 172.21.15.32:0/2276454721 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:51:58.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:58.083+0000 7fdb3ce14700 1 -- 172.21.15.32:0/2276454721 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb28005040 con 0x7fdb380fb940 2024-05-12T03:51:58.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:58.083+0000 7fdb3ce14700 1 --2- 172.21.15.32:0/2276454721 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb380fb940 0x7fdb3810bf00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdb28009300 tx=0x7fdb28009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:51:58.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:58.083+0000 7fdb357fa700 1 -- 172.21.15.32:0/2276454721 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdb28009600 con 0x7fdb380fb940 2024-05-12T03:51:58.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:58.083+0000 7fdb357fa700 1 -- 172.21.15.32:0/2276454721 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdb28009760 con 0x7fdb380fb940 2024-05-12T03:51:58.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:58.083+0000 7fdb3f078700 1 -- 172.21.15.32:0/2276454721 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb3810c640 con 0x7fdb380fb940 2024-05-12T03:51:58.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:58.083+0000 7fdb357fa700 1 -- 172.21.15.32:0/2276454721 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdb2801c5b0 con 0x7fdb380fb940 2024-05-12T03:51:58.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:58.083+0000 7fdb3f078700 1 -- 172.21.15.32:0/2276454721 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb3810d0d0 con 0x7fdb380fb940 2024-05-12T03:51:58.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:58.084+0000 7fdb357fa700 1 -- 172.21.15.32:0/2276454721 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7fdb2801c870 con 0x7fdb380fb940 2024-05-12T03:51:58.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:58.084+0000 7fdb3f078700 1 -- 172.21.15.32:0/2276454721 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb38101d10 con 0x7fdb380fb940 2024-05-12T03:51:58.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:58.084+0000 7fdb357fa700 1 --2- 172.21.15.32:0/2276454721 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7fdb20036880 0x7fdb20038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:51:58.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:58.084+0000 7fdb357fa700 1 -- 172.21.15.32:0/2276454721 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fdb28049c60 con 0x7fdb380fb940 2024-05-12T03:51:58.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:58.084+0000 7fdb37fff700 1 --2- 172.21.15.32:0/2276454721 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7fdb20036880 0x7fdb20038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:51:58.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:58.085+0000 7fdb37fff700 1 --2- 172.21.15.32:0/2276454721 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7fdb20036880 0x7fdb20038d30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c009990 tx=0x7fdb2c006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:51:58.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:58.089+0000 7fdb357fa700 1 -- 172.21.15.32:0/2276454721 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fdb28011730 con 0x7fdb380fb940 2024-05-12T03:51:58.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:58.294+0000 7fdb3f078700 1 -- 172.21.15.32:0/2276454721 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fdb3805f5c0 con 0x7fdb380fb940 2024-05-12T03:51:58.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:58.407+0000 7fdb357fa700 1 -- 172.21.15.32:0/2276454721 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7fdb28011370 con 0x7fdb380fb940 2024-05-12T03:51:58.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:58.409+0000 7fdb3f078700 1 -- 172.21.15.32:0/2276454721 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7fdb20036880 msgr2=0x7fdb20038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:58.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:58.409+0000 7fdb3f078700 1 --2- 172.21.15.32:0/2276454721 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7fdb20036880 0x7fdb20038d30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c009990 tx=0x7fdb2c006e40 comp rx=0 tx=0).stop 2024-05-12T03:51:58.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:58.409+0000 7fdb3f078700 1 -- 172.21.15.32:0/2276454721 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb380fb940 msgr2=0x7fdb3810bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:51:58.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:58.409+0000 7fdb3f078700 1 --2- 172.21.15.32:0/2276454721 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb380fb940 0x7fdb3810bf00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdb28009300 tx=0x7fdb28009e30 comp rx=0 tx=0).stop 2024-05-12T03:51:58.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:58.409+0000 7fdb3f078700 1 -- 172.21.15.32:0/2276454721 shutdown_connections 2024-05-12T03:51:58.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:58.409+0000 7fdb3f078700 1 --2- 172.21.15.32:0/2276454721 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7fdb20036880 0x7fdb20038d30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:58.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:58.409+0000 7fdb3f078700 1 --2- 172.21.15.32:0/2276454721 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb380fb940 0x7fdb3810bf00 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:51:58.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:58.409+0000 7fdb3f078700 1 -- 172.21.15.32:0/2276454721 >> 172.21.15.32:0/2276454721 conn(0x7fdb380f70a0 msgr2=0x7fdb380f7940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:51:58.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:58.410+0000 7fdb3f078700 1 -- 172.21.15.32:0/2276454721 shutdown_connections 2024-05-12T03:51:58.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:51:58.410+0000 7fdb3f078700 1 -- 172.21.15.32:0/2276454721 wait complete. 2024-05-12T03:51:58.410 INFO:teuthology.orchestra.run.smithi032.stderr:adjusted tunables profile to default 2024-05-12T03:51:58.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:58 smithi032 ceph-mon[40008]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:51:58.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:58 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/2276454721' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-12T03:51:59.286 INFO:tasks.cephadm:Adding mon.smithi032 on smithi032 2024-05-12T03:51:59.286 INFO:tasks.cephadm:Adding mon.smithi033 on smithi033 2024-05-12T03:51:59.286 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- ceph orch apply mon '2;smithi032:172.21.15.32=smithi032;smithi033:172.21.15.33=smithi033' 2024-05-12T03:51:59.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:59 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/2276454721' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-12T03:51:59.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:51:59 smithi032 ceph-mon[40008]: osdmap e4: 0 total, 0 up, 0 in 2024-05-12T03:51:59.579 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-12T03:51:59.648 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-12T03:52:00.789 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:00 smithi032 ceph-mon[40008]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:52:00.789 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:00 smithi032 ceph-mon[40008]: from='mgr.14160 172.21.15.32:0/1781291074' entity='mgr.smithi032.ukatkq' 2024-05-12T03:52:00.789 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:00 smithi032 ceph-mon[40008]: from='mgr.14160 172.21.15.32:0/1781291074' entity='mgr.smithi032.ukatkq' 2024-05-12T03:52:00.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:00 smithi032 ceph-mon[40008]: from='mgr.14160 172.21.15.32:0/1781291074' entity='mgr.smithi032.ukatkq' 2024-05-12T03:52:00.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:00 smithi032 ceph-mon[40008]: from='mgr.14160 172.21.15.32:0/1781291074' entity='mgr.smithi032.ukatkq' 2024-05-12T03:52:00.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:00 smithi032 ceph-mon[40008]: from='mgr.14160 172.21.15.32:0/1781291074' entity='mgr.smithi032.ukatkq' 2024-05-12T03:52:00.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:00 smithi032 ceph-mon[40008]: from='mgr.14160 172.21.15.32:0/1781291074' entity='mgr.smithi032.ukatkq' 2024-05-12T03:52:00.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:00 smithi032 ceph-mon[40008]: from='mgr.14160 172.21.15.32:0/1781291074' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-05-12T03:52:00.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:00 smithi032 ceph-mon[40008]: from='mgr.14160 172.21.15.32:0/1781291074' entity='mgr.smithi032.ukatkq' 2024-05-12T03:52:00.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:00 smithi032 ceph-mon[40008]: from='mgr.14160 172.21.15.32:0/1781291074' entity='mgr.smithi032.ukatkq' 2024-05-12T03:52:01.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:01.448+0000 7f9332e4c700 1 -- 172.21.15.33:0/236803839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f932c0fb940 msgr2=0x7f932c0fbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:52:01.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:01.448+0000 7f9332e4c700 1 --2- 172.21.15.33:0/236803839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f932c0fb940 0x7f932c0fbd50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f93200048c0 tx=0x7f9320009e30 comp rx=0 tx=0).stop 2024-05-12T03:52:01.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:01.449+0000 7f9332e4c700 1 -- 172.21.15.33:0/236803839 shutdown_connections 2024-05-12T03:52:01.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:01.449+0000 7f9332e4c700 1 --2- 172.21.15.33:0/236803839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f932c0fb940 0x7f932c0fbd50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:52:01.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:01.449+0000 7f9332e4c700 1 -- 172.21.15.33:0/236803839 >> 172.21.15.33:0/236803839 conn(0x7f932c0f70a0 msgr2=0x7f932c0f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:52:01.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:01.449+0000 7f9332e4c700 1 -- 172.21.15.33:0/236803839 shutdown_connections 2024-05-12T03:52:01.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:01.449+0000 7f9332e4c700 1 -- 172.21.15.33:0/236803839 wait complete. 2024-05-12T03:52:01.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:01.449+0000 7f9332e4c700 1 Processor -- start 2024-05-12T03:52:01.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:01.449+0000 7f9332e4c700 1 -- start start 2024-05-12T03:52:01.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:01.450+0000 7f9332e4c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f932c0fb940 0x7f932c10bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:52:01.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:01.450+0000 7f9332e4c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f932c10c410 con 0x7f932c0fb940 2024-05-12T03:52:01.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:01.450+0000 7f9330be8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f932c0fb940 0x7f932c10bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:52:01.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:01.450+0000 7f9330be8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f932c0fb940 0x7f932c10bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.33:47284/0 (socket says 172.21.15.33:47284) 2024-05-12T03:52:01.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:01.450+0000 7f9330be8700 1 -- 172.21.15.33:0/3551690507 learned_addr learned my addr 172.21.15.33:0/3551690507 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-12T03:52:01.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:01.450+0000 7f9330be8700 1 -- 172.21.15.33:0/3551690507 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9320005040 con 0x7f932c0fb940 2024-05-12T03:52:01.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:01.451+0000 7f9330be8700 1 --2- 172.21.15.33:0/3551690507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f932c0fb940 0x7f932c10bed0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9320009300 tx=0x7f9320009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:52:01.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:01.451+0000 7f93297fa700 1 -- 172.21.15.33:0/3551690507 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9320009600 con 0x7f932c0fb940 2024-05-12T03:52:01.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:01.451+0000 7f9332e4c700 1 -- 172.21.15.33:0/3551690507 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f932c10c610 con 0x7f932c0fb940 2024-05-12T03:52:01.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:01.451+0000 7f93297fa700 1 -- 172.21.15.33:0/3551690507 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9320009760 con 0x7f932c0fb940 2024-05-12T03:52:01.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:01.451+0000 7f93297fa700 1 -- 172.21.15.33:0/3551690507 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f932001c5b0 con 0x7f932c0fb940 2024-05-12T03:52:01.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:01.451+0000 7f9332e4c700 1 -- 172.21.15.33:0/3551690507 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f932c10d180 con 0x7f932c0fb940 2024-05-12T03:52:01.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:01.452+0000 7f93297fa700 1 -- 172.21.15.33:0/3551690507 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7f9320013420 con 0x7f932c0fb940 2024-05-12T03:52:01.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:01.452+0000 7f9332e4c700 1 -- 172.21.15.33:0/3551690507 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f932c101d10 con 0x7f932c0fb940 2024-05-12T03:52:01.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:01.452+0000 7f93297fa700 1 --2- 172.21.15.33:0/3551690507 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f931c036880 0x7f931c038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:52:01.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:01.452+0000 7f93297fa700 1 -- 172.21.15.33:0/3551690507 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f9320049af0 con 0x7f932c0fb940 2024-05-12T03:52:01.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:01.452+0000 7f932bfff700 1 --2- 172.21.15.33:0/3551690507 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f931c036880 0x7f931c038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:52:01.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:01.453+0000 7f932bfff700 1 --2- 172.21.15.33:0/3551690507 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f931c036880 0x7f931c038d30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9318009990 tx=0x7f9318006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:52:01.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:01.458+0000 7f93297fa700 1 -- 172.21.15.33:0/3551690507 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f9320013960 con 0x7f932c0fb940 2024-05-12T03:52:01.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:01.664+0000 7f9332e4c700 1 -- 172.21.15.33:0/3551690507 --> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi032:172.21.15.32=smithi032;smithi033:172.21.15.33=smithi033", "target": ["mon-mgr", ""]}) v1 -- 0x7f932c10d470 con 0x7f931c036880 2024-05-12T03:52:01.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:01.670+0000 7f93297fa700 1 -- 172.21.15.33:0/3551690507 <== mgr.14160 v2:172.21.15.32:6800/1988399583 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f932c10d470 con 0x7f931c036880 2024-05-12T03:52:01.671 INFO:teuthology.orchestra.run.smithi033.stdout:Scheduled mon update... 2024-05-12T03:52:01.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:01.672+0000 7f9332e4c700 1 -- 172.21.15.33:0/3551690507 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f931c036880 msgr2=0x7f931c038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:52:01.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:01.672+0000 7f9332e4c700 1 --2- 172.21.15.33:0/3551690507 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f931c036880 0x7f931c038d30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9318009990 tx=0x7f9318006e40 comp rx=0 tx=0).stop 2024-05-12T03:52:01.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:01.672+0000 7f9332e4c700 1 -- 172.21.15.33:0/3551690507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f932c0fb940 msgr2=0x7f932c10bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:52:01.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:01.672+0000 7f9332e4c700 1 --2- 172.21.15.33:0/3551690507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f932c0fb940 0x7f932c10bed0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9320009300 tx=0x7f9320009e30 comp rx=0 tx=0).stop 2024-05-12T03:52:01.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:01.672+0000 7f9332e4c700 1 -- 172.21.15.33:0/3551690507 shutdown_connections 2024-05-12T03:52:01.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:01.672+0000 7f9332e4c700 1 --2- 172.21.15.33:0/3551690507 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f931c036880 0x7f931c038d30 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:52:01.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:01.672+0000 7f9332e4c700 1 --2- 172.21.15.33:0/3551690507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f932c0fb940 0x7f932c10bed0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:52:01.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:01.672+0000 7f9332e4c700 1 -- 172.21.15.33:0/3551690507 >> 172.21.15.33:0/3551690507 conn(0x7f932c0f70a0 msgr2=0x7f932c0f7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:52:01.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:01.672+0000 7f9332e4c700 1 -- 172.21.15.33:0/3551690507 shutdown_connections 2024-05-12T03:52:01.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:01.672+0000 7f9332e4c700 1 -- 172.21.15.33:0/3551690507 wait complete. 2024-05-12T03:52:01.789 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:01 smithi032 ceph-mon[40008]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-05-12T03:52:01.789 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:01 smithi032 ceph-mon[40008]: Deploying daemon grafana.smithi032 on smithi032 2024-05-12T03:52:02.354 DEBUG:teuthology.orchestra.run.smithi033:mon.smithi033> sudo journalctl -f -n 0 -u ceph-a3e178f8-1012-11ef-bc98-c7b262605968@mon.smithi033.service 2024-05-12T03:52:02.356 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-12T03:52:02.357 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-12T03:52:02.390 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:-- Logs begin at Sun 2024-05-12 03:38:52 UTC. -- 2024-05-12T03:52:02.674 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-12T03:52:02.738 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-12T03:52:02.789 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:02 smithi032 ceph-mon[40008]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:52:02.789 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:02 smithi032 ceph-mon[40008]: from='mgr.14160 172.21.15.32:0/1781291074' entity='mgr.smithi032.ukatkq' 2024-05-12T03:52:03.789 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:03 smithi032 ceph-mon[40008]: from='client.14192 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi032:172.21.15.32=smithi032;smithi033:172.21.15.33=smithi033", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T03:52:03.789 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:03 smithi032 ceph-mon[40008]: Saving service mon spec with placement smithi032:172.21.15.32=smithi032;smithi033:172.21.15.33=smithi033;count:2 2024-05-12T03:52:04.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:04.454+0000 7f87e9a4c700 1 -- 172.21.15.33:0/1096272094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87e40fb940 msgr2=0x7f87e40fbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:52:04.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:04.454+0000 7f87e9a4c700 1 --2- 172.21.15.33:0/1096272094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87e40fb940 0x7f87e40fbd50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f87d40048c0 tx=0x7f87d4009e30 comp rx=0 tx=0).stop 2024-05-12T03:52:04.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:04.454+0000 7f87e9a4c700 1 -- 172.21.15.33:0/1096272094 shutdown_connections 2024-05-12T03:52:04.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:04.454+0000 7f87e9a4c700 1 --2- 172.21.15.33:0/1096272094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87e40fb940 0x7f87e40fbd50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:52:04.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:04.454+0000 7f87e9a4c700 1 -- 172.21.15.33:0/1096272094 >> 172.21.15.33:0/1096272094 conn(0x7f87e40f70a0 msgr2=0x7f87e40f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:52:04.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:04.454+0000 7f87e9a4c700 1 -- 172.21.15.33:0/1096272094 shutdown_connections 2024-05-12T03:52:04.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:04.454+0000 7f87e9a4c700 1 -- 172.21.15.33:0/1096272094 wait complete. 2024-05-12T03:52:04.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:04.455+0000 7f87e9a4c700 1 Processor -- start 2024-05-12T03:52:04.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:04.455+0000 7f87e9a4c700 1 -- start start 2024-05-12T03:52:04.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:04.455+0000 7f87e9a4c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87e40fb940 0x7f87e4065bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:52:04.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:04.455+0000 7f87e9a4c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e410daa0 con 0x7f87e40fb940 2024-05-12T03:52:04.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:04.455+0000 7f87e2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87e40fb940 0x7f87e4065bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:52:04.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:04.455+0000 7f87e2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87e40fb940 0x7f87e4065bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.33:47304/0 (socket says 172.21.15.33:47304) 2024-05-12T03:52:04.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:04.455+0000 7f87e2ffd700 1 -- 172.21.15.33:0/1947551675 learned_addr learned my addr 172.21.15.33:0/1947551675 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-12T03:52:04.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:04.455+0000 7f87e2ffd700 1 -- 172.21.15.33:0/1947551675 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d4005040 con 0x7f87e40fb940 2024-05-12T03:52:04.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:04.456+0000 7f87e2ffd700 1 --2- 172.21.15.33:0/1947551675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87e40fb940 0x7f87e4065bb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f87d4009300 tx=0x7f87d4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:52:04.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:04.456+0000 7f87db7fe700 1 -- 172.21.15.33:0/1947551675 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f87d4009600 con 0x7f87e40fb940 2024-05-12T03:52:04.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:04.456+0000 7f87db7fe700 1 -- 172.21.15.33:0/1947551675 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f87d4009760 con 0x7f87e40fb940 2024-05-12T03:52:04.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:04.456+0000 7f87e9a4c700 1 -- 172.21.15.33:0/1947551675 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87e41120b0 con 0x7f87e40fb940 2024-05-12T03:52:04.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:04.456+0000 7f87db7fe700 1 -- 172.21.15.33:0/1947551675 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f87d401c5b0 con 0x7f87e40fb940 2024-05-12T03:52:04.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:04.456+0000 7f87e9a4c700 1 -- 172.21.15.33:0/1947551675 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87e41124e0 con 0x7f87e40fb940 2024-05-12T03:52:04.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:04.457+0000 7f87db7fe700 1 -- 172.21.15.33:0/1947551675 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7f87d4013420 con 0x7f87e40fb940 2024-05-12T03:52:04.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:04.457+0000 7f87e9a4c700 1 -- 172.21.15.33:0/1947551675 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87e4106040 con 0x7f87e40fb940 2024-05-12T03:52:04.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:04.457+0000 7f87db7fe700 1 --2- 172.21.15.33:0/1947551675 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f87d00368d0 0x7f87d0038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:52:04.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:04.457+0000 7f87db7fe700 1 -- 172.21.15.33:0/1947551675 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f87d4049ae0 con 0x7f87e40fb940 2024-05-12T03:52:04.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:04.457+0000 7f87e27fc700 1 --2- 172.21.15.33:0/1947551675 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f87d00368d0 0x7f87d0038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:52:04.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:04.457+0000 7f87e27fc700 1 --2- 172.21.15.33:0/1947551675 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f87d00368d0 0x7f87d0038d80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f87cc009990 tx=0x7f87cc006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:52:04.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:04.462+0000 7f87db7fe700 1 -- 172.21.15.33:0/1947551675 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f87d4013960 con 0x7f87e40fb940 2024-05-12T03:52:04.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:04.743+0000 7f87e9a4c700 1 -- 172.21.15.33:0/1947551675 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f87e405f5c0 con 0x7f87e40fb940 2024-05-12T03:52:04.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:04.743+0000 7f87db7fe700 1 -- 172.21.15.33:0/1947551675 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f87d4011370 con 0x7f87e40fb940 2024-05-12T03:52:04.744 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:52:04.744 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":1,"fsid":"a3e178f8-1012-11ef-bc98-c7b262605968","modified":"2024-05-12T03:50:15.576069Z","created":"2024-05-12T03:50:15.576069Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi032","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:3300","nonce":0},{"type":"v1","addr":"172.21.15.32:6789","nonce":0}]},"addr":"172.21.15.32:6789/0","public_addr":"172.21.15.32:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-12T03:52:04.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:04.745+0000 7f87e9a4c700 1 -- 172.21.15.33:0/1947551675 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f87d00368d0 msgr2=0x7f87d0038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:52:04.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:04.745+0000 7f87e9a4c700 1 --2- 172.21.15.33:0/1947551675 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f87d00368d0 0x7f87d0038d80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f87cc009990 tx=0x7f87cc006e40 comp rx=0 tx=0).stop 2024-05-12T03:52:04.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:04.745+0000 7f87e9a4c700 1 -- 172.21.15.33:0/1947551675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87e40fb940 msgr2=0x7f87e4065bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:52:04.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:04.745+0000 7f87e9a4c700 1 --2- 172.21.15.33:0/1947551675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87e40fb940 0x7f87e4065bb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f87d4009300 tx=0x7f87d4009e30 comp rx=0 tx=0).stop 2024-05-12T03:52:04.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:04.745+0000 7f87e9a4c700 1 -- 172.21.15.33:0/1947551675 shutdown_connections 2024-05-12T03:52:04.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:04.745+0000 7f87e9a4c700 1 --2- 172.21.15.33:0/1947551675 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f87d00368d0 0x7f87d0038d80 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:52:04.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:04.745+0000 7f87e9a4c700 1 --2- 172.21.15.33:0/1947551675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87e40fb940 0x7f87e4065bb0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:52:04.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:04.745+0000 7f87e9a4c700 1 -- 172.21.15.33:0/1947551675 >> 172.21.15.33:0/1947551675 conn(0x7f87e40f70a0 msgr2=0x7f87e40f7940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:52:04.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:04.745+0000 7f87e9a4c700 1 -- 172.21.15.33:0/1947551675 shutdown_connections 2024-05-12T03:52:04.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:04.745+0000 7f87e9a4c700 1 -- 172.21.15.33:0/1947551675 wait complete. 2024-05-12T03:52:04.747 INFO:teuthology.orchestra.run.smithi033.stderr:dumped monmap epoch 1 2024-05-12T03:52:04.789 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:04 smithi032 ceph-mon[40008]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:52:05.789 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:05 smithi032 ceph-mon[40008]: from='client.? 172.21.15.33:0/1947551675' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-12T03:52:06.430 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-12T03:52:06.430 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-12T03:52:06.733 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-12T03:52:06.789 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:06 smithi032 ceph-mon[40008]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:52:06.791 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-12T03:52:08.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:08.655+0000 7fc87598e700 1 -- 172.21.15.33:0/2524877861 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8700fb930 msgr2=0x7fc8700fbd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:52:08.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:08.655+0000 7fc87598e700 1 --2- 172.21.15.33:0/2524877861 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8700fb930 0x7fc8700fbd40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc8600048c0 tx=0x7fc860009e30 comp rx=0 tx=0).stop 2024-05-12T03:52:08.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:08.655+0000 7fc87598e700 1 -- 172.21.15.33:0/2524877861 shutdown_connections 2024-05-12T03:52:08.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:08.655+0000 7fc87598e700 1 --2- 172.21.15.33:0/2524877861 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8700fb930 0x7fc8700fbd40 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:52:08.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:08.655+0000 7fc87598e700 1 -- 172.21.15.33:0/2524877861 >> 172.21.15.33:0/2524877861 conn(0x7fc8700f7070 msgr2=0x7fc8700f94c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:52:08.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:08.655+0000 7fc87598e700 1 -- 172.21.15.33:0/2524877861 shutdown_connections 2024-05-12T03:52:08.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:08.655+0000 7fc87598e700 1 -- 172.21.15.33:0/2524877861 wait complete. 2024-05-12T03:52:08.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:08.656+0000 7fc87598e700 1 Processor -- start 2024-05-12T03:52:08.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:08.656+0000 7fc87598e700 1 -- start start 2024-05-12T03:52:08.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:08.657+0000 7fc87598e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8700fb930 0x7fc87010bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:52:08.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:08.657+0000 7fc87598e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc87010c400 con 0x7fc8700fb930 2024-05-12T03:52:08.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:08.657+0000 7fc86effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8700fb930 0x7fc87010bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:52:08.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:08.657+0000 7fc86effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8700fb930 0x7fc87010bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.33:47322/0 (socket says 172.21.15.33:47322) 2024-05-12T03:52:08.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:08.657+0000 7fc86effd700 1 -- 172.21.15.33:0/1329654762 learned_addr learned my addr 172.21.15.33:0/1329654762 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-12T03:52:08.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:08.657+0000 7fc86effd700 1 -- 172.21.15.33:0/1329654762 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc860005040 con 0x7fc8700fb930 2024-05-12T03:52:08.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:08.657+0000 7fc86effd700 1 --2- 172.21.15.33:0/1329654762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8700fb930 0x7fc87010bec0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc860000c00 tx=0x7fc860009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:52:08.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:08.658+0000 7fc8577fe700 1 -- 172.21.15.33:0/1329654762 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc860009620 con 0x7fc8700fb930 2024-05-12T03:52:08.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:08.658+0000 7fc87598e700 1 -- 172.21.15.33:0/1329654762 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc87010c600 con 0x7fc8700fb930 2024-05-12T03:52:08.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:08.658+0000 7fc8577fe700 1 -- 172.21.15.33:0/1329654762 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc860009780 con 0x7fc8700fb930 2024-05-12T03:52:08.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:08.658+0000 7fc8577fe700 1 -- 172.21.15.33:0/1329654762 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc86001c5b0 con 0x7fc8700fb930 2024-05-12T03:52:08.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:08.658+0000 7fc87598e700 1 -- 172.21.15.33:0/1329654762 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc87010d170 con 0x7fc8700fb930 2024-05-12T03:52:08.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:08.658+0000 7fc8577fe700 1 -- 172.21.15.33:0/1329654762 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7fc86001c7f0 con 0x7fc8700fb930 2024-05-12T03:52:08.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:08.659+0000 7fc8577fe700 1 --2- 172.21.15.33:0/1329654762 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7fc858036880 0x7fc858038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:52:08.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:08.659+0000 7fc8577fe700 1 -- 172.21.15.33:0/1329654762 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fc860049b50 con 0x7fc8700fb930 2024-05-12T03:52:08.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:08.659+0000 7fc86e7fc700 1 --2- 172.21.15.33:0/1329654762 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7fc858036880 0x7fc858038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:52:08.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:08.659+0000 7fc86e7fc700 1 --2- 172.21.15.33:0/1329654762 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7fc858036880 0x7fc858038d30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc864009990 tx=0x7fc864006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:52:08.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:08.659+0000 7fc87598e700 1 -- 172.21.15.33:0/1329654762 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc870101d00 con 0x7fc8700fb930 2024-05-12T03:52:08.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:08.665+0000 7fc8577fe700 1 -- 172.21.15.33:0/1329654762 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fc860011dd0 con 0x7fc8700fb930 2024-05-12T03:52:08.789 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:08 smithi032 ceph-mon[40008]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:52:08.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:08.941+0000 7fc87598e700 1 -- 172.21.15.33:0/1329654762 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc87005f5c0 con 0x7fc8700fb930 2024-05-12T03:52:08.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:08.942+0000 7fc8577fe700 1 -- 172.21.15.33:0/1329654762 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fc87005f5c0 con 0x7fc8700fb930 2024-05-12T03:52:08.943 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:52:08.943 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":1,"fsid":"a3e178f8-1012-11ef-bc98-c7b262605968","modified":"2024-05-12T03:50:15.576069Z","created":"2024-05-12T03:50:15.576069Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi032","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:3300","nonce":0},{"type":"v1","addr":"172.21.15.32:6789","nonce":0}]},"addr":"172.21.15.32:6789/0","public_addr":"172.21.15.32:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-12T03:52:08.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:08.944+0000 7fc87598e700 1 -- 172.21.15.33:0/1329654762 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7fc858036880 msgr2=0x7fc858038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:52:08.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:08.944+0000 7fc87598e700 1 --2- 172.21.15.33:0/1329654762 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7fc858036880 0x7fc858038d30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc864009990 tx=0x7fc864006e40 comp rx=0 tx=0).stop 2024-05-12T03:52:08.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:08.944+0000 7fc87598e700 1 -- 172.21.15.33:0/1329654762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8700fb930 msgr2=0x7fc87010bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:52:08.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:08.944+0000 7fc87598e700 1 --2- 172.21.15.33:0/1329654762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8700fb930 0x7fc87010bec0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc860000c00 tx=0x7fc860009e30 comp rx=0 tx=0).stop 2024-05-12T03:52:08.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:08.944+0000 7fc87598e700 1 -- 172.21.15.33:0/1329654762 shutdown_connections 2024-05-12T03:52:08.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:08.944+0000 7fc87598e700 1 --2- 172.21.15.33:0/1329654762 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7fc858036880 0x7fc858038d30 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:52:08.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:08.944+0000 7fc87598e700 1 --2- 172.21.15.33:0/1329654762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8700fb930 0x7fc87010bec0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:52:08.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:08.944+0000 7fc87598e700 1 -- 172.21.15.33:0/1329654762 >> 172.21.15.33:0/1329654762 conn(0x7fc8700f7070 msgr2=0x7fc8700f7950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:52:08.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:08.944+0000 7fc87598e700 1 -- 172.21.15.33:0/1329654762 shutdown_connections 2024-05-12T03:52:08.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:08.944+0000 7fc87598e700 1 -- 172.21.15.33:0/1329654762 wait complete. 2024-05-12T03:52:08.946 INFO:teuthology.orchestra.run.smithi033.stderr:dumped monmap epoch 1 2024-05-12T03:52:09.789 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:09 smithi032 ceph-mon[40008]: from='client.? 172.21.15.33:0/1329654762' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-12T03:52:10.639 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-12T03:52:10.639 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-12T03:52:10.789 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:10 smithi032 ceph-mon[40008]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:52:10.942 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-12T03:52:11.012 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-12T03:52:12.789 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:12 smithi032 ceph-mon[40008]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:52:12.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:12.819+0000 7f367ad99700 1 -- 172.21.15.33:0/476010362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36740fca30 msgr2=0x7f36740fce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:52:12.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:12.819+0000 7f367ad99700 1 --2- 172.21.15.33:0/476010362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36740fca30 0x7f36740fce40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f36680048c0 tx=0x7f3668009e30 comp rx=0 tx=0).stop 2024-05-12T03:52:12.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:12.819+0000 7f367ad99700 1 -- 172.21.15.33:0/476010362 shutdown_connections 2024-05-12T03:52:12.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:12.819+0000 7f367ad99700 1 --2- 172.21.15.33:0/476010362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36740fca30 0x7f36740fce40 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:52:12.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:12.819+0000 7f367ad99700 1 -- 172.21.15.33:0/476010362 >> 172.21.15.33:0/476010362 conn(0x7f36740f8170 msgr2=0x7f36740fa5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:52:12.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:12.819+0000 7f367ad99700 1 -- 172.21.15.33:0/476010362 shutdown_connections 2024-05-12T03:52:12.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:12.819+0000 7f367ad99700 1 -- 172.21.15.33:0/476010362 wait complete. 2024-05-12T03:52:12.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:12.820+0000 7f367ad99700 1 Processor -- start 2024-05-12T03:52:12.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:12.820+0000 7f367ad99700 1 -- start start 2024-05-12T03:52:12.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:12.820+0000 7f367ad99700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36740fca30 0x7f367410cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:52:12.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:12.820+0000 7f367ad99700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f367410d500 con 0x7f36740fca30 2024-05-12T03:52:12.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:12.820+0000 7f3678b35700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36740fca30 0x7f367410cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:52:12.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:12.820+0000 7f3678b35700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36740fca30 0x7f367410cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.33:49002/0 (socket says 172.21.15.33:49002) 2024-05-12T03:52:12.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:12.820+0000 7f3678b35700 1 -- 172.21.15.33:0/3512265047 learned_addr learned my addr 172.21.15.33:0/3512265047 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-12T03:52:12.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:12.821+0000 7f3678b35700 1 -- 172.21.15.33:0/3512265047 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3668005040 con 0x7f36740fca30 2024-05-12T03:52:12.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:12.821+0000 7f3678b35700 1 --2- 172.21.15.33:0/3512265047 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36740fca30 0x7f367410cfc0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3668009300 tx=0x7f3668009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:52:12.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:12.821+0000 7f36717fa700 1 -- 172.21.15.33:0/3512265047 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3668009600 con 0x7f36740fca30 2024-05-12T03:52:12.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:12.821+0000 7f367ad99700 1 -- 172.21.15.33:0/3512265047 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f367410d700 con 0x7f36740fca30 2024-05-12T03:52:12.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:12.821+0000 7f367ad99700 1 -- 172.21.15.33:0/3512265047 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f367410e270 con 0x7f36740fca30 2024-05-12T03:52:12.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:12.821+0000 7f36717fa700 1 -- 172.21.15.33:0/3512265047 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3668009760 con 0x7f36740fca30 2024-05-12T03:52:12.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:12.821+0000 7f36717fa700 1 -- 172.21.15.33:0/3512265047 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f36680255c0 con 0x7f36740fca30 2024-05-12T03:52:12.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:12.822+0000 7f367ad99700 1 -- 172.21.15.33:0/3512265047 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3674102e00 con 0x7f36740fca30 2024-05-12T03:52:12.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:12.822+0000 7f36717fa700 1 -- 172.21.15.33:0/3512265047 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7f3668013960 con 0x7f36740fca30 2024-05-12T03:52:12.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:12.822+0000 7f36717fa700 1 --2- 172.21.15.33:0/3512265047 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f3664036880 0x7f3664038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:52:12.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:12.822+0000 7f36717fa700 1 -- 172.21.15.33:0/3512265047 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f3668049ae0 con 0x7f36740fca30 2024-05-12T03:52:12.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:12.822+0000 7f3673fff700 1 --2- 172.21.15.33:0/3512265047 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f3664036880 0x7f3664038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:52:12.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:12.823+0000 7f3673fff700 1 --2- 172.21.15.33:0/3512265047 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f3664036880 0x7f3664038d30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3660009990 tx=0x7f3660006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:52:12.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:12.828+0000 7f36717fa700 1 -- 172.21.15.33:0/3512265047 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f36680098d0 con 0x7f36740fca30 2024-05-12T03:52:13.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:13.107+0000 7f367ad99700 1 -- 172.21.15.33:0/3512265047 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f367405f5c0 con 0x7f36740fca30 2024-05-12T03:52:13.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:13.108+0000 7f36717fa700 1 -- 172.21.15.33:0/3512265047 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f3668023300 con 0x7f36740fca30 2024-05-12T03:52:13.109 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:52:13.109 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":1,"fsid":"a3e178f8-1012-11ef-bc98-c7b262605968","modified":"2024-05-12T03:50:15.576069Z","created":"2024-05-12T03:50:15.576069Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi032","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:3300","nonce":0},{"type":"v1","addr":"172.21.15.32:6789","nonce":0}]},"addr":"172.21.15.32:6789/0","public_addr":"172.21.15.32:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-12T03:52:13.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:13.110+0000 7f367ad99700 1 -- 172.21.15.33:0/3512265047 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f3664036880 msgr2=0x7f3664038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:52:13.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:13.110+0000 7f367ad99700 1 --2- 172.21.15.33:0/3512265047 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f3664036880 0x7f3664038d30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3660009990 tx=0x7f3660006e40 comp rx=0 tx=0).stop 2024-05-12T03:52:13.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:13.110+0000 7f367ad99700 1 -- 172.21.15.33:0/3512265047 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36740fca30 msgr2=0x7f367410cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:52:13.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:13.110+0000 7f367ad99700 1 --2- 172.21.15.33:0/3512265047 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36740fca30 0x7f367410cfc0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3668009300 tx=0x7f3668009e30 comp rx=0 tx=0).stop 2024-05-12T03:52:13.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:13.110+0000 7f367ad99700 1 -- 172.21.15.33:0/3512265047 shutdown_connections 2024-05-12T03:52:13.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:13.110+0000 7f367ad99700 1 --2- 172.21.15.33:0/3512265047 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f3664036880 0x7f3664038d30 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:52:13.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:13.110+0000 7f367ad99700 1 --2- 172.21.15.33:0/3512265047 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36740fca30 0x7f367410cfc0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:52:13.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:13.110+0000 7f367ad99700 1 -- 172.21.15.33:0/3512265047 >> 172.21.15.33:0/3512265047 conn(0x7f36740f8170 msgr2=0x7f36740f8a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:52:13.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:13.110+0000 7f367ad99700 1 -- 172.21.15.33:0/3512265047 shutdown_connections 2024-05-12T03:52:13.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:13.110+0000 7f367ad99700 1 -- 172.21.15.33:0/3512265047 wait complete. 2024-05-12T03:52:13.112 INFO:teuthology.orchestra.run.smithi033.stderr:dumped monmap epoch 1 2024-05-12T03:52:13.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:13 smithi032 ceph-mon[40008]: from='client.? 172.21.15.33:0/3512265047' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-12T03:52:14.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:14 smithi032 ceph-mon[40008]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:52:14.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:14 smithi032 ceph-mon[40008]: from='mgr.14160 172.21.15.32:0/1781291074' entity='mgr.smithi032.ukatkq' 2024-05-12T03:52:14.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:14 smithi032 ceph-mon[40008]: from='mgr.14160 172.21.15.32:0/1781291074' entity='mgr.smithi032.ukatkq' 2024-05-12T03:52:14.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:14 smithi032 ceph-mon[40008]: from='mgr.14160 172.21.15.32:0/1781291074' entity='mgr.smithi032.ukatkq' 2024-05-12T03:52:14.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:14 smithi032 ceph-mon[40008]: from='mgr.14160 172.21.15.32:0/1781291074' entity='mgr.smithi032.ukatkq' 2024-05-12T03:52:14.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:14 smithi032 ceph-mon[40008]: from='mgr.14160 172.21.15.32:0/1781291074' entity='mgr.smithi032.ukatkq' 2024-05-12T03:52:14.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:14 smithi032 ceph-mon[40008]: from='mgr.14160 172.21.15.32:0/1781291074' entity='mgr.smithi032.ukatkq' 2024-05-12T03:52:14.855 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-12T03:52:14.856 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-12T03:52:15.154 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-12T03:52:15.215 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-12T03:52:15.789 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:15 smithi032 ceph-mon[40008]: Deploying daemon prometheus.smithi032 on smithi032 2024-05-12T03:52:15.789 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:15 smithi032 ceph-mon[40008]: from='mgr.14160 172.21.15.32:0/1781291074' entity='mgr.smithi032.ukatkq' 2024-05-12T03:52:16.789 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:16 smithi032 ceph-mon[40008]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:52:16.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:16.850+0000 7f182b31d700 1 -- 172.21.15.33:0/2215188622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18240fb940 msgr2=0x7f18240fbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:52:16.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:16.850+0000 7f182b31d700 1 --2- 172.21.15.33:0/2215188622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18240fb940 0x7f18240fbd50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f18200048c0 tx=0x7f1820009e30 comp rx=0 tx=0).stop 2024-05-12T03:52:16.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:16.850+0000 7f182b31d700 1 -- 172.21.15.33:0/2215188622 shutdown_connections 2024-05-12T03:52:16.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:16.850+0000 7f182b31d700 1 --2- 172.21.15.33:0/2215188622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18240fb940 0x7f18240fbd50 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:52:16.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:16.850+0000 7f182b31d700 1 -- 172.21.15.33:0/2215188622 >> 172.21.15.33:0/2215188622 conn(0x7f18240f70a0 msgr2=0x7f18240f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:52:16.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:16.850+0000 7f182b31d700 1 -- 172.21.15.33:0/2215188622 shutdown_connections 2024-05-12T03:52:16.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:16.850+0000 7f182b31d700 1 -- 172.21.15.33:0/2215188622 wait complete. 2024-05-12T03:52:16.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:16.851+0000 7f182b31d700 1 Processor -- start 2024-05-12T03:52:16.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:16.851+0000 7f182b31d700 1 -- start start 2024-05-12T03:52:16.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:16.851+0000 7f182b31d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18240fb940 0x7f1824065bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:52:16.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:16.851+0000 7f182b31d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f182410fc90 con 0x7f18240fb940 2024-05-12T03:52:16.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:16.851+0000 7f18290b9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18240fb940 0x7f1824065bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:52:16.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:16.851+0000 7f18290b9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18240fb940 0x7f1824065bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.33:49012/0 (socket says 172.21.15.33:49012) 2024-05-12T03:52:16.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:16.851+0000 7f18290b9700 1 -- 172.21.15.33:0/2490746919 learned_addr learned my addr 172.21.15.33:0/2490746919 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-12T03:52:16.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:16.851+0000 7f18290b9700 1 -- 172.21.15.33:0/2490746919 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1820005040 con 0x7f18240fb940 2024-05-12T03:52:16.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:16.852+0000 7f18290b9700 1 --2- 172.21.15.33:0/2490746919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18240fb940 0x7f1824065bd0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1820009300 tx=0x7f1820009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:52:16.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:16.852+0000 7f1819ffb700 1 -- 172.21.15.33:0/2490746919 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1820009600 con 0x7f18240fb940 2024-05-12T03:52:16.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:16.852+0000 7f1819ffb700 1 -- 172.21.15.33:0/2490746919 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1820009760 con 0x7f18240fb940 2024-05-12T03:52:16.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:16.852+0000 7f182b31d700 1 -- 172.21.15.33:0/2490746919 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f182410fdd0 con 0x7f18240fb940 2024-05-12T03:52:16.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:16.852+0000 7f1819ffb700 1 -- 172.21.15.33:0/2490746919 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f182001c5b0 con 0x7f18240fb940 2024-05-12T03:52:16.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:16.852+0000 7f182b31d700 1 -- 172.21.15.33:0/2490746919 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18241102c0 con 0x7f18240fb940 2024-05-12T03:52:16.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:16.853+0000 7f1819ffb700 1 -- 172.21.15.33:0/2490746919 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7f182001c820 con 0x7f18240fb940 2024-05-12T03:52:16.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:16.853+0000 7f1819ffb700 1 --2- 172.21.15.33:0/2490746919 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f18100368d0 0x7f1810038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:52:16.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:16.853+0000 7f1819ffb700 1 -- 172.21.15.33:0/2490746919 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f1820049b00 con 0x7f18240fb940 2024-05-12T03:52:16.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:16.853+0000 7f182b31d700 1 -- 172.21.15.33:0/2490746919 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1824106040 con 0x7f18240fb940 2024-05-12T03:52:16.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:16.853+0000 7f18288b8700 1 --2- 172.21.15.33:0/2490746919 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f18100368d0 0x7f1810038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:52:16.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:16.854+0000 7f18288b8700 1 --2- 172.21.15.33:0/2490746919 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f18100368d0 0x7f1810038d80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1814009990 tx=0x7f1814006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:52:16.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:16.859+0000 7f1819ffb700 1 -- 172.21.15.33:0/2490746919 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f1820023300 con 0x7f18240fb940 2024-05-12T03:52:17.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:17.134+0000 7f182b31d700 1 -- 172.21.15.33:0/2490746919 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f18241015e0 con 0x7f18240fb940 2024-05-12T03:52:17.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:17.135+0000 7f1819ffb700 1 -- 172.21.15.33:0/2490746919 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f1820011b60 con 0x7f18240fb940 2024-05-12T03:52:17.136 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:52:17.136 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":1,"fsid":"a3e178f8-1012-11ef-bc98-c7b262605968","modified":"2024-05-12T03:50:15.576069Z","created":"2024-05-12T03:50:15.576069Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi032","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:3300","nonce":0},{"type":"v1","addr":"172.21.15.32:6789","nonce":0}]},"addr":"172.21.15.32:6789/0","public_addr":"172.21.15.32:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-12T03:52:17.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:17.138+0000 7f182b31d700 1 -- 172.21.15.33:0/2490746919 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f18100368d0 msgr2=0x7f1810038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:52:17.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:17.138+0000 7f182b31d700 1 --2- 172.21.15.33:0/2490746919 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f18100368d0 0x7f1810038d80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1814009990 tx=0x7f1814006e40 comp rx=0 tx=0).stop 2024-05-12T03:52:17.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:17.138+0000 7f182b31d700 1 -- 172.21.15.33:0/2490746919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18240fb940 msgr2=0x7f1824065bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:52:17.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:17.138+0000 7f182b31d700 1 --2- 172.21.15.33:0/2490746919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18240fb940 0x7f1824065bd0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1820009300 tx=0x7f1820009e30 comp rx=0 tx=0).stop 2024-05-12T03:52:17.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:17.138+0000 7f182b31d700 1 -- 172.21.15.33:0/2490746919 shutdown_connections 2024-05-12T03:52:17.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:17.138+0000 7f182b31d700 1 --2- 172.21.15.33:0/2490746919 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f18100368d0 0x7f1810038d80 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:52:17.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:17.138+0000 7f182b31d700 1 --2- 172.21.15.33:0/2490746919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18240fb940 0x7f1824065bd0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:52:17.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:17.138+0000 7f182b31d700 1 -- 172.21.15.33:0/2490746919 >> 172.21.15.33:0/2490746919 conn(0x7f18240f70a0 msgr2=0x7f18240f7940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:52:17.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:17.138+0000 7f182b31d700 1 -- 172.21.15.33:0/2490746919 shutdown_connections 2024-05-12T03:52:17.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:17.138+0000 7f182b31d700 1 -- 172.21.15.33:0/2490746919 wait complete. 2024-05-12T03:52:17.139 INFO:teuthology.orchestra.run.smithi033.stderr:dumped monmap epoch 1 2024-05-12T03:52:17.789 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:17 smithi032 ceph-mon[40008]: from='client.? 172.21.15.33:0/2490746919' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-12T03:52:18.789 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:18 smithi032 ceph-mon[40008]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:52:18.806 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-12T03:52:18.806 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-12T03:52:19.108 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-12T03:52:19.173 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-12T03:52:20.789 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:20 smithi032 ceph-mon[40008]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:52:20.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:20.958+0000 7f7bac7d2700 1 -- 172.21.15.33:0/2229872997 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ba40fb940 msgr2=0x7f7ba40fbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:52:20.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:20.958+0000 7f7bac7d2700 1 --2- 172.21.15.33:0/2229872997 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ba40fb940 0x7f7ba40fbd50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7b940048c0 tx=0x7f7b94009e30 comp rx=0 tx=0).stop 2024-05-12T03:52:20.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:20.958+0000 7f7bac7d2700 1 -- 172.21.15.33:0/2229872997 shutdown_connections 2024-05-12T03:52:20.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:20.958+0000 7f7bac7d2700 1 --2- 172.21.15.33:0/2229872997 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ba40fb940 0x7f7ba40fbd50 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:52:20.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:20.958+0000 7f7bac7d2700 1 -- 172.21.15.33:0/2229872997 >> 172.21.15.33:0/2229872997 conn(0x7f7ba40f70a0 msgr2=0x7f7ba40f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:52:20.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:20.958+0000 7f7bac7d2700 1 -- 172.21.15.33:0/2229872997 shutdown_connections 2024-05-12T03:52:20.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:20.958+0000 7f7bac7d2700 1 -- 172.21.15.33:0/2229872997 wait complete. 2024-05-12T03:52:20.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:20.959+0000 7f7bac7d2700 1 Processor -- start 2024-05-12T03:52:20.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:20.959+0000 7f7bac7d2700 1 -- start start 2024-05-12T03:52:20.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:20.959+0000 7f7bac7d2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ba40fb940 0x7f7ba4065bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:52:20.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:20.959+0000 7f7bac7d2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba410fc90 con 0x7f7ba40fb940 2024-05-12T03:52:20.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:20.960+0000 7f7baa56e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ba40fb940 0x7f7ba4065bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:52:20.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:20.960+0000 7f7baa56e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ba40fb940 0x7f7ba4065bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.33:49032/0 (socket says 172.21.15.33:49032) 2024-05-12T03:52:20.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:20.960+0000 7f7baa56e700 1 -- 172.21.15.33:0/3953471695 learned_addr learned my addr 172.21.15.33:0/3953471695 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-12T03:52:20.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:20.960+0000 7f7baa56e700 1 -- 172.21.15.33:0/3953471695 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b94005040 con 0x7f7ba40fb940 2024-05-12T03:52:20.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:20.960+0000 7f7baa56e700 1 --2- 172.21.15.33:0/3953471695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ba40fb940 0x7f7ba4065bd0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7b94009300 tx=0x7f7b94009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:52:20.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:20.960+0000 7f7b9affd700 1 -- 172.21.15.33:0/3953471695 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7b94009600 con 0x7f7ba40fb940 2024-05-12T03:52:20.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:20.960+0000 7f7b9affd700 1 -- 172.21.15.33:0/3953471695 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7b94009760 con 0x7f7ba40fb940 2024-05-12T03:52:20.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:20.960+0000 7f7bac7d2700 1 -- 172.21.15.33:0/3953471695 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ba410fdd0 con 0x7f7ba40fb940 2024-05-12T03:52:20.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:20.960+0000 7f7b9affd700 1 -- 172.21.15.33:0/3953471695 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7b9401c5b0 con 0x7f7ba40fb940 2024-05-12T03:52:20.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:20.960+0000 7f7bac7d2700 1 -- 172.21.15.33:0/3953471695 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ba41102c0 con 0x7f7ba40fb940 2024-05-12T03:52:20.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:20.961+0000 7f7b9affd700 1 -- 172.21.15.33:0/3953471695 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7f7b9401c870 con 0x7f7ba40fb940 2024-05-12T03:52:20.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:20.961+0000 7f7b9affd700 1 --2- 172.21.15.33:0/3953471695 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f7b90036880 0x7f7b90038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:52:20.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:20.961+0000 7f7b9affd700 1 -- 172.21.15.33:0/3953471695 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f7b94049b30 con 0x7f7ba40fb940 2024-05-12T03:52:20.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:20.961+0000 7f7bac7d2700 1 -- 172.21.15.33:0/3953471695 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ba4106040 con 0x7f7ba40fb940 2024-05-12T03:52:20.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:20.961+0000 7f7ba9d6d700 1 --2- 172.21.15.33:0/3953471695 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f7b90036880 0x7f7b90038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:52:20.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:20.962+0000 7f7ba9d6d700 1 --2- 172.21.15.33:0/3953471695 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f7b90036880 0x7f7b90038d30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7ba0009990 tx=0x7f7ba0006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:52:20.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:20.967+0000 7f7b9affd700 1 -- 172.21.15.33:0/3953471695 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f7b94011dd0 con 0x7f7ba40fb940 2024-05-12T03:52:21.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:21.246+0000 7f7bac7d2700 1 -- 172.21.15.33:0/3953471695 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f7ba41015e0 con 0x7f7ba40fb940 2024-05-12T03:52:21.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:21.246+0000 7f7b9affd700 1 -- 172.21.15.33:0/3953471695 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f7b94011370 con 0x7f7ba40fb940 2024-05-12T03:52:21.246 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:52:21.246 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":1,"fsid":"a3e178f8-1012-11ef-bc98-c7b262605968","modified":"2024-05-12T03:50:15.576069Z","created":"2024-05-12T03:50:15.576069Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi032","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:3300","nonce":0},{"type":"v1","addr":"172.21.15.32:6789","nonce":0}]},"addr":"172.21.15.32:6789/0","public_addr":"172.21.15.32:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-12T03:52:21.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:21.248+0000 7f7bac7d2700 1 -- 172.21.15.33:0/3953471695 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f7b90036880 msgr2=0x7f7b90038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:52:21.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:21.248+0000 7f7bac7d2700 1 --2- 172.21.15.33:0/3953471695 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f7b90036880 0x7f7b90038d30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7ba0009990 tx=0x7f7ba0006e40 comp rx=0 tx=0).stop 2024-05-12T03:52:21.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:21.248+0000 7f7bac7d2700 1 -- 172.21.15.33:0/3953471695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ba40fb940 msgr2=0x7f7ba4065bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:52:21.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:21.248+0000 7f7bac7d2700 1 --2- 172.21.15.33:0/3953471695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ba40fb940 0x7f7ba4065bd0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7b94009300 tx=0x7f7b94009e30 comp rx=0 tx=0).stop 2024-05-12T03:52:21.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:21.249+0000 7f7bac7d2700 1 -- 172.21.15.33:0/3953471695 shutdown_connections 2024-05-12T03:52:21.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:21.249+0000 7f7bac7d2700 1 --2- 172.21.15.33:0/3953471695 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f7b90036880 0x7f7b90038d30 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:52:21.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:21.249+0000 7f7bac7d2700 1 --2- 172.21.15.33:0/3953471695 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ba40fb940 0x7f7ba4065bd0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:52:21.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:21.249+0000 7f7bac7d2700 1 -- 172.21.15.33:0/3953471695 >> 172.21.15.33:0/3953471695 conn(0x7f7ba40f70a0 msgr2=0x7f7ba40f7940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:52:21.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:21.249+0000 7f7bac7d2700 1 -- 172.21.15.33:0/3953471695 shutdown_connections 2024-05-12T03:52:21.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:21.249+0000 7f7bac7d2700 1 -- 172.21.15.33:0/3953471695 wait complete. 2024-05-12T03:52:21.249 INFO:teuthology.orchestra.run.smithi033.stderr:dumped monmap epoch 1 2024-05-12T03:52:21.789 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:21 smithi032 ceph-mon[40008]: from='client.? 172.21.15.33:0/3953471695' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-12T03:52:22.732 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:22 smithi032 ceph-mon[40008]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:52:22.851 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-12T03:52:22.852 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-12T03:52:23.152 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-12T03:52:23.213 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-12T03:52:24.789 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:24 smithi032 ceph-mon[40008]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:52:24.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:24.894+0000 7fb68e026700 1 -- 172.21.15.33:0/3575872526 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6880669c0 msgr2=0x7fb6880ff630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:52:24.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:24.894+0000 7fb68e026700 1 --2- 172.21.15.33:0/3575872526 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6880669c0 0x7fb6880ff630 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb6780048c0 tx=0x7fb678009e30 comp rx=0 tx=0).stop 2024-05-12T03:52:24.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:24.894+0000 7fb68e026700 1 -- 172.21.15.33:0/3575872526 shutdown_connections 2024-05-12T03:52:24.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:24.894+0000 7fb68e026700 1 --2- 172.21.15.33:0/3575872526 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6880669c0 0x7fb6880ff630 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:52:24.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:24.894+0000 7fb68e026700 1 -- 172.21.15.33:0/3575872526 >> 172.21.15.33:0/3575872526 conn(0x7fb6880f70a0 msgr2=0x7fb6880f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:52:24.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:24.894+0000 7fb68e026700 1 -- 172.21.15.33:0/3575872526 shutdown_connections 2024-05-12T03:52:24.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:24.894+0000 7fb68e026700 1 -- 172.21.15.33:0/3575872526 wait complete. 2024-05-12T03:52:24.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:24.894+0000 7fb68e026700 1 Processor -- start 2024-05-12T03:52:24.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:24.895+0000 7fb68e026700 1 -- start start 2024-05-12T03:52:24.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:24.895+0000 7fb68e026700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6880669c0 0x7fb6880fcf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:52:24.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:24.895+0000 7fb68e026700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6880fd4c0 con 0x7fb6880669c0 2024-05-12T03:52:24.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:24.895+0000 7fb6877fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6880669c0 0x7fb6880fcf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:52:24.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:24.895+0000 7fb6877fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6880669c0 0x7fb6880fcf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.33:54968/0 (socket says 172.21.15.33:54968) 2024-05-12T03:52:24.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:24.895+0000 7fb6877fe700 1 -- 172.21.15.33:0/2296376131 learned_addr learned my addr 172.21.15.33:0/2296376131 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-12T03:52:24.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:24.895+0000 7fb6877fe700 1 -- 172.21.15.33:0/2296376131 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb678005040 con 0x7fb6880669c0 2024-05-12T03:52:24.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:24.896+0000 7fb6877fe700 1 --2- 172.21.15.33:0/2296376131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6880669c0 0x7fb6880fcf80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb67800b3b0 tx=0x7fb678009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:52:24.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:24.896+0000 7fb66ffff700 1 -- 172.21.15.33:0/2296376131 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb6780095d0 con 0x7fb6880669c0 2024-05-12T03:52:24.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:24.896+0000 7fb68e026700 1 -- 172.21.15.33:0/2296376131 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6880fb590 con 0x7fb6880669c0 2024-05-12T03:52:24.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:24.896+0000 7fb66ffff700 1 -- 172.21.15.33:0/2296376131 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb678009730 con 0x7fb6880669c0 2024-05-12T03:52:24.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:24.896+0000 7fb66ffff700 1 -- 172.21.15.33:0/2296376131 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb67801c5b0 con 0x7fb6880669c0 2024-05-12T03:52:24.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:24.896+0000 7fb68e026700 1 -- 172.21.15.33:0/2296376131 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6880fba80 con 0x7fb6880669c0 2024-05-12T03:52:24.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:24.897+0000 7fb68e026700 1 -- 172.21.15.33:0/2296376131 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb68805f5c0 con 0x7fb6880669c0 2024-05-12T03:52:24.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:24.897+0000 7fb66ffff700 1 -- 172.21.15.33:0/2296376131 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7fb678013960 con 0x7fb6880669c0 2024-05-12T03:52:24.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:24.897+0000 7fb66ffff700 1 --2- 172.21.15.33:0/2296376131 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7fb670036880 0x7fb670038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:52:24.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:24.897+0000 7fb66ffff700 1 -- 172.21.15.33:0/2296376131 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fb6780499d0 con 0x7fb6880669c0 2024-05-12T03:52:24.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:24.897+0000 7fb686ffd700 1 --2- 172.21.15.33:0/2296376131 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7fb670036880 0x7fb670038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:52:24.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:24.898+0000 7fb686ffd700 1 --2- 172.21.15.33:0/2296376131 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7fb670036880 0x7fb670038d30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb67c009990 tx=0x7fb67c006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:52:24.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:24.902+0000 7fb66ffff700 1 -- 172.21.15.33:0/2296376131 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb678011960 con 0x7fb6880669c0 2024-05-12T03:52:25.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:25.176+0000 7fb68e026700 1 -- 172.21.15.33:0/2296376131 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fb688106b50 con 0x7fb6880669c0 2024-05-12T03:52:25.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:25.177+0000 7fb66ffff700 1 -- 172.21.15.33:0/2296376131 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fb678011370 con 0x7fb6880669c0 2024-05-12T03:52:25.176 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:52:25.177 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":1,"fsid":"a3e178f8-1012-11ef-bc98-c7b262605968","modified":"2024-05-12T03:50:15.576069Z","created":"2024-05-12T03:50:15.576069Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi032","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:3300","nonce":0},{"type":"v1","addr":"172.21.15.32:6789","nonce":0}]},"addr":"172.21.15.32:6789/0","public_addr":"172.21.15.32:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-12T03:52:25.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:25.178+0000 7fb68e026700 1 -- 172.21.15.33:0/2296376131 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7fb670036880 msgr2=0x7fb670038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:52:25.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:25.178+0000 7fb68e026700 1 --2- 172.21.15.33:0/2296376131 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7fb670036880 0x7fb670038d30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb67c009990 tx=0x7fb67c006e40 comp rx=0 tx=0).stop 2024-05-12T03:52:25.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:25.178+0000 7fb68e026700 1 -- 172.21.15.33:0/2296376131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6880669c0 msgr2=0x7fb6880fcf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:52:25.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:25.178+0000 7fb68e026700 1 --2- 172.21.15.33:0/2296376131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6880669c0 0x7fb6880fcf80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb67800b3b0 tx=0x7fb678009e30 comp rx=0 tx=0).stop 2024-05-12T03:52:25.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:25.179+0000 7fb68e026700 1 -- 172.21.15.33:0/2296376131 shutdown_connections 2024-05-12T03:52:25.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:25.179+0000 7fb68e026700 1 --2- 172.21.15.33:0/2296376131 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7fb670036880 0x7fb670038d30 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:52:25.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:25.179+0000 7fb68e026700 1 --2- 172.21.15.33:0/2296376131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6880669c0 0x7fb6880fcf80 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:52:25.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:25.179+0000 7fb68e026700 1 -- 172.21.15.33:0/2296376131 >> 172.21.15.33:0/2296376131 conn(0x7fb6880f70a0 msgr2=0x7fb6880f7900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:52:25.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:25.179+0000 7fb68e026700 1 -- 172.21.15.33:0/2296376131 shutdown_connections 2024-05-12T03:52:25.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:25.179+0000 7fb68e026700 1 -- 172.21.15.33:0/2296376131 wait complete. 2024-05-12T03:52:25.179 INFO:teuthology.orchestra.run.smithi033.stderr:dumped monmap epoch 1 2024-05-12T03:52:25.789 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:25 smithi032 ceph-mon[40008]: from='client.? 172.21.15.33:0/2296376131' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-12T03:52:26.751 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:26 smithi032 ceph-mon[40008]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:52:26.842 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-12T03:52:26.842 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-12T03:52:27.137 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-12T03:52:27.202 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-12T03:52:28.789 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:28 smithi032 ceph-mon[40008]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:52:29.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:28.941+0000 7fa281e4c700 1 -- 172.21.15.33:0/788799308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa27c065d30 msgr2=0x7fa27c0ff630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:52:29.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:28.941+0000 7fa281e4c700 1 --2- 172.21.15.33:0/788799308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa27c065d30 0x7fa27c0ff630 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa26c0048c0 tx=0x7fa26c009e30 comp rx=0 tx=0).stop 2024-05-12T03:52:29.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:28.941+0000 7fa281e4c700 1 -- 172.21.15.33:0/788799308 shutdown_connections 2024-05-12T03:52:29.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:28.941+0000 7fa281e4c700 1 --2- 172.21.15.33:0/788799308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa27c065d30 0x7fa27c0ff630 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:52:29.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:28.941+0000 7fa281e4c700 1 -- 172.21.15.33:0/788799308 >> 172.21.15.33:0/788799308 conn(0x7fa27c0f70a0 msgr2=0x7fa27c0f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:52:29.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:28.941+0000 7fa281e4c700 1 -- 172.21.15.33:0/788799308 shutdown_connections 2024-05-12T03:52:29.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:28.941+0000 7fa281e4c700 1 -- 172.21.15.33:0/788799308 wait complete. 2024-05-12T03:52:29.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:28.942+0000 7fa281e4c700 1 Processor -- start 2024-05-12T03:52:29.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:28.942+0000 7fa281e4c700 1 -- start start 2024-05-12T03:52:29.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:28.942+0000 7fa281e4c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa27c065d30 0x7fa27c10cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:52:29.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:28.942+0000 7fa281e4c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27c10d4b0 con 0x7fa27c065d30 2024-05-12T03:52:29.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:28.943+0000 7fa27b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa27c065d30 0x7fa27c10cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:52:29.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:28.943+0000 7fa27b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa27c065d30 0x7fa27c10cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.33:54982/0 (socket says 172.21.15.33:54982) 2024-05-12T03:52:29.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:28.943+0000 7fa27b7fe700 1 -- 172.21.15.33:0/1530095046 learned_addr learned my addr 172.21.15.33:0/1530095046 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-12T03:52:29.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:28.943+0000 7fa27b7fe700 1 -- 172.21.15.33:0/1530095046 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa26c005040 con 0x7fa27c065d30 2024-05-12T03:52:29.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:28.943+0000 7fa27b7fe700 1 --2- 172.21.15.33:0/1530095046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa27c065d30 0x7fa27c10cf70 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa26c00b3b0 tx=0x7fa26c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:52:29.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:28.943+0000 7fa263fff700 1 -- 172.21.15.33:0/1530095046 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa26c0095d0 con 0x7fa27c065d30 2024-05-12T03:52:29.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:28.943+0000 7fa281e4c700 1 -- 172.21.15.33:0/1530095046 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa27c10d6b0 con 0x7fa27c065d30 2024-05-12T03:52:29.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:28.943+0000 7fa263fff700 1 -- 172.21.15.33:0/1530095046 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa26c009730 con 0x7fa27c065d30 2024-05-12T03:52:29.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:28.943+0000 7fa263fff700 1 -- 172.21.15.33:0/1530095046 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa26c01c5b0 con 0x7fa27c065d30 2024-05-12T03:52:29.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:28.943+0000 7fa281e4c700 1 -- 172.21.15.33:0/1530095046 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa27c10e220 con 0x7fa27c065d30 2024-05-12T03:52:29.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:28.944+0000 7fa263fff700 1 -- 172.21.15.33:0/1530095046 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7fa26c013960 con 0x7fa27c065d30 2024-05-12T03:52:29.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:28.944+0000 7fa281e4c700 1 -- 172.21.15.33:0/1530095046 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa27c102e00 con 0x7fa27c065d30 2024-05-12T03:52:29.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:28.944+0000 7fa263fff700 1 --2- 172.21.15.33:0/1530095046 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7fa264036880 0x7fa264038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:52:29.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:28.944+0000 7fa263fff700 1 -- 172.21.15.33:0/1530095046 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fa26c049ab0 con 0x7fa27c065d30 2024-05-12T03:52:29.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:28.945+0000 7fa27affd700 1 --2- 172.21.15.33:0/1530095046 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7fa264036880 0x7fa264038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:52:29.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:28.945+0000 7fa27affd700 1 --2- 172.21.15.33:0/1530095046 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7fa264036880 0x7fa264038d30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa270009990 tx=0x7fa270006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:52:29.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:28.950+0000 7fa263fff700 1 -- 172.21.15.33:0/1530095046 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fa26c011940 con 0x7fa27c065d30 2024-05-12T03:52:29.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:29.229+0000 7fa281e4c700 1 -- 172.21.15.33:0/1530095046 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa27c10ec30 con 0x7fa27c065d30 2024-05-12T03:52:29.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:29.229+0000 7fa263fff700 1 -- 172.21.15.33:0/1530095046 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fa26c011370 con 0x7fa27c065d30 2024-05-12T03:52:29.230 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:52:29.230 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":1,"fsid":"a3e178f8-1012-11ef-bc98-c7b262605968","modified":"2024-05-12T03:50:15.576069Z","created":"2024-05-12T03:50:15.576069Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi032","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:3300","nonce":0},{"type":"v1","addr":"172.21.15.32:6789","nonce":0}]},"addr":"172.21.15.32:6789/0","public_addr":"172.21.15.32:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-12T03:52:29.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:29.231+0000 7fa281e4c700 1 -- 172.21.15.33:0/1530095046 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7fa264036880 msgr2=0x7fa264038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:52:29.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:29.231+0000 7fa281e4c700 1 --2- 172.21.15.33:0/1530095046 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7fa264036880 0x7fa264038d30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa270009990 tx=0x7fa270006e40 comp rx=0 tx=0).stop 2024-05-12T03:52:29.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:29.231+0000 7fa281e4c700 1 -- 172.21.15.33:0/1530095046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa27c065d30 msgr2=0x7fa27c10cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:52:29.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:29.231+0000 7fa281e4c700 1 --2- 172.21.15.33:0/1530095046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa27c065d30 0x7fa27c10cf70 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa26c00b3b0 tx=0x7fa26c009e30 comp rx=0 tx=0).stop 2024-05-12T03:52:29.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:29.231+0000 7fa281e4c700 1 -- 172.21.15.33:0/1530095046 shutdown_connections 2024-05-12T03:52:29.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:29.231+0000 7fa281e4c700 1 --2- 172.21.15.33:0/1530095046 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7fa264036880 0x7fa264038d30 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:52:29.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:29.231+0000 7fa281e4c700 1 --2- 172.21.15.33:0/1530095046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa27c065d30 0x7fa27c10cf70 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:52:29.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:29.231+0000 7fa281e4c700 1 -- 172.21.15.33:0/1530095046 >> 172.21.15.33:0/1530095046 conn(0x7fa27c0f70a0 msgr2=0x7fa27c0f7920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:52:29.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:29.232+0000 7fa281e4c700 1 -- 172.21.15.33:0/1530095046 shutdown_connections 2024-05-12T03:52:29.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:29.232+0000 7fa281e4c700 1 -- 172.21.15.33:0/1530095046 wait complete. 2024-05-12T03:52:29.232 INFO:teuthology.orchestra.run.smithi033.stderr:dumped monmap epoch 1 2024-05-12T03:52:29.789 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:29 smithi032 ceph-mon[40008]: from='client.? 172.21.15.33:0/1530095046' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-12T03:52:30.736 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:30 smithi032 ceph-mon[40008]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:52:30.959 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-12T03:52:30.959 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-12T03:52:31.252 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-12T03:52:31.313 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-12T03:52:32.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:32 smithi032 ceph-mon[40008]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:52:32.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:32 smithi032 ceph-mon[40008]: from='mgr.14160 172.21.15.32:0/1781291074' entity='mgr.smithi032.ukatkq' 2024-05-12T03:52:32.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:32 smithi032 ceph-mon[40008]: from='mgr.14160 172.21.15.32:0/1781291074' entity='mgr.smithi032.ukatkq' 2024-05-12T03:52:32.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:32 smithi032 ceph-mon[40008]: from='mgr.14160 172.21.15.32:0/1781291074' entity='mgr.smithi032.ukatkq' 2024-05-12T03:52:32.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:32 smithi032 ceph-mon[40008]: from='mgr.14160 172.21.15.32:0/1781291074' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-05-12T03:52:33.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:33.099+0000 7fc8fce3f700 1 -- 172.21.15.33:0/1010171421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8f80fb940 msgr2=0x7fc8f80fbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:52:33.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:33.099+0000 7fc8fce3f700 1 --2- 172.21.15.33:0/1010171421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8f80fb940 0x7fc8f80fbd50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc8e80048c0 tx=0x7fc8e8009e30 comp rx=0 tx=0).stop 2024-05-12T03:52:33.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:33.099+0000 7fc8fce3f700 1 -- 172.21.15.33:0/1010171421 shutdown_connections 2024-05-12T03:52:33.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:33.099+0000 7fc8fce3f700 1 --2- 172.21.15.33:0/1010171421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8f80fb940 0x7fc8f80fbd50 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:52:33.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:33.099+0000 7fc8fce3f700 1 -- 172.21.15.33:0/1010171421 >> 172.21.15.33:0/1010171421 conn(0x7fc8f80f70a0 msgr2=0x7fc8f80f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:52:33.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:33.099+0000 7fc8fce3f700 1 -- 172.21.15.33:0/1010171421 shutdown_connections 2024-05-12T03:52:33.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:33.099+0000 7fc8fce3f700 1 -- 172.21.15.33:0/1010171421 wait complete. 2024-05-12T03:52:33.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:33.100+0000 7fc8fce3f700 1 Processor -- start 2024-05-12T03:52:33.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:33.100+0000 7fc8fce3f700 1 -- start start 2024-05-12T03:52:33.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:33.100+0000 7fc8fce3f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8f80fb940 0x7fc8f810bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:52:33.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:33.100+0000 7fc8fce3f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f810c440 con 0x7fc8f80fb940 2024-05-12T03:52:33.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:33.100+0000 7fc8f659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8f80fb940 0x7fc8f810bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:52:33.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:33.100+0000 7fc8f659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8f80fb940 0x7fc8f810bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.33:41326/0 (socket says 172.21.15.33:41326) 2024-05-12T03:52:33.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:33.100+0000 7fc8f659c700 1 -- 172.21.15.33:0/1602297915 learned_addr learned my addr 172.21.15.33:0/1602297915 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-12T03:52:33.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:33.100+0000 7fc8f659c700 1 -- 172.21.15.33:0/1602297915 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8e8005040 con 0x7fc8f80fb940 2024-05-12T03:52:33.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:33.101+0000 7fc8f659c700 1 --2- 172.21.15.33:0/1602297915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8f80fb940 0x7fc8f810bf00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc8e8009300 tx=0x7fc8e8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:52:33.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:33.101+0000 7fc8e6ffd700 1 -- 172.21.15.33:0/1602297915 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc8e8009600 con 0x7fc8f80fb940 2024-05-12T03:52:33.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:33.101+0000 7fc8fce3f700 1 -- 172.21.15.33:0/1602297915 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8f810c640 con 0x7fc8f80fb940 2024-05-12T03:52:33.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:33.101+0000 7fc8e6ffd700 1 -- 172.21.15.33:0/1602297915 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc8e8009760 con 0x7fc8f80fb940 2024-05-12T03:52:33.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:33.101+0000 7fc8e6ffd700 1 -- 172.21.15.33:0/1602297915 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc8e801c5b0 con 0x7fc8f80fb940 2024-05-12T03:52:33.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:33.101+0000 7fc8fce3f700 1 -- 172.21.15.33:0/1602297915 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8f810d0d0 con 0x7fc8f80fb940 2024-05-12T03:52:33.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:33.102+0000 7fc8e6ffd700 1 -- 172.21.15.33:0/1602297915 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 14) v1 ==== 43446+0+0 (secure 0 0 0) 0x7fc8e8013420 con 0x7fc8f80fb940 2024-05-12T03:52:33.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:33.102+0000 7fc8fce3f700 1 -- 172.21.15.33:0/1602297915 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8f8101d50 con 0x7fc8f80fb940 2024-05-12T03:52:33.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:33.102+0000 7fc8e6ffd700 1 --2- 172.21.15.33:0/1602297915 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7fc8e00368d0 0x7fc8e0038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:52:33.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:33.102+0000 7fc8e6ffd700 1 -- 172.21.15.33:0/1602297915 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fc8e8049c10 con 0x7fc8f80fb940 2024-05-12T03:52:33.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:33.102+0000 7fc8f5d9b700 1 -- 172.21.15.33:0/1602297915 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7fc8e00368d0 msgr2=0x7fc8e0038d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/1988399583 2024-05-12T03:52:33.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:33.102+0000 7fc8f5d9b700 1 --2- 172.21.15.33:0/1602297915 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7fc8e00368d0 0x7fc8e0038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-12T03:52:33.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:33.107+0000 7fc8e6ffd700 1 -- 172.21.15.33:0/1602297915 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fc8e8025400 con 0x7fc8f80fb940 2024-05-12T03:52:33.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:33.303+0000 7fc8f5d9b700 1 -- 172.21.15.33:0/1602297915 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7fc8e00368d0 msgr2=0x7fc8e0038d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/1988399583 2024-05-12T03:52:33.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:33.303+0000 7fc8f5d9b700 1 --2- 172.21.15.33:0/1602297915 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7fc8e00368d0 0x7fc8e0038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-12T03:52:33.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:33.384+0000 7fc8fce3f700 1 -- 172.21.15.33:0/1602297915 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc8f810db80 con 0x7fc8f80fb940 2024-05-12T03:52:33.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:33.385+0000 7fc8e6ffd700 1 -- 172.21.15.33:0/1602297915 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fc8e8011370 con 0x7fc8f80fb940 2024-05-12T03:52:33.385 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:52:33.385 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":1,"fsid":"a3e178f8-1012-11ef-bc98-c7b262605968","modified":"2024-05-12T03:50:15.576069Z","created":"2024-05-12T03:50:15.576069Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi032","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:3300","nonce":0},{"type":"v1","addr":"172.21.15.32:6789","nonce":0}]},"addr":"172.21.15.32:6789/0","public_addr":"172.21.15.32:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-12T03:52:33.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:33.387+0000 7fc8fce3f700 1 -- 172.21.15.33:0/1602297915 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7fc8e00368d0 msgr2=0x7fc8e0038d80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-12T03:52:33.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:33.387+0000 7fc8fce3f700 1 --2- 172.21.15.33:0/1602297915 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7fc8e00368d0 0x7fc8e0038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:52:33.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:33.387+0000 7fc8fce3f700 1 -- 172.21.15.33:0/1602297915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8f80fb940 msgr2=0x7fc8f810bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:52:33.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:33.387+0000 7fc8fce3f700 1 --2- 172.21.15.33:0/1602297915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8f80fb940 0x7fc8f810bf00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc8e8009300 tx=0x7fc8e8009e30 comp rx=0 tx=0).stop 2024-05-12T03:52:33.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:33.387+0000 7fc8fce3f700 1 -- 172.21.15.33:0/1602297915 shutdown_connections 2024-05-12T03:52:33.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:33.387+0000 7fc8fce3f700 1 --2- 172.21.15.33:0/1602297915 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7fc8e00368d0 0x7fc8e0038d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:52:33.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:33.387+0000 7fc8fce3f700 1 --2- 172.21.15.33:0/1602297915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8f80fb940 0x7fc8f810bf00 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:52:33.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:33.387+0000 7fc8fce3f700 1 -- 172.21.15.33:0/1602297915 >> 172.21.15.33:0/1602297915 conn(0x7fc8f80f70a0 msgr2=0x7fc8f80f7940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:52:33.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:33.387+0000 7fc8fce3f700 1 -- 172.21.15.33:0/1602297915 shutdown_connections 2024-05-12T03:52:33.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:33.387+0000 7fc8fce3f700 1 -- 172.21.15.33:0/1602297915 wait complete. 2024-05-12T03:52:33.388 INFO:teuthology.orchestra.run.smithi033.stderr:dumped monmap epoch 1 2024-05-12T03:52:34.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:34 smithi032 ceph-mon[40008]: from='mgr.14160 172.21.15.32:0/1781291074' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-05-12T03:52:34.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:34 smithi032 ceph-mon[40008]: mgrmap e14: smithi032.ukatkq(active, since 67s) 2024-05-12T03:52:34.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:34 smithi032 ceph-mon[40008]: from='client.? 172.21.15.33:0/1602297915' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-12T03:52:35.143 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-12T03:52:35.143 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-12T03:52:35.445 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-12T03:52:35.512 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-12T03:52:37.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:37.283+0000 7f1160920700 1 -- 172.21.15.33:0/461045865 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f115c0fe090 msgr2=0x7f115c0fe4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:52:37.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:37.283+0000 7f1160920700 1 --2- 172.21.15.33:0/461045865 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f115c0fe090 0x7f115c0fe4a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f114c0048c0 tx=0x7f114c009e30 comp rx=0 tx=0).stop 2024-05-12T03:52:37.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:37.283+0000 7f1160920700 1 -- 172.21.15.33:0/461045865 shutdown_connections 2024-05-12T03:52:37.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:37.283+0000 7f1160920700 1 --2- 172.21.15.33:0/461045865 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f115c0fe090 0x7f115c0fe4a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:52:37.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:37.283+0000 7f1160920700 1 -- 172.21.15.33:0/461045865 >> 172.21.15.33:0/461045865 conn(0x7f115c0f6850 msgr2=0x7f115c0f8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:52:37.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:37.283+0000 7f1160920700 1 -- 172.21.15.33:0/461045865 shutdown_connections 2024-05-12T03:52:37.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:37.283+0000 7f1160920700 1 -- 172.21.15.33:0/461045865 wait complete. 2024-05-12T03:52:37.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:37.284+0000 7f1160920700 1 Processor -- start 2024-05-12T03:52:37.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:37.284+0000 7f1160920700 1 -- start start 2024-05-12T03:52:37.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:37.284+0000 7f1160920700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f115c0fe090 0x7f115c10cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:52:37.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:37.284+0000 7f1160920700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f115c10d520 con 0x7f115c0fe090 2024-05-12T03:52:37.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:37.284+0000 7f115a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f115c0fe090 0x7f115c10cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:52:37.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:37.284+0000 7f115a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f115c0fe090 0x7f115c10cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.33:41342/0 (socket says 172.21.15.33:41342) 2024-05-12T03:52:37.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:37.284+0000 7f115a59c700 1 -- 172.21.15.33:0/1271280659 learned_addr learned my addr 172.21.15.33:0/1271280659 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-12T03:52:37.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:37.284+0000 7f115a59c700 1 -- 172.21.15.33:0/1271280659 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f114c005040 con 0x7f115c0fe090 2024-05-12T03:52:37.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:37.284+0000 7f115a59c700 1 --2- 172.21.15.33:0/1271280659 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f115c0fe090 0x7f115c10cfe0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f114c009300 tx=0x7f114c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:52:37.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:37.285+0000 7f114affd700 1 -- 172.21.15.33:0/1271280659 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f114c009600 con 0x7f115c0fe090 2024-05-12T03:52:37.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:37.285+0000 7f114affd700 1 -- 172.21.15.33:0/1271280659 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f114c009760 con 0x7f115c0fe090 2024-05-12T03:52:37.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:37.285+0000 7f1160920700 1 -- 172.21.15.33:0/1271280659 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f115c10d720 con 0x7f115c0fe090 2024-05-12T03:52:37.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:37.285+0000 7f114affd700 1 -- 172.21.15.33:0/1271280659 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f114c01c5b0 con 0x7f115c0fe090 2024-05-12T03:52:37.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:37.285+0000 7f1160920700 1 -- 172.21.15.33:0/1271280659 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f115c10e290 con 0x7f115c0fe090 2024-05-12T03:52:37.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:37.285+0000 7f114affd700 1 -- 172.21.15.33:0/1271280659 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 14) v1 ==== 43446+0+0 (secure 0 0 0) 0x7f114c013420 con 0x7f115c0fe090 2024-05-12T03:52:37.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:37.285+0000 7f1160920700 1 -- 172.21.15.33:0/1271280659 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f115c102e20 con 0x7f115c0fe090 2024-05-12T03:52:37.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:37.286+0000 7f114affd700 1 --2- 172.21.15.33:0/1271280659 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f11440368d0 0x7f1144038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:52:37.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:37.286+0000 7f114affd700 1 -- 172.21.15.33:0/1271280659 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f114c049bc0 con 0x7f115c0fe090 2024-05-12T03:52:37.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:37.286+0000 7f1159d9b700 1 -- 172.21.15.33:0/1271280659 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f11440368d0 msgr2=0x7f1144038d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/1988399583 2024-05-12T03:52:37.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:37.286+0000 7f1159d9b700 1 --2- 172.21.15.33:0/1271280659 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f11440368d0 0x7f1144038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-12T03:52:37.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:37.294+0000 7f114affd700 1 -- 172.21.15.33:0/1271280659 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f114c025400 con 0x7f115c0fe090 2024-05-12T03:52:37.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:37.487+0000 7f1159d9b700 1 -- 172.21.15.33:0/1271280659 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f11440368d0 msgr2=0x7f1144038d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/1988399583 2024-05-12T03:52:37.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:37.487+0000 7f1159d9b700 1 --2- 172.21.15.33:0/1271280659 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f11440368d0 0x7f1144038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-12T03:52:37.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:37.569+0000 7f1160920700 1 -- 172.21.15.33:0/1271280659 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f115c10ed60 con 0x7f115c0fe090 2024-05-12T03:52:37.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:37.570+0000 7f114affd700 1 -- 172.21.15.33:0/1271280659 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f114c049e60 con 0x7f115c0fe090 2024-05-12T03:52:37.570 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:52:37.570 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":1,"fsid":"a3e178f8-1012-11ef-bc98-c7b262605968","modified":"2024-05-12T03:50:15.576069Z","created":"2024-05-12T03:50:15.576069Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi032","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:3300","nonce":0},{"type":"v1","addr":"172.21.15.32:6789","nonce":0}]},"addr":"172.21.15.32:6789/0","public_addr":"172.21.15.32:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-12T03:52:37.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:37.572+0000 7f1160920700 1 -- 172.21.15.33:0/1271280659 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f11440368d0 msgr2=0x7f1144038d80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-12T03:52:37.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:37.572+0000 7f1160920700 1 --2- 172.21.15.33:0/1271280659 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f11440368d0 0x7f1144038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:52:37.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:37.572+0000 7f1160920700 1 -- 172.21.15.33:0/1271280659 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f115c0fe090 msgr2=0x7f115c10cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:52:37.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:37.572+0000 7f1160920700 1 --2- 172.21.15.33:0/1271280659 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f115c0fe090 0x7f115c10cfe0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f114c009300 tx=0x7f114c009e30 comp rx=0 tx=0).stop 2024-05-12T03:52:37.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:37.572+0000 7f1160920700 1 -- 172.21.15.33:0/1271280659 shutdown_connections 2024-05-12T03:52:37.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:37.572+0000 7f1160920700 1 --2- 172.21.15.33:0/1271280659 >> [v2:172.21.15.32:6800/1988399583,v1:172.21.15.32:6801/1988399583] conn(0x7f11440368d0 0x7f1144038d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:52:37.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:37.572+0000 7f1160920700 1 --2- 172.21.15.33:0/1271280659 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f115c0fe090 0x7f115c10cfe0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:52:37.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:37.572+0000 7f1160920700 1 -- 172.21.15.33:0/1271280659 >> 172.21.15.33:0/1271280659 conn(0x7f115c0f6850 msgr2=0x7f115c0f7080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:52:37.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:37.572+0000 7f1160920700 1 -- 172.21.15.33:0/1271280659 shutdown_connections 2024-05-12T03:52:37.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:37.572+0000 7f1160920700 1 -- 172.21.15.33:0/1271280659 wait complete. 2024-05-12T03:52:37.573 INFO:teuthology.orchestra.run.smithi033.stderr:dumped monmap epoch 1 2024-05-12T03:52:37.789 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:37 smithi032 ceph-mon[40008]: from='client.? 172.21.15.33:0/1271280659' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-12T03:52:39.221 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-12T03:52:39.221 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-12T03:52:39.530 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-12T03:52:39.598 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-12T03:52:39.926 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:39 smithi032 ceph-mon[40008]: Active manager daemon smithi032.ukatkq restarted 2024-05-12T03:52:39.926 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:39 smithi032 ceph-mon[40008]: Activating manager daemon smithi032.ukatkq 2024-05-12T03:52:39.926 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:39 smithi032 ceph-mon[40008]: osdmap e5: 0 total, 0 up, 0 in 2024-05-12T03:52:39.926 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:39 smithi032 ceph-mon[40008]: mgrmap e15: smithi032.ukatkq(active, starting, since 0.00310115s) 2024-05-12T03:52:39.926 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:39 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mon metadata", "id": "smithi032"}]: dispatch 2024-05-12T03:52:39.927 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:39 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mgr metadata", "who": "smithi032.ukatkq", "id": "smithi032.ukatkq"}]: dispatch 2024-05-12T03:52:39.927 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:39 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-12T03:52:39.927 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:39 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-12T03:52:39.927 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:39 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-12T03:52:39.927 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:39 smithi032 ceph-mon[40008]: Manager daemon smithi032.ukatkq is now available 2024-05-12T03:52:39.927 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:39 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T03:52:39.928 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:39 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T03:52:41.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:40 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:52:41.215 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:40 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T03:52:41.215 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:40 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi032.ukatkq/mirror_snapshot_schedule"}]: dispatch 2024-05-12T03:52:41.215 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:40 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi032.ukatkq/trash_purge_schedule"}]: dispatch 2024-05-12T03:52:41.216 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:40 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:52:41.216 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:40 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:52:41.216 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:40 smithi032 ceph-mon[40008]: mgrmap e16: smithi032.ukatkq(active, since 1.00543s) 2024-05-12T03:52:41.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:41.400+0000 7fe0786cc700 1 -- 172.21.15.33:0/432290744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe070071de0 msgr2=0x7fe0700702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:52:41.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:41.400+0000 7fe0786cc700 1 --2- 172.21.15.33:0/432290744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe070071de0 0x7fe0700702f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe06c0048c0 tx=0x7fe06c009e30 comp rx=0 tx=0).stop 2024-05-12T03:52:41.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:41.400+0000 7fe0786cc700 1 -- 172.21.15.33:0/432290744 shutdown_connections 2024-05-12T03:52:41.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:41.400+0000 7fe0786cc700 1 --2- 172.21.15.33:0/432290744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe070071de0 0x7fe0700702f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:52:41.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:41.400+0000 7fe0786cc700 1 -- 172.21.15.33:0/432290744 >> 172.21.15.33:0/432290744 conn(0x7fe0700f81c0 msgr2=0x7fe0700fa5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:52:41.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:41.400+0000 7fe0786cc700 1 -- 172.21.15.33:0/432290744 shutdown_connections 2024-05-12T03:52:41.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:41.401+0000 7fe0786cc700 1 -- 172.21.15.33:0/432290744 wait complete. 2024-05-12T03:52:41.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:41.401+0000 7fe0786cc700 1 Processor -- start 2024-05-12T03:52:41.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:41.401+0000 7fe0786cc700 1 -- start start 2024-05-12T03:52:41.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:41.401+0000 7fe0786cc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe070071de0 0x7fe07010f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:52:41.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:41.401+0000 7fe0786cc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe07010f740 con 0x7fe070071de0 2024-05-12T03:52:41.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:41.402+0000 7fe076468700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe070071de0 0x7fe07010f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:52:41.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:41.402+0000 7fe076468700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe070071de0 0x7fe07010f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.33:41356/0 (socket says 172.21.15.33:41356) 2024-05-12T03:52:41.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:41.402+0000 7fe076468700 1 -- 172.21.15.33:0/2388128302 learned_addr learned my addr 172.21.15.33:0/2388128302 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-12T03:52:41.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:41.402+0000 7fe076468700 1 -- 172.21.15.33:0/2388128302 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe06c005040 con 0x7fe070071de0 2024-05-12T03:52:41.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:41.402+0000 7fe076468700 1 --2- 172.21.15.33:0/2388128302 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe070071de0 0x7fe07010f200 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe06c00b860 tx=0x7fe06c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:52:41.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:41.402+0000 7fe066ffd700 1 -- 172.21.15.33:0/2388128302 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe06c009580 con 0x7fe070071de0 2024-05-12T03:52:41.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:41.402+0000 7fe066ffd700 1 -- 172.21.15.33:0/2388128302 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe06c0096e0 con 0x7fe070071de0 2024-05-12T03:52:41.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:41.402+0000 7fe0786cc700 1 -- 172.21.15.33:0/2388128302 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe07010f940 con 0x7fe070071de0 2024-05-12T03:52:41.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:41.402+0000 7fe066ffd700 1 -- 172.21.15.33:0/2388128302 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe06c01c5b0 con 0x7fe070071de0 2024-05-12T03:52:41.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:41.402+0000 7fe0786cc700 1 -- 172.21.15.33:0/2388128302 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0701104b0 con 0x7fe070071de0 2024-05-12T03:52:41.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:41.403+0000 7fe066ffd700 1 -- 172.21.15.33:0/2388128302 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 43297+0+0 (secure 0 0 0) 0x7fe06c01c710 con 0x7fe070071de0 2024-05-12T03:52:41.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:41.403+0000 7fe0786cc700 1 -- 172.21.15.33:0/2388128302 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe070105020 con 0x7fe070071de0 2024-05-12T03:52:41.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:41.403+0000 7fe066ffd700 1 --2- 172.21.15.33:0/2388128302 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fe05c036750 0x7fe05c038c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:52:41.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:41.403+0000 7fe066ffd700 1 -- 172.21.15.33:0/2388128302 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fe06c049960 con 0x7fe070071de0 2024-05-12T03:52:41.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:41.403+0000 7fe075c67700 1 --2- 172.21.15.33:0/2388128302 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fe05c036750 0x7fe05c038c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:52:41.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:41.404+0000 7fe075c67700 1 --2- 172.21.15.33:0/2388128302 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fe05c036750 0x7fe05c038c00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe060009990 tx=0x7fe060006e40 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:52:41.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:41.408+0000 7fe066ffd700 1 -- 172.21.15.33:0/2388128302 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fe06c025400 con 0x7fe070071de0 2024-05-12T03:52:41.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:41.696+0000 7fe0786cc700 1 -- 172.21.15.33:0/2388128302 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fe07005f5c0 con 0x7fe070071de0 2024-05-12T03:52:41.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:41.696+0000 7fe066ffd700 1 -- 172.21.15.33:0/2388128302 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fe06c011b60 con 0x7fe070071de0 2024-05-12T03:52:41.696 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:52:41.697 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":1,"fsid":"a3e178f8-1012-11ef-bc98-c7b262605968","modified":"2024-05-12T03:50:15.576069Z","created":"2024-05-12T03:50:15.576069Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi032","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:3300","nonce":0},{"type":"v1","addr":"172.21.15.32:6789","nonce":0}]},"addr":"172.21.15.32:6789/0","public_addr":"172.21.15.32:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-12T03:52:41.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:41.700+0000 7fe0786cc700 1 -- 172.21.15.33:0/2388128302 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fe05c036750 msgr2=0x7fe05c038c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:52:41.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:41.700+0000 7fe0786cc700 1 --2- 172.21.15.33:0/2388128302 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fe05c036750 0x7fe05c038c00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe060009990 tx=0x7fe060006e40 comp rx=0 tx=0).stop 2024-05-12T03:52:41.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:41.700+0000 7fe0786cc700 1 -- 172.21.15.33:0/2388128302 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe070071de0 msgr2=0x7fe07010f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:52:41.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:41.700+0000 7fe0786cc700 1 --2- 172.21.15.33:0/2388128302 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe070071de0 0x7fe07010f200 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe06c00b860 tx=0x7fe06c009e30 comp rx=0 tx=0).stop 2024-05-12T03:52:41.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:41.700+0000 7fe0786cc700 1 -- 172.21.15.33:0/2388128302 shutdown_connections 2024-05-12T03:52:41.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:41.700+0000 7fe0786cc700 1 --2- 172.21.15.33:0/2388128302 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fe05c036750 0x7fe05c038c00 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:52:41.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:41.700+0000 7fe0786cc700 1 --2- 172.21.15.33:0/2388128302 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe070071de0 0x7fe07010f200 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:52:41.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:41.700+0000 7fe0786cc700 1 -- 172.21.15.33:0/2388128302 >> 172.21.15.33:0/2388128302 conn(0x7fe0700f81c0 msgr2=0x7fe0700f8a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:52:41.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:41.700+0000 7fe0786cc700 1 -- 172.21.15.33:0/2388128302 shutdown_connections 2024-05-12T03:52:41.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:41.700+0000 7fe0786cc700 1 -- 172.21.15.33:0/2388128302 wait complete. 2024-05-12T03:52:41.701 INFO:teuthology.orchestra.run.smithi033.stderr:dumped monmap epoch 1 2024-05-12T03:52:42.274 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:41 smithi032 ceph-mon[40008]: [12/May/2024:03:52:40] ENGINE Bus STARTING 2024-05-12T03:52:42.274 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:41 smithi032 ceph-mon[40008]: [12/May/2024:03:52:40] ENGINE Serving on https://172.21.15.32:7150 2024-05-12T03:52:42.274 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:41 smithi032 ceph-mon[40008]: [12/May/2024:03:52:40] ENGINE Bus STARTED 2024-05-12T03:52:42.274 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:41 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:52:42.275 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:41 smithi032 ceph-mon[40008]: from='client.? 172.21.15.33:0/2388128302' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-12T03:52:42.275 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:41 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:52:43.444 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-12T03:52:43.445 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-12T03:52:43.831 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-12T03:52:44.039 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:43 smithi032 ceph-mon[40008]: mgrmap e17: smithi032.ukatkq(active, since 2s) 2024-05-12T03:52:44.288 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-12T03:52:45.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:45 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:52:45.542 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:45 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:52:45.542 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:45 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:52:45.542 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:45 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:52:45.542 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:45 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "who": "osd/host:smithi032", "name": "osd_memory_target"}]: dispatch 2024-05-12T03:52:46.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:46.928+0000 7fd837d13700 1 -- 172.21.15.33:0/1268397803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8300fda70 msgr2=0x7fd8300fde40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:52:46.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:46.928+0000 7fd837d13700 1 --2- 172.21.15.33:0/1268397803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8300fda70 0x7fd8300fde40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd82c0048c0 tx=0x7fd82c009e30 comp rx=0 tx=0).stop 2024-05-12T03:52:46.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:46.928+0000 7fd837d13700 1 -- 172.21.15.33:0/1268397803 shutdown_connections 2024-05-12T03:52:46.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:46.928+0000 7fd837d13700 1 --2- 172.21.15.33:0/1268397803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8300fda70 0x7fd8300fde40 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:52:46.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:46.928+0000 7fd837d13700 1 -- 172.21.15.33:0/1268397803 >> 172.21.15.33:0/1268397803 conn(0x7fd830071800 msgr2=0x7fd830071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:52:46.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:46.928+0000 7fd837d13700 1 -- 172.21.15.33:0/1268397803 shutdown_connections 2024-05-12T03:52:46.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:46.928+0000 7fd837d13700 1 -- 172.21.15.33:0/1268397803 wait complete. 2024-05-12T03:52:46.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:46.928+0000 7fd837d13700 1 Processor -- start 2024-05-12T03:52:46.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:46.928+0000 7fd837d13700 1 -- start start 2024-05-12T03:52:46.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:46.929+0000 7fd837d13700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8300fda70 0x7fd8300fb930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:52:46.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:46.929+0000 7fd837d13700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8300fbe70 con 0x7fd8300fda70 2024-05-12T03:52:46.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:46.929+0000 7fd835aaf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8300fda70 0x7fd8300fb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:52:46.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:46.929+0000 7fd835aaf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8300fda70 0x7fd8300fb930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.33:58448/0 (socket says 172.21.15.33:58448) 2024-05-12T03:52:46.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:46.929+0000 7fd835aaf700 1 -- 172.21.15.33:0/731156585 learned_addr learned my addr 172.21.15.33:0/731156585 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-12T03:52:46.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:46.929+0000 7fd835aaf700 1 -- 172.21.15.33:0/731156585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd82c005040 con 0x7fd8300fda70 2024-05-12T03:52:46.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:46.929+0000 7fd835aaf700 1 --2- 172.21.15.33:0/731156585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8300fda70 0x7fd8300fb930 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd82c007e30 tx=0x7fd82c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:52:46.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:46.930+0000 7fd8267fc700 1 -- 172.21.15.33:0/731156585 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd82c0093c0 con 0x7fd8300fda70 2024-05-12T03:52:46.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:46.930+0000 7fd8267fc700 1 -- 172.21.15.33:0/731156585 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd82c013420 con 0x7fd8300fda70 2024-05-12T03:52:46.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:46.930+0000 7fd837d13700 1 -- 172.21.15.33:0/731156585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8300fc070 con 0x7fd8300fda70 2024-05-12T03:52:46.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:46.930+0000 7fd8267fc700 1 -- 172.21.15.33:0/731156585 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd82c01c5b0 con 0x7fd8300fda70 2024-05-12T03:52:46.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:46.930+0000 7fd837d13700 1 -- 172.21.15.33:0/731156585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8300fc4e0 con 0x7fd8300fda70 2024-05-12T03:52:46.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:46.930+0000 7fd8267fc700 1 -- 172.21.15.33:0/731156585 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 43489+0+0 (secure 0 0 0) 0x7fd82c01c7e0 con 0x7fd8300fda70 2024-05-12T03:52:46.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:46.931+0000 7fd8267fc700 1 --2- 172.21.15.33:0/731156585 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fd81c036a00 0x7fd81c038eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:52:46.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:46.931+0000 7fd8267fc700 1 -- 172.21.15.33:0/731156585 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fd82c04ab80 con 0x7fd8300fda70 2024-05-12T03:52:46.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:46.931+0000 7fd8352ae700 1 --2- 172.21.15.33:0/731156585 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fd81c036a00 0x7fd81c038eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:52:46.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:46.931+0000 7fd837d13700 1 -- 172.21.15.33:0/731156585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd830102730 con 0x7fd8300fda70 2024-05-12T03:52:46.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:46.931+0000 7fd8352ae700 1 --2- 172.21.15.33:0/731156585 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fd81c036a00 0x7fd81c038eb0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd820009990 tx=0x7fd820006e40 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:52:46.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:46.938+0000 7fd8267fc700 1 -- 172.21.15.33:0/731156585 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd82c023300 con 0x7fd8300fda70 2024-05-12T03:52:47.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:47.237+0000 7fd837d13700 1 -- 172.21.15.33:0/731156585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd8300fc720 con 0x7fd8300fda70 2024-05-12T03:52:47.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:47.238+0000 7fd8267fc700 1 -- 172.21.15.33:0/731156585 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fd82c011b10 con 0x7fd8300fda70 2024-05-12T03:52:47.238 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:52:47.238 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":1,"fsid":"a3e178f8-1012-11ef-bc98-c7b262605968","modified":"2024-05-12T03:50:15.576069Z","created":"2024-05-12T03:50:15.576069Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi032","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:3300","nonce":0},{"type":"v1","addr":"172.21.15.32:6789","nonce":0}]},"addr":"172.21.15.32:6789/0","public_addr":"172.21.15.32:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-12T03:52:47.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:47.241+0000 7fd837d13700 1 -- 172.21.15.33:0/731156585 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fd81c036a00 msgr2=0x7fd81c038eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:52:47.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:47.241+0000 7fd837d13700 1 --2- 172.21.15.33:0/731156585 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fd81c036a00 0x7fd81c038eb0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd820009990 tx=0x7fd820006e40 comp rx=0 tx=0).stop 2024-05-12T03:52:47.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:47.241+0000 7fd837d13700 1 -- 172.21.15.33:0/731156585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8300fda70 msgr2=0x7fd8300fb930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:52:47.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:47.241+0000 7fd837d13700 1 --2- 172.21.15.33:0/731156585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8300fda70 0x7fd8300fb930 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd82c007e30 tx=0x7fd82c009e30 comp rx=0 tx=0).stop 2024-05-12T03:52:47.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:47.241+0000 7fd837d13700 1 -- 172.21.15.33:0/731156585 shutdown_connections 2024-05-12T03:52:47.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:47.241+0000 7fd837d13700 1 --2- 172.21.15.33:0/731156585 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fd81c036a00 0x7fd81c038eb0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:52:47.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:47.241+0000 7fd837d13700 1 --2- 172.21.15.33:0/731156585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8300fda70 0x7fd8300fb930 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:52:47.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:47.241+0000 7fd837d13700 1 -- 172.21.15.33:0/731156585 >> 172.21.15.33:0/731156585 conn(0x7fd830071800 msgr2=0x7fd8300fa7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:52:47.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:47.241+0000 7fd837d13700 1 -- 172.21.15.33:0/731156585 shutdown_connections 2024-05-12T03:52:47.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:47.241+0000 7fd837d13700 1 -- 172.21.15.33:0/731156585 wait complete. 2024-05-12T03:52:47.242 INFO:teuthology.orchestra.run.smithi033.stderr:dumped monmap epoch 1 2024-05-12T03:52:47.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:46 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:52:47.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:46 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:52:47.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:46 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:52:47.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:46 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:52:47.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:46 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "who": "osd/host:smithi033", "name": "osd_memory_target"}]: dispatch 2024-05-12T03:52:47.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:46 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:52:47.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:46 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T03:52:47.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:46 smithi032 ceph-mon[40008]: Updating smithi032:/etc/ceph/ceph.conf 2024-05-12T03:52:47.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:46 smithi032 ceph-mon[40008]: Updating smithi033:/etc/ceph/ceph.conf 2024-05-12T03:52:48.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:47 smithi032 ceph-mon[40008]: Updating smithi032:/var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-12T03:52:48.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:47 smithi032 ceph-mon[40008]: Updating smithi033:/var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-12T03:52:48.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:47 smithi032 ceph-mon[40008]: Updating smithi032:/etc/ceph/ceph.client.admin.keyring 2024-05-12T03:52:48.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:47 smithi032 ceph-mon[40008]: Updating smithi033:/etc/ceph/ceph.client.admin.keyring 2024-05-12T03:52:48.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:47 smithi032 ceph-mon[40008]: Updating smithi032:/var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-12T03:52:48.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:47 smithi032 ceph-mon[40008]: Updating smithi033:/var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-12T03:52:48.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:47 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:52:48.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:47 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:52:48.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:47 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:52:48.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:47 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:52:48.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:47 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:52:48.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:47 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi033", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-12T03:52:48.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:47 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi033", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-05-12T03:52:48.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:47 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:52:48.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:47 smithi032 ceph-mon[40008]: Deploying daemon crash.smithi033 on smithi033 2024-05-12T03:52:48.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:47 smithi032 ceph-mon[40008]: from='client.? 172.21.15.33:0/731156585' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-12T03:52:49.596 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-12T03:52:49.596 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-12T03:52:50.263 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-12T03:52:51.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:51 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:52:51.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:51 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:52:51.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:51 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:52:51.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:51 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:52:51.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:51 smithi032 ceph-mon[40008]: Deploying daemon node-exporter.smithi033 on smithi033 2024-05-12T03:52:51.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:51.908+0000 7f52ef2e4700 1 -- 172.21.15.33:0/2352497987 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52e8102a50 msgr2=0x7f52e8102e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:52:51.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:51.908+0000 7f52ef2e4700 1 --2- 172.21.15.33:0/2352497987 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52e8102a50 0x7f52e8102e20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f52e40048c0 tx=0x7f52e4009e30 comp rx=0 tx=0).stop 2024-05-12T03:52:51.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:51.908+0000 7f52ef2e4700 1 -- 172.21.15.33:0/2352497987 shutdown_connections 2024-05-12T03:52:51.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:51.908+0000 7f52ef2e4700 1 --2- 172.21.15.33:0/2352497987 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52e8102a50 0x7f52e8102e20 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:52:51.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:51.908+0000 7f52ef2e4700 1 -- 172.21.15.33:0/2352497987 >> 172.21.15.33:0/2352497987 conn(0x7f52e8073090 msgr2=0x7f52e8073490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:52:51.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:51.908+0000 7f52ef2e4700 1 -- 172.21.15.33:0/2352497987 shutdown_connections 2024-05-12T03:52:51.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:51.908+0000 7f52ef2e4700 1 -- 172.21.15.33:0/2352497987 wait complete. 2024-05-12T03:52:51.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:51.909+0000 7f52ef2e4700 1 Processor -- start 2024-05-12T03:52:51.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:51.909+0000 7f52ef2e4700 1 -- start start 2024-05-12T03:52:51.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:51.909+0000 7f52ef2e4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52e8102a50 0x7f52e8113050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:52:51.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:51.909+0000 7f52ef2e4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e8113590 con 0x7f52e8102a50 2024-05-12T03:52:51.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:51.909+0000 7f52ed080700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52e8102a50 0x7f52e8113050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:52:51.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:51.909+0000 7f52ed080700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52e8102a50 0x7f52e8113050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.33:41052/0 (socket says 172.21.15.33:41052) 2024-05-12T03:52:51.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:51.909+0000 7f52ed080700 1 -- 172.21.15.33:0/1484425685 learned_addr learned my addr 172.21.15.33:0/1484425685 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-12T03:52:51.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:51.910+0000 7f52ed080700 1 -- 172.21.15.33:0/1484425685 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52e4005040 con 0x7f52e8102a50 2024-05-12T03:52:51.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:51.910+0000 7f52ed080700 1 --2- 172.21.15.33:0/1484425685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52e8102a50 0x7f52e8113050 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f52e4007e80 tx=0x7f52e4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:52:51.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:51.910+0000 7f52ddffb700 1 -- 172.21.15.33:0/1484425685 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f52e40093c0 con 0x7f52e8102a50 2024-05-12T03:52:51.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:51.910+0000 7f52ef2e4700 1 -- 172.21.15.33:0/1484425685 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52e8113790 con 0x7f52e8102a50 2024-05-12T03:52:51.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:51.910+0000 7f52ddffb700 1 -- 172.21.15.33:0/1484425685 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f52e4013420 con 0x7f52e8102a50 2024-05-12T03:52:51.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:51.910+0000 7f52ddffb700 1 -- 172.21.15.33:0/1484425685 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f52e401c5b0 con 0x7f52e8102a50 2024-05-12T03:52:51.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:51.910+0000 7f52ef2e4700 1 -- 172.21.15.33:0/1484425685 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52e8113c80 con 0x7f52e8102a50 2024-05-12T03:52:51.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:51.911+0000 7f52ddffb700 1 -- 172.21.15.33:0/1484425685 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 43489+0+0 (secure 0 0 0) 0x7f52e401c830 con 0x7f52e8102a50 2024-05-12T03:52:51.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:51.911+0000 7f52ddffb700 1 --2- 172.21.15.33:0/1484425685 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f52d4036970 0x7f52d4038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:52:51.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:51.911+0000 7f52ddffb700 1 -- 172.21.15.33:0/1484425685 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f52e4049b10 con 0x7f52e8102a50 2024-05-12T03:52:51.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:51.911+0000 7f52ec87f700 1 --2- 172.21.15.33:0/1484425685 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f52d4036970 0x7f52d4038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:52:51.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:51.912+0000 7f52ec87f700 1 --2- 172.21.15.33:0/1484425685 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f52d4036970 0x7f52d4038e20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f52d8009990 tx=0x7f52d8006e40 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:52:51.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:51.915+0000 7f52ef2e4700 1 -- 172.21.15.33:0/1484425685 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52e80fe350 con 0x7f52e8102a50 2024-05-12T03:52:51.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:51.920+0000 7f52ddffb700 1 -- 172.21.15.33:0/1484425685 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f52e4023300 con 0x7f52e8102a50 2024-05-12T03:52:52.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:52.202+0000 7f52ef2e4700 1 -- 172.21.15.33:0/1484425685 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f52e804bf00 con 0x7f52e8102a50 2024-05-12T03:52:52.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:52.203+0000 7f52ddffb700 1 -- 172.21.15.33:0/1484425685 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f52e4034090 con 0x7f52e8102a50 2024-05-12T03:52:52.203 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:52:52.203 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":1,"fsid":"a3e178f8-1012-11ef-bc98-c7b262605968","modified":"2024-05-12T03:50:15.576069Z","created":"2024-05-12T03:50:15.576069Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi032","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:3300","nonce":0},{"type":"v1","addr":"172.21.15.32:6789","nonce":0}]},"addr":"172.21.15.32:6789/0","public_addr":"172.21.15.32:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-12T03:52:52.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:52.205+0000 7f52ef2e4700 1 -- 172.21.15.33:0/1484425685 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f52d4036970 msgr2=0x7f52d4038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:52:52.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:52.205+0000 7f52ef2e4700 1 --2- 172.21.15.33:0/1484425685 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f52d4036970 0x7f52d4038e20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f52d8009990 tx=0x7f52d8006e40 comp rx=0 tx=0).stop 2024-05-12T03:52:52.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:52.205+0000 7f52ef2e4700 1 -- 172.21.15.33:0/1484425685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52e8102a50 msgr2=0x7f52e8113050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:52:52.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:52.205+0000 7f52ef2e4700 1 --2- 172.21.15.33:0/1484425685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52e8102a50 0x7f52e8113050 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f52e4007e80 tx=0x7f52e4009e30 comp rx=0 tx=0).stop 2024-05-12T03:52:52.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:52.205+0000 7f52ef2e4700 1 -- 172.21.15.33:0/1484425685 shutdown_connections 2024-05-12T03:52:52.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:52.205+0000 7f52ef2e4700 1 --2- 172.21.15.33:0/1484425685 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f52d4036970 0x7f52d4038e20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:52:52.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:52.205+0000 7f52ef2e4700 1 --2- 172.21.15.33:0/1484425685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52e8102a50 0x7f52e8113050 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:52:52.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:52.205+0000 7f52ef2e4700 1 -- 172.21.15.33:0/1484425685 >> 172.21.15.33:0/1484425685 conn(0x7f52e8073090 msgr2=0x7f52e80f9f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:52:52.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:52.205+0000 7f52ef2e4700 1 -- 172.21.15.33:0/1484425685 shutdown_connections 2024-05-12T03:52:52.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:52.205+0000 7f52ef2e4700 1 -- 172.21.15.33:0/1484425685 wait complete. 2024-05-12T03:52:52.207 INFO:teuthology.orchestra.run.smithi033.stderr:dumped monmap epoch 1 2024-05-12T03:52:52.789 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:52 smithi032 ceph-mon[40008]: from='client.? 172.21.15.33:0/1484425685' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-12T03:52:54.067 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-12T03:52:54.068 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-12T03:52:54.333 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-12T03:52:56.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:55 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:52:56.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:55 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T03:52:56.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:55 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:52:56.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:55 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:52:56.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:55 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:52:56.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:55 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:52:56.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:55 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi033.pcvxwy", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-12T03:52:56.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:55 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi033.pcvxwy", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-12T03:52:56.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:55 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-12T03:52:56.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:55 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:52:56.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:56.930+0000 7f9665b31700 1 -- 172.21.15.33:0/2200337901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9660102aa0 msgr2=0x7f9660102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:52:56.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:56.930+0000 7f9665b31700 1 --2- 172.21.15.33:0/2200337901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9660102aa0 0x7f9660102e70 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f96500048c0 tx=0x7f9650009e30 comp rx=0 tx=0).stop 2024-05-12T03:52:56.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:56.930+0000 7f9665b31700 1 -- 172.21.15.33:0/2200337901 shutdown_connections 2024-05-12T03:52:56.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:56.930+0000 7f9665b31700 1 --2- 172.21.15.33:0/2200337901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9660102aa0 0x7f9660102e70 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:52:56.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:56.930+0000 7f9665b31700 1 -- 172.21.15.33:0/2200337901 >> 172.21.15.33:0/2200337901 conn(0x7f96600f85c0 msgr2=0x7f96600fa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:52:56.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:56.930+0000 7f9665b31700 1 -- 172.21.15.33:0/2200337901 shutdown_connections 2024-05-12T03:52:56.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:56.930+0000 7f9665b31700 1 -- 172.21.15.33:0/2200337901 wait complete. 2024-05-12T03:52:56.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:56.931+0000 7f9665b31700 1 Processor -- start 2024-05-12T03:52:56.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:56.931+0000 7f9665b31700 1 -- start start 2024-05-12T03:52:56.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:56.932+0000 7f9665b31700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9660102aa0 0x7f9660119620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:52:56.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:56.932+0000 7f9665b31700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9660119b60 con 0x7f9660102aa0 2024-05-12T03:52:56.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:56.932+0000 7f965f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9660102aa0 0x7f9660119620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:52:56.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:56.932+0000 7f965f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9660102aa0 0x7f9660119620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.33:41066/0 (socket says 172.21.15.33:41066) 2024-05-12T03:52:56.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:56.932+0000 7f965f7fe700 1 -- 172.21.15.33:0/716853394 learned_addr learned my addr 172.21.15.33:0/716853394 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-12T03:52:56.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:56.932+0000 7f965f7fe700 1 -- 172.21.15.33:0/716853394 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9650005040 con 0x7f9660102aa0 2024-05-12T03:52:56.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:56.933+0000 7f965f7fe700 1 --2- 172.21.15.33:0/716853394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9660102aa0 0x7f9660119620 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9650007e30 tx=0x7f9650009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:52:56.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:56.933+0000 7f9647fff700 1 -- 172.21.15.33:0/716853394 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f96500093c0 con 0x7f9660102aa0 2024-05-12T03:52:56.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:56.933+0000 7f9647fff700 1 -- 172.21.15.33:0/716853394 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9650013420 con 0x7f9660102aa0 2024-05-12T03:52:56.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:56.933+0000 7f9665b31700 1 -- 172.21.15.33:0/716853394 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9660119d60 con 0x7f9660102aa0 2024-05-12T03:52:56.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:56.933+0000 7f9647fff700 1 -- 172.21.15.33:0/716853394 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f965001c5b0 con 0x7f9660102aa0 2024-05-12T03:52:56.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:56.933+0000 7f9665b31700 1 -- 172.21.15.33:0/716853394 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f966011a1d0 con 0x7f9660102aa0 2024-05-12T03:52:56.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:56.934+0000 7f9647fff700 1 -- 172.21.15.33:0/716853394 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 43489+0+0 (secure 0 0 0) 0x7f965001c830 con 0x7f9660102aa0 2024-05-12T03:52:56.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:56.934+0000 7f9665b31700 1 -- 172.21.15.33:0/716853394 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96600fe1a0 con 0x7f9660102aa0 2024-05-12T03:52:56.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:56.934+0000 7f9647fff700 1 --2- 172.21.15.33:0/716853394 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f9648036960 0x7f9648038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:52:56.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:56.934+0000 7f9647fff700 1 -- 172.21.15.33:0/716853394 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f9650049c30 con 0x7f9660102aa0 2024-05-12T03:52:56.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:56.934+0000 7f965effd700 1 --2- 172.21.15.33:0/716853394 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f9648036960 0x7f9648038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:52:56.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:56.935+0000 7f965effd700 1 --2- 172.21.15.33:0/716853394 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f9648036960 0x7f9648038e10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9654009990 tx=0x7f9654006e40 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:52:56.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:56.939+0000 7f9647fff700 1 -- 172.21.15.33:0/716853394 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f9650011db0 con 0x7f9660102aa0 2024-05-12T03:52:57.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:57.224+0000 7f9665b31700 1 -- 172.21.15.33:0/716853394 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f966004bed0 con 0x7f9660102aa0 2024-05-12T03:52:57.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:57.225+0000 7f9647fff700 1 -- 172.21.15.33:0/716853394 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f9650049440 con 0x7f9660102aa0 2024-05-12T03:52:57.228 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:52:57.228 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":1,"fsid":"a3e178f8-1012-11ef-bc98-c7b262605968","modified":"2024-05-12T03:50:15.576069Z","created":"2024-05-12T03:50:15.576069Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi032","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:3300","nonce":0},{"type":"v1","addr":"172.21.15.32:6789","nonce":0}]},"addr":"172.21.15.32:6789/0","public_addr":"172.21.15.32:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-12T03:52:57.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:57.227+0000 7f9665b31700 1 -- 172.21.15.33:0/716853394 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f9648036960 msgr2=0x7f9648038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:52:57.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:57.227+0000 7f9665b31700 1 --2- 172.21.15.33:0/716853394 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f9648036960 0x7f9648038e10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9654009990 tx=0x7f9654006e40 comp rx=0 tx=0).stop 2024-05-12T03:52:57.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:57.227+0000 7f9665b31700 1 -- 172.21.15.33:0/716853394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9660102aa0 msgr2=0x7f9660119620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:52:57.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:57.227+0000 7f9665b31700 1 --2- 172.21.15.33:0/716853394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9660102aa0 0x7f9660119620 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9650007e30 tx=0x7f9650009e30 comp rx=0 tx=0).stop 2024-05-12T03:52:57.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:57.227+0000 7f9665b31700 1 -- 172.21.15.33:0/716853394 shutdown_connections 2024-05-12T03:52:57.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:57.227+0000 7f9665b31700 1 --2- 172.21.15.33:0/716853394 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f9648036960 0x7f9648038e10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:52:57.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:57.227+0000 7f9665b31700 1 --2- 172.21.15.33:0/716853394 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9660102aa0 0x7f9660119620 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:52:57.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:57.227+0000 7f9665b31700 1 -- 172.21.15.33:0/716853394 >> 172.21.15.33:0/716853394 conn(0x7f96600f85c0 msgr2=0x7f96600f9280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:52:57.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:57.227+0000 7f9665b31700 1 -- 172.21.15.33:0/716853394 shutdown_connections 2024-05-12T03:52:57.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:52:57.227+0000 7f9665b31700 1 -- 172.21.15.33:0/716853394 wait complete. 2024-05-12T03:52:57.229 INFO:teuthology.orchestra.run.smithi033.stderr:dumped monmap epoch 1 2024-05-12T03:52:57.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:56 smithi032 ceph-mon[40008]: Deploying daemon mgr.smithi033.pcvxwy on smithi033 2024-05-12T03:52:58.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:52:57 smithi032 ceph-mon[40008]: from='client.? 172.21.15.33:0/716853394' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-12T03:53:00.017 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-12T03:53:00.018 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-12T03:53:00.305 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-12T03:53:01.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:00 smithi032 ceph-mon[40008]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:53:01.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:00 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:01.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:00 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:01.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:00 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:01.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:00 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:01.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:00 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:01.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:00 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-12T03:53:01.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:00 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:53:01.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:00 smithi032 ceph-mon[40008]: Deploying daemon mon.smithi033 on smithi033 2024-05-12T03:53:03.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:02 smithi032 ceph-mon[40008]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:53:04.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:04.071+0000 7fe73cd7a700 1 -- 172.21.15.33:0/794851433 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7380fda70 msgr2=0x7fe7380fde40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:53:04.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:04.071+0000 7fe73cd7a700 1 --2- 172.21.15.33:0/794851433 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7380fda70 0x7fe7380fde40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe7280048c0 tx=0x7fe728009e30 comp rx=0 tx=0).stop 2024-05-12T03:53:04.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:04.071+0000 7fe73cd7a700 1 -- 172.21.15.33:0/794851433 shutdown_connections 2024-05-12T03:53:04.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:04.071+0000 7fe73cd7a700 1 --2- 172.21.15.33:0/794851433 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7380fda70 0x7fe7380fde40 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:53:04.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:04.071+0000 7fe73cd7a700 1 -- 172.21.15.33:0/794851433 >> 172.21.15.33:0/794851433 conn(0x7fe738071800 msgr2=0x7fe738071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:53:04.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:04.071+0000 7fe73cd7a700 1 -- 172.21.15.33:0/794851433 shutdown_connections 2024-05-12T03:53:04.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:04.071+0000 7fe73cd7a700 1 -- 172.21.15.33:0/794851433 wait complete. 2024-05-12T03:53:04.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:04.071+0000 7fe73cd7a700 1 Processor -- start 2024-05-12T03:53:04.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:04.071+0000 7fe73cd7a700 1 -- start start 2024-05-12T03:53:04.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:04.072+0000 7fe73cd7a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7380fda70 0x7fe738113060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:53:04.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:04.072+0000 7fe73cd7a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7381135a0 con 0x7fe7380fda70 2024-05-12T03:53:04.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:04.072+0000 7fe73659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7380fda70 0x7fe738113060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:53:04.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:04.072+0000 7fe73659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7380fda70 0x7fe738113060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.33:60112/0 (socket says 172.21.15.33:60112) 2024-05-12T03:53:04.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:04.072+0000 7fe73659c700 1 -- 172.21.15.33:0/1707397750 learned_addr learned my addr 172.21.15.33:0/1707397750 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-12T03:53:04.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:04.072+0000 7fe73659c700 1 -- 172.21.15.33:0/1707397750 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe728005040 con 0x7fe7380fda70 2024-05-12T03:53:04.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:04.073+0000 7fe73659c700 1 --2- 172.21.15.33:0/1707397750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7380fda70 0x7fe738113060 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe728007e80 tx=0x7fe728009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:53:04.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:04.073+0000 7fe726ffd700 1 -- 172.21.15.33:0/1707397750 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe7280093c0 con 0x7fe7380fda70 2024-05-12T03:53:04.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:04.073+0000 7fe73cd7a700 1 -- 172.21.15.33:0/1707397750 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7381137a0 con 0x7fe7380fda70 2024-05-12T03:53:04.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:04.073+0000 7fe726ffd700 1 -- 172.21.15.33:0/1707397750 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe728013420 con 0x7fe7380fda70 2024-05-12T03:53:04.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:04.073+0000 7fe726ffd700 1 -- 172.21.15.33:0/1707397750 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe72801c5b0 con 0x7fe7380fda70 2024-05-12T03:53:04.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:04.073+0000 7fe73cd7a700 1 -- 172.21.15.33:0/1707397750 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe738113c10 con 0x7fe7380fda70 2024-05-12T03:53:04.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:04.074+0000 7fe73cd7a700 1 -- 172.21.15.33:0/1707397750 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7380658b0 con 0x7fe7380fda70 2024-05-12T03:53:04.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:04.074+0000 7fe726ffd700 1 -- 172.21.15.33:0/1707397750 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 43489+0+0 (secure 0 0 0) 0x7fe728009520 con 0x7fe7380fda70 2024-05-12T03:53:04.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:04.074+0000 7fe726ffd700 1 --2- 172.21.15.33:0/1707397750 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fe720036970 0x7fe720038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:53:04.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:04.074+0000 7fe726ffd700 1 -- 172.21.15.33:0/1707397750 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fe728049a80 con 0x7fe7380fda70 2024-05-12T03:53:04.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:04.074+0000 7fe735d9b700 1 --2- 172.21.15.33:0/1707397750 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fe720036970 0x7fe720038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:53:04.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:04.074+0000 7fe735d9b700 1 --2- 172.21.15.33:0/1707397750 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fe720036970 0x7fe720038e20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe72c009990 tx=0x7fe72c006e40 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:53:04.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:04.079+0000 7fe726ffd700 1 -- 172.21.15.33:0/1707397750 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fe728013a00 con 0x7fe7380fda70 2024-05-12T03:53:04.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:04.354+0000 7fe73cd7a700 1 -- 172.21.15.33:0/1707397750 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fe73805f5c0 con 0x7fe7380fda70 2024-05-12T03:53:04.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:04.355+0000 7fe726ffd700 1 -- 172.21.15.33:0/1707397750 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fe728011370 con 0x7fe7380fda70 2024-05-12T03:53:04.355 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:53:04.355 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":1,"fsid":"a3e178f8-1012-11ef-bc98-c7b262605968","modified":"2024-05-12T03:50:15.576069Z","created":"2024-05-12T03:50:15.576069Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi032","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:3300","nonce":0},{"type":"v1","addr":"172.21.15.32:6789","nonce":0}]},"addr":"172.21.15.32:6789/0","public_addr":"172.21.15.32:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-12T03:53:04.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:04.357+0000 7fe73cd7a700 1 -- 172.21.15.33:0/1707397750 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fe720036970 msgr2=0x7fe720038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:53:04.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:04.357+0000 7fe73cd7a700 1 --2- 172.21.15.33:0/1707397750 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fe720036970 0x7fe720038e20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe72c009990 tx=0x7fe72c006e40 comp rx=0 tx=0).stop 2024-05-12T03:53:04.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:04.357+0000 7fe73cd7a700 1 -- 172.21.15.33:0/1707397750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7380fda70 msgr2=0x7fe738113060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:53:04.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:04.357+0000 7fe73cd7a700 1 --2- 172.21.15.33:0/1707397750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7380fda70 0x7fe738113060 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe728007e80 tx=0x7fe728009e30 comp rx=0 tx=0).stop 2024-05-12T03:53:04.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:04.357+0000 7fe73cd7a700 1 -- 172.21.15.33:0/1707397750 shutdown_connections 2024-05-12T03:53:04.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:04.357+0000 7fe73cd7a700 1 --2- 172.21.15.33:0/1707397750 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fe720036970 0x7fe720038e20 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:53:04.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:04.357+0000 7fe73cd7a700 1 --2- 172.21.15.33:0/1707397750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7380fda70 0x7fe738113060 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:53:04.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:04.357+0000 7fe73cd7a700 1 -- 172.21.15.33:0/1707397750 >> 172.21.15.33:0/1707397750 conn(0x7fe738071800 msgr2=0x7fe7380fa7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:53:04.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:04.357+0000 7fe73cd7a700 1 -- 172.21.15.33:0/1707397750 shutdown_connections 2024-05-12T03:53:04.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:04.357+0000 7fe73cd7a700 1 -- 172.21.15.33:0/1707397750 wait complete. 2024-05-12T03:53:04.358 INFO:teuthology.orchestra.run.smithi033.stderr:dumped monmap epoch 1 2024-05-12T03:53:04.848 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:04 smithi033 systemd[1]: Starting Ceph mon.smithi033 for a3e178f8-1012-11ef-bc98-c7b262605968... 2024-05-12T03:53:05.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:04 smithi032 ceph-mon[40008]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:53:05.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:04 smithi032 ceph-mon[40008]: from='client.? 172.21.15.33:0/1707397750' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-12T03:53:06.078 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: set uid:gid to 167:167 (ceph:ceph) 2024-05-12T03:53:06.078 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable), process ceph-mon, pid 7 2024-05-12T03:53:06.078 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: pidfile_write: ignore empty --pid-file 2024-05-12T03:53:06.078 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: load: jerasure load: lrc 2024-05-12T03:53:06.078 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: RocksDB version: 6.15.5 2024-05-12T03:53:06.078 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-05-12T03:53:06.078 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Compile date May 10 2024 2024-05-12T03:53:06.078 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: DB SUMMARY 2024-05-12T03:53:06.078 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: DB Session ID: 42UNW9WH9AKFQ0NMNPRD 2024-05-12T03:53:06.078 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: CURRENT file: CURRENT 2024-05-12T03:53:06.078 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: IDENTITY file: IDENTITY 2024-05-12T03:53:06.078 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: MANIFEST file: MANIFEST-000003 size: 57 Bytes 2024-05-12T03:53:06.079 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi033/store.db dir, Total Num: 0, files: 2024-05-12T03:53:06.079 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi033/store.db: 000004.log size: 511 ; 2024-05-12T03:53:06.079 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.error_if_exists: 0 2024-05-12T03:53:06.079 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.create_if_missing: 0 2024-05-12T03:53:06.079 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.paranoid_checks: 1 2024-05-12T03:53:06.079 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-12T03:53:06.079 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.env: 0x55f0ff38ab00 2024-05-12T03:53:06.079 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.fs: Posix File System 2024-05-12T03:53:06.079 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.info_log: 0x55f1017db620 2024-05-12T03:53:06.079 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.max_file_opening_threads: 16 2024-05-12T03:53:06.079 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.statistics: (nil) 2024-05-12T03:53:06.079 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.use_fsync: 0 2024-05-12T03:53:06.079 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.max_log_file_size: 0 2024-05-12T03:53:06.079 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-12T03:53:06.079 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-12T03:53:06.079 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.keep_log_file_num: 1000 2024-05-12T03:53:06.080 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.recycle_log_file_num: 0 2024-05-12T03:53:06.080 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.allow_fallocate: 1 2024-05-12T03:53:06.080 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.allow_mmap_reads: 0 2024-05-12T03:53:06.080 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.allow_mmap_writes: 0 2024-05-12T03:53:06.080 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.use_direct_reads: 0 2024-05-12T03:53:06.080 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-12T03:53:06.080 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.create_missing_column_families: 0 2024-05-12T03:53:06.080 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.db_log_dir: 2024-05-12T03:53:06.080 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-smithi033/store.db 2024-05-12T03:53:06.080 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-12T03:53:06.080 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-12T03:53:06.082 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-12T03:53:06.082 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-12T03:53:06.082 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-12T03:53:06.082 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-12T03:53:06.082 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.advise_random_on_open: 1 2024-05-12T03:53:06.082 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.db_write_buffer_size: 0 2024-05-12T03:53:06.082 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.write_buffer_manager: 0x55f1024cea20 2024-05-12T03:53:06.082 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-12T03:53:06.083 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-05-12T03:53:06.083 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-12T03:53:06.083 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-12T03:53:06.083 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.rate_limiter: (nil) 2024-05-12T03:53:06.083 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-12T03:53:06.083 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.wal_recovery_mode: 2 2024-05-12T03:53:06.083 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.enable_thread_tracking: 0 2024-05-12T03:53:06.083 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.enable_pipelined_write: 0 2024-05-12T03:53:06.083 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.unordered_write: 0 2024-05-12T03:53:06.083 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-12T03:53:06.083 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-12T03:53:06.083 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-12T03:53:06.083 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-12T03:53:06.083 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.row_cache: None 2024-05-12T03:53:06.084 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.wal_filter: None 2024-05-12T03:53:06.084 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-12T03:53:06.084 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.allow_ingest_behind: 0 2024-05-12T03:53:06.084 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.preserve_deletes: 0 2024-05-12T03:53:06.084 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.two_write_queues: 0 2024-05-12T03:53:06.084 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.manual_wal_flush: 0 2024-05-12T03:53:06.084 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.atomic_flush: 0 2024-05-12T03:53:06.084 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-12T03:53:06.084 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-12T03:53:06.084 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-12T03:53:06.084 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.log_readahead_size: 0 2024-05-12T03:53:06.084 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-12T03:53:06.084 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.best_efforts_recovery: 0 2024-05-12T03:53:06.084 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-12T03:53:06.085 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-12T03:53:06.085 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.allow_data_in_errors: 0 2024-05-12T03:53:06.085 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.db_host_id: __hostname__ 2024-05-12T03:53:06.085 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.max_background_jobs: 2 2024-05-12T03:53:06.085 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.max_background_compactions: -1 2024-05-12T03:53:06.085 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.max_subcompactions: 1 2024-05-12T03:53:06.085 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-12T03:53:06.085 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-12T03:53:06.085 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-12T03:53:06.085 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.max_total_wal_size: 0 2024-05-12T03:53:06.085 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-12T03:53:06.085 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-12T03:53:06.085 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-12T03:53:06.085 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-12T03:53:06.085 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.max_open_files: -1 2024-05-12T03:53:06.086 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.bytes_per_sync: 0 2024-05-12T03:53:06.086 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-12T03:53:06.086 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-12T03:53:06.086 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.compaction_readahead_size: 0 2024-05-12T03:53:06.086 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.max_background_flushes: -1 2024-05-12T03:53:06.086 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Compression algorithms supported: 2024-05-12T03:53:06.086 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-12T03:53:06.086 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: kZSTD supported: 0 2024-05-12T03:53:06.086 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: kXpressCompression supported: 0 2024-05-12T03:53:06.086 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: kLZ4HCCompression supported: 1 2024-05-12T03:53:06.086 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: kZlibCompression supported: 1 2024-05-12T03:53:06.086 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: kSnappyCompression supported: 1 2024-05-12T03:53:06.086 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: kLZ4Compression supported: 1 2024-05-12T03:53:06.087 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: kBZip2Compression supported: 0 2024-05-12T03:53:06.087 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-12T03:53:06.087 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi033/store.db/MANIFEST-000003 2024-05-12T03:53:06.087 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-05-12T03:53:06.087 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-12T03:53:06.087 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.merge_operator: 2024-05-12T03:53:06.087 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.compaction_filter: None 2024-05-12T03:53:06.087 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.compaction_filter_factory: None 2024-05-12T03:53:06.087 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.sst_partitioner_factory: None 2024-05-12T03:53:06.087 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-12T03:53:06.087 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-12T03:53:06.087 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55f1017aef58) 2024-05-12T03:53:06.087 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: cache_index_and_filter_blocks: 1 2024-05-12T03:53:06.087 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-12T03:53:06.087 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-12T03:53:06.088 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: pin_top_level_index_and_filter: 1 2024-05-12T03:53:06.088 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: index_type: 0 2024-05-12T03:53:06.088 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: data_block_index_type: 0 2024-05-12T03:53:06.088 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: index_shortening: 1 2024-05-12T03:53:06.088 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-12T03:53:06.088 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: hash_index_allow_collision: 1 2024-05-12T03:53:06.088 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: checksum: 1 2024-05-12T03:53:06.088 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: no_block_cache: 0 2024-05-12T03:53:06.088 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: block_cache: 0x55f1017e1090 2024-05-12T03:53:06.088 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: block_cache_name: BinnedLRUCache 2024-05-12T03:53:06.088 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: block_cache_options: 2024-05-12T03:53:06.088 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: capacity : 536870912 2024-05-12T03:53:06.088 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: num_shard_bits : 4 2024-05-12T03:53:06.089 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: strict_capacity_limit : 0 2024-05-12T03:53:06.089 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: high_pri_pool_ratio: 0.000 2024-05-12T03:53:06.089 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: block_cache_compressed: (nil) 2024-05-12T03:53:06.089 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: persistent_cache: (nil) 2024-05-12T03:53:06.089 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: block_size: 4096 2024-05-12T03:53:06.089 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: block_size_deviation: 10 2024-05-12T03:53:06.089 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: block_restart_interval: 16 2024-05-12T03:53:06.089 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: index_block_restart_interval: 1 2024-05-12T03:53:06.089 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: metadata_block_size: 4096 2024-05-12T03:53:06.089 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: partition_filters: 0 2024-05-12T03:53:06.089 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: use_delta_encoding: 1 2024-05-12T03:53:06.089 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-05-12T03:53:06.089 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: whole_key_filtering: 1 2024-05-12T03:53:06.090 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: verify_compression: 0 2024-05-12T03:53:06.090 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: read_amp_bytes_per_bit: 0 2024-05-12T03:53:06.090 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: format_version: 4 2024-05-12T03:53:06.090 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: enable_index_compression: 1 2024-05-12T03:53:06.090 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: block_align: 0 2024-05-12T03:53:06.090 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.write_buffer_size: 33554432 2024-05-12T03:53:06.090 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.max_write_buffer_number: 2 2024-05-12T03:53:06.090 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.compression: NoCompression 2024-05-12T03:53:06.090 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.bottommost_compression: Disabled 2024-05-12T03:53:06.090 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.prefix_extractor: nullptr 2024-05-12T03:53:06.090 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-12T03:53:06.090 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.num_levels: 7 2024-05-12T03:53:06.090 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-12T03:53:06.091 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-12T03:53:06.091 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-12T03:53:06.091 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-12T03:53:06.091 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-12T03:53:06.091 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-12T03:53:06.091 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-12T03:53:06.091 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-12T03:53:06.091 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-12T03:53:06.091 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-12T03:53:06.091 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-12T03:53:06.091 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.compression_opts.level: 32767 2024-05-12T03:53:06.091 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.compression_opts.strategy: 0 2024-05-12T03:53:06.091 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-12T03:53:06.091 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-12T03:53:06.092 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-12T03:53:06.092 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.compression_opts.enabled: false 2024-05-12T03:53:06.092 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-12T03:53:06.092 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-12T03:53:06.092 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-12T03:53:06.092 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.target_file_size_base: 67108864 2024-05-12T03:53:06.092 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-12T03:53:06.092 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-12T03:53:06.092 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-12T03:53:06.092 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-12T03:53:06.092 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-12T03:53:06.092 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-12T03:53:06.092 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-12T03:53:06.092 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-12T03:53:06.093 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-12T03:53:06.093 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-12T03:53:06.093 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-12T03:53:06.093 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-12T03:53:06.093 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-12T03:53:06.093 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.arena_block_size: 4194304 2024-05-12T03:53:06.093 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-12T03:53:06.093 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-12T03:53:06.093 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-05-12T03:53:06.093 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.disable_auto_compactions: 0 2024-05-12T03:53:06.093 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-12T03:53:06.093 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-12T03:53:06.093 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-12T03:53:06.093 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-12T03:53:06.093 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-12T03:53:06.094 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-12T03:53:06.094 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-12T03:53:06.094 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-12T03:53:06.094 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-12T03:53:06.094 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-12T03:53:06.094 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.table_properties_collectors: 2024-05-12T03:53:06.094 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.inplace_update_support: 0 2024-05-12T03:53:06.094 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-12T03:53:06.094 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-12T03:53:06.094 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-12T03:53:06.094 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-12T03:53:06.094 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.bloom_locality: 0 2024-05-12T03:53:06.094 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.max_successive_merges: 0 2024-05-12T03:53:06.094 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-12T03:53:06.095 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.paranoid_file_checks: 0 2024-05-12T03:53:06.095 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.force_consistency_checks: 1 2024-05-12T03:53:06.095 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.report_bg_io_stats: 0 2024-05-12T03:53:06.095 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.ttl: 2592000 2024-05-12T03:53:06.095 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-12T03:53:06.095 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.enable_blob_files: false 2024-05-12T03:53:06.095 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.min_blob_size: 0 2024-05-12T03:53:06.095 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.blob_file_size: 268435456 2024-05-12T03:53:06.095 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-12T03:53:06.095 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-12T03:53:06.095 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-12T03:53:06.095 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi033/store.db/MANIFEST-000003 succeeded,manifest_file_number is 3, next_file_number is 5, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-12T03:53:06.095 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 0 2024-05-12T03:53:06.095 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: [db/version_set.cc:4082] Creating manifest 7 2024-05-12T03:53:06.095 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715485985745463, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-12T03:53:06.096 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #4 mode 2 2024-05-12T03:53:06.096 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-05-12T03:53:06.096 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715485985745749, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1545, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1715485985, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "3c4959c9-cbf5-4cf4-92f0-0bee4772d0b1", "db_session_id": "42UNW9WH9AKFQ0NMNPRD"}} 2024-05-12T03:53:06.096 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: [db/version_set.cc:4082] Creating manifest 9 2024-05-12T03:53:06.096 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715485985746093, "job": 1, "event": "recovery_finished"} 2024-05-12T03:53:06.096 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-smithi033/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-12T03:53:06.096 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x55f101775180 2024-05-12T03:53:06.096 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: DB pointer 0x55f10184a000 2024-05-12T03:53:06.096 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-05-12T03:53:06.096 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-05-12T03:53:06.096 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: ** DB Stats ** 2024-05-12T03:53:06.097 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-12T03:53:06.097 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-12T03:53:06.097 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-12T03:53:06.097 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-12T03:53:06.097 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-12T03:53:06.097 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-05-12T03:53:06.097 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-12T03:53:06.097 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: 2024-05-12T03:53:06.097 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: ** Compaction Stats [default] ** 2024-05-12T03:53:06.098 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-05-12T03:53:06.098 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-12T03:53:06.098 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: L0 1/0 1.51 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.4 0.00 0.00 1 0.000 0 0 2024-05-12T03:53:06.098 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Sum 1/0 1.51 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.4 0.00 0.00 1 0.000 0 0 2024-05-12T03:53:06.098 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.4 0.00 0.00 1 0.000 0 0 2024-05-12T03:53:06.098 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: 2024-05-12T03:53:06.098 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: ** Compaction Stats [default] ** 2024-05-12T03:53:06.098 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-05-12T03:53:06.098 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-12T03:53:06.098 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 5.4 0.00 0.00 1 0.000 0 0 2024-05-12T03:53:06.098 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-12T03:53:06.099 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-12T03:53:06.099 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-12T03:53:06.099 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-12T03:53:06.099 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-12T03:53:06.099 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-12T03:53:06.099 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Cumulative compaction: 0.00 GB write, 0.54 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-12T03:53:06.099 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Interval compaction: 0.00 GB write, 0.54 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-12T03:53:06.099 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-12T03:53:06.099 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: 2024-05-12T03:53:06.099 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-12T03:53:06.099 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: 2024-05-12T03:53:06.099 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: ** Compaction Stats [default] ** 2024-05-12T03:53:06.099 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-05-12T03:53:06.099 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-12T03:53:06.100 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: L0 1/0 1.51 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.4 0.00 0.00 1 0.000 0 0 2024-05-12T03:53:06.100 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Sum 1/0 1.51 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.4 0.00 0.00 1 0.000 0 0 2024-05-12T03:53:06.100 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-05-12T03:53:06.100 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: 2024-05-12T03:53:06.100 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: ** Compaction Stats [default] ** 2024-05-12T03:53:06.100 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-05-12T03:53:06.100 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-12T03:53:06.100 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 5.4 0.00 0.00 1 0.000 0 0 2024-05-12T03:53:06.100 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-12T03:53:06.100 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-12T03:53:06.100 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-12T03:53:06.100 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-12T03:53:06.100 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-12T03:53:06.100 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-12T03:53:06.100 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Cumulative compaction: 0.00 GB write, 0.54 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-12T03:53:06.101 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-12T03:53:06.101 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-12T03:53:06.101 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: 2024-05-12T03:53:06.101 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-12T03:53:06.101 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: mon.smithi033 does not exist in monmap, will attempt to join an existing cluster 2024-05-12T03:53:06.101 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: using public_addr v2:172.21.15.33:0/0 -> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] 2024-05-12T03:53:06.101 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: starting mon.smithi033 rank -1 at public addrs [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] at bind addrs [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi033 fsid a3e178f8-1012-11ef-bc98-c7b262605968 2024-05-12T03:53:06.101 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: mon.smithi033@-1(???) e0 preinit fsid a3e178f8-1012-11ef-bc98-c7b262605968 2024-05-12T03:53:06.101 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: mon.smithi033@-1(synchronizing).mds e1 new map 2024-05-12T03:53:06.101 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: mon.smithi033@-1(synchronizing).mds e1 print_map 2024-05-12T03:53:06.101 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: e1 2024-05-12T03:53:06.101 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-12T03:53:06.101 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-12T03:53:06.101 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: legacy client fscid: -1 2024-05-12T03:53:06.101 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: 2024-05-12T03:53:06.102 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: No filesystems configured 2024-05-12T03:53:06.102 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: mon.smithi033@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-12T03:53:06.102 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: mon.smithi033@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-12T03:53:06.102 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: mon.smithi033@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-12T03:53:06.102 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: mon.smithi033@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-12T03:53:06.102 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: mon.smithi033@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-12T03:53:06.102 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: mon.smithi033@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-12T03:53:06.102 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: mon.smithi033@-1(synchronizing).osd e5 e5: 0 total, 0 up, 0 in 2024-05-12T03:53:06.102 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: mon.smithi033@-1(synchronizing).osd e5 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-12T03:53:06.102 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: mon.smithi033@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-05-12T03:53:06.102 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: mon.smithi033@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-05-12T03:53:06.102 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: mon.smithi033@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-05-12T03:53:06.102 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: from='client.? 172.21.15.33:0/716853394' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-12T03:53:06.102 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:53:06.102 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:06.103 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:06.103 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:06.103 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:06.103 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:06.103 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-12T03:53:06.103 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:53:06.103 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: Deploying daemon mon.smithi033 on smithi033 2024-05-12T03:53:06.103 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:53:06.103 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:53:06.103 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: from='client.? 172.21.15.33:0/1707397750' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-12T03:53:06.103 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 ceph-mon[49525]: mon.smithi033@-1(synchronizing).paxosservice(auth 1..6) refresh upgraded, format 0 -> 3 2024-05-12T03:53:06.103 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 bash[49482]: 707654d85748775d5c372cebcb353dfb60e6616c3b14f18ebd8c34d014a47caa 2024-05-12T03:53:06.103 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:05 smithi033 systemd[1]: Started Ceph mon.smithi033 for a3e178f8-1012-11ef-bc98-c7b262605968. 2024-05-12T03:53:06.126 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-05-12T03:53:06.126 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-12T03:53:06.412 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi033/config 2024-05-12T03:53:11.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:10 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mon metadata", "id": "smithi032"}]: dispatch 2024-05-12T03:53:11.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:10 smithi033 ceph-mon[49525]: mon.smithi032 calling monitor election 2024-05-12T03:53:11.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:10 smithi033 ceph-mon[49525]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:53:11.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:10 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mon metadata", "id": "smithi033"}]: dispatch 2024-05-12T03:53:11.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:10 smithi033 ceph-mon[49525]: from='mgr.? 172.21.15.33:0/1711909816' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi033.pcvxwy/crt"}]: dispatch 2024-05-12T03:53:11.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:10 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mon metadata", "id": "smithi033"}]: dispatch 2024-05-12T03:53:11.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:10 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mon metadata", "id": "smithi033"}]: dispatch 2024-05-12T03:53:11.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:10 smithi033 ceph-mon[49525]: mon.smithi033 calling monitor election 2024-05-12T03:53:11.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:10 smithi033 ceph-mon[49525]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:53:11.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:10 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mon metadata", "id": "smithi033"}]: dispatch 2024-05-12T03:53:11.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:10 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mon metadata", "id": "smithi033"}]: dispatch 2024-05-12T03:53:11.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:10 smithi033 ceph-mon[49525]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:53:11.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:10 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T03:53:11.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:10 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mon metadata", "id": "smithi033"}]: dispatch 2024-05-12T03:53:11.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:10 smithi033 ceph-mon[49525]: mon.smithi032 is new leader, mons smithi032,smithi033 in quorum (ranks 0,1) 2024-05-12T03:53:11.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:10 smithi033 ceph-mon[49525]: monmap e2: 2 mons at {smithi032=[v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0],smithi033=[v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-12T03:53:11.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:10 smithi033 ceph-mon[49525]: fsmap 2024-05-12T03:53:11.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:10 smithi033 ceph-mon[49525]: osdmap e5: 0 total, 0 up, 0 in 2024-05-12T03:53:11.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:10 smithi033 ceph-mon[49525]: mgrmap e17: smithi032.ukatkq(active, since 31s) 2024-05-12T03:53:11.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:10 smithi033 ceph-mon[49525]: overall HEALTH_OK 2024-05-12T03:53:11.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:10 smithi033 ceph-mon[49525]: Standby manager daemon smithi033.pcvxwy started 2024-05-12T03:53:11.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:10 smithi033 ceph-mon[49525]: from='mgr.? 172.21.15.33:0/1711909816' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-12T03:53:11.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:10 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:11.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:10 smithi033 ceph-mon[49525]: from='mgr.? 172.21.15.33:0/1711909816' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi033.pcvxwy/key"}]: dispatch 2024-05-12T03:53:11.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:10 smithi033 ceph-mon[49525]: from='mgr.? 172.21.15.33:0/1711909816' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-12T03:53:11.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:10 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:11.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:10 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:11.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:10 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:11.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:11.224+0000 7fc347711700 1 -- 172.21.15.33:0/1210489363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3400fdba0 msgr2=0x7fc32c0055d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:53:11.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:11.224+0000 7fc347711700 1 --2- 172.21.15.33:0/1210489363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3400fdba0 0x7fc32c0055d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc33c007ea0 tx=0x7fc33c00b040 comp rx=0 tx=0).stop 2024-05-12T03:53:11.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:11.224+0000 7fc347711700 1 -- 172.21.15.33:0/1210489363 shutdown_connections 2024-05-12T03:53:11.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:11.224+0000 7fc347711700 1 --2- 172.21.15.33:0/1210489363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3400fdba0 0x7fc32c0055d0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:53:11.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:11.224+0000 7fc347711700 1 -- 172.21.15.33:0/1210489363 >> 172.21.15.33:0/1210489363 conn(0x7fc340071970 msgr2=0x7fc340071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:53:11.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:11.225+0000 7fc347711700 1 -- 172.21.15.33:0/1210489363 shutdown_connections 2024-05-12T03:53:11.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:11.225+0000 7fc347711700 1 -- 172.21.15.33:0/1210489363 wait complete. 2024-05-12T03:53:11.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:11.225+0000 7fc347711700 1 Processor -- start 2024-05-12T03:53:11.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:11.225+0000 7fc347711700 1 -- start start 2024-05-12T03:53:11.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:11.225+0000 7fc347711700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3400fdba0 0x7fc3400fba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:53:11.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:11.225+0000 7fc347711700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3400fbfa0 0x7fc3400fc410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:53:11.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:11.225+0000 7fc347711700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc340119540 con 0x7fc3400fdba0 2024-05-12T03:53:11.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:11.225+0000 7fc347711700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc340119680 con 0x7fc3400fbfa0 2024-05-12T03:53:11.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:11.226+0000 7fc3454ad700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3400fdba0 0x7fc3400fba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:53:11.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:11.226+0000 7fc344cac700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3400fbfa0 0x7fc3400fc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:53:11.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:11.226+0000 7fc3454ad700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3400fdba0 0x7fc3400fba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.33:60180/0 (socket says 172.21.15.33:60180) 2024-05-12T03:53:11.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:11.226+0000 7fc3454ad700 1 -- 172.21.15.33:0/3504832674 learned_addr learned my addr 172.21.15.33:0/3504832674 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-12T03:53:11.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:11.226+0000 7fc344cac700 1 -- 172.21.15.33:0/3504832674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3400fbfa0 msgr2=0x7fc3400fc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-12T03:53:11.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:11.226+0000 7fc344cac700 1 -- 172.21.15.33:0/3504832674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3400fbfa0 msgr2=0x7fc3400fc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-12T03:53:11.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:11.226+0000 7fc344cac700 1 --2- 172.21.15.33:0/3504832674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3400fbfa0 0x7fc3400fc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-05-12T03:53:11.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:11.226+0000 7fc344cac700 1 --2- 172.21.15.33:0/3504832674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3400fbfa0 0x7fc3400fc410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-12T03:53:11.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:11.226+0000 7fc3454ad700 1 -- 172.21.15.33:0/3504832674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3400fbfa0 msgr2=0x7fc3400fc410 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-12T03:53:11.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:11.226+0000 7fc3454ad700 1 --2- 172.21.15.33:0/3504832674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3400fbfa0 0x7fc3400fc410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:53:11.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:11.226+0000 7fc3454ad700 1 -- 172.21.15.33:0/3504832674 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc33c005040 con 0x7fc3400fdba0 2024-05-12T03:53:11.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:11.226+0000 7fc3454ad700 1 --2- 172.21.15.33:0/3504832674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3400fdba0 0x7fc3400fba60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc33c007d60 tx=0x7fc33c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:53:11.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:11.226+0000 7fc331ffb700 1 -- 172.21.15.33:0/3504832674 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc33c026030 con 0x7fc3400fdba0 2024-05-12T03:53:11.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:11.226+0000 7fc331ffb700 1 -- 172.21.15.33:0/3504832674 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc33c0097e0 con 0x7fc3400fdba0 2024-05-12T03:53:11.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:11.226+0000 7fc347711700 1 -- 172.21.15.33:0/3504832674 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc340119820 con 0x7fc3400fdba0 2024-05-12T03:53:11.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:11.226+0000 7fc331ffb700 1 -- 172.21.15.33:0/3504832674 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc33c01b030 con 0x7fc3400fdba0 2024-05-12T03:53:11.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:11.227+0000 7fc347711700 1 -- 172.21.15.33:0/3504832674 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc340119d60 con 0x7fc3400fdba0 2024-05-12T03:53:11.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:11.227+0000 7fc347711700 1 -- 172.21.15.33:0/3504832674 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc340102320 con 0x7fc3400fdba0 2024-05-12T03:53:11.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:11.228+0000 7fc331ffb700 1 -- 172.21.15.33:0/3504832674 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 86573+0+0 (secure 0 0 0) 0x7fc33c029070 con 0x7fc3400fdba0 2024-05-12T03:53:11.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:11.228+0000 7fc331ffb700 1 --2- 172.21.15.33:0/3504832674 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fc3240693b0 0x7fc32406b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:53:11.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:11.228+0000 7fc331ffb700 1 -- 172.21.15.33:0/3504832674 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fc33c027070 con 0x7fc3400fdba0 2024-05-12T03:53:11.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:11.229+0000 7fc344cac700 1 --2- 172.21.15.33:0/3504832674 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fc3240693b0 0x7fc32406b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:53:11.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:11.229+0000 7fc344cac700 1 --2- 172.21.15.33:0/3504832674 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fc3240693b0 0x7fc32406b860 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc334006f30 tx=0x7fc334008040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:53:11.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:11.233+0000 7fc331ffb700 1 -- 172.21.15.33:0/3504832674 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fc33c054e60 con 0x7fc3400fdba0 2024-05-12T03:53:11.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:10 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mon metadata", "id": "smithi032"}]: dispatch 2024-05-12T03:53:11.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:10 smithi032 ceph-mon[40008]: mon.smithi032 calling monitor election 2024-05-12T03:53:11.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:10 smithi032 ceph-mon[40008]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:53:11.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:10 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mon metadata", "id": "smithi033"}]: dispatch 2024-05-12T03:53:11.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:10 smithi032 ceph-mon[40008]: from='mgr.? 172.21.15.33:0/1711909816' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi033.pcvxwy/crt"}]: dispatch 2024-05-12T03:53:11.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:10 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mon metadata", "id": "smithi033"}]: dispatch 2024-05-12T03:53:11.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:10 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mon metadata", "id": "smithi033"}]: dispatch 2024-05-12T03:53:11.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:10 smithi032 ceph-mon[40008]: mon.smithi033 calling monitor election 2024-05-12T03:53:11.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:10 smithi032 ceph-mon[40008]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:53:11.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:10 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mon metadata", "id": "smithi033"}]: dispatch 2024-05-12T03:53:11.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:10 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mon metadata", "id": "smithi033"}]: dispatch 2024-05-12T03:53:11.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:10 smithi032 ceph-mon[40008]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:53:11.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:10 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T03:53:11.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:10 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mon metadata", "id": "smithi033"}]: dispatch 2024-05-12T03:53:11.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:10 smithi032 ceph-mon[40008]: mon.smithi032 is new leader, mons smithi032,smithi033 in quorum (ranks 0,1) 2024-05-12T03:53:11.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:10 smithi032 ceph-mon[40008]: monmap e2: 2 mons at {smithi032=[v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0],smithi033=[v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-12T03:53:11.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:10 smithi032 ceph-mon[40008]: fsmap 2024-05-12T03:53:11.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:10 smithi032 ceph-mon[40008]: osdmap e5: 0 total, 0 up, 0 in 2024-05-12T03:53:11.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:10 smithi032 ceph-mon[40008]: mgrmap e17: smithi032.ukatkq(active, since 31s) 2024-05-12T03:53:11.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:10 smithi032 ceph-mon[40008]: overall HEALTH_OK 2024-05-12T03:53:11.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:10 smithi032 ceph-mon[40008]: Standby manager daemon smithi033.pcvxwy started 2024-05-12T03:53:11.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:10 smithi032 ceph-mon[40008]: from='mgr.? 172.21.15.33:0/1711909816' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-12T03:53:11.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:10 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:11.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:10 smithi032 ceph-mon[40008]: from='mgr.? 172.21.15.33:0/1711909816' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi033.pcvxwy/key"}]: dispatch 2024-05-12T03:53:11.292 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:10 smithi032 ceph-mon[40008]: from='mgr.? 172.21.15.33:0/1711909816' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-12T03:53:11.292 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:10 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:11.292 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:10 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:11.292 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:10 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:11.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:11.530+0000 7fc347711700 1 -- 172.21.15.33:0/3504832674 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc3400fd180 con 0x7fc3400fdba0 2024-05-12T03:53:11.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:11.530+0000 7fc331ffb700 1 -- 172.21.15.33:0/3504832674 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1013 (secure 0 0 0) 0x7fc33c058480 con 0x7fc3400fdba0 2024-05-12T03:53:11.531 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-12T03:53:11.531 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":2,"fsid":"a3e178f8-1012-11ef-bc98-c7b262605968","modified":"2024-05-12T03:53:05.776719Z","created":"2024-05-12T03:50:15.576069Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi032","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:3300","nonce":0},{"type":"v1","addr":"172.21.15.32:6789","nonce":0}]},"addr":"172.21.15.32:6789/0","public_addr":"172.21.15.32:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"smithi033","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-12T03:53:11.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:11.533+0000 7fc347711700 1 -- 172.21.15.33:0/3504832674 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fc3240693b0 msgr2=0x7fc32406b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:53:11.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:11.533+0000 7fc347711700 1 --2- 172.21.15.33:0/3504832674 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fc3240693b0 0x7fc32406b860 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc334006f30 tx=0x7fc334008040 comp rx=0 tx=0).stop 2024-05-12T03:53:11.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:11.533+0000 7fc347711700 1 -- 172.21.15.33:0/3504832674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3400fdba0 msgr2=0x7fc3400fba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:53:11.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:11.533+0000 7fc347711700 1 --2- 172.21.15.33:0/3504832674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3400fdba0 0x7fc3400fba60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc33c007d60 tx=0x7fc33c00b040 comp rx=0 tx=0).stop 2024-05-12T03:53:11.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:11.533+0000 7fc347711700 1 -- 172.21.15.33:0/3504832674 shutdown_connections 2024-05-12T03:53:11.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:11.533+0000 7fc347711700 1 --2- 172.21.15.33:0/3504832674 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fc3240693b0 0x7fc32406b860 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:53:11.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:11.533+0000 7fc347711700 1 --2- 172.21.15.33:0/3504832674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc3400fdba0 0x7fc3400fba60 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:53:11.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:11.533+0000 7fc347711700 1 --2- 172.21.15.33:0/3504832674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3400fbfa0 0x7fc3400fc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:53:11.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:11.533+0000 7fc347711700 1 -- 172.21.15.33:0/3504832674 >> 172.21.15.33:0/3504832674 conn(0x7fc340071970 msgr2=0x7fc340106010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:53:11.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:11.533+0000 7fc347711700 1 -- 172.21.15.33:0/3504832674 shutdown_connections 2024-05-12T03:53:11.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:53:11.533+0000 7fc347711700 1 -- 172.21.15.33:0/3504832674 wait complete. 2024-05-12T03:53:11.535 INFO:teuthology.orchestra.run.smithi033.stderr:dumped monmap epoch 2 2024-05-12T03:53:12.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:11 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:12.616 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:11 smithi033 ceph-mon[49525]: mgrmap e18: smithi032.ukatkq(active, since 31s), standbys: smithi033.pcvxwy 2024-05-12T03:53:12.616 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:11 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mgr metadata", "who": "smithi033.pcvxwy", "id": "smithi033.pcvxwy"}]: dispatch 2024-05-12T03:53:12.616 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:11 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T03:53:12.617 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:11 smithi033 ceph-mon[49525]: from='client.? 172.21.15.33:0/3504832674' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-12T03:53:12.617 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:11 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mon metadata", "id": "smithi033"}]: dispatch 2024-05-12T03:53:12.617 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:11 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:12.617 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:11 smithi032 ceph-mon[40008]: mgrmap e18: smithi032.ukatkq(active, since 31s), standbys: smithi033.pcvxwy 2024-05-12T03:53:12.617 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:11 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mgr metadata", "who": "smithi033.pcvxwy", "id": "smithi033.pcvxwy"}]: dispatch 2024-05-12T03:53:12.617 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:11 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T03:53:12.617 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:11 smithi032 ceph-mon[40008]: from='client.? 172.21.15.33:0/3504832674' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-12T03:53:12.618 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:11 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mon metadata", "id": "smithi033"}]: dispatch 2024-05-12T03:53:12.691 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-12T03:53:12.691 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- ceph config generate-minimal-conf 2024-05-12T03:53:12.984 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T03:53:13.039 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:12 smithi032 ceph-mon[40008]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:53:13.154 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:12 smithi033 ceph-mon[49525]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:53:14.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:14.846+0000 7fe1ba653700 1 -- 172.21.15.32:0/2906682235 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1b40fda90 msgr2=0x7fe1b40fde60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:53:14.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:14.846+0000 7fe1ba653700 1 --2- 172.21.15.32:0/2906682235 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1b40fda90 0x7fe1b40fde60 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe1a40048c0 tx=0x7fe1a4009e30 comp rx=0 tx=0).stop 2024-05-12T03:53:14.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:14.846+0000 7fe1ba653700 1 -- 172.21.15.32:0/2906682235 shutdown_connections 2024-05-12T03:53:14.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:14.846+0000 7fe1ba653700 1 --2- 172.21.15.32:0/2906682235 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1b40fda90 0x7fe1b40fde60 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:53:14.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:14.846+0000 7fe1ba653700 1 -- 172.21.15.32:0/2906682235 >> 172.21.15.32:0/2906682235 conn(0x7fe1b4071800 msgr2=0x7fe1b4071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:53:14.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:14.846+0000 7fe1ba653700 1 -- 172.21.15.32:0/2906682235 shutdown_connections 2024-05-12T03:53:14.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:14.846+0000 7fe1ba653700 1 -- 172.21.15.32:0/2906682235 wait complete. 2024-05-12T03:53:14.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:14.846+0000 7fe1ba653700 1 Processor -- start 2024-05-12T03:53:14.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:14.846+0000 7fe1ba653700 1 -- start start 2024-05-12T03:53:14.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:14.847+0000 7fe1ba653700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1b40fda90 0x7fe1b41131c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:53:14.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:14.847+0000 7fe1ba653700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1b4113700 0x7fe1b410d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:53:14.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:14.847+0000 7fe1ba653700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b4113d10 con 0x7fe1b40fda90 2024-05-12T03:53:14.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:14.847+0000 7fe1ba653700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b410d660 con 0x7fe1b4113700 2024-05-12T03:53:14.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:14.847+0000 7fe1b3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1b40fda90 0x7fe1b41131c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:53:14.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:14.847+0000 7fe1b3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1b40fda90 0x7fe1b41131c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51680/0 (socket says 172.21.15.32:51680) 2024-05-12T03:53:14.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:14.847+0000 7fe1b3fff700 1 -- 172.21.15.32:0/2924173624 learned_addr learned my addr 172.21.15.32:0/2924173624 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:53:14.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:14.847+0000 7fe1b3fff700 1 -- 172.21.15.32:0/2924173624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1b4113700 msgr2=0x7fe1b410d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:53:14.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:14.847+0000 7fe1b3fff700 1 --2- 172.21.15.32:0/2924173624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1b4113700 0x7fe1b410d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:53:14.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:14.847+0000 7fe1b3fff700 1 -- 172.21.15.32:0/2924173624 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1a4005040 con 0x7fe1b40fda90 2024-05-12T03:53:14.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:14.847+0000 7fe1b3fff700 1 --2- 172.21.15.32:0/2924173624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1b40fda90 0x7fe1b41131c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe1a400bff0 tx=0x7fe1a4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:53:14.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:14.847+0000 7fe1b0ff9700 1 -- 172.21.15.32:0/2924173624 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1a400e040 con 0x7fe1b40fda90 2024-05-12T03:53:14.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:14.847+0000 7fe1b0ff9700 1 -- 172.21.15.32:0/2924173624 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe1a4009930 con 0x7fe1b40fda90 2024-05-12T03:53:14.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:14.847+0000 7fe1ba653700 1 -- 172.21.15.32:0/2924173624 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1b410d8e0 con 0x7fe1b40fda90 2024-05-12T03:53:14.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:14.847+0000 7fe1b0ff9700 1 -- 172.21.15.32:0/2924173624 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1a4013e30 con 0x7fe1b40fda90 2024-05-12T03:53:14.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:14.847+0000 7fe1ba653700 1 -- 172.21.15.32:0/2924173624 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1b410de20 con 0x7fe1b40fda90 2024-05-12T03:53:14.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:14.848+0000 7fe1ba653700 1 -- 172.21.15.32:0/2924173624 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1b41027f0 con 0x7fe1b40fda90 2024-05-12T03:53:14.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:14.849+0000 7fe1b0ff9700 1 -- 172.21.15.32:0/2924173624 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 86573+0+0 (secure 0 0 0) 0x7fe1a401c4d0 con 0x7fe1b40fda90 2024-05-12T03:53:14.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:14.849+0000 7fe1b0ff9700 1 --2- 172.21.15.32:0/2924173624 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fe19c069360 0x7fe19c06b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:53:14.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:14.849+0000 7fe1b0ff9700 1 -- 172.21.15.32:0/2924173624 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fe1a4087450 con 0x7fe1b40fda90 2024-05-12T03:53:14.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:14.849+0000 7fe1b37fe700 1 --2- 172.21.15.32:0/2924173624 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fe19c069360 0x7fe19c06b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:53:14.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:14.849+0000 7fe1b37fe700 1 --2- 172.21.15.32:0/2924173624 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fe19c069360 0x7fe19c06b810 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8006fd0 tx=0x7fe1a8006dd0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:53:14.870 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:14.854+0000 7fe1b0ff9700 1 -- 172.21.15.32:0/2924173624 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fe1a4058270 con 0x7fe1b40fda90 2024-05-12T03:53:15.039 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:14 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:15.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:14 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:15.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:14 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:53:15.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:14 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T03:53:15.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:14 smithi032 ceph-mon[40008]: Updating smithi032:/etc/ceph/ceph.conf 2024-05-12T03:53:15.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:14 smithi032 ceph-mon[40008]: Updating smithi033:/etc/ceph/ceph.conf 2024-05-12T03:53:15.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:14 smithi032 ceph-mon[40008]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:53:15.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:14 smithi032 ceph-mon[40008]: Updating smithi032:/var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-12T03:53:15.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:14 smithi032 ceph-mon[40008]: Updating smithi033:/var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-12T03:53:15.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:14 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:15.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:14 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:15.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:14 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:15.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:14 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:15.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:14 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:15.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:14 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-12T03:53:15.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:14 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-12T03:53:15.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:14 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:53:15.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:15.056+0000 7fe1ba653700 1 -- 172.21.15.32:0/2924173624 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fe1b4106c70 con 0x7fe1b40fda90 2024-05-12T03:53:15.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:15.057+0000 7fe1b0ff9700 1 -- 172.21.15.32:0/2924173624 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v10) v1 ==== 76+0+223 (secure 0 0 0) 0x7fe1a4086c00 con 0x7fe1b40fda90 2024-05-12T03:53:15.058 INFO:teuthology.orchestra.run.smithi032.stdout:# minimal ceph.conf for a3e178f8-1012-11ef-bc98-c7b262605968 2024-05-12T03:53:15.058 INFO:teuthology.orchestra.run.smithi032.stdout:[global] 2024-05-12T03:53:15.058 INFO:teuthology.orchestra.run.smithi032.stdout: fsid = a3e178f8-1012-11ef-bc98-c7b262605968 2024-05-12T03:53:15.058 INFO:teuthology.orchestra.run.smithi032.stdout: mon_host = [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] 2024-05-12T03:53:15.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:15.058+0000 7fe1ba653700 1 -- 172.21.15.32:0/2924173624 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fe19c069360 msgr2=0x7fe19c06b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:53:15.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:15.058+0000 7fe1ba653700 1 --2- 172.21.15.32:0/2924173624 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fe19c069360 0x7fe19c06b810 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8006fd0 tx=0x7fe1a8006dd0 comp rx=0 tx=0).stop 2024-05-12T03:53:15.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:15.058+0000 7fe1ba653700 1 -- 172.21.15.32:0/2924173624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1b40fda90 msgr2=0x7fe1b41131c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:53:15.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:15.058+0000 7fe1ba653700 1 --2- 172.21.15.32:0/2924173624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1b40fda90 0x7fe1b41131c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe1a400bff0 tx=0x7fe1a4009e30 comp rx=0 tx=0).stop 2024-05-12T03:53:15.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:15.059+0000 7fe1ba653700 1 -- 172.21.15.32:0/2924173624 shutdown_connections 2024-05-12T03:53:15.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:15.059+0000 7fe1ba653700 1 --2- 172.21.15.32:0/2924173624 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fe19c069360 0x7fe19c06b810 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:53:15.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:15.059+0000 7fe1ba653700 1 --2- 172.21.15.32:0/2924173624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1b40fda90 0x7fe1b41131c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:53:15.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:15.059+0000 7fe1ba653700 1 --2- 172.21.15.32:0/2924173624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1b4113700 0x7fe1b410d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:53:15.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:15.059+0000 7fe1ba653700 1 -- 172.21.15.32:0/2924173624 >> 172.21.15.32:0/2924173624 conn(0x7fe1b4071800 msgr2=0x7fe1b40faa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:53:15.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:15.059+0000 7fe1ba653700 1 -- 172.21.15.32:0/2924173624 shutdown_connections 2024-05-12T03:53:15.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:15.059+0000 7fe1ba653700 1 -- 172.21.15.32:0/2924173624 wait complete. 2024-05-12T03:53:15.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:14 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:15.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:14 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:15.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:14 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:53:15.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:14 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T03:53:15.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:14 smithi033 ceph-mon[49525]: Updating smithi032:/etc/ceph/ceph.conf 2024-05-12T03:53:15.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:14 smithi033 ceph-mon[49525]: Updating smithi033:/etc/ceph/ceph.conf 2024-05-12T03:53:15.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:14 smithi033 ceph-mon[49525]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:53:15.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:14 smithi033 ceph-mon[49525]: Updating smithi032:/var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-12T03:53:15.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:14 smithi033 ceph-mon[49525]: Updating smithi033:/var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-12T03:53:15.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:14 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:15.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:14 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:15.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:14 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:15.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:14 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:15.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:14 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:15.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:14 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-12T03:53:15.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:14 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-12T03:53:15.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:14 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:53:15.722 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-12T03:53:15.722 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-05-12T03:53:15.722 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/ceph/ceph.conf 2024-05-12T03:53:15.754 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-05-12T03:53:15.754 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-12T03:53:15.822 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-12T03:53:15.823 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/ceph/ceph.conf 2024-05-12T03:53:15.860 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-12T03:53:15.861 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-12T03:53:15.935 INFO:tasks.cephadm:Deploying OSDs... 2024-05-12T03:53:15.935 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-05-12T03:53:15.935 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/scratch_devs of=/dev/stdout 2024-05-12T03:53:15.954 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-12T03:53:15.954 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_1 2024-05-12T03:53:16.015 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-12T03:53:16.015 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-12T03:53:16.015 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 6h/6d Inode: 167381 Links: 1 2024-05-12T03:53:16.015 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-12T03:53:16.015 INFO:teuthology.orchestra.run.smithi032.stdout:Context: system_u:object_r:device_t:s0 2024-05-12T03:53:16.016 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2024-05-12 03:52:31.331625470 +0000 2024-05-12T03:53:16.016 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2024-05-12 03:45:12.472242864 +0000 2024-05-12T03:53:16.016 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2024-05-12 03:45:12.472242864 +0000 2024-05-12T03:53:16.016 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: - 2024-05-12T03:53:16.016 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-12T03:53:16.039 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:15 smithi032 ceph-mon[40008]: Reconfiguring mon.smithi032 (unknown last config time)... 2024-05-12T03:53:16.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:15 smithi032 ceph-mon[40008]: Reconfiguring daemon mon.smithi032 on smithi032 2024-05-12T03:53:16.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:15 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:16.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:15 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/2924173624' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:53:16.066 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2024-05-12T03:53:16.066 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2024-05-12T03:53:16.066 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.000102718 s, 5.0 MB/s 2024-05-12T03:53:16.067 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-12T03:53:16.126 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_2 2024-05-12T03:53:16.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:15 smithi033 ceph-mon[49525]: Reconfiguring mon.smithi032 (unknown last config time)... 2024-05-12T03:53:16.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:15 smithi033 ceph-mon[49525]: Reconfiguring daemon mon.smithi032 on smithi032 2024-05-12T03:53:16.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:15 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:16.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:15 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/2924173624' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:53:16.184 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-12T03:53:16.184 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-12T03:53:16.184 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 6h/6d Inode: 171430 Links: 1 2024-05-12T03:53:16.184 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-12T03:53:16.184 INFO:teuthology.orchestra.run.smithi032.stdout:Context: system_u:object_r:device_t:s0 2024-05-12T03:53:16.184 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2024-05-12 03:52:31.331625470 +0000 2024-05-12T03:53:16.184 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2024-05-12 03:45:12.807236280 +0000 2024-05-12T03:53:16.184 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2024-05-12 03:45:12.807236280 +0000 2024-05-12T03:53:16.184 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: - 2024-05-12T03:53:16.184 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-12T03:53:16.251 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2024-05-12T03:53:16.251 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2024-05-12T03:53:16.251 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 9.7369e-05 s, 5.3 MB/s 2024-05-12T03:53:16.252 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-12T03:53:16.310 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_3 2024-05-12T03:53:16.369 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-12T03:53:16.369 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-12T03:53:16.369 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 6h/6d Inode: 168438 Links: 1 2024-05-12T03:53:16.369 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-12T03:53:16.369 INFO:teuthology.orchestra.run.smithi032.stdout:Context: system_u:object_r:device_t:s0 2024-05-12T03:53:16.369 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2024-05-12 03:52:31.331625470 +0000 2024-05-12T03:53:16.369 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2024-05-12 03:45:13.153229479 +0000 2024-05-12T03:53:16.369 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2024-05-12 03:45:13.153229479 +0000 2024-05-12T03:53:16.369 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: - 2024-05-12T03:53:16.369 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-12T03:53:16.435 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2024-05-12T03:53:16.435 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2024-05-12T03:53:16.435 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 9.3988e-05 s, 5.4 MB/s 2024-05-12T03:53:16.436 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-12T03:53:16.496 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_4 2024-05-12T03:53:16.554 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-12T03:53:16.555 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-12T03:53:16.555 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 6h/6d Inode: 171541 Links: 1 2024-05-12T03:53:16.555 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-12T03:53:16.555 INFO:teuthology.orchestra.run.smithi032.stdout:Context: system_u:object_r:device_t:s0 2024-05-12T03:53:16.555 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2024-05-12 03:52:31.331625470 +0000 2024-05-12T03:53:16.555 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2024-05-12 03:45:13.507222522 +0000 2024-05-12T03:53:16.555 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2024-05-12 03:45:13.507222522 +0000 2024-05-12T03:53:16.555 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: - 2024-05-12T03:53:16.555 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-12T03:53:16.622 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2024-05-12T03:53:16.622 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2024-05-12T03:53:16.622 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.000101803 s, 5.0 MB/s 2024-05-12T03:53:16.623 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-12T03:53:16.682 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-12T03:53:16.682 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/scratch_devs of=/dev/stdout 2024-05-12T03:53:16.705 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-12T03:53:16.705 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_1 2024-05-12T03:53:16.768 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-12T03:53:16.768 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-12T03:53:16.768 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 6h/6d Inode: 169340 Links: 1 2024-05-12T03:53:16.768 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-12T03:53:16.768 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2024-05-12T03:53:16.768 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-05-12 03:53:04.620012917 +0000 2024-05-12T03:53:16.768 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-05-12 03:45:12.984495507 +0000 2024-05-12T03:53:16.768 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-05-12 03:45:12.984495507 +0000 2024-05-12T03:53:16.768 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: - 2024-05-12T03:53:16.769 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-12T03:53:16.840 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-05-12T03:53:16.840 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-05-12T03:53:16.840 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000226769 s, 2.3 MB/s 2024-05-12T03:53:16.841 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-12T03:53:16.903 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_2 2024-05-12T03:53:16.964 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-12T03:53:16.964 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-12T03:53:16.964 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 6h/6d Inode: 172203 Links: 1 2024-05-12T03:53:16.964 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-12T03:53:16.964 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2024-05-12T03:53:16.964 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-05-12 03:53:04.620012917 +0000 2024-05-12T03:53:16.964 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-05-12 03:45:13.330490085 +0000 2024-05-12T03:53:16.964 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-05-12 03:45:13.330490085 +0000 2024-05-12T03:53:16.964 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: - 2024-05-12T03:53:16.964 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-12T03:53:17.032 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-05-12T03:53:17.032 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-05-12T03:53:17.032 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000219844 s, 2.3 MB/s 2024-05-12T03:53:17.033 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-12T03:53:17.094 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_3 2024-05-12T03:53:17.108 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:16 smithi032 ceph-mon[40008]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:53:17.108 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:16 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:17.108 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:16 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:17.108 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:16 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi032.ukatkq", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-12T03:53:17.108 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:16 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-12T03:53:17.108 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:16 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:53:17.154 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-12T03:53:17.154 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-12T03:53:17.154 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 6h/6d Inode: 172252 Links: 1 2024-05-12T03:53:17.154 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-12T03:53:17.154 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2024-05-12T03:53:17.155 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-05-12 03:53:04.621012901 +0000 2024-05-12T03:53:17.155 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-05-12 03:45:13.691484427 +0000 2024-05-12T03:53:17.155 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-05-12 03:45:13.691484427 +0000 2024-05-12T03:53:17.155 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: - 2024-05-12T03:53:17.155 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-12T03:53:17.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:16 smithi033 ceph-mon[49525]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:53:17.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:16 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:17.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:16 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:17.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:16 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi032.ukatkq", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-12T03:53:17.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:16 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-12T03:53:17.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:16 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:53:17.188 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-05-12T03:53:17.188 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-05-12T03:53:17.188 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000193084 s, 2.7 MB/s 2024-05-12T03:53:17.190 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-12T03:53:17.249 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_4 2024-05-12T03:53:17.309 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-12T03:53:17.309 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-12T03:53:17.309 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 6h/6d Inode: 172309 Links: 1 2024-05-12T03:53:17.309 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-12T03:53:17.309 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2024-05-12T03:53:17.309 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-05-12 03:53:04.621012901 +0000 2024-05-12T03:53:17.309 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-05-12 03:45:14.044478895 +0000 2024-05-12T03:53:17.309 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-05-12 03:45:14.044478895 +0000 2024-05-12T03:53:17.309 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: - 2024-05-12T03:53:17.309 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-12T03:53:17.377 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-05-12T03:53:17.378 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-05-12T03:53:17.378 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000215862 s, 2.4 MB/s 2024-05-12T03:53:17.379 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-12T03:53:17.442 INFO:tasks.cephadm:Deploying osd.0 on smithi032 with /dev/vg_nvme/lv_4... 2024-05-12T03:53:17.443 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-12T03:53:18.039 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:17 smithi032 ceph-mon[40008]: Reconfiguring mgr.smithi032.ukatkq (unknown last config time)... 2024-05-12T03:53:18.039 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:17 smithi032 ceph-mon[40008]: Reconfiguring daemon mgr.smithi032.ukatkq on smithi032 2024-05-12T03:53:18.149 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T03:53:18.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:17 smithi033 ceph-mon[49525]: Reconfiguring mgr.smithi032.ukatkq (unknown last config time)... 2024-05-12T03:53:18.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:17 smithi033 ceph-mon[49525]: Reconfiguring daemon mgr.smithi032.ukatkq on smithi032 2024-05-12T03:53:19.039 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:18 smithi032 ceph-mon[40008]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:53:19.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:18 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:19.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:18 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:19.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:18 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi032", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-12T03:53:19.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:18 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:53:19.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:18 smithi033 ceph-mon[49525]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:53:19.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:18 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:19.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:18 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:19.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:18 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi032", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-12T03:53:19.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:18 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:53:19.933 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:19 smithi032 ceph-mon[40008]: Reconfiguring crash.smithi032 (monmap changed)... 2024-05-12T03:53:19.934 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:19 smithi032 ceph-mon[40008]: Reconfiguring daemon crash.smithi032 on smithi032 2024-05-12T03:53:20.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:19 smithi033 ceph-mon[49525]: Reconfiguring crash.smithi032 (monmap changed)... 2024-05-12T03:53:20.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:19 smithi033 ceph-mon[49525]: Reconfiguring daemon crash.smithi032 on smithi032 2024-05-12T03:53:21.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:20 smithi032 ceph-mon[40008]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:53:21.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:20 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:21.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:20 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:21.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:20 smithi032 ceph-mon[40008]: Reconfiguring alertmanager.smithi032 (dependencies changed)... 2024-05-12T03:53:21.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:20 smithi032 ceph-mon[40008]: Reconfiguring daemon alertmanager.smithi032 on smithi032 2024-05-12T03:53:21.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:20 smithi033 ceph-mon[49525]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:53:21.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:20 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:21.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:20 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:21.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:20 smithi033 ceph-mon[49525]: Reconfiguring alertmanager.smithi032 (dependencies changed)... 2024-05-12T03:53:21.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:20 smithi033 ceph-mon[49525]: Reconfiguring daemon alertmanager.smithi032 on smithi032 2024-05-12T03:53:21.698 INFO:teuthology.orchestra.run.smithi032.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-12T03:53:21.698 INFO:teuthology.orchestra.run.smithi032.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-12T03:53:21.698 INFO:teuthology.orchestra.run.smithi032.stderr: stderr: 10+0 records in 2024-05-12T03:53:21.698 INFO:teuthology.orchestra.run.smithi032.stderr:10+0 records out 2024-05-12T03:53:21.698 INFO:teuthology.orchestra.run.smithi032.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00803887 s, 1.3 GB/s 2024-05-12T03:53:21.698 INFO:teuthology.orchestra.run.smithi032.stderr:--> Zapping successful for: 2024-05-12T03:53:22.413 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi032:vg_nvme/lv_4 2024-05-12T03:53:23.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:22 smithi032 ceph-mon[40008]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:53:23.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:22 smithi033 ceph-mon[49525]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:53:24.438 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T03:53:25.286 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:24 smithi032 ceph-mon[40008]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:53:25.286 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:24 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T03:53:25.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:24 smithi033 ceph-mon[49525]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:53:25.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:24 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T03:53:26.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:26.891+0000 7ff767daa700 1 -- 172.21.15.32:0/1713130931 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7600fbcb0 msgr2=0x7ff7600fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:53:26.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:26.891+0000 7ff767daa700 1 --2- 172.21.15.32:0/1713130931 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7600fbcb0 0x7ff7600fc0c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff75c004940 tx=0x7ff75c009e30 comp rx=0 tx=0).stop 2024-05-12T03:53:26.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:26.891+0000 7ff767daa700 1 -- 172.21.15.32:0/1713130931 shutdown_connections 2024-05-12T03:53:26.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:26.891+0000 7ff767daa700 1 --2- 172.21.15.32:0/1713130931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7600fcf60 0x7ff7600fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:53:26.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:26.891+0000 7ff767daa700 1 --2- 172.21.15.32:0/1713130931 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7600fbcb0 0x7ff7600fc0c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:53:26.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:26.891+0000 7ff767daa700 1 -- 172.21.15.32:0/1713130931 >> 172.21.15.32:0/1713130931 conn(0x7ff7600f7430 msgr2=0x7ff7600f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:53:26.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:26.892+0000 7ff767daa700 1 -- 172.21.15.32:0/1713130931 shutdown_connections 2024-05-12T03:53:26.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:26.892+0000 7ff767daa700 1 -- 172.21.15.32:0/1713130931 wait complete. 2024-05-12T03:53:26.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:26.893+0000 7ff767daa700 1 Processor -- start 2024-05-12T03:53:26.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:26.893+0000 7ff767daa700 1 -- start start 2024-05-12T03:53:26.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:26.893+0000 7ff767daa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7600fbcb0 0x7ff7601105d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:53:26.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:26.893+0000 7ff767daa700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7600fcf60 0x7ff760112b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:53:26.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:26.893+0000 7ff767daa700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7600663e0 con 0x7ff7600fbcb0 2024-05-12T03:53:26.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:26.893+0000 7ff767daa700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff760066520 con 0x7ff7600fcf60 2024-05-12T03:53:26.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:26.893+0000 7ff765b46700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7600fbcb0 0x7ff7601105d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:53:26.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:26.893+0000 7ff765b46700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7600fbcb0 0x7ff7601105d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47974/0 (socket says 172.21.15.32:47974) 2024-05-12T03:53:26.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:26.893+0000 7ff765b46700 1 -- 172.21.15.32:0/1747442396 learned_addr learned my addr 172.21.15.32:0/1747442396 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:53:26.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:26.893+0000 7ff765345700 1 --2- 172.21.15.32:0/1747442396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7600fcf60 0x7ff760112b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:53:26.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:26.893+0000 7ff765b46700 1 -- 172.21.15.32:0/1747442396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7600fcf60 msgr2=0x7ff760112b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:53:26.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:26.893+0000 7ff765b46700 1 --2- 172.21.15.32:0/1747442396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7600fcf60 0x7ff760112b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:53:26.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:26.893+0000 7ff765b46700 1 -- 172.21.15.32:0/1747442396 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff75c005040 con 0x7ff7600fbcb0 2024-05-12T03:53:26.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:26.893+0000 7ff765b46700 1 --2- 172.21.15.32:0/1747442396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7600fbcb0 0x7ff7601105d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff75c009380 tx=0x7ff75c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:53:26.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:26.894+0000 7ff7527fc700 1 -- 172.21.15.32:0/1747442396 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff75c00e040 con 0x7ff7600fbcb0 2024-05-12T03:53:26.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:26.894+0000 7ff7527fc700 1 -- 172.21.15.32:0/1747442396 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff75c004290 con 0x7ff7600fbcb0 2024-05-12T03:53:26.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:26.894+0000 7ff7527fc700 1 -- 172.21.15.32:0/1747442396 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff75c01c3f0 con 0x7ff7600fbcb0 2024-05-12T03:53:26.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:26.894+0000 7ff767daa700 1 -- 172.21.15.32:0/1747442396 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff760113120 con 0x7ff7600fbcb0 2024-05-12T03:53:26.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:26.894+0000 7ff767daa700 1 -- 172.21.15.32:0/1747442396 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7601136e0 con 0x7ff7600fbcb0 2024-05-12T03:53:26.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:26.895+0000 7ff767daa700 1 -- 172.21.15.32:0/1747442396 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff76005f5c0 con 0x7ff7600fbcb0 2024-05-12T03:53:26.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:26.895+0000 7ff7527fc700 1 -- 172.21.15.32:0/1747442396 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 86573+0+0 (secure 0 0 0) 0x7ff75c013420 con 0x7ff7600fbcb0 2024-05-12T03:53:26.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:26.895+0000 7ff7527fc700 1 --2- 172.21.15.32:0/1747442396 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7ff74c069210 0x7ff74c06b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:53:26.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:26.896+0000 7ff7527fc700 1 -- 172.21.15.32:0/1747442396 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7ff75c0872b0 con 0x7ff7600fbcb0 2024-05-12T03:53:26.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:26.896+0000 7ff765345700 1 --2- 172.21.15.32:0/1747442396 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7ff74c069210 0x7ff74c06b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:53:26.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:26.896+0000 7ff765345700 1 --2- 172.21.15.32:0/1747442396 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7ff74c069210 0x7ff74c06b6c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff754009bf0 tx=0x7ff7540091a0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:53:26.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:26.901+0000 7ff7527fc700 1 -- 172.21.15.32:0/1747442396 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ff75c054760 con 0x7ff7600fbcb0 2024-05-12T03:53:27.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:27.113+0000 7ff767daa700 1 -- 172.21.15.32:0/1747442396 --> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7ff760101320 con 0x7ff74c069210 2024-05-12T03:53:27.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:26 smithi032 ceph-mon[40008]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:53:27.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:26 smithi033 ceph-mon[49525]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:53:28.154 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:27 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-12T03:53:28.154 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:27 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-12T03:53:28.154 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:27 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:53:28.154 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:27 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:28.154 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:27 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:28.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:27 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-12T03:53:28.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:27 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-12T03:53:28.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:27 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:53:28.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:27 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:28.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:27 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:29.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:28 smithi032 ceph-mon[40008]: from='client.14242 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T03:53:29.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:28 smithi032 ceph-mon[40008]: Reconfiguring grafana.smithi032 (dependencies changed)... 2024-05-12T03:53:29.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:28 smithi032 ceph-mon[40008]: Reconfiguring daemon grafana.smithi032 on smithi032 2024-05-12T03:53:29.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:28 smithi032 ceph-mon[40008]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:53:29.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:28 smithi033 ceph-mon[49525]: from='client.14242 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T03:53:29.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:28 smithi033 ceph-mon[49525]: Reconfiguring grafana.smithi032 (dependencies changed)... 2024-05-12T03:53:29.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:28 smithi033 ceph-mon[49525]: Reconfiguring daemon grafana.smithi032 on smithi032 2024-05-12T03:53:29.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:28 smithi033 ceph-mon[49525]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:53:31.032 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:30 smithi032 ceph-mon[40008]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:53:31.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:30 smithi033 ceph-mon[49525]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:53:33.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:33 smithi032 ceph-mon[40008]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:53:33.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:33 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:33.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:33 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:33.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:33 smithi033 ceph-mon[49525]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:53:33.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:33 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:33.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:33 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:34.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:34 smithi032 ceph-mon[40008]: Reconfiguring prometheus.smithi032 (dependencies changed)... 2024-05-12T03:53:34.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:34 smithi032 ceph-mon[40008]: Reconfiguring daemon prometheus.smithi032 on smithi032 2024-05-12T03:53:34.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:34 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/713476674' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc534129-ef70-487d-b8d1-f98fc170c1d5"}]: dispatch 2024-05-12T03:53:34.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:34 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/713476674' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fc534129-ef70-487d-b8d1-f98fc170c1d5"}]': finished 2024-05-12T03:53:34.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:34 smithi032 ceph-mon[40008]: osdmap e6: 1 total, 0 up, 1 in 2024-05-12T03:53:34.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:34 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-12T03:53:34.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:34 smithi033 ceph-mon[49525]: Reconfiguring prometheus.smithi032 (dependencies changed)... 2024-05-12T03:53:34.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:34 smithi033 ceph-mon[49525]: Reconfiguring daemon prometheus.smithi032 on smithi032 2024-05-12T03:53:34.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:34 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/713476674' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc534129-ef70-487d-b8d1-f98fc170c1d5"}]: dispatch 2024-05-12T03:53:34.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:34 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/713476674' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fc534129-ef70-487d-b8d1-f98fc170c1d5"}]': finished 2024-05-12T03:53:34.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:34 smithi033 ceph-mon[49525]: osdmap e6: 1 total, 0 up, 1 in 2024-05-12T03:53:34.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:34 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-12T03:53:35.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:35 smithi032 ceph-mon[40008]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:53:35.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:35 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/91191835' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-12T03:53:35.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:35 smithi033 ceph-mon[49525]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:53:35.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:35 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/91191835' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-12T03:53:37.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:37 smithi032 ceph-mon[40008]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:53:37.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:37 smithi033 ceph-mon[49525]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:53:39.093 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:39 smithi032 ceph-mon[40008]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:53:39.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:39 smithi033 ceph-mon[49525]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:53:40.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:40 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T03:53:40.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:40 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T03:53:41.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:41 smithi032 ceph-mon[40008]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:53:41.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:41 smithi033 ceph-mon[49525]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:53:43.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:43 smithi032 ceph-mon[40008]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:53:43.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:43 smithi033 ceph-mon[49525]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:53:44.972 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:44 smithi032 ceph-mon[40008]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:53:44.972 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:44 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:44.972 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:44 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:44.972 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:44 smithi032 ceph-mon[40008]: Reconfiguring crash.smithi033 (monmap changed)... 2024-05-12T03:53:44.972 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:44 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi033", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-12T03:53:44.972 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:44 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:53:44.972 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:44 smithi032 ceph-mon[40008]: Reconfiguring daemon crash.smithi033 on smithi033 2024-05-12T03:53:45.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:44 smithi033 ceph-mon[49525]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:53:45.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:44 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:45.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:44 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:45.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:44 smithi033 ceph-mon[49525]: Reconfiguring crash.smithi033 (monmap changed)... 2024-05-12T03:53:45.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:44 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi033", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-12T03:53:45.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:44 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:53:45.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:44 smithi033 ceph-mon[49525]: Reconfiguring daemon crash.smithi033 on smithi033 2024-05-12T03:53:46.105 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:45 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-12T03:53:46.105 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:45 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:53:46.106 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:45 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:46.106 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:45 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:46.106 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:45 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi033.pcvxwy", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-12T03:53:46.106 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:45 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-12T03:53:46.106 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:45 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:53:46.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:45 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-12T03:53:46.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:45 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:53:46.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:45 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:46.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:45 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:46.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:45 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi033.pcvxwy", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-12T03:53:46.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:45 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-12T03:53:46.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:45 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:53:47.125 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:46 smithi032 ceph-mon[40008]: Deploying daemon osd.0 on smithi032 2024-05-12T03:53:47.125 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:46 smithi032 ceph-mon[40008]: Reconfiguring mgr.smithi033.pcvxwy (monmap changed)... 2024-05-12T03:53:47.125 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:46 smithi032 ceph-mon[40008]: Reconfiguring daemon mgr.smithi033.pcvxwy on smithi033 2024-05-12T03:53:47.125 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:46 smithi032 ceph-mon[40008]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:53:47.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:46 smithi033 ceph-mon[49525]: Deploying daemon osd.0 on smithi032 2024-05-12T03:53:47.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:46 smithi033 ceph-mon[49525]: Reconfiguring mgr.smithi033.pcvxwy (monmap changed)... 2024-05-12T03:53:47.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:46 smithi033 ceph-mon[49525]: Reconfiguring daemon mgr.smithi033.pcvxwy on smithi033 2024-05-12T03:53:47.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:46 smithi033 ceph-mon[49525]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:53:48.168 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:48 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:48.168 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:48 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:48.168 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:48 smithi032 ceph-mon[40008]: Reconfiguring mon.smithi033 (monmap changed)... 2024-05-12T03:53:48.168 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:48 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-12T03:53:48.168 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:48 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-12T03:53:48.169 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:48 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:53:48.169 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:48 smithi032 ceph-mon[40008]: Reconfiguring daemon mon.smithi033 on smithi033 2024-05-12T03:53:48.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:48 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:48.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:48 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:48.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:48 smithi033 ceph-mon[49525]: Reconfiguring mon.smithi033 (monmap changed)... 2024-05-12T03:53:48.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:48 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-12T03:53:48.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:48 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-12T03:53:48.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:48 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:53:48.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:48 smithi033 ceph-mon[49525]: Reconfiguring daemon mon.smithi033 on smithi033 2024-05-12T03:53:49.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:49 smithi033 ceph-mon[49525]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:53:49.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:49 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:49.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:49 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:49.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:49 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-12T03:53:49.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:49 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi032.front.sepia.ceph.com:9093"}]: dispatch 2024-05-12T03:53:49.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:49 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:49.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:49 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-12T03:53:49.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:49 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi032.front.sepia.ceph.com:3000"}]: dispatch 2024-05-12T03:53:49.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:49 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:49.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:49 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-12T03:53:49.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:49 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi032.front.sepia.ceph.com:9095"}]: dispatch 2024-05-12T03:53:49.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:49 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:49.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:49 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T03:53:49.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:49 smithi032 ceph-mon[40008]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:53:49.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:49 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:49.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:49 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:49.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:49 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-12T03:53:49.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:49 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi032.front.sepia.ceph.com:9093"}]: dispatch 2024-05-12T03:53:49.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:49 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:49.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:49 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-12T03:53:49.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:49 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi032.front.sepia.ceph.com:3000"}]: dispatch 2024-05-12T03:53:49.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:49 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:49.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:49 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-12T03:53:49.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:49 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi032.front.sepia.ceph.com:9095"}]: dispatch 2024-05-12T03:53:49.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:49 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:49.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:49 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T03:53:50.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:50 smithi032 ceph-mon[40008]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-12T03:53:50.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:50 smithi032 ceph-mon[40008]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi032.front.sepia.ceph.com:9093"}]: dispatch 2024-05-12T03:53:50.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:50 smithi032 ceph-mon[40008]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-12T03:53:50.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:50 smithi032 ceph-mon[40008]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi032.front.sepia.ceph.com:3000"}]: dispatch 2024-05-12T03:53:50.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:50 smithi032 ceph-mon[40008]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-12T03:53:50.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:50 smithi032 ceph-mon[40008]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi032.front.sepia.ceph.com:9095"}]: dispatch 2024-05-12T03:53:50.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:50 smithi033 ceph-mon[49525]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-12T03:53:50.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:50 smithi033 ceph-mon[49525]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi032.front.sepia.ceph.com:9093"}]: dispatch 2024-05-12T03:53:50.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:50 smithi033 ceph-mon[49525]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-12T03:53:50.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:50 smithi033 ceph-mon[49525]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi032.front.sepia.ceph.com:3000"}]: dispatch 2024-05-12T03:53:50.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:50 smithi033 ceph-mon[49525]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-12T03:53:50.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:50 smithi033 ceph-mon[49525]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi032.front.sepia.ceph.com:9095"}]: dispatch 2024-05-12T03:53:51.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:51 smithi032 ceph-mon[40008]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:53:51.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:51 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:51.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:51 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:51.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:51 smithi033 ceph-mon[49525]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:53:51.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:51 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:51.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:51 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:52.528 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:52 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:52.528 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:52 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:52.528 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:52 smithi032 ceph-mon[40008]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:53:52.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:52 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:52.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:52 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:52.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:52 smithi033 ceph-mon[49525]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:53:55.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:54 smithi032 ceph-mon[40008]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:53:55.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:54 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T03:53:55.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:54 smithi033 ceph-mon[49525]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:53:55.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:54 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T03:53:55.500 INFO:teuthology.orchestra.run.smithi032.stdout:Created osd(s) 0 on host 'smithi032' 2024-05-12T03:53:55.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:55.499+0000 7ff7527fc700 1 -- 172.21.15.32:0/1747442396 <== mgr.14208 v2:172.21.15.32:6800/4089797100 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff760101320 con 0x7ff74c069210 2024-05-12T03:53:55.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:55.501+0000 7ff767daa700 1 -- 172.21.15.32:0/1747442396 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7ff74c069210 msgr2=0x7ff74c06b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:53:55.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:55.501+0000 7ff767daa700 1 --2- 172.21.15.32:0/1747442396 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7ff74c069210 0x7ff74c06b6c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff754009bf0 tx=0x7ff7540091a0 comp rx=0 tx=0).stop 2024-05-12T03:53:55.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:55.501+0000 7ff767daa700 1 -- 172.21.15.32:0/1747442396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7600fbcb0 msgr2=0x7ff7601105d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:53:55.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:55.501+0000 7ff767daa700 1 --2- 172.21.15.32:0/1747442396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7600fbcb0 0x7ff7601105d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff75c009380 tx=0x7ff75c009e30 comp rx=0 tx=0).stop 2024-05-12T03:53:55.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:55.501+0000 7ff767daa700 1 -- 172.21.15.32:0/1747442396 shutdown_connections 2024-05-12T03:53:55.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:55.501+0000 7ff767daa700 1 --2- 172.21.15.32:0/1747442396 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7ff74c069210 0x7ff74c06b6c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:53:55.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:55.501+0000 7ff767daa700 1 --2- 172.21.15.32:0/1747442396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7600fbcb0 0x7ff7601105d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:53:55.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:55.501+0000 7ff767daa700 1 --2- 172.21.15.32:0/1747442396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7600fcf60 0x7ff760112b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:53:55.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:55.501+0000 7ff767daa700 1 -- 172.21.15.32:0/1747442396 >> 172.21.15.32:0/1747442396 conn(0x7ff7600f7430 msgr2=0x7ff7600f8b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:53:55.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:55.501+0000 7ff767daa700 1 -- 172.21.15.32:0/1747442396 shutdown_connections 2024-05-12T03:53:55.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:53:55.501+0000 7ff767daa700 1 -- 172.21.15.32:0/1747442396 wait complete. 2024-05-12T03:53:56.131 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:55 smithi032 ceph-mon[40008]: from='osd.0 [v2:172.21.15.32:6802/3497755677,v1:172.21.15.32:6803/3497755677]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-12T03:53:56.131 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:55 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:56.132 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:55 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:56.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:55 smithi033 ceph-mon[49525]: from='osd.0 [v2:172.21.15.32:6802/3497755677,v1:172.21.15.32:6803/3497755677]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-12T03:53:56.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:55 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:56.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:55 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:56.873 DEBUG:teuthology.orchestra.run.smithi032:osd.0> sudo journalctl -f -n 0 -u ceph-a3e178f8-1012-11ef-bc98-c7b262605968@osd.0.service 2024-05-12T03:53:56.875 INFO:tasks.cephadm:Deploying osd.1 on smithi032 with /dev/vg_nvme/lv_3... 2024-05-12T03:53:56.875 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-12T03:53:56.906 INFO:journalctl@ceph.osd.0.smithi032.stdout:-- Logs begin at Sun 2024-05-12 03:39:04 UTC. -- 2024-05-12T03:53:57.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:56 smithi033 ceph-mon[49525]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:53:57.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:56 smithi033 ceph-mon[49525]: from='osd.0 [v2:172.21.15.32:6802/3497755677,v1:172.21.15.32:6803/3497755677]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-12T03:53:57.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:56 smithi033 ceph-mon[49525]: osdmap e7: 1 total, 0 up, 1 in 2024-05-12T03:53:57.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:56 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-12T03:53:57.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:56 smithi033 ceph-mon[49525]: from='osd.0 [v2:172.21.15.32:6802/3497755677,v1:172.21.15.32:6803/3497755677]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2024-05-12T03:53:57.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:56 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:57.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:56 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:57.191 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T03:53:57.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:56 smithi032 ceph-mon[40008]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:53:57.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:56 smithi032 ceph-mon[40008]: from='osd.0 [v2:172.21.15.32:6802/3497755677,v1:172.21.15.32:6803/3497755677]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-12T03:53:57.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:56 smithi032 ceph-mon[40008]: osdmap e7: 1 total, 0 up, 1 in 2024-05-12T03:53:57.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:56 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-12T03:53:57.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:56 smithi032 ceph-mon[40008]: from='osd.0 [v2:172.21.15.32:6802/3497755677,v1:172.21.15.32:6803/3497755677]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2024-05-12T03:53:57.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:56 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:57.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:56 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:53:58.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:57 smithi033 ceph-mon[49525]: from='osd.0 [v2:172.21.15.32:6802/3497755677,v1:172.21.15.32:6803/3497755677]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2024-05-12T03:53:58.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:57 smithi033 ceph-mon[49525]: osdmap e8: 1 total, 0 up, 1 in 2024-05-12T03:53:58.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:57 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-12T03:53:58.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:57 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-12T03:53:58.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:57 smithi032 ceph-mon[40008]: from='osd.0 [v2:172.21.15.32:6802/3497755677,v1:172.21.15.32:6803/3497755677]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2024-05-12T03:53:58.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:57 smithi032 ceph-mon[40008]: osdmap e8: 1 total, 0 up, 1 in 2024-05-12T03:53:58.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:57 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-12T03:53:58.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:57 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-12T03:53:59.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:58 smithi033 ceph-mon[49525]: purged_snaps scrub starts 2024-05-12T03:53:59.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:58 smithi033 ceph-mon[49525]: purged_snaps scrub ok 2024-05-12T03:53:59.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:58 smithi033 ceph-mon[49525]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:53:59.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:58 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-12T03:53:59.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:58 smithi033 ceph-mon[49525]: osd.0 [v2:172.21.15.32:6802/3497755677,v1:172.21.15.32:6803/3497755677] boot 2024-05-12T03:53:59.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:58 smithi033 ceph-mon[49525]: osdmap e9: 1 total, 1 up, 1 in 2024-05-12T03:53:59.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:58 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-12T03:53:59.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:58 smithi032 ceph-mon[40008]: purged_snaps scrub starts 2024-05-12T03:53:59.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:58 smithi032 ceph-mon[40008]: purged_snaps scrub ok 2024-05-12T03:53:59.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:58 smithi032 ceph-mon[40008]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-12T03:53:59.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:58 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-12T03:53:59.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:58 smithi032 ceph-mon[40008]: osd.0 [v2:172.21.15.32:6802/3497755677,v1:172.21.15.32:6803/3497755677] boot 2024-05-12T03:53:59.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:58 smithi032 ceph-mon[40008]: osdmap e9: 1 total, 1 up, 1 in 2024-05-12T03:53:59.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:58 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-12T03:54:00.039 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:53:59 smithi032 ceph-mon[40008]: osdmap e10: 1 total, 1 up, 1 in 2024-05-12T03:54:00.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:53:59 smithi033 ceph-mon[49525]: osdmap e10: 1 total, 1 up, 1 in 2024-05-12T03:54:00.345 INFO:teuthology.orchestra.run.smithi032.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-12T03:54:00.345 INFO:teuthology.orchestra.run.smithi032.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-12T03:54:00.345 INFO:teuthology.orchestra.run.smithi032.stderr: stderr: 10+0 records in 2024-05-12T03:54:00.345 INFO:teuthology.orchestra.run.smithi032.stderr:10+0 records out 2024-05-12T03:54:00.345 INFO:teuthology.orchestra.run.smithi032.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0078201 s, 1.3 GB/s 2024-05-12T03:54:00.345 INFO:teuthology.orchestra.run.smithi032.stderr:--> Zapping successful for: 2024-05-12T03:54:01.132 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:00 smithi032 ceph-mon[40008]: pgmap v38: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-12T03:54:01.155 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi032:vg_nvme/lv_3 2024-05-12T03:54:01.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:00 smithi033 ceph-mon[49525]: pgmap v38: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-12T03:54:02.055 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T03:54:03.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:03 smithi033 ceph-mon[49525]: pgmap v39: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-12T03:54:03.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:03 smithi033 ceph-mon[49525]: Detected new or changed devices on smithi032 2024-05-12T03:54:03.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:03 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:54:03.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:03 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:54:03.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:03 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-12T03:54:03.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:03 smithi033 ceph-mon[49525]: Adjusting osd_memory_target on smithi032 to 12902M 2024-05-12T03:54:03.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:03 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:54:03.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:03 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:54:03.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:03 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T03:54:03.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:03 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:54:03.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:03 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T03:54:03.225 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:03 smithi032 ceph-mon[40008]: pgmap v39: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-12T03:54:03.225 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:03 smithi032 ceph-mon[40008]: Detected new or changed devices on smithi032 2024-05-12T03:54:03.225 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:03 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:54:03.225 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:03 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:54:03.226 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:03 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-12T03:54:03.226 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:03 smithi032 ceph-mon[40008]: Adjusting osd_memory_target on smithi032 to 12902M 2024-05-12T03:54:03.226 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:03 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:54:03.226 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:03 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:54:03.226 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:03 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T03:54:03.226 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:03 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:54:03.226 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:03 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T03:54:03.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:03.714+0000 7fa35f621700 1 -- 172.21.15.32:0/54840276 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3580fcf10 msgr2=0x7fa3580fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:54:03.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:03.714+0000 7fa35f621700 1 --2- 172.21.15.32:0/54840276 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3580fcf10 0x7fa3580fd320 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa354004940 tx=0x7fa354009e30 comp rx=0 tx=0).stop 2024-05-12T03:54:03.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:03.714+0000 7fa35f621700 1 -- 172.21.15.32:0/54840276 shutdown_connections 2024-05-12T03:54:03.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:03.714+0000 7fa35f621700 1 --2- 172.21.15.32:0/54840276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3580fe080 0x7fa3580fe4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:54:03.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:03.714+0000 7fa35f621700 1 --2- 172.21.15.32:0/54840276 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3580fcf10 0x7fa3580fd320 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:54:03.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:03.714+0000 7fa35f621700 1 -- 172.21.15.32:0/54840276 >> 172.21.15.32:0/54840276 conn(0x7fa3580f8560 msgr2=0x7fa3580fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:54:03.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:03.714+0000 7fa35f621700 1 -- 172.21.15.32:0/54840276 shutdown_connections 2024-05-12T03:54:03.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:03.714+0000 7fa35f621700 1 -- 172.21.15.32:0/54840276 wait complete. 2024-05-12T03:54:03.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:03.714+0000 7fa35f621700 1 Processor -- start 2024-05-12T03:54:03.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:03.714+0000 7fa35f621700 1 -- start start 2024-05-12T03:54:03.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:03.714+0000 7fa35f621700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3580fcf10 0x7fa35810d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:54:03.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:03.714+0000 7fa35f621700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3580fe080 0x7fa35810de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:54:03.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:03.714+0000 7fa35f621700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa35810e4e0 con 0x7fa3580fcf10 2024-05-12T03:54:03.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:03.714+0000 7fa35f621700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa35810ecd0 con 0x7fa3580fe080 2024-05-12T03:54:03.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:03.715+0000 7fa35d3bd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3580fcf10 0x7fa35810d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:54:03.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:03.715+0000 7fa35d3bd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3580fcf10 0x7fa35810d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55178/0 (socket says 172.21.15.32:55178) 2024-05-12T03:54:03.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:03.715+0000 7fa35d3bd700 1 -- 172.21.15.32:0/3469896883 learned_addr learned my addr 172.21.15.32:0/3469896883 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:54:03.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:03.715+0000 7fa35cbbc700 1 --2- 172.21.15.32:0/3469896883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3580fe080 0x7fa35810de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:54:03.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:03.715+0000 7fa35d3bd700 1 -- 172.21.15.32:0/3469896883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3580fe080 msgr2=0x7fa35810de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:54:03.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:03.715+0000 7fa35d3bd700 1 --2- 172.21.15.32:0/3469896883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3580fe080 0x7fa35810de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:54:03.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:03.715+0000 7fa35d3bd700 1 -- 172.21.15.32:0/3469896883 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa354005040 con 0x7fa3580fcf10 2024-05-12T03:54:03.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:03.715+0000 7fa35d3bd700 1 --2- 172.21.15.32:0/3469896883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3580fcf10 0x7fa35810d910 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa35400b3b0 tx=0x7fa354009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:54:03.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:03.715+0000 7fa34dffb700 1 -- 172.21.15.32:0/3469896883 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa35400e040 con 0x7fa3580fcf10 2024-05-12T03:54:03.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:03.715+0000 7fa34dffb700 1 -- 172.21.15.32:0/3469896883 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa35400bd20 con 0x7fa3580fcf10 2024-05-12T03:54:03.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:03.715+0000 7fa35f621700 1 -- 172.21.15.32:0/3469896883 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa35810ef50 con 0x7fa3580fcf10 2024-05-12T03:54:03.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:03.715+0000 7fa34dffb700 1 -- 172.21.15.32:0/3469896883 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa35401c630 con 0x7fa3580fcf10 2024-05-12T03:54:03.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:03.715+0000 7fa35f621700 1 -- 172.21.15.32:0/3469896883 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa35810f510 con 0x7fa3580fcf10 2024-05-12T03:54:03.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:03.716+0000 7fa35f621700 1 -- 172.21.15.32:0/3469896883 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa35805f5c0 con 0x7fa3580fcf10 2024-05-12T03:54:03.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:03.717+0000 7fa34dffb700 1 -- 172.21.15.32:0/3469896883 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 86573+0+0 (secure 0 0 0) 0x7fa354013420 con 0x7fa3580fcf10 2024-05-12T03:54:03.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:03.717+0000 7fa34dffb700 1 --2- 172.21.15.32:0/3469896883 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fa344069210 0x7fa34406b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:54:03.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:03.717+0000 7fa35cbbc700 1 --2- 172.21.15.32:0/3469896883 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fa344069210 0x7fa34406b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:54:03.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:03.717+0000 7fa34dffb700 1 -- 172.21.15.32:0/3469896883 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1978+0+0 (secure 0 0 0) 0x7fa3540870a0 con 0x7fa3580fcf10 2024-05-12T03:54:03.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:03.718+0000 7fa35cbbc700 1 --2- 172.21.15.32:0/3469896883 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fa344069210 0x7fa34406b6c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa348006fd0 tx=0x7fa348008040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:54:03.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:03.722+0000 7fa34dffb700 1 -- 172.21.15.32:0/3469896883 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fa354057c60 con 0x7fa3580fcf10 2024-05-12T03:54:03.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:03.930+0000 7fa35f621700 1 -- 172.21.15.32:0/3469896883 --> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fa35810f7f0 con 0x7fa344069210 2024-05-12T03:54:04.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:04 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-12T03:54:04.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:04 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-12T03:54:04.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:04 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:54:04.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:04 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-12T03:54:04.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:04 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-12T03:54:04.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:04 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:54:05.134 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:05 smithi032 ceph-mon[40008]: pgmap v40: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-12T03:54:05.134 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:05 smithi032 ceph-mon[40008]: from='client.14260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T03:54:05.134 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:05 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:54:05.134 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:05 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:54:05.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:05 smithi033 ceph-mon[49525]: pgmap v40: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-12T03:54:05.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:05 smithi033 ceph-mon[49525]: from='client.14260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T03:54:05.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:05 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:54:05.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:05 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:54:07.039 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:07 smithi032 ceph-mon[40008]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-12T03:54:07.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:07 smithi033 ceph-mon[49525]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-12T03:54:08.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:08 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/18614233' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8c35b6a-5afc-447b-b318-d56097de28d8"}]: dispatch 2024-05-12T03:54:08.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:08 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/18614233' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f8c35b6a-5afc-447b-b318-d56097de28d8"}]': finished 2024-05-12T03:54:08.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:08 smithi032 ceph-mon[40008]: osdmap e11: 2 total, 1 up, 2 in 2024-05-12T03:54:08.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:08 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-12T03:54:08.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:08 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/18614233' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8c35b6a-5afc-447b-b318-d56097de28d8"}]: dispatch 2024-05-12T03:54:08.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:08 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/18614233' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f8c35b6a-5afc-447b-b318-d56097de28d8"}]': finished 2024-05-12T03:54:08.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:08 smithi033 ceph-mon[49525]: osdmap e11: 2 total, 1 up, 2 in 2024-05-12T03:54:08.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:08 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-12T03:54:09.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:09 smithi032 ceph-mon[40008]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-12T03:54:09.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:09 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/2622011517' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-12T03:54:09.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:09 smithi033 ceph-mon[49525]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-12T03:54:09.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:09 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/2622011517' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-12T03:54:10.413 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:10 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:54:10.414 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:10 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:54:10.414 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:10 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:54:10.414 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:10 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T03:54:10.414 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:10 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:54:10.414 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:10 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T03:54:10.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:10 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:54:10.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:10 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:54:10.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:10 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:54:10.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:10 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T03:54:10.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:10 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:54:10.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:10 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T03:54:11.407 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:11 smithi032 ceph-mon[40008]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-12T03:54:11.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:11 smithi033 ceph-mon[49525]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-12T03:54:13.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:13 smithi033 ceph-mon[49525]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-12T03:54:13.491 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:13 smithi032 ceph-mon[40008]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-12T03:54:15.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:15 smithi033 ceph-mon[49525]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-12T03:54:15.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:15 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-12T03:54:15.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:15 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:54:15.453 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:15 smithi032 ceph-mon[40008]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-12T03:54:15.453 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:15 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-12T03:54:15.453 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:15 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:54:16.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:16 smithi033 ceph-mon[49525]: Deploying daemon osd.1 on smithi032 2024-05-12T03:54:16.439 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:16 smithi032 ceph-mon[40008]: Deploying daemon osd.1 on smithi032 2024-05-12T03:54:17.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:17 smithi033 ceph-mon[49525]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-12T03:54:17.472 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:17 smithi032 ceph-mon[40008]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-12T03:54:18.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:18 smithi033 ceph-mon[49525]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-12T03:54:18.492 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:18 smithi032 ceph-mon[40008]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-12T03:54:20.962 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:20 smithi032 ceph-mon[40008]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-12T03:54:20.962 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:20 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:54:20.962 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:20 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:54:20.962 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:20 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T03:54:21.096 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:20 smithi033 ceph-mon[49525]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-12T03:54:21.096 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:20 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:54:21.096 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:20 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:54:21.096 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:20 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T03:54:23.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:22 smithi032 ceph-mon[40008]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-12T03:54:23.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:22 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:54:23.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:22 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:54:23.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:22 smithi033 ceph-mon[49525]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-12T03:54:23.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:22 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:54:23.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:22 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:54:24.540 INFO:teuthology.orchestra.run.smithi032.stdout:Created osd(s) 1 on host 'smithi032' 2024-05-12T03:54:24.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:24.539+0000 7fa34dffb700 1 -- 172.21.15.32:0/3469896883 <== mgr.14208 v2:172.21.15.32:6800/4089797100 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fa35810f7f0 con 0x7fa344069210 2024-05-12T03:54:24.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:24.542+0000 7fa35f621700 1 -- 172.21.15.32:0/3469896883 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fa344069210 msgr2=0x7fa34406b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:54:24.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:24.542+0000 7fa35f621700 1 --2- 172.21.15.32:0/3469896883 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fa344069210 0x7fa34406b6c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa348006fd0 tx=0x7fa348008040 comp rx=0 tx=0).stop 2024-05-12T03:54:24.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:24.542+0000 7fa35f621700 1 -- 172.21.15.32:0/3469896883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3580fcf10 msgr2=0x7fa35810d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:54:24.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:24.542+0000 7fa35f621700 1 --2- 172.21.15.32:0/3469896883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3580fcf10 0x7fa35810d910 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa35400b3b0 tx=0x7fa354009e30 comp rx=0 tx=0).stop 2024-05-12T03:54:24.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:24.542+0000 7fa35f621700 1 -- 172.21.15.32:0/3469896883 shutdown_connections 2024-05-12T03:54:24.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:24.542+0000 7fa35f621700 1 --2- 172.21.15.32:0/3469896883 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fa344069210 0x7fa34406b6c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:54:24.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:24.542+0000 7fa35f621700 1 --2- 172.21.15.32:0/3469896883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3580fcf10 0x7fa35810d910 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:54:24.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:24.542+0000 7fa35f621700 1 --2- 172.21.15.32:0/3469896883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3580fe080 0x7fa35810de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:54:24.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:24.542+0000 7fa35f621700 1 -- 172.21.15.32:0/3469896883 >> 172.21.15.32:0/3469896883 conn(0x7fa3580f8560 msgr2=0x7fa3580f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:54:24.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:24.542+0000 7fa35f621700 1 -- 172.21.15.32:0/3469896883 shutdown_connections 2024-05-12T03:54:24.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:24.542+0000 7fa35f621700 1 -- 172.21.15.32:0/3469896883 wait complete. 2024-05-12T03:54:25.158 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:24 smithi032 ceph-mon[40008]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-12T03:54:25.158 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:24 smithi032 ceph-mon[40008]: from='osd.1 [v2:172.21.15.32:6810/3065624488,v1:172.21.15.32:6811/3065624488]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-12T03:54:25.158 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:24 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:54:25.158 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:24 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:54:25.158 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:24 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T03:54:25.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:24 smithi033 ceph-mon[49525]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-12T03:54:25.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:24 smithi033 ceph-mon[49525]: from='osd.1 [v2:172.21.15.32:6810/3065624488,v1:172.21.15.32:6811/3065624488]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-12T03:54:25.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:24 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:54:25.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:24 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:54:25.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:24 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T03:54:25.815 DEBUG:teuthology.orchestra.run.smithi032:osd.1> sudo journalctl -f -n 0 -u ceph-a3e178f8-1012-11ef-bc98-c7b262605968@osd.1.service 2024-05-12T03:54:25.817 INFO:tasks.cephadm:Deploying osd.2 on smithi032 with /dev/vg_nvme/lv_2... 2024-05-12T03:54:25.817 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-12T03:54:25.852 INFO:journalctl@ceph.osd.1.smithi032.stdout:-- Logs begin at Sun 2024-05-12 03:39:04 UTC. -- 2024-05-12T03:54:26.040 INFO:journalctl@ceph.osd.1.smithi032.stdout:May 12 03:54:25 smithi032 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-1[73194]: 2024-05-12T03:54:25.859+0000 7fb3f41fc700 -1 osd.1 0 waiting for initial osdmap 2024-05-12T03:54:26.040 INFO:journalctl@ceph.osd.1.smithi032.stdout:May 12 03:54:25 smithi032 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-1[73194]: 2024-05-12T03:54:25.866+0000 7fb3f07fa700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-12T03:54:26.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:25 smithi032 ceph-mon[40008]: from='osd.1 [v2:172.21.15.32:6810/3065624488,v1:172.21.15.32:6811/3065624488]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-12T03:54:26.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:25 smithi032 ceph-mon[40008]: osdmap e12: 2 total, 1 up, 2 in 2024-05-12T03:54:26.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:25 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-12T03:54:26.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:25 smithi032 ceph-mon[40008]: from='osd.1 [v2:172.21.15.32:6810/3065624488,v1:172.21.15.32:6811/3065624488]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2024-05-12T03:54:26.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:25 smithi033 ceph-mon[49525]: from='osd.1 [v2:172.21.15.32:6810/3065624488,v1:172.21.15.32:6811/3065624488]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-12T03:54:26.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:25 smithi033 ceph-mon[49525]: osdmap e12: 2 total, 1 up, 2 in 2024-05-12T03:54:26.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:25 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-12T03:54:26.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:25 smithi033 ceph-mon[49525]: from='osd.1 [v2:172.21.15.32:6810/3065624488,v1:172.21.15.32:6811/3065624488]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2024-05-12T03:54:26.246 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T03:54:27.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:26 smithi033 ceph-mon[49525]: pgmap v53: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-12T03:54:27.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:26 smithi033 ceph-mon[49525]: from='osd.1 [v2:172.21.15.32:6810/3065624488,v1:172.21.15.32:6811/3065624488]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2024-05-12T03:54:27.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:26 smithi033 ceph-mon[49525]: osdmap e13: 2 total, 1 up, 2 in 2024-05-12T03:54:27.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:26 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-12T03:54:27.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:26 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-12T03:54:27.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:26 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-12T03:54:27.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:26 smithi032 ceph-mon[40008]: pgmap v53: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-12T03:54:27.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:26 smithi032 ceph-mon[40008]: from='osd.1 [v2:172.21.15.32:6810/3065624488,v1:172.21.15.32:6811/3065624488]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2024-05-12T03:54:27.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:26 smithi032 ceph-mon[40008]: osdmap e13: 2 total, 1 up, 2 in 2024-05-12T03:54:27.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:26 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-12T03:54:27.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:26 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-12T03:54:27.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:26 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-12T03:54:28.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:27 smithi032 ceph-mon[40008]: purged_snaps scrub starts 2024-05-12T03:54:28.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:27 smithi032 ceph-mon[40008]: purged_snaps scrub ok 2024-05-12T03:54:28.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:27 smithi032 ceph-mon[40008]: osd.1 [v2:172.21.15.32:6810/3065624488,v1:172.21.15.32:6811/3065624488] boot 2024-05-12T03:54:28.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:27 smithi032 ceph-mon[40008]: osdmap e14: 2 total, 2 up, 2 in 2024-05-12T03:54:28.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:27 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-12T03:54:28.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:27 smithi033 ceph-mon[49525]: purged_snaps scrub starts 2024-05-12T03:54:28.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:27 smithi033 ceph-mon[49525]: purged_snaps scrub ok 2024-05-12T03:54:28.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:27 smithi033 ceph-mon[49525]: osd.1 [v2:172.21.15.32:6810/3065624488,v1:172.21.15.32:6811/3065624488] boot 2024-05-12T03:54:28.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:27 smithi033 ceph-mon[49525]: osdmap e14: 2 total, 2 up, 2 in 2024-05-12T03:54:28.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:27 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-12T03:54:29.044 INFO:teuthology.orchestra.run.smithi032.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-12T03:54:29.044 INFO:teuthology.orchestra.run.smithi032.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-12T03:54:29.044 INFO:teuthology.orchestra.run.smithi032.stderr: stderr: 10+0 records in 2024-05-12T03:54:29.044 INFO:teuthology.orchestra.run.smithi032.stderr:10+0 records out 2024-05-12T03:54:29.044 INFO:teuthology.orchestra.run.smithi032.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00827392 s, 1.3 GB/s 2024-05-12T03:54:29.044 INFO:teuthology.orchestra.run.smithi032.stderr:--> Zapping successful for: 2024-05-12T03:54:29.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:28 smithi033 ceph-mon[49525]: pgmap v56: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-12T03:54:29.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:28 smithi033 ceph-mon[49525]: osdmap e15: 2 total, 2 up, 2 in 2024-05-12T03:54:29.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:28 smithi032 ceph-mon[40008]: pgmap v56: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-05-12T03:54:29.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:28 smithi032 ceph-mon[40008]: osdmap e15: 2 total, 2 up, 2 in 2024-05-12T03:54:29.731 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi032:vg_nvme/lv_2 2024-05-12T03:54:30.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:30 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:54:30.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:30 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:54:30.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:30 smithi032 ceph-mon[40008]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-12T03:54:30.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:30 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:54:30.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:30 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:54:30.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:30 smithi033 ceph-mon[49525]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-12T03:54:30.996 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T03:54:33.107 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:32 smithi032 ceph-mon[40008]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-12T03:54:33.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:32 smithi033 ceph-mon[49525]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-12T03:54:33.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:33.336+0000 7f520fa5e700 1 -- 172.21.15.32:0/2591640089 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52080fce80 msgr2=0x7f52080fd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:54:33.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:33.336+0000 7f520fa5e700 1 --2- 172.21.15.32:0/2591640089 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52080fce80 0x7f52080fd290 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5204004940 tx=0x7f5204009e30 comp rx=0 tx=0).stop 2024-05-12T03:54:33.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:33.336+0000 7f520fa5e700 1 -- 172.21.15.32:0/2591640089 shutdown_connections 2024-05-12T03:54:33.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:33.336+0000 7f520fa5e700 1 --2- 172.21.15.32:0/2591640089 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52080fdff0 0x7f52080fe460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:54:33.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:33.336+0000 7f520fa5e700 1 --2- 172.21.15.32:0/2591640089 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52080fce80 0x7f52080fd290 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:54:33.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:33.336+0000 7f520fa5e700 1 -- 172.21.15.32:0/2591640089 >> 172.21.15.32:0/2591640089 conn(0x7f52080f8520 msgr2=0x7f52080fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:54:33.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:33.336+0000 7f520fa5e700 1 -- 172.21.15.32:0/2591640089 shutdown_connections 2024-05-12T03:54:33.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:33.337+0000 7f520fa5e700 1 -- 172.21.15.32:0/2591640089 wait complete. 2024-05-12T03:54:33.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:33.337+0000 7f520fa5e700 1 Processor -- start 2024-05-12T03:54:33.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:33.337+0000 7f520fa5e700 1 -- start start 2024-05-12T03:54:33.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:33.337+0000 7f520fa5e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52080fce80 0x7f520810d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:54:33.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:33.337+0000 7f520fa5e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52080fdff0 0x7f520810de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:54:33.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:33.337+0000 7f520fa5e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f520810e520 con 0x7f52080fdff0 2024-05-12T03:54:33.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:33.337+0000 7f520fa5e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f520810ecc0 con 0x7f52080fce80 2024-05-12T03:54:33.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:33.337+0000 7f520cff9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52080fdff0 0x7f520810de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:54:33.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:33.337+0000 7f520cff9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52080fdff0 0x7f520810de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47782/0 (socket says 172.21.15.32:47782) 2024-05-12T03:54:33.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:33.337+0000 7f520cff9700 1 -- 172.21.15.32:0/4093869129 learned_addr learned my addr 172.21.15.32:0/4093869129 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:54:33.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:33.338+0000 7f520cff9700 1 -- 172.21.15.32:0/4093869129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52080fce80 msgr2=0x7f520810d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:54:33.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:33.338+0000 7f520d7fa700 1 --2- 172.21.15.32:0/4093869129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52080fce80 0x7f520810d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:54:33.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:33.338+0000 7f520cff9700 1 --2- 172.21.15.32:0/4093869129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52080fce80 0x7f520810d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:54:33.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:33.338+0000 7f520cff9700 1 -- 172.21.15.32:0/4093869129 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5204005040 con 0x7f52080fdff0 2024-05-12T03:54:33.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:33.338+0000 7f520cff9700 1 --2- 172.21.15.32:0/4093869129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52080fdff0 0x7f520810de90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f51fc00b790 tx=0x7f51fc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:54:33.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:33.338+0000 7f51f9ffb700 1 -- 172.21.15.32:0/4093869129 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51fc00f770 con 0x7f52080fdff0 2024-05-12T03:54:33.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:33.338+0000 7f51f9ffb700 1 -- 172.21.15.32:0/4093869129 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f51fc00fd80 con 0x7f52080fdff0 2024-05-12T03:54:33.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:33.338+0000 7f520fa5e700 1 -- 172.21.15.32:0/4093869129 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f520810efa0 con 0x7f52080fdff0 2024-05-12T03:54:33.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:33.338+0000 7f51f9ffb700 1 -- 172.21.15.32:0/4093869129 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51fc00d310 con 0x7f52080fdff0 2024-05-12T03:54:33.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:33.338+0000 7f520fa5e700 1 -- 172.21.15.32:0/4093869129 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f520810f560 con 0x7f52080fdff0 2024-05-12T03:54:33.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:33.339+0000 7f520fa5e700 1 -- 172.21.15.32:0/4093869129 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f520805f5c0 con 0x7f52080fdff0 2024-05-12T03:54:33.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:33.339+0000 7f51f9ffb700 1 -- 172.21.15.32:0/4093869129 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 86573+0+0 (secure 0 0 0) 0x7f51fc010420 con 0x7f52080fdff0 2024-05-12T03:54:33.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:33.340+0000 7f51f9ffb700 1 --2- 172.21.15.32:0/4093869129 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f51f4069210 0x7f51f406b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:54:33.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:33.340+0000 7f51f9ffb700 1 -- 172.21.15.32:0/4093869129 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2410+0+0 (secure 0 0 0) 0x7f51fc086330 con 0x7f52080fdff0 2024-05-12T03:54:33.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:33.340+0000 7f520d7fa700 1 --2- 172.21.15.32:0/4093869129 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f51f4069210 0x7f51f406b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:54:33.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:33.340+0000 7f520d7fa700 1 --2- 172.21.15.32:0/4093869129 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f51f4069210 0x7f51f406b6c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5204004f90 tx=0x7f5204009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:54:33.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:33.344+0000 7f51f9ffb700 1 -- 172.21.15.32:0/4093869129 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f51fc056e60 con 0x7f52080fdff0 2024-05-12T03:54:33.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:33.549+0000 7f520fa5e700 1 -- 172.21.15.32:0/4093869129 --> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f520810f8a0 con 0x7f51f4069210 2024-05-12T03:54:34.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:33 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-12T03:54:34.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:33 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-12T03:54:34.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:33 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:54:34.164 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:33 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-12T03:54:34.164 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:33 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-12T03:54:34.164 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:33 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:54:35.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:34 smithi033 ceph-mon[49525]: from='client.14278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T03:54:35.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:34 smithi033 ceph-mon[49525]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-12T03:54:35.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:34 smithi033 ceph-mon[49525]: Detected new or changed devices on smithi032 2024-05-12T03:54:35.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:34 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:54:35.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:34 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:54:35.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:34 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-12T03:54:35.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:34 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-12T03:54:35.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:34 smithi033 ceph-mon[49525]: Adjusting osd_memory_target on smithi032 to 6451M 2024-05-12T03:54:35.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:34 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:54:35.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:34 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:54:35.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:34 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T03:54:35.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:34 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:54:35.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:34 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T03:54:35.217 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:34 smithi032 ceph-mon[40008]: from='client.14278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T03:54:35.218 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:34 smithi032 ceph-mon[40008]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-12T03:54:35.218 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:34 smithi032 ceph-mon[40008]: Detected new or changed devices on smithi032 2024-05-12T03:54:35.218 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:34 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:54:35.218 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:34 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:54:35.218 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:34 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-12T03:54:35.218 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:34 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-12T03:54:35.218 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:34 smithi032 ceph-mon[40008]: Adjusting osd_memory_target on smithi032 to 6451M 2024-05-12T03:54:35.218 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:34 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:54:35.218 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:34 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:54:35.218 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:34 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T03:54:35.218 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:34 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:54:35.218 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:34 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T03:54:37.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:36 smithi033 ceph-mon[49525]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-12T03:54:37.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:36 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/1733315222' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ed4b55d-cceb-494b-bdff-fef99d6c8cf9"}]: dispatch 2024-05-12T03:54:37.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:36 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/1733315222' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4ed4b55d-cceb-494b-bdff-fef99d6c8cf9"}]': finished 2024-05-12T03:54:37.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:36 smithi033 ceph-mon[49525]: osdmap e16: 3 total, 2 up, 3 in 2024-05-12T03:54:37.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:36 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-12T03:54:37.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:36 smithi032 ceph-mon[40008]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-12T03:54:37.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:36 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/1733315222' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ed4b55d-cceb-494b-bdff-fef99d6c8cf9"}]: dispatch 2024-05-12T03:54:37.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:36 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/1733315222' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4ed4b55d-cceb-494b-bdff-fef99d6c8cf9"}]': finished 2024-05-12T03:54:37.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:36 smithi032 ceph-mon[40008]: osdmap e16: 3 total, 2 up, 3 in 2024-05-12T03:54:37.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:36 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-12T03:54:38.039 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:37 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/3509554292' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-12T03:54:38.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:37 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/3509554292' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-12T03:54:39.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:38 smithi033 ceph-mon[49525]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-12T03:54:39.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:38 smithi032 ceph-mon[40008]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-12T03:54:40.761 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:40 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:54:40.761 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:40 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:54:40.761 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:40 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:54:40.761 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:40 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T03:54:40.762 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:40 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:54:40.762 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:40 smithi032 ceph-mon[40008]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-12T03:54:40.762 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:40 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T03:54:41.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:40 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:54:41.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:40 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:54:41.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:40 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:54:41.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:40 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T03:54:41.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:40 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:54:41.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:40 smithi033 ceph-mon[49525]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-12T03:54:41.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:40 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T03:54:43.023 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:42 smithi032 ceph-mon[40008]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-12T03:54:43.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:42 smithi033 ceph-mon[49525]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-12T03:54:45.039 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:44 smithi032 ceph-mon[40008]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-12T03:54:45.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:44 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-12T03:54:45.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:44 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:54:45.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:44 smithi032 ceph-mon[40008]: Deploying daemon osd.2 on smithi032 2024-05-12T03:54:45.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:44 smithi033 ceph-mon[49525]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-12T03:54:45.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:44 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-12T03:54:45.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:44 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:54:45.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:44 smithi033 ceph-mon[49525]: Deploying daemon osd.2 on smithi032 2024-05-12T03:54:47.042 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:46 smithi032 ceph-mon[40008]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-12T03:54:47.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:46 smithi033 ceph-mon[49525]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-12T03:54:49.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:48 smithi032 ceph-mon[40008]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-12T03:54:49.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:48 smithi033 ceph-mon[49525]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-12T03:54:51.141 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:50 smithi032 ceph-mon[40008]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-12T03:54:51.141 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:50 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:54:51.142 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:50 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:54:51.142 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:50 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T03:54:51.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:50 smithi033 ceph-mon[49525]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-12T03:54:51.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:50 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:54:51.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:50 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:54:51.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:50 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T03:54:53.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:52 smithi033 ceph-mon[49525]: pgmap v70: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-12T03:54:53.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:52 smithi032 ceph-mon[40008]: pgmap v70: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-12T03:54:54.211 INFO:teuthology.orchestra.run.smithi032.stdout:Created osd(s) 2 on host 'smithi032' 2024-05-12T03:54:54.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:54.209+0000 7f51f9ffb700 1 -- 172.21.15.32:0/4093869129 <== mgr.14208 v2:172.21.15.32:6800/4089797100 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f520810f8a0 con 0x7f51f4069210 2024-05-12T03:54:54.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:54.212+0000 7f520fa5e700 1 -- 172.21.15.32:0/4093869129 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f51f4069210 msgr2=0x7f51f406b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:54:54.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:54.212+0000 7f520fa5e700 1 --2- 172.21.15.32:0/4093869129 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f51f4069210 0x7f51f406b6c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5204004f90 tx=0x7f5204009e30 comp rx=0 tx=0).stop 2024-05-12T03:54:54.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:54.212+0000 7f520fa5e700 1 -- 172.21.15.32:0/4093869129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52080fdff0 msgr2=0x7f520810de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:54:54.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:54.212+0000 7f520fa5e700 1 --2- 172.21.15.32:0/4093869129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52080fdff0 0x7f520810de90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f51fc00b790 tx=0x7f51fc009250 comp rx=0 tx=0).stop 2024-05-12T03:54:54.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:54.212+0000 7f520fa5e700 1 -- 172.21.15.32:0/4093869129 shutdown_connections 2024-05-12T03:54:54.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:54.212+0000 7f520fa5e700 1 --2- 172.21.15.32:0/4093869129 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f51f4069210 0x7f51f406b6c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:54:54.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:54.212+0000 7f520fa5e700 1 --2- 172.21.15.32:0/4093869129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52080fce80 0x7f520810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:54:54.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:54.212+0000 7f520fa5e700 1 --2- 172.21.15.32:0/4093869129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52080fdff0 0x7f520810de90 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:54:54.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:54.212+0000 7f520fa5e700 1 -- 172.21.15.32:0/4093869129 >> 172.21.15.32:0/4093869129 conn(0x7f52080f8520 msgr2=0x7f52080f9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:54:54.213 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:54.212+0000 7f520fa5e700 1 -- 172.21.15.32:0/4093869129 shutdown_connections 2024-05-12T03:54:54.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:54:54.212+0000 7f520fa5e700 1 -- 172.21.15.32:0/4093869129 wait complete. 2024-05-12T03:54:55.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:54 smithi033 ceph-mon[49525]: pgmap v71: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-12T03:54:55.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:54 smithi033 ceph-mon[49525]: from='osd.2 [v2:172.21.15.32:6818/3261757113,v1:172.21.15.32:6819/3261757113]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-12T03:54:55.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:54 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:54:55.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:54 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:54:55.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:54 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T03:54:55.261 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:54 smithi032 ceph-mon[40008]: pgmap v71: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-12T03:54:55.261 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:54 smithi032 ceph-mon[40008]: from='osd.2 [v2:172.21.15.32:6818/3261757113,v1:172.21.15.32:6819/3261757113]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-12T03:54:55.261 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:54 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:54:55.261 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:54 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:54:55.261 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:54 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T03:54:55.285 DEBUG:teuthology.orchestra.run.smithi032:osd.2> sudo journalctl -f -n 0 -u ceph-a3e178f8-1012-11ef-bc98-c7b262605968@osd.2.service 2024-05-12T03:54:55.287 INFO:tasks.cephadm:Deploying osd.3 on smithi033 with /dev/vg_nvme/lv_4... 2024-05-12T03:54:55.288 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-12T03:54:55.320 INFO:journalctl@ceph.osd.2.smithi032.stdout:-- Logs begin at Sun 2024-05-12 03:39:04 UTC. -- 2024-05-12T03:54:55.569 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi033/config 2024-05-12T03:54:56.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:55 smithi033 ceph-mon[49525]: from='osd.2 [v2:172.21.15.32:6818/3261757113,v1:172.21.15.32:6819/3261757113]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-12T03:54:56.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:55 smithi033 ceph-mon[49525]: osdmap e17: 3 total, 2 up, 3 in 2024-05-12T03:54:56.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:55 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-12T03:54:56.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:55 smithi033 ceph-mon[49525]: from='osd.2 [v2:172.21.15.32:6818/3261757113,v1:172.21.15.32:6819/3261757113]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2024-05-12T03:54:56.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:55 smithi032 ceph-mon[40008]: from='osd.2 [v2:172.21.15.32:6818/3261757113,v1:172.21.15.32:6819/3261757113]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-12T03:54:56.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:55 smithi032 ceph-mon[40008]: osdmap e17: 3 total, 2 up, 3 in 2024-05-12T03:54:56.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:55 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-12T03:54:56.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:55 smithi032 ceph-mon[40008]: from='osd.2 [v2:172.21.15.32:6818/3261757113,v1:172.21.15.32:6819/3261757113]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2024-05-12T03:54:56.290 INFO:journalctl@ceph.osd.2.smithi032.stdout:May 12 03:54:55 smithi032 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-2[82340]: 2024-05-12T03:54:55.873+0000 7fc35a37f700 -1 osd.2 0 waiting for initial osdmap 2024-05-12T03:54:56.290 INFO:journalctl@ceph.osd.2.smithi032.stdout:May 12 03:54:55 smithi032 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-2[82340]: 2024-05-12T03:54:55.878+0000 7fc35517a700 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-12T03:54:56.904 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:56 smithi033 ceph-mon[49525]: pgmap v73: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-12T03:54:56.904 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:56 smithi033 ceph-mon[49525]: from='osd.2 [v2:172.21.15.32:6818/3261757113,v1:172.21.15.32:6819/3261757113]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2024-05-12T03:54:56.904 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:56 smithi033 ceph-mon[49525]: osdmap e18: 3 total, 2 up, 3 in 2024-05-12T03:54:56.904 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:56 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-12T03:54:56.905 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:56 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-12T03:54:57.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:56 smithi032 ceph-mon[40008]: pgmap v73: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-12T03:54:57.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:56 smithi032 ceph-mon[40008]: from='osd.2 [v2:172.21.15.32:6818/3261757113,v1:172.21.15.32:6819/3261757113]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2024-05-12T03:54:57.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:56 smithi032 ceph-mon[40008]: osdmap e18: 3 total, 2 up, 3 in 2024-05-12T03:54:57.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:56 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-12T03:54:57.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:56 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-12T03:54:57.583 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-12T03:54:57.583 INFO:teuthology.orchestra.run.smithi033.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-12T03:54:57.583 INFO:teuthology.orchestra.run.smithi033.stderr: stderr: 10+0 records in 2024-05-12T03:54:57.583 INFO:teuthology.orchestra.run.smithi033.stderr:10+0 records out 2024-05-12T03:54:57.583 INFO:teuthology.orchestra.run.smithi033.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0125037 s, 839 MB/s 2024-05-12T03:54:57.583 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping successful for: 2024-05-12T03:54:58.362 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi033:vg_nvme/lv_4 2024-05-12T03:54:58.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:58 smithi032 ceph-mon[40008]: purged_snaps scrub starts 2024-05-12T03:54:58.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:58 smithi032 ceph-mon[40008]: purged_snaps scrub ok 2024-05-12T03:54:58.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:58 smithi032 ceph-mon[40008]: osd.2 [v2:172.21.15.32:6818/3261757113,v1:172.21.15.32:6819/3261757113] boot 2024-05-12T03:54:58.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:58 smithi032 ceph-mon[40008]: osdmap e19: 3 total, 3 up, 3 in 2024-05-12T03:54:58.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:58 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-12T03:54:58.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:58 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:54:58.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:58 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:54:58.638 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi033/config 2024-05-12T03:54:58.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:58 smithi033 ceph-mon[49525]: purged_snaps scrub starts 2024-05-12T03:54:58.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:58 smithi033 ceph-mon[49525]: purged_snaps scrub ok 2024-05-12T03:54:58.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:58 smithi033 ceph-mon[49525]: osd.2 [v2:172.21.15.32:6818/3261757113,v1:172.21.15.32:6819/3261757113] boot 2024-05-12T03:54:58.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:58 smithi033 ceph-mon[49525]: osdmap e19: 3 total, 3 up, 3 in 2024-05-12T03:54:58.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:58 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-12T03:54:58.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:58 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:54:58.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:58 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:54:59.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:59 smithi033 ceph-mon[49525]: pgmap v76: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-12T03:54:59.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:59 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-05-12T03:54:59.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:59 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-05-12T03:54:59.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:59 smithi033 ceph-mon[49525]: osdmap e20: 3 total, 3 up, 3 in 2024-05-12T03:54:59.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:54:59 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-12T03:54:59.685 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:59 smithi032 ceph-mon[40008]: pgmap v76: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-05-12T03:54:59.685 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:59 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-05-12T03:54:59.686 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:59 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-05-12T03:54:59.686 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:59 smithi032 ceph-mon[40008]: osdmap e20: 3 total, 3 up, 3 in 2024-05-12T03:54:59.686 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:54:59 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-12T03:55:00.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:00 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-12T03:55:00.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:00 smithi033 ceph-mon[49525]: osdmap e21: 3 total, 3 up, 3 in 2024-05-12T03:55:00.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:00 smithi033 ceph-mon[49525]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-05-12T03:55:00.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:00 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-12T03:55:00.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:00 smithi032 ceph-mon[40008]: osdmap e21: 3 total, 3 up, 3 in 2024-05-12T03:55:00.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:00 smithi032 ceph-mon[40008]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-05-12T03:55:00.540 INFO:journalctl@ceph.osd.0.smithi032.stdout:May 12 03:55:00 smithi032 sudo[84539]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-12T03:55:00.540 INFO:journalctl@ceph.osd.0.smithi032.stdout:May 12 03:55:00 smithi032 sudo[84539]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-12T03:55:00.540 INFO:journalctl@ceph.osd.0.smithi032.stdout:May 12 03:55:00 smithi032 sudo[84539]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-12T03:55:00.541 INFO:journalctl@ceph.osd.0.smithi032.stdout:May 12 03:55:00 smithi032 sudo[84539]: pam_unix(sudo:session): session closed for user root 2024-05-12T03:55:00.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:00.704+0000 7f009d5bf700 1 -- 172.21.15.33:0/3700370733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00980fe410 msgr2=0x7f00980fe820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:55:00.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:00.704+0000 7f009d5bf700 1 --2- 172.21.15.33:0/3700370733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00980fe410 0x7f00980fe820 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f0088004940 tx=0x7f0088009e30 comp rx=0 tx=0).stop 2024-05-12T03:55:00.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:00.704+0000 7f009d5bf700 1 -- 172.21.15.33:0/3700370733 shutdown_connections 2024-05-12T03:55:00.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:00.704+0000 7f009d5bf700 1 --2- 172.21.15.33:0/3700370733 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00980fed60 0x7f00980faea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:55:00.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:00.704+0000 7f009d5bf700 1 --2- 172.21.15.33:0/3700370733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00980fe410 0x7f00980fe820 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:55:00.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:00.704+0000 7f009d5bf700 1 -- 172.21.15.33:0/3700370733 >> 172.21.15.33:0/3700370733 conn(0x7f00980f6bd0 msgr2=0x7f00980f8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:55:00.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:00.704+0000 7f009d5bf700 1 -- 172.21.15.33:0/3700370733 shutdown_connections 2024-05-12T03:55:00.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:00.704+0000 7f009d5bf700 1 -- 172.21.15.33:0/3700370733 wait complete. 2024-05-12T03:55:00.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:00.705+0000 7f009d5bf700 1 Processor -- start 2024-05-12T03:55:00.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:00.705+0000 7f009d5bf700 1 -- start start 2024-05-12T03:55:00.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:00.705+0000 7f009d5bf700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00980fe410 0x7f00980fda20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:55:00.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:00.705+0000 7f009d5bf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00980fed60 0x7f00980fc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:55:00.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:00.705+0000 7f009d5bf700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00980fe020 con 0x7f00980fed60 2024-05-12T03:55:00.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:00.705+0000 7f009d5bf700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00980fc610 con 0x7f00980fe410 2024-05-12T03:55:00.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:00.706+0000 7f0096ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00980fe410 0x7f00980fda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:55:00.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:00.706+0000 7f0096ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00980fe410 0x7f00980fda20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44864/0 (socket says 172.21.15.33:44864) 2024-05-12T03:55:00.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:00.706+0000 7f0096ffd700 1 -- 172.21.15.33:0/3565464991 learned_addr learned my addr 172.21.15.33:0/3565464991 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-12T03:55:00.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:00.706+0000 7f00967fc700 1 --2- 172.21.15.33:0/3565464991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00980fed60 0x7f00980fc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:55:00.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:00.706+0000 7f0096ffd700 1 -- 172.21.15.33:0/3565464991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00980fed60 msgr2=0x7f00980fc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:55:00.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:00.706+0000 7f0096ffd700 1 --2- 172.21.15.33:0/3565464991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00980fed60 0x7f00980fc0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:55:00.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:00.706+0000 7f0096ffd700 1 -- 172.21.15.33:0/3565464991 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0088005040 con 0x7f00980fe410 2024-05-12T03:55:00.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:00.706+0000 7f0096ffd700 1 --2- 172.21.15.33:0/3565464991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00980fe410 0x7f00980fda20 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f0088009fd0 tx=0x7f0088009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:55:00.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:00.706+0000 7f008f7fe700 1 -- 172.21.15.33:0/3565464991 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f008800e040 con 0x7f00980fe410 2024-05-12T03:55:00.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:00.706+0000 7f009d5bf700 1 -- 172.21.15.33:0/3565464991 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00980fc860 con 0x7f00980fe410 2024-05-12T03:55:00.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:00.706+0000 7f008f7fe700 1 -- 172.21.15.33:0/3565464991 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0088004290 con 0x7f00980fe410 2024-05-12T03:55:00.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:00.706+0000 7f008f7fe700 1 -- 172.21.15.33:0/3565464991 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f008801c3f0 con 0x7f00980fe410 2024-05-12T03:55:00.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:00.706+0000 7f009d5bf700 1 -- 172.21.15.33:0/3565464991 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00980fce20 con 0x7f00980fe410 2024-05-12T03:55:00.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:00.707+0000 7f009d5bf700 1 -- 172.21.15.33:0/3565464991 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f009804c710 con 0x7f00980fe410 2024-05-12T03:55:00.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:00.708+0000 7f008f7fe700 1 -- 172.21.15.33:0/3565464991 <== mon.1 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 86573+0+0 (secure 0 0 0) 0x7f0088013790 con 0x7f00980fe410 2024-05-12T03:55:00.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:00.708+0000 7f008f7fe700 1 --2- 172.21.15.33:0/3565464991 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f0084069210 0x7f008406b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:55:00.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:00.708+0000 7f008f7fe700 1 -- 172.21.15.33:0/3565464991 <== mon.1 v2:172.21.15.33:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 3228+0+0 (secure 0 0 0) 0x7f0088087b50 con 0x7f00980fe410 2024-05-12T03:55:00.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:00.708+0000 7f00967fc700 1 --2- 172.21.15.33:0/3565464991 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f0084069210 0x7f008406b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:55:00.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:00.709+0000 7f00967fc700 1 --2- 172.21.15.33:0/3565464991 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f0084069210 0x7f008406b6c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0080004f40 tx=0x7f008000b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:55:00.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:00.713+0000 7f008f7fe700 1 -- 172.21.15.33:0/3565464991 <== mon.1 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f0088058360 con 0x7f00980fe410 2024-05-12T03:55:00.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:00.917+0000 7f009d5bf700 1 -- 172.21.15.33:0/3565464991 --> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f0098102580 con 0x7f0084069210 2024-05-12T03:55:01.039 INFO:journalctl@ceph.osd.0.smithi032.stdout:May 12 03:55:00 smithi032 sudo[87680]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-12T03:55:01.039 INFO:journalctl@ceph.osd.0.smithi032.stdout:May 12 03:55:00 smithi032 sudo[87680]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-12T03:55:01.039 INFO:journalctl@ceph.osd.0.smithi032.stdout:May 12 03:55:00 smithi032 sudo[87680]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-12T03:55:01.039 INFO:journalctl@ceph.osd.0.smithi032.stdout:May 12 03:55:00 smithi032 sudo[87680]: pam_unix(sudo:session): session closed for user root 2024-05-12T03:55:01.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:01.319+0000 7f008f7fe700 1 -- 172.21.15.33:0/3565464991 <== mon.1 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f0088054ca0 con 0x7f00980fe410 2024-05-12T03:55:01.539 INFO:journalctl@ceph.osd.1.smithi032.stdout:May 12 03:55:01 smithi032 sudo[87705]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-12T03:55:01.540 INFO:journalctl@ceph.osd.1.smithi032.stdout:May 12 03:55:01 smithi032 sudo[87705]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-12T03:55:01.540 INFO:journalctl@ceph.osd.1.smithi032.stdout:May 12 03:55:01 smithi032 sudo[87705]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-12T03:55:01.540 INFO:journalctl@ceph.osd.1.smithi032.stdout:May 12 03:55:01 smithi032 sudo[87705]: pam_unix(sudo:session): session closed for user root 2024-05-12T03:55:01.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:01 smithi032 ceph-mon[40008]: osdmap e22: 3 total, 3 up, 3 in 2024-05-12T03:55:01.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:01 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:55:01.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:01 smithi032 ceph-mon[40008]: from='client.24129 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T03:55:01.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:01 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-12T03:55:01.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:01 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-12T03:55:01.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:01 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:55:01.648 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:01 smithi033 ceph-mon[49525]: osdmap e22: 3 total, 3 up, 3 in 2024-05-12T03:55:01.648 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:01 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:55:01.648 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:01 smithi033 ceph-mon[49525]: from='client.24129 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T03:55:01.648 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:01 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-12T03:55:01.648 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:01 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-12T03:55:01.648 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:01 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:55:02.039 INFO:journalctl@ceph.osd.1.smithi032.stdout:May 12 03:55:01 smithi032 sudo[87710]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-12T03:55:02.039 INFO:journalctl@ceph.osd.1.smithi032.stdout:May 12 03:55:01 smithi032 sudo[87710]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-12T03:55:02.040 INFO:journalctl@ceph.osd.1.smithi032.stdout:May 12 03:55:01 smithi032 sudo[87710]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-12T03:55:02.040 INFO:journalctl@ceph.osd.1.smithi032.stdout:May 12 03:55:01 smithi032 sudo[87710]: pam_unix(sudo:session): session closed for user root 2024-05-12T03:55:02.539 INFO:journalctl@ceph.osd.2.smithi032.stdout:May 12 03:55:02 smithi032 sudo[87849]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-12T03:55:02.539 INFO:journalctl@ceph.osd.2.smithi032.stdout:May 12 03:55:02 smithi032 sudo[87849]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-12T03:55:02.540 INFO:journalctl@ceph.osd.2.smithi032.stdout:May 12 03:55:02 smithi032 sudo[87849]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-12T03:55:02.540 INFO:journalctl@ceph.osd.2.smithi032.stdout:May 12 03:55:02 smithi032 sudo[87849]: pam_unix(sudo:session): session closed for user root 2024-05-12T03:55:02.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:02 smithi032 ceph-mon[40008]: mgrmap e19: smithi032.ukatkq(active, since 2m), standbys: smithi033.pcvxwy 2024-05-12T03:55:02.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:02 smithi032 ceph-mon[40008]: Detected new or changed devices on smithi032 2024-05-12T03:55:02.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:02 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:55:02.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:02 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:55:02.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:02 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-12T03:55:02.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:02 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-12T03:55:02.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:02 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-12T03:55:02.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:02 smithi032 ceph-mon[40008]: Adjusting osd_memory_target on smithi032 to 4300M 2024-05-12T03:55:02.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:02 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:55:02.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:02 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:55:02.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:02 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T03:55:02.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:02 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:55:02.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:02 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T03:55:02.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:02 smithi032 ceph-mon[40008]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-05-12T03:55:02.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:02 smithi033 ceph-mon[49525]: mgrmap e19: smithi032.ukatkq(active, since 2m), standbys: smithi033.pcvxwy 2024-05-12T03:55:02.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:02 smithi033 ceph-mon[49525]: Detected new or changed devices on smithi032 2024-05-12T03:55:02.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:02 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:55:02.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:02 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:55:02.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:02 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-12T03:55:02.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:02 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-12T03:55:02.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:02 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-12T03:55:02.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:02 smithi033 ceph-mon[49525]: Adjusting osd_memory_target on smithi032 to 4300M 2024-05-12T03:55:02.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:02 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:55:02.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:02 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:55:02.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:02 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T03:55:02.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:02 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:55:02.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:02 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T03:55:02.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:02 smithi033 ceph-mon[49525]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-05-12T03:55:02.993 INFO:journalctl@ceph.osd.2.smithi032.stdout:May 12 03:55:02 smithi032 sudo[87930]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-12T03:55:02.993 INFO:journalctl@ceph.osd.2.smithi032.stdout:May 12 03:55:02 smithi032 sudo[87930]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-12T03:55:02.993 INFO:journalctl@ceph.osd.2.smithi032.stdout:May 12 03:55:02 smithi032 sudo[87930]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-12T03:55:02.993 INFO:journalctl@ceph.osd.2.smithi032.stdout:May 12 03:55:02 smithi032 sudo[87930]: pam_unix(sudo:session): session closed for user root 2024-05-12T03:55:03.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:02 smithi032 sudo[88028]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-12T03:55:03.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:02 smithi032 sudo[88028]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-12T03:55:03.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:02 smithi032 sudo[88028]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-12T03:55:03.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:03 smithi032 sudo[88028]: pam_unix(sudo:session): session closed for user root 2024-05-12T03:55:03.789 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:03 smithi032 sudo[88065]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-12T03:55:03.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:03 smithi032 sudo[88065]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-12T03:55:03.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:03 smithi032 sudo[88065]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-12T03:55:03.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:03 smithi032 sudo[88065]: pam_unix(sudo:session): session closed for user root 2024-05-12T03:55:04.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:03 smithi033 sudo[53658]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-12T03:55:04.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:03 smithi033 sudo[53658]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-12T03:55:04.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:03 smithi033 sudo[53658]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-12T03:55:04.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:03 smithi033 sudo[53658]: pam_unix(sudo:session): session closed for user root 2024-05-12T03:55:04.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:04 smithi033 sudo[53661]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-12T03:55:04.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:04 smithi033 sudo[53661]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-12T03:55:04.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:04 smithi033 sudo[53661]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-12T03:55:04.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:04 smithi033 sudo[53661]: pam_unix(sudo:session): session closed for user root 2024-05-12T03:55:04.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:04 smithi033 ceph-mon[49525]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-12T03:55:04.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:04 smithi033 ceph-mon[49525]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-12T03:55:04.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:04 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mon metadata", "id": "smithi032"}]: dispatch 2024-05-12T03:55:04.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:04 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mon metadata", "id": "smithi033"}]: dispatch 2024-05-12T03:55:04.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:04 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mon metadata", "id": "smithi032"}]: dispatch 2024-05-12T03:55:04.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:04 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mon metadata", "id": "smithi033"}]: dispatch 2024-05-12T03:55:04.789 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:04 smithi032 ceph-mon[40008]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-12T03:55:04.789 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:04 smithi032 ceph-mon[40008]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-12T03:55:04.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:04 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mon metadata", "id": "smithi032"}]: dispatch 2024-05-12T03:55:04.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:04 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mon metadata", "id": "smithi033"}]: dispatch 2024-05-12T03:55:04.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:04 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mon metadata", "id": "smithi032"}]: dispatch 2024-05-12T03:55:04.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:04 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mon metadata", "id": "smithi033"}]: dispatch 2024-05-12T03:55:05.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:05 smithi033 ceph-mon[49525]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-12T03:55:05.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:05 smithi033 ceph-mon[49525]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-05-12T03:55:05.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:05 smithi033 ceph-mon[49525]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-12T03:55:05.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:05 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:55:05.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:05 smithi033 ceph-mon[49525]: from='client.? 172.21.15.33:0/1122266151' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef191267-c0a1-4be7-9e8b-761700af87a2"}]: dispatch 2024-05-12T03:55:05.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:05 smithi033 ceph-mon[49525]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef191267-c0a1-4be7-9e8b-761700af87a2"}]: dispatch 2024-05-12T03:55:05.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:05 smithi033 ceph-mon[49525]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ef191267-c0a1-4be7-9e8b-761700af87a2"}]': finished 2024-05-12T03:55:05.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:05 smithi033 ceph-mon[49525]: osdmap e23: 4 total, 3 up, 4 in 2024-05-12T03:55:05.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:05 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-12T03:55:05.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:05 smithi033 ceph-mon[49525]: from='client.? 172.21.15.33:0/4093909410' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-12T03:55:05.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:05 smithi032 ceph-mon[40008]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-12T03:55:05.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:05 smithi032 ceph-mon[40008]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-05-12T03:55:05.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:05 smithi032 ceph-mon[40008]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-12T03:55:05.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:05 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:55:05.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:05 smithi032 ceph-mon[40008]: from='client.? 172.21.15.33:0/1122266151' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef191267-c0a1-4be7-9e8b-761700af87a2"}]: dispatch 2024-05-12T03:55:05.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:05 smithi032 ceph-mon[40008]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef191267-c0a1-4be7-9e8b-761700af87a2"}]: dispatch 2024-05-12T03:55:05.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:05 smithi032 ceph-mon[40008]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ef191267-c0a1-4be7-9e8b-761700af87a2"}]': finished 2024-05-12T03:55:05.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:05 smithi032 ceph-mon[40008]: osdmap e23: 4 total, 3 up, 4 in 2024-05-12T03:55:05.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:05 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-12T03:55:05.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:05 smithi032 ceph-mon[40008]: from='client.? 172.21.15.33:0/4093909410' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-12T03:55:07.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:07 smithi033 ceph-mon[49525]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-05-12T03:55:07.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:07 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:55:07.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:07 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:55:07.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:07 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:55:07.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:07 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T03:55:07.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:07 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:55:07.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:07 smithi032 ceph-mon[40008]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-05-12T03:55:07.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:07 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:55:07.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:07 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:55:07.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:07 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:55:07.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:07 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T03:55:07.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:07 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:55:09.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:09 smithi033 ceph-mon[49525]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-05-12T03:55:09.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:09 smithi032 ceph-mon[40008]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-05-12T03:55:10.404 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:10 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T03:55:10.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:10 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T03:55:11.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:11 smithi033 ceph-mon[49525]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-05-12T03:55:11.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:11 smithi032 ceph-mon[40008]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-05-12T03:55:12.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:12 smithi033 ceph-mon[49525]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-05-12T03:55:12.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:12 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-12T03:55:12.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:12 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:55:12.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:12 smithi033 ceph-mon[49525]: Deploying daemon osd.3 on smithi033 2024-05-12T03:55:12.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:12 smithi032 ceph-mon[40008]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-05-12T03:55:12.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:12 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-12T03:55:12.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:12 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:55:12.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:12 smithi032 ceph-mon[40008]: Deploying daemon osd.3 on smithi033 2024-05-12T03:55:15.137 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:14 smithi033 ceph-mon[49525]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-05-12T03:55:15.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:14 smithi032 ceph-mon[40008]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-05-12T03:55:17.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:16 smithi033 ceph-mon[49525]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.6 KiB/s rd, 70 KiB/s wr, 4 op/s 2024-05-12T03:55:17.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:16 smithi032 ceph-mon[40008]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.6 KiB/s rd, 70 KiB/s wr, 4 op/s 2024-05-12T03:55:19.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:18 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:55:19.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:18 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:55:19.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:18 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T03:55:19.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:18 smithi032 ceph-mon[40008]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-05-12T03:55:19.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:18 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:55:19.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:18 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:55:19.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:18 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T03:55:19.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:18 smithi033 ceph-mon[49525]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-05-12T03:55:21.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:20 smithi033 ceph-mon[49525]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-05-12T03:55:21.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:20 smithi032 ceph-mon[40008]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-05-12T03:55:22.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:22.142+0000 7f008f7fe700 1 -- 172.21.15.33:0/3565464991 <== mgr.14208 v2:172.21.15.32:6800/4089797100 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f0098102580 con 0x7f0084069210 2024-05-12T03:55:22.144 INFO:teuthology.orchestra.run.smithi033.stdout:Created osd(s) 3 on host 'smithi033' 2024-05-12T03:55:22.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:22.145+0000 7f009d5bf700 1 -- 172.21.15.33:0/3565464991 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f0084069210 msgr2=0x7f008406b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:55:22.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:22.145+0000 7f009d5bf700 1 --2- 172.21.15.33:0/3565464991 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f0084069210 0x7f008406b6c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0080004f40 tx=0x7f008000b040 comp rx=0 tx=0).stop 2024-05-12T03:55:22.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:22.145+0000 7f009d5bf700 1 -- 172.21.15.33:0/3565464991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00980fe410 msgr2=0x7f00980fda20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:55:22.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:22.145+0000 7f009d5bf700 1 --2- 172.21.15.33:0/3565464991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00980fe410 0x7f00980fda20 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f0088009fd0 tx=0x7f0088009e30 comp rx=0 tx=0).stop 2024-05-12T03:55:22.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:22.145+0000 7f009d5bf700 1 -- 172.21.15.33:0/3565464991 shutdown_connections 2024-05-12T03:55:22.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:22.145+0000 7f009d5bf700 1 --2- 172.21.15.33:0/3565464991 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f0084069210 0x7f008406b6c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:55:22.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:22.145+0000 7f009d5bf700 1 --2- 172.21.15.33:0/3565464991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00980fe410 0x7f00980fda20 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:55:22.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:22.145+0000 7f009d5bf700 1 --2- 172.21.15.33:0/3565464991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00980fed60 0x7f00980fc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:55:22.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:22.145+0000 7f009d5bf700 1 -- 172.21.15.33:0/3565464991 >> 172.21.15.33:0/3565464991 conn(0x7f00980f6bd0 msgr2=0x7f00980f8310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:55:22.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:22.145+0000 7f009d5bf700 1 -- 172.21.15.33:0/3565464991 shutdown_connections 2024-05-12T03:55:22.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:22.145+0000 7f009d5bf700 1 -- 172.21.15.33:0/3565464991 wait complete. 2024-05-12T03:55:22.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:22 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:55:22.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:22 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:55:22.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:22 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:55:22.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:22 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T03:55:22.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:22 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:55:22.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:22 smithi033 ceph-mon[49525]: from='osd.3 [v2:172.21.15.33:6800/2331766478,v1:172.21.15.33:6801/2331766478]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-12T03:55:22.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:22 smithi033 ceph-mon[49525]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-12T03:55:22.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:22 smithi033 ceph-mon[49525]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-05-12T03:55:22.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:22 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:55:22.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:22 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:55:22.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:22 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:55:22.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:22 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T03:55:22.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:22 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:55:22.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:22 smithi032 ceph-mon[40008]: from='osd.3 [v2:172.21.15.33:6800/2331766478,v1:172.21.15.33:6801/2331766478]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-12T03:55:22.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:22 smithi032 ceph-mon[40008]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-12T03:55:22.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:22 smithi032 ceph-mon[40008]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-05-12T03:55:23.029 DEBUG:teuthology.orchestra.run.smithi033:osd.3> sudo journalctl -f -n 0 -u ceph-a3e178f8-1012-11ef-bc98-c7b262605968@osd.3.service 2024-05-12T03:55:23.032 INFO:tasks.cephadm:Deploying osd.4 on smithi033 with /dev/vg_nvme/lv_3... 2024-05-12T03:55:23.032 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-12T03:55:23.064 INFO:journalctl@ceph.osd.3.smithi033.stdout:-- Logs begin at Sun 2024-05-12 03:38:52 UTC. -- 2024-05-12T03:55:23.294 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 12 03:55:23 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-3[55966]: 2024-05-12T03:55:23.139+0000 7fb7cd518700 -1 osd.3 0 waiting for initial osdmap 2024-05-12T03:55:23.294 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 12 03:55:23 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-3[55966]: 2024-05-12T03:55:23.145+0000 7fb7c7b12700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-12T03:55:23.294 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:23 smithi033 ceph-mon[49525]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-12T03:55:23.294 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:23 smithi033 ceph-mon[49525]: osdmap e24: 4 total, 3 up, 4 in 2024-05-12T03:55:23.294 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:23 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-12T03:55:23.294 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:23 smithi033 ceph-mon[49525]: from='osd.3 [v2:172.21.15.33:6800/2331766478,v1:172.21.15.33:6801/2331766478]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-05-12T03:55:23.294 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:23 smithi033 ceph-mon[49525]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-05-12T03:55:23.295 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:23 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:55:23.295 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:23 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:55:23.295 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:23 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T03:55:23.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:23 smithi032 ceph-mon[40008]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-12T03:55:23.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:23 smithi032 ceph-mon[40008]: osdmap e24: 4 total, 3 up, 4 in 2024-05-12T03:55:23.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:23 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-12T03:55:23.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:23 smithi032 ceph-mon[40008]: from='osd.3 [v2:172.21.15.33:6800/2331766478,v1:172.21.15.33:6801/2331766478]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-05-12T03:55:23.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:23 smithi032 ceph-mon[40008]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-05-12T03:55:23.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:23 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:55:23.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:23 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:55:23.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:23 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T03:55:24.159 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi033/config 2024-05-12T03:55:24.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:24 smithi033 ceph-mon[49525]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-05-12T03:55:24.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:24 smithi033 ceph-mon[49525]: osdmap e25: 4 total, 3 up, 4 in 2024-05-12T03:55:24.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:24 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-12T03:55:24.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:24 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-12T03:55:24.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:24 smithi033 ceph-mon[49525]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-05-12T03:55:24.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:24 smithi032 ceph-mon[40008]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-05-12T03:55:24.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:24 smithi032 ceph-mon[40008]: osdmap e25: 4 total, 3 up, 4 in 2024-05-12T03:55:24.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:24 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-12T03:55:24.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:24 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-12T03:55:24.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:24 smithi032 ceph-mon[40008]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-05-12T03:55:25.304 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:25 smithi033 ceph-mon[49525]: purged_snaps scrub starts 2024-05-12T03:55:25.304 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:25 smithi033 ceph-mon[49525]: purged_snaps scrub ok 2024-05-12T03:55:25.304 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:25 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-12T03:55:25.304 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:25 smithi033 ceph-mon[49525]: osd.3 [v2:172.21.15.33:6800/2331766478,v1:172.21.15.33:6801/2331766478] boot 2024-05-12T03:55:25.304 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:25 smithi033 ceph-mon[49525]: osdmap e26: 4 total, 4 up, 4 in 2024-05-12T03:55:25.304 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:25 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-12T03:55:25.304 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:25 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T03:55:25.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:25 smithi032 ceph-mon[40008]: purged_snaps scrub starts 2024-05-12T03:55:25.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:25 smithi032 ceph-mon[40008]: purged_snaps scrub ok 2024-05-12T03:55:25.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:25 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-12T03:55:25.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:25 smithi032 ceph-mon[40008]: osd.3 [v2:172.21.15.33:6800/2331766478,v1:172.21.15.33:6801/2331766478] boot 2024-05-12T03:55:25.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:25 smithi032 ceph-mon[40008]: osdmap e26: 4 total, 4 up, 4 in 2024-05-12T03:55:25.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:25 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-12T03:55:25.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:25 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T03:55:26.351 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:26 smithi033 ceph-mon[49525]: osdmap e27: 4 total, 4 up, 4 in 2024-05-12T03:55:26.351 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:26 smithi033 ceph-mon[49525]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-12T03:55:26.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:26 smithi032 ceph-mon[40008]: osdmap e27: 4 total, 4 up, 4 in 2024-05-12T03:55:26.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:26 smithi032 ceph-mon[40008]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-12T03:55:27.155 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:27 smithi033 ceph-mon[49525]: osdmap e28: 4 total, 4 up, 4 in 2024-05-12T03:55:27.357 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-12T03:55:27.358 INFO:teuthology.orchestra.run.smithi033.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-12T03:55:27.358 INFO:teuthology.orchestra.run.smithi033.stderr: stderr: 10+0 records in 2024-05-12T03:55:27.358 INFO:teuthology.orchestra.run.smithi033.stderr:10+0 records out 2024-05-12T03:55:27.358 INFO:teuthology.orchestra.run.smithi033.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0108032 s, 971 MB/s 2024-05-12T03:55:27.358 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping successful for: 2024-05-12T03:55:27.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:27 smithi033 ceph-mon[49525]: osdmap e29: 4 total, 4 up, 4 in 2024-05-12T03:55:27.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:27 smithi032 ceph-mon[40008]: osdmap e28: 4 total, 4 up, 4 in 2024-05-12T03:55:27.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:27 smithi032 ceph-mon[40008]: osdmap e29: 4 total, 4 up, 4 in 2024-05-12T03:55:28.598 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi033:vg_nvme/lv_3 2024-05-12T03:55:28.881 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi033/config 2024-05-12T03:55:28.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:28 smithi033 ceph-mon[49525]: Detected new or changed devices on smithi033 2024-05-12T03:55:28.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:28 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:55:28.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:28 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:55:28.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:28 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-12T03:55:28.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:28 smithi033 ceph-mon[49525]: Adjusting osd_memory_target on smithi033 to 15974M 2024-05-12T03:55:28.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:28 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:55:28.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:28 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:55:28.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:28 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T03:55:28.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:28 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:55:28.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:28 smithi033 ceph-mon[49525]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-12T03:55:28.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:28 smithi033 ceph-mon[49525]: osdmap e30: 4 total, 4 up, 4 in 2024-05-12T03:55:29.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:28 smithi032 ceph-mon[40008]: Detected new or changed devices on smithi033 2024-05-12T03:55:29.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:28 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:55:29.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:28 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:55:29.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:28 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-12T03:55:29.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:28 smithi032 ceph-mon[40008]: Adjusting osd_memory_target on smithi033 to 15974M 2024-05-12T03:55:29.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:28 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:55:29.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:28 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:55:29.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:28 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T03:55:29.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:28 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:55:29.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:28 smithi032 ceph-mon[40008]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-12T03:55:29.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:28 smithi032 ceph-mon[40008]: osdmap e30: 4 total, 4 up, 4 in 2024-05-12T03:55:30.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:30.867+0000 7fee60789700 1 -- 172.21.15.33:0/1809606560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee580fdfd0 msgr2=0x7fee580fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:55:30.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:30.867+0000 7fee60789700 1 --2- 172.21.15.33:0/1809606560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee580fdfd0 0x7fee580fe440 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fee54004940 tx=0x7fee54009e30 comp rx=0 tx=0).stop 2024-05-12T03:55:30.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:30.867+0000 7fee60789700 1 -- 172.21.15.33:0/1809606560 shutdown_connections 2024-05-12T03:55:30.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:30.867+0000 7fee60789700 1 --2- 172.21.15.33:0/1809606560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee580fdfd0 0x7fee580fe440 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:55:30.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:30.867+0000 7fee60789700 1 --2- 172.21.15.33:0/1809606560 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee580fce60 0x7fee580fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:55:30.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:30.867+0000 7fee60789700 1 -- 172.21.15.33:0/1809606560 >> 172.21.15.33:0/1809606560 conn(0x7fee580f8560 msgr2=0x7fee580fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:55:30.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:30.867+0000 7fee60789700 1 -- 172.21.15.33:0/1809606560 shutdown_connections 2024-05-12T03:55:30.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:30.867+0000 7fee60789700 1 -- 172.21.15.33:0/1809606560 wait complete. 2024-05-12T03:55:30.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:30.868+0000 7fee60789700 1 Processor -- start 2024-05-12T03:55:30.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:30.868+0000 7fee60789700 1 -- start start 2024-05-12T03:55:30.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:30.868+0000 7fee60789700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee580fce60 0x7fee5810d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:55:30.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:30.868+0000 7fee60789700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee580fdfd0 0x7fee5810de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:55:30.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:30.868+0000 7fee60789700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5810e460 con 0x7fee580fdfd0 2024-05-12T03:55:30.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:30.868+0000 7fee60789700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5810ecb0 con 0x7fee580fce60 2024-05-12T03:55:30.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:30.868+0000 7fee5e525700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee580fce60 0x7fee5810d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:55:30.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:30.868+0000 7fee5e525700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee580fce60 0x7fee5810d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59838/0 (socket says 172.21.15.33:59838) 2024-05-12T03:55:30.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:30.868+0000 7fee5e525700 1 -- 172.21.15.33:0/3180350046 learned_addr learned my addr 172.21.15.33:0/3180350046 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-12T03:55:30.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:30.868+0000 7fee5dd24700 1 --2- 172.21.15.33:0/3180350046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee580fdfd0 0x7fee5810de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:55:30.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:30.868+0000 7fee5e525700 1 -- 172.21.15.33:0/3180350046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee580fdfd0 msgr2=0x7fee5810de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:55:30.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:30.868+0000 7fee5e525700 1 --2- 172.21.15.33:0/3180350046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee580fdfd0 0x7fee5810de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:55:30.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:30.868+0000 7fee5e525700 1 -- 172.21.15.33:0/3180350046 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee54005040 con 0x7fee580fce60 2024-05-12T03:55:30.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:30.869+0000 7fee5e525700 1 --2- 172.21.15.33:0/3180350046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee580fce60 0x7fee5810d920 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fee4800c830 tx=0x7fee48004470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:55:30.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:30.869+0000 7fee4effd700 1 -- 172.21.15.33:0/3180350046 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee4800a760 con 0x7fee580fce60 2024-05-12T03:55:30.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:30.869+0000 7fee4effd700 1 -- 172.21.15.33:0/3180350046 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fee4800ad70 con 0x7fee580fce60 2024-05-12T03:55:30.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:30.869+0000 7fee60789700 1 -- 172.21.15.33:0/3180350046 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee5810ef90 con 0x7fee580fce60 2024-05-12T03:55:30.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:30.869+0000 7fee4effd700 1 -- 172.21.15.33:0/3180350046 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee480113a0 con 0x7fee580fce60 2024-05-12T03:55:30.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:30.869+0000 7fee60789700 1 -- 172.21.15.33:0/3180350046 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee5810f4d0 con 0x7fee580fce60 2024-05-12T03:55:30.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:30.870+0000 7fee60789700 1 -- 172.21.15.33:0/3180350046 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee5805f5c0 con 0x7fee580fce60 2024-05-12T03:55:30.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:30.871+0000 7fee4effd700 1 -- 172.21.15.33:0/3180350046 <== mon.1 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fee48012400 con 0x7fee580fce60 2024-05-12T03:55:30.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:30.871+0000 7fee4effd700 1 --2- 172.21.15.33:0/3180350046 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fee440692b0 0x7fee4406b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:55:30.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:30.871+0000 7fee4effd700 1 -- 172.21.15.33:0/3180350046 <== mon.1 v2:172.21.15.33:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 3770+0+0 (secure 0 0 0) 0x7fee48086b90 con 0x7fee580fce60 2024-05-12T03:55:30.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:30.871+0000 7fee5dd24700 1 --2- 172.21.15.33:0/3180350046 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fee440692b0 0x7fee4406b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:55:30.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:30.872+0000 7fee5dd24700 1 --2- 172.21.15.33:0/3180350046 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fee440692b0 0x7fee4406b760 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fee5400b900 tx=0x7fee54009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:55:30.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:30.876+0000 7fee4effd700 1 -- 172.21.15.33:0/3180350046 <== mon.1 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fee48056ff0 con 0x7fee580fce60 2024-05-12T03:55:31.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:31.078+0000 7fee60789700 1 -- 172.21.15.33:0/3180350046 --> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fee5810f910 con 0x7fee440692b0 2024-05-12T03:55:31.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:30 smithi033 ceph-mon[49525]: pgmap v103: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-05-12T03:55:31.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:30 smithi032 ceph-mon[40008]: pgmap v103: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-05-12T03:55:32.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:31 smithi033 ceph-mon[49525]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T03:55:32.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:31 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-12T03:55:32.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:31 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-12T03:55:32.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:31 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:55:32.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:31 smithi032 ceph-mon[40008]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T03:55:32.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:31 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-12T03:55:32.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:31 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-12T03:55:32.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:31 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:55:33.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:32 smithi033 ceph-mon[49525]: pgmap v104: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-12T03:55:33.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:32 smithi032 ceph-mon[40008]: pgmap v104: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-12T03:55:34.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:33 smithi033 ceph-mon[49525]: from='client.? 172.21.15.33:0/2911251637' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fe45f168-076a-4888-895d-bb710bd0da73"}]: dispatch 2024-05-12T03:55:34.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:33 smithi033 ceph-mon[49525]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fe45f168-076a-4888-895d-bb710bd0da73"}]: dispatch 2024-05-12T03:55:34.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:33 smithi033 ceph-mon[49525]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fe45f168-076a-4888-895d-bb710bd0da73"}]': finished 2024-05-12T03:55:34.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:33 smithi033 ceph-mon[49525]: osdmap e31: 5 total, 4 up, 5 in 2024-05-12T03:55:34.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:33 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-12T03:55:34.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:33 smithi032 ceph-mon[40008]: from='client.? 172.21.15.33:0/2911251637' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fe45f168-076a-4888-895d-bb710bd0da73"}]: dispatch 2024-05-12T03:55:34.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:33 smithi032 ceph-mon[40008]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fe45f168-076a-4888-895d-bb710bd0da73"}]: dispatch 2024-05-12T03:55:34.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:33 smithi032 ceph-mon[40008]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fe45f168-076a-4888-895d-bb710bd0da73"}]': finished 2024-05-12T03:55:34.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:33 smithi032 ceph-mon[40008]: osdmap e31: 5 total, 4 up, 5 in 2024-05-12T03:55:34.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:33 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-12T03:55:35.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:34 smithi033 ceph-mon[49525]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-05-12T03:55:35.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:34 smithi033 ceph-mon[49525]: from='client.? 172.21.15.33:0/3609727255' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-12T03:55:35.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:34 smithi032 ceph-mon[40008]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-05-12T03:55:35.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:34 smithi032 ceph-mon[40008]: from='client.? 172.21.15.33:0/3609727255' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-12T03:55:37.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:36 smithi033 ceph-mon[49525]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-12T03:55:37.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:36 smithi032 ceph-mon[40008]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-12T03:55:39.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:38 smithi033 ceph-mon[49525]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-12T03:55:39.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:38 smithi032 ceph-mon[40008]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-12T03:55:40.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:39 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T03:55:40.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:39 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T03:55:41.130 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:40 smithi033 ceph-mon[49525]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-12T03:55:41.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:40 smithi032 ceph-mon[40008]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-12T03:55:42.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:41 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-12T03:55:42.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:41 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:55:42.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:41 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-12T03:55:42.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:41 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:55:43.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:42 smithi033 ceph-mon[49525]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-12T03:55:43.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:42 smithi033 ceph-mon[49525]: Deploying daemon osd.4 on smithi033 2024-05-12T03:55:43.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:42 smithi032 ceph-mon[40008]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-12T03:55:43.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:42 smithi032 ceph-mon[40008]: Deploying daemon osd.4 on smithi033 2024-05-12T03:55:45.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:44 smithi033 ceph-mon[49525]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-12T03:55:45.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:44 smithi032 ceph-mon[40008]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-12T03:55:47.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:46 smithi033 ceph-mon[49525]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-12T03:55:47.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:46 smithi032 ceph-mon[40008]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-12T03:55:48.789 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:48 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:55:48.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:48 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T03:55:48.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:48 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:55:48.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:48 smithi032 ceph-mon[40008]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-12T03:55:48.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:48 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:55:48.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:48 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T03:55:48.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:48 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:55:48.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:48 smithi033 ceph-mon[49525]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-12T03:55:51.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:50 smithi033 ceph-mon[49525]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-12T03:55:51.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:50 smithi032 ceph-mon[40008]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-12T03:55:51.950 INFO:teuthology.orchestra.run.smithi033.stdout:Created osd(s) 4 on host 'smithi033' 2024-05-12T03:55:51.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:51.950+0000 7fee4effd700 1 -- 172.21.15.33:0/3180350046 <== mgr.14208 v2:172.21.15.32:6800/4089797100 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fee5810f910 con 0x7fee440692b0 2024-05-12T03:55:51.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:51.952+0000 7fee60789700 1 -- 172.21.15.33:0/3180350046 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fee440692b0 msgr2=0x7fee4406b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:55:51.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:51.952+0000 7fee60789700 1 --2- 172.21.15.33:0/3180350046 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fee440692b0 0x7fee4406b760 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fee5400b900 tx=0x7fee54009e30 comp rx=0 tx=0).stop 2024-05-12T03:55:51.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:51.952+0000 7fee60789700 1 -- 172.21.15.33:0/3180350046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee580fce60 msgr2=0x7fee5810d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:55:51.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:51.952+0000 7fee60789700 1 --2- 172.21.15.33:0/3180350046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee580fce60 0x7fee5810d920 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fee4800c830 tx=0x7fee48004470 comp rx=0 tx=0).stop 2024-05-12T03:55:51.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:51.952+0000 7fee60789700 1 -- 172.21.15.33:0/3180350046 shutdown_connections 2024-05-12T03:55:51.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:51.952+0000 7fee60789700 1 --2- 172.21.15.33:0/3180350046 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fee440692b0 0x7fee4406b760 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:55:51.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:51.952+0000 7fee60789700 1 --2- 172.21.15.33:0/3180350046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee580fce60 0x7fee5810d920 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:55:51.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:51.952+0000 7fee60789700 1 --2- 172.21.15.33:0/3180350046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee580fdfd0 0x7fee5810de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:55:51.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:51.952+0000 7fee60789700 1 -- 172.21.15.33:0/3180350046 >> 172.21.15.33:0/3180350046 conn(0x7fee580f8560 msgr2=0x7fee580f9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:55:51.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:51.952+0000 7fee60789700 1 -- 172.21.15.33:0/3180350046 shutdown_connections 2024-05-12T03:55:51.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:55:51.952+0000 7fee60789700 1 -- 172.21.15.33:0/3180350046 wait complete. 2024-05-12T03:55:52.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:51 smithi033 ceph-mon[49525]: from='osd.4 [v2:172.21.15.33:6808/2404279705,v1:172.21.15.33:6809/2404279705]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-12T03:55:52.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:51 smithi033 ceph-mon[49525]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-12T03:55:52.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:51 smithi032 ceph-mon[40008]: from='osd.4 [v2:172.21.15.33:6808/2404279705,v1:172.21.15.33:6809/2404279705]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-12T03:55:52.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:51 smithi032 ceph-mon[40008]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-12T03:55:52.947 DEBUG:teuthology.orchestra.run.smithi033:osd.4> sudo journalctl -f -n 0 -u ceph-a3e178f8-1012-11ef-bc98-c7b262605968@osd.4.service 2024-05-12T03:55:52.949 INFO:tasks.cephadm:Deploying osd.5 on smithi033 with /dev/vg_nvme/lv_2... 2024-05-12T03:55:52.950 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-12T03:55:52.986 INFO:journalctl@ceph.osd.4.smithi033.stdout:-- Logs begin at Sun 2024-05-12 03:38:52 UTC. -- 2024-05-12T03:55:53.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:52 smithi033 ceph-mon[49525]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-12T03:55:53.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:52 smithi033 ceph-mon[49525]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-12T03:55:53.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:52 smithi033 ceph-mon[49525]: osdmap e32: 5 total, 4 up, 5 in 2024-05-12T03:55:53.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:52 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-12T03:55:53.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:52 smithi033 ceph-mon[49525]: from='osd.4 [v2:172.21.15.33:6808/2404279705,v1:172.21.15.33:6809/2404279705]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-05-12T03:55:53.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:52 smithi033 ceph-mon[49525]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-05-12T03:55:53.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:52 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:55:53.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:52 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:55:53.276 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi033/config 2024-05-12T03:55:53.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:52 smithi032 ceph-mon[40008]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-12T03:55:53.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:52 smithi032 ceph-mon[40008]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-12T03:55:53.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:52 smithi032 ceph-mon[40008]: osdmap e32: 5 total, 4 up, 5 in 2024-05-12T03:55:53.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:52 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-12T03:55:53.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:52 smithi032 ceph-mon[40008]: from='osd.4 [v2:172.21.15.33:6808/2404279705,v1:172.21.15.33:6809/2404279705]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-05-12T03:55:53.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:52 smithi032 ceph-mon[40008]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-05-12T03:55:53.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:52 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:55:53.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:52 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:55:54.150 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:53 smithi033 ceph-mon[49525]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-05-12T03:55:54.150 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:53 smithi033 ceph-mon[49525]: osdmap e33: 5 total, 4 up, 5 in 2024-05-12T03:55:54.150 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:53 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-12T03:55:54.150 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:53 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-12T03:55:54.150 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:53 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:55:54.150 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:53 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:55:54.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:53 smithi032 ceph-mon[40008]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-05-12T03:55:54.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:53 smithi032 ceph-mon[40008]: osdmap e33: 5 total, 4 up, 5 in 2024-05-12T03:55:54.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:53 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-12T03:55:54.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:53 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-12T03:55:54.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:53 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:55:54.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:53 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:55:55.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:54 smithi033 ceph-mon[49525]: purged_snaps scrub starts 2024-05-12T03:55:55.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:54 smithi033 ceph-mon[49525]: purged_snaps scrub ok 2024-05-12T03:55:55.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:54 smithi033 ceph-mon[49525]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-12T03:55:55.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:54 smithi033 ceph-mon[49525]: osd.4 [v2:172.21.15.33:6808/2404279705,v1:172.21.15.33:6809/2404279705] boot 2024-05-12T03:55:55.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:54 smithi033 ceph-mon[49525]: osdmap e34: 5 total, 5 up, 5 in 2024-05-12T03:55:55.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:54 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-12T03:55:55.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:54 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T03:55:55.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:54 smithi032 ceph-mon[40008]: purged_snaps scrub starts 2024-05-12T03:55:55.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:54 smithi032 ceph-mon[40008]: purged_snaps scrub ok 2024-05-12T03:55:55.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:54 smithi032 ceph-mon[40008]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-05-12T03:55:55.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:54 smithi032 ceph-mon[40008]: osd.4 [v2:172.21.15.33:6808/2404279705,v1:172.21.15.33:6809/2404279705] boot 2024-05-12T03:55:55.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:54 smithi032 ceph-mon[40008]: osdmap e34: 5 total, 5 up, 5 in 2024-05-12T03:55:55.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:54 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-12T03:55:55.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:54 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T03:55:56.056 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-12T03:55:56.057 INFO:teuthology.orchestra.run.smithi033.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-12T03:55:56.057 INFO:teuthology.orchestra.run.smithi033.stderr: stderr: 10+0 records in 2024-05-12T03:55:56.057 INFO:teuthology.orchestra.run.smithi033.stderr:10+0 records out 2024-05-12T03:55:56.057 INFO:teuthology.orchestra.run.smithi033.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0109577 s, 957 MB/s 2024-05-12T03:55:56.057 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping successful for: 2024-05-12T03:55:56.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:56 smithi033 ceph-mon[49525]: osdmap e35: 5 total, 5 up, 5 in 2024-05-12T03:55:56.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:56 smithi033 ceph-mon[49525]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-12T03:55:56.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:56 smithi032 ceph-mon[40008]: osdmap e35: 5 total, 5 up, 5 in 2024-05-12T03:55:56.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:56 smithi032 ceph-mon[40008]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-12T03:55:56.973 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi033:vg_nvme/lv_2 2024-05-12T03:55:57.440 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi033/config 2024-05-12T03:55:59.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:55:58 smithi033 ceph-mon[49525]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-12T03:55:59.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:55:58 smithi032 ceph-mon[40008]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-12T03:56:00.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:00.192+0000 7f258e891700 1 -- 172.21.15.33:0/2351437252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2588070830 msgr2=0x7f2588070ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:56:00.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:00.192+0000 7f258e891700 1 --2- 172.21.15.33:0/2351437252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2588070830 0x7f2588070ca0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f257c004970 tx=0x7f257c009e30 comp rx=0 tx=0).stop 2024-05-12T03:56:00.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:00.192+0000 7f258e891700 1 -- 172.21.15.33:0/2351437252 shutdown_connections 2024-05-12T03:56:00.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:00.192+0000 7f258e891700 1 --2- 172.21.15.33:0/2351437252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2588070830 0x7f2588070ca0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:00.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:00.192+0000 7f258e891700 1 --2- 172.21.15.33:0/2351437252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2588071de0 0x7f25880702f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:00.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:00.192+0000 7f258e891700 1 -- 172.21.15.33:0/2351437252 >> 172.21.15.33:0/2351437252 conn(0x7f25880f8560 msgr2=0x7f25880fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:56:00.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:00.192+0000 7f258e891700 1 -- 172.21.15.33:0/2351437252 shutdown_connections 2024-05-12T03:56:00.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:00.192+0000 7f258e891700 1 -- 172.21.15.33:0/2351437252 wait complete. 2024-05-12T03:56:00.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:00.192+0000 7f258e891700 1 Processor -- start 2024-05-12T03:56:00.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:00.193+0000 7f258e891700 1 -- start start 2024-05-12T03:56:00.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:00.193+0000 7f258e891700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2588070830 0x7f258810faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:56:00.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:00.193+0000 7f258e891700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2588071de0 0x7f258810ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:56:00.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:00.193+0000 7f258e891700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25881105e0 con 0x7f2588070830 2024-05-12T03:56:00.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:00.193+0000 7f258e891700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2588110e30 con 0x7f2588071de0 2024-05-12T03:56:00.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:00.193+0000 7f25877fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2588071de0 0x7f258810ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:56:00.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:00.193+0000 7f25877fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2588071de0 0x7f258810ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54694/0 (socket says 172.21.15.33:54694) 2024-05-12T03:56:00.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:00.193+0000 7f25877fe700 1 -- 172.21.15.33:0/1852871054 learned_addr learned my addr 172.21.15.33:0/1852871054 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-12T03:56:00.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:00.193+0000 7f2587fff700 1 --2- 172.21.15.33:0/1852871054 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2588070830 0x7f258810faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:56:00.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:00.193+0000 7f25877fe700 1 -- 172.21.15.33:0/1852871054 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2588070830 msgr2=0x7f258810faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:56:00.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:00.193+0000 7f25877fe700 1 --2- 172.21.15.33:0/1852871054 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2588070830 0x7f258810faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:00.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:00.193+0000 7f25877fe700 1 -- 172.21.15.33:0/1852871054 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f257c005040 con 0x7f2588071de0 2024-05-12T03:56:00.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:00.193+0000 7f25877fe700 1 --2- 172.21.15.33:0/1852871054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2588071de0 0x7f258810ffe0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f257c00b820 tx=0x7f257c009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:56:00.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:00.194+0000 7f2584ff9700 1 -- 172.21.15.33:0/1852871054 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f257c00e040 con 0x7f2588071de0 2024-05-12T03:56:00.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:00.194+0000 7f2584ff9700 1 -- 172.21.15.33:0/1852871054 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f257c013420 con 0x7f2588071de0 2024-05-12T03:56:00.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:00.194+0000 7f258e891700 1 -- 172.21.15.33:0/1852871054 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25881110b0 con 0x7f2588071de0 2024-05-12T03:56:00.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:00.194+0000 7f2584ff9700 1 -- 172.21.15.33:0/1852871054 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f257c01c630 con 0x7f2588071de0 2024-05-12T03:56:00.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:00.194+0000 7f258e891700 1 -- 172.21.15.33:0/1852871054 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25881115f0 con 0x7f2588071de0 2024-05-12T03:56:00.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:00.195+0000 7f258e891700 1 -- 172.21.15.33:0/1852871054 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f258805f5c0 con 0x7f2588071de0 2024-05-12T03:56:00.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:00.195+0000 7f2584ff9700 1 -- 172.21.15.33:0/1852871054 <== mon.1 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f257c0139e0 con 0x7f2588071de0 2024-05-12T03:56:00.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:00.196+0000 7f2584ff9700 1 --2- 172.21.15.33:0/1852871054 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f25700692a0 0x7f257006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:56:00.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:00.196+0000 7f2584ff9700 1 -- 172.21.15.33:0/1852871054 <== mon.1 v2:172.21.15.33:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4202+0+0 (secure 0 0 0) 0x7f257c087e20 con 0x7f2588071de0 2024-05-12T03:56:00.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:00.196+0000 7f2587fff700 1 --2- 172.21.15.33:0/1852871054 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f25700692a0 0x7f257006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:56:00.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:00.196+0000 7f2587fff700 1 --2- 172.21.15.33:0/1852871054 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f25700692a0 0x7f257006b750 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2578009b80 tx=0x7f257800b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:56:00.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:00.200+0000 7f2584ff9700 1 -- 172.21.15.33:0/1852871054 <== mon.1 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f257c058170 con 0x7f2588071de0 2024-05-12T03:56:00.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:00.402+0000 7f258e891700 1 -- 172.21.15.33:0/1852871054 --> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f2588111970 con 0x7f25700692a0 2024-05-12T03:56:01.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:00 smithi033 ceph-mon[49525]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-12T03:56:01.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:00 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-12T03:56:01.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:00 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-12T03:56:01.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:00 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:56:01.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:00 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:56:01.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:00 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:56:01.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:00 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-12T03:56:01.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:00 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-12T03:56:01.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:00 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:56:01.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:00 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:56:01.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:00 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T03:56:01.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:00 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:56:01.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:00 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T03:56:01.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:00 smithi032 ceph-mon[40008]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-12T03:56:01.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:00 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-12T03:56:01.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:00 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-12T03:56:01.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:00 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:56:01.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:00 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:56:01.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:00 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:56:01.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:00 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-12T03:56:01.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:00 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-12T03:56:01.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:00 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:56:01.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:00 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:56:01.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:00 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T03:56:01.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:00 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:56:01.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:00 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T03:56:02.111 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:01 smithi033 ceph-mon[49525]: from='client.24167 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T03:56:02.111 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:01 smithi033 ceph-mon[49525]: Detected new or changed devices on smithi033 2024-05-12T03:56:02.111 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:01 smithi033 ceph-mon[49525]: Adjusting osd_memory_target on smithi033 to 7987M 2024-05-12T03:56:02.111 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:01 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:56:02.111 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:01 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T03:56:02.112 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:01 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:56:02.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:01 smithi032 ceph-mon[40008]: from='client.24167 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T03:56:02.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:01 smithi032 ceph-mon[40008]: Detected new or changed devices on smithi033 2024-05-12T03:56:02.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:01 smithi032 ceph-mon[40008]: Adjusting osd_memory_target on smithi033 to 7987M 2024-05-12T03:56:02.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:01 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:56:02.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:01 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T03:56:02.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:01 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:56:03.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:02 smithi033 ceph-mon[49525]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-12T03:56:03.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:02 smithi032 ceph-mon[40008]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-12T03:56:04.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:03 smithi033 ceph-mon[49525]: from='client.? 172.21.15.33:0/2159634422' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f7a0e14e-a866-4163-a7df-f8a43b860bc0"}]: dispatch 2024-05-12T03:56:04.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:03 smithi033 ceph-mon[49525]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f7a0e14e-a866-4163-a7df-f8a43b860bc0"}]: dispatch 2024-05-12T03:56:04.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:03 smithi033 ceph-mon[49525]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f7a0e14e-a866-4163-a7df-f8a43b860bc0"}]': finished 2024-05-12T03:56:04.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:03 smithi033 ceph-mon[49525]: osdmap e36: 6 total, 5 up, 6 in 2024-05-12T03:56:04.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:03 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-12T03:56:04.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:03 smithi032 ceph-mon[40008]: from='client.? 172.21.15.33:0/2159634422' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f7a0e14e-a866-4163-a7df-f8a43b860bc0"}]: dispatch 2024-05-12T03:56:04.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:03 smithi032 ceph-mon[40008]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f7a0e14e-a866-4163-a7df-f8a43b860bc0"}]: dispatch 2024-05-12T03:56:04.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:03 smithi032 ceph-mon[40008]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f7a0e14e-a866-4163-a7df-f8a43b860bc0"}]': finished 2024-05-12T03:56:04.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:03 smithi032 ceph-mon[40008]: osdmap e36: 6 total, 5 up, 6 in 2024-05-12T03:56:04.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:03 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-12T03:56:05.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:04 smithi033 ceph-mon[49525]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-12T03:56:05.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:04 smithi033 ceph-mon[49525]: from='client.? 172.21.15.33:0/1711595040' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-12T03:56:05.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:04 smithi032 ceph-mon[40008]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-12T03:56:05.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:04 smithi032 ceph-mon[40008]: from='client.? 172.21.15.33:0/1711595040' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-12T03:56:07.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:06 smithi033 ceph-mon[49525]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-12T03:56:07.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:06 smithi032 ceph-mon[40008]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-12T03:56:08.959 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:08 smithi033 ceph-mon[49525]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-12T03:56:09.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:08 smithi032 ceph-mon[40008]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-12T03:56:10.043 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:09 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T03:56:10.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:09 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T03:56:11.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:10 smithi033 ceph-mon[49525]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-12T03:56:11.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:10 smithi032 ceph-mon[40008]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-12T03:56:12.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:11 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-12T03:56:12.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:11 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:56:12.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:11 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-12T03:56:12.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:11 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:56:13.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:12 smithi033 ceph-mon[49525]: Deploying daemon osd.5 on smithi033 2024-05-12T03:56:13.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:12 smithi033 ceph-mon[49525]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-12T03:56:13.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:12 smithi032 ceph-mon[40008]: Deploying daemon osd.5 on smithi033 2024-05-12T03:56:13.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:12 smithi032 ceph-mon[40008]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-12T03:56:15.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:14 smithi033 ceph-mon[49525]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-12T03:56:15.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:14 smithi032 ceph-mon[40008]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-12T03:56:17.152 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:16 smithi033 ceph-mon[49525]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-12T03:56:17.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:16 smithi032 ceph-mon[40008]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-12T03:56:18.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:18 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:56:18.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:18 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:56:18.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:18 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T03:56:18.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:18 smithi033 ceph-mon[49525]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-12T03:56:18.789 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:18 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:56:18.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:18 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:56:18.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:18 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T03:56:18.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:18 smithi032 ceph-mon[40008]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-12T03:56:21.156 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:20 smithi033 ceph-mon[49525]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-12T03:56:21.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:20 smithi032 ceph-mon[40008]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-12T03:56:21.924 INFO:teuthology.orchestra.run.smithi033.stdout:Created osd(s) 5 on host 'smithi033' 2024-05-12T03:56:21.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:21.921+0000 7f2584ff9700 1 -- 172.21.15.33:0/1852871054 <== mgr.14208 v2:172.21.15.32:6800/4089797100 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f2588111970 con 0x7f25700692a0 2024-05-12T03:56:21.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:21.924+0000 7f258e891700 1 -- 172.21.15.33:0/1852871054 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f25700692a0 msgr2=0x7f257006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:56:21.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:21.924+0000 7f258e891700 1 --2- 172.21.15.33:0/1852871054 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f25700692a0 0x7f257006b750 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2578009b80 tx=0x7f257800b040 comp rx=0 tx=0).stop 2024-05-12T03:56:21.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:21.924+0000 7f258e891700 1 -- 172.21.15.33:0/1852871054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2588071de0 msgr2=0x7f258810ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:56:21.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:21.924+0000 7f258e891700 1 --2- 172.21.15.33:0/1852871054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2588071de0 0x7f258810ffe0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f257c00b820 tx=0x7f257c009e30 comp rx=0 tx=0).stop 2024-05-12T03:56:21.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:21.924+0000 7f258e891700 1 -- 172.21.15.33:0/1852871054 shutdown_connections 2024-05-12T03:56:21.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:21.924+0000 7f258e891700 1 --2- 172.21.15.33:0/1852871054 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f25700692a0 0x7f257006b750 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:21.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:21.924+0000 7f258e891700 1 --2- 172.21.15.33:0/1852871054 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2588070830 0x7f258810faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:21.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:21.924+0000 7f258e891700 1 --2- 172.21.15.33:0/1852871054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2588071de0 0x7f258810ffe0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:21.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:21.924+0000 7f258e891700 1 -- 172.21.15.33:0/1852871054 >> 172.21.15.33:0/1852871054 conn(0x7f25880f8560 msgr2=0x7f25880f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:56:21.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:21.925+0000 7f258e891700 1 -- 172.21.15.33:0/1852871054 shutdown_connections 2024-05-12T03:56:21.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:21.925+0000 7f258e891700 1 -- 172.21.15.33:0/1852871054 wait complete. 2024-05-12T03:56:22.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:21 smithi033 ceph-mon[49525]: from='osd.5 [v2:172.21.15.33:6816/2123556026,v1:172.21.15.33:6817/2123556026]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-12T03:56:22.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:21 smithi033 ceph-mon[49525]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-12T03:56:22.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:21 smithi032 ceph-mon[40008]: from='osd.5 [v2:172.21.15.33:6816/2123556026,v1:172.21.15.33:6817/2123556026]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-12T03:56:22.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:21 smithi032 ceph-mon[40008]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-12T03:56:22.721 DEBUG:teuthology.orchestra.run.smithi033:osd.5> sudo journalctl -f -n 0 -u ceph-a3e178f8-1012-11ef-bc98-c7b262605968@osd.5.service 2024-05-12T03:56:22.723 INFO:tasks.cephadm:Waiting for 6 OSDs to come up... 2024-05-12T03:56:22.724 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- ceph osd stat -f json 2024-05-12T03:56:22.754 INFO:journalctl@ceph.osd.5.smithi033.stdout:-- Logs begin at Sun 2024-05-12 03:38:52 UTC. -- 2024-05-12T03:56:23.024 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T03:56:23.039 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:22 smithi032 ceph-mon[40008]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-12T03:56:23.039 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:22 smithi032 ceph-mon[40008]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-12T03:56:23.039 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:22 smithi032 ceph-mon[40008]: osdmap e37: 6 total, 5 up, 6 in 2024-05-12T03:56:23.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:22 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-12T03:56:23.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:22 smithi032 ceph-mon[40008]: from='osd.5 [v2:172.21.15.33:6816/2123556026,v1:172.21.15.33:6817/2123556026]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-05-12T03:56:23.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:22 smithi032 ceph-mon[40008]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-05-12T03:56:23.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:22 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:56:23.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:22 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:56:23.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:22 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:56:23.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:22 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:56:23.161 INFO:journalctl@ceph.osd.5.smithi033.stdout:May 12 03:56:22 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-5[72931]: 2024-05-12T03:56:22.879+0000 7f70bb1fe700 -1 osd.5 0 waiting for initial osdmap 2024-05-12T03:56:23.161 INFO:journalctl@ceph.osd.5.smithi033.stdout:May 12 03:56:22 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-5[72931]: 2024-05-12T03:56:22.888+0000 7f70b7ffd700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-12T03:56:23.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:22 smithi033 ceph-mon[49525]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-12T03:56:23.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:22 smithi033 ceph-mon[49525]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-12T03:56:23.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:22 smithi033 ceph-mon[49525]: osdmap e37: 6 total, 5 up, 6 in 2024-05-12T03:56:23.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:22 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-12T03:56:23.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:22 smithi033 ceph-mon[49525]: from='osd.5 [v2:172.21.15.33:6816/2123556026,v1:172.21.15.33:6817/2123556026]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-05-12T03:56:23.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:22 smithi033 ceph-mon[49525]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-05-12T03:56:23.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:22 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:56:23.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:22 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:56:23.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:22 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:56:23.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:22 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:56:24.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:23 smithi032 ceph-mon[40008]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-05-12T03:56:24.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:23 smithi032 ceph-mon[40008]: osdmap e38: 6 total, 5 up, 6 in 2024-05-12T03:56:24.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:23 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-12T03:56:24.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:23 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-12T03:56:24.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:23 smithi033 ceph-mon[49525]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-05-12T03:56:24.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:23 smithi033 ceph-mon[49525]: osdmap e38: 6 total, 5 up, 6 in 2024-05-12T03:56:24.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:23 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-12T03:56:24.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:23 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-12T03:56:24.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:24 smithi033 ceph-mon[49525]: purged_snaps scrub starts 2024-05-12T03:56:24.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:24 smithi033 ceph-mon[49525]: purged_snaps scrub ok 2024-05-12T03:56:24.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:24 smithi033 ceph-mon[49525]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-12T03:56:24.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:24 smithi033 ceph-mon[49525]: osd.5 [v2:172.21.15.33:6816/2123556026,v1:172.21.15.33:6817/2123556026] boot 2024-05-12T03:56:24.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:24 smithi033 ceph-mon[49525]: osdmap e39: 6 total, 6 up, 6 in 2024-05-12T03:56:24.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:24 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-12T03:56:24.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:24 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T03:56:25.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:25.260+0000 7f4869e0b700 1 -- 172.21.15.32:0/3662375197 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48640fdff0 msgr2=0x7f48640fe460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:56:25.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:25.260+0000 7f4869e0b700 1 --2- 172.21.15.32:0/3662375197 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48640fdff0 0x7f48640fe460 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4858004940 tx=0x7f4858009e30 comp rx=0 tx=0).stop 2024-05-12T03:56:25.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:25.260+0000 7f4869e0b700 1 -- 172.21.15.32:0/3662375197 shutdown_connections 2024-05-12T03:56:25.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:25.260+0000 7f4869e0b700 1 --2- 172.21.15.32:0/3662375197 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48640fdff0 0x7f48640fe460 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:25.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:25.260+0000 7f4869e0b700 1 --2- 172.21.15.32:0/3662375197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48640fce80 0x7f48640fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:25.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:25.260+0000 7f4869e0b700 1 -- 172.21.15.32:0/3662375197 >> 172.21.15.32:0/3662375197 conn(0x7f48640f8580 msgr2=0x7f48640fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:56:25.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:25.260+0000 7f4869e0b700 1 -- 172.21.15.32:0/3662375197 shutdown_connections 2024-05-12T03:56:25.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:25.260+0000 7f4869e0b700 1 -- 172.21.15.32:0/3662375197 wait complete. 2024-05-12T03:56:25.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:25.261+0000 7f4869e0b700 1 Processor -- start 2024-05-12T03:56:25.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:25.261+0000 7f4869e0b700 1 -- start start 2024-05-12T03:56:25.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:25.261+0000 7f4869e0b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48640fce80 0x7f486410d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:56:25.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:25.261+0000 7f4869e0b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48640fdff0 0x7f486410dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:56:25.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:25.261+0000 7f4869e0b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f486410e330 con 0x7f48640fce80 2024-05-12T03:56:25.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:25.261+0000 7f4869e0b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f486410e4a0 con 0x7f48640fdff0 2024-05-12T03:56:25.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:25.261+0000 7f48637fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48640fce80 0x7f486410d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:56:25.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:25.261+0000 7f48637fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48640fce80 0x7f486410d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43390/0 (socket says 172.21.15.32:43390) 2024-05-12T03:56:25.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:25.261+0000 7f48637fe700 1 -- 172.21.15.32:0/2517344912 learned_addr learned my addr 172.21.15.32:0/2517344912 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:56:25.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:25.261+0000 7f4862ffd700 1 --2- 172.21.15.32:0/2517344912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48640fdff0 0x7f486410dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:56:25.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:25.262+0000 7f48637fe700 1 -- 172.21.15.32:0/2517344912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48640fdff0 msgr2=0x7f486410dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:56:25.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:25.262+0000 7f48637fe700 1 --2- 172.21.15.32:0/2517344912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48640fdff0 0x7f486410dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:25.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:25.262+0000 7f48637fe700 1 -- 172.21.15.32:0/2517344912 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4858005040 con 0x7f48640fce80 2024-05-12T03:56:25.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:25.262+0000 7f48637fe700 1 --2- 172.21.15.32:0/2517344912 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48640fce80 0x7f486410d7f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f485400eb40 tx=0x7f485400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:56:25.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:25.262+0000 7f484bfff700 1 -- 172.21.15.32:0/2517344912 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4854007cc0 con 0x7f48640fce80 2024-05-12T03:56:25.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:25.262+0000 7f484bfff700 1 -- 172.21.15.32:0/2517344912 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f485400f040 con 0x7f48640fce80 2024-05-12T03:56:25.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:25.262+0000 7f4869e0b700 1 -- 172.21.15.32:0/2517344912 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4864112ea0 con 0x7f48640fce80 2024-05-12T03:56:25.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:25.262+0000 7f484bfff700 1 -- 172.21.15.32:0/2517344912 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f485400c990 con 0x7f48640fce80 2024-05-12T03:56:25.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:25.262+0000 7f4869e0b700 1 -- 172.21.15.32:0/2517344912 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48641133d0 con 0x7f48640fce80 2024-05-12T03:56:25.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:25.263+0000 7f4869e0b700 1 -- 172.21.15.32:0/2517344912 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f486404c710 con 0x7f48640fce80 2024-05-12T03:56:25.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:25.263+0000 7f484bfff700 1 -- 172.21.15.32:0/2517344912 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f485400cc00 con 0x7f48640fce80 2024-05-12T03:56:25.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:25.264+0000 7f484bfff700 1 --2- 172.21.15.32:0/2517344912 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f484c0692a0 0x7f484c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:56:25.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:25.264+0000 7f484bfff700 1 -- 172.21.15.32:0/2517344912 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f4854086a20 con 0x7f48640fce80 2024-05-12T03:56:25.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:25.264+0000 7f4862ffd700 1 --2- 172.21.15.32:0/2517344912 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f484c0692a0 0x7f484c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:56:25.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:25.264+0000 7f4862ffd700 1 --2- 172.21.15.32:0/2517344912 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f484c0692a0 0x7f484c06b750 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4858004890 tx=0x7f4858009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:56:25.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:25.269+0000 7f484bfff700 1 -- 172.21.15.32:0/2517344912 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f485404e250 con 0x7f48640fce80 2024-05-12T03:56:25.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:24 smithi032 ceph-mon[40008]: purged_snaps scrub starts 2024-05-12T03:56:25.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:24 smithi032 ceph-mon[40008]: purged_snaps scrub ok 2024-05-12T03:56:25.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:24 smithi032 ceph-mon[40008]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-05-12T03:56:25.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:24 smithi032 ceph-mon[40008]: osd.5 [v2:172.21.15.33:6816/2123556026,v1:172.21.15.33:6817/2123556026] boot 2024-05-12T03:56:25.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:24 smithi032 ceph-mon[40008]: osdmap e39: 6 total, 6 up, 6 in 2024-05-12T03:56:25.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:24 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-12T03:56:25.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:24 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T03:56:25.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:25.467+0000 7f4869e0b700 1 -- 172.21.15.32:0/2517344912 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f48641135c0 con 0x7f48640fce80 2024-05-12T03:56:25.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:25.467+0000 7f484bfff700 1 -- 172.21.15.32:0/2517344912 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v40) v1 ==== 74+0+130 (secure 0 0 0) 0x7f4854056b50 con 0x7f48640fce80 2024-05-12T03:56:25.468 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:56:25.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:25.470+0000 7f4869e0b700 1 -- 172.21.15.32:0/2517344912 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f484c0692a0 msgr2=0x7f484c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:56:25.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:25.470+0000 7f4869e0b700 1 --2- 172.21.15.32:0/2517344912 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f484c0692a0 0x7f484c06b750 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4858004890 tx=0x7f4858009e30 comp rx=0 tx=0).stop 2024-05-12T03:56:25.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:25.470+0000 7f4869e0b700 1 -- 172.21.15.32:0/2517344912 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48640fce80 msgr2=0x7f486410d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:56:25.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:25.470+0000 7f4869e0b700 1 --2- 172.21.15.32:0/2517344912 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48640fce80 0x7f486410d7f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f485400eb40 tx=0x7f485400b040 comp rx=0 tx=0).stop 2024-05-12T03:56:25.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:25.470+0000 7f4869e0b700 1 -- 172.21.15.32:0/2517344912 shutdown_connections 2024-05-12T03:56:25.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:25.470+0000 7f4869e0b700 1 --2- 172.21.15.32:0/2517344912 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f484c0692a0 0x7f484c06b750 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:25.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:25.470+0000 7f4869e0b700 1 --2- 172.21.15.32:0/2517344912 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48640fce80 0x7f486410d7f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:25.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:25.470+0000 7f4869e0b700 1 --2- 172.21.15.32:0/2517344912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48640fdff0 0x7f486410dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:25.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:25.470+0000 7f4869e0b700 1 -- 172.21.15.32:0/2517344912 >> 172.21.15.32:0/2517344912 conn(0x7f48640f8580 msgr2=0x7f48640f9fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:56:25.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:25.470+0000 7f4869e0b700 1 -- 172.21.15.32:0/2517344912 shutdown_connections 2024-05-12T03:56:25.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:25.470+0000 7f4869e0b700 1 -- 172.21.15.32:0/2517344912 wait complete. 2024-05-12T03:56:26.144 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:25 smithi032 ceph-mon[40008]: osdmap e40: 6 total, 6 up, 6 in 2024-05-12T03:56:26.145 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:25 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/2517344912' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-12T03:56:26.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:25 smithi033 ceph-mon[49525]: osdmap e40: 6 total, 6 up, 6 in 2024-05-12T03:56:26.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:25 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/2517344912' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-12T03:56:26.165 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":40,"num_osds":6,"num_up_osds":6,"osd_up_since":1715486183,"num_in_osds":6,"osd_in_since":1715486163,"num_remapped_pgs":0} 2024-05-12T03:56:26.166 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-12T03:56:26.444 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T03:56:27.039 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:26 smithi032 ceph-mon[40008]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:56:27.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:26 smithi033 ceph-mon[49525]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:56:28.241 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:28 smithi032 ceph-mon[40008]: Detected new or changed devices on smithi033 2024-05-12T03:56:28.242 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:28 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:56:28.242 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:28 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:56:28.242 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:28 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-12T03:56:28.242 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:28 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-12T03:56:28.242 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:28 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-12T03:56:28.242 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:28 smithi032 ceph-mon[40008]: Adjusting osd_memory_target on smithi033 to 5324M 2024-05-12T03:56:28.242 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:28 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:56:28.242 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:28 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:56:28.242 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:28 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T03:56:28.242 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:28 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:56:28.242 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:28 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T03:56:28.242 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:28 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:56:28.242 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:28 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T03:56:28.243 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:28 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:56:28.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:28 smithi033 ceph-mon[49525]: Detected new or changed devices on smithi033 2024-05-12T03:56:28.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:28 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:56:28.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:28 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:56:28.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:28 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-12T03:56:28.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:28 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-12T03:56:28.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:28 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-12T03:56:28.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:28 smithi033 ceph-mon[49525]: Adjusting osd_memory_target on smithi033 to 5324M 2024-05-12T03:56:28.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:28 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:56:28.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:28 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:56:28.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:28 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T03:56:28.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:28 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:56:28.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:28 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T03:56:28.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:28 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:56:28.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:28 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T03:56:28.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:28 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:56:28.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:28.517+0000 7f972a099700 1 -- 172.21.15.32:0/2118747923 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97240fbd40 msgr2=0x7f97240fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:56:28.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:28.517+0000 7f972a099700 1 --2- 172.21.15.32:0/2118747923 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97240fbd40 0x7f97240fc1b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f970c004940 tx=0x7f970c009e30 comp rx=0 tx=0).stop 2024-05-12T03:56:28.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:28.517+0000 7f972a099700 1 -- 172.21.15.32:0/2118747923 shutdown_connections 2024-05-12T03:56:28.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:28.517+0000 7f972a099700 1 --2- 172.21.15.32:0/2118747923 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97240fbd40 0x7f97240fc1b0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:28.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:28.517+0000 7f972a099700 1 --2- 172.21.15.32:0/2118747923 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9724101d90 0x7f9724102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:28.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:28.517+0000 7f972a099700 1 -- 172.21.15.32:0/2118747923 >> 172.21.15.32:0/2118747923 conn(0x7f9724071d00 msgr2=0x7f9724072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:56:28.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:28.517+0000 7f972a099700 1 -- 172.21.15.32:0/2118747923 shutdown_connections 2024-05-12T03:56:28.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:28.517+0000 7f972a099700 1 -- 172.21.15.32:0/2118747923 wait complete. 2024-05-12T03:56:28.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:28.518+0000 7f972a099700 1 Processor -- start 2024-05-12T03:56:28.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:28.518+0000 7f972a099700 1 -- start start 2024-05-12T03:56:28.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:28.518+0000 7f972a099700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97240fbd40 0x7f972410ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:56:28.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:28.518+0000 7f972a099700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9724101d90 0x7f972410ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:56:28.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:28.518+0000 7f972a099700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972410f5c0 con 0x7f97240fbd40 2024-05-12T03:56:28.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:28.518+0000 7f972a099700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972410f700 con 0x7f9724101d90 2024-05-12T03:56:28.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:28.518+0000 7f97237fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97240fbd40 0x7f972410ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:56:28.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:28.518+0000 7f97237fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97240fbd40 0x7f972410ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43402/0 (socket says 172.21.15.32:43402) 2024-05-12T03:56:28.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:28.518+0000 7f97237fe700 1 -- 172.21.15.32:0/3215345247 learned_addr learned my addr 172.21.15.32:0/3215345247 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:56:28.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:28.518+0000 7f9722ffd700 1 --2- 172.21.15.32:0/3215345247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9724101d90 0x7f972410ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:56:28.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:28.519+0000 7f97237fe700 1 -- 172.21.15.32:0/3215345247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9724101d90 msgr2=0x7f972410ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:56:28.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:28.519+0000 7f97237fe700 1 --2- 172.21.15.32:0/3215345247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9724101d90 0x7f972410ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:28.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:28.519+0000 7f97237fe700 1 -- 172.21.15.32:0/3215345247 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f970c005040 con 0x7f97240fbd40 2024-05-12T03:56:28.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:28.519+0000 7f97237fe700 1 --2- 172.21.15.32:0/3215345247 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97240fbd40 0x7f972410ea30 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9714002a50 tx=0x7f97140046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:56:28.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:28.519+0000 7f971bfff700 1 -- 172.21.15.32:0/3215345247 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9714007cc0 con 0x7f97240fbd40 2024-05-12T03:56:28.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:28.519+0000 7f972a099700 1 -- 172.21.15.32:0/3215345247 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9724117aa0 con 0x7f97240fbd40 2024-05-12T03:56:28.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:28.519+0000 7f971bfff700 1 -- 172.21.15.32:0/3215345247 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f971400f040 con 0x7f97240fbd40 2024-05-12T03:56:28.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:28.519+0000 7f971bfff700 1 -- 172.21.15.32:0/3215345247 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9714009540 con 0x7f97240fbd40 2024-05-12T03:56:28.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:28.519+0000 7f972a099700 1 -- 172.21.15.32:0/3215345247 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9724118000 con 0x7f97240fbd40 2024-05-12T03:56:28.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:28.520+0000 7f972a099700 1 -- 172.21.15.32:0/3215345247 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97240fd430 con 0x7f97240fbd40 2024-05-12T03:56:28.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:28.520+0000 7f971bfff700 1 -- 172.21.15.32:0/3215345247 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f97140096a0 con 0x7f97240fbd40 2024-05-12T03:56:28.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:28.521+0000 7f971bfff700 1 --2- 172.21.15.32:0/3215345247 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f97100692a0 0x7f971006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:56:28.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:28.521+0000 7f971bfff700 1 -- 172.21.15.32:0/3215345247 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f9714086c70 con 0x7f97240fbd40 2024-05-12T03:56:28.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:28.521+0000 7f9722ffd700 1 --2- 172.21.15.32:0/3215345247 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f97100692a0 0x7f971006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:56:28.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:28.521+0000 7f9722ffd700 1 --2- 172.21.15.32:0/3215345247 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f97100692a0 0x7f971006b750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f970c00b3b0 tx=0x7f970c009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:56:28.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:28.525+0000 7f971bfff700 1 -- 172.21.15.32:0/3215345247 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f9714053260 con 0x7f97240fbd40 2024-05-12T03:56:28.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:28.723+0000 7f972a099700 1 -- 172.21.15.32:0/3215345247 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f972405f5c0 con 0x7f97240fbd40 2024-05-12T03:56:28.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:28.723+0000 7f971bfff700 1 -- 172.21.15.32:0/3215345247 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11059 (secure 0 0 0) 0x7f9714056880 con 0x7f97240fbd40 2024-05-12T03:56:28.725 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:56:28.725 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":40,"fsid":"a3e178f8-1012-11ef-bc98-c7b262605968","created":"2024-05-12T03:50:18.356038+0000","modified":"2024-05-12T03:56:24.885995+0000","last_up_change":"2024-05-12T03:56:23.877862+0000","last_in_change":"2024-05-12T03:56:03.301729+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-12T03:54:58.296702+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"fc534129-ef70-487d-b8d1-f98fc170c1d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":3497755677},{"type":"v1","addr":"172.21.15.32:6803","nonce":3497755677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":3497755677},{"type":"v1","addr":"172.21.15.32:6805","nonce":3497755677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":3497755677},{"type":"v1","addr":"172.21.15.32:6809","nonce":3497755677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":3497755677},{"type":"v1","addr":"172.21.15.32:6807","nonce":3497755677}]},"public_addr":"172.21.15.32:6803/3497755677","cluster_addr":"172.21.15.32:6805/3497755677","heartbeat_back_addr":"172.21.15.32:6809/3497755677","heartbeat_front_addr":"172.21.15.32:6807/3497755677","state":["exists","up"]},{"osd":1,"uuid":"f8c35b6a-5afc-447b-b318-d56097de28d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":3065624488},{"type":"v1","addr":"172.21.15.32:6811","nonce":3065624488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":3065624488},{"type":"v1","addr":"172.21.15.32:6813","nonce":3065624488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3065624488},{"type":"v1","addr":"172.21.15.32:6817","nonce":3065624488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":3065624488},{"type":"v1","addr":"172.21.15.32:6815","nonce":3065624488}]},"public_addr":"172.21.15.32:6811/3065624488","cluster_addr":"172.21.15.32:6813/3065624488","heartbeat_back_addr":"172.21.15.32:6817/3065624488","heartbeat_front_addr":"172.21.15.32:6815/3065624488","state":["exists","up"]},{"osd":2,"uuid":"4ed4b55d-cceb-494b-bdff-fef99d6c8cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3261757113},{"type":"v1","addr":"172.21.15.32:6819","nonce":3261757113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":3261757113},{"type":"v1","addr":"172.21.15.32:6821","nonce":3261757113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":3261757113},{"type":"v1","addr":"172.21.15.32:6825","nonce":3261757113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3261757113},{"type":"v1","addr":"172.21.15.32:6823","nonce":3261757113}]},"public_addr":"172.21.15.32:6819/3261757113","cluster_addr":"172.21.15.32:6821/3261757113","heartbeat_back_addr":"172.21.15.32:6825/3261757113","heartbeat_front_addr":"172.21.15.32:6823/3261757113","state":["exists","up"]},{"osd":3,"uuid":"ef191267-c0a1-4be7-9e8b-761700af87a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2331766478},{"type":"v1","addr":"172.21.15.33:6801","nonce":2331766478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2331766478},{"type":"v1","addr":"172.21.15.33:6803","nonce":2331766478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2331766478},{"type":"v1","addr":"172.21.15.33:6807","nonce":2331766478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2331766478},{"type":"v1","addr":"172.21.15.33:6805","nonce":2331766478}]},"public_addr":"172.21.15.33:6801/2331766478","cluster_addr":"172.21.15.33:6803/2331766478","heartbeat_back_addr":"172.21.15.33:6807/2331766478","heartbeat_front_addr":"172.21.15.33:6805/2331766478","state":["exists","up"]},{"osd":4,"uuid":"fe45f168-076a-4888-895d-bb710bd0da73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":2404279705},{"type":"v1","addr":"172.21.15.33:6809","nonce":2404279705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":2404279705},{"type":"v1","addr":"172.21.15.33:6811","nonce":2404279705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":2404279705},{"type":"v1","addr":"172.21.15.33:6815","nonce":2404279705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":2404279705},{"type":"v1","addr":"172.21.15.33:6813","nonce":2404279705}]},"public_addr":"172.21.15.33:6809/2404279705","cluster_addr":"172.21.15.33:6811/2404279705","heartbeat_back_addr":"172.21.15.33:6815/2404279705","heartbeat_front_addr":"172.21.15.33:6813/2404279705","state":["exists","up"]},{"osd":5,"uuid":"f7a0e14e-a866-4163-a7df-f8a43b860bc0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2123556026},{"type":"v1","addr":"172.21.15.33:6817","nonce":2123556026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":2123556026},{"type":"v1","addr":"172.21.15.33:6819","nonce":2123556026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":2123556026},{"type":"v1","addr":"172.21.15.33:6823","nonce":2123556026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2123556026},{"type":"v1","addr":"172.21.15.33:6821","nonce":2123556026}]},"public_addr":"172.21.15.33:6817/2123556026","cluster_addr":"172.21.15.33:6819/2123556026","heartbeat_back_addr":"172.21.15.33:6823/2123556026","heartbeat_front_addr":"172.21.15.33:6821/2123556026","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-12T03:53:56.134194+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-12T03:54:25.231975+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-12T03:54:54.991052+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-12T03:55:22.680656+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-12T03:55:52.446678+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-12T03:56:22.128957+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6800/1988399583":"2024-05-13T03:52:39.772717+0000","172.21.15.32:0/3185268518":"2024-05-13T03:52:39.772717+0000","172.21.15.32:0/1073406471":"2024-05-13T03:52:39.772717+0000","172.21.15.32:6801/2967201790":"2024-05-13T03:51:25.269581+0000","172.21.15.32:6800/2967201790":"2024-05-13T03:51:25.269581+0000","172.21.15.32:6801/1988399583":"2024-05-13T03:52:39.772717+0000","172.21.15.32:0/2900983644":"2024-05-13T03:52:39.772717+0000","172.21.15.32:0/165225041":"2024-05-13T03:51:25.269581+0000","172.21.15.32:0/4293156271":"2024-05-13T03:51:25.269581+0000","172.21.15.32:6800/1121526649":"2024-05-13T03:50:49.378828+0000","172.21.15.32:0/2098237559":"2024-05-13T03:50:49.378828+0000","172.21.15.32:0/1463926533":"2024-05-13T03:50:49.378828+0000","172.21.15.32:6801/1121526649":"2024-05-13T03:50:49.378828+0000","172.21.15.32:0/625670655":"2024-05-13T03:50:49.378828+0000","172.21.15.32:0/1356801576":"2024-05-13T03:51:25.269581+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-12T03:56:28.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:28.726+0000 7f972a099700 1 -- 172.21.15.32:0/3215345247 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f97100692a0 msgr2=0x7f971006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:56:28.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:28.726+0000 7f972a099700 1 --2- 172.21.15.32:0/3215345247 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f97100692a0 0x7f971006b750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f970c00b3b0 tx=0x7f970c009e30 comp rx=0 tx=0).stop 2024-05-12T03:56:28.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:28.726+0000 7f972a099700 1 -- 172.21.15.32:0/3215345247 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97240fbd40 msgr2=0x7f972410ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:56:28.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:28.726+0000 7f972a099700 1 --2- 172.21.15.32:0/3215345247 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97240fbd40 0x7f972410ea30 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9714002a50 tx=0x7f97140046b0 comp rx=0 tx=0).stop 2024-05-12T03:56:28.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:28.726+0000 7f972a099700 1 -- 172.21.15.32:0/3215345247 shutdown_connections 2024-05-12T03:56:28.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:28.726+0000 7f972a099700 1 --2- 172.21.15.32:0/3215345247 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f97100692a0 0x7f971006b750 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:28.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:28.726+0000 7f972a099700 1 --2- 172.21.15.32:0/3215345247 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97240fbd40 0x7f972410ea30 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:28.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:28.726+0000 7f972a099700 1 --2- 172.21.15.32:0/3215345247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9724101d90 0x7f972410ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:28.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:28.726+0000 7f972a099700 1 -- 172.21.15.32:0/3215345247 >> 172.21.15.32:0/3215345247 conn(0x7f9724071d00 msgr2=0x7f9724104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:56:28.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:28.726+0000 7f972a099700 1 -- 172.21.15.32:0/3215345247 shutdown_connections 2024-05-12T03:56:28.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:28.726+0000 7f972a099700 1 -- 172.21.15.32:0/3215345247 wait complete. 2024-05-12T03:56:29.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:29 smithi032 ceph-mon[40008]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:56:29.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:29 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/3215345247' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-12T03:56:29.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:29 smithi033 ceph-mon[49525]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:56:29.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:29 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/3215345247' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-12T03:56:29.605 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-12T03:54:58.296702+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-05-12T03:56:29.605 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-12T03:56:29.885 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T03:56:31.122 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:31 smithi032 ceph-mon[40008]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:56:31.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:31 smithi033 ceph-mon[49525]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:56:31.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:31.603+0000 7f6f37198700 1 -- 172.21.15.32:0/2373495166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f30101cd0 msgr2=0x7f6f301020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:56:31.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:31.603+0000 7f6f37198700 1 --2- 172.21.15.32:0/2373495166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f30101cd0 0x7f6f301020a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6f24004940 tx=0x7f6f24009e30 comp rx=0 tx=0).stop 2024-05-12T03:56:31.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:31.604+0000 7f6f37198700 1 -- 172.21.15.32:0/2373495166 shutdown_connections 2024-05-12T03:56:31.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:31.604+0000 7f6f37198700 1 --2- 172.21.15.32:0/2373495166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f30102670 0x7f6f300fb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:31.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:31.604+0000 7f6f37198700 1 --2- 172.21.15.32:0/2373495166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f30101cd0 0x7f6f301020a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:31.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:31.604+0000 7f6f37198700 1 -- 172.21.15.32:0/2373495166 >> 172.21.15.32:0/2373495166 conn(0x7f6f30071460 msgr2=0x7f6f30071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:56:31.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:31.604+0000 7f6f37198700 1 -- 172.21.15.32:0/2373495166 shutdown_connections 2024-05-12T03:56:31.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:31.604+0000 7f6f37198700 1 -- 172.21.15.32:0/2373495166 wait complete. 2024-05-12T03:56:31.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:31.604+0000 7f6f37198700 1 Processor -- start 2024-05-12T03:56:31.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:31.604+0000 7f6f37198700 1 -- start start 2024-05-12T03:56:31.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:31.605+0000 7f6f37198700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f30101cd0 0x7f6f30113a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:56:31.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:31.605+0000 7f6f37198700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f30102670 0x7f6f30113fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:56:31.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:31.605+0000 7f6f37198700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f301146b0 con 0x7f6f30102670 2024-05-12T03:56:31.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:31.605+0000 7f6f37198700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3010da10 con 0x7f6f30101cd0 2024-05-12T03:56:31.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:31.605+0000 7f6f2ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f30102670 0x7f6f30113fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:56:31.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:31.605+0000 7f6f2ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f30102670 0x7f6f30113fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43424/0 (socket says 172.21.15.32:43424) 2024-05-12T03:56:31.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:31.605+0000 7f6f2ffff700 1 -- 172.21.15.32:0/215074665 learned_addr learned my addr 172.21.15.32:0/215074665 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:56:31.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:31.605+0000 7f6f34f34700 1 --2- 172.21.15.32:0/215074665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f30101cd0 0x7f6f30113a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:56:31.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:31.605+0000 7f6f2ffff700 1 -- 172.21.15.32:0/215074665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f30101cd0 msgr2=0x7f6f30113a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:56:31.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:31.605+0000 7f6f2ffff700 1 --2- 172.21.15.32:0/215074665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f30101cd0 0x7f6f30113a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:31.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:31.605+0000 7f6f2ffff700 1 -- 172.21.15.32:0/215074665 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f24005040 con 0x7f6f30102670 2024-05-12T03:56:31.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:31.605+0000 7f6f2ffff700 1 --2- 172.21.15.32:0/215074665 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f30102670 0x7f6f30113fd0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c00b760 tx=0x7f6f1c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:56:31.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:31.605+0000 7f6f2d7fa700 1 -- 172.21.15.32:0/215074665 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f1c00f6f0 con 0x7f6f30102670 2024-05-12T03:56:31.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:31.605+0000 7f6f2d7fa700 1 -- 172.21.15.32:0/215074665 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6f1c00fd00 con 0x7f6f30102670 2024-05-12T03:56:31.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:31.605+0000 7f6f2d7fa700 1 -- 172.21.15.32:0/215074665 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f1c011e60 con 0x7f6f30102670 2024-05-12T03:56:31.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:31.605+0000 7f6f37198700 1 -- 172.21.15.32:0/215074665 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f3010dcf0 con 0x7f6f30102670 2024-05-12T03:56:31.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:31.605+0000 7f6f37198700 1 -- 172.21.15.32:0/215074665 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f3010e2b0 con 0x7f6f30102670 2024-05-12T03:56:31.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:31.606+0000 7f6f37198700 1 -- 172.21.15.32:0/215074665 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f30065200 con 0x7f6f30102670 2024-05-12T03:56:31.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:31.607+0000 7f6f2d7fa700 1 -- 172.21.15.32:0/215074665 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f6f1c011690 con 0x7f6f30102670 2024-05-12T03:56:31.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:31.607+0000 7f6f2d7fa700 1 --2- 172.21.15.32:0/215074665 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f6f200692a0 0x7f6f2006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:56:31.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:31.607+0000 7f6f34f34700 1 --2- 172.21.15.32:0/215074665 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f6f200692a0 0x7f6f2006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:56:31.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:31.607+0000 7f6f2d7fa700 1 -- 172.21.15.32:0/215074665 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f6f1c086c20 con 0x7f6f30102670 2024-05-12T03:56:31.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:31.608+0000 7f6f34f34700 1 --2- 172.21.15.32:0/215074665 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f6f200692a0 0x7f6f2006b750 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6f24004f90 tx=0x7f6f24009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:56:31.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:31.612+0000 7f6f2d7fa700 1 -- 172.21.15.32:0/215074665 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f6f1c056dc0 con 0x7f6f30102670 2024-05-12T03:56:31.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:31.804+0000 7f6f37198700 1 -- 172.21.15.32:0/215074665 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6f3005f5c0 con 0x7f6f30102670 2024-05-12T03:56:31.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:31.805+0000 7f6f2d7fa700 1 -- 172.21.15.32:0/215074665 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6f1c056950 con 0x7f6f30102670 2024-05-12T03:56:31.806 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-05-12T03:56:31.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:31.807+0000 7f6f37198700 1 -- 172.21.15.32:0/215074665 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f6f200692a0 msgr2=0x7f6f2006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:56:31.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:31.807+0000 7f6f37198700 1 --2- 172.21.15.32:0/215074665 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f6f200692a0 0x7f6f2006b750 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6f24004f90 tx=0x7f6f24009e30 comp rx=0 tx=0).stop 2024-05-12T03:56:31.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:31.807+0000 7f6f37198700 1 -- 172.21.15.32:0/215074665 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f30102670 msgr2=0x7f6f30113fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:56:31.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:31.807+0000 7f6f37198700 1 --2- 172.21.15.32:0/215074665 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f30102670 0x7f6f30113fd0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c00b760 tx=0x7f6f1c009220 comp rx=0 tx=0).stop 2024-05-12T03:56:31.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:31.807+0000 7f6f37198700 1 -- 172.21.15.32:0/215074665 shutdown_connections 2024-05-12T03:56:31.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:31.807+0000 7f6f37198700 1 --2- 172.21.15.32:0/215074665 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f6f200692a0 0x7f6f2006b750 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:31.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:31.807+0000 7f6f37198700 1 --2- 172.21.15.32:0/215074665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f30101cd0 0x7f6f30113a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:31.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:31.807+0000 7f6f37198700 1 --2- 172.21.15.32:0/215074665 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f30102670 0x7f6f30113fd0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:31.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:31.807+0000 7f6f37198700 1 -- 172.21.15.32:0/215074665 >> 172.21.15.32:0/215074665 conn(0x7f6f30071460 msgr2=0x7f6f300f9430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:56:31.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:31.807+0000 7f6f37198700 1 -- 172.21.15.32:0/215074665 shutdown_connections 2024-05-12T03:56:31.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:31.807+0000 7f6f37198700 1 -- 172.21.15.32:0/215074665 wait complete. 2024-05-12T03:56:32.223 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:32 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/215074665' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-12T03:56:32.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:32 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/215074665' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-12T03:56:32.839 INFO:tasks.cephadm:Setting up client nodes... 2024-05-12T03:56:32.839 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-12T03:56:33.125 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T03:56:33.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:33 smithi032 ceph-mon[40008]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:56:33.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:33 smithi033 ceph-mon[49525]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:56:35.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:35.094+0000 7f8e37139700 1 -- 172.21.15.32:0/532714296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e300fcfa0 msgr2=0x7f8e300fd410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:56:35.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:35.094+0000 7f8e37139700 1 --2- 172.21.15.32:0/532714296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e300fcfa0 0x7f8e300fd410 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c004940 tx=0x7f8e1c009e30 comp rx=0 tx=0).stop 2024-05-12T03:56:35.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:35.094+0000 7f8e37139700 1 -- 172.21.15.32:0/532714296 shutdown_connections 2024-05-12T03:56:35.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:35.094+0000 7f8e37139700 1 --2- 172.21.15.32:0/532714296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e300fcfa0 0x7f8e300fd410 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:35.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:35.094+0000 7f8e37139700 1 --2- 172.21.15.32:0/532714296 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e300fbcf0 0x7f8e300fc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:35.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:35.095+0000 7f8e37139700 1 -- 172.21.15.32:0/532714296 >> 172.21.15.32:0/532714296 conn(0x7f8e300f7430 msgr2=0x7f8e300f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:56:35.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:35.095+0000 7f8e37139700 1 -- 172.21.15.32:0/532714296 shutdown_connections 2024-05-12T03:56:35.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:35.095+0000 7f8e37139700 1 -- 172.21.15.32:0/532714296 wait complete. 2024-05-12T03:56:35.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:35.095+0000 7f8e37139700 1 Processor -- start 2024-05-12T03:56:35.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:35.095+0000 7f8e37139700 1 -- start start 2024-05-12T03:56:35.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:35.095+0000 7f8e37139700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e300fbcf0 0x7f8e30065fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:56:35.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:35.096+0000 7f8e37139700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e300fcfa0 0x7f8e30112620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:56:35.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:35.096+0000 7f8e37139700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e30112b60 con 0x7f8e300fbcf0 2024-05-12T03:56:35.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:35.096+0000 7f8e37139700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e30112ca0 con 0x7f8e300fcfa0 2024-05-12T03:56:35.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:35.096+0000 7f8e34ed5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e300fbcf0 0x7f8e30065fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:56:35.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:35.096+0000 7f8e34ed5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e300fbcf0 0x7f8e30065fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39518/0 (socket says 172.21.15.32:39518) 2024-05-12T03:56:35.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:35.096+0000 7f8e34ed5700 1 -- 172.21.15.32:0/3245653057 learned_addr learned my addr 172.21.15.32:0/3245653057 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:56:35.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:35.096+0000 7f8e2ffff700 1 --2- 172.21.15.32:0/3245653057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e300fcfa0 0x7f8e30112620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:56:35.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:35.096+0000 7f8e34ed5700 1 -- 172.21.15.32:0/3245653057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e300fcfa0 msgr2=0x7f8e30112620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:56:35.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:35.096+0000 7f8e34ed5700 1 --2- 172.21.15.32:0/3245653057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e300fcfa0 0x7f8e30112620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:35.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:35.096+0000 7f8e34ed5700 1 -- 172.21.15.32:0/3245653057 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e1c005040 con 0x7f8e300fbcf0 2024-05-12T03:56:35.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:35.096+0000 7f8e34ed5700 1 --2- 172.21.15.32:0/3245653057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e300fbcf0 0x7f8e30065fe0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8e2400c830 tx=0x7f8e24004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:56:35.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:35.096+0000 7f8e2d7fa700 1 -- 172.21.15.32:0/3245653057 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e240106f0 con 0x7f8e300fbcf0 2024-05-12T03:56:35.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:35.096+0000 7f8e37139700 1 -- 172.21.15.32:0/3245653057 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e30112f00 con 0x7f8e300fbcf0 2024-05-12T03:56:35.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:35.096+0000 7f8e2d7fa700 1 -- 172.21.15.32:0/3245653057 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8e24010d00 con 0x7f8e300fbcf0 2024-05-12T03:56:35.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:35.096+0000 7f8e2d7fa700 1 -- 172.21.15.32:0/3245653057 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e24012e90 con 0x7f8e300fbcf0 2024-05-12T03:56:35.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:35.096+0000 7f8e37139700 1 -- 172.21.15.32:0/3245653057 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e30113430 con 0x7f8e300fbcf0 2024-05-12T03:56:35.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:35.098+0000 7f8e2d7fa700 1 -- 172.21.15.32:0/3245653057 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f8e24010850 con 0x7f8e300fbcf0 2024-05-12T03:56:35.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:35.098+0000 7f8e37139700 1 -- 172.21.15.32:0/3245653057 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e3004c710 con 0x7f8e300fbcf0 2024-05-12T03:56:35.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:35.098+0000 7f8e2d7fa700 1 --2- 172.21.15.32:0/3245653057 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f8e200692a0 0x7f8e2006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:56:35.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:35.098+0000 7f8e2d7fa700 1 -- 172.21.15.32:0/3245653057 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f8e24086d60 con 0x7f8e300fbcf0 2024-05-12T03:56:35.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:35.098+0000 7f8e2ffff700 1 --2- 172.21.15.32:0/3245653057 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f8e200692a0 0x7f8e2006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:56:35.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:35.099+0000 7f8e2ffff700 1 --2- 172.21.15.32:0/3245653057 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f8e200692a0 0x7f8e2006b750 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c004890 tx=0x7f8e1c009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:56:35.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:35.103+0000 7f8e2d7fa700 1 -- 172.21.15.32:0/3245653057 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f8e24056f00 con 0x7f8e300fbcf0 2024-05-12T03:56:35.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:35 smithi032 ceph-mon[40008]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:56:35.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:35.377+0000 7f8e37139700 1 -- 172.21.15.32:0/3245653057 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f8e301046a0 con 0x7f8e300fbcf0 2024-05-12T03:56:35.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:35.381+0000 7f8e2d7fa700 1 -- 172.21.15.32:0/3245653057 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v14) v1 ==== 170+0+59 (secure 0 0 0) 0x7f8e24056a90 con 0x7f8e300fbcf0 2024-05-12T03:56:35.383 INFO:teuthology.orchestra.run.smithi032.stdout:[client.0] 2024-05-12T03:56:35.383 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDzPUBm05CnFhAAmZ9zHUeGu713AfHF2xd7Mw== 2024-05-12T03:56:35.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:35.383+0000 7f8e37139700 1 -- 172.21.15.32:0/3245653057 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f8e200692a0 msgr2=0x7f8e2006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:56:35.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:35.383+0000 7f8e37139700 1 --2- 172.21.15.32:0/3245653057 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f8e200692a0 0x7f8e2006b750 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c004890 tx=0x7f8e1c009e30 comp rx=0 tx=0).stop 2024-05-12T03:56:35.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:35.383+0000 7f8e37139700 1 -- 172.21.15.32:0/3245653057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e300fbcf0 msgr2=0x7f8e30065fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:56:35.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:35.383+0000 7f8e37139700 1 --2- 172.21.15.32:0/3245653057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e300fbcf0 0x7f8e30065fe0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8e2400c830 tx=0x7f8e24004470 comp rx=0 tx=0).stop 2024-05-12T03:56:35.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:35.383+0000 7f8e37139700 1 -- 172.21.15.32:0/3245653057 shutdown_connections 2024-05-12T03:56:35.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:35.383+0000 7f8e37139700 1 --2- 172.21.15.32:0/3245653057 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f8e200692a0 0x7f8e2006b750 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:35.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:35.383+0000 7f8e37139700 1 --2- 172.21.15.32:0/3245653057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e300fbcf0 0x7f8e30065fe0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:35.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:35.383+0000 7f8e37139700 1 --2- 172.21.15.32:0/3245653057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e300fcfa0 0x7f8e30112620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:35.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:35.383+0000 7f8e37139700 1 -- 172.21.15.32:0/3245653057 >> 172.21.15.32:0/3245653057 conn(0x7f8e300f7430 msgr2=0x7f8e300f8bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:56:35.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:35.383+0000 7f8e37139700 1 -- 172.21.15.32:0/3245653057 shutdown_connections 2024-05-12T03:56:35.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:35.383+0000 7f8e37139700 1 -- 172.21.15.32:0/3245653057 wait complete. 2024-05-12T03:56:35.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:35 smithi033 ceph-mon[49525]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:56:36.099 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-05-12T03:56:36.100 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-12T03:56:36.100 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-12T03:56:36.146 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-12T03:56:36.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:36 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/3245653057' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-12T03:56:36.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:36 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/3245653057' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-12T03:56:36.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:36 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/3245653057' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-12T03:56:36.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:36 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/3245653057' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-12T03:56:36.433 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi033/config 2024-05-12T03:56:37.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:37 smithi032 ceph-mon[40008]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:56:37.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:37 smithi033 ceph-mon[49525]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:56:38.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:38.294+0000 7f378b1d2700 1 -- 172.21.15.33:0/3350796663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37840fff80 msgr2=0x7f3784102360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:56:38.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:38.294+0000 7f378b1d2700 1 --2- 172.21.15.33:0/3350796663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37840fff80 0x7f3784102360 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3770004940 tx=0x7f3770009e30 comp rx=0 tx=0).stop 2024-05-12T03:56:38.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:38.294+0000 7f378b1d2700 1 -- 172.21.15.33:0/3350796663 shutdown_connections 2024-05-12T03:56:38.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:38.294+0000 7f378b1d2700 1 --2- 172.21.15.33:0/3350796663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37840fff80 0x7f3784102360 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:38.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:38.294+0000 7f378b1d2700 1 --2- 172.21.15.33:0/3350796663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3784066200 0x7f37840ffa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:38.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:38.294+0000 7f378b1d2700 1 -- 172.21.15.33:0/3350796663 >> 172.21.15.33:0/3350796663 conn(0x7f37840f7430 msgr2=0x7f37840f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:56:38.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:38.294+0000 7f378b1d2700 1 -- 172.21.15.33:0/3350796663 shutdown_connections 2024-05-12T03:56:38.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:38.294+0000 7f378b1d2700 1 -- 172.21.15.33:0/3350796663 wait complete. 2024-05-12T03:56:38.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:38.294+0000 7f378b1d2700 1 Processor -- start 2024-05-12T03:56:38.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:38.294+0000 7f378b1d2700 1 -- start start 2024-05-12T03:56:38.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:38.295+0000 7f378b1d2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3784066200 0x7f378410d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:56:38.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:38.295+0000 7f378b1d2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37840fff80 0x7f378410dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:56:38.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:38.295+0000 7f378b1d2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f378410e3e0 con 0x7f37840fff80 2024-05-12T03:56:38.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:38.295+0000 7f378b1d2700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f378410e550 con 0x7f3784066200 2024-05-12T03:56:38.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:38.295+0000 7f3788f6e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3784066200 0x7f378410d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:56:38.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:38.295+0000 7f3788f6e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3784066200 0x7f378410d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33944/0 (socket says 172.21.15.33:33944) 2024-05-12T03:56:38.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:38.295+0000 7f3788f6e700 1 -- 172.21.15.33:0/4057329347 learned_addr learned my addr 172.21.15.33:0/4057329347 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-12T03:56:38.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:38.295+0000 7f3783fff700 1 --2- 172.21.15.33:0/4057329347 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37840fff80 0x7f378410dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:56:38.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:38.295+0000 7f3788f6e700 1 -- 172.21.15.33:0/4057329347 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37840fff80 msgr2=0x7f378410dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:56:38.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:38.295+0000 7f3788f6e700 1 --2- 172.21.15.33:0/4057329347 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37840fff80 0x7f378410dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:38.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:38.295+0000 7f3788f6e700 1 -- 172.21.15.33:0/4057329347 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3770005040 con 0x7f3784066200 2024-05-12T03:56:38.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:38.295+0000 7f3788f6e700 1 --2- 172.21.15.33:0/4057329347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3784066200 0x7f378410d8a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3778002a50 tx=0x7f3778004600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:56:38.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:38.295+0000 7f37817fa700 1 -- 172.21.15.33:0/4057329347 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3778007cc0 con 0x7f3784066200 2024-05-12T03:56:38.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:38.296+0000 7f37817fa700 1 -- 172.21.15.33:0/4057329347 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f377800f040 con 0x7f3784066200 2024-05-12T03:56:38.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:38.296+0000 7f378b1d2700 1 -- 172.21.15.33:0/4057329347 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3784112fb0 con 0x7f3784066200 2024-05-12T03:56:38.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:38.296+0000 7f37817fa700 1 -- 172.21.15.33:0/4057329347 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f377800d990 con 0x7f3784066200 2024-05-12T03:56:38.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:38.296+0000 7f378b1d2700 1 -- 172.21.15.33:0/4057329347 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37841134e0 con 0x7f3784066200 2024-05-12T03:56:38.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:38.296+0000 7f378b1d2700 1 -- 172.21.15.33:0/4057329347 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f378404c710 con 0x7f3784066200 2024-05-12T03:56:38.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:38.297+0000 7f37817fa700 1 -- 172.21.15.33:0/4057329347 <== mon.1 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f3778009710 con 0x7f3784066200 2024-05-12T03:56:38.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:38.297+0000 7f37817fa700 1 --2- 172.21.15.33:0/4057329347 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f37740692a0 0x7f377406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:56:38.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:38.298+0000 7f37817fa700 1 -- 172.21.15.33:0/4057329347 <== mon.1 v2:172.21.15.33:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f3778086bb0 con 0x7f3784066200 2024-05-12T03:56:38.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:38.298+0000 7f3783fff700 1 --2- 172.21.15.33:0/4057329347 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f37740692a0 0x7f377406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:56:38.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:38.298+0000 7f3783fff700 1 --2- 172.21.15.33:0/4057329347 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f37740692a0 0x7f377406b750 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3770000c00 tx=0x7f3770009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:56:38.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:38.302+0000 7f37817fa700 1 -- 172.21.15.33:0/4057329347 <== mon.1 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f377808d080 con 0x7f3784066200 2024-05-12T03:56:38.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:38.577+0000 7f378b1d2700 1 -- 172.21.15.33:0/4057329347 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f3784113760 con 0x7f3784066200 2024-05-12T03:56:38.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:38.581+0000 7f37817fa700 1 -- 172.21.15.33:0/4057329347 <== mon.1 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v15) v1 ==== 170+0+59 (secure 0 0 0) 0x7f37780567c0 con 0x7f3784066200 2024-05-12T03:56:38.582 INFO:teuthology.orchestra.run.smithi033.stdout:[client.1] 2024-05-12T03:56:38.582 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQD2PUBm6G+CIhAA48ZrqVGha6PkssxFVVBB5g== 2024-05-12T03:56:38.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:38.583+0000 7f378b1d2700 1 -- 172.21.15.33:0/4057329347 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f37740692a0 msgr2=0x7f377406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:56:38.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:38.583+0000 7f378b1d2700 1 --2- 172.21.15.33:0/4057329347 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f37740692a0 0x7f377406b750 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3770000c00 tx=0x7f3770009e30 comp rx=0 tx=0).stop 2024-05-12T03:56:38.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:38.583+0000 7f378b1d2700 1 -- 172.21.15.33:0/4057329347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3784066200 msgr2=0x7f378410d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:56:38.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:38.583+0000 7f378b1d2700 1 --2- 172.21.15.33:0/4057329347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3784066200 0x7f378410d8a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3778002a50 tx=0x7f3778004600 comp rx=0 tx=0).stop 2024-05-12T03:56:38.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:38.584+0000 7f378b1d2700 1 -- 172.21.15.33:0/4057329347 shutdown_connections 2024-05-12T03:56:38.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:38.584+0000 7f378b1d2700 1 --2- 172.21.15.33:0/4057329347 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f37740692a0 0x7f377406b750 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:38.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:38.584+0000 7f378b1d2700 1 --2- 172.21.15.33:0/4057329347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3784066200 0x7f378410d8a0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:38.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:38.584+0000 7f378b1d2700 1 --2- 172.21.15.33:0/4057329347 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37840fff80 0x7f378410dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:38.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:38.584+0000 7f378b1d2700 1 -- 172.21.15.33:0/4057329347 >> 172.21.15.33:0/4057329347 conn(0x7f37840f7430 msgr2=0x7f37840f7870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:56:38.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:38.584+0000 7f378b1d2700 1 -- 172.21.15.33:0/4057329347 shutdown_connections 2024-05-12T03:56:38.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-12T03:56:38.584+0000 7f378b1d2700 1 -- 172.21.15.33:0/4057329347 wait complete. 2024-05-12T03:56:39.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:39 smithi032 ceph-mon[40008]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:56:39.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:39 smithi032 ceph-mon[40008]: from='client.? 172.21.15.33:0/4057329347' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-12T03:56:39.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:39 smithi032 ceph-mon[40008]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-12T03:56:39.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:39 smithi032 ceph-mon[40008]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-12T03:56:39.304 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:39 smithi033 ceph-mon[49525]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:56:39.304 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:39 smithi033 ceph-mon[49525]: from='client.? 172.21.15.33:0/4057329347' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-12T03:56:39.304 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:39 smithi033 ceph-mon[49525]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-12T03:56:39.304 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:39 smithi033 ceph-mon[49525]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-12T03:56:39.329 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-12T03:56:39.329 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-05-12T03:56:39.330 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-05-12T03:56:39.375 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-12T03:56:39.375 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-12T03:56:39.375 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- ceph mgr dump --format=json 2024-05-12T03:56:39.664 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T03:56:40.211 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:40 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T03:56:40.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:40 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T03:56:41.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:41 smithi032 ceph-mon[40008]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:56:41.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:41 smithi033 ceph-mon[49525]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:56:41.683 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:41.683+0000 7fafbe781700 1 -- 172.21.15.32:0/282704535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafb80fbd40 msgr2=0x7fafb80fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:56:41.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:41.683+0000 7fafbe781700 1 --2- 172.21.15.32:0/282704535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafb80fbd40 0x7fafb80fc1b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fafac004940 tx=0x7fafac009e30 comp rx=0 tx=0).stop 2024-05-12T03:56:41.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:41.683+0000 7fafbe781700 1 -- 172.21.15.32:0/282704535 shutdown_connections 2024-05-12T03:56:41.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:41.683+0000 7fafbe781700 1 --2- 172.21.15.32:0/282704535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafb80fbd40 0x7fafb80fc1b0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:41.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:41.683+0000 7fafbe781700 1 --2- 172.21.15.32:0/282704535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafb8101d90 0x7fafb8102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:41.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:41.683+0000 7fafbe781700 1 -- 172.21.15.32:0/282704535 >> 172.21.15.32:0/282704535 conn(0x7fafb8071d00 msgr2=0x7fafb8072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:56:41.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:41.683+0000 7fafbe781700 1 -- 172.21.15.32:0/282704535 shutdown_connections 2024-05-12T03:56:41.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:41.683+0000 7fafbe781700 1 -- 172.21.15.32:0/282704535 wait complete. 2024-05-12T03:56:41.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:41.683+0000 7fafbe781700 1 Processor -- start 2024-05-12T03:56:41.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:41.684+0000 7fafbe781700 1 -- start start 2024-05-12T03:56:41.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:41.684+0000 7fafbe781700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafb80fbd40 0x7fafb8102b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:56:41.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:41.684+0000 7fafbe781700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafb8101d90 0x7fafb8103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:56:41.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:41.684+0000 7fafbe781700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb81035f0 con 0x7fafb80fbd40 2024-05-12T03:56:41.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:41.684+0000 7fafbe781700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb8103730 con 0x7fafb8101d90 2024-05-12T03:56:41.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:41.684+0000 7fafb7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafb80fbd40 0x7fafb8102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:56:41.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:41.684+0000 7fafb7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafb80fbd40 0x7fafb8102b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39538/0 (socket says 172.21.15.32:39538) 2024-05-12T03:56:41.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:41.684+0000 7fafb7fff700 1 -- 172.21.15.32:0/975438936 learned_addr learned my addr 172.21.15.32:0/975438936 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:56:41.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:41.684+0000 7fafb77fe700 1 --2- 172.21.15.32:0/975438936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafb8101d90 0x7fafb8103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:56:41.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:41.684+0000 7fafb7fff700 1 -- 172.21.15.32:0/975438936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafb8101d90 msgr2=0x7fafb8103060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:56:41.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:41.684+0000 7fafb7fff700 1 --2- 172.21.15.32:0/975438936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafb8101d90 0x7fafb8103060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:41.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:41.684+0000 7fafb7fff700 1 -- 172.21.15.32:0/975438936 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafac005040 con 0x7fafb80fbd40 2024-05-12T03:56:41.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:41.684+0000 7fafb7fff700 1 --2- 172.21.15.32:0/975438936 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafb80fbd40 0x7fafb8102b20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fafa8002a50 tx=0x7fafa80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:56:41.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:41.685+0000 7fafb4ff9700 1 -- 172.21.15.32:0/975438936 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fafa8007cc0 con 0x7fafb80fbd40 2024-05-12T03:56:41.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:41.685+0000 7fafbe781700 1 -- 172.21.15.32:0/975438936 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafb8119db0 con 0x7fafb80fbd40 2024-05-12T03:56:41.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:41.685+0000 7fafb4ff9700 1 -- 172.21.15.32:0/975438936 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fafa800f040 con 0x7fafb80fbd40 2024-05-12T03:56:41.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:41.685+0000 7fafb4ff9700 1 -- 172.21.15.32:0/975438936 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fafa800d9c0 con 0x7fafb80fbd40 2024-05-12T03:56:41.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:41.685+0000 7fafbe781700 1 -- 172.21.15.32:0/975438936 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafb811a230 con 0x7fafb80fbd40 2024-05-12T03:56:41.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:41.686+0000 7fafbe781700 1 -- 172.21.15.32:0/975438936 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafb80fd430 con 0x7fafb80fbd40 2024-05-12T03:56:41.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:41.686+0000 7fafb4ff9700 1 -- 172.21.15.32:0/975438936 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fafa800db60 con 0x7fafb80fbd40 2024-05-12T03:56:41.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:41.686+0000 7fafb4ff9700 1 --2- 172.21.15.32:0/975438936 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fafa00692a0 0x7fafa006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:56:41.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:41.687+0000 7fafb4ff9700 1 -- 172.21.15.32:0/975438936 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fafa80864d0 con 0x7fafb80fbd40 2024-05-12T03:56:41.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:41.687+0000 7fafb77fe700 1 --2- 172.21.15.32:0/975438936 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fafa00692a0 0x7fafa006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:56:41.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:41.687+0000 7fafb77fe700 1 --2- 172.21.15.32:0/975438936 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fafa00692a0 0x7fafa006b750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fafb8104460 tx=0x7fafac009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:56:41.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:41.691+0000 7fafb4ff9700 1 -- 172.21.15.32:0/975438936 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fafa804d250 con 0x7fafb80fbd40 2024-05-12T03:56:41.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:41.943+0000 7fafbe781700 1 -- 172.21.15.32:0/975438936 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fafb8104210 con 0x7fafb80fbd40 2024-05-12T03:56:41.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:41.946+0000 7fafb4ff9700 1 -- 172.21.15.32:0/975438936 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v19) v1 ==== 74+0+167178 (secure 0 0 0) 0x7fafa804ad20 con 0x7fafb80fbd40 2024-05-12T03:56:41.948 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:56:41.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:41.948+0000 7fafbe781700 1 -- 172.21.15.32:0/975438936 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fafa00692a0 msgr2=0x7fafa006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:56:41.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:41.948+0000 7fafbe781700 1 --2- 172.21.15.32:0/975438936 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fafa00692a0 0x7fafa006b750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fafb8104460 tx=0x7fafac009e30 comp rx=0 tx=0).stop 2024-05-12T03:56:41.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:41.948+0000 7fafbe781700 1 -- 172.21.15.32:0/975438936 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafb80fbd40 msgr2=0x7fafb8102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:56:41.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:41.948+0000 7fafbe781700 1 --2- 172.21.15.32:0/975438936 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafb80fbd40 0x7fafb8102b20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fafa8002a50 tx=0x7fafa80046b0 comp rx=0 tx=0).stop 2024-05-12T03:56:41.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:41.948+0000 7fafbe781700 1 -- 172.21.15.32:0/975438936 shutdown_connections 2024-05-12T03:56:41.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:41.948+0000 7fafbe781700 1 --2- 172.21.15.32:0/975438936 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fafa00692a0 0x7fafa006b750 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:41.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:41.948+0000 7fafbe781700 1 --2- 172.21.15.32:0/975438936 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafb80fbd40 0x7fafb8102b20 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:41.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:41.948+0000 7fafbe781700 1 --2- 172.21.15.32:0/975438936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafb8101d90 0x7fafb8103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:41.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:41.948+0000 7fafbe781700 1 -- 172.21.15.32:0/975438936 >> 172.21.15.32:0/975438936 conn(0x7fafb8071d00 msgr2=0x7fafb80655e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:56:41.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:41.949+0000 7fafbe781700 1 -- 172.21.15.32:0/975438936 shutdown_connections 2024-05-12T03:56:41.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:41.949+0000 7fafbe781700 1 -- 172.21.15.32:0/975438936 wait complete. 2024-05-12T03:56:42.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:42 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/975438936' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-12T03:56:42.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:42 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/975438936' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-12T03:56:42.767 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":19,"flags":0,"active_gid":14208,"active_name":"smithi032.ukatkq","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":4089797100},{"type":"v1","addr":"172.21.15.32:6801","nonce":4089797100}]},"active_addr":"172.21.15.32:6801/4089797100","active_change":"2024-05-12T03:52:39.772838+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":14228,"name":"smithi033.pcvxwy","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.32:8443/","prometheus":"http://172.21.15.32:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":2236672133}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":1834879835}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":993128492}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":3343483246}]}]} 2024-05-12T03:56:42.770 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-12T03:56:42.770 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-12T03:56:42.771 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-12T03:56:43.055 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T03:56:43.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:43 smithi032 ceph-mon[40008]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:56:43.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:43 smithi033 ceph-mon[49525]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:56:45.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:45.109+0000 7f810cdad700 1 -- 172.21.15.32:0/4124045077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8108102e50 msgr2=0x7f8108103220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:56:45.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:45.109+0000 7f810cdad700 1 --2- 172.21.15.32:0/4124045077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8108102e50 0x7f8108103220 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f80f8004940 tx=0x7f80f8009e30 comp rx=0 tx=0).stop 2024-05-12T03:56:45.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:45.109+0000 7f810cdad700 1 -- 172.21.15.32:0/4124045077 shutdown_connections 2024-05-12T03:56:45.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:45.109+0000 7f810cdad700 1 --2- 172.21.15.32:0/4124045077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8108065640 0x7f8108065ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:45.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:45.109+0000 7f810cdad700 1 --2- 172.21.15.32:0/4124045077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8108102e50 0x7f8108103220 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:45.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:45.109+0000 7f810cdad700 1 -- 172.21.15.32:0/4124045077 >> 172.21.15.32:0/4124045077 conn(0x7f8108071d00 msgr2=0x7f8108072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:56:45.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:45.109+0000 7f810cdad700 1 -- 172.21.15.32:0/4124045077 shutdown_connections 2024-05-12T03:56:45.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:45.109+0000 7f810cdad700 1 -- 172.21.15.32:0/4124045077 wait complete. 2024-05-12T03:56:45.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:45.110+0000 7f810cdad700 1 Processor -- start 2024-05-12T03:56:45.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:45.110+0000 7f810cdad700 1 -- start start 2024-05-12T03:56:45.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:45.110+0000 7f810cdad700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8108065640 0x7f8108113a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:56:45.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:45.110+0000 7f810cdad700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8108102e50 0x7f8108113fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:56:45.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:45.110+0000 7f810cdad700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8108114620 con 0x7f8108065640 2024-05-12T03:56:45.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:45.110+0000 7f810cdad700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f810810da10 con 0x7f8108102e50 2024-05-12T03:56:45.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:45.110+0000 7f810659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8108065640 0x7f8108113a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:56:45.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:45.110+0000 7f810659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8108065640 0x7f8108113a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55024/0 (socket says 172.21.15.32:55024) 2024-05-12T03:56:45.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:45.110+0000 7f810659c700 1 -- 172.21.15.32:0/522568352 learned_addr learned my addr 172.21.15.32:0/522568352 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:56:45.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:45.110+0000 7f810659c700 1 -- 172.21.15.32:0/522568352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8108102e50 msgr2=0x7f8108113fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:56:45.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:45.110+0000 7f810659c700 1 --2- 172.21.15.32:0/522568352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8108102e50 0x7f8108113fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:45.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:45.110+0000 7f810659c700 1 -- 172.21.15.32:0/522568352 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80f8005040 con 0x7f8108065640 2024-05-12T03:56:45.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:45.110+0000 7f810659c700 1 --2- 172.21.15.32:0/522568352 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8108065640 0x7f8108113a90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f80f8009390 tx=0x7f80f8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:56:45.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:45.111+0000 7f80f6ffd700 1 -- 172.21.15.32:0/522568352 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80f800e040 con 0x7f8108065640 2024-05-12T03:56:45.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:45.111+0000 7f80f6ffd700 1 -- 172.21.15.32:0/522568352 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f80f8013420 con 0x7f8108065640 2024-05-12T03:56:45.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:45.111+0000 7f810cdad700 1 -- 172.21.15.32:0/522568352 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f810810dc90 con 0x7f8108065640 2024-05-12T03:56:45.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:45.111+0000 7f80f6ffd700 1 -- 172.21.15.32:0/522568352 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80f801c630 con 0x7f8108065640 2024-05-12T03:56:45.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:45.111+0000 7f810cdad700 1 -- 172.21.15.32:0/522568352 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f810810e250 con 0x7f8108065640 2024-05-12T03:56:45.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:45.112+0000 7f810cdad700 1 -- 172.21.15.32:0/522568352 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81080fe540 con 0x7f8108065640 2024-05-12T03:56:45.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:45.112+0000 7f80f6ffd700 1 -- 172.21.15.32:0/522568352 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f80f80139e0 con 0x7f8108065640 2024-05-12T03:56:45.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:45.113+0000 7f80f6ffd700 1 --2- 172.21.15.32:0/522568352 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f80f00692b0 0x7f80f006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:56:45.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:45.113+0000 7f80f6ffd700 1 -- 172.21.15.32:0/522568352 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f80f80880d0 con 0x7f8108065640 2024-05-12T03:56:45.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:45.113+0000 7f8105d9b700 1 --2- 172.21.15.32:0/522568352 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f80f00692b0 0x7f80f006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:56:45.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:45.113+0000 7f8105d9b700 1 --2- 172.21.15.32:0/522568352 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f80f00692b0 0x7f80f006b760 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f810810ebd0 tx=0x7f80fc009ce0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:56:45.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:45.117+0000 7f80f6ffd700 1 -- 172.21.15.32:0/522568352 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f80f8058260 con 0x7f8108065640 2024-05-12T03:56:45.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:45 smithi032 ceph-mon[40008]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:56:45.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:45.313+0000 7f810cdad700 1 -- 172.21.15.32:0/522568352 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f810804bed0 con 0x7f8108065640 2024-05-12T03:56:45.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:45.314+0000 7f80f6ffd700 1 -- 172.21.15.32:0/522568352 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11059 (secure 0 0 0) 0x7f80f8057df0 con 0x7f8108065640 2024-05-12T03:56:45.314 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:56:45.314 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":40,"fsid":"a3e178f8-1012-11ef-bc98-c7b262605968","created":"2024-05-12T03:50:18.356038+0000","modified":"2024-05-12T03:56:24.885995+0000","last_up_change":"2024-05-12T03:56:23.877862+0000","last_in_change":"2024-05-12T03:56:03.301729+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-12T03:54:58.296702+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"fc534129-ef70-487d-b8d1-f98fc170c1d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":3497755677},{"type":"v1","addr":"172.21.15.32:6803","nonce":3497755677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":3497755677},{"type":"v1","addr":"172.21.15.32:6805","nonce":3497755677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":3497755677},{"type":"v1","addr":"172.21.15.32:6809","nonce":3497755677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":3497755677},{"type":"v1","addr":"172.21.15.32:6807","nonce":3497755677}]},"public_addr":"172.21.15.32:6803/3497755677","cluster_addr":"172.21.15.32:6805/3497755677","heartbeat_back_addr":"172.21.15.32:6809/3497755677","heartbeat_front_addr":"172.21.15.32:6807/3497755677","state":["exists","up"]},{"osd":1,"uuid":"f8c35b6a-5afc-447b-b318-d56097de28d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":3065624488},{"type":"v1","addr":"172.21.15.32:6811","nonce":3065624488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":3065624488},{"type":"v1","addr":"172.21.15.32:6813","nonce":3065624488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3065624488},{"type":"v1","addr":"172.21.15.32:6817","nonce":3065624488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":3065624488},{"type":"v1","addr":"172.21.15.32:6815","nonce":3065624488}]},"public_addr":"172.21.15.32:6811/3065624488","cluster_addr":"172.21.15.32:6813/3065624488","heartbeat_back_addr":"172.21.15.32:6817/3065624488","heartbeat_front_addr":"172.21.15.32:6815/3065624488","state":["exists","up"]},{"osd":2,"uuid":"4ed4b55d-cceb-494b-bdff-fef99d6c8cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3261757113},{"type":"v1","addr":"172.21.15.32:6819","nonce":3261757113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":3261757113},{"type":"v1","addr":"172.21.15.32:6821","nonce":3261757113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":3261757113},{"type":"v1","addr":"172.21.15.32:6825","nonce":3261757113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3261757113},{"type":"v1","addr":"172.21.15.32:6823","nonce":3261757113}]},"public_addr":"172.21.15.32:6819/3261757113","cluster_addr":"172.21.15.32:6821/3261757113","heartbeat_back_addr":"172.21.15.32:6825/3261757113","heartbeat_front_addr":"172.21.15.32:6823/3261757113","state":["exists","up"]},{"osd":3,"uuid":"ef191267-c0a1-4be7-9e8b-761700af87a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2331766478},{"type":"v1","addr":"172.21.15.33:6801","nonce":2331766478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2331766478},{"type":"v1","addr":"172.21.15.33:6803","nonce":2331766478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2331766478},{"type":"v1","addr":"172.21.15.33:6807","nonce":2331766478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2331766478},{"type":"v1","addr":"172.21.15.33:6805","nonce":2331766478}]},"public_addr":"172.21.15.33:6801/2331766478","cluster_addr":"172.21.15.33:6803/2331766478","heartbeat_back_addr":"172.21.15.33:6807/2331766478","heartbeat_front_addr":"172.21.15.33:6805/2331766478","state":["exists","up"]},{"osd":4,"uuid":"fe45f168-076a-4888-895d-bb710bd0da73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":2404279705},{"type":"v1","addr":"172.21.15.33:6809","nonce":2404279705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":2404279705},{"type":"v1","addr":"172.21.15.33:6811","nonce":2404279705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":2404279705},{"type":"v1","addr":"172.21.15.33:6815","nonce":2404279705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":2404279705},{"type":"v1","addr":"172.21.15.33:6813","nonce":2404279705}]},"public_addr":"172.21.15.33:6809/2404279705","cluster_addr":"172.21.15.33:6811/2404279705","heartbeat_back_addr":"172.21.15.33:6815/2404279705","heartbeat_front_addr":"172.21.15.33:6813/2404279705","state":["exists","up"]},{"osd":5,"uuid":"f7a0e14e-a866-4163-a7df-f8a43b860bc0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2123556026},{"type":"v1","addr":"172.21.15.33:6817","nonce":2123556026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":2123556026},{"type":"v1","addr":"172.21.15.33:6819","nonce":2123556026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":2123556026},{"type":"v1","addr":"172.21.15.33:6823","nonce":2123556026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2123556026},{"type":"v1","addr":"172.21.15.33:6821","nonce":2123556026}]},"public_addr":"172.21.15.33:6817/2123556026","cluster_addr":"172.21.15.33:6819/2123556026","heartbeat_back_addr":"172.21.15.33:6823/2123556026","heartbeat_front_addr":"172.21.15.33:6821/2123556026","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-12T03:53:56.134194+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-12T03:54:25.231975+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-12T03:54:54.991052+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-12T03:55:22.680656+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-12T03:55:52.446678+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-12T03:56:22.128957+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6800/1988399583":"2024-05-13T03:52:39.772717+0000","172.21.15.32:0/3185268518":"2024-05-13T03:52:39.772717+0000","172.21.15.32:0/1073406471":"2024-05-13T03:52:39.772717+0000","172.21.15.32:6801/2967201790":"2024-05-13T03:51:25.269581+0000","172.21.15.32:6800/2967201790":"2024-05-13T03:51:25.269581+0000","172.21.15.32:6801/1988399583":"2024-05-13T03:52:39.772717+0000","172.21.15.32:0/2900983644":"2024-05-13T03:52:39.772717+0000","172.21.15.32:0/165225041":"2024-05-13T03:51:25.269581+0000","172.21.15.32:0/4293156271":"2024-05-13T03:51:25.269581+0000","172.21.15.32:6800/1121526649":"2024-05-13T03:50:49.378828+0000","172.21.15.32:0/2098237559":"2024-05-13T03:50:49.378828+0000","172.21.15.32:0/1463926533":"2024-05-13T03:50:49.378828+0000","172.21.15.32:6801/1121526649":"2024-05-13T03:50:49.378828+0000","172.21.15.32:0/625670655":"2024-05-13T03:50:49.378828+0000","172.21.15.32:0/1356801576":"2024-05-13T03:51:25.269581+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-12T03:56:45.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:45.316+0000 7f810cdad700 1 -- 172.21.15.32:0/522568352 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f80f00692b0 msgr2=0x7f80f006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:56:45.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:45.316+0000 7f810cdad700 1 --2- 172.21.15.32:0/522568352 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f80f00692b0 0x7f80f006b760 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f810810ebd0 tx=0x7f80fc009ce0 comp rx=0 tx=0).stop 2024-05-12T03:56:45.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:45.316+0000 7f810cdad700 1 -- 172.21.15.32:0/522568352 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8108065640 msgr2=0x7f8108113a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:56:45.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:45.316+0000 7f810cdad700 1 --2- 172.21.15.32:0/522568352 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8108065640 0x7f8108113a90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f80f8009390 tx=0x7f80f8009e30 comp rx=0 tx=0).stop 2024-05-12T03:56:45.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:45.316+0000 7f810cdad700 1 -- 172.21.15.32:0/522568352 shutdown_connections 2024-05-12T03:56:45.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:45.316+0000 7f810cdad700 1 --2- 172.21.15.32:0/522568352 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f80f00692b0 0x7f80f006b760 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:45.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:45.316+0000 7f810cdad700 1 --2- 172.21.15.32:0/522568352 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8108065640 0x7f8108113a90 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:45.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:45.316+0000 7f810cdad700 1 --2- 172.21.15.32:0/522568352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8108102e50 0x7f8108113fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:45.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:45.316+0000 7f810cdad700 1 -- 172.21.15.32:0/522568352 >> 172.21.15.32:0/522568352 conn(0x7f8108071d00 msgr2=0x7f8108105b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:56:45.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:45.316+0000 7f810cdad700 1 -- 172.21.15.32:0/522568352 shutdown_connections 2024-05-12T03:56:45.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:45.316+0000 7f810cdad700 1 -- 172.21.15.32:0/522568352 wait complete. 2024-05-12T03:56:45.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:45 smithi033 ceph-mon[49525]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:56:46.108 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-12T03:56:46.108 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-12T03:56:46.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:46 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/522568352' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-12T03:56:46.388 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T03:56:46.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:46 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/522568352' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-12T03:56:47.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:47 smithi033 ceph-mon[49525]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:56:47.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:47 smithi032 ceph-mon[40008]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:56:48.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:48.375+0000 7f7dd3006700 1 -- 172.21.15.32:0/249975868 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7dcc101ce0 msgr2=0x7f7dcc1020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:56:48.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:48.375+0000 7f7dd3006700 1 --2- 172.21.15.32:0/249975868 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7dcc101ce0 0x7f7dcc1020b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0004940 tx=0x7f7dc0009e30 comp rx=0 tx=0).stop 2024-05-12T03:56:48.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:48.375+0000 7f7dd3006700 1 -- 172.21.15.32:0/249975868 shutdown_connections 2024-05-12T03:56:48.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:48.375+0000 7f7dd3006700 1 --2- 172.21.15.32:0/249975868 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7dcc102680 0x7f7dcc0fbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:48.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:48.375+0000 7f7dd3006700 1 --2- 172.21.15.32:0/249975868 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7dcc101ce0 0x7f7dcc1020b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:48.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:48.375+0000 7f7dd3006700 1 -- 172.21.15.32:0/249975868 >> 172.21.15.32:0/249975868 conn(0x7f7dcc0f7850 msgr2=0x7f7dcc0f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:56:48.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:48.375+0000 7f7dd3006700 1 -- 172.21.15.32:0/249975868 shutdown_connections 2024-05-12T03:56:48.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:48.375+0000 7f7dd3006700 1 -- 172.21.15.32:0/249975868 wait complete. 2024-05-12T03:56:48.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:48.375+0000 7f7dd3006700 1 Processor -- start 2024-05-12T03:56:48.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:48.375+0000 7f7dd3006700 1 -- start start 2024-05-12T03:56:48.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:48.376+0000 7f7dd3006700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7dcc101ce0 0x7f7dcc10ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:56:48.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:48.376+0000 7f7dd3006700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7dcc102680 0x7f7dcc10ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:56:48.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:48.376+0000 7f7dd3006700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dcc10f5a0 con 0x7f7dcc102680 2024-05-12T03:56:48.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:48.376+0000 7f7dd3006700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dcc10f6e0 con 0x7f7dcc101ce0 2024-05-12T03:56:48.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:48.376+0000 7f7dcbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7dcc102680 0x7f7dcc10ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:56:48.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:48.376+0000 7f7dcbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7dcc102680 0x7f7dcc10ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55054/0 (socket says 172.21.15.32:55054) 2024-05-12T03:56:48.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:48.376+0000 7f7dcbfff700 1 -- 172.21.15.32:0/3656410001 learned_addr learned my addr 172.21.15.32:0/3656410001 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:56:48.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:48.376+0000 7f7dd0da2700 1 --2- 172.21.15.32:0/3656410001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7dcc101ce0 0x7f7dcc10ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:56:48.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:48.376+0000 7f7dcbfff700 1 -- 172.21.15.32:0/3656410001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7dcc101ce0 msgr2=0x7f7dcc10ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:56:48.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:48.376+0000 7f7dcbfff700 1 --2- 172.21.15.32:0/3656410001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7dcc101ce0 0x7f7dcc10ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:48.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:48.376+0000 7f7dcbfff700 1 -- 172.21.15.32:0/3656410001 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dc0005040 con 0x7f7dcc102680 2024-05-12T03:56:48.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:48.376+0000 7f7dd0da2700 1 --2- 172.21.15.32:0/3656410001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7dcc101ce0 0x7f7dcc10ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-12T03:56:48.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:48.376+0000 7f7dcbfff700 1 --2- 172.21.15.32:0/3656410001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7dcc102680 0x7f7dcc10ef50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7db800c890 tx=0x7f7db80044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:56:48.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:48.377+0000 7f7dc97fa700 1 -- 172.21.15.32:0/3656410001 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7db800a7d0 con 0x7f7dcc102680 2024-05-12T03:56:48.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:48.377+0000 7f7dd3006700 1 -- 172.21.15.32:0/3656410001 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dcc117a80 con 0x7f7dcc102680 2024-05-12T03:56:48.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:48.377+0000 7f7dc97fa700 1 -- 172.21.15.32:0/3656410001 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7db800ade0 con 0x7f7dcc102680 2024-05-12T03:56:48.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:48.377+0000 7f7dc97fa700 1 -- 172.21.15.32:0/3656410001 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7db80113a0 con 0x7f7dcc102680 2024-05-12T03:56:48.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:48.377+0000 7f7dd3006700 1 -- 172.21.15.32:0/3656410001 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dcc117fe0 con 0x7f7dcc102680 2024-05-12T03:56:48.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:48.378+0000 7f7dd3006700 1 -- 172.21.15.32:0/3656410001 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dcc0fd430 con 0x7f7dcc102680 2024-05-12T03:56:48.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:48.378+0000 7f7dc97fa700 1 -- 172.21.15.32:0/3656410001 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f7db8012400 con 0x7f7dcc102680 2024-05-12T03:56:48.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:48.379+0000 7f7dc97fa700 1 --2- 172.21.15.32:0/3656410001 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f7dbc0692a0 0x7f7dbc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:56:48.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:48.379+0000 7f7dc97fa700 1 -- 172.21.15.32:0/3656410001 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f7db8086f60 con 0x7f7dcc102680 2024-05-12T03:56:48.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:48.379+0000 7f7dd0da2700 1 --2- 172.21.15.32:0/3656410001 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f7dbc0692a0 0x7f7dbc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:56:48.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:48.379+0000 7f7dd0da2700 1 --2- 172.21.15.32:0/3656410001 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f7dbc0692a0 0x7f7dbc06b750 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0004560 tx=0x7f7dc0009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:56:48.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:48.383+0000 7f7dc97fa700 1 -- 172.21.15.32:0/3656410001 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f7db80535d0 con 0x7f7dcc102680 2024-05-12T03:56:48.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:48.577+0000 7f7dd3006700 1 -- 172.21.15.32:0/3656410001 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7dcc05f5c0 con 0x7f7dcc102680 2024-05-12T03:56:48.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:48.578+0000 7f7dc97fa700 1 -- 172.21.15.32:0/3656410001 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11059 (secure 0 0 0) 0x7f7db8056bf0 con 0x7f7dcc102680 2024-05-12T03:56:48.578 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:56:48.578 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":40,"fsid":"a3e178f8-1012-11ef-bc98-c7b262605968","created":"2024-05-12T03:50:18.356038+0000","modified":"2024-05-12T03:56:24.885995+0000","last_up_change":"2024-05-12T03:56:23.877862+0000","last_in_change":"2024-05-12T03:56:03.301729+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-12T03:54:58.296702+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"fc534129-ef70-487d-b8d1-f98fc170c1d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":3497755677},{"type":"v1","addr":"172.21.15.32:6803","nonce":3497755677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":3497755677},{"type":"v1","addr":"172.21.15.32:6805","nonce":3497755677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":3497755677},{"type":"v1","addr":"172.21.15.32:6809","nonce":3497755677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":3497755677},{"type":"v1","addr":"172.21.15.32:6807","nonce":3497755677}]},"public_addr":"172.21.15.32:6803/3497755677","cluster_addr":"172.21.15.32:6805/3497755677","heartbeat_back_addr":"172.21.15.32:6809/3497755677","heartbeat_front_addr":"172.21.15.32:6807/3497755677","state":["exists","up"]},{"osd":1,"uuid":"f8c35b6a-5afc-447b-b318-d56097de28d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":3065624488},{"type":"v1","addr":"172.21.15.32:6811","nonce":3065624488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":3065624488},{"type":"v1","addr":"172.21.15.32:6813","nonce":3065624488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3065624488},{"type":"v1","addr":"172.21.15.32:6817","nonce":3065624488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":3065624488},{"type":"v1","addr":"172.21.15.32:6815","nonce":3065624488}]},"public_addr":"172.21.15.32:6811/3065624488","cluster_addr":"172.21.15.32:6813/3065624488","heartbeat_back_addr":"172.21.15.32:6817/3065624488","heartbeat_front_addr":"172.21.15.32:6815/3065624488","state":["exists","up"]},{"osd":2,"uuid":"4ed4b55d-cceb-494b-bdff-fef99d6c8cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3261757113},{"type":"v1","addr":"172.21.15.32:6819","nonce":3261757113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":3261757113},{"type":"v1","addr":"172.21.15.32:6821","nonce":3261757113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":3261757113},{"type":"v1","addr":"172.21.15.32:6825","nonce":3261757113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3261757113},{"type":"v1","addr":"172.21.15.32:6823","nonce":3261757113}]},"public_addr":"172.21.15.32:6819/3261757113","cluster_addr":"172.21.15.32:6821/3261757113","heartbeat_back_addr":"172.21.15.32:6825/3261757113","heartbeat_front_addr":"172.21.15.32:6823/3261757113","state":["exists","up"]},{"osd":3,"uuid":"ef191267-c0a1-4be7-9e8b-761700af87a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2331766478},{"type":"v1","addr":"172.21.15.33:6801","nonce":2331766478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2331766478},{"type":"v1","addr":"172.21.15.33:6803","nonce":2331766478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2331766478},{"type":"v1","addr":"172.21.15.33:6807","nonce":2331766478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2331766478},{"type":"v1","addr":"172.21.15.33:6805","nonce":2331766478}]},"public_addr":"172.21.15.33:6801/2331766478","cluster_addr":"172.21.15.33:6803/2331766478","heartbeat_back_addr":"172.21.15.33:6807/2331766478","heartbeat_front_addr":"172.21.15.33:6805/2331766478","state":["exists","up"]},{"osd":4,"uuid":"fe45f168-076a-4888-895d-bb710bd0da73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":2404279705},{"type":"v1","addr":"172.21.15.33:6809","nonce":2404279705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":2404279705},{"type":"v1","addr":"172.21.15.33:6811","nonce":2404279705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":2404279705},{"type":"v1","addr":"172.21.15.33:6815","nonce":2404279705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":2404279705},{"type":"v1","addr":"172.21.15.33:6813","nonce":2404279705}]},"public_addr":"172.21.15.33:6809/2404279705","cluster_addr":"172.21.15.33:6811/2404279705","heartbeat_back_addr":"172.21.15.33:6815/2404279705","heartbeat_front_addr":"172.21.15.33:6813/2404279705","state":["exists","up"]},{"osd":5,"uuid":"f7a0e14e-a866-4163-a7df-f8a43b860bc0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2123556026},{"type":"v1","addr":"172.21.15.33:6817","nonce":2123556026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":2123556026},{"type":"v1","addr":"172.21.15.33:6819","nonce":2123556026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":2123556026},{"type":"v1","addr":"172.21.15.33:6823","nonce":2123556026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2123556026},{"type":"v1","addr":"172.21.15.33:6821","nonce":2123556026}]},"public_addr":"172.21.15.33:6817/2123556026","cluster_addr":"172.21.15.33:6819/2123556026","heartbeat_back_addr":"172.21.15.33:6823/2123556026","heartbeat_front_addr":"172.21.15.33:6821/2123556026","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-12T03:53:56.134194+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-12T03:54:25.231975+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-12T03:54:54.991052+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-12T03:55:22.680656+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-12T03:55:52.446678+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-12T03:56:22.128957+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6800/1988399583":"2024-05-13T03:52:39.772717+0000","172.21.15.32:0/3185268518":"2024-05-13T03:52:39.772717+0000","172.21.15.32:0/1073406471":"2024-05-13T03:52:39.772717+0000","172.21.15.32:6801/2967201790":"2024-05-13T03:51:25.269581+0000","172.21.15.32:6800/2967201790":"2024-05-13T03:51:25.269581+0000","172.21.15.32:6801/1988399583":"2024-05-13T03:52:39.772717+0000","172.21.15.32:0/2900983644":"2024-05-13T03:52:39.772717+0000","172.21.15.32:0/165225041":"2024-05-13T03:51:25.269581+0000","172.21.15.32:0/4293156271":"2024-05-13T03:51:25.269581+0000","172.21.15.32:6800/1121526649":"2024-05-13T03:50:49.378828+0000","172.21.15.32:0/2098237559":"2024-05-13T03:50:49.378828+0000","172.21.15.32:0/1463926533":"2024-05-13T03:50:49.378828+0000","172.21.15.32:6801/1121526649":"2024-05-13T03:50:49.378828+0000","172.21.15.32:0/625670655":"2024-05-13T03:50:49.378828+0000","172.21.15.32:0/1356801576":"2024-05-13T03:51:25.269581+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-12T03:56:48.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:48.580+0000 7f7dd3006700 1 -- 172.21.15.32:0/3656410001 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f7dbc0692a0 msgr2=0x7f7dbc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:56:48.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:48.580+0000 7f7dd3006700 1 --2- 172.21.15.32:0/3656410001 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f7dbc0692a0 0x7f7dbc06b750 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0004560 tx=0x7f7dc0009e30 comp rx=0 tx=0).stop 2024-05-12T03:56:48.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:48.580+0000 7f7dd3006700 1 -- 172.21.15.32:0/3656410001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7dcc102680 msgr2=0x7f7dcc10ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:56:48.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:48.580+0000 7f7dd3006700 1 --2- 172.21.15.32:0/3656410001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7dcc102680 0x7f7dcc10ef50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7db800c890 tx=0x7f7db80044c0 comp rx=0 tx=0).stop 2024-05-12T03:56:48.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:48.580+0000 7f7dd3006700 1 -- 172.21.15.32:0/3656410001 shutdown_connections 2024-05-12T03:56:48.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:48.580+0000 7f7dd3006700 1 --2- 172.21.15.32:0/3656410001 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f7dbc0692a0 0x7f7dbc06b750 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:48.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:48.580+0000 7f7dd3006700 1 --2- 172.21.15.32:0/3656410001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7dcc101ce0 0x7f7dcc10ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:48.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:48.580+0000 7f7dd3006700 1 --2- 172.21.15.32:0/3656410001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7dcc102680 0x7f7dcc10ef50 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:48.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:48.580+0000 7f7dd3006700 1 -- 172.21.15.32:0/3656410001 >> 172.21.15.32:0/3656410001 conn(0x7f7dcc0f7850 msgr2=0x7f7dcc0f8940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:56:48.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:48.580+0000 7f7dd3006700 1 -- 172.21.15.32:0/3656410001 shutdown_connections 2024-05-12T03:56:48.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:48.580+0000 7f7dd3006700 1 -- 172.21.15.32:0/3656410001 wait complete. 2024-05-12T03:56:49.240 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:49 smithi032 ceph-mon[40008]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:56:49.240 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:49 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/3656410001' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-12T03:56:49.262 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-12T03:56:49.263 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-12T03:56:49.263 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-12T03:56:49.264 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-12T03:56:49.264 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-12T03:56:49.265 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-12T03:56:49.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:49 smithi033 ceph-mon[49525]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:56:49.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:49 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/3656410001' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-12T03:56:49.611 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T03:56:49.639 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T03:56:51.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:51 smithi033 ceph-mon[49525]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:56:51.457 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:51 smithi032 ceph-mon[40008]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:56:51.990 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T03:56:51.991 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T03:56:51.991 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T03:56:51.992 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T03:56:52.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:52.217+0000 7fa7bd2ba700 1 -- 172.21.15.32:0/769835242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7b8102de0 msgr2=0x7fa7b81031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:56:52.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:52.217+0000 7fa7bd2ba700 1 --2- 172.21.15.32:0/769835242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7b8102de0 0x7fa7b81031b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8004940 tx=0x7fa7a8009e30 comp rx=0 tx=0).stop 2024-05-12T03:56:52.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:52.217+0000 7fa7bd2ba700 1 -- 172.21.15.32:0/769835242 shutdown_connections 2024-05-12T03:56:52.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:52.217+0000 7fa7bd2ba700 1 --2- 172.21.15.32:0/769835242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7b8103780 0x7fa7b80fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:52.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:52.217+0000 7fa7bd2ba700 1 --2- 172.21.15.32:0/769835242 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7b8102de0 0x7fa7b81031b0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:52.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:52.217+0000 7fa7bd2ba700 1 -- 172.21.15.32:0/769835242 >> 172.21.15.32:0/769835242 conn(0x7fa7b80f8990 msgr2=0x7fa7b80fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:56:52.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:52.217+0000 7fa7bd2ba700 1 -- 172.21.15.32:0/769835242 shutdown_connections 2024-05-12T03:56:52.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:52.217+0000 7fa7bd2ba700 1 -- 172.21.15.32:0/769835242 wait complete. 2024-05-12T03:56:52.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:52.217+0000 7fa7bd2ba700 1 Processor -- start 2024-05-12T03:56:52.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:52.217+0000 7fa7bd2ba700 1 -- start start 2024-05-12T03:56:52.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:52.218+0000 7fa7bd2ba700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7b8102de0 0x7fa7b8113890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:56:52.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:52.218+0000 7fa7bd2ba700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7b8103780 0x7fa7b8113dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:56:52.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:52.218+0000 7fa7bd2ba700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b8114420 con 0x7fa7b8102de0 2024-05-12T03:56:52.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:52.218+0000 7fa7bd2ba700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b810da20 con 0x7fa7b8103780 2024-05-12T03:56:52.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:52.218+0000 7fa7b6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7b8102de0 0x7fa7b8113890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:56:52.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:52.218+0000 7fa7b6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7b8102de0 0x7fa7b8113890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34132/0 (socket says 172.21.15.32:34132) 2024-05-12T03:56:52.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:52.218+0000 7fa7b6ffd700 1 -- 172.21.15.32:0/3296107036 learned_addr learned my addr 172.21.15.32:0/3296107036 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:56:52.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:52.218+0000 7fa7b67fc700 1 --2- 172.21.15.32:0/3296107036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7b8103780 0x7fa7b8113dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:56:52.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:52.218+0000 7fa7b6ffd700 1 -- 172.21.15.32:0/3296107036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7b8103780 msgr2=0x7fa7b8113dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:56:52.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:52.218+0000 7fa7b6ffd700 1 --2- 172.21.15.32:0/3296107036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7b8103780 0x7fa7b8113dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:52.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:52.218+0000 7fa7b6ffd700 1 -- 172.21.15.32:0/3296107036 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a8005040 con 0x7fa7b8102de0 2024-05-12T03:56:52.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:52.218+0000 7fa7b6ffd700 1 --2- 172.21.15.32:0/3296107036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7b8102de0 0x7fa7b8113890 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8000c00 tx=0x7fa7a8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:56:52.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:52.218+0000 7fa79f7fe700 1 -- 172.21.15.32:0/3296107036 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7a800e040 con 0x7fa7b8102de0 2024-05-12T03:56:52.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:52.218+0000 7fa7bd2ba700 1 -- 172.21.15.32:0/3296107036 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7b810dca0 con 0x7fa7b8102de0 2024-05-12T03:56:52.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:52.218+0000 7fa79f7fe700 1 -- 172.21.15.32:0/3296107036 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa7a8013420 con 0x7fa7b8102de0 2024-05-12T03:56:52.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:52.218+0000 7fa79f7fe700 1 -- 172.21.15.32:0/3296107036 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7a801c630 con 0x7fa7b8102de0 2024-05-12T03:56:52.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:52.218+0000 7fa7bd2ba700 1 -- 172.21.15.32:0/3296107036 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7b810e260 con 0x7fa7b8102de0 2024-05-12T03:56:52.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:52.219+0000 7fa7bd2ba700 1 -- 172.21.15.32:0/3296107036 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa7b810e740 con 0x7fa7b8102de0 2024-05-12T03:56:52.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:52.220+0000 7fa79f7fe700 1 -- 172.21.15.32:0/3296107036 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fa7a8013590 con 0x7fa7b8102de0 2024-05-12T03:56:52.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:52.220+0000 7fa79f7fe700 1 --2- 172.21.15.32:0/3296107036 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fa7a00692a0 0x7fa7a006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:56:52.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:52.220+0000 7fa79f7fe700 1 -- 172.21.15.32:0/3296107036 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fa7a8087fc0 con 0x7fa7b8102de0 2024-05-12T03:56:52.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:52.220+0000 7fa7b67fc700 1 --2- 172.21.15.32:0/3296107036 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fa7a00692a0 0x7fa7a006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:56:52.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:52.220+0000 7fa79f7fe700 1 --2- 172.21.15.32:0/3296107036 >> [v2:172.21.15.33:6816/2123556026,v1:172.21.15.33:6817/2123556026] conn(0x7fa7a006ef70 0x7fa7a0071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:56:52.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:52.220+0000 7fa79f7fe700 1 -- 172.21.15.32:0/3296107036 --> [v2:172.21.15.33:6816/2123556026,v1:172.21.15.33:6817/2123556026] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa7a0071a30 con 0x7fa7a006ef70 2024-05-12T03:56:52.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:52.220+0000 7fa79f7fe700 1 -- 172.21.15.32:0/3296107036 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa7a808a550 con 0x7fa7b8102de0 2024-05-12T03:56:52.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:52.221+0000 7fa7b77fe700 1 --2- 172.21.15.32:0/3296107036 >> [v2:172.21.15.33:6816/2123556026,v1:172.21.15.33:6817/2123556026] conn(0x7fa7a006ef70 0x7fa7a0071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:56:52.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:52.221+0000 7fa7b67fc700 1 --2- 172.21.15.32:0/3296107036 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fa7a00692a0 0x7fa7a006b750 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa7b810ebe0 tx=0x7fa7ac0043c0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:56:52.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:52.221+0000 7fa7b77fe700 1 --2- 172.21.15.32:0/3296107036 >> [v2:172.21.15.33:6816/2123556026,v1:172.21.15.33:6817/2123556026] conn(0x7fa7a006ef70 0x7fa7a0071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:56:52.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:52.222+0000 7fa79f7fe700 1 -- 172.21.15.32:0/3296107036 <== osd.5 v2:172.21.15.33:6816/2123556026 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fa7a0071a30 con 0x7fa7a006ef70 2024-05-12T03:56:52.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:52.237+0000 7fa7bd2ba700 1 -- 172.21.15.32:0/3296107036 --> [v2:172.21.15.33:6816/2123556026,v1:172.21.15.33:6817/2123556026] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa7b80fa3b0 con 0x7fa7a006ef70 2024-05-12T03:56:52.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:52.238+0000 7fa79f7fe700 1 -- 172.21.15.32:0/3296107036 <== osd.5 v2:172.21.15.33:6816/2123556026 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fa7b80fa3b0 con 0x7fa7a006ef70 2024-05-12T03:56:52.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:52.238+0000 7fa7bd2ba700 1 -- 172.21.15.32:0/3296107036 >> [v2:172.21.15.33:6816/2123556026,v1:172.21.15.33:6817/2123556026] conn(0x7fa7a006ef70 msgr2=0x7fa7a0071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:56:52.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:52.238+0000 7fa7bd2ba700 1 --2- 172.21.15.32:0/3296107036 >> [v2:172.21.15.33:6816/2123556026,v1:172.21.15.33:6817/2123556026] conn(0x7fa7a006ef70 0x7fa7a0071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:52.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:52.238+0000 7fa7bd2ba700 1 -- 172.21.15.32:0/3296107036 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fa7a00692a0 msgr2=0x7fa7a006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:56:52.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:52.238+0000 7fa7bd2ba700 1 --2- 172.21.15.32:0/3296107036 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fa7a00692a0 0x7fa7a006b750 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa7b810ebe0 tx=0x7fa7ac0043c0 comp rx=0 tx=0).stop 2024-05-12T03:56:52.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:52.238+0000 7fa7bd2ba700 1 -- 172.21.15.32:0/3296107036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7b8102de0 msgr2=0x7fa7b8113890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:56:52.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:52.238+0000 7fa7bd2ba700 1 --2- 172.21.15.32:0/3296107036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7b8102de0 0x7fa7b8113890 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8000c00 tx=0x7fa7a8009e30 comp rx=0 tx=0).stop 2024-05-12T03:56:52.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:52.238+0000 7fa7bd2ba700 1 -- 172.21.15.32:0/3296107036 shutdown_connections 2024-05-12T03:56:52.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:52.238+0000 7fa7bd2ba700 1 --2- 172.21.15.32:0/3296107036 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fa7a00692a0 0x7fa7a006b750 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:52.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:52.238+0000 7fa7bd2ba700 1 --2- 172.21.15.32:0/3296107036 >> [v2:172.21.15.33:6816/2123556026,v1:172.21.15.33:6817/2123556026] conn(0x7fa7a006ef70 0x7fa7a0071380 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:52.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:52.238+0000 7fa7bd2ba700 1 --2- 172.21.15.32:0/3296107036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7b8102de0 0x7fa7b8113890 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:52.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:52.238+0000 7fa7bd2ba700 1 --2- 172.21.15.32:0/3296107036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7b8103780 0x7fa7b8113dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:52.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:52.238+0000 7fa7bd2ba700 1 -- 172.21.15.32:0/3296107036 >> 172.21.15.32:0/3296107036 conn(0x7fa7b80f8990 msgr2=0x7fa7b80f99e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:56:52.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:52.239+0000 7fa7bd2ba700 1 -- 172.21.15.32:0/3296107036 shutdown_connections 2024-05-12T03:56:52.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:52.239+0000 7fa7bd2ba700 1 -- 172.21.15.32:0/3296107036 wait complete. 2024-05-12T03:56:53.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:53 smithi033 ceph-mon[49525]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:56:53.460 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:53 smithi032 ceph-mon[40008]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:56:53.485 INFO:teuthology.orchestra.run.smithi032.stdout:167503724553 2024-05-12T03:56:53.485 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-12T03:56:53.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:53.919+0000 7f2d6eeb5700 1 -- 172.21.15.32:0/2322271791 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d68101ce0 msgr2=0x7f2d681020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:56:53.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:53.919+0000 7f2d6eeb5700 1 --2- 172.21.15.32:0/2322271791 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d68101ce0 0x7f2d681020b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c004940 tx=0x7f2d5c009e30 comp rx=0 tx=0).stop 2024-05-12T03:56:53.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:53.919+0000 7f2d6eeb5700 1 -- 172.21.15.32:0/2322271791 shutdown_connections 2024-05-12T03:56:53.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:53.919+0000 7f2d6eeb5700 1 --2- 172.21.15.32:0/2322271791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d68102680 0x7f2d680fbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:53.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:53.919+0000 7f2d6eeb5700 1 --2- 172.21.15.32:0/2322271791 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d68101ce0 0x7f2d681020b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:53.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:53.919+0000 7f2d6eeb5700 1 -- 172.21.15.32:0/2322271791 >> 172.21.15.32:0/2322271791 conn(0x7f2d680f7850 msgr2=0x7f2d680f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:56:53.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:53.919+0000 7f2d6eeb5700 1 -- 172.21.15.32:0/2322271791 shutdown_connections 2024-05-12T03:56:53.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:53.919+0000 7f2d6eeb5700 1 -- 172.21.15.32:0/2322271791 wait complete. 2024-05-12T03:56:53.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:53.920+0000 7f2d6eeb5700 1 Processor -- start 2024-05-12T03:56:53.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:53.920+0000 7f2d6eeb5700 1 -- start start 2024-05-12T03:56:53.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:53.920+0000 7f2d6eeb5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d68101ce0 0x7f2d6810ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:56:53.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:53.920+0000 7f2d6eeb5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d68102680 0x7f2d6810efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:56:53.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:53.920+0000 7f2d6eeb5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6810f600 con 0x7f2d68102680 2024-05-12T03:56:53.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:53.920+0000 7f2d6eeb5700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6810f740 con 0x7f2d68101ce0 2024-05-12T03:56:53.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:53.920+0000 7f2d67fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d68102680 0x7f2d6810efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:56:53.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:53.920+0000 7f2d67fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d68102680 0x7f2d6810efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34140/0 (socket says 172.21.15.32:34140) 2024-05-12T03:56:53.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:53.920+0000 7f2d67fff700 1 -- 172.21.15.32:0/610996372 learned_addr learned my addr 172.21.15.32:0/610996372 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:56:53.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:53.920+0000 7f2d6cc51700 1 --2- 172.21.15.32:0/610996372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d68101ce0 0x7f2d6810ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:56:53.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:53.920+0000 7f2d67fff700 1 -- 172.21.15.32:0/610996372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d68101ce0 msgr2=0x7f2d6810ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:56:53.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:53.920+0000 7f2d67fff700 1 --2- 172.21.15.32:0/610996372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d68101ce0 0x7f2d6810ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:53.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:53.920+0000 7f2d67fff700 1 -- 172.21.15.32:0/610996372 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d5c005040 con 0x7f2d68102680 2024-05-12T03:56:53.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:53.921+0000 7f2d6cc51700 1 --2- 172.21.15.32:0/610996372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d68101ce0 0x7f2d6810ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-12T03:56:53.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:53.921+0000 7f2d67fff700 1 --2- 172.21.15.32:0/610996372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d68102680 0x7f2d6810efb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2d54002a50 tx=0x7f2d54004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:56:53.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:53.921+0000 7f2d657fa700 1 -- 172.21.15.32:0/610996372 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d54007cc0 con 0x7f2d68102680 2024-05-12T03:56:53.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:53.921+0000 7f2d6eeb5700 1 -- 172.21.15.32:0/610996372 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d68117ae0 con 0x7f2d68102680 2024-05-12T03:56:53.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:53.921+0000 7f2d657fa700 1 -- 172.21.15.32:0/610996372 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2d5400f040 con 0x7f2d68102680 2024-05-12T03:56:53.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:53.921+0000 7f2d657fa700 1 -- 172.21.15.32:0/610996372 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d5400d990 con 0x7f2d68102680 2024-05-12T03:56:53.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:53.921+0000 7f2d6eeb5700 1 -- 172.21.15.32:0/610996372 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d68118040 con 0x7f2d68102680 2024-05-12T03:56:53.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:53.922+0000 7f2d6eeb5700 1 -- 172.21.15.32:0/610996372 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2d680f9360 con 0x7f2d68102680 2024-05-12T03:56:53.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:53.922+0000 7f2d657fa700 1 -- 172.21.15.32:0/610996372 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f2d54009710 con 0x7f2d68102680 2024-05-12T03:56:53.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:53.923+0000 7f2d657fa700 1 --2- 172.21.15.32:0/610996372 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f2d580692a0 0x7f2d5806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:56:53.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:53.923+0000 7f2d657fa700 1 -- 172.21.15.32:0/610996372 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f2d54086c60 con 0x7f2d68102680 2024-05-12T03:56:53.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:53.923+0000 7f2d6cc51700 1 --2- 172.21.15.32:0/610996372 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f2d580692a0 0x7f2d5806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:56:53.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:53.923+0000 7f2d657fa700 1 --2- 172.21.15.32:0/610996372 >> [v2:172.21.15.33:6808/2404279705,v1:172.21.15.33:6809/2404279705] conn(0x7f2d5806ef70 0x7f2d58071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:56:53.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:53.923+0000 7f2d657fa700 1 -- 172.21.15.32:0/610996372 --> [v2:172.21.15.33:6808/2404279705,v1:172.21.15.33:6809/2404279705] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2d58071a30 con 0x7f2d5806ef70 2024-05-12T03:56:53.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:53.923+0000 7f2d657fa700 1 -- 172.21.15.32:0/610996372 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d5408c260 con 0x7f2d68102680 2024-05-12T03:56:53.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:53.923+0000 7f2d6cc51700 1 --2- 172.21.15.32:0/610996372 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f2d580692a0 0x7f2d5806b750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c004560 tx=0x7f2d5c009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:56:53.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:53.923+0000 7f2d6d452700 1 --2- 172.21.15.32:0/610996372 >> [v2:172.21.15.33:6808/2404279705,v1:172.21.15.33:6809/2404279705] conn(0x7f2d5806ef70 0x7f2d58071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:56:53.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:53.924+0000 7f2d6d452700 1 --2- 172.21.15.32:0/610996372 >> [v2:172.21.15.33:6808/2404279705,v1:172.21.15.33:6809/2404279705] conn(0x7f2d5806ef70 0x7f2d58071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:56:53.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:53.924+0000 7f2d657fa700 1 -- 172.21.15.32:0/610996372 <== osd.4 v2:172.21.15.33:6808/2404279705 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f2d58071a30 con 0x7f2d5806ef70 2024-05-12T03:56:53.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:53.947+0000 7f2d6eeb5700 1 -- 172.21.15.32:0/610996372 --> [v2:172.21.15.33:6808/2404279705,v1:172.21.15.33:6809/2404279705] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2d680fd430 con 0x7f2d5806ef70 2024-05-12T03:56:53.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:53.948+0000 7f2d657fa700 1 -- 172.21.15.32:0/610996372 <== osd.4 v2:172.21.15.33:6808/2404279705 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f2d680fd430 con 0x7f2d5806ef70 2024-05-12T03:56:53.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:53.948+0000 7f2d6eeb5700 1 -- 172.21.15.32:0/610996372 >> [v2:172.21.15.33:6808/2404279705,v1:172.21.15.33:6809/2404279705] conn(0x7f2d5806ef70 msgr2=0x7f2d58071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:56:53.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:53.948+0000 7f2d6eeb5700 1 --2- 172.21.15.32:0/610996372 >> [v2:172.21.15.33:6808/2404279705,v1:172.21.15.33:6809/2404279705] conn(0x7f2d5806ef70 0x7f2d58071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:53.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:53.948+0000 7f2d6eeb5700 1 -- 172.21.15.32:0/610996372 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f2d580692a0 msgr2=0x7f2d5806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:56:53.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:53.949+0000 7f2d6eeb5700 1 --2- 172.21.15.32:0/610996372 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f2d580692a0 0x7f2d5806b750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c004560 tx=0x7f2d5c009e30 comp rx=0 tx=0).stop 2024-05-12T03:56:53.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:53.949+0000 7f2d6eeb5700 1 -- 172.21.15.32:0/610996372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d68102680 msgr2=0x7f2d6810efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:56:53.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:53.949+0000 7f2d6eeb5700 1 --2- 172.21.15.32:0/610996372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d68102680 0x7f2d6810efb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2d54002a50 tx=0x7f2d54004600 comp rx=0 tx=0).stop 2024-05-12T03:56:53.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:53.949+0000 7f2d6eeb5700 1 -- 172.21.15.32:0/610996372 shutdown_connections 2024-05-12T03:56:53.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:53.949+0000 7f2d6eeb5700 1 --2- 172.21.15.32:0/610996372 >> [v2:172.21.15.33:6808/2404279705,v1:172.21.15.33:6809/2404279705] conn(0x7f2d5806ef70 0x7f2d58071380 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:53.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:53.949+0000 7f2d6eeb5700 1 --2- 172.21.15.32:0/610996372 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f2d580692a0 0x7f2d5806b750 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:53.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:53.949+0000 7f2d6eeb5700 1 --2- 172.21.15.32:0/610996372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d68101ce0 0x7f2d6810ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:53.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:53.949+0000 7f2d6eeb5700 1 --2- 172.21.15.32:0/610996372 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d68102680 0x7f2d6810efb0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:53.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:53.949+0000 7f2d6eeb5700 1 -- 172.21.15.32:0/610996372 >> 172.21.15.32:0/610996372 conn(0x7f2d680f7850 msgr2=0x7f2d680f8990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:56:53.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:53.949+0000 7f2d6eeb5700 1 -- 172.21.15.32:0/610996372 shutdown_connections 2024-05-12T03:56:53.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:53.949+0000 7f2d6eeb5700 1 -- 172.21.15.32:0/610996372 wait complete. 2024-05-12T03:56:55.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:55 smithi033 ceph-mon[49525]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:56:55.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:55 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T03:56:55.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:55 smithi032 ceph-mon[40008]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:56:55.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:55 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T03:56:57.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:57 smithi033 ceph-mon[49525]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:56:57.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:57 smithi032 ceph-mon[40008]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:56:57.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:57.612+0000 7fbfe2005700 1 -- 172.21.15.32:0/1740421060 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfdc103780 msgr2=0x7fbfdc0fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:56:57.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:57.612+0000 7fbfe2005700 1 --2- 172.21.15.32:0/1740421060 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfdc103780 0x7fbfdc0fcdf0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbfd0004940 tx=0x7fbfd0009e30 comp rx=0 tx=0).stop 2024-05-12T03:56:57.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:57.612+0000 7fbfe2005700 1 -- 172.21.15.32:0/1740421060 shutdown_connections 2024-05-12T03:56:57.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:57.612+0000 7fbfe2005700 1 --2- 172.21.15.32:0/1740421060 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfdc103780 0x7fbfdc0fcdf0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:57.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:57.612+0000 7fbfe2005700 1 --2- 172.21.15.32:0/1740421060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfdc102de0 0x7fbfdc1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:57.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:57.612+0000 7fbfe2005700 1 -- 172.21.15.32:0/1740421060 >> 172.21.15.32:0/1740421060 conn(0x7fbfdc0f89b0 msgr2=0x7fbfdc0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:56:57.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:57.612+0000 7fbfe2005700 1 -- 172.21.15.32:0/1740421060 shutdown_connections 2024-05-12T03:56:57.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:57.612+0000 7fbfe2005700 1 -- 172.21.15.32:0/1740421060 wait complete. 2024-05-12T03:56:57.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:57.613+0000 7fbfe2005700 1 Processor -- start 2024-05-12T03:56:57.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:57.613+0000 7fbfe2005700 1 -- start start 2024-05-12T03:56:57.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:57.613+0000 7fbfe2005700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfdc102de0 0x7fbfdc070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:56:57.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:57.613+0000 7fbfe2005700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfdc103780 0x7fbfdc0707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:56:57.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:57.613+0000 7fbfe2005700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfdc070d30 con 0x7fbfdc103780 2024-05-12T03:56:57.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:57.613+0000 7fbfe2005700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfdc070e70 con 0x7fbfdc102de0 2024-05-12T03:56:57.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:57.613+0000 7fbfdaffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfdc103780 0x7fbfdc0707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:56:57.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:57.613+0000 7fbfdaffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfdc103780 0x7fbfdc0707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34158/0 (socket says 172.21.15.32:34158) 2024-05-12T03:56:57.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:57.613+0000 7fbfdaffd700 1 -- 172.21.15.32:0/3284587613 learned_addr learned my addr 172.21.15.32:0/3284587613 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:56:57.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:57.613+0000 7fbfdb7fe700 1 --2- 172.21.15.32:0/3284587613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfdc102de0 0x7fbfdc070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:56:57.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:57.613+0000 7fbfdaffd700 1 -- 172.21.15.32:0/3284587613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfdc102de0 msgr2=0x7fbfdc070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:56:57.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:57.613+0000 7fbfdaffd700 1 --2- 172.21.15.32:0/3284587613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfdc102de0 0x7fbfdc070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:57.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:57.613+0000 7fbfdaffd700 1 -- 172.21.15.32:0/3284587613 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd0005040 con 0x7fbfdc103780 2024-05-12T03:56:57.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:57.614+0000 7fbfdaffd700 1 --2- 172.21.15.32:0/3284587613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfdc103780 0x7fbfdc0707a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fbfd0009380 tx=0x7fbfd0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:56:57.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:57.614+0000 7fbfc3fff700 1 -- 172.21.15.32:0/3284587613 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbfd000e040 con 0x7fbfdc103780 2024-05-12T03:56:57.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:57.614+0000 7fbfc3fff700 1 -- 172.21.15.32:0/3284587613 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbfd0013420 con 0x7fbfdc103780 2024-05-12T03:56:57.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:57.614+0000 7fbfe2005700 1 -- 172.21.15.32:0/3284587613 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfdc0710c0 con 0x7fbfdc103780 2024-05-12T03:56:57.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:57.614+0000 7fbfc3fff700 1 -- 172.21.15.32:0/3284587613 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbfd001c630 con 0x7fbfdc103780 2024-05-12T03:56:57.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:57.614+0000 7fbfe2005700 1 -- 172.21.15.32:0/3284587613 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfdc11a210 con 0x7fbfdc103780 2024-05-12T03:56:57.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:57.615+0000 7fbfe2005700 1 -- 172.21.15.32:0/3284587613 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbfdc0fa480 con 0x7fbfdc103780 2024-05-12T03:56:57.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:57.616+0000 7fbfc3fff700 1 -- 172.21.15.32:0/3284587613 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fbfd00139e0 con 0x7fbfdc103780 2024-05-12T03:56:57.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:57.616+0000 7fbfc3fff700 1 --2- 172.21.15.32:0/3284587613 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fbfc40692a0 0x7fbfc406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:56:57.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:57.616+0000 7fbfc3fff700 1 -- 172.21.15.32:0/3284587613 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fbfd0088b40 con 0x7fbfdc103780 2024-05-12T03:56:57.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:57.617+0000 7fbfdb7fe700 1 --2- 172.21.15.32:0/3284587613 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fbfc40692a0 0x7fbfc406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:56:57.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:57.617+0000 7fbfc3fff700 1 --2- 172.21.15.32:0/3284587613 >> [v2:172.21.15.33:6800/2331766478,v1:172.21.15.33:6801/2331766478] conn(0x7fbfc406ef70 0x7fbfc4071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:56:57.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:57.617+0000 7fbfc3fff700 1 -- 172.21.15.32:0/3284587613 --> [v2:172.21.15.33:6800/2331766478,v1:172.21.15.33:6801/2331766478] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbfc4071a30 con 0x7fbfc406ef70 2024-05-12T03:56:57.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:57.617+0000 7fbfc3fff700 1 -- 172.21.15.32:0/3284587613 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbfd0088e80 con 0x7fbfdc103780 2024-05-12T03:56:57.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:57.617+0000 7fbfdb7fe700 1 --2- 172.21.15.32:0/3284587613 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fbfc40692a0 0x7fbfc406b750 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc007fd0 tx=0x7fbfcc009040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:56:57.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:57.617+0000 7fbfdbfff700 1 --2- 172.21.15.32:0/3284587613 >> [v2:172.21.15.33:6800/2331766478,v1:172.21.15.33:6801/2331766478] conn(0x7fbfc406ef70 0x7fbfc4071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:56:57.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:57.617+0000 7fbfdbfff700 1 --2- 172.21.15.32:0/3284587613 >> [v2:172.21.15.33:6800/2331766478,v1:172.21.15.33:6801/2331766478] conn(0x7fbfc406ef70 0x7fbfc4071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:56:57.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:57.618+0000 7fbfc3fff700 1 -- 172.21.15.32:0/3284587613 <== osd.3 v2:172.21.15.33:6800/2331766478 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fbfc4071a30 con 0x7fbfc406ef70 2024-05-12T03:56:57.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:57.633+0000 7fbfe2005700 1 -- 172.21.15.32:0/3284587613 --> [v2:172.21.15.33:6800/2331766478,v1:172.21.15.33:6801/2331766478] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbfdc0fe530 con 0x7fbfc406ef70 2024-05-12T03:56:57.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:57.633+0000 7fbfc3fff700 1 -- 172.21.15.32:0/3284587613 <== osd.3 v2:172.21.15.33:6800/2331766478 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fbfdc0fe530 con 0x7fbfc406ef70 2024-05-12T03:56:57.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:57.634+0000 7fbfe2005700 1 -- 172.21.15.32:0/3284587613 >> [v2:172.21.15.33:6800/2331766478,v1:172.21.15.33:6801/2331766478] conn(0x7fbfc406ef70 msgr2=0x7fbfc4071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:56:57.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:57.634+0000 7fbfe2005700 1 --2- 172.21.15.32:0/3284587613 >> [v2:172.21.15.33:6800/2331766478,v1:172.21.15.33:6801/2331766478] conn(0x7fbfc406ef70 0x7fbfc4071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:57.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:57.634+0000 7fbfe2005700 1 -- 172.21.15.32:0/3284587613 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fbfc40692a0 msgr2=0x7fbfc406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:56:57.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:57.634+0000 7fbfe2005700 1 --2- 172.21.15.32:0/3284587613 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fbfc40692a0 0x7fbfc406b750 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc007fd0 tx=0x7fbfcc009040 comp rx=0 tx=0).stop 2024-05-12T03:56:57.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:57.634+0000 7fbfe2005700 1 -- 172.21.15.32:0/3284587613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfdc103780 msgr2=0x7fbfdc0707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:56:57.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:57.634+0000 7fbfe2005700 1 --2- 172.21.15.32:0/3284587613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfdc103780 0x7fbfdc0707a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fbfd0009380 tx=0x7fbfd0009e30 comp rx=0 tx=0).stop 2024-05-12T03:56:57.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:57.634+0000 7fbfe2005700 1 -- 172.21.15.32:0/3284587613 shutdown_connections 2024-05-12T03:56:57.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:57.634+0000 7fbfe2005700 1 --2- 172.21.15.32:0/3284587613 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fbfc40692a0 0x7fbfc406b750 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:57.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:57.634+0000 7fbfe2005700 1 --2- 172.21.15.32:0/3284587613 >> [v2:172.21.15.33:6800/2331766478,v1:172.21.15.33:6801/2331766478] conn(0x7fbfc406ef70 0x7fbfc4071380 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:57.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:57.634+0000 7fbfe2005700 1 --2- 172.21.15.32:0/3284587613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfdc102de0 0x7fbfdc070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:57.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:57.634+0000 7fbfe2005700 1 --2- 172.21.15.32:0/3284587613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbfdc103780 0x7fbfdc0707a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:56:57.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:57.634+0000 7fbfe2005700 1 -- 172.21.15.32:0/3284587613 >> 172.21.15.32:0/3284587613 conn(0x7fbfdc0f89b0 msgr2=0x7fbfdc0f9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:56:57.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:57.634+0000 7fbfe2005700 1 -- 172.21.15.32:0/3284587613 shutdown_connections 2024-05-12T03:56:57.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:57.634+0000 7fbfe2005700 1 -- 172.21.15.32:0/3284587613 wait complete. 2024-05-12T03:56:59.254 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T03:56:59.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:56:59 smithi033 ceph-mon[49525]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:56:59.451 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:56:59 smithi032 ceph-mon[40008]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:57:00.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:00.678+0000 7fd9be0c4700 1 -- 172.21.15.32:0/1390118805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9b8102df0 msgr2=0x7fd9b81031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:00.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:00.678+0000 7fd9be0c4700 1 --2- 172.21.15.32:0/1390118805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9b8102df0 0x7fd9b81031c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd9a8004940 tx=0x7fd9a8009e30 comp rx=0 tx=0).stop 2024-05-12T03:57:00.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:00.678+0000 7fd9be0c4700 1 -- 172.21.15.32:0/1390118805 shutdown_connections 2024-05-12T03:57:00.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:00.678+0000 7fd9be0c4700 1 --2- 172.21.15.32:0/1390118805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9b8103790 0x7fd9b80fce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:00.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:00.678+0000 7fd9be0c4700 1 --2- 172.21.15.32:0/1390118805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9b8102df0 0x7fd9b81031c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:00.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:00.678+0000 7fd9be0c4700 1 -- 172.21.15.32:0/1390118805 >> 172.21.15.32:0/1390118805 conn(0x7fd9b80f8980 msgr2=0x7fd9b80fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:57:00.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:00.678+0000 7fd9be0c4700 1 -- 172.21.15.32:0/1390118805 shutdown_connections 2024-05-12T03:57:00.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:00.678+0000 7fd9be0c4700 1 -- 172.21.15.32:0/1390118805 wait complete. 2024-05-12T03:57:00.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:00.679+0000 7fd9be0c4700 1 Processor -- start 2024-05-12T03:57:00.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:00.679+0000 7fd9be0c4700 1 -- start start 2024-05-12T03:57:00.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:00.679+0000 7fd9be0c4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9b8102df0 0x7fd9b8113a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:00.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:00.679+0000 7fd9be0c4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9b8103790 0x7fd9b8113f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:00.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:00.679+0000 7fd9be0c4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b81145b0 con 0x7fd9b8102df0 2024-05-12T03:57:00.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:00.679+0000 7fd9be0c4700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b810d9a0 con 0x7fd9b8103790 2024-05-12T03:57:00.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:00.679+0000 7fd9b77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9b8102df0 0x7fd9b8113a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:00.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:00.679+0000 7fd9b77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9b8102df0 0x7fd9b8113a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34206/0 (socket says 172.21.15.32:34206) 2024-05-12T03:57:00.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:00.679+0000 7fd9b77fe700 1 -- 172.21.15.32:0/2212313867 learned_addr learned my addr 172.21.15.32:0/2212313867 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:57:00.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:00.679+0000 7fd9b6ffd700 1 --2- 172.21.15.32:0/2212313867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9b8103790 0x7fd9b8113f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:00.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:00.680+0000 7fd9b77fe700 1 -- 172.21.15.32:0/2212313867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9b8103790 msgr2=0x7fd9b8113f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:00.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:00.680+0000 7fd9b77fe700 1 --2- 172.21.15.32:0/2212313867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9b8103790 0x7fd9b8113f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:00.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:00.680+0000 7fd9b77fe700 1 -- 172.21.15.32:0/2212313867 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9a8005040 con 0x7fd9b8102df0 2024-05-12T03:57:00.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:00.680+0000 7fd9b77fe700 1 --2- 172.21.15.32:0/2212313867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9b8102df0 0x7fd9b8113a20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd9a8009180 tx=0x7fd9a8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:57:00.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:00.680+0000 7fd99ffff700 1 -- 172.21.15.32:0/2212313867 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9a800e040 con 0x7fd9b8102df0 2024-05-12T03:57:00.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:00.680+0000 7fd99ffff700 1 -- 172.21.15.32:0/2212313867 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd9a8004290 con 0x7fd9b8102df0 2024-05-12T03:57:00.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:00.680+0000 7fd9be0c4700 1 -- 172.21.15.32:0/2212313867 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9b810dc20 con 0x7fd9b8102df0 2024-05-12T03:57:00.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:00.680+0000 7fd99ffff700 1 -- 172.21.15.32:0/2212313867 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9a801c3f0 con 0x7fd9b8102df0 2024-05-12T03:57:00.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:00.680+0000 7fd9be0c4700 1 -- 172.21.15.32:0/2212313867 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9b810e1e0 con 0x7fd9b8102df0 2024-05-12T03:57:00.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:00.681+0000 7fd9be0c4700 1 -- 172.21.15.32:0/2212313867 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd9b810e6c0 con 0x7fd9b8102df0 2024-05-12T03:57:00.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:00.681+0000 7fd99ffff700 1 -- 172.21.15.32:0/2212313867 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fd9a800bb50 con 0x7fd9b8102df0 2024-05-12T03:57:00.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:00.682+0000 7fd99ffff700 1 --2- 172.21.15.32:0/2212313867 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fd9a00692a0 0x7fd9a006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:00.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:00.682+0000 7fd99ffff700 1 -- 172.21.15.32:0/2212313867 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fd9a8088190 con 0x7fd9b8102df0 2024-05-12T03:57:00.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:00.682+0000 7fd9b6ffd700 1 --2- 172.21.15.32:0/2212313867 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fd9a00692a0 0x7fd9a006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:00.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:00.682+0000 7fd99ffff700 1 --2- 172.21.15.32:0/2212313867 >> [v2:172.21.15.32:6810/3065624488,v1:172.21.15.32:6811/3065624488] conn(0x7fd9a006ef70 0x7fd9a0071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:00.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:00.682+0000 7fd99ffff700 1 -- 172.21.15.32:0/2212313867 --> [v2:172.21.15.32:6810/3065624488,v1:172.21.15.32:6811/3065624488] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd9a0071a30 con 0x7fd9a006ef70 2024-05-12T03:57:00.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:00.682+0000 7fd99ffff700 1 -- 172.21.15.32:0/2212313867 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9a808a7e0 con 0x7fd9b8102df0 2024-05-12T03:57:00.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:00.682+0000 7fd9b7fff700 1 --2- 172.21.15.32:0/2212313867 >> [v2:172.21.15.32:6810/3065624488,v1:172.21.15.32:6811/3065624488] conn(0x7fd9a006ef70 0x7fd9a0071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:00.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:00.682+0000 7fd9b6ffd700 1 --2- 172.21.15.32:0/2212313867 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fd9a00692a0 0x7fd9a006b750 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac009fd0 tx=0x7fd9ac009f60 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:57:00.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:00.683+0000 7fd9b7fff700 1 --2- 172.21.15.32:0/2212313867 >> [v2:172.21.15.32:6810/3065624488,v1:172.21.15.32:6811/3065624488] conn(0x7fd9a006ef70 0x7fd9a0071380 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:57:00.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:00.683+0000 7fd99ffff700 1 -- 172.21.15.32:0/2212313867 <== osd.1 v2:172.21.15.32:6810/3065624488 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fd9a0071a30 con 0x7fd9a006ef70 2024-05-12T03:57:00.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:00.698+0000 7fd9be0c4700 1 -- 172.21.15.32:0/2212313867 --> [v2:172.21.15.32:6810/3065624488,v1:172.21.15.32:6811/3065624488] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd9b80fa350 con 0x7fd9a006ef70 2024-05-12T03:57:00.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:00.698+0000 7fd99ffff700 1 -- 172.21.15.32:0/2212313867 <== osd.1 v2:172.21.15.32:6810/3065624488 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd9b80fa350 con 0x7fd9a006ef70 2024-05-12T03:57:00.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:00.699+0000 7fd9be0c4700 1 -- 172.21.15.32:0/2212313867 >> [v2:172.21.15.32:6810/3065624488,v1:172.21.15.32:6811/3065624488] conn(0x7fd9a006ef70 msgr2=0x7fd9a0071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:00.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:00.699+0000 7fd9be0c4700 1 --2- 172.21.15.32:0/2212313867 >> [v2:172.21.15.32:6810/3065624488,v1:172.21.15.32:6811/3065624488] conn(0x7fd9a006ef70 0x7fd9a0071380 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:00.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:00.699+0000 7fd9be0c4700 1 -- 172.21.15.32:0/2212313867 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fd9a00692a0 msgr2=0x7fd9a006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:00.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:00.699+0000 7fd9be0c4700 1 --2- 172.21.15.32:0/2212313867 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fd9a00692a0 0x7fd9a006b750 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac009fd0 tx=0x7fd9ac009f60 comp rx=0 tx=0).stop 2024-05-12T03:57:00.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:00.699+0000 7fd9be0c4700 1 -- 172.21.15.32:0/2212313867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9b8102df0 msgr2=0x7fd9b8113a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:00.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:00.699+0000 7fd9be0c4700 1 --2- 172.21.15.32:0/2212313867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9b8102df0 0x7fd9b8113a20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd9a8009180 tx=0x7fd9a8009e30 comp rx=0 tx=0).stop 2024-05-12T03:57:00.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:00.699+0000 7fd9be0c4700 1 -- 172.21.15.32:0/2212313867 shutdown_connections 2024-05-12T03:57:00.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:00.699+0000 7fd9be0c4700 1 --2- 172.21.15.32:0/2212313867 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fd9a00692a0 0x7fd9a006b750 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:00.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:00.699+0000 7fd9be0c4700 1 --2- 172.21.15.32:0/2212313867 >> [v2:172.21.15.32:6810/3065624488,v1:172.21.15.32:6811/3065624488] conn(0x7fd9a006ef70 0x7fd9a0071380 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:00.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:00.699+0000 7fd9be0c4700 1 --2- 172.21.15.32:0/2212313867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9b8102df0 0x7fd9b8113a20 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:00.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:00.699+0000 7fd9be0c4700 1 --2- 172.21.15.32:0/2212313867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9b8103790 0x7fd9b8113f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:00.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:00.699+0000 7fd9be0c4700 1 -- 172.21.15.32:0/2212313867 >> 172.21.15.32:0/2212313867 conn(0x7fd9b80f8980 msgr2=0x7fd9b80f9980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:57:00.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:00.699+0000 7fd9be0c4700 1 -- 172.21.15.32:0/2212313867 shutdown_connections 2024-05-12T03:57:00.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:00.699+0000 7fd9be0c4700 1 -- 172.21.15.32:0/2212313867 wait complete. 2024-05-12T03:57:01.015 INFO:teuthology.orchestra.run.smithi032.stdout:111669149718 2024-05-12T03:57:01.016 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-12T03:57:01.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:01 smithi032 ceph-mon[40008]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:57:01.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:01 smithi033 ceph-mon[49525]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:57:01.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:59.936+0000 7f6e1b93c700 1 -- 172.21.15.32:0/1800382627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e140fbd40 msgr2=0x7f6e140fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:01.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:59.936+0000 7f6e1b93c700 1 --2- 172.21.15.32:0/1800382627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e140fbd40 0x7f6e140fc1b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6e10004940 tx=0x7f6e10009e30 comp rx=0 tx=0).stop 2024-05-12T03:57:01.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:59.936+0000 7f6e1b93c700 1 -- 172.21.15.32:0/1800382627 shutdown_connections 2024-05-12T03:57:01.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:59.936+0000 7f6e1b93c700 1 --2- 172.21.15.32:0/1800382627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e140fbd40 0x7f6e140fc1b0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:01.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:59.936+0000 7f6e1b93c700 1 --2- 172.21.15.32:0/1800382627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e14101d90 0x7f6e14102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:01.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:59.936+0000 7f6e1b93c700 1 -- 172.21.15.32:0/1800382627 >> 172.21.15.32:0/1800382627 conn(0x7f6e14071d00 msgr2=0x7f6e14072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:57:01.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:59.936+0000 7f6e1b93c700 1 -- 172.21.15.32:0/1800382627 shutdown_connections 2024-05-12T03:57:01.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:59.936+0000 7f6e1b93c700 1 -- 172.21.15.32:0/1800382627 wait complete. 2024-05-12T03:57:01.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:59.936+0000 7f6e1b93c700 1 Processor -- start 2024-05-12T03:57:01.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:59.936+0000 7f6e1b93c700 1 -- start start 2024-05-12T03:57:01.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:59.937+0000 7f6e1b93c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e140fbd40 0x7f6e1410ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:01.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:59.937+0000 7f6e1b93c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e14101d90 0x7f6e1410ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:01.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:59.937+0000 7f6e1b93c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1410f5e0 con 0x7f6e14101d90 2024-05-12T03:57:01.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:59.937+0000 7f6e1b93c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1410f720 con 0x7f6e140fbd40 2024-05-12T03:57:01.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:59.937+0000 7f6e18ed7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e14101d90 0x7f6e1410ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:01.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:59.937+0000 7f6e18ed7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e14101d90 0x7f6e1410ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34174/0 (socket says 172.21.15.32:34174) 2024-05-12T03:57:01.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:59.937+0000 7f6e18ed7700 1 -- 172.21.15.32:0/953859907 learned_addr learned my addr 172.21.15.32:0/953859907 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:57:01.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:59.937+0000 7f6e196d8700 1 --2- 172.21.15.32:0/953859907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e140fbd40 0x7f6e1410ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:01.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:59.937+0000 7f6e18ed7700 1 -- 172.21.15.32:0/953859907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e140fbd40 msgr2=0x7f6e1410ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:01.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:59.937+0000 7f6e18ed7700 1 --2- 172.21.15.32:0/953859907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e140fbd40 0x7f6e1410ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:01.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:59.937+0000 7f6e18ed7700 1 -- 172.21.15.32:0/953859907 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e10005040 con 0x7f6e14101d90 2024-05-12T03:57:01.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:59.937+0000 7f6e18ed7700 1 --2- 172.21.15.32:0/953859907 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e14101d90 0x7f6e1410ef90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6e10009180 tx=0x7f6e10009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:57:01.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:59.937+0000 7f6e09ffb700 1 -- 172.21.15.32:0/953859907 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e1000e040 con 0x7f6e14101d90 2024-05-12T03:57:01.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:59.937+0000 7f6e1b93c700 1 -- 172.21.15.32:0/953859907 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e14117ac0 con 0x7f6e14101d90 2024-05-12T03:57:01.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:59.937+0000 7f6e09ffb700 1 -- 172.21.15.32:0/953859907 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6e10013420 con 0x7f6e14101d90 2024-05-12T03:57:01.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:59.937+0000 7f6e09ffb700 1 -- 172.21.15.32:0/953859907 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e1001c630 con 0x7f6e14101d90 2024-05-12T03:57:01.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:59.938+0000 7f6e1b93c700 1 -- 172.21.15.32:0/953859907 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e14117fd0 con 0x7f6e14101d90 2024-05-12T03:57:01.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:59.938+0000 7f6dff7fe700 1 -- 172.21.15.32:0/953859907 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6e1405e6a0 con 0x7f6e14101d90 2024-05-12T03:57:01.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:59.939+0000 7f6e09ffb700 1 -- 172.21.15.32:0/953859907 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f6e100139e0 con 0x7f6e14101d90 2024-05-12T03:57:01.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:59.939+0000 7f6e09ffb700 1 --2- 172.21.15.32:0/953859907 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f6e000692a0 0x7f6e0006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:01.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:59.939+0000 7f6e09ffb700 1 -- 172.21.15.32:0/953859907 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f6e10088090 con 0x7f6e14101d90 2024-05-12T03:57:01.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:59.939+0000 7f6e196d8700 1 --2- 172.21.15.32:0/953859907 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f6e000692a0 0x7f6e0006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:01.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:59.939+0000 7f6e09ffb700 1 --2- 172.21.15.32:0/953859907 >> [v2:172.21.15.32:6802/3497755677,v1:172.21.15.32:6803/3497755677] conn(0x7f6e0006ef70 0x7f6e00071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:01.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:59.939+0000 7f6e09ffb700 1 -- 172.21.15.32:0/953859907 --> [v2:172.21.15.32:6802/3497755677,v1:172.21.15.32:6803/3497755677] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6e00071a30 con 0x7f6e0006ef70 2024-05-12T03:57:01.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:59.939+0000 7f6e09ffb700 1 -- 172.21.15.32:0/953859907 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e1008a680 con 0x7f6e14101d90 2024-05-12T03:57:01.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:59.940+0000 7f6e19ed9700 1 --2- 172.21.15.32:0/953859907 >> [v2:172.21.15.32:6802/3497755677,v1:172.21.15.32:6803/3497755677] conn(0x7f6e0006ef70 0x7f6e00071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:01.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:59.940+0000 7f6e196d8700 1 --2- 172.21.15.32:0/953859907 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f6e000692a0 0x7f6e0006b750 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6e04007fd0 tx=0x7f6e04009040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:57:01.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:59.940+0000 7f6e19ed9700 1 --2- 172.21.15.32:0/953859907 >> [v2:172.21.15.32:6802/3497755677,v1:172.21.15.32:6803/3497755677] conn(0x7f6e0006ef70 0x7f6e00071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:57:01.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:59.941+0000 7f6e09ffb700 1 -- 172.21.15.32:0/953859907 <== osd.0 v2:172.21.15.32:6802/3497755677 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f6e00071a30 con 0x7f6e0006ef70 2024-05-12T03:57:01.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:59.955+0000 7f6dff7fe700 1 -- 172.21.15.32:0/953859907 --> [v2:172.21.15.32:6802/3497755677,v1:172.21.15.32:6803/3497755677] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6e1405ead0 con 0x7f6e0006ef70 2024-05-12T03:57:01.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:59.956+0000 7f6e09ffb700 1 -- 172.21.15.32:0/953859907 <== osd.0 v2:172.21.15.32:6802/3497755677 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6e1405ead0 con 0x7f6e0006ef70 2024-05-12T03:57:01.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:59.956+0000 7f6dff7fe700 1 -- 172.21.15.32:0/953859907 >> [v2:172.21.15.32:6802/3497755677,v1:172.21.15.32:6803/3497755677] conn(0x7f6e0006ef70 msgr2=0x7f6e00071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:01.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:59.956+0000 7f6dff7fe700 1 --2- 172.21.15.32:0/953859907 >> [v2:172.21.15.32:6802/3497755677,v1:172.21.15.32:6803/3497755677] conn(0x7f6e0006ef70 0x7f6e00071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:01.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:59.956+0000 7f6dff7fe700 1 -- 172.21.15.32:0/953859907 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f6e000692a0 msgr2=0x7f6e0006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:01.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:59.956+0000 7f6dff7fe700 1 --2- 172.21.15.32:0/953859907 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f6e000692a0 0x7f6e0006b750 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6e04007fd0 tx=0x7f6e04009040 comp rx=0 tx=0).stop 2024-05-12T03:57:01.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:59.956+0000 7f6dff7fe700 1 -- 172.21.15.32:0/953859907 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e14101d90 msgr2=0x7f6e1410ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:01.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:59.956+0000 7f6dff7fe700 1 --2- 172.21.15.32:0/953859907 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e14101d90 0x7f6e1410ef90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6e10009180 tx=0x7f6e10009e30 comp rx=0 tx=0).stop 2024-05-12T03:57:01.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:59.956+0000 7f6dff7fe700 1 -- 172.21.15.32:0/953859907 shutdown_connections 2024-05-12T03:57:01.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:59.956+0000 7f6dff7fe700 1 --2- 172.21.15.32:0/953859907 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f6e000692a0 0x7f6e0006b750 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:01.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:59.956+0000 7f6dff7fe700 1 --2- 172.21.15.32:0/953859907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e140fbd40 0x7f6e1410ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:01.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:59.956+0000 7f6dff7fe700 1 --2- 172.21.15.32:0/953859907 >> [v2:172.21.15.32:6802/3497755677,v1:172.21.15.32:6803/3497755677] conn(0x7f6e0006ef70 0x7f6e00071380 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:01.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:59.956+0000 7f6dff7fe700 1 --2- 172.21.15.32:0/953859907 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e14101d90 0x7f6e1410ef90 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:01.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:59.956+0000 7f6dff7fe700 1 -- 172.21.15.32:0/953859907 >> 172.21.15.32:0/953859907 conn(0x7f6e14071d00 msgr2=0x7f6e14104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:57:01.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:59.956+0000 7f6dff7fe700 1 -- 172.21.15.32:0/953859907 shutdown_connections 2024-05-12T03:57:01.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:56:59.956+0000 7f6dff7fe700 1 -- 172.21.15.32:0/953859907 wait complete. 2024-05-12T03:57:02.369 INFO:teuthology.orchestra.run.smithi032.stdout:146028888078 2024-05-12T03:57:02.370 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-12T03:57:03.168 INFO:teuthology.orchestra.run.smithi032.stdout:38654705704 2024-05-12T03:57:03.168 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-12T03:57:03.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:03 smithi033 ceph-mon[49525]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:57:03.450 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:03 smithi032 ceph-mon[40008]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:57:04.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.352+0000 7f1077d24700 1 -- 172.21.15.32:0/2065002629 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10700fde70 msgr2=0x7f10700fe240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:04.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.352+0000 7f1077d24700 1 --2- 172.21.15.32:0/2065002629 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10700fde70 0x7f10700fe240 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1060004940 tx=0x7f1060009e30 comp rx=0 tx=0).stop 2024-05-12T03:57:04.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.352+0000 7f1077d24700 1 -- 172.21.15.32:0/2065002629 shutdown_connections 2024-05-12T03:57:04.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.352+0000 7f1077d24700 1 --2- 172.21.15.32:0/2065002629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10700fe810 0x7f1070106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:04.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.352+0000 7f1077d24700 1 --2- 172.21.15.32:0/2065002629 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10700fde70 0x7f10700fe240 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:04.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.352+0000 7f1077d24700 1 -- 172.21.15.32:0/2065002629 >> 172.21.15.32:0/2065002629 conn(0x7f10700f78e0 msgr2=0x7f10700f9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:57:04.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.352+0000 7f1077d24700 1 -- 172.21.15.32:0/2065002629 shutdown_connections 2024-05-12T03:57:04.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.352+0000 7f1077d24700 1 -- 172.21.15.32:0/2065002629 wait complete. 2024-05-12T03:57:04.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.353+0000 7f1077d24700 1 Processor -- start 2024-05-12T03:57:04.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.353+0000 7f1077d24700 1 -- start start 2024-05-12T03:57:04.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.353+0000 7f1077d24700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10700fde70 0x7f1070113860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:04.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.353+0000 7f1077d24700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10700fe810 0x7f1070113da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:04.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.353+0000 7f1077d24700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10701143f0 con 0x7f10700fde70 2024-05-12T03:57:04.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.353+0000 7f1077d24700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f107010d9f0 con 0x7f10700fe810 2024-05-12T03:57:04.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.353+0000 7f1075ac0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10700fde70 0x7f1070113860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:04.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.353+0000 7f1075ac0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10700fde70 0x7f1070113860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37610/0 (socket says 172.21.15.32:37610) 2024-05-12T03:57:04.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.353+0000 7f1075ac0700 1 -- 172.21.15.32:0/516815900 learned_addr learned my addr 172.21.15.32:0/516815900 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:57:04.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.353+0000 7f10752bf700 1 --2- 172.21.15.32:0/516815900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10700fe810 0x7f1070113da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:04.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.353+0000 7f1075ac0700 1 -- 172.21.15.32:0/516815900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10700fe810 msgr2=0x7f1070113da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:04.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.353+0000 7f1075ac0700 1 --2- 172.21.15.32:0/516815900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10700fe810 0x7f1070113da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:04.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.353+0000 7f1075ac0700 1 -- 172.21.15.32:0/516815900 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1060005040 con 0x7f10700fde70 2024-05-12T03:57:04.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.354+0000 7f1075ac0700 1 --2- 172.21.15.32:0/516815900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10700fde70 0x7f1070113860 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1060009380 tx=0x7f1060009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:57:04.388 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.354+0000 7f10667fc700 1 -- 172.21.15.32:0/516815900 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f106000e040 con 0x7f10700fde70 2024-05-12T03:57:04.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.354+0000 7f1077d24700 1 -- 172.21.15.32:0/516815900 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f107010dc70 con 0x7f10700fde70 2024-05-12T03:57:04.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.354+0000 7f10667fc700 1 -- 172.21.15.32:0/516815900 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1060004290 con 0x7f10700fde70 2024-05-12T03:57:04.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.354+0000 7f10667fc700 1 -- 172.21.15.32:0/516815900 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1060013e90 con 0x7f10700fde70 2024-05-12T03:57:04.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.354+0000 7f1077d24700 1 -- 172.21.15.32:0/516815900 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f107010e1b0 con 0x7f10700fde70 2024-05-12T03:57:04.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.355+0000 7f1077d24700 1 -- 172.21.15.32:0/516815900 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1070066ad0 con 0x7f10700fde70 2024-05-12T03:57:04.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.355+0000 7f10667fc700 1 -- 172.21.15.32:0/516815900 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f10600136b0 con 0x7f10700fde70 2024-05-12T03:57:04.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.356+0000 7f10667fc700 1 --2- 172.21.15.32:0/516815900 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f105c0692a0 0x7f105c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:04.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.356+0000 7f10667fc700 1 -- 172.21.15.32:0/516815900 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f10600881c0 con 0x7f10700fde70 2024-05-12T03:57:04.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.356+0000 7f10752bf700 1 --2- 172.21.15.32:0/516815900 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f105c0692a0 0x7f105c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:04.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.356+0000 7f10752bf700 1 --2- 172.21.15.32:0/516815900 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f105c0692a0 0x7f105c06b750 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f107010ebb0 tx=0x7f106c0043c0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:57:04.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.361+0000 7f10667fc700 1 -- 172.21.15.32:0/516815900 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f1060058220 con 0x7f10700fde70 2024-05-12T03:57:04.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.558+0000 7f1077d24700 1 -- 172.21.15.32:0/516815900 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f1070109e10 con 0x7f10700fde70 2024-05-12T03:57:04.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.558+0000 7f10667fc700 1 -- 172.21.15.32:0/516815900 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1060057db0 con 0x7f10700fde70 2024-05-12T03:57:04.558 INFO:teuthology.orchestra.run.smithi032.stdout:167503724555 2024-05-12T03:57:04.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.560+0000 7f1077d24700 1 -- 172.21.15.32:0/516815900 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f105c0692a0 msgr2=0x7f105c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:04.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.561+0000 7f1077d24700 1 --2- 172.21.15.32:0/516815900 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f105c0692a0 0x7f105c06b750 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f107010ebb0 tx=0x7f106c0043c0 comp rx=0 tx=0).stop 2024-05-12T03:57:04.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.561+0000 7f1077d24700 1 -- 172.21.15.32:0/516815900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10700fde70 msgr2=0x7f1070113860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:04.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.561+0000 7f1077d24700 1 --2- 172.21.15.32:0/516815900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10700fde70 0x7f1070113860 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1060009380 tx=0x7f1060009e30 comp rx=0 tx=0).stop 2024-05-12T03:57:04.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.561+0000 7f1077d24700 1 -- 172.21.15.32:0/516815900 shutdown_connections 2024-05-12T03:57:04.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.561+0000 7f1077d24700 1 --2- 172.21.15.32:0/516815900 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f105c0692a0 0x7f105c06b750 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:04.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.561+0000 7f1077d24700 1 --2- 172.21.15.32:0/516815900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10700fde70 0x7f1070113860 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:04.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.561+0000 7f1077d24700 1 --2- 172.21.15.32:0/516815900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10700fe810 0x7f1070113da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:04.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.561+0000 7f1077d24700 1 -- 172.21.15.32:0/516815900 >> 172.21.15.32:0/516815900 conn(0x7f10700f78e0 msgr2=0x7f1070103750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:57:04.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.561+0000 7f1077d24700 1 -- 172.21.15.32:0/516815900 shutdown_connections 2024-05-12T03:57:04.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.561+0000 7f1077d24700 1 -- 172.21.15.32:0/516815900 wait complete. 2024-05-12T03:57:04.767 INFO:teuthology.orchestra.run.smithi032.stdout:60129542177 2024-05-12T03:57:04.768 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-12T03:57:05.339 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:05 smithi032 ceph-mon[40008]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:57:05.340 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:05 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/516815900' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-12T03:57:05.362 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724553 got 167503724555 for osd.5 2024-05-12T03:57:05.362 DEBUG:teuthology.parallel:result is None 2024-05-12T03:57:05.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:05 smithi033 ceph-mon[49525]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:57:05.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:05 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/516815900' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-12T03:57:05.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.255+0000 7f1066de6700 1 -- 172.21.15.32:0/2036987915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1060102670 msgr2=0x7f10600fb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:05.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.255+0000 7f1066de6700 1 --2- 172.21.15.32:0/2036987915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1060102670 0x7f10600fb600 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f104c004940 tx=0x7f104c009e30 comp rx=0 tx=0).stop 2024-05-12T03:57:05.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.255+0000 7f1066de6700 1 -- 172.21.15.32:0/2036987915 shutdown_connections 2024-05-12T03:57:05.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.255+0000 7f1066de6700 1 --2- 172.21.15.32:0/2036987915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1060102670 0x7f10600fb600 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:05.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.255+0000 7f1066de6700 1 --2- 172.21.15.32:0/2036987915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1060101cd0 0x7f10601020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:05.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.255+0000 7f1066de6700 1 -- 172.21.15.32:0/2036987915 >> 172.21.15.32:0/2036987915 conn(0x7f1060071460 msgr2=0x7f1060071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:57:05.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.255+0000 7f1066de6700 1 -- 172.21.15.32:0/2036987915 shutdown_connections 2024-05-12T03:57:05.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.255+0000 7f1066de6700 1 -- 172.21.15.32:0/2036987915 wait complete. 2024-05-12T03:57:05.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.256+0000 7f1066de6700 1 Processor -- start 2024-05-12T03:57:05.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.256+0000 7f1066de6700 1 -- start start 2024-05-12T03:57:05.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.256+0000 7f1066de6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1060101cd0 0x7f1060113af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:05.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.256+0000 7f1066de6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1060102670 0x7f1060114030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:05.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.256+0000 7f1066de6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1060114680 con 0x7f1060101cd0 2024-05-12T03:57:05.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.256+0000 7f1066de6700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106010da70 con 0x7f1060102670 2024-05-12T03:57:05.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.257+0000 7f1064b82700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1060101cd0 0x7f1060113af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:05.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.257+0000 7f1064b82700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1060101cd0 0x7f1060113af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37596/0 (socket says 172.21.15.32:37596) 2024-05-12T03:57:05.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.257+0000 7f1064b82700 1 -- 172.21.15.32:0/520742123 learned_addr learned my addr 172.21.15.32:0/520742123 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:57:05.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.257+0000 7f105ffff700 1 --2- 172.21.15.32:0/520742123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1060102670 0x7f1060114030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:05.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.257+0000 7f1064b82700 1 -- 172.21.15.32:0/520742123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1060102670 msgr2=0x7f1060114030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:05.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.257+0000 7f1064b82700 1 --2- 172.21.15.32:0/520742123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1060102670 0x7f1060114030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:05.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.257+0000 7f1064b82700 1 -- 172.21.15.32:0/520742123 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f104c005040 con 0x7f1060101cd0 2024-05-12T03:57:05.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.257+0000 7f1064b82700 1 --2- 172.21.15.32:0/520742123 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1060101cd0 0x7f1060113af0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1054002a50 tx=0x7f10540046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:57:05.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.257+0000 7f105d7fa700 1 -- 172.21.15.32:0/520742123 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1054007cc0 con 0x7f1060101cd0 2024-05-12T03:57:05.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.257+0000 7f105d7fa700 1 -- 172.21.15.32:0/520742123 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f105400f040 con 0x7f1060101cd0 2024-05-12T03:57:05.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.257+0000 7f1066de6700 1 -- 172.21.15.32:0/520742123 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f106010dd50 con 0x7f1060101cd0 2024-05-12T03:57:05.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.257+0000 7f105d7fa700 1 -- 172.21.15.32:0/520742123 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f105400d9c0 con 0x7f1060101cd0 2024-05-12T03:57:05.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.257+0000 7f1066de6700 1 -- 172.21.15.32:0/520742123 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f106010e2e0 con 0x7f1060101cd0 2024-05-12T03:57:05.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.258+0000 7f1066de6700 1 -- 172.21.15.32:0/520742123 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f106010e7d0 con 0x7f1060101cd0 2024-05-12T03:57:05.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.259+0000 7f105d7fa700 1 -- 172.21.15.32:0/520742123 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f105400db40 con 0x7f1060101cd0 2024-05-12T03:57:05.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.259+0000 7f105d7fa700 1 --2- 172.21.15.32:0/520742123 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f10500692a0 0x7f105006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:05.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.260+0000 7f105d7fa700 1 -- 172.21.15.32:0/520742123 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f1054086c00 con 0x7f1060101cd0 2024-05-12T03:57:05.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.260+0000 7f105ffff700 1 --2- 172.21.15.32:0/520742123 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f10500692a0 0x7f105006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:05.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.260+0000 7f105d7fa700 1 --2- 172.21.15.32:0/520742123 >> [v2:172.21.15.32:6818/3261757113,v1:172.21.15.32:6819/3261757113] conn(0x7f105006ef70 0x7f1050071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:05.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.260+0000 7f105d7fa700 1 -- 172.21.15.32:0/520742123 --> [v2:172.21.15.32:6818/3261757113,v1:172.21.15.32:6819/3261757113] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1050071a30 con 0x7f105006ef70 2024-05-12T03:57:05.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.260+0000 7f105d7fa700 1 -- 172.21.15.32:0/520742123 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f105408b220 con 0x7f1060101cd0 2024-05-12T03:57:05.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.260+0000 7f1065383700 1 --2- 172.21.15.32:0/520742123 >> [v2:172.21.15.32:6818/3261757113,v1:172.21.15.32:6819/3261757113] conn(0x7f105006ef70 0x7f1050071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:05.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.260+0000 7f105ffff700 1 --2- 172.21.15.32:0/520742123 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f10500692a0 0x7f105006b750 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f104c00b3b0 tx=0x7f104c009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:57:05.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.260+0000 7f1065383700 1 --2- 172.21.15.32:0/520742123 >> [v2:172.21.15.32:6818/3261757113,v1:172.21.15.32:6819/3261757113] conn(0x7f105006ef70 0x7f1050071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:57:05.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.261+0000 7f105d7fa700 1 -- 172.21.15.32:0/520742123 <== osd.2 v2:172.21.15.32:6818/3261757113 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f1050071a30 con 0x7f105006ef70 2024-05-12T03:57:05.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.276+0000 7f1066de6700 1 -- 172.21.15.32:0/520742123 --> [v2:172.21.15.32:6818/3261757113,v1:172.21.15.32:6819/3261757113] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f10601013a0 con 0x7f105006ef70 2024-05-12T03:57:05.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.276+0000 7f105d7fa700 1 -- 172.21.15.32:0/520742123 <== osd.2 v2:172.21.15.32:6818/3261757113 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f10601013a0 con 0x7f105006ef70 2024-05-12T03:57:05.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.277+0000 7f1066de6700 1 -- 172.21.15.32:0/520742123 >> [v2:172.21.15.32:6818/3261757113,v1:172.21.15.32:6819/3261757113] conn(0x7f105006ef70 msgr2=0x7f1050071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:05.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.277+0000 7f1066de6700 1 --2- 172.21.15.32:0/520742123 >> [v2:172.21.15.32:6818/3261757113,v1:172.21.15.32:6819/3261757113] conn(0x7f105006ef70 0x7f1050071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:05.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.277+0000 7f1066de6700 1 -- 172.21.15.32:0/520742123 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f10500692a0 msgr2=0x7f105006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:05.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.277+0000 7f1066de6700 1 --2- 172.21.15.32:0/520742123 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f10500692a0 0x7f105006b750 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f104c00b3b0 tx=0x7f104c009e30 comp rx=0 tx=0).stop 2024-05-12T03:57:05.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.277+0000 7f1066de6700 1 -- 172.21.15.32:0/520742123 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1060101cd0 msgr2=0x7f1060113af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:05.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.277+0000 7f1066de6700 1 --2- 172.21.15.32:0/520742123 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1060101cd0 0x7f1060113af0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1054002a50 tx=0x7f10540046b0 comp rx=0 tx=0).stop 2024-05-12T03:57:05.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.277+0000 7f1066de6700 1 -- 172.21.15.32:0/520742123 shutdown_connections 2024-05-12T03:57:05.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.277+0000 7f1066de6700 1 --2- 172.21.15.32:0/520742123 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f10500692a0 0x7f105006b750 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:05.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.277+0000 7f1066de6700 1 --2- 172.21.15.32:0/520742123 >> [v2:172.21.15.32:6818/3261757113,v1:172.21.15.32:6819/3261757113] conn(0x7f105006ef70 0x7f1050071380 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:05.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.277+0000 7f1066de6700 1 --2- 172.21.15.32:0/520742123 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1060101cd0 0x7f1060113af0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:05.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.277+0000 7f1066de6700 1 --2- 172.21.15.32:0/520742123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1060102670 0x7f1060114030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:05.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.277+0000 7f1066de6700 1 -- 172.21.15.32:0/520742123 >> 172.21.15.32:0/520742123 conn(0x7f1060071460 msgr2=0x7f10601009d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:57:05.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.277+0000 7f1066de6700 1 -- 172.21.15.32:0/520742123 shutdown_connections 2024-05-12T03:57:05.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:04.277+0000 7f1066de6700 1 -- 172.21.15.32:0/520742123 wait complete. 2024-05-12T03:57:05.576 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T03:57:05.577 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T03:57:06.049 INFO:teuthology.orchestra.run.smithi032.stdout:81604378652 2024-05-12T03:57:06.049 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-12T03:57:06.359 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T03:57:07.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:07 smithi033 ceph-mon[49525]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:57:07.505 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:07 smithi032 ceph-mon[40008]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:57:08.116 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T03:57:09.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:09.381+0000 7faaabb8e700 1 -- 172.21.15.32:0/2347358244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaa4101cd0 msgr2=0x7faaa41020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:09.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:09.381+0000 7faaabb8e700 1 --2- 172.21.15.32:0/2347358244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaa4101cd0 0x7faaa41020a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7faaa0004940 tx=0x7faaa0009e30 comp rx=0 tx=0).stop 2024-05-12T03:57:09.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:09.382+0000 7faaabb8e700 1 -- 172.21.15.32:0/2347358244 shutdown_connections 2024-05-12T03:57:09.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:09.382+0000 7faaabb8e700 1 --2- 172.21.15.32:0/2347358244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaa4102670 0x7faaa40fb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:09.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:09.382+0000 7faaabb8e700 1 --2- 172.21.15.32:0/2347358244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaa4101cd0 0x7faaa41020a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:09.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:09.382+0000 7faaabb8e700 1 -- 172.21.15.32:0/2347358244 >> 172.21.15.32:0/2347358244 conn(0x7faaa4071460 msgr2=0x7faaa4071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:57:09.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:09.382+0000 7faaabb8e700 1 -- 172.21.15.32:0/2347358244 shutdown_connections 2024-05-12T03:57:09.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:09.382+0000 7faaabb8e700 1 -- 172.21.15.32:0/2347358244 wait complete. 2024-05-12T03:57:09.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:09.382+0000 7faaabb8e700 1 Processor -- start 2024-05-12T03:57:09.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:09.382+0000 7faaabb8e700 1 -- start start 2024-05-12T03:57:09.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:09.382+0000 7faaabb8e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaa4101cd0 0x7faaa4113a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:09.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:09.382+0000 7faaabb8e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaa4102670 0x7faaa4113fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:09.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:09.382+0000 7faaabb8e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa4114680 con 0x7faaa4102670 2024-05-12T03:57:09.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:09.382+0000 7faaabb8e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa410d9e0 con 0x7faaa4101cd0 2024-05-12T03:57:09.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:09.382+0000 7faaa9129700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaa4102670 0x7faaa4113fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:09.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:09.383+0000 7faaa9129700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaa4102670 0x7faaa4113fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37640/0 (socket says 172.21.15.32:37640) 2024-05-12T03:57:09.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:09.383+0000 7faaa9129700 1 -- 172.21.15.32:0/3765777469 learned_addr learned my addr 172.21.15.32:0/3765777469 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:57:09.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:09.383+0000 7faaa992a700 1 --2- 172.21.15.32:0/3765777469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaa4101cd0 0x7faaa4113a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:09.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:09.383+0000 7faaa9129700 1 -- 172.21.15.32:0/3765777469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaa4101cd0 msgr2=0x7faaa4113a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:09.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:09.383+0000 7faaa9129700 1 --2- 172.21.15.32:0/3765777469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaa4101cd0 0x7faaa4113a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:09.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:09.383+0000 7faaa9129700 1 -- 172.21.15.32:0/3765777469 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaa0005040 con 0x7faaa4102670 2024-05-12T03:57:09.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:09.383+0000 7faaa9129700 1 --2- 172.21.15.32:0/3765777469 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaa4102670 0x7faaa4113fa0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7faa9400b760 tx=0x7faa94009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:57:09.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:09.383+0000 7faa9a7fc700 1 -- 172.21.15.32:0/3765777469 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa9400f6f0 con 0x7faaa4102670 2024-05-12T03:57:09.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:09.383+0000 7faaabb8e700 1 -- 172.21.15.32:0/3765777469 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaa410dcc0 con 0x7faaa4102670 2024-05-12T03:57:09.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:09.383+0000 7faa9a7fc700 1 -- 172.21.15.32:0/3765777469 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7faa9400fd00 con 0x7faaa4102670 2024-05-12T03:57:09.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:09.383+0000 7faa9a7fc700 1 -- 172.21.15.32:0/3765777469 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa94011e60 con 0x7faaa4102670 2024-05-12T03:57:09.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:09.383+0000 7faaabb8e700 1 -- 172.21.15.32:0/3765777469 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaa410e280 con 0x7faaa4102670 2024-05-12T03:57:09.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:09.384+0000 7faaabb8e700 1 -- 172.21.15.32:0/3765777469 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaa4065200 con 0x7faaa4102670 2024-05-12T03:57:09.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:09.385+0000 7faa9a7fc700 1 -- 172.21.15.32:0/3765777469 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7faa94011690 con 0x7faaa4102670 2024-05-12T03:57:09.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:09.385+0000 7faa9a7fc700 1 --2- 172.21.15.32:0/3765777469 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7faa900692a0 0x7faa9006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:09.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:09.385+0000 7faa9a7fc700 1 -- 172.21.15.32:0/3765777469 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7faa94086d10 con 0x7faaa4102670 2024-05-12T03:57:09.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:09.385+0000 7faaa992a700 1 --2- 172.21.15.32:0/3765777469 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7faa900692a0 0x7faa9006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:09.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:09.386+0000 7faaa992a700 1 --2- 172.21.15.32:0/3765777469 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7faa900692a0 0x7faa9006b750 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7faaa0004f90 tx=0x7faaa0009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:57:09.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:09.390+0000 7faa9a7fc700 1 -- 172.21.15.32:0/3765777469 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7faa94056e10 con 0x7faaa4102670 2024-05-12T03:57:09.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:09 smithi033 ceph-mon[49525]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:57:09.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:09 smithi032 ceph-mon[40008]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:57:09.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:09.585+0000 7faaabb8e700 1 -- 172.21.15.32:0/3765777469 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7faaa404bed0 con 0x7faaa4102670 2024-05-12T03:57:09.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:09.586+0000 7faa9a7fc700 1 -- 172.21.15.32:0/3765777469 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7faa940569a0 con 0x7faaa4102670 2024-05-12T03:57:09.586 INFO:teuthology.orchestra.run.smithi032.stdout:38654705705 2024-05-12T03:57:09.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:09.588+0000 7faaabb8e700 1 -- 172.21.15.32:0/3765777469 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7faa900692a0 msgr2=0x7faa9006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:09.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:09.588+0000 7faaabb8e700 1 --2- 172.21.15.32:0/3765777469 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7faa900692a0 0x7faa9006b750 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7faaa0004f90 tx=0x7faaa0009e30 comp rx=0 tx=0).stop 2024-05-12T03:57:09.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:09.588+0000 7faaabb8e700 1 -- 172.21.15.32:0/3765777469 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaa4102670 msgr2=0x7faaa4113fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:09.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:09.588+0000 7faaabb8e700 1 --2- 172.21.15.32:0/3765777469 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaa4102670 0x7faaa4113fa0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7faa9400b760 tx=0x7faa94009220 comp rx=0 tx=0).stop 2024-05-12T03:57:09.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:09.588+0000 7faaabb8e700 1 -- 172.21.15.32:0/3765777469 shutdown_connections 2024-05-12T03:57:09.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:09.588+0000 7faaabb8e700 1 --2- 172.21.15.32:0/3765777469 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7faa900692a0 0x7faa9006b750 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:09.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:09.588+0000 7faaabb8e700 1 --2- 172.21.15.32:0/3765777469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaa4101cd0 0x7faaa4113a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:09.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:09.588+0000 7faaabb8e700 1 --2- 172.21.15.32:0/3765777469 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faaa4102670 0x7faaa4113fa0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:09.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:09.588+0000 7faaabb8e700 1 -- 172.21.15.32:0/3765777469 >> 172.21.15.32:0/3765777469 conn(0x7faaa4071460 msgr2=0x7faaa40f9430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:57:09.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:09.588+0000 7faaabb8e700 1 -- 172.21.15.32:0/3765777469 shutdown_connections 2024-05-12T03:57:09.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:09.588+0000 7faaabb8e700 1 -- 172.21.15.32:0/3765777469 wait complete. 2024-05-12T03:57:10.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:10 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/3765777469' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-12T03:57:10.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:10 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T03:57:10.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:10 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/3765777469' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-12T03:57:10.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:10 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T03:57:10.945 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T03:57:11.176 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:11 smithi032 ceph-mon[40008]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:57:11.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:11 smithi033 ceph-mon[49525]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:57:12.178 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705704 got 38654705705 for osd.0 2024-05-12T03:57:12.178 DEBUG:teuthology.parallel:result is None 2024-05-12T03:57:12.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:11.826+0000 7f75307d1700 1 -- 172.21.15.32:0/1157297440 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7528065640 msgr2=0x7f7528065ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:12.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:11.827+0000 7f75307d1700 1 --2- 172.21.15.32:0/1157297440 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7528065640 0x7f7528065ab0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f751c004940 tx=0x7f751c009e30 comp rx=0 tx=0).stop 2024-05-12T03:57:12.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:11.827+0000 7f75307d1700 1 -- 172.21.15.32:0/1157297440 shutdown_connections 2024-05-12T03:57:12.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:11.827+0000 7f75307d1700 1 --2- 172.21.15.32:0/1157297440 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7528065640 0x7f7528065ab0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:12.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:11.827+0000 7f75307d1700 1 --2- 172.21.15.32:0/1157297440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7528102e50 0x7f7528103220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:12.289 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:11.827+0000 7f75307d1700 1 -- 172.21.15.32:0/1157297440 >> 172.21.15.32:0/1157297440 conn(0x7f7528071d00 msgr2=0x7f7528072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:57:12.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:11.827+0000 7f75307d1700 1 -- 172.21.15.32:0/1157297440 shutdown_connections 2024-05-12T03:57:12.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:11.827+0000 7f75307d1700 1 -- 172.21.15.32:0/1157297440 wait complete. 2024-05-12T03:57:12.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:11.827+0000 7f75307d1700 1 Processor -- start 2024-05-12T03:57:12.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:11.827+0000 7f75307d1700 1 -- start start 2024-05-12T03:57:12.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:11.827+0000 7f75307d1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7528065640 0x7f7528113a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:12.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:11.827+0000 7f75307d1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7528102e50 0x7f7528113f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:12.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:11.828+0000 7f75307d1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75281145e0 con 0x7f7528065640 2024-05-12T03:57:12.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:11.828+0000 7f75307d1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f752810d9d0 con 0x7f7528102e50 2024-05-12T03:57:12.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:11.828+0000 7f752e56d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7528065640 0x7f7528113a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:12.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:11.828+0000 7f752e56d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7528065640 0x7f7528113a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37650/0 (socket says 172.21.15.32:37650) 2024-05-12T03:57:12.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:11.828+0000 7f752e56d700 1 -- 172.21.15.32:0/3254460983 learned_addr learned my addr 172.21.15.32:0/3254460983 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:57:12.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:11.828+0000 7f752dd6c700 1 --2- 172.21.15.32:0/3254460983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7528102e50 0x7f7528113f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:12.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:11.828+0000 7f752e56d700 1 -- 172.21.15.32:0/3254460983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7528102e50 msgr2=0x7f7528113f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:12.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:11.828+0000 7f752e56d700 1 --2- 172.21.15.32:0/3254460983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7528102e50 0x7f7528113f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:12.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:11.828+0000 7f752e56d700 1 -- 172.21.15.32:0/3254460983 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f751c005040 con 0x7f7528065640 2024-05-12T03:57:12.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:11.828+0000 7f752e56d700 1 --2- 172.21.15.32:0/3254460983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7528065640 0x7f7528113a50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f752400eb70 tx=0x7f752400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:57:12.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:11.828+0000 7f751affd700 1 -- 172.21.15.32:0/3254460983 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7524007cc0 con 0x7f7528065640 2024-05-12T03:57:12.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:11.828+0000 7f75307d1700 1 -- 172.21.15.32:0/3254460983 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f752810dc50 con 0x7f7528065640 2024-05-12T03:57:12.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:11.828+0000 7f751affd700 1 -- 172.21.15.32:0/3254460983 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f752400f040 con 0x7f7528065640 2024-05-12T03:57:12.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:11.828+0000 7f751affd700 1 -- 172.21.15.32:0/3254460983 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f752400c9f0 con 0x7f7528065640 2024-05-12T03:57:12.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:11.828+0000 7f75307d1700 1 -- 172.21.15.32:0/3254460983 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f752810e210 con 0x7f7528065640 2024-05-12T03:57:12.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:11.829+0000 7f75307d1700 1 -- 172.21.15.32:0/3254460983 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75280fe540 con 0x7f7528065640 2024-05-12T03:57:12.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:11.830+0000 7f751affd700 1 -- 172.21.15.32:0/3254460983 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f752400cb50 con 0x7f7528065640 2024-05-12T03:57:12.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:11.830+0000 7f751affd700 1 --2- 172.21.15.32:0/3254460983 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f75140692b0 0x7f751406b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:12.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:11.830+0000 7f751affd700 1 -- 172.21.15.32:0/3254460983 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f7524086a10 con 0x7f7528065640 2024-05-12T03:57:12.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:11.830+0000 7f752dd6c700 1 --2- 172.21.15.32:0/3254460983 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f75140692b0 0x7f751406b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:12.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:11.830+0000 7f752dd6c700 1 --2- 172.21.15.32:0/3254460983 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f75140692b0 0x7f751406b760 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f752810eb90 tx=0x7f751c009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:57:12.291 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:11.834+0000 7f751affd700 1 -- 172.21.15.32:0/3254460983 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f7524056a90 con 0x7f7528065640 2024-05-12T03:57:12.292 INFO:teuthology.orchestra.run.smithi032.stdout:146028888081 2024-05-12T03:57:12.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:12.031+0000 7f75307d1700 1 -- 172.21.15.32:0/3254460983 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f7528109de0 con 0x7f7528065640 2024-05-12T03:57:12.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:12.031+0000 7f751affd700 1 -- 172.21.15.32:0/3254460983 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7524056620 con 0x7f7528065640 2024-05-12T03:57:12.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:12.033+0000 7f75307d1700 1 -- 172.21.15.32:0/3254460983 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f75140692b0 msgr2=0x7f751406b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:12.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:12.033+0000 7f75307d1700 1 --2- 172.21.15.32:0/3254460983 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f75140692b0 0x7f751406b760 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f752810eb90 tx=0x7f751c009e30 comp rx=0 tx=0).stop 2024-05-12T03:57:12.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:12.033+0000 7f75307d1700 1 -- 172.21.15.32:0/3254460983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7528065640 msgr2=0x7f7528113a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:12.292 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:12.033+0000 7f75307d1700 1 --2- 172.21.15.32:0/3254460983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7528065640 0x7f7528113a50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f752400eb70 tx=0x7f752400b040 comp rx=0 tx=0).stop 2024-05-12T03:57:12.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:12.033+0000 7f75307d1700 1 -- 172.21.15.32:0/3254460983 shutdown_connections 2024-05-12T03:57:12.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:12.033+0000 7f75307d1700 1 --2- 172.21.15.32:0/3254460983 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f75140692b0 0x7f751406b760 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:12.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:12.033+0000 7f75307d1700 1 --2- 172.21.15.32:0/3254460983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7528065640 0x7f7528113a50 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:12.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:12.033+0000 7f75307d1700 1 --2- 172.21.15.32:0/3254460983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7528102e50 0x7f7528113f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:12.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:12.033+0000 7f75307d1700 1 -- 172.21.15.32:0/3254460983 >> 172.21.15.32:0/3254460983 conn(0x7f7528071d00 msgr2=0x7f75280fa160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:57:12.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:12.033+0000 7f75307d1700 1 -- 172.21.15.32:0/3254460983 shutdown_connections 2024-05-12T03:57:12.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:12.034+0000 7f75307d1700 1 -- 172.21.15.32:0/3254460983 wait complete. 2024-05-12T03:57:12.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:12 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/3254460983' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-12T03:57:12.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:12 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/3254460983' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-12T03:57:13.332 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:13 smithi032 ceph-mon[40008]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:57:13.333 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888078 got 146028888081 for osd.4 2024-05-12T03:57:13.334 DEBUG:teuthology.parallel:result is None 2024-05-12T03:57:13.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:13 smithi033 ceph-mon[49525]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:57:13.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:12.880+0000 7ffa3c2e3700 1 -- 172.21.15.32:0/586668886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa34101cd0 msgr2=0x7ffa341020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:13.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:12.880+0000 7ffa3c2e3700 1 --2- 172.21.15.32:0/586668886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa34101cd0 0x7ffa341020a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ffa28004940 tx=0x7ffa28009e30 comp rx=0 tx=0).stop 2024-05-12T03:57:13.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:12.880+0000 7ffa3c2e3700 1 -- 172.21.15.32:0/586668886 shutdown_connections 2024-05-12T03:57:13.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:12.880+0000 7ffa3c2e3700 1 --2- 172.21.15.32:0/586668886 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa34102670 0x7ffa340fb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:13.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:12.880+0000 7ffa3c2e3700 1 --2- 172.21.15.32:0/586668886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa34101cd0 0x7ffa341020a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:13.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:12.880+0000 7ffa3c2e3700 1 -- 172.21.15.32:0/586668886 >> 172.21.15.32:0/586668886 conn(0x7ffa34071460 msgr2=0x7ffa34071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:57:13.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:12.880+0000 7ffa3c2e3700 1 -- 172.21.15.32:0/586668886 shutdown_connections 2024-05-12T03:57:13.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:12.880+0000 7ffa3c2e3700 1 -- 172.21.15.32:0/586668886 wait complete. 2024-05-12T03:57:13.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:12.881+0000 7ffa3c2e3700 1 Processor -- start 2024-05-12T03:57:13.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:12.881+0000 7ffa3c2e3700 1 -- start start 2024-05-12T03:57:13.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:12.881+0000 7ffa3c2e3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa34101cd0 0x7ffa340fc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:13.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:12.881+0000 7ffa3c2e3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa34102670 0x7ffa340fcd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:13.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:12.881+0000 7ffa3c2e3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa340fd2b0 con 0x7ffa34101cd0 2024-05-12T03:57:13.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:12.881+0000 7ffa3c2e3700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa340fd3f0 con 0x7ffa34102670 2024-05-12T03:57:13.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:12.881+0000 7ffa3a07f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa34101cd0 0x7ffa340fc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:13.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:12.881+0000 7ffa3a07f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa34101cd0 0x7ffa340fc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58480/0 (socket says 172.21.15.32:58480) 2024-05-12T03:57:13.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:12.881+0000 7ffa3a07f700 1 -- 172.21.15.32:0/4026509733 learned_addr learned my addr 172.21.15.32:0/4026509733 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:57:13.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:12.881+0000 7ffa3987e700 1 --2- 172.21.15.32:0/4026509733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa34102670 0x7ffa340fcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:13.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:12.881+0000 7ffa3a07f700 1 -- 172.21.15.32:0/4026509733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa34102670 msgr2=0x7ffa340fcd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:13.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:12.881+0000 7ffa3a07f700 1 --2- 172.21.15.32:0/4026509733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa34102670 0x7ffa340fcd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:13.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:12.881+0000 7ffa3a07f700 1 -- 172.21.15.32:0/4026509733 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa28005040 con 0x7ffa34101cd0 2024-05-12T03:57:13.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:12.882+0000 7ffa3a07f700 1 --2- 172.21.15.32:0/4026509733 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa34101cd0 0x7ffa340fc7e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ffa2800b3b0 tx=0x7ffa28009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:57:13.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:12.882+0000 7ffa26ffd700 1 -- 172.21.15.32:0/4026509733 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa2800e040 con 0x7ffa34101cd0 2024-05-12T03:57:14.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:12.882+0000 7ffa3c2e3700 1 -- 172.21.15.32:0/4026509733 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa340fd640 con 0x7ffa34101cd0 2024-05-12T03:57:14.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:12.882+0000 7ffa26ffd700 1 -- 172.21.15.32:0/4026509733 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ffa28013420 con 0x7ffa34101cd0 2024-05-12T03:57:14.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:12.882+0000 7ffa26ffd700 1 -- 172.21.15.32:0/4026509733 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa2801c630 con 0x7ffa34101cd0 2024-05-12T03:57:14.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:12.882+0000 7ffa3c2e3700 1 -- 172.21.15.32:0/4026509733 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa3411a1d0 con 0x7ffa34101cd0 2024-05-12T03:57:14.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:12.883+0000 7ffa3c2e3700 1 -- 172.21.15.32:0/4026509733 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa34065200 con 0x7ffa34101cd0 2024-05-12T03:57:14.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:12.883+0000 7ffa26ffd700 1 -- 172.21.15.32:0/4026509733 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7ffa28013970 con 0x7ffa34101cd0 2024-05-12T03:57:14.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:12.884+0000 7ffa26ffd700 1 --2- 172.21.15.32:0/4026509733 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7ffa200692a0 0x7ffa2006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:14.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:12.884+0000 7ffa26ffd700 1 -- 172.21.15.32:0/4026509733 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7ffa28088060 con 0x7ffa34101cd0 2024-05-12T03:57:14.000 INFO:teuthology.orchestra.run.smithi032.stdout:111669149721 2024-05-12T03:57:14.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:12.884+0000 7ffa3987e700 1 --2- 172.21.15.32:0/4026509733 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7ffa200692a0 0x7ffa2006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:14.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:12.884+0000 7ffa3987e700 1 --2- 172.21.15.32:0/4026509733 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7ffa200692a0 0x7ffa2006b750 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ffa30007fd0 tx=0x7ffa30009040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:57:14.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:12.888+0000 7ffa26ffd700 1 -- 172.21.15.32:0/4026509733 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ffa280546d0 con 0x7ffa34101cd0 2024-05-12T03:57:14.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:13.087+0000 7ffa3c2e3700 1 -- 172.21.15.32:0/4026509733 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ffa3404bed0 con 0x7ffa34101cd0 2024-05-12T03:57:14.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:13.087+0000 7ffa26ffd700 1 -- 172.21.15.32:0/4026509733 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ffa28057cf0 con 0x7ffa34101cd0 2024-05-12T03:57:14.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:13.089+0000 7ffa3c2e3700 1 -- 172.21.15.32:0/4026509733 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7ffa200692a0 msgr2=0x7ffa2006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:14.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:13.089+0000 7ffa3c2e3700 1 --2- 172.21.15.32:0/4026509733 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7ffa200692a0 0x7ffa2006b750 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ffa30007fd0 tx=0x7ffa30009040 comp rx=0 tx=0).stop 2024-05-12T03:57:14.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:13.089+0000 7ffa3c2e3700 1 -- 172.21.15.32:0/4026509733 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa34101cd0 msgr2=0x7ffa340fc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:14.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:13.089+0000 7ffa3c2e3700 1 --2- 172.21.15.32:0/4026509733 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa34101cd0 0x7ffa340fc7e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ffa2800b3b0 tx=0x7ffa28009e30 comp rx=0 tx=0).stop 2024-05-12T03:57:14.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:13.089+0000 7ffa3c2e3700 1 -- 172.21.15.32:0/4026509733 shutdown_connections 2024-05-12T03:57:14.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:13.089+0000 7ffa3c2e3700 1 --2- 172.21.15.32:0/4026509733 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7ffa200692a0 0x7ffa2006b750 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:14.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:13.089+0000 7ffa3c2e3700 1 --2- 172.21.15.32:0/4026509733 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa34101cd0 0x7ffa340fc7e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:14.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:13.089+0000 7ffa3c2e3700 1 --2- 172.21.15.32:0/4026509733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa34102670 0x7ffa340fcd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:14.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:13.089+0000 7ffa3c2e3700 1 -- 172.21.15.32:0/4026509733 >> 172.21.15.32:0/4026509733 conn(0x7ffa34071460 msgr2=0x7ffa340f9430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:57:14.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:13.089+0000 7ffa3c2e3700 1 -- 172.21.15.32:0/4026509733 shutdown_connections 2024-05-12T03:57:14.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:13.089+0000 7ffa3c2e3700 1 -- 172.21.15.32:0/4026509733 wait complete. 2024-05-12T03:57:14.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:14 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/4026509733' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-12T03:57:14.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:14 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/4026509733' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-12T03:57:14.779 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149718 got 111669149721 for osd.3 2024-05-12T03:57:14.779 DEBUG:teuthology.parallel:result is None 2024-05-12T03:57:15.332 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:15 smithi032 ceph-mon[40008]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:57:15.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:15 smithi033 ceph-mon[49525]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:57:15.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:15.564+0000 7fa44b593700 1 -- 172.21.15.32:0/4100521532 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa444101d90 msgr2=0x7fa444102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:15.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:15.564+0000 7fa44b593700 1 --2- 172.21.15.32:0/4100521532 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa444101d90 0x7fa444102160 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa440004940 tx=0x7fa440009e30 comp rx=0 tx=0).stop 2024-05-12T03:57:15.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:15.564+0000 7fa44b593700 1 -- 172.21.15.32:0/4100521532 shutdown_connections 2024-05-12T03:57:15.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:15.564+0000 7fa44b593700 1 --2- 172.21.15.32:0/4100521532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4440fbd40 0x7fa4440fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:15.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:15.564+0000 7fa44b593700 1 --2- 172.21.15.32:0/4100521532 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa444101d90 0x7fa444102160 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:15.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:15.564+0000 7fa44b593700 1 -- 172.21.15.32:0/4100521532 >> 172.21.15.32:0/4100521532 conn(0x7fa444071d00 msgr2=0x7fa444072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:57:15.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:15.564+0000 7fa44b593700 1 -- 172.21.15.32:0/4100521532 shutdown_connections 2024-05-12T03:57:15.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:15.564+0000 7fa44b593700 1 -- 172.21.15.32:0/4100521532 wait complete. 2024-05-12T03:57:15.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:15.565+0000 7fa44b593700 1 Processor -- start 2024-05-12T03:57:15.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:15.565+0000 7fa44b593700 1 -- start start 2024-05-12T03:57:15.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:15.565+0000 7fa44b593700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4440fbd40 0x7fa44410ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:15.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:15.565+0000 7fa44b593700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa444101d90 0x7fa44410f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:15.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:15.565+0000 7fa44b593700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa44410f630 con 0x7fa4440fbd40 2024-05-12T03:57:15.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:15.565+0000 7fa44b593700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa44410f770 con 0x7fa444101d90 2024-05-12T03:57:15.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:15.565+0000 7fa44932f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4440fbd40 0x7fa44410ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:15.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:15.565+0000 7fa44932f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4440fbd40 0x7fa44410ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58498/0 (socket says 172.21.15.32:58498) 2024-05-12T03:57:15.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:15.565+0000 7fa44932f700 1 -- 172.21.15.32:0/43919097 learned_addr learned my addr 172.21.15.32:0/43919097 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:57:15.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:15.565+0000 7fa448b2e700 1 --2- 172.21.15.32:0/43919097 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa444101d90 0x7fa44410f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:15.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:15.565+0000 7fa44932f700 1 -- 172.21.15.32:0/43919097 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa444101d90 msgr2=0x7fa44410f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:15.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:15.565+0000 7fa44932f700 1 --2- 172.21.15.32:0/43919097 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa444101d90 0x7fa44410f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:15.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:15.565+0000 7fa44932f700 1 -- 172.21.15.32:0/43919097 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa440005040 con 0x7fa4440fbd40 2024-05-12T03:57:15.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:15.565+0000 7fa44932f700 1 --2- 172.21.15.32:0/43919097 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4440fbd40 0x7fa44410ebb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa44000b3b0 tx=0x7fa440009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:57:15.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:15.566+0000 7fa439ffb700 1 -- 172.21.15.32:0/43919097 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa44000e040 con 0x7fa4440fbd40 2024-05-12T03:57:15.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:15.566+0000 7fa439ffb700 1 -- 172.21.15.32:0/43919097 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa440013420 con 0x7fa4440fbd40 2024-05-12T03:57:15.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:15.566+0000 7fa439ffb700 1 -- 172.21.15.32:0/43919097 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa44001c630 con 0x7fa4440fbd40 2024-05-12T03:57:15.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:15.566+0000 7fa44b593700 1 -- 172.21.15.32:0/43919097 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa444117a60 con 0x7fa4440fbd40 2024-05-12T03:57:15.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:15.566+0000 7fa44b593700 1 -- 172.21.15.32:0/43919097 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa444117ff0 con 0x7fa4440fbd40 2024-05-12T03:57:15.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:15.567+0000 7fa44b593700 1 -- 172.21.15.32:0/43919097 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4440fd430 con 0x7fa4440fbd40 2024-05-12T03:57:15.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:15.567+0000 7fa439ffb700 1 -- 172.21.15.32:0/43919097 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fa440013970 con 0x7fa4440fbd40 2024-05-12T03:57:15.934 INFO:teuthology.orchestra.run.smithi032.stdout:60129542180 2024-05-12T03:57:15.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:15.568+0000 7fa439ffb700 1 --2- 172.21.15.32:0/43919097 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fa4300692a0 0x7fa43006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:15.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:15.568+0000 7fa439ffb700 1 -- 172.21.15.32:0/43919097 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fa440088100 con 0x7fa4440fbd40 2024-05-12T03:57:15.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:15.568+0000 7fa448b2e700 1 --2- 172.21.15.32:0/43919097 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fa4300692a0 0x7fa43006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:15.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:15.568+0000 7fa448b2e700 1 --2- 172.21.15.32:0/43919097 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fa4300692a0 0x7fa43006b750 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa444066000 tx=0x7fa434009ee0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:57:15.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:15.572+0000 7fa439ffb700 1 -- 172.21.15.32:0/43919097 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fa440054770 con 0x7fa4440fbd40 2024-05-12T03:57:15.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:15.771+0000 7fa44b593700 1 -- 172.21.15.32:0/43919097 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa44404bf00 con 0x7fa4440fbd40 2024-05-12T03:57:15.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:15.771+0000 7fa439ffb700 1 -- 172.21.15.32:0/43919097 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa440057d90 con 0x7fa4440fbd40 2024-05-12T03:57:15.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:15.775+0000 7fa44b593700 1 -- 172.21.15.32:0/43919097 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fa4300692a0 msgr2=0x7fa43006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:15.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:15.775+0000 7fa44b593700 1 --2- 172.21.15.32:0/43919097 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fa4300692a0 0x7fa43006b750 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa444066000 tx=0x7fa434009ee0 comp rx=0 tx=0).stop 2024-05-12T03:57:15.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:15.775+0000 7fa44b593700 1 -- 172.21.15.32:0/43919097 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4440fbd40 msgr2=0x7fa44410ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:15.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:15.775+0000 7fa44b593700 1 --2- 172.21.15.32:0/43919097 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4440fbd40 0x7fa44410ebb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa44000b3b0 tx=0x7fa440009e30 comp rx=0 tx=0).stop 2024-05-12T03:57:15.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:15.775+0000 7fa44b593700 1 -- 172.21.15.32:0/43919097 shutdown_connections 2024-05-12T03:57:15.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:15.775+0000 7fa44b593700 1 --2- 172.21.15.32:0/43919097 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fa4300692a0 0x7fa43006b750 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:15.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:15.775+0000 7fa44b593700 1 --2- 172.21.15.32:0/43919097 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4440fbd40 0x7fa44410ebb0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:15.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:15.775+0000 7fa44b593700 1 --2- 172.21.15.32:0/43919097 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa444101d90 0x7fa44410f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:15.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:15.775+0000 7fa44b593700 1 -- 172.21.15.32:0/43919097 >> 172.21.15.32:0/43919097 conn(0x7fa444071d00 msgr2=0x7fa444104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:57:15.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:15.775+0000 7fa44b593700 1 -- 172.21.15.32:0/43919097 shutdown_connections 2024-05-12T03:57:15.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:15.775+0000 7fa44b593700 1 -- 172.21.15.32:0/43919097 wait complete. 2024-05-12T03:57:16.345 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:16 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/43919097' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-12T03:57:16.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:16 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/43919097' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-12T03:57:16.895 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542177 got 60129542180 for osd.1 2024-05-12T03:57:16.896 DEBUG:teuthology.parallel:result is None 2024-05-12T03:57:16.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:16.586+0000 7fcaafd8f700 1 -- 172.21.15.32:0/1909404936 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcaa8102de0 msgr2=0x7fcaa81031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:16.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:16.586+0000 7fcaafd8f700 1 --2- 172.21.15.32:0/1909404936 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcaa8102de0 0x7fcaa81031b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4004940 tx=0x7fcaa4009e30 comp rx=0 tx=0).stop 2024-05-12T03:57:16.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:16.586+0000 7fcaafd8f700 1 -- 172.21.15.32:0/1909404936 shutdown_connections 2024-05-12T03:57:16.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:16.586+0000 7fcaafd8f700 1 --2- 172.21.15.32:0/1909404936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcaa8103780 0x7fcaa80fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:16.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:16.586+0000 7fcaafd8f700 1 --2- 172.21.15.32:0/1909404936 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcaa8102de0 0x7fcaa81031b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:16.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:16.586+0000 7fcaafd8f700 1 -- 172.21.15.32:0/1909404936 >> 172.21.15.32:0/1909404936 conn(0x7fcaa80f8990 msgr2=0x7fcaa80fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:57:16.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:16.587+0000 7fcaafd8f700 1 -- 172.21.15.32:0/1909404936 shutdown_connections 2024-05-12T03:57:16.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:16.587+0000 7fcaafd8f700 1 -- 172.21.15.32:0/1909404936 wait complete. 2024-05-12T03:57:16.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:16.587+0000 7fcaafd8f700 1 Processor -- start 2024-05-12T03:57:16.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:16.587+0000 7fcaafd8f700 1 -- start start 2024-05-12T03:57:16.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:16.588+0000 7fcaafd8f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcaa8102de0 0x7fcaa8113860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:16.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:16.588+0000 7fcaafd8f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcaa8103780 0x7fcaa8113da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:16.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:16.588+0000 7fcaafd8f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa81143f0 con 0x7fcaa8103780 2024-05-12T03:57:16.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:16.588+0000 7fcaafd8f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa810d9f0 con 0x7fcaa8102de0 2024-05-12T03:57:17.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:16.588+0000 7fcaad32a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcaa8103780 0x7fcaa8113da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:17.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:16.588+0000 7fcaad32a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcaa8103780 0x7fcaa8113da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58510/0 (socket says 172.21.15.32:58510) 2024-05-12T03:57:17.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:16.588+0000 7fcaad32a700 1 -- 172.21.15.32:0/2767230868 learned_addr learned my addr 172.21.15.32:0/2767230868 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:57:17.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:16.588+0000 7fcaadb2b700 1 --2- 172.21.15.32:0/2767230868 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcaa8102de0 0x7fcaa8113860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:17.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:16.588+0000 7fcaad32a700 1 -- 172.21.15.32:0/2767230868 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcaa8102de0 msgr2=0x7fcaa8113860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:17.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:16.588+0000 7fcaad32a700 1 --2- 172.21.15.32:0/2767230868 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcaa8102de0 0x7fcaa8113860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:17.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:16.588+0000 7fcaad32a700 1 -- 172.21.15.32:0/2767230868 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa4005040 con 0x7fcaa8103780 2024-05-12T03:57:17.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:16.588+0000 7fcaad32a700 1 --2- 172.21.15.32:0/2767230868 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcaa8103780 0x7fcaa8113da0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fca98002a50 tx=0x7fca98004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:57:17.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:16.588+0000 7fca9e7fc700 1 -- 172.21.15.32:0/2767230868 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca98007cc0 con 0x7fcaa8103780 2024-05-12T03:57:17.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:16.588+0000 7fcaafd8f700 1 -- 172.21.15.32:0/2767230868 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaa810dcd0 con 0x7fcaa8103780 2024-05-12T03:57:17.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:16.588+0000 7fca9e7fc700 1 -- 172.21.15.32:0/2767230868 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fca9800f040 con 0x7fcaa8103780 2024-05-12T03:57:17.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:16.588+0000 7fca9e7fc700 1 -- 172.21.15.32:0/2767230868 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca9800d990 con 0x7fcaa8103780 2024-05-12T03:57:17.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:16.588+0000 7fcaafd8f700 1 -- 172.21.15.32:0/2767230868 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaa810e290 con 0x7fcaa8103780 2024-05-12T03:57:17.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:16.589+0000 7fcaafd8f700 1 -- 172.21.15.32:0/2767230868 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaa80fe530 con 0x7fcaa8103780 2024-05-12T03:57:17.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:16.590+0000 7fca9e7fc700 1 -- 172.21.15.32:0/2767230868 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fca98009710 con 0x7fcaa8103780 2024-05-12T03:57:17.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:16.590+0000 7fca9e7fc700 1 --2- 172.21.15.32:0/2767230868 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fca940692a0 0x7fca9406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:17.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:16.590+0000 7fca9e7fc700 1 -- 172.21.15.32:0/2767230868 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fca98086cd0 con 0x7fcaa8103780 2024-05-12T03:57:17.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:16.590+0000 7fcaadb2b700 1 --2- 172.21.15.32:0/2767230868 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fca940692a0 0x7fca9406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:17.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:16.591+0000 7fcaadb2b700 1 --2- 172.21.15.32:0/2767230868 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fca940692a0 0x7fca9406b750 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4004f90 tx=0x7fcaa4009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:57:17.001 INFO:teuthology.orchestra.run.smithi032.stdout:81604378654 2024-05-12T03:57:17.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:16.595+0000 7fca9e7fc700 1 -- 172.21.15.32:0/2767230868 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fca98056ac0 con 0x7fcaa8103780 2024-05-12T03:57:17.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:16.793+0000 7fcaafd8f700 1 -- 172.21.15.32:0/2767230868 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fcaa810eee0 con 0x7fcaa8103780 2024-05-12T03:57:17.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:16.793+0000 7fca9e7fc700 1 -- 172.21.15.32:0/2767230868 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fca980568e0 con 0x7fcaa8103780 2024-05-12T03:57:17.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:16.795+0000 7fcaafd8f700 1 -- 172.21.15.32:0/2767230868 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fca940692a0 msgr2=0x7fca9406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:17.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:16.795+0000 7fcaafd8f700 1 --2- 172.21.15.32:0/2767230868 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fca940692a0 0x7fca9406b750 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4004f90 tx=0x7fcaa4009e30 comp rx=0 tx=0).stop 2024-05-12T03:57:17.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:16.795+0000 7fcaafd8f700 1 -- 172.21.15.32:0/2767230868 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcaa8103780 msgr2=0x7fcaa8113da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:17.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:16.795+0000 7fcaafd8f700 1 --2- 172.21.15.32:0/2767230868 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcaa8103780 0x7fcaa8113da0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fca98002a50 tx=0x7fca98004600 comp rx=0 tx=0).stop 2024-05-12T03:57:17.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:16.796+0000 7fcaafd8f700 1 -- 172.21.15.32:0/2767230868 shutdown_connections 2024-05-12T03:57:17.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:16.796+0000 7fcaafd8f700 1 --2- 172.21.15.32:0/2767230868 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fca940692a0 0x7fca9406b750 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:17.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:16.796+0000 7fcaafd8f700 1 --2- 172.21.15.32:0/2767230868 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcaa8102de0 0x7fcaa8113860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:17.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:16.796+0000 7fcaafd8f700 1 --2- 172.21.15.32:0/2767230868 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcaa8103780 0x7fcaa8113da0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:17.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:16.796+0000 7fcaafd8f700 1 -- 172.21.15.32:0/2767230868 >> 172.21.15.32:0/2767230868 conn(0x7fcaa80f8990 msgr2=0x7fcaa80f99c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:57:17.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:16.796+0000 7fcaafd8f700 1 -- 172.21.15.32:0/2767230868 shutdown_connections 2024-05-12T03:57:17.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:16.796+0000 7fcaafd8f700 1 -- 172.21.15.32:0/2767230868 wait complete. 2024-05-12T03:57:17.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:17 smithi033 ceph-mon[49525]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:57:17.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:17 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/2767230868' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-12T03:57:17.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:17 smithi032 ceph-mon[40008]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:57:17.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:17 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/2767230868' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-12T03:57:17.595 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378652 got 81604378654 for osd.2 2024-05-12T03:57:17.596 DEBUG:teuthology.parallel:result is None 2024-05-12T03:57:17.596 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-12T03:57:17.596 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-12T03:57:17.874 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T03:57:19.381 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:19 smithi032 ceph-mon[40008]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:57:19.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:19 smithi033 ceph-mon[49525]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:57:19.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:19.636+0000 7f54796d6700 1 -- 172.21.15.32:0/2984186514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54741037c0 msgr2=0x7f54740fb3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:19.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:19.636+0000 7f54796d6700 1 --2- 172.21.15.32:0/2984186514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54741037c0 0x7f54740fb3c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5468004940 tx=0x7f5468009e30 comp rx=0 tx=0).stop 2024-05-12T03:57:19.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:19.636+0000 7f54796d6700 1 -- 172.21.15.32:0/2984186514 shutdown_connections 2024-05-12T03:57:19.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:19.636+0000 7f54796d6700 1 --2- 172.21.15.32:0/2984186514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54741037c0 0x7f54740fb3c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:19.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:19.636+0000 7f54796d6700 1 --2- 172.21.15.32:0/2984186514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5474102e20 0x7f54741031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:19.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:19.636+0000 7f54796d6700 1 -- 172.21.15.32:0/2984186514 >> 172.21.15.32:0/2984186514 conn(0x7f5474072380 msgr2=0x7f5474072780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:57:19.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:19.637+0000 7f54796d6700 1 -- 172.21.15.32:0/2984186514 shutdown_connections 2024-05-12T03:57:19.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:19.637+0000 7f54796d6700 1 -- 172.21.15.32:0/2984186514 wait complete. 2024-05-12T03:57:19.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:19.637+0000 7f54796d6700 1 Processor -- start 2024-05-12T03:57:19.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:19.637+0000 7f54796d6700 1 -- start start 2024-05-12T03:57:19.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:19.637+0000 7f54796d6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5474102e20 0x7f54741138e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:19.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:19.637+0000 7f54796d6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54741037c0 0x7f5474113e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:19.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:19.638+0000 7f54796d6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5474114470 con 0x7f54741037c0 2024-05-12T03:57:19.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:19.638+0000 7f54796d6700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f547410da70 con 0x7f5474102e20 2024-05-12T03:57:19.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:19.638+0000 7f54727fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54741037c0 0x7f5474113e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:19.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:19.638+0000 7f54727fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54741037c0 0x7f5474113e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58514/0 (socket says 172.21.15.32:58514) 2024-05-12T03:57:19.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:19.638+0000 7f54727fc700 1 -- 172.21.15.32:0/1766452456 learned_addr learned my addr 172.21.15.32:0/1766452456 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:57:19.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:19.638+0000 7f5472ffd700 1 --2- 172.21.15.32:0/1766452456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5474102e20 0x7f54741138e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:19.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:19.638+0000 7f54727fc700 1 -- 172.21.15.32:0/1766452456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5474102e20 msgr2=0x7f54741138e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:19.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:19.638+0000 7f54727fc700 1 --2- 172.21.15.32:0/1766452456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5474102e20 0x7f54741138e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:19.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:19.638+0000 7f54727fc700 1 -- 172.21.15.32:0/1766452456 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5468005040 con 0x7f54741037c0 2024-05-12T03:57:19.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:19.638+0000 7f54727fc700 1 --2- 172.21.15.32:0/1766452456 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54741037c0 0x7f5474113e20 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f54680093b0 tx=0x7f5468009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:57:19.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:19.638+0000 7f545b7fe700 1 -- 172.21.15.32:0/1766452456 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f546800e040 con 0x7f54741037c0 2024-05-12T03:57:19.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:19.638+0000 7f54796d6700 1 -- 172.21.15.32:0/1766452456 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f547410dcf0 con 0x7f54741037c0 2024-05-12T03:57:19.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:19.638+0000 7f545b7fe700 1 -- 172.21.15.32:0/1766452456 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5468013420 con 0x7f54741037c0 2024-05-12T03:57:19.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:19.638+0000 7f545b7fe700 1 -- 172.21.15.32:0/1766452456 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f546801c630 con 0x7f54741037c0 2024-05-12T03:57:19.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:19.638+0000 7f54796d6700 1 -- 172.21.15.32:0/1766452456 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f547410e230 con 0x7f54741037c0 2024-05-12T03:57:19.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:19.639+0000 7f54796d6700 1 -- 172.21.15.32:0/1766452456 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54740fe6e0 con 0x7f54741037c0 2024-05-12T03:57:19.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:19.640+0000 7f545b7fe700 1 -- 172.21.15.32:0/1766452456 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f54680139e0 con 0x7f54741037c0 2024-05-12T03:57:19.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:19.640+0000 7f545b7fe700 1 --2- 172.21.15.32:0/1766452456 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f545c0692a0 0x7f545c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:19.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:19.640+0000 7f545b7fe700 1 -- 172.21.15.32:0/1766452456 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f5468087ea0 con 0x7f54741037c0 2024-05-12T03:57:19.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:19.640+0000 7f5472ffd700 1 --2- 172.21.15.32:0/1766452456 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f545c0692a0 0x7f545c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:19.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:19.641+0000 7f5472ffd700 1 --2- 172.21.15.32:0/1766452456 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f545c0692a0 0x7f545c06b750 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5474065e90 tx=0x7f5464009040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:57:19.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:19.645+0000 7f545b7fe700 1 -- 172.21.15.32:0/1766452456 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f54680580f0 con 0x7f54741037c0 2024-05-12T03:57:19.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:19.835+0000 7f54796d6700 1 -- 172.21.15.32:0/1766452456 --> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5474109de0 con 0x7f545c0692a0 2024-05-12T03:57:19.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:19.837+0000 7f545b7fe700 1 -- 172.21.15.32:0/1766452456 <== mgr.14208 v2:172.21.15.32:6800/4089797100 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+28033 (secure 0 0 0) 0x7f5474109de0 con 0x7f545c0692a0 2024-05-12T03:57:19.838 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:57:19.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:19.839+0000 7f54796d6700 1 -- 172.21.15.32:0/1766452456 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f545c0692a0 msgr2=0x7f545c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:19.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:19.839+0000 7f54796d6700 1 --2- 172.21.15.32:0/1766452456 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f545c0692a0 0x7f545c06b750 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5474065e90 tx=0x7f5464009040 comp rx=0 tx=0).stop 2024-05-12T03:57:19.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:19.839+0000 7f54796d6700 1 -- 172.21.15.32:0/1766452456 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54741037c0 msgr2=0x7f5474113e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:19.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:19.839+0000 7f54796d6700 1 --2- 172.21.15.32:0/1766452456 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54741037c0 0x7f5474113e20 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f54680093b0 tx=0x7f5468009e30 comp rx=0 tx=0).stop 2024-05-12T03:57:19.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:19.839+0000 7f54796d6700 1 -- 172.21.15.32:0/1766452456 shutdown_connections 2024-05-12T03:57:19.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:19.839+0000 7f54796d6700 1 --2- 172.21.15.32:0/1766452456 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f545c0692a0 0x7f545c06b750 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:19.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:19.839+0000 7f54796d6700 1 --2- 172.21.15.32:0/1766452456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5474102e20 0x7f54741138e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:19.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:19.839+0000 7f54796d6700 1 --2- 172.21.15.32:0/1766452456 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54741037c0 0x7f5474113e20 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:19.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:19.839+0000 7f54796d6700 1 -- 172.21.15.32:0/1766452456 >> 172.21.15.32:0/1766452456 conn(0x7f5474072380 msgr2=0x7f54740f9f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:57:19.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:19.839+0000 7f54796d6700 1 -- 172.21.15.32:0/1766452456 shutdown_connections 2024-05-12T03:57:19.840 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:19.839+0000 7f54796d6700 1 -- 172.21.15.32:0/1766452456 wait complete. 2024-05-12T03:57:19.841 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-05-12T03:57:20.457 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":168,"stamp":"2024-05-12T03:57:19.825878+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":1788020,"kb_used_data":3868,"kb_used_omap":9,"kb_used_meta":1784054,"kb_avail":560682892,"statfs":{"total":575970213888,"available":574139281408,"internally_reserved":0,"allocated":3960832,"data_stored":2605278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10224,"internal_metadata":1826871312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001849"},"pg_stats":[{"pgid":"1.0","version":"22'117","reported_seq":189,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-12T03:56:23.883509+0000","last_change":"2024-05-12T03:55:53.879553+0000","last_active":"2024-05-12T03:56:23.883509+0000","last_peered":"2024-05-12T03:56:23.883509+0000","last_clean":"2024-05-12T03:56:23.883509+0000","last_became_active":"2024-05-12T03:55:53.879241+0000","last_became_peered":"2024-05-12T03:55:53.879241+0000","last_unstale":"2024-05-12T03:56:23.883509+0000","last_undegraded":"2024-05-12T03:56:23.883509+0000","last_fullsized":"2024-05-12T03:56:23.883509+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-12T03:54:58.296798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-12T03:54:58.296798+0000","last_clean_scrub_stamp":"2024-05-12T03:54:58.296798+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T07:40:49.777112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":39,"seq":167503724558,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297712,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447440,"statfs":{"total":95995035648,"available":95690178560,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]}]},{"osd":4,"up_from":34,"seq":146028888083,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297712,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447440,"statfs":{"total":95995035648,"available":95690178560,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun May 12 03:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.59399999999999997}]},{"osd":1,"last update":"Sun May 12 03:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.51700000000000002}]},{"osd":2,"last update":"Sun May 12 03:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.57099999999999995}]},{"osd":3,"last update":"Sun May 12 03:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.47699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46700000000000003}]}]},{"osd":3,"up_from":26,"seq":111669149722,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298288,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446864,"statfs":{"total":95995035648,"available":95689588736,"internally_reserved":0,"allocated":954368,"data_stored":729238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun May 12 03:56:24 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.46800000000000003}]},{"osd":1,"last update":"Sun May 12 03:56:24 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.48999999999999999}]},{"osd":2,"last update":"Sun May 12 03:56:24 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.42899999999999999}]},{"osd":4,"last update":"Sun May 12 03:56:55 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.55700000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]}]},{"osd":2,"up_from":19,"seq":81604378655,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297716,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447436,"statfs":{"total":95995035648,"available":95690174464,"internally_reserved":0,"allocated":360448,"data_stored":138532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun May 12 03:57:03 2024","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.36299999999999999,"15min":0.36499999999999999},"min":{"1min":0.246,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.46300000000000002,"5min":0.502,"15min":0.502},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.33300000000000002,"5min":0.379,"15min":0.38700000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.45600000000000002,"5min":0.59099999999999997,"15min":0.59099999999999997},"last":0.379}]},{"osd":1,"last update":"Sun May 12 03:57:03 2024","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.38600000000000001,"15min":0.39200000000000002},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.54400000000000004,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.39500000000000002,"15min":0.40200000000000002},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.52800000000000002,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.438}]},{"osd":3,"last update":"Sun May 12 03:56:24 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.69099999999999995}]},{"osd":4,"last update":"Sun May 12 03:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.61799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]}]},{"osd":0,"up_from":9,"seq":38654705708,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298296,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446856,"statfs":{"total":95995035648,"available":95689580544,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun May 12 03:56:31 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.32000000000000001,"15min":0.312},"min":{"1min":0.26400000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.496,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.379,"5min":0.34399999999999997,"15min":0.33800000000000002},"min":{"1min":0.26900000000000002,"5min":0.224,"15min":0.224},"max":{"1min":0.54600000000000004,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.50700000000000001}]},{"osd":2,"last update":"Sun May 12 03:57:01 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.39100000000000001,"15min":0.39400000000000002},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.52300000000000002,"5min":0.622,"15min":0.622},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.374,"5min":0.40699999999999997,"15min":0.41299999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.55700000000000005,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.54800000000000004}]},{"osd":3,"last update":"Sun May 12 03:56:29 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.70699999999999996}]},{"osd":4,"last update":"Sun May 12 03:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.751},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.77200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]}]},{"osd":1,"up_from":14,"seq":60129542181,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298296,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446856,"statfs":{"total":95995035648,"available":95689580544,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun May 12 03:56:33 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.33400000000000002,"15min":0.32800000000000001},"min":{"1min":0.26600000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.35699999999999998,"15min":0.35299999999999998},"min":{"1min":0.25900000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.36199999999999999}]},{"osd":2,"last update":"Sun May 12 03:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.379,"15min":0.38100000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"last":0.38},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.40600000000000003,"15min":0.40799999999999997},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.40100000000000002}]},{"osd":3,"last update":"Sun May 12 03:56:24 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.46400000000000002}]},{"osd":4,"last update":"Sun May 12 03:56:55 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.66100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-12T03:57:20.459 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-12T03:57:20.739 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T03:57:21.230 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:21 smithi032 ceph-mon[40008]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:57:21.230 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:21 smithi032 ceph-mon[40008]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-12T03:57:21.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:21 smithi033 ceph-mon[49525]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:57:21.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:21 smithi033 ceph-mon[49525]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-12T03:57:22.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:22.598+0000 7effed93c700 1 -- 172.21.15.32:0/2577829941 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effe8101d00 msgr2=0x7effe81020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:22.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:22.598+0000 7effed93c700 1 --2- 172.21.15.32:0/2577829941 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effe8101d00 0x7effe81020d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7effd8004940 tx=0x7effd8009e30 comp rx=0 tx=0).stop 2024-05-12T03:57:22.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:22.598+0000 7effed93c700 1 -- 172.21.15.32:0/2577829941 shutdown_connections 2024-05-12T03:57:22.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:22.598+0000 7effed93c700 1 --2- 172.21.15.32:0/2577829941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effe81026a0 0x7effe80fbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:22.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:22.598+0000 7effed93c700 1 --2- 172.21.15.32:0/2577829941 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effe8101d00 0x7effe81020d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:22.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:22.598+0000 7effed93c700 1 -- 172.21.15.32:0/2577829941 >> 172.21.15.32:0/2577829941 conn(0x7effe80f7870 msgr2=0x7effe80f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:57:22.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:22.598+0000 7effed93c700 1 -- 172.21.15.32:0/2577829941 shutdown_connections 2024-05-12T03:57:22.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:22.598+0000 7effed93c700 1 -- 172.21.15.32:0/2577829941 wait complete. 2024-05-12T03:57:22.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:22.598+0000 7effed93c700 1 Processor -- start 2024-05-12T03:57:22.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:22.599+0000 7effed93c700 1 -- start start 2024-05-12T03:57:22.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:22.599+0000 7effed93c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effe8101d00 0x7effe81138e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:22.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:22.599+0000 7effed93c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effe81026a0 0x7effe8113e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:22.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:22.599+0000 7effed93c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe8114470 con 0x7effe81026a0 2024-05-12T03:57:22.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:22.599+0000 7effed93c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe810da70 con 0x7effe8101d00 2024-05-12T03:57:22.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:22.599+0000 7effe67fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effe81026a0 0x7effe8113e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:22.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:22.599+0000 7effe67fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effe81026a0 0x7effe8113e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:40994/0 (socket says 172.21.15.32:40994) 2024-05-12T03:57:22.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:22.599+0000 7effe67fc700 1 -- 172.21.15.32:0/1764479842 learned_addr learned my addr 172.21.15.32:0/1764479842 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:57:22.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:22.599+0000 7effe6ffd700 1 --2- 172.21.15.32:0/1764479842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effe8101d00 0x7effe81138e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:22.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:22.599+0000 7effe67fc700 1 -- 172.21.15.32:0/1764479842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effe8101d00 msgr2=0x7effe81138e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:22.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:22.599+0000 7effe67fc700 1 --2- 172.21.15.32:0/1764479842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effe8101d00 0x7effe81138e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:22.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:22.599+0000 7effe67fc700 1 -- 172.21.15.32:0/1764479842 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd8005040 con 0x7effe81026a0 2024-05-12T03:57:22.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:22.599+0000 7effe67fc700 1 --2- 172.21.15.32:0/1764479842 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effe81026a0 0x7effe8113e20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7effdc00c800 tx=0x7effdc004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:57:22.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:22.599+0000 7effcf7fe700 1 -- 172.21.15.32:0/1764479842 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7effdc0106f0 con 0x7effe81026a0 2024-05-12T03:57:22.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:22.600+0000 7effcf7fe700 1 -- 172.21.15.32:0/1764479842 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7effdc010d00 con 0x7effe81026a0 2024-05-12T03:57:22.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:22.600+0000 7effcf7fe700 1 -- 172.21.15.32:0/1764479842 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7effdc0113a0 con 0x7effe81026a0 2024-05-12T03:57:22.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:22.600+0000 7effed93c700 1 -- 172.21.15.32:0/1764479842 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effe810dd50 con 0x7effe81026a0 2024-05-12T03:57:22.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:22.600+0000 7effed93c700 1 -- 172.21.15.32:0/1764479842 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effe810e310 con 0x7effe81026a0 2024-05-12T03:57:22.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:22.600+0000 7effed93c700 1 -- 172.21.15.32:0/1764479842 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effe80fd450 con 0x7effe81026a0 2024-05-12T03:57:22.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:22.601+0000 7effcf7fe700 1 -- 172.21.15.32:0/1764479842 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7effdc010850 con 0x7effe81026a0 2024-05-12T03:57:22.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:22.602+0000 7effcf7fe700 1 --2- 172.21.15.32:0/1764479842 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7effd00692a0 0x7effd006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:22.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:22.602+0000 7effcf7fe700 1 -- 172.21.15.32:0/1764479842 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7effdc087960 con 0x7effe81026a0 2024-05-12T03:57:22.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:22.602+0000 7effe6ffd700 1 --2- 172.21.15.32:0/1764479842 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7effd00692a0 0x7effd006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:22.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:22.602+0000 7effe6ffd700 1 --2- 172.21.15.32:0/1764479842 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7effd00692a0 0x7effd006b750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7effd8004f90 tx=0x7effd8009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:57:22.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:22.606+0000 7effcf7fe700 1 -- 172.21.15.32:0/1764479842 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7effdc057b00 con 0x7effe81026a0 2024-05-12T03:57:22.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:22.795+0000 7effed93c700 1 -- 172.21.15.32:0/1764479842 --> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7effe8108d60 con 0x7effd00692a0 2024-05-12T03:57:22.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:22.796+0000 7effcf7fe700 1 -- 172.21.15.32:0/1764479842 <== mgr.14208 v2:172.21.15.32:6800/4089797100 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+28033 (secure 0 0 0) 0x7effd8007e00 con 0x7effd00692a0 2024-05-12T03:57:22.798 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:57:22.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:22.798+0000 7effed93c700 1 -- 172.21.15.32:0/1764479842 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7effd00692a0 msgr2=0x7effd006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:22.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:22.798+0000 7effed93c700 1 --2- 172.21.15.32:0/1764479842 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7effd00692a0 0x7effd006b750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7effd8004f90 tx=0x7effd8009e30 comp rx=0 tx=0).stop 2024-05-12T03:57:22.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:22.798+0000 7effed93c700 1 -- 172.21.15.32:0/1764479842 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effe81026a0 msgr2=0x7effe8113e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:22.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:22.798+0000 7effed93c700 1 --2- 172.21.15.32:0/1764479842 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effe81026a0 0x7effe8113e20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7effdc00c800 tx=0x7effdc004470 comp rx=0 tx=0).stop 2024-05-12T03:57:22.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:22.798+0000 7effed93c700 1 -- 172.21.15.32:0/1764479842 shutdown_connections 2024-05-12T03:57:22.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:22.798+0000 7effed93c700 1 --2- 172.21.15.32:0/1764479842 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7effd00692a0 0x7effd006b750 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:22.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:22.798+0000 7effed93c700 1 --2- 172.21.15.32:0/1764479842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effe8101d00 0x7effe81138e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:22.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:22.798+0000 7effed93c700 1 --2- 172.21.15.32:0/1764479842 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effe81026a0 0x7effe8113e20 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:22.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:22.798+0000 7effed93c700 1 -- 172.21.15.32:0/1764479842 >> 172.21.15.32:0/1764479842 conn(0x7effe80f7870 msgr2=0x7effe8066290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:57:22.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:22.799+0000 7effed93c700 1 -- 172.21.15.32:0/1764479842 shutdown_connections 2024-05-12T03:57:22.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:22.799+0000 7effed93c700 1 -- 172.21.15.32:0/1764479842 wait complete. 2024-05-12T03:57:22.801 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-05-12T03:57:23.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:23 smithi033 ceph-mon[49525]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:57:23.492 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:23 smithi032 ceph-mon[40008]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:57:23.516 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":169,"stamp":"2024-05-12T03:57:21.826225+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":1788020,"kb_used_data":3868,"kb_used_omap":9,"kb_used_meta":1784054,"kb_avail":560682892,"statfs":{"total":575970213888,"available":574139281408,"internally_reserved":0,"allocated":3960832,"data_stored":2605278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10224,"internal_metadata":1826871312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001796"},"pg_stats":[{"pgid":"1.0","version":"22'117","reported_seq":189,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-12T03:56:23.883509+0000","last_change":"2024-05-12T03:55:53.879553+0000","last_active":"2024-05-12T03:56:23.883509+0000","last_peered":"2024-05-12T03:56:23.883509+0000","last_clean":"2024-05-12T03:56:23.883509+0000","last_became_active":"2024-05-12T03:55:53.879241+0000","last_became_peered":"2024-05-12T03:55:53.879241+0000","last_unstale":"2024-05-12T03:56:23.883509+0000","last_undegraded":"2024-05-12T03:56:23.883509+0000","last_fullsized":"2024-05-12T03:56:23.883509+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-12T03:54:58.296798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-12T03:54:58.296798+0000","last_clean_scrub_stamp":"2024-05-12T03:54:58.296798+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T07:40:49.777112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":39,"seq":167503724559,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297712,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447440,"statfs":{"total":95995035648,"available":95690178560,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]}]},{"osd":4,"up_from":34,"seq":146028888084,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297712,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447440,"statfs":{"total":95995035648,"available":95690178560,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun May 12 03:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.29899999999999999}]},{"osd":1,"last update":"Sun May 12 03:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.44900000000000001}]},{"osd":2,"last update":"Sun May 12 03:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.40600000000000003}]},{"osd":3,"last update":"Sun May 12 03:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.42699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]}]},{"osd":3,"up_from":26,"seq":111669149723,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298288,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446864,"statfs":{"total":95995035648,"available":95689588736,"internally_reserved":0,"allocated":954368,"data_stored":729238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun May 12 03:56:24 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.40200000000000002}]},{"osd":1,"last update":"Sun May 12 03:56:24 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.48499999999999999}]},{"osd":2,"last update":"Sun May 12 03:56:24 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.41499999999999998}]},{"osd":4,"last update":"Sun May 12 03:56:55 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.50800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]}]},{"osd":2,"up_from":19,"seq":81604378655,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297716,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447436,"statfs":{"total":95995035648,"available":95690174464,"internally_reserved":0,"allocated":360448,"data_stored":138532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun May 12 03:57:03 2024","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.36299999999999999,"15min":0.36499999999999999},"min":{"1min":0.246,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.46300000000000002,"5min":0.502,"15min":0.502},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.33300000000000002,"5min":0.379,"15min":0.38700000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.45600000000000002,"5min":0.59099999999999997,"15min":0.59099999999999997},"last":0.379}]},{"osd":1,"last update":"Sun May 12 03:57:03 2024","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.38600000000000001,"15min":0.39200000000000002},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.54400000000000004,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.39500000000000002,"15min":0.40200000000000002},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.52800000000000002,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.438}]},{"osd":3,"last update":"Sun May 12 03:56:24 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.69099999999999995}]},{"osd":4,"last update":"Sun May 12 03:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.61799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]}]},{"osd":0,"up_from":9,"seq":38654705708,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298296,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446856,"statfs":{"total":95995035648,"available":95689580544,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun May 12 03:56:31 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.32000000000000001,"15min":0.312},"min":{"1min":0.26400000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.496,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.379,"5min":0.34399999999999997,"15min":0.33800000000000002},"min":{"1min":0.26900000000000002,"5min":0.224,"15min":0.224},"max":{"1min":0.54600000000000004,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.50700000000000001}]},{"osd":2,"last update":"Sun May 12 03:57:01 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.39100000000000001,"15min":0.39400000000000002},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.52300000000000002,"5min":0.622,"15min":0.622},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.374,"5min":0.40699999999999997,"15min":0.41299999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.55700000000000005,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.54800000000000004}]},{"osd":3,"last update":"Sun May 12 03:56:29 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.70699999999999996}]},{"osd":4,"last update":"Sun May 12 03:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.751},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.77200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]}]},{"osd":1,"up_from":14,"seq":60129542181,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298296,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446856,"statfs":{"total":95995035648,"available":95689580544,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun May 12 03:56:33 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.33400000000000002,"15min":0.32800000000000001},"min":{"1min":0.26600000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.35699999999999998,"15min":0.35299999999999998},"min":{"1min":0.25900000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.36199999999999999}]},{"osd":2,"last update":"Sun May 12 03:57:06 2024","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.379,"15min":0.38100000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"last":0.38},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.40600000000000003,"15min":0.40799999999999997},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.40100000000000002}]},{"osd":3,"last update":"Sun May 12 03:56:24 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.46400000000000002}]},{"osd":4,"last update":"Sun May 12 03:56:55 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.66100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-12T03:57:23.517 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-12T03:57:23.517 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-12T03:57:23.517 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-12T03:57:23.517 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- ceph health --format=json 2024-05-12T03:57:23.801 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T03:57:24.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:24 smithi032 ceph-mon[40008]: from='client.14422 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-12T03:57:24.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:24 smithi033 ceph-mon[49525]: from='client.14422 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-12T03:57:25.388 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:25 smithi032 ceph-mon[40008]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:57:25.388 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:25 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T03:57:25.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:25 smithi033 ceph-mon[49525]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:57:25.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:25 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T03:57:25.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:25.656+0000 7f8c60646700 1 -- 172.21.15.32:0/1012922543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c580fdde0 msgr2=0x7f8c580fe1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:25.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:25.656+0000 7f8c60646700 1 --2- 172.21.15.32:0/1012922543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c580fdde0 0x7f8c580fe1b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c004940 tx=0x7f8c4c009e30 comp rx=0 tx=0).stop 2024-05-12T03:57:25.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:25.656+0000 7f8c60646700 1 -- 172.21.15.32:0/1012922543 shutdown_connections 2024-05-12T03:57:25.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:25.656+0000 7f8c60646700 1 --2- 172.21.15.32:0/1012922543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c580fe780 0x7f8c58106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:25.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:25.656+0000 7f8c60646700 1 --2- 172.21.15.32:0/1012922543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c580fdde0 0x7f8c580fe1b0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:25.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:25.656+0000 7f8c60646700 1 -- 172.21.15.32:0/1012922543 >> 172.21.15.32:0/1012922543 conn(0x7f8c580f7890 msgr2=0x7f8c580f9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:57:25.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:25.656+0000 7f8c60646700 1 -- 172.21.15.32:0/1012922543 shutdown_connections 2024-05-12T03:57:25.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:25.656+0000 7f8c60646700 1 -- 172.21.15.32:0/1012922543 wait complete. 2024-05-12T03:57:25.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:25.657+0000 7f8c60646700 1 Processor -- start 2024-05-12T03:57:25.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:25.657+0000 7f8c60646700 1 -- start start 2024-05-12T03:57:25.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:25.657+0000 7f8c60646700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c580fdde0 0x7f8c58115bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:25.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:25.657+0000 7f8c60646700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c580fe780 0x7f8c58116100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:25.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:25.657+0000 7f8c60646700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c58116750 con 0x7f8c580fdde0 2024-05-12T03:57:25.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:25.657+0000 7f8c60646700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5810fbd0 con 0x7f8c580fe780 2024-05-12T03:57:25.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:25.657+0000 7f8c5e3e2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c580fdde0 0x7f8c58115bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:25.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:25.657+0000 7f8c5e3e2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c580fdde0 0x7f8c58115bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41014/0 (socket says 172.21.15.32:41014) 2024-05-12T03:57:25.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:25.657+0000 7f8c5e3e2700 1 -- 172.21.15.32:0/1047709737 learned_addr learned my addr 172.21.15.32:0/1047709737 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:57:25.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:25.657+0000 7f8c5dbe1700 1 --2- 172.21.15.32:0/1047709737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c580fe780 0x7f8c58116100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:25.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:25.657+0000 7f8c5e3e2700 1 -- 172.21.15.32:0/1047709737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c580fe780 msgr2=0x7f8c58116100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:25.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:25.657+0000 7f8c5e3e2700 1 --2- 172.21.15.32:0/1047709737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c580fe780 0x7f8c58116100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:25.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:25.657+0000 7f8c5e3e2700 1 -- 172.21.15.32:0/1047709737 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c4c005040 con 0x7f8c580fdde0 2024-05-12T03:57:25.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:25.658+0000 7f8c5e3e2700 1 --2- 172.21.15.32:0/1047709737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c580fdde0 0x7f8c58115bc0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c009380 tx=0x7f8c4c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:57:25.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:25.658+0000 7f8c4affd700 1 -- 172.21.15.32:0/1047709737 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c4c00e040 con 0x7f8c580fdde0 2024-05-12T03:57:25.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:25.658+0000 7f8c4affd700 1 -- 172.21.15.32:0/1047709737 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8c4c004290 con 0x7f8c580fdde0 2024-05-12T03:57:25.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:25.658+0000 7f8c4affd700 1 -- 172.21.15.32:0/1047709737 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c4c013e90 con 0x7f8c580fdde0 2024-05-12T03:57:25.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:25.658+0000 7f8c60646700 1 -- 172.21.15.32:0/1047709737 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c5810fe50 con 0x7f8c580fdde0 2024-05-12T03:57:25.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:25.658+0000 7f8c60646700 1 -- 172.21.15.32:0/1047709737 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c58110390 con 0x7f8c580fdde0 2024-05-12T03:57:25.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:25.659+0000 7f8c60646700 1 -- 172.21.15.32:0/1047709737 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c58066750 con 0x7f8c580fdde0 2024-05-12T03:57:25.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:25.659+0000 7f8c4affd700 1 -- 172.21.15.32:0/1047709737 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f8c4c0136b0 con 0x7f8c580fdde0 2024-05-12T03:57:25.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:25.660+0000 7f8c4affd700 1 --2- 172.21.15.32:0/1047709737 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f8c440692a0 0x7f8c4406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:25.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:25.660+0000 7f8c4affd700 1 -- 172.21.15.32:0/1047709737 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f8c4c088120 con 0x7f8c580fdde0 2024-05-12T03:57:25.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:25.660+0000 7f8c5dbe1700 1 --2- 172.21.15.32:0/1047709737 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f8c440692a0 0x7f8c4406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:25.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:25.660+0000 7f8c5dbe1700 1 --2- 172.21.15.32:0/1047709737 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f8c440692a0 0x7f8c4406b750 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8c58103840 tx=0x7f8c5400b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:57:25.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:25.664+0000 7f8c4affd700 1 -- 172.21.15.32:0/1047709737 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f8c4c054790 con 0x7f8c580fdde0 2024-05-12T03:57:25.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:25.903+0000 7f8c60646700 1 -- 172.21.15.32:0/1047709737 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f8c5805f5c0 con 0x7f8c580fdde0 2024-05-12T03:57:25.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:25.903+0000 7f8c4affd700 1 -- 172.21.15.32:0/1047709737 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f8c4c057db0 con 0x7f8c580fdde0 2024-05-12T03:57:25.904 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:57:25.905 INFO:teuthology.orchestra.run.smithi032.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-12T03:57:25.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:25.905+0000 7f8c60646700 1 -- 172.21.15.32:0/1047709737 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f8c440692a0 msgr2=0x7f8c4406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:25.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:25.905+0000 7f8c60646700 1 --2- 172.21.15.32:0/1047709737 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f8c440692a0 0x7f8c4406b750 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8c58103840 tx=0x7f8c5400b040 comp rx=0 tx=0).stop 2024-05-12T03:57:25.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:25.905+0000 7f8c60646700 1 -- 172.21.15.32:0/1047709737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c580fdde0 msgr2=0x7f8c58115bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:25.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:25.905+0000 7f8c60646700 1 --2- 172.21.15.32:0/1047709737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c580fdde0 0x7f8c58115bc0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c009380 tx=0x7f8c4c009e30 comp rx=0 tx=0).stop 2024-05-12T03:57:25.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:25.905+0000 7f8c60646700 1 -- 172.21.15.32:0/1047709737 shutdown_connections 2024-05-12T03:57:25.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:25.905+0000 7f8c60646700 1 --2- 172.21.15.32:0/1047709737 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f8c440692a0 0x7f8c4406b750 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:25.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:25.905+0000 7f8c60646700 1 --2- 172.21.15.32:0/1047709737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c580fdde0 0x7f8c58115bc0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:25.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:25.905+0000 7f8c60646700 1 --2- 172.21.15.32:0/1047709737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c580fe780 0x7f8c58116100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:25.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:25.905+0000 7f8c60646700 1 -- 172.21.15.32:0/1047709737 >> 172.21.15.32:0/1047709737 conn(0x7f8c580f7890 msgr2=0x7f8c58106960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:57:25.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:25.905+0000 7f8c60646700 1 -- 172.21.15.32:0/1047709737 shutdown_connections 2024-05-12T03:57:25.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:25.906+0000 7f8c60646700 1 -- 172.21.15.32:0/1047709737 wait complete. 2024-05-12T03:57:26.232 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:26 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/1047709737' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-12T03:57:26.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:26 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/1047709737' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-12T03:57:26.541 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-12T03:57:26.541 INFO:tasks.cephadm:Setup complete, yielding 2024-05-12T03:57:26.542 INFO:teuthology.run_tasks:Running task print... 2024-05-12T03:57:26.552 INFO:teuthology.task.print:**** done end installing quincy cephadm ... 2024-05-12T03:57:26.552 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-12T03:57:26.593 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi032.front.sepia.ceph.com 2024-05-12T03:57:26.594 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- bash -c 'ceph config set mgr mgr/cephadm/use_repo_digest true --force' 2024-05-12T03:57:26.870 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T03:57:27.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:27 smithi033 ceph-mon[49525]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:57:27.400 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:27 smithi032 ceph-mon[40008]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:57:28.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:28 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T03:57:28.517 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:28 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T03:57:28.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:28.774+0000 7f7dba274700 1 -- 172.21.15.32:0/4287115826 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7db40fce60 msgr2=0x7f7db40fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:28.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:28.774+0000 7f7dba274700 1 --2- 172.21.15.32:0/4287115826 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7db40fce60 0x7f7db40fd270 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c004940 tx=0x7f7d9c009e30 comp rx=0 tx=0).stop 2024-05-12T03:57:28.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:28.774+0000 7f7dba274700 1 -- 172.21.15.32:0/4287115826 shutdown_connections 2024-05-12T03:57:28.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:28.774+0000 7f7dba274700 1 --2- 172.21.15.32:0/4287115826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7db40fdfd0 0x7f7db40fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:28.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:28.774+0000 7f7dba274700 1 --2- 172.21.15.32:0/4287115826 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7db40fce60 0x7f7db40fd270 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:28.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:28.774+0000 7f7dba274700 1 -- 172.21.15.32:0/4287115826 >> 172.21.15.32:0/4287115826 conn(0x7f7db40f8540 msgr2=0x7f7db40fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:57:28.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:28.774+0000 7f7dba274700 1 -- 172.21.15.32:0/4287115826 shutdown_connections 2024-05-12T03:57:28.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:28.774+0000 7f7dba274700 1 -- 172.21.15.32:0/4287115826 wait complete. 2024-05-12T03:57:28.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:28.775+0000 7f7dba274700 1 Processor -- start 2024-05-12T03:57:28.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:28.775+0000 7f7dba274700 1 -- start start 2024-05-12T03:57:28.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:28.775+0000 7f7dba274700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7db40fce60 0x7f7db410d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:28.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:28.775+0000 7f7dba274700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7db40fdfd0 0x7f7db410de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:28.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:28.775+0000 7f7dba274700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db410e460 con 0x7f7db40fce60 2024-05-12T03:57:28.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:28.775+0000 7f7dba274700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db410ecb0 con 0x7f7db40fdfd0 2024-05-12T03:57:28.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:28.775+0000 7f7db3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7db40fce60 0x7f7db410d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:28.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:28.775+0000 7f7db3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7db40fce60 0x7f7db410d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41034/0 (socket says 172.21.15.32:41034) 2024-05-12T03:57:28.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:28.775+0000 7f7db3fff700 1 -- 172.21.15.32:0/2034340100 learned_addr learned my addr 172.21.15.32:0/2034340100 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:57:28.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:28.775+0000 7f7db37fe700 1 --2- 172.21.15.32:0/2034340100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7db40fdfd0 0x7f7db410de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:28.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:28.775+0000 7f7db3fff700 1 -- 172.21.15.32:0/2034340100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7db40fdfd0 msgr2=0x7f7db410de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:28.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:28.775+0000 7f7db3fff700 1 --2- 172.21.15.32:0/2034340100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7db40fdfd0 0x7f7db410de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:28.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:28.775+0000 7f7db3fff700 1 -- 172.21.15.32:0/2034340100 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d9c005040 con 0x7f7db40fce60 2024-05-12T03:57:28.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:28.775+0000 7f7db37fe700 1 --2- 172.21.15.32:0/2034340100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7db40fdfd0 0x7f7db410de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-12T03:57:28.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:28.776+0000 7f7db3fff700 1 --2- 172.21.15.32:0/2034340100 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7db40fce60 0x7f7db410d920 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c00b3b0 tx=0x7f7d9c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:57:28.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:28.776+0000 7f7db0ff9700 1 -- 172.21.15.32:0/2034340100 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d9c00e040 con 0x7f7db40fce60 2024-05-12T03:57:28.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:28.776+0000 7f7dba274700 1 -- 172.21.15.32:0/2034340100 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7db410ef30 con 0x7f7db40fce60 2024-05-12T03:57:28.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:28.776+0000 7f7db0ff9700 1 -- 172.21.15.32:0/2034340100 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7d9c00bd20 con 0x7f7db40fce60 2024-05-12T03:57:28.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:28.776+0000 7f7db0ff9700 1 -- 172.21.15.32:0/2034340100 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d9c01c630 con 0x7f7db40fce60 2024-05-12T03:57:28.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:28.776+0000 7f7dba274700 1 -- 172.21.15.32:0/2034340100 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7db410f470 con 0x7f7db40fce60 2024-05-12T03:57:28.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:28.777+0000 7f7dba274700 1 -- 172.21.15.32:0/2034340100 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7db405f5c0 con 0x7f7db40fce60 2024-05-12T03:57:28.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:28.777+0000 7f7db0ff9700 1 -- 172.21.15.32:0/2034340100 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f7d9c013420 con 0x7f7db40fce60 2024-05-12T03:57:28.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:28.778+0000 7f7db0ff9700 1 --2- 172.21.15.32:0/2034340100 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f7da00692a0 0x7f7da006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:28.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:28.778+0000 7f7db0ff9700 1 -- 172.21.15.32:0/2034340100 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f7d9c087c40 con 0x7f7db40fce60 2024-05-12T03:57:28.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:28.778+0000 7f7db37fe700 1 --2- 172.21.15.32:0/2034340100 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f7da00692a0 0x7f7da006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:28.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:28.778+0000 7f7db37fe700 1 --2- 172.21.15.32:0/2034340100 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f7da00692a0 0x7f7da006b750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7da4009fd0 tx=0x7f7da4009ee0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:57:28.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:28.782+0000 7f7db0ff9700 1 -- 172.21.15.32:0/2034340100 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f7d9c057cc0 con 0x7f7db40fce60 2024-05-12T03:57:28.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:28.977+0000 7f7dba274700 1 -- 172.21.15.32:0/2034340100 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/use_repo_digest}] v 0) v1 -- 0x7f7db404bed0 con 0x7f7db40fce60 2024-05-12T03:57:28.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:28.982+0000 7f7db0ff9700 1 -- 172.21.15.32:0/2034340100 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/use_repo_digest}]=0 v20)=0 v20) v1 ==== 143+0+0 (secure 0 0 0) 0x7f7d9c057850 con 0x7f7db40fce60 2024-05-12T03:57:28.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:28.984+0000 7f7dba274700 1 -- 172.21.15.32:0/2034340100 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f7da00692a0 msgr2=0x7f7da006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:28.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:28.984+0000 7f7dba274700 1 --2- 172.21.15.32:0/2034340100 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f7da00692a0 0x7f7da006b750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7da4009fd0 tx=0x7f7da4009ee0 comp rx=0 tx=0).stop 2024-05-12T03:57:28.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:28.984+0000 7f7dba274700 1 -- 172.21.15.32:0/2034340100 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7db40fce60 msgr2=0x7f7db410d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:28.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:28.984+0000 7f7dba274700 1 --2- 172.21.15.32:0/2034340100 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7db40fce60 0x7f7db410d920 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c00b3b0 tx=0x7f7d9c009e30 comp rx=0 tx=0).stop 2024-05-12T03:57:28.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:28.984+0000 7f7dba274700 1 -- 172.21.15.32:0/2034340100 shutdown_connections 2024-05-12T03:57:28.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:28.984+0000 7f7dba274700 1 --2- 172.21.15.32:0/2034340100 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f7da00692a0 0x7f7da006b750 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:28.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:28.984+0000 7f7dba274700 1 --2- 172.21.15.32:0/2034340100 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7db40fce60 0x7f7db410d920 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:28.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:28.984+0000 7f7dba274700 1 --2- 172.21.15.32:0/2034340100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7db40fdfd0 0x7f7db410de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:28.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:28.984+0000 7f7dba274700 1 -- 172.21.15.32:0/2034340100 >> 172.21.15.32:0/2034340100 conn(0x7f7db40f8540 msgr2=0x7f7db40f9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:57:28.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:28.984+0000 7f7dba274700 1 -- 172.21.15.32:0/2034340100 shutdown_connections 2024-05-12T03:57:28.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:28.984+0000 7f7dba274700 1 -- 172.21.15.32:0/2034340100 wait complete. 2024-05-12T03:57:29.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:29 smithi032 ceph-mon[40008]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:57:29.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:29 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:57:29.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:29 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T03:57:29.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:29 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:57:29.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:29 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/2034340100' entity='client.admin' 2024-05-12T03:57:29.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:29 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T03:57:29.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:29 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:57:29.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:29 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T03:57:29.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:29 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:57:29.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:29 smithi033 ceph-mon[49525]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:57:29.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:29 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:57:29.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:29 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T03:57:29.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:29 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:57:29.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:29 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/2034340100' entity='client.admin' 2024-05-12T03:57:29.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:29 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T03:57:29.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:29 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:57:29.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:29 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T03:57:29.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:29 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:57:29.663 INFO:teuthology.run_tasks:Running task print... 2024-05-12T03:57:29.673 INFO:teuthology.task.print:**** done cephadm.shell ceph config set mgr... 2024-05-12T03:57:29.673 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-12T03:57:29.710 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi032.front.sepia.ceph.com 2024-05-12T03:57:29.710 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- bash -c 'ceph orch status' 2024-05-12T03:57:29.992 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T03:57:31.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:31 smithi032 ceph-mon[40008]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:57:31.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:31 smithi033 ceph-mon[49525]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:57:31.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:31.942+0000 7fc44a8f1700 1 -- 172.21.15.32:0/2484895538 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4440fff60 msgr2=0x7fc444102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:31.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:31.942+0000 7fc44a8f1700 1 --2- 172.21.15.32:0/2484895538 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4440fff60 0x7fc444102340 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc438004940 tx=0x7fc438009e30 comp rx=0 tx=0).stop 2024-05-12T03:57:31.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:31.942+0000 7fc44a8f1700 1 -- 172.21.15.32:0/2484895538 shutdown_connections 2024-05-12T03:57:31.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:31.942+0000 7fc44a8f1700 1 --2- 172.21.15.32:0/2484895538 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4440fff60 0x7fc444102340 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:31.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:31.942+0000 7fc44a8f1700 1 --2- 172.21.15.32:0/2484895538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc444066240 0x7fc4440ffa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:31.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:31.942+0000 7fc44a8f1700 1 -- 172.21.15.32:0/2484895538 >> 172.21.15.32:0/2484895538 conn(0x7fc4440f7430 msgr2=0x7fc4440f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:57:31.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:31.942+0000 7fc44a8f1700 1 -- 172.21.15.32:0/2484895538 shutdown_connections 2024-05-12T03:57:31.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:31.942+0000 7fc44a8f1700 1 -- 172.21.15.32:0/2484895538 wait complete. 2024-05-12T03:57:31.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:31.943+0000 7fc44a8f1700 1 Processor -- start 2024-05-12T03:57:31.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:31.943+0000 7fc44a8f1700 1 -- start start 2024-05-12T03:57:31.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:31.943+0000 7fc44a8f1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc444066240 0x7fc44410d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:31.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:31.943+0000 7fc44a8f1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4440fff60 0x7fc44410de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:31.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:31.943+0000 7fc44a8f1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44410e460 con 0x7fc444066240 2024-05-12T03:57:31.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:31.943+0000 7fc44a8f1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44410ecb0 con 0x7fc4440fff60 2024-05-12T03:57:31.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:31.943+0000 7fc443fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc444066240 0x7fc44410d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:31.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:31.943+0000 7fc443fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc444066240 0x7fc44410d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42418/0 (socket says 172.21.15.32:42418) 2024-05-12T03:57:31.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:31.943+0000 7fc443fff700 1 -- 172.21.15.32:0/2314683502 learned_addr learned my addr 172.21.15.32:0/2314683502 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:57:31.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:31.943+0000 7fc4437fe700 1 --2- 172.21.15.32:0/2314683502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4440fff60 0x7fc44410de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:31.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:31.943+0000 7fc443fff700 1 -- 172.21.15.32:0/2314683502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4440fff60 msgr2=0x7fc44410de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:31.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:31.943+0000 7fc443fff700 1 --2- 172.21.15.32:0/2314683502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4440fff60 0x7fc44410de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:31.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:31.944+0000 7fc443fff700 1 -- 172.21.15.32:0/2314683502 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc438005040 con 0x7fc444066240 2024-05-12T03:57:31.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:31.944+0000 7fc443fff700 1 --2- 172.21.15.32:0/2314683502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc444066240 0x7fc44410d920 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc434002a50 tx=0x7fc434004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:57:31.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:31.944+0000 7fc440ff9700 1 -- 172.21.15.32:0/2314683502 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc434007cc0 con 0x7fc444066240 2024-05-12T03:57:31.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:31.944+0000 7fc44a8f1700 1 -- 172.21.15.32:0/2314683502 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc44410ef90 con 0x7fc444066240 2024-05-12T03:57:31.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:31.944+0000 7fc440ff9700 1 -- 172.21.15.32:0/2314683502 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc43400f040 con 0x7fc444066240 2024-05-12T03:57:31.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:31.944+0000 7fc440ff9700 1 -- 172.21.15.32:0/2314683502 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc43400d8d0 con 0x7fc444066240 2024-05-12T03:57:31.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:31.944+0000 7fc44a8f1700 1 -- 172.21.15.32:0/2314683502 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc44410f4d0 con 0x7fc444066240 2024-05-12T03:57:31.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:31.945+0000 7fc44a8f1700 1 -- 172.21.15.32:0/2314683502 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc44405f5c0 con 0x7fc444066240 2024-05-12T03:57:31.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:31.945+0000 7fc440ff9700 1 -- 172.21.15.32:0/2314683502 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fc43400db40 con 0x7fc444066240 2024-05-12T03:57:31.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:31.946+0000 7fc440ff9700 1 --2- 172.21.15.32:0/2314683502 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fc42c0692a0 0x7fc42c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:31.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:31.946+0000 7fc440ff9700 1 -- 172.21.15.32:0/2314683502 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fc434086bc0 con 0x7fc444066240 2024-05-12T03:57:31.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:31.946+0000 7fc4437fe700 1 --2- 172.21.15.32:0/2314683502 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fc42c0692a0 0x7fc42c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:31.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:31.946+0000 7fc4437fe700 1 --2- 172.21.15.32:0/2314683502 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fc42c0692a0 0x7fc42c06b750 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc43800b900 tx=0x7fc438009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:57:31.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:31.950+0000 7fc440ff9700 1 -- 172.21.15.32:0/2314683502 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fc434056c40 con 0x7fc444066240 2024-05-12T03:57:32.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:32.160+0000 7fc44a8f1700 1 -- 172.21.15.32:0/2314683502 --> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc44410f850 con 0x7fc42c0692a0 2024-05-12T03:57:32.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:32.161+0000 7fc440ff9700 1 -- 172.21.15.32:0/2314683502 <== mgr.14208 v2:172.21.15.32:6800/4089797100 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fc44410f850 con 0x7fc42c0692a0 2024-05-12T03:57:32.162 INFO:teuthology.orchestra.run.smithi032.stdout:Backend: cephadm 2024-05-12T03:57:32.162 INFO:teuthology.orchestra.run.smithi032.stdout:Available: Yes 2024-05-12T03:57:32.162 INFO:teuthology.orchestra.run.smithi032.stdout:Paused: No 2024-05-12T03:57:32.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:32.164+0000 7fc44a8f1700 1 -- 172.21.15.32:0/2314683502 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fc42c0692a0 msgr2=0x7fc42c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:32.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:32.164+0000 7fc44a8f1700 1 --2- 172.21.15.32:0/2314683502 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fc42c0692a0 0x7fc42c06b750 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc43800b900 tx=0x7fc438009e30 comp rx=0 tx=0).stop 2024-05-12T03:57:32.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:32.164+0000 7fc44a8f1700 1 -- 172.21.15.32:0/2314683502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc444066240 msgr2=0x7fc44410d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:32.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:32.164+0000 7fc44a8f1700 1 --2- 172.21.15.32:0/2314683502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc444066240 0x7fc44410d920 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc434002a50 tx=0x7fc434004600 comp rx=0 tx=0).stop 2024-05-12T03:57:32.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:32.164+0000 7fc44a8f1700 1 -- 172.21.15.32:0/2314683502 shutdown_connections 2024-05-12T03:57:32.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:32.164+0000 7fc44a8f1700 1 --2- 172.21.15.32:0/2314683502 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fc42c0692a0 0x7fc42c06b750 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:32.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:32.164+0000 7fc44a8f1700 1 --2- 172.21.15.32:0/2314683502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc444066240 0x7fc44410d920 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:32.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:32.164+0000 7fc44a8f1700 1 --2- 172.21.15.32:0/2314683502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4440fff60 0x7fc44410de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:32.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:32.164+0000 7fc44a8f1700 1 -- 172.21.15.32:0/2314683502 >> 172.21.15.32:0/2314683502 conn(0x7fc4440f7430 msgr2=0x7fc4440f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:57:32.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:32.164+0000 7fc44a8f1700 1 -- 172.21.15.32:0/2314683502 shutdown_connections 2024-05-12T03:57:32.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:32.164+0000 7fc44a8f1700 1 -- 172.21.15.32:0/2314683502 wait complete. 2024-05-12T03:57:32.871 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-12T03:57:33.152 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T03:57:33.378 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:33 smithi032 ceph-mon[40008]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:57:33.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:33 smithi033 ceph-mon[49525]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:57:34.402 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:34 smithi032 ceph-mon[40008]: from='client.14434 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T03:57:34.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:34 smithi033 ceph-mon[49525]: from='client.14434 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T03:57:34.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:34.983+0000 7f11e345e700 1 -- 172.21.15.32:0/1395769774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11dc0fce80 msgr2=0x7f11dc0fd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:34.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:34.983+0000 7f11e345e700 1 --2- 172.21.15.32:0/1395769774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11dc0fce80 0x7f11dc0fd290 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f11cc004940 tx=0x7f11cc009e30 comp rx=0 tx=0).stop 2024-05-12T03:57:34.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:34.983+0000 7f11e345e700 1 -- 172.21.15.32:0/1395769774 shutdown_connections 2024-05-12T03:57:34.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:34.983+0000 7f11e345e700 1 --2- 172.21.15.32:0/1395769774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11dc0fdff0 0x7f11dc0fe460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:34.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:34.983+0000 7f11e345e700 1 --2- 172.21.15.32:0/1395769774 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11dc0fce80 0x7f11dc0fd290 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:34.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:34.983+0000 7f11e345e700 1 -- 172.21.15.32:0/1395769774 >> 172.21.15.32:0/1395769774 conn(0x7f11dc0f8520 msgr2=0x7f11dc0fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:57:34.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:34.983+0000 7f11e345e700 1 -- 172.21.15.32:0/1395769774 shutdown_connections 2024-05-12T03:57:34.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:34.983+0000 7f11e345e700 1 -- 172.21.15.32:0/1395769774 wait complete. 2024-05-12T03:57:34.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:34.984+0000 7f11e345e700 1 Processor -- start 2024-05-12T03:57:34.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:34.984+0000 7f11e345e700 1 -- start start 2024-05-12T03:57:34.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:34.984+0000 7f11e345e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11dc0fce80 0x7f11dc10d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:34.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:34.984+0000 7f11e345e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11dc0fdff0 0x7f11dc10de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:34.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:34.984+0000 7f11e345e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11dc10e450 con 0x7f11dc0fce80 2024-05-12T03:57:34.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:34.984+0000 7f11e345e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11dc10eca0 con 0x7f11dc0fdff0 2024-05-12T03:57:34.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:34.984+0000 7f11e11fa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11dc0fce80 0x7f11dc10d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:34.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:34.984+0000 7f11e11fa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11dc0fce80 0x7f11dc10d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42438/0 (socket says 172.21.15.32:42438) 2024-05-12T03:57:34.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:34.984+0000 7f11e11fa700 1 -- 172.21.15.32:0/2705242658 learned_addr learned my addr 172.21.15.32:0/2705242658 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:57:34.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:34.984+0000 7f11e11fa700 1 -- 172.21.15.32:0/2705242658 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11dc0fdff0 msgr2=0x7f11dc10de50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-12T03:57:34.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:34.984+0000 7f11e11fa700 1 --2- 172.21.15.32:0/2705242658 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11dc0fdff0 0x7f11dc10de50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:34.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:34.984+0000 7f11e11fa700 1 -- 172.21.15.32:0/2705242658 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11cc005040 con 0x7f11dc0fce80 2024-05-12T03:57:34.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:34.984+0000 7f11e11fa700 1 --2- 172.21.15.32:0/2705242658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11dc0fce80 0x7f11dc10d910 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f11cc00b3b0 tx=0x7f11cc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:57:34.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:34.985+0000 7f11d1ffb700 1 -- 172.21.15.32:0/2705242658 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11cc00e040 con 0x7f11dc0fce80 2024-05-12T03:57:34.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:34.985+0000 7f11d1ffb700 1 -- 172.21.15.32:0/2705242658 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f11cc00bd20 con 0x7f11dc0fce80 2024-05-12T03:57:34.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:34.985+0000 7f11e345e700 1 -- 172.21.15.32:0/2705242658 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11dc10ef20 con 0x7f11dc0fce80 2024-05-12T03:57:34.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:34.985+0000 7f11d1ffb700 1 -- 172.21.15.32:0/2705242658 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11cc01c630 con 0x7f11dc0fce80 2024-05-12T03:57:34.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:34.985+0000 7f11e345e700 1 -- 172.21.15.32:0/2705242658 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11dc10f460 con 0x7f11dc0fce80 2024-05-12T03:57:34.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:34.986+0000 7f11e345e700 1 -- 172.21.15.32:0/2705242658 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11dc05f5c0 con 0x7f11dc0fce80 2024-05-12T03:57:34.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:34.986+0000 7f11d1ffb700 1 -- 172.21.15.32:0/2705242658 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f11cc013420 con 0x7f11dc0fce80 2024-05-12T03:57:34.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:34.987+0000 7f11d1ffb700 1 --2- 172.21.15.32:0/2705242658 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f11c80692a0 0x7f11c806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:34.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:34.987+0000 7f11d1ffb700 1 -- 172.21.15.32:0/2705242658 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f11cc087cc0 con 0x7f11dc0fce80 2024-05-12T03:57:34.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:34.987+0000 7f11e09f9700 1 --2- 172.21.15.32:0/2705242658 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f11c80692a0 0x7f11c806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:34.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:34.987+0000 7f11e09f9700 1 --2- 172.21.15.32:0/2705242658 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f11c80692a0 0x7f11c806b750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f11d8004b70 tx=0x7f11d8007cc0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:57:34.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:34.991+0000 7f11d1ffb700 1 -- 172.21.15.32:0/2705242658 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f11cc08d020 con 0x7f11dc0fce80 2024-05-12T03:57:35.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:35.197+0000 7f11e345e700 1 -- 172.21.15.32:0/2705242658 --> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f11dc10f720 con 0x7f11c80692a0 2024-05-12T03:57:35.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:35.205+0000 7f11d1ffb700 1 -- 172.21.15.32:0/2705242658 <== mgr.14208 v2:172.21.15.32:6800/4089797100 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+2718 (secure 0 0 0) 0x7f11dc10f720 con 0x7f11c80692a0 2024-05-12T03:57:35.205 INFO:teuthology.orchestra.run.smithi032.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-12T03:57:35.205 INFO:teuthology.orchestra.run.smithi032.stdout:alertmanager.smithi032 smithi032 *:9093,9094 running (4m) 2m ago 5m 25.6M - 0.25.0 c8568f914cd2 5c80dfe17d2e 2024-05-12T03:57:35.205 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi032 smithi032 running (5m) 2m ago 5m 6651k - 17.2.7-939-gc190bdee 284803f7d9cf a478874e9dfa 2024-05-12T03:57:35.205 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi033 smithi033 running (4m) 72s ago 4m 6651k - 17.2.7-939-gc190bdee 284803f7d9cf 34c62d004c7e 2024-05-12T03:57:35.205 INFO:teuthology.orchestra.run.smithi032.stdout:grafana.smithi032 smithi032 *:3000 running (4m) 2m ago 5m 76.8M - 9.4.7 954c08fa6188 ac9076563478 2024-05-12T03:57:35.205 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi032.ukatkq smithi032 *:9283 running (7m) 2m ago 7m 474M - 17.2.7-939-gc190bdee 284803f7d9cf d9763dc47404 2024-05-12T03:57:35.205 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi033.pcvxwy smithi033 *:8443,9283 running (4m) 72s ago 4m 433M - 17.2.7-939-gc190bdee 284803f7d9cf 70d285a35adb 2024-05-12T03:57:35.205 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi032 smithi032 running (7m) 2m ago 7m 42.5M 2048M 17.2.7-939-gc190bdee 284803f7d9cf d7fea430c6e2 2024-05-12T03:57:35.206 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi033 smithi033 running (4m) 72s ago 4m 38.9M 2048M 17.2.7-939-gc190bdee 284803f7d9cf 707654d85748 2024-05-12T03:57:35.206 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi032 smithi032 *:9100 running (5m) 2m ago 5m 17.5M - 1.5.0 0da6a335fe13 6aa48ce262de 2024-05-12T03:57:35.206 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi033 smithi033 *:9100 running (4m) 72s ago 4m 16.7M - 1.5.0 0da6a335fe13 1144caed00e9 2024-05-12T03:57:35.206 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0 smithi032 running (3m) 2m ago 3m 38.1M 4300M 17.2.7-939-gc190bdee 284803f7d9cf 75c78a47f8e5 2024-05-12T03:57:35.206 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1 smithi032 running (3m) 2m ago 3m 41.4M 4300M 17.2.7-939-gc190bdee 284803f7d9cf 87cfd247d8d6 2024-05-12T03:57:35.206 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2 smithi032 running (2m) 2m ago 2m 40.0M 4300M 17.2.7-939-gc190bdee 284803f7d9cf 7aef7affb8ba 2024-05-12T03:57:35.206 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3 smithi033 running (2m) 72s ago 2m 39.1M 5324M 17.2.7-939-gc190bdee 284803f7d9cf 3a92b5e3348e 2024-05-12T03:57:35.206 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4 smithi033 running (108s) 72s ago 107s 39.5M 5324M 17.2.7-939-gc190bdee 284803f7d9cf 0896268b4591 2024-05-12T03:57:35.206 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5 smithi033 running (78s) 72s ago 77s 10.1M 5324M 17.2.7-939-gc190bdee 284803f7d9cf 57e65a64c821 2024-05-12T03:57:35.206 INFO:teuthology.orchestra.run.smithi032.stdout:prometheus.smithi032 smithi032 *:9095 running (3m) 2m ago 5m 38.1M - 2.43.0 a07b618ecd1d 0b3ce3323a49 2024-05-12T03:57:35.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:35.207+0000 7f11e345e700 1 -- 172.21.15.32:0/2705242658 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f11c80692a0 msgr2=0x7f11c806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:35.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:35.207+0000 7f11e345e700 1 --2- 172.21.15.32:0/2705242658 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f11c80692a0 0x7f11c806b750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f11d8004b70 tx=0x7f11d8007cc0 comp rx=0 tx=0).stop 2024-05-12T03:57:35.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:35.207+0000 7f11e345e700 1 -- 172.21.15.32:0/2705242658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11dc0fce80 msgr2=0x7f11dc10d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:35.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:35.207+0000 7f11e345e700 1 --2- 172.21.15.32:0/2705242658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11dc0fce80 0x7f11dc10d910 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f11cc00b3b0 tx=0x7f11cc009e30 comp rx=0 tx=0).stop 2024-05-12T03:57:35.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:35.207+0000 7f11e345e700 1 -- 172.21.15.32:0/2705242658 shutdown_connections 2024-05-12T03:57:35.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:35.207+0000 7f11e345e700 1 --2- 172.21.15.32:0/2705242658 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f11c80692a0 0x7f11c806b750 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:35.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:35.207+0000 7f11e345e700 1 --2- 172.21.15.32:0/2705242658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11dc0fce80 0x7f11dc10d910 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:35.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:35.207+0000 7f11e345e700 1 --2- 172.21.15.32:0/2705242658 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11dc0fdff0 0x7f11dc10de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:35.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:35.207+0000 7f11e345e700 1 -- 172.21.15.32:0/2705242658 >> 172.21.15.32:0/2705242658 conn(0x7f11dc0f8520 msgr2=0x7f11dc0f9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:57:35.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:35.207+0000 7f11e345e700 1 -- 172.21.15.32:0/2705242658 shutdown_connections 2024-05-12T03:57:35.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:35.207+0000 7f11e345e700 1 -- 172.21.15.32:0/2705242658 wait complete. 2024-05-12T03:57:35.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:35 smithi033 ceph-mon[49525]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:57:35.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:35 smithi032 ceph-mon[40008]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:57:35.951 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-12T03:57:36.232 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T03:57:37.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:37 smithi033 ceph-mon[49525]: from='client.14438 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T03:57:37.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:37 smithi033 ceph-mon[49525]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:57:37.458 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:37 smithi032 ceph-mon[40008]: from='client.14438 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T03:57:37.459 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:37 smithi032 ceph-mon[40008]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:57:38.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:38.040+0000 7f3d3a521700 1 -- 172.21.15.32:0/2306467685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d340fcf60 msgr2=0x7f3d340fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:38.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:38.040+0000 7f3d3a521700 1 --2- 172.21.15.32:0/2306467685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d340fcf60 0x7f3d340fd3d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3d28004940 tx=0x7f3d28009e30 comp rx=0 tx=0).stop 2024-05-12T03:57:38.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:38.040+0000 7f3d3a521700 1 -- 172.21.15.32:0/2306467685 shutdown_connections 2024-05-12T03:57:38.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:38.040+0000 7f3d3a521700 1 --2- 172.21.15.32:0/2306467685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d340fcf60 0x7f3d340fd3d0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:38.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:38.040+0000 7f3d3a521700 1 --2- 172.21.15.32:0/2306467685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d340fbcb0 0x7f3d340fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:38.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:38.040+0000 7f3d3a521700 1 -- 172.21.15.32:0/2306467685 >> 172.21.15.32:0/2306467685 conn(0x7f3d340f7430 msgr2=0x7f3d340f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:57:38.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:38.040+0000 7f3d3a521700 1 -- 172.21.15.32:0/2306467685 shutdown_connections 2024-05-12T03:57:38.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:38.040+0000 7f3d3a521700 1 -- 172.21.15.32:0/2306467685 wait complete. 2024-05-12T03:57:38.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:38.040+0000 7f3d3a521700 1 Processor -- start 2024-05-12T03:57:38.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:38.041+0000 7f3d3a521700 1 -- start start 2024-05-12T03:57:38.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:38.041+0000 7f3d3a521700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d340fbcb0 0x7f3d3410c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:38.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:38.041+0000 7f3d3a521700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d340fcf60 0x7f3d3410ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:38.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:38.041+0000 7f3d3a521700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3410d2c0 con 0x7f3d340fcf60 2024-05-12T03:57:38.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:38.041+0000 7f3d3a521700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3410db10 con 0x7f3d340fbcb0 2024-05-12T03:57:38.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:38.041+0000 7f3d337fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d340fcf60 0x7f3d3410ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:38.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:38.041+0000 7f3d337fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d340fcf60 0x7f3d3410ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42452/0 (socket says 172.21.15.32:42452) 2024-05-12T03:57:38.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:38.041+0000 7f3d337fe700 1 -- 172.21.15.32:0/2077948461 learned_addr learned my addr 172.21.15.32:0/2077948461 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:57:38.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:38.041+0000 7f3d33fff700 1 --2- 172.21.15.32:0/2077948461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d340fbcb0 0x7f3d3410c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:38.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:38.041+0000 7f3d337fe700 1 -- 172.21.15.32:0/2077948461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d340fbcb0 msgr2=0x7f3d3410c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:38.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:38.041+0000 7f3d337fe700 1 --2- 172.21.15.32:0/2077948461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d340fbcb0 0x7f3d3410c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:38.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:38.041+0000 7f3d337fe700 1 -- 172.21.15.32:0/2077948461 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d28005040 con 0x7f3d340fcf60 2024-05-12T03:57:38.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:38.041+0000 7f3d337fe700 1 --2- 172.21.15.32:0/2077948461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d340fcf60 0x7f3d3410ccc0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3d280137b0 tx=0x7f3d28009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:57:38.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:38.042+0000 7f3d30ff9700 1 -- 172.21.15.32:0/2077948461 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d28021070 con 0x7f3d340fcf60 2024-05-12T03:57:38.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:38.042+0000 7f3d3a521700 1 -- 172.21.15.32:0/2077948461 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d3410dd90 con 0x7f3d340fcf60 2024-05-12T03:57:38.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:38.042+0000 7f3d30ff9700 1 -- 172.21.15.32:0/2077948461 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3d2801c400 con 0x7f3d340fcf60 2024-05-12T03:57:38.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:38.042+0000 7f3d30ff9700 1 -- 172.21.15.32:0/2077948461 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d2800e070 con 0x7f3d340fcf60 2024-05-12T03:57:38.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:38.042+0000 7f3d3a521700 1 -- 172.21.15.32:0/2077948461 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d3410e270 con 0x7f3d340fcf60 2024-05-12T03:57:38.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:38.043+0000 7f3d3a521700 1 -- 172.21.15.32:0/2077948461 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d3405f5c0 con 0x7f3d340fcf60 2024-05-12T03:57:38.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:38.043+0000 7f3d30ff9700 1 -- 172.21.15.32:0/2077948461 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f3d2801c570 con 0x7f3d340fcf60 2024-05-12T03:57:38.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:38.044+0000 7f3d30ff9700 1 --2- 172.21.15.32:0/2077948461 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f3d1c0692b0 0x7f3d1c06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:38.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:38.044+0000 7f3d33fff700 1 --2- 172.21.15.32:0/2077948461 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f3d1c0692b0 0x7f3d1c06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:38.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:38.044+0000 7f3d30ff9700 1 -- 172.21.15.32:0/2077948461 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f3d28087ff0 con 0x7f3d340fcf60 2024-05-12T03:57:38.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:38.044+0000 7f3d33fff700 1 --2- 172.21.15.32:0/2077948461 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f3d1c0692b0 0x7f3d1c06b760 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3d24009b80 tx=0x7f3d2400b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:57:38.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:38.048+0000 7f3d30ff9700 1 -- 172.21.15.32:0/2077948461 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f3d280580f0 con 0x7f3d340fcf60 2024-05-12T03:57:38.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:38.252+0000 7f3d3a521700 1 -- 172.21.15.32:0/2077948461 --> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f3d3410e5a0 con 0x7f3d1c0692b0 2024-05-12T03:57:38.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:38.256+0000 7f3d30ff9700 1 -- 172.21.15.32:0/2077948461 <== mgr.14208 v2:172.21.15.32:6800/4089797100 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+1161 (secure 0 0 0) 0x7f3d3410e5a0 con 0x7f3d1c0692b0 2024-05-12T03:57:38.256 INFO:teuthology.orchestra.run.smithi032.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-12T03:57:38.257 INFO:teuthology.orchestra.run.smithi032.stdout:alertmanager ?:9093,9094 1/1 2m ago 6m count:1 2024-05-12T03:57:38.257 INFO:teuthology.orchestra.run.smithi032.stdout:crash 2/2 2m ago 6m * 2024-05-12T03:57:38.257 INFO:teuthology.orchestra.run.smithi032.stdout:grafana ?:3000 1/1 2m ago 6m count:1 2024-05-12T03:57:38.257 INFO:teuthology.orchestra.run.smithi032.stdout:mgr 2/2 2m ago 6m count:2 2024-05-12T03:57:38.257 INFO:teuthology.orchestra.run.smithi032.stdout:mon 2/2 2m ago 5m smithi032:172.21.15.32=smithi032;smithi033:172.21.15.33=smithi033;count:2 2024-05-12T03:57:38.257 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter ?:9100 2/2 2m ago 6m * 2024-05-12T03:57:38.257 INFO:teuthology.orchestra.run.smithi032.stdout:osd 6 2m ago - 2024-05-12T03:57:38.257 INFO:teuthology.orchestra.run.smithi032.stdout:prometheus ?:9095 1/1 2m ago 6m count:1 2024-05-12T03:57:38.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:38.258+0000 7f3d3a521700 1 -- 172.21.15.32:0/2077948461 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f3d1c0692b0 msgr2=0x7f3d1c06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:38.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:38.258+0000 7f3d3a521700 1 --2- 172.21.15.32:0/2077948461 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f3d1c0692b0 0x7f3d1c06b760 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3d24009b80 tx=0x7f3d2400b040 comp rx=0 tx=0).stop 2024-05-12T03:57:38.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:38.258+0000 7f3d3a521700 1 -- 172.21.15.32:0/2077948461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d340fcf60 msgr2=0x7f3d3410ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:38.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:38.258+0000 7f3d3a521700 1 --2- 172.21.15.32:0/2077948461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d340fcf60 0x7f3d3410ccc0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3d280137b0 tx=0x7f3d28009e30 comp rx=0 tx=0).stop 2024-05-12T03:57:38.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:38.258+0000 7f3d3a521700 1 -- 172.21.15.32:0/2077948461 shutdown_connections 2024-05-12T03:57:38.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:38.258+0000 7f3d3a521700 1 --2- 172.21.15.32:0/2077948461 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f3d1c0692b0 0x7f3d1c06b760 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:38.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:38.258+0000 7f3d3a521700 1 --2- 172.21.15.32:0/2077948461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d340fbcb0 0x7f3d3410c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:38.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:38.258+0000 7f3d3a521700 1 --2- 172.21.15.32:0/2077948461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d340fcf60 0x7f3d3410ccc0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:38.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:38.258+0000 7f3d3a521700 1 -- 172.21.15.32:0/2077948461 >> 172.21.15.32:0/2077948461 conn(0x7f3d340f7430 msgr2=0x7f3d340f8b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:57:38.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:38.258+0000 7f3d3a521700 1 -- 172.21.15.32:0/2077948461 shutdown_connections 2024-05-12T03:57:38.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:38.258+0000 7f3d3a521700 1 -- 172.21.15.32:0/2077948461 wait complete. 2024-05-12T03:57:38.937 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-12T03:57:39.216 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T03:57:39.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:39 smithi032 ceph-mon[40008]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:57:39.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:39 smithi033 ceph-mon[49525]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:57:40.396 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:40 smithi032 ceph-mon[40008]: from='client.14442 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T03:57:40.396 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:40 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T03:57:40.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:40 smithi033 ceph-mon[49525]: from='client.14442 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T03:57:40.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:40 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T03:57:40.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:40.924+0000 7f3cee351700 1 -- 172.21.15.32:0/4087490634 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ce80fcf60 msgr2=0x7f3ce80fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:40.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:40.924+0000 7f3cee351700 1 --2- 172.21.15.32:0/4087490634 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ce80fcf60 0x7f3ce80fd3d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc004940 tx=0x7f3cdc009e30 comp rx=0 tx=0).stop 2024-05-12T03:57:40.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:40.924+0000 7f3cee351700 1 -- 172.21.15.32:0/4087490634 shutdown_connections 2024-05-12T03:57:40.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:40.924+0000 7f3cee351700 1 --2- 172.21.15.32:0/4087490634 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ce80fcf60 0x7f3ce80fd3d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:40.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:40.924+0000 7f3cee351700 1 --2- 172.21.15.32:0/4087490634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ce80fbcb0 0x7f3ce80fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:40.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:40.924+0000 7f3cee351700 1 -- 172.21.15.32:0/4087490634 >> 172.21.15.32:0/4087490634 conn(0x7f3ce80f7430 msgr2=0x7f3ce80f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:57:40.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:40.924+0000 7f3cee351700 1 -- 172.21.15.32:0/4087490634 shutdown_connections 2024-05-12T03:57:40.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:40.924+0000 7f3cee351700 1 -- 172.21.15.32:0/4087490634 wait complete. 2024-05-12T03:57:40.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:40.925+0000 7f3cee351700 1 Processor -- start 2024-05-12T03:57:40.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:40.925+0000 7f3cee351700 1 -- start start 2024-05-12T03:57:40.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:40.925+0000 7f3cee351700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ce80fbcb0 0x7f3ce80ffd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:40.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:40.925+0000 7f3cee351700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ce80fcf60 0x7f3ce80fe2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:40.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:40.925+0000 7f3cee351700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce80fe8c0 con 0x7f3ce80fbcb0 2024-05-12T03:57:40.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:40.925+0000 7f3cee351700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce80fea00 con 0x7f3ce80fcf60 2024-05-12T03:57:40.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:40.925+0000 7f3ce7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ce80fbcb0 0x7f3ce80ffd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:40.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:40.925+0000 7f3ce7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ce80fbcb0 0x7f3ce80ffd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42462/0 (socket says 172.21.15.32:42462) 2024-05-12T03:57:40.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:40.925+0000 7f3ce7fff700 1 -- 172.21.15.32:0/3066409223 learned_addr learned my addr 172.21.15.32:0/3066409223 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:57:40.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:40.926+0000 7f3ce77fe700 1 --2- 172.21.15.32:0/3066409223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ce80fcf60 0x7f3ce80fe2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:40.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:40.926+0000 7f3ce7fff700 1 -- 172.21.15.32:0/3066409223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ce80fcf60 msgr2=0x7f3ce80fe2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:40.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:40.926+0000 7f3ce7fff700 1 --2- 172.21.15.32:0/3066409223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ce80fcf60 0x7f3ce80fe2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:40.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:40.926+0000 7f3ce7fff700 1 -- 172.21.15.32:0/3066409223 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cdc005040 con 0x7f3ce80fbcb0 2024-05-12T03:57:40.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:40.926+0000 7f3ce77fe700 1 --2- 172.21.15.32:0/3066409223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ce80fcf60 0x7f3ce80fe2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-12T03:57:40.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:40.926+0000 7f3ce7fff700 1 --2- 172.21.15.32:0/3066409223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ce80fbcb0 0x7f3ce80ffd80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3cd800b790 tx=0x7f3cd8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:57:40.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:40.926+0000 7f3ce4ff9700 1 -- 172.21.15.32:0/3066409223 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3cd800f770 con 0x7f3ce80fbcb0 2024-05-12T03:57:40.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:40.926+0000 7f3cee351700 1 -- 172.21.15.32:0/3066409223 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ce80fece0 con 0x7f3ce80fbcb0 2024-05-12T03:57:40.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:40.926+0000 7f3ce4ff9700 1 -- 172.21.15.32:0/3066409223 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3cd800fd80 con 0x7f3ce80fbcb0 2024-05-12T03:57:40.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:40.926+0000 7f3ce4ff9700 1 -- 172.21.15.32:0/3066409223 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3cd800d310 con 0x7f3ce80fbcb0 2024-05-12T03:57:40.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:40.926+0000 7f3cee351700 1 -- 172.21.15.32:0/3066409223 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ce8115990 con 0x7f3ce80fbcb0 2024-05-12T03:57:40.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:40.927+0000 7f3cee351700 1 -- 172.21.15.32:0/3066409223 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ce8101af0 con 0x7f3ce80fbcb0 2024-05-12T03:57:40.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:40.928+0000 7f3ce4ff9700 1 -- 172.21.15.32:0/3066409223 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f3cd8010420 con 0x7f3ce80fbcb0 2024-05-12T03:57:40.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:40.928+0000 7f3ce4ff9700 1 --2- 172.21.15.32:0/3066409223 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f3cd00692a0 0x7f3cd006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:40.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:40.928+0000 7f3ce4ff9700 1 -- 172.21.15.32:0/3066409223 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f3cd8086e00 con 0x7f3ce80fbcb0 2024-05-12T03:57:40.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:40.928+0000 7f3ce77fe700 1 --2- 172.21.15.32:0/3066409223 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f3cd00692a0 0x7f3cd006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:40.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:40.929+0000 7f3ce77fe700 1 --2- 172.21.15.32:0/3066409223 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f3cd00692a0 0x7f3cd006b750 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc00b3b0 tx=0x7f3cdc009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:57:40.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:40.932+0000 7f3ce4ff9700 1 -- 172.21.15.32:0/3066409223 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f3cd8053470 con 0x7f3ce80fbcb0 2024-05-12T03:57:41.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:41.131+0000 7f3cee351700 1 -- 172.21.15.32:0/3066409223 --> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f3ce805e3a0 con 0x7f3cd00692a0 2024-05-12T03:57:41.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:41.133+0000 7f3ce4ff9700 1 -- 172.21.15.32:0/3066409223 <== mgr.14208 v2:172.21.15.32:6800/4089797100 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+145 (secure 0 0 0) 0x7f3ce805e3a0 con 0x7f3cd00692a0 2024-05-12T03:57:41.133 INFO:teuthology.orchestra.run.smithi032.stdout:HOST ADDR LABELS STATUS 2024-05-12T03:57:41.133 INFO:teuthology.orchestra.run.smithi032.stdout:smithi032 172.21.15.32 2024-05-12T03:57:41.133 INFO:teuthology.orchestra.run.smithi032.stdout:smithi033 172.21.15.33 2024-05-12T03:57:41.133 INFO:teuthology.orchestra.run.smithi032.stdout:2 hosts in cluster 2024-05-12T03:57:41.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:41.135+0000 7f3cee351700 1 -- 172.21.15.32:0/3066409223 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f3cd00692a0 msgr2=0x7f3cd006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:41.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:41.135+0000 7f3cee351700 1 --2- 172.21.15.32:0/3066409223 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f3cd00692a0 0x7f3cd006b750 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc00b3b0 tx=0x7f3cdc009e30 comp rx=0 tx=0).stop 2024-05-12T03:57:41.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:41.135+0000 7f3cee351700 1 -- 172.21.15.32:0/3066409223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ce80fbcb0 msgr2=0x7f3ce80ffd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:41.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:41.135+0000 7f3cee351700 1 --2- 172.21.15.32:0/3066409223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ce80fbcb0 0x7f3ce80ffd80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3cd800b790 tx=0x7f3cd8009250 comp rx=0 tx=0).stop 2024-05-12T03:57:41.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:41.135+0000 7f3cee351700 1 -- 172.21.15.32:0/3066409223 shutdown_connections 2024-05-12T03:57:41.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:41.135+0000 7f3cee351700 1 --2- 172.21.15.32:0/3066409223 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f3cd00692a0 0x7f3cd006b750 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:41.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:41.135+0000 7f3cee351700 1 --2- 172.21.15.32:0/3066409223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3ce80fbcb0 0x7f3ce80ffd80 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:41.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:41.135+0000 7f3cee351700 1 --2- 172.21.15.32:0/3066409223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ce80fcf60 0x7f3ce80fe2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:41.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:41.135+0000 7f3cee351700 1 -- 172.21.15.32:0/3066409223 >> 172.21.15.32:0/3066409223 conn(0x7f3ce80f7430 msgr2=0x7f3ce8065bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:57:41.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:41.135+0000 7f3cee351700 1 -- 172.21.15.32:0/3066409223 shutdown_connections 2024-05-12T03:57:41.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:41.135+0000 7f3cee351700 1 -- 172.21.15.32:0/3066409223 wait complete. 2024-05-12T03:57:41.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:41 smithi033 ceph-mon[49525]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:57:41.489 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:41 smithi032 ceph-mon[40008]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:57:41.799 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-12T03:57:42.079 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T03:57:42.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:42 smithi033 ceph-mon[49525]: from='client.14446 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T03:57:42.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:42 smithi032 ceph-mon[40008]: from='client.14446 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T03:57:43.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:43 smithi032 ceph-mon[40008]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:57:43.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:43 smithi033 ceph-mon[49525]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:57:43.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:43.758+0000 7f0add786700 1 -- 172.21.15.32:0/1660962254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ad80fe420 msgr2=0x7f0ad80fe830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:43.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:43.758+0000 7f0add786700 1 --2- 172.21.15.32:0/1660962254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ad80fe420 0x7f0ad80fe830 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0ac8004940 tx=0x7f0ac8009e30 comp rx=0 tx=0).stop 2024-05-12T03:57:43.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:43.758+0000 7f0add786700 1 -- 172.21.15.32:0/1660962254 shutdown_connections 2024-05-12T03:57:43.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:43.758+0000 7f0add786700 1 --2- 172.21.15.32:0/1660962254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ad80fed70 0x7f0ad80faeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:43.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:43.758+0000 7f0add786700 1 --2- 172.21.15.32:0/1660962254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ad80fe420 0x7f0ad80fe830 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:43.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:43.758+0000 7f0add786700 1 -- 172.21.15.32:0/1660962254 >> 172.21.15.32:0/1660962254 conn(0x7f0ad80f6be0 msgr2=0x7f0ad80f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:57:43.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:43.758+0000 7f0add786700 1 -- 172.21.15.32:0/1660962254 shutdown_connections 2024-05-12T03:57:43.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:43.758+0000 7f0add786700 1 -- 172.21.15.32:0/1660962254 wait complete. 2024-05-12T03:57:43.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:43.759+0000 7f0add786700 1 Processor -- start 2024-05-12T03:57:43.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:43.759+0000 7f0add786700 1 -- start start 2024-05-12T03:57:43.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:43.759+0000 7f0add786700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ad80fe420 0x7f0ad810d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:43.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:43.759+0000 7f0add786700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ad80fed70 0x7f0ad810dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:43.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:43.759+0000 7f0add786700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad810e360 con 0x7f0ad80fed70 2024-05-12T03:57:43.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:43.759+0000 7f0add786700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad810e4d0 con 0x7f0ad80fe420 2024-05-12T03:57:43.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:43.759+0000 7f0ad67fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ad80fed70 0x7f0ad810dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:43.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:43.759+0000 7f0ad67fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ad80fed70 0x7f0ad810dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54734/0 (socket says 172.21.15.32:54734) 2024-05-12T03:57:43.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:43.759+0000 7f0ad67fc700 1 -- 172.21.15.32:0/1714167334 learned_addr learned my addr 172.21.15.32:0/1714167334 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:57:43.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:43.760+0000 7f0ad6ffd700 1 --2- 172.21.15.32:0/1714167334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ad80fe420 0x7f0ad810d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:43.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:43.760+0000 7f0ad67fc700 1 -- 172.21.15.32:0/1714167334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ad80fe420 msgr2=0x7f0ad810d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:43.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:43.760+0000 7f0ad67fc700 1 --2- 172.21.15.32:0/1714167334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ad80fe420 0x7f0ad810d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:43.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:43.760+0000 7f0ad67fc700 1 -- 172.21.15.32:0/1714167334 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ac8005040 con 0x7f0ad80fed70 2024-05-12T03:57:43.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:43.760+0000 7f0ad67fc700 1 --2- 172.21.15.32:0/1714167334 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ad80fed70 0x7f0ad810dd60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0acc00c830 tx=0x7f0acc004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:57:43.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:43.760+0000 7f0abf7fe700 1 -- 172.21.15.32:0/1714167334 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0acc00a760 con 0x7f0ad80fed70 2024-05-12T03:57:43.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:43.760+0000 7f0abf7fe700 1 -- 172.21.15.32:0/1714167334 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0acc00ad70 con 0x7f0ad80fed70 2024-05-12T03:57:43.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:43.760+0000 7f0add786700 1 -- 172.21.15.32:0/1714167334 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ad8112f30 con 0x7f0ad80fed70 2024-05-12T03:57:43.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:43.760+0000 7f0abf7fe700 1 -- 172.21.15.32:0/1714167334 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0acc0113a0 con 0x7f0ad80fed70 2024-05-12T03:57:43.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:43.760+0000 7f0add786700 1 -- 172.21.15.32:0/1714167334 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ad81134c0 con 0x7f0ad80fed70 2024-05-12T03:57:43.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:43.761+0000 7f0add786700 1 -- 172.21.15.32:0/1714167334 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ad804c710 con 0x7f0ad80fed70 2024-05-12T03:57:43.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:43.762+0000 7f0abf7fe700 1 -- 172.21.15.32:0/1714167334 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f0acc012400 con 0x7f0ad80fed70 2024-05-12T03:57:43.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:43.762+0000 7f0abf7fe700 1 --2- 172.21.15.32:0/1714167334 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f0ac00692a0 0x7f0ac006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:43.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:43.762+0000 7f0abf7fe700 1 -- 172.21.15.32:0/1714167334 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f0acc086ef0 con 0x7f0ad80fed70 2024-05-12T03:57:43.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:43.762+0000 7f0ad6ffd700 1 --2- 172.21.15.32:0/1714167334 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f0ac00692a0 0x7f0ac006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:43.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:43.762+0000 7f0ad6ffd700 1 --2- 172.21.15.32:0/1714167334 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f0ac00692a0 0x7f0ac006b750 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0ac800b3b0 tx=0x7f0ac8009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:57:43.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:43.766+0000 7f0abf7fe700 1 -- 172.21.15.32:0/1714167334 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f0acc056ff0 con 0x7f0ad80fed70 2024-05-12T03:57:43.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:43.973+0000 7f0add786700 1 -- 172.21.15.32:0/1714167334 --> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f0ad8102530 con 0x7f0ac00692a0 2024-05-12T03:57:43.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:43.974+0000 7f0abf7fe700 1 -- 172.21.15.32:0/1714167334 <== mgr.14208 v2:172.21.15.32:6800/4089797100 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+399 (secure 0 0 0) 0x7f0ad8102530 con 0x7f0ac00692a0 2024-05-12T03:57:43.974 INFO:teuthology.orchestra.run.smithi032.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-12T03:57:43.974 INFO:teuthology.orchestra.run.smithi032.stdout:smithi032 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB747402DN480DGN 447G No 2m ago Has a FileSystem, LVM detected 2024-05-12T03:57:43.974 INFO:teuthology.orchestra.run.smithi032.stdout:smithi033 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5330008Q400BGN 372G No 76s ago Has a FileSystem, LVM detected 2024-05-12T03:57:43.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:43.976+0000 7f0add786700 1 -- 172.21.15.32:0/1714167334 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f0ac00692a0 msgr2=0x7f0ac006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:43.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:43.976+0000 7f0add786700 1 --2- 172.21.15.32:0/1714167334 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f0ac00692a0 0x7f0ac006b750 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0ac800b3b0 tx=0x7f0ac8009e30 comp rx=0 tx=0).stop 2024-05-12T03:57:43.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:43.976+0000 7f0add786700 1 -- 172.21.15.32:0/1714167334 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ad80fed70 msgr2=0x7f0ad810dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:43.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:43.976+0000 7f0add786700 1 --2- 172.21.15.32:0/1714167334 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ad80fed70 0x7f0ad810dd60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0acc00c830 tx=0x7f0acc004470 comp rx=0 tx=0).stop 2024-05-12T03:57:43.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:43.977+0000 7f0add786700 1 -- 172.21.15.32:0/1714167334 shutdown_connections 2024-05-12T03:57:43.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:43.977+0000 7f0add786700 1 --2- 172.21.15.32:0/1714167334 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f0ac00692a0 0x7f0ac006b750 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:43.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:43.977+0000 7f0add786700 1 --2- 172.21.15.32:0/1714167334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ad80fe420 0x7f0ad810d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:43.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:43.977+0000 7f0add786700 1 --2- 172.21.15.32:0/1714167334 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ad80fed70 0x7f0ad810dd60 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:43.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:43.977+0000 7f0add786700 1 -- 172.21.15.32:0/1714167334 >> 172.21.15.32:0/1714167334 conn(0x7f0ad80f6be0 msgr2=0x7f0ad80f8360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:57:43.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:43.977+0000 7f0add786700 1 -- 172.21.15.32:0/1714167334 shutdown_connections 2024-05-12T03:57:43.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:43.977+0000 7f0add786700 1 -- 172.21.15.32:0/1714167334 wait complete. 2024-05-12T03:57:44.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:44 smithi032 ceph-mon[40008]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:57:44.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:44 smithi032 ceph-mon[40008]: from='client.14450 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T03:57:44.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:44 smithi033 ceph-mon[49525]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:57:44.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:44 smithi033 ceph-mon[49525]: from='client.14450 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T03:57:44.621 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-12T03:57:44.631 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi032.front.sepia.ceph.com 2024-05-12T03:57:44.631 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- bash -c 'ceph fs volume create cephfs --placement=4' 2024-05-12T03:57:44.912 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T03:57:46.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:46.802+0000 7f53928aa700 1 -- 172.21.15.32:0/1142979505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f538c0fbcb0 msgr2=0x7f538c0fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:46.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:46.802+0000 7f53928aa700 1 --2- 172.21.15.32:0/1142979505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f538c0fbcb0 0x7f538c0fc0c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f537c004940 tx=0x7f537c009e30 comp rx=0 tx=0).stop 2024-05-12T03:57:46.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:46.802+0000 7f53928aa700 1 -- 172.21.15.32:0/1142979505 shutdown_connections 2024-05-12T03:57:46.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:46.802+0000 7f53928aa700 1 --2- 172.21.15.32:0/1142979505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f538c0fcf60 0x7f538c0fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:46.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:46.802+0000 7f53928aa700 1 --2- 172.21.15.32:0/1142979505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f538c0fbcb0 0x7f538c0fc0c0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:46.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:46.802+0000 7f53928aa700 1 -- 172.21.15.32:0/1142979505 >> 172.21.15.32:0/1142979505 conn(0x7f538c0f7430 msgr2=0x7f538c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:57:46.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:46.802+0000 7f53928aa700 1 -- 172.21.15.32:0/1142979505 shutdown_connections 2024-05-12T03:57:46.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:46.802+0000 7f53928aa700 1 -- 172.21.15.32:0/1142979505 wait complete. 2024-05-12T03:57:46.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:46.803+0000 7f53928aa700 1 Processor -- start 2024-05-12T03:57:46.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:46.803+0000 7f53928aa700 1 -- start start 2024-05-12T03:57:46.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:46.803+0000 7f53928aa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f538c0fbcb0 0x7f538c10d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:46.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:46.803+0000 7f53928aa700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f538c0fcf60 0x7f538c10def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:46.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:46.803+0000 7f53928aa700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f538c10e580 con 0x7f538c0fbcb0 2024-05-12T03:57:46.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:46.803+0000 7f53928aa700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f538c10ecd0 con 0x7f538c0fcf60 2024-05-12T03:57:46.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:46.803+0000 7f538bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f538c0fbcb0 0x7f538c10d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:46.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:46.803+0000 7f538bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f538c0fbcb0 0x7f538c10d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54752/0 (socket says 172.21.15.32:54752) 2024-05-12T03:57:46.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:46.803+0000 7f538bfff700 1 -- 172.21.15.32:0/1822370388 learned_addr learned my addr 172.21.15.32:0/1822370388 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:57:46.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:46.803+0000 7f538bfff700 1 -- 172.21.15.32:0/1822370388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f538c0fcf60 msgr2=0x7f538c10def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:46.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:46.803+0000 7f538b7fe700 1 --2- 172.21.15.32:0/1822370388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f538c0fcf60 0x7f538c10def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:46.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:46.803+0000 7f538bfff700 1 --2- 172.21.15.32:0/1822370388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f538c0fcf60 0x7f538c10def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:46.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:46.803+0000 7f538bfff700 1 -- 172.21.15.32:0/1822370388 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f537c005040 con 0x7f538c0fbcb0 2024-05-12T03:57:46.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:46.803+0000 7f538bfff700 1 --2- 172.21.15.32:0/1822370388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f538c0fbcb0 0x7f538c10d9b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f537c00b3b0 tx=0x7f537c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:57:46.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:46.804+0000 7f5388ff9700 1 -- 172.21.15.32:0/1822370388 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f537c00e040 con 0x7f538c0fbcb0 2024-05-12T03:57:46.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:46.804+0000 7f53928aa700 1 -- 172.21.15.32:0/1822370388 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f538c10ef50 con 0x7f538c0fbcb0 2024-05-12T03:57:46.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:46.804+0000 7f5388ff9700 1 -- 172.21.15.32:0/1822370388 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f537c00bd20 con 0x7f538c0fbcb0 2024-05-12T03:57:46.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:46.804+0000 7f5388ff9700 1 -- 172.21.15.32:0/1822370388 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f537c01c630 con 0x7f538c0fbcb0 2024-05-12T03:57:46.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:46.804+0000 7f53928aa700 1 -- 172.21.15.32:0/1822370388 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f538c10f510 con 0x7f538c0fbcb0 2024-05-12T03:57:46.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:46.805+0000 7f53928aa700 1 -- 172.21.15.32:0/1822370388 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f538c101af0 con 0x7f538c0fbcb0 2024-05-12T03:57:46.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:46.805+0000 7f5388ff9700 1 -- 172.21.15.32:0/1822370388 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f537c013420 con 0x7f538c0fbcb0 2024-05-12T03:57:46.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:46.806+0000 7f5388ff9700 1 --2- 172.21.15.32:0/1822370388 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f53780692a0 0x7f537806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:46.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:46.806+0000 7f5388ff9700 1 -- 172.21.15.32:0/1822370388 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f537c087c40 con 0x7f538c0fbcb0 2024-05-12T03:57:46.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:46.806+0000 7f538b7fe700 1 --2- 172.21.15.32:0/1822370388 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f53780692a0 0x7f537806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:46.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:46.806+0000 7f538b7fe700 1 --2- 172.21.15.32:0/1822370388 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f53780692a0 0x7f537806b750 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5374006fd0 tx=0x7f5374008040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:57:46.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:46.810+0000 7f5388ff9700 1 -- 172.21.15.32:0/1822370388 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f537c057cc0 con 0x7f538c0fbcb0 2024-05-12T03:57:47.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:47.035+0000 7f53928aa700 1 -- 172.21.15.32:0/1822370388 --> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f538c104890 con 0x7f53780692a0 2024-05-12T03:57:47.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:46 smithi033 ceph-mon[49525]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:57:47.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:46 smithi032 ceph-mon[40008]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:57:48.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:47 smithi033 ceph-mon[49525]: from='client.14454 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T03:57:48.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:47 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-05-12T03:57:48.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:47 smithi032 ceph-mon[40008]: from='client.14454 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T03:57:48.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:47 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-05-12T03:57:48.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:48.923+0000 7f5388ff9700 1 -- 172.21.15.32:0/1822370388 <== mgr.14208 v2:172.21.15.32:6800/4089797100 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f5374007dd0 con 0x7f53780692a0 2024-05-12T03:57:48.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:48.926+0000 7f53928aa700 1 -- 172.21.15.32:0/1822370388 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f53780692a0 msgr2=0x7f537806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:48.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:48.926+0000 7f53928aa700 1 --2- 172.21.15.32:0/1822370388 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f53780692a0 0x7f537806b750 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5374006fd0 tx=0x7f5374008040 comp rx=0 tx=0).stop 2024-05-12T03:57:48.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:48.926+0000 7f53928aa700 1 -- 172.21.15.32:0/1822370388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f538c0fbcb0 msgr2=0x7f538c10d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:48.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:48.926+0000 7f53928aa700 1 --2- 172.21.15.32:0/1822370388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f538c0fbcb0 0x7f538c10d9b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f537c00b3b0 tx=0x7f537c009e30 comp rx=0 tx=0).stop 2024-05-12T03:57:48.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:48.926+0000 7f53928aa700 1 -- 172.21.15.32:0/1822370388 shutdown_connections 2024-05-12T03:57:48.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:48.926+0000 7f53928aa700 1 --2- 172.21.15.32:0/1822370388 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f53780692a0 0x7f537806b750 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:48.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:48.926+0000 7f53928aa700 1 --2- 172.21.15.32:0/1822370388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f538c0fbcb0 0x7f538c10d9b0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:48.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:48.926+0000 7f53928aa700 1 --2- 172.21.15.32:0/1822370388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f538c0fcf60 0x7f538c10def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:48.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:48.926+0000 7f53928aa700 1 -- 172.21.15.32:0/1822370388 >> 172.21.15.32:0/1822370388 conn(0x7f538c0f7430 msgr2=0x7f538c065b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:57:48.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:48.926+0000 7f53928aa700 1 -- 172.21.15.32:0/1822370388 shutdown_connections 2024-05-12T03:57:48.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:48.926+0000 7f53928aa700 1 -- 172.21.15.32:0/1822370388 wait complete. 2024-05-12T03:57:49.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:48 smithi033 ceph-mon[49525]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:57:49.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:48 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-05-12T03:57:49.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:48 smithi033 ceph-mon[49525]: osdmap e41: 6 total, 6 up, 6 in 2024-05-12T03:57:49.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:48 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-05-12T03:57:49.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:48 smithi032 ceph-mon[40008]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:57:49.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:48 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-05-12T03:57:49.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:48 smithi032 ceph-mon[40008]: osdmap e41: 6 total, 6 up, 6 in 2024-05-12T03:57:49.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:48 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-05-12T03:57:49.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:48 smithi032 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-mon-smithi032[39985]: 2024-05-12T03:57:48.904+0000 7f54eacc5700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-12T03:57:49.674 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-12T03:57:49.958 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:49 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-05-12T03:57:49.959 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:49 smithi032 ceph-mon[40008]: osdmap e42: 6 total, 6 up, 6 in 2024-05-12T03:57:49.959 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:49 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-05-12T03:57:49.959 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:49 smithi032 ceph-mon[40008]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-12T03:57:49.959 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:49 smithi032 ceph-mon[40008]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-12T03:57:49.959 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:49 smithi032 ceph-mon[40008]: osdmap e43: 6 total, 6 up, 6 in 2024-05-12T03:57:49.959 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:49 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-05-12T03:57:49.959 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:49 smithi032 ceph-mon[40008]: fsmap cephfs:0 2024-05-12T03:57:49.959 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:49 smithi032 ceph-mon[40008]: Saving service mds.cephfs spec with placement count:4 2024-05-12T03:57:49.959 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:49 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:57:49.959 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:49 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T03:57:49.959 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:49 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:57:49.959 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:49 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T03:57:49.959 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:49 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:57:49.959 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:49 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi032.ialtqb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-12T03:57:49.959 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:49 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi032.ialtqb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-12T03:57:49.959 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:49 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:57:49.960 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:49 smithi032 ceph-mon[40008]: Deploying daemon mds.cephfs.smithi032.ialtqb on smithi032 2024-05-12T03:57:50.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:49 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-05-12T03:57:50.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:49 smithi033 ceph-mon[49525]: osdmap e42: 6 total, 6 up, 6 in 2024-05-12T03:57:50.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:49 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-05-12T03:57:50.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:49 smithi033 ceph-mon[49525]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-12T03:57:50.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:49 smithi033 ceph-mon[49525]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-12T03:57:50.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:49 smithi033 ceph-mon[49525]: osdmap e43: 6 total, 6 up, 6 in 2024-05-12T03:57:50.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:49 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-05-12T03:57:50.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:49 smithi033 ceph-mon[49525]: fsmap cephfs:0 2024-05-12T03:57:50.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:49 smithi033 ceph-mon[49525]: Saving service mds.cephfs spec with placement count:4 2024-05-12T03:57:50.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:49 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:57:50.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:49 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T03:57:50.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:49 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:57:50.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:49 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T03:57:50.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:49 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:57:50.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:49 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi032.ialtqb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-12T03:57:50.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:49 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi032.ialtqb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-12T03:57:50.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:49 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:57:50.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:49 smithi033 ceph-mon[49525]: Deploying daemon mds.cephfs.smithi032.ialtqb on smithi032 2024-05-12T03:57:50.499 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T03:57:51.219 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:50 smithi032 ceph-mon[40008]: pgmap v186: 65 pgs: 3 creating+peering, 1 active+clean, 61 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:57:51.219 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:50 smithi032 ceph-mon[40008]: osdmap e44: 6 total, 6 up, 6 in 2024-05-12T03:57:51.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:50 smithi033 ceph-mon[49525]: pgmap v186: 65 pgs: 3 creating+peering, 1 active+clean, 61 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:57:51.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:50 smithi033 ceph-mon[49525]: osdmap e44: 6 total, 6 up, 6 in 2024-05-12T03:57:52.214 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:51 smithi032 ceph-mon[40008]: osdmap e45: 6 total, 6 up, 6 in 2024-05-12T03:57:52.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:51 smithi033 ceph-mon[49525]: osdmap e45: 6 total, 6 up, 6 in 2024-05-12T03:57:53.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:53.066+0000 7f9e801f2700 1 -- 172.21.15.32:0/1214079754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e780fcf60 msgr2=0x7f9e780fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:53.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:53.066+0000 7f9e801f2700 1 --2- 172.21.15.32:0/1214079754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e780fcf60 0x7f9e780fd3d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c004940 tx=0x7f9e6c009e30 comp rx=0 tx=0).stop 2024-05-12T03:57:53.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:53.066+0000 7f9e801f2700 1 -- 172.21.15.32:0/1214079754 shutdown_connections 2024-05-12T03:57:53.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:53.066+0000 7f9e801f2700 1 --2- 172.21.15.32:0/1214079754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e780fcf60 0x7f9e780fd3d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:53.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:53.066+0000 7f9e801f2700 1 --2- 172.21.15.32:0/1214079754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e780fbcb0 0x7f9e780fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:53.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:53.066+0000 7f9e801f2700 1 -- 172.21.15.32:0/1214079754 >> 172.21.15.32:0/1214079754 conn(0x7f9e780f7430 msgr2=0x7f9e780f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:57:53.075 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:53.066+0000 7f9e801f2700 1 -- 172.21.15.32:0/1214079754 shutdown_connections 2024-05-12T03:57:53.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:53.066+0000 7f9e801f2700 1 -- 172.21.15.32:0/1214079754 wait complete. 2024-05-12T03:57:53.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:53.067+0000 7f9e801f2700 1 Processor -- start 2024-05-12T03:57:53.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:53.067+0000 7f9e801f2700 1 -- start start 2024-05-12T03:57:53.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:53.067+0000 7f9e801f2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e780fbcb0 0x7f9e781105f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:53.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:53.067+0000 7f9e801f2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e780fcf60 0x7f9e78112b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:53.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:53.067+0000 7f9e801f2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e78066330 con 0x7f9e780fbcb0 2024-05-12T03:57:53.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:53.067+0000 7f9e801f2700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e78066470 con 0x7f9e780fcf60 2024-05-12T03:57:53.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:53.067+0000 7f9e7df8e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e780fbcb0 0x7f9e781105f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:53.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:53.067+0000 7f9e7df8e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e780fbcb0 0x7f9e781105f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41374/0 (socket says 172.21.15.32:41374) 2024-05-12T03:57:53.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:53.067+0000 7f9e7df8e700 1 -- 172.21.15.32:0/4157595693 learned_addr learned my addr 172.21.15.32:0/4157595693 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:57:53.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:53.067+0000 7f9e7d78d700 1 --2- 172.21.15.32:0/4157595693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e780fcf60 0x7f9e78112b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:53.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:53.067+0000 7f9e7df8e700 1 -- 172.21.15.32:0/4157595693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e780fcf60 msgr2=0x7f9e78112b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:53.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:53.067+0000 7f9e7df8e700 1 --2- 172.21.15.32:0/4157595693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e780fcf60 0x7f9e78112b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:53.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:53.067+0000 7f9e7df8e700 1 -- 172.21.15.32:0/4157595693 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e6c005040 con 0x7f9e780fbcb0 2024-05-12T03:57:53.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:53.067+0000 7f9e7df8e700 1 --2- 172.21.15.32:0/4157595693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e780fbcb0 0x7f9e781105f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9e7400eb70 tx=0x7f9e7400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:57:53.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:53.068+0000 7f9e6a7fc700 1 -- 172.21.15.32:0/4157595693 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e74007cc0 con 0x7f9e780fbcb0 2024-05-12T03:57:53.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:53.068+0000 7f9e801f2700 1 -- 172.21.15.32:0/4157595693 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e78113080 con 0x7f9e780fbcb0 2024-05-12T03:57:53.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:53.068+0000 7f9e6a7fc700 1 -- 172.21.15.32:0/4157595693 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9e7400f040 con 0x7f9e780fbcb0 2024-05-12T03:57:53.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:53.068+0000 7f9e6a7fc700 1 -- 172.21.15.32:0/4157595693 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e740095f0 con 0x7f9e780fbcb0 2024-05-12T03:57:53.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:53.068+0000 7f9e801f2700 1 -- 172.21.15.32:0/4157595693 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e78113640 con 0x7f9e780fbcb0 2024-05-12T03:57:53.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:53.068+0000 7f9e801f2700 1 -- 172.21.15.32:0/4157595693 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e7805f5c0 con 0x7f9e780fbcb0 2024-05-12T03:57:53.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:53.069+0000 7f9e6a7fc700 1 -- 172.21.15.32:0/4157595693 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f9e74009750 con 0x7f9e780fbcb0 2024-05-12T03:57:53.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:53.069+0000 7f9e6a7fc700 1 --2- 172.21.15.32:0/4157595693 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f9e640692a0 0x7f9e6406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:53.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:53.070+0000 7f9e6a7fc700 1 -- 172.21.15.32:0/4157595693 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f9e74086e10 con 0x7f9e780fbcb0 2024-05-12T03:57:53.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:53.070+0000 7f9e7d78d700 1 --2- 172.21.15.32:0/4157595693 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f9e640692a0 0x7f9e6406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:53.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:53.070+0000 7f9e7d78d700 1 --2- 172.21.15.32:0/4157595693 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f9e640692a0 0x7f9e6406b750 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c0137b0 tx=0x7f9e6c009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:57:53.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:53.074+0000 7f9e6a7fc700 1 -- 172.21.15.32:0/4157595693 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f9e740530c0 con 0x7f9e780fbcb0 2024-05-12T03:57:53.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:53.314+0000 7f9e801f2700 1 -- 172.21.15.32:0/4157595693 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9e78113960 con 0x7f9e780fbcb0 2024-05-12T03:57:53.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:53.315+0000 7f9e6a7fc700 1 -- 172.21.15.32:0/4157595693 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 3 v3) v1 ==== 75+0+1247 (secure 0 0 0) 0x7f9e740566e0 con 0x7f9e780fbcb0 2024-05-12T03:57:53.315 INFO:teuthology.orchestra.run.smithi032.stdout:e3 2024-05-12T03:57:53.315 INFO:teuthology.orchestra.run.smithi032.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-12T03:57:53.315 INFO:teuthology.orchestra.run.smithi032.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-12T03:57:53.315 INFO:teuthology.orchestra.run.smithi032.stdout:legacy client fscid: 1 2024-05-12T03:57:53.315 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:57:53.315 INFO:teuthology.orchestra.run.smithi032.stdout:Filesystem 'cephfs' (1) 2024-05-12T03:57:53.316 INFO:teuthology.orchestra.run.smithi032.stdout:fs_name cephfs 2024-05-12T03:57:53.316 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 3 2024-05-12T03:57:53.316 INFO:teuthology.orchestra.run.smithi032.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-05-12T03:57:53.316 INFO:teuthology.orchestra.run.smithi032.stdout:created 2024-05-12T03:57:48.904614+0000 2024-05-12T03:57:53.316 INFO:teuthology.orchestra.run.smithi032.stdout:modified 2024-05-12T03:57:52.466401+0000 2024-05-12T03:57:53.316 INFO:teuthology.orchestra.run.smithi032.stdout:tableserver 0 2024-05-12T03:57:53.316 INFO:teuthology.orchestra.run.smithi032.stdout:root 0 2024-05-12T03:57:53.316 INFO:teuthology.orchestra.run.smithi032.stdout:session_timeout 60 2024-05-12T03:57:53.316 INFO:teuthology.orchestra.run.smithi032.stdout:session_autoclose 300 2024-05-12T03:57:53.316 INFO:teuthology.orchestra.run.smithi032.stdout:max_file_size 1099511627776 2024-05-12T03:57:53.316 INFO:teuthology.orchestra.run.smithi032.stdout:required_client_features {} 2024-05-12T03:57:53.316 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure 0 2024-05-12T03:57:53.317 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure_osd_epoch 0 2024-05-12T03:57:53.317 INFO:teuthology.orchestra.run.smithi032.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-12T03:57:53.317 INFO:teuthology.orchestra.run.smithi032.stdout:max_mds 1 2024-05-12T03:57:53.317 INFO:teuthology.orchestra.run.smithi032.stdout:in 0 2024-05-12T03:57:53.317 INFO:teuthology.orchestra.run.smithi032.stdout:up {0=14458} 2024-05-12T03:57:53.317 INFO:teuthology.orchestra.run.smithi032.stdout:failed 2024-05-12T03:57:53.317 INFO:teuthology.orchestra.run.smithi032.stdout:damaged 2024-05-12T03:57:53.317 INFO:teuthology.orchestra.run.smithi032.stdout:stopped 2024-05-12T03:57:53.317 INFO:teuthology.orchestra.run.smithi032.stdout:data_pools [3] 2024-05-12T03:57:53.317 INFO:teuthology.orchestra.run.smithi032.stdout:metadata_pool 2 2024-05-12T03:57:53.317 INFO:teuthology.orchestra.run.smithi032.stdout:inline_data disabled 2024-05-12T03:57:53.317 INFO:teuthology.orchestra.run.smithi032.stdout:balancer 2024-05-12T03:57:53.317 INFO:teuthology.orchestra.run.smithi032.stdout:standby_count_wanted 0 2024-05-12T03:57:53.317 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.ialtqb{0:14458} state up:creating seq 1 addr [v2:172.21.15.32:6826/1907818042,v1:172.21.15.32:6827/1907818042] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T03:57:53.318 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:57:53.318 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:57:53.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:53.316+0000 7f9e801f2700 1 -- 172.21.15.32:0/4157595693 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f9e640692a0 msgr2=0x7f9e6406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:53.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:53.316+0000 7f9e801f2700 1 --2- 172.21.15.32:0/4157595693 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f9e640692a0 0x7f9e6406b750 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c0137b0 tx=0x7f9e6c009e30 comp rx=0 tx=0).stop 2024-05-12T03:57:53.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:53.316+0000 7f9e801f2700 1 -- 172.21.15.32:0/4157595693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e780fbcb0 msgr2=0x7f9e781105f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:53.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:53.317+0000 7f9e801f2700 1 --2- 172.21.15.32:0/4157595693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e780fbcb0 0x7f9e781105f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9e7400eb70 tx=0x7f9e7400b040 comp rx=0 tx=0).stop 2024-05-12T03:57:53.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:53.317+0000 7f9e801f2700 1 -- 172.21.15.32:0/4157595693 shutdown_connections 2024-05-12T03:57:53.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:53.317+0000 7f9e801f2700 1 --2- 172.21.15.32:0/4157595693 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f9e640692a0 0x7f9e6406b750 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:53.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:53.317+0000 7f9e801f2700 1 --2- 172.21.15.32:0/4157595693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e780fbcb0 0x7f9e781105f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:53.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:53.317+0000 7f9e801f2700 1 --2- 172.21.15.32:0/4157595693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e780fcf60 0x7f9e78112b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:53.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:53.317+0000 7f9e801f2700 1 -- 172.21.15.32:0/4157595693 >> 172.21.15.32:0/4157595693 conn(0x7f9e780f7430 msgr2=0x7f9e780f8b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:57:53.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:53.317+0000 7f9e801f2700 1 -- 172.21.15.32:0/4157595693 shutdown_connections 2024-05-12T03:57:53.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:53.317+0000 7f9e801f2700 1 -- 172.21.15.32:0/4157595693 wait complete. 2024-05-12T03:57:53.319 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 3 2024-05-12T03:57:53.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:53 smithi033 ceph-mon[49525]: pgmap v189: 65 pgs: 14 creating+peering, 24 active+clean, 27 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:57:53.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:53 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:57:53.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:53 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:57:53.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:53 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:57:53.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:53 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi033.jgbjhk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-12T03:57:53.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:53 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi033.jgbjhk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-12T03:57:53.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:53 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:57:53.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:53 smithi033 ceph-mon[49525]: daemon mds.cephfs.smithi032.ialtqb assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-05-12T03:57:53.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:53 smithi033 ceph-mon[49525]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-12T03:57:53.413 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:53 smithi033 ceph-mon[49525]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-12T03:57:53.413 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:53 smithi033 ceph-mon[49525]: Cluster is now healthy 2024-05-12T03:57:53.413 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:53 smithi033 ceph-mon[49525]: mds.? [v2:172.21.15.32:6826/1907818042,v1:172.21.15.32:6827/1907818042] up:boot 2024-05-12T03:57:53.413 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:53 smithi033 ceph-mon[49525]: fsmap cephfs:1 {0=cephfs.smithi032.ialtqb=up:creating} 2024-05-12T03:57:53.413 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:53 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi032.ialtqb"}]: dispatch 2024-05-12T03:57:53.413 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:53 smithi033 ceph-mon[49525]: daemon mds.cephfs.smithi032.ialtqb is now active in filesystem cephfs as rank 0 2024-05-12T03:57:53.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:53 smithi032 ceph-mon[40008]: pgmap v189: 65 pgs: 14 creating+peering, 24 active+clean, 27 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-05-12T03:57:53.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:53 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:57:53.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:53 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:57:53.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:53 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:57:53.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:53 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi033.jgbjhk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-12T03:57:53.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:53 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi033.jgbjhk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-12T03:57:53.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:53 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:57:53.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:53 smithi032 ceph-mon[40008]: daemon mds.cephfs.smithi032.ialtqb assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-05-12T03:57:53.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:53 smithi032 ceph-mon[40008]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-12T03:57:53.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:53 smithi032 ceph-mon[40008]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-12T03:57:53.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:53 smithi032 ceph-mon[40008]: Cluster is now healthy 2024-05-12T03:57:53.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:53 smithi032 ceph-mon[40008]: mds.? [v2:172.21.15.32:6826/1907818042,v1:172.21.15.32:6827/1907818042] up:boot 2024-05-12T03:57:53.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:53 smithi032 ceph-mon[40008]: fsmap cephfs:1 {0=cephfs.smithi032.ialtqb=up:creating} 2024-05-12T03:57:53.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:53 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi032.ialtqb"}]: dispatch 2024-05-12T03:57:53.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:53 smithi032 ceph-mon[40008]: daemon mds.cephfs.smithi032.ialtqb is now active in filesystem cephfs as rank 0 2024-05-12T03:57:54.023 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-12T03:57:54.034 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi032.front.sepia.ceph.com 2024-05-12T03:57:54.034 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- bash -c 'ceph fs set cephfs max_mds 1' 2024-05-12T03:57:54.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:54 smithi032 ceph-mon[40008]: Deploying daemon mds.cephfs.smithi033.jgbjhk on smithi033 2024-05-12T03:57:54.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:54 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/4157595693' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-12T03:57:54.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:54 smithi032 ceph-mon[40008]: mds.? [v2:172.21.15.32:6826/1907818042,v1:172.21.15.32:6827/1907818042] up:active 2024-05-12T03:57:54.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:54 smithi032 ceph-mon[40008]: fsmap cephfs:1 {0=cephfs.smithi032.ialtqb=up:active} 2024-05-12T03:57:54.315 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T03:57:54.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:54 smithi033 ceph-mon[49525]: Deploying daemon mds.cephfs.smithi033.jgbjhk on smithi033 2024-05-12T03:57:54.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:54 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/4157595693' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-12T03:57:54.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:54 smithi033 ceph-mon[49525]: mds.? [v2:172.21.15.32:6826/1907818042,v1:172.21.15.32:6827/1907818042] up:active 2024-05-12T03:57:54.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:54 smithi033 ceph-mon[49525]: fsmap cephfs:1 {0=cephfs.smithi032.ialtqb=up:active} 2024-05-12T03:57:55.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:55 smithi033 ceph-mon[49525]: pgmap v190: 65 pgs: 14 creating+peering, 45 active+clean, 6 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 414 B/s wr, 4 op/s 2024-05-12T03:57:55.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:55 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T03:57:55.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:55 smithi032 ceph-mon[40008]: pgmap v190: 65 pgs: 14 creating+peering, 45 active+clean, 6 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 414 B/s wr, 4 op/s 2024-05-12T03:57:55.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:55 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T03:57:56.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:56.340+0000 7f97cbbdf700 1 -- 172.21.15.32:0/2160866748 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97c40fce80 msgr2=0x7f97c40fd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:56.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:56.340+0000 7f97cbbdf700 1 --2- 172.21.15.32:0/2160866748 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97c40fce80 0x7f97c40fd290 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f97b4004940 tx=0x7f97b4009e30 comp rx=0 tx=0).stop 2024-05-12T03:57:56.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:56.340+0000 7f97cbbdf700 1 -- 172.21.15.32:0/2160866748 shutdown_connections 2024-05-12T03:57:56.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:56.340+0000 7f97cbbdf700 1 --2- 172.21.15.32:0/2160866748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97c40fe080 0x7f97c40fe4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:56.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:56.340+0000 7f97cbbdf700 1 --2- 172.21.15.32:0/2160866748 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97c40fce80 0x7f97c40fd290 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:56.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:56.340+0000 7f97cbbdf700 1 -- 172.21.15.32:0/2160866748 >> 172.21.15.32:0/2160866748 conn(0x7f97c40f8590 msgr2=0x7f97c40fa9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:57:56.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:56.340+0000 7f97cbbdf700 1 -- 172.21.15.32:0/2160866748 shutdown_connections 2024-05-12T03:57:56.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:56.340+0000 7f97cbbdf700 1 -- 172.21.15.32:0/2160866748 wait complete. 2024-05-12T03:57:56.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:56.341+0000 7f97cbbdf700 1 Processor -- start 2024-05-12T03:57:56.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:56.341+0000 7f97cbbdf700 1 -- start start 2024-05-12T03:57:56.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:56.341+0000 7f97cbbdf700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97c40fce80 0x7f97c410d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:56.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:56.341+0000 7f97cbbdf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97c40fe080 0x7f97c410de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:56.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:56.341+0000 7f97cbbdf700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c410e490 con 0x7f97c40fe080 2024-05-12T03:57:56.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:56.341+0000 7f97cbbdf700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c410ece0 con 0x7f97c40fce80 2024-05-12T03:57:56.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:56.341+0000 7f97c917a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97c40fe080 0x7f97c410de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:56.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:56.341+0000 7f97c917a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97c40fe080 0x7f97c410de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41400/0 (socket says 172.21.15.32:41400) 2024-05-12T03:57:56.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:56.341+0000 7f97c917a700 1 -- 172.21.15.32:0/3885746558 learned_addr learned my addr 172.21.15.32:0/3885746558 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:57:56.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:56.341+0000 7f97c997b700 1 --2- 172.21.15.32:0/3885746558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97c40fce80 0x7f97c410d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:56.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:56.341+0000 7f97c917a700 1 -- 172.21.15.32:0/3885746558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97c40fce80 msgr2=0x7f97c410d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:56.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:56.341+0000 7f97c917a700 1 --2- 172.21.15.32:0/3885746558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97c40fce80 0x7f97c410d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:56.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:56.341+0000 7f97c917a700 1 -- 172.21.15.32:0/3885746558 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b4005040 con 0x7f97c40fe080 2024-05-12T03:57:56.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:56.341+0000 7f97c917a700 1 --2- 172.21.15.32:0/3885746558 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97c40fe080 0x7f97c410de90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f97c0002a50 tx=0x7f97c00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:57:56.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:56.342+0000 7f97ba7fc700 1 -- 172.21.15.32:0/3885746558 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97c0007cc0 con 0x7f97c40fe080 2024-05-12T03:57:56.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:56.342+0000 7f97cbbdf700 1 -- 172.21.15.32:0/3885746558 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97c410efc0 con 0x7f97c40fe080 2024-05-12T03:57:56.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:56.342+0000 7f97ba7fc700 1 -- 172.21.15.32:0/3885746558 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f97c000f040 con 0x7f97c40fe080 2024-05-12T03:57:56.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:56.342+0000 7f97ba7fc700 1 -- 172.21.15.32:0/3885746558 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97c000d9c0 con 0x7f97c40fe080 2024-05-12T03:57:56.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:56.342+0000 7f97cbbdf700 1 -- 172.21.15.32:0/3885746558 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97c410f500 con 0x7f97c40fe080 2024-05-12T03:57:56.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:56.343+0000 7f97cbbdf700 1 -- 172.21.15.32:0/3885746558 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97c405f5c0 con 0x7f97c40fe080 2024-05-12T03:57:56.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:56.343+0000 7f97ba7fc700 1 -- 172.21.15.32:0/3885746558 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f97c000db20 con 0x7f97c40fe080 2024-05-12T03:57:56.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:56.343+0000 7f97ba7fc700 1 --2- 172.21.15.32:0/3885746558 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f97b00692a0 0x7f97b006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:57:56.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:56.343+0000 7f97ba7fc700 1 -- 172.21.15.32:0/3885746558 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f97c0086ec0 con 0x7f97c40fe080 2024-05-12T03:57:56.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:56.344+0000 7f97c997b700 1 --2- 172.21.15.32:0/3885746558 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f97b00692a0 0x7f97b006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:57:56.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:56.344+0000 7f97c997b700 1 --2- 172.21.15.32:0/3885746558 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f97b00692a0 0x7f97b006b750 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f97b4004890 tx=0x7f97b4009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:57:56.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:56.348+0000 7f97ba7fc700 1 -- 172.21.15.32:0/3885746558 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f97c0056c00 con 0x7f97c40fe080 2024-05-12T03:57:56.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:56.595+0000 7f97cbbdf700 1 -- 172.21.15.32:0/3885746558 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"} v 0) v1 -- 0x7f97c410f7c0 con 0x7f97c40fe080 2024-05-12T03:57:56.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:56 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:57:56.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:56 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:57:56.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:56 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:57:56.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:56 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi032.qpsawh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-12T03:57:56.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:56 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi032.qpsawh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-12T03:57:56.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:56 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:57:56.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:56 smithi033 ceph-mon[49525]: Deploying daemon mds.cephfs.smithi032.qpsawh on smithi032 2024-05-12T03:57:56.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:56 smithi033 ceph-mon[49525]: pgmap v191: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s wr, 6 op/s 2024-05-12T03:57:56.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:56 smithi033 ceph-mon[49525]: mds.? [v2:172.21.15.33:6824/55417471,v1:172.21.15.33:6825/55417471] up:boot 2024-05-12T03:57:56.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:56 smithi033 ceph-mon[49525]: fsmap cephfs:1 {0=cephfs.smithi032.ialtqb=up:active} 1 up:standby 2024-05-12T03:57:56.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:56 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi033.jgbjhk"}]: dispatch 2024-05-12T03:57:56.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:56 smithi033 ceph-mon[49525]: fsmap cephfs:1 {0=cephfs.smithi032.ialtqb=up:active} 1 up:standby 2024-05-12T03:57:56.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:56 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:57:56.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:56 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:57:56.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:56 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:57:56.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:56 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi032.qpsawh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-12T03:57:56.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:56 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi032.qpsawh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-12T03:57:56.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:56 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:57:56.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:56 smithi032 ceph-mon[40008]: Deploying daemon mds.cephfs.smithi032.qpsawh on smithi032 2024-05-12T03:57:56.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:56 smithi032 ceph-mon[40008]: pgmap v191: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s wr, 6 op/s 2024-05-12T03:57:56.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:56 smithi032 ceph-mon[40008]: mds.? [v2:172.21.15.33:6824/55417471,v1:172.21.15.33:6825/55417471] up:boot 2024-05-12T03:57:56.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:56 smithi032 ceph-mon[40008]: fsmap cephfs:1 {0=cephfs.smithi032.ialtqb=up:active} 1 up:standby 2024-05-12T03:57:56.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:56 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi033.jgbjhk"}]: dispatch 2024-05-12T03:57:56.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:56 smithi032 ceph-mon[40008]: fsmap cephfs:1 {0=cephfs.smithi032.ialtqb=up:active} 1 up:standby 2024-05-12T03:57:57.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:57.331+0000 7f97ba7fc700 1 -- 172.21.15.32:0/3885746558 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]=0 v7) v1 ==== 105+0+0 (secure 0 0 0) 0x7f97c0056790 con 0x7f97c40fe080 2024-05-12T03:57:57.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:57.334+0000 7f97cbbdf700 1 -- 172.21.15.32:0/3885746558 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f97b00692a0 msgr2=0x7f97b006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:57.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:57.334+0000 7f97cbbdf700 1 --2- 172.21.15.32:0/3885746558 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f97b00692a0 0x7f97b006b750 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f97b4004890 tx=0x7f97b4009e30 comp rx=0 tx=0).stop 2024-05-12T03:57:57.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:57.334+0000 7f97cbbdf700 1 -- 172.21.15.32:0/3885746558 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97c40fe080 msgr2=0x7f97c410de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:57:57.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:57.334+0000 7f97cbbdf700 1 --2- 172.21.15.32:0/3885746558 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97c40fe080 0x7f97c410de90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f97c0002a50 tx=0x7f97c00046b0 comp rx=0 tx=0).stop 2024-05-12T03:57:57.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:57.334+0000 7f97cbbdf700 1 -- 172.21.15.32:0/3885746558 shutdown_connections 2024-05-12T03:57:57.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:57.334+0000 7f97cbbdf700 1 --2- 172.21.15.32:0/3885746558 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f97b00692a0 0x7f97b006b750 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:57.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:57.334+0000 7f97cbbdf700 1 --2- 172.21.15.32:0/3885746558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97c40fce80 0x7f97c410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:57.334 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:57.334+0000 7f97cbbdf700 1 --2- 172.21.15.32:0/3885746558 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97c40fe080 0x7f97c410de90 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:57:57.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:57.334+0000 7f97cbbdf700 1 -- 172.21.15.32:0/3885746558 >> 172.21.15.32:0/3885746558 conn(0x7f97c40f8590 msgr2=0x7f97c40f9d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:57:57.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:57.334+0000 7f97cbbdf700 1 -- 172.21.15.32:0/3885746558 shutdown_connections 2024-05-12T03:57:57.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:57:57.334+0000 7f97cbbdf700 1 -- 172.21.15.32:0/3885746558 wait complete. 2024-05-12T03:57:57.477 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:57 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/3885746558' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-05-12T03:57:57.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:57 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/3885746558' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-05-12T03:57:58.023 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-12T03:57:58.033 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi032.front.sepia.ceph.com 2024-05-12T03:57:58.033 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- bash -c 'ceph fs set cephfs allow_standby_replay true' 2024-05-12T03:57:58.606 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:58 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/3885746558' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-05-12T03:57:58.606 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:58 smithi032 ceph-mon[40008]: fsmap cephfs:1 {0=cephfs.smithi032.ialtqb=up:active} 1 up:standby 2024-05-12T03:57:58.606 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:58 smithi032 ceph-mon[40008]: pgmap v192: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s wr, 5 op/s 2024-05-12T03:57:58.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:58 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/3885746558' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-05-12T03:57:58.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:58 smithi033 ceph-mon[49525]: fsmap cephfs:1 {0=cephfs.smithi032.ialtqb=up:active} 1 up:standby 2024-05-12T03:57:58.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:58 smithi033 ceph-mon[49525]: pgmap v192: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s wr, 5 op/s 2024-05-12T03:57:58.770 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T03:58:00.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:59 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:58:00.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:59 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:58:00.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:59 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:58:00.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:59 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi033.hglwml", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-12T03:58:00.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:59 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi033.hglwml", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-12T03:58:00.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:59 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:58:00.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:59 smithi032 ceph-mon[40008]: Deploying daemon mds.cephfs.smithi033.hglwml on smithi033 2024-05-12T03:58:00.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:59 smithi032 ceph-mon[40008]: mds.? [v2:172.21.15.32:6828/3100155296,v1:172.21.15.32:6829/3100155296] up:boot 2024-05-12T03:58:00.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:59 smithi032 ceph-mon[40008]: fsmap cephfs:1 {0=cephfs.smithi032.ialtqb=up:active} 2 up:standby 2024-05-12T03:58:00.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:57:59 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi032.qpsawh"}]: dispatch 2024-05-12T03:58:00.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:59 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:58:00.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:59 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:58:00.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:59 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:58:00.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:59 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi033.hglwml", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-12T03:58:00.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:59 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi033.hglwml", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-12T03:58:00.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:59 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:58:00.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:59 smithi033 ceph-mon[49525]: Deploying daemon mds.cephfs.smithi033.hglwml on smithi033 2024-05-12T03:58:00.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:59 smithi033 ceph-mon[49525]: mds.? [v2:172.21.15.32:6828/3100155296,v1:172.21.15.32:6829/3100155296] up:boot 2024-05-12T03:58:00.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:59 smithi033 ceph-mon[49525]: fsmap cephfs:1 {0=cephfs.smithi032.ialtqb=up:active} 2 up:standby 2024-05-12T03:58:00.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:57:59 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi032.qpsawh"}]: dispatch 2024-05-12T03:58:00.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:00.798+0000 7f502143e700 1 -- 172.21.15.32:0/3471552452 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f501c0fed70 msgr2=0x7f501c0faeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:00.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:00.798+0000 7f502143e700 1 --2- 172.21.15.32:0/3471552452 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f501c0fed70 0x7f501c0faeb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f500c004940 tx=0x7f500c009e30 comp rx=0 tx=0).stop 2024-05-12T03:58:00.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:00.798+0000 7f502143e700 1 -- 172.21.15.32:0/3471552452 shutdown_connections 2024-05-12T03:58:00.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:00.798+0000 7f502143e700 1 --2- 172.21.15.32:0/3471552452 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f501c0fed70 0x7f501c0faeb0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:00.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:00.798+0000 7f502143e700 1 --2- 172.21.15.32:0/3471552452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f501c0fe420 0x7f501c0fe830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:00.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:00.798+0000 7f502143e700 1 -- 172.21.15.32:0/3471552452 >> 172.21.15.32:0/3471552452 conn(0x7f501c0f6be0 msgr2=0x7f501c0f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:58:00.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:00.798+0000 7f502143e700 1 -- 172.21.15.32:0/3471552452 shutdown_connections 2024-05-12T03:58:00.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:00.798+0000 7f502143e700 1 -- 172.21.15.32:0/3471552452 wait complete. 2024-05-12T03:58:00.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:00.799+0000 7f502143e700 1 Processor -- start 2024-05-12T03:58:00.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:00.799+0000 7f502143e700 1 -- start start 2024-05-12T03:58:00.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:00.799+0000 7f502143e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f501c0fe420 0x7f501c10d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:00.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:00.799+0000 7f502143e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f501c0fed70 0x7f501c10dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:00.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:00.799+0000 7f502143e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f501c10e320 con 0x7f501c0fe420 2024-05-12T03:58:00.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:00.799+0000 7f502143e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f501c10e490 con 0x7f501c0fed70 2024-05-12T03:58:00.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:00.799+0000 7f501affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f501c0fe420 0x7f501c10d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:00.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:00.799+0000 7f501affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f501c0fe420 0x7f501c10d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41422/0 (socket says 172.21.15.32:41422) 2024-05-12T03:58:00.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:00.800+0000 7f501affd700 1 -- 172.21.15.32:0/2025407270 learned_addr learned my addr 172.21.15.32:0/2025407270 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:58:00.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:00.800+0000 7f501a7fc700 1 --2- 172.21.15.32:0/2025407270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f501c0fed70 0x7f501c10dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:00.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:00.800+0000 7f501affd700 1 -- 172.21.15.32:0/2025407270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f501c0fed70 msgr2=0x7f501c10dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:00.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:00.800+0000 7f501affd700 1 --2- 172.21.15.32:0/2025407270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f501c0fed70 0x7f501c10dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:00.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:00.800+0000 7f501affd700 1 -- 172.21.15.32:0/2025407270 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f500c005040 con 0x7f501c0fe420 2024-05-12T03:58:00.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:00.800+0000 7f501affd700 1 --2- 172.21.15.32:0/2025407270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f501c0fe420 0x7f501c10d7e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f501000eb40 tx=0x7f501000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:58:00.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:00.800+0000 7f50037fe700 1 -- 172.21.15.32:0/2025407270 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5010007cc0 con 0x7f501c0fe420 2024-05-12T03:58:00.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:00.800+0000 7f502143e700 1 -- 172.21.15.32:0/2025407270 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f501c112e40 con 0x7f501c0fe420 2024-05-12T03:58:00.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:00.800+0000 7f502143e700 1 -- 172.21.15.32:0/2025407270 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f501c113370 con 0x7f501c0fe420 2024-05-12T03:58:00.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:00.800+0000 7f50037fe700 1 -- 172.21.15.32:0/2025407270 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f501000f040 con 0x7f501c0fe420 2024-05-12T03:58:00.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:00.800+0000 7f50037fe700 1 -- 172.21.15.32:0/2025407270 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f501000c990 con 0x7f501c0fe420 2024-05-12T03:58:00.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:00.801+0000 7f502143e700 1 -- 172.21.15.32:0/2025407270 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f501c04c710 con 0x7f501c0fe420 2024-05-12T03:58:00.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:00.802+0000 7f50037fe700 1 -- 172.21.15.32:0/2025407270 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f501000caf0 con 0x7f501c0fe420 2024-05-12T03:58:00.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:00.802+0000 7f50037fe700 1 --2- 172.21.15.32:0/2025407270 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f50040692a0 0x7f500406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:00.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:00.802+0000 7f50037fe700 1 -- 172.21.15.32:0/2025407270 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f5010086d30 con 0x7f501c0fe420 2024-05-12T03:58:00.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:00.802+0000 7f501a7fc700 1 --2- 172.21.15.32:0/2025407270 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f50040692a0 0x7f500406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:00.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:00.803+0000 7f501a7fc700 1 --2- 172.21.15.32:0/2025407270 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f50040692a0 0x7f500406b750 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f500c004890 tx=0x7f500c009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:58:00.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:00.807+0000 7f50037fe700 1 -- 172.21.15.32:0/2025407270 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f501004e250 con 0x7f501c0fe420 2024-05-12T03:58:00.999 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:00 smithi033 ceph-mon[49525]: pgmap v193: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s wr, 4 op/s 2024-05-12T03:58:01.039 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:00 smithi032 ceph-mon[40008]: pgmap v193: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s wr, 4 op/s 2024-05-12T03:58:01.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:01.053+0000 7f502143e700 1 -- 172.21.15.32:0/2025407270 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f501c105790 con 0x7f501c0fe420 2024-05-12T03:58:01.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:01.794+0000 7f50037fe700 1 -- 172.21.15.32:0/2025407270 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v9) v1 ==== 121+0+0 (secure 0 0 0) 0x7f5010056b00 con 0x7f501c0fe420 2024-05-12T03:58:01.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:01.797+0000 7f502143e700 1 -- 172.21.15.32:0/2025407270 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f50040692a0 msgr2=0x7f500406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:01.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:01.797+0000 7f502143e700 1 --2- 172.21.15.32:0/2025407270 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f50040692a0 0x7f500406b750 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f500c004890 tx=0x7f500c009e30 comp rx=0 tx=0).stop 2024-05-12T03:58:01.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:01.797+0000 7f502143e700 1 -- 172.21.15.32:0/2025407270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f501c0fe420 msgr2=0x7f501c10d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:01.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:01.797+0000 7f502143e700 1 --2- 172.21.15.32:0/2025407270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f501c0fe420 0x7f501c10d7e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f501000eb40 tx=0x7f501000b040 comp rx=0 tx=0).stop 2024-05-12T03:58:01.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:01.797+0000 7f502143e700 1 -- 172.21.15.32:0/2025407270 shutdown_connections 2024-05-12T03:58:01.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:01.797+0000 7f502143e700 1 --2- 172.21.15.32:0/2025407270 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f50040692a0 0x7f500406b750 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:01.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:01.797+0000 7f502143e700 1 --2- 172.21.15.32:0/2025407270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f501c0fe420 0x7f501c10d7e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:01.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:01.797+0000 7f502143e700 1 --2- 172.21.15.32:0/2025407270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f501c0fed70 0x7f501c10dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:01.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:01.797+0000 7f502143e700 1 -- 172.21.15.32:0/2025407270 >> 172.21.15.32:0/2025407270 conn(0x7f501c0f6be0 msgr2=0x7f501c0f82b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:58:01.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:01.797+0000 7f502143e700 1 -- 172.21.15.32:0/2025407270 shutdown_connections 2024-05-12T03:58:01.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:01.797+0000 7f502143e700 1 -- 172.21.15.32:0/2025407270 wait complete. 2024-05-12T03:58:01.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:01 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/2025407270' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-12T03:58:01.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:01 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:58:01.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:01 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:58:01.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:01 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:58:01.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:01 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:58:02.039 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:01 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/2025407270' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-12T03:58:02.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:01 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:58:02.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:01 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:58:02.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:01 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:58:02.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:01 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:58:02.592 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-12T03:58:02.603 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi032.front.sepia.ceph.com 2024-05-12T03:58:02.603 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- bash -c 'ceph fs set cephfs inline_data true --yes-i-really-really-mean-it' 2024-05-12T03:58:02.833 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:02 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/2025407270' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-12T03:58:02.833 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:02 smithi032 ceph-mon[40008]: mds.? [v2:172.21.15.33:6826/2250494767,v1:172.21.15.33:6827/2250494767] up:boot 2024-05-12T03:58:02.833 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:02 smithi032 ceph-mon[40008]: fsmap cephfs:1 {0=cephfs.smithi032.ialtqb=up:active} 3 up:standby 2024-05-12T03:58:02.833 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:02 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:58:02.833 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:02 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi033.hglwml"}]: dispatch 2024-05-12T03:58:02.834 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:02 smithi032 ceph-mon[40008]: fsmap cephfs:1 {0=cephfs.smithi032.ialtqb=up:active} 1 up:standby-replay 2 up:standby 2024-05-12T03:58:02.834 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:02 smithi032 ceph-mon[40008]: pgmap v194: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s wr, 4 op/s 2024-05-12T03:58:02.834 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:02 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T03:58:03.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:02 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/2025407270' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-12T03:58:03.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:02 smithi033 ceph-mon[49525]: mds.? [v2:172.21.15.33:6826/2250494767,v1:172.21.15.33:6827/2250494767] up:boot 2024-05-12T03:58:03.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:02 smithi033 ceph-mon[49525]: fsmap cephfs:1 {0=cephfs.smithi032.ialtqb=up:active} 3 up:standby 2024-05-12T03:58:03.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:02 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:58:03.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:02 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi033.hglwml"}]: dispatch 2024-05-12T03:58:03.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:02 smithi033 ceph-mon[49525]: fsmap cephfs:1 {0=cephfs.smithi032.ialtqb=up:active} 1 up:standby-replay 2 up:standby 2024-05-12T03:58:03.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:02 smithi033 ceph-mon[49525]: pgmap v194: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s wr, 4 op/s 2024-05-12T03:58:03.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:02 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T03:58:03.483 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T03:58:03.994 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:03 smithi032 ceph-mon[40008]: mds.? [v2:172.21.15.32:6828/3100155296,v1:172.21.15.32:6829/3100155296] up:standby 2024-05-12T03:58:03.994 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:03 smithi032 ceph-mon[40008]: Dropping low affinity standby-replay daemon mds.cephfs.smithi033.hglwml in favor of higher affinity standby. 2024-05-12T03:58:03.994 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:03 smithi032 ceph-mon[40008]: fsmap cephfs:1 {0=cephfs.smithi032.ialtqb=up:active} 1 up:standby-replay 2 up:standby 2024-05-12T03:58:03.994 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:03 smithi032 ceph-mon[40008]: Dropping low affinity active daemon mds.cephfs.smithi032.ialtqb in favor of higher affinity standby. 2024-05-12T03:58:03.994 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:03 smithi032 ceph-mon[40008]: Replacing daemon mds.cephfs.smithi032.ialtqb as rank 0 with standby daemon mds.cephfs.smithi032.qpsawh 2024-05-12T03:58:03.994 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:03 smithi032 ceph-mon[40008]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-12T03:58:03.994 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:03 smithi032 ceph-mon[40008]: fsmap cephfs:1 {0=cephfs.smithi032.ialtqb=up:active} 1 up:standby-replay 1 up:standby 2024-05-12T03:58:03.995 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:03 smithi032 ceph-mon[40008]: osdmap e46: 6 total, 6 up, 6 in 2024-05-12T03:58:03.995 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:03 smithi032 ceph-mon[40008]: fsmap cephfs:1/1 {0=cephfs.smithi032.qpsawh=up:replay} 1 up:standby 2024-05-12T03:58:04.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:03 smithi033 ceph-mon[49525]: mds.? [v2:172.21.15.32:6828/3100155296,v1:172.21.15.32:6829/3100155296] up:standby 2024-05-12T03:58:04.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:03 smithi033 ceph-mon[49525]: Dropping low affinity standby-replay daemon mds.cephfs.smithi033.hglwml in favor of higher affinity standby. 2024-05-12T03:58:04.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:03 smithi033 ceph-mon[49525]: fsmap cephfs:1 {0=cephfs.smithi032.ialtqb=up:active} 1 up:standby-replay 2 up:standby 2024-05-12T03:58:04.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:03 smithi033 ceph-mon[49525]: Dropping low affinity active daemon mds.cephfs.smithi032.ialtqb in favor of higher affinity standby. 2024-05-12T03:58:04.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:03 smithi033 ceph-mon[49525]: Replacing daemon mds.cephfs.smithi032.ialtqb as rank 0 with standby daemon mds.cephfs.smithi032.qpsawh 2024-05-12T03:58:04.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:03 smithi033 ceph-mon[49525]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-12T03:58:04.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:03 smithi033 ceph-mon[49525]: fsmap cephfs:1 {0=cephfs.smithi032.ialtqb=up:active} 1 up:standby-replay 1 up:standby 2024-05-12T03:58:04.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:03 smithi033 ceph-mon[49525]: osdmap e46: 6 total, 6 up, 6 in 2024-05-12T03:58:04.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:03 smithi033 ceph-mon[49525]: fsmap cephfs:1/1 {0=cephfs.smithi032.qpsawh=up:replay} 1 up:standby 2024-05-12T03:58:04.958 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:04 smithi032 ceph-mon[40008]: mds.? [v2:172.21.15.32:6828/3100155296,v1:172.21.15.32:6829/3100155296] up:reconnect 2024-05-12T03:58:04.959 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:04 smithi032 ceph-mon[40008]: mds.? [v2:172.21.15.33:6826/433858238,v1:172.21.15.33:6827/433858238] up:boot 2024-05-12T03:58:04.959 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:04 smithi032 ceph-mon[40008]: mds.? [v2:172.21.15.32:6826/3703132329,v1:172.21.15.32:6827/3703132329] up:boot 2024-05-12T03:58:04.959 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:04 smithi032 ceph-mon[40008]: mds.? [v2:172.21.15.33:6824/55417471,v1:172.21.15.33:6825/55417471] up:standby 2024-05-12T03:58:04.959 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:04 smithi032 ceph-mon[40008]: fsmap cephfs:1/1 {0=cephfs.smithi032.qpsawh=up:reconnect} 3 up:standby 2024-05-12T03:58:04.959 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:04 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi033.hglwml"}]: dispatch 2024-05-12T03:58:04.959 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:04 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi032.ialtqb"}]: dispatch 2024-05-12T03:58:04.959 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:04 smithi032 ceph-mon[40008]: pgmap v196: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 511 B/s rd, 1.1 KiB/s wr, 2 op/s 2024-05-12T03:58:04.959 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:04 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:58:04.959 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:04 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:58:05.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:04 smithi033 ceph-mon[49525]: mds.? [v2:172.21.15.32:6828/3100155296,v1:172.21.15.32:6829/3100155296] up:reconnect 2024-05-12T03:58:05.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:04 smithi033 ceph-mon[49525]: mds.? [v2:172.21.15.33:6826/433858238,v1:172.21.15.33:6827/433858238] up:boot 2024-05-12T03:58:05.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:04 smithi033 ceph-mon[49525]: mds.? [v2:172.21.15.32:6826/3703132329,v1:172.21.15.32:6827/3703132329] up:boot 2024-05-12T03:58:05.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:04 smithi033 ceph-mon[49525]: mds.? [v2:172.21.15.33:6824/55417471,v1:172.21.15.33:6825/55417471] up:standby 2024-05-12T03:58:05.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:04 smithi033 ceph-mon[49525]: fsmap cephfs:1/1 {0=cephfs.smithi032.qpsawh=up:reconnect} 3 up:standby 2024-05-12T03:58:05.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:04 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi033.hglwml"}]: dispatch 2024-05-12T03:58:05.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:04 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi032.ialtqb"}]: dispatch 2024-05-12T03:58:05.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:04 smithi033 ceph-mon[49525]: pgmap v196: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 511 B/s rd, 1.1 KiB/s wr, 2 op/s 2024-05-12T03:58:05.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:04 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:58:05.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:04 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:58:06.039 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:05 smithi032 ceph-mon[40008]: mds.? [v2:172.21.15.32:6828/3100155296,v1:172.21.15.32:6829/3100155296] up:rejoin 2024-05-12T03:58:06.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:05 smithi032 ceph-mon[40008]: fsmap cephfs:1/1 {0=cephfs.smithi032.qpsawh=up:rejoin} 3 up:standby 2024-05-12T03:58:06.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:05 smithi032 ceph-mon[40008]: daemon mds.cephfs.smithi032.qpsawh is now active in filesystem cephfs as rank 0 2024-05-12T03:58:06.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:05 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:58:06.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:06.062+0000 7fcd317bd700 1 -- 172.21.15.32:0/708106147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd2c0fcf60 msgr2=0x7fcd2c0fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:06.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:06.062+0000 7fcd317bd700 1 --2- 172.21.15.32:0/708106147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd2c0fcf60 0x7fcd2c0fd3d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fcd14004940 tx=0x7fcd14009e30 comp rx=0 tx=0).stop 2024-05-12T03:58:06.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:06.062+0000 7fcd317bd700 1 -- 172.21.15.32:0/708106147 shutdown_connections 2024-05-12T03:58:06.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:06.062+0000 7fcd317bd700 1 --2- 172.21.15.32:0/708106147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd2c0fcf60 0x7fcd2c0fd3d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:06.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:06.062+0000 7fcd317bd700 1 --2- 172.21.15.32:0/708106147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd2c0fbcb0 0x7fcd2c0fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:06.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:06.062+0000 7fcd317bd700 1 -- 172.21.15.32:0/708106147 >> 172.21.15.32:0/708106147 conn(0x7fcd2c0f7430 msgr2=0x7fcd2c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:58:06.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:06.062+0000 7fcd317bd700 1 -- 172.21.15.32:0/708106147 shutdown_connections 2024-05-12T03:58:06.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:06.062+0000 7fcd317bd700 1 -- 172.21.15.32:0/708106147 wait complete. 2024-05-12T03:58:06.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:06.063+0000 7fcd317bd700 1 Processor -- start 2024-05-12T03:58:06.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:06.063+0000 7fcd317bd700 1 -- start start 2024-05-12T03:58:06.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:06.063+0000 7fcd317bd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd2c0fbcb0 0x7fcd2c10c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:06.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:06.063+0000 7fcd317bd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd2c0fcf60 0x7fcd2c10cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:06.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:06.063+0000 7fcd317bd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2c10d360 con 0x7fcd2c0fbcb0 2024-05-12T03:58:06.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:06.063+0000 7fcd317bd700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2c10dbb0 con 0x7fcd2c0fcf60 2024-05-12T03:58:06.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:06.063+0000 7fcd2affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd2c0fbcb0 0x7fcd2c10c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:06.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:06.063+0000 7fcd2affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd2c0fbcb0 0x7fcd2c10c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51402/0 (socket says 172.21.15.32:51402) 2024-05-12T03:58:06.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:06.063+0000 7fcd2affd700 1 -- 172.21.15.32:0/2282734974 learned_addr learned my addr 172.21.15.32:0/2282734974 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:58:06.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:06.063+0000 7fcd2a7fc700 1 --2- 172.21.15.32:0/2282734974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd2c0fcf60 0x7fcd2c10cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:06.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:06.063+0000 7fcd2affd700 1 -- 172.21.15.32:0/2282734974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd2c0fcf60 msgr2=0x7fcd2c10cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:06.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:06.063+0000 7fcd2affd700 1 --2- 172.21.15.32:0/2282734974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd2c0fcf60 0x7fcd2c10cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:06.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:06.063+0000 7fcd2affd700 1 -- 172.21.15.32:0/2282734974 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd14005040 con 0x7fcd2c0fbcb0 2024-05-12T03:58:06.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:06.063+0000 7fcd2affd700 1 --2- 172.21.15.32:0/2282734974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd2c0fbcb0 0x7fcd2c10c820 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c00fce0 tx=0x7fcd1c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:58:06.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:06.064+0000 7fcd237fe700 1 -- 172.21.15.32:0/2282734974 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd1c007ea0 con 0x7fcd2c0fbcb0 2024-05-12T03:58:06.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:06.064+0000 7fcd237fe700 1 -- 172.21.15.32:0/2282734974 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcd1c00dd80 con 0x7fcd2c0fbcb0 2024-05-12T03:58:06.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:06.064+0000 7fcd317bd700 1 -- 172.21.15.32:0/2282734974 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd2c10de30 con 0x7fcd2c0fbcb0 2024-05-12T03:58:06.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:06.064+0000 7fcd237fe700 1 -- 172.21.15.32:0/2282734974 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd1c012460 con 0x7fcd2c0fbcb0 2024-05-12T03:58:06.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:06.064+0000 7fcd317bd700 1 -- 172.21.15.32:0/2282734974 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd2c10e3f0 con 0x7fcd2c0fbcb0 2024-05-12T03:58:06.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:06.065+0000 7fcd317bd700 1 -- 172.21.15.32:0/2282734974 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd2c05f5c0 con 0x7fcd2c0fbcb0 2024-05-12T03:58:06.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:06.065+0000 7fcd237fe700 1 -- 172.21.15.32:0/2282734974 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fcd1c009360 con 0x7fcd2c0fbcb0 2024-05-12T03:58:06.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:06.065+0000 7fcd237fe700 1 --2- 172.21.15.32:0/2282734974 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fcd180692a0 0x7fcd1806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:06.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:06.065+0000 7fcd237fe700 1 -- 172.21.15.32:0/2282734974 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fcd1c089890 con 0x7fcd2c0fbcb0 2024-05-12T03:58:06.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:06.066+0000 7fcd2a7fc700 1 --2- 172.21.15.32:0/2282734974 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fcd180692a0 0x7fcd1806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:06.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:06.066+0000 7fcd2a7fc700 1 --2- 172.21.15.32:0/2282734974 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fcd180692a0 0x7fcd1806b750 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fcd1400b3b0 tx=0x7fcd14009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:58:06.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:06.070+0000 7fcd237fe700 1 -- 172.21.15.32:0/2282734974 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fcd1c059580 con 0x7fcd2c0fbcb0 2024-05-12T03:58:06.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:05 smithi033 ceph-mon[49525]: mds.? [v2:172.21.15.32:6828/3100155296,v1:172.21.15.32:6829/3100155296] up:rejoin 2024-05-12T03:58:06.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:05 smithi033 ceph-mon[49525]: fsmap cephfs:1/1 {0=cephfs.smithi032.qpsawh=up:rejoin} 3 up:standby 2024-05-12T03:58:06.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:05 smithi033 ceph-mon[49525]: daemon mds.cephfs.smithi032.qpsawh is now active in filesystem cephfs as rank 0 2024-05-12T03:58:06.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:05 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:58:06.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:06.316+0000 7fcd317bd700 1 -- 172.21.15.32:0/2282734974 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fcd2c1047b0 con 0x7fcd2c0fbcb0 2024-05-12T03:58:06.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:06.825+0000 7fcd237fe700 1 -- 172.21.15.32:0/2282734974 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]=0 inline data enabled v18) v1 ==== 168+0+0 (secure 0 0 0) 0x7fcd1c059110 con 0x7fcd2c0fbcb0 2024-05-12T03:58:06.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:06.827+0000 7fcd317bd700 1 -- 172.21.15.32:0/2282734974 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fcd180692a0 msgr2=0x7fcd1806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:06.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:06.827+0000 7fcd317bd700 1 --2- 172.21.15.32:0/2282734974 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fcd180692a0 0x7fcd1806b750 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fcd1400b3b0 tx=0x7fcd14009e30 comp rx=0 tx=0).stop 2024-05-12T03:58:06.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:06.827+0000 7fcd317bd700 1 -- 172.21.15.32:0/2282734974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd2c0fbcb0 msgr2=0x7fcd2c10c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:06.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:06.827+0000 7fcd317bd700 1 --2- 172.21.15.32:0/2282734974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd2c0fbcb0 0x7fcd2c10c820 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c00fce0 tx=0x7fcd1c0046b0 comp rx=0 tx=0).stop 2024-05-12T03:58:06.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:06.827+0000 7fcd317bd700 1 -- 172.21.15.32:0/2282734974 shutdown_connections 2024-05-12T03:58:06.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:06.827+0000 7fcd317bd700 1 --2- 172.21.15.32:0/2282734974 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fcd180692a0 0x7fcd1806b750 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:06.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:06.827+0000 7fcd317bd700 1 --2- 172.21.15.32:0/2282734974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd2c0fbcb0 0x7fcd2c10c820 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:06.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:06.827+0000 7fcd317bd700 1 --2- 172.21.15.32:0/2282734974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd2c0fcf60 0x7fcd2c10cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:06.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:06.827+0000 7fcd317bd700 1 -- 172.21.15.32:0/2282734974 >> 172.21.15.32:0/2282734974 conn(0x7fcd2c0f7430 msgr2=0x7fcd2c0f8bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:58:06.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:06.828+0000 7fcd317bd700 1 -- 172.21.15.32:0/2282734974 shutdown_connections 2024-05-12T03:58:06.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:06.828+0000 7fcd317bd700 1 -- 172.21.15.32:0/2282734974 wait complete. 2024-05-12T03:58:06.828 INFO:teuthology.orchestra.run.smithi032.stderr:inline data enabled 2024-05-12T03:58:07.076 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:06 smithi032 ceph-mon[40008]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-12T03:58:07.076 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:06 smithi032 ceph-mon[40008]: Cluster is now healthy 2024-05-12T03:58:07.076 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:06 smithi032 ceph-mon[40008]: mds.? [v2:172.21.15.32:6828/3100155296,v1:172.21.15.32:6829/3100155296] up:active 2024-05-12T03:58:07.076 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:06 smithi032 ceph-mon[40008]: fsmap cephfs:1 {0=cephfs.smithi032.qpsawh=up:active} 3 up:standby 2024-05-12T03:58:07.076 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:06 smithi032 ceph-mon[40008]: fsmap cephfs:1 {0=cephfs.smithi032.qpsawh=up:active} 1 up:standby-replay 2 up:standby 2024-05-12T03:58:07.076 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:06 smithi032 ceph-mon[40008]: pgmap v197: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-05-12T03:58:07.076 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:06 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/2282734974' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-12T03:58:07.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:06 smithi033 ceph-mon[49525]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-12T03:58:07.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:06 smithi033 ceph-mon[49525]: Cluster is now healthy 2024-05-12T03:58:07.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:06 smithi033 ceph-mon[49525]: mds.? [v2:172.21.15.32:6828/3100155296,v1:172.21.15.32:6829/3100155296] up:active 2024-05-12T03:58:07.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:06 smithi033 ceph-mon[49525]: fsmap cephfs:1 {0=cephfs.smithi032.qpsawh=up:active} 3 up:standby 2024-05-12T03:58:07.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:06 smithi033 ceph-mon[49525]: fsmap cephfs:1 {0=cephfs.smithi032.qpsawh=up:active} 1 up:standby-replay 2 up:standby 2024-05-12T03:58:07.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:06 smithi033 ceph-mon[49525]: pgmap v197: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-05-12T03:58:07.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:06 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/2282734974' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-12T03:58:08.094 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:07 smithi032 ceph-mon[40008]: Health check failed: 1 filesystem with deprecated feature inline_data (FS_INLINE_DATA_DEPRECATED) 2024-05-12T03:58:08.094 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:07 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/2282734974' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]': finished 2024-05-12T03:58:08.095 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:07 smithi032 ceph-mon[40008]: fsmap cephfs:1 {0=cephfs.smithi032.qpsawh=up:active} 1 up:standby-replay 2 up:standby 2024-05-12T03:58:08.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:07 smithi033 ceph-mon[49525]: Health check failed: 1 filesystem with deprecated feature inline_data (FS_INLINE_DATA_DEPRECATED) 2024-05-12T03:58:08.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:07 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/2282734974' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]': finished 2024-05-12T03:58:08.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:07 smithi033 ceph-mon[49525]: fsmap cephfs:1 {0=cephfs.smithi032.qpsawh=up:active} 1 up:standby-replay 2 up:standby 2024-05-12T03:58:08.354 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-12T03:58:08.364 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi032.front.sepia.ceph.com 2024-05-12T03:58:08.365 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-12T03:58:08.883 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T03:58:09.083 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:08 smithi032 ceph-mon[40008]: pgmap v198: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-05-12T03:58:09.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:08 smithi033 ceph-mon[49525]: pgmap v198: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-05-12T03:58:10.461 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:10 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:58:10.461 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:10 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:58:10.461 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:10 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:58:10.461 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:10 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T03:58:10.461 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:10 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:58:10.461 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:10 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T03:58:10.461 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:10 smithi032 ceph-mon[40008]: pgmap v199: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 5.2 KiB/s rd, 5 op/s 2024-05-12T03:58:10.461 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:10 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:58:10.461 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:10 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T03:58:10.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:10 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:58:10.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:10 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:58:10.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:10 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:58:10.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:10 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T03:58:10.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:10 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:58:10.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:10 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T03:58:10.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:10 smithi033 ceph-mon[49525]: pgmap v199: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 5.2 KiB/s rd, 5 op/s 2024-05-12T03:58:10.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:10 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:58:10.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:10 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T03:58:11.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:10.936+0000 7f19b23a1700 1 -- 172.21.15.32:0/343081524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19ac066b50 msgr2=0x7f19ac0ff800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:11.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:10.936+0000 7f19b23a1700 1 --2- 172.21.15.32:0/343081524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19ac066b50 0x7f19ac0ff800 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f19a8004940 tx=0x7f19a8009e30 comp rx=0 tx=0).stop 2024-05-12T03:58:11.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:10.936+0000 7f19b23a1700 1 -- 172.21.15.32:0/343081524 shutdown_connections 2024-05-12T03:58:11.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:10.936+0000 7f19b23a1700 1 --2- 172.21.15.32:0/343081524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19ac0ffd40 0x7f19ac102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:11.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:10.936+0000 7f19b23a1700 1 --2- 172.21.15.32:0/343081524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19ac066b50 0x7f19ac0ff800 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:11.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:10.936+0000 7f19b23a1700 1 -- 172.21.15.32:0/343081524 >> 172.21.15.32:0/343081524 conn(0x7f19ac0f7270 msgr2=0x7f19ac0f96a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:58:11.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:10.936+0000 7f19b23a1700 1 -- 172.21.15.32:0/343081524 shutdown_connections 2024-05-12T03:58:11.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:10.936+0000 7f19b23a1700 1 -- 172.21.15.32:0/343081524 wait complete. 2024-05-12T03:58:11.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:10.937+0000 7f19b23a1700 1 Processor -- start 2024-05-12T03:58:11.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:10.937+0000 7f19b23a1700 1 -- start start 2024-05-12T03:58:11.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:10.937+0000 7f19b23a1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19ac066b50 0x7f19ac0fd230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:11.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:10.937+0000 7f19b23a1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19ac0ffd40 0x7f19ac0fb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:11.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:10.937+0000 7f19b23a1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ac0fbcb0 con 0x7f19ac0ffd40 2024-05-12T03:58:11.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:10.937+0000 7f19b23a1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ac0fbe20 con 0x7f19ac066b50 2024-05-12T03:58:11.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:10.937+0000 7f19b0b9e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19ac0ffd40 0x7f19ac0fb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:11.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:10.937+0000 7f19b0b9e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19ac0ffd40 0x7f19ac0fb700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51420/0 (socket says 172.21.15.32:51420) 2024-05-12T03:58:11.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:10.937+0000 7f19b0b9e700 1 -- 172.21.15.32:0/4230483479 learned_addr learned my addr 172.21.15.32:0/4230483479 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:58:11.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:10.937+0000 7f19b139f700 1 --2- 172.21.15.32:0/4230483479 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19ac066b50 0x7f19ac0fd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:11.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:10.937+0000 7f19b0b9e700 1 -- 172.21.15.32:0/4230483479 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19ac066b50 msgr2=0x7f19ac0fd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:11.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:10.937+0000 7f19b0b9e700 1 --2- 172.21.15.32:0/4230483479 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19ac066b50 0x7f19ac0fd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:11.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:10.937+0000 7f19b0b9e700 1 -- 172.21.15.32:0/4230483479 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19a8005040 con 0x7f19ac0ffd40 2024-05-12T03:58:11.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:10.938+0000 7f19b0b9e700 1 --2- 172.21.15.32:0/4230483479 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19ac0ffd40 0x7f19ac0fb700 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f199c002a50 tx=0x7f199c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:58:11.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:10.938+0000 7f19a1ffb700 1 -- 172.21.15.32:0/4230483479 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f199c007cc0 con 0x7f19ac0ffd40 2024-05-12T03:58:11.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:10.938+0000 7f19b23a1700 1 -- 172.21.15.32:0/4230483479 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19ac0fc100 con 0x7f19ac0ffd40 2024-05-12T03:58:11.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:10.938+0000 7f19a1ffb700 1 -- 172.21.15.32:0/4230483479 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f199c00f040 con 0x7f19ac0ffd40 2024-05-12T03:58:11.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:10.938+0000 7f19a1ffb700 1 -- 172.21.15.32:0/4230483479 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f199c00d9c0 con 0x7f19ac0ffd40 2024-05-12T03:58:11.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:10.938+0000 7f19b23a1700 1 -- 172.21.15.32:0/4230483479 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19ac1157e0 con 0x7f19ac0ffd40 2024-05-12T03:58:11.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:10.939+0000 7f19b23a1700 1 -- 172.21.15.32:0/4230483479 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19ac05f5c0 con 0x7f19ac0ffd40 2024-05-12T03:58:11.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:10.939+0000 7f19a1ffb700 1 -- 172.21.15.32:0/4230483479 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f199c00db20 con 0x7f19ac0ffd40 2024-05-12T03:58:11.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:10.939+0000 7f19a1ffb700 1 --2- 172.21.15.32:0/4230483479 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f19980692b0 0x7f199806b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:11.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:10.940+0000 7f19a1ffb700 1 -- 172.21.15.32:0/4230483479 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f199c086f10 con 0x7f19ac0ffd40 2024-05-12T03:58:11.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:10.940+0000 7f19b139f700 1 --2- 172.21.15.32:0/4230483479 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f19980692b0 0x7f199806b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:11.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:10.940+0000 7f19b139f700 1 --2- 172.21.15.32:0/4230483479 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f19980692b0 0x7f199806b760 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f19a8000c00 tx=0x7f19a8009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:58:11.037 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:10.944+0000 7f19a1ffb700 1 -- 172.21.15.32:0/4230483479 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f199c053170 con 0x7f19ac0ffd40 2024-05-12T03:58:11.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:11.184+0000 7f19b23a1700 1 -- 172.21.15.32:0/4230483479 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f19ac04bed0 con 0x7f19ac0ffd40 2024-05-12T03:58:11.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:11.185+0000 7f19a1ffb700 1 -- 172.21.15.32:0/4230483479 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1828 (secure 0 0 0) 0x7f199c056790 con 0x7f19ac0ffd40 2024-05-12T03:58:11.186 INFO:teuthology.orchestra.run.smithi032.stdout:e18 2024-05-12T03:58:11.186 INFO:teuthology.orchestra.run.smithi032.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-12T03:58:11.186 INFO:teuthology.orchestra.run.smithi032.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-12T03:58:11.186 INFO:teuthology.orchestra.run.smithi032.stdout:legacy client fscid: 1 2024-05-12T03:58:11.186 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:58:11.186 INFO:teuthology.orchestra.run.smithi032.stdout:Filesystem 'cephfs' (1) 2024-05-12T03:58:11.186 INFO:teuthology.orchestra.run.smithi032.stdout:fs_name cephfs 2024-05-12T03:58:11.186 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 18 2024-05-12T03:58:11.186 INFO:teuthology.orchestra.run.smithi032.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-12T03:58:11.187 INFO:teuthology.orchestra.run.smithi032.stdout:created 2024-05-12T03:57:48.904614+0000 2024-05-12T03:58:11.187 INFO:teuthology.orchestra.run.smithi032.stdout:modified 2024-05-12T03:58:06.821591+0000 2024-05-12T03:58:11.187 INFO:teuthology.orchestra.run.smithi032.stdout:tableserver 0 2024-05-12T03:58:11.187 INFO:teuthology.orchestra.run.smithi032.stdout:root 0 2024-05-12T03:58:11.187 INFO:teuthology.orchestra.run.smithi032.stdout:session_timeout 60 2024-05-12T03:58:11.187 INFO:teuthology.orchestra.run.smithi032.stdout:session_autoclose 300 2024-05-12T03:58:11.187 INFO:teuthology.orchestra.run.smithi032.stdout:max_file_size 1099511627776 2024-05-12T03:58:11.187 INFO:teuthology.orchestra.run.smithi032.stdout:required_client_features {} 2024-05-12T03:58:11.187 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure 0 2024-05-12T03:58:11.187 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure_osd_epoch 46 2024-05-12T03:58:11.187 INFO:teuthology.orchestra.run.smithi032.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-12T03:58:11.187 INFO:teuthology.orchestra.run.smithi032.stdout:max_mds 1 2024-05-12T03:58:11.187 INFO:teuthology.orchestra.run.smithi032.stdout:in 0 2024-05-12T03:58:11.187 INFO:teuthology.orchestra.run.smithi032.stdout:up {0=14474} 2024-05-12T03:58:11.187 INFO:teuthology.orchestra.run.smithi032.stdout:failed 2024-05-12T03:58:11.187 INFO:teuthology.orchestra.run.smithi032.stdout:damaged 2024-05-12T03:58:11.188 INFO:teuthology.orchestra.run.smithi032.stdout:stopped 2024-05-12T03:58:11.188 INFO:teuthology.orchestra.run.smithi032.stdout:data_pools [3] 2024-05-12T03:58:11.188 INFO:teuthology.orchestra.run.smithi032.stdout:metadata_pool 2 2024-05-12T03:58:11.188 INFO:teuthology.orchestra.run.smithi032.stdout:inline_data enabled 2024-05-12T03:58:11.188 INFO:teuthology.orchestra.run.smithi032.stdout:balancer 2024-05-12T03:58:11.188 INFO:teuthology.orchestra.run.smithi032.stdout:standby_count_wanted 1 2024-05-12T03:58:11.188 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.qpsawh{0:14474} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.32:6828/3100155296,v1:172.21.15.32:6829/3100155296] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T03:58:11.188 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi033.jgbjhk{0:24267} state up:standby-replay seq 3 join_fscid=1 addr [v2:172.21.15.33:6824/55417471,v1:172.21.15.33:6825/55417471] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T03:58:11.188 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:58:11.188 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:58:11.188 INFO:teuthology.orchestra.run.smithi032.stdout:Standby daemons: 2024-05-12T03:58:11.188 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:58:11.188 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi033.hglwml{-1:24283} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/433858238,v1:172.21.15.33:6827/433858238] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T03:58:11.188 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.ialtqb{-1:24287} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.32:6826/3703132329,v1:172.21.15.32:6827/3703132329] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T03:58:11.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:11.187+0000 7f19b23a1700 1 -- 172.21.15.32:0/4230483479 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f19980692b0 msgr2=0x7f199806b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:11.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:11.187+0000 7f19b23a1700 1 --2- 172.21.15.32:0/4230483479 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f19980692b0 0x7f199806b760 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f19a8000c00 tx=0x7f19a8009e30 comp rx=0 tx=0).stop 2024-05-12T03:58:11.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:11.187+0000 7f19b23a1700 1 -- 172.21.15.32:0/4230483479 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19ac0ffd40 msgr2=0x7f19ac0fb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:11.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:11.187+0000 7f19b23a1700 1 --2- 172.21.15.32:0/4230483479 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19ac0ffd40 0x7f19ac0fb700 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f199c002a50 tx=0x7f199c0046b0 comp rx=0 tx=0).stop 2024-05-12T03:58:11.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:11.187+0000 7f19b23a1700 1 -- 172.21.15.32:0/4230483479 shutdown_connections 2024-05-12T03:58:11.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:11.187+0000 7f19b23a1700 1 --2- 172.21.15.32:0/4230483479 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f19980692b0 0x7f199806b760 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:11.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:11.187+0000 7f19b23a1700 1 --2- 172.21.15.32:0/4230483479 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19ac066b50 0x7f19ac0fd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:11.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:11.187+0000 7f19b23a1700 1 --2- 172.21.15.32:0/4230483479 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19ac0ffd40 0x7f19ac0fb700 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:11.190 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:11.187+0000 7f19b23a1700 1 -- 172.21.15.32:0/4230483479 >> 172.21.15.32:0/4230483479 conn(0x7f19ac0f7270 msgr2=0x7f19ac0f7690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:58:11.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:11.187+0000 7f19b23a1700 1 -- 172.21.15.32:0/4230483479 shutdown_connections 2024-05-12T03:58:11.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:11.187+0000 7f19b23a1700 1 -- 172.21.15.32:0/4230483479 wait complete. 2024-05-12T03:58:11.191 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 18 2024-05-12T03:58:11.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:11 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/4230483479' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-12T03:58:11.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:11 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/4230483479' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-12T03:58:11.999 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- bash -c 'ceph --format=json fs dump | jq -e ".filesystems | length == 1"' 2024-05-12T03:58:12.734 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T03:58:13.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:12 smithi033 ceph-mon[49525]: pgmap v200: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 7.4 KiB/s rd, 7 op/s 2024-05-12T03:58:13.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:12 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:58:13.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:12 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:58:13.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:12 smithi032 ceph-mon[40008]: pgmap v200: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 7.4 KiB/s rd, 7 op/s 2024-05-12T03:58:13.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:12 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:58:13.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:12 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:58:15.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:14 smithi033 ceph-mon[49525]: pgmap v201: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 6.7 KiB/s rd, 0 B/s wr, 7 op/s 2024-05-12T03:58:15.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:14 smithi032 ceph-mon[40008]: pgmap v201: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 6.7 KiB/s rd, 0 B/s wr, 7 op/s 2024-05-12T03:58:15.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:15.367+0000 7fa424234700 1 -- 172.21.15.32:0/1387553850 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa41c0fe080 msgr2=0x7fa41c0fe4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:15.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:15.367+0000 7fa424234700 1 --2- 172.21.15.32:0/1387553850 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa41c0fe080 0x7fa41c0fe4f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa40c004940 tx=0x7fa40c009e30 comp rx=0 tx=0).stop 2024-05-12T03:58:15.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:15.367+0000 7fa424234700 1 -- 172.21.15.32:0/1387553850 shutdown_connections 2024-05-12T03:58:15.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:15.367+0000 7fa424234700 1 --2- 172.21.15.32:0/1387553850 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa41c0fe080 0x7fa41c0fe4f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:15.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:15.368+0000 7fa424234700 1 --2- 172.21.15.32:0/1387553850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa41c0fce80 0x7fa41c0fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:15.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:15.368+0000 7fa424234700 1 -- 172.21.15.32:0/1387553850 >> 172.21.15.32:0/1387553850 conn(0x7fa41c0f8590 msgr2=0x7fa41c0fa9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:58:15.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:15.368+0000 7fa424234700 1 -- 172.21.15.32:0/1387553850 shutdown_connections 2024-05-12T03:58:15.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:15.368+0000 7fa424234700 1 -- 172.21.15.32:0/1387553850 wait complete. 2024-05-12T03:58:15.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:15.368+0000 7fa424234700 1 Processor -- start 2024-05-12T03:58:15.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:15.368+0000 7fa424234700 1 -- start start 2024-05-12T03:58:15.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:15.368+0000 7fa424234700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa41c0fce80 0x7fa41c071de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:15.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:15.368+0000 7fa424234700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa41c0fe080 0x7fa41c0702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:15.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:15.368+0000 7fa424234700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa41c070860 con 0x7fa41c0fce80 2024-05-12T03:58:15.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:15.368+0000 7fa424234700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa41c0709d0 con 0x7fa41c0fe080 2024-05-12T03:58:15.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:15.369+0000 7fa421fd0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa41c0fce80 0x7fa41c071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:15.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:15.369+0000 7fa421fd0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa41c0fce80 0x7fa41c071de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57836/0 (socket says 172.21.15.32:57836) 2024-05-12T03:58:15.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:15.369+0000 7fa421fd0700 1 -- 172.21.15.32:0/721888629 learned_addr learned my addr 172.21.15.32:0/721888629 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:58:15.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:15.369+0000 7fa4217cf700 1 --2- 172.21.15.32:0/721888629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa41c0fe080 0x7fa41c0702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:15.769 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:15.369+0000 7fa421fd0700 1 -- 172.21.15.32:0/721888629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa41c0fe080 msgr2=0x7fa41c0702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:15.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:15.369+0000 7fa421fd0700 1 --2- 172.21.15.32:0/721888629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa41c0fe080 0x7fa41c0702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:15.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:15.369+0000 7fa421fd0700 1 -- 172.21.15.32:0/721888629 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa40c005040 con 0x7fa41c0fce80 2024-05-12T03:58:15.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:15.369+0000 7fa421fd0700 1 --2- 172.21.15.32:0/721888629 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa41c0fce80 0x7fa41c071de0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa41800c800 tx=0x7fa418004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:58:15.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:15.369+0000 7fa4127fc700 1 -- 172.21.15.32:0/721888629 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa41800a760 con 0x7fa41c0fce80 2024-05-12T03:58:15.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:15.369+0000 7fa424234700 1 -- 172.21.15.32:0/721888629 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa41c070cb0 con 0x7fa41c0fce80 2024-05-12T03:58:15.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:15.369+0000 7fa4127fc700 1 -- 172.21.15.32:0/721888629 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa41800ad70 con 0x7fa41c0fce80 2024-05-12T03:58:15.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:15.369+0000 7fa4127fc700 1 -- 172.21.15.32:0/721888629 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa4180113a0 con 0x7fa41c0fce80 2024-05-12T03:58:15.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:15.369+0000 7fa424234700 1 -- 172.21.15.32:0/721888629 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa41c115980 con 0x7fa41c0fce80 2024-05-12T03:58:15.770 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:15.370+0000 7fa424234700 1 -- 172.21.15.32:0/721888629 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa41c05f5c0 con 0x7fa41c0fce80 2024-05-12T03:58:15.771 INFO:teuthology.orchestra.run.smithi032.stdout:true 2024-05-12T03:58:15.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:15.371+0000 7fa4127fc700 1 -- 172.21.15.32:0/721888629 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fa418012400 con 0x7fa41c0fce80 2024-05-12T03:58:15.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:15.371+0000 7fa4127fc700 1 --2- 172.21.15.32:0/721888629 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fa4080692a0 0x7fa40806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:15.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:15.371+0000 7fa4217cf700 1 --2- 172.21.15.32:0/721888629 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fa4080692a0 0x7fa40806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:15.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:15.371+0000 7fa4127fc700 1 -- 172.21.15.32:0/721888629 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fa418087280 con 0x7fa41c0fce80 2024-05-12T03:58:15.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:15.371+0000 7fa4217cf700 1 --2- 172.21.15.32:0/721888629 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fa4080692a0 0x7fa40806b750 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa40c00b3b0 tx=0x7fa40c009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:58:15.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:15.375+0000 7fa4127fc700 1 -- 172.21.15.32:0/721888629 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fa418053560 con 0x7fa41c0fce80 2024-05-12T03:58:15.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:15.614+0000 7fa424234700 1 -- 172.21.15.32:0/721888629 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa41c04bed0 con 0x7fa41c0fce80 2024-05-12T03:58:15.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:15.614+0000 7fa4127fc700 1 -- 172.21.15.32:0/721888629 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4687 (secure 0 0 0) 0x7fa418056b80 con 0x7fa41c0fce80 2024-05-12T03:58:15.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:15.616+0000 7fa424234700 1 -- 172.21.15.32:0/721888629 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fa4080692a0 msgr2=0x7fa40806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:15.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:15.616+0000 7fa424234700 1 --2- 172.21.15.32:0/721888629 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fa4080692a0 0x7fa40806b750 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa40c00b3b0 tx=0x7fa40c009e30 comp rx=0 tx=0).stop 2024-05-12T03:58:15.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:15.616+0000 7fa424234700 1 -- 172.21.15.32:0/721888629 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa41c0fce80 msgr2=0x7fa41c071de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:15.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:15.616+0000 7fa424234700 1 --2- 172.21.15.32:0/721888629 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa41c0fce80 0x7fa41c071de0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa41800c800 tx=0x7fa418004470 comp rx=0 tx=0).stop 2024-05-12T03:58:15.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:15.616+0000 7fa424234700 1 -- 172.21.15.32:0/721888629 shutdown_connections 2024-05-12T03:58:15.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:15.616+0000 7fa424234700 1 --2- 172.21.15.32:0/721888629 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fa4080692a0 0x7fa40806b750 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:15.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:15.616+0000 7fa424234700 1 --2- 172.21.15.32:0/721888629 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa41c0fce80 0x7fa41c071de0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:15.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:15.616+0000 7fa424234700 1 --2- 172.21.15.32:0/721888629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa41c0fe080 0x7fa41c0702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:15.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:15.616+0000 7fa424234700 1 -- 172.21.15.32:0/721888629 >> 172.21.15.32:0/721888629 conn(0x7fa41c0f8590 msgr2=0x7fa41c0f9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:58:15.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:15.616+0000 7fa424234700 1 -- 172.21.15.32:0/721888629 shutdown_connections 2024-05-12T03:58:15.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:15.616+0000 7fa424234700 1 -- 172.21.15.32:0/721888629 wait complete. 2024-05-12T03:58:15.772 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 18 2024-05-12T03:58:16.039 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:15 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/721888629' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-12T03:58:16.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:15 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/721888629' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-12T03:58:16.526 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- bash -c 'while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done' 2024-05-12T03:58:17.075 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:16 smithi032 ceph-mon[40008]: pgmap v202: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 6.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-05-12T03:58:17.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:16 smithi033 ceph-mon[49525]: pgmap v202: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 6.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-05-12T03:58:17.193 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T03:58:19.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:19.161+0000 7f11d0fde700 1 -- 172.21.15.32:0/3975341494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11cc0fce80 msgr2=0x7f11cc0fd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:19.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:19.161+0000 7f11d0fde700 1 --2- 172.21.15.32:0/3975341494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11cc0fce80 0x7f11cc0fd290 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f11bc004940 tx=0x7f11bc009e30 comp rx=0 tx=0).stop 2024-05-12T03:58:19.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:19.161+0000 7f11d0fde700 1 -- 172.21.15.32:0/3975341494 shutdown_connections 2024-05-12T03:58:19.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:19.161+0000 7f11d0fde700 1 --2- 172.21.15.32:0/3975341494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11cc0fe080 0x7f11cc0fe4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:19.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:19.161+0000 7f11d0fde700 1 --2- 172.21.15.32:0/3975341494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11cc0fce80 0x7f11cc0fd290 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:19.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:19.162+0000 7f11d0fde700 1 -- 172.21.15.32:0/3975341494 >> 172.21.15.32:0/3975341494 conn(0x7f11cc0f8590 msgr2=0x7f11cc0fa9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:58:19.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:19.162+0000 7f11d0fde700 1 -- 172.21.15.32:0/3975341494 shutdown_connections 2024-05-12T03:58:19.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:19.162+0000 7f11d0fde700 1 -- 172.21.15.32:0/3975341494 wait complete. 2024-05-12T03:58:19.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:19.162+0000 7f11d0fde700 1 Processor -- start 2024-05-12T03:58:19.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:19.162+0000 7f11d0fde700 1 -- start start 2024-05-12T03:58:19.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:19.162+0000 7f11d0fde700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11cc0fce80 0x7f11cc10d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:19.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:19.162+0000 7f11d0fde700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11cc0fe080 0x7f11cc10de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:19.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:19.162+0000 7f11d0fde700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11cc10e490 con 0x7f11cc0fce80 2024-05-12T03:58:19.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:19.162+0000 7f11d0fde700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11cc10ece0 con 0x7f11cc0fe080 2024-05-12T03:58:19.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:19.163+0000 7f11ca59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11cc0fce80 0x7f11cc10d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:19.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:19.163+0000 7f11ca59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11cc0fce80 0x7f11cc10d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57844/0 (socket says 172.21.15.32:57844) 2024-05-12T03:58:19.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:19.163+0000 7f11ca59c700 1 -- 172.21.15.32:0/2929838092 learned_addr learned my addr 172.21.15.32:0/2929838092 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:58:19.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:19.163+0000 7f11c9d9b700 1 --2- 172.21.15.32:0/2929838092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11cc0fe080 0x7f11cc10de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:19.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:19.163+0000 7f11ca59c700 1 -- 172.21.15.32:0/2929838092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11cc0fe080 msgr2=0x7f11cc10de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:19.186 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:19.163+0000 7f11ca59c700 1 --2- 172.21.15.32:0/2929838092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11cc0fe080 0x7f11cc10de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:19.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:19.163+0000 7f11ca59c700 1 -- 172.21.15.32:0/2929838092 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11bc005040 con 0x7f11cc0fce80 2024-05-12T03:58:19.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:19.163+0000 7f11ca59c700 1 --2- 172.21.15.32:0/2929838092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11cc0fce80 0x7f11cc10d950 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f11bc000c00 tx=0x7f11bc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:58:19.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:19.163+0000 7f11c2ffd700 1 -- 172.21.15.32:0/2929838092 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11bc00e040 con 0x7f11cc0fce80 2024-05-12T03:58:19.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:19.163+0000 7f11c2ffd700 1 -- 172.21.15.32:0/2929838092 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f11bc00bd20 con 0x7f11cc0fce80 2024-05-12T03:58:19.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:19.163+0000 7f11d0fde700 1 -- 172.21.15.32:0/2929838092 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11cc10ef60 con 0x7f11cc0fce80 2024-05-12T03:58:19.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:19.163+0000 7f11c2ffd700 1 -- 172.21.15.32:0/2929838092 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11bc01c630 con 0x7f11cc0fce80 2024-05-12T03:58:19.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:19.163+0000 7f11d0fde700 1 -- 172.21.15.32:0/2929838092 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11cc10f4a0 con 0x7f11cc0fce80 2024-05-12T03:58:19.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:19.164+0000 7f11d0fde700 1 -- 172.21.15.32:0/2929838092 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11cc05f5c0 con 0x7f11cc0fce80 2024-05-12T03:58:19.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:19.165+0000 7f11c2ffd700 1 -- 172.21.15.32:0/2929838092 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f11bc013420 con 0x7f11cc0fce80 2024-05-12T03:58:19.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:19.165+0000 7f11c2ffd700 1 --2- 172.21.15.32:0/2929838092 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f11b80692a0 0x7f11b806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:19.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:19.165+0000 7f11c2ffd700 1 -- 172.21.15.32:0/2929838092 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f11bc0884e0 con 0x7f11cc0fce80 2024-05-12T03:58:19.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:19.165+0000 7f11c9d9b700 1 --2- 172.21.15.32:0/2929838092 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f11b80692a0 0x7f11b806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:19.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:19.165+0000 7f11c9d9b700 1 --2- 172.21.15.32:0/2929838092 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f11b80692a0 0x7f11b806b750 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f11b4004c40 tx=0x7f11b4009040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:58:19.187 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:19.170+0000 7f11c2ffd700 1 -- 172.21.15.32:0/2929838092 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f11bc0582f0 con 0x7f11cc0fce80 2024-05-12T03:58:19.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:19 smithi032 ceph-mon[40008]: pgmap v203: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 4.7 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-12T03:58:19.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:19 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:58:19.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:19 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:58:19.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:19 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:58:19.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:19 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T03:58:19.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:19 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:58:19.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:19 smithi033 ceph-mon[49525]: pgmap v203: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 4.7 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-12T03:58:19.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:19 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:58:19.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:19 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:58:19.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:19 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:58:19.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:19 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T03:58:19.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:19 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:58:19.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:19.450+0000 7f11d0fde700 1 -- 172.21.15.32:0/2929838092 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mds versions", "format": "json"} v 0) v1 -- 0x7f11cc04bed0 con 0x7f11cc0fce80 2024-05-12T03:58:19.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:19.450+0000 7f11c2ffd700 1 -- 172.21.15.32:0/2929838092 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mds versions", "format": "json"}]=0 v18) v1 ==== 78+0+99 (secure 0 0 0) 0x7f11bc057e80 con 0x7f11cc0fce80 2024-05-12T03:58:19.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:19.452+0000 7f11d0fde700 1 -- 172.21.15.32:0/2929838092 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f11b80692a0 msgr2=0x7f11b806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:19.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:19.452+0000 7f11d0fde700 1 --2- 172.21.15.32:0/2929838092 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f11b80692a0 0x7f11b806b750 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f11b4004c40 tx=0x7f11b4009040 comp rx=0 tx=0).stop 2024-05-12T03:58:19.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:19.452+0000 7f11d0fde700 1 -- 172.21.15.32:0/2929838092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11cc0fce80 msgr2=0x7f11cc10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:19.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:19.452+0000 7f11d0fde700 1 --2- 172.21.15.32:0/2929838092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11cc0fce80 0x7f11cc10d950 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f11bc000c00 tx=0x7f11bc009e30 comp rx=0 tx=0).stop 2024-05-12T03:58:19.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:19.453+0000 7f11d0fde700 1 -- 172.21.15.32:0/2929838092 shutdown_connections 2024-05-12T03:58:19.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:19.453+0000 7f11d0fde700 1 --2- 172.21.15.32:0/2929838092 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f11b80692a0 0x7f11b806b750 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:19.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:19.453+0000 7f11d0fde700 1 --2- 172.21.15.32:0/2929838092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11cc0fce80 0x7f11cc10d950 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:19.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:19.453+0000 7f11d0fde700 1 --2- 172.21.15.32:0/2929838092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11cc0fe080 0x7f11cc10de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:19.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:19.453+0000 7f11d0fde700 1 -- 172.21.15.32:0/2929838092 >> 172.21.15.32:0/2929838092 conn(0x7f11cc0f8590 msgr2=0x7f11cc0f9d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:58:19.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:19.453+0000 7f11d0fde700 1 -- 172.21.15.32:0/2929838092 shutdown_connections 2024-05-12T03:58:19.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:19.453+0000 7f11d0fde700 1 -- 172.21.15.32:0/2929838092 wait complete. 2024-05-12T03:58:19.464 INFO:teuthology.orchestra.run.smithi032.stdout:true 2024-05-12T03:58:20.244 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:20 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/2929838092' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-05-12T03:58:20.268 INFO:teuthology.run_tasks:Running task fs.pre_upgrade_save... 2024-05-12T03:58:20.281 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-12T03:58:20.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:20 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/2929838092' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-05-12T03:58:20.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.441+0000 7f9a58178700 1 -- 172.21.15.32:0/1946766688 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a500fce70 msgr2=0x7f9a500fd2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:20.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.441+0000 7f9a58178700 1 --2- 172.21.15.32:0/1946766688 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a500fce70 0x7f9a500fd2e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9a40004940 tx=0x7f9a40009e30 comp rx=0 tx=0).stop 2024-05-12T03:58:20.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.441+0000 7f9a58178700 1 -- 172.21.15.32:0/1946766688 shutdown_connections 2024-05-12T03:58:20.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.441+0000 7f9a58178700 1 --2- 172.21.15.32:0/1946766688 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a500fce70 0x7f9a500fd2e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:20.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.441+0000 7f9a58178700 1 --2- 172.21.15.32:0/1946766688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a500fbbc0 0x7f9a500fbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:20.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.441+0000 7f9a58178700 1 -- 172.21.15.32:0/1946766688 >> 172.21.15.32:0/1946766688 conn(0x7f9a500f7340 msgr2=0x7f9a500f9750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:58:20.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.441+0000 7f9a58178700 1 -- 172.21.15.32:0/1946766688 shutdown_connections 2024-05-12T03:58:20.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.441+0000 7f9a58178700 1 -- 172.21.15.32:0/1946766688 wait complete. 2024-05-12T03:58:20.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.442+0000 7f9a58178700 1 Processor -- start 2024-05-12T03:58:20.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.442+0000 7f9a58178700 1 -- start start 2024-05-12T03:58:20.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.442+0000 7f9a58178700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a500fbbc0 0x7f9a50066530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:20.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.442+0000 7f9a58178700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a500fce70 0x7f9a50114970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:20.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.442+0000 7f9a58178700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5010e410 con 0x7f9a500fce70 2024-05-12T03:58:20.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.442+0000 7f9a58178700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a50066aa0 con 0x7f9a500fbbc0 2024-05-12T03:58:20.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.442+0000 7f9a55713700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a500fce70 0x7f9a50114970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:20.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.442+0000 7f9a55713700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a500fce70 0x7f9a50114970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57862/0 (socket says 172.21.15.32:57862) 2024-05-12T03:58:20.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.442+0000 7f9a55713700 1 -- 172.21.15.32:0/2032213644 learned_addr learned my addr 172.21.15.32:0/2032213644 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:58:20.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.442+0000 7f9a55f14700 1 --2- 172.21.15.32:0/2032213644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a500fbbc0 0x7f9a50066530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:20.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.442+0000 7f9a55713700 1 -- 172.21.15.32:0/2032213644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a500fbbc0 msgr2=0x7f9a50066530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:20.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.442+0000 7f9a55713700 1 --2- 172.21.15.32:0/2032213644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a500fbbc0 0x7f9a50066530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:20.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.442+0000 7f9a55713700 1 -- 172.21.15.32:0/2032213644 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a40005040 con 0x7f9a500fce70 2024-05-12T03:58:20.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.442+0000 7f9a55713700 1 --2- 172.21.15.32:0/2032213644 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a500fce70 0x7f9a50114970 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9a4000b3b0 tx=0x7f9a40009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:58:20.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.442+0000 7f9a467fc700 1 -- 172.21.15.32:0/2032213644 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a4000e040 con 0x7f9a500fce70 2024-05-12T03:58:20.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.443+0000 7f9a467fc700 1 -- 172.21.15.32:0/2032213644 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9a4000bc50 con 0x7f9a500fce70 2024-05-12T03:58:20.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.443+0000 7f9a58178700 1 -- 172.21.15.32:0/2032213644 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a50114f70 con 0x7f9a500fce70 2024-05-12T03:58:20.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.443+0000 7f9a467fc700 1 -- 172.21.15.32:0/2032213644 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a4001c630 con 0x7f9a500fce70 2024-05-12T03:58:20.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.443+0000 7f9a58178700 1 -- 172.21.15.32:0/2032213644 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a50115500 con 0x7f9a500fce70 2024-05-12T03:58:20.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.444+0000 7f9a58178700 1 -- 172.21.15.32:0/2032213644 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a500640c0 con 0x7f9a500fce70 2024-05-12T03:58:20.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.444+0000 7f9a467fc700 1 -- 172.21.15.32:0/2032213644 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f9a40013420 con 0x7f9a500fce70 2024-05-12T03:58:20.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.444+0000 7f9a467fc700 1 --2- 172.21.15.32:0/2032213644 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f9a3c0692a0 0x7f9a3c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:20.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.445+0000 7f9a467fc700 1 -- 172.21.15.32:0/2032213644 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f9a40087ee0 con 0x7f9a500fce70 2024-05-12T03:58:20.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.445+0000 7f9a55f14700 1 --2- 172.21.15.32:0/2032213644 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f9a3c0692a0 0x7f9a3c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:20.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.445+0000 7f9a55f14700 1 --2- 172.21.15.32:0/2032213644 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f9a3c0692a0 0x7f9a3c06b750 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c004f40 tx=0x7f9a4c00b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:58:20.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.449+0000 7f9a467fc700 1 -- 172.21.15.32:0/2032213644 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f9a40057c50 con 0x7f9a500fce70 2024-05-12T03:58:20.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.647+0000 7f9a58178700 1 -- 172.21.15.32:0/2032213644 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9a50104570 con 0x7f9a500fce70 2024-05-12T03:58:20.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.648+0000 7f9a467fc700 1 -- 172.21.15.32:0/2032213644 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+14374 (secure 0 0 0) 0x7f9a400577e0 con 0x7f9a500fce70 2024-05-12T03:58:20.649 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:58:20.649 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":46,"fsid":"a3e178f8-1012-11ef-bc98-c7b262605968","created":"2024-05-12T03:50:18.356038+0000","modified":"2024-05-12T03:58:02.805659+0000","last_up_change":"2024-05-12T03:56:23.877862+0000","last_in_change":"2024-05-12T03:56:03.301729+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-12T03:54:58.296702+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-05-12T03:57:47.465323+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-05-12T03:57:48.324643+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"fc534129-ef70-487d-b8d1-f98fc170c1d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":3497755677},{"type":"v1","addr":"172.21.15.32:6803","nonce":3497755677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":3497755677},{"type":"v1","addr":"172.21.15.32:6805","nonce":3497755677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":3497755677},{"type":"v1","addr":"172.21.15.32:6809","nonce":3497755677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":3497755677},{"type":"v1","addr":"172.21.15.32:6807","nonce":3497755677}]},"public_addr":"172.21.15.32:6803/3497755677","cluster_addr":"172.21.15.32:6805/3497755677","heartbeat_back_addr":"172.21.15.32:6809/3497755677","heartbeat_front_addr":"172.21.15.32:6807/3497755677","state":["exists","up"]},{"osd":1,"uuid":"f8c35b6a-5afc-447b-b318-d56097de28d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":3065624488},{"type":"v1","addr":"172.21.15.32:6811","nonce":3065624488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":3065624488},{"type":"v1","addr":"172.21.15.32:6813","nonce":3065624488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":3065624488},{"type":"v1","addr":"172.21.15.32:6817","nonce":3065624488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":3065624488},{"type":"v1","addr":"172.21.15.32:6815","nonce":3065624488}]},"public_addr":"172.21.15.32:6811/3065624488","cluster_addr":"172.21.15.32:6813/3065624488","heartbeat_back_addr":"172.21.15.32:6817/3065624488","heartbeat_front_addr":"172.21.15.32:6815/3065624488","state":["exists","up"]},{"osd":2,"uuid":"4ed4b55d-cceb-494b-bdff-fef99d6c8cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":3261757113},{"type":"v1","addr":"172.21.15.32:6819","nonce":3261757113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":3261757113},{"type":"v1","addr":"172.21.15.32:6821","nonce":3261757113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":3261757113},{"type":"v1","addr":"172.21.15.32:6825","nonce":3261757113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":3261757113},{"type":"v1","addr":"172.21.15.32:6823","nonce":3261757113}]},"public_addr":"172.21.15.32:6819/3261757113","cluster_addr":"172.21.15.32:6821/3261757113","heartbeat_back_addr":"172.21.15.32:6825/3261757113","heartbeat_front_addr":"172.21.15.32:6823/3261757113","state":["exists","up"]},{"osd":3,"uuid":"ef191267-c0a1-4be7-9e8b-761700af87a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2331766478},{"type":"v1","addr":"172.21.15.33:6801","nonce":2331766478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":2331766478},{"type":"v1","addr":"172.21.15.33:6803","nonce":2331766478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":2331766478},{"type":"v1","addr":"172.21.15.33:6807","nonce":2331766478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":2331766478},{"type":"v1","addr":"172.21.15.33:6805","nonce":2331766478}]},"public_addr":"172.21.15.33:6801/2331766478","cluster_addr":"172.21.15.33:6803/2331766478","heartbeat_back_addr":"172.21.15.33:6807/2331766478","heartbeat_front_addr":"172.21.15.33:6805/2331766478","state":["exists","up"]},{"osd":4,"uuid":"fe45f168-076a-4888-895d-bb710bd0da73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":2404279705},{"type":"v1","addr":"172.21.15.33:6809","nonce":2404279705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":2404279705},{"type":"v1","addr":"172.21.15.33:6811","nonce":2404279705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":2404279705},{"type":"v1","addr":"172.21.15.33:6815","nonce":2404279705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":2404279705},{"type":"v1","addr":"172.21.15.33:6813","nonce":2404279705}]},"public_addr":"172.21.15.33:6809/2404279705","cluster_addr":"172.21.15.33:6811/2404279705","heartbeat_back_addr":"172.21.15.33:6815/2404279705","heartbeat_front_addr":"172.21.15.33:6813/2404279705","state":["exists","up"]},{"osd":5,"uuid":"f7a0e14e-a866-4163-a7df-f8a43b860bc0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2123556026},{"type":"v1","addr":"172.21.15.33:6817","nonce":2123556026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":2123556026},{"type":"v1","addr":"172.21.15.33:6819","nonce":2123556026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":2123556026},{"type":"v1","addr":"172.21.15.33:6823","nonce":2123556026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2123556026},{"type":"v1","addr":"172.21.15.33:6821","nonce":2123556026}]},"public_addr":"172.21.15.33:6817/2123556026","cluster_addr":"172.21.15.33:6819/2123556026","heartbeat_back_addr":"172.21.15.33:6823/2123556026","heartbeat_front_addr":"172.21.15.33:6821/2123556026","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-12T03:53:56.134194+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-12T03:54:25.231975+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-12T03:54:54.991052+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-12T03:55:22.680656+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-12T03:55:52.446678+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-05-12T03:56:22.128957+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/1356801576":"2024-05-13T03:51:25.269581+0000","172.21.15.32:0/625670655":"2024-05-13T03:50:49.378828+0000","172.21.15.32:6801/1121526649":"2024-05-13T03:50:49.378828+0000","172.21.15.32:0/1463926533":"2024-05-13T03:50:49.378828+0000","172.21.15.32:0/2098237559":"2024-05-13T03:50:49.378828+0000","172.21.15.32:6800/1121526649":"2024-05-13T03:50:49.378828+0000","172.21.15.32:6827/1907818042":"2024-05-13T03:58:02.805639+0000","172.21.15.32:0/4293156271":"2024-05-13T03:51:25.269581+0000","172.21.15.32:0/165225041":"2024-05-13T03:51:25.269581+0000","172.21.15.32:6801/1988399583":"2024-05-13T03:52:39.772717+0000","172.21.15.32:0/2900983644":"2024-05-13T03:52:39.772717+0000","172.21.15.32:6800/2967201790":"2024-05-13T03:51:25.269581+0000","172.21.15.32:6801/2967201790":"2024-05-13T03:51:25.269581+0000","172.21.15.32:0/1073406471":"2024-05-13T03:52:39.772717+0000","172.21.15.32:0/3185268518":"2024-05-13T03:52:39.772717+0000","172.21.15.32:6800/1988399583":"2024-05-13T03:52:39.772717+0000","172.21.15.32:6826/1907818042":"2024-05-13T03:58:02.805639+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-12T03:58:20.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.650+0000 7f9a58178700 1 -- 172.21.15.32:0/2032213644 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f9a3c0692a0 msgr2=0x7f9a3c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:20.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.650+0000 7f9a58178700 1 --2- 172.21.15.32:0/2032213644 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f9a3c0692a0 0x7f9a3c06b750 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c004f40 tx=0x7f9a4c00b040 comp rx=0 tx=0).stop 2024-05-12T03:58:20.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.650+0000 7f9a58178700 1 -- 172.21.15.32:0/2032213644 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a500fce70 msgr2=0x7f9a50114970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:20.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.650+0000 7f9a58178700 1 --2- 172.21.15.32:0/2032213644 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a500fce70 0x7f9a50114970 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9a4000b3b0 tx=0x7f9a40009e30 comp rx=0 tx=0).stop 2024-05-12T03:58:20.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.650+0000 7f9a58178700 1 -- 172.21.15.32:0/2032213644 shutdown_connections 2024-05-12T03:58:20.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.650+0000 7f9a58178700 1 --2- 172.21.15.32:0/2032213644 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f9a3c0692a0 0x7f9a3c06b750 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:20.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.650+0000 7f9a58178700 1 --2- 172.21.15.32:0/2032213644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a500fbbc0 0x7f9a50066530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:20.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.650+0000 7f9a58178700 1 --2- 172.21.15.32:0/2032213644 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a500fce70 0x7f9a50114970 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:20.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.650+0000 7f9a58178700 1 -- 172.21.15.32:0/2032213644 >> 172.21.15.32:0/2032213644 conn(0x7f9a500f7340 msgr2=0x7f9a500f8ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:58:20.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.650+0000 7f9a58178700 1 -- 172.21.15.32:0/2032213644 shutdown_connections 2024-05-12T03:58:20.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.650+0000 7f9a58178700 1 -- 172.21.15.32:0/2032213644 wait complete. 2024-05-12T03:58:20.664 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-12T03:54:58.296702+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-05-12T03:57:47.465323+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-05-12T03:57:48.324643+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-12T03:58:20.664 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-12T03:58:20.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.787+0000 7fabbbd03700 1 -- 172.21.15.32:0/4152711621 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabb40fce30 msgr2=0x7fabb40fd2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:20.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.787+0000 7fabbbd03700 1 --2- 172.21.15.32:0/4152711621 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabb40fce30 0x7fabb40fd2a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7faba8004940 tx=0x7faba8009e30 comp rx=0 tx=0).stop 2024-05-12T03:58:20.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.787+0000 7fabbbd03700 1 -- 172.21.15.32:0/4152711621 shutdown_connections 2024-05-12T03:58:20.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.787+0000 7fabbbd03700 1 --2- 172.21.15.32:0/4152711621 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabb40fce30 0x7fabb40fd2a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:20.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.787+0000 7fabbbd03700 1 --2- 172.21.15.32:0/4152711621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabb40fbb80 0x7fabb40fbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:20.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.787+0000 7fabbbd03700 1 -- 172.21.15.32:0/4152711621 >> 172.21.15.32:0/4152711621 conn(0x7fabb40f7340 msgr2=0x7fabb40f9750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:58:20.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.787+0000 7fabbbd03700 1 -- 172.21.15.32:0/4152711621 shutdown_connections 2024-05-12T03:58:20.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.787+0000 7fabbbd03700 1 -- 172.21.15.32:0/4152711621 wait complete. 2024-05-12T03:58:20.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.788+0000 7fabbbd03700 1 Processor -- start 2024-05-12T03:58:20.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.788+0000 7fabbbd03700 1 -- start start 2024-05-12T03:58:20.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.788+0000 7fabbbd03700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabb40fbb80 0x7fabb4065800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:20.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.788+0000 7fabbbd03700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabb40fce30 0x7fabb4112520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:20.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.788+0000 7fabbbd03700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb4112a60 con 0x7fabb40fbb80 2024-05-12T03:58:20.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.788+0000 7fabbbd03700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb4112ba0 con 0x7fabb40fce30 2024-05-12T03:58:20.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.788+0000 7fabb9a9f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabb40fbb80 0x7fabb4065800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:20.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.788+0000 7fabb9a9f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabb40fbb80 0x7fabb4065800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57870/0 (socket says 172.21.15.32:57870) 2024-05-12T03:58:20.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.788+0000 7fabb9a9f700 1 -- 172.21.15.32:0/2058389156 learned_addr learned my addr 172.21.15.32:0/2058389156 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:58:20.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.788+0000 7fabb929e700 1 --2- 172.21.15.32:0/2058389156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabb40fce30 0x7fabb4112520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:20.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.789+0000 7fabb9a9f700 1 -- 172.21.15.32:0/2058389156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabb40fce30 msgr2=0x7fabb4112520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:20.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.789+0000 7fabb9a9f700 1 --2- 172.21.15.32:0/2058389156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabb40fce30 0x7fabb4112520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:20.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.789+0000 7fabb9a9f700 1 -- 172.21.15.32:0/2058389156 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba8005040 con 0x7fabb40fbb80 2024-05-12T03:58:20.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.789+0000 7fabb929e700 1 --2- 172.21.15.32:0/2058389156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabb40fce30 0x7fabb4112520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-12T03:58:20.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.789+0000 7fabb9a9f700 1 --2- 172.21.15.32:0/2058389156 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabb40fbb80 0x7fabb4065800 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fabb0002a50 tx=0x7fabb00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:58:20.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.789+0000 7faba67fc700 1 -- 172.21.15.32:0/2058389156 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fabb0007cc0 con 0x7fabb40fbb80 2024-05-12T03:58:20.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.789+0000 7fabbbd03700 1 -- 172.21.15.32:0/2058389156 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabb4112e00 con 0x7fabb40fbb80 2024-05-12T03:58:20.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.789+0000 7faba67fc700 1 -- 172.21.15.32:0/2058389156 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fabb000f040 con 0x7fabb40fbb80 2024-05-12T03:58:20.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.789+0000 7faba67fc700 1 -- 172.21.15.32:0/2058389156 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fabb0009540 con 0x7fabb40fbb80 2024-05-12T03:58:20.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.789+0000 7fabbbd03700 1 -- 172.21.15.32:0/2058389156 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabb4113340 con 0x7fabb40fbb80 2024-05-12T03:58:20.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.790+0000 7fabbbd03700 1 -- 172.21.15.32:0/2058389156 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabb404c710 con 0x7fabb40fbb80 2024-05-12T03:58:20.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.790+0000 7faba67fc700 1 -- 172.21.15.32:0/2058389156 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fabb00096a0 con 0x7fabb40fbb80 2024-05-12T03:58:20.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.791+0000 7faba67fc700 1 --2- 172.21.15.32:0/2058389156 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7faba00692a0 0x7faba006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:20.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.791+0000 7faba67fc700 1 -- 172.21.15.32:0/2058389156 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fabb0087000 con 0x7fabb40fbb80 2024-05-12T03:58:20.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.791+0000 7fabb929e700 1 --2- 172.21.15.32:0/2058389156 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7faba00692a0 0x7faba006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:20.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.791+0000 7fabb929e700 1 --2- 172.21.15.32:0/2058389156 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7faba00692a0 0x7faba006b750 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7faba8004f90 tx=0x7faba8009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:58:20.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.795+0000 7faba67fc700 1 -- 172.21.15.32:0/2058389156 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fabb0056cf0 con 0x7fabb40fbb80 2024-05-12T03:58:20.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.991+0000 7fabbbd03700 1 -- 172.21.15.32:0/2058389156 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fabb4113560 con 0x7fabb40fbb80 2024-05-12T03:58:20.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.992+0000 7faba67fc700 1 -- 172.21.15.32:0/2058389156 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) v1 ==== 93+0+10 (secure 0 0 0) 0x7fabb0056880 con 0x7fabb40fbb80 2024-05-12T03:58:20.993 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-05-12T03:58:20.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.994+0000 7fabbbd03700 1 -- 172.21.15.32:0/2058389156 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7faba00692a0 msgr2=0x7faba006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:20.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.994+0000 7fabbbd03700 1 --2- 172.21.15.32:0/2058389156 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7faba00692a0 0x7faba006b750 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7faba8004f90 tx=0x7faba8009e30 comp rx=0 tx=0).stop 2024-05-12T03:58:20.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.994+0000 7fabbbd03700 1 -- 172.21.15.32:0/2058389156 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabb40fbb80 msgr2=0x7fabb4065800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:20.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.994+0000 7fabbbd03700 1 --2- 172.21.15.32:0/2058389156 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabb40fbb80 0x7fabb4065800 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fabb0002a50 tx=0x7fabb00046b0 comp rx=0 tx=0).stop 2024-05-12T03:58:20.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.994+0000 7fabbbd03700 1 -- 172.21.15.32:0/2058389156 shutdown_connections 2024-05-12T03:58:20.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.994+0000 7fabbbd03700 1 --2- 172.21.15.32:0/2058389156 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7faba00692a0 0x7faba006b750 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:20.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.994+0000 7fabbbd03700 1 --2- 172.21.15.32:0/2058389156 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabb40fbb80 0x7fabb4065800 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:20.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.994+0000 7fabbbd03700 1 --2- 172.21.15.32:0/2058389156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabb40fce30 0x7fabb4112520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:20.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.994+0000 7fabbbd03700 1 -- 172.21.15.32:0/2058389156 >> 172.21.15.32:0/2058389156 conn(0x7fabb40f7340 msgr2=0x7fabb40f8ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:58:20.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.994+0000 7fabbbd03700 1 -- 172.21.15.32:0/2058389156 shutdown_connections 2024-05-12T03:58:20.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:20.994+0000 7fabbbd03700 1 -- 172.21.15.32:0/2058389156 wait complete. 2024-05-12T03:58:21.007 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-05-12T03:58:21.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.131+0000 7f053c0bb700 1 -- 172.21.15.32:0/3607843944 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05340fbb80 msgr2=0x7f05340fbf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:21.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.131+0000 7f053c0bb700 1 --2- 172.21.15.32:0/3607843944 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05340fbb80 0x7f05340fbf90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0524004940 tx=0x7f0524009e30 comp rx=0 tx=0).stop 2024-05-12T03:58:21.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.131+0000 7f053c0bb700 1 -- 172.21.15.32:0/3607843944 shutdown_connections 2024-05-12T03:58:21.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.131+0000 7f053c0bb700 1 --2- 172.21.15.32:0/3607843944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05340fce30 0x7f05340fd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:21.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.131+0000 7f053c0bb700 1 --2- 172.21.15.32:0/3607843944 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05340fbb80 0x7f05340fbf90 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:21.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.131+0000 7f053c0bb700 1 -- 172.21.15.32:0/3607843944 >> 172.21.15.32:0/3607843944 conn(0x7f05340f7340 msgr2=0x7f05340f9750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:58:21.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.131+0000 7f053c0bb700 1 -- 172.21.15.32:0/3607843944 shutdown_connections 2024-05-12T03:58:21.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.131+0000 7f053c0bb700 1 -- 172.21.15.32:0/3607843944 wait complete. 2024-05-12T03:58:21.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.131+0000 7f053c0bb700 1 Processor -- start 2024-05-12T03:58:21.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.131+0000 7f053c0bb700 1 -- start start 2024-05-12T03:58:21.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.132+0000 7f053c0bb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05340fbb80 0x7f053410d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:21.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.132+0000 7f053c0bb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05340fce30 0x7f053410dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:21.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.132+0000 7f053c0bb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f053410e200 con 0x7f05340fce30 2024-05-12T03:58:21.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.132+0000 7f053c0bb700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f053410e370 con 0x7f05340fbb80 2024-05-12T03:58:21.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.132+0000 7f0539656700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05340fce30 0x7f053410dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:21.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.132+0000 7f0539656700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05340fce30 0x7f053410dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57900/0 (socket says 172.21.15.32:57900) 2024-05-12T03:58:21.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.132+0000 7f0539656700 1 -- 172.21.15.32:0/1386918522 learned_addr learned my addr 172.21.15.32:0/1386918522 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:58:21.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.132+0000 7f0539e57700 1 --2- 172.21.15.32:0/1386918522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05340fbb80 0x7f053410d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:21.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.132+0000 7f0539656700 1 -- 172.21.15.32:0/1386918522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05340fbb80 msgr2=0x7f053410d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:21.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.132+0000 7f0539656700 1 --2- 172.21.15.32:0/1386918522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05340fbb80 0x7f053410d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:21.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.132+0000 7f0539656700 1 -- 172.21.15.32:0/1386918522 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0524005040 con 0x7f05340fce30 2024-05-12T03:58:21.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.132+0000 7f0539656700 1 --2- 172.21.15.32:0/1386918522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05340fce30 0x7f053410dc00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f053000eb40 tx=0x7f053000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:58:21.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.132+0000 7f052a7fc700 1 -- 172.21.15.32:0/1386918522 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0530007cc0 con 0x7f05340fce30 2024-05-12T03:58:21.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.132+0000 7f052a7fc700 1 -- 172.21.15.32:0/1386918522 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f053000f040 con 0x7f05340fce30 2024-05-12T03:58:21.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.132+0000 7f053c0bb700 1 -- 172.21.15.32:0/1386918522 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0534112d20 con 0x7f05340fce30 2024-05-12T03:58:21.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.132+0000 7f052a7fc700 1 -- 172.21.15.32:0/1386918522 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f053000c990 con 0x7f05340fce30 2024-05-12T03:58:21.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.132+0000 7f053c0bb700 1 -- 172.21.15.32:0/1386918522 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05341132e0 con 0x7f05340fce30 2024-05-12T03:58:21.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.133+0000 7f053c0bb700 1 -- 172.21.15.32:0/1386918522 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05341019c0 con 0x7f05340fce30 2024-05-12T03:58:21.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.134+0000 7f052a7fc700 1 -- 172.21.15.32:0/1386918522 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f053000cb10 con 0x7f05340fce30 2024-05-12T03:58:21.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.134+0000 7f052a7fc700 1 --2- 172.21.15.32:0/1386918522 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f05200692a0 0x7f052006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:21.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.134+0000 7f052a7fc700 1 -- 172.21.15.32:0/1386918522 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f0530086ce0 con 0x7f05340fce30 2024-05-12T03:58:21.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.134+0000 7f0539e57700 1 --2- 172.21.15.32:0/1386918522 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f05200692a0 0x7f052006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:21.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.135+0000 7f0539e57700 1 --2- 172.21.15.32:0/1386918522 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f05200692a0 0x7f052006b750 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0524013680 tx=0x7f0524009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:58:21.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.139+0000 7f052a7fc700 1 -- 172.21.15.32:0/1386918522 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f053004e250 con 0x7f05340fce30 2024-05-12T03:58:21.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:21 smithi032 ceph-mon[40008]: pgmap v204: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 4.7 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-12T03:58:21.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:21 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/2032213644' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-12T03:58:21.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:21 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/2058389156' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-12T03:58:21.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.331+0000 7f053c0bb700 1 -- 172.21.15.32:0/1386918522 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f053404c710 con 0x7f05340fce30 2024-05-12T03:58:21.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.332+0000 7f052a7fc700 1 -- 172.21.15.32:0/1386918522 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v46) v1 ==== 107+0+11 (secure 0 0 0) 0x7f0530056b00 con 0x7f05340fce30 2024-05-12T03:58:21.333 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-05-12T03:58:21.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.334+0000 7f053c0bb700 1 -- 172.21.15.32:0/1386918522 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f05200692a0 msgr2=0x7f052006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:21.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.334+0000 7f053c0bb700 1 --2- 172.21.15.32:0/1386918522 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f05200692a0 0x7f052006b750 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0524013680 tx=0x7f0524009e30 comp rx=0 tx=0).stop 2024-05-12T03:58:21.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.334+0000 7f053c0bb700 1 -- 172.21.15.32:0/1386918522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05340fce30 msgr2=0x7f053410dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:21.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.334+0000 7f053c0bb700 1 --2- 172.21.15.32:0/1386918522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05340fce30 0x7f053410dc00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f053000eb40 tx=0x7f053000b040 comp rx=0 tx=0).stop 2024-05-12T03:58:21.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.334+0000 7f053c0bb700 1 -- 172.21.15.32:0/1386918522 shutdown_connections 2024-05-12T03:58:21.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.334+0000 7f053c0bb700 1 --2- 172.21.15.32:0/1386918522 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f05200692a0 0x7f052006b750 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:21.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.334+0000 7f053c0bb700 1 --2- 172.21.15.32:0/1386918522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05340fbb80 0x7f053410d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:21.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.334+0000 7f053c0bb700 1 --2- 172.21.15.32:0/1386918522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05340fce30 0x7f053410dc00 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:21.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.334+0000 7f053c0bb700 1 -- 172.21.15.32:0/1386918522 >> 172.21.15.32:0/1386918522 conn(0x7f05340f7340 msgr2=0x7f0534065390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:58:21.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.334+0000 7f053c0bb700 1 -- 172.21.15.32:0/1386918522 shutdown_connections 2024-05-12T03:58:21.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.334+0000 7f053c0bb700 1 -- 172.21.15.32:0/1386918522 wait complete. 2024-05-12T03:58:21.347 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-05-12T03:58:21.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:21 smithi033 ceph-mon[49525]: pgmap v204: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 4.7 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-12T03:58:21.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:21 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/2032213644' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-12T03:58:21.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:21 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/2058389156' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-12T03:58:21.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.471+0000 7f8bdfcde700 1 -- 172.21.15.32:0/3824249799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bd8065a00 msgr2=0x7f8bd80ff8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:21.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.471+0000 7f8bdfcde700 1 --2- 172.21.15.32:0/3824249799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bd8065a00 0x7f8bd80ff8f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8bc8004940 tx=0x7f8bc8009e30 comp rx=0 tx=0).stop 2024-05-12T03:58:21.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.471+0000 7f8bdfcde700 1 -- 172.21.15.32:0/3824249799 shutdown_connections 2024-05-12T03:58:21.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.471+0000 7f8bdfcde700 1 --2- 172.21.15.32:0/3824249799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bd80ffe30 0x7f8bd8102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:21.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.471+0000 7f8bdfcde700 1 --2- 172.21.15.32:0/3824249799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bd8065a00 0x7f8bd80ff8f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:21.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.471+0000 7f8bdfcde700 1 -- 172.21.15.32:0/3824249799 >> 172.21.15.32:0/3824249799 conn(0x7f8bd80f7340 msgr2=0x7f8bd80f9750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:58:21.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.471+0000 7f8bdfcde700 1 -- 172.21.15.32:0/3824249799 shutdown_connections 2024-05-12T03:58:21.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.471+0000 7f8bdfcde700 1 -- 172.21.15.32:0/3824249799 wait complete. 2024-05-12T03:58:21.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.472+0000 7f8bdfcde700 1 Processor -- start 2024-05-12T03:58:21.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.472+0000 7f8bdfcde700 1 -- start start 2024-05-12T03:58:21.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.472+0000 7f8bdfcde700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bd8065a00 0x7f8bd810d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:21.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.472+0000 7f8bdfcde700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bd80ffe30 0x7f8bd810dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:21.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.472+0000 7f8bdfcde700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd810e350 con 0x7f8bd8065a00 2024-05-12T03:58:21.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.472+0000 7f8bdfcde700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd810e4c0 con 0x7f8bd80ffe30 2024-05-12T03:58:21.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.472+0000 7f8bdda7a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bd8065a00 0x7f8bd810d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:21.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.472+0000 7f8bdda7a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bd8065a00 0x7f8bd810d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57910/0 (socket says 172.21.15.32:57910) 2024-05-12T03:58:21.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.472+0000 7f8bdda7a700 1 -- 172.21.15.32:0/3136232784 learned_addr learned my addr 172.21.15.32:0/3136232784 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:58:21.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.472+0000 7f8bdda7a700 1 -- 172.21.15.32:0/3136232784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bd80ffe30 msgr2=0x7f8bd810dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:21.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.472+0000 7f8bdda7a700 1 --2- 172.21.15.32:0/3136232784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bd80ffe30 0x7f8bd810dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:21.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.472+0000 7f8bdda7a700 1 -- 172.21.15.32:0/3136232784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc8005040 con 0x7f8bd8065a00 2024-05-12T03:58:21.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.472+0000 7f8bdda7a700 1 --2- 172.21.15.32:0/3136232784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bd8065a00 0x7f8bd810d810 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8bc800b3b0 tx=0x7f8bc8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:58:21.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.473+0000 7f8bce7fc700 1 -- 172.21.15.32:0/3136232784 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bc800e040 con 0x7f8bd8065a00 2024-05-12T03:58:21.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.473+0000 7f8bce7fc700 1 -- 172.21.15.32:0/3136232784 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8bc8004290 con 0x7f8bd8065a00 2024-05-12T03:58:21.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.473+0000 7f8bdfcde700 1 -- 172.21.15.32:0/3136232784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bd8112e70 con 0x7f8bd8065a00 2024-05-12T03:58:21.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.473+0000 7f8bce7fc700 1 -- 172.21.15.32:0/3136232784 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bc801c3f0 con 0x7f8bd8065a00 2024-05-12T03:58:21.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.473+0000 7f8bdfcde700 1 -- 172.21.15.32:0/3136232784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bd81133d0 con 0x7f8bd8065a00 2024-05-12T03:58:21.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.474+0000 7f8bdfcde700 1 -- 172.21.15.32:0/3136232784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bd804c710 con 0x7f8bd8065a00 2024-05-12T03:58:21.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.474+0000 7f8bce7fc700 1 -- 172.21.15.32:0/3136232784 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f8bc800bc50 con 0x7f8bd8065a00 2024-05-12T03:58:21.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.475+0000 7f8bce7fc700 1 --2- 172.21.15.32:0/3136232784 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f8bc40692a0 0x7f8bc406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:21.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.475+0000 7f8bce7fc700 1 -- 172.21.15.32:0/3136232784 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f8bc8088fd0 con 0x7f8bd8065a00 2024-05-12T03:58:21.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.475+0000 7f8bdd279700 1 --2- 172.21.15.32:0/3136232784 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f8bc40692a0 0x7f8bc406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:21.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.475+0000 7f8bdd279700 1 --2- 172.21.15.32:0/3136232784 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f8bc40692a0 0x7f8bc406b750 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4004ad0 tx=0x7f8bd4007d40 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:58:21.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.479+0000 7f8bce7fc700 1 -- 172.21.15.32:0/3136232784 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f8bc8058de0 con 0x7f8bd8065a00 2024-05-12T03:58:21.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.676+0000 7f8bdfcde700 1 -- 172.21.15.32:0/3136232784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f8bd81136a0 con 0x7f8bd8065a00 2024-05-12T03:58:21.677 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.676+0000 7f8bce7fc700 1 -- 172.21.15.32:0/3136232784 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v46) v1 ==== 107+0+11 (secure 0 0 0) 0x7f8bc8058970 con 0x7f8bd8065a00 2024-05-12T03:58:21.678 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-05-12T03:58:21.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.679+0000 7f8bdfcde700 1 -- 172.21.15.32:0/3136232784 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f8bc40692a0 msgr2=0x7f8bc406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:21.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.679+0000 7f8bdfcde700 1 --2- 172.21.15.32:0/3136232784 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f8bc40692a0 0x7f8bc406b750 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4004ad0 tx=0x7f8bd4007d40 comp rx=0 tx=0).stop 2024-05-12T03:58:21.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.679+0000 7f8bdfcde700 1 -- 172.21.15.32:0/3136232784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bd8065a00 msgr2=0x7f8bd810d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:21.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.679+0000 7f8bdfcde700 1 --2- 172.21.15.32:0/3136232784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bd8065a00 0x7f8bd810d810 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8bc800b3b0 tx=0x7f8bc8009e30 comp rx=0 tx=0).stop 2024-05-12T03:58:21.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.679+0000 7f8bdfcde700 1 -- 172.21.15.32:0/3136232784 shutdown_connections 2024-05-12T03:58:21.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.679+0000 7f8bdfcde700 1 --2- 172.21.15.32:0/3136232784 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f8bc40692a0 0x7f8bc406b750 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:21.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.679+0000 7f8bdfcde700 1 --2- 172.21.15.32:0/3136232784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bd8065a00 0x7f8bd810d810 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:21.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.679+0000 7f8bdfcde700 1 --2- 172.21.15.32:0/3136232784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bd80ffe30 0x7f8bd810dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:21.680 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.679+0000 7f8bdfcde700 1 -- 172.21.15.32:0/3136232784 >> 172.21.15.32:0/3136232784 conn(0x7f8bd80f7340 msgr2=0x7f8bd80f9720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:58:21.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.679+0000 7f8bdfcde700 1 -- 172.21.15.32:0/3136232784 shutdown_connections 2024-05-12T03:58:21.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.679+0000 7f8bdfcde700 1 -- 172.21.15.32:0/3136232784 wait complete. 2024-05-12T03:58:21.692 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-12T03:58:21.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.812+0000 7fe5b4054700 1 -- 172.21.15.32:0/2213102046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5ac0fe2f0 msgr2=0x7fe5ac0fe700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:21.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.812+0000 7fe5b4054700 1 --2- 172.21.15.32:0/2213102046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5ac0fe2f0 0x7fe5ac0fe700 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4004940 tx=0x7fe5a4009e30 comp rx=0 tx=0).stop 2024-05-12T03:58:21.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.812+0000 7fe5b4054700 1 -- 172.21.15.32:0/2213102046 shutdown_connections 2024-05-12T03:58:21.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.812+0000 7fe5b4054700 1 --2- 172.21.15.32:0/2213102046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5ac0fec40 0x7fe5ac0fad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:21.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.812+0000 7fe5b4054700 1 --2- 172.21.15.32:0/2213102046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5ac0fe2f0 0x7fe5ac0fe700 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:21.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.812+0000 7fe5b4054700 1 -- 172.21.15.32:0/2213102046 >> 172.21.15.32:0/2213102046 conn(0x7fe5ac0f6a90 msgr2=0x7fe5ac0f8ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:58:21.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.812+0000 7fe5b4054700 1 -- 172.21.15.32:0/2213102046 shutdown_connections 2024-05-12T03:58:21.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.812+0000 7fe5b4054700 1 -- 172.21.15.32:0/2213102046 wait complete. 2024-05-12T03:58:21.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.813+0000 7fe5b4054700 1 Processor -- start 2024-05-12T03:58:21.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.813+0000 7fe5b4054700 1 -- start start 2024-05-12T03:58:21.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.813+0000 7fe5b4054700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5ac0fe2f0 0x7fe5ac10d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:21.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.813+0000 7fe5b4054700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5ac0fec40 0x7fe5ac10dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:21.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.813+0000 7fe5b4054700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ac10e290 con 0x7fe5ac0fe2f0 2024-05-12T03:58:21.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.813+0000 7fe5b4054700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ac10e400 con 0x7fe5ac0fec40 2024-05-12T03:58:21.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.813+0000 7fe5b1df0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5ac0fe2f0 0x7fe5ac10d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:21.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.813+0000 7fe5b1df0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5ac0fe2f0 0x7fe5ac10d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57930/0 (socket says 172.21.15.32:57930) 2024-05-12T03:58:21.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.813+0000 7fe5b1df0700 1 -- 172.21.15.32:0/921847699 learned_addr learned my addr 172.21.15.32:0/921847699 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:58:21.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.813+0000 7fe5b15ef700 1 --2- 172.21.15.32:0/921847699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5ac0fec40 0x7fe5ac10dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:21.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.813+0000 7fe5b1df0700 1 -- 172.21.15.32:0/921847699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5ac0fec40 msgr2=0x7fe5ac10dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:21.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.813+0000 7fe5b1df0700 1 --2- 172.21.15.32:0/921847699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5ac0fec40 0x7fe5ac10dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:21.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.813+0000 7fe5b1df0700 1 -- 172.21.15.32:0/921847699 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5a4005040 con 0x7fe5ac0fe2f0 2024-05-12T03:58:21.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.814+0000 7fe5b1df0700 1 --2- 172.21.15.32:0/921847699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5ac0fe2f0 0x7fe5ac10d750 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4009fd0 tx=0x7fe5a4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:58:21.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.814+0000 7fe5aa7fc700 1 -- 172.21.15.32:0/921847699 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5a400e040 con 0x7fe5ac0fe2f0 2024-05-12T03:58:21.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.814+0000 7fe5aa7fc700 1 -- 172.21.15.32:0/921847699 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe5a4004290 con 0x7fe5ac0fe2f0 2024-05-12T03:58:21.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.814+0000 7fe5b4054700 1 -- 172.21.15.32:0/921847699 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5ac112e00 con 0x7fe5ac0fe2f0 2024-05-12T03:58:21.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.814+0000 7fe5aa7fc700 1 -- 172.21.15.32:0/921847699 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5a401c3f0 con 0x7fe5ac0fe2f0 2024-05-12T03:58:21.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.814+0000 7fe5b4054700 1 -- 172.21.15.32:0/921847699 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5ac113340 con 0x7fe5ac0fe2f0 2024-05-12T03:58:21.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.815+0000 7fe5b4054700 1 -- 172.21.15.32:0/921847699 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5ac0640c0 con 0x7fe5ac0fe2f0 2024-05-12T03:58:21.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.815+0000 7fe5aa7fc700 1 -- 172.21.15.32:0/921847699 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fe5a400bc50 con 0x7fe5ac0fe2f0 2024-05-12T03:58:21.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.816+0000 7fe5aa7fc700 1 --2- 172.21.15.32:0/921847699 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fe5940692a0 0x7fe59406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:21.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.816+0000 7fe5b15ef700 1 --2- 172.21.15.32:0/921847699 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fe5940692a0 0x7fe59406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:21.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.816+0000 7fe5aa7fc700 1 -- 172.21.15.32:0/921847699 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fe5a40885e0 con 0x7fe5ac0fe2f0 2024-05-12T03:58:21.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.816+0000 7fe5b15ef700 1 --2- 172.21.15.32:0/921847699 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fe5940692a0 0x7fe59406b750 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0007fd0 tx=0x7fe5a0009040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:58:21.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:21.821+0000 7fe5aa7fc700 1 -- 172.21.15.32:0/921847699 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fe5a40583f0 con 0x7fe5ac0fe2f0 2024-05-12T03:58:22.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:22.065+0000 7fe5b4054700 1 -- 172.21.15.32:0/921847699 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe5ac105660 con 0x7fe5ac0fe2f0 2024-05-12T03:58:22.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:22.066+0000 7fe5aa7fc700 1 -- 172.21.15.32:0/921847699 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4687 (secure 0 0 0) 0x7fe5a4057f80 con 0x7fe5ac0fe2f0 2024-05-12T03:58:22.067 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:58:22.067 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24283,"name":"cephfs.smithi033.hglwml","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6827/433858238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":433858238},{"type":"v1","addr":"172.21.15.33:6827","nonce":433858238}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":24287,"name":"cephfs.smithi032.ialtqb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6827/3703132329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3703132329},{"type":"v1","addr":"172.21.15.32:6827","nonce":3703132329}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":18,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-12T03:57:48.904614+0000","modified":"2024-05-12T03:58:06.821591+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14474},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14474":{"gid":14474,"name":"cephfs.smithi032.qpsawh","rank":0,"incarnation":13,"state":"up:active","state_seq":5,"addr":"172.21.15.32:6829/3100155296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":3100155296},{"type":"v1","addr":"172.21.15.32:6829","nonce":3100155296}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24267":{"gid":24267,"name":"cephfs.smithi033.jgbjhk","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":3,"addr":"172.21.15.33:6825/55417471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":55417471},{"type":"v1","addr":"172.21.15.33:6825","nonce":55417471}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-05-12T03:58:22.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:22.068+0000 7fe5b4054700 1 -- 172.21.15.32:0/921847699 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fe5940692a0 msgr2=0x7fe59406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:22.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:22.068+0000 7fe5b4054700 1 --2- 172.21.15.32:0/921847699 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fe5940692a0 0x7fe59406b750 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0007fd0 tx=0x7fe5a0009040 comp rx=0 tx=0).stop 2024-05-12T03:58:22.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:22.068+0000 7fe5b4054700 1 -- 172.21.15.32:0/921847699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5ac0fe2f0 msgr2=0x7fe5ac10d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:22.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:22.068+0000 7fe5b4054700 1 --2- 172.21.15.32:0/921847699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5ac0fe2f0 0x7fe5ac10d750 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4009fd0 tx=0x7fe5a4009e30 comp rx=0 tx=0).stop 2024-05-12T03:58:22.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:22.068+0000 7fe5b4054700 1 -- 172.21.15.32:0/921847699 shutdown_connections 2024-05-12T03:58:22.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:22.068+0000 7fe5b4054700 1 --2- 172.21.15.32:0/921847699 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fe5940692a0 0x7fe59406b750 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:22.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:22.068+0000 7fe5b4054700 1 --2- 172.21.15.32:0/921847699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5ac0fe2f0 0x7fe5ac10d750 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:22.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:22.068+0000 7fe5b4054700 1 --2- 172.21.15.32:0/921847699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5ac0fec40 0x7fe5ac10dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:22.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:22.068+0000 7fe5b4054700 1 -- 172.21.15.32:0/921847699 >> 172.21.15.32:0/921847699 conn(0x7fe5ac0f6a90 msgr2=0x7fe5ac0f8290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:58:22.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:22.068+0000 7fe5b4054700 1 -- 172.21.15.32:0/921847699 shutdown_connections 2024-05-12T03:58:22.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:22.068+0000 7fe5b4054700 1 -- 172.21.15.32:0/921847699 wait complete. 2024-05-12T03:58:22.070 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 18 2024-05-12T03:58:22.081 DEBUG:tasks.fs:fs fscid=1,name=cephfs state = {'epoch': 18, 'max_mds': 1, 'flags': 50} 2024-05-12T03:58:22.081 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-05-12T03:58:22.106 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-05-12T03:58:22.106 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.1': {}} 2024-05-12T03:58:22.106 INFO:tasks.ceph_fuse:client.0 config is {} 2024-05-12T03:58:22.106 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-12T03:58:22.106 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-12T03:58:22.106 INFO:tasks.ceph_fuse:client.1 config is {} 2024-05-12T03:58:22.107 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-12T03:58:22.107 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-12T03:58:22.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-12T03:58:22.107 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2024-05-12T03:58:22.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-12T03:58:22.126 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link delete ceph-brx 2024-05-12T03:58:22.195 INFO:teuthology.orchestra.run.smithi032.stderr:Cannot find device "ceph-brx" 2024-05-12T03:58:22.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-12T03:58:22.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-12T03:58:22.197 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2024-05-12T03:58:22.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-12T03:58:22.221 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link delete ceph-brx 2024-05-12T03:58:22.301 INFO:teuthology.orchestra.run.smithi033.stderr:Cannot find device "ceph-brx" 2024-05-12T03:58:22.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-12T03:58:22.303 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-05-12T03:58:22.303 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- ceph fs ls 2024-05-12T03:58:22.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:22 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/1386918522' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-05-12T03:58:22.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:22 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/3136232784' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-05-12T03:58:22.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:22 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/921847699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-12T03:58:22.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:22 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/1386918522' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-05-12T03:58:22.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:22 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/3136232784' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-05-12T03:58:22.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:22 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/921847699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-12T03:58:22.582 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T03:58:23.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:23 smithi033 ceph-mon[49525]: pgmap v205: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 0 B/s wr, 3 op/s 2024-05-12T03:58:23.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:23 smithi032 ceph-mon[40008]: pgmap v205: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 0 B/s wr, 3 op/s 2024-05-12T03:58:24.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:24.582+0000 7f4bc77ad700 1 -- 172.21.15.32:0/402022324 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bc00fbd40 msgr2=0x7f4bc00fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:24.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:24.582+0000 7f4bc77ad700 1 --2- 172.21.15.32:0/402022324 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bc00fbd40 0x7f4bc00fc1b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc004940 tx=0x7f4bbc009e30 comp rx=0 tx=0).stop 2024-05-12T03:58:24.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:24.582+0000 7f4bc77ad700 1 -- 172.21.15.32:0/402022324 shutdown_connections 2024-05-12T03:58:24.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:24.582+0000 7f4bc77ad700 1 --2- 172.21.15.32:0/402022324 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bc00fbd40 0x7f4bc00fc1b0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:24.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:24.582+0000 7f4bc77ad700 1 --2- 172.21.15.32:0/402022324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bc0101d90 0x7f4bc0102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:24.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:24.582+0000 7f4bc77ad700 1 -- 172.21.15.32:0/402022324 >> 172.21.15.32:0/402022324 conn(0x7f4bc0071d00 msgr2=0x7f4bc0072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:58:24.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:24.582+0000 7f4bc77ad700 1 -- 172.21.15.32:0/402022324 shutdown_connections 2024-05-12T03:58:24.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:24.582+0000 7f4bc77ad700 1 -- 172.21.15.32:0/402022324 wait complete. 2024-05-12T03:58:24.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:24.583+0000 7f4bc77ad700 1 Processor -- start 2024-05-12T03:58:24.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:24.583+0000 7f4bc77ad700 1 -- start start 2024-05-12T03:58:24.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:24.583+0000 7f4bc77ad700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bc00fbd40 0x7f4bc010ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:24.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:24.583+0000 7f4bc77ad700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bc0101d90 0x7f4bc010ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:24.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:24.583+0000 7f4bc77ad700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc010f5c0 con 0x7f4bc0101d90 2024-05-12T03:58:24.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:24.583+0000 7f4bc77ad700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc010f700 con 0x7f4bc00fbd40 2024-05-12T03:58:24.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:24.583+0000 7f4bc4d48700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bc0101d90 0x7f4bc010ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:24.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:24.583+0000 7f4bc4d48700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bc0101d90 0x7f4bc010ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57848/0 (socket says 172.21.15.32:57848) 2024-05-12T03:58:24.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:24.583+0000 7f4bc4d48700 1 -- 172.21.15.32:0/165660178 learned_addr learned my addr 172.21.15.32:0/165660178 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:58:24.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:24.583+0000 7f4bc5549700 1 --2- 172.21.15.32:0/165660178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bc00fbd40 0x7f4bc010ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:24.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:24.583+0000 7f4bc4d48700 1 -- 172.21.15.32:0/165660178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bc00fbd40 msgr2=0x7f4bc010ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:24.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:24.583+0000 7f4bc4d48700 1 --2- 172.21.15.32:0/165660178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bc00fbd40 0x7f4bc010ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:24.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:24.583+0000 7f4bc4d48700 1 -- 172.21.15.32:0/165660178 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bbc005040 con 0x7f4bc0101d90 2024-05-12T03:58:24.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:24.584+0000 7f4bc4d48700 1 --2- 172.21.15.32:0/165660178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bc0101d90 0x7f4bc010ef70 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc00b3b0 tx=0x7f4bbc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:58:24.585 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:24.584+0000 7f4bb1ffb700 1 -- 172.21.15.32:0/165660178 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4bbc00e040 con 0x7f4bc0101d90 2024-05-12T03:58:24.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:24.584+0000 7f4bb1ffb700 1 -- 172.21.15.32:0/165660178 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4bbc013420 con 0x7f4bc0101d90 2024-05-12T03:58:24.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:24.584+0000 7f4bc77ad700 1 -- 172.21.15.32:0/165660178 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bc0117aa0 con 0x7f4bc0101d90 2024-05-12T03:58:24.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:24.584+0000 7f4bb1ffb700 1 -- 172.21.15.32:0/165660178 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4bbc01c610 con 0x7f4bc0101d90 2024-05-12T03:58:24.586 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:24.584+0000 7f4bc77ad700 1 -- 172.21.15.32:0/165660178 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bc0117fb0 con 0x7f4bc0101d90 2024-05-12T03:58:24.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:24.585+0000 7f4bc77ad700 1 -- 172.21.15.32:0/165660178 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bc00fd430 con 0x7f4bc0101d90 2024-05-12T03:58:24.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:24.586+0000 7f4bb1ffb700 1 -- 172.21.15.32:0/165660178 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f4bbc013a00 con 0x7f4bc0101d90 2024-05-12T03:58:24.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:24.586+0000 7f4bb1ffb700 1 --2- 172.21.15.32:0/165660178 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f4bac0692a0 0x7f4bac06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:24.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:24.586+0000 7f4bb1ffb700 1 -- 172.21.15.32:0/165660178 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f4bbc088330 con 0x7f4bc0101d90 2024-05-12T03:58:24.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:24.586+0000 7f4bc5549700 1 --2- 172.21.15.32:0/165660178 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f4bac0692a0 0x7f4bac06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:24.587 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:24.586+0000 7f4bc5549700 1 --2- 172.21.15.32:0/165660178 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f4bac0692a0 0x7f4bac06b750 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4004f40 tx=0x7f4bb400b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:58:24.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:24.591+0000 7f4bb1ffb700 1 -- 172.21.15.32:0/165660178 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f4bbc0546b0 con 0x7f4bc0101d90 2024-05-12T03:58:24.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:24.834+0000 7f4bc77ad700 1 -- 172.21.15.32:0/165660178 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4bc005f5c0 con 0x7f4bc0101d90 2024-05-12T03:58:24.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:24.834+0000 7f4bb1ffb700 1 -- 172.21.15.32:0/165660178 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) v1 ==== 53+0+83 (secure 0 0 0) 0x7f4bbc057cd0 con 0x7f4bc0101d90 2024-05-12T03:58:24.834 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-05-12T03:58:24.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:24.836+0000 7f4bc77ad700 1 -- 172.21.15.32:0/165660178 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f4bac0692a0 msgr2=0x7f4bac06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:24.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:24.836+0000 7f4bc77ad700 1 --2- 172.21.15.32:0/165660178 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f4bac0692a0 0x7f4bac06b750 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4004f40 tx=0x7f4bb400b040 comp rx=0 tx=0).stop 2024-05-12T03:58:24.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:24.836+0000 7f4bc77ad700 1 -- 172.21.15.32:0/165660178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bc0101d90 msgr2=0x7f4bc010ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:24.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:24.836+0000 7f4bc77ad700 1 --2- 172.21.15.32:0/165660178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bc0101d90 0x7f4bc010ef70 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc00b3b0 tx=0x7f4bbc009e30 comp rx=0 tx=0).stop 2024-05-12T03:58:24.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:24.836+0000 7f4bc77ad700 1 -- 172.21.15.32:0/165660178 shutdown_connections 2024-05-12T03:58:24.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:24.836+0000 7f4bc77ad700 1 --2- 172.21.15.32:0/165660178 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f4bac0692a0 0x7f4bac06b750 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:24.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:24.836+0000 7f4bc77ad700 1 --2- 172.21.15.32:0/165660178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bc00fbd40 0x7f4bc010ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:24.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:24.836+0000 7f4bc77ad700 1 --2- 172.21.15.32:0/165660178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bc0101d90 0x7f4bc010ef70 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:24.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:24.836+0000 7f4bc77ad700 1 -- 172.21.15.32:0/165660178 >> 172.21.15.32:0/165660178 conn(0x7f4bc0071d00 msgr2=0x7f4bc0104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:58:24.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:24.836+0000 7f4bc77ad700 1 -- 172.21.15.32:0/165660178 shutdown_connections 2024-05-12T03:58:24.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:24.836+0000 7f4bc77ad700 1 -- 172.21.15.32:0/165660178 wait complete. 2024-05-12T03:58:25.193 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:25 smithi032 ceph-mon[40008]: pgmap v206: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-05-12T03:58:25.193 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:25 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/165660178' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-12T03:58:25.193 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:25 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T03:58:25.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:25 smithi033 ceph-mon[49525]: pgmap v206: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-05-12T03:58:25.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:25 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/165660178' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-12T03:58:25.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:25 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T03:58:25.626 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-12T03:58:25.627 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-12T03:58:25.627 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2024-05-12T03:58:25.627 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-12T03:58:25.627 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-12T03:58:25.627 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-12T03:58:25.627 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-12T03:58:25.627 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-12T03:58:25.627 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-12T03:58:25.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-12T03:58:25.627 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2024-05-12T03:58:25.650 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-12T03:58:25.650 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-12T03:58:25.650 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2024-05-12T03:58:25.650 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-05-12T03:58:25.650 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2024-05-12T03:58:25.650 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-05-12T03:58:25.650 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-12T03:58:25.650 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2024-05-12T03:58:25.650 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2024-05-12T03:58:25.650 INFO:teuthology.orchestra.run.smithi032.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-12T03:58:25.650 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2024-05-12T03:58:25.650 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2024-05-12T03:58:25.651 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-12T03:58:25.651 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 42047sec preferred_lft 42047sec 2024-05-12T03:58:25.651 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link 2024-05-12T03:58:25.651 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-05-12T03:58:25.651 INFO:teuthology.orchestra.run.smithi032.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-12T03:58:25.651 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2024-05-12T03:58:25.651 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2024-05-12T03:58:25.651 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-12T03:58:25.651 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2024-05-12T03:58:25.651 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2024-05-12T03:58:25.651 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-12T03:58:25.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-12T03:58:25.652 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2024-05-12T03:58:25.652 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-05-12T03:58:25.652 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link add name ceph-brx type bridge 2024-05-12T03:58:25.652 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip addr flush dev ceph-brx 2024-05-12T03:58:25.652 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set ceph-brx up 2024-05-12T03:58:25.652 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-12T03:58:25.652 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-05-12T03:58:25.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-12T03:58:25.847 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-12T03:58:25.878 INFO:teuthology.orchestra.run.smithi032.stdout:1 2024-05-12T03:58:25.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-12T03:58:25.879 DEBUG:teuthology.orchestra.run.smithi032:> route 2024-05-12T03:58:25.951 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2024-05-12T03:58:25.951 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-12T03:58:25.951 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-12T03:58:25.951 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-12T03:58:25.951 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-12T03:58:25.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-12T03:58:25.952 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2024-05-12T03:58:25.952 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-05-12T03:58:25.952 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-12T03:58:25.952 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-12T03:58:25.952 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-12T03:58:25.952 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-05-12T03:58:26.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-12T03:58:26.237 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2024-05-12T03:58:26.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-12T03:58:26.256 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list-id 2024-05-12T03:58:26.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-12T03:58:26.314 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2024-05-12T03:58:26.314 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-05-12T03:58:26.314 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-12T03:58:26.314 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-12T03:58:26.314 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-05-12T03:58:26.410 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-12T03:58:26.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-12T03:58:26.411 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2024-05-12T03:58:26.411 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-05-12T03:58:26.411 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-12T03:58:26.411 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-12T03:58:26.411 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-12T03:58:26.411 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-12T03:58:26.411 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-12T03:58:26.411 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-05-12T03:58:26.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-12T03:58:26.603 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2024-05-12T03:58:26.604 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-05-12T03:58:26.604 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.0 up 2024-05-12T03:58:26.604 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set dev brx.0 master ceph-brx 2024-05-12T03:58:26.604 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-05-12T03:58:26.702 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-12T03:58:26.702 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-12T03:58:26.702 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-12T03:58:26.720 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-12T03:58:26.720 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-12T03:58:26.721 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-12T03:58:26.779 DEBUG:teuthology.orchestra.run.smithi032:> sudo modprobe fuse 2024-05-12T03:58:26.847 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-12T03:58:26.847 DEBUG:teuthology.orchestra.run.smithi032:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-12T03:58:26.915 INFO:teuthology.orchestra.run.smithi032.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-12T03:58:26.916 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-12T03:58:26.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-12T03:58:26.917 DEBUG:teuthology.orchestra.run.smithi032:> ls /sys/fs/fuse/connections 2024-05-12T03:58:26.975 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-12T03:58:26.975 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-12T03:58:27.018 DEBUG:teuthology.orchestra.run.smithi032:> sudo modprobe fuse 2024-05-12T03:58:27.045 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-12T03:58:27.045 DEBUG:teuthology.orchestra.run.smithi032:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-12T03:58:27.113 INFO:teuthology.orchestra.run.smithi032.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-12T03:58:27.142 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-12T03:58:27.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-12T03:58:27.142 DEBUG:teuthology.orchestra.run.smithi032:> ls /sys/fs/fuse/connections 2024-05-12T03:58:27.143 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:2024-05-12T03:58:27.125+0000 7faf8ed67540 -1 init, newargv = 0x561f88125d70 newargc=15 2024-05-12T03:58:27.143 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[100208]: starting ceph client 2024-05-12T03:58:27.143 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[100208]: starting fuse 2024-05-12T03:58:27.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:27 smithi033 ceph-mon[49525]: pgmap v207: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T03:58:27.201 INFO:teuthology.orchestra.run.smithi032.stdout:161 2024-05-12T03:58:27.202 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [161] 2024-05-12T03:58:27.202 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-05-12T03:58:27.202 DEBUG:teuthology.orchestra.run.smithi032:> import glob 2024-05-12T03:58:27.202 DEBUG:teuthology.orchestra.run.smithi032:> import re 2024-05-12T03:58:27.202 DEBUG:teuthology.orchestra.run.smithi032:> import os 2024-05-12T03:58:27.202 DEBUG:teuthology.orchestra.run.smithi032:> import subprocess 2024-05-12T03:58:27.202 DEBUG:teuthology.orchestra.run.smithi032:> 2024-05-12T03:58:27.202 DEBUG:teuthology.orchestra.run.smithi032:> def _find_admin_socket(client_name): 2024-05-12T03:58:27.203 DEBUG:teuthology.orchestra.run.smithi032:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-12T03:58:27.203 DEBUG:teuthology.orchestra.run.smithi032:> files = glob.glob(asok_path) 2024-05-12T03:58:27.203 DEBUG:teuthology.orchestra.run.smithi032:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-12T03:58:27.203 DEBUG:teuthology.orchestra.run.smithi032:> 2024-05-12T03:58:27.203 DEBUG:teuthology.orchestra.run.smithi032:> # Given a non-glob path, it better be there 2024-05-12T03:58:27.203 DEBUG:teuthology.orchestra.run.smithi032:> if "*" not in asok_path: 2024-05-12T03:58:27.203 DEBUG:teuthology.orchestra.run.smithi032:> assert(len(files) == 1) 2024-05-12T03:58:27.203 DEBUG:teuthology.orchestra.run.smithi032:> return files[0] 2024-05-12T03:58:27.203 DEBUG:teuthology.orchestra.run.smithi032:> 2024-05-12T03:58:27.203 DEBUG:teuthology.orchestra.run.smithi032:> for f in files: 2024-05-12T03:58:27.203 DEBUG:teuthology.orchestra.run.smithi032:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-12T03:58:27.203 DEBUG:teuthology.orchestra.run.smithi032:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-12T03:58:27.203 DEBUG:teuthology.orchestra.run.smithi032:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-12T03:58:27.204 DEBUG:teuthology.orchestra.run.smithi032:> contents = proc_f.read() 2024-05-12T03:58:27.204 DEBUG:teuthology.orchestra.run.smithi032:> if mountpoint in contents: 2024-05-12T03:58:27.204 DEBUG:teuthology.orchestra.run.smithi032:> return f 2024-05-12T03:58:27.204 DEBUG:teuthology.orchestra.run.smithi032:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-12T03:58:27.204 DEBUG:teuthology.orchestra.run.smithi032:> 2024-05-12T03:58:27.204 DEBUG:teuthology.orchestra.run.smithi032:> print(_find_admin_socket("client.0")) 2024-05-12T03:58:27.204 DEBUG:teuthology.orchestra.run.smithi032:> ' 2024-05-12T03:58:27.267 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:27 smithi032 ceph-mon[40008]: pgmap v207: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T03:58:27.327 INFO:teuthology.orchestra.run.smithi032.stdout:/var/run/ceph/ceph-client.0.100208.asok 2024-05-12T03:58:27.510 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.100208.asok 2024-05-12T03:58:27.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-12T03:58:27.510 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.100208.asok status 2024-05-12T03:58:27.668 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-05-12T03:58:27.668 INFO:teuthology.orchestra.run.smithi032.stdout: "metadata": { 2024-05-12T03:58:27.668 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_sha1": "c190bdeec0cb2c5a9760aa085c602e6994249982", 2024-05-12T03:58:27.668 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_version": "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)", 2024-05-12T03:58:27.668 INFO:teuthology.orchestra.run.smithi032.stdout: "entity_id": "0", 2024-05-12T03:58:27.668 INFO:teuthology.orchestra.run.smithi032.stdout: "hostname": "smithi032", 2024-05-12T03:58:27.668 INFO:teuthology.orchestra.run.smithi032.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-12T03:58:27.668 INFO:teuthology.orchestra.run.smithi032.stdout: "pid": "100208", 2024-05-12T03:58:27.668 INFO:teuthology.orchestra.run.smithi032.stdout: "root": "/" 2024-05-12T03:58:27.668 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T03:58:27.668 INFO:teuthology.orchestra.run.smithi032.stdout: "dentry_count": 0, 2024-05-12T03:58:27.668 INFO:teuthology.orchestra.run.smithi032.stdout: "dentry_pinned_count": 0, 2024-05-12T03:58:27.668 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 14532, 2024-05-12T03:58:27.669 INFO:teuthology.orchestra.run.smithi032.stdout: "inst": { 2024-05-12T03:58:27.669 INFO:teuthology.orchestra.run.smithi032.stdout: "name": { 2024-05-12T03:58:27.669 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "client", 2024-05-12T03:58:27.669 INFO:teuthology.orchestra.run.smithi032.stdout: "num": 14532 2024-05-12T03:58:27.669 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T03:58:27.669 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": { 2024-05-12T03:58:27.669 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-05-12T03:58:27.669 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "192.168.0.1:0", 2024-05-12T03:58:27.669 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3762506211 2024-05-12T03:58:27.669 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-05-12T03:58:27.669 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T03:58:27.669 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": { 2024-05-12T03:58:27.669 INFO:teuthology.orchestra.run.smithi032.stdout: "type": "v1", 2024-05-12T03:58:27.669 INFO:teuthology.orchestra.run.smithi032.stdout: "addr": "192.168.0.1:0", 2024-05-12T03:58:27.670 INFO:teuthology.orchestra.run.smithi032.stdout: "nonce": 3762506211 2024-05-12T03:58:27.670 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T03:58:27.670 INFO:teuthology.orchestra.run.smithi032.stdout: "inst_str": "client.14532 192.168.0.1:0/3762506211", 2024-05-12T03:58:27.670 INFO:teuthology.orchestra.run.smithi032.stdout: "addr_str": "192.168.0.1:0/3762506211", 2024-05-12T03:58:27.670 INFO:teuthology.orchestra.run.smithi032.stdout: "inode_count": 1, 2024-05-12T03:58:27.670 INFO:teuthology.orchestra.run.smithi032.stdout: "mds_epoch": 18, 2024-05-12T03:58:27.670 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_epoch": 46, 2024-05-12T03:58:27.670 INFO:teuthology.orchestra.run.smithi032.stdout: "osd_epoch_barrier": 0, 2024-05-12T03:58:27.670 INFO:teuthology.orchestra.run.smithi032.stdout: "blocklisted": false, 2024-05-12T03:58:27.670 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs" 2024-05-12T03:58:27.670 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-05-12T03:58:27.678 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- ceph fs ls 2024-05-12T03:58:27.954 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T03:58:29.234 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:29 smithi032 ceph-mon[40008]: pgmap v208: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T03:58:29.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:29 smithi033 ceph-mon[49525]: pgmap v208: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T03:58:29.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:29.787+0000 7fa0e870e700 1 -- 172.21.15.32:0/2925600607 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0e0101d90 msgr2=0x7fa0e0102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:29.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:29.787+0000 7fa0e870e700 1 --2- 172.21.15.32:0/2925600607 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0e0101d90 0x7fa0e0102160 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa0d0004940 tx=0x7fa0d0009e30 comp rx=0 tx=0).stop 2024-05-12T03:58:29.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:29.787+0000 7fa0e870e700 1 -- 172.21.15.32:0/2925600607 shutdown_connections 2024-05-12T03:58:29.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:29.787+0000 7fa0e870e700 1 --2- 172.21.15.32:0/2925600607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0e00fbd40 0x7fa0e00fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:29.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:29.787+0000 7fa0e870e700 1 --2- 172.21.15.32:0/2925600607 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0e0101d90 0x7fa0e0102160 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:29.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:29.787+0000 7fa0e870e700 1 -- 172.21.15.32:0/2925600607 >> 172.21.15.32:0/2925600607 conn(0x7fa0e0071d00 msgr2=0x7fa0e0072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:58:29.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:29.787+0000 7fa0e870e700 1 -- 172.21.15.32:0/2925600607 shutdown_connections 2024-05-12T03:58:29.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:29.787+0000 7fa0e870e700 1 -- 172.21.15.32:0/2925600607 wait complete. 2024-05-12T03:58:29.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:29.788+0000 7fa0e870e700 1 Processor -- start 2024-05-12T03:58:29.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:29.788+0000 7fa0e870e700 1 -- start start 2024-05-12T03:58:29.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:29.788+0000 7fa0e870e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0e00fbd40 0x7fa0e0113a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:29.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:29.788+0000 7fa0e870e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0e0101d90 0x7fa0e0113fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:29.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:29.788+0000 7fa0e870e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e01146a0 con 0x7fa0e0101d90 2024-05-12T03:58:29.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:29.788+0000 7fa0e870e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e010da50 con 0x7fa0e00fbd40 2024-05-12T03:58:29.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:29.788+0000 7fa0e5ca9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0e0101d90 0x7fa0e0113fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:29.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:29.788+0000 7fa0e5ca9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0e0101d90 0x7fa0e0113fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57872/0 (socket says 172.21.15.32:57872) 2024-05-12T03:58:29.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:29.788+0000 7fa0e5ca9700 1 -- 172.21.15.32:0/4293687008 learned_addr learned my addr 172.21.15.32:0/4293687008 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:58:29.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:29.788+0000 7fa0e64aa700 1 --2- 172.21.15.32:0/4293687008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0e00fbd40 0x7fa0e0113a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:29.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:29.788+0000 7fa0e5ca9700 1 -- 172.21.15.32:0/4293687008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0e00fbd40 msgr2=0x7fa0e0113a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:29.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:29.788+0000 7fa0e5ca9700 1 --2- 172.21.15.32:0/4293687008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0e00fbd40 0x7fa0e0113a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:29.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:29.788+0000 7fa0e5ca9700 1 -- 172.21.15.32:0/4293687008 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0d0005040 con 0x7fa0e0101d90 2024-05-12T03:58:29.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:29.788+0000 7fa0e5ca9700 1 --2- 172.21.15.32:0/4293687008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0e0101d90 0x7fa0e0113fc0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc00b790 tx=0x7fa0dc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:58:29.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:29.788+0000 7fa0d6ffd700 1 -- 172.21.15.32:0/4293687008 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa0dc00f6f0 con 0x7fa0e0101d90 2024-05-12T03:58:29.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:29.789+0000 7fa0d6ffd700 1 -- 172.21.15.32:0/4293687008 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa0dc00fd00 con 0x7fa0e0101d90 2024-05-12T03:58:29.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:29.789+0000 7fa0e870e700 1 -- 172.21.15.32:0/4293687008 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0e010dd30 con 0x7fa0e0101d90 2024-05-12T03:58:29.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:29.789+0000 7fa0d6ffd700 1 -- 172.21.15.32:0/4293687008 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa0dc0103a0 con 0x7fa0e0101d90 2024-05-12T03:58:29.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:29.789+0000 7fa0e870e700 1 -- 172.21.15.32:0/4293687008 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0e010e2f0 con 0x7fa0e0101d90 2024-05-12T03:58:29.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:29.789+0000 7fa0e870e700 1 -- 172.21.15.32:0/4293687008 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0e00fd430 con 0x7fa0e0101d90 2024-05-12T03:58:29.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:29.791+0000 7fa0d6ffd700 1 -- 172.21.15.32:0/4293687008 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fa0dc00f850 con 0x7fa0e0101d90 2024-05-12T03:58:29.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:29.791+0000 7fa0d6ffd700 1 --2- 172.21.15.32:0/4293687008 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fa0cc0692a0 0x7fa0cc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:29.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:29.791+0000 7fa0d6ffd700 1 -- 172.21.15.32:0/4293687008 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fa0dc087c20 con 0x7fa0e0101d90 2024-05-12T03:58:29.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:29.791+0000 7fa0e64aa700 1 --2- 172.21.15.32:0/4293687008 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fa0cc0692a0 0x7fa0cc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:29.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:29.792+0000 7fa0e64aa700 1 --2- 172.21.15.32:0/4293687008 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fa0cc0692a0 0x7fa0cc06b750 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa0d0004560 tx=0x7fa0d0009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:58:29.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:29.795+0000 7fa0d6ffd700 1 -- 172.21.15.32:0/4293687008 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fa0dc057970 con 0x7fa0e0101d90 2024-05-12T03:58:30.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:30.037+0000 7fa0e870e700 1 -- 172.21.15.32:0/4293687008 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa0e005f5c0 con 0x7fa0e0101d90 2024-05-12T03:58:30.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:30.038+0000 7fa0d6ffd700 1 -- 172.21.15.32:0/4293687008 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) v1 ==== 53+0+83 (secure 0 0 0) 0x7fa0dc057500 con 0x7fa0e0101d90 2024-05-12T03:58:30.038 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-05-12T03:58:30.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:30.040+0000 7fa0e870e700 1 -- 172.21.15.32:0/4293687008 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fa0cc0692a0 msgr2=0x7fa0cc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:30.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:30.040+0000 7fa0e870e700 1 --2- 172.21.15.32:0/4293687008 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fa0cc0692a0 0x7fa0cc06b750 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa0d0004560 tx=0x7fa0d0009e30 comp rx=0 tx=0).stop 2024-05-12T03:58:30.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:30.040+0000 7fa0e870e700 1 -- 172.21.15.32:0/4293687008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0e0101d90 msgr2=0x7fa0e0113fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:30.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:30.040+0000 7fa0e870e700 1 --2- 172.21.15.32:0/4293687008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0e0101d90 0x7fa0e0113fc0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc00b790 tx=0x7fa0dc009250 comp rx=0 tx=0).stop 2024-05-12T03:58:30.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:30.040+0000 7fa0e870e700 1 -- 172.21.15.32:0/4293687008 shutdown_connections 2024-05-12T03:58:30.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:30.040+0000 7fa0e870e700 1 --2- 172.21.15.32:0/4293687008 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fa0cc0692a0 0x7fa0cc06b750 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:30.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:30.040+0000 7fa0e870e700 1 --2- 172.21.15.32:0/4293687008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0e00fbd40 0x7fa0e0113a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:30.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:30.040+0000 7fa0e870e700 1 --2- 172.21.15.32:0/4293687008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0e0101d90 0x7fa0e0113fc0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:30.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:30.040+0000 7fa0e870e700 1 -- 172.21.15.32:0/4293687008 >> 172.21.15.32:0/4293687008 conn(0x7fa0e0071d00 msgr2=0x7fa0e00fa850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:58:30.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:30.040+0000 7fa0e870e700 1 -- 172.21.15.32:0/4293687008 shutdown_connections 2024-05-12T03:58:30.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:30.040+0000 7fa0e870e700 1 -- 172.21.15.32:0/4293687008 wait complete. 2024-05-12T03:58:30.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:30 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/4293687008' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-12T03:58:30.426 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:30 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/4293687008' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-12T03:58:30.949 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-12T03:58:30.949 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-12T03:58:30.949 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2024-05-12T03:58:30.949 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-12T03:58:30.949 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-12T03:58:30.949 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-12T03:58:30.949 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-12T03:58:30.949 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-12T03:58:30.949 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-12T03:58:30.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-12T03:58:30.949 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2024-05-12T03:58:30.974 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-12T03:58:30.974 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-12T03:58:30.974 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2024-05-12T03:58:30.974 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-05-12T03:58:30.974 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2024-05-12T03:58:30.974 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-05-12T03:58:30.974 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-12T03:58:30.974 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2024-05-12T03:58:30.974 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f0 2024-05-12T03:58:30.974 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-12T03:58:30.974 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2024-05-12T03:58:30.974 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f1 2024-05-12T03:58:30.975 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-12T03:58:30.975 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2024-05-12T03:58:30.975 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f0 2024-05-12T03:58:30.975 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-12T03:58:30.975 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 42036sec preferred_lft 42036sec 2024-05-12T03:58:30.975 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link 2024-05-12T03:58:30.975 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-05-12T03:58:30.975 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-12T03:58:30.975 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2024-05-12T03:58:30.975 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f1 2024-05-12T03:58:30.975 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-12T03:58:30.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-12T03:58:30.976 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2024-05-12T03:58:30.976 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-05-12T03:58:30.976 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link add name ceph-brx type bridge 2024-05-12T03:58:30.976 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip addr flush dev ceph-brx 2024-05-12T03:58:30.976 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set ceph-brx up 2024-05-12T03:58:30.976 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-12T03:58:30.976 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-05-12T03:58:31.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-12T03:58:31.148 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-12T03:58:31.179 INFO:teuthology.orchestra.run.smithi033.stdout:1 2024-05-12T03:58:31.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-12T03:58:31.180 DEBUG:teuthology.orchestra.run.smithi033:> route 2024-05-12T03:58:31.265 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2024-05-12T03:58:31.265 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-12T03:58:31.265 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-12T03:58:31.265 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-12T03:58:31.265 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-12T03:58:31.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-12T03:58:31.266 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2024-05-12T03:58:31.266 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-05-12T03:58:31.266 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-12T03:58:31.266 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-12T03:58:31.266 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-12T03:58:31.266 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-05-12T03:58:31.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:31 smithi032 ceph-mon[40008]: pgmap v209: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T03:58:31.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:31 smithi033 ceph-mon[49525]: pgmap v209: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T03:58:31.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-12T03:58:31.521 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2024-05-12T03:58:31.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-12T03:58:31.540 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list-id 2024-05-12T03:58:31.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-12T03:58:31.599 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2024-05-12T03:58:31.599 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-05-12T03:58:31.599 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-12T03:58:31.599 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-05-12T03:58:31.599 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-05-12T03:58:31.696 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-05-12T03:58:31.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-12T03:58:31.696 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2024-05-12T03:58:31.696 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-05-12T03:58:31.696 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-05-12T03:58:31.697 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-12T03:58:31.697 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-05-12T03:58:31.697 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-05-12T03:58:31.697 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-05-12T03:58:31.697 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-05-12T03:58:31.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-12T03:58:31.914 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2024-05-12T03:58:31.914 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-05-12T03:58:31.914 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 up 2024-05-12T03:58:31.914 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set dev brx.0 master ceph-brx 2024-05-12T03:58:31.914 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-05-12T03:58:32.013 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-05-12T03:58:32.013 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-12T03:58:32.013 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-12T03:58:32.032 INFO:teuthology.orchestra.run.smithi033.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-12T03:58:32.032 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-12T03:58:32.032 DEBUG:teuthology.orchestra.run.smithi033:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-12T03:58:32.090 DEBUG:teuthology.orchestra.run.smithi033:> sudo modprobe fuse 2024-05-12T03:58:32.160 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-12T03:58:32.161 DEBUG:teuthology.orchestra.run.smithi033:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-12T03:58:32.228 INFO:teuthology.orchestra.run.smithi033.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-12T03:58:32.229 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-12T03:58:32.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-12T03:58:32.230 DEBUG:teuthology.orchestra.run.smithi033:> ls /sys/fs/fuse/connections 2024-05-12T03:58:32.288 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-12T03:58:32.288 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-05-12T03:58:32.330 DEBUG:teuthology.orchestra.run.smithi033:> sudo modprobe fuse 2024-05-12T03:58:32.357 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-12T03:58:32.358 DEBUG:teuthology.orchestra.run.smithi033:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-12T03:58:32.426 INFO:teuthology.orchestra.run.smithi033.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-12T03:58:32.427 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-12T03:58:32.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-12T03:58:32.427 DEBUG:teuthology.orchestra.run.smithi033:> ls /sys/fs/fuse/connections 2024-05-12T03:58:32.440 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi033.stderr:2024-05-12T03:58:32.439+0000 7f10c8cdf540 -1 init, newargv = 0x55c87a9a1b80 newargc=15 2024-05-12T03:58:32.440 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi033.stderr:ceph-fuse[81523]: starting ceph client 2024-05-12T03:58:32.445 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi033.stderr:ceph-fuse[81523]: starting fuse 2024-05-12T03:58:32.461 INFO:teuthology.orchestra.run.smithi033.stdout:125 2024-05-12T03:58:32.461 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [125] 2024-05-12T03:58:32.461 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-05-12T03:58:32.461 DEBUG:teuthology.orchestra.run.smithi033:> import glob 2024-05-12T03:58:32.461 DEBUG:teuthology.orchestra.run.smithi033:> import re 2024-05-12T03:58:32.461 DEBUG:teuthology.orchestra.run.smithi033:> import os 2024-05-12T03:58:32.462 DEBUG:teuthology.orchestra.run.smithi033:> import subprocess 2024-05-12T03:58:32.462 DEBUG:teuthology.orchestra.run.smithi033:> 2024-05-12T03:58:32.462 DEBUG:teuthology.orchestra.run.smithi033:> def _find_admin_socket(client_name): 2024-05-12T03:58:32.462 DEBUG:teuthology.orchestra.run.smithi033:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-05-12T03:58:32.462 DEBUG:teuthology.orchestra.run.smithi033:> files = glob.glob(asok_path) 2024-05-12T03:58:32.462 DEBUG:teuthology.orchestra.run.smithi033:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-05-12T03:58:32.462 DEBUG:teuthology.orchestra.run.smithi033:> 2024-05-12T03:58:32.462 DEBUG:teuthology.orchestra.run.smithi033:> # Given a non-glob path, it better be there 2024-05-12T03:58:32.462 DEBUG:teuthology.orchestra.run.smithi033:> if "*" not in asok_path: 2024-05-12T03:58:32.462 DEBUG:teuthology.orchestra.run.smithi033:> assert(len(files) == 1) 2024-05-12T03:58:32.462 DEBUG:teuthology.orchestra.run.smithi033:> return files[0] 2024-05-12T03:58:32.462 DEBUG:teuthology.orchestra.run.smithi033:> 2024-05-12T03:58:32.462 DEBUG:teuthology.orchestra.run.smithi033:> for f in files: 2024-05-12T03:58:32.462 DEBUG:teuthology.orchestra.run.smithi033:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-12T03:58:32.462 DEBUG:teuthology.orchestra.run.smithi033:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-12T03:58:32.462 DEBUG:teuthology.orchestra.run.smithi033:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-12T03:58:32.463 DEBUG:teuthology.orchestra.run.smithi033:> contents = proc_f.read() 2024-05-12T03:58:32.463 DEBUG:teuthology.orchestra.run.smithi033:> if mountpoint in contents: 2024-05-12T03:58:32.463 DEBUG:teuthology.orchestra.run.smithi033:> return f 2024-05-12T03:58:32.463 DEBUG:teuthology.orchestra.run.smithi033:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-12T03:58:32.463 DEBUG:teuthology.orchestra.run.smithi033:> 2024-05-12T03:58:32.463 DEBUG:teuthology.orchestra.run.smithi033:> print(_find_admin_socket("client.1")) 2024-05-12T03:58:32.463 DEBUG:teuthology.orchestra.run.smithi033:> ' 2024-05-12T03:58:32.586 INFO:teuthology.orchestra.run.smithi033.stdout:/var/run/ceph/ceph-client.1.81523.asok 2024-05-12T03:58:32.768 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.81523.asok 2024-05-12T03:58:32.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-12T03:58:32.768 DEBUG:teuthology.orchestra.run.smithi033:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.81523.asok status 2024-05-12T03:58:32.926 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-12T03:58:32.926 INFO:teuthology.orchestra.run.smithi033.stdout: "metadata": { 2024-05-12T03:58:32.926 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_sha1": "c190bdeec0cb2c5a9760aa085c602e6994249982", 2024-05-12T03:58:32.926 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_version": "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)", 2024-05-12T03:58:32.927 INFO:teuthology.orchestra.run.smithi033.stdout: "entity_id": "1", 2024-05-12T03:58:32.927 INFO:teuthology.orchestra.run.smithi033.stdout: "hostname": "smithi033", 2024-05-12T03:58:32.927 INFO:teuthology.orchestra.run.smithi033.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-05-12T03:58:32.927 INFO:teuthology.orchestra.run.smithi033.stdout: "pid": "81523", 2024-05-12T03:58:32.927 INFO:teuthology.orchestra.run.smithi033.stdout: "root": "/" 2024-05-12T03:58:32.927 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-12T03:58:32.927 INFO:teuthology.orchestra.run.smithi033.stdout: "dentry_count": 0, 2024-05-12T03:58:32.927 INFO:teuthology.orchestra.run.smithi033.stdout: "dentry_pinned_count": 0, 2024-05-12T03:58:32.927 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24323, 2024-05-12T03:58:32.927 INFO:teuthology.orchestra.run.smithi033.stdout: "inst": { 2024-05-12T03:58:32.927 INFO:teuthology.orchestra.run.smithi033.stdout: "name": { 2024-05-12T03:58:32.927 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "client", 2024-05-12T03:58:32.927 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 24323 2024-05-12T03:58:32.927 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-12T03:58:32.928 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": { 2024-05-12T03:58:32.928 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-12T03:58:32.928 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "192.168.0.1:0", 2024-05-12T03:58:32.928 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 1200139116 2024-05-12T03:58:32.928 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-12T03:58:32.928 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-12T03:58:32.928 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": { 2024-05-12T03:58:32.928 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-12T03:58:32.928 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "192.168.0.1:0", 2024-05-12T03:58:32.928 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 1200139116 2024-05-12T03:58:32.928 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-12T03:58:32.928 INFO:teuthology.orchestra.run.smithi033.stdout: "inst_str": "client.24323 192.168.0.1:0/1200139116", 2024-05-12T03:58:32.928 INFO:teuthology.orchestra.run.smithi033.stdout: "addr_str": "192.168.0.1:0/1200139116", 2024-05-12T03:58:32.928 INFO:teuthology.orchestra.run.smithi033.stdout: "inode_count": 1, 2024-05-12T03:58:32.928 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_epoch": 18, 2024-05-12T03:58:32.929 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_epoch": 46, 2024-05-12T03:58:32.929 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_epoch_barrier": 0, 2024-05-12T03:58:32.929 INFO:teuthology.orchestra.run.smithi033.stdout: "blocklisted": false, 2024-05-12T03:58:32.929 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs" 2024-05-12T03:58:32.929 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-12T03:58:32.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-12T03:58:32.936 DEBUG:teuthology.orchestra.run.smithi032:> stat --file-system '--printf=%T 2024-05-12T03:58:32.936 DEBUG:teuthology.orchestra.run.smithi032:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-12T03:58:32.961 INFO:teuthology.orchestra.run.smithi032.stdout:fuseblk 2024-05-12T03:58:32.961 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-12T03:58:32.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-12T03:58:32.961 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-12T03:58:33.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-12T03:58:33.040 DEBUG:teuthology.orchestra.run.smithi033:> stat --file-system '--printf=%T 2024-05-12T03:58:33.040 DEBUG:teuthology.orchestra.run.smithi033:> ' -- /home/ubuntu/cephtest/mnt.1 2024-05-12T03:58:33.059 INFO:teuthology.orchestra.run.smithi033.stdout:fuseblk 2024-05-12T03:58:33.059 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-05-12T03:58:33.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-12T03:58:33.059 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-12T03:58:33.131 INFO:teuthology.run_tasks:Running task print... 2024-05-12T03:58:33.141 INFO:teuthology.task.print:**** done client 2024-05-12T03:58:33.141 INFO:teuthology.run_tasks:Running task parallel... 2024-05-12T03:58:33.186 INFO:teuthology.task.parallel:starting parallel... 2024-05-12T03:58:33.187 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-05-12T03:58:33.187 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-05-12T03:58:33.188 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi032.front.sepia.ceph.com 2024-05-12T03:58:33.188 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -e sha1=58249552a22f437b8cb49ed6b1068e1158bed605 -- bash -c 'ceph config set mgr mgr/orchestrator/fail_fs false || true' 2024-05-12T03:58:33.188 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-05-12T03:58:33.188 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-05-12T03:58:33.192 INFO:tasks.workunit:Pulling workunits from ref fccfb453906af8847786c471919d347112ea0360 2024-05-12T03:58:33.192 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-12T03:58:33.192 INFO:tasks.workunit:timeout=3h 2024-05-12T03:58:33.192 INFO:tasks.workunit:cleanup=True 2024-05-12T03:58:33.192 DEBUG:teuthology.orchestra.run.smithi032:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-12T03:58:33.217 INFO:teuthology.orchestra.run.smithi032.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-05-12T03:58:33.217 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-05-12T03:58:33.217 INFO:teuthology.orchestra.run.smithi032.stdout:Device: a1h/161d Inode: 1 Links: 2 2024-05-12T03:58:33.217 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-12T03:58:33.218 INFO:teuthology.orchestra.run.smithi032.stdout:Context: system_u:object_r:fusefs_t:s0 2024-05-12T03:58:33.218 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-05-12T03:58:33.218 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2024-05-12 03:57:52.470002872 +0000 2024-05-12T03:58:33.218 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2024-05-12 03:58:33.037399103 +0000 2024-05-12T03:58:33.218 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: - 2024-05-12T03:58:33.218 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-05-12T03:58:33.218 DEBUG:teuthology.orchestra.run.smithi032:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-05-12T03:58:33.284 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:33 smithi032 ceph-mon[40008]: pgmap v210: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-05-12T03:58:33.294 DEBUG:teuthology.orchestra.run.smithi033:> stat -- /home/ubuntu/cephtest/mnt.1 2024-05-12T03:58:33.314 INFO:teuthology.orchestra.run.smithi033.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-05-12T03:58:33.314 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-05-12T03:58:33.314 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 7dh/125d Inode: 1 Links: 3 2024-05-12T03:58:33.314 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-12T03:58:33.314 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:fusefs_t:s0 2024-05-12T03:58:33.314 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-05-12T03:58:33.315 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-05-12 03:58:33.288512695 +0000 2024-05-12T03:58:33.315 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-05-12 03:58:33.288512695 +0000 2024-05-12T03:58:33.315 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: - 2024-05-12T03:58:33.315 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-05-12T03:58:33.315 DEBUG:teuthology.orchestra.run.smithi033:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-05-12T03:58:33.380 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:33 smithi033 ceph-mon[49525]: pgmap v210: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-05-12T03:58:33.389 DEBUG:teuthology.orchestra.run.smithi032:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout fccfb453906af8847786c471919d347112ea0360 2024-05-12T03:58:33.389 DEBUG:teuthology.orchestra.run.smithi033:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout fccfb453906af8847786c471919d347112ea0360 2024-05-12T03:58:33.408 INFO:tasks.workunit.client.0.smithi032.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-12T03:58:33.447 INFO:tasks.workunit.client.1.smithi033.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-05-12T03:58:33.485 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T03:58:35.273 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:35 smithi032 ceph-mon[40008]: pgmap v211: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-05-12T03:58:35.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:35 smithi033 ceph-mon[49525]: pgmap v211: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-05-12T03:58:35.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:35.607+0000 7f50ac14f700 1 -- 172.21.15.32:0/2125110004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50a40fcf60 msgr2=0x7f50a40fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:35.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:35.607+0000 7f50ac14f700 1 --2- 172.21.15.32:0/2125110004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50a40fcf60 0x7f50a40fd3d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5098004940 tx=0x7f5098009e30 comp rx=0 tx=0).stop 2024-05-12T03:58:35.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:35.607+0000 7f50ac14f700 1 -- 172.21.15.32:0/2125110004 shutdown_connections 2024-05-12T03:58:35.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:35.607+0000 7f50ac14f700 1 --2- 172.21.15.32:0/2125110004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50a40fcf60 0x7f50a40fd3d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:35.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:35.607+0000 7f50ac14f700 1 --2- 172.21.15.32:0/2125110004 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50a40fbcb0 0x7f50a40fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:35.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:35.607+0000 7f50ac14f700 1 -- 172.21.15.32:0/2125110004 >> 172.21.15.32:0/2125110004 conn(0x7f50a40f7430 msgr2=0x7f50a40f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:58:35.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:35.607+0000 7f50ac14f700 1 -- 172.21.15.32:0/2125110004 shutdown_connections 2024-05-12T03:58:35.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:35.607+0000 7f50ac14f700 1 -- 172.21.15.32:0/2125110004 wait complete. 2024-05-12T03:58:35.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:35.608+0000 7f50ac14f700 1 Processor -- start 2024-05-12T03:58:35.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:35.608+0000 7f50ac14f700 1 -- start start 2024-05-12T03:58:35.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:35.608+0000 7f50ac14f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50a40fbcb0 0x7f50a40ffd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:35.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:35.608+0000 7f50ac14f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50a40fcf60 0x7f50a40fe2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:35.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:35.608+0000 7f50ac14f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a40fe830 con 0x7f50a40fbcb0 2024-05-12T03:58:35.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:35.608+0000 7f50ac14f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a40fe970 con 0x7f50a40fcf60 2024-05-12T03:58:35.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:35.608+0000 7f50a9eeb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50a40fbcb0 0x7f50a40ffd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:35.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:35.608+0000 7f50a9eeb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50a40fbcb0 0x7f50a40ffd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47908/0 (socket says 172.21.15.32:47908) 2024-05-12T03:58:35.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:35.608+0000 7f50a9eeb700 1 -- 172.21.15.32:0/4210463403 learned_addr learned my addr 172.21.15.32:0/4210463403 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:58:35.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:35.608+0000 7f50a96ea700 1 --2- 172.21.15.32:0/4210463403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50a40fcf60 0x7f50a40fe2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:35.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:35.609+0000 7f50a9eeb700 1 -- 172.21.15.32:0/4210463403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50a40fcf60 msgr2=0x7f50a40fe2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:35.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:35.609+0000 7f50a9eeb700 1 --2- 172.21.15.32:0/4210463403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50a40fcf60 0x7f50a40fe2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:35.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:35.609+0000 7f50a9eeb700 1 -- 172.21.15.32:0/4210463403 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5098005040 con 0x7f50a40fbcb0 2024-05-12T03:58:35.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:35.609+0000 7f50a9eeb700 1 --2- 172.21.15.32:0/4210463403 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50a40fbcb0 0x7f50a40ffd80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f50a0002a50 tx=0x7f50a00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:58:35.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:35.609+0000 7f50967fc700 1 -- 172.21.15.32:0/4210463403 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f50a0007cc0 con 0x7f50a40fbcb0 2024-05-12T03:58:35.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:35.609+0000 7f50967fc700 1 -- 172.21.15.32:0/4210463403 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f50a000f040 con 0x7f50a40fbcb0 2024-05-12T03:58:35.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:35.609+0000 7f50ac14f700 1 -- 172.21.15.32:0/4210463403 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50a40fec50 con 0x7f50a40fbcb0 2024-05-12T03:58:35.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:35.609+0000 7f50967fc700 1 -- 172.21.15.32:0/4210463403 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f50a000d9c0 con 0x7f50a40fbcb0 2024-05-12T03:58:35.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:35.609+0000 7f50ac14f700 1 -- 172.21.15.32:0/4210463403 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50a41159d0 con 0x7f50a40fbcb0 2024-05-12T03:58:35.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:35.610+0000 7f50ac14f700 1 -- 172.21.15.32:0/4210463403 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50a4101af0 con 0x7f50a40fbcb0 2024-05-12T03:58:35.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:35.610+0000 7f50967fc700 1 -- 172.21.15.32:0/4210463403 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f50a000db20 con 0x7f50a40fbcb0 2024-05-12T03:58:35.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:35.611+0000 7f50967fc700 1 --2- 172.21.15.32:0/4210463403 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f50900692a0 0x7f509006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:35.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:35.611+0000 7f50967fc700 1 -- 172.21.15.32:0/4210463403 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f50a0086f10 con 0x7f50a40fbcb0 2024-05-12T03:58:35.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:35.611+0000 7f50a96ea700 1 --2- 172.21.15.32:0/4210463403 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f50900692a0 0x7f509006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:35.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:35.612+0000 7f50a96ea700 1 --2- 172.21.15.32:0/4210463403 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f50900692a0 0x7f509006b750 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f509800b900 tx=0x7f5098009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:58:35.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:35.616+0000 7f50967fc700 1 -- 172.21.15.32:0/4210463403 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f50a0053170 con 0x7f50a40fbcb0 2024-05-12T03:58:35.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:35.809+0000 7f50ac14f700 1 -- 172.21.15.32:0/4210463403 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mgr/orchestrator/fail_fs}] v 0) v1 -- 0x7f50a405f5c0 con 0x7f50a40fbcb0 2024-05-12T03:58:35.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:35.809+0000 7f50967fc700 1 -- 172.21.15.32:0/4210463403 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/orchestrator/fail_fs}]=-22 unrecognized config option 'mgr/orchestrator/fail_fs' v21)=-22 unrecognized config option 'mgr/orchestrator/fail_fs' v21) v1 ==== 179+0+0 (secure 0 0 0) 0x7f50a0056790 con 0x7f50a40fbcb0 2024-05-12T03:58:35.811 INFO:teuthology.orchestra.run.smithi032.stderr:Error EINVAL: unrecognized config option 'mgr/orchestrator/fail_fs' 2024-05-12T03:58:36.735 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-05-12T03:58:36.736 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi032.front.sepia.ceph.com 2024-05-12T03:58:36.736 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -e sha1=58249552a22f437b8cb49ed6b1068e1158bed605 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-05-12T03:58:37.058 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T03:58:37.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:37 smithi033 ceph-mon[49525]: pgmap v212: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 682 B/s wr, 3 op/s 2024-05-12T03:58:37.498 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:37 smithi032 ceph-mon[40008]: pgmap v212: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 682 B/s wr, 3 op/s 2024-05-12T03:58:38.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:38.946+0000 7f163f8be700 1 -- 172.21.15.32:0/3488303769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1638066e90 msgr2=0x7f16380ffa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:38.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:38.946+0000 7f163f8be700 1 --2- 172.21.15.32:0/3488303769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1638066e90 0x7f16380ffa70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1634004940 tx=0x7f1634009e30 comp rx=0 tx=0).stop 2024-05-12T03:58:38.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:38.946+0000 7f163f8be700 1 -- 172.21.15.32:0/3488303769 shutdown_connections 2024-05-12T03:58:38.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:38.946+0000 7f163f8be700 1 --2- 172.21.15.32:0/3488303769 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16380fffb0 0x7f1638102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:38.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:38.946+0000 7f163f8be700 1 --2- 172.21.15.32:0/3488303769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1638066e90 0x7f16380ffa70 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:38.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:38.946+0000 7f163f8be700 1 -- 172.21.15.32:0/3488303769 >> 172.21.15.32:0/3488303769 conn(0x7f16380f7440 msgr2=0x7f16380f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:58:38.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:38.946+0000 7f163f8be700 1 -- 172.21.15.32:0/3488303769 shutdown_connections 2024-05-12T03:58:38.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:38.946+0000 7f163f8be700 1 -- 172.21.15.32:0/3488303769 wait complete. 2024-05-12T03:58:38.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:38.946+0000 7f163f8be700 1 Processor -- start 2024-05-12T03:58:38.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:38.946+0000 7f163f8be700 1 -- start start 2024-05-12T03:58:38.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:38.947+0000 7f163f8be700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1638066e90 0x7f16380fd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:38.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:38.947+0000 7f163f8be700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16380fffb0 0x7f16380fb970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:38.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:38.947+0000 7f163f8be700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16380fbeb0 con 0x7f1638066e90 2024-05-12T03:58:38.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:38.947+0000 7f163f8be700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16380fc020 con 0x7f16380fffb0 2024-05-12T03:58:38.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:38.947+0000 7f163d65a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1638066e90 0x7f16380fd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:38.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:38.947+0000 7f163d65a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1638066e90 0x7f16380fd2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47932/0 (socket says 172.21.15.32:47932) 2024-05-12T03:58:38.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:38.947+0000 7f163d65a700 1 -- 172.21.15.32:0/525169670 learned_addr learned my addr 172.21.15.32:0/525169670 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:58:38.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:38.947+0000 7f163ce59700 1 --2- 172.21.15.32:0/525169670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16380fffb0 0x7f16380fb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:38.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:38.947+0000 7f163d65a700 1 -- 172.21.15.32:0/525169670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16380fffb0 msgr2=0x7f16380fb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:38.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:38.947+0000 7f163d65a700 1 --2- 172.21.15.32:0/525169670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16380fffb0 0x7f16380fb970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:38.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:38.947+0000 7f163d65a700 1 -- 172.21.15.32:0/525169670 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1634005040 con 0x7f1638066e90 2024-05-12T03:58:38.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:38.947+0000 7f163d65a700 1 --2- 172.21.15.32:0/525169670 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1638066e90 0x7f16380fd2c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1634009fd0 tx=0x7f1634009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:58:38.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:38.947+0000 7f162dffb700 1 -- 172.21.15.32:0/525169670 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f163400e040 con 0x7f1638066e90 2024-05-12T03:58:38.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:38.947+0000 7f163f8be700 1 -- 172.21.15.32:0/525169670 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16380fc2a0 con 0x7f1638066e90 2024-05-12T03:58:38.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:38.947+0000 7f162dffb700 1 -- 172.21.15.32:0/525169670 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1634013420 con 0x7f1638066e90 2024-05-12T03:58:38.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:38.947+0000 7f162dffb700 1 -- 172.21.15.32:0/525169670 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f163401c630 con 0x7f1638066e90 2024-05-12T03:58:38.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:38.948+0000 7f163f8be700 1 -- 172.21.15.32:0/525169670 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1638115a30 con 0x7f1638066e90 2024-05-12T03:58:38.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:38.948+0000 7f163f8be700 1 -- 172.21.15.32:0/525169670 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f163804c710 con 0x7f1638066e90 2024-05-12T03:58:38.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:38.949+0000 7f162dffb700 1 -- 172.21.15.32:0/525169670 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f16340139e0 con 0x7f1638066e90 2024-05-12T03:58:38.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:38.950+0000 7f162dffb700 1 --2- 172.21.15.32:0/525169670 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f1624069370 0x7f162406b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:38.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:38.950+0000 7f163ce59700 1 --2- 172.21.15.32:0/525169670 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f1624069370 0x7f162406b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:38.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:38.950+0000 7f162dffb700 1 -- 172.21.15.32:0/525169670 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f1634088410 con 0x7f1638066e90 2024-05-12T03:58:38.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:38.950+0000 7f163ce59700 1 --2- 172.21.15.32:0/525169670 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f1624069370 0x7f162406b820 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1638067350 tx=0x7f16280091a0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:58:38.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:38.954+0000 7f162dffb700 1 -- 172.21.15.32:0/525169670 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f1634058160 con 0x7f1638066e90 2024-05-12T03:58:39.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:39.149+0000 7f163f8be700 1 -- 172.21.15.32:0/525169670 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7f1638105810 con 0x7f1638066e90 2024-05-12T03:58:39.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:39.150+0000 7f162dffb700 1 -- 172.21.15.32:0/525169670 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v21)=0 v21) v1 ==== 155+0+0 (secure 0 0 0) 0x7f1634057cf0 con 0x7f1638066e90 2024-05-12T03:58:39.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:39.152+0000 7f163f8be700 1 -- 172.21.15.32:0/525169670 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f1624069370 msgr2=0x7f162406b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:39.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:39.152+0000 7f163f8be700 1 --2- 172.21.15.32:0/525169670 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f1624069370 0x7f162406b820 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1638067350 tx=0x7f16280091a0 comp rx=0 tx=0).stop 2024-05-12T03:58:39.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:39.152+0000 7f163f8be700 1 -- 172.21.15.32:0/525169670 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1638066e90 msgr2=0x7f16380fd2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:39.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:39.152+0000 7f163f8be700 1 --2- 172.21.15.32:0/525169670 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1638066e90 0x7f16380fd2c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1634009fd0 tx=0x7f1634009e30 comp rx=0 tx=0).stop 2024-05-12T03:58:39.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:39.152+0000 7f163f8be700 1 -- 172.21.15.32:0/525169670 shutdown_connections 2024-05-12T03:58:39.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:39.152+0000 7f163f8be700 1 --2- 172.21.15.32:0/525169670 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f1624069370 0x7f162406b820 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:39.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:39.152+0000 7f163f8be700 1 --2- 172.21.15.32:0/525169670 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1638066e90 0x7f16380fd2c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:39.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:39.152+0000 7f163f8be700 1 --2- 172.21.15.32:0/525169670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16380fffb0 0x7f16380fb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:39.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:39.152+0000 7f163f8be700 1 -- 172.21.15.32:0/525169670 >> 172.21.15.32:0/525169670 conn(0x7f16380f7440 msgr2=0x7f16380f7810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:58:39.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:39.152+0000 7f163f8be700 1 -- 172.21.15.32:0/525169670 shutdown_connections 2024-05-12T03:58:39.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:39.152+0000 7f163f8be700 1 -- 172.21.15.32:0/525169670 wait complete. 2024-05-12T03:58:39.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:39 smithi033 ceph-mon[49525]: pgmap v213: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 682 B/s wr, 2 op/s 2024-05-12T03:58:39.521 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:39 smithi032 ceph-mon[40008]: pgmap v213: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 682 B/s wr, 2 op/s 2024-05-12T03:58:39.965 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -e sha1=58249552a22f437b8cb49ed6b1068e1158bed605 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-05-12T03:58:40.247 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T03:58:40.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:40 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T03:58:40.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:40 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T03:58:41.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:41 smithi033 ceph-mon[49525]: pgmap v214: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 767 B/s wr, 2 op/s 2024-05-12T03:58:41.536 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:41 smithi032 ceph-mon[40008]: pgmap v214: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 767 B/s wr, 2 op/s 2024-05-12T03:58:42.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:42.128+0000 7f47ea1ea700 1 -- 172.21.15.32:0/1103555190 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47e40ffee0 msgr2=0x7f47e4102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:42.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:42.128+0000 7f47ea1ea700 1 --2- 172.21.15.32:0/1103555190 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47e40ffee0 0x7f47e4102300 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f47d4004940 tx=0x7f47d4009e30 comp rx=0 tx=0).stop 2024-05-12T03:58:42.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:42.128+0000 7f47ea1ea700 1 -- 172.21.15.32:0/1103555190 shutdown_connections 2024-05-12T03:58:42.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:42.128+0000 7f47ea1ea700 1 --2- 172.21.15.32:0/1103555190 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47e40ffee0 0x7f47e4102300 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:42.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:42.128+0000 7f47ea1ea700 1 --2- 172.21.15.32:0/1103555190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47e40662c0 0x7f47e40ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:42.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:42.128+0000 7f47ea1ea700 1 -- 172.21.15.32:0/1103555190 >> 172.21.15.32:0/1103555190 conn(0x7f47e40f7430 msgr2=0x7f47e40f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:58:42.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:42.128+0000 7f47ea1ea700 1 -- 172.21.15.32:0/1103555190 shutdown_connections 2024-05-12T03:58:42.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:42.128+0000 7f47ea1ea700 1 -- 172.21.15.32:0/1103555190 wait complete. 2024-05-12T03:58:42.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:42.129+0000 7f47ea1ea700 1 Processor -- start 2024-05-12T03:58:42.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:42.129+0000 7f47ea1ea700 1 -- start start 2024-05-12T03:58:42.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:42.129+0000 7f47ea1ea700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47e40662c0 0x7f47e410d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:42.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:42.129+0000 7f47ea1ea700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47e40ffee0 0x7f47e410dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:42.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:42.129+0000 7f47ea1ea700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e410e550 con 0x7f47e40ffee0 2024-05-12T03:58:42.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:42.129+0000 7f47ea1ea700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e410ed40 con 0x7f47e40662c0 2024-05-12T03:58:42.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:42.130+0000 7f47e2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47e40ffee0 0x7f47e410dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:42.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:42.130+0000 7f47e2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47e40ffee0 0x7f47e410dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34556/0 (socket says 172.21.15.32:34556) 2024-05-12T03:58:42.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:42.130+0000 7f47e2ffd700 1 -- 172.21.15.32:0/1215199320 learned_addr learned my addr 172.21.15.32:0/1215199320 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:58:42.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:42.130+0000 7f47e37fe700 1 --2- 172.21.15.32:0/1215199320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47e40662c0 0x7f47e410d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:42.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:42.130+0000 7f47e2ffd700 1 -- 172.21.15.32:0/1215199320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47e40662c0 msgr2=0x7f47e410d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:42.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:42.130+0000 7f47e2ffd700 1 --2- 172.21.15.32:0/1215199320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47e40662c0 0x7f47e410d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:42.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:42.130+0000 7f47e2ffd700 1 -- 172.21.15.32:0/1215199320 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47d4005040 con 0x7f47e40ffee0 2024-05-12T03:58:42.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:42.130+0000 7f47e2ffd700 1 --2- 172.21.15.32:0/1215199320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47e40ffee0 0x7f47e410dec0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f47d400b900 tx=0x7f47d4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:58:42.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:42.131+0000 7f47cbfff700 1 -- 172.21.15.32:0/1215199320 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47d400e040 con 0x7f47e40ffee0 2024-05-12T03:58:42.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:42.131+0000 7f47ea1ea700 1 -- 172.21.15.32:0/1215199320 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47e410efc0 con 0x7f47e40ffee0 2024-05-12T03:58:42.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:42.131+0000 7f47cbfff700 1 -- 172.21.15.32:0/1215199320 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f47d4004290 con 0x7f47e40ffee0 2024-05-12T03:58:42.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:42.131+0000 7f47cbfff700 1 -- 172.21.15.32:0/1215199320 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47d401c3f0 con 0x7f47e40ffee0 2024-05-12T03:58:42.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:42.131+0000 7f47ea1ea700 1 -- 172.21.15.32:0/1215199320 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47e410f580 con 0x7f47e40ffee0 2024-05-12T03:58:42.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:42.132+0000 7f47ea1ea700 1 -- 172.21.15.32:0/1215199320 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47e405f5c0 con 0x7f47e40ffee0 2024-05-12T03:58:42.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:42.132+0000 7f47cbfff700 1 -- 172.21.15.32:0/1215199320 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f47d401c6b0 con 0x7f47e40ffee0 2024-05-12T03:58:42.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:42.133+0000 7f47cbfff700 1 --2- 172.21.15.32:0/1215199320 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f47cc0692b0 0x7f47cc06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:42.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:42.133+0000 7f47cbfff700 1 -- 172.21.15.32:0/1215199320 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f47d40884c0 con 0x7f47e40ffee0 2024-05-12T03:58:42.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:42.133+0000 7f47e37fe700 1 --2- 172.21.15.32:0/1215199320 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f47cc0692b0 0x7f47cc06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:42.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:42.133+0000 7f47e37fe700 1 --2- 172.21.15.32:0/1215199320 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f47cc0692b0 0x7f47cc06b760 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f47e4066000 tx=0x7f47d80091a0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:58:42.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:42.137+0000 7f47cbfff700 1 -- 172.21.15.32:0/1215199320 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f47d4058230 con 0x7f47e40ffee0 2024-05-12T03:58:42.329 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:42.329+0000 7f47ea1ea700 1 -- 172.21.15.32:0/1215199320 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7f47e404bed0 con 0x7f47e40ffee0 2024-05-12T03:58:42.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:42.330+0000 7f47cbfff700 1 -- 172.21.15.32:0/1215199320 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v21)=0 v21) v1 ==== 163+0+0 (secure 0 0 0) 0x7f47d4057dc0 con 0x7f47e40ffee0 2024-05-12T03:58:42.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:42.332+0000 7f47ea1ea700 1 -- 172.21.15.32:0/1215199320 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f47cc0692b0 msgr2=0x7f47cc06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:42.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:42.332+0000 7f47ea1ea700 1 --2- 172.21.15.32:0/1215199320 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f47cc0692b0 0x7f47cc06b760 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f47e4066000 tx=0x7f47d80091a0 comp rx=0 tx=0).stop 2024-05-12T03:58:42.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:42.332+0000 7f47ea1ea700 1 -- 172.21.15.32:0/1215199320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47e40ffee0 msgr2=0x7f47e410dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:42.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:42.332+0000 7f47ea1ea700 1 --2- 172.21.15.32:0/1215199320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47e40ffee0 0x7f47e410dec0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f47d400b900 tx=0x7f47d4009e30 comp rx=0 tx=0).stop 2024-05-12T03:58:42.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:42.332+0000 7f47ea1ea700 1 -- 172.21.15.32:0/1215199320 shutdown_connections 2024-05-12T03:58:42.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:42.332+0000 7f47ea1ea700 1 --2- 172.21.15.32:0/1215199320 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f47cc0692b0 0x7f47cc06b760 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:42.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:42.332+0000 7f47ea1ea700 1 --2- 172.21.15.32:0/1215199320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47e40662c0 0x7f47e410d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:42.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:42.332+0000 7f47ea1ea700 1 --2- 172.21.15.32:0/1215199320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47e40ffee0 0x7f47e410dec0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:42.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:42.332+0000 7f47ea1ea700 1 -- 172.21.15.32:0/1215199320 >> 172.21.15.32:0/1215199320 conn(0x7f47e40f7430 msgr2=0x7f47e40f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:58:42.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:42.332+0000 7f47ea1ea700 1 -- 172.21.15.32:0/1215199320 shutdown_connections 2024-05-12T03:58:42.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:42.332+0000 7f47ea1ea700 1 -- 172.21.15.32:0/1215199320 wait complete. 2024-05-12T03:58:43.116 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -e sha1=58249552a22f437b8cb49ed6b1068e1158bed605 -- bash -c 'ceph config set global log_to_journald false --force' 2024-05-12T03:58:43.399 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T03:58:43.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:43 smithi033 ceph-mon[49525]: pgmap v215: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 938 B/s wr, 3 op/s 2024-05-12T03:58:43.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:43 smithi032 ceph-mon[40008]: pgmap v215: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 938 B/s wr, 3 op/s 2024-05-12T03:58:45.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:45.309+0000 7f4087c19700 1 -- 172.21.15.32:0/2175317985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40800fcf60 msgr2=0x7f40800fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:45.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:45.309+0000 7f4087c19700 1 --2- 172.21.15.32:0/2175317985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40800fcf60 0x7f40800fd3d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f407c004940 tx=0x7f407c009e30 comp rx=0 tx=0).stop 2024-05-12T03:58:45.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:45.309+0000 7f4087c19700 1 -- 172.21.15.32:0/2175317985 shutdown_connections 2024-05-12T03:58:45.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:45.309+0000 7f4087c19700 1 --2- 172.21.15.32:0/2175317985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40800fcf60 0x7f40800fd3d0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:45.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:45.309+0000 7f4087c19700 1 --2- 172.21.15.32:0/2175317985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40800fbcb0 0x7f40800fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:45.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:45.309+0000 7f4087c19700 1 -- 172.21.15.32:0/2175317985 >> 172.21.15.32:0/2175317985 conn(0x7f40800f7430 msgr2=0x7f40800f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:58:45.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:45.309+0000 7f4087c19700 1 -- 172.21.15.32:0/2175317985 shutdown_connections 2024-05-12T03:58:45.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:45.309+0000 7f4087c19700 1 -- 172.21.15.32:0/2175317985 wait complete. 2024-05-12T03:58:45.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:45.310+0000 7f4087c19700 1 Processor -- start 2024-05-12T03:58:45.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:45.310+0000 7f4087c19700 1 -- start start 2024-05-12T03:58:45.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:45.310+0000 7f4087c19700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40800fbcb0 0x7f40800661b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:45.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:45.310+0000 7f4087c19700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40800fcf60 0x7f4080112620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:45.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:45.310+0000 7f4087c19700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4080112ce0 con 0x7f40800fbcb0 2024-05-12T03:58:45.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:45.310+0000 7f4087c19700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4080112e50 con 0x7f40800fcf60 2024-05-12T03:58:45.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:45.310+0000 7f40859b5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40800fbcb0 0x7f40800661b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:45.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:45.310+0000 7f40859b5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40800fbcb0 0x7f40800661b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34576/0 (socket says 172.21.15.32:34576) 2024-05-12T03:58:45.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:45.310+0000 7f40859b5700 1 -- 172.21.15.32:0/3052357701 learned_addr learned my addr 172.21.15.32:0/3052357701 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:58:45.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:45.310+0000 7f40851b4700 1 --2- 172.21.15.32:0/3052357701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40800fcf60 0x7f4080112620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:45.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:45.310+0000 7f40859b5700 1 -- 172.21.15.32:0/3052357701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40800fcf60 msgr2=0x7f4080112620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:45.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:45.310+0000 7f40859b5700 1 --2- 172.21.15.32:0/3052357701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40800fcf60 0x7f4080112620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:45.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:45.310+0000 7f40859b5700 1 -- 172.21.15.32:0/3052357701 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f407c005040 con 0x7f40800fbcb0 2024-05-12T03:58:45.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:45.311+0000 7f40851b4700 1 --2- 172.21.15.32:0/3052357701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40800fcf60 0x7f4080112620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-12T03:58:45.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:45.311+0000 7f40859b5700 1 --2- 172.21.15.32:0/3052357701 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40800fbcb0 0x7f40800661b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f407000b790 tx=0x7f4070009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:58:45.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:45.311+0000 7f40767fc700 1 -- 172.21.15.32:0/3052357701 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f407000f770 con 0x7f40800fbcb0 2024-05-12T03:58:45.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:45.311+0000 7f40767fc700 1 -- 172.21.15.32:0/3052357701 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f407000fd80 con 0x7f40800fbcb0 2024-05-12T03:58:45.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:45.311+0000 7f4087c19700 1 -- 172.21.15.32:0/3052357701 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4080113130 con 0x7f40800fbcb0 2024-05-12T03:58:45.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:45.311+0000 7f40767fc700 1 -- 172.21.15.32:0/3052357701 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f407000d310 con 0x7f40800fbcb0 2024-05-12T03:58:45.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:45.311+0000 7f4087c19700 1 -- 172.21.15.32:0/3052357701 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40801136f0 con 0x7f40800fbcb0 2024-05-12T03:58:45.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:45.312+0000 7f4087c19700 1 -- 172.21.15.32:0/3052357701 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f408005f5c0 con 0x7f40800fbcb0 2024-05-12T03:58:45.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:45.312+0000 7f40767fc700 1 -- 172.21.15.32:0/3052357701 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f4070010420 con 0x7f40800fbcb0 2024-05-12T03:58:45.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:45.313+0000 7f40767fc700 1 --2- 172.21.15.32:0/3052357701 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f406c0692a0 0x7f406c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:45.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:45.313+0000 7f40767fc700 1 -- 172.21.15.32:0/3052357701 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f4070087090 con 0x7f40800fbcb0 2024-05-12T03:58:45.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:45.313+0000 7f40851b4700 1 --2- 172.21.15.32:0/3052357701 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f406c0692a0 0x7f406c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:45.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:45.313+0000 7f40851b4700 1 --2- 172.21.15.32:0/3052357701 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f406c0692a0 0x7f406c06b750 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f407c00b3b0 tx=0x7f407c009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:58:45.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:45.317+0000 7f40767fc700 1 -- 172.21.15.32:0/3052357701 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f4070053410 con 0x7f40800fbcb0 2024-05-12T03:58:45.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:45 smithi033 ceph-mon[49525]: pgmap v216: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 853 B/s wr, 2 op/s 2024-05-12T03:58:45.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:45.512+0000 7f4087c19700 1 -- 172.21.15.32:0/3052357701 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7f408004bed0 con 0x7f40800fbcb0 2024-05-12T03:58:45.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:45.513+0000 7f40767fc700 1 -- 172.21.15.32:0/3052357701 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v21)=0 v21) v1 ==== 135+0+0 (secure 0 0 0) 0x7f4070056a30 con 0x7f40800fbcb0 2024-05-12T03:58:45.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:45.514+0000 7f4087c19700 1 -- 172.21.15.32:0/3052357701 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f406c0692a0 msgr2=0x7f406c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:45.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:45.514+0000 7f4087c19700 1 --2- 172.21.15.32:0/3052357701 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f406c0692a0 0x7f406c06b750 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f407c00b3b0 tx=0x7f407c009e30 comp rx=0 tx=0).stop 2024-05-12T03:58:45.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:45.514+0000 7f4087c19700 1 -- 172.21.15.32:0/3052357701 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40800fbcb0 msgr2=0x7f40800661b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:45.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:45.515+0000 7f4087c19700 1 --2- 172.21.15.32:0/3052357701 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40800fbcb0 0x7f40800661b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f407000b790 tx=0x7f4070009250 comp rx=0 tx=0).stop 2024-05-12T03:58:45.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:45.515+0000 7f4087c19700 1 -- 172.21.15.32:0/3052357701 shutdown_connections 2024-05-12T03:58:45.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:45.515+0000 7f4087c19700 1 --2- 172.21.15.32:0/3052357701 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f406c0692a0 0x7f406c06b750 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:45.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:45.515+0000 7f4087c19700 1 --2- 172.21.15.32:0/3052357701 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40800fbcb0 0x7f40800661b0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:45.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:45.515+0000 7f4087c19700 1 --2- 172.21.15.32:0/3052357701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40800fcf60 0x7f4080112620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:45.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:45.515+0000 7f4087c19700 1 -- 172.21.15.32:0/3052357701 >> 172.21.15.32:0/3052357701 conn(0x7f40800f7430 msgr2=0x7f40800f8b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:58:45.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:45.515+0000 7f4087c19700 1 -- 172.21.15.32:0/3052357701 shutdown_connections 2024-05-12T03:58:45.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:45.515+0000 7f4087c19700 1 -- 172.21.15.32:0/3052357701 wait complete. 2024-05-12T03:58:45.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:45 smithi032 ceph-mon[40008]: pgmap v216: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 853 B/s wr, 2 op/s 2024-05-12T03:58:46.386 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -e sha1=58249552a22f437b8cb49ed6b1068e1158bed605 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-05-12T03:58:46.666 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T03:58:47.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:47 smithi033 ceph-mon[49525]: pgmap v217: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-05-12T03:58:47.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:47 smithi032 ceph-mon[40008]: pgmap v217: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-05-12T03:58:48.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:48.522+0000 7f7c5ff59700 1 -- 172.21.15.32:0/548907143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c58066260 msgr2=0x7f7c580ffa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:48.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:48.522+0000 7f7c5ff59700 1 --2- 172.21.15.32:0/548907143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c58066260 0x7f7c580ffa00 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7c48004940 tx=0x7f7c48009e30 comp rx=0 tx=0).stop 2024-05-12T03:58:48.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:48.522+0000 7f7c5ff59700 1 -- 172.21.15.32:0/548907143 shutdown_connections 2024-05-12T03:58:48.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:48.522+0000 7f7c5ff59700 1 --2- 172.21.15.32:0/548907143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c580fff40 0x7f7c58102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:48.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:48.522+0000 7f7c5ff59700 1 --2- 172.21.15.32:0/548907143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c58066260 0x7f7c580ffa00 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:48.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:48.522+0000 7f7c5ff59700 1 -- 172.21.15.32:0/548907143 >> 172.21.15.32:0/548907143 conn(0x7f7c580f7430 msgr2=0x7f7c580f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:58:48.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:48.522+0000 7f7c5ff59700 1 -- 172.21.15.32:0/548907143 shutdown_connections 2024-05-12T03:58:48.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:48.522+0000 7f7c5ff59700 1 -- 172.21.15.32:0/548907143 wait complete. 2024-05-12T03:58:48.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:48.523+0000 7f7c5ff59700 1 Processor -- start 2024-05-12T03:58:48.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:48.523+0000 7f7c5ff59700 1 -- start start 2024-05-12T03:58:48.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:48.523+0000 7f7c5ff59700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c58066260 0x7f7c5810d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:48.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:48.523+0000 7f7c5ff59700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c580fff40 0x7f7c5810df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:48.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:48.523+0000 7f7c5ff59700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5810e530 con 0x7f7c58066260 2024-05-12T03:58:48.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:48.523+0000 7f7c5ff59700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5810ed30 con 0x7f7c580fff40 2024-05-12T03:58:48.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:48.523+0000 7f7c5dcf5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c58066260 0x7f7c5810d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:48.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:48.523+0000 7f7c5dcf5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c58066260 0x7f7c5810d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34584/0 (socket says 172.21.15.32:34584) 2024-05-12T03:58:48.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:48.523+0000 7f7c5dcf5700 1 -- 172.21.15.32:0/3520046283 learned_addr learned my addr 172.21.15.32:0/3520046283 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:58:48.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:48.523+0000 7f7c5d4f4700 1 --2- 172.21.15.32:0/3520046283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c580fff40 0x7f7c5810df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:48.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:48.524+0000 7f7c5dcf5700 1 -- 172.21.15.32:0/3520046283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c580fff40 msgr2=0x7f7c5810df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:48.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:48.524+0000 7f7c5dcf5700 1 --2- 172.21.15.32:0/3520046283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c580fff40 0x7f7c5810df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:48.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:48.524+0000 7f7c5dcf5700 1 -- 172.21.15.32:0/3520046283 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c48005040 con 0x7f7c58066260 2024-05-12T03:58:48.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:48.524+0000 7f7c5dcf5700 1 --2- 172.21.15.32:0/3520046283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c58066260 0x7f7c5810d9f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7c4800b3b0 tx=0x7f7c48009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:58:48.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:48.524+0000 7f7c4e7fc700 1 -- 172.21.15.32:0/3520046283 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c4800e040 con 0x7f7c58066260 2024-05-12T03:58:48.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:48.524+0000 7f7c4e7fc700 1 -- 172.21.15.32:0/3520046283 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7c4800bc50 con 0x7f7c58066260 2024-05-12T03:58:48.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:48.524+0000 7f7c5ff59700 1 -- 172.21.15.32:0/3520046283 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c5810efb0 con 0x7f7c58066260 2024-05-12T03:58:48.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:48.524+0000 7f7c4e7fc700 1 -- 172.21.15.32:0/3520046283 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c4801c3f0 con 0x7f7c58066260 2024-05-12T03:58:48.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:48.524+0000 7f7c5ff59700 1 -- 172.21.15.32:0/3520046283 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c5810f4f0 con 0x7f7c58066260 2024-05-12T03:58:48.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:48.525+0000 7f7c5ff59700 1 -- 172.21.15.32:0/3520046283 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c5805f5c0 con 0x7f7c58066260 2024-05-12T03:58:48.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:48.525+0000 7f7c4e7fc700 1 -- 172.21.15.32:0/3520046283 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f7c48013420 con 0x7f7c58066260 2024-05-12T03:58:48.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:48.526+0000 7f7c4e7fc700 1 --2- 172.21.15.32:0/3520046283 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f7c440692a0 0x7f7c4406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:48.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:48.526+0000 7f7c4e7fc700 1 -- 172.21.15.32:0/3520046283 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f7c48087ec0 con 0x7f7c58066260 2024-05-12T03:58:48.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:48.526+0000 7f7c5d4f4700 1 --2- 172.21.15.32:0/3520046283 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f7c440692a0 0x7f7c4406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:48.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:48.526+0000 7f7c5d4f4700 1 --2- 172.21.15.32:0/3520046283 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f7c440692a0 0x7f7c4406b750 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7c540041d0 tx=0x7f7c540043c0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:58:48.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:48.530+0000 7f7c4e7fc700 1 -- 172.21.15.32:0/3520046283 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f7c48057c30 con 0x7f7c58066260 2024-05-12T03:58:48.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:48.745+0000 7f7c5ff59700 1 -- 172.21.15.32:0/3520046283 --> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:58249552a22f437b8cb49ed6b1068e1158bed605", "target": ["mon-mgr", ""]}) v1 -- 0x7f7c5810f870 con 0x7f7c440692a0 2024-05-12T03:58:48.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:48.750+0000 7f7c4e7fc700 1 -- 172.21.15.32:0/3520046283 <== mgr.14208 v2:172.21.15.32:6800/4089797100 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7f7c5810f870 con 0x7f7c440692a0 2024-05-12T03:58:48.751 INFO:teuthology.orchestra.run.smithi032.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:58249552a22f437b8cb49ed6b1068e1158bed605 2024-05-12T03:58:48.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:48.753+0000 7f7c5ff59700 1 -- 172.21.15.32:0/3520046283 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f7c440692a0 msgr2=0x7f7c4406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:48.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:48.753+0000 7f7c5ff59700 1 --2- 172.21.15.32:0/3520046283 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f7c440692a0 0x7f7c4406b750 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7c540041d0 tx=0x7f7c540043c0 comp rx=0 tx=0).stop 2024-05-12T03:58:48.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:48.753+0000 7f7c5ff59700 1 -- 172.21.15.32:0/3520046283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c58066260 msgr2=0x7f7c5810d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:48.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:48.753+0000 7f7c5ff59700 1 --2- 172.21.15.32:0/3520046283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c58066260 0x7f7c5810d9f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7c4800b3b0 tx=0x7f7c48009e30 comp rx=0 tx=0).stop 2024-05-12T03:58:48.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:48.753+0000 7f7c5ff59700 1 -- 172.21.15.32:0/3520046283 shutdown_connections 2024-05-12T03:58:48.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:48.753+0000 7f7c5ff59700 1 --2- 172.21.15.32:0/3520046283 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f7c440692a0 0x7f7c4406b750 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:48.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:48.753+0000 7f7c5ff59700 1 --2- 172.21.15.32:0/3520046283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c58066260 0x7f7c5810d9f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:48.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:48.753+0000 7f7c5ff59700 1 --2- 172.21.15.32:0/3520046283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c580fff40 0x7f7c5810df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:48.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:48.753+0000 7f7c5ff59700 1 -- 172.21.15.32:0/3520046283 >> 172.21.15.32:0/3520046283 conn(0x7f7c580f7430 msgr2=0x7f7c580f9810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:58:48.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:48.753+0000 7f7c5ff59700 1 -- 172.21.15.32:0/3520046283 shutdown_connections 2024-05-12T03:58:48.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:48.753+0000 7f7c5ff59700 1 -- 172.21.15.32:0/3520046283 wait complete. 2024-05-12T03:58:49.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:49 smithi033 ceph-mon[49525]: pgmap v218: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 597 B/s wr, 2 op/s 2024-05-12T03:58:49.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:49 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:58:49.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:49 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T03:58:49.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:49 smithi032 ceph-mon[40008]: pgmap v218: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 597 B/s wr, 2 op/s 2024-05-12T03:58:49.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:49 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:58:49.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:49 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T03:58:49.620 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-05-12T03:58:49.621 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi032.front.sepia.ceph.com 2024-05-12T03:58:49.622 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -e sha1=58249552a22f437b8cb49ed6b1068e1158bed605 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done' 2024-05-12T03:58:49.905 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T03:58:50.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:50 smithi033 ceph-mon[49525]: from='client.14558 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:58249552a22f437b8cb49ed6b1068e1158bed605", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T03:58:50.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:50 smithi033 ceph-mon[49525]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:58249552a22f437b8cb49ed6b1068e1158bed605 2024-05-12T03:58:50.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:50 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:58:50.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:50 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T03:58:50.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:50 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:58:50.538 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:50 smithi032 ceph-mon[40008]: from='client.14558 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:58249552a22f437b8cb49ed6b1068e1158bed605", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T03:58:50.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:50 smithi032 ceph-mon[40008]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:58249552a22f437b8cb49ed6b1068e1158bed605 2024-05-12T03:58:50.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:50 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:58:50.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:50 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T03:58:50.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:50 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:58:51.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:51 smithi033 ceph-mon[49525]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:58249552a22f437b8cb49ed6b1068e1158bed605 2024-05-12T03:58:51.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:51 smithi033 ceph-mon[49525]: pgmap v219: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 682 B/s wr, 2 op/s 2024-05-12T03:58:51.454 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:51 smithi032 ceph-mon[40008]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:58249552a22f437b8cb49ed6b1068e1158bed605 2024-05-12T03:58:51.455 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:51 smithi032 ceph-mon[40008]: pgmap v219: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 682 B/s wr, 2 op/s 2024-05-12T03:58:52.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.036+0000 7fd8e81cb700 1 -- 172.21.15.32:0/4050852347 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8e00662c0 msgr2=0x7fd8e00ff9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:52.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.036+0000 7fd8e81cb700 1 --2- 172.21.15.32:0/4050852347 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8e00662c0 0x7fd8e00ff9a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc004940 tx=0x7fd8dc009e30 comp rx=0 tx=0).stop 2024-05-12T03:58:52.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.037+0000 7fd8e81cb700 1 -- 172.21.15.32:0/4050852347 shutdown_connections 2024-05-12T03:58:52.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.037+0000 7fd8e81cb700 1 --2- 172.21.15.32:0/4050852347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8e00ffee0 0x7fd8e0102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:52.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.037+0000 7fd8e81cb700 1 --2- 172.21.15.32:0/4050852347 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8e00662c0 0x7fd8e00ff9a0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:52.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.037+0000 7fd8e81cb700 1 -- 172.21.15.32:0/4050852347 >> 172.21.15.32:0/4050852347 conn(0x7fd8e00f7430 msgr2=0x7fd8e00f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:58:52.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.037+0000 7fd8e81cb700 1 -- 172.21.15.32:0/4050852347 shutdown_connections 2024-05-12T03:58:52.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.037+0000 7fd8e81cb700 1 -- 172.21.15.32:0/4050852347 wait complete. 2024-05-12T03:58:52.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.037+0000 7fd8e81cb700 1 Processor -- start 2024-05-12T03:58:52.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.037+0000 7fd8e81cb700 1 -- start start 2024-05-12T03:58:52.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.037+0000 7fd8e81cb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8e00662c0 0x7fd8e00fd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:52.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.037+0000 7fd8e81cb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8e00ffee0 0x7fd8e00fb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:52.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.038+0000 7fd8e81cb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e00fbe50 con 0x7fd8e00662c0 2024-05-12T03:58:52.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.038+0000 7fd8e81cb700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e00fbfc0 con 0x7fd8e00ffee0 2024-05-12T03:58:52.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.038+0000 7fd8e5f67700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8e00662c0 0x7fd8e00fd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:52.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.038+0000 7fd8e5f67700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8e00662c0 0x7fd8e00fd3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43644/0 (socket says 172.21.15.32:43644) 2024-05-12T03:58:52.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.038+0000 7fd8e5f67700 1 -- 172.21.15.32:0/3286088681 learned_addr learned my addr 172.21.15.32:0/3286088681 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:58:52.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.038+0000 7fd8e5766700 1 --2- 172.21.15.32:0/3286088681 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8e00ffee0 0x7fd8e00fb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:52.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.038+0000 7fd8e5f67700 1 -- 172.21.15.32:0/3286088681 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8e00ffee0 msgr2=0x7fd8e00fb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:52.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.038+0000 7fd8e5f67700 1 --2- 172.21.15.32:0/3286088681 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8e00ffee0 0x7fd8e00fb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:52.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.038+0000 7fd8e5f67700 1 -- 172.21.15.32:0/3286088681 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8dc005040 con 0x7fd8e00662c0 2024-05-12T03:58:52.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.038+0000 7fd8e5f67700 1 --2- 172.21.15.32:0/3286088681 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8e00662c0 0x7fd8e00fd3d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc000c00 tx=0x7fd8dc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:58:52.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.038+0000 7fd8d67fc700 1 -- 172.21.15.32:0/3286088681 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd8dc00e070 con 0x7fd8e00662c0 2024-05-12T03:58:52.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.038+0000 7fd8d67fc700 1 -- 172.21.15.32:0/3286088681 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd8dc0093f0 con 0x7fd8e00662c0 2024-05-12T03:58:52.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.038+0000 7fd8d67fc700 1 -- 172.21.15.32:0/3286088681 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd8dc00b410 con 0x7fd8e00662c0 2024-05-12T03:58:52.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.038+0000 7fd8e81cb700 1 -- 172.21.15.32:0/3286088681 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8e00fc240 con 0x7fd8e00662c0 2024-05-12T03:58:52.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.038+0000 7fd8e81cb700 1 -- 172.21.15.32:0/3286088681 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8e01158e0 con 0x7fd8e00662c0 2024-05-12T03:58:52.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.039+0000 7fd8e81cb700 1 -- 172.21.15.32:0/3286088681 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8e005f5c0 con 0x7fd8e00662c0 2024-05-12T03:58:52.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.040+0000 7fd8d67fc700 1 -- 172.21.15.32:0/3286088681 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fd8dc01c400 con 0x7fd8e00662c0 2024-05-12T03:58:52.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.040+0000 7fd8d67fc700 1 --2- 172.21.15.32:0/3286088681 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fd8cc0692a0 0x7fd8cc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:52.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.041+0000 7fd8d67fc700 1 -- 172.21.15.32:0/3286088681 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fd8dc0887d0 con 0x7fd8e00662c0 2024-05-12T03:58:52.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.041+0000 7fd8e5766700 1 --2- 172.21.15.32:0/3286088681 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fd8cc0692a0 0x7fd8cc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:52.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.041+0000 7fd8e5766700 1 --2- 172.21.15.32:0/3286088681 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fd8cc0692a0 0x7fd8cc06b750 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd8d0009fd0 tx=0x7fd8d000b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:58:52.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.045+0000 7fd8d67fc700 1 -- 172.21.15.32:0/3286088681 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd8dc054b50 con 0x7fd8e00662c0 2024-05-12T03:58:52.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.277+0000 7fd8e81cb700 1 -- 172.21.15.32:0/3286088681 --> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd8e00f8f20 con 0x7fd8cc0692a0 2024-05-12T03:58:52.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.278+0000 7fd8d67fc700 1 -- 172.21.15.32:0/3286088681 <== mgr.14208 v2:172.21.15.32:6800/4089797100 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7fd8e00f8f20 con 0x7fd8cc0692a0 2024-05-12T03:58:52.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.280+0000 7fd8e81cb700 1 -- 172.21.15.32:0/3286088681 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fd8cc0692a0 msgr2=0x7fd8cc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:52.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.280+0000 7fd8e81cb700 1 --2- 172.21.15.32:0/3286088681 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fd8cc0692a0 0x7fd8cc06b750 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd8d0009fd0 tx=0x7fd8d000b040 comp rx=0 tx=0).stop 2024-05-12T03:58:52.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.280+0000 7fd8e81cb700 1 -- 172.21.15.32:0/3286088681 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8e00662c0 msgr2=0x7fd8e00fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:52.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.280+0000 7fd8e81cb700 1 --2- 172.21.15.32:0/3286088681 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8e00662c0 0x7fd8e00fd3d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc000c00 tx=0x7fd8dc009e30 comp rx=0 tx=0).stop 2024-05-12T03:58:52.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.280+0000 7fd8e81cb700 1 -- 172.21.15.32:0/3286088681 shutdown_connections 2024-05-12T03:58:52.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.280+0000 7fd8e81cb700 1 --2- 172.21.15.32:0/3286088681 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fd8cc0692a0 0x7fd8cc06b750 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:52.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.280+0000 7fd8e81cb700 1 --2- 172.21.15.32:0/3286088681 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8e00662c0 0x7fd8e00fd3d0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:52.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.280+0000 7fd8e81cb700 1 --2- 172.21.15.32:0/3286088681 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8e00ffee0 0x7fd8e00fb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:52.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.280+0000 7fd8e81cb700 1 -- 172.21.15.32:0/3286088681 >> 172.21.15.32:0/3286088681 conn(0x7fd8e00f7430 msgr2=0x7fd8e00f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:58:52.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.280+0000 7fd8e81cb700 1 -- 172.21.15.32:0/3286088681 shutdown_connections 2024-05-12T03:58:52.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.280+0000 7fd8e81cb700 1 -- 172.21.15.32:0/3286088681 wait complete. 2024-05-12T03:58:52.291 INFO:teuthology.orchestra.run.smithi032.stdout:true 2024-05-12T03:58:52.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.395+0000 7fc6b79c6700 1 -- 172.21.15.32:0/2114647229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6b00ffee0 msgr2=0x7fc6b0102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:52.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.395+0000 7fc6b79c6700 1 --2- 172.21.15.32:0/2114647229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6b00ffee0 0x7fc6b0102300 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0004940 tx=0x7fc6a0009e30 comp rx=0 tx=0).stop 2024-05-12T03:58:52.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.395+0000 7fc6b79c6700 1 -- 172.21.15.32:0/2114647229 shutdown_connections 2024-05-12T03:58:52.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.395+0000 7fc6b79c6700 1 --2- 172.21.15.32:0/2114647229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6b00ffee0 0x7fc6b0102300 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:52.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.395+0000 7fc6b79c6700 1 --2- 172.21.15.32:0/2114647229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6b00662c0 0x7fc6b00ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:52.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.395+0000 7fc6b79c6700 1 -- 172.21.15.32:0/2114647229 >> 172.21.15.32:0/2114647229 conn(0x7fc6b00f7430 msgr2=0x7fc6b00f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:58:52.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.395+0000 7fc6b79c6700 1 -- 172.21.15.32:0/2114647229 shutdown_connections 2024-05-12T03:58:52.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.395+0000 7fc6b79c6700 1 -- 172.21.15.32:0/2114647229 wait complete. 2024-05-12T03:58:52.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.396+0000 7fc6b79c6700 1 Processor -- start 2024-05-12T03:58:52.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.396+0000 7fc6b79c6700 1 -- start start 2024-05-12T03:58:52.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.396+0000 7fc6b79c6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6b00662c0 0x7fc6b00fd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:52.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.396+0000 7fc6b79c6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6b00ffee0 0x7fc6b00fb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:52.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.396+0000 7fc6b79c6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b00fbde0 con 0x7fc6b00ffee0 2024-05-12T03:58:52.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.396+0000 7fc6b79c6700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b00fbf20 con 0x7fc6b00662c0 2024-05-12T03:58:52.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.396+0000 7fc6b4f61700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6b00ffee0 0x7fc6b00fb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:52.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.396+0000 7fc6b4f61700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6b00ffee0 0x7fc6b00fb8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43654/0 (socket says 172.21.15.32:43654) 2024-05-12T03:58:52.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.396+0000 7fc6b4f61700 1 -- 172.21.15.32:0/2713876617 learned_addr learned my addr 172.21.15.32:0/2713876617 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:58:52.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.396+0000 7fc6b5762700 1 --2- 172.21.15.32:0/2713876617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6b00662c0 0x7fc6b00fd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:52.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.396+0000 7fc6b4f61700 1 -- 172.21.15.32:0/2713876617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6b00662c0 msgr2=0x7fc6b00fd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:52.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.396+0000 7fc6b4f61700 1 --2- 172.21.15.32:0/2713876617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6b00662c0 0x7fc6b00fd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:52.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.396+0000 7fc6b4f61700 1 -- 172.21.15.32:0/2713876617 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6a0005040 con 0x7fc6b00ffee0 2024-05-12T03:58:52.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.396+0000 7fc6b4f61700 1 --2- 172.21.15.32:0/2713876617 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6b00ffee0 0x7fc6b00fb8a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0009fd0 tx=0x7fc6a0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:58:52.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.397+0000 7fc6a5ffb700 1 -- 172.21.15.32:0/2713876617 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc6a000e040 con 0x7fc6b00ffee0 2024-05-12T03:58:52.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.397+0000 7fc6a5ffb700 1 -- 172.21.15.32:0/2713876617 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc6a0004290 con 0x7fc6b00ffee0 2024-05-12T03:58:52.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.397+0000 7fc6a5ffb700 1 -- 172.21.15.32:0/2713876617 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc6a001c3f0 con 0x7fc6b00ffee0 2024-05-12T03:58:52.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.397+0000 7fc6b79c6700 1 -- 172.21.15.32:0/2713876617 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6b00fc1a0 con 0x7fc6b00ffee0 2024-05-12T03:58:52.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.397+0000 7fc6b79c6700 1 -- 172.21.15.32:0/2713876617 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6b0115a10 con 0x7fc6b00ffee0 2024-05-12T03:58:52.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.398+0000 7fc6b79c6700 1 -- 172.21.15.32:0/2713876617 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6b005f5c0 con 0x7fc6b00ffee0 2024-05-12T03:58:52.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.399+0000 7fc6a5ffb700 1 -- 172.21.15.32:0/2713876617 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fc6a000bc50 con 0x7fc6b00ffee0 2024-05-12T03:58:52.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.399+0000 7fc6a5ffb700 1 --2- 172.21.15.32:0/2713876617 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fc69c0692b0 0x7fc69c06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:52.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.400+0000 7fc6a5ffb700 1 -- 172.21.15.32:0/2713876617 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fc6a0088010 con 0x7fc6b00ffee0 2024-05-12T03:58:52.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.400+0000 7fc6b5762700 1 --2- 172.21.15.32:0/2713876617 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fc69c0692b0 0x7fc69c06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:52.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.400+0000 7fc6b5762700 1 --2- 172.21.15.32:0/2713876617 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fc69c0692b0 0x7fc69c06b760 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac0041d0 tx=0x7fc6ac009040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:58:52.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.407+0000 7fc6a5ffb700 1 -- 172.21.15.32:0/2713876617 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fc6a0054390 con 0x7fc6b00ffee0 2024-05-12T03:58:52.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.624+0000 7fc6b79c6700 1 -- 172.21.15.32:0/2713876617 --> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc6b0101b50 con 0x7fc69c0692b0 2024-05-12T03:58:52.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.625+0000 7fc6a5ffb700 1 -- 172.21.15.32:0/2713876617 <== mgr.14208 v2:172.21.15.32:6800/4089797100 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7fc6b0101b50 con 0x7fc69c0692b0 2024-05-12T03:58:52.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.627+0000 7fc6b79c6700 1 -- 172.21.15.32:0/2713876617 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fc69c0692b0 msgr2=0x7fc69c06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:52.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.627+0000 7fc6b79c6700 1 --2- 172.21.15.32:0/2713876617 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fc69c0692b0 0x7fc69c06b760 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac0041d0 tx=0x7fc6ac009040 comp rx=0 tx=0).stop 2024-05-12T03:58:52.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.627+0000 7fc6b79c6700 1 -- 172.21.15.32:0/2713876617 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6b00ffee0 msgr2=0x7fc6b00fb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:52.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.627+0000 7fc6b79c6700 1 --2- 172.21.15.32:0/2713876617 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6b00ffee0 0x7fc6b00fb8a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0009fd0 tx=0x7fc6a0009e30 comp rx=0 tx=0).stop 2024-05-12T03:58:52.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.627+0000 7fc6b79c6700 1 -- 172.21.15.32:0/2713876617 shutdown_connections 2024-05-12T03:58:52.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.627+0000 7fc6b79c6700 1 --2- 172.21.15.32:0/2713876617 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fc69c0692b0 0x7fc69c06b760 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:52.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.627+0000 7fc6b79c6700 1 --2- 172.21.15.32:0/2713876617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6b00662c0 0x7fc6b00fd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:52.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.627+0000 7fc6b79c6700 1 --2- 172.21.15.32:0/2713876617 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6b00ffee0 0x7fc6b00fb8a0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:52.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.627+0000 7fc6b79c6700 1 -- 172.21.15.32:0/2713876617 >> 172.21.15.32:0/2713876617 conn(0x7fc6b00f7430 msgr2=0x7fc6b00f9810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:58:52.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.627+0000 7fc6b79c6700 1 -- 172.21.15.32:0/2713876617 shutdown_connections 2024-05-12T03:58:52.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.627+0000 7fc6b79c6700 1 -- 172.21.15.32:0/2713876617 wait complete. 2024-05-12T03:58:52.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.739+0000 7fae76387700 1 -- 172.21.15.32:0/2561097619 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae700fcf60 msgr2=0x7fae700fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:52.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.739+0000 7fae76387700 1 --2- 172.21.15.32:0/2561097619 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae700fcf60 0x7fae700fd3d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fae64004940 tx=0x7fae64009e30 comp rx=0 tx=0).stop 2024-05-12T03:58:52.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.740+0000 7fae76387700 1 -- 172.21.15.32:0/2561097619 shutdown_connections 2024-05-12T03:58:52.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.740+0000 7fae76387700 1 --2- 172.21.15.32:0/2561097619 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae700fcf60 0x7fae700fd3d0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:52.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.740+0000 7fae76387700 1 --2- 172.21.15.32:0/2561097619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae700fbcb0 0x7fae700fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:52.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.740+0000 7fae76387700 1 -- 172.21.15.32:0/2561097619 >> 172.21.15.32:0/2561097619 conn(0x7fae700f7430 msgr2=0x7fae700f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:58:52.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.740+0000 7fae76387700 1 -- 172.21.15.32:0/2561097619 shutdown_connections 2024-05-12T03:58:52.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.740+0000 7fae76387700 1 -- 172.21.15.32:0/2561097619 wait complete. 2024-05-12T03:58:52.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.740+0000 7fae76387700 1 Processor -- start 2024-05-12T03:58:52.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.740+0000 7fae76387700 1 -- start start 2024-05-12T03:58:52.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.741+0000 7fae76387700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae700fbcb0 0x7fae7010da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:52.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.741+0000 7fae76387700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae700fcf60 0x7fae7010df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:52.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.741+0000 7fae76387700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7010e540 con 0x7fae700fcf60 2024-05-12T03:58:52.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.741+0000 7fae76387700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7010ecf0 con 0x7fae700fbcb0 2024-05-12T03:58:52.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.741+0000 7fae6f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae700fcf60 0x7fae7010df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:52.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.741+0000 7fae6f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae700fcf60 0x7fae7010df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43670/0 (socket says 172.21.15.32:43670) 2024-05-12T03:58:52.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.741+0000 7fae6f7fe700 1 -- 172.21.15.32:0/2766367391 learned_addr learned my addr 172.21.15.32:0/2766367391 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:58:52.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.741+0000 7fae6ffff700 1 --2- 172.21.15.32:0/2766367391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae700fbcb0 0x7fae7010da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:52.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.741+0000 7fae6f7fe700 1 -- 172.21.15.32:0/2766367391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae700fbcb0 msgr2=0x7fae7010da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:52.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.741+0000 7fae6f7fe700 1 --2- 172.21.15.32:0/2766367391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae700fbcb0 0x7fae7010da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:52.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.741+0000 7fae6f7fe700 1 -- 172.21.15.32:0/2766367391 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae64005040 con 0x7fae700fcf60 2024-05-12T03:58:52.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.741+0000 7fae6f7fe700 1 --2- 172.21.15.32:0/2766367391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae700fcf60 0x7fae7010df40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fae6400b3b0 tx=0x7fae64009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:58:52.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.741+0000 7fae6cff9700 1 -- 172.21.15.32:0/2766367391 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae6400e040 con 0x7fae700fcf60 2024-05-12T03:58:52.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.741+0000 7fae6cff9700 1 -- 172.21.15.32:0/2766367391 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fae6400bc50 con 0x7fae700fcf60 2024-05-12T03:58:52.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.741+0000 7fae76387700 1 -- 172.21.15.32:0/2766367391 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae7010ef70 con 0x7fae700fcf60 2024-05-12T03:58:52.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.741+0000 7fae6cff9700 1 -- 172.21.15.32:0/2766367391 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae6401c630 con 0x7fae700fcf60 2024-05-12T03:58:52.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.741+0000 7fae76387700 1 -- 172.21.15.32:0/2766367391 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae7010f4b0 con 0x7fae700fcf60 2024-05-12T03:58:52.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.743+0000 7fae76387700 1 -- 172.21.15.32:0/2766367391 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae70101af0 con 0x7fae700fcf60 2024-05-12T03:58:52.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.743+0000 7fae6cff9700 1 -- 172.21.15.32:0/2766367391 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fae64013420 con 0x7fae700fcf60 2024-05-12T03:58:52.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.743+0000 7fae6cff9700 1 --2- 172.21.15.32:0/2766367391 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fae580692a0 0x7fae5806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:52.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.744+0000 7fae6ffff700 1 --2- 172.21.15.32:0/2766367391 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fae580692a0 0x7fae5806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:52.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.744+0000 7fae6cff9700 1 -- 172.21.15.32:0/2766367391 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fae64087f80 con 0x7fae700fcf60 2024-05-12T03:58:52.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.744+0000 7fae6ffff700 1 --2- 172.21.15.32:0/2766367391 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fae580692a0 0x7fae5806b750 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fae60004f40 tx=0x7fae6000b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:58:52.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.748+0000 7fae6cff9700 1 -- 172.21.15.32:0/2766367391 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fae64057c50 con 0x7fae700fcf60 2024-05-12T03:58:52.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.972+0000 7fae76387700 1 -- 172.21.15.32:0/2766367391 --> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fae70104770 con 0x7fae580692a0 2024-05-12T03:58:52.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.982+0000 7fae6cff9700 1 -- 172.21.15.32:0/2766367391 <== mgr.14208 v2:172.21.15.32:6800/4089797100 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3366 (secure 0 0 0) 0x7fae70104770 con 0x7fae580692a0 2024-05-12T03:58:52.983 INFO:teuthology.orchestra.run.smithi032.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-12T03:58:52.983 INFO:teuthology.orchestra.run.smithi032.stdout:alertmanager.smithi032 smithi032 *:9093,9094 running (5m) 34s ago 6m 27.7M - 0.25.0 c8568f914cd2 5c80dfe17d2e 2024-05-12T03:58:52.983 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi032 smithi032 running (7m) 34s ago 7m 6651k - 17.2.7-939-gc190bdee 284803f7d9cf a478874e9dfa 2024-05-12T03:58:52.983 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi033 smithi033 running (6m) 41s ago 6m 6651k - 17.2.7-939-gc190bdee 284803f7d9cf 34c62d004c7e 2024-05-12T03:58:52.983 INFO:teuthology.orchestra.run.smithi032.stdout:grafana.smithi032 smithi032 *:3000 running (5m) 34s ago 6m 85.7M - 9.4.7 954c08fa6188 ac9076563478 2024-05-12T03:58:52.983 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.ialtqb smithi032 running (61s) 34s ago 60s 10.4M - 17.2.7-939-gc190bdee 284803f7d9cf 0683abf00e3e 2024-05-12T03:58:52.983 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.qpsawh smithi032 running (54s) 34s ago 54s 16.6M - 17.2.7-939-gc190bdee 284803f7d9cf 95a7db5c4fcd 2024-05-12T03:58:52.983 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi033.hglwml smithi033 running (51s) 41s ago 51s 10.4M - 17.2.7-939-gc190bdee 284803f7d9cf c97902b05025 2024-05-12T03:58:52.983 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi033.jgbjhk smithi033 running (58s) 41s ago 57s 15.2M - 17.2.7-939-gc190bdee 284803f7d9cf af97052cb97a 2024-05-12T03:58:52.983 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi032.ukatkq smithi032 *:9283 running (8m) 34s ago 8m 483M - 17.2.7-939-gc190bdee 284803f7d9cf d9763dc47404 2024-05-12T03:58:52.983 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi033.pcvxwy smithi033 *:8443,9283 running (5m) 41s ago 5m 434M - 17.2.7-939-gc190bdee 284803f7d9cf 70d285a35adb 2024-05-12T03:58:52.984 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi032 smithi032 running (8m) 34s ago 8m 45.4M 2048M 17.2.7-939-gc190bdee 284803f7d9cf d7fea430c6e2 2024-05-12T03:58:52.984 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi033 smithi033 running (5m) 41s ago 5m 34.4M 2048M 17.2.7-939-gc190bdee 284803f7d9cf 707654d85748 2024-05-12T03:58:52.984 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi032 smithi032 *:9100 running (7m) 34s ago 7m 17.8M - 1.5.0 0da6a335fe13 6aa48ce262de 2024-05-12T03:58:52.984 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi033 smithi033 *:9100 running (5m) 41s ago 5m 16.9M - 1.5.0 0da6a335fe13 1144caed00e9 2024-05-12T03:58:52.984 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0 smithi032 running (5m) 34s ago 5m 44.7M 4300M 17.2.7-939-gc190bdee 284803f7d9cf 75c78a47f8e5 2024-05-12T03:58:52.984 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1 smithi032 running (4m) 34s ago 4m 48.5M 4300M 17.2.7-939-gc190bdee 284803f7d9cf 87cfd247d8d6 2024-05-12T03:58:52.984 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2 smithi032 running (4m) 34s ago 4m 43.8M 4300M 17.2.7-939-gc190bdee 284803f7d9cf 7aef7affb8ba 2024-05-12T03:58:52.984 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3 smithi033 running (3m) 41s ago 3m 44.0M 5324M 17.2.7-939-gc190bdee 284803f7d9cf 3a92b5e3348e 2024-05-12T03:58:52.984 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4 smithi033 running (3m) 41s ago 3m 44.2M 5324M 17.2.7-939-gc190bdee 284803f7d9cf 0896268b4591 2024-05-12T03:58:52.984 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5 smithi033 running (2m) 41s ago 2m 40.7M 5324M 17.2.7-939-gc190bdee 284803f7d9cf 57e65a64c821 2024-05-12T03:58:52.984 INFO:teuthology.orchestra.run.smithi032.stdout:prometheus.smithi032 smithi032 *:9095 running (5m) 34s ago 6m 52.6M - 2.43.0 a07b618ecd1d 0b3ce3323a49 2024-05-12T03:58:52.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.985+0000 7fae76387700 1 -- 172.21.15.32:0/2766367391 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fae580692a0 msgr2=0x7fae5806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:52.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.985+0000 7fae76387700 1 --2- 172.21.15.32:0/2766367391 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fae580692a0 0x7fae5806b750 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fae60004f40 tx=0x7fae6000b040 comp rx=0 tx=0).stop 2024-05-12T03:58:52.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.985+0000 7fae76387700 1 -- 172.21.15.32:0/2766367391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae700fcf60 msgr2=0x7fae7010df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:52.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.985+0000 7fae76387700 1 --2- 172.21.15.32:0/2766367391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae700fcf60 0x7fae7010df40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fae6400b3b0 tx=0x7fae64009e30 comp rx=0 tx=0).stop 2024-05-12T03:58:52.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.985+0000 7fae76387700 1 -- 172.21.15.32:0/2766367391 shutdown_connections 2024-05-12T03:58:52.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.985+0000 7fae76387700 1 --2- 172.21.15.32:0/2766367391 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fae580692a0 0x7fae5806b750 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:52.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.985+0000 7fae76387700 1 --2- 172.21.15.32:0/2766367391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae700fbcb0 0x7fae7010da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:52.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.985+0000 7fae76387700 1 --2- 172.21.15.32:0/2766367391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae700fcf60 0x7fae7010df40 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:52.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.985+0000 7fae76387700 1 -- 172.21.15.32:0/2766367391 >> 172.21.15.32:0/2766367391 conn(0x7fae700f7430 msgr2=0x7fae70065e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:58:52.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.985+0000 7fae76387700 1 -- 172.21.15.32:0/2766367391 shutdown_connections 2024-05-12T03:58:52.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:52.985+0000 7fae76387700 1 -- 172.21.15.32:0/2766367391 wait complete. 2024-05-12T03:58:53.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.094+0000 7f112275d700 1 -- 172.21.15.32:0/457304567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f111c0fefd0 msgr2=0x7f111c0ff3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:53.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.094+0000 7f112275d700 1 --2- 172.21.15.32:0/457304567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f111c0fefd0 0x7f111c0ff3e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f110c004940 tx=0x7f110c009e30 comp rx=0 tx=0).stop 2024-05-12T03:58:53.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.094+0000 7f112275d700 1 -- 172.21.15.32:0/457304567 shutdown_connections 2024-05-12T03:58:53.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.094+0000 7f112275d700 1 --2- 172.21.15.32:0/457304567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f111c100280 0x7f111c1006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:53.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.094+0000 7f112275d700 1 --2- 172.21.15.32:0/457304567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f111c0fefd0 0x7f111c0ff3e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:53.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.094+0000 7f112275d700 1 -- 172.21.15.32:0/457304567 >> 172.21.15.32:0/457304567 conn(0x7f111c0fa8f0 msgr2=0x7f111c0fccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:58:53.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.095+0000 7f112275d700 1 -- 172.21.15.32:0/457304567 shutdown_connections 2024-05-12T03:58:53.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.095+0000 7f112275d700 1 -- 172.21.15.32:0/457304567 wait complete. 2024-05-12T03:58:53.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.095+0000 7f112275d700 1 Processor -- start 2024-05-12T03:58:53.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.095+0000 7f112275d700 1 -- start start 2024-05-12T03:58:53.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.095+0000 7f112275d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f111c0fefd0 0x7f111c10fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:53.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.095+0000 7f112275d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f111c100280 0x7f111c110090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:53.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.095+0000 7f112275d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f111c110690 con 0x7f111c100280 2024-05-12T03:58:53.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.095+0000 7f112275d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f111c110e90 con 0x7f111c0fefd0 2024-05-12T03:58:53.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.096+0000 7f111b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f111c100280 0x7f111c110090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:53.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.096+0000 7f111b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f111c100280 0x7f111c110090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43690/0 (socket says 172.21.15.32:43690) 2024-05-12T03:58:53.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.096+0000 7f111b7fe700 1 -- 172.21.15.32:0/237985536 learned_addr learned my addr 172.21.15.32:0/237985536 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:58:53.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.096+0000 7f111bfff700 1 --2- 172.21.15.32:0/237985536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f111c0fefd0 0x7f111c10fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:53.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.096+0000 7f111b7fe700 1 -- 172.21.15.32:0/237985536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f111c0fefd0 msgr2=0x7f111c10fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:53.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.096+0000 7f111b7fe700 1 --2- 172.21.15.32:0/237985536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f111c0fefd0 0x7f111c10fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:53.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.096+0000 7f111b7fe700 1 -- 172.21.15.32:0/237985536 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f110c005040 con 0x7f111c100280 2024-05-12T03:58:53.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.096+0000 7f111bfff700 1 --2- 172.21.15.32:0/237985536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f111c0fefd0 0x7f111c10fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-12T03:58:53.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.096+0000 7f111b7fe700 1 --2- 172.21.15.32:0/237985536 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f111c100280 0x7f111c110090 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1110002a50 tx=0x7f11100046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:58:53.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.096+0000 7f1118ff9700 1 -- 172.21.15.32:0/237985536 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1110007cc0 con 0x7f111c100280 2024-05-12T03:58:53.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.096+0000 7f112275d700 1 -- 172.21.15.32:0/237985536 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f111c111170 con 0x7f111c100280 2024-05-12T03:58:53.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.096+0000 7f1118ff9700 1 -- 172.21.15.32:0/237985536 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f111000f040 con 0x7f111c100280 2024-05-12T03:58:53.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.096+0000 7f1118ff9700 1 -- 172.21.15.32:0/237985536 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1110009540 con 0x7f111c100280 2024-05-12T03:58:53.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.096+0000 7f112275d700 1 -- 172.21.15.32:0/237985536 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f111c1116b0 con 0x7f111c100280 2024-05-12T03:58:53.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.097+0000 7f112275d700 1 -- 172.21.15.32:0/237985536 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f111c05f5c0 con 0x7f111c100280 2024-05-12T03:58:53.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.098+0000 7f1118ff9700 1 -- 172.21.15.32:0/237985536 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f11100096a0 con 0x7f111c100280 2024-05-12T03:58:53.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.099+0000 7f1118ff9700 1 --2- 172.21.15.32:0/237985536 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f11040692a0 0x7f110406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:53.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.099+0000 7f1118ff9700 1 -- 172.21.15.32:0/237985536 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f11100870a0 con 0x7f111c100280 2024-05-12T03:58:53.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.099+0000 7f111bfff700 1 --2- 172.21.15.32:0/237985536 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f11040692a0 0x7f110406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:53.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.099+0000 7f111bfff700 1 --2- 172.21.15.32:0/237985536 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f11040692a0 0x7f110406b750 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f110c00b3b0 tx=0x7f110c009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:58:53.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.103+0000 7f1118ff9700 1 -- 172.21.15.32:0/237985536 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f1110056cf0 con 0x7f111c100280 2024-05-12T03:58:53.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.398+0000 7f112275d700 1 -- 172.21.15.32:0/237985536 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f111c1119d0 con 0x7f111c100280 2024-05-12T03:58:53.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.399+0000 7f1118ff9700 1 -- 172.21.15.32:0/237985536 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7f1110056880 con 0x7f111c100280 2024-05-12T03:58:53.399 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-05-12T03:58:53.399 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": { 2024-05-12T03:58:53.399 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 2 2024-05-12T03:58:53.399 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T03:58:53.399 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": { 2024-05-12T03:58:53.400 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 2 2024-05-12T03:58:53.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T03:58:53.400 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": { 2024-05-12T03:58:53.400 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 6 2024-05-12T03:58:53.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T03:58:53.400 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": { 2024-05-12T03:58:53.400 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 4 2024-05-12T03:58:53.400 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T03:58:53.400 INFO:teuthology.orchestra.run.smithi032.stdout: "overall": { 2024-05-12T03:58:53.400 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 14 2024-05-12T03:58:53.400 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-05-12T03:58:53.400 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-05-12T03:58:53.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.402+0000 7f112275d700 1 -- 172.21.15.32:0/237985536 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f11040692a0 msgr2=0x7f110406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:53.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.402+0000 7f112275d700 1 --2- 172.21.15.32:0/237985536 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f11040692a0 0x7f110406b750 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f110c00b3b0 tx=0x7f110c009e30 comp rx=0 tx=0).stop 2024-05-12T03:58:53.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.402+0000 7f112275d700 1 -- 172.21.15.32:0/237985536 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f111c100280 msgr2=0x7f111c110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:53.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.402+0000 7f112275d700 1 --2- 172.21.15.32:0/237985536 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f111c100280 0x7f111c110090 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1110002a50 tx=0x7f11100046b0 comp rx=0 tx=0).stop 2024-05-12T03:58:53.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.402+0000 7f112275d700 1 -- 172.21.15.32:0/237985536 shutdown_connections 2024-05-12T03:58:53.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.402+0000 7f112275d700 1 --2- 172.21.15.32:0/237985536 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f11040692a0 0x7f110406b750 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:53.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.402+0000 7f112275d700 1 --2- 172.21.15.32:0/237985536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f111c0fefd0 0x7f111c10fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:53.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.402+0000 7f112275d700 1 --2- 172.21.15.32:0/237985536 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f111c100280 0x7f111c110090 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:53.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.402+0000 7f112275d700 1 -- 172.21.15.32:0/237985536 >> 172.21.15.32:0/237985536 conn(0x7f111c0fa8f0 msgr2=0x7f111c0fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:58:53.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.402+0000 7f112275d700 1 -- 172.21.15.32:0/237985536 shutdown_connections 2024-05-12T03:58:53.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.402+0000 7f112275d700 1 -- 172.21.15.32:0/237985536 wait complete. 2024-05-12T03:58:53.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:53 smithi033 ceph-mon[49525]: pgmap v220: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 597 B/s wr, 3 op/s 2024-05-12T03:58:53.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.514+0000 7f1d20771700 1 -- 172.21.15.32:0/295954227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d180fce80 msgr2=0x7f1d180fd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:53.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.514+0000 7f1d20771700 1 --2- 172.21.15.32:0/295954227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d180fce80 0x7f1d180fd290 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1d14004940 tx=0x7f1d14009e30 comp rx=0 tx=0).stop 2024-05-12T03:58:53.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.514+0000 7f1d20771700 1 -- 172.21.15.32:0/295954227 shutdown_connections 2024-05-12T03:58:53.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.514+0000 7f1d20771700 1 --2- 172.21.15.32:0/295954227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d180fe080 0x7f1d180fe4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:53.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.514+0000 7f1d20771700 1 --2- 172.21.15.32:0/295954227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d180fce80 0x7f1d180fd290 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:53.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.514+0000 7f1d20771700 1 -- 172.21.15.32:0/295954227 >> 172.21.15.32:0/295954227 conn(0x7f1d180f8590 msgr2=0x7f1d180fa9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:58:53.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.514+0000 7f1d20771700 1 -- 172.21.15.32:0/295954227 shutdown_connections 2024-05-12T03:58:53.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.514+0000 7f1d20771700 1 -- 172.21.15.32:0/295954227 wait complete. 2024-05-12T03:58:53.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.515+0000 7f1d20771700 1 Processor -- start 2024-05-12T03:58:53.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.515+0000 7f1d20771700 1 -- start start 2024-05-12T03:58:53.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.515+0000 7f1d20771700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d180fce80 0x7f1d1810d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:53.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.515+0000 7f1d20771700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d180fe080 0x7f1d1810de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:53.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.515+0000 7f1d20771700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1810e490 con 0x7f1d180fe080 2024-05-12T03:58:53.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.515+0000 7f1d20771700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1810ece0 con 0x7f1d180fce80 2024-05-12T03:58:53.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.515+0000 7f1d1dd0c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d180fe080 0x7f1d1810de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:53.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.515+0000 7f1d1dd0c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d180fe080 0x7f1d1810de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43708/0 (socket says 172.21.15.32:43708) 2024-05-12T03:58:53.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.515+0000 7f1d1dd0c700 1 -- 172.21.15.32:0/2133030951 learned_addr learned my addr 172.21.15.32:0/2133030951 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:58:53.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.515+0000 7f1d1e50d700 1 --2- 172.21.15.32:0/2133030951 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d180fce80 0x7f1d1810d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:53.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.515+0000 7f1d1dd0c700 1 -- 172.21.15.32:0/2133030951 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d180fce80 msgr2=0x7f1d1810d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:53.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.516+0000 7f1d1dd0c700 1 --2- 172.21.15.32:0/2133030951 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d180fce80 0x7f1d1810d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:53.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.516+0000 7f1d1dd0c700 1 -- 172.21.15.32:0/2133030951 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d14005040 con 0x7f1d180fe080 2024-05-12T03:58:53.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.516+0000 7f1d1dd0c700 1 --2- 172.21.15.32:0/2133030951 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d180fe080 0x7f1d1810de90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1d0800c830 tx=0x7f1d08004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:58:53.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.516+0000 7f1d0effd700 1 -- 172.21.15.32:0/2133030951 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d0800a760 con 0x7f1d180fe080 2024-05-12T03:58:53.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.516+0000 7f1d20771700 1 -- 172.21.15.32:0/2133030951 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d1810efc0 con 0x7f1d180fe080 2024-05-12T03:58:53.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.516+0000 7f1d0effd700 1 -- 172.21.15.32:0/2133030951 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1d0800ad70 con 0x7f1d180fe080 2024-05-12T03:58:53.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.516+0000 7f1d0effd700 1 -- 172.21.15.32:0/2133030951 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d080113a0 con 0x7f1d180fe080 2024-05-12T03:58:53.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.516+0000 7f1d20771700 1 -- 172.21.15.32:0/2133030951 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d1810f500 con 0x7f1d180fe080 2024-05-12T03:58:53.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.517+0000 7f1d20771700 1 -- 172.21.15.32:0/2133030951 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d1805f5c0 con 0x7f1d180fe080 2024-05-12T03:58:53.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.518+0000 7f1d0effd700 1 -- 172.21.15.32:0/2133030951 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f1d08012400 con 0x7f1d180fe080 2024-05-12T03:58:53.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.518+0000 7f1d0effd700 1 --2- 172.21.15.32:0/2133030951 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f1d040692a0 0x7f1d0406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:53.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.518+0000 7f1d0effd700 1 -- 172.21.15.32:0/2133030951 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f1d08087280 con 0x7f1d180fe080 2024-05-12T03:58:53.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.518+0000 7f1d1e50d700 1 --2- 172.21.15.32:0/2133030951 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f1d040692a0 0x7f1d0406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:53.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.519+0000 7f1d1e50d700 1 --2- 172.21.15.32:0/2133030951 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f1d040692a0 0x7f1d0406b750 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1d1400b900 tx=0x7f1d14009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:58:53.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.522+0000 7f1d0effd700 1 -- 172.21.15.32:0/2133030951 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f1d08056ff0 con 0x7f1d180fe080 2024-05-12T03:58:53.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:53 smithi032 ceph-mon[40008]: pgmap v220: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 597 B/s wr, 3 op/s 2024-05-12T03:58:53.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.777+0000 7f1d20771700 1 -- 172.21.15.32:0/2133030951 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f1d1810f880 con 0x7f1d180fe080 2024-05-12T03:58:53.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.777+0000 7f1d0effd700 1 -- 172.21.15.32:0/2133030951 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1828 (secure 0 0 0) 0x7f1d08056b80 con 0x7f1d180fe080 2024-05-12T03:58:53.777 INFO:teuthology.orchestra.run.smithi032.stdout:e18 2024-05-12T03:58:53.777 INFO:teuthology.orchestra.run.smithi032.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-12T03:58:53.777 INFO:teuthology.orchestra.run.smithi032.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-12T03:58:53.777 INFO:teuthology.orchestra.run.smithi032.stdout:legacy client fscid: 1 2024-05-12T03:58:53.778 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:58:53.778 INFO:teuthology.orchestra.run.smithi032.stdout:Filesystem 'cephfs' (1) 2024-05-12T03:58:53.778 INFO:teuthology.orchestra.run.smithi032.stdout:fs_name cephfs 2024-05-12T03:58:53.778 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 18 2024-05-12T03:58:53.778 INFO:teuthology.orchestra.run.smithi032.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-12T03:58:53.778 INFO:teuthology.orchestra.run.smithi032.stdout:created 2024-05-12T03:57:48.904614+0000 2024-05-12T03:58:53.778 INFO:teuthology.orchestra.run.smithi032.stdout:modified 2024-05-12T03:58:06.821591+0000 2024-05-12T03:58:53.778 INFO:teuthology.orchestra.run.smithi032.stdout:tableserver 0 2024-05-12T03:58:53.778 INFO:teuthology.orchestra.run.smithi032.stdout:root 0 2024-05-12T03:58:53.778 INFO:teuthology.orchestra.run.smithi032.stdout:session_timeout 60 2024-05-12T03:58:53.778 INFO:teuthology.orchestra.run.smithi032.stdout:session_autoclose 300 2024-05-12T03:58:53.778 INFO:teuthology.orchestra.run.smithi032.stdout:max_file_size 1099511627776 2024-05-12T03:58:53.778 INFO:teuthology.orchestra.run.smithi032.stdout:required_client_features {} 2024-05-12T03:58:53.778 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure 0 2024-05-12T03:58:53.779 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure_osd_epoch 46 2024-05-12T03:58:53.779 INFO:teuthology.orchestra.run.smithi032.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-12T03:58:53.779 INFO:teuthology.orchestra.run.smithi032.stdout:max_mds 1 2024-05-12T03:58:53.779 INFO:teuthology.orchestra.run.smithi032.stdout:in 0 2024-05-12T03:58:53.779 INFO:teuthology.orchestra.run.smithi032.stdout:up {0=14474} 2024-05-12T03:58:53.779 INFO:teuthology.orchestra.run.smithi032.stdout:failed 2024-05-12T03:58:53.779 INFO:teuthology.orchestra.run.smithi032.stdout:damaged 2024-05-12T03:58:53.779 INFO:teuthology.orchestra.run.smithi032.stdout:stopped 2024-05-12T03:58:53.779 INFO:teuthology.orchestra.run.smithi032.stdout:data_pools [3] 2024-05-12T03:58:53.779 INFO:teuthology.orchestra.run.smithi032.stdout:metadata_pool 2 2024-05-12T03:58:53.779 INFO:teuthology.orchestra.run.smithi032.stdout:inline_data enabled 2024-05-12T03:58:53.779 INFO:teuthology.orchestra.run.smithi032.stdout:balancer 2024-05-12T03:58:53.779 INFO:teuthology.orchestra.run.smithi032.stdout:standby_count_wanted 1 2024-05-12T03:58:53.780 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.qpsawh{0:14474} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.32:6828/3100155296,v1:172.21.15.32:6829/3100155296] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T03:58:53.780 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi033.jgbjhk{0:24267} state up:standby-replay seq 3 join_fscid=1 addr [v2:172.21.15.33:6824/55417471,v1:172.21.15.33:6825/55417471] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T03:58:53.780 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:58:53.780 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:58:53.780 INFO:teuthology.orchestra.run.smithi032.stdout:Standby daemons: 2024-05-12T03:58:53.780 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:58:53.780 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi033.hglwml{-1:24283} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/433858238,v1:172.21.15.33:6827/433858238] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T03:58:53.780 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.ialtqb{-1:24287} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.32:6826/3703132329,v1:172.21.15.32:6827/3703132329] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T03:58:53.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.779+0000 7f1d20771700 1 -- 172.21.15.32:0/2133030951 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f1d040692a0 msgr2=0x7f1d0406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:53.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.779+0000 7f1d20771700 1 --2- 172.21.15.32:0/2133030951 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f1d040692a0 0x7f1d0406b750 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1d1400b900 tx=0x7f1d14009e30 comp rx=0 tx=0).stop 2024-05-12T03:58:53.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.779+0000 7f1d20771700 1 -- 172.21.15.32:0/2133030951 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d180fe080 msgr2=0x7f1d1810de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:53.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.779+0000 7f1d20771700 1 --2- 172.21.15.32:0/2133030951 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d180fe080 0x7f1d1810de90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1d0800c830 tx=0x7f1d08004470 comp rx=0 tx=0).stop 2024-05-12T03:58:53.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.779+0000 7f1d20771700 1 -- 172.21.15.32:0/2133030951 shutdown_connections 2024-05-12T03:58:53.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.779+0000 7f1d20771700 1 --2- 172.21.15.32:0/2133030951 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f1d040692a0 0x7f1d0406b750 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:53.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.779+0000 7f1d20771700 1 --2- 172.21.15.32:0/2133030951 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d180fce80 0x7f1d1810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:53.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.779+0000 7f1d20771700 1 --2- 172.21.15.32:0/2133030951 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d180fe080 0x7f1d1810de90 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:53.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.779+0000 7f1d20771700 1 -- 172.21.15.32:0/2133030951 >> 172.21.15.32:0/2133030951 conn(0x7f1d180f8590 msgr2=0x7f1d180f9d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:58:53.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.779+0000 7f1d20771700 1 -- 172.21.15.32:0/2133030951 shutdown_connections 2024-05-12T03:58:53.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.780+0000 7f1d20771700 1 -- 172.21.15.32:0/2133030951 wait complete. 2024-05-12T03:58:53.781 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 18 2024-05-12T03:58:53.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.890+0000 7f57b9248700 1 -- 172.21.15.32:0/2916616220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57b40fdfd0 msgr2=0x7f57b40fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:53.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.890+0000 7f57b9248700 1 --2- 172.21.15.32:0/2916616220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57b40fdfd0 0x7f57b40fe440 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f57a8004940 tx=0x7f57a8009e30 comp rx=0 tx=0).stop 2024-05-12T03:58:53.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.890+0000 7f57b9248700 1 -- 172.21.15.32:0/2916616220 shutdown_connections 2024-05-12T03:58:53.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.890+0000 7f57b9248700 1 --2- 172.21.15.32:0/2916616220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57b40fdfd0 0x7f57b40fe440 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:53.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.890+0000 7f57b9248700 1 --2- 172.21.15.32:0/2916616220 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b40fce60 0x7f57b40fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:53.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.890+0000 7f57b9248700 1 -- 172.21.15.32:0/2916616220 >> 172.21.15.32:0/2916616220 conn(0x7f57b40f8540 msgr2=0x7f57b40fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:58:53.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.890+0000 7f57b9248700 1 -- 172.21.15.32:0/2916616220 shutdown_connections 2024-05-12T03:58:53.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.890+0000 7f57b9248700 1 -- 172.21.15.32:0/2916616220 wait complete. 2024-05-12T03:58:53.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.891+0000 7f57b9248700 1 Processor -- start 2024-05-12T03:58:53.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.891+0000 7f57b9248700 1 -- start start 2024-05-12T03:58:53.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.891+0000 7f57b9248700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b40fce60 0x7f57b410d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:53.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.891+0000 7f57b9248700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57b40fdfd0 0x7f57b410de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:53.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.891+0000 7f57b9248700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b410e400 con 0x7f57b40fdfd0 2024-05-12T03:58:53.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.891+0000 7f57b9248700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b410ec50 con 0x7f57b40fce60 2024-05-12T03:58:53.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.891+0000 7f57b259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57b40fdfd0 0x7f57b410de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:53.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.891+0000 7f57b259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57b40fdfd0 0x7f57b410de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43722/0 (socket says 172.21.15.32:43722) 2024-05-12T03:58:53.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.891+0000 7f57b259c700 1 -- 172.21.15.32:0/688700960 learned_addr learned my addr 172.21.15.32:0/688700960 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:58:53.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.892+0000 7f57b2d9d700 1 --2- 172.21.15.32:0/688700960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b40fce60 0x7f57b410d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:53.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.892+0000 7f57b259c700 1 -- 172.21.15.32:0/688700960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b40fce60 msgr2=0x7f57b410d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:53.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.892+0000 7f57b259c700 1 --2- 172.21.15.32:0/688700960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b40fce60 0x7f57b410d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:53.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.892+0000 7f57b259c700 1 -- 172.21.15.32:0/688700960 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a8005040 con 0x7f57b40fdfd0 2024-05-12T03:58:53.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.892+0000 7f57b259c700 1 --2- 172.21.15.32:0/688700960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57b40fdfd0 0x7f57b410de00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f57a80137b0 tx=0x7f57a8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:58:53.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.892+0000 7f579b7fe700 1 -- 172.21.15.32:0/688700960 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f57a800e070 con 0x7f57b40fdfd0 2024-05-12T03:58:53.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.892+0000 7f57b9248700 1 -- 172.21.15.32:0/688700960 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57b410eed0 con 0x7f57b40fdfd0 2024-05-12T03:58:53.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.892+0000 7f579b7fe700 1 -- 172.21.15.32:0/688700960 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f57a80093f0 con 0x7f57b40fdfd0 2024-05-12T03:58:53.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.892+0000 7f579b7fe700 1 -- 172.21.15.32:0/688700960 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f57a800b410 con 0x7f57b40fdfd0 2024-05-12T03:58:53.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.892+0000 7f57b9248700 1 -- 172.21.15.32:0/688700960 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57b410f3b0 con 0x7f57b40fdfd0 2024-05-12T03:58:53.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.893+0000 7f57b9248700 1 -- 172.21.15.32:0/688700960 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57b405f5c0 con 0x7f57b40fdfd0 2024-05-12T03:58:53.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.894+0000 7f579b7fe700 1 -- 172.21.15.32:0/688700960 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f57a8013c40 con 0x7f57b40fdfd0 2024-05-12T03:58:53.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.894+0000 7f579b7fe700 1 --2- 172.21.15.32:0/688700960 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f579c0692a0 0x7f579c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:53.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.894+0000 7f579b7fe700 1 -- 172.21.15.32:0/688700960 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f57a80888b0 con 0x7f57b40fdfd0 2024-05-12T03:58:53.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.894+0000 7f57b2d9d700 1 --2- 172.21.15.32:0/688700960 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f579c0692a0 0x7f579c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:53.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.895+0000 7f57b2d9d700 1 --2- 172.21.15.32:0/688700960 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f579c0692a0 0x7f579c06b750 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f57a4004600 tx=0x7f57a400b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:58:53.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:53.898+0000 7f579b7fe700 1 -- 172.21.15.32:0/688700960 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f57a8058620 con 0x7f57b40fdfd0 2024-05-12T03:58:54.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:54.140+0000 7f57b9248700 1 -- 172.21.15.32:0/688700960 --> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f57b410f6e0 con 0x7f579c0692a0 2024-05-12T03:58:54.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:54.141+0000 7f579b7fe700 1 -- 172.21.15.32:0/688700960 <== mgr.14208 v2:172.21.15.32:6800/4089797100 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f57b410f6e0 con 0x7f579c0692a0 2024-05-12T03:58:54.141 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-05-12T03:58:54.141 INFO:teuthology.orchestra.run.smithi032.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:58249552a22f437b8cb49ed6b1068e1158bed605", 2024-05-12T03:58:54.141 INFO:teuthology.orchestra.run.smithi032.stdout: "in_progress": true, 2024-05-12T03:58:54.141 INFO:teuthology.orchestra.run.smithi032.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-12T03:58:54.141 INFO:teuthology.orchestra.run.smithi032.stdout: "services_complete": [], 2024-05-12T03:58:54.141 INFO:teuthology.orchestra.run.smithi032.stdout: "progress": "", 2024-05-12T03:58:54.141 INFO:teuthology.orchestra.run.smithi032.stdout: "message": "Doing first pull of quay.ceph.io/ceph-ci/ceph:58249552a22f437b8cb49ed6b1068e1158bed605 image", 2024-05-12T03:58:54.141 INFO:teuthology.orchestra.run.smithi032.stdout: "is_paused": false 2024-05-12T03:58:54.141 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-05-12T03:58:54.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:54.143+0000 7f57b9248700 1 -- 172.21.15.32:0/688700960 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f579c0692a0 msgr2=0x7f579c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:54.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:54.143+0000 7f57b9248700 1 --2- 172.21.15.32:0/688700960 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f579c0692a0 0x7f579c06b750 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f57a4004600 tx=0x7f57a400b040 comp rx=0 tx=0).stop 2024-05-12T03:58:54.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:54.143+0000 7f57b9248700 1 -- 172.21.15.32:0/688700960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57b40fdfd0 msgr2=0x7f57b410de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:54.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:54.143+0000 7f57b9248700 1 --2- 172.21.15.32:0/688700960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57b40fdfd0 0x7f57b410de00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f57a80137b0 tx=0x7f57a8009e30 comp rx=0 tx=0).stop 2024-05-12T03:58:54.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:54.145+0000 7f57b9248700 1 -- 172.21.15.32:0/688700960 shutdown_connections 2024-05-12T03:58:54.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:54.145+0000 7f57b9248700 1 --2- 172.21.15.32:0/688700960 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f579c0692a0 0x7f579c06b750 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:54.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:54.145+0000 7f57b9248700 1 --2- 172.21.15.32:0/688700960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b40fce60 0x7f57b410d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:54.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:54.145+0000 7f57b9248700 1 --2- 172.21.15.32:0/688700960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57b40fdfd0 0x7f57b410de00 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:54.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:54.145+0000 7f57b9248700 1 -- 172.21.15.32:0/688700960 >> 172.21.15.32:0/688700960 conn(0x7f57b40f8540 msgr2=0x7f57b40fa000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:58:54.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:54.145+0000 7f57b9248700 1 -- 172.21.15.32:0/688700960 shutdown_connections 2024-05-12T03:58:54.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:54.145+0000 7f57b9248700 1 -- 172.21.15.32:0/688700960 wait complete. 2024-05-12T03:58:54.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:54.266+0000 7f59f368e700 1 -- 172.21.15.32:0/740495667 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59ec0fe080 msgr2=0x7f59ec0fe4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:54.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:54.266+0000 7f59f368e700 1 --2- 172.21.15.32:0/740495667 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59ec0fe080 0x7f59ec0fe4f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f59dc004940 tx=0x7f59dc009e30 comp rx=0 tx=0).stop 2024-05-12T03:58:54.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:54.266+0000 7f59f368e700 1 -- 172.21.15.32:0/740495667 shutdown_connections 2024-05-12T03:58:54.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:54.266+0000 7f59f368e700 1 --2- 172.21.15.32:0/740495667 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59ec0fe080 0x7f59ec0fe4f0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:54.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:54.266+0000 7f59f368e700 1 --2- 172.21.15.32:0/740495667 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59ec0fce80 0x7f59ec0fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:54.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:54.266+0000 7f59f368e700 1 -- 172.21.15.32:0/740495667 >> 172.21.15.32:0/740495667 conn(0x7f59ec0f8590 msgr2=0x7f59ec0fa9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:58:54.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:54.266+0000 7f59f368e700 1 -- 172.21.15.32:0/740495667 shutdown_connections 2024-05-12T03:58:54.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:54.266+0000 7f59f368e700 1 -- 172.21.15.32:0/740495667 wait complete. 2024-05-12T03:58:54.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:54.267+0000 7f59f368e700 1 Processor -- start 2024-05-12T03:58:54.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:54.267+0000 7f59f368e700 1 -- start start 2024-05-12T03:58:54.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:54.267+0000 7f59f368e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59ec0fce80 0x7f59ec10d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:54.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:54.267+0000 7f59f368e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59ec0fe080 0x7f59ec10de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:54.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:54.267+0000 7f59f368e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59ec10e500 con 0x7f59ec0fce80 2024-05-12T03:58:54.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:54.267+0000 7f59f368e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59ec10ecf0 con 0x7f59ec0fe080 2024-05-12T03:58:54.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:54.267+0000 7f59f142a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59ec0fce80 0x7f59ec10d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:54.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:54.267+0000 7f59f142a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59ec0fce80 0x7f59ec10d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43740/0 (socket says 172.21.15.32:43740) 2024-05-12T03:58:54.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:54.267+0000 7f59f142a700 1 -- 172.21.15.32:0/1007355647 learned_addr learned my addr 172.21.15.32:0/1007355647 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:58:54.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:54.268+0000 7f59f0c29700 1 --2- 172.21.15.32:0/1007355647 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59ec0fe080 0x7f59ec10de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:54.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:54.268+0000 7f59f142a700 1 -- 172.21.15.32:0/1007355647 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59ec0fe080 msgr2=0x7f59ec10de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:54.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:54.268+0000 7f59f142a700 1 --2- 172.21.15.32:0/1007355647 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59ec0fe080 0x7f59ec10de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:54.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:54.268+0000 7f59f142a700 1 -- 172.21.15.32:0/1007355647 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59dc005040 con 0x7f59ec0fce80 2024-05-12T03:58:54.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:54.268+0000 7f59f142a700 1 --2- 172.21.15.32:0/1007355647 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59ec0fce80 0x7f59ec10d930 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f59e800b790 tx=0x7f59e8009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:58:54.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:54.268+0000 7f59e1ffb700 1 -- 172.21.15.32:0/1007355647 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f59e800f6f0 con 0x7f59ec0fce80 2024-05-12T03:58:54.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:54.268+0000 7f59e1ffb700 1 -- 172.21.15.32:0/1007355647 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f59e800fd00 con 0x7f59ec0fce80 2024-05-12T03:58:54.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:54.268+0000 7f59e1ffb700 1 -- 172.21.15.32:0/1007355647 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f59e80103a0 con 0x7f59ec0fce80 2024-05-12T03:58:54.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:54.268+0000 7f59f368e700 1 -- 172.21.15.32:0/1007355647 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59ec10efd0 con 0x7f59ec0fce80 2024-05-12T03:58:54.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:54.268+0000 7f59f368e700 1 -- 172.21.15.32:0/1007355647 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59ec10f540 con 0x7f59ec0fce80 2024-05-12T03:58:54.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:54.270+0000 7f59f368e700 1 -- 172.21.15.32:0/1007355647 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59ec05f5c0 con 0x7f59ec0fce80 2024-05-12T03:58:54.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:54.270+0000 7f59e1ffb700 1 -- 172.21.15.32:0/1007355647 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f59e800f850 con 0x7f59ec0fce80 2024-05-12T03:58:54.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:54.271+0000 7f59e1ffb700 1 --2- 172.21.15.32:0/1007355647 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f59d80692b0 0x7f59d806b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:58:54.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:54.271+0000 7f59e1ffb700 1 -- 172.21.15.32:0/1007355647 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f59e8087050 con 0x7f59ec0fce80 2024-05-12T03:58:54.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:54.271+0000 7f59f0c29700 1 --2- 172.21.15.32:0/1007355647 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f59d80692b0 0x7f59d806b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:58:54.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:54.271+0000 7f59f0c29700 1 --2- 172.21.15.32:0/1007355647 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f59d80692b0 0x7f59d806b760 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f59dc00b900 tx=0x7f59dc009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:58:54.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:54.275+0000 7f59e1ffb700 1 -- 172.21.15.32:0/1007355647 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f59e8056d40 con 0x7f59ec0fce80 2024-05-12T03:58:54.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:54 smithi033 ceph-mon[49525]: from='client.14562 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T03:58:54.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:54 smithi033 ceph-mon[49525]: from='client.14566 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T03:58:54.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:54 smithi033 ceph-mon[49525]: from='client.14570 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T03:58:54.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:54 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/237985536' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T03:58:54.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:54 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/2133030951' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-12T03:58:54.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:54 smithi032 ceph-mon[40008]: from='client.14562 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T03:58:54.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:54 smithi032 ceph-mon[40008]: from='client.14566 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T03:58:54.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:54 smithi032 ceph-mon[40008]: from='client.14570 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T03:58:54.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:54 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/237985536' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T03:58:54.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:54 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/2133030951' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-12T03:58:54.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:54.576+0000 7f59f368e700 1 -- 172.21.15.32:0/1007355647 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f59ec04bed0 con 0x7f59ec0fce80 2024-05-12T03:58:54.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:54.577+0000 7f59e1ffb700 1 -- 172.21.15.32:0/1007355647 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f59e80568d0 con 0x7f59ec0fce80 2024-05-12T03:58:54.577 INFO:teuthology.orchestra.run.smithi032.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-12T03:58:54.577 INFO:teuthology.orchestra.run.smithi032.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-12T03:58:54.577 INFO:teuthology.orchestra.run.smithi032.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-12T03:58:54.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:54.579+0000 7f59f368e700 1 -- 172.21.15.32:0/1007355647 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f59d80692b0 msgr2=0x7f59d806b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:54.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:54.579+0000 7f59f368e700 1 --2- 172.21.15.32:0/1007355647 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f59d80692b0 0x7f59d806b760 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f59dc00b900 tx=0x7f59dc009e30 comp rx=0 tx=0).stop 2024-05-12T03:58:54.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:54.579+0000 7f59f368e700 1 -- 172.21.15.32:0/1007355647 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59ec0fce80 msgr2=0x7f59ec10d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:58:54.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:54.579+0000 7f59f368e700 1 --2- 172.21.15.32:0/1007355647 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59ec0fce80 0x7f59ec10d930 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f59e800b790 tx=0x7f59e8009220 comp rx=0 tx=0).stop 2024-05-12T03:58:54.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:54.579+0000 7f59f368e700 1 -- 172.21.15.32:0/1007355647 shutdown_connections 2024-05-12T03:58:54.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:54.579+0000 7f59f368e700 1 --2- 172.21.15.32:0/1007355647 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f59d80692b0 0x7f59d806b760 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:54.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:54.579+0000 7f59f368e700 1 --2- 172.21.15.32:0/1007355647 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59ec0fce80 0x7f59ec10d930 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:54.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:54.579+0000 7f59f368e700 1 --2- 172.21.15.32:0/1007355647 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59ec0fe080 0x7f59ec10de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:58:54.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:54.579+0000 7f59f368e700 1 -- 172.21.15.32:0/1007355647 >> 172.21.15.32:0/1007355647 conn(0x7f59ec0f8590 msgr2=0x7f59ec0f9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:58:54.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:54.579+0000 7f59f368e700 1 -- 172.21.15.32:0/1007355647 shutdown_connections 2024-05-12T03:58:54.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:58:54.579+0000 7f59f368e700 1 -- 172.21.15.32:0/1007355647 wait complete. 2024-05-12T03:58:55.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:55 smithi033 ceph-mon[49525]: pgmap v221: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 426 B/s wr, 2 op/s 2024-05-12T03:58:55.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:55 smithi033 ceph-mon[49525]: from='client.14582 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T03:58:55.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:55 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/1007355647' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-12T03:58:55.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:55 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T03:58:55.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:55 smithi032 ceph-mon[40008]: pgmap v221: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 426 B/s wr, 2 op/s 2024-05-12T03:58:55.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:55 smithi032 ceph-mon[40008]: from='client.14582 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T03:58:55.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:55 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/1007355647' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-12T03:58:55.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:55 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T03:58:56.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:56 smithi033 ceph-mon[49525]: pgmap v222: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 341 B/s wr, 2 op/s 2024-05-12T03:58:56.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:56 smithi032 ceph-mon[40008]: pgmap v222: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 341 B/s wr, 2 op/s 2024-05-12T03:58:59.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:58:58 smithi033 ceph-mon[49525]: pgmap v223: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 85 B/s wr, 1 op/s 2024-05-12T03:58:59.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:58:58 smithi032 ceph-mon[40008]: pgmap v223: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 85 B/s wr, 1 op/s 2024-05-12T03:59:01.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:59:00 smithi033 ceph-mon[49525]: pgmap v224: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-05-12T03:59:01.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:59:00 smithi032 ceph-mon[40008]: pgmap v224: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-05-12T03:59:03.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:59:02 smithi033 ceph-mon[49525]: pgmap v225: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T03:59:03.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:59:02 smithi032 ceph-mon[40008]: pgmap v225: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T03:59:05.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:59:04 smithi033 ceph-mon[49525]: pgmap v226: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T03:59:05.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:59:04 smithi032 ceph-mon[40008]: pgmap v226: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T03:59:07.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:59:06 smithi033 ceph-mon[49525]: pgmap v227: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T03:59:07.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:59:06 smithi032 ceph-mon[40008]: pgmap v227: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T03:59:09.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:59:08 smithi033 ceph-mon[49525]: pgmap v228: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T03:59:09.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:59:08 smithi032 ceph-mon[40008]: pgmap v228: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T03:59:10.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:59:09 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T03:59:10.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:59:09 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T03:59:11.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:59:10 smithi032 ceph-mon[40008]: pgmap v229: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T03:59:11.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:59:10 smithi033 ceph-mon[49525]: pgmap v229: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T03:59:13.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:59:12 smithi032 ceph-mon[40008]: pgmap v230: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T03:59:13.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:59:12 smithi033 ceph-mon[49525]: pgmap v230: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T03:59:15.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:59:14 smithi032 ceph-mon[40008]: pgmap v231: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T03:59:15.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:59:14 smithi033 ceph-mon[49525]: pgmap v231: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T03:59:17.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:59:16 smithi033 ceph-mon[49525]: pgmap v232: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T03:59:17.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:59:16 smithi032 ceph-mon[40008]: pgmap v232: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T03:59:19.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:59:18 smithi032 ceph-mon[40008]: pgmap v233: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T03:59:19.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:59:18 smithi033 ceph-mon[49525]: pgmap v233: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T03:59:21.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:59:20 smithi032 ceph-mon[40008]: pgmap v234: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T03:59:21.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:59:20 smithi033 ceph-mon[49525]: pgmap v234: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T03:59:23.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:59:22 smithi032 ceph-mon[40008]: pgmap v235: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T03:59:23.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:59:22 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:59:23.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:59:22 smithi032 ceph-mon[40008]: Upgrade: Target is version 18.2.2-1272-g58249552 (unknown) 2024-05-12T03:59:23.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:59:22 smithi032 ceph-mon[40008]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:19c9734256aa43a60e203feb242f50bf8b7f58666d520536af43cc4fee96173a, digests ['quay.ceph.io/ceph-ci/ceph@sha256:19c9734256aa43a60e203feb242f50bf8b7f58666d520536af43cc4fee96173a'] 2024-05-12T03:59:23.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:59:22 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T03:59:23.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:59:22 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:59:23.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:59:22 smithi032 ceph-mon[40008]: Upgrade: Need to upgrade myself (mgr.smithi032.ukatkq) 2024-05-12T03:59:23.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:59:22 smithi033 ceph-mon[49525]: pgmap v235: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T03:59:23.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:59:22 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:59:23.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:59:22 smithi033 ceph-mon[49525]: Upgrade: Target is version 18.2.2-1272-g58249552 (unknown) 2024-05-12T03:59:23.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:59:22 smithi033 ceph-mon[49525]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:19c9734256aa43a60e203feb242f50bf8b7f58666d520536af43cc4fee96173a, digests ['quay.ceph.io/ceph-ci/ceph@sha256:19c9734256aa43a60e203feb242f50bf8b7f58666d520536af43cc4fee96173a'] 2024-05-12T03:59:23.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:59:22 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T03:59:23.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:59:22 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:59:23.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:59:22 smithi033 ceph-mon[49525]: Upgrade: Need to upgrade myself (mgr.smithi032.ukatkq) 2024-05-12T03:59:24.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:59:23 smithi032 ceph-mon[40008]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:19c9734256aa43a60e203feb242f50bf8b7f58666d520536af43cc4fee96173a on smithi033 2024-05-12T03:59:24.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:59:23 smithi033 ceph-mon[49525]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:19c9734256aa43a60e203feb242f50bf8b7f58666d520536af43cc4fee96173a on smithi033 2024-05-12T03:59:24.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:24.698+0000 7f27ebf17700 1 -- 172.21.15.32:0/3234187519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27e40fdbf0 msgr2=0x7f27e40fc000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:59:24.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:24.698+0000 7f27ebf17700 1 --2- 172.21.15.32:0/3234187519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27e40fdbf0 0x7f27e40fc000 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f27d4004940 tx=0x7f27d4009e30 comp rx=0 tx=0).stop 2024-05-12T03:59:24.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:24.698+0000 7f27ebf17700 1 -- 172.21.15.32:0/3234187519 shutdown_connections 2024-05-12T03:59:24.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:24.698+0000 7f27ebf17700 1 --2- 172.21.15.32:0/3234187519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27e40fdbf0 0x7f27e40fc000 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:24.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:24.698+0000 7f27ebf17700 1 --2- 172.21.15.32:0/3234187519 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27e40fd2a0 0x7f27e40fd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:24.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:24.698+0000 7f27ebf17700 1 -- 172.21.15.32:0/3234187519 >> 172.21.15.32:0/3234187519 conn(0x7f27e40f6b80 msgr2=0x7f27e40f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:59:24.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:24.698+0000 7f27ebf17700 1 -- 172.21.15.32:0/3234187519 shutdown_connections 2024-05-12T03:59:24.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:24.698+0000 7f27ebf17700 1 -- 172.21.15.32:0/3234187519 wait complete. 2024-05-12T03:59:24.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:24.699+0000 7f27ebf17700 1 Processor -- start 2024-05-12T03:59:24.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:24.699+0000 7f27ebf17700 1 -- start start 2024-05-12T03:59:24.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:24.699+0000 7f27ebf17700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27e40fd2a0 0x7f27e40ffe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:59:24.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:24.699+0000 7f27ebf17700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27e40fdbf0 0x7f27e40fe350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:59:24.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:24.699+0000 7f27ebf17700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e40fe8e0 con 0x7f27e40fdbf0 2024-05-12T03:59:24.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:24.699+0000 7f27ebf17700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e40fea20 con 0x7f27e40fd2a0 2024-05-12T03:59:24.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:24.699+0000 7f27e94b2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27e40fdbf0 0x7f27e40fe350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:59:24.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:24.699+0000 7f27e94b2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27e40fdbf0 0x7f27e40fe350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33724/0 (socket says 172.21.15.32:33724) 2024-05-12T03:59:24.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:24.699+0000 7f27e9cb3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27e40fd2a0 0x7f27e40ffe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:59:24.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:24.699+0000 7f27e94b2700 1 -- 172.21.15.32:0/94302227 learned_addr learned my addr 172.21.15.32:0/94302227 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:59:24.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:24.700+0000 7f27e9cb3700 1 -- 172.21.15.32:0/94302227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27e40fdbf0 msgr2=0x7f27e40fe350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:59:24.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:24.700+0000 7f27e9cb3700 1 --2- 172.21.15.32:0/94302227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27e40fdbf0 0x7f27e40fe350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:24.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:24.700+0000 7f27e9cb3700 1 -- 172.21.15.32:0/94302227 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27d4005040 con 0x7f27e40fd2a0 2024-05-12T03:59:24.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:24.700+0000 7f27e9cb3700 1 --2- 172.21.15.32:0/94302227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27e40fd2a0 0x7f27e40ffe30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f27e0009fd0 tx=0x7f27e000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:59:24.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:24.700+0000 7f27da7fc700 1 -- 172.21.15.32:0/94302227 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f27e0004e90 con 0x7f27e40fd2a0 2024-05-12T03:59:24.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:24.700+0000 7f27ebf17700 1 -- 172.21.15.32:0/94302227 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27e40feca0 con 0x7f27e40fd2a0 2024-05-12T03:59:24.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:24.700+0000 7f27da7fc700 1 -- 172.21.15.32:0/94302227 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f27e000f040 con 0x7f27e40fd2a0 2024-05-12T03:59:24.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:24.700+0000 7f27da7fc700 1 -- 172.21.15.32:0/94302227 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f27e000c9b0 con 0x7f27e40fd2a0 2024-05-12T03:59:24.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:24.700+0000 7f27ebf17700 1 -- 172.21.15.32:0/94302227 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27e4115970 con 0x7f27e40fd2a0 2024-05-12T03:59:24.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:24.701+0000 7f27ebf17700 1 -- 172.21.15.32:0/94302227 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27e4101ba0 con 0x7f27e40fd2a0 2024-05-12T03:59:24.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:24.702+0000 7f27da7fc700 1 -- 172.21.15.32:0/94302227 <== mon.1 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f27e0007cc0 con 0x7f27e40fd2a0 2024-05-12T03:59:24.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:24.702+0000 7f27da7fc700 1 --2- 172.21.15.32:0/94302227 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f27d00692a0 0x7f27d006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:59:24.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:24.702+0000 7f27da7fc700 1 -- 172.21.15.32:0/94302227 <== mon.1 v2:172.21.15.33:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f27e0086b80 con 0x7f27e40fd2a0 2024-05-12T03:59:24.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:24.702+0000 7f27e94b2700 1 --2- 172.21.15.32:0/94302227 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f27d00692a0 0x7f27d006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:59:24.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:24.703+0000 7f27e94b2700 1 --2- 172.21.15.32:0/94302227 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f27d00692a0 0x7f27d006b750 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f27d40135e0 tx=0x7f27d4009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:59:24.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:24.710+0000 7f27da7fc700 1 -- 172.21.15.32:0/94302227 <== mon.1 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f27e004e250 con 0x7f27e40fd2a0 2024-05-12T03:59:25.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.053+0000 7f27ebf17700 1 -- 172.21.15.32:0/94302227 --> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f27e405e3a0 con 0x7f27d00692a0 2024-05-12T03:59:25.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.054+0000 7f27da7fc700 1 -- 172.21.15.32:0/94302227 <== mgr.14208 v2:172.21.15.32:6800/4089797100 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7f27e405e3a0 con 0x7f27d00692a0 2024-05-12T03:59:25.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.057+0000 7f27ebf17700 1 -- 172.21.15.32:0/94302227 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f27d00692a0 msgr2=0x7f27d006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:59:25.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.057+0000 7f27ebf17700 1 --2- 172.21.15.32:0/94302227 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f27d00692a0 0x7f27d006b750 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f27d40135e0 tx=0x7f27d4009e30 comp rx=0 tx=0).stop 2024-05-12T03:59:25.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.057+0000 7f27ebf17700 1 -- 172.21.15.32:0/94302227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27e40fd2a0 msgr2=0x7f27e40ffe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:59:25.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.057+0000 7f27ebf17700 1 --2- 172.21.15.32:0/94302227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27e40fd2a0 0x7f27e40ffe30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f27e0009fd0 tx=0x7f27e000b040 comp rx=0 tx=0).stop 2024-05-12T03:59:25.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.057+0000 7f27ebf17700 1 -- 172.21.15.32:0/94302227 shutdown_connections 2024-05-12T03:59:25.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.057+0000 7f27ebf17700 1 --2- 172.21.15.32:0/94302227 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f27d00692a0 0x7f27d006b750 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:25.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.057+0000 7f27ebf17700 1 --2- 172.21.15.32:0/94302227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27e40fd2a0 0x7f27e40ffe30 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:25.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.057+0000 7f27ebf17700 1 --2- 172.21.15.32:0/94302227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27e40fdbf0 0x7f27e40fe350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:25.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.057+0000 7f27ebf17700 1 -- 172.21.15.32:0/94302227 >> 172.21.15.32:0/94302227 conn(0x7f27e40f6b80 msgr2=0x7f27e40fb560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:59:25.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.057+0000 7f27ebf17700 1 -- 172.21.15.32:0/94302227 shutdown_connections 2024-05-12T03:59:25.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.057+0000 7f27ebf17700 1 -- 172.21.15.32:0/94302227 wait complete. 2024-05-12T03:59:25.069 INFO:teuthology.orchestra.run.smithi032.stdout:true 2024-05-12T03:59:25.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.207+0000 7fb4c194b700 1 -- 172.21.15.32:0/3176255512 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4bc0fcf60 msgr2=0x7fb4bc0fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:59:25.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.207+0000 7fb4c194b700 1 --2- 172.21.15.32:0/3176255512 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4bc0fcf60 0x7fb4bc0fd3d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0004940 tx=0x7fb4b0009e30 comp rx=0 tx=0).stop 2024-05-12T03:59:25.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.207+0000 7fb4c194b700 1 -- 172.21.15.32:0/3176255512 shutdown_connections 2024-05-12T03:59:25.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.207+0000 7fb4c194b700 1 --2- 172.21.15.32:0/3176255512 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4bc0fcf60 0x7fb4bc0fd3d0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:25.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.207+0000 7fb4c194b700 1 --2- 172.21.15.32:0/3176255512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4bc0fbcb0 0x7fb4bc0fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:25.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.207+0000 7fb4c194b700 1 -- 172.21.15.32:0/3176255512 >> 172.21.15.32:0/3176255512 conn(0x7fb4bc0f7430 msgr2=0x7fb4bc0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:59:25.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.207+0000 7fb4c194b700 1 -- 172.21.15.32:0/3176255512 shutdown_connections 2024-05-12T03:59:25.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.207+0000 7fb4c194b700 1 -- 172.21.15.32:0/3176255512 wait complete. 2024-05-12T03:59:25.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.208+0000 7fb4c194b700 1 Processor -- start 2024-05-12T03:59:25.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.208+0000 7fb4c194b700 1 -- start start 2024-05-12T03:59:25.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.208+0000 7fb4c194b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4bc0fbcb0 0x7fb4bc1105a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:59:25.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.208+0000 7fb4c194b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4bc0fcf60 0x7fb4bc112af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:59:25.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.208+0000 7fb4c194b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4bc066330 con 0x7fb4bc0fcf60 2024-05-12T03:59:25.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.208+0000 7fb4c194b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4bc066470 con 0x7fb4bc0fbcb0 2024-05-12T03:59:25.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.208+0000 7fb4ba7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4bc0fcf60 0x7fb4bc112af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:59:25.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.208+0000 7fb4ba7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4bc0fcf60 0x7fb4bc112af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33740/0 (socket says 172.21.15.32:33740) 2024-05-12T03:59:25.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.208+0000 7fb4ba7fc700 1 -- 172.21.15.32:0/1017193541 learned_addr learned my addr 172.21.15.32:0/1017193541 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:59:25.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.208+0000 7fb4baffd700 1 --2- 172.21.15.32:0/1017193541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4bc0fbcb0 0x7fb4bc1105a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:59:25.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.208+0000 7fb4ba7fc700 1 -- 172.21.15.32:0/1017193541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4bc0fbcb0 msgr2=0x7fb4bc1105a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:59:25.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.208+0000 7fb4ba7fc700 1 --2- 172.21.15.32:0/1017193541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4bc0fbcb0 0x7fb4bc1105a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:25.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.208+0000 7fb4ba7fc700 1 -- 172.21.15.32:0/1017193541 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4b0005040 con 0x7fb4bc0fcf60 2024-05-12T03:59:25.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.209+0000 7fb4ba7fc700 1 --2- 172.21.15.32:0/1017193541 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4bc0fcf60 0x7fb4bc112af0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0000c00 tx=0x7fb4b0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:59:25.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.209+0000 7fb4a37fe700 1 -- 172.21.15.32:0/1017193541 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb4b0021070 con 0x7fb4bc0fcf60 2024-05-12T03:59:25.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.209+0000 7fb4c194b700 1 -- 172.21.15.32:0/1017193541 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4bc113030 con 0x7fb4bc0fcf60 2024-05-12T03:59:25.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.209+0000 7fb4a37fe700 1 -- 172.21.15.32:0/1017193541 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb4b001c400 con 0x7fb4bc0fcf60 2024-05-12T03:59:25.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.209+0000 7fb4a37fe700 1 -- 172.21.15.32:0/1017193541 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb4b000e070 con 0x7fb4bc0fcf60 2024-05-12T03:59:25.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.209+0000 7fb4c194b700 1 -- 172.21.15.32:0/1017193541 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4bc113590 con 0x7fb4bc0fcf60 2024-05-12T03:59:25.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.210+0000 7fb4c194b700 1 -- 172.21.15.32:0/1017193541 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4bc05f5c0 con 0x7fb4bc0fcf60 2024-05-12T03:59:25.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.211+0000 7fb4a37fe700 1 -- 172.21.15.32:0/1017193541 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fb4b001c9e0 con 0x7fb4bc0fcf60 2024-05-12T03:59:25.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.212+0000 7fb4a37fe700 1 --2- 172.21.15.32:0/1017193541 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fb4a40692a0 0x7fb4a406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:59:25.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.212+0000 7fb4a37fe700 1 -- 172.21.15.32:0/1017193541 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fb4b0088ee0 con 0x7fb4bc0fcf60 2024-05-12T03:59:25.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.212+0000 7fb4baffd700 1 --2- 172.21.15.32:0/1017193541 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fb4a40692a0 0x7fb4a406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:59:25.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.212+0000 7fb4baffd700 1 --2- 172.21.15.32:0/1017193541 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fb4a40692a0 0x7fb4a406b750 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc065fb0 tx=0x7fb4ac00b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:59:25.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.219+0000 7fb4a37fe700 1 -- 172.21.15.32:0/1017193541 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb4b0055250 con 0x7fb4bc0fcf60 2024-05-12T03:59:25.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:59:24 smithi032 ceph-mon[40008]: pgmap v236: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T03:59:25.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:59:24 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T03:59:25.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:59:24 smithi033 ceph-mon[49525]: pgmap v236: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T03:59:25.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:59:24 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T03:59:25.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.553+0000 7fb4c194b700 1 -- 172.21.15.32:0/1017193541 --> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb4bc101340 con 0x7fb4a40692a0 2024-05-12T03:59:25.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.555+0000 7fb4a37fe700 1 -- 172.21.15.32:0/1017193541 <== mgr.14208 v2:172.21.15.32:6800/4089797100 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7fb4bc101340 con 0x7fb4a40692a0 2024-05-12T03:59:25.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.557+0000 7fb4c194b700 1 -- 172.21.15.32:0/1017193541 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fb4a40692a0 msgr2=0x7fb4a406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:59:25.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.557+0000 7fb4c194b700 1 --2- 172.21.15.32:0/1017193541 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fb4a40692a0 0x7fb4a406b750 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc065fb0 tx=0x7fb4ac00b040 comp rx=0 tx=0).stop 2024-05-12T03:59:25.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.557+0000 7fb4c194b700 1 -- 172.21.15.32:0/1017193541 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4bc0fcf60 msgr2=0x7fb4bc112af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:59:25.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.557+0000 7fb4c194b700 1 --2- 172.21.15.32:0/1017193541 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4bc0fcf60 0x7fb4bc112af0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0000c00 tx=0x7fb4b0009e30 comp rx=0 tx=0).stop 2024-05-12T03:59:25.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.557+0000 7fb4c194b700 1 -- 172.21.15.32:0/1017193541 shutdown_connections 2024-05-12T03:59:25.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.557+0000 7fb4c194b700 1 --2- 172.21.15.32:0/1017193541 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fb4a40692a0 0x7fb4a406b750 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:25.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.557+0000 7fb4c194b700 1 --2- 172.21.15.32:0/1017193541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4bc0fbcb0 0x7fb4bc1105a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:25.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.557+0000 7fb4c194b700 1 --2- 172.21.15.32:0/1017193541 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4bc0fcf60 0x7fb4bc112af0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:25.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.557+0000 7fb4c194b700 1 -- 172.21.15.32:0/1017193541 >> 172.21.15.32:0/1017193541 conn(0x7fb4bc0f7430 msgr2=0x7fb4bc0f8b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:59:25.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.557+0000 7fb4c194b700 1 -- 172.21.15.32:0/1017193541 shutdown_connections 2024-05-12T03:59:25.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.557+0000 7fb4c194b700 1 -- 172.21.15.32:0/1017193541 wait complete. 2024-05-12T03:59:25.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.688+0000 7f4aa6f1a700 1 -- 172.21.15.32:0/3034911248 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4aa00fbcb0 msgr2=0x7f4aa00fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:59:25.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.688+0000 7f4aa6f1a700 1 --2- 172.21.15.32:0/3034911248 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4aa00fbcb0 0x7f4aa00fc0c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4a94004940 tx=0x7f4a94009e30 comp rx=0 tx=0).stop 2024-05-12T03:59:25.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.688+0000 7f4aa6f1a700 1 -- 172.21.15.32:0/3034911248 shutdown_connections 2024-05-12T03:59:25.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.688+0000 7f4aa6f1a700 1 --2- 172.21.15.32:0/3034911248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4aa00fcf60 0x7f4aa00fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:25.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.688+0000 7f4aa6f1a700 1 --2- 172.21.15.32:0/3034911248 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4aa00fbcb0 0x7f4aa00fc0c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:25.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.688+0000 7f4aa6f1a700 1 -- 172.21.15.32:0/3034911248 >> 172.21.15.32:0/3034911248 conn(0x7f4aa00f7430 msgr2=0x7f4aa00f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:59:25.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.688+0000 7f4aa6f1a700 1 -- 172.21.15.32:0/3034911248 shutdown_connections 2024-05-12T03:59:25.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.688+0000 7f4aa6f1a700 1 -- 172.21.15.32:0/3034911248 wait complete. 2024-05-12T03:59:25.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.689+0000 7f4aa6f1a700 1 Processor -- start 2024-05-12T03:59:25.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.689+0000 7f4aa6f1a700 1 -- start start 2024-05-12T03:59:25.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.690+0000 7f4aa6f1a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4aa00fbcb0 0x7f4aa010b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:59:25.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.690+0000 7f4aa6f1a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4aa00fcf60 0x7f4aa010bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:59:25.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.690+0000 7f4aa6f1a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa010c180 con 0x7f4aa00fbcb0 2024-05-12T03:59:25.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.690+0000 7f4aa6f1a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa010c9d0 con 0x7f4aa00fcf60 2024-05-12T03:59:25.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.690+0000 7f4aa4cb6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4aa00fbcb0 0x7f4aa010b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:59:25.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.690+0000 7f4aa4cb6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4aa00fbcb0 0x7f4aa010b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33756/0 (socket says 172.21.15.32:33756) 2024-05-12T03:59:25.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.690+0000 7f4aa4cb6700 1 -- 172.21.15.32:0/2290746255 learned_addr learned my addr 172.21.15.32:0/2290746255 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:59:25.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.690+0000 7f4a9ffff700 1 --2- 172.21.15.32:0/2290746255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4aa00fcf60 0x7f4aa010bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:59:25.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.690+0000 7f4aa4cb6700 1 -- 172.21.15.32:0/2290746255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4aa00fcf60 msgr2=0x7f4aa010bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:59:25.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.690+0000 7f4aa4cb6700 1 --2- 172.21.15.32:0/2290746255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4aa00fcf60 0x7f4aa010bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:25.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.690+0000 7f4aa4cb6700 1 -- 172.21.15.32:0/2290746255 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a94005040 con 0x7f4aa00fbcb0 2024-05-12T03:59:25.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.690+0000 7f4aa4cb6700 1 --2- 172.21.15.32:0/2290746255 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4aa00fbcb0 0x7f4aa010b640 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4a940137b0 tx=0x7f4a94009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:59:25.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.691+0000 7f4a9d7fa700 1 -- 172.21.15.32:0/2290746255 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a9400e070 con 0x7f4aa00fbcb0 2024-05-12T03:59:25.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.691+0000 7f4a9d7fa700 1 -- 172.21.15.32:0/2290746255 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4a9401c400 con 0x7f4aa00fbcb0 2024-05-12T03:59:25.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.691+0000 7f4aa6f1a700 1 -- 172.21.15.32:0/2290746255 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4aa010cc50 con 0x7f4aa00fbcb0 2024-05-12T03:59:25.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.691+0000 7f4aa6f1a700 1 -- 172.21.15.32:0/2290746255 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4aa010d130 con 0x7f4aa00fbcb0 2024-05-12T03:59:25.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.691+0000 7f4a9d7fa700 1 -- 172.21.15.32:0/2290746255 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a9400b650 con 0x7f4aa00fbcb0 2024-05-12T03:59:25.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.692+0000 7f4aa6f1a700 1 -- 172.21.15.32:0/2290746255 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a840052e0 con 0x7f4aa00fbcb0 2024-05-12T03:59:25.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.693+0000 7f4a9d7fa700 1 -- 172.21.15.32:0/2290746255 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f4a9400b8c0 con 0x7f4aa00fbcb0 2024-05-12T03:59:25.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.693+0000 7f4a9d7fa700 1 --2- 172.21.15.32:0/2290746255 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f4a9006d6a0 0x7f4a9006fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:59:25.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.693+0000 7f4a9d7fa700 1 -- 172.21.15.32:0/2290746255 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f4a94088680 con 0x7f4aa00fbcb0 2024-05-12T03:59:25.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.693+0000 7f4a9ffff700 1 --2- 172.21.15.32:0/2290746255 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f4a9006d6a0 0x7f4a9006fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:59:25.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.694+0000 7f4a9ffff700 1 --2- 172.21.15.32:0/2290746255 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f4a9006d6a0 0x7f4a9006fb50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c009b80 tx=0x7f4a8c00b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:59:25.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:25.698+0000 7f4a9d7fa700 1 -- 172.21.15.32:0/2290746255 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f4a94058490 con 0x7f4aa00fbcb0 2024-05-12T03:59:26.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.013+0000 7f4aa6f1a700 1 -- 172.21.15.32:0/2290746255 --> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4a84000bf0 con 0x7f4a9006d6a0 2024-05-12T03:59:26.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.026+0000 7f4a9d7fa700 1 -- 172.21.15.32:0/2290746255 <== mgr.14208 v2:172.21.15.32:6800/4089797100 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3366 (secure 0 0 0) 0x7f4a84000bf0 con 0x7f4a9006d6a0 2024-05-12T03:59:26.026 INFO:teuthology.orchestra.run.smithi032.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-12T03:59:26.026 INFO:teuthology.orchestra.run.smithi032.stdout:alertmanager.smithi032 smithi032 *:9093,9094 running (5m) 67s ago 7m 27.7M - 0.25.0 c8568f914cd2 5c80dfe17d2e 2024-05-12T03:59:26.026 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi032 smithi032 running (7m) 67s ago 7m 6651k - 17.2.7-939-gc190bdee 284803f7d9cf a478874e9dfa 2024-05-12T03:59:26.026 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi033 smithi033 running (6m) 74s ago 6m 6651k - 17.2.7-939-gc190bdee 284803f7d9cf 34c62d004c7e 2024-05-12T03:59:26.026 INFO:teuthology.orchestra.run.smithi032.stdout:grafana.smithi032 smithi032 *:3000 running (5m) 67s ago 7m 85.7M - 9.4.7 954c08fa6188 ac9076563478 2024-05-12T03:59:26.026 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.ialtqb smithi032 running (94s) 67s ago 93s 10.4M - 17.2.7-939-gc190bdee 284803f7d9cf 0683abf00e3e 2024-05-12T03:59:26.026 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.qpsawh smithi032 running (87s) 67s ago 87s 16.6M - 17.2.7-939-gc190bdee 284803f7d9cf 95a7db5c4fcd 2024-05-12T03:59:26.026 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi033.hglwml smithi033 running (84s) 74s ago 84s 10.4M - 17.2.7-939-gc190bdee 284803f7d9cf c97902b05025 2024-05-12T03:59:26.026 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi033.jgbjhk smithi033 running (91s) 74s ago 90s 15.2M - 17.2.7-939-gc190bdee 284803f7d9cf af97052cb97a 2024-05-12T03:59:26.026 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi032.ukatkq smithi032 *:9283 running (8m) 67s ago 8m 483M - 17.2.7-939-gc190bdee 284803f7d9cf d9763dc47404 2024-05-12T03:59:26.027 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi033.pcvxwy smithi033 *:8443,9283 running (6m) 74s ago 6m 434M - 17.2.7-939-gc190bdee 284803f7d9cf 70d285a35adb 2024-05-12T03:59:26.027 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi032 smithi032 running (9m) 67s ago 9m 45.4M 2048M 17.2.7-939-gc190bdee 284803f7d9cf d7fea430c6e2 2024-05-12T03:59:26.027 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi033 smithi033 running (6m) 74s ago 6m 34.4M 2048M 17.2.7-939-gc190bdee 284803f7d9cf 707654d85748 2024-05-12T03:59:26.027 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi032 smithi032 *:9100 running (7m) 67s ago 7m 17.8M - 1.5.0 0da6a335fe13 6aa48ce262de 2024-05-12T03:59:26.027 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi033 smithi033 *:9100 running (6m) 74s ago 6m 16.9M - 1.5.0 0da6a335fe13 1144caed00e9 2024-05-12T03:59:26.027 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0 smithi032 running (5m) 67s ago 5m 44.7M 4300M 17.2.7-939-gc190bdee 284803f7d9cf 75c78a47f8e5 2024-05-12T03:59:26.027 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1 smithi032 running (5m) 67s ago 5m 48.5M 4300M 17.2.7-939-gc190bdee 284803f7d9cf 87cfd247d8d6 2024-05-12T03:59:26.027 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2 smithi032 running (4m) 67s ago 4m 43.8M 4300M 17.2.7-939-gc190bdee 284803f7d9cf 7aef7affb8ba 2024-05-12T03:59:26.027 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3 smithi033 running (4m) 74s ago 4m 44.0M 5324M 17.2.7-939-gc190bdee 284803f7d9cf 3a92b5e3348e 2024-05-12T03:59:26.027 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4 smithi033 running (3m) 74s ago 3m 44.2M 5324M 17.2.7-939-gc190bdee 284803f7d9cf 0896268b4591 2024-05-12T03:59:26.027 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5 smithi033 running (3m) 74s ago 3m 40.7M 5324M 17.2.7-939-gc190bdee 284803f7d9cf 57e65a64c821 2024-05-12T03:59:26.027 INFO:teuthology.orchestra.run.smithi032.stdout:prometheus.smithi032 smithi032 *:9095 running (5m) 67s ago 6m 52.6M - 2.43.0 a07b618ecd1d 0b3ce3323a49 2024-05-12T03:59:26.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.029+0000 7f4aa6f1a700 1 -- 172.21.15.32:0/2290746255 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f4a9006d6a0 msgr2=0x7f4a9006fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:59:26.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.029+0000 7f4aa6f1a700 1 --2- 172.21.15.32:0/2290746255 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f4a9006d6a0 0x7f4a9006fb50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c009b80 tx=0x7f4a8c00b040 comp rx=0 tx=0).stop 2024-05-12T03:59:26.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.029+0000 7f4aa6f1a700 1 -- 172.21.15.32:0/2290746255 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4aa00fbcb0 msgr2=0x7f4aa010b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:59:26.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.029+0000 7f4aa6f1a700 1 --2- 172.21.15.32:0/2290746255 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4aa00fbcb0 0x7f4aa010b640 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4a940137b0 tx=0x7f4a94009e30 comp rx=0 tx=0).stop 2024-05-12T03:59:26.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.029+0000 7f4aa6f1a700 1 -- 172.21.15.32:0/2290746255 shutdown_connections 2024-05-12T03:59:26.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.029+0000 7f4aa6f1a700 1 --2- 172.21.15.32:0/2290746255 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f4a9006d6a0 0x7f4a9006fb50 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:26.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.029+0000 7f4aa6f1a700 1 --2- 172.21.15.32:0/2290746255 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4aa00fbcb0 0x7f4aa010b640 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:26.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.029+0000 7f4aa6f1a700 1 --2- 172.21.15.32:0/2290746255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4aa00fcf60 0x7f4aa010bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:26.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.029+0000 7f4aa6f1a700 1 -- 172.21.15.32:0/2290746255 >> 172.21.15.32:0/2290746255 conn(0x7f4aa00f7430 msgr2=0x7f4aa00f8b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:59:26.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.029+0000 7f4aa6f1a700 1 -- 172.21.15.32:0/2290746255 shutdown_connections 2024-05-12T03:59:26.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.029+0000 7f4aa6f1a700 1 -- 172.21.15.32:0/2290746255 wait complete. 2024-05-12T03:59:26.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.142+0000 7fd26f6f4700 1 -- 172.21.15.32:0/1649410211 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2680662c0 msgr2=0x7fd2680ff9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:59:26.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.142+0000 7fd26f6f4700 1 --2- 172.21.15.32:0/1649410211 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2680662c0 0x7fd2680ff9a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd264004940 tx=0x7fd264009e30 comp rx=0 tx=0).stop 2024-05-12T03:59:26.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.142+0000 7fd26f6f4700 1 -- 172.21.15.32:0/1649410211 shutdown_connections 2024-05-12T03:59:26.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.142+0000 7fd26f6f4700 1 --2- 172.21.15.32:0/1649410211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2680ffee0 0x7fd268102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:26.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.142+0000 7fd26f6f4700 1 --2- 172.21.15.32:0/1649410211 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2680662c0 0x7fd2680ff9a0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:26.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.142+0000 7fd26f6f4700 1 -- 172.21.15.32:0/1649410211 >> 172.21.15.32:0/1649410211 conn(0x7fd2680f7430 msgr2=0x7fd2680f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:59:26.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.142+0000 7fd26f6f4700 1 -- 172.21.15.32:0/1649410211 shutdown_connections 2024-05-12T03:59:26.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.142+0000 7fd26f6f4700 1 -- 172.21.15.32:0/1649410211 wait complete. 2024-05-12T03:59:26.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.143+0000 7fd26f6f4700 1 Processor -- start 2024-05-12T03:59:26.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.143+0000 7fd26f6f4700 1 -- start start 2024-05-12T03:59:26.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.143+0000 7fd26f6f4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2680662c0 0x7fd26810d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:59:26.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.143+0000 7fd26f6f4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2680ffee0 0x7fd26810dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:59:26.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.143+0000 7fd26f6f4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd26810e4a0 con 0x7fd2680662c0 2024-05-12T03:59:26.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.143+0000 7fd26f6f4700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd26810ec50 con 0x7fd2680ffee0 2024-05-12T03:59:26.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.143+0000 7fd26d490700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2680662c0 0x7fd26810d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:59:26.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.143+0000 7fd26d490700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2680662c0 0x7fd26810d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33774/0 (socket says 172.21.15.32:33774) 2024-05-12T03:59:26.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.143+0000 7fd26d490700 1 -- 172.21.15.32:0/1390746432 learned_addr learned my addr 172.21.15.32:0/1390746432 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:59:26.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.143+0000 7fd26cc8f700 1 --2- 172.21.15.32:0/1390746432 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2680ffee0 0x7fd26810dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:59:26.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.144+0000 7fd26d490700 1 -- 172.21.15.32:0/1390746432 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2680ffee0 msgr2=0x7fd26810dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:59:26.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.144+0000 7fd26d490700 1 --2- 172.21.15.32:0/1390746432 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2680ffee0 0x7fd26810dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:26.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.144+0000 7fd26d490700 1 -- 172.21.15.32:0/1390746432 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd264005040 con 0x7fd2680662c0 2024-05-12T03:59:26.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.144+0000 7fd26cc8f700 1 --2- 172.21.15.32:0/1390746432 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2680ffee0 0x7fd26810dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-12T03:59:26.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.144+0000 7fd26d490700 1 --2- 172.21.15.32:0/1390746432 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2680662c0 0x7fd26810d960 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd2640137b0 tx=0x7fd264009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:59:26.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.144+0000 7fd25dffb700 1 -- 172.21.15.32:0/1390746432 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd264021070 con 0x7fd2680662c0 2024-05-12T03:59:26.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.144+0000 7fd26f6f4700 1 -- 172.21.15.32:0/1390746432 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd26810eed0 con 0x7fd2680662c0 2024-05-12T03:59:26.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.144+0000 7fd25dffb700 1 -- 172.21.15.32:0/1390746432 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd26401c400 con 0x7fd2680662c0 2024-05-12T03:59:26.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.144+0000 7fd25dffb700 1 -- 172.21.15.32:0/1390746432 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd26400e070 con 0x7fd2680662c0 2024-05-12T03:59:26.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.144+0000 7fd26f6f4700 1 -- 172.21.15.32:0/1390746432 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd26810f410 con 0x7fd2680662c0 2024-05-12T03:59:26.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.145+0000 7fd26f6f4700 1 -- 172.21.15.32:0/1390746432 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd26805f5c0 con 0x7fd2680662c0 2024-05-12T03:59:26.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.146+0000 7fd25dffb700 1 -- 172.21.15.32:0/1390746432 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fd26401c9c0 con 0x7fd2680662c0 2024-05-12T03:59:26.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.147+0000 7fd25dffb700 1 --2- 172.21.15.32:0/1390746432 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fd2540692a0 0x7fd25406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:59:26.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.147+0000 7fd25dffb700 1 -- 172.21.15.32:0/1390746432 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fd264088440 con 0x7fd2680662c0 2024-05-12T03:59:26.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.147+0000 7fd26cc8f700 1 --2- 172.21.15.32:0/1390746432 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fd2540692a0 0x7fd25406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:59:26.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.148+0000 7fd26cc8f700 1 --2- 172.21.15.32:0/1390746432 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fd2540692a0 0x7fd25406b750 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd258009b80 tx=0x7fd25800b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:59:26.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.151+0000 7fd25dffb700 1 -- 172.21.15.32:0/1390746432 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd2640581b0 con 0x7fd2680662c0 2024-05-12T03:59:26.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:59:25 smithi032 ceph-mon[40008]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T03:59:26.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:59:25 smithi033 ceph-mon[49525]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T03:59:26.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.517+0000 7fd26f6f4700 1 -- 172.21.15.32:0/1390746432 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd26810f790 con 0x7fd2680662c0 2024-05-12T03:59:26.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.517+0000 7fd25dffb700 1 -- 172.21.15.32:0/1390746432 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7fd264057d40 con 0x7fd2680662c0 2024-05-12T03:59:26.518 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-05-12T03:59:26.518 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": { 2024-05-12T03:59:26.518 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 2 2024-05-12T03:59:26.518 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T03:59:26.518 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": { 2024-05-12T03:59:26.518 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 2 2024-05-12T03:59:26.518 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T03:59:26.518 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": { 2024-05-12T03:59:26.518 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 6 2024-05-12T03:59:26.518 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T03:59:26.518 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": { 2024-05-12T03:59:26.518 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 4 2024-05-12T03:59:26.519 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T03:59:26.519 INFO:teuthology.orchestra.run.smithi032.stdout: "overall": { 2024-05-12T03:59:26.519 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 14 2024-05-12T03:59:26.519 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-05-12T03:59:26.519 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-05-12T03:59:26.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.520+0000 7fd26f6f4700 1 -- 172.21.15.32:0/1390746432 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fd2540692a0 msgr2=0x7fd25406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:59:26.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.520+0000 7fd26f6f4700 1 --2- 172.21.15.32:0/1390746432 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fd2540692a0 0x7fd25406b750 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd258009b80 tx=0x7fd25800b040 comp rx=0 tx=0).stop 2024-05-12T03:59:26.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.520+0000 7fd26f6f4700 1 -- 172.21.15.32:0/1390746432 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2680662c0 msgr2=0x7fd26810d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:59:26.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.520+0000 7fd26f6f4700 1 --2- 172.21.15.32:0/1390746432 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2680662c0 0x7fd26810d960 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd2640137b0 tx=0x7fd264009e30 comp rx=0 tx=0).stop 2024-05-12T03:59:26.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.520+0000 7fd26f6f4700 1 -- 172.21.15.32:0/1390746432 shutdown_connections 2024-05-12T03:59:26.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.520+0000 7fd26f6f4700 1 --2- 172.21.15.32:0/1390746432 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fd2540692a0 0x7fd25406b750 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:26.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.520+0000 7fd26f6f4700 1 --2- 172.21.15.32:0/1390746432 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2680662c0 0x7fd26810d960 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:26.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.520+0000 7fd26f6f4700 1 --2- 172.21.15.32:0/1390746432 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2680ffee0 0x7fd26810dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:26.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.520+0000 7fd26f6f4700 1 -- 172.21.15.32:0/1390746432 >> 172.21.15.32:0/1390746432 conn(0x7fd2680f7430 msgr2=0x7fd2680f9810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:59:26.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.520+0000 7fd26f6f4700 1 -- 172.21.15.32:0/1390746432 shutdown_connections 2024-05-12T03:59:26.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.520+0000 7fd26f6f4700 1 -- 172.21.15.32:0/1390746432 wait complete. 2024-05-12T03:59:26.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.630+0000 7f32ae7f8700 1 -- 172.21.15.32:0/2917148204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32a80fbcb0 msgr2=0x7f32a80fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:59:26.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.630+0000 7f32ae7f8700 1 --2- 172.21.15.32:0/2917148204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32a80fbcb0 0x7f32a80fc0c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3298004940 tx=0x7f3298009e30 comp rx=0 tx=0).stop 2024-05-12T03:59:26.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.630+0000 7f32ae7f8700 1 -- 172.21.15.32:0/2917148204 shutdown_connections 2024-05-12T03:59:26.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.630+0000 7f32ae7f8700 1 --2- 172.21.15.32:0/2917148204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32a80fcf60 0x7f32a80fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:26.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.630+0000 7f32ae7f8700 1 --2- 172.21.15.32:0/2917148204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32a80fbcb0 0x7f32a80fc0c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:26.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.630+0000 7f32ae7f8700 1 -- 172.21.15.32:0/2917148204 >> 172.21.15.32:0/2917148204 conn(0x7f32a80f7430 msgr2=0x7f32a80f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:59:26.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.630+0000 7f32ae7f8700 1 -- 172.21.15.32:0/2917148204 shutdown_connections 2024-05-12T03:59:26.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.631+0000 7f32ae7f8700 1 -- 172.21.15.32:0/2917148204 wait complete. 2024-05-12T03:59:26.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.631+0000 7f32ae7f8700 1 Processor -- start 2024-05-12T03:59:26.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.631+0000 7f32ae7f8700 1 -- start start 2024-05-12T03:59:26.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.632+0000 7f32ae7f8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32a80fbcb0 0x7f32a8103df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:59:26.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.632+0000 7f32ae7f8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32a80fcf60 0x7f32a81022c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:59:26.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.632+0000 7f32ae7f8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a81114e0 con 0x7f32a80fbcb0 2024-05-12T03:59:26.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.632+0000 7f32ae7f8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a8111650 con 0x7f32a80fcf60 2024-05-12T03:59:26.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.632+0000 7f32a7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32a80fbcb0 0x7f32a8103df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:59:26.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.632+0000 7f32a7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32a80fbcb0 0x7f32a8103df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33800/0 (socket says 172.21.15.32:33800) 2024-05-12T03:59:26.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.632+0000 7f32a7fff700 1 -- 172.21.15.32:0/2759184195 learned_addr learned my addr 172.21.15.32:0/2759184195 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:59:26.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.632+0000 7f32a77fe700 1 --2- 172.21.15.32:0/2759184195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32a80fcf60 0x7f32a81022c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:59:26.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.632+0000 7f32a77fe700 1 -- 172.21.15.32:0/2759184195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32a80fbcb0 msgr2=0x7f32a8103df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:59:26.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.632+0000 7f32a77fe700 1 --2- 172.21.15.32:0/2759184195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32a80fbcb0 0x7f32a8103df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:26.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.632+0000 7f32a77fe700 1 -- 172.21.15.32:0/2759184195 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3298005040 con 0x7f32a80fcf60 2024-05-12T03:59:26.633 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.632+0000 7f32a77fe700 1 --2- 172.21.15.32:0/2759184195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32a80fcf60 0x7f32a81022c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f329c009fd0 tx=0x7f329c00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:59:26.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.633+0000 7f32a4ff9700 1 -- 172.21.15.32:0/2759184195 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f329c004e90 con 0x7f32a80fcf60 2024-05-12T03:59:26.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.633+0000 7f32a4ff9700 1 -- 172.21.15.32:0/2759184195 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f329c00f040 con 0x7f32a80fcf60 2024-05-12T03:59:26.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.633+0000 7f32ae7f8700 1 -- 172.21.15.32:0/2759184195 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32a81118d0 con 0x7f32a80fcf60 2024-05-12T03:59:26.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.633+0000 7f32a4ff9700 1 -- 172.21.15.32:0/2759184195 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f329c00c9b0 con 0x7f32a80fcf60 2024-05-12T03:59:26.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.633+0000 7f32ae7f8700 1 -- 172.21.15.32:0/2759184195 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32a8111e90 con 0x7f32a80fcf60 2024-05-12T03:59:26.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.634+0000 7f32ae7f8700 1 -- 172.21.15.32:0/2759184195 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32a805f5c0 con 0x7f32a80fcf60 2024-05-12T03:59:26.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.635+0000 7f32a4ff9700 1 -- 172.21.15.32:0/2759184195 <== mon.1 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f329c007cc0 con 0x7f32a80fcf60 2024-05-12T03:59:26.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.636+0000 7f32a4ff9700 1 --2- 172.21.15.32:0/2759184195 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f32900692a0 0x7f329006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:59:26.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.636+0000 7f32a4ff9700 1 -- 172.21.15.32:0/2759184195 <== mon.1 v2:172.21.15.33:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f329c087210 con 0x7f32a80fcf60 2024-05-12T03:59:26.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.636+0000 7f32a7fff700 1 --2- 172.21.15.32:0/2759184195 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f32900692a0 0x7f329006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:59:26.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.636+0000 7f32a7fff700 1 --2- 172.21.15.32:0/2759184195 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f32900692a0 0x7f329006b750 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f329800b0d0 tx=0x7f3298009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:59:26.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.643+0000 7f32a4ff9700 1 -- 172.21.15.32:0/2759184195 <== mon.1 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f329c0534d0 con 0x7f32a80fcf60 2024-05-12T03:59:26.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.886+0000 7f32ae7f8700 1 -- 172.21.15.32:0/2759184195 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f32a8105760 con 0x7f32a80fcf60 2024-05-12T03:59:26.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.886+0000 7f32a4ff9700 1 -- 172.21.15.32:0/2759184195 <== mon.1 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1828 (secure 0 0 0) 0x7f329c056af0 con 0x7f32a80fcf60 2024-05-12T03:59:26.887 INFO:teuthology.orchestra.run.smithi032.stdout:e18 2024-05-12T03:59:26.887 INFO:teuthology.orchestra.run.smithi032.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-12T03:59:26.887 INFO:teuthology.orchestra.run.smithi032.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-12T03:59:26.887 INFO:teuthology.orchestra.run.smithi032.stdout:legacy client fscid: 1 2024-05-12T03:59:26.887 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:59:26.887 INFO:teuthology.orchestra.run.smithi032.stdout:Filesystem 'cephfs' (1) 2024-05-12T03:59:26.887 INFO:teuthology.orchestra.run.smithi032.stdout:fs_name cephfs 2024-05-12T03:59:26.887 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 18 2024-05-12T03:59:26.887 INFO:teuthology.orchestra.run.smithi032.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-12T03:59:26.887 INFO:teuthology.orchestra.run.smithi032.stdout:created 2024-05-12T03:57:48.904614+0000 2024-05-12T03:59:26.888 INFO:teuthology.orchestra.run.smithi032.stdout:modified 2024-05-12T03:58:06.821591+0000 2024-05-12T03:59:26.888 INFO:teuthology.orchestra.run.smithi032.stdout:tableserver 0 2024-05-12T03:59:26.888 INFO:teuthology.orchestra.run.smithi032.stdout:root 0 2024-05-12T03:59:26.888 INFO:teuthology.orchestra.run.smithi032.stdout:session_timeout 60 2024-05-12T03:59:26.888 INFO:teuthology.orchestra.run.smithi032.stdout:session_autoclose 300 2024-05-12T03:59:26.888 INFO:teuthology.orchestra.run.smithi032.stdout:max_file_size 1099511627776 2024-05-12T03:59:26.888 INFO:teuthology.orchestra.run.smithi032.stdout:required_client_features {} 2024-05-12T03:59:26.888 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure 0 2024-05-12T03:59:26.888 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure_osd_epoch 46 2024-05-12T03:59:26.888 INFO:teuthology.orchestra.run.smithi032.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-12T03:59:26.888 INFO:teuthology.orchestra.run.smithi032.stdout:max_mds 1 2024-05-12T03:59:26.888 INFO:teuthology.orchestra.run.smithi032.stdout:in 0 2024-05-12T03:59:26.888 INFO:teuthology.orchestra.run.smithi032.stdout:up {0=14474} 2024-05-12T03:59:26.888 INFO:teuthology.orchestra.run.smithi032.stdout:failed 2024-05-12T03:59:26.888 INFO:teuthology.orchestra.run.smithi032.stdout:damaged 2024-05-12T03:59:26.888 INFO:teuthology.orchestra.run.smithi032.stdout:stopped 2024-05-12T03:59:26.889 INFO:teuthology.orchestra.run.smithi032.stdout:data_pools [3] 2024-05-12T03:59:26.889 INFO:teuthology.orchestra.run.smithi032.stdout:metadata_pool 2 2024-05-12T03:59:26.889 INFO:teuthology.orchestra.run.smithi032.stdout:inline_data enabled 2024-05-12T03:59:26.889 INFO:teuthology.orchestra.run.smithi032.stdout:balancer 2024-05-12T03:59:26.889 INFO:teuthology.orchestra.run.smithi032.stdout:standby_count_wanted 1 2024-05-12T03:59:26.889 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.qpsawh{0:14474} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.32:6828/3100155296,v1:172.21.15.32:6829/3100155296] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T03:59:26.889 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi033.jgbjhk{0:24267} state up:standby-replay seq 3 join_fscid=1 addr [v2:172.21.15.33:6824/55417471,v1:172.21.15.33:6825/55417471] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T03:59:26.889 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:59:26.889 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:59:26.889 INFO:teuthology.orchestra.run.smithi032.stdout:Standby daemons: 2024-05-12T03:59:26.889 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T03:59:26.889 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi033.hglwml{-1:24283} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/433858238,v1:172.21.15.33:6827/433858238] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T03:59:26.889 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.ialtqb{-1:24287} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.32:6826/3703132329,v1:172.21.15.32:6827/3703132329] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T03:59:26.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.889+0000 7f32ae7f8700 1 -- 172.21.15.32:0/2759184195 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f32900692a0 msgr2=0x7f329006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:59:26.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.889+0000 7f32ae7f8700 1 --2- 172.21.15.32:0/2759184195 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f32900692a0 0x7f329006b750 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f329800b0d0 tx=0x7f3298009e30 comp rx=0 tx=0).stop 2024-05-12T03:59:26.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.889+0000 7f32ae7f8700 1 -- 172.21.15.32:0/2759184195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32a80fcf60 msgr2=0x7f32a81022c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:59:26.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.889+0000 7f32ae7f8700 1 --2- 172.21.15.32:0/2759184195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32a80fcf60 0x7f32a81022c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f329c009fd0 tx=0x7f329c00b040 comp rx=0 tx=0).stop 2024-05-12T03:59:26.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.889+0000 7f32ae7f8700 1 -- 172.21.15.32:0/2759184195 shutdown_connections 2024-05-12T03:59:26.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.889+0000 7f32ae7f8700 1 --2- 172.21.15.32:0/2759184195 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f32900692a0 0x7f329006b750 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:26.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.889+0000 7f32ae7f8700 1 --2- 172.21.15.32:0/2759184195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32a80fbcb0 0x7f32a8103df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:26.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.889+0000 7f32ae7f8700 1 --2- 172.21.15.32:0/2759184195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32a80fcf60 0x7f32a81022c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:26.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.889+0000 7f32ae7f8700 1 -- 172.21.15.32:0/2759184195 >> 172.21.15.32:0/2759184195 conn(0x7f32a80f7430 msgr2=0x7f32a80f8b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:59:26.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.889+0000 7f32ae7f8700 1 -- 172.21.15.32:0/2759184195 shutdown_connections 2024-05-12T03:59:26.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:26.889+0000 7f32ae7f8700 1 -- 172.21.15.32:0/2759184195 wait complete. 2024-05-12T03:59:26.891 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 18 2024-05-12T03:59:27.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.025+0000 7f14d1a50700 1 -- 172.21.15.32:0/3954546178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14cc0fe080 msgr2=0x7f14cc0fe4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:59:27.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.025+0000 7f14d1a50700 1 --2- 172.21.15.32:0/3954546178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14cc0fe080 0x7f14cc0fe4f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f14c0004940 tx=0x7f14c0009e30 comp rx=0 tx=0).stop 2024-05-12T03:59:27.025 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.025+0000 7f14d1a50700 1 -- 172.21.15.32:0/3954546178 shutdown_connections 2024-05-12T03:59:27.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.025+0000 7f14d1a50700 1 --2- 172.21.15.32:0/3954546178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14cc0fe080 0x7f14cc0fe4f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:27.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.025+0000 7f14d1a50700 1 --2- 172.21.15.32:0/3954546178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14cc0fcf10 0x7f14cc0fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:27.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.025+0000 7f14d1a50700 1 -- 172.21.15.32:0/3954546178 >> 172.21.15.32:0/3954546178 conn(0x7f14cc0f8560 msgr2=0x7f14cc0fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:59:27.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.025+0000 7f14d1a50700 1 -- 172.21.15.32:0/3954546178 shutdown_connections 2024-05-12T03:59:27.026 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.025+0000 7f14d1a50700 1 -- 172.21.15.32:0/3954546178 wait complete. 2024-05-12T03:59:27.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.026+0000 7f14d1a50700 1 Processor -- start 2024-05-12T03:59:27.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.026+0000 7f14d1a50700 1 -- start start 2024-05-12T03:59:27.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.027+0000 7f14d1a50700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14cc0fcf10 0x7f14cc10d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:59:27.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.027+0000 7f14d1a50700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14cc0fe080 0x7f14cc10de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:59:27.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.027+0000 7f14d1a50700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14cc10e4e0 con 0x7f14cc0fe080 2024-05-12T03:59:27.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.027+0000 7f14d1a50700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14cc10ecd0 con 0x7f14cc0fcf10 2024-05-12T03:59:27.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.027+0000 7f14ca7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14cc0fe080 0x7f14cc10de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:59:27.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.027+0000 7f14caffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14cc0fcf10 0x7f14cc10d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:59:27.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.027+0000 7f14ca7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14cc0fe080 0x7f14cc10de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33814/0 (socket says 172.21.15.32:33814) 2024-05-12T03:59:27.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.027+0000 7f14ca7fc700 1 -- 172.21.15.32:0/1398374095 learned_addr learned my addr 172.21.15.32:0/1398374095 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:59:27.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.027+0000 7f14ca7fc700 1 -- 172.21.15.32:0/1398374095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14cc0fcf10 msgr2=0x7f14cc10d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:59:27.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.027+0000 7f14ca7fc700 1 --2- 172.21.15.32:0/1398374095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14cc0fcf10 0x7f14cc10d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:27.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.027+0000 7f14ca7fc700 1 -- 172.21.15.32:0/1398374095 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14c0005040 con 0x7f14cc0fe080 2024-05-12T03:59:27.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.027+0000 7f14ca7fc700 1 --2- 172.21.15.32:0/1398374095 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14cc0fe080 0x7f14cc10de50 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f14c000b900 tx=0x7f14c0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:59:27.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.028+0000 7f14b37fe700 1 -- 172.21.15.32:0/1398374095 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f14c000e040 con 0x7f14cc0fe080 2024-05-12T03:59:27.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.028+0000 7f14d1a50700 1 -- 172.21.15.32:0/1398374095 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14cc10ef50 con 0x7f14cc0fe080 2024-05-12T03:59:27.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.028+0000 7f14d1a50700 1 -- 172.21.15.32:0/1398374095 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14cc10f510 con 0x7f14cc0fe080 2024-05-12T03:59:27.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.028+0000 7f14b37fe700 1 -- 172.21.15.32:0/1398374095 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f14c000bc50 con 0x7f14cc0fe080 2024-05-12T03:59:27.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.028+0000 7f14b37fe700 1 -- 172.21.15.32:0/1398374095 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f14c001c630 con 0x7f14cc0fe080 2024-05-12T03:59:27.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.029+0000 7f14d1a50700 1 -- 172.21.15.32:0/1398374095 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14b80052e0 con 0x7f14cc0fe080 2024-05-12T03:59:27.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.033+0000 7f14b37fe700 1 -- 172.21.15.32:0/1398374095 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f14c0013420 con 0x7f14cc0fe080 2024-05-12T03:59:27.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.034+0000 7f14b37fe700 1 --2- 172.21.15.32:0/1398374095 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f14b4069370 0x7f14b406b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:59:27.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.034+0000 7f14b37fe700 1 -- 172.21.15.32:0/1398374095 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f14c0088e50 con 0x7f14cc0fe080 2024-05-12T03:59:27.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.034+0000 7f14b37fe700 1 -- 172.21.15.32:0/1398374095 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f14c0089240 con 0x7f14cc0fe080 2024-05-12T03:59:27.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.034+0000 7f14caffd700 1 --2- 172.21.15.32:0/1398374095 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f14b4069370 0x7f14b406b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:59:27.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.035+0000 7f14caffd700 1 --2- 172.21.15.32:0/1398374095 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f14b4069370 0x7f14b406b820 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f14bc009bf0 tx=0x7f14bc0091a0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:59:27.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:59:26 smithi033 ceph-mon[49525]: from='client.14594 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T03:59:27.164 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:59:26 smithi033 ceph-mon[49525]: pgmap v237: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T03:59:27.164 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:59:26 smithi033 ceph-mon[49525]: from='client.14598 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T03:59:27.164 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:59:26 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/1390746432' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T03:59:27.164 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:59:26 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/2759184195' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-12T03:59:27.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:59:26 smithi032 ceph-mon[40008]: from='client.14594 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T03:59:27.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:59:26 smithi032 ceph-mon[40008]: pgmap v237: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T03:59:27.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:59:26 smithi032 ceph-mon[40008]: from='client.14598 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T03:59:27.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:59:26 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/1390746432' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T03:59:27.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:59:26 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/2759184195' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-12T03:59:27.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.316+0000 7f14d1a50700 1 -- 172.21.15.32:0/1398374095 --> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f14b8000bf0 con 0x7f14b4069370 2024-05-12T03:59:27.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.318+0000 7f14b37fe700 1 -- 172.21.15.32:0/1398374095 <== mgr.14208 v2:172.21.15.32:6800/4089797100 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7f14b8000bf0 con 0x7f14b4069370 2024-05-12T03:59:27.318 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-05-12T03:59:27.318 INFO:teuthology.orchestra.run.smithi032.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:19c9734256aa43a60e203feb242f50bf8b7f58666d520536af43cc4fee96173a", 2024-05-12T03:59:27.318 INFO:teuthology.orchestra.run.smithi032.stdout: "in_progress": true, 2024-05-12T03:59:27.318 INFO:teuthology.orchestra.run.smithi032.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-12T03:59:27.318 INFO:teuthology.orchestra.run.smithi032.stdout: "services_complete": [], 2024-05-12T03:59:27.318 INFO:teuthology.orchestra.run.smithi032.stdout: "progress": "0/21 daemons upgraded", 2024-05-12T03:59:27.318 INFO:teuthology.orchestra.run.smithi032.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:19c9734256aa43a60e203feb242f50bf8b7f58666d520536af43cc4fee96173a image on host smithi033", 2024-05-12T03:59:27.319 INFO:teuthology.orchestra.run.smithi032.stdout: "is_paused": false 2024-05-12T03:59:27.319 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-05-12T03:59:27.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.320+0000 7f14d1a50700 1 -- 172.21.15.32:0/1398374095 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f14b4069370 msgr2=0x7f14b406b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:59:27.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.320+0000 7f14d1a50700 1 --2- 172.21.15.32:0/1398374095 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f14b4069370 0x7f14b406b820 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f14bc009bf0 tx=0x7f14bc0091a0 comp rx=0 tx=0).stop 2024-05-12T03:59:27.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.320+0000 7f14d1a50700 1 -- 172.21.15.32:0/1398374095 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14cc0fe080 msgr2=0x7f14cc10de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:59:27.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.320+0000 7f14d1a50700 1 --2- 172.21.15.32:0/1398374095 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14cc0fe080 0x7f14cc10de50 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f14c000b900 tx=0x7f14c0009e30 comp rx=0 tx=0).stop 2024-05-12T03:59:27.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.320+0000 7f14d1a50700 1 -- 172.21.15.32:0/1398374095 shutdown_connections 2024-05-12T03:59:27.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.320+0000 7f14d1a50700 1 --2- 172.21.15.32:0/1398374095 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f14b4069370 0x7f14b406b820 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:27.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.320+0000 7f14d1a50700 1 --2- 172.21.15.32:0/1398374095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14cc0fcf10 0x7f14cc10d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:27.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.320+0000 7f14d1a50700 1 --2- 172.21.15.32:0/1398374095 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14cc0fe080 0x7f14cc10de50 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:27.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.320+0000 7f14d1a50700 1 -- 172.21.15.32:0/1398374095 >> 172.21.15.32:0/1398374095 conn(0x7f14cc0f8560 msgr2=0x7f14cc0f9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:59:27.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.320+0000 7f14d1a50700 1 -- 172.21.15.32:0/1398374095 shutdown_connections 2024-05-12T03:59:27.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.320+0000 7f14d1a50700 1 -- 172.21.15.32:0/1398374095 wait complete. 2024-05-12T03:59:27.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.434+0000 7f38a4a7b700 1 -- 172.21.15.32:0/4060051252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38a00fbcb0 msgr2=0x7f38a00fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:59:27.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.434+0000 7f38a4a7b700 1 --2- 172.21.15.32:0/4060051252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38a00fbcb0 0x7f38a00fc0c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f3890004940 tx=0x7f3890009e30 comp rx=0 tx=0).stop 2024-05-12T03:59:27.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.434+0000 7f38a4a7b700 1 -- 172.21.15.32:0/4060051252 shutdown_connections 2024-05-12T03:59:27.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.434+0000 7f38a4a7b700 1 --2- 172.21.15.32:0/4060051252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38a00fcf60 0x7f38a00fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:27.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.434+0000 7f38a4a7b700 1 --2- 172.21.15.32:0/4060051252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38a00fbcb0 0x7f38a00fc0c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:27.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.434+0000 7f38a4a7b700 1 -- 172.21.15.32:0/4060051252 >> 172.21.15.32:0/4060051252 conn(0x7f38a00f7430 msgr2=0x7f38a00f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:59:27.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.434+0000 7f38a4a7b700 1 -- 172.21.15.32:0/4060051252 shutdown_connections 2024-05-12T03:59:27.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.434+0000 7f38a4a7b700 1 -- 172.21.15.32:0/4060051252 wait complete. 2024-05-12T03:59:27.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.434+0000 7f38a4a7b700 1 Processor -- start 2024-05-12T03:59:27.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.435+0000 7f38a4a7b700 1 -- start start 2024-05-12T03:59:27.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.435+0000 7f38a4a7b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38a00fbcb0 0x7f38a0103d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:59:27.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.435+0000 7f38a4a7b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38a00fcf60 0x7f38a01022c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:59:27.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.435+0000 7f38a4a7b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a0102850 con 0x7f38a00fcf60 2024-05-12T03:59:27.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.435+0000 7f38a4a7b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a0102990 con 0x7f38a00fbcb0 2024-05-12T03:59:27.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.435+0000 7f389dd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38a00fcf60 0x7f38a01022c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:59:27.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.435+0000 7f389dd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38a00fcf60 0x7f38a01022c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33838/0 (socket says 172.21.15.32:33838) 2024-05-12T03:59:27.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.435+0000 7f389dd9b700 1 -- 172.21.15.32:0/2853026894 learned_addr learned my addr 172.21.15.32:0/2853026894 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:59:27.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.435+0000 7f389dd9b700 1 -- 172.21.15.32:0/2853026894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38a00fbcb0 msgr2=0x7f38a0103d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:59:27.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.435+0000 7f389e59c700 1 --2- 172.21.15.32:0/2853026894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38a00fbcb0 0x7f38a0103d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:59:27.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.435+0000 7f389dd9b700 1 --2- 172.21.15.32:0/2853026894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38a00fbcb0 0x7f38a0103d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:27.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.435+0000 7f389dd9b700 1 -- 172.21.15.32:0/2853026894 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3890005040 con 0x7f38a00fcf60 2024-05-12T03:59:27.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.435+0000 7f389e59c700 1 --2- 172.21.15.32:0/2853026894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38a00fbcb0 0x7f38a0103d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-12T03:59:27.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.436+0000 7f389dd9b700 1 --2- 172.21.15.32:0/2853026894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38a00fcf60 0x7f38a01022c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3894002a50 tx=0x7f38940046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:59:27.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.436+0000 7f388effd700 1 -- 172.21.15.32:0/2853026894 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3894007cc0 con 0x7f38a00fcf60 2024-05-12T03:59:27.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.436+0000 7f38a4a7b700 1 -- 172.21.15.32:0/2853026894 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38a0102c40 con 0x7f38a00fcf60 2024-05-12T03:59:27.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.436+0000 7f388effd700 1 -- 172.21.15.32:0/2853026894 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f389400f040 con 0x7f38a00fcf60 2024-05-12T03:59:27.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.436+0000 7f388effd700 1 -- 172.21.15.32:0/2853026894 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3894009540 con 0x7f38a00fcf60 2024-05-12T03:59:27.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.436+0000 7f38a4a7b700 1 -- 172.21.15.32:0/2853026894 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38a0115930 con 0x7f38a00fcf60 2024-05-12T03:59:27.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.437+0000 7f38a4a7b700 1 -- 172.21.15.32:0/2853026894 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38a005f5c0 con 0x7f38a00fcf60 2024-05-12T03:59:27.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.438+0000 7f388effd700 1 -- 172.21.15.32:0/2853026894 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f38940096a0 con 0x7f38a00fcf60 2024-05-12T03:59:27.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.439+0000 7f388effd700 1 --2- 172.21.15.32:0/2853026894 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f38880692a0 0x7f388806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:59:27.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.439+0000 7f388effd700 1 -- 172.21.15.32:0/2853026894 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f3894086d90 con 0x7f38a00fcf60 2024-05-12T03:59:27.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.439+0000 7f389e59c700 1 --2- 172.21.15.32:0/2853026894 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f38880692a0 0x7f388806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:59:27.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.440+0000 7f389e59c700 1 --2- 172.21.15.32:0/2853026894 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f38880692a0 0x7f388806b750 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3890009fd0 tx=0x7f3890009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:59:27.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.447+0000 7f388effd700 1 -- 172.21.15.32:0/2853026894 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f3894052ff0 con 0x7f38a00fcf60 2024-05-12T03:59:27.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.728+0000 7f38a4a7b700 1 -- 172.21.15.32:0/2853026894 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f38a004bed0 con 0x7f38a00fcf60 2024-05-12T03:59:27.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.728+0000 7f388effd700 1 -- 172.21.15.32:0/2853026894 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f3894056610 con 0x7f38a00fcf60 2024-05-12T03:59:27.729 INFO:teuthology.orchestra.run.smithi032.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-12T03:59:27.729 INFO:teuthology.orchestra.run.smithi032.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-12T03:59:27.729 INFO:teuthology.orchestra.run.smithi032.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-12T03:59:27.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.731+0000 7f38a4a7b700 1 -- 172.21.15.32:0/2853026894 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f38880692a0 msgr2=0x7f388806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:59:27.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.731+0000 7f38a4a7b700 1 --2- 172.21.15.32:0/2853026894 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f38880692a0 0x7f388806b750 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3890009fd0 tx=0x7f3890009e30 comp rx=0 tx=0).stop 2024-05-12T03:59:27.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.731+0000 7f38a4a7b700 1 -- 172.21.15.32:0/2853026894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38a00fcf60 msgr2=0x7f38a01022c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:59:27.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.731+0000 7f38a4a7b700 1 --2- 172.21.15.32:0/2853026894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38a00fcf60 0x7f38a01022c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3894002a50 tx=0x7f38940046b0 comp rx=0 tx=0).stop 2024-05-12T03:59:27.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.732+0000 7f38a4a7b700 1 -- 172.21.15.32:0/2853026894 shutdown_connections 2024-05-12T03:59:27.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.732+0000 7f38a4a7b700 1 --2- 172.21.15.32:0/2853026894 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f38880692a0 0x7f388806b750 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:27.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.732+0000 7f38a4a7b700 1 --2- 172.21.15.32:0/2853026894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38a00fbcb0 0x7f38a0103d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:27.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.732+0000 7f38a4a7b700 1 --2- 172.21.15.32:0/2853026894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38a00fcf60 0x7f38a01022c0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:27.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.732+0000 7f38a4a7b700 1 -- 172.21.15.32:0/2853026894 >> 172.21.15.32:0/2853026894 conn(0x7f38a00f7430 msgr2=0x7f38a00f8bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:59:27.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.732+0000 7f38a4a7b700 1 -- 172.21.15.32:0/2853026894 shutdown_connections 2024-05-12T03:59:27.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:27.732+0000 7f38a4a7b700 1 -- 172.21.15.32:0/2853026894 wait complete. 2024-05-12T03:59:28.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:59:27 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/2853026894' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-12T03:59:28.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:59:27 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/2853026894' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-12T03:59:29.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:59:28 smithi032 ceph-mon[40008]: from='client.14610 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T03:59:29.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:59:28 smithi032 ceph-mon[40008]: pgmap v238: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T03:59:29.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:59:28 smithi033 ceph-mon[49525]: from='client.14610 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T03:59:29.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:59:28 smithi033 ceph-mon[49525]: pgmap v238: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T03:59:31.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:59:30 smithi032 ceph-mon[40008]: pgmap v239: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T03:59:31.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:59:30 smithi033 ceph-mon[49525]: pgmap v239: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T03:59:33.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:59:32 smithi032 ceph-mon[40008]: pgmap v240: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T03:59:33.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:59:32 smithi033 ceph-mon[49525]: pgmap v240: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T03:59:35.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:59:34 smithi032 ceph-mon[40008]: pgmap v241: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T03:59:35.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:59:34 smithi033 ceph-mon[49525]: pgmap v241: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T03:59:37.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:59:36 smithi033 ceph-mon[49525]: pgmap v242: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T03:59:37.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:59:36 smithi032 ceph-mon[40008]: pgmap v242: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T03:59:39.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:59:38 smithi032 ceph-mon[40008]: pgmap v243: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T03:59:39.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:59:38 smithi033 ceph-mon[49525]: pgmap v243: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T03:59:40.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:59:39 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T03:59:40.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:59:39 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T03:59:41.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:59:40 smithi032 ceph-mon[40008]: pgmap v244: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T03:59:41.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:59:40 smithi033 ceph-mon[49525]: pgmap v244: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T03:59:43.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:59:42 smithi032 ceph-mon[40008]: pgmap v245: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T03:59:43.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:59:42 smithi033 ceph-mon[49525]: pgmap v245: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T03:59:45.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:59:44 smithi032 ceph-mon[40008]: pgmap v246: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T03:59:45.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:59:44 smithi033 ceph-mon[49525]: pgmap v246: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T03:59:47.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:59:46 smithi033 ceph-mon[49525]: pgmap v247: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T03:59:47.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:59:46 smithi032 ceph-mon[40008]: pgmap v247: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T03:59:48.621 INFO:tasks.workunit.client.0.smithi032.stderr:Updating files: 83% (10520/12534) Updating files: 84% (10529/12534) Updating files: 85% (10654/12534) Updating files: 86% (10780/12534) Updating files: 87% (10905/12534) Updating files: 88% (11030/12534) Updating files: 89% (11156/12534) Updating files: 90% (11281/12534) Updating files: 91% (11406/12534) Updating files: 92% (11532/12534) Updating files: 93% (11657/12534) Updating files: 94% (11782/12534) Updating files: 95% (11908/12534) Updating files: 96% (12033/12534) Updating files: 97% (12158/12534) Updating files: 98% (12284/12534) Updating files: 99% (12409/12534) Updating files: 100% (12534/12534) Updating files: 100% (12534/12534), done. 2024-05-12T03:59:49.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:59:48 smithi032 ceph-mon[40008]: pgmap v248: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T03:59:49.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:59:48 smithi033 ceph-mon[49525]: pgmap v248: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T03:59:49.584 INFO:tasks.workunit.client.0.smithi032.stderr:Note: switching to 'fccfb453906af8847786c471919d347112ea0360'. 2024-05-12T03:59:49.584 INFO:tasks.workunit.client.0.smithi032.stderr: 2024-05-12T03:59:49.584 INFO:tasks.workunit.client.0.smithi032.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-12T03:59:49.584 INFO:tasks.workunit.client.0.smithi032.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-12T03:59:49.584 INFO:tasks.workunit.client.0.smithi032.stderr:state without impacting any branches by switching back to a branch. 2024-05-12T03:59:49.584 INFO:tasks.workunit.client.0.smithi032.stderr: 2024-05-12T03:59:49.584 INFO:tasks.workunit.client.0.smithi032.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-12T03:59:49.584 INFO:tasks.workunit.client.0.smithi032.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-12T03:59:49.584 INFO:tasks.workunit.client.0.smithi032.stderr: 2024-05-12T03:59:49.584 INFO:tasks.workunit.client.0.smithi032.stderr: git switch -c 2024-05-12T03:59:49.584 INFO:tasks.workunit.client.0.smithi032.stderr: 2024-05-12T03:59:49.584 INFO:tasks.workunit.client.0.smithi032.stderr:Or undo this operation with: 2024-05-12T03:59:49.584 INFO:tasks.workunit.client.0.smithi032.stderr: 2024-05-12T03:59:49.585 INFO:tasks.workunit.client.0.smithi032.stderr: git switch - 2024-05-12T03:59:49.585 INFO:tasks.workunit.client.0.smithi032.stderr: 2024-05-12T03:59:49.585 INFO:tasks.workunit.client.0.smithi032.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-12T03:59:49.585 INFO:tasks.workunit.client.0.smithi032.stderr: 2024-05-12T03:59:49.585 INFO:tasks.workunit.client.0.smithi032.stderr:HEAD is now at fccfb453906 Merge pull request #54550 from k0ste/wip-62819-reef 2024-05-12T03:59:49.591 DEBUG:teuthology.orchestra.run.smithi032:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-12T03:59:49.636 INFO:tasks.workunit.client.0.smithi032.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-12T03:59:49.638 INFO:tasks.workunit.client.0.smithi032.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-12T03:59:49.638 INFO:tasks.workunit.client.0.smithi032.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-12T03:59:51.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:59:50 smithi032 ceph-mon[40008]: pgmap v249: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T03:59:51.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:59:50 smithi033 ceph-mon[49525]: pgmap v249: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T03:59:53.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:59:52 smithi032 ceph-mon[40008]: pgmap v250: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T03:59:53.293 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:59:52 smithi033 ceph-mon[49525]: pgmap v250: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T03:59:53.430 INFO:tasks.workunit.client.0.smithi032.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-12T03:59:53.513 INFO:tasks.workunit.client.0.smithi032.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-12T03:59:53.555 INFO:tasks.workunit.client.0.smithi032.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-12T03:59:53.556 INFO:tasks.workunit.client.0.smithi032.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-12T03:59:53.556 INFO:tasks.workunit.client.0.smithi032.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-12T03:59:53.595 INFO:tasks.workunit.client.0.smithi032.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-12T03:59:53.599 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-05-12T03:59:53.599 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-12T03:59:53.618 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-05-12T03:59:53.621 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-05-12T03:59:53.621 DEBUG:teuthology.orchestra.run.smithi032:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=fccfb453906af8847786c471919d347112ea0360 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-05-12T03:59:53.659 INFO:tasks.workunit.client.1.smithi033.stderr:Updating files: 90% (11282/12534) Updating files: 91% (11406/12534) Updating files: 92% (11532/12534) Updating files: 93% (11657/12534) Updating files: 94% (11782/12534) Updating files: 95% (11908/12534) Updating files: 96% (12033/12534) Updating files: 97% (12158/12534) Updating files: 98% (12284/12534) Updating files: 99% (12409/12534) Updating files: 100% (12534/12534) Updating files: 100% (12534/12534), done. 2024-05-12T03:59:53.686 INFO:tasks.workunit.client.0.smithi032.stderr:+ mkdir -p fsstress 2024-05-12T03:59:53.688 INFO:tasks.workunit.client.0.smithi032.stderr:+ pushd fsstress 2024-05-12T03:59:53.689 INFO:tasks.workunit.client.0.smithi032.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-05-12T03:59:53.690 INFO:tasks.workunit.client.0.smithi032.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-05-12T03:59:54.357 INFO:tasks.workunit.client.0.smithi032.stderr:+ tar xzf ltp-full.tgz 2024-05-12T03:59:54.567 INFO:tasks.workunit.client.1.smithi033.stderr:Note: switching to 'fccfb453906af8847786c471919d347112ea0360'. 2024-05-12T03:59:54.568 INFO:tasks.workunit.client.1.smithi033.stderr: 2024-05-12T03:59:54.568 INFO:tasks.workunit.client.1.smithi033.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-12T03:59:54.568 INFO:tasks.workunit.client.1.smithi033.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-12T03:59:54.568 INFO:tasks.workunit.client.1.smithi033.stderr:state without impacting any branches by switching back to a branch. 2024-05-12T03:59:54.568 INFO:tasks.workunit.client.1.smithi033.stderr: 2024-05-12T03:59:54.568 INFO:tasks.workunit.client.1.smithi033.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-12T03:59:54.568 INFO:tasks.workunit.client.1.smithi033.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-12T03:59:54.568 INFO:tasks.workunit.client.1.smithi033.stderr: 2024-05-12T03:59:54.568 INFO:tasks.workunit.client.1.smithi033.stderr: git switch -c 2024-05-12T03:59:54.568 INFO:tasks.workunit.client.1.smithi033.stderr: 2024-05-12T03:59:54.568 INFO:tasks.workunit.client.1.smithi033.stderr:Or undo this operation with: 2024-05-12T03:59:54.568 INFO:tasks.workunit.client.1.smithi033.stderr: 2024-05-12T03:59:54.568 INFO:tasks.workunit.client.1.smithi033.stderr: git switch - 2024-05-12T03:59:54.569 INFO:tasks.workunit.client.1.smithi033.stderr: 2024-05-12T03:59:54.569 INFO:tasks.workunit.client.1.smithi033.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-12T03:59:54.569 INFO:tasks.workunit.client.1.smithi033.stderr: 2024-05-12T03:59:54.569 INFO:tasks.workunit.client.1.smithi033.stderr:HEAD is now at fccfb453906 Merge pull request #54550 from k0ste/wip-62819-reef 2024-05-12T03:59:54.574 DEBUG:teuthology.orchestra.run.smithi033:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-05-12T03:59:54.641 INFO:tasks.workunit.client.1.smithi033.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-12T03:59:54.643 INFO:tasks.workunit.client.1.smithi033.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-05-12T03:59:54.644 INFO:tasks.workunit.client.1.smithi033.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-12T03:59:54.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:59:54 smithi033 ceph-mon[49525]: pgmap v251: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T03:59:54.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:59:54 smithi033 ceph-mon[49525]: Upgrade: Updating mgr.smithi033.pcvxwy 2024-05-12T03:59:54.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:59:54 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:59:54.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:59:54 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi033.pcvxwy", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-12T03:59:54.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:59:54 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-12T03:59:54.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:59:54 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:59:54.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:59:54 smithi033 ceph-mon[49525]: Deploying daemon mgr.smithi033.pcvxwy on smithi033 2024-05-12T03:59:54.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:59:54 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T03:59:55.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:59:54 smithi032 ceph-mon[40008]: pgmap v251: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T03:59:55.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:59:54 smithi032 ceph-mon[40008]: Upgrade: Updating mgr.smithi033.pcvxwy 2024-05-12T03:59:55.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:59:54 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T03:59:55.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:59:54 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi033.pcvxwy", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-12T03:59:55.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:59:54 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-12T03:59:55.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:59:54 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T03:59:55.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:59:54 smithi032 ceph-mon[40008]: Deploying daemon mgr.smithi033.pcvxwy on smithi033 2024-05-12T03:59:55.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:59:54 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T03:59:55.940 INFO:tasks.workunit.client.1.smithi033.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-12T03:59:56.083 INFO:tasks.workunit.client.1.smithi033.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-12T03:59:56.125 INFO:tasks.workunit.client.1.smithi033.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-05-12T03:59:56.127 INFO:tasks.workunit.client.1.smithi033.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-05-12T03:59:56.127 INFO:tasks.workunit.client.1.smithi033.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-12T03:59:56.165 INFO:tasks.workunit.client.1.smithi033.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-05-12T03:59:56.169 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-12T03:59:56.169 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-05-12T03:59:56.228 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.1... 2024-05-12T03:59:56.229 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-05-12T03:59:56.229 DEBUG:teuthology.orchestra.run.smithi033:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=fccfb453906af8847786c471919d347112ea0360 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh 2024-05-12T03:59:56.297 INFO:tasks.workunit.client.1.smithi033.stderr:+ mkdir -p fsstress 2024-05-12T03:59:56.299 INFO:tasks.workunit.client.1.smithi033.stderr:+ pushd fsstress 2024-05-12T03:59:56.300 INFO:tasks.workunit.client.1.smithi033.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-05-12T03:59:56.301 INFO:tasks.workunit.client.1.smithi033.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-05-12T03:59:57.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:59:56 smithi033 ceph-mon[49525]: pgmap v252: 65 pgs: 65 active+clean; 7.3 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 577 KiB/s wr, 7 op/s 2024-05-12T03:59:57.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:59:56 smithi032 ceph-mon[40008]: pgmap v252: 65 pgs: 65 active+clean; 7.3 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 577 KiB/s wr, 7 op/s 2024-05-12T03:59:57.345 INFO:tasks.workunit.client.1.smithi033.stderr:+ tar xzf ltp-full.tgz 2024-05-12T03:59:57.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:57.860+0000 7f7795c33700 1 -- 172.21.15.32:0/534660439 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77900fc290 msgr2=0x7f77900fc6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:59:57.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:57.860+0000 7f7795c33700 1 --2- 172.21.15.32:0/534660439 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77900fc290 0x7f77900fc6a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7780004940 tx=0x7f7780009e30 comp rx=0 tx=0).stop 2024-05-12T03:59:57.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:57.860+0000 7f7795c33700 1 -- 172.21.15.32:0/534660439 shutdown_connections 2024-05-12T03:59:57.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:57.860+0000 7f7795c33700 1 --2- 172.21.15.32:0/534660439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77900fcbe0 0x7f77900fd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:57.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:57.860+0000 7f7795c33700 1 --2- 172.21.15.32:0/534660439 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77900fc290 0x7f77900fc6a0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:57.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:57.860+0000 7f7795c33700 1 -- 172.21.15.32:0/534660439 >> 172.21.15.32:0/534660439 conn(0x7f77900f6b80 msgr2=0x7f77900f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:59:57.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:57.860+0000 7f7795c33700 1 -- 172.21.15.32:0/534660439 shutdown_connections 2024-05-12T03:59:57.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:57.860+0000 7f7795c33700 1 -- 172.21.15.32:0/534660439 wait complete. 2024-05-12T03:59:57.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:57.861+0000 7f7795c33700 1 Processor -- start 2024-05-12T03:59:57.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:57.861+0000 7f7795c33700 1 -- start start 2024-05-12T03:59:57.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:57.861+0000 7f7795c33700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77900fc290 0x7f77900ffec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:59:57.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:57.861+0000 7f7795c33700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77900fcbe0 0x7f77900fe390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:59:57.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:57.861+0000 7f7795c33700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77900fe9d0 con 0x7f77900fcbe0 2024-05-12T03:59:57.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:57.861+0000 7f7795c33700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77900feb40 con 0x7f77900fc290 2024-05-12T03:59:57.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:57.861+0000 7f778f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77900fc290 0x7f77900ffec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:59:57.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:57.861+0000 7f778f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77900fc290 0x7f77900ffec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.32:60150/0 (socket says 172.21.15.32:60150) 2024-05-12T03:59:57.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:57.861+0000 7f778f7fe700 1 -- 172.21.15.32:0/1116535993 learned_addr learned my addr 172.21.15.32:0/1116535993 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:59:57.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:57.861+0000 7f778effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77900fcbe0 0x7f77900fe390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:59:57.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:57.862+0000 7f778effd700 1 -- 172.21.15.32:0/1116535993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77900fc290 msgr2=0x7f77900ffec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:59:57.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:57.862+0000 7f778effd700 1 --2- 172.21.15.32:0/1116535993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77900fc290 0x7f77900ffec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:57.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:57.862+0000 7f778effd700 1 -- 172.21.15.32:0/1116535993 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7780005040 con 0x7f77900fcbe0 2024-05-12T03:59:57.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:57.862+0000 7f778f7fe700 1 --2- 172.21.15.32:0/1116535993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77900fc290 0x7f77900ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-12T03:59:57.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:57.862+0000 7f778effd700 1 --2- 172.21.15.32:0/1116535993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77900fcbe0 0x7f77900fe390 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f778400b580 tx=0x7f77840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:59:57.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:57.862+0000 7f7777fff700 1 -- 172.21.15.32:0/1116535993 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f778400f6f0 con 0x7f77900fcbe0 2024-05-12T03:59:57.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:57.862+0000 7f7777fff700 1 -- 172.21.15.32:0/1116535993 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f778400fd00 con 0x7f77900fcbe0 2024-05-12T03:59:57.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:57.862+0000 7f7795c33700 1 -- 172.21.15.32:0/1116535993 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77900fee20 con 0x7f77900fcbe0 2024-05-12T03:59:57.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:57.862+0000 7f7777fff700 1 -- 172.21.15.32:0/1116535993 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77840103a0 con 0x7f77900fcbe0 2024-05-12T03:59:57.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:57.862+0000 7f7795c33700 1 -- 172.21.15.32:0/1116535993 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7790115990 con 0x7f77900fcbe0 2024-05-12T03:59:57.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:57.864+0000 7f7795c33700 1 -- 172.21.15.32:0/1116535993 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7790101be0 con 0x7f77900fcbe0 2024-05-12T03:59:57.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:57.865+0000 7f7777fff700 1 -- 172.21.15.32:0/1116535993 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f778400f850 con 0x7f77900fcbe0 2024-05-12T03:59:57.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:57.865+0000 7f7777fff700 1 --2- 172.21.15.32:0/1116535993 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f77780692b0 0x7f777806b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:59:57.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:57.866+0000 7f7777fff700 1 -- 172.21.15.32:0/1116535993 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f7784087a50 con 0x7f77900fcbe0 2024-05-12T03:59:57.867 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:57.867+0000 7f778f7fe700 1 --2- 172.21.15.32:0/1116535993 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f77780692b0 0x7f777806b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:59:57.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:57.867+0000 7f778f7fe700 1 --2- 172.21.15.32:0/1116535993 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f77780692b0 0x7f777806b760 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7780004f90 tx=0x7f7780009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:59:57.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:57.871+0000 7f7777fff700 1 -- 172.21.15.32:0/1116535993 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f7784053dd0 con 0x7f77900fcbe0 2024-05-12T03:59:58.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.145+0000 7f7795c33700 1 -- 172.21.15.32:0/1116535993 --> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f779005e3a0 con 0x7f77780692b0 2024-05-12T03:59:58.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.146+0000 7f7777fff700 1 -- 172.21.15.32:0/1116535993 <== mgr.14208 v2:172.21.15.32:6800/4089797100 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7f779005e3a0 con 0x7f77780692b0 2024-05-12T03:59:58.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.148+0000 7f7795c33700 1 -- 172.21.15.32:0/1116535993 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f77780692b0 msgr2=0x7f777806b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:59:58.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.148+0000 7f7795c33700 1 --2- 172.21.15.32:0/1116535993 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f77780692b0 0x7f777806b760 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7780004f90 tx=0x7f7780009e30 comp rx=0 tx=0).stop 2024-05-12T03:59:58.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.148+0000 7f7795c33700 1 -- 172.21.15.32:0/1116535993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77900fcbe0 msgr2=0x7f77900fe390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:59:58.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.148+0000 7f7795c33700 1 --2- 172.21.15.32:0/1116535993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77900fcbe0 0x7f77900fe390 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f778400b580 tx=0x7f77840091a0 comp rx=0 tx=0).stop 2024-05-12T03:59:58.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.148+0000 7f7795c33700 1 -- 172.21.15.32:0/1116535993 shutdown_connections 2024-05-12T03:59:58.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.148+0000 7f7795c33700 1 --2- 172.21.15.32:0/1116535993 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f77780692b0 0x7f777806b760 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:58.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.149+0000 7f7795c33700 1 --2- 172.21.15.32:0/1116535993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77900fc290 0x7f77900ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:58.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.149+0000 7f7795c33700 1 --2- 172.21.15.32:0/1116535993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f77900fcbe0 0x7f77900fe390 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:58.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.149+0000 7f7795c33700 1 -- 172.21.15.32:0/1116535993 >> 172.21.15.32:0/1116535993 conn(0x7f77900f6b80 msgr2=0x7f77900fb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:59:58.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.149+0000 7f7795c33700 1 -- 172.21.15.32:0/1116535993 shutdown_connections 2024-05-12T03:59:58.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.149+0000 7f7795c33700 1 -- 172.21.15.32:0/1116535993 wait complete. 2024-05-12T03:59:58.163 INFO:teuthology.orchestra.run.smithi032.stdout:true 2024-05-12T03:59:58.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.279+0000 7f1704e88700 1 -- 172.21.15.32:0/3816980958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17000fcf60 msgr2=0x7f17000fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:59:58.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.279+0000 7f1704e88700 1 --2- 172.21.15.32:0/3816980958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17000fcf60 0x7f17000fd3d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f16f4004940 tx=0x7f16f4009e30 comp rx=0 tx=0).stop 2024-05-12T03:59:58.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.279+0000 7f1704e88700 1 -- 172.21.15.32:0/3816980958 shutdown_connections 2024-05-12T03:59:58.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.279+0000 7f1704e88700 1 --2- 172.21.15.32:0/3816980958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17000fcf60 0x7f17000fd3d0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:58.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.279+0000 7f1704e88700 1 --2- 172.21.15.32:0/3816980958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17000fbcb0 0x7f17000fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:58.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.279+0000 7f1704e88700 1 -- 172.21.15.32:0/3816980958 >> 172.21.15.32:0/3816980958 conn(0x7f17000f7430 msgr2=0x7f17000f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:59:58.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.279+0000 7f1704e88700 1 -- 172.21.15.32:0/3816980958 shutdown_connections 2024-05-12T03:59:58.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.279+0000 7f1704e88700 1 -- 172.21.15.32:0/3816980958 wait complete. 2024-05-12T03:59:58.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.279+0000 7f1704e88700 1 Processor -- start 2024-05-12T03:59:58.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.280+0000 7f1704e88700 1 -- start start 2024-05-12T03:59:58.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.280+0000 7f1704e88700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17000fbcb0 0x7f1700113f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:59:58.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.280+0000 7f1704e88700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17000fcf60 0x7f1700114480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:59:58.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.280+0000 7f1704e88700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1700115140 con 0x7f17000fbcb0 2024-05-12T03:59:58.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.280+0000 7f1704e88700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17001152b0 con 0x7f17000fcf60 2024-05-12T03:59:58.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.280+0000 7f16fe59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17000fbcb0 0x7f1700113f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:59:58.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.280+0000 7f16fe59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17000fbcb0 0x7f1700113f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58150/0 (socket says 172.21.15.32:58150) 2024-05-12T03:59:58.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.280+0000 7f16fe59c700 1 -- 172.21.15.32:0/580331791 learned_addr learned my addr 172.21.15.32:0/580331791 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:59:58.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.280+0000 7f16fdd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17000fcf60 0x7f1700114480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:59:58.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.280+0000 7f16fe59c700 1 -- 172.21.15.32:0/580331791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17000fcf60 msgr2=0x7f1700114480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:59:58.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.280+0000 7f16fe59c700 1 --2- 172.21.15.32:0/580331791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17000fcf60 0x7f1700114480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:58.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.280+0000 7f16fe59c700 1 -- 172.21.15.32:0/580331791 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16f4005040 con 0x7f17000fbcb0 2024-05-12T03:59:58.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.280+0000 7f16fdd9b700 1 --2- 172.21.15.32:0/580331791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17000fcf60 0x7f1700114480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-12T03:59:58.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.281+0000 7f16fe59c700 1 --2- 172.21.15.32:0/580331791 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17000fbcb0 0x7f1700113f40 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f16f000eb40 tx=0x7f16f000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:59:58.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.281+0000 7f16eeffd700 1 -- 172.21.15.32:0/580331791 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f16f0007cc0 con 0x7f17000fbcb0 2024-05-12T03:59:58.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.281+0000 7f1704e88700 1 -- 172.21.15.32:0/580331791 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1700115530 con 0x7f17000fbcb0 2024-05-12T03:59:58.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.281+0000 7f16eeffd700 1 -- 172.21.15.32:0/580331791 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f16f000f040 con 0x7f17000fbcb0 2024-05-12T03:59:58.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.281+0000 7f16eeffd700 1 -- 172.21.15.32:0/580331791 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f16f000c990 con 0x7f17000fbcb0 2024-05-12T03:59:58.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.281+0000 7f1704e88700 1 -- 172.21.15.32:0/580331791 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1700115af0 con 0x7f17000fbcb0 2024-05-12T03:59:58.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.282+0000 7f1704e88700 1 -- 172.21.15.32:0/580331791 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1700105f70 con 0x7f17000fbcb0 2024-05-12T03:59:58.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.283+0000 7f16eeffd700 1 -- 172.21.15.32:0/580331791 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f16f000cc00 con 0x7f17000fbcb0 2024-05-12T03:59:58.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.284+0000 7f16eeffd700 1 --2- 172.21.15.32:0/580331791 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f16e80692a0 0x7f16e806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:59:58.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.284+0000 7f16eeffd700 1 -- 172.21.15.32:0/580331791 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f16f0087000 con 0x7f17000fbcb0 2024-05-12T03:59:58.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.284+0000 7f16fdd9b700 1 --2- 172.21.15.32:0/580331791 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f16e80692a0 0x7f16e806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:59:58.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.284+0000 7f16fdd9b700 1 --2- 172.21.15.32:0/580331791 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f16e80692a0 0x7f16e806b750 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f16f4004890 tx=0x7f16f4009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:59:58.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.287+0000 7f16eeffd700 1 -- 172.21.15.32:0/580331791 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f16f0056e20 con 0x7f17000fbcb0 2024-05-12T03:59:58.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.578+0000 7f1704e88700 1 -- 172.21.15.32:0/580331791 --> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1700115d80 con 0x7f16e80692a0 2024-05-12T03:59:58.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.579+0000 7f16eeffd700 1 -- 172.21.15.32:0/580331791 <== mgr.14208 v2:172.21.15.32:6800/4089797100 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7f1700115d80 con 0x7f16e80692a0 2024-05-12T03:59:58.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.582+0000 7f1704e88700 1 -- 172.21.15.32:0/580331791 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f16e80692a0 msgr2=0x7f16e806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:59:58.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.582+0000 7f1704e88700 1 --2- 172.21.15.32:0/580331791 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f16e80692a0 0x7f16e806b750 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f16f4004890 tx=0x7f16f4009e30 comp rx=0 tx=0).stop 2024-05-12T03:59:58.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.582+0000 7f1704e88700 1 -- 172.21.15.32:0/580331791 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17000fbcb0 msgr2=0x7f1700113f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:59:58.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.582+0000 7f1704e88700 1 --2- 172.21.15.32:0/580331791 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17000fbcb0 0x7f1700113f40 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f16f000eb40 tx=0x7f16f000b040 comp rx=0 tx=0).stop 2024-05-12T03:59:58.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.582+0000 7f1704e88700 1 -- 172.21.15.32:0/580331791 shutdown_connections 2024-05-12T03:59:58.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.582+0000 7f1704e88700 1 --2- 172.21.15.32:0/580331791 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f16e80692a0 0x7f16e806b750 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:58.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.582+0000 7f1704e88700 1 --2- 172.21.15.32:0/580331791 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17000fbcb0 0x7f1700113f40 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:58.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.582+0000 7f1704e88700 1 --2- 172.21.15.32:0/580331791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17000fcf60 0x7f1700114480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:58.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.582+0000 7f1704e88700 1 -- 172.21.15.32:0/580331791 >> 172.21.15.32:0/580331791 conn(0x7f17000f7430 msgr2=0x7f17000f8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:59:58.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.582+0000 7f1704e88700 1 -- 172.21.15.32:0/580331791 shutdown_connections 2024-05-12T03:59:58.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.582+0000 7f1704e88700 1 -- 172.21.15.32:0/580331791 wait complete. 2024-05-12T03:59:58.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.722+0000 7f9290a8d700 1 -- 172.21.15.32:0/1508150868 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f928c0fcf60 msgr2=0x7f928c0fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:59:58.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.722+0000 7f9290a8d700 1 --2- 172.21.15.32:0/1508150868 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f928c0fcf60 0x7f928c0fd3d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9280004940 tx=0x7f9280009e30 comp rx=0 tx=0).stop 2024-05-12T03:59:58.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.722+0000 7f9290a8d700 1 -- 172.21.15.32:0/1508150868 shutdown_connections 2024-05-12T03:59:58.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.722+0000 7f9290a8d700 1 --2- 172.21.15.32:0/1508150868 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f928c0fcf60 0x7f928c0fd3d0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:58.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.722+0000 7f9290a8d700 1 --2- 172.21.15.32:0/1508150868 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f928c0fbcb0 0x7f928c0fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:58.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.722+0000 7f9290a8d700 1 -- 172.21.15.32:0/1508150868 >> 172.21.15.32:0/1508150868 conn(0x7f928c0f7430 msgr2=0x7f928c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:59:58.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.722+0000 7f9290a8d700 1 -- 172.21.15.32:0/1508150868 shutdown_connections 2024-05-12T03:59:58.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.722+0000 7f9290a8d700 1 -- 172.21.15.32:0/1508150868 wait complete. 2024-05-12T03:59:58.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.723+0000 7f9290a8d700 1 Processor -- start 2024-05-12T03:59:58.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.723+0000 7f9290a8d700 1 -- start start 2024-05-12T03:59:58.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.723+0000 7f9290a8d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f928c0fbcb0 0x7f928c10b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:59:58.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.723+0000 7f9290a8d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f928c0fcf60 0x7f928c10bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:59:58.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.723+0000 7f9290a8d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928c10c1e0 con 0x7f928c0fbcb0 2024-05-12T03:59:58.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.723+0000 7f9290a8d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928c111c90 con 0x7f928c0fcf60 2024-05-12T03:59:58.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.723+0000 7f9289d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f928c0fcf60 0x7f928c10bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:59:58.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.723+0000 7f9289d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f928c0fcf60 0x7f928c10bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.32:60182/0 (socket says 172.21.15.32:60182) 2024-05-12T03:59:58.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.723+0000 7f9289d9b700 1 -- 172.21.15.32:0/48637032 learned_addr learned my addr 172.21.15.32:0/48637032 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:59:58.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.723+0000 7f9289d9b700 1 -- 172.21.15.32:0/48637032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f928c0fbcb0 msgr2=0x7f928c10b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:59:58.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.723+0000 7f9289d9b700 1 --2- 172.21.15.32:0/48637032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f928c0fbcb0 0x7f928c10b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:58.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.723+0000 7f9289d9b700 1 -- 172.21.15.32:0/48637032 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9280005040 con 0x7f928c0fcf60 2024-05-12T03:59:58.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.724+0000 7f9289d9b700 1 --2- 172.21.15.32:0/48637032 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f928c0fcf60 0x7f928c10bb90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9280007d40 tx=0x7f9280009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:59:58.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.724+0000 7f927affd700 1 -- 172.21.15.32:0/48637032 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f928000e030 con 0x7f928c0fcf60 2024-05-12T03:59:58.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.724+0000 7f927affd700 1 -- 172.21.15.32:0/48637032 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f92800093f0 con 0x7f928c0fcf60 2024-05-12T03:59:58.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.724+0000 7f927affd700 1 -- 172.21.15.32:0/48637032 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f928000b410 con 0x7f928c0fcf60 2024-05-12T03:59:58.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.725+0000 7f9290a8d700 1 -- 172.21.15.32:0/48637032 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f928c111f10 con 0x7f928c0fcf60 2024-05-12T03:59:58.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.725+0000 7f9290a8d700 1 -- 172.21.15.32:0/48637032 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f928c1124a0 con 0x7f928c0fcf60 2024-05-12T03:59:58.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.726+0000 7f9290a8d700 1 -- 172.21.15.32:0/48637032 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f928c05f5c0 con 0x7f928c0fcf60 2024-05-12T03:59:58.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.726+0000 7f927affd700 1 -- 172.21.15.32:0/48637032 <== mon.1 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f928001c400 con 0x7f928c0fcf60 2024-05-12T03:59:58.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.727+0000 7f927affd700 1 --2- 172.21.15.32:0/48637032 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f92740692a0 0x7f927406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:59:58.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.727+0000 7f927affd700 1 -- 172.21.15.32:0/48637032 <== mon.1 v2:172.21.15.33:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f9280088330 con 0x7f928c0fcf60 2024-05-12T03:59:58.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.727+0000 7f928a59c700 1 --2- 172.21.15.32:0/48637032 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f92740692a0 0x7f927406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:59:58.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.728+0000 7f928a59c700 1 --2- 172.21.15.32:0/48637032 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f92740692a0 0x7f927406b750 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f928c10b3e0 tx=0x7f927c00b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:59:58.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:58.732+0000 7f927affd700 1 -- 172.21.15.32:0/48637032 <== mon.1 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f9280054770 con 0x7f928c0fcf60 2024-05-12T03:59:59.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.056+0000 7f9290a8d700 1 -- 172.21.15.32:0/48637032 --> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f928c101420 con 0x7f92740692a0 2024-05-12T03:59:59.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.072+0000 7f927affd700 1 -- 172.21.15.32:0/48637032 <== mgr.14208 v2:172.21.15.32:6800/4089797100 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3410 (secure 0 0 0) 0x7f928c101420 con 0x7f92740692a0 2024-05-12T03:59:59.073 INFO:teuthology.orchestra.run.smithi032.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-12T03:59:59.073 INFO:teuthology.orchestra.run.smithi032.stdout:alertmanager.smithi032 smithi032 *:9093,9094 running (6m) 100s ago 7m 27.7M - 0.25.0 c8568f914cd2 5c80dfe17d2e 2024-05-12T03:59:59.073 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi032 smithi032 running (8m) 100s ago 8m 6651k - 17.2.7-939-gc190bdee 284803f7d9cf a478874e9dfa 2024-05-12T03:59:59.073 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi033 smithi033 running (7m) 107s ago 7m 6651k - 17.2.7-939-gc190bdee 284803f7d9cf 34c62d004c7e 2024-05-12T03:59:59.073 INFO:teuthology.orchestra.run.smithi032.stdout:grafana.smithi032 smithi032 *:3000 running (6m) 100s ago 7m 85.7M - 9.4.7 954c08fa6188 ac9076563478 2024-05-12T03:59:59.073 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.ialtqb smithi032 running (2m) 100s ago 2m 10.4M - 17.2.7-939-gc190bdee 284803f7d9cf 0683abf00e3e 2024-05-12T03:59:59.073 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.qpsawh smithi032 running (2m) 100s ago 2m 16.6M - 17.2.7-939-gc190bdee 284803f7d9cf 95a7db5c4fcd 2024-05-12T03:59:59.073 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi033.hglwml smithi033 running (117s) 107s ago 117s 10.4M - 17.2.7-939-gc190bdee 284803f7d9cf c97902b05025 2024-05-12T03:59:59.073 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi033.jgbjhk smithi033 running (2m) 107s ago 2m 15.2M - 17.2.7-939-gc190bdee 284803f7d9cf af97052cb97a 2024-05-12T03:59:59.074 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi032.ukatkq smithi032 *:9283 running (9m) 100s ago 9m 483M - 17.2.7-939-gc190bdee 284803f7d9cf d9763dc47404 2024-05-12T03:59:59.074 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi033.pcvxwy smithi033 *:8443,9283 starting - - - - 2024-05-12T03:59:59.074 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi032 smithi032 running (9m) 100s ago 9m 45.4M 2048M 17.2.7-939-gc190bdee 284803f7d9cf d7fea430c6e2 2024-05-12T03:59:59.074 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi033 smithi033 running (6m) 107s ago 6m 34.4M 2048M 17.2.7-939-gc190bdee 284803f7d9cf 707654d85748 2024-05-12T03:59:59.074 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi032 smithi032 *:9100 running (8m) 100s ago 8m 17.8M - 1.5.0 0da6a335fe13 6aa48ce262de 2024-05-12T03:59:59.074 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi033 smithi033 *:9100 running (7m) 107s ago 7m 16.9M - 1.5.0 0da6a335fe13 1144caed00e9 2024-05-12T03:59:59.074 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0 smithi032 running (6m) 100s ago 6m 44.7M 4300M 17.2.7-939-gc190bdee 284803f7d9cf 75c78a47f8e5 2024-05-12T03:59:59.074 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1 smithi032 running (5m) 100s ago 5m 48.5M 4300M 17.2.7-939-gc190bdee 284803f7d9cf 87cfd247d8d6 2024-05-12T03:59:59.074 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2 smithi032 running (5m) 100s ago 5m 43.8M 4300M 17.2.7-939-gc190bdee 284803f7d9cf 7aef7affb8ba 2024-05-12T03:59:59.074 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3 smithi033 running (4m) 107s ago 4m 44.0M 5324M 17.2.7-939-gc190bdee 284803f7d9cf 3a92b5e3348e 2024-05-12T03:59:59.074 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4 smithi033 running (4m) 107s ago 4m 44.2M 5324M 17.2.7-939-gc190bdee 284803f7d9cf 0896268b4591 2024-05-12T03:59:59.074 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5 smithi033 running (3m) 107s ago 3m 40.7M 5324M 17.2.7-939-gc190bdee 284803f7d9cf 57e65a64c821 2024-05-12T03:59:59.074 INFO:teuthology.orchestra.run.smithi032.stdout:prometheus.smithi032 smithi032 *:9095 running (6m) 100s ago 7m 52.6M - 2.43.0 a07b618ecd1d 0b3ce3323a49 2024-05-12T03:59:59.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.075+0000 7f9290a8d700 1 -- 172.21.15.32:0/48637032 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f92740692a0 msgr2=0x7f927406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:59:59.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.075+0000 7f9290a8d700 1 --2- 172.21.15.32:0/48637032 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f92740692a0 0x7f927406b750 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f928c10b3e0 tx=0x7f927c00b040 comp rx=0 tx=0).stop 2024-05-12T03:59:59.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.075+0000 7f9290a8d700 1 -- 172.21.15.32:0/48637032 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f928c0fcf60 msgr2=0x7f928c10bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:59:59.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.075+0000 7f9290a8d700 1 --2- 172.21.15.32:0/48637032 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f928c0fcf60 0x7f928c10bb90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9280007d40 tx=0x7f9280009e30 comp rx=0 tx=0).stop 2024-05-12T03:59:59.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.075+0000 7f9290a8d700 1 -- 172.21.15.32:0/48637032 shutdown_connections 2024-05-12T03:59:59.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.075+0000 7f9290a8d700 1 --2- 172.21.15.32:0/48637032 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f92740692a0 0x7f927406b750 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:59.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.075+0000 7f9290a8d700 1 --2- 172.21.15.32:0/48637032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f928c0fbcb0 0x7f928c10b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:59.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.075+0000 7f9290a8d700 1 --2- 172.21.15.32:0/48637032 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f928c0fcf60 0x7f928c10bb90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:59.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.075+0000 7f9290a8d700 1 -- 172.21.15.32:0/48637032 >> 172.21.15.32:0/48637032 conn(0x7f928c0f7430 msgr2=0x7f928c065bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:59:59.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.075+0000 7f9290a8d700 1 -- 172.21.15.32:0/48637032 shutdown_connections 2024-05-12T03:59:59.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.075+0000 7f9290a8d700 1 -- 172.21.15.32:0/48637032 wait complete. 2024-05-12T03:59:59.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:59:58 smithi033 ceph-mon[49525]: pgmap v253: 65 pgs: 65 active+clean; 7.3 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 577 KiB/s wr, 6 op/s 2024-05-12T03:59:59.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 03:59:58 smithi033 ceph-mon[49525]: from='client.14618 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T03:59:59.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.219+0000 7fe4b5b31700 1 -- 172.21.15.32:0/3602961184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4b00fc090 msgr2=0x7fe4b00fc4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:59:59.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.219+0000 7fe4b5b31700 1 --2- 172.21.15.32:0/3602961184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4b00fc090 0x7fe4b00fc4a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8009ce0 tx=0x7fe4a8004d10 comp rx=0 tx=0).stop 2024-05-12T03:59:59.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.219+0000 7fe4b5b31700 1 -- 172.21.15.32:0/3602961184 shutdown_connections 2024-05-12T03:59:59.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.219+0000 7fe4b5b31700 1 --2- 172.21.15.32:0/3602961184 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4b00fab20 0x7fe4b00faf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:59.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.219+0000 7fe4b5b31700 1 --2- 172.21.15.32:0/3602961184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4b00fc090 0x7fe4b00fc4a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:59.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.219+0000 7fe4b5b31700 1 -- 172.21.15.32:0/3602961184 >> 172.21.15.32:0/3602961184 conn(0x7fe4b00f6740 msgr2=0x7fe4b00f8b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:59:59.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.220+0000 7fe4b5b31700 1 -- 172.21.15.32:0/3602961184 shutdown_connections 2024-05-12T03:59:59.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.220+0000 7fe4b5b31700 1 -- 172.21.15.32:0/3602961184 wait complete. 2024-05-12T03:59:59.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.220+0000 7fe4b5b31700 1 Processor -- start 2024-05-12T03:59:59.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.221+0000 7fe4b5b31700 1 -- start start 2024-05-12T03:59:59.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.221+0000 7fe4b5b31700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4b00fab20 0x7fe4b010b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:59:59.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.221+0000 7fe4b5b31700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4b00fc090 0x7fe4b010bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:59:59.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.221+0000 7fe4b5b31700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b010c3c0 con 0x7fe4b00fab20 2024-05-12T03:59:59.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.221+0000 7fe4b5b31700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b0110d50 con 0x7fe4b00fc090 2024-05-12T03:59:59.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.221+0000 7fe4af7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4b00fab20 0x7fe4b010b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:59:59.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.221+0000 7fe4af7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4b00fab20 0x7fe4b010b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58178/0 (socket says 172.21.15.32:58178) 2024-05-12T03:59:59.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.221+0000 7fe4af7fe700 1 -- 172.21.15.32:0/2202679195 learned_addr learned my addr 172.21.15.32:0/2202679195 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:59:59.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.221+0000 7fe4aeffd700 1 --2- 172.21.15.32:0/2202679195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4b00fc090 0x7fe4b010bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:59:59.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.222+0000 7fe4af7fe700 1 -- 172.21.15.32:0/2202679195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4b00fc090 msgr2=0x7fe4b010bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:59:59.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.222+0000 7fe4af7fe700 1 --2- 172.21.15.32:0/2202679195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4b00fc090 0x7fe4b010bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:59.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.222+0000 7fe4af7fe700 1 -- 172.21.15.32:0/2202679195 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4a8009940 con 0x7fe4b00fab20 2024-05-12T03:59:59.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.222+0000 7fe4aeffd700 1 --2- 172.21.15.32:0/2202679195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4b00fc090 0x7fe4b010bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-12T03:59:59.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.222+0000 7fe4af7fe700 1 --2- 172.21.15.32:0/2202679195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4b00fab20 0x7fe4b010b7a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8005010 tx=0x7fe4a8004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:59:59.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.222+0000 7fe497fff700 1 -- 172.21.15.32:0/2202679195 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4a800e040 con 0x7fe4b00fab20 2024-05-12T03:59:59.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.222+0000 7fe497fff700 1 -- 172.21.15.32:0/2202679195 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe4a80128c0 con 0x7fe4b00fab20 2024-05-12T03:59:59.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.222+0000 7fe497fff700 1 -- 172.21.15.32:0/2202679195 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4a801c760 con 0x7fe4b00fab20 2024-05-12T03:59:59.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.222+0000 7fe4b5b31700 1 -- 172.21.15.32:0/2202679195 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4b0110fd0 con 0x7fe4b00fab20 2024-05-12T03:59:59.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.222+0000 7fe4b5b31700 1 -- 172.21.15.32:0/2202679195 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4b0111590 con 0x7fe4b00fab20 2024-05-12T03:59:59.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.224+0000 7fe4b5b31700 1 -- 172.21.15.32:0/2202679195 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4b0102f60 con 0x7fe4b00fab20 2024-05-12T03:59:59.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.225+0000 7fe497fff700 1 -- 172.21.15.32:0/2202679195 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fe4a8012a30 con 0x7fe4b00fab20 2024-05-12T03:59:59.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.226+0000 7fe497fff700 1 --2- 172.21.15.32:0/2202679195 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fe4980692e0 0x7fe49806b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:59:59.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.226+0000 7fe497fff700 1 -- 172.21.15.32:0/2202679195 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fe4a8089280 con 0x7fe4b00fab20 2024-05-12T03:59:59.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.226+0000 7fe4aeffd700 1 --2- 172.21.15.32:0/2202679195 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fe4980692e0 0x7fe49806b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:59:59.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.226+0000 7fe4aeffd700 1 --2- 172.21.15.32:0/2202679195 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fe4980692e0 0x7fe49806b790 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4006fd0 tx=0x7fe4a4008040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:59:59.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.229+0000 7fe497fff700 1 -- 172.21.15.32:0/2202679195 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fe4a8055540 con 0x7fe4b00fab20 2024-05-12T03:59:59.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:59:58 smithi032 ceph-mon[40008]: pgmap v253: 65 pgs: 65 active+clean; 7.3 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 577 KiB/s wr, 6 op/s 2024-05-12T03:59:59.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 03:59:58 smithi032 ceph-mon[40008]: from='client.14618 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T03:59:59.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.623+0000 7fe4b5b31700 1 -- 172.21.15.32:0/2202679195 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe4b01079d0 con 0x7fe4b00fab20 2024-05-12T03:59:59.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.624+0000 7fe497fff700 1 -- 172.21.15.32:0/2202679195 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7fe4a8058b60 con 0x7fe4b00fab20 2024-05-12T03:59:59.625 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-05-12T03:59:59.625 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": { 2024-05-12T03:59:59.625 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 2 2024-05-12T03:59:59.625 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T03:59:59.625 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": { 2024-05-12T03:59:59.625 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 2 2024-05-12T03:59:59.626 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T03:59:59.626 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": { 2024-05-12T03:59:59.626 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 6 2024-05-12T03:59:59.626 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T03:59:59.626 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": { 2024-05-12T03:59:59.626 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 4 2024-05-12T03:59:59.626 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T03:59:59.626 INFO:teuthology.orchestra.run.smithi032.stdout: "overall": { 2024-05-12T03:59:59.626 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 14 2024-05-12T03:59:59.626 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-05-12T03:59:59.626 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-05-12T03:59:59.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.626+0000 7fe4b5b31700 1 -- 172.21.15.32:0/2202679195 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fe4980692e0 msgr2=0x7fe49806b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:59:59.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.626+0000 7fe4b5b31700 1 --2- 172.21.15.32:0/2202679195 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fe4980692e0 0x7fe49806b790 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4006fd0 tx=0x7fe4a4008040 comp rx=0 tx=0).stop 2024-05-12T03:59:59.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.626+0000 7fe4b5b31700 1 -- 172.21.15.32:0/2202679195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4b00fab20 msgr2=0x7fe4b010b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:59:59.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.626+0000 7fe4b5b31700 1 --2- 172.21.15.32:0/2202679195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4b00fab20 0x7fe4b010b7a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8005010 tx=0x7fe4a8004d10 comp rx=0 tx=0).stop 2024-05-12T03:59:59.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.626+0000 7fe4b5b31700 1 -- 172.21.15.32:0/2202679195 shutdown_connections 2024-05-12T03:59:59.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.626+0000 7fe4b5b31700 1 --2- 172.21.15.32:0/2202679195 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7fe4980692e0 0x7fe49806b790 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:59.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.626+0000 7fe4b5b31700 1 --2- 172.21.15.32:0/2202679195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4b00fab20 0x7fe4b010b7a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:59.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.626+0000 7fe4b5b31700 1 --2- 172.21.15.32:0/2202679195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4b00fc090 0x7fe4b010bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:59.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.626+0000 7fe4b5b31700 1 -- 172.21.15.32:0/2202679195 >> 172.21.15.32:0/2202679195 conn(0x7fe4b00f6740 msgr2=0x7fe4b00f7e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:59:59.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.626+0000 7fe4b5b31700 1 -- 172.21.15.32:0/2202679195 shutdown_connections 2024-05-12T03:59:59.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.626+0000 7fe4b5b31700 1 -- 172.21.15.32:0/2202679195 wait complete. 2024-05-12T03:59:59.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.757+0000 7f4b6a291700 1 -- 172.21.15.32:0/3507491856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b640fbcb0 msgr2=0x7f4b640fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:59:59.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.757+0000 7f4b6a291700 1 --2- 172.21.15.32:0/3507491856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b640fbcb0 0x7f4b640fc0c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4b54004940 tx=0x7f4b54009e30 comp rx=0 tx=0).stop 2024-05-12T03:59:59.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.757+0000 7f4b6a291700 1 -- 172.21.15.32:0/3507491856 shutdown_connections 2024-05-12T03:59:59.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.757+0000 7f4b6a291700 1 --2- 172.21.15.32:0/3507491856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b640fcf60 0x7f4b640fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:59.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.757+0000 7f4b6a291700 1 --2- 172.21.15.32:0/3507491856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b640fbcb0 0x7f4b640fc0c0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:59.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.757+0000 7f4b6a291700 1 -- 172.21.15.32:0/3507491856 >> 172.21.15.32:0/3507491856 conn(0x7f4b640f7430 msgr2=0x7f4b640f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T03:59:59.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.757+0000 7f4b6a291700 1 -- 172.21.15.32:0/3507491856 shutdown_connections 2024-05-12T03:59:59.759 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.757+0000 7f4b6a291700 1 -- 172.21.15.32:0/3507491856 wait complete. 2024-05-12T03:59:59.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.758+0000 7f4b6a291700 1 Processor -- start 2024-05-12T03:59:59.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.758+0000 7f4b6a291700 1 -- start start 2024-05-12T03:59:59.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.758+0000 7f4b6a291700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b640fbcb0 0x7f4b6410e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:59:59.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.758+0000 7f4b6a291700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b640fcf60 0x7f4b64110870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:59:59.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.758+0000 7f4b6a291700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b64066390 con 0x7f4b640fbcb0 2024-05-12T03:59:59.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.758+0000 7f4b6a291700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b640664d0 con 0x7f4b640fcf60 2024-05-12T03:59:59.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.758+0000 7f4b637fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b640fcf60 0x7f4b64110870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:59:59.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.759+0000 7f4b637fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b640fcf60 0x7f4b64110870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.32:60216/0 (socket says 172.21.15.32:60216) 2024-05-12T03:59:59.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.759+0000 7f4b637fe700 1 -- 172.21.15.32:0/4089866534 learned_addr learned my addr 172.21.15.32:0/4089866534 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T03:59:59.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.759+0000 7f4b63fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b640fbcb0 0x7f4b6410e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:59:59.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.759+0000 7f4b637fe700 1 -- 172.21.15.32:0/4089866534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b640fbcb0 msgr2=0x7f4b6410e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T03:59:59.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.759+0000 7f4b637fe700 1 --2- 172.21.15.32:0/4089866534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b640fbcb0 0x7f4b6410e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T03:59:59.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.759+0000 7f4b637fe700 1 -- 172.21.15.32:0/4089866534 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b54005040 con 0x7f4b640fcf60 2024-05-12T03:59:59.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.759+0000 7f4b63fff700 1 --2- 172.21.15.32:0/4089866534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b640fbcb0 0x7f4b6410e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-12T03:59:59.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.759+0000 7f4b637fe700 1 --2- 172.21.15.32:0/4089866534 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b640fcf60 0x7f4b64110870 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4b5800eb70 tx=0x7f4b5800b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:59:59.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.759+0000 7f4b60ff9700 1 -- 172.21.15.32:0/4089866534 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b58007cc0 con 0x7f4b640fcf60 2024-05-12T03:59:59.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.759+0000 7f4b6a291700 1 -- 172.21.15.32:0/4089866534 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b64110e10 con 0x7f4b640fcf60 2024-05-12T03:59:59.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.759+0000 7f4b60ff9700 1 -- 172.21.15.32:0/4089866534 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4b5800f040 con 0x7f4b640fcf60 2024-05-12T03:59:59.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.759+0000 7f4b6a291700 1 -- 172.21.15.32:0/4089866534 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b641113d0 con 0x7f4b640fcf60 2024-05-12T03:59:59.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.759+0000 7f4b60ff9700 1 -- 172.21.15.32:0/4089866534 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b5800c9f0 con 0x7f4b640fcf60 2024-05-12T03:59:59.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.760+0000 7f4b6a291700 1 -- 172.21.15.32:0/4089866534 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b6405f5c0 con 0x7f4b640fcf60 2024-05-12T03:59:59.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.761+0000 7f4b60ff9700 1 -- 172.21.15.32:0/4089866534 <== mon.1 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f4b580090d0 con 0x7f4b640fcf60 2024-05-12T03:59:59.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.761+0000 7f4b60ff9700 1 --2- 172.21.15.32:0/4089866534 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f4b4c0692a0 0x7f4b4c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T03:59:59.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.761+0000 7f4b60ff9700 1 -- 172.21.15.32:0/4089866534 <== mon.1 v2:172.21.15.33:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f4b58086c80 con 0x7f4b640fcf60 2024-05-12T03:59:59.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.761+0000 7f4b63fff700 1 --2- 172.21.15.32:0/4089866534 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f4b4c0692a0 0x7f4b4c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T03:59:59.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.762+0000 7f4b63fff700 1 --2- 172.21.15.32:0/4089866534 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f4b4c0692a0 0x7f4b4c06b750 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4b54013680 tx=0x7f4b54009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T03:59:59.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T03:59:59.766+0000 7f4b60ff9700 1 -- 172.21.15.32:0/4089866534 <== mon.1 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f4b58052f60 con 0x7f4b640fcf60 2024-05-12T04:00:00.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.097+0000 7f4b6a291700 1 -- 172.21.15.32:0/4089866534 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4b641116e0 con 0x7f4b640fcf60 2024-05-12T04:00:00.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.098+0000 7f4b60ff9700 1 -- 172.21.15.32:0/4089866534 <== mon.1 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1828 (secure 0 0 0) 0x7f4b58019740 con 0x7f4b640fcf60 2024-05-12T04:00:00.099 INFO:teuthology.orchestra.run.smithi032.stdout:e18 2024-05-12T04:00:00.099 INFO:teuthology.orchestra.run.smithi032.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-12T04:00:00.099 INFO:teuthology.orchestra.run.smithi032.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-12T04:00:00.099 INFO:teuthology.orchestra.run.smithi032.stdout:legacy client fscid: 1 2024-05-12T04:00:00.099 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:00:00.099 INFO:teuthology.orchestra.run.smithi032.stdout:Filesystem 'cephfs' (1) 2024-05-12T04:00:00.099 INFO:teuthology.orchestra.run.smithi032.stdout:fs_name cephfs 2024-05-12T04:00:00.099 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 18 2024-05-12T04:00:00.100 INFO:teuthology.orchestra.run.smithi032.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-12T04:00:00.100 INFO:teuthology.orchestra.run.smithi032.stdout:created 2024-05-12T03:57:48.904614+0000 2024-05-12T04:00:00.100 INFO:teuthology.orchestra.run.smithi032.stdout:modified 2024-05-12T03:58:06.821591+0000 2024-05-12T04:00:00.100 INFO:teuthology.orchestra.run.smithi032.stdout:tableserver 0 2024-05-12T04:00:00.100 INFO:teuthology.orchestra.run.smithi032.stdout:root 0 2024-05-12T04:00:00.100 INFO:teuthology.orchestra.run.smithi032.stdout:session_timeout 60 2024-05-12T04:00:00.100 INFO:teuthology.orchestra.run.smithi032.stdout:session_autoclose 300 2024-05-12T04:00:00.100 INFO:teuthology.orchestra.run.smithi032.stdout:max_file_size 1099511627776 2024-05-12T04:00:00.100 INFO:teuthology.orchestra.run.smithi032.stdout:required_client_features {} 2024-05-12T04:00:00.100 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure 0 2024-05-12T04:00:00.100 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure_osd_epoch 46 2024-05-12T04:00:00.100 INFO:teuthology.orchestra.run.smithi032.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-12T04:00:00.100 INFO:teuthology.orchestra.run.smithi032.stdout:max_mds 1 2024-05-12T04:00:00.101 INFO:teuthology.orchestra.run.smithi032.stdout:in 0 2024-05-12T04:00:00.101 INFO:teuthology.orchestra.run.smithi032.stdout:up {0=14474} 2024-05-12T04:00:00.101 INFO:teuthology.orchestra.run.smithi032.stdout:failed 2024-05-12T04:00:00.101 INFO:teuthology.orchestra.run.smithi032.stdout:damaged 2024-05-12T04:00:00.101 INFO:teuthology.orchestra.run.smithi032.stdout:stopped 2024-05-12T04:00:00.101 INFO:teuthology.orchestra.run.smithi032.stdout:data_pools [3] 2024-05-12T04:00:00.101 INFO:teuthology.orchestra.run.smithi032.stdout:metadata_pool 2 2024-05-12T04:00:00.101 INFO:teuthology.orchestra.run.smithi032.stdout:inline_data enabled 2024-05-12T04:00:00.101 INFO:teuthology.orchestra.run.smithi032.stdout:balancer 2024-05-12T04:00:00.101 INFO:teuthology.orchestra.run.smithi032.stdout:standby_count_wanted 1 2024-05-12T04:00:00.101 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.qpsawh{0:14474} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.32:6828/3100155296,v1:172.21.15.32:6829/3100155296] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:00:00.101 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi033.jgbjhk{0:24267} state up:standby-replay seq 3 join_fscid=1 addr [v2:172.21.15.33:6824/55417471,v1:172.21.15.33:6825/55417471] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:00:00.102 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:00:00.102 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:00:00.102 INFO:teuthology.orchestra.run.smithi032.stdout:Standby daemons: 2024-05-12T04:00:00.102 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:00:00.102 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi033.hglwml{-1:24283} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/433858238,v1:172.21.15.33:6827/433858238] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:00:00.102 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.ialtqb{-1:24287} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.32:6826/3703132329,v1:172.21.15.32:6827/3703132329] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:00:00.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.100+0000 7f4b6a291700 1 -- 172.21.15.32:0/4089866534 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f4b4c0692a0 msgr2=0x7f4b4c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:00:00.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.100+0000 7f4b6a291700 1 --2- 172.21.15.32:0/4089866534 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f4b4c0692a0 0x7f4b4c06b750 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4b54013680 tx=0x7f4b54009e30 comp rx=0 tx=0).stop 2024-05-12T04:00:00.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.100+0000 7f4b6a291700 1 -- 172.21.15.32:0/4089866534 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b640fcf60 msgr2=0x7f4b64110870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:00:00.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.100+0000 7f4b6a291700 1 --2- 172.21.15.32:0/4089866534 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b640fcf60 0x7f4b64110870 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4b5800eb70 tx=0x7f4b5800b040 comp rx=0 tx=0).stop 2024-05-12T04:00:00.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.100+0000 7f4b6a291700 1 -- 172.21.15.32:0/4089866534 shutdown_connections 2024-05-12T04:00:00.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.100+0000 7f4b6a291700 1 --2- 172.21.15.32:0/4089866534 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f4b4c0692a0 0x7f4b4c06b750 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:00:00.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.100+0000 7f4b6a291700 1 --2- 172.21.15.32:0/4089866534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b640fbcb0 0x7f4b6410e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:00:00.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.100+0000 7f4b6a291700 1 --2- 172.21.15.32:0/4089866534 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b640fcf60 0x7f4b64110870 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:00:00.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.101+0000 7f4b6a291700 1 -- 172.21.15.32:0/4089866534 >> 172.21.15.32:0/4089866534 conn(0x7f4b640f7430 msgr2=0x7f4b640f8b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:00:00.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.101+0000 7f4b6a291700 1 -- 172.21.15.32:0/4089866534 shutdown_connections 2024-05-12T04:00:00.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.101+0000 7f4b6a291700 1 -- 172.21.15.32:0/4089866534 wait complete. 2024-05-12T04:00:00.104 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 18 2024-05-12T04:00:00.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.221+0000 7f2981235700 1 -- 172.21.15.32:0/1578644729 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f297c0fff50 msgr2=0x7f297c102330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:00:00.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.221+0000 7f2981235700 1 --2- 172.21.15.32:0/1578644729 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f297c0fff50 0x7f297c102330 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2970004940 tx=0x7f2970009e30 comp rx=0 tx=0).stop 2024-05-12T04:00:00.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.221+0000 7f2981235700 1 -- 172.21.15.32:0/1578644729 shutdown_connections 2024-05-12T04:00:00.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.221+0000 7f2981235700 1 --2- 172.21.15.32:0/1578644729 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f297c0fff50 0x7f297c102330 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:00:00.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.221+0000 7f2981235700 1 --2- 172.21.15.32:0/1578644729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f297c066b10 0x7f297c0ffa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:00:00.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.221+0000 7f2981235700 1 -- 172.21.15.32:0/1578644729 >> 172.21.15.32:0/1578644729 conn(0x7f297c0f7440 msgr2=0x7f297c0f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:00:00.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.221+0000 7f2981235700 1 -- 172.21.15.32:0/1578644729 shutdown_connections 2024-05-12T04:00:00.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.221+0000 7f2981235700 1 -- 172.21.15.32:0/1578644729 wait complete. 2024-05-12T04:00:00.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.222+0000 7f2981235700 1 Processor -- start 2024-05-12T04:00:00.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.222+0000 7f2981235700 1 -- start start 2024-05-12T04:00:00.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.222+0000 7f2981235700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f297c066b10 0x7f297c10d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:00:00.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.222+0000 7f2981235700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f297c0fff50 0x7f297c10de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:00:00.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.222+0000 7f2981235700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f297c10e410 con 0x7f297c0fff50 2024-05-12T04:00:00.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.222+0000 7f2981235700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f297c10ec60 con 0x7f297c066b10 2024-05-12T04:00:00.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.222+0000 7f297ad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f297c066b10 0x7f297c10d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:00:00.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.222+0000 7f297ad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f297c066b10 0x7f297c10d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.32:60236/0 (socket says 172.21.15.32:60236) 2024-05-12T04:00:00.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.222+0000 7f297ad9d700 1 -- 172.21.15.32:0/3813667618 learned_addr learned my addr 172.21.15.32:0/3813667618 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:00:00.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.222+0000 7f297a59c700 1 --2- 172.21.15.32:0/3813667618 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f297c0fff50 0x7f297c10de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:00:00.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.222+0000 7f297ad9d700 1 -- 172.21.15.32:0/3813667618 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f297c0fff50 msgr2=0x7f297c10de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:00:00.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.222+0000 7f297ad9d700 1 --2- 172.21.15.32:0/3813667618 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f297c0fff50 0x7f297c10de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:00:00.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.222+0000 7f297ad9d700 1 -- 172.21.15.32:0/3813667618 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2970005040 con 0x7f297c066b10 2024-05-12T04:00:00.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.223+0000 7f297ad9d700 1 --2- 172.21.15.32:0/3813667618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f297c066b10 0x7f297c10d8d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f296c00eb70 tx=0x7f296c00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:00:00.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.223+0000 7f29637fe700 1 -- 172.21.15.32:0/3813667618 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f296c007cc0 con 0x7f297c066b10 2024-05-12T04:00:00.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.223+0000 7f2981235700 1 -- 172.21.15.32:0/3813667618 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f297c10ee60 con 0x7f297c066b10 2024-05-12T04:00:00.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.223+0000 7f2981235700 1 -- 172.21.15.32:0/3813667618 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f297c10f420 con 0x7f297c066b10 2024-05-12T04:00:00.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.223+0000 7f29637fe700 1 -- 172.21.15.32:0/3813667618 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f296c00f040 con 0x7f297c066b10 2024-05-12T04:00:00.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.223+0000 7f29637fe700 1 -- 172.21.15.32:0/3813667618 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f296c00ca10 con 0x7f297c066b10 2024-05-12T04:00:00.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.224+0000 7f29637fe700 1 -- 172.21.15.32:0/3813667618 <== mon.1 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f296c00cb70 con 0x7f297c066b10 2024-05-12T04:00:00.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.225+0000 7f29637fe700 1 --2- 172.21.15.32:0/3813667618 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f29640692a0 0x7f296406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:00:00.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.225+0000 7f29637fe700 1 -- 172.21.15.32:0/3813667618 <== mon.1 v2:172.21.15.33:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f296c087c10 con 0x7f297c066b10 2024-05-12T04:00:00.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.225+0000 7f297a59c700 1 --2- 172.21.15.32:0/3813667618 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f29640692a0 0x7f296406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:00:00.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.225+0000 7f2981235700 1 -- 172.21.15.32:0/3813667618 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29680052e0 con 0x7f297c066b10 2024-05-12T04:00:00.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.226+0000 7f297a59c700 1 --2- 172.21.15.32:0/3813667618 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f29640692a0 0x7f296406b750 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2970013600 tx=0x7f2970009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:00:00.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.231+0000 7f29637fe700 1 -- 172.21.15.32:0/3813667618 <== mon.1 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f296c04e250 con 0x7f297c066b10 2024-05-12T04:00:00.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:00 smithi032 ceph-mon[40008]: from='client.14622 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:00:00.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:00 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T04:00:00.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:00 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T04:00:00.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:00 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:00:00.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:00 smithi032 ceph-mon[40008]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:00:00.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:00 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/2202679195' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:00:00.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:00 smithi032 ceph-mon[40008]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-12T04:00:00.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:00 smithi032 ceph-mon[40008]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-12T04:00:00.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:00 smithi032 ceph-mon[40008]: fs cephfs has deprecated feature inline_data enabled. 2024-05-12T04:00:00.333 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:00 smithi033 ceph-mon[49525]: from='client.14622 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:00:00.333 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:00 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T04:00:00.333 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:00 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T04:00:00.333 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:00 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:00:00.333 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:00 smithi033 ceph-mon[49525]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:00:00.333 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:00 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/2202679195' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:00:00.333 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:00 smithi033 ceph-mon[49525]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-12T04:00:00.333 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:00 smithi033 ceph-mon[49525]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-12T04:00:00.333 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:00 smithi033 ceph-mon[49525]: fs cephfs has deprecated feature inline_data enabled. 2024-05-12T04:00:00.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.469+0000 7f2981235700 1 -- 172.21.15.32:0/3813667618 --> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2968000bf0 con 0x7f29640692a0 2024-05-12T04:00:00.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.472+0000 7f29637fe700 1 -- 172.21.15.32:0/3813667618 <== mgr.14208 v2:172.21.15.32:6800/4089797100 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+328 (secure 0 0 0) 0x7f2968000bf0 con 0x7f29640692a0 2024-05-12T04:00:00.473 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-05-12T04:00:00.473 INFO:teuthology.orchestra.run.smithi032.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:19c9734256aa43a60e203feb242f50bf8b7f58666d520536af43cc4fee96173a", 2024-05-12T04:00:00.473 INFO:teuthology.orchestra.run.smithi032.stdout: "in_progress": true, 2024-05-12T04:00:00.474 INFO:teuthology.orchestra.run.smithi032.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-12T04:00:00.474 INFO:teuthology.orchestra.run.smithi032.stdout: "services_complete": [], 2024-05-12T04:00:00.474 INFO:teuthology.orchestra.run.smithi032.stdout: "progress": "", 2024-05-12T04:00:00.474 INFO:teuthology.orchestra.run.smithi032.stdout: "message": "Currently upgrading mgr daemons", 2024-05-12T04:00:00.474 INFO:teuthology.orchestra.run.smithi032.stdout: "is_paused": false 2024-05-12T04:00:00.474 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-05-12T04:00:00.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.475+0000 7f2981235700 1 -- 172.21.15.32:0/3813667618 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f29640692a0 msgr2=0x7f296406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:00:00.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.475+0000 7f2981235700 1 --2- 172.21.15.32:0/3813667618 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f29640692a0 0x7f296406b750 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2970013600 tx=0x7f2970009e30 comp rx=0 tx=0).stop 2024-05-12T04:00:00.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.475+0000 7f2981235700 1 -- 172.21.15.32:0/3813667618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f297c066b10 msgr2=0x7f297c10d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:00:00.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.475+0000 7f2981235700 1 --2- 172.21.15.32:0/3813667618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f297c066b10 0x7f297c10d8d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f296c00eb70 tx=0x7f296c00b040 comp rx=0 tx=0).stop 2024-05-12T04:00:00.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.475+0000 7f2981235700 1 -- 172.21.15.32:0/3813667618 shutdown_connections 2024-05-12T04:00:00.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.475+0000 7f2981235700 1 --2- 172.21.15.32:0/3813667618 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f29640692a0 0x7f296406b750 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:00:00.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.475+0000 7f2981235700 1 --2- 172.21.15.32:0/3813667618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f297c066b10 0x7f297c10d8d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:00:00.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.475+0000 7f2981235700 1 --2- 172.21.15.32:0/3813667618 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f297c0fff50 0x7f297c10de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:00:00.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.475+0000 7f2981235700 1 -- 172.21.15.32:0/3813667618 >> 172.21.15.32:0/3813667618 conn(0x7f297c0f7440 msgr2=0x7f297c0f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:00:00.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.476+0000 7f2981235700 1 -- 172.21.15.32:0/3813667618 shutdown_connections 2024-05-12T04:00:00.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.476+0000 7f2981235700 1 -- 172.21.15.32:0/3813667618 wait complete. 2024-05-12T04:00:00.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.621+0000 7f75ae12c700 1 -- 172.21.15.32:0/2413230112 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75a80fbcb0 msgr2=0x7f75a80fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:00:00.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.621+0000 7f75ae12c700 1 --2- 172.21.15.32:0/2413230112 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75a80fbcb0 0x7f75a80fc0c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f75980049c0 tx=0x7f7598009e30 comp rx=0 tx=0).stop 2024-05-12T04:00:00.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.622+0000 7f75ae12c700 1 -- 172.21.15.32:0/2413230112 shutdown_connections 2024-05-12T04:00:00.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.622+0000 7f75ae12c700 1 --2- 172.21.15.32:0/2413230112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75a80fcf60 0x7f75a80fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:00:00.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.622+0000 7f75ae12c700 1 --2- 172.21.15.32:0/2413230112 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75a80fbcb0 0x7f75a80fc0c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:00:00.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.622+0000 7f75ae12c700 1 -- 172.21.15.32:0/2413230112 >> 172.21.15.32:0/2413230112 conn(0x7f75a80f7430 msgr2=0x7f75a80f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:00:00.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.622+0000 7f75ae12c700 1 -- 172.21.15.32:0/2413230112 shutdown_connections 2024-05-12T04:00:00.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.622+0000 7f75ae12c700 1 -- 172.21.15.32:0/2413230112 wait complete. 2024-05-12T04:00:00.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.622+0000 7f75ae12c700 1 Processor -- start 2024-05-12T04:00:00.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.622+0000 7f75ae12c700 1 -- start start 2024-05-12T04:00:00.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.623+0000 7f75ae12c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75a80fbcb0 0x7f75a810f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:00:00.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.623+0000 7f75ae12c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75a80fcf60 0x7f75a810ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:00:00.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.623+0000 7f75ae12c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a8110510 con 0x7f75a80fcf60 2024-05-12T04:00:00.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.623+0000 7f75ae12c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a8110680 con 0x7f75a80fbcb0 2024-05-12T04:00:00.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.623+0000 7f75a6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75a80fcf60 0x7f75a810ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:00:00.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.623+0000 7f75a77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75a80fbcb0 0x7f75a810f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:00:00.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.623+0000 7f75a6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75a80fcf60 0x7f75a810ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58220/0 (socket says 172.21.15.32:58220) 2024-05-12T04:00:00.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.623+0000 7f75a6ffd700 1 -- 172.21.15.32:0/1664051126 learned_addr learned my addr 172.21.15.32:0/1664051126 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:00:00.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.623+0000 7f75a77fe700 1 -- 172.21.15.32:0/1664051126 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75a80fcf60 msgr2=0x7f75a810ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:00:00.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.623+0000 7f75a77fe700 1 --2- 172.21.15.32:0/1664051126 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75a80fcf60 0x7f75a810ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:00:00.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.623+0000 7f75a77fe700 1 -- 172.21.15.32:0/1664051126 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7598005040 con 0x7f75a80fbcb0 2024-05-12T04:00:00.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.623+0000 7f75a6ffd700 1 --2- 172.21.15.32:0/1664051126 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75a80fcf60 0x7f75a810ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-12T04:00:00.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.623+0000 7f75a77fe700 1 --2- 172.21.15.32:0/1664051126 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75a80fbcb0 0x7f75a810f9d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7598009630 tx=0x7f7598009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:00:00.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.624+0000 7f758ffff700 1 -- 172.21.15.32:0/1664051126 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7598011480 con 0x7f75a80fbcb0 2024-05-12T04:00:00.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.624+0000 7f75ae12c700 1 -- 172.21.15.32:0/1664051126 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75a8115080 con 0x7f75a80fbcb0 2024-05-12T04:00:00.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.624+0000 7f758ffff700 1 -- 172.21.15.32:0/1664051126 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7598011a90 con 0x7f75a80fbcb0 2024-05-12T04:00:00.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.624+0000 7f758ffff700 1 -- 172.21.15.32:0/1664051126 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f759801b920 con 0x7f75a80fbcb0 2024-05-12T04:00:00.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.624+0000 7f75ae12c700 1 -- 172.21.15.32:0/1664051126 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75a8115560 con 0x7f75a80fbcb0 2024-05-12T04:00:00.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.625+0000 7f75ae12c700 1 -- 172.21.15.32:0/1664051126 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75a804c710 con 0x7f75a80fbcb0 2024-05-12T04:00:00.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.625+0000 7f758ffff700 1 -- 172.21.15.32:0/1664051126 <== mon.1 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f759801bb90 con 0x7f75a80fbcb0 2024-05-12T04:00:00.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.626+0000 7f758ffff700 1 --2- 172.21.15.32:0/1664051126 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f75900692a0 0x7f759006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:00:00.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.626+0000 7f758ffff700 1 -- 172.21.15.32:0/1664051126 <== mon.1 v2:172.21.15.33:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f7598089110 con 0x7f75a80fbcb0 2024-05-12T04:00:00.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.626+0000 7f75a6ffd700 1 --2- 172.21.15.32:0/1664051126 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f75900692a0 0x7f759006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:00:00.630 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.626+0000 7f75a6ffd700 1 --2- 172.21.15.32:0/1664051126 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f75900692a0 0x7f759006b750 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f759c0041d0 tx=0x7f759c00b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:00:00.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:00.630+0000 7f758ffff700 1 -- 172.21.15.32:0/1664051126 <== mon.1 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f7598058e70 con 0x7f75a80fbcb0 2024-05-12T04:00:01.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:01.066+0000 7f75ae12c700 1 -- 172.21.15.32:0/1664051126 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f75a8115930 con 0x7f75a80fbcb0 2024-05-12T04:00:01.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:01.067+0000 7f758ffff700 1 -- 172.21.15.32:0/1664051126 <== mon.1 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f7598058a00 con 0x7f75a80fbcb0 2024-05-12T04:00:01.068 INFO:teuthology.orchestra.run.smithi032.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-12T04:00:01.068 INFO:teuthology.orchestra.run.smithi032.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-12T04:00:01.068 INFO:teuthology.orchestra.run.smithi032.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-12T04:00:01.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:01.069+0000 7f75ae12c700 1 -- 172.21.15.32:0/1664051126 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f75900692a0 msgr2=0x7f759006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:00:01.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:01.069+0000 7f75ae12c700 1 --2- 172.21.15.32:0/1664051126 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f75900692a0 0x7f759006b750 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f759c0041d0 tx=0x7f759c00b040 comp rx=0 tx=0).stop 2024-05-12T04:00:01.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:01.069+0000 7f75ae12c700 1 -- 172.21.15.32:0/1664051126 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75a80fbcb0 msgr2=0x7f75a810f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:00:01.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:01.069+0000 7f75ae12c700 1 --2- 172.21.15.32:0/1664051126 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75a80fbcb0 0x7f75a810f9d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7598009630 tx=0x7f7598009e30 comp rx=0 tx=0).stop 2024-05-12T04:00:01.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:01.069+0000 7f75ae12c700 1 -- 172.21.15.32:0/1664051126 shutdown_connections 2024-05-12T04:00:01.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:01.069+0000 7f75ae12c700 1 --2- 172.21.15.32:0/1664051126 >> [v2:172.21.15.32:6800/4089797100,v1:172.21.15.32:6801/4089797100] conn(0x7f75900692a0 0x7f759006b750 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:00:01.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:01.069+0000 7f75ae12c700 1 --2- 172.21.15.32:0/1664051126 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75a80fbcb0 0x7f75a810f9d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:00:01.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:01.069+0000 7f75ae12c700 1 --2- 172.21.15.32:0/1664051126 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75a80fcf60 0x7f75a810ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:00:01.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:01.069+0000 7f75ae12c700 1 -- 172.21.15.32:0/1664051126 >> 172.21.15.32:0/1664051126 conn(0x7f75a80f7430 msgr2=0x7f75a80f8ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:00:01.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:01.070+0000 7f75ae12c700 1 -- 172.21.15.32:0/1664051126 shutdown_connections 2024-05-12T04:00:01.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:01.070+0000 7f75ae12c700 1 -- 172.21.15.32:0/1664051126 wait complete. 2024-05-12T04:00:01.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:01 smithi033 ceph-mon[49525]: pgmap v254: 65 pgs: 65 active+clean; 19 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 1.6 MiB/s wr, 30 op/s 2024-05-12T04:00:01.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:01 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/4089866534' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-12T04:00:01.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:01 smithi032 ceph-mon[40008]: pgmap v254: 65 pgs: 65 active+clean; 19 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 1.6 MiB/s wr, 30 op/s 2024-05-12T04:00:01.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:01 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/4089866534' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-12T04:00:02.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:02 smithi032 ceph-mon[40008]: from='client.24399 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:00:02.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:02 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/1664051126' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-12T04:00:02.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:02 smithi033 ceph-mon[49525]: from='client.24399 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:00:02.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:02 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/1664051126' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-12T04:00:03.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:03 smithi033 ceph-mon[49525]: pgmap v255: 65 pgs: 65 active+clean; 29 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 342 KiB/s rd, 2.4 MiB/s wr, 72 op/s 2024-05-12T04:00:03.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:03 smithi032 ceph-mon[40008]: pgmap v255: 65 pgs: 65 active+clean; 29 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 342 KiB/s rd, 2.4 MiB/s wr, 72 op/s 2024-05-12T04:00:05.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:05 smithi032 ceph-mon[40008]: pgmap v256: 65 pgs: 65 active+clean; 34 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 683 KiB/s rd, 2.9 MiB/s wr, 91 op/s 2024-05-12T04:00:05.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:05 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T04:00:05.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:05 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T04:00:05.380 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:05 smithi033 ceph-mon[49525]: pgmap v256: 65 pgs: 65 active+clean; 34 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 683 KiB/s rd, 2.9 MiB/s wr, 91 op/s 2024-05-12T04:00:05.380 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:05 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T04:00:05.380 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:05 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T04:00:06.362 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:06 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T04:00:06.362 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:06 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T04:00:06.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:06 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T04:00:06.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:06 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T04:00:07.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:07 smithi033 ceph-mon[49525]: pgmap v257: 65 pgs: 65 active+clean; 43 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 3.6 MiB/s wr, 133 op/s 2024-05-12T04:00:07.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:07 smithi033 ceph-mon[49525]: Standby manager daemon smithi033.pcvxwy restarted 2024-05-12T04:00:07.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:07 smithi033 ceph-mon[49525]: Standby manager daemon smithi033.pcvxwy started 2024-05-12T04:00:07.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:07 smithi033 ceph-mon[49525]: from='mgr.? 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi033.pcvxwy/crt"}]: dispatch 2024-05-12T04:00:07.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:07 smithi033 ceph-mon[49525]: from='mgr.? 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-12T04:00:07.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:07 smithi033 ceph-mon[49525]: from='mgr.? 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi033.pcvxwy/key"}]: dispatch 2024-05-12T04:00:07.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:07 smithi033 ceph-mon[49525]: from='mgr.? 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-12T04:00:07.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:07 smithi032 ceph-mon[40008]: pgmap v257: 65 pgs: 65 active+clean; 43 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 3.6 MiB/s wr, 133 op/s 2024-05-12T04:00:07.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:07 smithi032 ceph-mon[40008]: Standby manager daemon smithi033.pcvxwy restarted 2024-05-12T04:00:07.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:07 smithi032 ceph-mon[40008]: Standby manager daemon smithi033.pcvxwy started 2024-05-12T04:00:07.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:07 smithi032 ceph-mon[40008]: from='mgr.? 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi033.pcvxwy/crt"}]: dispatch 2024-05-12T04:00:07.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:07 smithi032 ceph-mon[40008]: from='mgr.? 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-12T04:00:07.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:07 smithi032 ceph-mon[40008]: from='mgr.? 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi033.pcvxwy/key"}]: dispatch 2024-05-12T04:00:07.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:07 smithi032 ceph-mon[40008]: from='mgr.? 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-12T04:00:08.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:08 smithi033 ceph-mon[49525]: mgrmap e20: smithi032.ukatkq(active, since 7m), standbys: smithi033.pcvxwy 2024-05-12T04:00:08.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:08 smithi032 ceph-mon[40008]: mgrmap e20: smithi032.ukatkq(active, since 7m), standbys: smithi033.pcvxwy 2024-05-12T04:00:09.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:09 smithi033 ceph-mon[49525]: pgmap v258: 65 pgs: 65 active+clean; 43 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 1023 KiB/s rd, 3.1 MiB/s wr, 128 op/s 2024-05-12T04:00:09.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:09 smithi032 ceph-mon[40008]: pgmap v258: 65 pgs: 65 active+clean; 43 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 1023 KiB/s rd, 3.1 MiB/s wr, 128 op/s 2024-05-12T04:00:10.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:10 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T04:00:10.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:10 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T04:00:10.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:10 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:00:10.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:10 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T04:00:10.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:10 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T04:00:10.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:10 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:00:10.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:10 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mgr fail", "who": "smithi032.ukatkq"}]: dispatch 2024-05-12T04:00:10.913 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:10 smithi033 ceph-mon[49525]: Activating manager daemon smithi033.pcvxwy 2024-05-12T04:00:10.913 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:10 smithi033 ceph-mon[49525]: osdmap e47: 6 total, 6 up, 6 in 2024-05-12T04:00:10.913 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:10 smithi033 ceph-mon[49525]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "mgr fail", "who": "smithi032.ukatkq"}]': finished 2024-05-12T04:00:10.913 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:10 smithi033 ceph-mon[49525]: mgrmap e21: smithi033.pcvxwy(active, starting, since 0.00618852s) 2024-05-12T04:00:10.913 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:10 smithi033 ceph-mon[49525]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "mon metadata", "id": "smithi032"}]: dispatch 2024-05-12T04:00:10.913 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:10 smithi033 ceph-mon[49525]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "mon metadata", "id": "smithi033"}]: dispatch 2024-05-12T04:00:10.913 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:10 smithi033 ceph-mon[49525]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi032.qpsawh"}]: dispatch 2024-05-12T04:00:10.913 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:10 smithi033 ceph-mon[49525]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi033.jgbjhk"}]: dispatch 2024-05-12T04:00:10.913 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:10 smithi033 ceph-mon[49525]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi033.hglwml"}]: dispatch 2024-05-12T04:00:10.913 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:10 smithi033 ceph-mon[49525]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi032.ialtqb"}]: dispatch 2024-05-12T04:00:10.913 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:10 smithi033 ceph-mon[49525]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "mgr metadata", "who": "smithi033.pcvxwy", "id": "smithi033.pcvxwy"}]: dispatch 2024-05-12T04:00:10.913 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:10 smithi033 ceph-mon[49525]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-12T04:00:10.913 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:10 smithi033 ceph-mon[49525]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-12T04:00:10.913 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:10 smithi033 ceph-mon[49525]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-12T04:00:10.914 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:10 smithi033 ceph-mon[49525]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-12T04:00:10.914 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:10 smithi033 ceph-mon[49525]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-12T04:00:10.914 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:10 smithi033 ceph-mon[49525]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-12T04:00:10.914 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:10 smithi033 ceph-mon[49525]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-12T04:00:10.914 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:10 smithi033 ceph-mon[49525]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-12T04:00:10.914 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:10 smithi033 ceph-mon[49525]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-12T04:00:10.914 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:10 smithi033 ceph-mon[49525]: Manager daemon smithi033.pcvxwy is now available 2024-05-12T04:00:10.914 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:10 smithi033 ceph-mon[49525]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:00:10.914 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:10 smithi033 ceph-mon[49525]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:00:10.914 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:10 smithi033 ceph-mon[49525]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi033.pcvxwy/mirror_snapshot_schedule"}]: dispatch 2024-05-12T04:00:10.914 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:10 smithi033 ceph-mon[49525]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi033.pcvxwy/mirror_snapshot_schedule"}]: dispatch 2024-05-12T04:00:10.914 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:10 smithi033 ceph-mon[49525]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi033.pcvxwy/trash_purge_schedule"}]: dispatch 2024-05-12T04:00:10.914 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:10 smithi033 ceph-mon[49525]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi033.pcvxwy/trash_purge_schedule"}]: dispatch 2024-05-12T04:00:11.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:10 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T04:00:11.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:10 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T04:00:11.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:10 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:00:11.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:10 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T04:00:11.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:10 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' 2024-05-12T04:00:11.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:10 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:00:11.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:10 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mgr fail", "who": "smithi032.ukatkq"}]: dispatch 2024-05-12T04:00:11.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:10 smithi032 ceph-mon[40008]: Activating manager daemon smithi033.pcvxwy 2024-05-12T04:00:11.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:10 smithi032 ceph-mon[40008]: osdmap e47: 6 total, 6 up, 6 in 2024-05-12T04:00:11.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:10 smithi032 ceph-mon[40008]: from='mgr.14208 172.21.15.32:0/2306213730' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "mgr fail", "who": "smithi032.ukatkq"}]': finished 2024-05-12T04:00:11.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:10 smithi032 ceph-mon[40008]: mgrmap e21: smithi033.pcvxwy(active, starting, since 0.00618852s) 2024-05-12T04:00:11.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:10 smithi032 ceph-mon[40008]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "mon metadata", "id": "smithi032"}]: dispatch 2024-05-12T04:00:11.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:10 smithi032 ceph-mon[40008]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "mon metadata", "id": "smithi033"}]: dispatch 2024-05-12T04:00:11.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:10 smithi032 ceph-mon[40008]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi032.qpsawh"}]: dispatch 2024-05-12T04:00:11.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:10 smithi032 ceph-mon[40008]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi033.jgbjhk"}]: dispatch 2024-05-12T04:00:11.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:10 smithi032 ceph-mon[40008]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi033.hglwml"}]: dispatch 2024-05-12T04:00:11.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:10 smithi032 ceph-mon[40008]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi032.ialtqb"}]: dispatch 2024-05-12T04:00:11.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:10 smithi032 ceph-mon[40008]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "mgr metadata", "who": "smithi033.pcvxwy", "id": "smithi033.pcvxwy"}]: dispatch 2024-05-12T04:00:11.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:10 smithi032 ceph-mon[40008]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-12T04:00:11.042 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:10 smithi032 ceph-mon[40008]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-12T04:00:11.042 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:10 smithi032 ceph-mon[40008]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-12T04:00:11.042 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:10 smithi032 ceph-mon[40008]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-12T04:00:11.042 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:10 smithi032 ceph-mon[40008]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-12T04:00:11.042 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:10 smithi032 ceph-mon[40008]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-12T04:00:11.042 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:10 smithi032 ceph-mon[40008]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-12T04:00:11.042 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:10 smithi032 ceph-mon[40008]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-12T04:00:11.042 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:10 smithi032 ceph-mon[40008]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-12T04:00:11.042 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:10 smithi032 ceph-mon[40008]: Manager daemon smithi033.pcvxwy is now available 2024-05-12T04:00:11.042 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:10 smithi032 ceph-mon[40008]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:00:11.042 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:10 smithi032 ceph-mon[40008]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:00:11.042 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:10 smithi032 ceph-mon[40008]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi033.pcvxwy/mirror_snapshot_schedule"}]: dispatch 2024-05-12T04:00:11.042 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:10 smithi032 ceph-mon[40008]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi033.pcvxwy/mirror_snapshot_schedule"}]: dispatch 2024-05-12T04:00:11.042 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:10 smithi032 ceph-mon[40008]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi033.pcvxwy/trash_purge_schedule"}]: dispatch 2024-05-12T04:00:11.042 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:10 smithi032 ceph-mon[40008]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi033.pcvxwy/trash_purge_schedule"}]: dispatch 2024-05-12T04:00:12.039 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:11 smithi032 ceph-mon[40008]: Deploying cephadm binary to smithi032 2024-05-12T04:00:12.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:11 smithi032 ceph-mon[40008]: Deploying cephadm binary to smithi033 2024-05-12T04:00:12.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:11 smithi032 ceph-mon[40008]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:00:12.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:11 smithi032 ceph-mon[40008]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:00:12.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:11 smithi032 ceph-mon[40008]: mgrmap e22: smithi033.pcvxwy(active, since 1.00854s) 2024-05-12T04:00:12.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:11 smithi032 ceph-mon[40008]: pgmap v3: 65 pgs: 65 active+clean; 70 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail 2024-05-12T04:00:12.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:11 smithi033 ceph-mon[49525]: Deploying cephadm binary to smithi032 2024-05-12T04:00:12.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:11 smithi033 ceph-mon[49525]: Deploying cephadm binary to smithi033 2024-05-12T04:00:12.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:11 smithi033 ceph-mon[49525]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:00:12.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:11 smithi033 ceph-mon[49525]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:00:12.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:11 smithi033 ceph-mon[49525]: mgrmap e22: smithi033.pcvxwy(active, since 1.00854s) 2024-05-12T04:00:12.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:11 smithi033 ceph-mon[49525]: pgmap v3: 65 pgs: 65 active+clean; 70 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail 2024-05-12T04:00:13.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:12 smithi032 ceph-mon[40008]: [12/May/2024:04:00:11] ENGINE Bus STARTING 2024-05-12T04:00:13.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:12 smithi032 ceph-mon[40008]: [12/May/2024:04:00:11] ENGINE Serving on http://172.21.15.33:8765 2024-05-12T04:00:13.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:12 smithi032 ceph-mon[40008]: [12/May/2024:04:00:11] ENGINE Serving on https://172.21.15.33:7150 2024-05-12T04:00:13.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:12 smithi032 ceph-mon[40008]: [12/May/2024:04:00:11] ENGINE Bus STARTED 2024-05-12T04:00:13.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:12 smithi032 ceph-mon[40008]: pgmap v4: 65 pgs: 65 active+clean; 70 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail 2024-05-12T04:00:13.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:12 smithi032 ceph-mon[40008]: mgrmap e23: smithi033.pcvxwy(active, since 2s) 2024-05-12T04:00:13.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:12 smithi033 ceph-mon[49525]: [12/May/2024:04:00:11] ENGINE Bus STARTING 2024-05-12T04:00:13.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:12 smithi033 ceph-mon[49525]: [12/May/2024:04:00:11] ENGINE Serving on http://172.21.15.33:8765 2024-05-12T04:00:13.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:12 smithi033 ceph-mon[49525]: [12/May/2024:04:00:11] ENGINE Serving on https://172.21.15.33:7150 2024-05-12T04:00:13.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:12 smithi033 ceph-mon[49525]: [12/May/2024:04:00:11] ENGINE Bus STARTED 2024-05-12T04:00:13.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:12 smithi033 ceph-mon[49525]: pgmap v4: 65 pgs: 65 active+clean; 70 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail 2024-05-12T04:00:13.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:12 smithi033 ceph-mon[49525]: mgrmap e23: smithi033.pcvxwy(active, since 2s) 2024-05-12T04:00:15.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:14 smithi032 ceph-mon[40008]: pgmap v5: 65 pgs: 65 active+clean; 70 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail 2024-05-12T04:00:15.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:14 smithi032 ceph-mon[40008]: mgrmap e24: smithi033.pcvxwy(active, since 4s) 2024-05-12T04:00:15.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:14 smithi033 ceph-mon[49525]: pgmap v5: 65 pgs: 65 active+clean; 70 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail 2024-05-12T04:00:15.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:14 smithi033 ceph-mon[49525]: mgrmap e24: smithi033.pcvxwy(active, since 4s) 2024-05-12T04:00:17.013 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:16 smithi032 ceph-mon[40008]: pgmap v6: 65 pgs: 65 active+clean; 71 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 286 KiB/s wr, 19 op/s 2024-05-12T04:00:17.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:16 smithi033 ceph-mon[49525]: pgmap v6: 65 pgs: 65 active+clean; 71 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 286 KiB/s wr, 19 op/s 2024-05-12T04:00:18.889 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:18 smithi032 ceph-mon[40008]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:00:18.889 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:18 smithi032 ceph-mon[40008]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:00:18.889 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:18 smithi032 ceph-mon[40008]: pgmap v7: 65 pgs: 65 active+clean; 90 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 135 op/s 2024-05-12T04:00:18.889 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:18 smithi032 ceph-mon[40008]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:00:18.889 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:18 smithi032 ceph-mon[40008]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:00:18.889 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:18 smithi032 ceph-mon[40008]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:00:18.889 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:18 smithi032 ceph-mon[40008]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:00:19.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:18 smithi033 ceph-mon[49525]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:00:19.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:18 smithi033 ceph-mon[49525]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:00:19.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:18 smithi033 ceph-mon[49525]: pgmap v7: 65 pgs: 65 active+clean; 90 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 135 op/s 2024-05-12T04:00:19.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:18 smithi033 ceph-mon[49525]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:00:19.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:18 smithi033 ceph-mon[49525]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:00:19.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:18 smithi033 ceph-mon[49525]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:00:19.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:18 smithi033 ceph-mon[49525]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:00:20.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:19 smithi032 ceph-mon[40008]: Standby manager daemon smithi032.ukatkq started 2024-05-12T04:00:20.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:19 smithi032 ceph-mon[40008]: from='mgr.? 172.21.15.32:0/3338675547' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi032.ukatkq/crt"}]: dispatch 2024-05-12T04:00:20.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:19 smithi032 ceph-mon[40008]: from='mgr.? 172.21.15.32:0/3338675547' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-12T04:00:20.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:19 smithi032 ceph-mon[40008]: from='mgr.? 172.21.15.32:0/3338675547' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi032.ukatkq/key"}]: dispatch 2024-05-12T04:00:20.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:19 smithi032 ceph-mon[40008]: from='mgr.? 172.21.15.32:0/3338675547' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-12T04:00:20.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:19 smithi032 ceph-mon[40008]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:00:20.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:19 smithi032 ceph-mon[40008]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:00:20.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:19 smithi032 ceph-mon[40008]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-12T04:00:20.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:19 smithi032 ceph-mon[40008]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-12T04:00:20.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:19 smithi032 ceph-mon[40008]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-12T04:00:20.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:19 smithi032 ceph-mon[40008]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-12T04:00:20.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:19 smithi032 ceph-mon[40008]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-12T04:00:20.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:19 smithi032 ceph-mon[40008]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-12T04:00:20.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:19 smithi032 ceph-mon[40008]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:00:20.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:19 smithi033 ceph-mon[49525]: Standby manager daemon smithi032.ukatkq started 2024-05-12T04:00:20.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:19 smithi033 ceph-mon[49525]: from='mgr.? 172.21.15.32:0/3338675547' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi032.ukatkq/crt"}]: dispatch 2024-05-12T04:00:20.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:19 smithi033 ceph-mon[49525]: from='mgr.? 172.21.15.32:0/3338675547' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-12T04:00:20.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:19 smithi033 ceph-mon[49525]: from='mgr.? 172.21.15.32:0/3338675547' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi032.ukatkq/key"}]: dispatch 2024-05-12T04:00:20.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:19 smithi033 ceph-mon[49525]: from='mgr.? 172.21.15.32:0/3338675547' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-12T04:00:20.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:19 smithi033 ceph-mon[49525]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:00:20.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:19 smithi033 ceph-mon[49525]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:00:20.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:19 smithi033 ceph-mon[49525]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-12T04:00:20.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:19 smithi033 ceph-mon[49525]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-12T04:00:20.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:19 smithi033 ceph-mon[49525]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-12T04:00:20.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:19 smithi033 ceph-mon[49525]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-12T04:00:20.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:19 smithi033 ceph-mon[49525]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-12T04:00:20.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:19 smithi033 ceph-mon[49525]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-12T04:00:20.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:19 smithi033 ceph-mon[49525]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:00:20.998 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:20 smithi033 ceph-mon[49525]: Adjusting osd_memory_target on smithi032 to 1570M 2024-05-12T04:00:20.998 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:20 smithi033 ceph-mon[49525]: mgrmap e25: smithi033.pcvxwy(active, since 9s), standbys: smithi032.ukatkq 2024-05-12T04:00:20.998 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:20 smithi033 ceph-mon[49525]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "mgr metadata", "who": "smithi032.ukatkq", "id": "smithi032.ukatkq"}]: dispatch 2024-05-12T04:00:20.998 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:20 smithi033 ceph-mon[49525]: pgmap v8: 65 pgs: 65 active+clean; 90 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 105 op/s 2024-05-12T04:00:21.039 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:20 smithi032 ceph-mon[40008]: Adjusting osd_memory_target on smithi032 to 1570M 2024-05-12T04:00:21.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:20 smithi032 ceph-mon[40008]: mgrmap e25: smithi033.pcvxwy(active, since 9s), standbys: smithi032.ukatkq 2024-05-12T04:00:21.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:20 smithi032 ceph-mon[40008]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "mgr metadata", "who": "smithi032.ukatkq", "id": "smithi032.ukatkq"}]: dispatch 2024-05-12T04:00:21.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:20 smithi032 ceph-mon[40008]: pgmap v8: 65 pgs: 65 active+clean; 90 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 105 op/s 2024-05-12T04:00:23.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:22 smithi033 ceph-mon[49525]: pgmap v9: 65 pgs: 65 active+clean; 104 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 152 op/s 2024-05-12T04:00:23.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:22 smithi032 ceph-mon[40008]: pgmap v9: 65 pgs: 65 active+clean; 104 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 152 op/s 2024-05-12T04:00:25.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:24 smithi033 ceph-mon[49525]: pgmap v10: 65 pgs: 65 active+clean; 104 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 139 op/s 2024-05-12T04:00:25.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:24 smithi032 ceph-mon[40008]: pgmap v10: 65 pgs: 65 active+clean; 104 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 139 op/s 2024-05-12T04:00:26.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:25 smithi033 ceph-mon[49525]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:00:26.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:25 smithi032 ceph-mon[40008]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:00:27.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:26 smithi033 ceph-mon[49525]: pgmap v11: 65 pgs: 65 active+clean; 108 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 159 op/s 2024-05-12T04:00:27.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:26 smithi032 ceph-mon[40008]: pgmap v11: 65 pgs: 65 active+clean; 108 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 159 op/s 2024-05-12T04:00:29.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:28 smithi033 ceph-mon[49525]: pgmap v12: 65 pgs: 65 active+clean; 118 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.7 MiB/s rd, 4.0 MiB/s wr, 202 op/s 2024-05-12T04:00:29.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:28 smithi032 ceph-mon[40008]: pgmap v12: 65 pgs: 65 active+clean; 118 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.7 MiB/s rd, 4.0 MiB/s wr, 202 op/s 2024-05-12T04:00:31.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:30 smithi033 ceph-mon[49525]: pgmap v13: 65 pgs: 65 active+clean; 118 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 131 op/s 2024-05-12T04:00:31.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.220+0000 7ff12dc7c700 1 -- 172.21.15.32:0/686095940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1280fcf60 msgr2=0x7ff1280fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:00:31.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.220+0000 7ff12dc7c700 1 --2- 172.21.15.32:0/686095940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1280fcf60 0x7ff1280fd3d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff11c0049c0 tx=0x7ff11c009e30 comp rx=0 tx=0).stop 2024-05-12T04:00:31.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.220+0000 7ff12dc7c700 1 -- 172.21.15.32:0/686095940 shutdown_connections 2024-05-12T04:00:31.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.220+0000 7ff12dc7c700 1 --2- 172.21.15.32:0/686095940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1280fcf60 0x7ff1280fd3d0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:00:31.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.220+0000 7ff12dc7c700 1 --2- 172.21.15.32:0/686095940 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1280fbcb0 0x7ff1280fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:00:31.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.220+0000 7ff12dc7c700 1 -- 172.21.15.32:0/686095940 >> 172.21.15.32:0/686095940 conn(0x7ff1280f7430 msgr2=0x7ff1280f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:00:31.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.221+0000 7ff12dc7c700 1 -- 172.21.15.32:0/686095940 shutdown_connections 2024-05-12T04:00:31.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.221+0000 7ff12dc7c700 1 -- 172.21.15.32:0/686095940 wait complete. 2024-05-12T04:00:31.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.221+0000 7ff12dc7c700 1 Processor -- start 2024-05-12T04:00:31.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.221+0000 7ff12dc7c700 1 -- start start 2024-05-12T04:00:31.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.222+0000 7ff12dc7c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1280fbcb0 0x7ff12810b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:00:31.222 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.222+0000 7ff12dc7c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1280fcf60 0x7ff12810bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:00:31.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.222+0000 7ff12dc7c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12810c1d0 con 0x7ff1280fcf60 2024-05-12T04:00:31.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.222+0000 7ff12dc7c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12810ca20 con 0x7ff1280fbcb0 2024-05-12T04:00:31.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.222+0000 7ff126ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1280fcf60 0x7ff12810bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:00:31.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.222+0000 7ff126ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1280fcf60 0x7ff12810bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57814/0 (socket says 172.21.15.32:57814) 2024-05-12T04:00:31.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.222+0000 7ff126ffd700 1 -- 172.21.15.32:0/524739699 learned_addr learned my addr 172.21.15.32:0/524739699 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:00:31.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.222+0000 7ff1277fe700 1 --2- 172.21.15.32:0/524739699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1280fbcb0 0x7ff12810b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:00:31.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.222+0000 7ff1277fe700 1 -- 172.21.15.32:0/524739699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1280fcf60 msgr2=0x7ff12810bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:00:31.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.222+0000 7ff1277fe700 1 --2- 172.21.15.32:0/524739699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1280fcf60 0x7ff12810bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:00:31.223 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.222+0000 7ff1277fe700 1 -- 172.21.15.32:0/524739699 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff11c005040 con 0x7ff1280fbcb0 2024-05-12T04:00:31.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.223+0000 7ff1277fe700 1 --2- 172.21.15.32:0/524739699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1280fbcb0 0x7ff12810b690 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff118009fd0 tx=0x7ff11800b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:00:31.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.223+0000 7ff10ffff700 1 -- 172.21.15.32:0/524739699 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff11800e070 con 0x7ff1280fbcb0 2024-05-12T04:00:31.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.223+0000 7ff10ffff700 1 -- 172.21.15.32:0/524739699 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff1180090d0 con 0x7ff1280fbcb0 2024-05-12T04:00:31.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.223+0000 7ff12dc7c700 1 -- 172.21.15.32:0/524739699 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff12810cca0 con 0x7ff1280fbcb0 2024-05-12T04:00:31.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.223+0000 7ff10ffff700 1 -- 172.21.15.32:0/524739699 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff118012780 con 0x7ff1280fbcb0 2024-05-12T04:00:31.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.223+0000 7ff12dc7c700 1 -- 172.21.15.32:0/524739699 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff12810d1e0 con 0x7ff1280fbcb0 2024-05-12T04:00:31.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.224+0000 7ff12dc7c700 1 -- 172.21.15.32:0/524739699 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff12805f5c0 con 0x7ff1280fbcb0 2024-05-12T04:00:31.224 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.225+0000 7ff10ffff700 1 -- 172.21.15.32:0/524739699 <== mon.1 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) v1 ==== 88772+0+0 (secure 0 0 0) 0x7ff118009240 con 0x7ff1280fbcb0 2024-05-12T04:00:31.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.225+0000 7ff10ffff700 1 --2- 172.21.15.32:0/524739699 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7ff11007bed0 0x7ff11007e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:00:31.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.225+0000 7ff10ffff700 1 -- 172.21.15.32:0/524739699 <== mon.1 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7ff118089690 con 0x7ff1280fbcb0 2024-05-12T04:00:31.225 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.225+0000 7ff126ffd700 1 --2- 172.21.15.32:0/524739699 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7ff11007bed0 0x7ff11007e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:00:31.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.226+0000 7ff126ffd700 1 --2- 172.21.15.32:0/524739699 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7ff11007bed0 0x7ff11007e380 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff11c004f80 tx=0x7ff11c009e30 comp rx=0 tx=0).ready entity=mgr.24389 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:00:31.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.230+0000 7ff10ffff700 1 -- 172.21.15.32:0/524739699 <== mon.1 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181777 (secure 0 0 0) 0x7ff118057570 con 0x7ff1280fbcb0 2024-05-12T04:00:31.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:30 smithi032 ceph-mon[40008]: pgmap v13: 65 pgs: 65 active+clean; 118 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 131 op/s 2024-05-12T04:00:31.535 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.535+0000 7ff12dc7c700 1 -- 172.21.15.32:0/524739699 --> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff12810d600 con 0x7ff11007bed0 2024-05-12T04:00:31.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.537+0000 7ff10ffff700 1 -- 172.21.15.32:0/524739699 <== mgr.24389 v2:172.21.15.33:6828/1598726962 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7ff12810d600 con 0x7ff11007bed0 2024-05-12T04:00:31.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.539+0000 7ff12dc7c700 1 -- 172.21.15.32:0/524739699 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7ff11007bed0 msgr2=0x7ff11007e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:00:31.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.539+0000 7ff12dc7c700 1 --2- 172.21.15.32:0/524739699 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7ff11007bed0 0x7ff11007e380 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff11c004f80 tx=0x7ff11c009e30 comp rx=0 tx=0).stop 2024-05-12T04:00:31.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.539+0000 7ff12dc7c700 1 -- 172.21.15.32:0/524739699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1280fbcb0 msgr2=0x7ff12810b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:00:31.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.539+0000 7ff12dc7c700 1 --2- 172.21.15.32:0/524739699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1280fbcb0 0x7ff12810b690 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff118009fd0 tx=0x7ff11800b040 comp rx=0 tx=0).stop 2024-05-12T04:00:31.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.539+0000 7ff12dc7c700 1 -- 172.21.15.32:0/524739699 shutdown_connections 2024-05-12T04:00:31.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.539+0000 7ff12dc7c700 1 --2- 172.21.15.32:0/524739699 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7ff11007bed0 0x7ff11007e380 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:00:31.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.539+0000 7ff12dc7c700 1 --2- 172.21.15.32:0/524739699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1280fbcb0 0x7ff12810b690 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:00:31.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.539+0000 7ff12dc7c700 1 --2- 172.21.15.32:0/524739699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1280fcf60 0x7ff12810bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:00:31.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.539+0000 7ff12dc7c700 1 -- 172.21.15.32:0/524739699 >> 172.21.15.32:0/524739699 conn(0x7ff1280f7430 msgr2=0x7ff1280f8c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:00:31.540 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.540+0000 7ff12dc7c700 1 -- 172.21.15.32:0/524739699 shutdown_connections 2024-05-12T04:00:31.541 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.540+0000 7ff12dc7c700 1 -- 172.21.15.32:0/524739699 wait complete. 2024-05-12T04:00:31.552 INFO:teuthology.orchestra.run.smithi032.stdout:true 2024-05-12T04:00:31.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.670+0000 7f3e4e26f700 1 -- 172.21.15.32:0/862922846 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e480fbcb0 msgr2=0x7f3e480fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:00:31.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.670+0000 7f3e4e26f700 1 --2- 172.21.15.32:0/862922846 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e480fbcb0 0x7f3e480fc0c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3e38004940 tx=0x7f3e38009e30 comp rx=0 tx=0).stop 2024-05-12T04:00:31.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.670+0000 7f3e4e26f700 1 -- 172.21.15.32:0/862922846 shutdown_connections 2024-05-12T04:00:31.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.670+0000 7f3e4e26f700 1 --2- 172.21.15.32:0/862922846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e480fcf60 0x7f3e480fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:00:31.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.670+0000 7f3e4e26f700 1 --2- 172.21.15.32:0/862922846 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e480fbcb0 0x7f3e480fc0c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:00:31.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.670+0000 7f3e4e26f700 1 -- 172.21.15.32:0/862922846 >> 172.21.15.32:0/862922846 conn(0x7f3e480f7430 msgr2=0x7f3e480f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:00:31.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.670+0000 7f3e4e26f700 1 -- 172.21.15.32:0/862922846 shutdown_connections 2024-05-12T04:00:31.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.670+0000 7f3e4e26f700 1 -- 172.21.15.32:0/862922846 wait complete. 2024-05-12T04:00:31.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.671+0000 7f3e4e26f700 1 Processor -- start 2024-05-12T04:00:31.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.671+0000 7f3e4e26f700 1 -- start start 2024-05-12T04:00:31.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.671+0000 7f3e4e26f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e480fbcb0 0x7f3e481105b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:00:31.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.671+0000 7f3e4e26f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e480fcf60 0x7f3e48112b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:00:31.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.671+0000 7f3e4e26f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e48066330 con 0x7f3e480fbcb0 2024-05-12T04:00:31.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.671+0000 7f3e4e26f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e48066470 con 0x7f3e480fcf60 2024-05-12T04:00:31.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.672+0000 7f3e47fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e480fbcb0 0x7f3e481105b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:00:31.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.672+0000 7f3e477fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e480fcf60 0x7f3e48112b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:00:31.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.672+0000 7f3e477fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e480fcf60 0x7f3e48112b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.32:52070/0 (socket says 172.21.15.32:52070) 2024-05-12T04:00:31.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.672+0000 7f3e47fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e480fbcb0 0x7f3e481105b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57826/0 (socket says 172.21.15.32:57826) 2024-05-12T04:00:31.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.672+0000 7f3e477fe700 1 -- 172.21.15.32:0/554943827 learned_addr learned my addr 172.21.15.32:0/554943827 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:00:31.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.672+0000 7f3e477fe700 1 -- 172.21.15.32:0/554943827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e480fbcb0 msgr2=0x7f3e481105b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:00:31.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.672+0000 7f3e477fe700 1 --2- 172.21.15.32:0/554943827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e480fbcb0 0x7f3e481105b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:00:31.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.672+0000 7f3e477fe700 1 -- 172.21.15.32:0/554943827 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e3c005040 con 0x7f3e480fcf60 2024-05-12T04:00:31.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.672+0000 7f3e477fe700 1 --2- 172.21.15.32:0/554943827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e480fcf60 0x7f3e48112b00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c010b50 tx=0x7f3e3c009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:00:31.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.672+0000 7f3e44ff9700 1 -- 172.21.15.32:0/554943827 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e3c015070 con 0x7f3e480fcf60 2024-05-12T04:00:31.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.672+0000 7f3e44ff9700 1 -- 172.21.15.32:0/554943827 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3e3c009750 con 0x7f3e480fcf60 2024-05-12T04:00:31.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.672+0000 7f3e4e26f700 1 -- 172.21.15.32:0/554943827 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e38005040 con 0x7f3e480fcf60 2024-05-12T04:00:31.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.672+0000 7f3e44ff9700 1 -- 172.21.15.32:0/554943827 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e3c0185b0 con 0x7f3e480fcf60 2024-05-12T04:00:31.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.672+0000 7f3e4e26f700 1 -- 172.21.15.32:0/554943827 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e4810c820 con 0x7f3e480fcf60 2024-05-12T04:00:31.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.673+0000 7f3e4e26f700 1 -- 172.21.15.32:0/554943827 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e4805f5c0 con 0x7f3e480fcf60 2024-05-12T04:00:31.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.675+0000 7f3e44ff9700 1 -- 172.21.15.32:0/554943827 <== mon.1 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) v1 ==== 88772+0+0 (secure 0 0 0) 0x7f3e3c0193c0 con 0x7f3e480fcf60 2024-05-12T04:00:31.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.675+0000 7f3e44ff9700 1 --2- 172.21.15.32:0/554943827 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f3e3006aec0 0x7f3e3006d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:00:31.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.675+0000 7f3e44ff9700 1 -- 172.21.15.32:0/554943827 <== mon.1 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f3e3c08d950 con 0x7f3e480fcf60 2024-05-12T04:00:31.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.675+0000 7f3e47fff700 1 --2- 172.21.15.32:0/554943827 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f3e3006aec0 0x7f3e3006d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:00:31.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.676+0000 7f3e47fff700 1 --2- 172.21.15.32:0/554943827 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f3e3006aec0 0x7f3e3006d370 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3e380137d0 tx=0x7f3e38009e30 comp rx=0 tx=0).ready entity=mgr.24389 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:00:31.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.681+0000 7f3e44ff9700 1 -- 172.21.15.32:0/554943827 <== mon.1 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181777 (secure 0 0 0) 0x7f3e3c057d20 con 0x7f3e480fcf60 2024-05-12T04:00:31.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.952+0000 7f3e4e26f700 1 -- 172.21.15.32:0/554943827 --> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3e48101320 con 0x7f3e3006aec0 2024-05-12T04:00:31.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.953+0000 7f3e44ff9700 1 -- 172.21.15.32:0/554943827 <== mgr.24389 v2:172.21.15.33:6828/1598726962 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7f3e48101320 con 0x7f3e3006aec0 2024-05-12T04:00:31.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.956+0000 7f3e4e26f700 1 -- 172.21.15.32:0/554943827 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f3e3006aec0 msgr2=0x7f3e3006d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:00:31.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.956+0000 7f3e4e26f700 1 --2- 172.21.15.32:0/554943827 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f3e3006aec0 0x7f3e3006d370 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3e380137d0 tx=0x7f3e38009e30 comp rx=0 tx=0).stop 2024-05-12T04:00:31.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.956+0000 7f3e4e26f700 1 -- 172.21.15.32:0/554943827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e480fcf60 msgr2=0x7f3e48112b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:00:31.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.956+0000 7f3e4e26f700 1 --2- 172.21.15.32:0/554943827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e480fcf60 0x7f3e48112b00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c010b50 tx=0x7f3e3c009e30 comp rx=0 tx=0).stop 2024-05-12T04:00:31.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.956+0000 7f3e4e26f700 1 -- 172.21.15.32:0/554943827 shutdown_connections 2024-05-12T04:00:31.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.956+0000 7f3e4e26f700 1 --2- 172.21.15.32:0/554943827 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f3e3006aec0 0x7f3e3006d370 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:00:31.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.956+0000 7f3e4e26f700 1 --2- 172.21.15.32:0/554943827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e480fbcb0 0x7f3e481105b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:00:31.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.956+0000 7f3e4e26f700 1 --2- 172.21.15.32:0/554943827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e480fcf60 0x7f3e48112b00 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:00:31.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.956+0000 7f3e4e26f700 1 -- 172.21.15.32:0/554943827 >> 172.21.15.32:0/554943827 conn(0x7f3e480f7430 msgr2=0x7f3e480f8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:00:31.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.956+0000 7f3e4e26f700 1 -- 172.21.15.32:0/554943827 shutdown_connections 2024-05-12T04:00:31.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:31.956+0000 7f3e4e26f700 1 -- 172.21.15.32:0/554943827 wait complete. 2024-05-12T04:00:32.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.078+0000 7ff302c3d700 1 -- 172.21.15.32:0/1058829696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2fc0ffee0 msgr2=0x7ff2fc102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:00:32.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.078+0000 7ff302c3d700 1 --2- 172.21.15.32:0/1058829696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2fc0ffee0 0x7ff2fc102300 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8004940 tx=0x7ff2e8009e30 comp rx=0 tx=0).stop 2024-05-12T04:00:32.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.078+0000 7ff302c3d700 1 -- 172.21.15.32:0/1058829696 shutdown_connections 2024-05-12T04:00:32.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.078+0000 7ff302c3d700 1 --2- 172.21.15.32:0/1058829696 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2fc0ffee0 0x7ff2fc102300 secure :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8004940 tx=0x7ff2e8009e30 comp rx=0 tx=0).stop 2024-05-12T04:00:32.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.078+0000 7ff302c3d700 1 --2- 172.21.15.32:0/1058829696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2fc0662c0 0x7ff2fc0ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:00:32.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.078+0000 7ff302c3d700 1 -- 172.21.15.32:0/1058829696 >> 172.21.15.32:0/1058829696 conn(0x7ff2fc0f7430 msgr2=0x7ff2fc0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:00:32.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.078+0000 7ff302c3d700 1 -- 172.21.15.32:0/1058829696 shutdown_connections 2024-05-12T04:00:32.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.078+0000 7ff302c3d700 1 -- 172.21.15.32:0/1058829696 wait complete. 2024-05-12T04:00:32.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.079+0000 7ff302c3d700 1 Processor -- start 2024-05-12T04:00:32.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.079+0000 7ff302c3d700 1 -- start start 2024-05-12T04:00:32.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.079+0000 7ff302c3d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2fc0662c0 0x7ff2fc0fd360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:00:32.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.079+0000 7ff302c3d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2fc0fb8a0 0x7ff2fc0fbd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:00:32.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.079+0000 7ff302c3d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2fc0fc250 con 0x7ff2fc0662c0 2024-05-12T04:00:32.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.079+0000 7ff302c3d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2fc0fc3c0 con 0x7ff2fc0fb8a0 2024-05-12T04:00:32.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.079+0000 7ff3009d9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2fc0662c0 0x7ff2fc0fd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:00:32.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.079+0000 7ff3009d9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2fc0662c0 0x7ff2fc0fd360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39890/0 (socket says 172.21.15.32:39890) 2024-05-12T04:00:32.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.079+0000 7ff3009d9700 1 -- 172.21.15.32:0/218856245 learned_addr learned my addr 172.21.15.32:0/218856245 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:00:32.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.079+0000 7ff2fbfff700 1 --2- 172.21.15.32:0/218856245 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2fc0fb8a0 0x7ff2fc0fbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:00:32.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.080+0000 7ff3009d9700 1 -- 172.21.15.32:0/218856245 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2fc0fb8a0 msgr2=0x7ff2fc0fbd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:00:32.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.080+0000 7ff3009d9700 1 --2- 172.21.15.32:0/218856245 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2fc0fb8a0 0x7ff2fc0fbd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:00:32.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.080+0000 7ff3009d9700 1 -- 172.21.15.32:0/218856245 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2e8005040 con 0x7ff2fc0662c0 2024-05-12T04:00:32.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.080+0000 7ff3009d9700 1 --2- 172.21.15.32:0/218856245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2fc0662c0 0x7ff2fc0fd360 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff2f0002a50 tx=0x7ff2f0004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:00:32.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.080+0000 7ff2f97fa700 1 -- 172.21.15.32:0/218856245 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2f0007cc0 con 0x7ff2fc0662c0 2024-05-12T04:00:32.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.080+0000 7ff2f97fa700 1 -- 172.21.15.32:0/218856245 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff2f000f040 con 0x7ff2fc0662c0 2024-05-12T04:00:32.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.080+0000 7ff302c3d700 1 -- 172.21.15.32:0/218856245 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2fc0fc6a0 con 0x7ff2fc0662c0 2024-05-12T04:00:32.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.080+0000 7ff2f97fa700 1 -- 172.21.15.32:0/218856245 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2f000d920 con 0x7ff2fc0662c0 2024-05-12T04:00:32.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.080+0000 7ff302c3d700 1 -- 172.21.15.32:0/218856245 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2fc115e50 con 0x7ff2fc0662c0 2024-05-12T04:00:32.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.081+0000 7ff302c3d700 1 -- 172.21.15.32:0/218856245 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2fc05f5c0 con 0x7ff2fc0662c0 2024-05-12T04:00:32.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.083+0000 7ff2f97fa700 1 -- 172.21.15.32:0/218856245 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 25) v1 ==== 88772+0+0 (secure 0 0 0) 0x7ff2f0009680 con 0x7ff2fc0662c0 2024-05-12T04:00:32.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.083+0000 7ff2f97fa700 1 --2- 172.21.15.32:0/218856245 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7ff2ec06af90 0x7ff2ec06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:00:32.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.083+0000 7ff2f97fa700 1 -- 172.21.15.32:0/218856245 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7ff2f0089b60 con 0x7ff2fc0662c0 2024-05-12T04:00:32.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.083+0000 7ff2fbfff700 1 --2- 172.21.15.32:0/218856245 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7ff2ec06af90 0x7ff2ec06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:00:32.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.084+0000 7ff2fbfff700 1 --2- 172.21.15.32:0/218856245 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7ff2ec06af90 0x7ff2ec06d440 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8000c00 tx=0x7ff2e8009e30 comp rx=0 tx=0).ready entity=mgr.24389 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:00:32.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.089+0000 7ff2f97fa700 1 -- 172.21.15.32:0/218856245 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181777 (secure 0 0 0) 0x7ff2f0054070 con 0x7ff2fc0662c0 2024-05-12T04:00:32.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.410+0000 7ff302c3d700 1 -- 172.21.15.32:0/218856245 --> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff2fc101880 con 0x7ff2ec06af90 2024-05-12T04:00:32.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.423+0000 7ff2f97fa700 1 -- 172.21.15.32:0/218856245 <== mgr.24389 v2:172.21.15.33:6828/1598726962 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7ff2fc101880 con 0x7ff2ec06af90 2024-05-12T04:00:32.423 INFO:teuthology.orchestra.run.smithi032.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-12T04:00:32.423 INFO:teuthology.orchestra.run.smithi032.stdout:alertmanager.smithi032 smithi032 *:9093,9094 running (7m) 14s ago 8m 27.8M - 0.25.0 c8568f914cd2 5c80dfe17d2e 2024-05-12T04:00:32.423 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi032 smithi032 running (8m) 14s ago 8m 6651k - 17.2.7-939-gc190bdee 284803f7d9cf a478874e9dfa 2024-05-12T04:00:32.423 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi033 smithi033 running (7m) 14s ago 7m 6651k - 17.2.7-939-gc190bdee 284803f7d9cf 34c62d004c7e 2024-05-12T04:00:32.423 INFO:teuthology.orchestra.run.smithi032.stdout:grafana.smithi032 smithi032 *:3000 running (7m) 14s ago 8m 87.5M - 9.4.7 954c08fa6188 ac9076563478 2024-05-12T04:00:32.423 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.ialtqb smithi032 running (2m) 14s ago 2m 11.6M - 17.2.7-939-gc190bdee 284803f7d9cf 0683abf00e3e 2024-05-12T04:00:32.423 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.qpsawh smithi032 running (2m) 14s ago 2m 104M - 17.2.7-939-gc190bdee 284803f7d9cf 95a7db5c4fcd 2024-05-12T04:00:32.424 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi033.hglwml smithi033 running (2m) 14s ago 2m 11.6M - 17.2.7-939-gc190bdee 284803f7d9cf c97902b05025 2024-05-12T04:00:32.424 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi033.jgbjhk smithi033 running (2m) 14s ago 2m 82.6M - 17.2.7-939-gc190bdee 284803f7d9cf af97052cb97a 2024-05-12T04:00:32.424 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi032.ukatkq smithi032 *:9283 running (10m) 14s ago 10m 158M - 17.2.7-939-gc190bdee 284803f7d9cf d9763dc47404 2024-05-12T04:00:32.424 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi033.pcvxwy smithi033 *:8443,9283 running (34s) 14s ago 7m 484M - 18.2.2-1272-g58249552 ddcb0a9d6fde b0bff3ab143f 2024-05-12T04:00:32.424 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi032 smithi032 running (10m) 14s ago 10m 49.9M 2048M 17.2.7-939-gc190bdee 284803f7d9cf d7fea430c6e2 2024-05-12T04:00:32.424 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi033 smithi033 running (7m) 14s ago 7m 40.1M 2048M 17.2.7-939-gc190bdee 284803f7d9cf 707654d85748 2024-05-12T04:00:32.424 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi032 smithi032 *:9100 running (8m) 14s ago 8m 18.7M - 1.5.0 0da6a335fe13 6aa48ce262de 2024-05-12T04:00:32.424 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi033 smithi033 *:9100 running (7m) 14s ago 7m 17.4M - 1.5.0 0da6a335fe13 1144caed00e9 2024-05-12T04:00:32.424 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0 smithi032 running (6m) 14s ago 6m 66.2M 1570M 17.2.7-939-gc190bdee 284803f7d9cf 75c78a47f8e5 2024-05-12T04:00:32.424 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1 smithi032 running (6m) 14s ago 6m 67.5M 1570M 17.2.7-939-gc190bdee 284803f7d9cf 87cfd247d8d6 2024-05-12T04:00:32.424 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2 smithi032 running (5m) 14s ago 5m 57.7M 1570M 17.2.7-939-gc190bdee 284803f7d9cf 7aef7affb8ba 2024-05-12T04:00:32.424 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3 smithi033 running (5m) 14s ago 5m 77.0M 5324M 17.2.7-939-gc190bdee 284803f7d9cf 3a92b5e3348e 2024-05-12T04:00:32.424 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4 smithi033 running (4m) 14s ago 4m 63.5M 5324M 17.2.7-939-gc190bdee 284803f7d9cf 0896268b4591 2024-05-12T04:00:32.424 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5 smithi033 running (4m) 14s ago 4m 64.4M 5324M 17.2.7-939-gc190bdee 284803f7d9cf 57e65a64c821 2024-05-12T04:00:32.424 INFO:teuthology.orchestra.run.smithi032.stdout:prometheus.smithi032 smithi032 *:9095 running (6m) 14s ago 8m 57.7M - 2.43.0 a07b618ecd1d 0b3ce3323a49 2024-05-12T04:00:32.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.425+0000 7ff302c3d700 1 -- 172.21.15.32:0/218856245 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7ff2ec06af90 msgr2=0x7ff2ec06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:00:32.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.425+0000 7ff302c3d700 1 --2- 172.21.15.32:0/218856245 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7ff2ec06af90 0x7ff2ec06d440 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8000c00 tx=0x7ff2e8009e30 comp rx=0 tx=0).stop 2024-05-12T04:00:32.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.425+0000 7ff302c3d700 1 -- 172.21.15.32:0/218856245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2fc0662c0 msgr2=0x7ff2fc0fd360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:00:32.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.425+0000 7ff302c3d700 1 --2- 172.21.15.32:0/218856245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2fc0662c0 0x7ff2fc0fd360 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff2f0002a50 tx=0x7ff2f0004600 comp rx=0 tx=0).stop 2024-05-12T04:00:32.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.425+0000 7ff302c3d700 1 -- 172.21.15.32:0/218856245 shutdown_connections 2024-05-12T04:00:32.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.425+0000 7ff302c3d700 1 --2- 172.21.15.32:0/218856245 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7ff2ec06af90 0x7ff2ec06d440 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:00:32.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.425+0000 7ff302c3d700 1 --2- 172.21.15.32:0/218856245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2fc0662c0 0x7ff2fc0fd360 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:00:32.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.425+0000 7ff302c3d700 1 --2- 172.21.15.32:0/218856245 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2fc0fb8a0 0x7ff2fc0fbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:00:32.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.425+0000 7ff302c3d700 1 -- 172.21.15.32:0/218856245 >> 172.21.15.32:0/218856245 conn(0x7ff2fc0f7430 msgr2=0x7ff2fc0f9810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:00:32.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.426+0000 7ff302c3d700 1 -- 172.21.15.32:0/218856245 shutdown_connections 2024-05-12T04:00:32.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.426+0000 7ff302c3d700 1 -- 172.21.15.32:0/218856245 wait complete. 2024-05-12T04:00:32.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.565+0000 7f3f144f4700 1 -- 172.21.15.32:0/3174486059 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f0c06ed90 msgr2=0x7f3f0c10b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:00:32.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.565+0000 7f3f144f4700 1 --2- 172.21.15.32:0/3174486059 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f0c06ed90 0x7f3f0c10b190 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3f08004940 tx=0x7f3f08009e30 comp rx=0 tx=0).stop 2024-05-12T04:00:32.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.565+0000 7f3f144f4700 1 -- 172.21.15.32:0/3174486059 shutdown_connections 2024-05-12T04:00:32.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.565+0000 7f3f144f4700 1 --2- 172.21.15.32:0/3174486059 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f0c06ed90 0x7f3f0c10b190 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:00:32.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.565+0000 7f3f144f4700 1 --2- 172.21.15.32:0/3174486059 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f0c06e440 0x7f3f0c06e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:00:32.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.565+0000 7f3f144f4700 1 -- 172.21.15.32:0/3174486059 >> 172.21.15.32:0/3174486059 conn(0x7f3f0c069dc0 msgr2=0x7f3f0c06c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:00:32.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.566+0000 7f3f144f4700 1 -- 172.21.15.32:0/3174486059 shutdown_connections 2024-05-12T04:00:32.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.566+0000 7f3f144f4700 1 -- 172.21.15.32:0/3174486059 wait complete. 2024-05-12T04:00:32.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.566+0000 7f3f144f4700 1 Processor -- start 2024-05-12T04:00:32.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.566+0000 7f3f144f4700 1 -- start start 2024-05-12T04:00:32.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.567+0000 7f3f144f4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f0c06e440 0x7f3f0c11c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:00:32.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.567+0000 7f3f144f4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f0c06ed90 0x7f3f0c11cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:00:32.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.567+0000 7f3f144f4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c11d100 con 0x7f3f0c06ed90 2024-05-12T04:00:32.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.567+0000 7f3f144f4700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c11d270 con 0x7f3f0c06e440 2024-05-12T04:00:32.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.567+0000 7f3f11a8f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f0c06ed90 0x7f3f0c11cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:00:32.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.567+0000 7f3f11a8f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f0c06ed90 0x7f3f0c11cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39900/0 (socket says 172.21.15.32:39900) 2024-05-12T04:00:32.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.567+0000 7f3f11a8f700 1 -- 172.21.15.32:0/2367982367 learned_addr learned my addr 172.21.15.32:0/2367982367 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:00:32.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.567+0000 7f3f12290700 1 --2- 172.21.15.32:0/2367982367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f0c06e440 0x7f3f0c11c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:00:32.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.567+0000 7f3f11a8f700 1 -- 172.21.15.32:0/2367982367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f0c06e440 msgr2=0x7f3f0c11c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:00:32.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.567+0000 7f3f11a8f700 1 --2- 172.21.15.32:0/2367982367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f0c06e440 0x7f3f0c11c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:00:32.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.567+0000 7f3f11a8f700 1 -- 172.21.15.32:0/2367982367 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f08005040 con 0x7f3f0c06ed90 2024-05-12T04:00:32.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.567+0000 7f3f11a8f700 1 --2- 172.21.15.32:0/2367982367 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f0c06ed90 0x7f3f0c11cb00 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3f08009380 tx=0x7f3f08009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:00:32.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.568+0000 7f3f02ffd700 1 -- 172.21.15.32:0/2367982367 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f0800e040 con 0x7f3f0c06ed90 2024-05-12T04:00:32.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.568+0000 7f3f144f4700 1 -- 172.21.15.32:0/2367982367 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f0c121c70 con 0x7f3f0c06ed90 2024-05-12T04:00:32.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.568+0000 7f3f144f4700 1 -- 172.21.15.32:0/2367982367 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f0c122230 con 0x7f3f0c06ed90 2024-05-12T04:00:32.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.568+0000 7f3f02ffd700 1 -- 172.21.15.32:0/2367982367 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3f0800bd20 con 0x7f3f0c06ed90 2024-05-12T04:00:32.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.568+0000 7f3f02ffd700 1 -- 172.21.15.32:0/2367982367 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f0801c630 con 0x7f3f0c06ed90 2024-05-12T04:00:32.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.569+0000 7f3f144f4700 1 -- 172.21.15.32:0/2367982367 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ef00052e0 con 0x7f3f0c06ed90 2024-05-12T04:00:32.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.569+0000 7f3f02ffd700 1 -- 172.21.15.32:0/2367982367 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 25) v1 ==== 88772+0+0 (secure 0 0 0) 0x7f3f08013420 con 0x7f3f0c06ed90 2024-05-12T04:00:32.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.570+0000 7f3f02ffd700 1 --2- 172.21.15.32:0/2367982367 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f3ef806aed0 0x7f3ef806d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:00:32.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.570+0000 7f3f02ffd700 1 -- 172.21.15.32:0/2367982367 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f3f0808a280 con 0x7f3f0c06ed90 2024-05-12T04:00:32.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.570+0000 7f3f12290700 1 --2- 172.21.15.32:0/2367982367 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f3ef806aed0 0x7f3ef806d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:00:32.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.571+0000 7f3f12290700 1 --2- 172.21.15.32:0/2367982367 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f3ef806aed0 0x7f3ef806d380 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3efc009fd0 tx=0x7f3efc00b040 comp rx=0 tx=0).ready entity=mgr.24389 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:00:32.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.576+0000 7f3f02ffd700 1 -- 172.21.15.32:0/2367982367 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181777 (secure 0 0 0) 0x7f3f08058340 con 0x7f3f0c06ed90 2024-05-12T04:00:32.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.970+0000 7f3f144f4700 1 -- 172.21.15.32:0/2367982367 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3ef0005860 con 0x7f3f0c06ed90 2024-05-12T04:00:32.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.970+0000 7f3f02ffd700 1 -- 172.21.15.32:0/2367982367 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+850 (secure 0 0 0) 0x7f3f08057ed0 con 0x7f3f0c06ed90 2024-05-12T04:00:32.970 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-05-12T04:00:32.971 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": { 2024-05-12T04:00:32.971 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 2 2024-05-12T04:00:32.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:00:32.971 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": { 2024-05-12T04:00:32.971 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 1, 2024-05-12T04:00:32.971 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 1 2024-05-12T04:00:32.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:00:32.971 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": { 2024-05-12T04:00:32.971 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 6 2024-05-12T04:00:32.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:00:32.971 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": { 2024-05-12T04:00:32.971 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 4 2024-05-12T04:00:32.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:00:32.971 INFO:teuthology.orchestra.run.smithi032.stdout: "overall": { 2024-05-12T04:00:32.972 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 13, 2024-05-12T04:00:32.972 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 1 2024-05-12T04:00:32.972 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-05-12T04:00:32.972 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-05-12T04:00:32.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.973+0000 7f3f144f4700 1 -- 172.21.15.32:0/2367982367 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f3ef806aed0 msgr2=0x7f3ef806d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:00:32.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.973+0000 7f3f144f4700 1 --2- 172.21.15.32:0/2367982367 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f3ef806aed0 0x7f3ef806d380 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3efc009fd0 tx=0x7f3efc00b040 comp rx=0 tx=0).stop 2024-05-12T04:00:32.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.973+0000 7f3f144f4700 1 -- 172.21.15.32:0/2367982367 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f0c06ed90 msgr2=0x7f3f0c11cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:00:32.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.973+0000 7f3f144f4700 1 --2- 172.21.15.32:0/2367982367 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f0c06ed90 0x7f3f0c11cb00 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3f08009380 tx=0x7f3f08009e30 comp rx=0 tx=0).stop 2024-05-12T04:00:32.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.973+0000 7f3f144f4700 1 -- 172.21.15.32:0/2367982367 shutdown_connections 2024-05-12T04:00:32.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.973+0000 7f3f144f4700 1 --2- 172.21.15.32:0/2367982367 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f3ef806aed0 0x7f3ef806d380 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:00:32.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.973+0000 7f3f144f4700 1 --2- 172.21.15.32:0/2367982367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f0c06e440 0x7f3f0c11c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:00:32.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.973+0000 7f3f144f4700 1 --2- 172.21.15.32:0/2367982367 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f0c06ed90 0x7f3f0c11cb00 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:00:32.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.973+0000 7f3f144f4700 1 -- 172.21.15.32:0/2367982367 >> 172.21.15.32:0/2367982367 conn(0x7f3f0c069dc0 msgr2=0x7f3f0c107260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:00:32.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.974+0000 7f3f144f4700 1 -- 172.21.15.32:0/2367982367 shutdown_connections 2024-05-12T04:00:32.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:32.974+0000 7f3f144f4700 1 -- 172.21.15.32:0/2367982367 wait complete. 2024-05-12T04:00:33.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.102+0000 7fea4f2d5700 1 -- 172.21.15.32:0/2578973792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea480fbcb0 msgr2=0x7fea480fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:00:33.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.102+0000 7fea4f2d5700 1 --2- 172.21.15.32:0/2578973792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea480fbcb0 0x7fea480fc0c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fea44004940 tx=0x7fea44009e30 comp rx=0 tx=0).stop 2024-05-12T04:00:33.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.103+0000 7fea4f2d5700 1 -- 172.21.15.32:0/2578973792 shutdown_connections 2024-05-12T04:00:33.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.103+0000 7fea4f2d5700 1 --2- 172.21.15.32:0/2578973792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea480fcf60 0x7fea480fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:00:33.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.103+0000 7fea4f2d5700 1 --2- 172.21.15.32:0/2578973792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea480fbcb0 0x7fea480fc0c0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:00:33.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.103+0000 7fea4f2d5700 1 -- 172.21.15.32:0/2578973792 >> 172.21.15.32:0/2578973792 conn(0x7fea480f7430 msgr2=0x7fea480f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:00:33.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.103+0000 7fea4f2d5700 1 -- 172.21.15.32:0/2578973792 shutdown_connections 2024-05-12T04:00:33.103 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.103+0000 7fea4f2d5700 1 -- 172.21.15.32:0/2578973792 wait complete. 2024-05-12T04:00:33.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.103+0000 7fea4f2d5700 1 Processor -- start 2024-05-12T04:00:33.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.103+0000 7fea4f2d5700 1 -- start start 2024-05-12T04:00:33.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.104+0000 7fea4f2d5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea480fbcb0 0x7fea4810d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:00:33.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.104+0000 7fea4f2d5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea480fcf60 0x7fea4810de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:00:33.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.104+0000 7fea4f2d5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4810e470 con 0x7fea480fcf60 2024-05-12T04:00:33.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.104+0000 7fea4f2d5700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4810ec20 con 0x7fea480fbcb0 2024-05-12T04:00:33.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.104+0000 7fea4c870700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea480fcf60 0x7fea4810de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:00:33.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.104+0000 7fea4c870700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea480fcf60 0x7fea4810de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39908/0 (socket says 172.21.15.32:39908) 2024-05-12T04:00:33.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.104+0000 7fea4c870700 1 -- 172.21.15.32:0/2178161133 learned_addr learned my addr 172.21.15.32:0/2178161133 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:00:33.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.104+0000 7fea4d071700 1 --2- 172.21.15.32:0/2178161133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea480fbcb0 0x7fea4810d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:00:33.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.104+0000 7fea4c870700 1 -- 172.21.15.32:0/2178161133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea480fbcb0 msgr2=0x7fea4810d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:00:33.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.104+0000 7fea4c870700 1 --2- 172.21.15.32:0/2178161133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea480fbcb0 0x7fea4810d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:00:33.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.104+0000 7fea4c870700 1 -- 172.21.15.32:0/2178161133 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea44005040 con 0x7fea480fcf60 2024-05-12T04:00:33.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.104+0000 7fea4c870700 1 --2- 172.21.15.32:0/2178161133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea480fcf60 0x7fea4810de70 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fea3800eb40 tx=0x7fea3800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:00:33.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.104+0000 7fea3dffb700 1 -- 172.21.15.32:0/2178161133 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea38007cc0 con 0x7fea480fcf60 2024-05-12T04:00:33.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.105+0000 7fea4f2d5700 1 -- 172.21.15.32:0/2178161133 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea4810eea0 con 0x7fea480fcf60 2024-05-12T04:00:33.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.105+0000 7fea3dffb700 1 -- 172.21.15.32:0/2178161133 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fea3800f040 con 0x7fea480fcf60 2024-05-12T04:00:33.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.105+0000 7fea3dffb700 1 -- 172.21.15.32:0/2178161133 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea3800c990 con 0x7fea480fcf60 2024-05-12T04:00:33.106 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.105+0000 7fea4f2d5700 1 -- 172.21.15.32:0/2178161133 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea4810f3e0 con 0x7fea480fcf60 2024-05-12T04:00:33.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.106+0000 7fea4f2d5700 1 -- 172.21.15.32:0/2178161133 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea48101af0 con 0x7fea480fcf60 2024-05-12T04:00:33.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.106+0000 7fea3dffb700 1 -- 172.21.15.32:0/2178161133 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 25) v1 ==== 88772+0+0 (secure 0 0 0) 0x7fea3800cc00 con 0x7fea480fcf60 2024-05-12T04:00:33.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.107+0000 7fea3dffb700 1 --2- 172.21.15.32:0/2178161133 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7fea3406aec0 0x7fea3406d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:00:33.107 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.107+0000 7fea3dffb700 1 -- 172.21.15.32:0/2178161133 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fea38089430 con 0x7fea480fcf60 2024-05-12T04:00:33.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.107+0000 7fea4d071700 1 --2- 172.21.15.32:0/2178161133 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7fea3406aec0 0x7fea3406d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:00:33.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.108+0000 7fea4d071700 1 --2- 172.21.15.32:0/2178161133 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7fea3406aec0 0x7fea3406d370 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fea44004890 tx=0x7fea44009e30 comp rx=0 tx=0).ready entity=mgr.24389 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:00:33.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.115+0000 7fea3dffb700 1 -- 172.21.15.32:0/2178161133 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181777 (secure 0 0 0) 0x7fea38057290 con 0x7fea480fcf60 2024-05-12T04:00:33.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:32 smithi033 ceph-mon[49525]: from='client.24429 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:00:33.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:32 smithi033 ceph-mon[49525]: pgmap v14: 65 pgs: 65 active+clean; 137 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.6 MiB/s rd, 4.0 MiB/s wr, 221 op/s 2024-05-12T04:00:33.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:32 smithi032 ceph-mon[40008]: from='client.24429 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:00:33.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:32 smithi032 ceph-mon[40008]: pgmap v14: 65 pgs: 65 active+clean; 137 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.6 MiB/s rd, 4.0 MiB/s wr, 221 op/s 2024-05-12T04:00:33.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.467+0000 7fea4f2d5700 1 -- 172.21.15.32:0/2178161133 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fea4805f5c0 con 0x7fea480fcf60 2024-05-12T04:00:33.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.468+0000 7fea3dffb700 1 -- 172.21.15.32:0/2178161133 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1828 (secure 0 0 0) 0x7fea38056e20 con 0x7fea480fcf60 2024-05-12T04:00:33.468 INFO:teuthology.orchestra.run.smithi032.stdout:e18 2024-05-12T04:00:33.468 INFO:teuthology.orchestra.run.smithi032.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-12T04:00:33.469 INFO:teuthology.orchestra.run.smithi032.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-12T04:00:33.469 INFO:teuthology.orchestra.run.smithi032.stdout:legacy client fscid: 1 2024-05-12T04:00:33.469 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:00:33.469 INFO:teuthology.orchestra.run.smithi032.stdout:Filesystem 'cephfs' (1) 2024-05-12T04:00:33.469 INFO:teuthology.orchestra.run.smithi032.stdout:fs_name cephfs 2024-05-12T04:00:33.469 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 18 2024-05-12T04:00:33.469 INFO:teuthology.orchestra.run.smithi032.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-12T04:00:33.469 INFO:teuthology.orchestra.run.smithi032.stdout:created 2024-05-12T03:57:48.904614+0000 2024-05-12T04:00:33.469 INFO:teuthology.orchestra.run.smithi032.stdout:modified 2024-05-12T03:58:06.821591+0000 2024-05-12T04:00:33.469 INFO:teuthology.orchestra.run.smithi032.stdout:tableserver 0 2024-05-12T04:00:33.469 INFO:teuthology.orchestra.run.smithi032.stdout:root 0 2024-05-12T04:00:33.469 INFO:teuthology.orchestra.run.smithi032.stdout:session_timeout 60 2024-05-12T04:00:33.470 INFO:teuthology.orchestra.run.smithi032.stdout:session_autoclose 300 2024-05-12T04:00:33.470 INFO:teuthology.orchestra.run.smithi032.stdout:max_file_size 1099511627776 2024-05-12T04:00:33.470 INFO:teuthology.orchestra.run.smithi032.stdout:required_client_features {} 2024-05-12T04:00:33.470 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure 0 2024-05-12T04:00:33.470 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure_osd_epoch 46 2024-05-12T04:00:33.470 INFO:teuthology.orchestra.run.smithi032.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-12T04:00:33.470 INFO:teuthology.orchestra.run.smithi032.stdout:max_mds 1 2024-05-12T04:00:33.470 INFO:teuthology.orchestra.run.smithi032.stdout:in 0 2024-05-12T04:00:33.470 INFO:teuthology.orchestra.run.smithi032.stdout:up {0=14474} 2024-05-12T04:00:33.470 INFO:teuthology.orchestra.run.smithi032.stdout:failed 2024-05-12T04:00:33.470 INFO:teuthology.orchestra.run.smithi032.stdout:damaged 2024-05-12T04:00:33.470 INFO:teuthology.orchestra.run.smithi032.stdout:stopped 2024-05-12T04:00:33.470 INFO:teuthology.orchestra.run.smithi032.stdout:data_pools [3] 2024-05-12T04:00:33.471 INFO:teuthology.orchestra.run.smithi032.stdout:metadata_pool 2 2024-05-12T04:00:33.471 INFO:teuthology.orchestra.run.smithi032.stdout:inline_data enabled 2024-05-12T04:00:33.471 INFO:teuthology.orchestra.run.smithi032.stdout:balancer 2024-05-12T04:00:33.471 INFO:teuthology.orchestra.run.smithi032.stdout:standby_count_wanted 1 2024-05-12T04:00:33.471 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.qpsawh{0:14474} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.32:6828/3100155296,v1:172.21.15.32:6829/3100155296] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:00:33.471 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi033.jgbjhk{0:24267} state up:standby-replay seq 3 join_fscid=1 addr [v2:172.21.15.33:6824/55417471,v1:172.21.15.33:6825/55417471] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:00:33.471 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:00:33.471 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:00:33.471 INFO:teuthology.orchestra.run.smithi032.stdout:Standby daemons: 2024-05-12T04:00:33.471 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:00:33.471 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi033.hglwml{-1:24283} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/433858238,v1:172.21.15.33:6827/433858238] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:00:33.471 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.ialtqb{-1:24287} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.32:6826/3703132329,v1:172.21.15.32:6827/3703132329] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:00:33.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.471+0000 7fea4f2d5700 1 -- 172.21.15.32:0/2178161133 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7fea3406aec0 msgr2=0x7fea3406d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:00:33.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.471+0000 7fea4f2d5700 1 --2- 172.21.15.32:0/2178161133 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7fea3406aec0 0x7fea3406d370 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fea44004890 tx=0x7fea44009e30 comp rx=0 tx=0).stop 2024-05-12T04:00:33.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.471+0000 7fea4f2d5700 1 -- 172.21.15.32:0/2178161133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea480fcf60 msgr2=0x7fea4810de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:00:33.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.471+0000 7fea4f2d5700 1 --2- 172.21.15.32:0/2178161133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea480fcf60 0x7fea4810de70 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fea3800eb40 tx=0x7fea3800b040 comp rx=0 tx=0).stop 2024-05-12T04:00:33.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.471+0000 7fea4f2d5700 1 -- 172.21.15.32:0/2178161133 shutdown_connections 2024-05-12T04:00:33.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.471+0000 7fea4f2d5700 1 --2- 172.21.15.32:0/2178161133 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7fea3406aec0 0x7fea3406d370 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:00:33.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.471+0000 7fea4f2d5700 1 --2- 172.21.15.32:0/2178161133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea480fbcb0 0x7fea4810d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:00:33.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.471+0000 7fea4f2d5700 1 --2- 172.21.15.32:0/2178161133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea480fcf60 0x7fea4810de70 secure :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fea3800eb40 tx=0x7fea3800b040 comp rx=0 tx=0).stop 2024-05-12T04:00:33.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.471+0000 7fea4f2d5700 1 -- 172.21.15.32:0/2178161133 >> 172.21.15.32:0/2178161133 conn(0x7fea480f7430 msgr2=0x7fea48065b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:00:33.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.471+0000 7fea4f2d5700 1 -- 172.21.15.32:0/2178161133 shutdown_connections 2024-05-12T04:00:33.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.471+0000 7fea4f2d5700 1 -- 172.21.15.32:0/2178161133 wait complete. 2024-05-12T04:00:33.473 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 18 2024-05-12T04:00:33.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.603+0000 7f0793264700 1 -- 172.21.15.32:0/1324251356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f078c0fbcc0 msgr2=0x7f078c0fc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:00:33.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.603+0000 7f0793264700 1 --2- 172.21.15.32:0/1324251356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f078c0fbcc0 0x7f078c0fc0d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f0780004940 tx=0x7f0780009e30 comp rx=0 tx=0).stop 2024-05-12T04:00:33.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.603+0000 7f0793264700 1 -- 172.21.15.32:0/1324251356 shutdown_connections 2024-05-12T04:00:33.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.603+0000 7f0793264700 1 --2- 172.21.15.32:0/1324251356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f078c0fcf70 0x7f078c0fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:00:33.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.603+0000 7f0793264700 1 --2- 172.21.15.32:0/1324251356 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f078c0fbcc0 0x7f078c0fc0d0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:00:33.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.603+0000 7f0793264700 1 -- 172.21.15.32:0/1324251356 >> 172.21.15.32:0/1324251356 conn(0x7f078c0f7420 msgr2=0x7f078c0f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:00:33.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.603+0000 7f0793264700 1 -- 172.21.15.32:0/1324251356 shutdown_connections 2024-05-12T04:00:33.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.603+0000 7f0793264700 1 -- 172.21.15.32:0/1324251356 wait complete. 2024-05-12T04:00:33.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.604+0000 7f0793264700 1 Processor -- start 2024-05-12T04:00:33.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.604+0000 7f0793264700 1 -- start start 2024-05-12T04:00:33.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.604+0000 7f0793264700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f078c0fbcc0 0x7f078c113f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:00:33.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.604+0000 7f0793264700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f078c0fcf70 0x7f078c114460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:00:33.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.604+0000 7f0793264700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078c1151d0 con 0x7f078c0fbcc0 2024-05-12T04:00:33.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.604+0000 7f0793264700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078c115340 con 0x7f078c0fcf70 2024-05-12T04:00:33.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.605+0000 7f078bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f078c0fcf70 0x7f078c114460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:00:33.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.605+0000 7f0791000700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f078c0fbcc0 0x7f078c113f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:00:33.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.605+0000 7f078bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f078c0fcf70 0x7f078c114460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.32:58014/0 (socket says 172.21.15.32:58014) 2024-05-12T04:00:33.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.605+0000 7f078bfff700 1 -- 172.21.15.32:0/4212367404 learned_addr learned my addr 172.21.15.32:0/4212367404 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:00:33.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.605+0000 7f0791000700 1 -- 172.21.15.32:0/4212367404 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f078c0fcf70 msgr2=0x7f078c114460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:00:33.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.605+0000 7f0791000700 1 --2- 172.21.15.32:0/4212367404 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f078c0fcf70 0x7f078c114460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:00:33.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.605+0000 7f0791000700 1 -- 172.21.15.32:0/4212367404 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0780005040 con 0x7f078c0fbcc0 2024-05-12T04:00:33.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.605+0000 7f078bfff700 1 --2- 172.21.15.32:0/4212367404 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f078c0fcf70 0x7f078c114460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-12T04:00:33.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.605+0000 7f0791000700 1 --2- 172.21.15.32:0/4212367404 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f078c0fbcc0 0x7f078c113f20 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f07800137b0 tx=0x7f0780009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:00:33.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.606+0000 7f07897fa700 1 -- 172.21.15.32:0/4212367404 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0780021070 con 0x7f078c0fbcc0 2024-05-12T04:00:33.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.606+0000 7f07897fa700 1 -- 172.21.15.32:0/4212367404 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f078001c400 con 0x7f078c0fbcc0 2024-05-12T04:00:33.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.606+0000 7f0793264700 1 -- 172.21.15.32:0/4212367404 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f078c115560 con 0x7f078c0fbcc0 2024-05-12T04:00:33.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.606+0000 7f07897fa700 1 -- 172.21.15.32:0/4212367404 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f078000e070 con 0x7f078c0fbcc0 2024-05-12T04:00:33.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.606+0000 7f0793264700 1 -- 172.21.15.32:0/4212367404 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f078c115b20 con 0x7f078c0fbcc0 2024-05-12T04:00:33.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.607+0000 7f0793264700 1 -- 172.21.15.32:0/4212367404 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07700052e0 con 0x7f078c0fbcc0 2024-05-12T04:00:33.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.608+0000 7f07897fa700 1 -- 172.21.15.32:0/4212367404 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 25) v1 ==== 88772+0+0 (secure 0 0 0) 0x7f078000b650 con 0x7f078c0fbcc0 2024-05-12T04:00:33.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.608+0000 7f07897fa700 1 --2- 172.21.15.32:0/4212367404 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f077c06af90 0x7f077c06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:00:33.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.608+0000 7f07897fa700 1 -- 172.21.15.32:0/4212367404 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f078008a830 con 0x7f078c0fbcc0 2024-05-12T04:00:33.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.609+0000 7f078bfff700 1 --2- 172.21.15.32:0/4212367404 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f077c06af90 0x7f077c06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:00:33.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.609+0000 7f078bfff700 1 --2- 172.21.15.32:0/4212367404 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f077c06af90 0x7f077c06d440 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f07780041d0 tx=0x7f077800b040 comp rx=0 tx=0).ready entity=mgr.24389 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:00:33.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.616+0000 7f07897fa700 1 -- 172.21.15.32:0/4212367404 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181777 (secure 0 0 0) 0x7f0780054f60 con 0x7f078c0fbcc0 2024-05-12T04:00:33.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:33 smithi033 ceph-mon[49525]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:00:33.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:33 smithi033 ceph-mon[49525]: from='client.14680 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:00:33.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:33 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/2367982367' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:00:33.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:33 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/2178161133' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-12T04:00:33.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.941+0000 7f0793264700 1 -- 172.21.15.32:0/4212367404 --> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0770000bf0 con 0x7f077c06af90 2024-05-12T04:00:33.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.942+0000 7f07897fa700 1 -- 172.21.15.32:0/4212367404 <== mgr.24389 v2:172.21.15.33:6828/1598726962 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7f0770000bf0 con 0x7f077c06af90 2024-05-12T04:00:33.942 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-05-12T04:00:33.942 INFO:teuthology.orchestra.run.smithi032.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:19c9734256aa43a60e203feb242f50bf8b7f58666d520536af43cc4fee96173a", 2024-05-12T04:00:33.942 INFO:teuthology.orchestra.run.smithi032.stdout: "in_progress": true, 2024-05-12T04:00:33.942 INFO:teuthology.orchestra.run.smithi032.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-12T04:00:33.942 INFO:teuthology.orchestra.run.smithi032.stdout: "services_complete": [], 2024-05-12T04:00:33.942 INFO:teuthology.orchestra.run.smithi032.stdout: "progress": "1/21 daemons upgraded", 2024-05-12T04:00:33.942 INFO:teuthology.orchestra.run.smithi032.stdout: "message": "", 2024-05-12T04:00:33.942 INFO:teuthology.orchestra.run.smithi032.stdout: "is_paused": false 2024-05-12T04:00:33.942 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-05-12T04:00:33.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.945+0000 7f0793264700 1 -- 172.21.15.32:0/4212367404 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f077c06af90 msgr2=0x7f077c06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:00:33.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.945+0000 7f0793264700 1 --2- 172.21.15.32:0/4212367404 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f077c06af90 0x7f077c06d440 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f07780041d0 tx=0x7f077800b040 comp rx=0 tx=0).stop 2024-05-12T04:00:33.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.945+0000 7f0793264700 1 -- 172.21.15.32:0/4212367404 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f078c0fbcc0 msgr2=0x7f078c113f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:00:33.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.945+0000 7f0793264700 1 --2- 172.21.15.32:0/4212367404 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f078c0fbcc0 0x7f078c113f20 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f07800137b0 tx=0x7f0780009e30 comp rx=0 tx=0).stop 2024-05-12T04:00:33.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.945+0000 7f0793264700 1 -- 172.21.15.32:0/4212367404 shutdown_connections 2024-05-12T04:00:33.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.945+0000 7f0793264700 1 --2- 172.21.15.32:0/4212367404 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f077c06af90 0x7f077c06d440 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:00:33.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.945+0000 7f0793264700 1 --2- 172.21.15.32:0/4212367404 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f078c0fbcc0 0x7f078c113f20 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:00:33.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.945+0000 7f0793264700 1 --2- 172.21.15.32:0/4212367404 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f078c0fcf70 0x7f078c114460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:00:33.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.945+0000 7f0793264700 1 -- 172.21.15.32:0/4212367404 >> 172.21.15.32:0/4212367404 conn(0x7f078c0f7420 msgr2=0x7f078c0f8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:00:33.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.945+0000 7f0793264700 1 -- 172.21.15.32:0/4212367404 shutdown_connections 2024-05-12T04:00:33.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:33.945+0000 7f0793264700 1 -- 172.21.15.32:0/4212367404 wait complete. 2024-05-12T04:00:34.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:34.073+0000 7f12d3c21700 1 -- 172.21.15.32:0/638035316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12cc106880 msgr2=0x7f12cc106cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:00:34.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:34.073+0000 7f12d3c21700 1 --2- 172.21.15.32:0/638035316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12cc106880 0x7f12cc106cf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f12bc004940 tx=0x7f12bc009e30 comp rx=0 tx=0).stop 2024-05-12T04:00:34.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:34.074+0000 7f12d3c21700 1 -- 172.21.15.32:0/638035316 shutdown_connections 2024-05-12T04:00:34.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:34.074+0000 7f12d3c21700 1 --2- 172.21.15.32:0/638035316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12cc106880 0x7f12cc106cf0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:00:34.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:34.074+0000 7f12d3c21700 1 --2- 172.21.15.32:0/638035316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12cc1055d0 0x7f12cc1059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:00:34.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:34.074+0000 7f12d3c21700 1 -- 172.21.15.32:0/638035316 >> 172.21.15.32:0/638035316 conn(0x7f12cc06bf50 msgr2=0x7f12cc06e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:00:34.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:34.074+0000 7f12d3c21700 1 -- 172.21.15.32:0/638035316 shutdown_connections 2024-05-12T04:00:34.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:34.074+0000 7f12d3c21700 1 -- 172.21.15.32:0/638035316 wait complete. 2024-05-12T04:00:34.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:34.074+0000 7f12d3c21700 1 Processor -- start 2024-05-12T04:00:34.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:34.074+0000 7f12d3c21700 1 -- start start 2024-05-12T04:00:34.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:34.075+0000 7f12d3c21700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12cc1055d0 0x7f12cc068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:00:34.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:34.075+0000 7f12d3c21700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12cc106880 0x7f12cc0674c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:00:34.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:34.075+0000 7f12d3c21700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc069450 con 0x7f12cc106880 2024-05-12T04:00:34.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:34.075+0000 7f12d3c21700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc119e20 con 0x7f12cc1055d0 2024-05-12T04:00:34.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:34.075+0000 7f12d11bc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12cc106880 0x7f12cc0674c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:00:34.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:34.075+0000 7f12d11bc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12cc106880 0x7f12cc0674c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39922/0 (socket says 172.21.15.32:39922) 2024-05-12T04:00:34.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:34.075+0000 7f12d11bc700 1 -- 172.21.15.32:0/2065743168 learned_addr learned my addr 172.21.15.32:0/2065743168 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:00:34.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:34.075+0000 7f12d19bd700 1 --2- 172.21.15.32:0/2065743168 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12cc1055d0 0x7f12cc068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:00:34.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:34.075+0000 7f12d11bc700 1 -- 172.21.15.32:0/2065743168 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12cc1055d0 msgr2=0x7f12cc068dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:00:34.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:34.075+0000 7f12d11bc700 1 --2- 172.21.15.32:0/2065743168 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12cc1055d0 0x7f12cc068dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:00:34.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:34.075+0000 7f12d11bc700 1 -- 172.21.15.32:0/2065743168 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12bc005040 con 0x7f12cc106880 2024-05-12T04:00:34.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:34.075+0000 7f12d19bd700 1 --2- 172.21.15.32:0/2065743168 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12cc1055d0 0x7f12cc068dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-12T04:00:34.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:34.075+0000 7f12d11bc700 1 --2- 172.21.15.32:0/2065743168 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12cc106880 0x7f12cc0674c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f12bc009180 tx=0x7f12bc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:00:34.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:34.075+0000 7f12c27fc700 1 -- 172.21.15.32:0/2065743168 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12bc00e040 con 0x7f12cc106880 2024-05-12T04:00:34.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:34.075+0000 7f12c27fc700 1 -- 172.21.15.32:0/2065743168 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f12bc013420 con 0x7f12cc106880 2024-05-12T04:00:34.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:34.075+0000 7f12d3c21700 1 -- 172.21.15.32:0/2065743168 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12cc067a00 con 0x7f12cc106880 2024-05-12T04:00:34.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:34.075+0000 7f12c27fc700 1 -- 172.21.15.32:0/2065743168 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12bc01c630 con 0x7f12cc106880 2024-05-12T04:00:34.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:34.076+0000 7f12d3c21700 1 -- 172.21.15.32:0/2065743168 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12cc067fc0 con 0x7f12cc106880 2024-05-12T04:00:34.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:34.077+0000 7f12d3c21700 1 -- 172.21.15.32:0/2065743168 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12cc10e810 con 0x7f12cc106880 2024-05-12T04:00:34.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:34.078+0000 7f12c27fc700 1 -- 172.21.15.32:0/2065743168 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 25) v1 ==== 88772+0+0 (secure 0 0 0) 0x7f12bc0139e0 con 0x7f12cc106880 2024-05-12T04:00:34.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:34.079+0000 7f12c27fc700 1 --2- 172.21.15.32:0/2065743168 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f12b806aed0 0x7f12b806d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:00:34.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:34.079+0000 7f12c27fc700 1 -- 172.21.15.32:0/2065743168 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f12bc08b580 con 0x7f12cc106880 2024-05-12T04:00:34.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:34.079+0000 7f12d19bd700 1 --2- 172.21.15.32:0/2065743168 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f12b806aed0 0x7f12b806d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:00:34.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:34.080+0000 7f12d19bd700 1 --2- 172.21.15.32:0/2065743168 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f12b806aed0 0x7f12b806d380 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f12c8009fd0 tx=0x7f12c80091a0 comp rx=0 tx=0).ready entity=mgr.24389 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:00:34.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:34.083+0000 7f12c27fc700 1 -- 172.21.15.32:0/2065743168 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181777 (secure 0 0 0) 0x7f12bc059520 con 0x7f12cc106880 2024-05-12T04:00:34.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:33 smithi032 ceph-mon[40008]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:00:34.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:33 smithi032 ceph-mon[40008]: from='client.14680 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:00:34.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:33 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/2367982367' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:00:34.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:33 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/2178161133' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-12T04:00:34.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:34.534+0000 7f12d3c21700 1 -- 172.21.15.32:0/2065743168 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f12cc04c710 con 0x7f12cc106880 2024-05-12T04:00:34.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:34.534+0000 7f12c27fc700 1 -- 172.21.15.32:0/2065743168 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f12bc0590b0 con 0x7f12cc106880 2024-05-12T04:00:34.534 INFO:teuthology.orchestra.run.smithi032.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-12T04:00:34.534 INFO:teuthology.orchestra.run.smithi032.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-12T04:00:34.534 INFO:teuthology.orchestra.run.smithi032.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-12T04:00:34.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:34.537+0000 7f12d3c21700 1 -- 172.21.15.32:0/2065743168 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f12b806aed0 msgr2=0x7f12b806d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:00:34.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:34.537+0000 7f12d3c21700 1 --2- 172.21.15.32:0/2065743168 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f12b806aed0 0x7f12b806d380 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f12c8009fd0 tx=0x7f12c80091a0 comp rx=0 tx=0).stop 2024-05-12T04:00:34.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:34.537+0000 7f12d3c21700 1 -- 172.21.15.32:0/2065743168 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12cc106880 msgr2=0x7f12cc0674c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:00:34.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:34.537+0000 7f12d3c21700 1 --2- 172.21.15.32:0/2065743168 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12cc106880 0x7f12cc0674c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f12bc009180 tx=0x7f12bc009e30 comp rx=0 tx=0).stop 2024-05-12T04:00:34.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:34.537+0000 7f12d3c21700 1 -- 172.21.15.32:0/2065743168 shutdown_connections 2024-05-12T04:00:34.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:34.537+0000 7f12d3c21700 1 --2- 172.21.15.32:0/2065743168 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f12b806aed0 0x7f12b806d380 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:00:34.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:34.537+0000 7f12d3c21700 1 --2- 172.21.15.32:0/2065743168 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12cc1055d0 0x7f12cc068dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:00:34.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:34.537+0000 7f12d3c21700 1 --2- 172.21.15.32:0/2065743168 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12cc106880 0x7f12cc0674c0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:00:34.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:34.537+0000 7f12d3c21700 1 -- 172.21.15.32:0/2065743168 >> 172.21.15.32:0/2065743168 conn(0x7f12cc06bf50 msgr2=0x7f12cc06d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:00:34.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:34.537+0000 7f12d3c21700 1 -- 172.21.15.32:0/2065743168 shutdown_connections 2024-05-12T04:00:34.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:00:34.537+0000 7f12d3c21700 1 -- 172.21.15.32:0/2065743168 wait complete. 2024-05-12T04:00:35.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:34 smithi033 ceph-mon[49525]: pgmap v15: 65 pgs: 65 active+clean; 137 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 160 op/s 2024-05-12T04:00:35.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:34 smithi033 ceph-mon[49525]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:00:35.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:34 smithi033 ceph-mon[49525]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:00:35.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:34 smithi033 ceph-mon[49525]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-12T04:00:35.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:34 smithi033 ceph-mon[49525]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-12T04:00:35.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:34 smithi033 ceph-mon[49525]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-12T04:00:35.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:34 smithi033 ceph-mon[49525]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-12T04:00:35.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:34 smithi033 ceph-mon[49525]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-12T04:00:35.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:34 smithi033 ceph-mon[49525]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-12T04:00:35.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:34 smithi033 ceph-mon[49525]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:00:35.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:34 smithi033 ceph-mon[49525]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:00:35.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:34 smithi033 ceph-mon[49525]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T04:00:35.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:34 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/2065743168' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-12T04:00:35.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:34 smithi032 ceph-mon[40008]: pgmap v15: 65 pgs: 65 active+clean; 137 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 160 op/s 2024-05-12T04:00:35.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:34 smithi032 ceph-mon[40008]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:00:35.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:34 smithi032 ceph-mon[40008]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:00:35.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:34 smithi032 ceph-mon[40008]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-12T04:00:35.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:34 smithi032 ceph-mon[40008]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-12T04:00:35.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:34 smithi032 ceph-mon[40008]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-12T04:00:35.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:34 smithi032 ceph-mon[40008]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-12T04:00:35.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:34 smithi032 ceph-mon[40008]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-12T04:00:35.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:34 smithi032 ceph-mon[40008]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-12T04:00:35.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:34 smithi032 ceph-mon[40008]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:00:35.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:34 smithi032 ceph-mon[40008]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:00:35.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:34 smithi032 ceph-mon[40008]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T04:00:35.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:34 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/2065743168' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-12T04:00:36.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:35 smithi032 ceph-mon[40008]: from='client.14692 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:00:36.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:35 smithi032 ceph-mon[40008]: Adjusting osd_memory_target on smithi033 to 2594M 2024-05-12T04:00:36.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:35 smithi032 ceph-mon[40008]: Updating smithi032:/etc/ceph/ceph.conf 2024-05-12T04:00:36.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:35 smithi032 ceph-mon[40008]: Updating smithi033:/etc/ceph/ceph.conf 2024-05-12T04:00:36.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:35 smithi032 ceph-mon[40008]: Updating smithi033:/var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-12T04:00:36.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:35 smithi032 ceph-mon[40008]: Updating smithi032:/var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-12T04:00:36.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:35 smithi032 ceph-mon[40008]: Updating smithi033:/etc/ceph/ceph.client.admin.keyring 2024-05-12T04:00:36.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:35 smithi032 ceph-mon[40008]: Updating smithi032:/etc/ceph/ceph.client.admin.keyring 2024-05-12T04:00:36.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:35 smithi032 ceph-mon[40008]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:00:36.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:35 smithi032 ceph-mon[40008]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:00:36.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:35 smithi032 ceph-mon[40008]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:00:36.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:35 smithi032 ceph-mon[40008]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:00:36.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:35 smithi032 ceph-mon[40008]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:00:36.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:35 smithi032 ceph-mon[40008]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:00:36.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:35 smithi032 ceph-mon[40008]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:00:36.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:35 smithi032 ceph-mon[40008]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:00:36.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:35 smithi032 ceph-mon[40008]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:00:36.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:35 smithi033 ceph-mon[49525]: from='client.14692 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:00:36.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:35 smithi033 ceph-mon[49525]: Adjusting osd_memory_target on smithi033 to 2594M 2024-05-12T04:00:36.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:35 smithi033 ceph-mon[49525]: Updating smithi032:/etc/ceph/ceph.conf 2024-05-12T04:00:36.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:35 smithi033 ceph-mon[49525]: Updating smithi033:/etc/ceph/ceph.conf 2024-05-12T04:00:36.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:35 smithi033 ceph-mon[49525]: Updating smithi033:/var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-12T04:00:36.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:35 smithi033 ceph-mon[49525]: Updating smithi032:/var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-12T04:00:36.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:35 smithi033 ceph-mon[49525]: Updating smithi033:/etc/ceph/ceph.client.admin.keyring 2024-05-12T04:00:36.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:35 smithi033 ceph-mon[49525]: Updating smithi032:/etc/ceph/ceph.client.admin.keyring 2024-05-12T04:00:36.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:35 smithi033 ceph-mon[49525]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:00:36.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:35 smithi033 ceph-mon[49525]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:00:36.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:35 smithi033 ceph-mon[49525]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:00:36.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:35 smithi033 ceph-mon[49525]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:00:36.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:35 smithi033 ceph-mon[49525]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:00:36.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:35 smithi033 ceph-mon[49525]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:00:36.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:35 smithi033 ceph-mon[49525]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:00:36.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:35 smithi033 ceph-mon[49525]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:00:36.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:35 smithi033 ceph-mon[49525]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:00:37.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:36 smithi033 ceph-mon[49525]: Updating smithi033:/var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-12T04:00:37.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:36 smithi033 ceph-mon[49525]: Updating smithi032:/var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-12T04:00:37.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:36 smithi033 ceph-mon[49525]: Reconfiguring node-exporter.smithi032 (dependencies changed)... 2024-05-12T04:00:37.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:36 smithi033 ceph-mon[49525]: Reconfiguring daemon node-exporter.smithi032 on smithi032 2024-05-12T04:00:37.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:36 smithi033 ceph-mon[49525]: pgmap v16: 65 pgs: 65 active+clean; 139 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 187 op/s 2024-05-12T04:00:37.177 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:36 smithi032 ceph-mon[40008]: Updating smithi033:/var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-12T04:00:37.177 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:36 smithi032 ceph-mon[40008]: Updating smithi032:/var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-12T04:00:37.177 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:36 smithi032 ceph-mon[40008]: Reconfiguring node-exporter.smithi032 (dependencies changed)... 2024-05-12T04:00:37.177 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:36 smithi032 ceph-mon[40008]: Reconfiguring daemon node-exporter.smithi032 on smithi032 2024-05-12T04:00:37.177 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:36 smithi032 ceph-mon[40008]: pgmap v16: 65 pgs: 65 active+clean; 139 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 187 op/s 2024-05-12T04:00:38.789 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:38 smithi032 ceph-mon[40008]: pgmap v17: 65 pgs: 65 active+clean; 153 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.4 MiB/s rd, 3.8 MiB/s wr, 236 op/s 2024-05-12T04:00:38.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:38 smithi033 ceph-mon[49525]: pgmap v17: 65 pgs: 65 active+clean; 153 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.4 MiB/s rd, 3.8 MiB/s wr, 236 op/s 2024-05-12T04:00:39.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:39 smithi032 ceph-mon[40008]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:00:39.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:39 smithi032 ceph-mon[40008]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:00:39.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:39 smithi032 ceph-mon[40008]: Reconfiguring alertmanager.smithi032 (dependencies changed)... 2024-05-12T04:00:39.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:39 smithi032 ceph-mon[40008]: Reconfiguring daemon alertmanager.smithi032 on smithi032 2024-05-12T04:00:39.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:39 smithi033 ceph-mon[49525]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:00:39.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:39 smithi033 ceph-mon[49525]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:00:39.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:39 smithi033 ceph-mon[49525]: Reconfiguring alertmanager.smithi032 (dependencies changed)... 2024-05-12T04:00:39.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:39 smithi033 ceph-mon[49525]: Reconfiguring daemon alertmanager.smithi032 on smithi032 2024-05-12T04:00:40.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:40 smithi033 ceph-mon[49525]: pgmap v18: 65 pgs: 65 active+clean; 153 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 185 op/s 2024-05-12T04:00:40.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:40 smithi033 ceph-mon[49525]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:00:41.039 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:40 smithi032 ceph-mon[40008]: pgmap v18: 65 pgs: 65 active+clean; 153 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 185 op/s 2024-05-12T04:00:41.039 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:40 smithi032 ceph-mon[40008]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:00:43.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:42 smithi033 ceph-mon[49525]: pgmap v19: 65 pgs: 65 active+clean; 170 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 2.5 MiB/s rd, 4.5 MiB/s wr, 337 op/s 2024-05-12T04:00:43.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:42 smithi032 ceph-mon[40008]: pgmap v19: 65 pgs: 65 active+clean; 170 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 2.5 MiB/s rd, 4.5 MiB/s wr, 337 op/s 2024-05-12T04:00:45.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:44 smithi033 ceph-mon[49525]: pgmap v20: 65 pgs: 65 active+clean; 170 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 247 op/s 2024-05-12T04:00:45.205 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:44 smithi032 ceph-mon[40008]: pgmap v20: 65 pgs: 65 active+clean; 170 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 247 op/s 2024-05-12T04:00:47.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:46 smithi033 ceph-mon[49525]: pgmap v21: 65 pgs: 65 active+clean; 174 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 293 op/s 2024-05-12T04:00:47.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:46 smithi032 ceph-mon[40008]: pgmap v21: 65 pgs: 65 active+clean; 174 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 293 op/s 2024-05-12T04:00:49.020 INFO:tasks.workunit.client.0.smithi032.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-05-12T04:00:49.028 INFO:tasks.workunit.client.0.smithi032.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-05-12T04:00:49.028 INFO:tasks.workunit.client.0.smithi032.stderr:+ make 2024-05-12T04:00:49.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:48 smithi033 ceph-mon[49525]: pgmap v22: 65 pgs: 65 active+clean; 189 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 2.4 MiB/s rd, 4.3 MiB/s wr, 359 op/s 2024-05-12T04:00:49.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:48 smithi032 ceph-mon[40008]: pgmap v22: 65 pgs: 65 active+clean; 189 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 2.4 MiB/s rd, 4.3 MiB/s wr, 359 op/s 2024-05-12T04:00:49.616 INFO:tasks.workunit.client.0.smithi032.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-05-12T04:00:51.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:50 smithi033 ceph-mon[49525]: pgmap v23: 65 pgs: 65 active+clean; 189 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 291 op/s 2024-05-12T04:00:51.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:50 smithi032 ceph-mon[40008]: pgmap v23: 65 pgs: 65 active+clean; 189 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 291 op/s 2024-05-12T04:00:51.923 INFO:tasks.workunit.client.0.smithi032.stderr:++ readlink -f fsstress 2024-05-12T04:00:51.925 INFO:tasks.workunit.client.0.smithi032.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-05-12T04:00:51.925 INFO:tasks.workunit.client.0.smithi032.stderr:+ popd 2024-05-12T04:00:51.926 INFO:tasks.workunit.client.0.smithi032.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-05-12T04:00:51.926 INFO:tasks.workunit.client.0.smithi032.stderr:+ popd 2024-05-12T04:00:51.926 INFO:tasks.workunit.client.0.smithi032.stdout:~/cephtest/mnt.0/client.0/tmp 2024-05-12T04:00:51.927 INFO:tasks.workunit.client.0.smithi032.stderr:++ mktemp -d -p . 2024-05-12T04:00:51.929 INFO:tasks.workunit.client.0.smithi032.stderr:+ T=./tmp.P7RwIzMTK8 2024-05-12T04:00:51.929 INFO:tasks.workunit.client.0.smithi032.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.P7RwIzMTK8 -l 1 -n 1000 -p 10 -v 2024-05-12T04:00:51.932 INFO:tasks.workunit.client.0.smithi032.stdout:seed = 1716387945 2024-05-12T04:00:51.938 INFO:tasks.workunit.client.0.smithi032.stdout:1/0: write - no filename 2024-05-12T04:00:51.940 INFO:tasks.workunit.client.0.smithi032.stdout:0/0: symlink l0 0 2024-05-12T04:00:51.940 INFO:tasks.workunit.client.0.smithi032.stdout:2/0: chown . 6048716 1 2024-05-12T04:00:51.941 INFO:tasks.workunit.client.0.smithi032.stdout:2/1: dread - no filename 2024-05-12T04:00:51.941 INFO:tasks.workunit.client.0.smithi032.stdout:2/2: fdatasync - no filename 2024-05-12T04:00:51.941 INFO:tasks.workunit.client.0.smithi032.stdout:1/1: symlink l0 0 2024-05-12T04:00:51.943 INFO:tasks.workunit.client.0.smithi032.stdout:1/2: mkdir d1 0 2024-05-12T04:00:51.945 INFO:tasks.workunit.client.0.smithi032.stdout:2/3: creat f0 x:0 0 0 2024-05-12T04:00:51.946 INFO:tasks.workunit.client.0.smithi032.stdout:4/0: dread - no filename 2024-05-12T04:00:51.946 INFO:tasks.workunit.client.0.smithi032.stdout:4/1: stat - no entries 2024-05-12T04:00:51.949 INFO:tasks.workunit.client.0.smithi032.stdout:2/4: mkdir d1 0 2024-05-12T04:00:51.949 INFO:tasks.workunit.client.0.smithi032.stdout:2/5: fdatasync f0 0 2024-05-12T04:00:51.949 INFO:tasks.workunit.client.0.smithi032.stdout:2/6: rename d1 to d1/d2 22 2024-05-12T04:00:51.950 INFO:tasks.workunit.client.0.smithi032.stdout:2/7: chown f0 11675509 1 2024-05-12T04:00:51.950 INFO:tasks.workunit.client.0.smithi032.stdout:5/0: stat - no entries 2024-05-12T04:00:51.951 INFO:tasks.workunit.client.0.smithi032.stdout:0/1: link l0 l1 0 2024-05-12T04:00:51.952 INFO:tasks.workunit.client.0.smithi032.stdout:3/0: creat f0 x:0 0 0 2024-05-12T04:00:51.953 INFO:tasks.workunit.client.0.smithi032.stdout:4/2: mkdir d0 0 2024-05-12T04:00:51.955 INFO:tasks.workunit.client.0.smithi032.stdout:6/0: read - no filename 2024-05-12T04:00:51.956 INFO:tasks.workunit.client.0.smithi032.stdout:2/8: rename f0 to d1/f3 0 2024-05-12T04:00:51.956 INFO:tasks.workunit.client.0.smithi032.stdout:2/9: chown d1/f3 1 1 2024-05-12T04:00:51.957 INFO:tasks.workunit.client.0.smithi032.stdout:2/10: dread - d1/f3 zero size 2024-05-12T04:00:51.957 INFO:tasks.workunit.client.0.smithi032.stdout:2/11: truncate d1/f3 692768 0 2024-05-12T04:00:51.958 INFO:tasks.workunit.client.0.smithi032.stdout:0/2: mknod c2 0 2024-05-12T04:00:51.958 INFO:tasks.workunit.client.0.smithi032.stdout:0/3: write - no filename 2024-05-12T04:00:51.959 INFO:tasks.workunit.client.0.smithi032.stdout:3/1: rename f0 to f1 0 2024-05-12T04:00:51.960 INFO:tasks.workunit.client.0.smithi032.stdout:1/3: link l0 d1/l2 0 2024-05-12T04:00:51.960 INFO:tasks.workunit.client.0.smithi032.stdout:1/4: write - no filename 2024-05-12T04:00:51.960 INFO:tasks.workunit.client.0.smithi032.stdout:1/5: stat d1 0 2024-05-12T04:00:51.963 INFO:tasks.workunit.client.0.smithi032.stdout:7/0: chown . 1669941 1 2024-05-12T04:00:51.963 INFO:tasks.workunit.client.0.smithi032.stdout:7/1: dread - no filename 2024-05-12T04:00:51.963 INFO:tasks.workunit.client.0.smithi032.stdout:5/1: mknod c0 0 2024-05-12T04:00:51.964 INFO:tasks.workunit.client.0.smithi032.stdout:2/12: write d1/f3 [479796,114614] 0 2024-05-12T04:00:51.965 INFO:tasks.workunit.client.0.smithi032.stdout:3/2: creat f2 x:0 0 0 2024-05-12T04:00:51.969 INFO:tasks.workunit.client.0.smithi032.stdout:4/3: mknod d0/c1 0 2024-05-12T04:00:51.969 INFO:tasks.workunit.client.0.smithi032.stdout:4/4: dread - no filename 2024-05-12T04:00:51.970 INFO:tasks.workunit.client.0.smithi032.stdout:4/5: write - no filename 2024-05-12T04:00:51.970 INFO:tasks.workunit.client.0.smithi032.stdout:4/6: truncate - no filename 2024-05-12T04:00:51.970 INFO:tasks.workunit.client.0.smithi032.stdout:6/1: symlink l0 0 2024-05-12T04:00:51.970 INFO:tasks.workunit.client.0.smithi032.stdout:6/2: dwrite - no filename 2024-05-12T04:00:51.970 INFO:tasks.workunit.client.0.smithi032.stdout:6/3: truncate - no filename 2024-05-12T04:00:51.970 INFO:tasks.workunit.client.0.smithi032.stdout:6/4: write - no filename 2024-05-12T04:00:51.970 INFO:tasks.workunit.client.0.smithi032.stdout:8/0: truncate - no filename 2024-05-12T04:00:51.970 INFO:tasks.workunit.client.0.smithi032.stdout:8/1: dwrite - no filename 2024-05-12T04:00:51.971 INFO:tasks.workunit.client.0.smithi032.stdout:5/2: creat f1 x:0 0 0 2024-05-12T04:00:51.972 INFO:tasks.workunit.client.0.smithi032.stdout:0/4: unlink l0 0 2024-05-12T04:00:51.973 INFO:tasks.workunit.client.0.smithi032.stdout:2/13: creat d1/f4 x:0 0 0 2024-05-12T04:00:51.974 INFO:tasks.workunit.client.0.smithi032.stdout:1/6: creat d1/f3 x:0 0 0 2024-05-12T04:00:51.975 INFO:tasks.workunit.client.0.smithi032.stdout:3/3: creat f3 x:0 0 0 2024-05-12T04:00:51.976 INFO:tasks.workunit.client.0.smithi032.stdout:1/7: write d1/f3 [420912,122056] 0 2024-05-12T04:00:51.976 INFO:tasks.workunit.client.0.smithi032.stdout:4/7: creat d0/f2 x:0 0 0 2024-05-12T04:00:51.976 INFO:tasks.workunit.client.0.smithi032.stdout:3/4: truncate f2 594143 0 2024-05-12T04:00:51.977 INFO:tasks.workunit.client.0.smithi032.stdout:6/5: symlink l1 0 2024-05-12T04:00:51.977 INFO:tasks.workunit.client.0.smithi032.stdout:6/6: dread - no filename 2024-05-12T04:00:51.978 INFO:tasks.workunit.client.0.smithi032.stdout:6/7: chown l1 3172240 1 2024-05-12T04:00:51.978 INFO:tasks.workunit.client.0.smithi032.stdout:7/2: mknod c0 0 2024-05-12T04:00:51.980 INFO:tasks.workunit.client.0.smithi032.stdout:9/0: read - no filename 2024-05-12T04:00:51.980 INFO:tasks.workunit.client.0.smithi032.stdout:9/1: write - no filename 2024-05-12T04:00:51.980 INFO:tasks.workunit.client.0.smithi032.stdout:5/3: creat f2 x:0 0 0 2024-05-12T04:00:51.981 INFO:tasks.workunit.client.0.smithi032.stdout:0/5: mknod c3 0 2024-05-12T04:00:51.982 INFO:tasks.workunit.client.0.smithi032.stdout:2/14: creat d1/f5 x:0 0 0 2024-05-12T04:00:51.984 INFO:tasks.workunit.client.0.smithi032.stdout:3/5: symlink l4 0 2024-05-12T04:00:51.985 INFO:tasks.workunit.client.0.smithi032.stdout:3/6: chown f2 456 1 2024-05-12T04:00:51.986 INFO:tasks.workunit.client.0.smithi032.stdout:1/8: dwrite d1/f3 [0,4194304] 0 2024-05-12T04:00:51.986 INFO:tasks.workunit.client.0.smithi032.stdout:7/3: creat f1 x:0 0 0 2024-05-12T04:00:51.988 INFO:tasks.workunit.client.0.smithi032.stdout:8/2: mkdir d0 0 2024-05-12T04:00:51.988 INFO:tasks.workunit.client.0.smithi032.stdout:8/3: dwrite - no filename 2024-05-12T04:00:51.988 INFO:tasks.workunit.client.0.smithi032.stdout:8/4: rename d0 to d0/d1 22 2024-05-12T04:00:51.989 INFO:tasks.workunit.client.0.smithi032.stdout:5/4: creat f3 x:0 0 0 2024-05-12T04:00:51.990 INFO:tasks.workunit.client.0.smithi032.stdout:5/5: chown f1 248965 1 2024-05-12T04:00:51.991 INFO:tasks.workunit.client.0.smithi032.stdout:2/15: creat d1/f6 x:0 0 0 2024-05-12T04:00:51.995 INFO:tasks.workunit.client.0.smithi032.stdout:9/2: mkdir d0 0 2024-05-12T04:00:51.995 INFO:tasks.workunit.client.0.smithi032.stdout:9/3: truncate - no filename 2024-05-12T04:00:51.996 INFO:tasks.workunit.client.0.smithi032.stdout:5/6: mkdir d4 0 2024-05-12T04:00:51.997 INFO:tasks.workunit.client.0.smithi032.stdout:5/7: write f1 [519132,38261] 0 2024-05-12T04:00:51.997 INFO:tasks.workunit.client.0.smithi032.stdout:0/6: rename l1 to l4 0 2024-05-12T04:00:51.998 INFO:tasks.workunit.client.0.smithi032.stdout:5/8: truncate f1 1131330 0 2024-05-12T04:00:51.999 INFO:tasks.workunit.client.0.smithi032.stdout:3/7: link f3 f5 0 2024-05-12T04:00:51.999 INFO:tasks.workunit.client.0.smithi032.stdout:5/9: read - f3 zero size 2024-05-12T04:00:52.000 INFO:tasks.workunit.client.0.smithi032.stdout:3/8: truncate f2 820715 0 2024-05-12T04:00:52.000 INFO:tasks.workunit.client.0.smithi032.stdout:3/9: rmdir - no directory 2024-05-12T04:00:52.004 INFO:tasks.workunit.client.0.smithi032.stdout:6/8: link l1 l2 0 2024-05-12T04:00:52.004 INFO:tasks.workunit.client.0.smithi032.stdout:6/9: dwrite - no filename 2024-05-12T04:00:52.004 INFO:tasks.workunit.client.0.smithi032.stdout:6/10: dread - no filename 2024-05-12T04:00:52.005 INFO:tasks.workunit.client.0.smithi032.stdout:8/5: creat d0/f2 x:0 0 0 2024-05-12T04:00:52.006 INFO:tasks.workunit.client.0.smithi032.stdout:0/7: creat f5 x:0 0 0 2024-05-12T04:00:52.006 INFO:tasks.workunit.client.0.smithi032.stdout:2/16: link d1/f5 d1/f7 0 2024-05-12T04:00:52.007 INFO:tasks.workunit.client.0.smithi032.stdout:6/11: chown l0 125659604 1 2024-05-12T04:00:52.007 INFO:tasks.workunit.client.0.smithi032.stdout:6/12: dwrite - no filename 2024-05-12T04:00:52.008 INFO:tasks.workunit.client.0.smithi032.stdout:8/6: dread - d0/f2 zero size 2024-05-12T04:00:52.008 INFO:tasks.workunit.client.0.smithi032.stdout:8/7: readlink - no filename 2024-05-12T04:00:52.008 INFO:tasks.workunit.client.0.smithi032.stdout:6/13: readlink l0 0 2024-05-12T04:00:52.009 INFO:tasks.workunit.client.0.smithi032.stdout:3/10: unlink f5 0 2024-05-12T04:00:52.009 INFO:tasks.workunit.client.0.smithi032.stdout:0/8: dread - f5 zero size 2024-05-12T04:00:52.010 INFO:tasks.workunit.client.0.smithi032.stdout:1/9: link d1/l2 d1/l4 0 2024-05-12T04:00:52.011 INFO:tasks.workunit.client.0.smithi032.stdout:5/10: dwrite f3 [0,4194304] 0 2024-05-12T04:00:52.011 INFO:tasks.workunit.client.0.smithi032.stdout:5/11: readlink - no filename 2024-05-12T04:00:52.011 INFO:tasks.workunit.client.0.smithi032.stdout:8/8: write d0/f2 [428694,854] 0 2024-05-12T04:00:52.014 INFO:tasks.workunit.client.0.smithi032.stdout:0/9: write f5 [932364,90898] 0 2024-05-12T04:00:52.014 INFO:tasks.workunit.client.0.smithi032.stdout:9/4: getdents d0 0 2024-05-12T04:00:52.014 INFO:tasks.workunit.client.0.smithi032.stdout:9/5: write - no filename 2024-05-12T04:00:52.014 INFO:tasks.workunit.client.0.smithi032.stdout:9/6: dwrite - no filename 2024-05-12T04:00:52.014 INFO:tasks.workunit.client.0.smithi032.stdout:2/17: mkdir d1/d8 0 2024-05-12T04:00:52.015 INFO:tasks.workunit.client.0.smithi032.stdout:9/7: chown d0 862113 1 2024-05-12T04:00:52.015 INFO:tasks.workunit.client.0.smithi032.stdout:5/12: truncate f2 1007317 0 2024-05-12T04:00:52.016 INFO:tasks.workunit.client.0.smithi032.stdout:6/14: creat f3 x:0 0 0 2024-05-12T04:00:52.017 INFO:tasks.workunit.client.0.smithi032.stdout:9/8: stat d0 0 2024-05-12T04:00:52.017 INFO:tasks.workunit.client.0.smithi032.stdout:9/9: dread - no filename 2024-05-12T04:00:52.017 INFO:tasks.workunit.client.0.smithi032.stdout:9/10: dread - no filename 2024-05-12T04:00:52.017 INFO:tasks.workunit.client.0.smithi032.stdout:9/11: dread - no filename 2024-05-12T04:00:52.017 INFO:tasks.workunit.client.0.smithi032.stdout:9/12: dwrite - no filename 2024-05-12T04:00:52.017 INFO:tasks.workunit.client.0.smithi032.stdout:9/13: dread - no filename 2024-05-12T04:00:52.017 INFO:tasks.workunit.client.0.smithi032.stdout:3/11: creat f6 x:0 0 0 2024-05-12T04:00:52.018 INFO:tasks.workunit.client.0.smithi032.stdout:5/13: write f2 [97565,7523] 0 2024-05-12T04:00:52.019 INFO:tasks.workunit.client.0.smithi032.stdout:9/14: chown d0 11321393 1 2024-05-12T04:00:52.024 INFO:tasks.workunit.client.0.smithi032.stdout:5/14: dread f3 [0,4194304] 0 2024-05-12T04:00:52.024 INFO:tasks.workunit.client.0.smithi032.stdout:5/15: chown f3 6723461 1 2024-05-12T04:00:52.025 INFO:tasks.workunit.client.0.smithi032.stdout:0/10: rename c2 to c6 0 2024-05-12T04:00:52.030 INFO:tasks.workunit.client.0.smithi032.stdout:6/15: symlink l4 0 2024-05-12T04:00:52.034 INFO:tasks.workunit.client.0.smithi032.stdout:2/18: dwrite d1/f5 [0,4194304] 0 2024-05-12T04:00:52.037 INFO:tasks.workunit.client.0.smithi032.stdout:5/16: dwrite f3 [0,4194304] 0 2024-05-12T04:00:52.038 INFO:tasks.workunit.client.0.smithi032.stdout:5/17: chown f2 642315 1 2024-05-12T04:00:52.039 INFO:tasks.workunit.client.0.smithi032.stdout:0/11: symlink l7 0 2024-05-12T04:00:52.044 INFO:tasks.workunit.client.0.smithi032.stdout:5/18: read f1 [52506,12953] 0 2024-05-12T04:00:52.045 INFO:tasks.workunit.client.0.smithi032.stdout:5/19: chown f2 5476718 1 2024-05-12T04:00:52.045 INFO:tasks.workunit.client.0.smithi032.stdout:6/16: rename l0 to l5 0 2024-05-12T04:00:52.047 INFO:tasks.workunit.client.0.smithi032.stdout:6/17: write f3 [1030594,113933] 0 2024-05-12T04:00:52.048 INFO:tasks.workunit.client.0.smithi032.stdout:6/18: chown l4 2 1 2024-05-12T04:00:52.049 INFO:tasks.workunit.client.0.smithi032.stdout:6/19: chown l4 125 1 2024-05-12T04:00:52.052 INFO:tasks.workunit.client.0.smithi032.stdout:5/20: dread f3 [0,4194304] 0 2024-05-12T04:00:52.053 INFO:tasks.workunit.client.0.smithi032.stdout:9/15: symlink d0/l1 0 2024-05-12T04:00:52.057 INFO:tasks.workunit.client.0.smithi032.stdout:3/12: dwrite f3 [0,4194304] 0 2024-05-12T04:00:52.063 INFO:tasks.workunit.client.0.smithi032.stdout:6/20: dread f3 [0,4194304] 0 2024-05-12T04:00:52.065 INFO:tasks.workunit.client.0.smithi032.stdout:9/16: rename d0/l1 to d0/l2 0 2024-05-12T04:00:52.070 INFO:tasks.workunit.client.0.smithi032.stdout:3/13: unlink f2 0 2024-05-12T04:00:52.073 INFO:tasks.workunit.client.0.smithi032.stdout:2/19: rmdir d1/d8 0 2024-05-12T04:00:52.074 INFO:tasks.workunit.client.0.smithi032.stdout:2/20: read - d1/f4 zero size 2024-05-12T04:00:52.074 INFO:tasks.workunit.client.0.smithi032.stdout:5/21: mknod d4/c5 0 2024-05-12T04:00:52.076 INFO:tasks.workunit.client.0.smithi032.stdout:6/21: creat f6 x:0 0 0 2024-05-12T04:00:52.077 INFO:tasks.workunit.client.0.smithi032.stdout:6/22: write f6 [558550,79012] 0 2024-05-12T04:00:52.081 INFO:tasks.workunit.client.0.smithi032.stdout:3/14: creat f7 x:0 0 0 2024-05-12T04:00:52.082 INFO:tasks.workunit.client.0.smithi032.stdout:3/15: write f6 [622452,31683] 0 2024-05-12T04:00:52.082 INFO:tasks.workunit.client.0.smithi032.stdout:3/16: rmdir - no directory 2024-05-12T04:00:52.090 INFO:tasks.workunit.client.0.smithi032.stdout:5/22: link f2 d4/f6 0 2024-05-12T04:00:52.092 INFO:tasks.workunit.client.0.smithi032.stdout:5/23: write f3 [2519813,102368] 0 2024-05-12T04:00:52.138 INFO:tasks.workunit.client.0.smithi032.stdout:3/17: rename f1 to f8 0 2024-05-12T04:00:52.152 INFO:tasks.workunit.client.0.smithi032.stdout:3/18: symlink l9 0 2024-05-12T04:00:52.152 INFO:tasks.workunit.client.0.smithi032.stdout:3/19: dread - f8 zero size 2024-05-12T04:00:52.152 INFO:tasks.workunit.client.0.smithi032.stdout:3/20: read - f7 zero size 2024-05-12T04:00:52.175 INFO:tasks.workunit.client.0.smithi032.stdout:5/24: fsync f1 0 2024-05-12T04:00:52.200 INFO:tasks.workunit.client.0.smithi032.stdout:7/4: getdents . 0 2024-05-12T04:00:52.216 INFO:tasks.workunit.client.0.smithi032.stdout:8/9: fsync d0/f2 0 2024-05-12T04:00:52.218 INFO:tasks.workunit.client.0.smithi032.stdout:8/10: read d0/f2 [392925,66667] 0 2024-05-12T04:00:52.220 INFO:tasks.workunit.client.0.smithi032.stdout:2/21: fsync d1/f7 0 2024-05-12T04:00:52.226 INFO:tasks.workunit.client.0.smithi032.stdout:8/11: dwrite d0/f2 [0,4194304] 0 2024-05-12T04:00:52.241 INFO:tasks.workunit.client.0.smithi032.stdout:1/10: truncate d1/f3 3678462 0 2024-05-12T04:00:52.245 INFO:tasks.workunit.client.0.smithi032.stdout:0/12: getdents . 0 2024-05-12T04:00:52.251 INFO:tasks.workunit.client.0.smithi032.stdout:6/23: stat l5 0 2024-05-12T04:00:52.252 INFO:tasks.workunit.client.0.smithi032.stdout:0/13: dwrite f5 [0,4194304] 0 2024-05-12T04:00:52.253 INFO:tasks.workunit.client.0.smithi032.stdout:6/24: dread f3 [0,4194304] 0 2024-05-12T04:00:52.556 INFO:tasks.workunit.client.0.smithi032.stdout:3/21: mknod ca 0 2024-05-12T04:00:52.557 INFO:tasks.workunit.client.0.smithi032.stdout:3/22: write f6 [14330,122329] 0 2024-05-12T04:00:52.557 INFO:tasks.workunit.client.0.smithi032.stdout:3/23: fsync f7 0 2024-05-12T04:00:52.558 INFO:tasks.workunit.client.0.smithi032.stdout:5/25: mkdir d4/d7 0 2024-05-12T04:00:52.559 INFO:tasks.workunit.client.0.smithi032.stdout:7/5: creat f2 x:0 0 0 2024-05-12T04:00:52.559 INFO:tasks.workunit.client.0.smithi032.stdout:7/6: dread - f2 zero size 2024-05-12T04:00:52.563 INFO:tasks.workunit.client.0.smithi032.stdout:8/12: rmdir d0 39 2024-05-12T04:00:52.568 INFO:tasks.workunit.client.0.smithi032.stdout:7/7: dwrite f2 [0,4194304] 0 2024-05-12T04:00:52.574 INFO:tasks.workunit.client.0.smithi032.stdout:3/24: mknod cb 0 2024-05-12T04:00:52.579 INFO:tasks.workunit.client.0.smithi032.stdout:2/22: rename d1/f5 to d1/f9 0 2024-05-12T04:00:52.581 INFO:tasks.workunit.client.0.smithi032.stdout:2/23: dread - d1/f4 zero size 2024-05-12T04:00:52.584 INFO:tasks.workunit.client.0.smithi032.stdout:8/13: dread d0/f2 [0,4194304] 0 2024-05-12T04:00:52.585 INFO:tasks.workunit.client.0.smithi032.stdout:8/14: write d0/f2 [289557,15630] 0 2024-05-12T04:00:52.585 INFO:tasks.workunit.client.0.smithi032.stdout:1/11: rename d1/l4 to d1/l5 0 2024-05-12T04:00:52.587 INFO:tasks.workunit.client.0.smithi032.stdout:7/8: mkdir d3 0 2024-05-12T04:00:52.590 INFO:tasks.workunit.client.0.smithi032.stdout:6/25: unlink l2 0 2024-05-12T04:00:52.593 INFO:tasks.workunit.client.0.smithi032.stdout:5/26: getdents d4/d7 0 2024-05-12T04:00:52.593 INFO:tasks.workunit.client.0.smithi032.stdout:6/26: dread f6 [0,4194304] 0 2024-05-12T04:00:52.594 INFO:tasks.workunit.client.0.smithi032.stdout:6/27: dread f6 [0,4194304] 0 2024-05-12T04:00:52.598 INFO:tasks.workunit.client.0.smithi032.stdout:1/12: creat d1/f6 x:0 0 0 2024-05-12T04:00:52.599 INFO:tasks.workunit.client.0.smithi032.stdout:8/15: creat d0/f3 x:0 0 0 2024-05-12T04:00:52.600 INFO:tasks.workunit.client.0.smithi032.stdout:8/16: fdatasync d0/f3 0 2024-05-12T04:00:52.608 INFO:tasks.workunit.client.0.smithi032.stdout:6/28: fsync f3 0 2024-05-12T04:00:52.610 INFO:tasks.workunit.client.0.smithi032.stdout:2/24: link d1/f9 d1/fa 0 2024-05-12T04:00:52.616 INFO:tasks.workunit.client.0.smithi032.stdout:3/25: link ca cc 0 2024-05-12T04:00:52.619 INFO:tasks.workunit.client.0.smithi032.stdout:5/27: creat d4/d7/f8 x:0 0 0 2024-05-12T04:00:52.620 INFO:tasks.workunit.client.0.smithi032.stdout:5/28: dread - d4/d7/f8 zero size 2024-05-12T04:00:52.621 INFO:tasks.workunit.client.0.smithi032.stdout:6/29: unlink l4 0 2024-05-12T04:00:52.622 INFO:tasks.workunit.client.0.smithi032.stdout:2/25: creat d1/fb x:0 0 0 2024-05-12T04:00:52.625 INFO:tasks.workunit.client.0.smithi032.stdout:7/9: rmdir d3 0 2024-05-12T04:00:52.626 INFO:tasks.workunit.client.0.smithi032.stdout:7/10: chown f1 221 1 2024-05-12T04:00:52.627 INFO:tasks.workunit.client.0.smithi032.stdout:3/26: creat fd x:0 0 0 2024-05-12T04:00:52.630 INFO:tasks.workunit.client.0.smithi032.stdout:5/29: rename c0 to d4/c9 0 2024-05-12T04:00:52.632 INFO:tasks.workunit.client.0.smithi032.stdout:2/26: unlink d1/fb 0 2024-05-12T04:00:52.632 INFO:tasks.workunit.client.0.smithi032.stdout:5/30: chown d4/d7/f8 6766161 1 2024-05-12T04:00:52.635 INFO:tasks.workunit.client.0.smithi032.stdout:7/11: symlink l4 0 2024-05-12T04:00:52.635 INFO:tasks.workunit.client.0.smithi032.stdout:3/27: dwrite f7 [0,4194304] 0 2024-05-12T04:00:52.637 INFO:tasks.workunit.client.0.smithi032.stdout:3/28: dread - f8 zero size 2024-05-12T04:00:52.642 INFO:tasks.workunit.client.0.smithi032.stdout:5/31: dwrite f3 [0,4194304] 0 2024-05-12T04:00:52.643 INFO:tasks.workunit.client.0.smithi032.stdout:2/27: creat d1/fc x:0 0 0 2024-05-12T04:00:52.645 INFO:tasks.workunit.client.0.smithi032.stdout:2/28: write d1/f4 [516785,110997] 0 2024-05-12T04:00:52.652 INFO:tasks.workunit.client.0.smithi032.stdout:2/29: symlink d1/ld 0 2024-05-12T04:00:52.658 INFO:tasks.workunit.client.0.smithi032.stdout:2/30: fdatasync d1/f4 0 2024-05-12T04:00:52.659 INFO:tasks.workunit.client.0.smithi032.stdout:5/32: truncate f1 1023287 0 2024-05-12T04:00:52.661 INFO:tasks.workunit.client.0.smithi032.stdout:2/31: creat d1/fe x:0 0 0 2024-05-12T04:00:52.661 INFO:tasks.workunit.client.0.smithi032.stdout:2/32: readlink d1/ld 0 2024-05-12T04:00:52.858 INFO:tasks.workunit.client.0.smithi032.stdout:0/14: creat f8 x:0 0 0 2024-05-12T04:00:52.861 INFO:tasks.workunit.client.0.smithi032.stdout:5/33: dwrite f2 [0,4194304] 0 2024-05-12T04:00:52.867 INFO:tasks.workunit.client.0.smithi032.stdout:5/34: rename f3 to d4/fa 0 2024-05-12T04:00:52.868 INFO:tasks.workunit.client.0.smithi032.stdout:2/33: unlink d1/f9 0 2024-05-12T04:00:52.871 INFO:tasks.workunit.client.0.smithi032.stdout:5/35: mkdir d4/db 0 2024-05-12T04:00:52.874 INFO:tasks.workunit.client.0.smithi032.stdout:5/36: symlink d4/d7/lc 0 2024-05-12T04:00:52.878 INFO:tasks.workunit.client.0.smithi032.stdout:5/37: rename d4/d7/f8 to d4/fd 0 2024-05-12T04:00:52.879 INFO:tasks.workunit.client.0.smithi032.stdout:5/38: readlink d4/d7/lc 0 2024-05-12T04:00:52.882 INFO:tasks.workunit.client.0.smithi032.stdout:5/39: creat d4/fe x:0 0 0 2024-05-12T04:00:52.883 INFO:tasks.workunit.client.0.smithi032.stdout:1/13: fsync d1/f3 0 2024-05-12T04:00:52.884 INFO:tasks.workunit.client.0.smithi032.stdout:1/14: stat d1/f6 0 2024-05-12T04:00:52.884 INFO:tasks.workunit.client.0.smithi032.stdout:1/15: chown d1 376440006 1 2024-05-12T04:00:52.885 INFO:tasks.workunit.client.0.smithi032.stdout:5/40: write d4/fd [242460,127635] 0 2024-05-12T04:00:52.886 INFO:tasks.workunit.client.0.smithi032.stdout:8/17: truncate d0/f2 339009 0 2024-05-12T04:00:52.891 INFO:tasks.workunit.client.0.smithi032.stdout:5/41: unlink d4/fe 0 2024-05-12T04:00:52.892 INFO:tasks.workunit.client.0.smithi032.stdout:8/18: creat d0/f4 x:0 0 0 2024-05-12T04:00:52.899 INFO:tasks.workunit.client.0.smithi032.stdout:5/42: creat d4/db/ff x:0 0 0 2024-05-12T04:00:52.907 INFO:tasks.workunit.client.0.smithi032.stdout:5/43: link d4/c9 d4/d7/c10 0 2024-05-12T04:00:52.909 INFO:tasks.workunit.client.0.smithi032.stdout:5/44: readlink d4/d7/lc 0 2024-05-12T04:00:52.910 INFO:tasks.workunit.client.0.smithi032.stdout:5/45: fdatasync f2 0 2024-05-12T04:00:52.914 INFO:tasks.workunit.client.0.smithi032.stdout:3/29: dwrite f7 [4194304,4194304] 0 2024-05-12T04:00:52.914 INFO:tasks.workunit.client.0.smithi032.stdout:7/12: truncate f2 1624260 0 2024-05-12T04:00:52.917 INFO:tasks.workunit.client.0.smithi032.stdout:5/46: creat d4/d7/f11 x:0 0 0 2024-05-12T04:00:52.918 INFO:tasks.workunit.client.0.smithi032.stdout:5/47: chown d4 77801 1 2024-05-12T04:00:52.926 INFO:tasks.workunit.client.0.smithi032.stdout:5/48: dwrite d4/fd [0,4194304] 0 2024-05-12T04:00:52.928 INFO:tasks.workunit.client.0.smithi032.stdout:3/30: getdents . 0 2024-05-12T04:00:52.931 INFO:tasks.workunit.client.0.smithi032.stdout:3/31: write f8 [210583,66821] 0 2024-05-12T04:00:52.932 INFO:tasks.workunit.client.0.smithi032.stdout:5/49: link d4/db/ff d4/f12 0 2024-05-12T04:00:52.935 INFO:tasks.workunit.client.0.smithi032.stdout:3/32: mkdir de 0 2024-05-12T04:00:52.937 INFO:tasks.workunit.client.0.smithi032.stdout:5/50: creat d4/db/f13 x:0 0 0 2024-05-12T04:00:52.942 INFO:tasks.workunit.client.0.smithi032.stdout:5/51: unlink d4/c5 0 2024-05-12T04:00:52.943 INFO:tasks.workunit.client.0.smithi032.stdout:3/33: getdents de 0 2024-05-12T04:00:52.944 INFO:tasks.workunit.client.0.smithi032.stdout:3/34: write f8 [347300,98213] 0 2024-05-12T04:00:52.948 INFO:tasks.workunit.client.0.smithi032.stdout:3/35: chown cc 1483511 1 2024-05-12T04:00:52.949 INFO:tasks.workunit.client.0.smithi032.stdout:3/36: write f6 [621419,33455] 0 2024-05-12T04:00:52.951 INFO:tasks.workunit.client.0.smithi032.stdout:5/52: fdatasync f1 0 2024-05-12T04:00:52.954 INFO:tasks.workunit.client.0.smithi032.stdout:5/53: dread f2 [0,4194304] 0 2024-05-12T04:00:52.961 INFO:tasks.workunit.client.0.smithi032.stdout:3/37: rmdir de 0 2024-05-12T04:00:52.983 INFO:tasks.workunit.client.0.smithi032.stdout:2/34: getdents d1 0 2024-05-12T04:00:52.986 INFO:tasks.workunit.client.0.smithi032.stdout:0/15: link f8 f9 0 2024-05-12T04:00:52.994 INFO:tasks.workunit.client.0.smithi032.stdout:1/16: read d1/f3 [3140831,79190] 0 2024-05-12T04:00:53.001 INFO:tasks.workunit.client.0.smithi032.stdout:1/17: mkdir d1/d7 0 2024-05-12T04:00:53.005 INFO:tasks.workunit.client.0.smithi032.stdout:1/18: mknod d1/d7/c8 0 2024-05-12T04:00:53.006 INFO:tasks.workunit.client.0.smithi032.stdout:0/16: dwrite f8 [0,4194304] 0 2024-05-12T04:00:53.007 INFO:tasks.workunit.client.0.smithi032.stdout:0/17: rmdir - no directory 2024-05-12T04:00:53.010 INFO:tasks.workunit.client.0.smithi032.stdout:1/19: chown d1/l2 6972235 1 2024-05-12T04:00:53.011 INFO:tasks.workunit.client.0.smithi032.stdout:1/20: read - d1/f6 zero size 2024-05-12T04:00:53.014 INFO:tasks.workunit.client.0.smithi032.stdout:1/21: mkdir d1/d7/d9 0 2024-05-12T04:00:53.018 INFO:tasks.workunit.client.0.smithi032.stdout:1/22: dread d1/f3 [0,4194304] 0 2024-05-12T04:00:53.021 INFO:tasks.workunit.client.0.smithi032.stdout:1/23: fdatasync d1/f3 0 2024-05-12T04:00:53.028 INFO:tasks.workunit.client.0.smithi032.stdout:1/24: rmdir d1/d7/d9 0 2024-05-12T04:00:53.029 INFO:tasks.workunit.client.0.smithi032.stdout:1/25: write d1/f6 [560224,51901] 0 2024-05-12T04:00:53.030 INFO:tasks.workunit.client.0.smithi032.stdout:5/54: truncate f1 83915 0 2024-05-12T04:00:53.031 INFO:tasks.workunit.client.0.smithi032.stdout:1/26: mknod d1/d7/ca 0 2024-05-12T04:00:53.034 INFO:tasks.workunit.client.0.smithi032.stdout:5/55: mkdir d4/d7/d14 0 2024-05-12T04:00:53.035 INFO:tasks.workunit.client.0.smithi032.stdout:5/56: rename d4 to d4/d7/d14/d15 22 2024-05-12T04:00:53.036 INFO:tasks.workunit.client.0.smithi032.stdout:3/38: truncate f3 3819591 0 2024-05-12T04:00:53.037 INFO:tasks.workunit.client.0.smithi032.stdout:3/39: chown ca 7 1 2024-05-12T04:00:53.045 INFO:tasks.workunit.client.0.smithi032.stdout:0/18: creat fa x:0 0 0 2024-05-12T04:00:53.047 INFO:tasks.workunit.client.0.smithi032.stdout:3/40: getdents . 0 2024-05-12T04:00:53.048 INFO:tasks.workunit.client.0.smithi032.stdout:3/41: write f6 [1459446,5201] 0 2024-05-12T04:00:53.058 INFO:tasks.workunit.client.0.smithi032.stdout:2/35: dread d1/fa [0,4194304] 0 2024-05-12T04:00:53.067 INFO:tasks.workunit.client.0.smithi032.stdout:2/36: chown d1 129952 1 2024-05-12T04:00:53.069 INFO:tasks.workunit.client.0.smithi032.stdout:0/19: creat fb x:0 0 0 2024-05-12T04:00:53.074 INFO:tasks.workunit.client.0.smithi032.stdout:0/20: chown c3 59565 1 2024-05-12T04:00:53.077 INFO:tasks.workunit.client.0.smithi032.stdout:0/21: dread f9 [0,4194304] 0 2024-05-12T04:00:53.078 INFO:tasks.workunit.client.0.smithi032.stdout:3/42: read f3 [1901974,58286] 0 2024-05-12T04:00:53.081 INFO:tasks.workunit.client.0.smithi032.stdout:2/37: creat d1/ff x:0 0 0 2024-05-12T04:00:53.085 INFO:tasks.workunit.client.0.smithi032.stdout:3/43: dwrite f7 [4194304,4194304] 0 2024-05-12T04:00:53.092 INFO:tasks.workunit.client.0.smithi032.stdout:0/22: mknod cc 0 2024-05-12T04:00:53.092 INFO:tasks.workunit.client.0.smithi032.stdout:0/23: rmdir - no directory 2024-05-12T04:00:53.094 INFO:tasks.workunit.client.0.smithi032.stdout:3/44: creat ff x:0 0 0 2024-05-12T04:00:53.095 INFO:tasks.workunit.client.0.smithi032.stdout:3/45: readlink l4 0 2024-05-12T04:00:53.096 INFO:tasks.workunit.client.0.smithi032.stdout:0/24: write f8 [2096332,37053] 0 2024-05-12T04:00:53.098 INFO:tasks.workunit.client.0.smithi032.stdout:0/25: write f9 [1155446,124913] 0 2024-05-12T04:00:53.104 INFO:tasks.workunit.client.0.smithi032.stdout:3/46: dwrite f7 [4194304,4194304] 0 2024-05-12T04:00:53.112 INFO:tasks.workunit.client.0.smithi032.stdout:2/38: dread - d1/ff zero size 2024-05-12T04:00:53.114 INFO:tasks.workunit.client.0.smithi032.stdout:5/57: dwrite f1 [0,4194304] 0 2024-05-12T04:00:53.117 INFO:tasks.workunit.client.0.smithi032.stdout:3/47: dwrite f6 [0,4194304] 0 2024-05-12T04:00:53.119 INFO:tasks.workunit.client.0.smithi032.stdout:3/48: write f7 [2054170,7889] 0 2024-05-12T04:00:53.121 INFO:tasks.workunit.client.0.smithi032.stdout:0/26: truncate f9 3373209 0 2024-05-12T04:00:53.126 INFO:tasks.workunit.client.0.smithi032.stdout:3/49: symlink l10 0 2024-05-12T04:00:53.127 INFO:tasks.workunit.client.0.smithi032.stdout:0/27: fsync f5 0 2024-05-12T04:00:53.127 INFO:tasks.workunit.client.0.smithi032.stdout:0/28: rmdir - no directory 2024-05-12T04:00:53.129 INFO:tasks.workunit.client.0.smithi032.stdout:5/58: getdents d4 0 2024-05-12T04:00:53.130 INFO:tasks.workunit.client.0.smithi032.stdout:3/50: unlink l10 0 2024-05-12T04:00:53.133 INFO:tasks.workunit.client.0.smithi032.stdout:5/59: rmdir d4/db 39 2024-05-12T04:00:53.133 INFO:tasks.workunit.client.0.smithi032.stdout:5/60: readlink d4/d7/lc 0 2024-05-12T04:00:53.134 INFO:tasks.workunit.client.0.smithi032.stdout:3/51: rename l9 to l11 0 2024-05-12T04:00:53.135 INFO:tasks.workunit.client.0.smithi032.stdout:0/29: creat fd x:0 0 0 2024-05-12T04:00:53.136 INFO:tasks.workunit.client.0.smithi032.stdout:3/52: creat f12 x:0 0 0 2024-05-12T04:00:53.139 INFO:tasks.workunit.client.0.smithi032.stdout:3/53: rename cb to c13 0 2024-05-12T04:00:53.140 INFO:tasks.workunit.client.0.smithi032.stdout:3/54: write f12 [798638,91022] 0 2024-05-12T04:00:53.141 INFO:tasks.workunit.client.0.smithi032.stdout:0/30: mknod ce 0 2024-05-12T04:00:53.145 INFO:tasks.workunit.client.0.smithi032.stdout:0/31: mkdir df 0 2024-05-12T04:00:53.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:52 smithi033 ceph-mon[49525]: pgmap v24: 65 pgs: 65 active+clean; 207 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s rd, 4.6 MiB/s wr, 418 op/s 2024-05-12T04:00:53.167 INFO:tasks.workunit.client.0.smithi032.stdout:3/55: truncate f7 2105200 0 2024-05-12T04:00:53.171 INFO:tasks.workunit.client.0.smithi032.stdout:0/32: dwrite f8 [0,4194304] 0 2024-05-12T04:00:53.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:52 smithi032 ceph-mon[40008]: pgmap v24: 65 pgs: 65 active+clean; 207 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s rd, 4.6 MiB/s wr, 418 op/s 2024-05-12T04:00:53.418 INFO:tasks.workunit.client.0.smithi032.stdout:0/33: mkdir df/d10 0 2024-05-12T04:00:53.422 INFO:tasks.workunit.client.0.smithi032.stdout:3/56: getdents . 0 2024-05-12T04:00:53.423 INFO:tasks.workunit.client.0.smithi032.stdout:3/57: dread - fd zero size 2024-05-12T04:00:53.423 INFO:tasks.workunit.client.0.smithi032.stdout:3/58: readlink l4 0 2024-05-12T04:00:53.423 INFO:tasks.workunit.client.0.smithi032.stdout:0/34: mknod df/d10/c11 0 2024-05-12T04:00:53.424 INFO:tasks.workunit.client.0.smithi032.stdout:3/59: fsync f8 0 2024-05-12T04:00:53.428 INFO:tasks.workunit.client.0.smithi032.stdout:0/35: creat df/f12 x:0 0 0 2024-05-12T04:00:53.430 INFO:tasks.workunit.client.0.smithi032.stdout:0/36: write fb [40211,16107] 0 2024-05-12T04:00:53.431 INFO:tasks.workunit.client.0.smithi032.stdout:3/60: creat f14 x:0 0 0 2024-05-12T04:00:53.434 INFO:tasks.workunit.client.0.smithi032.stdout:0/37: rename f8 to df/d10/f13 0 2024-05-12T04:00:53.436 INFO:tasks.workunit.client.0.smithi032.stdout:3/61: mkdir d15 0 2024-05-12T04:00:53.440 INFO:tasks.workunit.client.0.smithi032.stdout:0/38: dwrite fa [0,4194304] 0 2024-05-12T04:00:53.445 INFO:tasks.workunit.client.0.smithi032.stdout:3/62: symlink d15/l16 0 2024-05-12T04:00:53.445 INFO:tasks.workunit.client.0.smithi032.stdout:3/63: fsync fd 0 2024-05-12T04:00:53.457 INFO:tasks.workunit.client.0.smithi032.stdout:0/39: link ce df/d10/c14 0 2024-05-12T04:00:53.462 INFO:tasks.workunit.client.0.smithi032.stdout:0/40: symlink df/d10/l15 0 2024-05-12T04:00:53.462 INFO:tasks.workunit.client.0.smithi032.stdout:0/41: stat df/f12 0 2024-05-12T04:00:53.465 INFO:tasks.workunit.client.0.smithi032.stdout:0/42: rmdir df/d10 39 2024-05-12T04:00:53.591 INFO:tasks.workunit.client.0.smithi032.stdout:0/43: fdatasync df/d10/f13 0 2024-05-12T04:00:53.606 INFO:tasks.workunit.client.0.smithi032.stdout:3/64: truncate f12 4027 0 2024-05-12T04:00:53.609 INFO:tasks.workunit.client.0.smithi032.stdout:3/65: creat d15/f17 x:0 0 0 2024-05-12T04:00:53.609 INFO:tasks.workunit.client.0.smithi032.stdout:3/66: truncate fd 842370 0 2024-05-12T04:00:53.611 INFO:tasks.workunit.client.0.smithi032.stdout:0/44: getdents df/d10 0 2024-05-12T04:00:53.611 INFO:tasks.workunit.client.0.smithi032.stdout:0/45: read - df/f12 zero size 2024-05-12T04:00:53.617 INFO:tasks.workunit.client.0.smithi032.stdout:0/46: dread df/d10/f13 [0,4194304] 0 2024-05-12T04:00:53.617 INFO:tasks.workunit.client.0.smithi032.stdout:0/47: fdatasync fa 0 2024-05-12T04:00:53.617 INFO:tasks.workunit.client.0.smithi032.stdout:0/48: dread fb [0,4194304] 0 2024-05-12T04:00:53.622 INFO:tasks.workunit.client.0.smithi032.stdout:0/49: mknod df/c16 0 2024-05-12T04:00:53.622 INFO:tasks.workunit.client.0.smithi032.stdout:0/50: dread - df/f12 zero size 2024-05-12T04:00:53.626 INFO:tasks.workunit.client.0.smithi032.stdout:0/51: unlink df/f12 0 2024-05-12T04:00:53.628 INFO:tasks.workunit.client.0.smithi032.stdout:0/52: mknod df/d10/c17 0 2024-05-12T04:00:53.628 INFO:tasks.workunit.client.0.smithi032.stdout:0/53: dread - fd zero size 2024-05-12T04:00:53.630 INFO:tasks.workunit.client.0.smithi032.stdout:0/54: fdatasync fb 0 2024-05-12T04:00:53.631 INFO:tasks.workunit.client.0.smithi032.stdout:0/55: dread fb [0,4194304] 0 2024-05-12T04:00:53.632 INFO:tasks.workunit.client.0.smithi032.stdout:0/56: chown df/d10/c17 0 1 2024-05-12T04:00:53.633 INFO:tasks.workunit.client.0.smithi032.stdout:0/57: write fd [142432,13626] 0 2024-05-12T04:00:53.638 INFO:tasks.workunit.client.0.smithi032.stdout:0/58: mknod df/c18 0 2024-05-12T04:00:53.641 INFO:tasks.workunit.client.0.smithi032.stdout:0/59: creat df/d10/f19 x:0 0 0 2024-05-12T04:00:53.641 INFO:tasks.workunit.client.0.smithi032.stdout:0/60: fdatasync f9 0 2024-05-12T04:00:53.661 INFO:tasks.workunit.client.0.smithi032.stdout:3/67: fsync f12 0 2024-05-12T04:00:53.664 INFO:tasks.workunit.client.0.smithi032.stdout:3/68: unlink ff 0 2024-05-12T04:00:53.669 INFO:tasks.workunit.client.0.smithi032.stdout:3/69: rename f8 to d15/f18 0 2024-05-12T04:00:53.674 INFO:tasks.workunit.client.0.smithi032.stdout:3/70: dread f7 [0,4194304] 0 2024-05-12T04:00:53.712 INFO:tasks.workunit.client.0.smithi032.stdout:0/61: getdents df 0 2024-05-12T04:00:53.715 INFO:tasks.workunit.client.0.smithi032.stdout:0/62: mkdir df/d10/d1a 0 2024-05-12T04:00:53.717 INFO:tasks.workunit.client.0.smithi032.stdout:0/63: creat df/d10/f1b x:0 0 0 2024-05-12T04:00:53.721 INFO:tasks.workunit.client.0.smithi032.stdout:0/64: rename df/c16 to df/d10/c1c 0 2024-05-12T04:00:53.765 INFO:tasks.workunit.client.0.smithi032.stdout:0/65: symlink df/l1d 0 2024-05-12T04:00:53.769 INFO:tasks.workunit.client.0.smithi032.stdout:0/66: dread fa [0,4194304] 0 2024-05-12T04:00:53.776 INFO:tasks.workunit.client.0.smithi032.stdout:0/67: rmdir df/d10/d1a 0 2024-05-12T04:00:53.779 INFO:tasks.workunit.client.0.smithi032.stdout:0/68: mknod df/d10/c1e 0 2024-05-12T04:00:53.780 INFO:tasks.workunit.client.0.smithi032.stdout:0/69: write fb [341879,56964] 0 2024-05-12T04:00:53.784 INFO:tasks.workunit.client.0.smithi032.stdout:0/70: dread f5 [0,4194304] 0 2024-05-12T04:00:53.785 INFO:tasks.workunit.client.0.smithi032.stdout:0/71: dread - df/d10/f1b zero size 2024-05-12T04:00:53.786 INFO:tasks.workunit.client.0.smithi032.stdout:0/72: truncate df/d10/f1b 817103 0 2024-05-12T04:00:53.787 INFO:tasks.workunit.client.0.smithi032.stdout:0/73: write f9 [1562158,70482] 0 2024-05-12T04:00:53.788 INFO:tasks.workunit.client.0.smithi032.stdout:0/74: write f5 [4483792,116263] 0 2024-05-12T04:00:53.791 INFO:tasks.workunit.client.0.smithi032.stdout:0/75: dread fb [0,4194304] 0 2024-05-12T04:00:53.793 INFO:tasks.workunit.client.0.smithi032.stdout:0/76: rmdir df 39 2024-05-12T04:00:53.796 INFO:tasks.workunit.client.0.smithi032.stdout:0/77: mkdir df/d10/d1f 0 2024-05-12T04:00:53.797 INFO:tasks.workunit.client.0.smithi032.stdout:0/78: write f9 [3608530,100728] 0 2024-05-12T04:00:53.799 INFO:tasks.workunit.client.0.smithi032.stdout:0/79: stat l4 0 2024-05-12T04:00:53.800 INFO:tasks.workunit.client.0.smithi032.stdout:0/80: write df/d10/f1b [1777604,51616] 0 2024-05-12T04:00:53.803 INFO:tasks.workunit.client.0.smithi032.stdout:0/81: mknod df/c20 0 2024-05-12T04:00:53.803 INFO:tasks.workunit.client.0.smithi032.stdout:0/82: read f5 [2022272,36042] 0 2024-05-12T04:00:53.806 INFO:tasks.workunit.client.0.smithi032.stdout:0/83: mknod df/d10/c21 0 2024-05-12T04:00:53.814 INFO:tasks.workunit.client.0.smithi032.stdout:0/84: dwrite f9 [4194304,4194304] 0 2024-05-12T04:00:53.818 INFO:tasks.workunit.client.0.smithi032.stdout:0/85: mkdir df/d22 0 2024-05-12T04:00:53.818 INFO:tasks.workunit.client.0.smithi032.stdout:0/86: chown df/d10/c11 2 1 2024-05-12T04:00:53.822 INFO:tasks.workunit.client.0.smithi032.stdout:0/87: symlink df/d10/d1f/l23 0 2024-05-12T04:00:53.825 INFO:tasks.workunit.client.0.smithi032.stdout:0/88: symlink df/l24 0 2024-05-12T04:00:53.827 INFO:tasks.workunit.client.0.smithi032.stdout:0/89: getdents df/d22 0 2024-05-12T04:00:53.850 INFO:tasks.workunit.client.0.smithi032.stdout:0/90: truncate df/d10/f13 5385438 0 2024-05-12T04:00:55.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:54 smithi033 ceph-mon[49525]: pgmap v25: 65 pgs: 65 active+clean; 207 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 266 op/s 2024-05-12T04:00:55.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:54 smithi032 ceph-mon[40008]: pgmap v25: 65 pgs: 65 active+clean; 207 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 266 op/s 2024-05-12T04:00:55.499 INFO:tasks.workunit.client.1.smithi033.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-05-12T04:00:55.503 INFO:tasks.workunit.client.1.smithi033.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-05-12T04:00:55.504 INFO:tasks.workunit.client.1.smithi033.stderr:+ make 2024-05-12T04:00:55.816 INFO:tasks.workunit.client.1.smithi033.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-05-12T04:00:56.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:55 smithi033 ceph-mon[49525]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:00:56.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:55 smithi032 ceph-mon[40008]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:00:57.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:56 smithi033 ceph-mon[49525]: pgmap v26: 65 pgs: 65 active+clean; 232 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 7.0 MiB/s wr, 289 op/s 2024-05-12T04:00:57.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:56 smithi032 ceph-mon[40008]: pgmap v26: 65 pgs: 65 active+clean; 232 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 7.0 MiB/s wr, 289 op/s 2024-05-12T04:00:58.159 INFO:tasks.workunit.client.1.smithi033.stderr:++ readlink -f fsstress 2024-05-12T04:00:58.161 INFO:tasks.workunit.client.1.smithi033.stderr:+ BIN=/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-05-12T04:00:58.161 INFO:tasks.workunit.client.1.smithi033.stderr:+ popd 2024-05-12T04:00:58.162 INFO:tasks.workunit.client.1.smithi033.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-05-12T04:00:58.162 INFO:tasks.workunit.client.1.smithi033.stderr:+ popd 2024-05-12T04:00:58.163 INFO:tasks.workunit.client.1.smithi033.stdout:~/cephtest/mnt.1/client.1/tmp 2024-05-12T04:00:58.163 INFO:tasks.workunit.client.1.smithi033.stderr:++ mktemp -d -p . 2024-05-12T04:00:58.166 INFO:tasks.workunit.client.1.smithi033.stderr:+ T=./tmp.TCxXXFL3v4 2024-05-12T04:00:58.166 INFO:tasks.workunit.client.1.smithi033.stderr:+ /home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.TCxXXFL3v4 -l 1 -n 1000 -p 10 -v 2024-05-12T04:00:58.170 INFO:tasks.workunit.client.1.smithi033.stdout:seed = 1715643653 2024-05-12T04:00:58.173 INFO:tasks.workunit.client.1.smithi033.stdout:0/0: write - no filename 2024-05-12T04:00:58.175 INFO:tasks.workunit.client.1.smithi033.stdout:1/0: chown . 2145959719 1 2024-05-12T04:00:58.175 INFO:tasks.workunit.client.1.smithi033.stdout:0/1: symlink l0 0 2024-05-12T04:00:58.175 INFO:tasks.workunit.client.1.smithi033.stdout:0/2: write - no filename 2024-05-12T04:00:58.193 INFO:tasks.workunit.client.1.smithi033.stdout:2/0: dread - no filename 2024-05-12T04:00:58.193 INFO:tasks.workunit.client.1.smithi033.stdout:2/1: dwrite - no filename 2024-05-12T04:00:58.193 INFO:tasks.workunit.client.1.smithi033.stdout:2/2: stat - no entries 2024-05-12T04:00:58.193 INFO:tasks.workunit.client.1.smithi033.stdout:2/3: fdatasync - no filename 2024-05-12T04:00:58.194 INFO:tasks.workunit.client.1.smithi033.stdout:0/3: creat f1 x:0 0 0 2024-05-12T04:00:58.194 INFO:tasks.workunit.client.1.smithi033.stdout:0/4: dread - f1 zero size 2024-05-12T04:00:58.195 INFO:tasks.workunit.client.1.smithi033.stdout:0/5: write f1 [654461,77323] 0 2024-05-12T04:00:58.198 INFO:tasks.workunit.client.1.smithi033.stdout:1/1: mknod c0 0 2024-05-12T04:00:58.198 INFO:tasks.workunit.client.1.smithi033.stdout:1/2: dread - no filename 2024-05-12T04:00:58.198 INFO:tasks.workunit.client.1.smithi033.stdout:1/3: write - no filename 2024-05-12T04:00:58.200 INFO:tasks.workunit.client.1.smithi033.stdout:0/6: unlink l0 0 2024-05-12T04:00:58.202 INFO:tasks.workunit.client.1.smithi033.stdout:4/0: dread - no filename 2024-05-12T04:00:58.203 INFO:tasks.workunit.client.1.smithi033.stdout:2/4: mknod c0 0 2024-05-12T04:00:58.207 INFO:tasks.workunit.client.1.smithi033.stdout:5/0: rename - no filename 2024-05-12T04:00:58.208 INFO:tasks.workunit.client.1.smithi033.stdout:3/0: mknod c0 0 2024-05-12T04:00:58.209 INFO:tasks.workunit.client.1.smithi033.stdout:0/7: dwrite f1 [0,4194304] 0 2024-05-12T04:00:58.209 INFO:tasks.workunit.client.1.smithi033.stdout:2/5: creat f1 x:0 0 0 2024-05-12T04:00:58.210 INFO:tasks.workunit.client.1.smithi033.stdout:0/8: chown f1 1 1 2024-05-12T04:00:58.212 INFO:tasks.workunit.client.1.smithi033.stdout:4/1: symlink l0 0 2024-05-12T04:00:58.212 INFO:tasks.workunit.client.1.smithi033.stdout:4/2: write - no filename 2024-05-12T04:00:58.212 INFO:tasks.workunit.client.1.smithi033.stdout:4/3: truncate - no filename 2024-05-12T04:00:58.212 INFO:tasks.workunit.client.1.smithi033.stdout:6/0: dwrite - no filename 2024-05-12T04:00:58.213 INFO:tasks.workunit.client.1.smithi033.stdout:3/1: rename c0 to c1 0 2024-05-12T04:00:58.213 INFO:tasks.workunit.client.1.smithi033.stdout:4/4: chown l0 38309922 1 2024-05-12T04:00:58.213 INFO:tasks.workunit.client.1.smithi033.stdout:4/5: dread - no filename 2024-05-12T04:00:58.213 INFO:tasks.workunit.client.1.smithi033.stdout:4/6: truncate - no filename 2024-05-12T04:00:58.213 INFO:tasks.workunit.client.1.smithi033.stdout:4/7: dread - no filename 2024-05-12T04:00:58.213 INFO:tasks.workunit.client.1.smithi033.stdout:4/8: write - no filename 2024-05-12T04:00:58.213 INFO:tasks.workunit.client.1.smithi033.stdout:4/9: dwrite - no filename 2024-05-12T04:00:58.218 INFO:tasks.workunit.client.1.smithi033.stdout:2/6: symlink l2 0 2024-05-12T04:00:58.218 INFO:tasks.workunit.client.1.smithi033.stdout:3/2: symlink l2 0 2024-05-12T04:00:58.219 INFO:tasks.workunit.client.1.smithi033.stdout:3/3: fdatasync - no filename 2024-05-12T04:00:58.219 INFO:tasks.workunit.client.1.smithi033.stdout:4/10: mknod c1 0 2024-05-12T04:00:58.220 INFO:tasks.workunit.client.1.smithi033.stdout:6/1: mknod c0 0 2024-05-12T04:00:58.220 INFO:tasks.workunit.client.1.smithi033.stdout:6/2: dwrite - no filename 2024-05-12T04:00:58.220 INFO:tasks.workunit.client.1.smithi033.stdout:6/3: write - no filename 2024-05-12T04:00:58.220 INFO:tasks.workunit.client.1.smithi033.stdout:7/0: write - no filename 2024-05-12T04:00:58.220 INFO:tasks.workunit.client.1.smithi033.stdout:7/1: readlink - no filename 2024-05-12T04:00:58.220 INFO:tasks.workunit.client.1.smithi033.stdout:7/2: dread - no filename 2024-05-12T04:00:58.221 INFO:tasks.workunit.client.1.smithi033.stdout:7/3: truncate - no filename 2024-05-12T04:00:58.221 INFO:tasks.workunit.client.1.smithi033.stdout:3/4: symlink l3 0 2024-05-12T04:00:58.221 INFO:tasks.workunit.client.1.smithi033.stdout:0/9: dread f1 [0,4194304] 0 2024-05-12T04:00:58.222 INFO:tasks.workunit.client.1.smithi033.stdout:3/5: chown l2 84660 1 2024-05-12T04:00:58.222 INFO:tasks.workunit.client.1.smithi033.stdout:3/6: write - no filename 2024-05-12T04:00:58.222 INFO:tasks.workunit.client.1.smithi033.stdout:8/0: write - no filename 2024-05-12T04:00:58.222 INFO:tasks.workunit.client.1.smithi033.stdout:2/7: creat f3 x:0 0 0 2024-05-12T04:00:58.224 INFO:tasks.workunit.client.1.smithi033.stdout:4/11: mknod c2 0 2024-05-12T04:00:58.224 INFO:tasks.workunit.client.1.smithi033.stdout:4/12: truncate - no filename 2024-05-12T04:00:58.224 INFO:tasks.workunit.client.1.smithi033.stdout:4/13: fsync - no filename 2024-05-12T04:00:58.224 INFO:tasks.workunit.client.1.smithi033.stdout:4/14: write - no filename 2024-05-12T04:00:58.224 INFO:tasks.workunit.client.1.smithi033.stdout:4/15: truncate - no filename 2024-05-12T04:00:58.224 INFO:tasks.workunit.client.1.smithi033.stdout:4/16: fdatasync - no filename 2024-05-12T04:00:58.224 INFO:tasks.workunit.client.1.smithi033.stdout:4/17: write - no filename 2024-05-12T04:00:58.225 INFO:tasks.workunit.client.1.smithi033.stdout:4/18: chown l0 2550508 1 2024-05-12T04:00:58.225 INFO:tasks.workunit.client.1.smithi033.stdout:4/19: dwrite - no filename 2024-05-12T04:00:58.228 INFO:tasks.workunit.client.1.smithi033.stdout:3/7: rename l2 to l4 0 2024-05-12T04:00:58.230 INFO:tasks.workunit.client.1.smithi033.stdout:7/4: symlink l0 0 2024-05-12T04:00:58.230 INFO:tasks.workunit.client.1.smithi033.stdout:7/5: rmdir - no directory 2024-05-12T04:00:58.231 INFO:tasks.workunit.client.1.smithi033.stdout:4/20: creat f3 x:0 0 0 2024-05-12T04:00:58.232 INFO:tasks.workunit.client.1.smithi033.stdout:4/21: chown c2 12 1 2024-05-12T04:00:58.233 INFO:tasks.workunit.client.1.smithi033.stdout:3/8: symlink l5 0 2024-05-12T04:00:58.234 INFO:tasks.workunit.client.1.smithi033.stdout:8/1: symlink l0 0 2024-05-12T04:00:58.234 INFO:tasks.workunit.client.1.smithi033.stdout:8/2: fdatasync - no filename 2024-05-12T04:00:58.234 INFO:tasks.workunit.client.1.smithi033.stdout:8/3: rmdir - no directory 2024-05-12T04:00:58.235 INFO:tasks.workunit.client.1.smithi033.stdout:8/4: chown l0 33868510 1 2024-05-12T04:00:58.235 INFO:tasks.workunit.client.1.smithi033.stdout:8/5: truncate - no filename 2024-05-12T04:00:58.237 INFO:tasks.workunit.client.1.smithi033.stdout:7/6: symlink l1 0 2024-05-12T04:00:58.237 INFO:tasks.workunit.client.1.smithi033.stdout:7/7: read - no filename 2024-05-12T04:00:58.238 INFO:tasks.workunit.client.1.smithi033.stdout:9/0: creat f0 x:0 0 0 2024-05-12T04:00:58.238 INFO:tasks.workunit.client.1.smithi033.stdout:9/1: chown f0 7048606 1 2024-05-12T04:00:58.239 INFO:tasks.workunit.client.1.smithi033.stdout:4/22: symlink l4 0 2024-05-12T04:00:58.240 INFO:tasks.workunit.client.1.smithi033.stdout:3/9: creat f6 x:0 0 0 2024-05-12T04:00:58.241 INFO:tasks.workunit.client.1.smithi033.stdout:3/10: dread - f6 zero size 2024-05-12T04:00:58.242 INFO:tasks.workunit.client.1.smithi033.stdout:8/6: mkdir d1 0 2024-05-12T04:00:58.242 INFO:tasks.workunit.client.1.smithi033.stdout:8/7: write - no filename 2024-05-12T04:00:58.242 INFO:tasks.workunit.client.1.smithi033.stdout:8/8: dread - no filename 2024-05-12T04:00:58.242 INFO:tasks.workunit.client.1.smithi033.stdout:8/9: write - no filename 2024-05-12T04:00:58.243 INFO:tasks.workunit.client.1.smithi033.stdout:2/8: link l2 l4 0 2024-05-12T04:00:58.243 INFO:tasks.workunit.client.1.smithi033.stdout:7/8: mknod c2 0 2024-05-12T04:00:58.244 INFO:tasks.workunit.client.1.smithi033.stdout:9/2: mknod c1 0 2024-05-12T04:00:58.244 INFO:tasks.workunit.client.1.smithi033.stdout:3/11: write f6 [9678,72105] 0 2024-05-12T04:00:58.245 INFO:tasks.workunit.client.1.smithi033.stdout:2/9: write f3 [1035399,95497] 0 2024-05-12T04:00:58.247 INFO:tasks.workunit.client.1.smithi033.stdout:7/9: creat f3 x:0 0 0 2024-05-12T04:00:58.247 INFO:tasks.workunit.client.1.smithi033.stdout:9/3: chown c1 1 1 2024-05-12T04:00:58.248 INFO:tasks.workunit.client.1.smithi033.stdout:2/10: stat f1 0 2024-05-12T04:00:58.248 INFO:tasks.workunit.client.1.smithi033.stdout:8/10: creat d1/f2 x:0 0 0 2024-05-12T04:00:58.250 INFO:tasks.workunit.client.1.smithi033.stdout:3/12: symlink l7 0 2024-05-12T04:00:58.251 INFO:tasks.workunit.client.1.smithi033.stdout:7/10: creat f4 x:0 0 0 2024-05-12T04:00:58.251 INFO:tasks.workunit.client.1.smithi033.stdout:4/23: dwrite f3 [0,4194304] 0 2024-05-12T04:00:58.251 INFO:tasks.workunit.client.1.smithi033.stdout:9/4: mkdir d2 0 2024-05-12T04:00:58.252 INFO:tasks.workunit.client.1.smithi033.stdout:3/13: write f6 [201472,19136] 0 2024-05-12T04:00:58.253 INFO:tasks.workunit.client.1.smithi033.stdout:2/11: mknod c5 0 2024-05-12T04:00:58.254 INFO:tasks.workunit.client.1.smithi033.stdout:4/24: write f3 [3066362,49138] 0 2024-05-12T04:00:58.255 INFO:tasks.workunit.client.1.smithi033.stdout:2/12: fdatasync f3 0 2024-05-12T04:00:58.255 INFO:tasks.workunit.client.1.smithi033.stdout:4/25: stat l4 0 2024-05-12T04:00:58.256 INFO:tasks.workunit.client.1.smithi033.stdout:7/11: rename l1 to l5 0 2024-05-12T04:00:58.256 INFO:tasks.workunit.client.1.smithi033.stdout:2/13: dread - f1 zero size 2024-05-12T04:00:58.260 INFO:tasks.workunit.client.1.smithi033.stdout:4/26: creat f5 x:0 0 0 2024-05-12T04:00:58.262 INFO:tasks.workunit.client.1.smithi033.stdout:2/14: creat f6 x:0 0 0 2024-05-12T04:00:58.262 INFO:tasks.workunit.client.1.smithi033.stdout:4/27: write f3 [3960321,37880] 0 2024-05-12T04:00:58.263 INFO:tasks.workunit.client.1.smithi033.stdout:9/5: symlink d2/l3 0 2024-05-12T04:00:58.265 INFO:tasks.workunit.client.1.smithi033.stdout:4/28: write f5 [833513,108636] 0 2024-05-12T04:00:58.265 INFO:tasks.workunit.client.1.smithi033.stdout:3/14: dread f6 [0,4194304] 0 2024-05-12T04:00:58.266 INFO:tasks.workunit.client.1.smithi033.stdout:7/12: dwrite f3 [0,4194304] 0 2024-05-12T04:00:58.269 INFO:tasks.workunit.client.1.smithi033.stdout:2/15: creat f7 x:0 0 0 2024-05-12T04:00:58.269 INFO:tasks.workunit.client.1.smithi033.stdout:9/6: mkdir d2/d4 0 2024-05-12T04:00:58.269 INFO:tasks.workunit.client.1.smithi033.stdout:9/7: stat f0 0 2024-05-12T04:00:58.270 INFO:tasks.workunit.client.1.smithi033.stdout:9/8: read - f0 zero size 2024-05-12T04:00:58.272 INFO:tasks.workunit.client.1.smithi033.stdout:3/15: fdatasync f6 0 2024-05-12T04:00:58.274 INFO:tasks.workunit.client.1.smithi033.stdout:7/13: mknod c6 0 2024-05-12T04:00:58.274 INFO:tasks.workunit.client.1.smithi033.stdout:2/16: symlink l8 0 2024-05-12T04:00:58.275 INFO:tasks.workunit.client.1.smithi033.stdout:9/9: rmdir d2 39 2024-05-12T04:00:58.276 INFO:tasks.workunit.client.1.smithi033.stdout:9/10: write f0 [827969,80022] 0 2024-05-12T04:00:58.276 INFO:tasks.workunit.client.1.smithi033.stdout:3/16: mknod c8 0 2024-05-12T04:00:58.278 INFO:tasks.workunit.client.1.smithi033.stdout:7/14: unlink f4 0 2024-05-12T04:00:58.280 INFO:tasks.workunit.client.1.smithi033.stdout:2/17: creat f9 x:0 0 0 2024-05-12T04:00:58.281 INFO:tasks.workunit.client.1.smithi033.stdout:2/18: write f7 [66698,87479] 0 2024-05-12T04:00:58.283 INFO:tasks.workunit.client.1.smithi033.stdout:7/15: creat f7 x:0 0 0 2024-05-12T04:00:58.283 INFO:tasks.workunit.client.1.smithi033.stdout:9/11: dread f0 [0,4194304] 0 2024-05-12T04:00:58.284 INFO:tasks.workunit.client.1.smithi033.stdout:7/16: chown c2 416987 1 2024-05-12T04:00:58.300 INFO:tasks.workunit.client.1.smithi033.stdout:7/17: dwrite f7 [0,4194304] 0 2024-05-12T04:00:58.371 INFO:tasks.workunit.client.1.smithi033.stdout:0/10: fsync f1 0 2024-05-12T04:00:58.376 INFO:tasks.workunit.client.1.smithi033.stdout:0/11: dwrite f1 [0,4194304] 0 2024-05-12T04:00:58.381 INFO:tasks.workunit.client.1.smithi033.stdout:0/12: dread f1 [0,4194304] 0 2024-05-12T04:00:58.387 INFO:tasks.workunit.client.1.smithi033.stdout:0/13: dwrite f1 [0,4194304] 0 2024-05-12T04:00:58.391 INFO:tasks.workunit.client.1.smithi033.stdout:2/19: fsync f1 0 2024-05-12T04:00:58.394 INFO:tasks.workunit.client.1.smithi033.stdout:2/20: write f6 [557554,82858] 0 2024-05-12T04:00:58.396 INFO:tasks.workunit.client.1.smithi033.stdout:0/14: dwrite f1 [0,4194304] 0 2024-05-12T04:00:58.402 INFO:tasks.workunit.client.1.smithi033.stdout:6/4: getdents . 0 2024-05-12T04:00:58.422 INFO:tasks.workunit.client.1.smithi033.stdout:8/11: rmdir d1 39 2024-05-12T04:00:58.429 INFO:tasks.workunit.client.1.smithi033.stdout:7/18: symlink l8 0 2024-05-12T04:00:58.432 INFO:tasks.workunit.client.1.smithi033.stdout:2/21: mknod ca 0 2024-05-12T04:00:58.432 INFO:tasks.workunit.client.1.smithi033.stdout:4/29: fsync f5 0 2024-05-12T04:00:58.432 INFO:tasks.workunit.client.1.smithi033.stdout:2/22: stat f3 0 2024-05-12T04:00:58.434 INFO:tasks.workunit.client.1.smithi033.stdout:6/5: unlink c0 0 2024-05-12T04:00:58.434 INFO:tasks.workunit.client.1.smithi033.stdout:6/6: rmdir - no directory 2024-05-12T04:00:58.434 INFO:tasks.workunit.client.1.smithi033.stdout:6/7: write - no filename 2024-05-12T04:00:58.435 INFO:tasks.workunit.client.1.smithi033.stdout:6/8: write - no filename 2024-05-12T04:00:58.435 INFO:tasks.workunit.client.1.smithi033.stdout:6/9: write - no filename 2024-05-12T04:00:58.436 INFO:tasks.workunit.client.1.smithi033.stdout:3/17: rename l4 to l9 0 2024-05-12T04:00:58.436 INFO:tasks.workunit.client.1.smithi033.stdout:3/18: readlink l3 0 2024-05-12T04:00:58.437 INFO:tasks.workunit.client.1.smithi033.stdout:8/12: truncate d1/f2 1013212 0 2024-05-12T04:00:58.439 INFO:tasks.workunit.client.1.smithi033.stdout:7/19: rename c2 to c9 0 2024-05-12T04:00:58.441 INFO:tasks.workunit.client.1.smithi033.stdout:2/23: symlink lb 0 2024-05-12T04:00:58.443 INFO:tasks.workunit.client.1.smithi033.stdout:0/15: rename f1 to f2 0 2024-05-12T04:00:58.443 INFO:tasks.workunit.client.1.smithi033.stdout:3/19: dwrite f6 [0,4194304] 0 2024-05-12T04:00:58.444 INFO:tasks.workunit.client.1.smithi033.stdout:9/12: mkdir d2/d4/d5 0 2024-05-12T04:00:58.446 INFO:tasks.workunit.client.1.smithi033.stdout:7/20: symlink la 0 2024-05-12T04:00:58.448 INFO:tasks.workunit.client.1.smithi033.stdout:2/24: mkdir dc 0 2024-05-12T04:00:58.451 INFO:tasks.workunit.client.1.smithi033.stdout:3/20: rename l3 to la 0 2024-05-12T04:00:58.454 INFO:tasks.workunit.client.1.smithi033.stdout:9/13: symlink d2/d4/l6 0 2024-05-12T04:00:58.455 INFO:tasks.workunit.client.1.smithi033.stdout:7/21: mknod cb 0 2024-05-12T04:00:58.456 INFO:tasks.workunit.client.1.smithi033.stdout:9/14: rename d2/d4 to d2/d4/d7 22 2024-05-12T04:00:58.458 INFO:tasks.workunit.client.1.smithi033.stdout:0/16: dwrite f2 [0,4194304] 0 2024-05-12T04:00:58.459 INFO:tasks.workunit.client.1.smithi033.stdout:3/21: dread f6 [0,4194304] 0 2024-05-12T04:00:58.460 INFO:tasks.workunit.client.1.smithi033.stdout:2/25: creat dc/fd x:0 0 0 2024-05-12T04:00:58.461 INFO:tasks.workunit.client.1.smithi033.stdout:7/22: rename l8 to lc 0 2024-05-12T04:00:58.461 INFO:tasks.workunit.client.1.smithi033.stdout:2/26: dread - f1 zero size 2024-05-12T04:00:58.462 INFO:tasks.workunit.client.1.smithi033.stdout:8/13: getdents d1 0 2024-05-12T04:00:58.464 INFO:tasks.workunit.client.1.smithi033.stdout:9/15: creat d2/d4/f8 x:0 0 0 2024-05-12T04:00:58.464 INFO:tasks.workunit.client.1.smithi033.stdout:0/17: symlink l3 0 2024-05-12T04:00:58.465 INFO:tasks.workunit.client.1.smithi033.stdout:8/14: dread d1/f2 [0,4194304] 0 2024-05-12T04:00:58.465 INFO:tasks.workunit.client.1.smithi033.stdout:3/22: fdatasync f6 0 2024-05-12T04:00:58.471 INFO:tasks.workunit.client.1.smithi033.stdout:9/16: unlink d2/d4/f8 0 2024-05-12T04:00:58.472 INFO:tasks.workunit.client.1.smithi033.stdout:0/18: creat f4 x:0 0 0 2024-05-12T04:00:58.472 INFO:tasks.workunit.client.1.smithi033.stdout:0/19: rmdir - no directory 2024-05-12T04:00:58.473 INFO:tasks.workunit.client.1.smithi033.stdout:8/15: symlink d1/l3 0 2024-05-12T04:00:58.474 INFO:tasks.workunit.client.1.smithi033.stdout:3/23: mkdir db 0 2024-05-12T04:00:58.478 INFO:tasks.workunit.client.1.smithi033.stdout:9/17: write f0 [1457852,45308] 0 2024-05-12T04:00:58.478 INFO:tasks.workunit.client.1.smithi033.stdout:8/16: mkdir d1/d4 0 2024-05-12T04:00:58.480 INFO:tasks.workunit.client.1.smithi033.stdout:2/27: link ca dc/ce 0 2024-05-12T04:00:58.481 INFO:tasks.workunit.client.1.smithi033.stdout:2/28: truncate f6 1162468 0 2024-05-12T04:00:58.481 INFO:tasks.workunit.client.1.smithi033.stdout:8/17: getdents d1/d4 0 2024-05-12T04:00:58.481 INFO:tasks.workunit.client.1.smithi033.stdout:8/18: chown d1/l3 16168204 1 2024-05-12T04:00:58.482 INFO:tasks.workunit.client.1.smithi033.stdout:2/29: mknod dc/cf 0 2024-05-12T04:00:58.485 INFO:tasks.workunit.client.1.smithi033.stdout:2/30: link f6 dc/f10 0 2024-05-12T04:00:58.487 INFO:tasks.workunit.client.1.smithi033.stdout:2/31: unlink dc/fd 0 2024-05-12T04:00:58.488 INFO:tasks.workunit.client.1.smithi033.stdout:2/32: symlink dc/l11 0 2024-05-12T04:00:58.490 INFO:tasks.workunit.client.1.smithi033.stdout:2/33: symlink dc/l12 0 2024-05-12T04:00:58.491 INFO:tasks.workunit.client.1.smithi033.stdout:2/34: mkdir dc/d13 0 2024-05-12T04:00:58.492 INFO:tasks.workunit.client.1.smithi033.stdout:2/35: dread - f1 zero size 2024-05-12T04:00:58.498 INFO:tasks.workunit.client.1.smithi033.stdout:2/36: dread dc/f10 [0,4194304] 0 2024-05-12T04:00:58.498 INFO:tasks.workunit.client.1.smithi033.stdout:2/37: readlink l2 0 2024-05-12T04:00:58.500 INFO:tasks.workunit.client.1.smithi033.stdout:2/38: creat dc/d13/f14 x:0 0 0 2024-05-12T04:00:58.500 INFO:tasks.workunit.client.1.smithi033.stdout:2/39: chown f3 76793355 1 2024-05-12T04:00:58.501 INFO:tasks.workunit.client.1.smithi033.stdout:2/40: creat dc/d13/f15 x:0 0 0 2024-05-12T04:00:58.502 INFO:tasks.workunit.client.1.smithi033.stdout:2/41: truncate dc/d13/f14 730802 0 2024-05-12T04:00:58.503 INFO:tasks.workunit.client.1.smithi033.stdout:2/42: dread - f9 zero size 2024-05-12T04:00:58.523 INFO:tasks.workunit.client.1.smithi033.stdout:7/23: write f7 [4711448,117937] 0 2024-05-12T04:00:58.524 INFO:tasks.workunit.client.1.smithi033.stdout:7/24: write f3 [3637471,130332] 0 2024-05-12T04:00:58.534 INFO:tasks.workunit.client.1.smithi033.stdout:3/24: rename la to db/lc 0 2024-05-12T04:00:58.535 INFO:tasks.workunit.client.1.smithi033.stdout:9/18: rename d2 to d2/d9 22 2024-05-12T04:00:58.536 INFO:tasks.workunit.client.1.smithi033.stdout:3/25: write f6 [4264703,5164] 0 2024-05-12T04:00:58.536 INFO:tasks.workunit.client.1.smithi033.stdout:7/25: getdents . 0 2024-05-12T04:00:58.538 INFO:tasks.workunit.client.1.smithi033.stdout:7/26: fdatasync f3 0 2024-05-12T04:00:58.538 INFO:tasks.workunit.client.1.smithi033.stdout:8/19: rename d1/l3 to d1/d4/l5 0 2024-05-12T04:00:58.542 INFO:tasks.workunit.client.1.smithi033.stdout:7/27: mknod cd 0 2024-05-12T04:00:58.542 INFO:tasks.workunit.client.1.smithi033.stdout:0/20: fsync f4 0 2024-05-12T04:00:58.543 INFO:tasks.workunit.client.1.smithi033.stdout:9/19: dwrite f0 [0,4194304] 0 2024-05-12T04:00:58.544 INFO:tasks.workunit.client.1.smithi033.stdout:0/21: write f2 [4136926,26868] 0 2024-05-12T04:00:58.547 INFO:tasks.workunit.client.1.smithi033.stdout:7/28: mkdir de 0 2024-05-12T04:00:58.548 INFO:tasks.workunit.client.1.smithi033.stdout:0/22: creat f5 x:0 0 0 2024-05-12T04:00:58.549 INFO:tasks.workunit.client.1.smithi033.stdout:0/23: truncate f4 926773 0 2024-05-12T04:00:58.555 INFO:tasks.workunit.client.1.smithi033.stdout:0/24: mkdir d6 0 2024-05-12T04:00:58.556 INFO:tasks.workunit.client.1.smithi033.stdout:0/25: chown f5 0 1 2024-05-12T04:00:58.556 INFO:tasks.workunit.client.1.smithi033.stdout:0/26: dread - f5 zero size 2024-05-12T04:00:58.557 INFO:tasks.workunit.client.1.smithi033.stdout:7/29: getdents de 0 2024-05-12T04:00:58.557 INFO:tasks.workunit.client.1.smithi033.stdout:0/27: readlink l3 0 2024-05-12T04:00:58.558 INFO:tasks.workunit.client.1.smithi033.stdout:7/30: write f7 [3491707,117578] 0 2024-05-12T04:00:58.558 INFO:tasks.workunit.client.1.smithi033.stdout:2/43: truncate f6 749861 0 2024-05-12T04:00:58.559 INFO:tasks.workunit.client.1.smithi033.stdout:7/31: write f3 [3225665,22904] 0 2024-05-12T04:00:58.560 INFO:tasks.workunit.client.1.smithi033.stdout:0/28: creat d6/f7 x:0 0 0 2024-05-12T04:00:58.561 INFO:tasks.workunit.client.1.smithi033.stdout:0/29: chown f5 53 1 2024-05-12T04:00:58.562 INFO:tasks.workunit.client.1.smithi033.stdout:0/30: chown f4 0 1 2024-05-12T04:00:58.564 INFO:tasks.workunit.client.1.smithi033.stdout:7/32: mkdir de/df 0 2024-05-12T04:00:58.564 INFO:tasks.workunit.client.1.smithi033.stdout:7/33: write f7 [2465436,37447] 0 2024-05-12T04:00:58.565 INFO:tasks.workunit.client.1.smithi033.stdout:0/31: symlink d6/l8 0 2024-05-12T04:00:58.567 INFO:tasks.workunit.client.1.smithi033.stdout:7/34: unlink lc 0 2024-05-12T04:00:58.568 INFO:tasks.workunit.client.1.smithi033.stdout:0/32: symlink d6/l9 0 2024-05-12T04:00:58.569 INFO:tasks.workunit.client.1.smithi033.stdout:3/26: truncate f6 2289206 0 2024-05-12T04:00:58.570 INFO:tasks.workunit.client.1.smithi033.stdout:0/33: mknod d6/ca 0 2024-05-12T04:00:58.572 INFO:tasks.workunit.client.1.smithi033.stdout:3/27: creat db/fd x:0 0 0 2024-05-12T04:00:58.573 INFO:tasks.workunit.client.1.smithi033.stdout:0/34: mkdir d6/db 0 2024-05-12T04:00:58.573 INFO:tasks.workunit.client.1.smithi033.stdout:9/20: truncate f0 2007052 0 2024-05-12T04:00:58.573 INFO:tasks.workunit.client.1.smithi033.stdout:3/28: fdatasync db/fd 0 2024-05-12T04:00:58.574 INFO:tasks.workunit.client.1.smithi033.stdout:8/20: dwrite d1/f2 [0,4194304] 0 2024-05-12T04:00:58.576 INFO:tasks.workunit.client.1.smithi033.stdout:7/35: dread f3 [0,4194304] 0 2024-05-12T04:00:58.576 INFO:tasks.workunit.client.1.smithi033.stdout:7/36: chown cb 583925 1 2024-05-12T04:00:58.577 INFO:tasks.workunit.client.1.smithi033.stdout:9/21: creat d2/fa x:0 0 0 2024-05-12T04:00:58.577 INFO:tasks.workunit.client.1.smithi033.stdout:7/37: fdatasync f7 0 2024-05-12T04:00:58.578 INFO:tasks.workunit.client.1.smithi033.stdout:9/22: chown d2/l3 62112 1 2024-05-12T04:00:58.579 INFO:tasks.workunit.client.1.smithi033.stdout:0/35: mkdir d6/dc 0 2024-05-12T04:00:58.580 INFO:tasks.workunit.client.1.smithi033.stdout:2/44: read dc/f10 [216020,28557] 0 2024-05-12T04:00:58.580 INFO:tasks.workunit.client.1.smithi033.stdout:8/21: mknod d1/d4/c6 0 2024-05-12T04:00:58.582 INFO:tasks.workunit.client.1.smithi033.stdout:0/36: chown d6/dc 10327 1 2024-05-12T04:00:58.582 INFO:tasks.workunit.client.1.smithi033.stdout:2/45: chown dc 1599 1 2024-05-12T04:00:58.582 INFO:tasks.workunit.client.1.smithi033.stdout:2/46: fdatasync f3 0 2024-05-12T04:00:58.583 INFO:tasks.workunit.client.1.smithi033.stdout:0/37: write f4 [615168,17477] 0 2024-05-12T04:00:58.584 INFO:tasks.workunit.client.1.smithi033.stdout:2/47: chown lb 5 1 2024-05-12T04:00:58.584 INFO:tasks.workunit.client.1.smithi033.stdout:3/29: unlink db/lc 0 2024-05-12T04:00:58.585 INFO:tasks.workunit.client.1.smithi033.stdout:2/48: dread - f1 zero size 2024-05-12T04:00:58.588 INFO:tasks.workunit.client.1.smithi033.stdout:7/38: creat de/df/f10 x:0 0 0 2024-05-12T04:00:58.589 INFO:tasks.workunit.client.1.smithi033.stdout:8/22: dwrite d1/f2 [0,4194304] 0 2024-05-12T04:00:58.590 INFO:tasks.workunit.client.1.smithi033.stdout:7/39: write f7 [4265138,91626] 0 2024-05-12T04:00:58.591 INFO:tasks.workunit.client.1.smithi033.stdout:7/40: chown f7 662216234 1 2024-05-12T04:00:58.592 INFO:tasks.workunit.client.1.smithi033.stdout:7/41: write f7 [5708738,9193] 0 2024-05-12T04:00:58.593 INFO:tasks.workunit.client.1.smithi033.stdout:9/23: symlink d2/d4/d5/lb 0 2024-05-12T04:00:58.595 INFO:tasks.workunit.client.1.smithi033.stdout:3/30: mknod db/ce 0 2024-05-12T04:00:58.597 INFO:tasks.workunit.client.1.smithi033.stdout:2/49: creat dc/d13/f16 x:0 0 0 2024-05-12T04:00:58.598 INFO:tasks.workunit.client.1.smithi033.stdout:8/23: chown d1/d4/l5 367556 1 2024-05-12T04:00:58.600 INFO:tasks.workunit.client.1.smithi033.stdout:7/42: write f3 [4629364,98964] 0 2024-05-12T04:00:58.601 INFO:tasks.workunit.client.1.smithi033.stdout:0/38: creat d6/db/fd x:0 0 0 2024-05-12T04:00:58.603 INFO:tasks.workunit.client.1.smithi033.stdout:9/24: unlink c1 0 2024-05-12T04:00:58.603 INFO:tasks.workunit.client.1.smithi033.stdout:9/25: chown f0 437061740 1 2024-05-12T04:00:58.604 INFO:tasks.workunit.client.1.smithi033.stdout:3/31: symlink db/lf 0 2024-05-12T04:00:58.604 INFO:tasks.workunit.client.1.smithi033.stdout:3/32: fsync db/fd 0 2024-05-12T04:00:58.606 INFO:tasks.workunit.client.1.smithi033.stdout:2/50: write f6 [463097,130352] 0 2024-05-12T04:00:58.607 INFO:tasks.workunit.client.1.smithi033.stdout:8/24: creat d1/d4/f7 x:0 0 0 2024-05-12T04:00:58.607 INFO:tasks.workunit.client.1.smithi033.stdout:2/51: fdatasync dc/d13/f14 0 2024-05-12T04:00:58.608 INFO:tasks.workunit.client.1.smithi033.stdout:7/43: unlink l0 0 2024-05-12T04:00:58.609 INFO:tasks.workunit.client.1.smithi033.stdout:2/52: dread dc/f10 [0,4194304] 0 2024-05-12T04:00:58.609 INFO:tasks.workunit.client.1.smithi033.stdout:0/39: mknod d6/db/ce 0 2024-05-12T04:00:58.611 INFO:tasks.workunit.client.1.smithi033.stdout:2/53: chown lb 33199160 1 2024-05-12T04:00:58.611 INFO:tasks.workunit.client.1.smithi033.stdout:9/26: mknod d2/d4/d5/cc 0 2024-05-12T04:00:58.611 INFO:tasks.workunit.client.1.smithi033.stdout:0/40: fdatasync d6/f7 0 2024-05-12T04:00:58.612 INFO:tasks.workunit.client.1.smithi033.stdout:9/27: read - d2/fa zero size 2024-05-12T04:00:58.613 INFO:tasks.workunit.client.1.smithi033.stdout:0/41: dread - d6/f7 zero size 2024-05-12T04:00:58.613 INFO:tasks.workunit.client.1.smithi033.stdout:2/54: chown dc/d13/f16 5717256 1 2024-05-12T04:00:58.615 INFO:tasks.workunit.client.1.smithi033.stdout:0/42: dread - d6/db/fd zero size 2024-05-12T04:00:58.616 INFO:tasks.workunit.client.1.smithi033.stdout:0/43: read - d6/f7 zero size 2024-05-12T04:00:58.616 INFO:tasks.workunit.client.1.smithi033.stdout:7/44: dwrite f3 [0,4194304] 0 2024-05-12T04:00:58.617 INFO:tasks.workunit.client.1.smithi033.stdout:0/44: fdatasync d6/db/fd 0 2024-05-12T04:00:58.617 INFO:tasks.workunit.client.1.smithi033.stdout:7/45: write f7 [1411889,69410] 0 2024-05-12T04:00:58.618 INFO:tasks.workunit.client.1.smithi033.stdout:2/55: mkdir dc/d17 0 2024-05-12T04:00:58.618 INFO:tasks.workunit.client.1.smithi033.stdout:7/46: chown de 7127100 1 2024-05-12T04:00:58.620 INFO:tasks.workunit.client.1.smithi033.stdout:7/47: write f3 [393178,104578] 0 2024-05-12T04:00:58.622 INFO:tasks.workunit.client.1.smithi033.stdout:3/33: link l7 db/l10 0 2024-05-12T04:00:58.622 INFO:tasks.workunit.client.1.smithi033.stdout:3/34: truncate db/fd 1048422 0 2024-05-12T04:00:58.625 INFO:tasks.workunit.client.1.smithi033.stdout:0/45: symlink d6/db/lf 0 2024-05-12T04:00:58.625 INFO:tasks.workunit.client.1.smithi033.stdout:3/35: dread db/fd [0,4194304] 0 2024-05-12T04:00:58.625 INFO:tasks.workunit.client.1.smithi033.stdout:8/25: fsync d1/d4/f7 0 2024-05-12T04:00:58.626 INFO:tasks.workunit.client.1.smithi033.stdout:3/36: write db/fd [354997,36167] 0 2024-05-12T04:00:58.628 INFO:tasks.workunit.client.1.smithi033.stdout:7/48: creat de/df/f11 x:0 0 0 2024-05-12T04:00:58.630 INFO:tasks.workunit.client.1.smithi033.stdout:9/28: rename d2/d4/d5 to d2/dd 0 2024-05-12T04:00:58.630 INFO:tasks.workunit.client.1.smithi033.stdout:2/56: getdents dc/d17 0 2024-05-12T04:00:58.631 INFO:tasks.workunit.client.1.smithi033.stdout:3/37: symlink db/l11 0 2024-05-12T04:00:58.632 INFO:tasks.workunit.client.1.smithi033.stdout:8/26: creat d1/d4/f8 x:0 0 0 2024-05-12T04:00:58.634 INFO:tasks.workunit.client.1.smithi033.stdout:0/46: dwrite d6/f7 [0,4194304] 0 2024-05-12T04:00:58.635 INFO:tasks.workunit.client.1.smithi033.stdout:9/29: write f0 [735763,27959] 0 2024-05-12T04:00:58.637 INFO:tasks.workunit.client.1.smithi033.stdout:7/49: rename f7 to de/f12 0 2024-05-12T04:00:58.639 INFO:tasks.workunit.client.1.smithi033.stdout:3/38: unlink c8 0 2024-05-12T04:00:58.641 INFO:tasks.workunit.client.1.smithi033.stdout:8/27: symlink d1/l9 0 2024-05-12T04:00:58.644 INFO:tasks.workunit.client.1.smithi033.stdout:8/28: fdatasync d1/d4/f7 0 2024-05-12T04:00:58.645 INFO:tasks.workunit.client.1.smithi033.stdout:9/30: mknod d2/d4/ce 0 2024-05-12T04:00:58.646 INFO:tasks.workunit.client.1.smithi033.stdout:0/47: dwrite f4 [0,4194304] 0 2024-05-12T04:00:58.649 INFO:tasks.workunit.client.1.smithi033.stdout:2/57: symlink dc/d17/l18 0 2024-05-12T04:00:58.651 INFO:tasks.workunit.client.1.smithi033.stdout:8/29: write d1/f2 [3231691,83032] 0 2024-05-12T04:00:58.651 INFO:tasks.workunit.client.1.smithi033.stdout:0/48: read f4 [675146,20160] 0 2024-05-12T04:00:58.651 INFO:tasks.workunit.client.1.smithi033.stdout:3/39: dread db/fd [0,4194304] 0 2024-05-12T04:00:58.654 INFO:tasks.workunit.client.1.smithi033.stdout:7/50: dwrite de/df/f11 [0,4194304] 0 2024-05-12T04:00:58.657 INFO:tasks.workunit.client.1.smithi033.stdout:7/51: rename de/df to de/df/d13 22 2024-05-12T04:00:58.658 INFO:tasks.workunit.client.1.smithi033.stdout:9/31: creat d2/d4/ff x:0 0 0 2024-05-12T04:00:58.661 INFO:tasks.workunit.client.1.smithi033.stdout:2/58: chown dc/ce 1 1 2024-05-12T04:00:58.663 INFO:tasks.workunit.client.1.smithi033.stdout:2/59: rmdir dc/d17 39 2024-05-12T04:00:58.663 INFO:tasks.workunit.client.1.smithi033.stdout:7/52: readlink l5 0 2024-05-12T04:00:58.664 INFO:tasks.workunit.client.1.smithi033.stdout:3/40: getdents db 0 2024-05-12T04:00:58.668 INFO:tasks.workunit.client.1.smithi033.stdout:7/53: symlink de/l14 0 2024-05-12T04:00:58.669 INFO:tasks.workunit.client.1.smithi033.stdout:8/30: dwrite d1/d4/f8 [0,4194304] 0 2024-05-12T04:00:58.670 INFO:tasks.workunit.client.1.smithi033.stdout:3/41: rename db/fd to db/f12 0 2024-05-12T04:00:58.671 INFO:tasks.workunit.client.1.smithi033.stdout:7/54: chown de/l14 3591225 1 2024-05-12T04:00:58.672 INFO:tasks.workunit.client.1.smithi033.stdout:0/49: truncate f4 781542 0 2024-05-12T04:00:58.673 INFO:tasks.workunit.client.1.smithi033.stdout:0/50: fsync d6/f7 0 2024-05-12T04:00:58.674 INFO:tasks.workunit.client.1.smithi033.stdout:7/55: read de/f12 [5260239,29504] 0 2024-05-12T04:00:58.675 INFO:tasks.workunit.client.1.smithi033.stdout:8/31: creat d1/fa x:0 0 0 2024-05-12T04:00:58.675 INFO:tasks.workunit.client.1.smithi033.stdout:9/32: dwrite f0 [0,4194304] 0 2024-05-12T04:00:58.676 INFO:tasks.workunit.client.1.smithi033.stdout:0/51: chown d6/l9 3847518 1 2024-05-12T04:00:58.676 INFO:tasks.workunit.client.1.smithi033.stdout:7/56: dread - de/df/f10 zero size 2024-05-12T04:00:58.676 INFO:tasks.workunit.client.1.smithi033.stdout:0/52: fdatasync f2 0 2024-05-12T04:00:58.683 INFO:tasks.workunit.client.1.smithi033.stdout:9/33: chown d2/dd/cc 167353038 1 2024-05-12T04:00:58.683 INFO:tasks.workunit.client.1.smithi033.stdout:9/34: chown f0 133584558 1 2024-05-12T04:00:58.695 INFO:tasks.workunit.client.1.smithi033.stdout:9/35: dwrite f0 [0,4194304] 0 2024-05-12T04:00:58.696 INFO:tasks.workunit.client.1.smithi033.stdout:9/36: chown d2/d4/ce 296468953 1 2024-05-12T04:00:58.706 INFO:tasks.workunit.client.1.smithi033.stdout:9/37: dwrite d2/d4/ff [0,4194304] 0 2024-05-12T04:00:58.710 INFO:tasks.workunit.client.1.smithi033.stdout:3/42: rename l9 to db/l13 0 2024-05-12T04:00:58.710 INFO:tasks.workunit.client.1.smithi033.stdout:8/32: mknod d1/cb 0 2024-05-12T04:00:58.714 INFO:tasks.workunit.client.1.smithi033.stdout:0/53: symlink d6/db/l10 0 2024-05-12T04:00:58.716 INFO:tasks.workunit.client.1.smithi033.stdout:9/38: creat d2/d4/f10 x:0 0 0 2024-05-12T04:00:58.717 INFO:tasks.workunit.client.1.smithi033.stdout:8/33: dread d1/f2 [0,4194304] 0 2024-05-12T04:00:58.718 INFO:tasks.workunit.client.1.smithi033.stdout:8/34: chown d1/d4/c6 1 1 2024-05-12T04:00:58.719 INFO:tasks.workunit.client.1.smithi033.stdout:0/54: fsync f4 0 2024-05-12T04:00:58.721 INFO:tasks.workunit.client.1.smithi033.stdout:9/39: creat d2/dd/f11 x:0 0 0 2024-05-12T04:00:58.722 INFO:tasks.workunit.client.1.smithi033.stdout:8/35: dread d1/d4/f8 [0,4194304] 0 2024-05-12T04:00:58.723 INFO:tasks.workunit.client.1.smithi033.stdout:0/55: mkdir d6/d11 0 2024-05-12T04:00:58.724 INFO:tasks.workunit.client.1.smithi033.stdout:8/36: truncate d1/d4/f7 391278 0 2024-05-12T04:00:58.724 INFO:tasks.workunit.client.1.smithi033.stdout:9/40: symlink d2/d4/l12 0 2024-05-12T04:00:58.727 INFO:tasks.workunit.client.1.smithi033.stdout:9/41: creat d2/dd/f13 x:0 0 0 2024-05-12T04:00:58.729 INFO:tasks.workunit.client.1.smithi033.stdout:9/42: mknod d2/d4/c14 0 2024-05-12T04:00:58.732 INFO:tasks.workunit.client.1.smithi033.stdout:9/43: unlink f0 0 2024-05-12T04:00:58.736 INFO:tasks.workunit.client.1.smithi033.stdout:9/44: dread d2/d4/ff [0,4194304] 0 2024-05-12T04:00:58.738 INFO:tasks.workunit.client.1.smithi033.stdout:9/45: creat d2/d4/f15 x:0 0 0 2024-05-12T04:00:58.738 INFO:tasks.workunit.client.1.smithi033.stdout:9/46: dread - d2/d4/f10 zero size 2024-05-12T04:00:58.739 INFO:tasks.workunit.client.1.smithi033.stdout:9/47: dread - d2/d4/f10 zero size 2024-05-12T04:00:58.740 INFO:tasks.workunit.client.1.smithi033.stdout:0/56: truncate d6/f7 2701812 0 2024-05-12T04:00:58.744 INFO:tasks.workunit.client.1.smithi033.stdout:3/43: unlink db/l13 0 2024-05-12T04:00:58.744 INFO:tasks.workunit.client.1.smithi033.stdout:0/57: mknod d6/dc/c12 0 2024-05-12T04:00:58.746 INFO:tasks.workunit.client.1.smithi033.stdout:0/58: creat d6/db/f13 x:0 0 0 2024-05-12T04:00:58.748 INFO:tasks.workunit.client.1.smithi033.stdout:3/44: rename c1 to db/c14 0 2024-05-12T04:00:58.749 INFO:tasks.workunit.client.1.smithi033.stdout:7/57: truncate de/f12 4472160 0 2024-05-12T04:00:58.750 INFO:tasks.workunit.client.1.smithi033.stdout:3/45: mkdir db/d15 0 2024-05-12T04:00:58.752 INFO:tasks.workunit.client.1.smithi033.stdout:0/59: rename d6/db/ce to d6/c14 0 2024-05-12T04:00:58.753 INFO:tasks.workunit.client.1.smithi033.stdout:7/58: dread f3 [0,4194304] 0 2024-05-12T04:00:58.753 INFO:tasks.workunit.client.1.smithi033.stdout:9/48: fsync d2/dd/f11 0 2024-05-12T04:00:58.756 INFO:tasks.workunit.client.1.smithi033.stdout:0/60: dread f2 [0,4194304] 0 2024-05-12T04:00:58.756 INFO:tasks.workunit.client.1.smithi033.stdout:9/49: creat d2/d4/f16 x:0 0 0 2024-05-12T04:00:58.759 INFO:tasks.workunit.client.1.smithi033.stdout:0/61: mkdir d6/d15 0 2024-05-12T04:00:58.760 INFO:tasks.workunit.client.1.smithi033.stdout:0/62: stat f2 0 2024-05-12T04:00:58.763 INFO:tasks.workunit.client.1.smithi033.stdout:0/63: creat d6/dc/f16 x:0 0 0 2024-05-12T04:00:58.766 INFO:tasks.workunit.client.1.smithi033.stdout:9/50: dwrite d2/dd/f11 [0,4194304] 0 2024-05-12T04:00:58.770 INFO:tasks.workunit.client.1.smithi033.stdout:9/51: unlink d2/d4/ce 0 2024-05-12T04:00:58.771 INFO:tasks.workunit.client.1.smithi033.stdout:7/59: fsync de/f12 0 2024-05-12T04:00:58.773 INFO:tasks.workunit.client.1.smithi033.stdout:3/46: getdents db 0 2024-05-12T04:00:58.776 INFO:tasks.workunit.client.1.smithi033.stdout:9/52: dread d2/dd/f11 [0,4194304] 0 2024-05-12T04:00:58.777 INFO:tasks.workunit.client.1.smithi033.stdout:7/60: rmdir de/df 39 2024-05-12T04:00:58.777 INFO:tasks.workunit.client.1.smithi033.stdout:9/53: stat d2 0 2024-05-12T04:00:58.778 INFO:tasks.workunit.client.1.smithi033.stdout:3/47: fdatasync f6 0 2024-05-12T04:00:58.779 INFO:tasks.workunit.client.1.smithi033.stdout:7/61: fsync de/f12 0 2024-05-12T04:00:58.780 INFO:tasks.workunit.client.1.smithi033.stdout:9/54: mknod d2/d4/c17 0 2024-05-12T04:00:58.780 INFO:tasks.workunit.client.1.smithi033.stdout:9/55: rename d2 to d2/d18 22 2024-05-12T04:00:58.781 INFO:tasks.workunit.client.1.smithi033.stdout:3/48: rmdir db 39 2024-05-12T04:00:58.782 INFO:tasks.workunit.client.1.smithi033.stdout:0/64: write d6/f7 [1817331,73649] 0 2024-05-12T04:00:58.782 INFO:tasks.workunit.client.1.smithi033.stdout:0/65: stat d6 0 2024-05-12T04:00:58.787 INFO:tasks.workunit.client.1.smithi033.stdout:9/56: creat d2/dd/f19 x:0 0 0 2024-05-12T04:00:58.787 INFO:tasks.workunit.client.1.smithi033.stdout:3/49: fsync f6 0 2024-05-12T04:00:58.788 INFO:tasks.workunit.client.1.smithi033.stdout:7/62: creat de/df/f15 x:0 0 0 2024-05-12T04:00:58.793 INFO:tasks.workunit.client.1.smithi033.stdout:3/50: write f6 [2586645,51159] 0 2024-05-12T04:00:58.797 INFO:tasks.workunit.client.1.smithi033.stdout:3/51: write f6 [359225,44001] 0 2024-05-12T04:00:58.797 INFO:tasks.workunit.client.1.smithi033.stdout:0/66: dwrite d6/db/fd [0,4194304] 0 2024-05-12T04:00:58.798 INFO:tasks.workunit.client.1.smithi033.stdout:7/63: link cb de/df/c16 0 2024-05-12T04:00:58.804 INFO:tasks.workunit.client.1.smithi033.stdout:9/57: dwrite d2/d4/f15 [0,4194304] 0 2024-05-12T04:00:58.808 INFO:tasks.workunit.client.1.smithi033.stdout:7/64: mkdir de/df/d17 0 2024-05-12T04:00:58.809 INFO:tasks.workunit.client.1.smithi033.stdout:9/58: mknod d2/dd/c1a 0 2024-05-12T04:00:58.815 INFO:tasks.workunit.client.1.smithi033.stdout:9/59: write d2/dd/f13 [895281,33251] 0 2024-05-12T04:00:58.819 INFO:tasks.workunit.client.1.smithi033.stdout:0/67: dwrite f2 [4194304,4194304] 0 2024-05-12T04:00:58.822 INFO:tasks.workunit.client.1.smithi033.stdout:3/52: dwrite db/f12 [0,4194304] 0 2024-05-12T04:00:58.829 INFO:tasks.workunit.client.1.smithi033.stdout:9/60: rename d2/dd/cc to d2/dd/c1b 0 2024-05-12T04:00:58.829 INFO:tasks.workunit.client.1.smithi033.stdout:0/68: dread f4 [0,4194304] 0 2024-05-12T04:00:58.830 INFO:tasks.workunit.client.1.smithi033.stdout:3/53: creat db/f16 x:0 0 0 2024-05-12T04:00:58.830 INFO:tasks.workunit.client.1.smithi033.stdout:0/69: dread f4 [0,4194304] 0 2024-05-12T04:00:58.830 INFO:tasks.workunit.client.1.smithi033.stdout:0/70: fsync f4 0 2024-05-12T04:00:58.830 INFO:tasks.workunit.client.1.smithi033.stdout:3/54: rename db/d15 to db/d17 0 2024-05-12T04:00:58.833 INFO:tasks.workunit.client.1.smithi033.stdout:0/71: chown d6/c14 6 1 2024-05-12T04:00:58.834 INFO:tasks.workunit.client.1.smithi033.stdout:3/55: creat db/d17/f18 x:0 0 0 2024-05-12T04:00:58.838 INFO:tasks.workunit.client.1.smithi033.stdout:9/61: dwrite d2/d4/ff [4194304,4194304] 0 2024-05-12T04:00:58.840 INFO:tasks.workunit.client.1.smithi033.stdout:9/62: write d2/dd/f19 [27506,25337] 0 2024-05-12T04:00:58.840 INFO:tasks.workunit.client.1.smithi033.stdout:3/56: mknod db/d17/c19 0 2024-05-12T04:00:58.841 INFO:tasks.workunit.client.1.smithi033.stdout:3/57: stat db 0 2024-05-12T04:00:58.842 INFO:tasks.workunit.client.1.smithi033.stdout:3/58: dread - db/f16 zero size 2024-05-12T04:00:58.844 INFO:tasks.workunit.client.1.smithi033.stdout:3/59: mknod db/d17/c1a 0 2024-05-12T04:00:58.846 INFO:tasks.workunit.client.1.smithi033.stdout:3/60: write f6 [3652570,66220] 0 2024-05-12T04:00:58.851 INFO:tasks.workunit.client.1.smithi033.stdout:3/61: creat db/d17/f1b x:0 0 0 2024-05-12T04:00:58.852 INFO:tasks.workunit.client.1.smithi033.stdout:0/72: truncate d6/db/fd 3823677 0 2024-05-12T04:00:58.852 INFO:tasks.workunit.client.1.smithi033.stdout:9/63: dwrite d2/d4/f16 [0,4194304] 0 2024-05-12T04:00:58.852 INFO:tasks.workunit.client.1.smithi033.stdout:3/62: stat db/d17/c19 0 2024-05-12T04:00:58.854 INFO:tasks.workunit.client.1.smithi033.stdout:3/63: write db/d17/f18 [491221,10390] 0 2024-05-12T04:00:58.859 INFO:tasks.workunit.client.1.smithi033.stdout:9/64: dread d2/d4/f16 [0,4194304] 0 2024-05-12T04:00:58.860 INFO:tasks.workunit.client.1.smithi033.stdout:3/64: rmdir db 39 2024-05-12T04:00:58.863 INFO:tasks.workunit.client.1.smithi033.stdout:9/65: rmdir d2/dd 39 2024-05-12T04:00:58.866 INFO:tasks.workunit.client.1.smithi033.stdout:0/73: write d6/db/fd [4123189,95173] 0 2024-05-12T04:00:58.866 INFO:tasks.workunit.client.1.smithi033.stdout:3/65: truncate db/f16 675654 0 2024-05-12T04:00:58.866 INFO:tasks.workunit.client.1.smithi033.stdout:9/66: rmdir d2/dd 39 2024-05-12T04:00:58.867 INFO:tasks.workunit.client.1.smithi033.stdout:9/67: readlink d2/d4/l12 0 2024-05-12T04:00:58.868 INFO:tasks.workunit.client.1.smithi033.stdout:3/66: creat db/d17/f1c x:0 0 0 2024-05-12T04:00:58.870 INFO:tasks.workunit.client.1.smithi033.stdout:0/74: truncate f4 641819 0 2024-05-12T04:00:58.871 INFO:tasks.workunit.client.1.smithi033.stdout:3/67: mknod db/d17/c1d 0 2024-05-12T04:00:58.874 INFO:tasks.workunit.client.1.smithi033.stdout:3/68: dread db/f16 [0,4194304] 0 2024-05-12T04:00:58.875 INFO:tasks.workunit.client.1.smithi033.stdout:0/75: symlink d6/d15/l17 0 2024-05-12T04:00:58.876 INFO:tasks.workunit.client.1.smithi033.stdout:3/69: rename db/d17 to db/d17/d1e 22 2024-05-12T04:00:58.877 INFO:tasks.workunit.client.1.smithi033.stdout:9/68: dwrite d2/fa [0,4194304] 0 2024-05-12T04:00:58.878 INFO:tasks.workunit.client.1.smithi033.stdout:0/76: creat d6/dc/f18 x:0 0 0 2024-05-12T04:00:58.879 INFO:tasks.workunit.client.1.smithi033.stdout:0/77: chown f5 155852449 1 2024-05-12T04:00:58.886 INFO:tasks.workunit.client.1.smithi033.stdout:9/69: dwrite d2/d4/f15 [0,4194304] 0 2024-05-12T04:00:58.886 INFO:tasks.workunit.client.1.smithi033.stdout:3/70: truncate db/f12 3458093 0 2024-05-12T04:00:58.888 INFO:tasks.workunit.client.1.smithi033.stdout:3/71: truncate f6 3737549 0 2024-05-12T04:00:58.890 INFO:tasks.workunit.client.1.smithi033.stdout:9/70: mknod d2/c1c 0 2024-05-12T04:00:58.893 INFO:tasks.workunit.client.1.smithi033.stdout:3/72: read db/d17/f18 [277671,75221] 0 2024-05-12T04:00:58.895 INFO:tasks.workunit.client.1.smithi033.stdout:9/71: rename d2 to d2/dd/d1d 22 2024-05-12T04:00:58.897 INFO:tasks.workunit.client.1.smithi033.stdout:0/78: dwrite f4 [0,4194304] 0 2024-05-12T04:00:58.897 INFO:tasks.workunit.client.1.smithi033.stdout:9/72: chown d2/dd/c1a 601 1 2024-05-12T04:00:58.899 INFO:tasks.workunit.client.1.smithi033.stdout:0/79: creat d6/db/f19 x:0 0 0 2024-05-12T04:00:58.902 INFO:tasks.workunit.client.1.smithi033.stdout:0/80: creat d6/db/f1a x:0 0 0 2024-05-12T04:00:58.904 INFO:tasks.workunit.client.1.smithi033.stdout:0/81: mknod d6/c1b 0 2024-05-12T04:00:58.912 INFO:tasks.workunit.client.1.smithi033.stdout:0/82: dwrite d6/db/f19 [0,4194304] 0 2024-05-12T04:00:58.912 INFO:tasks.workunit.client.1.smithi033.stdout:0/83: chown d6/l9 19031007 1 2024-05-12T04:00:58.915 INFO:tasks.workunit.client.1.smithi033.stdout:0/84: creat d6/d11/f1c x:0 0 0 2024-05-12T04:00:58.917 INFO:tasks.workunit.client.1.smithi033.stdout:0/85: creat d6/dc/f1d x:0 0 0 2024-05-12T04:00:58.923 INFO:tasks.workunit.client.1.smithi033.stdout:9/73: truncate d2/fa 999954 0 2024-05-12T04:00:58.926 INFO:tasks.workunit.client.1.smithi033.stdout:9/74: fdatasync d2/dd/f11 0 2024-05-12T04:00:58.926 INFO:tasks.workunit.client.1.smithi033.stdout:0/86: getdents d6/d11 0 2024-05-12T04:00:58.927 INFO:tasks.workunit.client.1.smithi033.stdout:0/87: symlink d6/dc/l1e 0 2024-05-12T04:00:58.928 INFO:tasks.workunit.client.1.smithi033.stdout:9/75: symlink d2/l1e 0 2024-05-12T04:00:58.929 INFO:tasks.workunit.client.1.smithi033.stdout:0/88: mkdir d6/db/d1f 0 2024-05-12T04:00:58.930 INFO:tasks.workunit.client.1.smithi033.stdout:9/76: creat d2/d4/f1f x:0 0 0 2024-05-12T04:00:58.931 INFO:tasks.workunit.client.1.smithi033.stdout:9/77: write d2/dd/f13 [892275,87113] 0 2024-05-12T04:00:58.933 INFO:tasks.workunit.client.1.smithi033.stdout:0/89: getdents d6/d15 0 2024-05-12T04:00:58.933 INFO:tasks.workunit.client.1.smithi033.stdout:0/90: stat d6/db/f13 0 2024-05-12T04:00:58.934 INFO:tasks.workunit.client.1.smithi033.stdout:9/78: creat d2/f20 x:0 0 0 2024-05-12T04:00:58.935 INFO:tasks.workunit.client.1.smithi033.stdout:0/91: creat d6/dc/f20 x:0 0 0 2024-05-12T04:00:58.935 INFO:tasks.workunit.client.1.smithi033.stdout:9/79: read d2/dd/f19 [37616,49038] 0 2024-05-12T04:00:58.937 INFO:tasks.workunit.client.1.smithi033.stdout:9/80: creat d2/d4/f21 x:0 0 0 2024-05-12T04:00:58.938 INFO:tasks.workunit.client.1.smithi033.stdout:9/81: truncate d2/dd/f13 1514636 0 2024-05-12T04:00:58.938 INFO:tasks.workunit.client.1.smithi033.stdout:0/92: rename d6/dc/f20 to d6/d15/f21 0 2024-05-12T04:00:58.939 INFO:tasks.workunit.client.1.smithi033.stdout:0/93: write d6/d11/f1c [76972,115329] 0 2024-05-12T04:00:58.942 INFO:tasks.workunit.client.1.smithi033.stdout:0/94: symlink d6/dc/l22 0 2024-05-12T04:00:58.951 INFO:tasks.workunit.client.1.smithi033.stdout:9/82: dwrite d2/dd/f19 [0,4194304] 0 2024-05-12T04:00:58.951 INFO:tasks.workunit.client.1.smithi033.stdout:0/95: dwrite f4 [0,4194304] 0 2024-05-12T04:00:58.955 INFO:tasks.workunit.client.1.smithi033.stdout:0/96: rename f5 to d6/f23 0 2024-05-12T04:00:58.957 INFO:tasks.workunit.client.1.smithi033.stdout:0/97: read - d6/db/f1a zero size 2024-05-12T04:00:58.959 INFO:tasks.workunit.client.1.smithi033.stdout:0/98: mknod d6/d15/c24 0 2024-05-12T04:00:58.961 INFO:tasks.workunit.client.1.smithi033.stdout:9/83: dwrite d2/d4/f15 [0,4194304] 0 2024-05-12T04:00:58.962 INFO:tasks.workunit.client.1.smithi033.stdout:0/99: symlink d6/dc/l25 0 2024-05-12T04:00:58.965 INFO:tasks.workunit.client.1.smithi033.stdout:0/100: creat d6/f26 x:0 0 0 2024-05-12T04:00:58.968 INFO:tasks.workunit.client.1.smithi033.stdout:0/101: creat d6/dc/f27 x:0 0 0 2024-05-12T04:00:58.970 INFO:tasks.workunit.client.1.smithi033.stdout:0/102: creat d6/d11/f28 x:0 0 0 2024-05-12T04:00:58.971 INFO:tasks.workunit.client.1.smithi033.stdout:0/103: write d6/db/fd [1664860,22254] 0 2024-05-12T04:00:58.973 INFO:tasks.workunit.client.1.smithi033.stdout:0/104: dread d6/d11/f1c [0,4194304] 0 2024-05-12T04:00:58.975 INFO:tasks.workunit.client.1.smithi033.stdout:0/105: creat d6/db/d1f/f29 x:0 0 0 2024-05-12T04:00:58.984 INFO:tasks.workunit.client.1.smithi033.stdout:0/106: dwrite d6/d11/f28 [0,4194304] 0 2024-05-12T04:00:58.984 INFO:tasks.workunit.client.1.smithi033.stdout:9/84: dread d2/fa [0,4194304] 0 2024-05-12T04:00:58.986 INFO:tasks.workunit.client.1.smithi033.stdout:9/85: write d2/dd/f13 [1112844,7448] 0 2024-05-12T04:00:58.988 INFO:tasks.workunit.client.1.smithi033.stdout:9/86: mknod d2/d4/c22 0 2024-05-12T04:00:58.989 INFO:tasks.workunit.client.1.smithi033.stdout:0/107: dread f2 [4194304,4194304] 0 2024-05-12T04:00:58.991 INFO:tasks.workunit.client.1.smithi033.stdout:0/108: write d6/d15/f21 [751262,84555] 0 2024-05-12T04:00:58.992 INFO:tasks.workunit.client.1.smithi033.stdout:9/87: unlink d2/d4/c14 0 2024-05-12T04:00:58.992 INFO:tasks.workunit.client.1.smithi033.stdout:0/109: fsync d6/f26 0 2024-05-12T04:00:58.994 INFO:tasks.workunit.client.1.smithi033.stdout:0/110: mknod d6/d11/c2a 0 2024-05-12T04:00:58.995 INFO:tasks.workunit.client.1.smithi033.stdout:9/88: symlink d2/l23 0 2024-05-12T04:00:58.996 INFO:tasks.workunit.client.1.smithi033.stdout:0/111: rmdir d6/db 39 2024-05-12T04:00:58.996 INFO:tasks.workunit.client.1.smithi033.stdout:9/89: read d2/dd/f13 [476354,120617] 0 2024-05-12T04:00:58.998 INFO:tasks.workunit.client.1.smithi033.stdout:0/112: chown d6/db/d1f/f29 101403452 1 2024-05-12T04:00:59.001 INFO:tasks.workunit.client.1.smithi033.stdout:9/90: rmdir d2/d4 39 2024-05-12T04:00:59.005 INFO:tasks.workunit.client.1.smithi033.stdout:0/113: getdents d6/d11 0 2024-05-12T04:00:59.008 INFO:tasks.workunit.client.1.smithi033.stdout:0/114: truncate d6/db/d1f/f29 448546 0 2024-05-12T04:00:59.010 INFO:tasks.workunit.client.1.smithi033.stdout:9/91: dwrite d2/d4/f15 [4194304,4194304] 0 2024-05-12T04:00:59.014 INFO:tasks.workunit.client.1.smithi033.stdout:0/115: dread d6/db/f19 [0,4194304] 0 2024-05-12T04:00:59.015 INFO:tasks.workunit.client.1.smithi033.stdout:9/92: creat d2/f24 x:0 0 0 2024-05-12T04:00:59.016 INFO:tasks.workunit.client.1.smithi033.stdout:9/93: chown d2/d4/f1f 514715155 1 2024-05-12T04:00:59.017 INFO:tasks.workunit.client.1.smithi033.stdout:0/116: mkdir d6/d2b 0 2024-05-12T04:00:59.019 INFO:tasks.workunit.client.1.smithi033.stdout:0/117: dread - d6/db/f13 zero size 2024-05-12T04:00:59.023 INFO:tasks.workunit.client.1.smithi033.stdout:0/118: rename d6/d2b to d6/db/d2c 0 2024-05-12T04:00:59.025 INFO:tasks.workunit.client.1.smithi033.stdout:9/94: dwrite d2/f20 [0,4194304] 0 2024-05-12T04:00:59.025 INFO:tasks.workunit.client.1.smithi033.stdout:0/119: chown d6/c1b 162939644 1 2024-05-12T04:00:59.033 INFO:tasks.workunit.client.1.smithi033.stdout:0/120: rmdir d6 39 2024-05-12T04:00:59.033 INFO:tasks.workunit.client.1.smithi033.stdout:9/95: dwrite d2/d4/ff [8388608,4194304] 0 2024-05-12T04:00:59.036 INFO:tasks.workunit.client.1.smithi033.stdout:0/121: symlink d6/db/l2d 0 2024-05-12T04:00:59.038 INFO:tasks.workunit.client.1.smithi033.stdout:0/122: mknod d6/db/d2c/c2e 0 2024-05-12T04:00:59.039 INFO:tasks.workunit.client.1.smithi033.stdout:0/123: fsync f4 0 2024-05-12T04:00:59.051 INFO:tasks.workunit.client.1.smithi033.stdout:9/96: fsync d2/d4/ff 0 2024-05-12T04:00:59.053 INFO:tasks.workunit.client.1.smithi033.stdout:9/97: creat d2/d4/f25 x:0 0 0 2024-05-12T04:00:59.059 INFO:tasks.workunit.client.1.smithi033.stdout:9/98: getdents d2/d4 0 2024-05-12T04:00:59.060 INFO:tasks.workunit.client.1.smithi033.stdout:9/99: dread - d2/d4/f25 zero size 2024-05-12T04:00:59.061 INFO:tasks.workunit.client.1.smithi033.stdout:9/100: rmdir d2 39 2024-05-12T04:00:59.068 INFO:tasks.workunit.client.1.smithi033.stdout:9/101: dwrite d2/d4/f16 [0,4194304] 0 2024-05-12T04:00:59.070 INFO:tasks.workunit.client.1.smithi033.stdout:9/102: unlink d2/d4/f21 0 2024-05-12T04:00:59.076 INFO:tasks.workunit.client.1.smithi033.stdout:9/103: dwrite d2/dd/f11 [0,4194304] 0 2024-05-12T04:00:59.084 INFO:tasks.workunit.client.1.smithi033.stdout:9/104: write d2/fa [1129622,3178] 0 2024-05-12T04:00:59.086 INFO:tasks.workunit.client.1.smithi033.stdout:9/105: creat d2/dd/f26 x:0 0 0 2024-05-12T04:00:59.087 INFO:tasks.workunit.client.1.smithi033.stdout:9/106: stat d2/d4/l12 0 2024-05-12T04:00:59.090 INFO:tasks.workunit.client.1.smithi033.stdout:9/107: rename d2/dd/f19 to d2/f27 0 2024-05-12T04:00:59.092 INFO:tasks.workunit.client.1.smithi033.stdout:9/108: creat d2/d4/f28 x:0 0 0 2024-05-12T04:00:59.094 INFO:tasks.workunit.client.1.smithi033.stdout:9/109: unlink d2/f20 0 2024-05-12T04:00:59.096 INFO:tasks.workunit.client.1.smithi033.stdout:9/110: mkdir d2/d4/d29 0 2024-05-12T04:00:59.103 INFO:tasks.workunit.client.1.smithi033.stdout:9/111: dwrite d2/d4/f16 [0,4194304] 0 2024-05-12T04:00:59.105 INFO:tasks.workunit.client.1.smithi033.stdout:9/112: creat d2/d4/d29/f2a x:0 0 0 2024-05-12T04:00:59.106 INFO:tasks.workunit.client.1.smithi033.stdout:9/113: stat d2/d4/f28 0 2024-05-12T04:00:59.107 INFO:tasks.workunit.client.1.smithi033.stdout:9/114: mkdir d2/d4/d2b 0 2024-05-12T04:00:59.109 INFO:tasks.workunit.client.1.smithi033.stdout:9/115: fsync d2/f27 0 2024-05-12T04:00:59.110 INFO:tasks.workunit.client.1.smithi033.stdout:9/116: creat d2/d4/f2c x:0 0 0 2024-05-12T04:00:59.112 INFO:tasks.workunit.client.1.smithi033.stdout:9/117: write d2/d4/d29/f2a [281516,128646] 0 2024-05-12T04:00:59.115 INFO:tasks.workunit.client.1.smithi033.stdout:9/118: mkdir d2/d2d 0 2024-05-12T04:00:59.117 INFO:tasks.workunit.client.1.smithi033.stdout:9/119: mknod d2/c2e 0 2024-05-12T04:00:59.117 INFO:tasks.workunit.client.1.smithi033.stdout:9/120: dread - d2/d4/f25 zero size 2024-05-12T04:00:59.122 INFO:tasks.workunit.client.1.smithi033.stdout:9/121: dwrite d2/fa [0,4194304] 0 2024-05-12T04:00:59.123 INFO:tasks.workunit.client.1.smithi033.stdout:9/122: write d2/d4/f1f [1012842,108967] 0 2024-05-12T04:00:59.124 INFO:tasks.workunit.client.1.smithi033.stdout:9/123: chown d2/d4/c17 386 1 2024-05-12T04:00:59.125 INFO:tasks.workunit.client.1.smithi033.stdout:9/124: truncate d2/d4/f1f 1591767 0 2024-05-12T04:00:59.127 INFO:tasks.workunit.client.1.smithi033.stdout:9/125: mkdir d2/d2d/d2f 0 2024-05-12T04:00:59.128 INFO:tasks.workunit.client.1.smithi033.stdout:9/126: dread - d2/d4/f10 zero size 2024-05-12T04:00:59.134 INFO:tasks.workunit.client.1.smithi033.stdout:9/127: dwrite d2/dd/f11 [0,4194304] 0 2024-05-12T04:00:59.135 INFO:tasks.workunit.client.1.smithi033.stdout:9/128: read d2/d4/f16 [925026,62730] 0 2024-05-12T04:00:59.138 INFO:tasks.workunit.client.1.smithi033.stdout:9/129: creat d2/d4/f30 x:0 0 0 2024-05-12T04:00:59.140 INFO:tasks.workunit.client.1.smithi033.stdout:9/130: write d2/d4/f30 [841204,50938] 0 2024-05-12T04:00:59.147 INFO:tasks.workunit.client.1.smithi033.stdout:9/131: dwrite d2/d4/f25 [0,4194304] 0 2024-05-12T04:00:59.150 INFO:tasks.workunit.client.1.smithi033.stdout:9/132: rename d2/c2e to d2/dd/c31 0 2024-05-12T04:00:59.154 INFO:tasks.workunit.client.1.smithi033.stdout:9/133: dread d2/dd/f13 [0,4194304] 0 2024-05-12T04:00:59.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:00:58 smithi033 ceph-mon[49525]: pgmap v27: 65 pgs: 65 active+clean; 278 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.1 MiB/s rd, 11 MiB/s wr, 306 op/s 2024-05-12T04:00:59.165 INFO:tasks.workunit.client.1.smithi033.stdout:9/134: truncate d2/d4/f30 381946 0 2024-05-12T04:00:59.167 INFO:tasks.workunit.client.1.smithi033.stdout:9/135: mkdir d2/d4/d29/d32 0 2024-05-12T04:00:59.168 INFO:tasks.workunit.client.1.smithi033.stdout:9/136: mkdir d2/d4/d2b/d33 0 2024-05-12T04:00:59.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:00:58 smithi032 ceph-mon[40008]: pgmap v27: 65 pgs: 65 active+clean; 278 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.1 MiB/s rd, 11 MiB/s wr, 306 op/s 2024-05-12T04:01:00.786 INFO:tasks.workunit.client.1.smithi033.stdout:5/1: sync 2024-05-12T04:01:00.786 INFO:tasks.workunit.client.1.smithi033.stdout:2/60: sync 2024-05-12T04:01:00.786 INFO:tasks.workunit.client.1.smithi033.stdout:0/124: sync 2024-05-12T04:01:00.786 INFO:tasks.workunit.client.1.smithi033.stdout:1/4: sync 2024-05-12T04:01:00.786 INFO:tasks.workunit.client.1.smithi033.stdout:9/137: sync 2024-05-12T04:01:00.786 INFO:tasks.workunit.client.1.smithi033.stdout:6/10: sync 2024-05-12T04:01:00.786 INFO:tasks.workunit.client.1.smithi033.stdout:6/11: fdatasync - no filename 2024-05-12T04:01:00.786 INFO:tasks.workunit.client.1.smithi033.stdout:6/12: write - no filename 2024-05-12T04:01:00.786 INFO:tasks.workunit.client.1.smithi033.stdout:7/65: sync 2024-05-12T04:01:00.787 INFO:tasks.workunit.client.1.smithi033.stdout:3/73: sync 2024-05-12T04:01:00.787 INFO:tasks.workunit.client.1.smithi033.stdout:4/30: sync 2024-05-12T04:01:00.787 INFO:tasks.workunit.client.1.smithi033.stdout:8/37: sync 2024-05-12T04:01:00.787 INFO:tasks.workunit.client.1.smithi033.stdout:2/61: fdatasync f3 0 2024-05-12T04:01:00.788 INFO:tasks.workunit.client.1.smithi033.stdout:6/13: symlink l1 0 2024-05-12T04:01:00.789 INFO:tasks.workunit.client.1.smithi033.stdout:6/14: write - no filename 2024-05-12T04:01:00.789 INFO:tasks.workunit.client.1.smithi033.stdout:6/15: dread - no filename 2024-05-12T04:01:00.789 INFO:tasks.workunit.client.1.smithi033.stdout:6/16: readlink l1 0 2024-05-12T04:01:00.791 INFO:tasks.workunit.client.1.smithi033.stdout:7/66: creat de/df/f18 x:0 0 0 2024-05-12T04:01:00.792 INFO:tasks.workunit.client.1.smithi033.stdout:7/67: fsync de/df/f15 0 2024-05-12T04:01:00.792 INFO:tasks.workunit.client.1.smithi033.stdout:7/68: stat f3 0 2024-05-12T04:01:00.792 INFO:tasks.workunit.client.1.smithi033.stdout:5/2: mknod c0 0 2024-05-12T04:01:00.793 INFO:tasks.workunit.client.1.smithi033.stdout:7/69: dread - de/df/f15 zero size 2024-05-12T04:01:00.794 INFO:tasks.workunit.client.1.smithi033.stdout:6/17: creat f2 x:0 0 0 2024-05-12T04:01:00.794 INFO:tasks.workunit.client.1.smithi033.stdout:6/18: write f2 [919767,48456] 0 2024-05-12T04:01:00.803 INFO:tasks.workunit.client.1.smithi033.stdout:6/19: dwrite f2 [0,4194304] 0 2024-05-12T04:01:00.804 INFO:tasks.workunit.client.1.smithi033.stdout:3/74: fsync db/d17/f1c 0 2024-05-12T04:01:00.804 INFO:tasks.workunit.client.1.smithi033.stdout:4/31: read f3 [1830590,40591] 0 2024-05-12T04:01:00.806 INFO:tasks.workunit.client.1.smithi033.stdout:8/38: write d1/d4/f7 [261772,60120] 0 2024-05-12T04:01:00.807 INFO:tasks.workunit.client.1.smithi033.stdout:5/3: unlink c0 0 2024-05-12T04:01:00.807 INFO:tasks.workunit.client.1.smithi033.stdout:5/4: dwrite - no filename 2024-05-12T04:01:00.807 INFO:tasks.workunit.client.1.smithi033.stdout:5/5: link - no file 2024-05-12T04:01:00.807 INFO:tasks.workunit.client.1.smithi033.stdout:7/70: dread f3 [4194304,4194304] 0 2024-05-12T04:01:00.807 INFO:tasks.workunit.client.1.smithi033.stdout:5/6: rmdir - no directory 2024-05-12T04:01:00.810 INFO:tasks.workunit.client.1.smithi033.stdout:9/138: mknod d2/d4/d29/d32/c34 0 2024-05-12T04:01:00.812 INFO:tasks.workunit.client.1.smithi033.stdout:6/20: symlink l3 0 2024-05-12T04:01:00.813 INFO:tasks.workunit.client.1.smithi033.stdout:4/32: mknod c6 0 2024-05-12T04:01:00.814 INFO:tasks.workunit.client.1.smithi033.stdout:9/139: fdatasync d2/dd/f11 0 2024-05-12T04:01:00.815 INFO:tasks.workunit.client.1.smithi033.stdout:3/75: mknod db/c1f 0 2024-05-12T04:01:00.815 INFO:tasks.workunit.client.1.smithi033.stdout:7/71: write de/df/f10 [394035,128816] 0 2024-05-12T04:01:00.817 INFO:tasks.workunit.client.1.smithi033.stdout:5/7: creat f1 x:0 0 0 2024-05-12T04:01:00.818 INFO:tasks.workunit.client.1.smithi033.stdout:1/5: link c0 c1 0 2024-05-12T04:01:00.823 INFO:tasks.workunit.client.1.smithi033.stdout:7/72: write de/df/f10 [1230869,55549] 0 2024-05-12T04:01:00.824 INFO:tasks.workunit.client.1.smithi033.stdout:4/33: symlink l7 0 2024-05-12T04:01:00.829 INFO:tasks.workunit.client.1.smithi033.stdout:2/62: dwrite f9 [0,4194304] 0 2024-05-12T04:01:00.829 INFO:tasks.workunit.client.1.smithi033.stdout:0/125: dwrite d6/d15/f21 [0,4194304] 0 2024-05-12T04:01:00.829 INFO:tasks.workunit.client.1.smithi033.stdout:3/76: rmdir db 39 2024-05-12T04:01:00.830 INFO:tasks.workunit.client.1.smithi033.stdout:5/8: rename f1 to f2 0 2024-05-12T04:01:00.830 INFO:tasks.workunit.client.1.smithi033.stdout:9/140: symlink d2/d4/d29/l35 0 2024-05-12T04:01:00.834 INFO:tasks.workunit.client.1.smithi033.stdout:5/9: dread - f2 zero size 2024-05-12T04:01:00.836 INFO:tasks.workunit.client.1.smithi033.stdout:0/126: creat d6/d11/f2f x:0 0 0 2024-05-12T04:01:00.840 INFO:tasks.workunit.client.1.smithi033.stdout:3/77: write db/d17/f1c [670459,42783] 0 2024-05-12T04:01:00.843 INFO:tasks.workunit.client.1.smithi033.stdout:2/63: rename dc/cf to dc/d17/c19 0 2024-05-12T04:01:00.843 INFO:tasks.workunit.client.1.smithi033.stdout:8/39: dwrite d1/f2 [0,4194304] 0 2024-05-12T04:01:00.845 INFO:tasks.workunit.client.1.smithi033.stdout:2/64: chown dc/l12 88902917 1 2024-05-12T04:01:00.847 INFO:tasks.workunit.client.1.smithi033.stdout:5/10: mkdir d3 0 2024-05-12T04:01:00.849 INFO:tasks.workunit.client.1.smithi033.stdout:8/40: write d1/fa [645028,84250] 0 2024-05-12T04:01:00.852 INFO:tasks.workunit.client.1.smithi033.stdout:9/141: mknod d2/d4/d2b/d33/c36 0 2024-05-12T04:01:00.855 INFO:tasks.workunit.client.1.smithi033.stdout:4/34: dwrite f5 [0,4194304] 0 2024-05-12T04:01:00.857 INFO:tasks.workunit.client.1.smithi033.stdout:5/11: mkdir d3/d4 0 2024-05-12T04:01:00.858 INFO:tasks.workunit.client.1.smithi033.stdout:5/12: chown d3 82 1 2024-05-12T04:01:00.858 INFO:tasks.workunit.client.1.smithi033.stdout:8/41: write d1/d4/f7 [1154902,62385] 0 2024-05-12T04:01:00.861 INFO:tasks.workunit.client.1.smithi033.stdout:4/35: fsync f3 0 2024-05-12T04:01:00.862 INFO:tasks.workunit.client.1.smithi033.stdout:4/36: chown c1 27859 1 2024-05-12T04:01:00.863 INFO:tasks.workunit.client.1.smithi033.stdout:9/142: dread d2/d4/f15 [0,4194304] 0 2024-05-12T04:01:00.865 INFO:tasks.workunit.client.1.smithi033.stdout:9/143: symlink d2/d4/d29/d32/l37 0 2024-05-12T04:01:00.867 INFO:tasks.workunit.client.1.smithi033.stdout:4/37: dread f5 [0,4194304] 0 2024-05-12T04:01:00.871 INFO:tasks.workunit.client.1.smithi033.stdout:4/38: rename c2 to c8 0 2024-05-12T04:01:00.873 INFO:tasks.workunit.client.1.smithi033.stdout:9/144: getdents d2/d2d 0 2024-05-12T04:01:00.875 INFO:tasks.workunit.client.1.smithi033.stdout:4/39: link f3 f9 0 2024-05-12T04:01:00.877 INFO:tasks.workunit.client.1.smithi033.stdout:4/40: unlink f5 0 2024-05-12T04:01:00.878 INFO:tasks.workunit.client.1.smithi033.stdout:4/41: chown l4 93040810 1 2024-05-12T04:01:00.878 INFO:tasks.workunit.client.1.smithi033.stdout:4/42: chown l4 69 1 2024-05-12T04:01:00.879 INFO:tasks.workunit.client.1.smithi033.stdout:9/145: creat d2/d2d/d2f/f38 x:0 0 0 2024-05-12T04:01:00.881 INFO:tasks.workunit.client.1.smithi033.stdout:4/43: creat fa x:0 0 0 2024-05-12T04:01:00.882 INFO:tasks.workunit.client.1.smithi033.stdout:9/146: mkdir d2/dd/d39 0 2024-05-12T04:01:00.883 INFO:tasks.workunit.client.1.smithi033.stdout:4/44: creat fb x:0 0 0 2024-05-12T04:01:00.884 INFO:tasks.workunit.client.1.smithi033.stdout:9/147: mkdir d2/d2d/d3a 0 2024-05-12T04:01:00.885 INFO:tasks.workunit.client.1.smithi033.stdout:4/45: fdatasync f9 0 2024-05-12T04:01:00.887 INFO:tasks.workunit.client.1.smithi033.stdout:4/46: unlink fb 0 2024-05-12T04:01:00.889 INFO:tasks.workunit.client.1.smithi033.stdout:4/47: creat fc x:0 0 0 2024-05-12T04:01:00.892 INFO:tasks.workunit.client.1.smithi033.stdout:4/48: mkdir dd 0 2024-05-12T04:01:00.892 INFO:tasks.workunit.client.1.smithi033.stdout:9/148: dwrite d2/d4/ff [8388608,4194304] 0 2024-05-12T04:01:00.893 INFO:tasks.workunit.client.1.smithi033.stdout:4/49: truncate fc 227571 0 2024-05-12T04:01:00.898 INFO:tasks.workunit.client.1.smithi033.stdout:4/50: creat dd/fe x:0 0 0 2024-05-12T04:01:00.899 INFO:tasks.workunit.client.1.smithi033.stdout:4/51: write f3 [3423960,130484] 0 2024-05-12T04:01:00.899 INFO:tasks.workunit.client.1.smithi033.stdout:4/52: readlink l7 0 2024-05-12T04:01:00.902 INFO:tasks.workunit.client.1.smithi033.stdout:4/53: mknod dd/cf 0 2024-05-12T04:01:00.904 INFO:tasks.workunit.client.1.smithi033.stdout:4/54: unlink c1 0 2024-05-12T04:01:00.907 INFO:tasks.workunit.client.1.smithi033.stdout:4/55: unlink dd/fe 0 2024-05-12T04:01:01.028 INFO:tasks.workunit.client.1.smithi033.stdout:7/73: rmdir de/df 39 2024-05-12T04:01:01.029 INFO:tasks.workunit.client.1.smithi033.stdout:6/21: fsync f2 0 2024-05-12T04:01:01.030 INFO:tasks.workunit.client.1.smithi033.stdout:6/22: readlink l3 0 2024-05-12T04:01:01.032 INFO:tasks.workunit.client.1.smithi033.stdout:7/74: rename la to de/df/l19 0 2024-05-12T04:01:01.033 INFO:tasks.workunit.client.1.smithi033.stdout:6/23: mkdir d4 0 2024-05-12T04:01:01.034 INFO:tasks.workunit.client.1.smithi033.stdout:1/6: sync 2024-05-12T04:01:01.037 INFO:tasks.workunit.client.1.smithi033.stdout:7/75: dread de/df/f10 [0,4194304] 0 2024-05-12T04:01:01.039 INFO:tasks.workunit.client.1.smithi033.stdout:1/7: rename c0 to c2 0 2024-05-12T04:01:01.040 INFO:tasks.workunit.client.1.smithi033.stdout:6/24: dwrite f2 [0,4194304] 0 2024-05-12T04:01:01.041 INFO:tasks.workunit.client.1.smithi033.stdout:1/8: creat f3 x:0 0 0 2024-05-12T04:01:01.042 INFO:tasks.workunit.client.1.smithi033.stdout:1/9: dread - f3 zero size 2024-05-12T04:01:01.044 INFO:tasks.workunit.client.1.smithi033.stdout:7/76: creat de/f1a x:0 0 0 2024-05-12T04:01:01.045 INFO:tasks.workunit.client.1.smithi033.stdout:6/25: mkdir d4/d5 0 2024-05-12T04:01:01.047 INFO:tasks.workunit.client.1.smithi033.stdout:1/10: creat f4 x:0 0 0 2024-05-12T04:01:01.048 INFO:tasks.workunit.client.1.smithi033.stdout:0/127: fsync d6/d11/f2f 0 2024-05-12T04:01:01.049 INFO:tasks.workunit.client.1.smithi033.stdout:6/26: symlink d4/l6 0 2024-05-12T04:01:01.052 INFO:tasks.workunit.client.1.smithi033.stdout:2/65: getdents dc 0 2024-05-12T04:01:01.052 INFO:tasks.workunit.client.1.smithi033.stdout:2/66: read - f1 zero size 2024-05-12T04:01:01.053 INFO:tasks.workunit.client.1.smithi033.stdout:7/77: mkdir de/df/d17/d1b 0 2024-05-12T04:01:01.053 INFO:tasks.workunit.client.1.smithi033.stdout:2/67: chown dc/l11 11 1 2024-05-12T04:01:01.054 INFO:tasks.workunit.client.1.smithi033.stdout:2/68: write dc/d13/f16 [726292,48333] 0 2024-05-12T04:01:01.055 INFO:tasks.workunit.client.1.smithi033.stdout:0/128: symlink d6/db/l30 0 2024-05-12T04:01:01.059 INFO:tasks.workunit.client.1.smithi033.stdout:7/78: rename de/df/f15 to de/df/d17/d1b/f1c 0 2024-05-12T04:01:01.062 INFO:tasks.workunit.client.1.smithi033.stdout:3/78: dwrite db/f16 [0,4194304] 0 2024-05-12T04:01:01.065 INFO:tasks.workunit.client.1.smithi033.stdout:7/79: symlink de/df/d17/d1b/l1d 0 2024-05-12T04:01:01.067 INFO:tasks.workunit.client.1.smithi033.stdout:3/79: rename db/d17/f18 to db/f20 0 2024-05-12T04:01:01.069 INFO:tasks.workunit.client.1.smithi033.stdout:7/80: mkdir de/df/d17/d1e 0 2024-05-12T04:01:01.071 INFO:tasks.workunit.client.1.smithi033.stdout:3/80: write db/f20 [1115790,116985] 0 2024-05-12T04:01:01.071 INFO:tasks.workunit.client.1.smithi033.stdout:0/129: getdents d6/dc 0 2024-05-12T04:01:01.072 INFO:tasks.workunit.client.1.smithi033.stdout:0/130: chown d6/d15 362075 1 2024-05-12T04:01:01.075 INFO:tasks.workunit.client.1.smithi033.stdout:3/81: unlink db/f16 0 2024-05-12T04:01:01.076 INFO:tasks.workunit.client.1.smithi033.stdout:0/131: fdatasync d6/d11/f1c 0 2024-05-12T04:01:01.076 INFO:tasks.workunit.client.1.smithi033.stdout:3/82: truncate db/f20 2275105 0 2024-05-12T04:01:01.077 INFO:tasks.workunit.client.1.smithi033.stdout:3/83: stat db/d17 0 2024-05-12T04:01:01.081 INFO:tasks.workunit.client.1.smithi033.stdout:4/56: rmdir dd 39 2024-05-12T04:01:01.082 INFO:tasks.workunit.client.1.smithi033.stdout:3/84: dread f6 [0,4194304] 0 2024-05-12T04:01:01.083 INFO:tasks.workunit.client.1.smithi033.stdout:3/85: mknod db/c21 0 2024-05-12T04:01:01.084 INFO:tasks.workunit.client.1.smithi033.stdout:3/86: write db/d17/f1b [628621,65033] 0 2024-05-12T04:01:01.087 INFO:tasks.workunit.client.1.smithi033.stdout:3/87: creat db/f22 x:0 0 0 2024-05-12T04:01:01.087 INFO:tasks.workunit.client.1.smithi033.stdout:3/88: stat l5 0 2024-05-12T04:01:01.088 INFO:tasks.workunit.client.1.smithi033.stdout:1/11: chown c2 833 1 2024-05-12T04:01:01.089 INFO:tasks.workunit.client.1.smithi033.stdout:1/12: rmdir - no directory 2024-05-12T04:01:01.089 INFO:tasks.workunit.client.1.smithi033.stdout:3/89: creat db/f23 x:0 0 0 2024-05-12T04:01:01.096 INFO:tasks.workunit.client.1.smithi033.stdout:7/81: rename de/df/d17/d1b/f1c to de/df/f1f 0 2024-05-12T04:01:01.097 INFO:tasks.workunit.client.1.smithi033.stdout:9/149: truncate d2/d4/f30 87718 0 2024-05-12T04:01:01.098 INFO:tasks.workunit.client.1.smithi033.stdout:6/27: truncate f2 1657207 0 2024-05-12T04:01:01.099 INFO:tasks.workunit.client.1.smithi033.stdout:7/82: write f3 [3999260,67151] 0 2024-05-12T04:01:01.099 INFO:tasks.workunit.client.1.smithi033.stdout:7/83: dread - de/df/f18 zero size 2024-05-12T04:01:01.101 INFO:tasks.workunit.client.1.smithi033.stdout:7/84: truncate de/df/f11 4814159 0 2024-05-12T04:01:01.101 INFO:tasks.workunit.client.1.smithi033.stdout:0/132: rename d6/f7 to d6/db/d2c/f31 0 2024-05-12T04:01:01.103 INFO:tasks.workunit.client.1.smithi033.stdout:9/150: symlink d2/dd/l3b 0 2024-05-12T04:01:01.103 INFO:tasks.workunit.client.1.smithi033.stdout:9/151: stat d2/l23 0 2024-05-12T04:01:01.104 INFO:tasks.workunit.client.1.smithi033.stdout:3/90: getdents db/d17 0 2024-05-12T04:01:01.104 INFO:tasks.workunit.client.1.smithi033.stdout:9/152: read - d2/d2d/d2f/f38 zero size 2024-05-12T04:01:01.108 INFO:tasks.workunit.client.1.smithi033.stdout:0/133: symlink d6/db/d1f/l32 0 2024-05-12T04:01:01.111 INFO:tasks.workunit.client.1.smithi033.stdout:3/91: rename db/lf to db/d17/l24 0 2024-05-12T04:01:01.112 INFO:tasks.workunit.client.1.smithi033.stdout:9/153: mkdir d2/d4/d29/d3c 0 2024-05-12T04:01:01.113 INFO:tasks.workunit.client.1.smithi033.stdout:9/154: stat d2/dd/lb 0 2024-05-12T04:01:01.113 INFO:tasks.workunit.client.1.smithi033.stdout:9/155: write d2/f24 [727928,118809] 0 2024-05-12T04:01:01.115 INFO:tasks.workunit.client.1.smithi033.stdout:4/57: truncate f9 2187300 0 2024-05-12T04:01:01.117 INFO:tasks.workunit.client.1.smithi033.stdout:0/134: creat d6/d11/f33 x:0 0 0 2024-05-12T04:01:01.120 INFO:tasks.workunit.client.1.smithi033.stdout:3/92: mkdir db/d17/d25 0 2024-05-12T04:01:01.122 INFO:tasks.workunit.client.1.smithi033.stdout:3/93: truncate db/d17/f1b 1498580 0 2024-05-12T04:01:01.125 INFO:tasks.workunit.client.1.smithi033.stdout:0/135: rmdir d6/db 39 2024-05-12T04:01:01.128 INFO:tasks.workunit.client.1.smithi033.stdout:9/156: dwrite d2/d4/f15 [0,4194304] 0 2024-05-12T04:01:01.131 INFO:tasks.workunit.client.1.smithi033.stdout:7/85: truncate f3 974232 0 2024-05-12T04:01:01.133 INFO:tasks.workunit.client.1.smithi033.stdout:4/58: getdents dd 0 2024-05-12T04:01:01.136 INFO:tasks.workunit.client.1.smithi033.stdout:9/157: getdents d2/dd/d39 0 2024-05-12T04:01:01.137 INFO:tasks.workunit.client.1.smithi033.stdout:3/94: truncate f6 3343606 0 2024-05-12T04:01:01.139 INFO:tasks.workunit.client.1.smithi033.stdout:0/136: dwrite d6/db/d2c/f31 [0,4194304] 0 2024-05-12T04:01:01.140 INFO:tasks.workunit.client.1.smithi033.stdout:9/158: mkdir d2/d4/d29/d3d 0 2024-05-12T04:01:01.142 INFO:tasks.workunit.client.1.smithi033.stdout:7/86: dread de/df/f11 [0,4194304] 0 2024-05-12T04:01:01.144 INFO:tasks.workunit.client.1.smithi033.stdout:9/159: symlink d2/d4/d29/l3e 0 2024-05-12T04:01:01.144 INFO:tasks.workunit.client.1.smithi033.stdout:7/87: chown cd 1 1 2024-05-12T04:01:01.145 INFO:tasks.workunit.client.1.smithi033.stdout:9/160: chown d2/d4/f2c 1083 1 2024-05-12T04:01:01.148 INFO:tasks.workunit.client.1.smithi033.stdout:3/95: link db/l10 db/l26 0 2024-05-12T04:01:01.149 INFO:tasks.workunit.client.1.smithi033.stdout:9/161: symlink d2/d2d/l3f 0 2024-05-12T04:01:01.151 INFO:tasks.workunit.client.1.smithi033.stdout:0/137: dwrite d6/dc/f16 [0,4194304] 0 2024-05-12T04:01:01.153 INFO:tasks.workunit.client.1.smithi033.stdout:7/88: truncate de/f12 919914 0 2024-05-12T04:01:01.153 INFO:tasks.workunit.client.1.smithi033.stdout:0/138: rename d6/db/d2c to d6/db/d2c/d34 22 2024-05-12T04:01:01.157 INFO:tasks.workunit.client.1.smithi033.stdout:9/162: dread d2/dd/f11 [0,4194304] 0 2024-05-12T04:01:01.158 INFO:tasks.workunit.client.1.smithi033.stdout:0/139: creat d6/d15/f35 x:0 0 0 2024-05-12T04:01:01.160 INFO:tasks.workunit.client.1.smithi033.stdout:3/96: chown l7 1 1 2024-05-12T04:01:01.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:00 smithi033 ceph-mon[49525]: pgmap v28: 65 pgs: 65 active+clean; 278 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 10 MiB/s wr, 213 op/s 2024-05-12T04:01:01.161 INFO:tasks.workunit.client.1.smithi033.stdout:7/89: mkdir de/d20 0 2024-05-12T04:01:01.162 INFO:tasks.workunit.client.1.smithi033.stdout:3/97: chown db/c14 2890 1 2024-05-12T04:01:01.166 INFO:tasks.workunit.client.1.smithi033.stdout:9/163: dwrite d2/dd/f26 [0,4194304] 0 2024-05-12T04:01:01.169 INFO:tasks.workunit.client.1.smithi033.stdout:7/90: unlink de/df/f11 0 2024-05-12T04:01:01.169 INFO:tasks.workunit.client.1.smithi033.stdout:4/59: dread f9 [0,4194304] 0 2024-05-12T04:01:01.169 INFO:tasks.workunit.client.1.smithi033.stdout:4/60: dread - fa zero size 2024-05-12T04:01:01.171 INFO:tasks.workunit.client.1.smithi033.stdout:9/164: dread d2/dd/f11 [0,4194304] 0 2024-05-12T04:01:01.171 INFO:tasks.workunit.client.1.smithi033.stdout:5/13: sync 2024-05-12T04:01:01.171 INFO:tasks.workunit.client.1.smithi033.stdout:5/14: readlink - no filename 2024-05-12T04:01:01.171 INFO:tasks.workunit.client.1.smithi033.stdout:0/140: symlink d6/l36 0 2024-05-12T04:01:01.171 INFO:tasks.workunit.client.1.smithi033.stdout:0/141: stat d6/dc/f18 0 2024-05-12T04:01:01.175 INFO:tasks.workunit.client.1.smithi033.stdout:4/61: truncate f9 2366046 0 2024-05-12T04:01:01.177 INFO:tasks.workunit.client.1.smithi033.stdout:5/15: write f2 [133322,79666] 0 2024-05-12T04:01:01.177 INFO:tasks.workunit.client.1.smithi033.stdout:5/16: chown d3 1157389 1 2024-05-12T04:01:01.179 INFO:tasks.workunit.client.1.smithi033.stdout:0/142: mknod d6/d15/c37 0 2024-05-12T04:01:01.180 INFO:tasks.workunit.client.1.smithi033.stdout:7/91: creat de/d20/f21 x:0 0 0 2024-05-12T04:01:01.181 INFO:tasks.workunit.client.1.smithi033.stdout:7/92: fdatasync de/f1a 0 2024-05-12T04:01:01.182 INFO:tasks.workunit.client.1.smithi033.stdout:7/93: chown de/df/d17/d1b/l1d 10241 1 2024-05-12T04:01:01.184 INFO:tasks.workunit.client.1.smithi033.stdout:9/165: link d2/d4/ff d2/d4/d29/d32/f40 0 2024-05-12T04:01:01.184 INFO:tasks.workunit.client.1.smithi033.stdout:0/143: dread d6/d11/f28 [0,4194304] 0 2024-05-12T04:01:01.185 INFO:tasks.workunit.client.1.smithi033.stdout:5/17: mknod d3/c5 0 2024-05-12T04:01:01.185 INFO:tasks.workunit.client.1.smithi033.stdout:7/94: fdatasync de/df/f18 0 2024-05-12T04:01:01.186 INFO:tasks.workunit.client.1.smithi033.stdout:9/166: stat d2/l1e 0 2024-05-12T04:01:01.186 INFO:tasks.workunit.client.1.smithi033.stdout:4/62: link c6 dd/c10 0 2024-05-12T04:01:01.188 INFO:tasks.workunit.client.1.smithi033.stdout:5/18: mknod d3/c6 0 2024-05-12T04:01:01.188 INFO:tasks.workunit.client.1.smithi033.stdout:0/144: chown d6/dc/f16 10 1 2024-05-12T04:01:01.188 INFO:tasks.workunit.client.1.smithi033.stdout:9/167: chown d2/d4/d2b/d33 315594 1 2024-05-12T04:01:01.189 INFO:tasks.workunit.client.1.smithi033.stdout:4/63: mkdir dd/d11 0 2024-05-12T04:01:01.190 INFO:tasks.workunit.client.1.smithi033.stdout:3/98: dwrite db/f12 [0,4194304] 0 2024-05-12T04:01:01.191 INFO:tasks.workunit.client.1.smithi033.stdout:0/145: chown d6/d11/f33 52 1 2024-05-12T04:01:01.191 INFO:tasks.workunit.client.1.smithi033.stdout:9/168: dread - d2/d2d/d2f/f38 zero size 2024-05-12T04:01:01.191 INFO:tasks.workunit.client.1.smithi033.stdout:5/19: rename f2 to d3/f7 0 2024-05-12T04:01:01.196 INFO:tasks.workunit.client.1.smithi033.stdout:9/169: unlink d2/d4/l12 0 2024-05-12T04:01:01.199 INFO:tasks.workunit.client.1.smithi033.stdout:3/99: mknod db/d17/d25/c27 0 2024-05-12T04:01:01.199 INFO:tasks.workunit.client.1.smithi033.stdout:9/170: rename d2/d2d/d2f to d2/d2d/d2f/d41 22 2024-05-12T04:01:01.199 INFO:tasks.workunit.client.1.smithi033.stdout:0/146: dwrite f4 [0,4194304] 0 2024-05-12T04:01:01.201 INFO:tasks.workunit.client.1.smithi033.stdout:5/20: rmdir d3/d4 0 2024-05-12T04:01:01.201 INFO:tasks.workunit.client.1.smithi033.stdout:3/100: write db/f22 [167483,118436] 0 2024-05-12T04:01:01.203 INFO:tasks.workunit.client.1.smithi033.stdout:5/21: chown d3/f7 65138627 1 2024-05-12T04:01:01.204 INFO:tasks.workunit.client.1.smithi033.stdout:9/171: rename d2/dd/l3b to d2/d4/l42 0 2024-05-12T04:01:01.208 INFO:tasks.workunit.client.1.smithi033.stdout:5/22: write d3/f7 [205821,26619] 0 2024-05-12T04:01:01.208 INFO:tasks.workunit.client.1.smithi033.stdout:9/172: creat d2/dd/d39/f43 x:0 0 0 2024-05-12T04:01:01.210 INFO:tasks.workunit.client.1.smithi033.stdout:5/23: rmdir d3 39 2024-05-12T04:01:01.214 INFO:tasks.workunit.client.1.smithi033.stdout:0/147: dwrite d6/f23 [0,4194304] 0 2024-05-12T04:01:01.215 INFO:tasks.workunit.client.1.smithi033.stdout:3/101: dread f6 [0,4194304] 0 2024-05-12T04:01:01.215 INFO:tasks.workunit.client.1.smithi033.stdout:5/24: chown d3/c6 342528919 1 2024-05-12T04:01:01.220 INFO:tasks.workunit.client.1.smithi033.stdout:3/102: mknod db/d17/d25/c28 0 2024-05-12T04:01:01.221 INFO:tasks.workunit.client.1.smithi033.stdout:5/25: chown d3/f7 1363054 1 2024-05-12T04:01:01.222 INFO:tasks.workunit.client.1.smithi033.stdout:9/173: dwrite d2/d4/f25 [4194304,4194304] 0 2024-05-12T04:01:01.223 INFO:tasks.workunit.client.1.smithi033.stdout:3/103: chown db/l26 10 1 2024-05-12T04:01:01.227 INFO:tasks.workunit.client.1.smithi033.stdout:3/104: symlink db/l29 0 2024-05-12T04:01:01.229 INFO:tasks.workunit.client.1.smithi033.stdout:9/174: truncate d2/d4/f1f 2499445 0 2024-05-12T04:01:01.229 INFO:tasks.workunit.client.1.smithi033.stdout:4/64: dwrite f3 [0,4194304] 0 2024-05-12T04:01:01.232 INFO:tasks.workunit.client.1.smithi033.stdout:3/105: symlink db/l2a 0 2024-05-12T04:01:01.234 INFO:tasks.workunit.client.1.smithi033.stdout:4/65: truncate fc 1155113 0 2024-05-12T04:01:01.234 INFO:tasks.workunit.client.1.smithi033.stdout:9/175: write d2/fa [1179747,112647] 0 2024-05-12T04:01:01.237 INFO:tasks.workunit.client.1.smithi033.stdout:3/106: write db/d17/f1c [1676880,57835] 0 2024-05-12T04:01:01.239 INFO:tasks.workunit.client.1.smithi033.stdout:5/26: dwrite d3/f7 [0,4194304] 0 2024-05-12T04:01:01.239 INFO:tasks.workunit.client.1.smithi033.stdout:5/27: readlink - no filename 2024-05-12T04:01:01.242 INFO:tasks.workunit.client.1.smithi033.stdout:0/148: dwrite d6/d11/f28 [0,4194304] 0 2024-05-12T04:01:01.246 INFO:tasks.workunit.client.1.smithi033.stdout:3/107: rename db/c21 to db/d17/d25/c2b 0 2024-05-12T04:01:01.246 INFO:tasks.workunit.client.1.smithi033.stdout:0/149: write d6/f23 [4503326,50136] 0 2024-05-12T04:01:01.249 INFO:tasks.workunit.client.1.smithi033.stdout:3/108: chown l5 7414830 1 2024-05-12T04:01:01.249 INFO:tasks.workunit.client.1.smithi033.stdout:0/150: stat d6/dc 0 2024-05-12T04:01:01.253 INFO:tasks.workunit.client.1.smithi033.stdout:4/66: dwrite f3 [0,4194304] 0 2024-05-12T04:01:01.254 INFO:tasks.workunit.client.1.smithi033.stdout:4/67: readlink l0 0 2024-05-12T04:01:01.256 INFO:tasks.workunit.client.1.smithi033.stdout:4/68: readlink l0 0 2024-05-12T04:01:01.256 INFO:tasks.workunit.client.1.smithi033.stdout:5/28: dwrite d3/f7 [0,4194304] 0 2024-05-12T04:01:01.258 INFO:tasks.workunit.client.1.smithi033.stdout:0/151: rename d6/dc/l22 to d6/l38 0 2024-05-12T04:01:01.260 INFO:tasks.workunit.client.1.smithi033.stdout:3/109: dread db/d17/f1b [0,4194304] 0 2024-05-12T04:01:01.262 INFO:tasks.workunit.client.1.smithi033.stdout:5/29: symlink d3/l8 0 2024-05-12T04:01:01.263 INFO:tasks.workunit.client.1.smithi033.stdout:0/152: mkdir d6/dc/d39 0 2024-05-12T04:01:01.265 INFO:tasks.workunit.client.1.smithi033.stdout:4/69: dwrite f9 [0,4194304] 0 2024-05-12T04:01:01.265 INFO:tasks.workunit.client.1.smithi033.stdout:5/30: creat d3/f9 x:0 0 0 2024-05-12T04:01:01.266 INFO:tasks.workunit.client.1.smithi033.stdout:0/153: write d6/dc/f1d [284389,23108] 0 2024-05-12T04:01:01.269 INFO:tasks.workunit.client.1.smithi033.stdout:0/154: symlink d6/db/d2c/l3a 0 2024-05-12T04:01:01.270 INFO:tasks.workunit.client.1.smithi033.stdout:0/155: write d6/dc/f27 [340890,86772] 0 2024-05-12T04:01:01.270 INFO:tasks.workunit.client.1.smithi033.stdout:5/31: mknod d3/ca 0 2024-05-12T04:01:01.271 INFO:tasks.workunit.client.1.smithi033.stdout:5/32: stat d3/c5 0 2024-05-12T04:01:01.272 INFO:tasks.workunit.client.1.smithi033.stdout:4/70: creat dd/d11/f12 x:0 0 0 2024-05-12T04:01:01.275 INFO:tasks.workunit.client.1.smithi033.stdout:0/156: mknod d6/d11/c3b 0 2024-05-12T04:01:01.275 INFO:tasks.workunit.client.1.smithi033.stdout:5/33: creat d3/fb x:0 0 0 2024-05-12T04:01:01.277 INFO:tasks.workunit.client.1.smithi033.stdout:0/157: write d6/d11/f2f [932572,34719] 0 2024-05-12T04:01:01.278 INFO:tasks.workunit.client.1.smithi033.stdout:5/34: fsync d3/f9 0 2024-05-12T04:01:01.278 INFO:tasks.workunit.client.1.smithi033.stdout:0/158: read - d6/d11/f33 zero size 2024-05-12T04:01:01.279 INFO:tasks.workunit.client.1.smithi033.stdout:3/110: truncate db/d17/f1b 317593 0 2024-05-12T04:01:01.280 INFO:tasks.workunit.client.1.smithi033.stdout:4/71: dwrite f3 [0,4194304] 0 2024-05-12T04:01:01.282 INFO:tasks.workunit.client.1.smithi033.stdout:0/159: creat d6/dc/f3c x:0 0 0 2024-05-12T04:01:01.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:00 smithi032 ceph-mon[40008]: pgmap v28: 65 pgs: 65 active+clean; 278 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 10 MiB/s wr, 213 op/s 2024-05-12T04:01:01.313 INFO:tasks.workunit.client.1.smithi033.stdout:1/13: sync 2024-05-12T04:01:01.313 INFO:tasks.workunit.client.1.smithi033.stdout:8/42: sync 2024-05-12T04:01:01.314 INFO:tasks.workunit.client.1.smithi033.stdout:2/69: sync 2024-05-12T04:01:01.341 INFO:tasks.workunit.client.1.smithi033.stdout:2/70: dread - dc/d13/f15 zero size 2024-05-12T04:01:01.342 INFO:tasks.workunit.client.1.smithi033.stdout:0/160: dread f2 [4194304,4194304] 0 2024-05-12T04:01:01.344 INFO:tasks.workunit.client.1.smithi033.stdout:2/71: dread dc/d13/f14 [0,4194304] 0 2024-05-12T04:01:01.345 INFO:tasks.workunit.client.1.smithi033.stdout:8/43: dread d1/d4/f7 [0,4194304] 0 2024-05-12T04:01:01.346 INFO:tasks.workunit.client.1.smithi033.stdout:1/14: dwrite f4 [0,4194304] 0 2024-05-12T04:01:01.348 INFO:tasks.workunit.client.1.smithi033.stdout:1/15: write f3 [424705,86371] 0 2024-05-12T04:01:01.348 INFO:tasks.workunit.client.1.smithi033.stdout:1/16: readlink - no filename 2024-05-12T04:01:01.348 INFO:tasks.workunit.client.1.smithi033.stdout:1/17: truncate f3 1128328 0 2024-05-12T04:01:01.375 INFO:tasks.workunit.client.1.smithi033.stdout:5/35: getdents d3 0 2024-05-12T04:01:01.376 INFO:tasks.workunit.client.1.smithi033.stdout:4/72: link dd/d11/f12 dd/f13 0 2024-05-12T04:01:01.387 INFO:tasks.workunit.client.1.smithi033.stdout:1/18: symlink l5 0 2024-05-12T04:01:01.387 INFO:tasks.workunit.client.1.smithi033.stdout:5/36: creat d3/fc x:0 0 0 2024-05-12T04:01:01.387 INFO:tasks.workunit.client.1.smithi033.stdout:1/19: chown c2 54885 1 2024-05-12T04:01:01.387 INFO:tasks.workunit.client.1.smithi033.stdout:5/37: chown d3/fb 11675282 1 2024-05-12T04:01:01.387 INFO:tasks.workunit.client.1.smithi033.stdout:5/38: chown d3/ca 75020 1 2024-05-12T04:01:01.390 INFO:tasks.workunit.client.1.smithi033.stdout:4/73: getdents dd/d11 0 2024-05-12T04:01:01.391 INFO:tasks.workunit.client.1.smithi033.stdout:1/20: dwrite f3 [0,4194304] 0 2024-05-12T04:01:01.395 INFO:tasks.workunit.client.1.smithi033.stdout:1/21: symlink l6 0 2024-05-12T04:01:01.396 INFO:tasks.workunit.client.1.smithi033.stdout:1/22: truncate f3 4745311 0 2024-05-12T04:01:01.397 INFO:tasks.workunit.client.1.smithi033.stdout:1/23: chown c1 320341 1 2024-05-12T04:01:01.401 INFO:tasks.workunit.client.1.smithi033.stdout:4/74: dwrite dd/d11/f12 [0,4194304] 0 2024-05-12T04:01:01.403 INFO:tasks.workunit.client.1.smithi033.stdout:4/75: rmdir dd 39 2024-05-12T04:01:01.403 INFO:tasks.workunit.client.1.smithi033.stdout:1/24: dread f4 [0,4194304] 0 2024-05-12T04:01:01.404 INFO:tasks.workunit.client.1.smithi033.stdout:4/76: truncate f3 4956084 0 2024-05-12T04:01:01.407 INFO:tasks.workunit.client.1.smithi033.stdout:1/25: rename l6 to l7 0 2024-05-12T04:01:01.410 INFO:tasks.workunit.client.1.smithi033.stdout:4/77: getdents dd/d11 0 2024-05-12T04:01:01.414 INFO:tasks.workunit.client.1.smithi033.stdout:4/78: dread dd/d11/f12 [0,4194304] 0 2024-05-12T04:01:01.415 INFO:tasks.workunit.client.1.smithi033.stdout:4/79: chown l0 827846002 1 2024-05-12T04:01:01.415 INFO:tasks.workunit.client.1.smithi033.stdout:4/80: chown fc 20000 1 2024-05-12T04:01:01.418 INFO:tasks.workunit.client.1.smithi033.stdout:4/81: link fc dd/d11/f14 0 2024-05-12T04:01:01.426 INFO:tasks.workunit.client.1.smithi033.stdout:3/111: link db/d17/f1b db/d17/f2c 0 2024-05-12T04:01:01.430 INFO:tasks.workunit.client.1.smithi033.stdout:5/39: getdents d3 0 2024-05-12T04:01:01.432 INFO:tasks.workunit.client.1.smithi033.stdout:4/82: write dd/d11/f12 [5078688,114894] 0 2024-05-12T04:01:01.433 INFO:tasks.workunit.client.1.smithi033.stdout:1/26: getdents . 0 2024-05-12T04:01:01.436 INFO:tasks.workunit.client.1.smithi033.stdout:5/40: truncate d3/f7 4303021 0 2024-05-12T04:01:01.437 INFO:tasks.workunit.client.1.smithi033.stdout:4/83: mkdir dd/d11/d15 0 2024-05-12T04:01:01.437 INFO:tasks.workunit.client.1.smithi033.stdout:3/112: write f6 [1649736,87039] 0 2024-05-12T04:01:01.438 INFO:tasks.workunit.client.1.smithi033.stdout:1/27: creat f8 x:0 0 0 2024-05-12T04:01:01.439 INFO:tasks.workunit.client.1.smithi033.stdout:0/161: dwrite d6/d15/f21 [4194304,4194304] 0 2024-05-12T04:01:01.441 INFO:tasks.workunit.client.1.smithi033.stdout:2/72: dwrite dc/f10 [0,4194304] 0 2024-05-12T04:01:01.441 INFO:tasks.workunit.client.1.smithi033.stdout:1/28: creat f9 x:0 0 0 2024-05-12T04:01:01.443 INFO:tasks.workunit.client.1.smithi033.stdout:0/162: stat d6/d15/l17 0 2024-05-12T04:01:01.444 INFO:tasks.workunit.client.1.smithi033.stdout:2/73: truncate dc/d13/f16 1789008 0 2024-05-12T04:01:01.444 INFO:tasks.workunit.client.1.smithi033.stdout:4/84: mknod dd/c16 0 2024-05-12T04:01:01.444 INFO:tasks.workunit.client.1.smithi033.stdout:5/41: rename d3/c6 to d3/cd 0 2024-05-12T04:01:01.446 INFO:tasks.workunit.client.1.smithi033.stdout:1/29: fdatasync f4 0 2024-05-12T04:01:01.447 INFO:tasks.workunit.client.1.smithi033.stdout:1/30: stat c2 0 2024-05-12T04:01:01.448 INFO:tasks.workunit.client.1.smithi033.stdout:2/74: write dc/f10 [1852753,126872] 0 2024-05-12T04:01:01.452 INFO:tasks.workunit.client.1.smithi033.stdout:2/75: creat dc/f1a x:0 0 0 2024-05-12T04:01:01.452 INFO:tasks.workunit.client.1.smithi033.stdout:0/163: symlink d6/dc/d39/l3d 0 2024-05-12T04:01:01.453 INFO:tasks.workunit.client.1.smithi033.stdout:5/42: link d3/ca d3/ce 0 2024-05-12T04:01:01.455 INFO:tasks.workunit.client.1.smithi033.stdout:0/164: fdatasync d6/d15/f35 0 2024-05-12T04:01:01.457 INFO:tasks.workunit.client.1.smithi033.stdout:0/165: read - d6/db/f13 zero size 2024-05-12T04:01:01.458 INFO:tasks.workunit.client.1.smithi033.stdout:5/43: creat d3/ff x:0 0 0 2024-05-12T04:01:01.459 INFO:tasks.workunit.client.1.smithi033.stdout:1/31: dwrite f4 [4194304,4194304] 0 2024-05-12T04:01:01.459 INFO:tasks.workunit.client.1.smithi033.stdout:1/32: rmdir - no directory 2024-05-12T04:01:01.459 INFO:tasks.workunit.client.1.smithi033.stdout:5/44: read - d3/fb zero size 2024-05-12T04:01:01.460 INFO:tasks.workunit.client.1.smithi033.stdout:1/33: chown f9 6 1 2024-05-12T04:01:01.462 INFO:tasks.workunit.client.1.smithi033.stdout:0/166: rename d6/d15/f35 to d6/db/d2c/f3e 0 2024-05-12T04:01:01.465 INFO:tasks.workunit.client.1.smithi033.stdout:5/45: creat d3/f10 x:0 0 0 2024-05-12T04:01:01.466 INFO:tasks.workunit.client.1.smithi033.stdout:3/113: write db/d17/f2c [655944,97775] 0 2024-05-12T04:01:01.468 INFO:tasks.workunit.client.1.smithi033.stdout:5/46: write d3/f7 [6527,29841] 0 2024-05-12T04:01:01.468 INFO:tasks.workunit.client.1.smithi033.stdout:4/85: write dd/d11/f14 [2121004,25573] 0 2024-05-12T04:01:01.473 INFO:tasks.workunit.client.1.smithi033.stdout:5/47: write d3/f7 [3334577,84379] 0 2024-05-12T04:01:01.474 INFO:tasks.workunit.client.1.smithi033.stdout:4/86: creat dd/f17 x:0 0 0 2024-05-12T04:01:01.474 INFO:tasks.workunit.client.1.smithi033.stdout:1/34: fsync f4 0 2024-05-12T04:01:01.477 INFO:tasks.workunit.client.1.smithi033.stdout:0/167: dwrite d6/d11/f1c [0,4194304] 0 2024-05-12T04:01:01.477 INFO:tasks.workunit.client.1.smithi033.stdout:4/87: fdatasync dd/f13 0 2024-05-12T04:01:01.478 INFO:tasks.workunit.client.1.smithi033.stdout:5/48: mknod d3/c11 0 2024-05-12T04:01:01.479 INFO:tasks.workunit.client.1.smithi033.stdout:4/88: chown dd 336476 1 2024-05-12T04:01:01.480 INFO:tasks.workunit.client.1.smithi033.stdout:1/35: rename f3 to fa 0 2024-05-12T04:01:01.480 INFO:tasks.workunit.client.1.smithi033.stdout:4/89: stat l7 0 2024-05-12T04:01:01.482 INFO:tasks.workunit.client.1.smithi033.stdout:3/114: dwrite db/d17/f1c [0,4194304] 0 2024-05-12T04:01:01.486 INFO:tasks.workunit.client.1.smithi033.stdout:4/90: unlink dd/c10 0 2024-05-12T04:01:01.489 INFO:tasks.workunit.client.1.smithi033.stdout:6/28: sync 2024-05-12T04:01:01.491 INFO:tasks.workunit.client.1.smithi033.stdout:7/95: sync 2024-05-12T04:01:01.495 INFO:tasks.workunit.client.1.smithi033.stdout:1/36: link c1 cb 0 2024-05-12T04:01:01.496 INFO:tasks.workunit.client.1.smithi033.stdout:7/96: read de/df/f10 [268006,2730] 0 2024-05-12T04:01:01.497 INFO:tasks.workunit.client.1.smithi033.stdout:0/168: getdents d6/dc 0 2024-05-12T04:01:01.497 INFO:tasks.workunit.client.1.smithi033.stdout:5/49: dwrite d3/f10 [0,4194304] 0 2024-05-12T04:01:01.498 INFO:tasks.workunit.client.1.smithi033.stdout:6/29: mknod d4/d5/c7 0 2024-05-12T04:01:01.499 INFO:tasks.workunit.client.1.smithi033.stdout:1/37: truncate f9 65604 0 2024-05-12T04:01:01.501 INFO:tasks.workunit.client.1.smithi033.stdout:3/115: dread db/f12 [0,4194304] 0 2024-05-12T04:01:01.501 INFO:tasks.workunit.client.1.smithi033.stdout:0/169: mknod d6/dc/c3f 0 2024-05-12T04:01:01.504 INFO:tasks.workunit.client.1.smithi033.stdout:6/30: creat d4/d5/f8 x:0 0 0 2024-05-12T04:01:01.504 INFO:tasks.workunit.client.1.smithi033.stdout:7/97: write de/df/f10 [406220,75266] 0 2024-05-12T04:01:01.506 INFO:tasks.workunit.client.1.smithi033.stdout:7/98: stat l5 0 2024-05-12T04:01:01.509 INFO:tasks.workunit.client.1.smithi033.stdout:4/91: dwrite dd/f13 [4194304,4194304] 0 2024-05-12T04:01:01.509 INFO:tasks.workunit.client.1.smithi033.stdout:3/116: creat db/d17/f2d x:0 0 0 2024-05-12T04:01:01.509 INFO:tasks.workunit.client.1.smithi033.stdout:6/31: creat d4/d5/f9 x:0 0 0 2024-05-12T04:01:01.509 INFO:tasks.workunit.client.1.smithi033.stdout:1/38: link l7 lc 0 2024-05-12T04:01:01.511 INFO:tasks.workunit.client.1.smithi033.stdout:7/99: write de/f1a [62140,67733] 0 2024-05-12T04:01:01.514 INFO:tasks.workunit.client.1.smithi033.stdout:6/32: creat d4/fa x:0 0 0 2024-05-12T04:01:01.516 INFO:tasks.workunit.client.1.smithi033.stdout:1/39: write fa [5532253,25359] 0 2024-05-12T04:01:01.517 INFO:tasks.workunit.client.1.smithi033.stdout:3/117: read db/d17/f2c [285980,96769] 0 2024-05-12T04:01:01.518 INFO:tasks.workunit.client.1.smithi033.stdout:5/50: dwrite d3/fc [0,4194304] 0 2024-05-12T04:01:01.518 INFO:tasks.workunit.client.1.smithi033.stdout:6/33: write d4/d5/f8 [777445,19787] 0 2024-05-12T04:01:01.520 INFO:tasks.workunit.client.1.smithi033.stdout:5/51: stat d3/f9 0 2024-05-12T04:01:01.520 INFO:tasks.workunit.client.1.smithi033.stdout:6/34: truncate f2 1720971 0 2024-05-12T04:01:01.522 INFO:tasks.workunit.client.1.smithi033.stdout:7/100: rename de/df/f18 to de/df/d17/f22 0 2024-05-12T04:01:01.525 INFO:tasks.workunit.client.1.smithi033.stdout:6/35: write d4/d5/f8 [1213663,110072] 0 2024-05-12T04:01:01.526 INFO:tasks.workunit.client.1.smithi033.stdout:5/52: chown d3/ca 1288 1 2024-05-12T04:01:01.527 INFO:tasks.workunit.client.1.smithi033.stdout:6/36: chown f2 1 1 2024-05-12T04:01:01.527 INFO:tasks.workunit.client.1.smithi033.stdout:1/40: dread fa [0,4194304] 0 2024-05-12T04:01:01.527 INFO:tasks.workunit.client.1.smithi033.stdout:7/101: getdents de/df/d17/d1e 0 2024-05-12T04:01:01.528 INFO:tasks.workunit.client.1.smithi033.stdout:5/53: mkdir d3/d12 0 2024-05-12T04:01:01.530 INFO:tasks.workunit.client.1.smithi033.stdout:6/37: write d4/fa [725390,25733] 0 2024-05-12T04:01:01.531 INFO:tasks.workunit.client.1.smithi033.stdout:7/102: chown f3 137531653 1 2024-05-12T04:01:01.531 INFO:tasks.workunit.client.1.smithi033.stdout:1/41: unlink f4 0 2024-05-12T04:01:01.531 INFO:tasks.workunit.client.1.smithi033.stdout:3/118: dwrite db/f22 [0,4194304] 0 2024-05-12T04:01:01.533 INFO:tasks.workunit.client.1.smithi033.stdout:1/42: dread f9 [0,4194304] 0 2024-05-12T04:01:01.534 INFO:tasks.workunit.client.1.smithi033.stdout:6/38: write d4/d5/f8 [1824848,101281] 0 2024-05-12T04:01:01.535 INFO:tasks.workunit.client.1.smithi033.stdout:7/103: creat de/df/d17/d1e/f23 x:0 0 0 2024-05-12T04:01:01.536 INFO:tasks.workunit.client.1.smithi033.stdout:6/39: write d4/d5/f9 [519093,121744] 0 2024-05-12T04:01:01.538 INFO:tasks.workunit.client.1.smithi033.stdout:3/119: rename db/d17/c1a to db/d17/d25/c2e 0 2024-05-12T04:01:01.540 INFO:tasks.workunit.client.1.smithi033.stdout:1/43: rename f9 to fd 0 2024-05-12T04:01:01.542 INFO:tasks.workunit.client.1.smithi033.stdout:7/104: mkdir de/df/d17/d1e/d24 0 2024-05-12T04:01:01.542 INFO:tasks.workunit.client.1.smithi033.stdout:7/105: chown de/df 78409 1 2024-05-12T04:01:01.543 INFO:tasks.workunit.client.1.smithi033.stdout:1/44: symlink le 0 2024-05-12T04:01:01.544 INFO:tasks.workunit.client.1.smithi033.stdout:3/120: dread db/d17/f1b [0,4194304] 0 2024-05-12T04:01:01.545 INFO:tasks.workunit.client.1.smithi033.stdout:7/106: dread de/f12 [0,4194304] 0 2024-05-12T04:01:01.545 INFO:tasks.workunit.client.1.smithi033.stdout:3/121: write db/d17/f1c [2730869,123039] 0 2024-05-12T04:01:01.549 INFO:tasks.workunit.client.1.smithi033.stdout:1/45: link c1 cf 0 2024-05-12T04:01:01.551 INFO:tasks.workunit.client.1.smithi033.stdout:1/46: rename c1 to c10 0 2024-05-12T04:01:01.566 INFO:tasks.workunit.client.1.smithi033.stdout:5/54: fdatasync d3/fc 0 2024-05-12T04:01:01.567 INFO:tasks.workunit.client.1.smithi033.stdout:4/92: truncate f9 1406134 0 2024-05-12T04:01:01.568 INFO:tasks.workunit.client.1.smithi033.stdout:4/93: dread - fa zero size 2024-05-12T04:01:01.568 INFO:tasks.workunit.client.1.smithi033.stdout:4/94: readlink l7 0 2024-05-12T04:01:01.569 INFO:tasks.workunit.client.1.smithi033.stdout:6/40: dread d4/fa [0,4194304] 0 2024-05-12T04:01:01.569 INFO:tasks.workunit.client.1.smithi033.stdout:4/95: dread - dd/f17 zero size 2024-05-12T04:01:01.570 INFO:tasks.workunit.client.1.smithi033.stdout:4/96: chown dd/d11/f12 49922653 1 2024-05-12T04:01:01.570 INFO:tasks.workunit.client.1.smithi033.stdout:5/55: mkdir d3/d12/d13 0 2024-05-12T04:01:01.571 INFO:tasks.workunit.client.1.smithi033.stdout:6/41: symlink d4/d5/lb 0 2024-05-12T04:01:01.573 INFO:tasks.workunit.client.1.smithi033.stdout:6/42: write d4/d5/f8 [961691,29212] 0 2024-05-12T04:01:01.574 INFO:tasks.workunit.client.1.smithi033.stdout:4/97: mknod dd/d11/d15/c18 0 2024-05-12T04:01:01.575 INFO:tasks.workunit.client.1.smithi033.stdout:3/122: rmdir db 39 2024-05-12T04:01:01.576 INFO:tasks.workunit.client.1.smithi033.stdout:6/43: write d4/d5/f8 [980773,1733] 0 2024-05-12T04:01:01.577 INFO:tasks.workunit.client.1.smithi033.stdout:6/44: fsync d4/d5/f9 0 2024-05-12T04:01:01.578 INFO:tasks.workunit.client.1.smithi033.stdout:6/45: stat d4/fa 0 2024-05-12T04:01:01.578 INFO:tasks.workunit.client.1.smithi033.stdout:5/56: dwrite d3/ff [0,4194304] 0 2024-05-12T04:01:01.579 INFO:tasks.workunit.client.1.smithi033.stdout:7/107: rmdir de 39 2024-05-12T04:01:01.580 INFO:tasks.workunit.client.1.smithi033.stdout:5/57: write d3/f10 [3178975,42274] 0 2024-05-12T04:01:01.583 INFO:tasks.workunit.client.1.smithi033.stdout:3/123: fdatasync db/f22 0 2024-05-12T04:01:01.584 INFO:tasks.workunit.client.1.smithi033.stdout:3/124: dread - db/f23 zero size 2024-05-12T04:01:01.584 INFO:tasks.workunit.client.1.smithi033.stdout:5/58: fsync d3/f10 0 2024-05-12T04:01:01.585 INFO:tasks.workunit.client.1.smithi033.stdout:5/59: write d3/fb [798538,125704] 0 2024-05-12T04:01:01.586 INFO:tasks.workunit.client.1.smithi033.stdout:1/47: chown c10 5025 1 2024-05-12T04:01:01.587 INFO:tasks.workunit.client.1.smithi033.stdout:5/60: truncate d3/ff 4911214 0 2024-05-12T04:01:01.587 INFO:tasks.workunit.client.1.smithi033.stdout:1/48: write f8 [635098,12704] 0 2024-05-12T04:01:01.588 INFO:tasks.workunit.client.1.smithi033.stdout:5/61: write d3/f9 [1036464,57663] 0 2024-05-12T04:01:01.590 INFO:tasks.workunit.client.1.smithi033.stdout:5/62: fsync d3/f10 0 2024-05-12T04:01:01.592 INFO:tasks.workunit.client.1.smithi033.stdout:7/108: read - de/d20/f21 zero size 2024-05-12T04:01:01.592 INFO:tasks.workunit.client.1.smithi033.stdout:5/63: chown d3/f7 2 1 2024-05-12T04:01:01.593 INFO:tasks.workunit.client.1.smithi033.stdout:3/125: mknod db/c2f 0 2024-05-12T04:01:01.594 INFO:tasks.workunit.client.1.smithi033.stdout:1/49: dread f8 [0,4194304] 0 2024-05-12T04:01:01.595 INFO:tasks.workunit.client.1.smithi033.stdout:7/109: truncate de/df/f10 2273093 0 2024-05-12T04:01:01.599 INFO:tasks.workunit.client.1.smithi033.stdout:1/50: fdatasync fa 0 2024-05-12T04:01:01.600 INFO:tasks.workunit.client.1.smithi033.stdout:6/46: link d4/d5/c7 d4/cc 0 2024-05-12T04:01:01.602 INFO:tasks.workunit.client.1.smithi033.stdout:6/47: readlink l3 0 2024-05-12T04:01:01.602 INFO:tasks.workunit.client.1.smithi033.stdout:1/51: mkdir d11 0 2024-05-12T04:01:01.604 INFO:tasks.workunit.client.1.smithi033.stdout:5/64: getdents d3/d12 0 2024-05-12T04:01:01.606 INFO:tasks.workunit.client.1.smithi033.stdout:1/52: chown f8 1584 1 2024-05-12T04:01:01.606 INFO:tasks.workunit.client.1.smithi033.stdout:6/48: mknod d4/d5/cd 0 2024-05-12T04:01:01.606 INFO:tasks.workunit.client.1.smithi033.stdout:3/126: dwrite db/d17/f1c [4194304,4194304] 0 2024-05-12T04:01:01.607 INFO:tasks.workunit.client.1.smithi033.stdout:4/98: truncate f9 1644622 0 2024-05-12T04:01:01.608 INFO:tasks.workunit.client.1.smithi033.stdout:1/53: readlink le 0 2024-05-12T04:01:01.609 INFO:tasks.workunit.client.1.smithi033.stdout:5/65: creat d3/d12/f14 x:0 0 0 2024-05-12T04:01:01.610 INFO:tasks.workunit.client.1.smithi033.stdout:7/110: dwrite de/df/f10 [0,4194304] 0 2024-05-12T04:01:01.612 INFO:tasks.workunit.client.1.smithi033.stdout:3/127: creat db/d17/d25/f30 x:0 0 0 2024-05-12T04:01:01.612 INFO:tasks.workunit.client.1.smithi033.stdout:6/49: rmdir d4 39 2024-05-12T04:01:01.615 INFO:tasks.workunit.client.1.smithi033.stdout:1/54: truncate f8 620512 0 2024-05-12T04:01:01.616 INFO:tasks.workunit.client.1.smithi033.stdout:5/66: mkdir d3/d15 0 2024-05-12T04:01:01.618 INFO:tasks.workunit.client.1.smithi033.stdout:6/50: fdatasync d4/d5/f9 0 2024-05-12T04:01:01.619 INFO:tasks.workunit.client.1.smithi033.stdout:6/51: readlink d4/d5/lb 0 2024-05-12T04:01:01.620 INFO:tasks.workunit.client.1.smithi033.stdout:3/128: rmdir db/d17 39 2024-05-12T04:01:01.622 INFO:tasks.workunit.client.1.smithi033.stdout:6/52: rmdir d4 39 2024-05-12T04:01:01.623 INFO:tasks.workunit.client.1.smithi033.stdout:9/176: sync 2024-05-12T04:01:01.625 INFO:tasks.workunit.client.1.smithi033.stdout:7/111: creat de/f25 x:0 0 0 2024-05-12T04:01:01.625 INFO:tasks.workunit.client.1.smithi033.stdout:1/55: stat f8 0 2024-05-12T04:01:01.625 INFO:tasks.workunit.client.1.smithi033.stdout:5/67: mkdir d3/d15/d16 0 2024-05-12T04:01:01.630 INFO:tasks.workunit.client.1.smithi033.stdout:9/177: rename d2/d4/f25 to d2/d4/d29/d32/f44 0 2024-05-12T04:01:01.630 INFO:tasks.workunit.client.1.smithi033.stdout:4/99: truncate fc 1775784 0 2024-05-12T04:01:01.633 INFO:tasks.workunit.client.1.smithi033.stdout:4/100: dread - fa zero size 2024-05-12T04:01:01.634 INFO:tasks.workunit.client.1.smithi033.stdout:6/53: symlink d4/d5/le 0 2024-05-12T04:01:01.634 INFO:tasks.workunit.client.1.smithi033.stdout:4/101: dread - dd/f17 zero size 2024-05-12T04:01:01.637 INFO:tasks.workunit.client.1.smithi033.stdout:1/56: mknod d11/c12 0 2024-05-12T04:01:01.637 INFO:tasks.workunit.client.1.smithi033.stdout:9/178: readlink d2/d4/l42 0 2024-05-12T04:01:01.638 INFO:tasks.workunit.client.1.smithi033.stdout:5/68: dwrite d3/f9 [0,4194304] 0 2024-05-12T04:01:01.639 INFO:tasks.workunit.client.1.smithi033.stdout:1/57: chown cf 10993673 1 2024-05-12T04:01:01.645 INFO:tasks.workunit.client.1.smithi033.stdout:3/129: getdents db/d17 0 2024-05-12T04:01:01.648 INFO:tasks.workunit.client.1.smithi033.stdout:9/179: creat d2/d2d/d3a/f45 x:0 0 0 2024-05-12T04:01:01.648 INFO:tasks.workunit.client.1.smithi033.stdout:7/112: dwrite f3 [0,4194304] 0 2024-05-12T04:01:01.649 INFO:tasks.workunit.client.1.smithi033.stdout:9/180: stat d2/d4/d2b/d33/c36 0 2024-05-12T04:01:01.653 INFO:tasks.workunit.client.1.smithi033.stdout:6/54: getdents d4/d5 0 2024-05-12T04:01:01.656 INFO:tasks.workunit.client.1.smithi033.stdout:3/130: link db/f20 db/f31 0 2024-05-12T04:01:01.657 INFO:tasks.workunit.client.1.smithi033.stdout:9/181: link d2/d4/f10 d2/d4/d29/f46 0 2024-05-12T04:01:01.658 INFO:tasks.workunit.client.1.smithi033.stdout:6/55: truncate d4/d5/f8 2922304 0 2024-05-12T04:01:01.659 INFO:tasks.workunit.client.1.smithi033.stdout:1/58: write f8 [266706,58674] 0 2024-05-12T04:01:01.660 INFO:tasks.workunit.client.1.smithi033.stdout:3/131: read db/d17/f2c [742973,65458] 0 2024-05-12T04:01:01.665 INFO:tasks.workunit.client.1.smithi033.stdout:9/182: dread d2/d4/d29/f2a [0,4194304] 0 2024-05-12T04:01:01.665 INFO:tasks.workunit.client.1.smithi033.stdout:7/113: dwrite de/d20/f21 [0,4194304] 0 2024-05-12T04:01:01.666 INFO:tasks.workunit.client.1.smithi033.stdout:6/56: dread d4/d5/f9 [0,4194304] 0 2024-05-12T04:01:01.668 INFO:tasks.workunit.client.1.smithi033.stdout:9/183: write d2/d4/f28 [98896,16582] 0 2024-05-12T04:01:01.674 INFO:tasks.workunit.client.1.smithi033.stdout:3/132: dread db/d17/f1c [4194304,4194304] 0 2024-05-12T04:01:01.675 INFO:tasks.workunit.client.1.smithi033.stdout:6/57: link d4/d5/c7 d4/cf 0 2024-05-12T04:01:01.678 INFO:tasks.workunit.client.1.smithi033.stdout:3/133: readlink db/l2a 0 2024-05-12T04:01:01.679 INFO:tasks.workunit.client.1.smithi033.stdout:4/102: dwrite dd/d11/f14 [0,4194304] 0 2024-05-12T04:01:01.679 INFO:tasks.workunit.client.1.smithi033.stdout:6/58: fdatasync d4/d5/f9 0 2024-05-12T04:01:01.684 INFO:tasks.workunit.client.1.smithi033.stdout:4/103: stat dd/d11/d15/c18 0 2024-05-12T04:01:01.685 INFO:tasks.workunit.client.1.smithi033.stdout:7/114: dwrite de/df/d17/f22 [0,4194304] 0 2024-05-12T04:01:01.685 INFO:tasks.workunit.client.1.smithi033.stdout:3/134: mkdir db/d17/d25/d32 0 2024-05-12T04:01:01.685 INFO:tasks.workunit.client.1.smithi033.stdout:9/184: dread d2/fa [0,4194304] 0 2024-05-12T04:01:01.686 INFO:tasks.workunit.client.1.smithi033.stdout:6/59: fsync d4/fa 0 2024-05-12T04:01:01.686 INFO:tasks.workunit.client.1.smithi033.stdout:3/135: chown db/d17/f2d 127828467 1 2024-05-12T04:01:01.687 INFO:tasks.workunit.client.1.smithi033.stdout:6/60: fsync d4/d5/f8 0 2024-05-12T04:01:01.688 INFO:tasks.workunit.client.1.smithi033.stdout:4/104: creat dd/d11/f19 x:0 0 0 2024-05-12T04:01:01.689 INFO:tasks.workunit.client.1.smithi033.stdout:6/61: write d4/fa [43270,75781] 0 2024-05-12T04:01:01.690 INFO:tasks.workunit.client.1.smithi033.stdout:3/136: symlink db/d17/l33 0 2024-05-12T04:01:01.691 INFO:tasks.workunit.client.1.smithi033.stdout:4/105: write dd/d11/f14 [1747795,83554] 0 2024-05-12T04:01:01.693 INFO:tasks.workunit.client.1.smithi033.stdout:7/115: creat de/f26 x:0 0 0 2024-05-12T04:01:01.697 INFO:tasks.workunit.client.1.smithi033.stdout:6/62: creat d4/d5/f10 x:0 0 0 2024-05-12T04:01:01.697 INFO:tasks.workunit.client.1.smithi033.stdout:4/106: symlink dd/l1a 0 2024-05-12T04:01:01.698 INFO:tasks.workunit.client.1.smithi033.stdout:7/116: rename de/df/d17 to de/df/d17/d27 22 2024-05-12T04:01:01.699 INFO:tasks.workunit.client.1.smithi033.stdout:3/137: mknod db/d17/d25/d32/c34 0 2024-05-12T04:01:01.700 INFO:tasks.workunit.client.1.smithi033.stdout:7/117: dread - de/f25 zero size 2024-05-12T04:01:01.700 INFO:tasks.workunit.client.1.smithi033.stdout:6/63: symlink d4/l11 0 2024-05-12T04:01:01.702 INFO:tasks.workunit.client.1.smithi033.stdout:9/185: dwrite d2/d2d/d2f/f38 [0,4194304] 0 2024-05-12T04:01:01.703 INFO:tasks.workunit.client.1.smithi033.stdout:7/118: stat de/df/d17/f22 0 2024-05-12T04:01:01.704 INFO:tasks.workunit.client.1.smithi033.stdout:6/64: mknod d4/c12 0 2024-05-12T04:01:01.707 INFO:tasks.workunit.client.1.smithi033.stdout:6/65: write d4/d5/f10 [169420,102187] 0 2024-05-12T04:01:01.709 INFO:tasks.workunit.client.1.smithi033.stdout:4/107: dwrite dd/f17 [0,4194304] 0 2024-05-12T04:01:01.710 INFO:tasks.workunit.client.1.smithi033.stdout:7/119: mkdir de/df/d17/d1b/d28 0 2024-05-12T04:01:01.711 INFO:tasks.workunit.client.1.smithi033.stdout:3/138: link db/d17/d25/d32/c34 db/d17/c35 0 2024-05-12T04:01:01.713 INFO:tasks.workunit.client.1.smithi033.stdout:9/186: creat d2/f47 x:0 0 0 2024-05-12T04:01:01.717 INFO:tasks.workunit.client.1.smithi033.stdout:4/108: rename dd/f17 to dd/f1b 0 2024-05-12T04:01:01.719 INFO:tasks.workunit.client.1.smithi033.stdout:7/120: creat de/df/f29 x:0 0 0 2024-05-12T04:01:01.722 INFO:tasks.workunit.client.1.smithi033.stdout:9/187: creat d2/d2d/f48 x:0 0 0 2024-05-12T04:01:01.725 INFO:tasks.workunit.client.1.smithi033.stdout:4/109: symlink dd/d11/d15/l1c 0 2024-05-12T04:01:01.726 INFO:tasks.workunit.client.1.smithi033.stdout:9/188: mkdir d2/d4/d2b/d49 0 2024-05-12T04:01:01.727 INFO:tasks.workunit.client.1.smithi033.stdout:4/110: write fc [3750738,40702] 0 2024-05-12T04:01:01.728 INFO:tasks.workunit.client.1.smithi033.stdout:7/121: dwrite de/df/d17/f22 [0,4194304] 0 2024-05-12T04:01:01.728 INFO:tasks.workunit.client.1.smithi033.stdout:3/139: write db/f31 [1357864,10516] 0 2024-05-12T04:01:01.729 INFO:tasks.workunit.client.1.smithi033.stdout:9/189: write d2/f47 [167978,97121] 0 2024-05-12T04:01:01.730 INFO:tasks.workunit.client.1.smithi033.stdout:6/66: fsync f2 0 2024-05-12T04:01:01.731 INFO:tasks.workunit.client.1.smithi033.stdout:4/111: creat dd/d11/f1d x:0 0 0 2024-05-12T04:01:01.732 INFO:tasks.workunit.client.1.smithi033.stdout:3/140: creat db/d17/f36 x:0 0 0 2024-05-12T04:01:01.735 INFO:tasks.workunit.client.1.smithi033.stdout:7/122: symlink de/df/d17/d1b/l2a 0 2024-05-12T04:01:01.735 INFO:tasks.workunit.client.1.smithi033.stdout:6/67: dread d4/d5/f10 [0,4194304] 0 2024-05-12T04:01:01.735 INFO:tasks.workunit.client.1.smithi033.stdout:4/112: dread f3 [0,4194304] 0 2024-05-12T04:01:01.736 INFO:tasks.workunit.client.1.smithi033.stdout:6/68: chown d4/l11 1597 1 2024-05-12T04:01:01.736 INFO:tasks.workunit.client.1.smithi033.stdout:7/123: chown de/f1a 50577 1 2024-05-12T04:01:01.737 INFO:tasks.workunit.client.1.smithi033.stdout:6/69: chown l3 0 1 2024-05-12T04:01:01.738 INFO:tasks.workunit.client.1.smithi033.stdout:3/141: mknod db/d17/d25/c37 0 2024-05-12T04:01:01.738 INFO:tasks.workunit.client.1.smithi033.stdout:7/124: dread - de/df/d17/d1e/f23 zero size 2024-05-12T04:01:01.739 INFO:tasks.workunit.client.1.smithi033.stdout:7/125: dread - de/df/d17/d1e/f23 zero size 2024-05-12T04:01:01.740 INFO:tasks.workunit.client.1.smithi033.stdout:7/126: truncate de/f25 829496 0 2024-05-12T04:01:01.741 INFO:tasks.workunit.client.1.smithi033.stdout:4/113: creat dd/d11/d15/f1e x:0 0 0 2024-05-12T04:01:01.741 INFO:tasks.workunit.client.1.smithi033.stdout:3/142: symlink db/d17/l38 0 2024-05-12T04:01:01.743 INFO:tasks.workunit.client.1.smithi033.stdout:4/114: dread - dd/d11/d15/f1e zero size 2024-05-12T04:01:01.745 INFO:tasks.workunit.client.1.smithi033.stdout:7/127: fdatasync de/f12 0 2024-05-12T04:01:01.746 INFO:tasks.workunit.client.1.smithi033.stdout:6/70: dread d4/d5/f8 [0,4194304] 0 2024-05-12T04:01:01.747 INFO:tasks.workunit.client.1.smithi033.stdout:3/143: unlink db/f12 0 2024-05-12T04:01:01.747 INFO:tasks.workunit.client.1.smithi033.stdout:6/71: dread d4/d5/f9 [0,4194304] 0 2024-05-12T04:01:01.748 INFO:tasks.workunit.client.1.smithi033.stdout:3/144: write db/d17/d25/f30 [796223,122351] 0 2024-05-12T04:01:01.749 INFO:tasks.workunit.client.1.smithi033.stdout:6/72: write d4/d5/f8 [3445448,15255] 0 2024-05-12T04:01:01.750 INFO:tasks.workunit.client.1.smithi033.stdout:9/190: truncate d2/d2d/d2f/f38 407154 0 2024-05-12T04:01:01.751 INFO:tasks.workunit.client.1.smithi033.stdout:7/128: mkdir de/d2b 0 2024-05-12T04:01:01.752 INFO:tasks.workunit.client.1.smithi033.stdout:7/129: truncate de/df/f29 169356 0 2024-05-12T04:01:01.753 INFO:tasks.workunit.client.1.smithi033.stdout:3/145: creat db/d17/d25/f39 x:0 0 0 2024-05-12T04:01:01.754 INFO:tasks.workunit.client.1.smithi033.stdout:6/73: mknod d4/d5/c13 0 2024-05-12T04:01:01.754 INFO:tasks.workunit.client.1.smithi033.stdout:6/74: fsync f2 0 2024-05-12T04:01:01.755 INFO:tasks.workunit.client.1.smithi033.stdout:6/75: truncate f2 2333299 0 2024-05-12T04:01:01.755 INFO:tasks.workunit.client.1.smithi033.stdout:9/191: symlink d2/d4/d2b/d33/l4a 0 2024-05-12T04:01:01.756 INFO:tasks.workunit.client.1.smithi033.stdout:3/146: mkdir db/d17/d3a 0 2024-05-12T04:01:01.757 INFO:tasks.workunit.client.1.smithi033.stdout:7/130: creat de/d2b/f2c x:0 0 0 2024-05-12T04:01:01.757 INFO:tasks.workunit.client.1.smithi033.stdout:9/192: creat d2/d4/d2b/f4b x:0 0 0 2024-05-12T04:01:01.758 INFO:tasks.workunit.client.1.smithi033.stdout:3/147: write db/d17/d25/f39 [28936,50843] 0 2024-05-12T04:01:01.759 INFO:tasks.workunit.client.1.smithi033.stdout:9/193: readlink d2/dd/lb 0 2024-05-12T04:01:01.761 INFO:tasks.workunit.client.1.smithi033.stdout:7/131: symlink de/df/d17/d1e/d24/l2d 0 2024-05-12T04:01:01.765 INFO:tasks.workunit.client.1.smithi033.stdout:6/76: dwrite d4/d5/f9 [0,4194304] 0 2024-05-12T04:01:01.765 INFO:tasks.workunit.client.1.smithi033.stdout:9/194: dread d2/d4/f30 [0,4194304] 0 2024-05-12T04:01:01.768 INFO:tasks.workunit.client.1.smithi033.stdout:7/132: symlink de/df/d17/d1e/d24/l2e 0 2024-05-12T04:01:01.769 INFO:tasks.workunit.client.1.smithi033.stdout:6/77: unlink l1 0 2024-05-12T04:01:01.770 INFO:tasks.workunit.client.1.smithi033.stdout:7/133: dread - de/f26 zero size 2024-05-12T04:01:01.770 INFO:tasks.workunit.client.1.smithi033.stdout:3/148: dwrite db/d17/f36 [0,4194304] 0 2024-05-12T04:01:01.772 INFO:tasks.workunit.client.1.smithi033.stdout:9/195: truncate d2/dd/f13 1179975 0 2024-05-12T04:01:01.774 INFO:tasks.workunit.client.1.smithi033.stdout:6/78: stat d4/cf 0 2024-05-12T04:01:01.776 INFO:tasks.workunit.client.1.smithi033.stdout:6/79: chown d4/c12 64302041 1 2024-05-12T04:01:01.777 INFO:tasks.workunit.client.1.smithi033.stdout:7/134: write de/df/f1f [978206,80223] 0 2024-05-12T04:01:01.778 INFO:tasks.workunit.client.1.smithi033.stdout:7/135: chown de/d2b/f2c 153 1 2024-05-12T04:01:01.779 INFO:tasks.workunit.client.1.smithi033.stdout:9/196: mkdir d2/d2d/d4c 0 2024-05-12T04:01:01.780 INFO:tasks.workunit.client.1.smithi033.stdout:6/80: read d4/d5/f10 [239051,13929] 0 2024-05-12T04:01:01.781 INFO:tasks.workunit.client.1.smithi033.stdout:3/149: dread db/f20 [0,4194304] 0 2024-05-12T04:01:01.783 INFO:tasks.workunit.client.1.smithi033.stdout:7/136: mkdir de/d2f 0 2024-05-12T04:01:01.786 INFO:tasks.workunit.client.1.smithi033.stdout:3/150: rename db/f23 to db/d17/d3a/f3b 0 2024-05-12T04:01:01.787 INFO:tasks.workunit.client.1.smithi033.stdout:7/137: truncate de/df/f29 347744 0 2024-05-12T04:01:01.788 INFO:tasks.workunit.client.1.smithi033.stdout:3/151: fsync db/d17/d25/f39 0 2024-05-12T04:01:01.789 INFO:tasks.workunit.client.1.smithi033.stdout:9/197: link d2/dd/c1b d2/d4/d29/d32/c4d 0 2024-05-12T04:01:01.790 INFO:tasks.workunit.client.1.smithi033.stdout:6/81: dwrite d4/fa [0,4194304] 0 2024-05-12T04:01:01.790 INFO:tasks.workunit.client.1.smithi033.stdout:9/198: readlink d2/l1e 0 2024-05-12T04:01:01.791 INFO:tasks.workunit.client.1.smithi033.stdout:6/82: fsync d4/d5/f8 0 2024-05-12T04:01:01.792 INFO:tasks.workunit.client.1.smithi033.stdout:3/152: truncate db/d17/d3a/f3b 715021 0 2024-05-12T04:01:01.792 INFO:tasks.workunit.client.1.smithi033.stdout:7/138: unlink de/df/f10 0 2024-05-12T04:01:01.792 INFO:tasks.workunit.client.1.smithi033.stdout:7/139: stat f3 0 2024-05-12T04:01:01.794 INFO:tasks.workunit.client.1.smithi033.stdout:9/199: creat d2/dd/d39/f4e x:0 0 0 2024-05-12T04:01:01.795 INFO:tasks.workunit.client.1.smithi033.stdout:7/140: mknod de/df/d17/d1b/c30 0 2024-05-12T04:01:01.796 INFO:tasks.workunit.client.1.smithi033.stdout:3/153: mkdir db/d17/d25/d32/d3c 0 2024-05-12T04:01:01.802 INFO:tasks.workunit.client.1.smithi033.stdout:6/83: dwrite d4/d5/f8 [4194304,4194304] 0 2024-05-12T04:01:01.802 INFO:tasks.workunit.client.1.smithi033.stdout:9/200: rename d2/d4/d29/d32/f40 to d2/f4f 0 2024-05-12T04:01:01.802 INFO:tasks.workunit.client.1.smithi033.stdout:8/44: sync 2024-05-12T04:01:01.802 INFO:tasks.workunit.client.1.smithi033.stdout:2/76: sync 2024-05-12T04:01:01.803 INFO:tasks.workunit.client.1.smithi033.stdout:8/45: chown l0 2 1 2024-05-12T04:01:01.804 INFO:tasks.workunit.client.1.smithi033.stdout:8/46: stat d1/d4/f7 0 2024-05-12T04:01:01.805 INFO:tasks.workunit.client.1.smithi033.stdout:7/141: rename cd to de/df/d17/d1e/c31 0 2024-05-12T04:01:01.807 INFO:tasks.workunit.client.1.smithi033.stdout:6/84: unlink d4/d5/c13 0 2024-05-12T04:01:01.810 INFO:tasks.workunit.client.1.smithi033.stdout:8/47: mknod d1/d4/cc 0 2024-05-12T04:01:01.810 INFO:tasks.workunit.client.1.smithi033.stdout:6/85: truncate d4/d5/f9 4978781 0 2024-05-12T04:01:01.812 INFO:tasks.workunit.client.1.smithi033.stdout:9/201: dwrite d2/d4/f2c [0,4194304] 0 2024-05-12T04:01:01.813 INFO:tasks.workunit.client.1.smithi033.stdout:2/77: creat dc/d13/f1b x:0 0 0 2024-05-12T04:01:01.813 INFO:tasks.workunit.client.1.smithi033.stdout:6/86: dread d4/d5/f10 [0,4194304] 0 2024-05-12T04:01:01.814 INFO:tasks.workunit.client.1.smithi033.stdout:9/202: stat d2/f27 0 2024-05-12T04:01:01.814 INFO:tasks.workunit.client.1.smithi033.stdout:7/142: symlink de/l32 0 2024-05-12T04:01:01.815 INFO:tasks.workunit.client.1.smithi033.stdout:2/78: write f6 [2061029,78764] 0 2024-05-12T04:01:01.815 INFO:tasks.workunit.client.1.smithi033.stdout:9/203: chown d2/f27 2658 1 2024-05-12T04:01:01.817 INFO:tasks.workunit.client.1.smithi033.stdout:8/48: write d1/d4/f8 [1148008,124757] 0 2024-05-12T04:01:01.818 INFO:tasks.workunit.client.1.smithi033.stdout:2/79: dread - dc/f1a zero size 2024-05-12T04:01:01.818 INFO:tasks.workunit.client.1.smithi033.stdout:6/87: rename d4/d5/c7 to d4/d5/c14 0 2024-05-12T04:01:01.820 INFO:tasks.workunit.client.1.smithi033.stdout:9/204: truncate d2/dd/d39/f4e 491065 0 2024-05-12T04:01:01.822 INFO:tasks.workunit.client.1.smithi033.stdout:6/88: write d4/d5/f9 [5028106,58651] 0 2024-05-12T04:01:01.823 INFO:tasks.workunit.client.1.smithi033.stdout:2/80: mkdir dc/d13/d1c 0 2024-05-12T04:01:01.824 INFO:tasks.workunit.client.1.smithi033.stdout:6/89: fsync d4/fa 0 2024-05-12T04:01:01.828 INFO:tasks.workunit.client.1.smithi033.stdout:7/143: link de/l14 de/df/d17/l33 0 2024-05-12T04:01:01.830 INFO:tasks.workunit.client.1.smithi033.stdout:8/49: dwrite d1/fa [0,4194304] 0 2024-05-12T04:01:01.832 INFO:tasks.workunit.client.1.smithi033.stdout:6/90: rename d4/d5/cd to d4/d5/c15 0 2024-05-12T04:01:01.834 INFO:tasks.workunit.client.1.smithi033.stdout:8/50: unlink d1/fa 0 2024-05-12T04:01:01.835 INFO:tasks.workunit.client.1.smithi033.stdout:2/81: dwrite f6 [0,4194304] 0 2024-05-12T04:01:01.836 INFO:tasks.workunit.client.1.smithi033.stdout:8/51: chown d1/cb 0 1 2024-05-12T04:01:01.837 INFO:tasks.workunit.client.1.smithi033.stdout:7/144: rename de/df/d17/f22 to de/df/d17/f34 0 2024-05-12T04:01:01.841 INFO:tasks.workunit.client.1.smithi033.stdout:6/91: creat d4/f16 x:0 0 0 2024-05-12T04:01:01.842 INFO:tasks.workunit.client.1.smithi033.stdout:6/92: chown d4/d5/f10 7774 1 2024-05-12T04:01:01.844 INFO:tasks.workunit.client.1.smithi033.stdout:2/82: creat dc/d13/f1d x:0 0 0 2024-05-12T04:01:01.846 INFO:tasks.workunit.client.1.smithi033.stdout:2/83: truncate dc/d13/f16 2430909 0 2024-05-12T04:01:01.850 INFO:tasks.workunit.client.1.smithi033.stdout:6/93: dwrite d4/d5/f8 [0,4194304] 0 2024-05-12T04:01:01.853 INFO:tasks.workunit.client.1.smithi033.stdout:8/52: getdents d1/d4 0 2024-05-12T04:01:01.858 INFO:tasks.workunit.client.1.smithi033.stdout:6/94: dwrite f2 [0,4194304] 0 2024-05-12T04:01:01.862 INFO:tasks.workunit.client.1.smithi033.stdout:8/53: creat d1/d4/fd x:0 0 0 2024-05-12T04:01:01.865 INFO:tasks.workunit.client.1.smithi033.stdout:7/145: getdents de/df/d17 0 2024-05-12T04:01:01.865 INFO:tasks.workunit.client.1.smithi033.stdout:7/146: chown de/d2f 376830333 1 2024-05-12T04:01:01.866 INFO:tasks.workunit.client.1.smithi033.stdout:2/84: link dc/ce dc/d13/c1e 0 2024-05-12T04:01:01.869 INFO:tasks.workunit.client.1.smithi033.stdout:6/95: rename d4/cf to d4/d5/c17 0 2024-05-12T04:01:01.873 INFO:tasks.workunit.client.1.smithi033.stdout:9/205: stat d2/dd/f13 0 2024-05-12T04:01:01.873 INFO:tasks.workunit.client.1.smithi033.stdout:7/147: dwrite de/f26 [0,4194304] 0 2024-05-12T04:01:01.878 INFO:tasks.workunit.client.1.smithi033.stdout:9/206: mknod d2/d4/d29/d3d/c50 0 2024-05-12T04:01:01.880 INFO:tasks.workunit.client.1.smithi033.stdout:6/96: dwrite d4/d5/f8 [4194304,4194304] 0 2024-05-12T04:01:01.880 INFO:tasks.workunit.client.1.smithi033.stdout:9/207: chown d2/d4/d2b/d33/l4a 1 1 2024-05-12T04:01:01.882 INFO:tasks.workunit.client.1.smithi033.stdout:9/208: chown d2/d4 14704 1 2024-05-12T04:01:01.882 INFO:tasks.workunit.client.1.smithi033.stdout:7/148: dread de/df/d17/f34 [0,4194304] 0 2024-05-12T04:01:01.885 INFO:tasks.workunit.client.1.smithi033.stdout:9/209: truncate d2/d4/d2b/f4b 431656 0 2024-05-12T04:01:01.889 INFO:tasks.workunit.client.1.smithi033.stdout:9/210: symlink d2/l51 0 2024-05-12T04:01:01.890 INFO:tasks.workunit.client.1.smithi033.stdout:6/97: dwrite d4/fa [0,4194304] 0 2024-05-12T04:01:01.890 INFO:tasks.workunit.client.1.smithi033.stdout:7/149: getdents de/df/d17 0 2024-05-12T04:01:01.893 INFO:tasks.workunit.client.1.smithi033.stdout:9/211: mkdir d2/d4/d29/d3c/d52 0 2024-05-12T04:01:01.896 INFO:tasks.workunit.client.1.smithi033.stdout:7/150: rename de/df/d17/d1e/d24/l2d to de/d20/l35 0 2024-05-12T04:01:01.898 INFO:tasks.workunit.client.1.smithi033.stdout:7/151: dread de/df/f29 [0,4194304] 0 2024-05-12T04:01:01.898 INFO:tasks.workunit.client.1.smithi033.stdout:6/98: dread d4/d5/f8 [4194304,4194304] 0 2024-05-12T04:01:01.899 INFO:tasks.workunit.client.1.smithi033.stdout:9/212: dread d2/dd/f26 [0,4194304] 0 2024-05-12T04:01:01.900 INFO:tasks.workunit.client.1.smithi033.stdout:6/99: creat d4/f18 x:0 0 0 2024-05-12T04:01:01.901 INFO:tasks.workunit.client.1.smithi033.stdout:7/152: truncate de/f12 477522 0 2024-05-12T04:01:01.901 INFO:tasks.workunit.client.1.smithi033.stdout:7/153: stat de/d2b 0 2024-05-12T04:01:01.902 INFO:tasks.workunit.client.1.smithi033.stdout:6/100: symlink d4/l19 0 2024-05-12T04:01:01.903 INFO:tasks.workunit.client.1.smithi033.stdout:9/213: truncate d2/d4/ff 1587069 0 2024-05-12T04:01:01.905 INFO:tasks.workunit.client.1.smithi033.stdout:6/101: dread d4/d5/f10 [0,4194304] 0 2024-05-12T04:01:01.909 INFO:tasks.workunit.client.1.smithi033.stdout:7/154: dwrite de/f1a [0,4194304] 0 2024-05-12T04:01:01.915 INFO:tasks.workunit.client.1.smithi033.stdout:6/102: dwrite d4/fa [0,4194304] 0 2024-05-12T04:01:01.917 INFO:tasks.workunit.client.1.smithi033.stdout:6/103: write d4/f18 [815856,58414] 0 2024-05-12T04:01:01.920 INFO:tasks.workunit.client.1.smithi033.stdout:9/214: dread d2/f27 [0,4194304] 0 2024-05-12T04:01:01.921 INFO:tasks.workunit.client.1.smithi033.stdout:7/155: dread de/d20/f21 [0,4194304] 0 2024-05-12T04:01:01.921 INFO:tasks.workunit.client.1.smithi033.stdout:6/104: mkdir d4/d1a 0 2024-05-12T04:01:01.924 INFO:tasks.workunit.client.1.smithi033.stdout:9/215: symlink d2/d2d/d2f/l53 0 2024-05-12T04:01:01.926 INFO:tasks.workunit.client.1.smithi033.stdout:9/216: write d2/dd/f11 [4152190,97236] 0 2024-05-12T04:01:01.929 INFO:tasks.workunit.client.1.smithi033.stdout:7/156: mkdir de/df/d17/d1b/d36 0 2024-05-12T04:01:01.929 INFO:tasks.workunit.client.1.smithi033.stdout:9/217: dread d2/d4/d29/f2a [0,4194304] 0 2024-05-12T04:01:01.930 INFO:tasks.workunit.client.1.smithi033.stdout:7/157: mkdir de/d2f/d37 0 2024-05-12T04:01:01.932 INFO:tasks.workunit.client.1.smithi033.stdout:9/218: dread d2/d4/d29/f2a [0,4194304] 0 2024-05-12T04:01:01.933 INFO:tasks.workunit.client.1.smithi033.stdout:7/158: truncate de/f25 1475059 0 2024-05-12T04:01:01.935 INFO:tasks.workunit.client.1.smithi033.stdout:7/159: mkdir de/d20/d38 0 2024-05-12T04:01:01.938 INFO:tasks.workunit.client.1.smithi033.stdout:7/160: creat de/df/d17/d1b/d28/f39 x:0 0 0 2024-05-12T04:01:01.939 INFO:tasks.workunit.client.1.smithi033.stdout:9/219: dwrite d2/f27 [0,4194304] 0 2024-05-12T04:01:01.941 INFO:tasks.workunit.client.1.smithi033.stdout:7/161: mknod de/d20/c3a 0 2024-05-12T04:01:01.942 INFO:tasks.workunit.client.1.smithi033.stdout:9/220: creat d2/dd/d39/f54 x:0 0 0 2024-05-12T04:01:01.946 INFO:tasks.workunit.client.1.smithi033.stdout:7/162: rename de/df/d17/d1e/c31 to de/d20/d38/c3b 0 2024-05-12T04:01:01.948 INFO:tasks.workunit.client.1.smithi033.stdout:7/163: truncate de/df/f1f 1059776 0 2024-05-12T04:01:01.949 INFO:tasks.workunit.client.1.smithi033.stdout:9/221: getdents d2 0 2024-05-12T04:01:01.950 INFO:tasks.workunit.client.1.smithi033.stdout:7/164: symlink de/d2f/d37/l3c 0 2024-05-12T04:01:01.951 INFO:tasks.workunit.client.1.smithi033.stdout:9/222: mkdir d2/d4/d55 0 2024-05-12T04:01:01.954 INFO:tasks.workunit.client.1.smithi033.stdout:7/165: link de/d20/f21 de/df/d17/d1b/d28/f3d 0 2024-05-12T04:01:01.956 INFO:tasks.workunit.client.1.smithi033.stdout:9/223: creat d2/d4/d2b/d49/f56 x:0 0 0 2024-05-12T04:01:01.959 INFO:tasks.workunit.client.1.smithi033.stdout:7/166: mknod de/df/d17/d1b/d36/c3e 0 2024-05-12T04:01:01.961 INFO:tasks.workunit.client.1.smithi033.stdout:7/167: chown de/d2f/d37/l3c 1 1 2024-05-12T04:01:01.964 INFO:tasks.workunit.client.1.smithi033.stdout:7/168: creat de/df/d17/d1e/f3f x:0 0 0 2024-05-12T04:01:01.965 INFO:tasks.workunit.client.1.smithi033.stdout:9/224: dwrite d2/dd/d39/f54 [0,4194304] 0 2024-05-12T04:01:01.968 INFO:tasks.workunit.client.1.smithi033.stdout:9/225: symlink d2/d2d/d3a/l57 0 2024-05-12T04:01:01.970 INFO:tasks.workunit.client.1.smithi033.stdout:7/169: mknod de/df/d17/d1b/d28/c40 0 2024-05-12T04:01:01.971 INFO:tasks.workunit.client.1.smithi033.stdout:9/226: symlink d2/d4/d2b/d33/l58 0 2024-05-12T04:01:01.972 INFO:tasks.workunit.client.1.smithi033.stdout:9/227: readlink d2/d4/d2b/d33/l4a 0 2024-05-12T04:01:01.973 INFO:tasks.workunit.client.1.smithi033.stdout:7/170: symlink de/df/d17/d1b/l41 0 2024-05-12T04:01:01.975 INFO:tasks.workunit.client.1.smithi033.stdout:9/228: rename d2/d4/d29/d32/f44 to d2/dd/d39/f59 0 2024-05-12T04:01:01.976 INFO:tasks.workunit.client.1.smithi033.stdout:7/171: mkdir de/d2f/d42 0 2024-05-12T04:01:01.977 INFO:tasks.workunit.client.1.smithi033.stdout:9/229: fsync d2/d4/f16 0 2024-05-12T04:01:01.978 INFO:tasks.workunit.client.1.smithi033.stdout:9/230: write d2/f24 [192538,87646] 0 2024-05-12T04:01:01.979 INFO:tasks.workunit.client.1.smithi033.stdout:7/172: write de/df/d17/d1e/f3f [960591,101017] 0 2024-05-12T04:01:01.980 INFO:tasks.workunit.client.1.smithi033.stdout:7/173: fsync de/df/f1f 0 2024-05-12T04:01:01.981 INFO:tasks.workunit.client.1.smithi033.stdout:9/231: creat d2/d2d/d3a/f5a x:0 0 0 2024-05-12T04:01:01.983 INFO:tasks.workunit.client.1.smithi033.stdout:7/174: write de/df/d17/d1b/d28/f39 [460584,24181] 0 2024-05-12T04:01:01.984 INFO:tasks.workunit.client.1.smithi033.stdout:9/232: creat d2/d4/d29/f5b x:0 0 0 2024-05-12T04:01:01.985 INFO:tasks.workunit.client.1.smithi033.stdout:7/175: mknod de/df/d17/d1e/c43 0 2024-05-12T04:01:01.986 INFO:tasks.workunit.client.1.smithi033.stdout:7/176: truncate f3 4922283 0 2024-05-12T04:01:01.989 INFO:tasks.workunit.client.1.smithi033.stdout:0/170: sync 2024-05-12T04:01:01.992 INFO:tasks.workunit.client.1.smithi033.stdout:0/171: creat d6/dc/d39/f40 x:0 0 0 2024-05-12T04:01:01.992 INFO:tasks.workunit.client.1.smithi033.stdout:7/177: link de/df/d17/d1b/l1d de/df/l44 0 2024-05-12T04:01:01.994 INFO:tasks.workunit.client.1.smithi033.stdout:9/233: dwrite d2/d2d/d3a/f5a [0,4194304] 0 2024-05-12T04:01:01.994 INFO:tasks.workunit.client.1.smithi033.stdout:0/172: stat d6/dc/d39/l3d 0 2024-05-12T04:01:01.998 INFO:tasks.workunit.client.1.smithi033.stdout:9/234: creat d2/d2d/f5c x:0 0 0 2024-05-12T04:01:02.000 INFO:tasks.workunit.client.1.smithi033.stdout:7/178: link de/f12 de/f45 0 2024-05-12T04:01:02.001 INFO:tasks.workunit.client.1.smithi033.stdout:9/235: mknod d2/d2d/d4c/c5d 0 2024-05-12T04:01:02.001 INFO:tasks.workunit.client.1.smithi033.stdout:7/179: chown de/df/d17/d1e/d24 267864 1 2024-05-12T04:01:02.002 INFO:tasks.workunit.client.1.smithi033.stdout:9/236: read d2/fa [3143423,31843] 0 2024-05-12T04:01:02.003 INFO:tasks.workunit.client.1.smithi033.stdout:9/237: fdatasync d2/d4/f2c 0 2024-05-12T04:01:02.006 INFO:tasks.workunit.client.1.smithi033.stdout:9/238: readlink d2/d4/d29/l35 0 2024-05-12T04:01:02.010 INFO:tasks.workunit.client.1.smithi033.stdout:9/239: fsync d2/d4/f1f 0 2024-05-12T04:01:02.010 INFO:tasks.workunit.client.1.smithi033.stdout:0/173: dwrite d6/db/f1a [0,4194304] 0 2024-05-12T04:01:02.010 INFO:tasks.workunit.client.1.smithi033.stdout:9/240: dread - d2/d4/f10 zero size 2024-05-12T04:01:02.013 INFO:tasks.workunit.client.1.smithi033.stdout:7/180: dread de/df/d17/f34 [0,4194304] 0 2024-05-12T04:01:02.016 INFO:tasks.workunit.client.1.smithi033.stdout:9/241: write d2/d4/d29/f2a [1037560,2802] 0 2024-05-12T04:01:02.019 INFO:tasks.workunit.client.1.smithi033.stdout:0/174: dwrite d6/db/f13 [0,4194304] 0 2024-05-12T04:01:02.022 INFO:tasks.workunit.client.1.smithi033.stdout:7/181: dread de/df/d17/f34 [0,4194304] 0 2024-05-12T04:01:02.023 INFO:tasks.workunit.client.1.smithi033.stdout:7/182: fsync f3 0 2024-05-12T04:01:02.024 INFO:tasks.workunit.client.1.smithi033.stdout:9/242: creat d2/d4/d29/d3c/d52/f5e x:0 0 0 2024-05-12T04:01:02.025 INFO:tasks.workunit.client.1.smithi033.stdout:9/243: fdatasync d2/d4/d29/f2a 0 2024-05-12T04:01:02.027 INFO:tasks.workunit.client.1.smithi033.stdout:7/183: mkdir de/df/d17/d1b/d46 0 2024-05-12T04:01:02.027 INFO:tasks.workunit.client.1.smithi033.stdout:9/244: dread d2/d4/f30 [0,4194304] 0 2024-05-12T04:01:02.029 INFO:tasks.workunit.client.1.smithi033.stdout:9/245: mkdir d2/d2d/d2f/d5f 0 2024-05-12T04:01:02.030 INFO:tasks.workunit.client.1.smithi033.stdout:9/246: write d2/d4/f2c [3892775,84331] 0 2024-05-12T04:01:02.031 INFO:tasks.workunit.client.1.smithi033.stdout:9/247: chown d2/dd/d39/f54 4 1 2024-05-12T04:01:02.031 INFO:tasks.workunit.client.1.smithi033.stdout:9/248: dread - d2/d4/f10 zero size 2024-05-12T04:01:02.033 INFO:tasks.workunit.client.1.smithi033.stdout:0/175: truncate d6/d11/f1c 2444845 0 2024-05-12T04:01:02.034 INFO:tasks.workunit.client.1.smithi033.stdout:9/249: truncate d2/dd/f13 1522902 0 2024-05-12T04:01:02.034 INFO:tasks.workunit.client.1.smithi033.stdout:0/176: mknod d6/dc/c41 0 2024-05-12T04:01:02.040 INFO:tasks.workunit.client.1.smithi033.stdout:9/250: write d2/d4/f10 [585163,20780] 0 2024-05-12T04:01:02.042 INFO:tasks.workunit.client.1.smithi033.stdout:9/251: symlink d2/d2d/l60 0 2024-05-12T04:01:02.044 INFO:tasks.workunit.client.1.smithi033.stdout:9/252: creat d2/d2d/d2f/d5f/f61 x:0 0 0 2024-05-12T04:01:02.047 INFO:tasks.workunit.client.1.smithi033.stdout:9/253: truncate d2/dd/f13 259867 0 2024-05-12T04:01:02.048 INFO:tasks.workunit.client.1.smithi033.stdout:9/254: write d2/dd/d39/f43 [72738,87505] 0 2024-05-12T04:01:02.051 INFO:tasks.workunit.client.1.smithi033.stdout:9/255: dread d2/f27 [0,4194304] 0 2024-05-12T04:01:02.053 INFO:tasks.workunit.client.1.smithi033.stdout:9/256: mkdir d2/d4/d29/d32/d62 0 2024-05-12T04:01:02.056 INFO:tasks.workunit.client.1.smithi033.stdout:9/257: getdents d2 0 2024-05-12T04:01:02.058 INFO:tasks.workunit.client.1.smithi033.stdout:9/258: rename d2/d4/l42 to d2/d4/d29/d3c/l63 0 2024-05-12T04:01:02.064 INFO:tasks.workunit.client.1.smithi033.stdout:9/259: dwrite d2/d2d/f48 [0,4194304] 0 2024-05-12T04:01:02.066 INFO:tasks.workunit.client.1.smithi033.stdout:9/260: mknod d2/c64 0 2024-05-12T04:01:02.071 INFO:tasks.workunit.client.1.smithi033.stdout:9/261: read d2/f4f [1513681,48537] 0 2024-05-12T04:01:02.073 INFO:tasks.workunit.client.1.smithi033.stdout:9/262: write d2/d4/d29/f2a [1580625,28794] 0 2024-05-12T04:01:02.074 INFO:tasks.workunit.client.1.smithi033.stdout:9/263: creat d2/d2d/d3a/f65 x:0 0 0 2024-05-12T04:01:02.080 INFO:tasks.workunit.client.1.smithi033.stdout:9/264: dwrite d2/dd/f26 [0,4194304] 0 2024-05-12T04:01:02.087 INFO:tasks.workunit.client.1.smithi033.stdout:5/69: sync 2024-05-12T04:01:02.087 INFO:tasks.workunit.client.1.smithi033.stdout:1/59: sync 2024-05-12T04:01:02.091 INFO:tasks.workunit.client.1.smithi033.stdout:5/70: dread d3/fb [0,4194304] 0 2024-05-12T04:01:02.093 INFO:tasks.workunit.client.1.smithi033.stdout:1/60: link f8 d11/f13 0 2024-05-12T04:01:02.094 INFO:tasks.workunit.client.1.smithi033.stdout:5/71: rmdir d3/d12 39 2024-05-12T04:01:02.095 INFO:tasks.workunit.client.1.smithi033.stdout:5/72: chown d3/c11 0 1 2024-05-12T04:01:02.096 INFO:tasks.workunit.client.1.smithi033.stdout:9/265: fdatasync d2/dd/f26 0 2024-05-12T04:01:02.097 INFO:tasks.workunit.client.1.smithi033.stdout:1/61: write f8 [1003539,69518] 0 2024-05-12T04:01:02.098 INFO:tasks.workunit.client.1.smithi033.stdout:9/266: mknod d2/d4/c66 0 2024-05-12T04:01:02.100 INFO:tasks.workunit.client.1.smithi033.stdout:5/73: mkdir d3/d17 0 2024-05-12T04:01:02.100 INFO:tasks.workunit.client.1.smithi033.stdout:9/267: write d2/d2d/d3a/f65 [184355,102503] 0 2024-05-12T04:01:02.101 INFO:tasks.workunit.client.1.smithi033.stdout:1/62: mkdir d11/d14 0 2024-05-12T04:01:02.102 INFO:tasks.workunit.client.1.smithi033.stdout:9/268: creat d2/dd/d39/f67 x:0 0 0 2024-05-12T04:01:02.103 INFO:tasks.workunit.client.1.smithi033.stdout:9/269: write d2/d4/f28 [375842,60207] 0 2024-05-12T04:01:02.104 INFO:tasks.workunit.client.1.smithi033.stdout:9/270: stat d2/d4/f28 0 2024-05-12T04:01:02.106 INFO:tasks.workunit.client.1.smithi033.stdout:1/63: creat d11/d14/f15 x:0 0 0 2024-05-12T04:01:02.110 INFO:tasks.workunit.client.1.smithi033.stdout:9/271: unlink d2/d2d/d4c/c5d 0 2024-05-12T04:01:02.111 INFO:tasks.workunit.client.1.smithi033.stdout:1/64: creat d11/d14/f16 x:0 0 0 2024-05-12T04:01:02.113 INFO:tasks.workunit.client.1.smithi033.stdout:9/272: mknod d2/d2d/d2f/c68 0 2024-05-12T04:01:02.114 INFO:tasks.workunit.client.1.smithi033.stdout:5/74: truncate d3/f7 1123174 0 2024-05-12T04:01:02.115 INFO:tasks.workunit.client.1.smithi033.stdout:5/75: write d3/f9 [3723243,103555] 0 2024-05-12T04:01:02.117 INFO:tasks.workunit.client.1.smithi033.stdout:1/65: truncate fa 701382 0 2024-05-12T04:01:02.121 INFO:tasks.workunit.client.1.smithi033.stdout:1/66: mkdir d11/d17 0 2024-05-12T04:01:02.122 INFO:tasks.workunit.client.1.smithi033.stdout:9/273: link d2/d2d/d3a/f45 d2/d2d/f69 0 2024-05-12T04:01:02.124 INFO:tasks.workunit.client.1.smithi033.stdout:1/67: creat d11/d17/f18 x:0 0 0 2024-05-12T04:01:02.126 INFO:tasks.workunit.client.1.smithi033.stdout:9/274: mknod d2/d4/d29/d3c/d52/c6a 0 2024-05-12T04:01:02.128 INFO:tasks.workunit.client.1.smithi033.stdout:9/275: truncate d2/d4/f28 926921 0 2024-05-12T04:01:02.130 INFO:tasks.workunit.client.1.smithi033.stdout:9/276: mknod d2/d4/d29/d32/c6b 0 2024-05-12T04:01:02.131 INFO:tasks.workunit.client.1.smithi033.stdout:9/277: mkdir d2/d2d/d4c/d6c 0 2024-05-12T04:01:02.138 INFO:tasks.workunit.client.1.smithi033.stdout:9/278: rename d2/d2d to d2/d4/d2b/d6d 0 2024-05-12T04:01:02.144 INFO:tasks.workunit.client.1.smithi033.stdout:9/279: dwrite d2/d4/d2b/d6d/d3a/f5a [0,4194304] 0 2024-05-12T04:01:02.148 INFO:tasks.workunit.client.1.smithi033.stdout:9/280: link d2/d4/d2b/d6d/d2f/c68 d2/d4/d2b/c6e 0 2024-05-12T04:01:02.151 INFO:tasks.workunit.client.1.smithi033.stdout:9/281: symlink d2/d4/d2b/d49/l6f 0 2024-05-12T04:01:02.151 INFO:tasks.workunit.client.1.smithi033.stdout:9/282: readlink d2/l1e 0 2024-05-12T04:01:02.152 INFO:tasks.workunit.client.1.smithi033.stdout:9/283: readlink d2/d4/d29/d3c/l63 0 2024-05-12T04:01:02.153 INFO:tasks.workunit.client.1.smithi033.stdout:9/284: chown d2/dd/d39/f54 9 1 2024-05-12T04:01:02.158 INFO:tasks.workunit.client.1.smithi033.stdout:9/285: write d2/d4/ff [97588,20409] 0 2024-05-12T04:01:02.159 INFO:tasks.workunit.client.1.smithi033.stdout:9/286: chown d2/d4/d29/l3e 3793 1 2024-05-12T04:01:02.161 INFO:tasks.workunit.client.1.smithi033.stdout:9/287: read d2/d4/f28 [15865,127308] 0 2024-05-12T04:01:02.162 INFO:tasks.workunit.client.1.smithi033.stdout:9/288: write d2/d4/f28 [880782,31713] 0 2024-05-12T04:01:02.171 INFO:tasks.workunit.client.1.smithi033.stdout:9/289: dwrite d2/dd/f13 [0,4194304] 0 2024-05-12T04:01:02.179 INFO:tasks.workunit.client.1.smithi033.stdout:9/290: dwrite d2/dd/f13 [0,4194304] 0 2024-05-12T04:01:02.181 INFO:tasks.workunit.client.1.smithi033.stdout:9/291: mkdir d2/d70 0 2024-05-12T04:01:02.182 INFO:tasks.workunit.client.1.smithi033.stdout:9/292: fdatasync d2/d4/d2b/f4b 0 2024-05-12T04:01:02.185 INFO:tasks.workunit.client.1.smithi033.stdout:9/293: symlink d2/d4/d29/d3c/d52/l71 0 2024-05-12T04:01:02.187 INFO:tasks.workunit.client.1.smithi033.stdout:9/294: mkdir d2/dd/d39/d72 0 2024-05-12T04:01:02.188 INFO:tasks.workunit.client.1.smithi033.stdout:9/295: dread d2/d4/f30 [0,4194304] 0 2024-05-12T04:01:02.189 INFO:tasks.workunit.client.1.smithi033.stdout:9/296: dread - d2/d4/d29/d3c/d52/f5e zero size 2024-05-12T04:01:02.192 INFO:tasks.workunit.client.1.smithi033.stdout:9/297: unlink d2/d4/f10 0 2024-05-12T04:01:02.194 INFO:tasks.workunit.client.1.smithi033.stdout:9/298: unlink d2/dd/d39/f4e 0 2024-05-12T04:01:02.198 INFO:tasks.workunit.client.1.smithi033.stdout:9/299: rename d2/d4/d29/d3d to d2/d4/d2b/d49/d73 0 2024-05-12T04:01:02.206 INFO:tasks.workunit.client.1.smithi033.stdout:9/300: getdents d2/d4/d2b/d49 0 2024-05-12T04:01:02.211 INFO:tasks.workunit.client.1.smithi033.stdout:9/301: write d2/d4/d2b/d6d/d2f/f38 [1200475,99917] 0 2024-05-12T04:01:02.213 INFO:tasks.workunit.client.1.smithi033.stdout:9/302: mknod d2/dd/c74 0 2024-05-12T04:01:02.215 INFO:tasks.workunit.client.1.smithi033.stdout:9/303: symlink d2/d4/d2b/d49/d73/l75 0 2024-05-12T04:01:02.217 INFO:tasks.workunit.client.1.smithi033.stdout:9/304: mkdir d2/d4/d55/d76 0 2024-05-12T04:01:02.221 INFO:tasks.workunit.client.1.smithi033.stdout:9/305: rename d2/d4/d29/l35 to d2/dd/d39/d72/l77 0 2024-05-12T04:01:02.223 INFO:tasks.workunit.client.1.smithi033.stdout:9/306: unlink d2/d4/c66 0 2024-05-12T04:01:02.229 INFO:tasks.workunit.client.1.smithi033.stdout:9/307: dwrite d2/d4/f28 [0,4194304] 0 2024-05-12T04:01:02.232 INFO:tasks.workunit.client.1.smithi033.stdout:9/308: read d2/d4/d29/f2a [1117981,73300] 0 2024-05-12T04:01:02.234 INFO:tasks.workunit.client.1.smithi033.stdout:9/309: stat d2/d4/d29/d3c/d52/f5e 0 2024-05-12T04:01:02.265 INFO:tasks.workunit.client.1.smithi033.stdout:3/154: sync 2024-05-12T04:01:02.265 INFO:tasks.workunit.client.1.smithi033.stdout:4/115: sync 2024-05-12T04:01:02.266 INFO:tasks.workunit.client.1.smithi033.stdout:3/155: dread db/d17/d25/f39 [0,4194304] 0 2024-05-12T04:01:02.267 INFO:tasks.workunit.client.1.smithi033.stdout:4/116: symlink dd/l1f 0 2024-05-12T04:01:02.268 INFO:tasks.workunit.client.1.smithi033.stdout:3/156: dread db/d17/f1b [0,4194304] 0 2024-05-12T04:01:02.269 INFO:tasks.workunit.client.1.smithi033.stdout:4/117: mknod dd/d11/d15/c20 0 2024-05-12T04:01:02.270 INFO:tasks.workunit.client.1.smithi033.stdout:4/118: dread f3 [0,4194304] 0 2024-05-12T04:01:02.271 INFO:tasks.workunit.client.1.smithi033.stdout:3/157: mkdir db/d17/d25/d32/d3c/d3d 0 2024-05-12T04:01:02.271 INFO:tasks.workunit.client.1.smithi033.stdout:4/119: fsync dd/d11/d15/f1e 0 2024-05-12T04:01:02.272 INFO:tasks.workunit.client.1.smithi033.stdout:9/310: write d2/d4/d29/f46 [937904,67058] 0 2024-05-12T04:01:02.272 INFO:tasks.workunit.client.1.smithi033.stdout:4/120: chown l0 120131 1 2024-05-12T04:01:02.274 INFO:tasks.workunit.client.1.smithi033.stdout:4/121: write dd/f1b [4447118,66342] 0 2024-05-12T04:01:02.276 INFO:tasks.workunit.client.1.smithi033.stdout:3/158: dread f6 [0,4194304] 0 2024-05-12T04:01:02.278 INFO:tasks.workunit.client.1.smithi033.stdout:3/159: fdatasync db/d17/f36 0 2024-05-12T04:01:02.281 INFO:tasks.workunit.client.1.smithi033.stdout:3/160: mkdir db/d17/d3e 0 2024-05-12T04:01:02.282 INFO:tasks.workunit.client.1.smithi033.stdout:9/311: dwrite d2/d4/d29/d3c/d52/f5e [0,4194304] 0 2024-05-12T04:01:02.283 INFO:tasks.workunit.client.1.smithi033.stdout:3/161: truncate db/f20 3117474 0 2024-05-12T04:01:02.283 INFO:tasks.workunit.client.1.smithi033.stdout:4/122: truncate dd/d11/f12 4719607 0 2024-05-12T04:01:02.284 INFO:tasks.workunit.client.1.smithi033.stdout:9/312: write d2/d4/d2b/d6d/d2f/f38 [1278214,59419] 0 2024-05-12T04:01:02.286 INFO:tasks.workunit.client.1.smithi033.stdout:4/123: creat dd/d11/f21 x:0 0 0 2024-05-12T04:01:02.286 INFO:tasks.workunit.client.1.smithi033.stdout:4/124: readlink l4 0 2024-05-12T04:01:02.286 INFO:tasks.workunit.client.1.smithi033.stdout:4/125: read - dd/d11/d15/f1e zero size 2024-05-12T04:01:02.287 INFO:tasks.workunit.client.1.smithi033.stdout:4/126: chown dd/d11/f1d 31 1 2024-05-12T04:01:02.287 INFO:tasks.workunit.client.1.smithi033.stdout:4/127: dread - dd/d11/f1d zero size 2024-05-12T04:01:02.288 INFO:tasks.workunit.client.1.smithi033.stdout:9/313: read d2/d4/f16 [3992223,99132] 0 2024-05-12T04:01:02.288 INFO:tasks.workunit.client.1.smithi033.stdout:4/128: dread - dd/d11/f21 zero size 2024-05-12T04:01:02.291 INFO:tasks.workunit.client.1.smithi033.stdout:9/314: rename d2/d4/d55 to d2/d4/d2b/d78 0 2024-05-12T04:01:02.293 INFO:tasks.workunit.client.1.smithi033.stdout:4/129: mkdir dd/d11/d22 0 2024-05-12T04:01:02.352 INFO:tasks.workunit.client.1.smithi033.stdout:4/130: chown dd 52 1 2024-05-12T04:01:02.355 INFO:tasks.workunit.client.1.smithi033.stdout:9/315: mkdir d2/dd/d79 0 2024-05-12T04:01:02.358 INFO:tasks.workunit.client.1.smithi033.stdout:9/316: rmdir d2/dd/d39 39 2024-05-12T04:01:02.366 INFO:tasks.workunit.client.1.smithi033.stdout:9/317: dwrite d2/dd/d39/f67 [0,4194304] 0 2024-05-12T04:01:02.373 INFO:tasks.workunit.client.1.smithi033.stdout:4/131: truncate dd/f13 5385932 0 2024-05-12T04:01:02.375 INFO:tasks.workunit.client.1.smithi033.stdout:4/132: fsync dd/d11/f14 0 2024-05-12T04:01:02.376 INFO:tasks.workunit.client.1.smithi033.stdout:9/318: dwrite d2/f24 [0,4194304] 0 2024-05-12T04:01:02.379 INFO:tasks.workunit.client.1.smithi033.stdout:9/319: truncate d2/d4/d29/f5b 833183 0 2024-05-12T04:01:02.381 INFO:tasks.workunit.client.1.smithi033.stdout:9/320: mkdir d2/d4/d2b/d6d/d7a 0 2024-05-12T04:01:02.382 INFO:tasks.workunit.client.1.smithi033.stdout:9/321: chown d2/l23 6646533 1 2024-05-12T04:01:02.383 INFO:tasks.workunit.client.1.smithi033.stdout:8/54: sync 2024-05-12T04:01:02.385 INFO:tasks.workunit.client.1.smithi033.stdout:8/55: mknod d1/d4/ce 0 2024-05-12T04:01:02.386 INFO:tasks.workunit.client.1.smithi033.stdout:9/322: mkdir d2/d70/d7b 0 2024-05-12T04:01:02.387 INFO:tasks.workunit.client.1.smithi033.stdout:9/323: fsync d2/f47 0 2024-05-12T04:01:02.387 INFO:tasks.workunit.client.1.smithi033.stdout:8/56: symlink d1/lf 0 2024-05-12T04:01:02.390 INFO:tasks.workunit.client.1.smithi033.stdout:8/57: mkdir d1/d10 0 2024-05-12T04:01:02.393 INFO:tasks.workunit.client.1.smithi033.stdout:4/133: dwrite dd/d11/f12 [0,4194304] 0 2024-05-12T04:01:02.394 INFO:tasks.workunit.client.1.smithi033.stdout:9/324: getdents d2/d4/d2b/d33 0 2024-05-12T04:01:02.395 INFO:tasks.workunit.client.1.smithi033.stdout:8/58: rmdir d1/d10 0 2024-05-12T04:01:02.395 INFO:tasks.workunit.client.1.smithi033.stdout:9/325: chown d2/d4/d29/f2a 9 1 2024-05-12T04:01:02.397 INFO:tasks.workunit.client.1.smithi033.stdout:8/59: read d1/d4/f7 [877878,127047] 0 2024-05-12T04:01:02.397 INFO:tasks.workunit.client.1.smithi033.stdout:9/326: rmdir d2/dd/d39 39 2024-05-12T04:01:02.401 INFO:tasks.workunit.client.1.smithi033.stdout:9/327: rmdir d2/d4/d2b/d33 39 2024-05-12T04:01:02.401 INFO:tasks.workunit.client.1.smithi033.stdout:8/60: read d1/d4/f8 [2475499,91661] 0 2024-05-12T04:01:02.402 INFO:tasks.workunit.client.1.smithi033.stdout:4/134: dwrite dd/d11/d15/f1e [0,4194304] 0 2024-05-12T04:01:02.403 INFO:tasks.workunit.client.1.smithi033.stdout:9/328: write d2/f24 [4979307,33510] 0 2024-05-12T04:01:02.409 INFO:tasks.workunit.client.1.smithi033.stdout:4/135: unlink dd/d11/f12 0 2024-05-12T04:01:02.415 INFO:tasks.workunit.client.1.smithi033.stdout:4/136: symlink dd/d11/d15/l23 0 2024-05-12T04:01:02.418 INFO:tasks.workunit.client.1.smithi033.stdout:4/137: rmdir dd/d11/d22 0 2024-05-12T04:01:02.420 INFO:tasks.workunit.client.1.smithi033.stdout:1/68: sync 2024-05-12T04:01:02.420 INFO:tasks.workunit.client.1.smithi033.stdout:6/105: sync 2024-05-12T04:01:02.420 INFO:tasks.workunit.client.1.smithi033.stdout:2/85: sync 2024-05-12T04:01:02.421 INFO:tasks.workunit.client.1.smithi033.stdout:1/69: chown d11/d14/f15 10 1 2024-05-12T04:01:02.421 INFO:tasks.workunit.client.1.smithi033.stdout:8/61: dread d1/f2 [0,4194304] 0 2024-05-12T04:01:02.421 INFO:tasks.workunit.client.1.smithi033.stdout:1/70: truncate f8 2017525 0 2024-05-12T04:01:02.422 INFO:tasks.workunit.client.1.smithi033.stdout:4/138: unlink dd/d11/d15/l1c 0 2024-05-12T04:01:02.423 INFO:tasks.workunit.client.1.smithi033.stdout:4/139: dread f9 [0,4194304] 0 2024-05-12T04:01:02.424 INFO:tasks.workunit.client.1.smithi033.stdout:2/86: mkdir dc/d13/d1f 0 2024-05-12T04:01:02.424 INFO:tasks.workunit.client.1.smithi033.stdout:9/329: truncate d2/dd/d39/f67 3919888 0 2024-05-12T04:01:02.424 INFO:tasks.workunit.client.1.smithi033.stdout:2/87: truncate f1 906877 0 2024-05-12T04:01:02.425 INFO:tasks.workunit.client.1.smithi033.stdout:2/88: write f7 [1109236,84996] 0 2024-05-12T04:01:02.426 INFO:tasks.workunit.client.1.smithi033.stdout:2/89: fsync dc/d13/f1d 0 2024-05-12T04:01:02.493 INFO:tasks.workunit.client.1.smithi033.stdout:1/71: creat d11/f19 x:0 0 0 2024-05-12T04:01:02.494 INFO:tasks.workunit.client.1.smithi033.stdout:2/90: write f1 [1019845,35045] 0 2024-05-12T04:01:02.495 INFO:tasks.workunit.client.1.smithi033.stdout:1/72: chown l5 25 1 2024-05-12T04:01:02.497 INFO:tasks.workunit.client.1.smithi033.stdout:1/73: chown cb 6 1 2024-05-12T04:01:02.498 INFO:tasks.workunit.client.1.smithi033.stdout:1/74: chown cf 941 1 2024-05-12T04:01:02.499 INFO:tasks.workunit.client.1.smithi033.stdout:2/91: rename l8 to dc/d13/d1c/l20 0 2024-05-12T04:01:02.499 INFO:tasks.workunit.client.1.smithi033.stdout:8/62: dwrite d1/f2 [0,4194304] 0 2024-05-12T04:01:02.501 INFO:tasks.workunit.client.1.smithi033.stdout:9/330: getdents d2 0 2024-05-12T04:01:02.501 INFO:tasks.workunit.client.1.smithi033.stdout:4/140: mknod dd/d11/c24 0 2024-05-12T04:01:02.501 INFO:tasks.workunit.client.1.smithi033.stdout:8/63: chown d1/cb 2919 1 2024-05-12T04:01:02.502 INFO:tasks.workunit.client.1.smithi033.stdout:4/141: dread - fa zero size 2024-05-12T04:01:02.502 INFO:tasks.workunit.client.1.smithi033.stdout:2/92: chown dc/ce 3255 1 2024-05-12T04:01:02.503 INFO:tasks.workunit.client.1.smithi033.stdout:4/142: fdatasync dd/d11/f1d 0 2024-05-12T04:01:02.508 INFO:tasks.workunit.client.1.smithi033.stdout:4/143: truncate fc 523667 0 2024-05-12T04:01:02.508 INFO:tasks.workunit.client.1.smithi033.stdout:9/331: dread d2/dd/f13 [0,4194304] 0 2024-05-12T04:01:02.509 INFO:tasks.workunit.client.1.smithi033.stdout:2/93: rename dc/d13/f14 to dc/d13/d1c/f21 0 2024-05-12T04:01:02.510 INFO:tasks.workunit.client.1.smithi033.stdout:4/144: fdatasync dd/d11/f14 0 2024-05-12T04:01:02.510 INFO:tasks.workunit.client.1.smithi033.stdout:4/145: fdatasync fa 0 2024-05-12T04:01:02.512 INFO:tasks.workunit.client.1.smithi033.stdout:9/332: creat d2/d4/d2b/d78/d76/f7c x:0 0 0 2024-05-12T04:01:02.513 INFO:tasks.workunit.client.1.smithi033.stdout:4/146: write dd/d11/f21 [638477,120241] 0 2024-05-12T04:01:02.516 INFO:tasks.workunit.client.1.smithi033.stdout:4/147: read dd/d11/d15/f1e [3561100,46939] 0 2024-05-12T04:01:02.518 INFO:tasks.workunit.client.1.smithi033.stdout:2/94: dwrite f9 [0,4194304] 0 2024-05-12T04:01:02.518 INFO:tasks.workunit.client.1.smithi033.stdout:8/64: write d1/d4/f7 [2119002,129989] 0 2024-05-12T04:01:02.521 INFO:tasks.workunit.client.1.smithi033.stdout:4/148: write dd/d11/f1d [456498,88772] 0 2024-05-12T04:01:02.523 INFO:tasks.workunit.client.1.smithi033.stdout:2/95: write dc/d13/d1c/f21 [184380,100431] 0 2024-05-12T04:01:02.525 INFO:tasks.workunit.client.1.smithi033.stdout:9/333: dread d2/dd/d39/f54 [0,4194304] 0 2024-05-12T04:01:02.526 INFO:tasks.workunit.client.1.smithi033.stdout:2/96: write f9 [4404611,65723] 0 2024-05-12T04:01:02.530 INFO:tasks.workunit.client.1.smithi033.stdout:2/97: write dc/f1a [764409,77251] 0 2024-05-12T04:01:02.535 INFO:tasks.workunit.client.1.smithi033.stdout:8/65: dwrite d1/d4/fd [0,4194304] 0 2024-05-12T04:01:02.537 INFO:tasks.workunit.client.1.smithi033.stdout:4/149: dwrite dd/f1b [0,4194304] 0 2024-05-12T04:01:02.543 INFO:tasks.workunit.client.1.smithi033.stdout:8/66: getdents d1 0 2024-05-12T04:01:02.544 INFO:tasks.workunit.client.1.smithi033.stdout:2/98: truncate f7 931637 0 2024-05-12T04:01:02.549 INFO:tasks.workunit.client.1.smithi033.stdout:9/334: dread d2/d4/f16 [0,4194304] 0 2024-05-12T04:01:02.549 INFO:tasks.workunit.client.1.smithi033.stdout:2/99: mknod dc/d13/d1f/c22 0 2024-05-12T04:01:02.554 INFO:tasks.workunit.client.1.smithi033.stdout:4/150: dwrite f3 [0,4194304] 0 2024-05-12T04:01:02.554 INFO:tasks.workunit.client.1.smithi033.stdout:8/67: dread d1/d4/f7 [0,4194304] 0 2024-05-12T04:01:02.554 INFO:tasks.workunit.client.1.smithi033.stdout:2/100: chown dc/d13/d1c/f21 1164 1 2024-05-12T04:01:02.558 INFO:tasks.workunit.client.1.smithi033.stdout:2/101: dread dc/f1a [0,4194304] 0 2024-05-12T04:01:02.558 INFO:tasks.workunit.client.1.smithi033.stdout:8/68: rename d1/d4/ce to d1/c11 0 2024-05-12T04:01:02.560 INFO:tasks.workunit.client.1.smithi033.stdout:9/335: symlink d2/d4/d2b/d6d/d4c/d6c/l7d 0 2024-05-12T04:01:02.564 INFO:tasks.workunit.client.1.smithi033.stdout:8/69: unlink d1/d4/f7 0 2024-05-12T04:01:02.564 INFO:tasks.workunit.client.1.smithi033.stdout:8/70: chown d1/lf 170 1 2024-05-12T04:01:02.568 INFO:tasks.workunit.client.1.smithi033.stdout:4/151: getdents dd/d11/d15 0 2024-05-12T04:01:02.568 INFO:tasks.workunit.client.1.smithi033.stdout:0/177: sync 2024-05-12T04:01:02.568 INFO:tasks.workunit.client.1.smithi033.stdout:7/184: sync 2024-05-12T04:01:02.568 INFO:tasks.workunit.client.1.smithi033.stdout:5/76: sync 2024-05-12T04:01:02.568 INFO:tasks.workunit.client.1.smithi033.stdout:3/162: sync 2024-05-12T04:01:02.569 INFO:tasks.workunit.client.1.smithi033.stdout:4/152: write dd/f1b [1349950,2316] 0 2024-05-12T04:01:02.570 INFO:tasks.workunit.client.1.smithi033.stdout:8/71: creat d1/f12 x:0 0 0 2024-05-12T04:01:02.572 INFO:tasks.workunit.client.1.smithi033.stdout:2/102: unlink dc/d17/c19 0 2024-05-12T04:01:02.573 INFO:tasks.workunit.client.1.smithi033.stdout:2/103: write f3 [1601857,34338] 0 2024-05-12T04:01:02.574 INFO:tasks.workunit.client.1.smithi033.stdout:3/163: dread f6 [0,4194304] 0 2024-05-12T04:01:02.574 INFO:tasks.workunit.client.1.smithi033.stdout:9/336: rename d2/d4/d2b/d6d/d4c/d6c to d2/d4/d2b/d33/d7e 0 2024-05-12T04:01:02.576 INFO:tasks.workunit.client.1.smithi033.stdout:0/178: rmdir d6/dc 39 2024-05-12T04:01:02.576 INFO:tasks.workunit.client.1.smithi033.stdout:9/337: chown d2/d4/d2b/d6d/d3a 123942308 1 2024-05-12T04:01:02.578 INFO:tasks.workunit.client.1.smithi033.stdout:5/77: write d3/fb [1138950,89167] 0 2024-05-12T04:01:02.579 INFO:tasks.workunit.client.1.smithi033.stdout:8/72: creat d1/f13 x:0 0 0 2024-05-12T04:01:02.580 INFO:tasks.workunit.client.1.smithi033.stdout:2/104: creat dc/d17/f23 x:0 0 0 2024-05-12T04:01:02.581 INFO:tasks.workunit.client.1.smithi033.stdout:4/153: dwrite dd/d11/f21 [0,4194304] 0 2024-05-12T04:01:02.581 INFO:tasks.workunit.client.1.smithi033.stdout:8/73: readlink d1/d4/l5 0 2024-05-12T04:01:02.582 INFO:tasks.workunit.client.1.smithi033.stdout:4/154: chown dd/d11/d15/f1e 246 1 2024-05-12T04:01:02.583 INFO:tasks.workunit.client.1.smithi033.stdout:9/338: unlink d2/d4/d2b/d6d/f48 0 2024-05-12T04:01:02.584 INFO:tasks.workunit.client.1.smithi033.stdout:5/78: read d3/f7 [626107,125607] 0 2024-05-12T04:01:02.587 INFO:tasks.workunit.client.1.smithi033.stdout:8/74: rename d1/lf to d1/d4/l14 0 2024-05-12T04:01:02.587 INFO:tasks.workunit.client.1.smithi033.stdout:7/185: link de/df/d17/d1b/l41 de/l47 0 2024-05-12T04:01:02.592 INFO:tasks.workunit.client.1.smithi033.stdout:4/155: symlink dd/d11/d15/l25 0 2024-05-12T04:01:02.597 INFO:tasks.workunit.client.1.smithi033.stdout:4/156: dread dd/d11/f21 [0,4194304] 0 2024-05-12T04:01:02.597 INFO:tasks.workunit.client.1.smithi033.stdout:5/79: unlink d3/cd 0 2024-05-12T04:01:02.597 INFO:tasks.workunit.client.1.smithi033.stdout:0/179: write d6/d11/f1c [322723,98873] 0 2024-05-12T04:01:02.599 INFO:tasks.workunit.client.1.smithi033.stdout:4/157: chown dd/d11/f1d 14 1 2024-05-12T04:01:02.599 INFO:tasks.workunit.client.1.smithi033.stdout:4/158: truncate fc 1293857 0 2024-05-12T04:01:02.602 INFO:tasks.workunit.client.1.smithi033.stdout:8/75: write d1/d4/f8 [4550749,62976] 0 2024-05-12T04:01:02.603 INFO:tasks.workunit.client.1.smithi033.stdout:7/186: rename de/f45 to de/df/f48 0 2024-05-12T04:01:02.604 INFO:tasks.workunit.client.1.smithi033.stdout:8/76: truncate d1/d4/f8 5608761 0 2024-05-12T04:01:02.604 INFO:tasks.workunit.client.1.smithi033.stdout:2/105: getdents dc/d13 0 2024-05-12T04:01:02.605 INFO:tasks.workunit.client.1.smithi033.stdout:0/180: readlink d6/dc/l1e 0 2024-05-12T04:01:02.606 INFO:tasks.workunit.client.1.smithi033.stdout:5/80: rename d3 to d3/d18 22 2024-05-12T04:01:02.606 INFO:tasks.workunit.client.1.smithi033.stdout:0/181: write d6/dc/f3c [593021,31097] 0 2024-05-12T04:01:02.607 INFO:tasks.workunit.client.1.smithi033.stdout:7/187: symlink de/df/l49 0 2024-05-12T04:01:02.607 INFO:tasks.workunit.client.1.smithi033.stdout:0/182: chown d6/dc/l25 55 1 2024-05-12T04:01:02.608 INFO:tasks.workunit.client.1.smithi033.stdout:8/77: mknod d1/c15 0 2024-05-12T04:01:02.608 INFO:tasks.workunit.client.1.smithi033.stdout:2/106: readlink dc/l12 0 2024-05-12T04:01:02.609 INFO:tasks.workunit.client.1.smithi033.stdout:9/339: getdents d2/d4/d2b/d78 0 2024-05-12T04:01:02.609 INFO:tasks.workunit.client.1.smithi033.stdout:7/188: write de/df/d17/d1e/f3f [21205,28052] 0 2024-05-12T04:01:02.610 INFO:tasks.workunit.client.1.smithi033.stdout:5/81: rmdir d3 39 2024-05-12T04:01:02.614 INFO:tasks.workunit.client.1.smithi033.stdout:0/183: mknod d6/c42 0 2024-05-12T04:01:02.615 INFO:tasks.workunit.client.1.smithi033.stdout:4/159: getdents dd/d11/d15 0 2024-05-12T04:01:02.617 INFO:tasks.workunit.client.1.smithi033.stdout:9/340: rename d2/f47 to d2/d4/d2b/d49/f7f 0 2024-05-12T04:01:02.621 INFO:tasks.workunit.client.1.smithi033.stdout:4/160: mknod dd/c26 0 2024-05-12T04:01:02.621 INFO:tasks.workunit.client.1.smithi033.stdout:0/184: truncate d6/db/d1f/f29 566136 0 2024-05-12T04:01:02.622 INFO:tasks.workunit.client.1.smithi033.stdout:7/189: getdents de/df/d17 0 2024-05-12T04:01:02.623 INFO:tasks.workunit.client.1.smithi033.stdout:8/78: dwrite d1/d4/fd [0,4194304] 0 2024-05-12T04:01:02.624 INFO:tasks.workunit.client.1.smithi033.stdout:2/107: dwrite dc/d13/f15 [0,4194304] 0 2024-05-12T04:01:02.625 INFO:tasks.workunit.client.1.smithi033.stdout:9/341: dread d2/dd/d39/f43 [0,4194304] 0 2024-05-12T04:01:02.625 INFO:tasks.workunit.client.1.smithi033.stdout:4/161: mknod dd/d11/c27 0 2024-05-12T04:01:02.626 INFO:tasks.workunit.client.1.smithi033.stdout:5/82: dread d3/f10 [0,4194304] 0 2024-05-12T04:01:02.630 INFO:tasks.workunit.client.1.smithi033.stdout:2/108: write dc/d13/d1c/f21 [535579,43010] 0 2024-05-12T04:01:02.630 INFO:tasks.workunit.client.1.smithi033.stdout:0/185: symlink d6/db/d2c/l43 0 2024-05-12T04:01:02.631 INFO:tasks.workunit.client.1.smithi033.stdout:7/190: unlink de/df/c16 0 2024-05-12T04:01:02.633 INFO:tasks.workunit.client.1.smithi033.stdout:0/186: chown d6/dc/f27 686714 1 2024-05-12T04:01:02.634 INFO:tasks.workunit.client.1.smithi033.stdout:5/83: symlink d3/d12/l19 0 2024-05-12T04:01:02.636 INFO:tasks.workunit.client.1.smithi033.stdout:8/79: dread d1/f2 [0,4194304] 0 2024-05-12T04:01:02.636 INFO:tasks.workunit.client.1.smithi033.stdout:0/187: symlink d6/d15/l44 0 2024-05-12T04:01:02.637 INFO:tasks.workunit.client.1.smithi033.stdout:2/109: link f3 dc/d13/d1f/f24 0 2024-05-12T04:01:02.637 INFO:tasks.workunit.client.1.smithi033.stdout:7/191: creat de/df/d17/d1b/d36/f4a x:0 0 0 2024-05-12T04:01:02.639 INFO:tasks.workunit.client.1.smithi033.stdout:0/188: readlink d6/l38 0 2024-05-12T04:01:02.642 INFO:tasks.workunit.client.1.smithi033.stdout:5/84: creat d3/d15/d16/f1a x:0 0 0 2024-05-12T04:01:02.642 INFO:tasks.workunit.client.1.smithi033.stdout:8/80: dread d1/d4/f8 [4194304,4194304] 0 2024-05-12T04:01:02.643 INFO:tasks.workunit.client.1.smithi033.stdout:2/110: rename f1 to dc/d13/d1c/f25 0 2024-05-12T04:01:02.644 INFO:tasks.workunit.client.1.smithi033.stdout:2/111: rename dc/d13 to dc/d13/d26 22 2024-05-12T04:01:02.645 INFO:tasks.workunit.client.1.smithi033.stdout:0/189: symlink d6/d15/l45 0 2024-05-12T04:01:02.646 INFO:tasks.workunit.client.1.smithi033.stdout:0/190: write d6/dc/f18 [618489,27755] 0 2024-05-12T04:01:02.646 INFO:tasks.workunit.client.1.smithi033.stdout:4/162: truncate fc 1157709 0 2024-05-12T04:01:02.647 INFO:tasks.workunit.client.1.smithi033.stdout:4/163: rename dd/d11/d15 to dd/d11/d15/d28 22 2024-05-12T04:01:02.648 INFO:tasks.workunit.client.1.smithi033.stdout:8/81: mkdir d1/d16 0 2024-05-12T04:01:02.649 INFO:tasks.workunit.client.1.smithi033.stdout:7/192: symlink de/df/d17/d1b/d46/l4b 0 2024-05-12T04:01:02.649 INFO:tasks.workunit.client.1.smithi033.stdout:7/193: chown de/df/d17/d1e/c43 9 1 2024-05-12T04:01:02.651 INFO:tasks.workunit.client.1.smithi033.stdout:0/191: creat d6/dc/f46 x:0 0 0 2024-05-12T04:01:02.651 INFO:tasks.workunit.client.1.smithi033.stdout:5/85: symlink d3/d17/l1b 0 2024-05-12T04:01:02.653 INFO:tasks.workunit.client.1.smithi033.stdout:0/192: write f4 [1064749,90529] 0 2024-05-12T04:01:02.654 INFO:tasks.workunit.client.1.smithi033.stdout:7/194: rmdir de/d2f 39 2024-05-12T04:01:02.654 INFO:tasks.workunit.client.1.smithi033.stdout:8/82: chown d1/c11 7289 1 2024-05-12T04:01:02.655 INFO:tasks.workunit.client.1.smithi033.stdout:0/193: chown d6/d11/c3b 1087098 1 2024-05-12T04:01:02.656 INFO:tasks.workunit.client.1.smithi033.stdout:4/164: fdatasync fc 0 2024-05-12T04:01:02.660 INFO:tasks.workunit.client.1.smithi033.stdout:7/195: creat de/df/d17/d1e/f4c x:0 0 0 2024-05-12T04:01:02.663 INFO:tasks.workunit.client.1.smithi033.stdout:7/196: stat de/df/d17/d1b/d28 0 2024-05-12T04:01:02.663 INFO:tasks.workunit.client.1.smithi033.stdout:4/165: read fc [650915,70151] 0 2024-05-12T04:01:02.664 INFO:tasks.workunit.client.1.smithi033.stdout:5/86: truncate d3/fc 2883260 0 2024-05-12T04:01:02.665 INFO:tasks.workunit.client.1.smithi033.stdout:0/194: dwrite d6/d11/f33 [0,4194304] 0 2024-05-12T04:01:02.666 INFO:tasks.workunit.client.1.smithi033.stdout:8/83: rename d1/d4/l14 to d1/d16/l17 0 2024-05-12T04:01:02.671 INFO:tasks.workunit.client.1.smithi033.stdout:7/197: symlink de/d20/d38/l4d 0 2024-05-12T04:01:02.671 INFO:tasks.workunit.client.1.smithi033.stdout:0/195: dread d6/dc/f27 [0,4194304] 0 2024-05-12T04:01:02.673 INFO:tasks.workunit.client.1.smithi033.stdout:2/112: dwrite dc/d13/d1f/f24 [0,4194304] 0 2024-05-12T04:01:02.675 INFO:tasks.workunit.client.1.smithi033.stdout:0/196: read - d6/dc/d39/f40 zero size 2024-05-12T04:01:02.676 INFO:tasks.workunit.client.1.smithi033.stdout:8/84: mknod d1/d16/c18 0 2024-05-12T04:01:02.679 INFO:tasks.workunit.client.1.smithi033.stdout:0/197: fdatasync d6/dc/f46 0 2024-05-12T04:01:02.680 INFO:tasks.workunit.client.1.smithi033.stdout:5/87: dwrite d3/d15/d16/f1a [0,4194304] 0 2024-05-12T04:01:02.681 INFO:tasks.workunit.client.1.smithi033.stdout:6/106: sync 2024-05-12T04:01:02.684 INFO:tasks.workunit.client.1.smithi033.stdout:2/113: mkdir dc/d17/d27 0 2024-05-12T04:01:02.684 INFO:tasks.workunit.client.1.smithi033.stdout:0/198: dread - d6/db/d2c/f3e zero size 2024-05-12T04:01:02.684 INFO:tasks.workunit.client.1.smithi033.stdout:7/198: dread f3 [0,4194304] 0 2024-05-12T04:01:02.686 INFO:tasks.workunit.client.1.smithi033.stdout:8/85: fsync d1/d4/f8 0 2024-05-12T04:01:02.689 INFO:tasks.workunit.client.1.smithi033.stdout:4/166: getdents dd/d11 0 2024-05-12T04:01:02.691 INFO:tasks.workunit.client.1.smithi033.stdout:0/199: creat d6/d11/f47 x:0 0 0 2024-05-12T04:01:02.691 INFO:tasks.workunit.client.1.smithi033.stdout:2/114: mknod dc/d13/d1c/c28 0 2024-05-12T04:01:02.692 INFO:tasks.workunit.client.1.smithi033.stdout:8/86: fsync d1/f2 0 2024-05-12T04:01:02.694 INFO:tasks.workunit.client.1.smithi033.stdout:7/199: creat de/d2f/f4e x:0 0 0 2024-05-12T04:01:02.695 INFO:tasks.workunit.client.1.smithi033.stdout:6/107: dwrite d4/f18 [0,4194304] 0 2024-05-12T04:01:02.697 INFO:tasks.workunit.client.1.smithi033.stdout:5/88: dwrite d3/d15/d16/f1a [0,4194304] 0 2024-05-12T04:01:02.698 INFO:tasks.workunit.client.1.smithi033.stdout:6/108: chown d4/l11 3980892 1 2024-05-12T04:01:02.698 INFO:tasks.workunit.client.1.smithi033.stdout:0/200: dread d6/dc/f18 [0,4194304] 0 2024-05-12T04:01:02.700 INFO:tasks.workunit.client.1.smithi033.stdout:4/167: creat dd/f29 x:0 0 0 2024-05-12T04:01:02.702 INFO:tasks.workunit.client.1.smithi033.stdout:8/87: dread d1/f2 [0,4194304] 0 2024-05-12T04:01:02.703 INFO:tasks.workunit.client.1.smithi033.stdout:8/88: chown d1/c11 40268 1 2024-05-12T04:01:02.704 INFO:tasks.workunit.client.1.smithi033.stdout:0/201: creat d6/db/d2c/f48 x:0 0 0 2024-05-12T04:01:02.704 INFO:tasks.workunit.client.1.smithi033.stdout:2/115: link l2 dc/d13/l29 0 2024-05-12T04:01:02.707 INFO:tasks.workunit.client.1.smithi033.stdout:7/200: dwrite de/df/d17/d1b/d28/f39 [0,4194304] 0 2024-05-12T04:01:02.707 INFO:tasks.workunit.client.1.smithi033.stdout:0/202: chown d6/db/f13 1 1 2024-05-12T04:01:02.707 INFO:tasks.workunit.client.1.smithi033.stdout:6/109: symlink d4/d1a/l1b 0 2024-05-12T04:01:02.707 INFO:tasks.workunit.client.1.smithi033.stdout:0/203: fdatasync d6/dc/d39/f40 0 2024-05-12T04:01:02.709 INFO:tasks.workunit.client.1.smithi033.stdout:5/89: creat d3/f1c x:0 0 0 2024-05-12T04:01:02.712 INFO:tasks.workunit.client.1.smithi033.stdout:8/89: rename d1/d4/f8 to d1/d16/f19 0 2024-05-12T04:01:02.714 INFO:tasks.workunit.client.1.smithi033.stdout:8/90: chown d1/d4/c6 576247245 1 2024-05-12T04:01:02.715 INFO:tasks.workunit.client.1.smithi033.stdout:5/90: creat d3/d12/f1d x:0 0 0 2024-05-12T04:01:02.716 INFO:tasks.workunit.client.1.smithi033.stdout:7/201: dwrite de/df/f1f [0,4194304] 0 2024-05-12T04:01:02.716 INFO:tasks.workunit.client.1.smithi033.stdout:4/168: creat dd/f2a x:0 0 0 2024-05-12T04:01:02.718 INFO:tasks.workunit.client.1.smithi033.stdout:5/91: truncate d3/d12/f14 201950 0 2024-05-12T04:01:02.719 INFO:tasks.workunit.client.1.smithi033.stdout:4/169: write dd/f29 [318105,96033] 0 2024-05-12T04:01:02.721 INFO:tasks.workunit.client.1.smithi033.stdout:0/204: rename d6/dc/c12 to d6/d11/c49 0 2024-05-12T04:01:02.723 INFO:tasks.workunit.client.1.smithi033.stdout:2/116: link lb dc/l2a 0 2024-05-12T04:01:02.728 INFO:tasks.workunit.client.1.smithi033.stdout:8/91: rename d1/d4/l5 to d1/d16/l1a 0 2024-05-12T04:01:02.729 INFO:tasks.workunit.client.1.smithi033.stdout:6/110: link d4/l19 d4/l1c 0 2024-05-12T04:01:02.731 INFO:tasks.workunit.client.1.smithi033.stdout:2/117: read dc/f10 [4168914,59498] 0 2024-05-12T04:01:02.732 INFO:tasks.workunit.client.1.smithi033.stdout:8/92: dread d1/f2 [0,4194304] 0 2024-05-12T04:01:02.736 INFO:tasks.workunit.client.1.smithi033.stdout:6/111: mknod d4/c1d 0 2024-05-12T04:01:02.737 INFO:tasks.workunit.client.1.smithi033.stdout:5/92: creat d3/f1e x:0 0 0 2024-05-12T04:01:02.740 INFO:tasks.workunit.client.1.smithi033.stdout:4/170: creat dd/f2b x:0 0 0 2024-05-12T04:01:02.741 INFO:tasks.workunit.client.1.smithi033.stdout:8/93: mkdir d1/d4/d1b 0 2024-05-12T04:01:02.744 INFO:tasks.workunit.client.1.smithi033.stdout:6/112: rename d4/d5/f8 to d4/d1a/f1e 0 2024-05-12T04:01:02.744 INFO:tasks.workunit.client.1.smithi033.stdout:5/93: dwrite d3/d12/f14 [0,4194304] 0 2024-05-12T04:01:02.746 INFO:tasks.workunit.client.1.smithi033.stdout:4/171: dread dd/d11/f21 [0,4194304] 0 2024-05-12T04:01:02.748 INFO:tasks.workunit.client.1.smithi033.stdout:7/202: truncate de/df/d17/d1b/d28/f39 821869 0 2024-05-12T04:01:02.748 INFO:tasks.workunit.client.1.smithi033.stdout:2/118: chown dc/l2a 1685 1 2024-05-12T04:01:02.748 INFO:tasks.workunit.client.1.smithi033.stdout:4/172: truncate dd/d11/f19 111144 0 2024-05-12T04:01:02.749 INFO:tasks.workunit.client.1.smithi033.stdout:0/205: getdents d6 0 2024-05-12T04:01:02.751 INFO:tasks.workunit.client.1.smithi033.stdout:8/94: creat d1/d4/f1c x:0 0 0 2024-05-12T04:01:02.751 INFO:tasks.workunit.client.1.smithi033.stdout:8/95: stat d1 0 2024-05-12T04:01:02.754 INFO:tasks.workunit.client.1.smithi033.stdout:4/173: dread dd/d11/f1d [0,4194304] 0 2024-05-12T04:01:02.755 INFO:tasks.workunit.client.1.smithi033.stdout:7/203: creat de/df/d17/d1e/d24/f4f x:0 0 0 2024-05-12T04:01:02.756 INFO:tasks.workunit.client.1.smithi033.stdout:0/206: rmdir d6/db/d1f 39 2024-05-12T04:01:02.758 INFO:tasks.workunit.client.1.smithi033.stdout:2/119: mknod dc/d17/d27/c2b 0 2024-05-12T04:01:02.758 INFO:tasks.workunit.client.1.smithi033.stdout:0/207: read - d6/db/d2c/f3e zero size 2024-05-12T04:01:02.759 INFO:tasks.workunit.client.1.smithi033.stdout:8/96: rename d1/d4/d1b to d1/d4/d1d 0 2024-05-12T04:01:02.759 INFO:tasks.workunit.client.1.smithi033.stdout:6/113: dwrite d4/d1a/f1e [8388608,4194304] 0 2024-05-12T04:01:02.760 INFO:tasks.workunit.client.1.smithi033.stdout:2/120: write dc/d13/f1b [45169,100344] 0 2024-05-12T04:01:02.763 INFO:tasks.workunit.client.1.smithi033.stdout:7/204: mkdir de/df/d17/d1b/d50 0 2024-05-12T04:01:02.767 INFO:tasks.workunit.client.1.smithi033.stdout:5/94: getdents d3/d12 0 2024-05-12T04:01:02.772 INFO:tasks.workunit.client.1.smithi033.stdout:7/205: creat de/df/d17/f51 x:0 0 0 2024-05-12T04:01:02.774 INFO:tasks.workunit.client.1.smithi033.stdout:6/114: write d4/f18 [5099507,18892] 0 2024-05-12T04:01:02.775 INFO:tasks.workunit.client.1.smithi033.stdout:1/75: sync 2024-05-12T04:01:02.775 INFO:tasks.workunit.client.1.smithi033.stdout:5/95: dwrite d3/fb [0,4194304] 0 2024-05-12T04:01:02.781 INFO:tasks.workunit.client.1.smithi033.stdout:1/76: rmdir d11 39 2024-05-12T04:01:02.781 INFO:tasks.workunit.client.1.smithi033.stdout:6/115: creat d4/f1f x:0 0 0 2024-05-12T04:01:02.781 INFO:tasks.workunit.client.1.smithi033.stdout:4/174: getdents dd/d11 0 2024-05-12T04:01:02.783 INFO:tasks.workunit.client.1.smithi033.stdout:5/96: creat d3/d12/f1f x:0 0 0 2024-05-12T04:01:02.784 INFO:tasks.workunit.client.1.smithi033.stdout:0/208: getdents d6/dc 0 2024-05-12T04:01:02.786 INFO:tasks.workunit.client.1.smithi033.stdout:4/175: unlink f9 0 2024-05-12T04:01:02.786 INFO:tasks.workunit.client.1.smithi033.stdout:7/206: dread de/d20/f21 [0,4194304] 0 2024-05-12T04:01:02.787 INFO:tasks.workunit.client.1.smithi033.stdout:1/77: symlink d11/l1a 0 2024-05-12T04:01:02.788 INFO:tasks.workunit.client.1.smithi033.stdout:4/176: stat l7 0 2024-05-12T04:01:02.788 INFO:tasks.workunit.client.1.smithi033.stdout:5/97: write d3/f1c [219812,102364] 0 2024-05-12T04:01:02.789 INFO:tasks.workunit.client.1.smithi033.stdout:0/209: fdatasync d6/db/d2c/f48 0 2024-05-12T04:01:02.789 INFO:tasks.workunit.client.1.smithi033.stdout:8/97: dwrite d1/d16/f19 [0,4194304] 0 2024-05-12T04:01:02.792 INFO:tasks.workunit.client.1.smithi033.stdout:1/78: mknod d11/d14/c1b 0 2024-05-12T04:01:02.793 INFO:tasks.workunit.client.1.smithi033.stdout:4/177: fdatasync dd/f1b 0 2024-05-12T04:01:02.794 INFO:tasks.workunit.client.1.smithi033.stdout:8/98: write d1/f13 [556661,5600] 0 2024-05-12T04:01:02.795 INFO:tasks.workunit.client.1.smithi033.stdout:2/121: dwrite f7 [0,4194304] 0 2024-05-12T04:01:02.795 INFO:tasks.workunit.client.1.smithi033.stdout:7/207: mknod de/df/d17/d1b/c52 0 2024-05-12T04:01:02.796 INFO:tasks.workunit.client.1.smithi033.stdout:0/210: write d6/db/d2c/f31 [101071,105284] 0 2024-05-12T04:01:02.802 INFO:tasks.workunit.client.1.smithi033.stdout:0/211: chown d6/dc/d39/f40 1697817671 1 2024-05-12T04:01:02.802 INFO:tasks.workunit.client.1.smithi033.stdout:4/178: chown c6 1167892969 1 2024-05-12T04:01:02.802 INFO:tasks.workunit.client.1.smithi033.stdout:7/208: unlink de/df/f1f 0 2024-05-12T04:01:02.803 INFO:tasks.workunit.client.1.smithi033.stdout:8/99: creat d1/d4/d1d/f1e x:0 0 0 2024-05-12T04:01:02.804 INFO:tasks.workunit.client.1.smithi033.stdout:2/122: fsync dc/d13/d1c/f25 0 2024-05-12T04:01:02.805 INFO:tasks.workunit.client.1.smithi033.stdout:0/212: write d6/d11/f33 [298044,54241] 0 2024-05-12T04:01:02.808 INFO:tasks.workunit.client.1.smithi033.stdout:4/179: mknod dd/d11/c2c 0 2024-05-12T04:01:02.808 INFO:tasks.workunit.client.1.smithi033.stdout:5/98: dwrite d3/ff [0,4194304] 0 2024-05-12T04:01:02.808 INFO:tasks.workunit.client.1.smithi033.stdout:8/100: mknod d1/d16/c1f 0 2024-05-12T04:01:02.809 INFO:tasks.workunit.client.1.smithi033.stdout:2/123: write dc/d13/d1c/f25 [506657,47199] 0 2024-05-12T04:01:02.811 INFO:tasks.workunit.client.1.smithi033.stdout:0/213: read - d6/db/d2c/f48 zero size 2024-05-12T04:01:02.815 INFO:tasks.workunit.client.1.smithi033.stdout:7/209: dwrite f3 [0,4194304] 0 2024-05-12T04:01:02.818 INFO:tasks.workunit.client.1.smithi033.stdout:4/180: creat dd/d11/d15/f2d x:0 0 0 2024-05-12T04:01:02.821 INFO:tasks.workunit.client.1.smithi033.stdout:8/101: mkdir d1/d16/d20 0 2024-05-12T04:01:02.823 INFO:tasks.workunit.client.1.smithi033.stdout:2/124: symlink dc/l2c 0 2024-05-12T04:01:02.824 INFO:tasks.workunit.client.1.smithi033.stdout:4/181: write dd/d11/f19 [83078,11846] 0 2024-05-12T04:01:02.825 INFO:tasks.workunit.client.1.smithi033.stdout:1/79: truncate f8 278219 0 2024-05-12T04:01:02.826 INFO:tasks.workunit.client.1.smithi033.stdout:2/125: chown lb 71715758 1 2024-05-12T04:01:02.827 INFO:tasks.workunit.client.1.smithi033.stdout:4/182: chown dd/c26 18866510 1 2024-05-12T04:01:02.829 INFO:tasks.workunit.client.1.smithi033.stdout:7/210: dread de/df/d17/d1b/d28/f3d [0,4194304] 0 2024-05-12T04:01:02.829 INFO:tasks.workunit.client.1.smithi033.stdout:2/126: rmdir dc/d17 39 2024-05-12T04:01:02.831 INFO:tasks.workunit.client.1.smithi033.stdout:8/102: dwrite d1/f13 [0,4194304] 0 2024-05-12T04:01:02.833 INFO:tasks.workunit.client.1.smithi033.stdout:4/183: creat dd/d11/d15/f2e x:0 0 0 2024-05-12T04:01:02.839 INFO:tasks.workunit.client.1.smithi033.stdout:2/127: symlink dc/d13/d1f/l2d 0 2024-05-12T04:01:02.841 INFO:tasks.workunit.client.1.smithi033.stdout:4/184: read dd/f13 [1613189,49206] 0 2024-05-12T04:01:02.843 INFO:tasks.workunit.client.1.smithi033.stdout:7/211: getdents de/df/d17/d1e 0 2024-05-12T04:01:02.843 INFO:tasks.workunit.client.1.smithi033.stdout:1/80: dwrite d11/d14/f15 [0,4194304] 0 2024-05-12T04:01:02.844 INFO:tasks.workunit.client.1.smithi033.stdout:4/185: fdatasync dd/f1b 0 2024-05-12T04:01:02.844 INFO:tasks.workunit.client.1.smithi033.stdout:2/128: chown dc/d13/d1f/l2d 32122 1 2024-05-12T04:01:02.845 INFO:tasks.workunit.client.1.smithi033.stdout:8/103: link d1/d16/c18 d1/d16/c21 0 2024-05-12T04:01:02.850 INFO:tasks.workunit.client.1.smithi033.stdout:4/186: creat dd/d11/f2f x:0 0 0 2024-05-12T04:01:02.853 INFO:tasks.workunit.client.1.smithi033.stdout:2/129: mknod dc/d17/d27/c2e 0 2024-05-12T04:01:02.857 INFO:tasks.workunit.client.1.smithi033.stdout:4/187: symlink dd/d11/d15/l30 0 2024-05-12T04:01:02.858 INFO:tasks.workunit.client.1.smithi033.stdout:7/212: dwrite de/df/d17/f51 [0,4194304] 0 2024-05-12T04:01:02.858 INFO:tasks.workunit.client.1.smithi033.stdout:0/214: dread d6/dc/f16 [0,4194304] 0 2024-05-12T04:01:02.860 INFO:tasks.workunit.client.1.smithi033.stdout:2/130: mkdir dc/d17/d27/d2f 0 2024-05-12T04:01:02.862 INFO:tasks.workunit.client.1.smithi033.stdout:8/104: dwrite d1/f13 [0,4194304] 0 2024-05-12T04:01:02.862 INFO:tasks.workunit.client.1.smithi033.stdout:4/188: mkdir dd/d11/d31 0 2024-05-12T04:01:02.862 INFO:tasks.workunit.client.1.smithi033.stdout:7/213: write de/df/d17/f51 [3448869,14398] 0 2024-05-12T04:01:02.869 INFO:tasks.workunit.client.1.smithi033.stdout:4/189: chown dd/d11/f2f 13157755 1 2024-05-12T04:01:02.870 INFO:tasks.workunit.client.1.smithi033.stdout:8/105: readlink l0 0 2024-05-12T04:01:02.870 INFO:tasks.workunit.client.1.smithi033.stdout:1/81: read f8 [81781,81649] 0 2024-05-12T04:01:02.870 INFO:tasks.workunit.client.1.smithi033.stdout:4/190: chown dd/d11/f2f 39450902 1 2024-05-12T04:01:02.871 INFO:tasks.workunit.client.1.smithi033.stdout:7/214: creat de/df/d17/d1b/d46/f53 x:0 0 0 2024-05-12T04:01:02.872 INFO:tasks.workunit.client.1.smithi033.stdout:1/82: dread fd [0,4194304] 0 2024-05-12T04:01:02.872 INFO:tasks.workunit.client.1.smithi033.stdout:2/131: symlink dc/d13/d1c/l30 0 2024-05-12T04:01:02.873 INFO:tasks.workunit.client.1.smithi033.stdout:4/191: creat dd/f32 x:0 0 0 2024-05-12T04:01:02.874 INFO:tasks.workunit.client.1.smithi033.stdout:7/215: symlink de/d2f/d42/l54 0 2024-05-12T04:01:02.875 INFO:tasks.workunit.client.1.smithi033.stdout:7/216: chown de/d20/c3a 3149 1 2024-05-12T04:01:02.876 INFO:tasks.workunit.client.1.smithi033.stdout:1/83: rename d11/d14 to d11/d1c 0 2024-05-12T04:01:02.877 INFO:tasks.workunit.client.1.smithi033.stdout:2/132: dread f9 [0,4194304] 0 2024-05-12T04:01:02.878 INFO:tasks.workunit.client.1.smithi033.stdout:1/84: dread fd [0,4194304] 0 2024-05-12T04:01:02.878 INFO:tasks.workunit.client.1.smithi033.stdout:7/217: dread de/f25 [0,4194304] 0 2024-05-12T04:01:02.878 INFO:tasks.workunit.client.1.smithi033.stdout:4/192: link dd/f13 dd/d11/d31/f33 0 2024-05-12T04:01:02.880 INFO:tasks.workunit.client.1.smithi033.stdout:4/193: stat dd/d11/c24 0 2024-05-12T04:01:02.883 INFO:tasks.workunit.client.1.smithi033.stdout:2/133: creat dc/f31 x:0 0 0 2024-05-12T04:01:02.884 INFO:tasks.workunit.client.1.smithi033.stdout:1/85: dread d11/d1c/f15 [0,4194304] 0 2024-05-12T04:01:02.885 INFO:tasks.workunit.client.1.smithi033.stdout:2/134: dread - dc/d13/f1d zero size 2024-05-12T04:01:02.886 INFO:tasks.workunit.client.1.smithi033.stdout:3/164: sync 2024-05-12T04:01:02.886 INFO:tasks.workunit.client.1.smithi033.stdout:9/342: sync 2024-05-12T04:01:02.888 INFO:tasks.workunit.client.1.smithi033.stdout:1/86: rename d11/d1c/f15 to d11/d17/f1d 0 2024-05-12T04:01:02.889 INFO:tasks.workunit.client.1.smithi033.stdout:4/194: dwrite dd/d11/f1d [0,4194304] 0 2024-05-12T04:01:02.889 INFO:tasks.workunit.client.1.smithi033.stdout:2/135: mkdir dc/d17/d32 0 2024-05-12T04:01:02.890 INFO:tasks.workunit.client.1.smithi033.stdout:2/136: readlink dc/l2c 0 2024-05-12T04:01:02.890 INFO:tasks.workunit.client.1.smithi033.stdout:4/195: chown l0 38 1 2024-05-12T04:01:02.892 INFO:tasks.workunit.client.1.smithi033.stdout:7/218: write de/df/f29 [630202,44883] 0 2024-05-12T04:01:02.901 INFO:tasks.workunit.client.1.smithi033.stdout:3/165: write f6 [716710,87472] 0 2024-05-12T04:01:02.903 INFO:tasks.workunit.client.1.smithi033.stdout:9/343: write d2/dd/d39/f54 [2940040,2293] 0 2024-05-12T04:01:02.903 INFO:tasks.workunit.client.1.smithi033.stdout:4/196: read f3 [1950238,36905] 0 2024-05-12T04:01:02.905 INFO:tasks.workunit.client.1.smithi033.stdout:2/137: dread dc/d13/f15 [0,4194304] 0 2024-05-12T04:01:02.906 INFO:tasks.workunit.client.1.smithi033.stdout:2/138: dread - dc/d17/f23 zero size 2024-05-12T04:01:02.907 INFO:tasks.workunit.client.1.smithi033.stdout:1/87: write d11/f13 [199540,88295] 0 2024-05-12T04:01:02.909 INFO:tasks.workunit.client.1.smithi033.stdout:4/197: creat dd/f34 x:0 0 0 2024-05-12T04:01:02.909 INFO:tasks.workunit.client.1.smithi033.stdout:7/219: write de/f12 [1371729,33475] 0 2024-05-12T04:01:02.910 INFO:tasks.workunit.client.1.smithi033.stdout:9/344: mknod d2/d4/d29/d32/c80 0 2024-05-12T04:01:02.911 INFO:tasks.workunit.client.1.smithi033.stdout:1/88: unlink fa 0 2024-05-12T04:01:02.914 INFO:tasks.workunit.client.1.smithi033.stdout:1/89: dread d11/f13 [0,4194304] 0 2024-05-12T04:01:02.916 INFO:tasks.workunit.client.1.smithi033.stdout:9/345: dread d2/d4/f16 [0,4194304] 0 2024-05-12T04:01:02.916 INFO:tasks.workunit.client.1.smithi033.stdout:1/90: chown lc 2231 1 2024-05-12T04:01:02.917 INFO:tasks.workunit.client.1.smithi033.stdout:1/91: truncate d11/f19 671569 0 2024-05-12T04:01:02.919 INFO:tasks.workunit.client.1.smithi033.stdout:9/346: creat d2/d4/d2b/d6d/d7a/f81 x:0 0 0 2024-05-12T04:01:02.919 INFO:tasks.workunit.client.1.smithi033.stdout:7/220: truncate de/df/d17/f51 1087491 0 2024-05-12T04:01:02.923 INFO:tasks.workunit.client.1.smithi033.stdout:4/198: dwrite dd/d11/d15/f1e [0,4194304] 0 2024-05-12T04:01:02.923 INFO:tasks.workunit.client.1.smithi033.stdout:9/347: unlink d2/d4/d2b/d33/d7e/l7d 0 2024-05-12T04:01:02.923 INFO:tasks.workunit.client.1.smithi033.stdout:1/92: link l5 d11/l1e 0 2024-05-12T04:01:02.924 INFO:tasks.workunit.client.1.smithi033.stdout:7/221: symlink de/df/d17/d1b/d50/l55 0 2024-05-12T04:01:02.925 INFO:tasks.workunit.client.1.smithi033.stdout:1/93: symlink d11/d1c/l1f 0 2024-05-12T04:01:02.927 INFO:tasks.workunit.client.1.smithi033.stdout:9/348: rename d2/d4/d29/d32/d62 to d2/dd/d39/d72/d82 0 2024-05-12T04:01:02.928 INFO:tasks.workunit.client.1.smithi033.stdout:7/222: mknod de/d2f/d42/c56 0 2024-05-12T04:01:02.929 INFO:tasks.workunit.client.1.smithi033.stdout:1/94: fsync fd 0 2024-05-12T04:01:02.935 INFO:tasks.workunit.client.1.smithi033.stdout:9/349: creat d2/d4/d2b/d6d/d4c/f83 x:0 0 0 2024-05-12T04:01:02.936 INFO:tasks.workunit.client.1.smithi033.stdout:7/223: creat de/df/d17/d1e/d24/f57 x:0 0 0 2024-05-12T04:01:02.937 INFO:tasks.workunit.client.1.smithi033.stdout:4/199: write dd/d11/d31/f33 [6187114,107401] 0 2024-05-12T04:01:02.938 INFO:tasks.workunit.client.1.smithi033.stdout:7/224: chown de/df/d17/d1e/d24/f57 4582 1 2024-05-12T04:01:02.939 INFO:tasks.workunit.client.1.smithi033.stdout:4/200: dread - dd/f2a zero size 2024-05-12T04:01:02.942 INFO:tasks.workunit.client.1.smithi033.stdout:7/225: rmdir de/d20/d38 39 2024-05-12T04:01:02.943 INFO:tasks.workunit.client.1.smithi033.stdout:4/201: chown dd/d11/f14 3865739 1 2024-05-12T04:01:02.943 INFO:tasks.workunit.client.1.smithi033.stdout:9/350: dwrite d2/d4/ff [0,4194304] 0 2024-05-12T04:01:02.945 INFO:tasks.workunit.client.1.smithi033.stdout:7/226: symlink de/d2f/d42/l58 0 2024-05-12T04:01:02.946 INFO:tasks.workunit.client.1.smithi033.stdout:1/95: dwrite f8 [0,4194304] 0 2024-05-12T04:01:02.950 INFO:tasks.workunit.client.1.smithi033.stdout:7/227: mknod de/df/c59 0 2024-05-12T04:01:02.952 INFO:tasks.workunit.client.1.smithi033.stdout:1/96: creat d11/d17/f20 x:0 0 0 2024-05-12T04:01:02.958 INFO:tasks.workunit.client.1.smithi033.stdout:7/228: creat de/df/d17/d1b/d28/f5a x:0 0 0 2024-05-12T04:01:02.958 INFO:tasks.workunit.client.1.smithi033.stdout:1/97: rename d11/d1c/f16 to d11/f21 0 2024-05-12T04:01:02.959 INFO:tasks.workunit.client.1.smithi033.stdout:1/98: write d11/d17/f18 [369188,130480] 0 2024-05-12T04:01:02.960 INFO:tasks.workunit.client.1.smithi033.stdout:9/351: dread d2/d4/d2b/d6d/d3a/f5a [0,4194304] 0 2024-05-12T04:01:02.963 INFO:tasks.workunit.client.1.smithi033.stdout:1/99: creat d11/d17/f22 x:0 0 0 2024-05-12T04:01:02.964 INFO:tasks.workunit.client.1.smithi033.stdout:9/352: mkdir d2/d4/d2b/d6d/d2f/d5f/d84 0 2024-05-12T04:01:02.966 INFO:tasks.workunit.client.1.smithi033.stdout:1/100: link d11/d17/f20 d11/d17/f23 0 2024-05-12T04:01:02.967 INFO:tasks.workunit.client.1.smithi033.stdout:9/353: mknod d2/d4/d29/c85 0 2024-05-12T04:01:02.971 INFO:tasks.workunit.client.1.smithi033.stdout:1/101: dread d11/d17/f1d [0,4194304] 0 2024-05-12T04:01:02.972 INFO:tasks.workunit.client.1.smithi033.stdout:1/102: rmdir d11 39 2024-05-12T04:01:02.975 INFO:tasks.workunit.client.1.smithi033.stdout:1/103: creat d11/d17/f24 x:0 0 0 2024-05-12T04:01:02.976 INFO:tasks.workunit.client.1.smithi033.stdout:1/104: mknod d11/d1c/c25 0 2024-05-12T04:01:02.978 INFO:tasks.workunit.client.1.smithi033.stdout:1/105: symlink d11/l26 0 2024-05-12T04:01:02.979 INFO:tasks.workunit.client.1.smithi033.stdout:7/229: write de/df/d17/d1b/d28/f39 [127798,94198] 0 2024-05-12T04:01:02.980 INFO:tasks.workunit.client.1.smithi033.stdout:1/106: truncate d11/d17/f23 819793 0 2024-05-12T04:01:02.982 INFO:tasks.workunit.client.1.smithi033.stdout:1/107: mkdir d11/d17/d27 0 2024-05-12T04:01:02.984 INFO:tasks.workunit.client.1.smithi033.stdout:1/108: mknod d11/c28 0 2024-05-12T04:01:02.985 INFO:tasks.workunit.client.1.smithi033.stdout:7/230: getdents de/d2f/d42 0 2024-05-12T04:01:02.987 INFO:tasks.workunit.client.1.smithi033.stdout:1/109: mknod d11/d17/c29 0 2024-05-12T04:01:02.987 INFO:tasks.workunit.client.1.smithi033.stdout:7/231: fdatasync de/df/d17/d1b/d36/f4a 0 2024-05-12T04:01:02.989 INFO:tasks.workunit.client.1.smithi033.stdout:9/354: dwrite d2/dd/d39/f67 [4194304,4194304] 0 2024-05-12T04:01:02.990 INFO:tasks.workunit.client.1.smithi033.stdout:1/110: mknod d11/d17/c2a 0 2024-05-12T04:01:02.992 INFO:tasks.workunit.client.1.smithi033.stdout:9/355: rename d2/d4/d2b/d78/d76/f7c to d2/d4/d29/d3c/d52/f86 0 2024-05-12T04:01:02.995 INFO:tasks.workunit.client.1.smithi033.stdout:7/232: getdents de 0 2024-05-12T04:01:02.997 INFO:tasks.workunit.client.1.smithi033.stdout:9/356: link d2/d4/d2b/d49/f56 d2/d70/f87 0 2024-05-12T04:01:03.039 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:02 smithi032 ceph-mon[40008]: pgmap v29: 65 pgs: 65 active+clean; 395 MiB data, 2.9 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s rd, 25 MiB/s wr, 239 op/s 2024-05-12T04:01:03.042 INFO:tasks.workunit.client.1.smithi033.stdout:6/116: sync 2024-05-12T04:01:03.061 INFO:tasks.workunit.client.1.smithi033.stdout:7/233: symlink de/df/d17/d1b/d28/l5b 0 2024-05-12T04:01:03.061 INFO:tasks.workunit.client.1.smithi033.stdout:1/111: truncate d11/d17/f23 425701 0 2024-05-12T04:01:03.114 INFO:tasks.workunit.client.1.smithi033.stdout:6/117: dwrite d4/f16 [0,4194304] 0 2024-05-12T04:01:03.115 INFO:tasks.workunit.client.1.smithi033.stdout:9/357: dwrite d2/d4/f2c [0,4194304] 0 2024-05-12T04:01:03.120 INFO:tasks.workunit.client.1.smithi033.stdout:6/118: creat d4/d5/f20 x:0 0 0 2024-05-12T04:01:03.120 INFO:tasks.workunit.client.1.smithi033.stdout:6/119: readlink l3 0 2024-05-12T04:01:03.120 INFO:tasks.workunit.client.1.smithi033.stdout:9/358: dread - d2/d4/d2b/d6d/d4c/f83 zero size 2024-05-12T04:01:03.122 INFO:tasks.workunit.client.1.smithi033.stdout:7/234: dwrite de/df/d17/f51 [0,4194304] 0 2024-05-12T04:01:03.125 INFO:tasks.workunit.client.1.smithi033.stdout:6/120: mknod d4/d5/c21 0 2024-05-12T04:01:03.126 INFO:tasks.workunit.client.1.smithi033.stdout:9/359: creat d2/d4/d2b/d78/f88 x:0 0 0 2024-05-12T04:01:03.127 INFO:tasks.workunit.client.1.smithi033.stdout:1/112: dwrite d11/d17/f20 [0,4194304] 0 2024-05-12T04:01:03.127 INFO:tasks.workunit.client.1.smithi033.stdout:7/235: creat de/d2f/f5c x:0 0 0 2024-05-12T04:01:03.130 INFO:tasks.workunit.client.1.smithi033.stdout:6/121: mknod d4/d5/c22 0 2024-05-12T04:01:03.136 INFO:tasks.workunit.client.1.smithi033.stdout:1/113: write d11/d17/f1d [1181076,113314] 0 2024-05-12T04:01:03.136 INFO:tasks.workunit.client.1.smithi033.stdout:7/236: fsync de/f25 0 2024-05-12T04:01:03.136 INFO:tasks.workunit.client.1.smithi033.stdout:6/122: truncate d4/d5/f10 862941 0 2024-05-12T04:01:03.136 INFO:tasks.workunit.client.1.smithi033.stdout:9/360: mknod d2/d4/d2b/c89 0 2024-05-12T04:01:03.138 INFO:tasks.workunit.client.1.smithi033.stdout:1/114: unlink d11/d17/f22 0 2024-05-12T04:01:03.139 INFO:tasks.workunit.client.1.smithi033.stdout:6/123: write d4/d5/f9 [2658676,77199] 0 2024-05-12T04:01:03.139 INFO:tasks.workunit.client.1.smithi033.stdout:1/115: readlink l7 0 2024-05-12T04:01:03.140 INFO:tasks.workunit.client.1.smithi033.stdout:9/361: write d2/d4/d2b/f4b [750584,68576] 0 2024-05-12T04:01:03.141 INFO:tasks.workunit.client.1.smithi033.stdout:1/116: dread - d11/f21 zero size 2024-05-12T04:01:03.143 INFO:tasks.workunit.client.1.smithi033.stdout:7/237: dread de/d20/f21 [0,4194304] 0 2024-05-12T04:01:03.143 INFO:tasks.workunit.client.1.smithi033.stdout:6/124: creat d4/d1a/f23 x:0 0 0 2024-05-12T04:01:03.145 INFO:tasks.workunit.client.1.smithi033.stdout:7/238: write de/df/d17/d1b/d28/f5a [586374,116335] 0 2024-05-12T04:01:03.146 INFO:tasks.workunit.client.1.smithi033.stdout:9/362: creat d2/d70/d7b/f8a x:0 0 0 2024-05-12T04:01:03.147 INFO:tasks.workunit.client.1.smithi033.stdout:1/117: creat d11/d17/d27/f2b x:0 0 0 2024-05-12T04:01:03.148 INFO:tasks.workunit.client.1.smithi033.stdout:6/125: rename d4/d5/c21 to d4/d5/c24 0 2024-05-12T04:01:03.148 INFO:tasks.workunit.client.1.smithi033.stdout:7/239: symlink de/df/l5d 0 2024-05-12T04:01:03.150 INFO:tasks.workunit.client.1.smithi033.stdout:9/363: symlink d2/d4/d2b/d6d/d7a/l8b 0 2024-05-12T04:01:03.150 INFO:tasks.workunit.client.1.smithi033.stdout:7/240: write f3 [844054,113991] 0 2024-05-12T04:01:03.150 INFO:tasks.workunit.client.1.smithi033.stdout:1/118: write d11/f21 [934628,56342] 0 2024-05-12T04:01:03.150 INFO:tasks.workunit.client.1.smithi033.stdout:6/126: mknod d4/c25 0 2024-05-12T04:01:03.152 INFO:tasks.workunit.client.1.smithi033.stdout:9/364: readlink d2/d4/d29/l3e 0 2024-05-12T04:01:03.153 INFO:tasks.workunit.client.1.smithi033.stdout:1/119: creat d11/d17/f2c x:0 0 0 2024-05-12T04:01:03.155 INFO:tasks.workunit.client.1.smithi033.stdout:7/241: write de/df/d17/f34 [520721,7157] 0 2024-05-12T04:01:03.155 INFO:tasks.workunit.client.1.smithi033.stdout:6/127: rename d4/d1a/f1e to d4/d1a/f26 0 2024-05-12T04:01:03.156 INFO:tasks.workunit.client.1.smithi033.stdout:9/365: creat d2/d4/d2b/d78/f8c x:0 0 0 2024-05-12T04:01:03.158 INFO:tasks.workunit.client.1.smithi033.stdout:7/242: fsync de/df/d17/d1e/d24/f4f 0 2024-05-12T04:01:03.159 INFO:tasks.workunit.client.1.smithi033.stdout:6/128: rename d4/f18 to d4/d5/f27 0 2024-05-12T04:01:03.159 INFO:tasks.workunit.client.1.smithi033.stdout:7/243: dread - de/d2b/f2c zero size 2024-05-12T04:01:03.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:02 smithi033 ceph-mon[49525]: pgmap v29: 65 pgs: 65 active+clean; 395 MiB data, 2.9 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s rd, 25 MiB/s wr, 239 op/s 2024-05-12T04:01:03.162 INFO:tasks.workunit.client.1.smithi033.stdout:6/129: truncate d4/d5/f27 5642652 0 2024-05-12T04:01:03.165 INFO:tasks.workunit.client.1.smithi033.stdout:1/120: dwrite d11/d17/d27/f2b [0,4194304] 0 2024-05-12T04:01:03.174 INFO:tasks.workunit.client.1.smithi033.stdout:7/244: dwrite de/df/d17/d1e/f3f [0,4194304] 0 2024-05-12T04:01:03.175 INFO:tasks.workunit.client.1.smithi033.stdout:6/130: dwrite d4/d1a/f26 [4194304,4194304] 0 2024-05-12T04:01:03.176 INFO:tasks.workunit.client.1.smithi033.stdout:1/121: dread d11/d17/d27/f2b [0,4194304] 0 2024-05-12T04:01:03.177 INFO:tasks.workunit.client.1.smithi033.stdout:6/131: chown d4/c12 436782 1 2024-05-12T04:01:03.177 INFO:tasks.workunit.client.1.smithi033.stdout:7/245: mknod de/d2f/d37/c5e 0 2024-05-12T04:01:03.178 INFO:tasks.workunit.client.1.smithi033.stdout:6/132: read f2 [177942,97984] 0 2024-05-12T04:01:03.178 INFO:tasks.workunit.client.1.smithi033.stdout:5/99: sync 2024-05-12T04:01:03.181 INFO:tasks.workunit.client.1.smithi033.stdout:1/122: creat d11/f2d x:0 0 0 2024-05-12T04:01:03.182 INFO:tasks.workunit.client.1.smithi033.stdout:1/123: chown d11/c28 596 1 2024-05-12T04:01:03.182 INFO:tasks.workunit.client.1.smithi033.stdout:5/100: truncate d3/d12/f14 4234037 0 2024-05-12T04:01:03.183 INFO:tasks.workunit.client.1.smithi033.stdout:7/246: unlink de/df/l44 0 2024-05-12T04:01:03.186 INFO:tasks.workunit.client.1.smithi033.stdout:6/133: mknod d4/d1a/c28 0 2024-05-12T04:01:03.187 INFO:tasks.workunit.client.1.smithi033.stdout:7/247: dread - de/df/d17/d1e/f4c zero size 2024-05-12T04:01:03.189 INFO:tasks.workunit.client.1.smithi033.stdout:6/134: fdatasync d4/f16 0 2024-05-12T04:01:03.192 INFO:tasks.workunit.client.1.smithi033.stdout:6/135: creat d4/f29 x:0 0 0 2024-05-12T04:01:03.192 INFO:tasks.workunit.client.1.smithi033.stdout:7/248: chown de/df/d17/d1b/d28/f39 438411 1 2024-05-12T04:01:03.193 INFO:tasks.workunit.client.1.smithi033.stdout:1/124: dwrite d11/f21 [0,4194304] 0 2024-05-12T04:01:03.193 INFO:tasks.workunit.client.1.smithi033.stdout:5/101: dread d3/d15/d16/f1a [0,4194304] 0 2024-05-12T04:01:03.195 INFO:tasks.workunit.client.1.smithi033.stdout:1/125: creat d11/d17/f2e x:0 0 0 2024-05-12T04:01:03.196 INFO:tasks.workunit.client.1.smithi033.stdout:7/249: mknod de/df/d17/d1b/d50/c5f 0 2024-05-12T04:01:03.197 INFO:tasks.workunit.client.1.smithi033.stdout:7/250: fsync de/df/d17/d1e/d24/f4f 0 2024-05-12T04:01:03.198 INFO:tasks.workunit.client.1.smithi033.stdout:1/126: mknod d11/c2f 0 2024-05-12T04:01:03.200 INFO:tasks.workunit.client.1.smithi033.stdout:1/127: write d11/d17/f2c [13358,84127] 0 2024-05-12T04:01:03.201 INFO:tasks.workunit.client.1.smithi033.stdout:5/102: link d3/f9 d3/d15/d16/f20 0 2024-05-12T04:01:03.203 INFO:tasks.workunit.client.1.smithi033.stdout:7/251: rename de/d2f/d37 to de/df/d60 0 2024-05-12T04:01:03.205 INFO:tasks.workunit.client.1.smithi033.stdout:1/128: creat d11/f30 x:0 0 0 2024-05-12T04:01:03.207 INFO:tasks.workunit.client.1.smithi033.stdout:5/103: mknod d3/d12/c21 0 2024-05-12T04:01:03.207 INFO:tasks.workunit.client.1.smithi033.stdout:1/129: write d11/d17/f20 [112134,26301] 0 2024-05-12T04:01:03.210 INFO:tasks.workunit.client.1.smithi033.stdout:6/136: dread d4/fa [0,4194304] 0 2024-05-12T04:01:03.213 INFO:tasks.workunit.client.1.smithi033.stdout:7/252: dwrite de/df/d17/d1e/d24/f4f [0,4194304] 0 2024-05-12T04:01:03.216 INFO:tasks.workunit.client.1.smithi033.stdout:1/130: dread d11/d17/f1d [0,4194304] 0 2024-05-12T04:01:03.216 INFO:tasks.workunit.client.1.smithi033.stdout:1/131: fdatasync d11/f21 0 2024-05-12T04:01:03.222 INFO:tasks.workunit.client.1.smithi033.stdout:7/253: rmdir de/d20/d38 39 2024-05-12T04:01:03.223 INFO:tasks.workunit.client.1.smithi033.stdout:1/132: creat d11/d17/d27/f31 x:0 0 0 2024-05-12T04:01:03.225 INFO:tasks.workunit.client.1.smithi033.stdout:6/137: getdents d4 0 2024-05-12T04:01:03.226 INFO:tasks.workunit.client.1.smithi033.stdout:6/138: chown d4/d5/f27 224 1 2024-05-12T04:01:03.226 INFO:tasks.workunit.client.1.smithi033.stdout:1/133: creat d11/d1c/f32 x:0 0 0 2024-05-12T04:01:03.229 INFO:tasks.workunit.client.1.smithi033.stdout:1/134: stat d11/d17/f2c 0 2024-05-12T04:01:03.230 INFO:tasks.workunit.client.1.smithi033.stdout:7/254: creat de/d20/d38/f61 x:0 0 0 2024-05-12T04:01:03.230 INFO:tasks.workunit.client.1.smithi033.stdout:6/139: mkdir d4/d1a/d2a 0 2024-05-12T04:01:03.230 INFO:tasks.workunit.client.1.smithi033.stdout:7/255: write de/d20/d38/f61 [228758,109686] 0 2024-05-12T04:01:03.231 INFO:tasks.workunit.client.1.smithi033.stdout:7/256: dread - de/d2f/f4e zero size 2024-05-12T04:01:03.231 INFO:tasks.workunit.client.1.smithi033.stdout:6/140: rename d4/d5/le to d4/d5/l2b 0 2024-05-12T04:01:03.232 INFO:tasks.workunit.client.1.smithi033.stdout:1/135: rmdir d11/d17 39 2024-05-12T04:01:03.233 INFO:tasks.workunit.client.1.smithi033.stdout:7/257: symlink de/df/l62 0 2024-05-12T04:01:03.236 INFO:tasks.workunit.client.1.smithi033.stdout:7/258: mknod de/df/d17/c63 0 2024-05-12T04:01:03.237 INFO:tasks.workunit.client.1.smithi033.stdout:1/136: unlink l5 0 2024-05-12T04:01:03.238 INFO:tasks.workunit.client.1.smithi033.stdout:6/141: rmdir d4/d1a/d2a 0 2024-05-12T04:01:03.239 INFO:tasks.workunit.client.1.smithi033.stdout:7/259: mkdir de/d20/d64 0 2024-05-12T04:01:03.240 INFO:tasks.workunit.client.1.smithi033.stdout:6/142: mknod d4/d5/c2c 0 2024-05-12T04:01:03.241 INFO:tasks.workunit.client.1.smithi033.stdout:7/260: mkdir de/df/d65 0 2024-05-12T04:01:03.242 INFO:tasks.workunit.client.1.smithi033.stdout:6/143: creat d4/d5/f2d x:0 0 0 2024-05-12T04:01:03.242 INFO:tasks.workunit.client.1.smithi033.stdout:6/144: rename d4 to d4/d2e 22 2024-05-12T04:01:03.248 INFO:tasks.workunit.client.1.smithi033.stdout:6/145: dread d4/d5/f10 [0,4194304] 0 2024-05-12T04:01:03.248 INFO:tasks.workunit.client.1.smithi033.stdout:7/261: dwrite de/df/d17/d1e/d24/f4f [0,4194304] 0 2024-05-12T04:01:03.252 INFO:tasks.workunit.client.1.smithi033.stdout:6/146: creat d4/f2f x:0 0 0 2024-05-12T04:01:03.260 INFO:tasks.workunit.client.1.smithi033.stdout:6/147: getdents d4/d5 0 2024-05-12T04:01:03.260 INFO:tasks.workunit.client.1.smithi033.stdout:7/262: truncate de/df/d17/d1b/d28/f3d 1025991 0 2024-05-12T04:01:03.261 INFO:tasks.workunit.client.1.smithi033.stdout:7/263: fsync de/df/d17/d1b/d46/f53 0 2024-05-12T04:01:03.268 INFO:tasks.workunit.client.1.smithi033.stdout:6/148: truncate f2 1278537 0 2024-05-12T04:01:03.271 INFO:tasks.workunit.client.1.smithi033.stdout:6/149: creat d4/f30 x:0 0 0 2024-05-12T04:01:03.273 INFO:tasks.workunit.client.1.smithi033.stdout:6/150: symlink d4/d5/l31 0 2024-05-12T04:01:03.274 INFO:tasks.workunit.client.1.smithi033.stdout:6/151: readlink d4/l11 0 2024-05-12T04:01:03.277 INFO:tasks.workunit.client.1.smithi033.stdout:6/152: creat d4/d1a/f32 x:0 0 0 2024-05-12T04:01:03.278 INFO:tasks.workunit.client.1.smithi033.stdout:7/264: dread de/f26 [0,4194304] 0 2024-05-12T04:01:03.280 INFO:tasks.workunit.client.1.smithi033.stdout:7/265: mknod de/d20/d64/c66 0 2024-05-12T04:01:03.282 INFO:tasks.workunit.client.1.smithi033.stdout:6/153: dwrite d4/d5/f9 [0,4194304] 0 2024-05-12T04:01:03.285 INFO:tasks.workunit.client.1.smithi033.stdout:6/154: write d4/f16 [512431,114723] 0 2024-05-12T04:01:03.285 INFO:tasks.workunit.client.1.smithi033.stdout:7/266: creat de/df/d17/d1b/d50/f67 x:0 0 0 2024-05-12T04:01:03.285 INFO:tasks.workunit.client.1.smithi033.stdout:6/155: fdatasync d4/d1a/f32 0 2024-05-12T04:01:03.289 INFO:tasks.workunit.client.1.smithi033.stdout:6/156: unlink d4/d5/c24 0 2024-05-12T04:01:03.289 INFO:tasks.workunit.client.1.smithi033.stdout:6/157: chown d4/l6 247081004 1 2024-05-12T04:01:03.289 INFO:tasks.workunit.client.1.smithi033.stdout:7/267: truncate de/f25 1665666 0 2024-05-12T04:01:03.292 INFO:tasks.workunit.client.1.smithi033.stdout:7/268: mknod de/df/d17/c68 0 2024-05-12T04:01:03.296 INFO:tasks.workunit.client.1.smithi033.stdout:7/269: mknod de/d20/d64/c69 0 2024-05-12T04:01:03.297 INFO:tasks.workunit.client.1.smithi033.stdout:6/158: dwrite d4/f1f [0,4194304] 0 2024-05-12T04:01:03.300 INFO:tasks.workunit.client.1.smithi033.stdout:7/270: creat de/df/d17/f6a x:0 0 0 2024-05-12T04:01:03.303 INFO:tasks.workunit.client.1.smithi033.stdout:7/271: read - de/df/d17/d1e/d24/f57 zero size 2024-05-12T04:01:03.303 INFO:tasks.workunit.client.1.smithi033.stdout:7/272: stat de/df/d17/c63 0 2024-05-12T04:01:03.303 INFO:tasks.workunit.client.1.smithi033.stdout:7/273: write de/f1a [293000,100559] 0 2024-05-12T04:01:03.303 INFO:tasks.workunit.client.1.smithi033.stdout:6/159: unlink d4/fa 0 2024-05-12T04:01:03.305 INFO:tasks.workunit.client.1.smithi033.stdout:7/274: write de/df/d17/d1b/d46/f53 [847584,88671] 0 2024-05-12T04:01:03.306 INFO:tasks.workunit.client.1.smithi033.stdout:7/275: chown de/d2f/f5c 5232962 1 2024-05-12T04:01:03.307 INFO:tasks.workunit.client.1.smithi033.stdout:7/276: stat de/df/d17/d1b/d28/c40 0 2024-05-12T04:01:03.308 INFO:tasks.workunit.client.1.smithi033.stdout:7/277: stat de/df/d17/d1b 0 2024-05-12T04:01:03.309 INFO:tasks.workunit.client.1.smithi033.stdout:7/278: chown de/df/d17/d1e/d24/f57 8397157 1 2024-05-12T04:01:03.318 INFO:tasks.workunit.client.1.smithi033.stdout:6/160: truncate d4/d1a/f26 9054916 0 2024-05-12T04:01:03.319 INFO:tasks.workunit.client.1.smithi033.stdout:7/279: write de/df/d17/d1b/d28/f3d [501654,97535] 0 2024-05-12T04:01:03.324 INFO:tasks.workunit.client.1.smithi033.stdout:7/280: link de/df/d60/l3c de/df/d65/l6b 0 2024-05-12T04:01:03.327 INFO:tasks.workunit.client.1.smithi033.stdout:6/161: rename f2 to d4/d5/f33 0 2024-05-12T04:01:03.329 INFO:tasks.workunit.client.1.smithi033.stdout:6/162: chown d4/cc 190 1 2024-05-12T04:01:03.329 INFO:tasks.workunit.client.1.smithi033.stdout:6/163: chown l3 57816820 1 2024-05-12T04:01:03.330 INFO:tasks.workunit.client.1.smithi033.stdout:7/281: rename de/df/d17/d1b/d28 to de/d2b/d6c 0 2024-05-12T04:01:03.331 INFO:tasks.workunit.client.1.smithi033.stdout:7/282: fdatasync de/df/d17/f6a 0 2024-05-12T04:01:03.332 INFO:tasks.workunit.client.1.smithi033.stdout:7/283: write de/df/d17/d1e/f3f [1391019,41755] 0 2024-05-12T04:01:03.333 INFO:tasks.workunit.client.1.smithi033.stdout:7/284: chown de/d20/f21 756816 1 2024-05-12T04:01:03.335 INFO:tasks.workunit.client.1.smithi033.stdout:7/285: dread - de/d2f/f4e zero size 2024-05-12T04:01:03.336 INFO:tasks.workunit.client.1.smithi033.stdout:7/286: dread - de/df/d17/d1b/d36/f4a zero size 2024-05-12T04:01:03.336 INFO:tasks.workunit.client.1.smithi033.stdout:6/164: write d4/d1a/f26 [4739112,63684] 0 2024-05-12T04:01:03.336 INFO:tasks.workunit.client.1.smithi033.stdout:7/287: stat de/df/d17/d1e/c43 0 2024-05-12T04:01:03.339 INFO:tasks.workunit.client.1.smithi033.stdout:7/288: creat de/df/d17/d1b/d46/f6d x:0 0 0 2024-05-12T04:01:03.344 INFO:tasks.workunit.client.1.smithi033.stdout:7/289: dwrite de/df/d17/d1e/d24/f57 [0,4194304] 0 2024-05-12T04:01:03.345 INFO:tasks.workunit.client.1.smithi033.stdout:0/215: sync 2024-05-12T04:01:03.345 INFO:tasks.workunit.client.1.smithi033.stdout:8/106: sync 2024-05-12T04:01:03.346 INFO:tasks.workunit.client.1.smithi033.stdout:0/216: dread - d6/db/d2c/f3e zero size 2024-05-12T04:01:03.348 INFO:tasks.workunit.client.1.smithi033.stdout:8/107: mknod d1/d16/c22 0 2024-05-12T04:01:03.350 INFO:tasks.workunit.client.1.smithi033.stdout:0/217: write d6/d15/f21 [982728,62957] 0 2024-05-12T04:01:03.350 INFO:tasks.workunit.client.1.smithi033.stdout:8/108: write d1/f13 [2438665,37627] 0 2024-05-12T04:01:03.354 INFO:tasks.workunit.client.1.smithi033.stdout:8/109: write d1/d4/f1c [846059,103440] 0 2024-05-12T04:01:03.360 INFO:tasks.workunit.client.1.smithi033.stdout:7/290: dwrite de/df/d17/d1e/d24/f57 [4194304,4194304] 0 2024-05-12T04:01:03.368 INFO:tasks.workunit.client.1.smithi033.stdout:0/218: dwrite d6/dc/f18 [0,4194304] 0 2024-05-12T04:01:03.369 INFO:tasks.workunit.client.1.smithi033.stdout:8/110: dread d1/f13 [0,4194304] 0 2024-05-12T04:01:03.372 INFO:tasks.workunit.client.1.smithi033.stdout:0/219: truncate d6/f26 294007 0 2024-05-12T04:01:03.372 INFO:tasks.workunit.client.1.smithi033.stdout:8/111: chown d1/d4/d1d 222146 1 2024-05-12T04:01:03.373 INFO:tasks.workunit.client.1.smithi033.stdout:7/291: dwrite de/df/d17/d1b/d46/f6d [0,4194304] 0 2024-05-12T04:01:03.373 INFO:tasks.workunit.client.1.smithi033.stdout:0/220: chown d6/l8 1978 1 2024-05-12T04:01:03.374 INFO:tasks.workunit.client.1.smithi033.stdout:0/221: readlink l3 0 2024-05-12T04:01:03.375 INFO:tasks.workunit.client.1.smithi033.stdout:0/222: chown d6/db/fd 1150609 1 2024-05-12T04:01:03.378 INFO:tasks.workunit.client.1.smithi033.stdout:7/292: unlink de/df/d17/d1e/f3f 0 2024-05-12T04:01:03.379 INFO:tasks.workunit.client.1.smithi033.stdout:0/223: write d6/dc/d39/f40 [505043,61855] 0 2024-05-12T04:01:03.383 INFO:tasks.workunit.client.1.smithi033.stdout:7/293: rmdir de/df/d17/d1b 39 2024-05-12T04:01:03.383 INFO:tasks.workunit.client.1.smithi033.stdout:0/224: mknod d6/db/d2c/c4a 0 2024-05-12T04:01:03.384 INFO:tasks.workunit.client.1.smithi033.stdout:8/112: dwrite d1/d4/d1d/f1e [0,4194304] 0 2024-05-12T04:01:03.386 INFO:tasks.workunit.client.1.smithi033.stdout:8/113: write d1/f2 [440142,70012] 0 2024-05-12T04:01:03.388 INFO:tasks.workunit.client.1.smithi033.stdout:0/225: rename d6/d15/l44 to d6/dc/l4b 0 2024-05-12T04:01:03.389 INFO:tasks.workunit.client.1.smithi033.stdout:8/114: mknod d1/d16/d20/c23 0 2024-05-12T04:01:03.394 INFO:tasks.workunit.client.1.smithi033.stdout:0/226: mknod d6/db/d1f/c4c 0 2024-05-12T04:01:03.395 INFO:tasks.workunit.client.1.smithi033.stdout:7/294: dwrite de/df/d17/d1b/d36/f4a [0,4194304] 0 2024-05-12T04:01:03.395 INFO:tasks.workunit.client.1.smithi033.stdout:8/115: mknod d1/d16/d20/c24 0 2024-05-12T04:01:03.399 INFO:tasks.workunit.client.1.smithi033.stdout:8/116: mknod d1/c25 0 2024-05-12T04:01:03.400 INFO:tasks.workunit.client.1.smithi033.stdout:8/117: write d1/d16/f19 [4248387,78026] 0 2024-05-12T04:01:03.401 INFO:tasks.workunit.client.1.smithi033.stdout:8/118: truncate d1/f12 652970 0 2024-05-12T04:01:03.404 INFO:tasks.workunit.client.1.smithi033.stdout:7/295: rename de/l47 to de/df/d17/d1b/d36/l6e 0 2024-05-12T04:01:03.405 INFO:tasks.workunit.client.1.smithi033.stdout:8/119: dread d1/d4/f1c [0,4194304] 0 2024-05-12T04:01:03.407 INFO:tasks.workunit.client.1.smithi033.stdout:7/296: creat de/df/d17/d1e/d24/f6f x:0 0 0 2024-05-12T04:01:03.408 INFO:tasks.workunit.client.1.smithi033.stdout:7/297: write de/f1a [2067848,75100] 0 2024-05-12T04:01:03.410 INFO:tasks.workunit.client.1.smithi033.stdout:8/120: link d1/d16/f19 d1/d4/d1d/f26 0 2024-05-12T04:01:03.410 INFO:tasks.workunit.client.1.smithi033.stdout:8/121: stat d1/c11 0 2024-05-12T04:01:03.410 INFO:tasks.workunit.client.1.smithi033.stdout:8/122: chown d1/d4/cc 2 1 2024-05-12T04:01:03.411 INFO:tasks.workunit.client.1.smithi033.stdout:7/298: mkdir de/d2f/d70 0 2024-05-12T04:01:03.412 INFO:tasks.workunit.client.1.smithi033.stdout:8/123: write d1/d4/f1c [272533,100431] 0 2024-05-12T04:01:03.413 INFO:tasks.workunit.client.1.smithi033.stdout:7/299: mknod de/d2f/c71 0 2024-05-12T04:01:03.417 INFO:tasks.workunit.client.1.smithi033.stdout:0/227: dread d6/db/f13 [0,4194304] 0 2024-05-12T04:01:03.419 INFO:tasks.workunit.client.1.smithi033.stdout:8/124: getdents d1/d4 0 2024-05-12T04:01:03.420 INFO:tasks.workunit.client.1.smithi033.stdout:0/228: write d6/d11/f28 [3542128,113445] 0 2024-05-12T04:01:03.420 INFO:tasks.workunit.client.1.smithi033.stdout:7/300: dwrite de/f1a [0,4194304] 0 2024-05-12T04:01:03.424 INFO:tasks.workunit.client.1.smithi033.stdout:8/125: mkdir d1/d16/d27 0 2024-05-12T04:01:03.425 INFO:tasks.workunit.client.1.smithi033.stdout:0/229: rmdir d6/dc 39 2024-05-12T04:01:03.427 INFO:tasks.workunit.client.1.smithi033.stdout:0/230: mknod d6/db/d2c/c4d 0 2024-05-12T04:01:03.428 INFO:tasks.workunit.client.1.smithi033.stdout:0/231: chown d6/db/l10 4182344 1 2024-05-12T04:01:03.434 INFO:tasks.workunit.client.1.smithi033.stdout:0/232: dwrite d6/d11/f33 [4194304,4194304] 0 2024-05-12T04:01:03.437 INFO:tasks.workunit.client.1.smithi033.stdout:0/233: symlink d6/db/d1f/l4e 0 2024-05-12T04:01:03.440 INFO:tasks.workunit.client.1.smithi033.stdout:0/234: creat d6/dc/d39/f4f x:0 0 0 2024-05-12T04:01:03.455 INFO:tasks.workunit.client.1.smithi033.stdout:0/235: dread f4 [0,4194304] 0 2024-05-12T04:01:03.460 INFO:tasks.workunit.client.1.smithi033.stdout:0/236: dwrite f4 [0,4194304] 0 2024-05-12T04:01:03.461 INFO:tasks.workunit.client.1.smithi033.stdout:0/237: write d6/dc/f1d [271603,67658] 0 2024-05-12T04:01:03.466 INFO:tasks.workunit.client.1.smithi033.stdout:0/238: dread d6/d11/f2f [0,4194304] 0 2024-05-12T04:01:03.469 INFO:tasks.workunit.client.1.smithi033.stdout:0/239: write d6/db/d2c/f3e [203835,80625] 0 2024-05-12T04:01:03.478 INFO:tasks.workunit.client.1.smithi033.stdout:0/240: dread d6/db/d2c/f3e [0,4194304] 0 2024-05-12T04:01:03.480 INFO:tasks.workunit.client.1.smithi033.stdout:0/241: symlink d6/db/l50 0 2024-05-12T04:01:03.488 INFO:tasks.workunit.client.1.smithi033.stdout:0/242: truncate d6/d11/f33 7337919 0 2024-05-12T04:01:03.489 INFO:tasks.workunit.client.1.smithi033.stdout:0/243: truncate d6/db/fd 5035308 0 2024-05-12T04:01:03.491 INFO:tasks.workunit.client.1.smithi033.stdout:0/244: creat d6/db/d1f/f51 x:0 0 0 2024-05-12T04:01:03.493 INFO:tasks.workunit.client.1.smithi033.stdout:0/245: rename d6/db/lf to d6/db/d2c/l52 0 2024-05-12T04:01:03.494 INFO:tasks.workunit.client.1.smithi033.stdout:0/246: readlink d6/db/d2c/l43 0 2024-05-12T04:01:03.495 INFO:tasks.workunit.client.1.smithi033.stdout:0/247: creat d6/dc/f53 x:0 0 0 2024-05-12T04:01:03.497 INFO:tasks.workunit.client.1.smithi033.stdout:0/248: creat d6/dc/f54 x:0 0 0 2024-05-12T04:01:03.498 INFO:tasks.workunit.client.1.smithi033.stdout:0/249: chown d6/d11/f28 39741061 1 2024-05-12T04:01:03.505 INFO:tasks.workunit.client.1.smithi033.stdout:0/250: dread d6/d11/f33 [4194304,4194304] 0 2024-05-12T04:01:03.507 INFO:tasks.workunit.client.1.smithi033.stdout:0/251: creat d6/d15/f55 x:0 0 0 2024-05-12T04:01:03.507 INFO:tasks.workunit.client.1.smithi033.stdout:0/252: read - d6/d15/f55 zero size 2024-05-12T04:01:03.515 INFO:tasks.workunit.client.1.smithi033.stdout:0/253: dread d6/d11/f1c [0,4194304] 0 2024-05-12T04:01:03.521 INFO:tasks.workunit.client.1.smithi033.stdout:0/254: truncate d6/db/fd 1216251 0 2024-05-12T04:01:03.524 INFO:tasks.workunit.client.1.smithi033.stdout:3/166: sync 2024-05-12T04:01:03.524 INFO:tasks.workunit.client.1.smithi033.stdout:2/139: sync 2024-05-12T04:01:03.525 INFO:tasks.workunit.client.1.smithi033.stdout:2/140: chown ca 98706 1 2024-05-12T04:01:03.526 INFO:tasks.workunit.client.1.smithi033.stdout:3/167: truncate db/d17/f1b 1567715 0 2024-05-12T04:01:03.527 INFO:tasks.workunit.client.1.smithi033.stdout:3/168: read db/f31 [2338264,29484] 0 2024-05-12T04:01:03.527 INFO:tasks.workunit.client.1.smithi033.stdout:2/141: symlink dc/d13/d1c/l33 0 2024-05-12T04:01:03.529 INFO:tasks.workunit.client.1.smithi033.stdout:2/142: unlink lb 0 2024-05-12T04:01:03.531 INFO:tasks.workunit.client.1.smithi033.stdout:0/255: dread f2 [0,4194304] 0 2024-05-12T04:01:03.532 INFO:tasks.workunit.client.1.smithi033.stdout:0/256: chown d6/d15 93008 1 2024-05-12T04:01:03.533 INFO:tasks.workunit.client.1.smithi033.stdout:0/257: chown d6/dc/f54 3158 1 2024-05-12T04:01:03.533 INFO:tasks.workunit.client.1.smithi033.stdout:2/143: mknod dc/d17/d32/c34 0 2024-05-12T04:01:03.533 INFO:tasks.workunit.client.1.smithi033.stdout:3/169: link db/l10 db/d17/d25/d32/l3f 0 2024-05-12T04:01:03.534 INFO:tasks.workunit.client.1.smithi033.stdout:2/144: write dc/d13/f1d [535068,130718] 0 2024-05-12T04:01:03.537 INFO:tasks.workunit.client.1.smithi033.stdout:0/258: link d6/d11/f47 d6/dc/f56 0 2024-05-12T04:01:03.538 INFO:tasks.workunit.client.1.smithi033.stdout:3/170: rename db/d17/d25/f30 to db/d17/d25/f40 0 2024-05-12T04:01:03.540 INFO:tasks.workunit.client.1.smithi033.stdout:2/145: rename dc/d17/d32/c34 to dc/c35 0 2024-05-12T04:01:03.542 INFO:tasks.workunit.client.1.smithi033.stdout:2/146: write dc/d13/f1d [1510420,40879] 0 2024-05-12T04:01:03.545 INFO:tasks.workunit.client.1.smithi033.stdout:0/259: rename d6/d15 to d6/d11/d57 0 2024-05-12T04:01:03.546 INFO:tasks.workunit.client.1.smithi033.stdout:3/171: dread db/f31 [0,4194304] 0 2024-05-12T04:01:03.547 INFO:tasks.workunit.client.1.smithi033.stdout:2/147: write dc/d13/f15 [2593340,125486] 0 2024-05-12T04:01:03.548 INFO:tasks.workunit.client.1.smithi033.stdout:0/260: fsync d6/db/f13 0 2024-05-12T04:01:03.549 INFO:tasks.workunit.client.1.smithi033.stdout:0/261: dread - d6/dc/f56 zero size 2024-05-12T04:01:03.549 INFO:tasks.workunit.client.1.smithi033.stdout:3/172: truncate db/d17/d25/f39 152153 0 2024-05-12T04:01:03.550 INFO:tasks.workunit.client.1.smithi033.stdout:3/173: read db/f31 [1220280,124730] 0 2024-05-12T04:01:03.551 INFO:tasks.workunit.client.1.smithi033.stdout:0/262: write d6/d11/d57/f55 [196352,55588] 0 2024-05-12T04:01:03.552 INFO:tasks.workunit.client.1.smithi033.stdout:2/148: chown dc/l2a 3 1 2024-05-12T04:01:03.555 INFO:tasks.workunit.client.1.smithi033.stdout:0/263: dread f2 [4194304,4194304] 0 2024-05-12T04:01:03.556 INFO:tasks.workunit.client.1.smithi033.stdout:0/264: chown f2 12642241 1 2024-05-12T04:01:03.556 INFO:tasks.workunit.client.1.smithi033.stdout:3/174: write db/d17/d3a/f3b [402979,78949] 0 2024-05-12T04:01:03.558 INFO:tasks.workunit.client.1.smithi033.stdout:3/175: dread - db/d17/f2d zero size 2024-05-12T04:01:03.560 INFO:tasks.workunit.client.1.smithi033.stdout:0/265: write f2 [8265853,130466] 0 2024-05-12T04:01:03.566 INFO:tasks.workunit.client.1.smithi033.stdout:3/176: dwrite db/f22 [4194304,4194304] 0 2024-05-12T04:01:03.568 INFO:tasks.workunit.client.1.smithi033.stdout:2/149: dread f3 [0,4194304] 0 2024-05-12T04:01:03.568 INFO:tasks.workunit.client.1.smithi033.stdout:3/177: truncate db/d17/f2d 90286 0 2024-05-12T04:01:03.572 INFO:tasks.workunit.client.1.smithi033.stdout:3/178: symlink db/d17/d25/d32/l41 0 2024-05-12T04:01:03.575 INFO:tasks.workunit.client.1.smithi033.stdout:3/179: chown db/d17/d25/c2b 3071040 1 2024-05-12T04:01:03.576 INFO:tasks.workunit.client.1.smithi033.stdout:3/180: chown db/d17/d25/d32 234 1 2024-05-12T04:01:03.576 INFO:tasks.workunit.client.1.smithi033.stdout:2/150: dwrite f7 [4194304,4194304] 0 2024-05-12T04:01:03.580 INFO:tasks.workunit.client.1.smithi033.stdout:3/181: mkdir db/d42 0 2024-05-12T04:01:03.582 INFO:tasks.workunit.client.1.smithi033.stdout:2/151: unlink dc/f1a 0 2024-05-12T04:01:03.585 INFO:tasks.workunit.client.1.smithi033.stdout:2/152: fsync f6 0 2024-05-12T04:01:03.589 INFO:tasks.workunit.client.1.smithi033.stdout:2/153: mknod dc/d13/c36 0 2024-05-12T04:01:03.589 INFO:tasks.workunit.client.1.smithi033.stdout:3/182: dwrite db/f20 [0,4194304] 0 2024-05-12T04:01:03.590 INFO:tasks.workunit.client.1.smithi033.stdout:2/154: stat dc/d13/c36 0 2024-05-12T04:01:03.590 INFO:tasks.workunit.client.1.smithi033.stdout:3/183: readlink db/d17/l33 0 2024-05-12T04:01:03.592 INFO:tasks.workunit.client.1.smithi033.stdout:2/155: chown dc/ce 3 1 2024-05-12T04:01:03.596 INFO:tasks.workunit.client.1.smithi033.stdout:3/184: rename db/d17/d25/d32/c34 to db/d17/d3a/c43 0 2024-05-12T04:01:03.596 INFO:tasks.workunit.client.1.smithi033.stdout:3/185: readlink db/l11 0 2024-05-12T04:01:03.599 INFO:tasks.workunit.client.1.smithi033.stdout:3/186: dread db/d17/d3a/f3b [0,4194304] 0 2024-05-12T04:01:03.601 INFO:tasks.workunit.client.1.smithi033.stdout:3/187: mkdir db/d17/d25/d32/d3c/d44 0 2024-05-12T04:01:03.603 INFO:tasks.workunit.client.1.smithi033.stdout:3/188: symlink db/d17/d25/d32/d3c/l45 0 2024-05-12T04:01:03.603 INFO:tasks.workunit.client.1.smithi033.stdout:3/189: readlink db/d17/l38 0 2024-05-12T04:01:03.610 INFO:tasks.workunit.client.1.smithi033.stdout:2/156: dread f6 [0,4194304] 0 2024-05-12T04:01:03.618 INFO:tasks.workunit.client.1.smithi033.stdout:2/157: dwrite dc/f31 [0,4194304] 0 2024-05-12T04:01:03.618 INFO:tasks.workunit.client.1.smithi033.stdout:3/190: dread db/d17/f1c [4194304,4194304] 0 2024-05-12T04:01:03.622 INFO:tasks.workunit.client.1.smithi033.stdout:3/191: truncate db/d17/d25/f40 1033003 0 2024-05-12T04:01:03.623 INFO:tasks.workunit.client.1.smithi033.stdout:3/192: write db/f31 [2956375,108587] 0 2024-05-12T04:01:03.624 INFO:tasks.workunit.client.1.smithi033.stdout:2/158: dread dc/f31 [0,4194304] 0 2024-05-12T04:01:03.624 INFO:tasks.workunit.client.1.smithi033.stdout:9/366: sync 2024-05-12T04:01:03.624 INFO:tasks.workunit.client.1.smithi033.stdout:5/104: sync 2024-05-12T04:01:03.625 INFO:tasks.workunit.client.1.smithi033.stdout:4/202: sync 2024-05-12T04:01:03.625 INFO:tasks.workunit.client.1.smithi033.stdout:3/193: chown db/l2a 2560 1 2024-05-12T04:01:03.626 INFO:tasks.workunit.client.1.smithi033.stdout:4/203: write dd/f1b [4817701,35557] 0 2024-05-12T04:01:03.628 INFO:tasks.workunit.client.1.smithi033.stdout:9/367: write d2/d4/d2b/d78/f8c [958314,51222] 0 2024-05-12T04:01:03.628 INFO:tasks.workunit.client.1.smithi033.stdout:5/105: creat d3/d17/f22 x:0 0 0 2024-05-12T04:01:03.628 INFO:tasks.workunit.client.1.smithi033.stdout:2/159: rmdir dc/d17/d27 39 2024-05-12T04:01:03.629 INFO:tasks.workunit.client.1.smithi033.stdout:3/194: write db/d17/d25/f40 [1339398,73627] 0 2024-05-12T04:01:03.638 INFO:tasks.workunit.client.1.smithi033.stdout:3/195: link f6 db/d17/d25/d32/f46 0 2024-05-12T04:01:03.638 INFO:tasks.workunit.client.1.smithi033.stdout:9/368: write d2/d4/f30 [707354,89566] 0 2024-05-12T04:01:03.638 INFO:tasks.workunit.client.1.smithi033.stdout:2/160: mkdir dc/d37 0 2024-05-12T04:01:03.639 INFO:tasks.workunit.client.1.smithi033.stdout:4/204: link dd/d11/f21 dd/f35 0 2024-05-12T04:01:03.642 INFO:tasks.workunit.client.1.smithi033.stdout:9/369: fsync d2/d4/d2b/d6d/d3a/f45 0 2024-05-12T04:01:03.643 INFO:tasks.workunit.client.1.smithi033.stdout:5/106: getdents d3/d12 0 2024-05-12T04:01:03.644 INFO:tasks.workunit.client.1.smithi033.stdout:5/107: stat d3/d12/f14 0 2024-05-12T04:01:03.645 INFO:tasks.workunit.client.1.smithi033.stdout:3/196: write f6 [1930125,12359] 0 2024-05-12T04:01:03.648 INFO:tasks.workunit.client.1.smithi033.stdout:2/161: creat dc/d17/d27/d2f/f38 x:0 0 0 2024-05-12T04:01:03.649 INFO:tasks.workunit.client.1.smithi033.stdout:9/370: mknod d2/d4/d2b/d6d/d2f/d5f/d84/c8d 0 2024-05-12T04:01:03.650 INFO:tasks.workunit.client.1.smithi033.stdout:4/205: creat dd/f36 x:0 0 0 2024-05-12T04:01:03.650 INFO:tasks.workunit.client.1.smithi033.stdout:9/371: stat d2/d4/d2b/d6d/d3a/f5a 0 2024-05-12T04:01:03.650 INFO:tasks.workunit.client.1.smithi033.stdout:5/108: mkdir d3/d12/d13/d23 0 2024-05-12T04:01:03.651 INFO:tasks.workunit.client.1.smithi033.stdout:3/197: mkdir db/d17/d25/d32/d3c/d3d/d47 0 2024-05-12T04:01:03.651 INFO:tasks.workunit.client.1.smithi033.stdout:4/206: write dd/f1b [113165,57053] 0 2024-05-12T04:01:03.652 INFO:tasks.workunit.client.1.smithi033.stdout:2/162: mkdir dc/d13/d39 0 2024-05-12T04:01:03.652 INFO:tasks.workunit.client.1.smithi033.stdout:5/109: write d3/f1c [629210,89508] 0 2024-05-12T04:01:03.653 INFO:tasks.workunit.client.1.smithi033.stdout:4/207: truncate fa 796518 0 2024-05-12T04:01:03.654 INFO:tasks.workunit.client.1.smithi033.stdout:9/372: creat d2/f8e x:0 0 0 2024-05-12T04:01:03.655 INFO:tasks.workunit.client.1.smithi033.stdout:4/208: readlink dd/d11/d15/l30 0 2024-05-12T04:01:03.657 INFO:tasks.workunit.client.1.smithi033.stdout:9/373: unlink d2/d4/d29/f5b 0 2024-05-12T04:01:03.657 INFO:tasks.workunit.client.1.smithi033.stdout:3/198: dread db/d17/f2c [0,4194304] 0 2024-05-12T04:01:03.658 INFO:tasks.workunit.client.1.smithi033.stdout:9/374: readlink d2/d4/d2b/d33/l58 0 2024-05-12T04:01:03.659 INFO:tasks.workunit.client.1.smithi033.stdout:2/163: rename dc/d37 to dc/d17/d3a 0 2024-05-12T04:01:03.663 INFO:tasks.workunit.client.1.smithi033.stdout:4/209: rename l0 to dd/d11/d31/l37 0 2024-05-12T04:01:03.666 INFO:tasks.workunit.client.1.smithi033.stdout:2/164: truncate dc/f10 104965 0 2024-05-12T04:01:03.668 INFO:tasks.workunit.client.1.smithi033.stdout:3/199: dread f6 [0,4194304] 0 2024-05-12T04:01:03.673 INFO:tasks.workunit.client.1.smithi033.stdout:3/200: creat db/d17/d3e/f48 x:0 0 0 2024-05-12T04:01:03.673 INFO:tasks.workunit.client.1.smithi033.stdout:4/210: rmdir dd/d11 39 2024-05-12T04:01:03.675 INFO:tasks.workunit.client.1.smithi033.stdout:3/201: fdatasync db/d17/d25/f39 0 2024-05-12T04:01:03.676 INFO:tasks.workunit.client.1.smithi033.stdout:5/110: dwrite d3/f7 [0,4194304] 0 2024-05-12T04:01:03.677 INFO:tasks.workunit.client.1.smithi033.stdout:5/111: dread - d3/d12/f1f zero size 2024-05-12T04:01:03.688 INFO:tasks.workunit.client.1.smithi033.stdout:9/375: truncate d2/dd/d39/f54 1954040 0 2024-05-12T04:01:03.710 INFO:tasks.workunit.client.1.smithi033.stdout:4/211: rmdir dd/d11/d31 39 2024-05-12T04:01:03.710 INFO:tasks.workunit.client.1.smithi033.stdout:4/212: readlink l4 0 2024-05-12T04:01:03.711 INFO:tasks.workunit.client.1.smithi033.stdout:5/112: creat d3/d12/f24 x:0 0 0 2024-05-12T04:01:03.711 INFO:tasks.workunit.client.1.smithi033.stdout:2/165: truncate f9 1457932 0 2024-05-12T04:01:03.712 INFO:tasks.workunit.client.1.smithi033.stdout:3/202: mkdir db/d17/d25/d32/d49 0 2024-05-12T04:01:03.713 INFO:tasks.workunit.client.1.smithi033.stdout:2/166: truncate dc/f31 4928145 0 2024-05-12T04:01:03.713 INFO:tasks.workunit.client.1.smithi033.stdout:9/376: symlink d2/d70/l8f 0 2024-05-12T04:01:03.717 INFO:tasks.workunit.client.1.smithi033.stdout:5/113: fsync d3/d15/d16/f20 0 2024-05-12T04:01:03.721 INFO:tasks.workunit.client.1.smithi033.stdout:4/213: mkdir dd/d11/d31/d38 0 2024-05-12T04:01:03.726 INFO:tasks.workunit.client.1.smithi033.stdout:5/114: dwrite d3/d12/f1f [0,4194304] 0 2024-05-12T04:01:03.758 INFO:tasks.workunit.client.1.smithi033.stdout:3/203: creat db/d17/d25/d32/d3c/d44/f4a x:0 0 0 2024-05-12T04:01:03.758 INFO:tasks.workunit.client.1.smithi033.stdout:3/204: stat db/d17/d3a 0 2024-05-12T04:01:03.760 INFO:tasks.workunit.client.1.smithi033.stdout:4/214: symlink dd/d11/d31/l39 0 2024-05-12T04:01:03.761 INFO:tasks.workunit.client.1.smithi033.stdout:5/115: mkdir d3/d12/d25 0 2024-05-12T04:01:03.768 INFO:tasks.workunit.client.1.smithi033.stdout:3/205: dwrite db/d17/d25/d32/d3c/d44/f4a [0,4194304] 0 2024-05-12T04:01:03.772 INFO:tasks.workunit.client.1.smithi033.stdout:5/116: dwrite d3/d12/f14 [0,4194304] 0 2024-05-12T04:01:03.774 INFO:tasks.workunit.client.1.smithi033.stdout:3/206: write db/d17/d3a/f3b [1731933,372] 0 2024-05-12T04:01:03.775 INFO:tasks.workunit.client.1.smithi033.stdout:2/167: mkdir dc/d3b 0 2024-05-12T04:01:03.775 INFO:tasks.workunit.client.1.smithi033.stdout:5/117: write d3/f7 [1278989,123820] 0 2024-05-12T04:01:03.776 INFO:tasks.workunit.client.1.smithi033.stdout:5/118: chown d3/c11 16718 1 2024-05-12T04:01:03.779 INFO:tasks.workunit.client.1.smithi033.stdout:2/168: mknod dc/d13/d1c/c3c 0 2024-05-12T04:01:03.784 INFO:tasks.workunit.client.1.smithi033.stdout:1/137: sync 2024-05-12T04:01:03.787 INFO:tasks.workunit.client.1.smithi033.stdout:2/169: rename dc/d17/d27/c2e to dc/d13/d39/c3d 0 2024-05-12T04:01:03.790 INFO:tasks.workunit.client.1.smithi033.stdout:5/119: write d3/d15/d16/f20 [39585,22672] 0 2024-05-12T04:01:03.791 INFO:tasks.workunit.client.1.smithi033.stdout:1/138: chown d11/d1c/c25 0 1 2024-05-12T04:01:03.792 INFO:tasks.workunit.client.1.smithi033.stdout:2/170: symlink dc/d13/d39/l3e 0 2024-05-12T04:01:03.795 INFO:tasks.workunit.client.1.smithi033.stdout:5/120: unlink d3/d12/f24 0 2024-05-12T04:01:03.795 INFO:tasks.workunit.client.1.smithi033.stdout:2/171: dread dc/d13/f1b [0,4194304] 0 2024-05-12T04:01:03.802 INFO:tasks.workunit.client.1.smithi033.stdout:9/377: dwrite d2/d4/d2b/d6d/f69 [0,4194304] 0 2024-05-12T04:01:03.806 INFO:tasks.workunit.client.1.smithi033.stdout:5/121: mkdir d3/d12/d25/d26 0 2024-05-12T04:01:03.806 INFO:tasks.workunit.client.1.smithi033.stdout:5/122: dread - d3/d12/f1d zero size 2024-05-12T04:01:03.807 INFO:tasks.workunit.client.1.smithi033.stdout:1/139: getdents d11/d1c 0 2024-05-12T04:01:03.808 INFO:tasks.workunit.client.1.smithi033.stdout:1/140: write f8 [3745693,16687] 0 2024-05-12T04:01:03.808 INFO:tasks.workunit.client.1.smithi033.stdout:5/123: read d3/d15/d16/f1a [3907034,58953] 0 2024-05-12T04:01:03.809 INFO:tasks.workunit.client.1.smithi033.stdout:5/124: chown d3/d17 198666254 1 2024-05-12T04:01:03.809 INFO:tasks.workunit.client.1.smithi033.stdout:2/172: creat dc/f3f x:0 0 0 2024-05-12T04:01:03.811 INFO:tasks.workunit.client.1.smithi033.stdout:9/378: creat d2/d4/d2b/d6d/d7a/f90 x:0 0 0 2024-05-12T04:01:03.811 INFO:tasks.workunit.client.1.smithi033.stdout:0/266: sync 2024-05-12T04:01:03.811 INFO:tasks.workunit.client.1.smithi033.stdout:8/126: sync 2024-05-12T04:01:03.811 INFO:tasks.workunit.client.1.smithi033.stdout:7/301: sync 2024-05-12T04:01:03.811 INFO:tasks.workunit.client.1.smithi033.stdout:6/165: sync 2024-05-12T04:01:03.815 INFO:tasks.workunit.client.1.smithi033.stdout:2/173: unlink dc/d13/f16 0 2024-05-12T04:01:03.816 INFO:tasks.workunit.client.1.smithi033.stdout:1/141: write d11/f19 [530219,46120] 0 2024-05-12T04:01:03.816 INFO:tasks.workunit.client.1.smithi033.stdout:9/379: fsync d2/dd/f26 0 2024-05-12T04:01:03.818 INFO:tasks.workunit.client.1.smithi033.stdout:7/302: write de/df/d17/d1b/d50/f67 [332718,21786] 0 2024-05-12T04:01:03.819 INFO:tasks.workunit.client.1.smithi033.stdout:0/267: dread d6/d11/d57/f55 [0,4194304] 0 2024-05-12T04:01:03.821 INFO:tasks.workunit.client.1.smithi033.stdout:8/127: mkdir d1/d16/d27/d28 0 2024-05-12T04:01:03.822 INFO:tasks.workunit.client.1.smithi033.stdout:5/125: dwrite d3/f10 [0,4194304] 0 2024-05-12T04:01:03.823 INFO:tasks.workunit.client.1.smithi033.stdout:6/166: rename d4/l1c to d4/d5/l34 0 2024-05-12T04:01:03.825 INFO:tasks.workunit.client.1.smithi033.stdout:1/142: fdatasync d11/f13 0 2024-05-12T04:01:03.825 INFO:tasks.workunit.client.1.smithi033.stdout:5/126: read - d3/d17/f22 zero size 2024-05-12T04:01:03.826 INFO:tasks.workunit.client.1.smithi033.stdout:2/174: mknod dc/c40 0 2024-05-12T04:01:03.827 INFO:tasks.workunit.client.1.smithi033.stdout:4/215: dwrite dd/d11/f21 [0,4194304] 0 2024-05-12T04:01:03.827 INFO:tasks.workunit.client.1.smithi033.stdout:2/175: readlink dc/d13/d39/l3e 0 2024-05-12T04:01:03.831 INFO:tasks.workunit.client.1.smithi033.stdout:0/268: mkdir d6/dc/d58 0 2024-05-12T04:01:03.834 INFO:tasks.workunit.client.1.smithi033.stdout:7/303: creat de/d2b/d6c/f72 x:0 0 0 2024-05-12T04:01:03.836 INFO:tasks.workunit.client.1.smithi033.stdout:6/167: creat d4/d5/f35 x:0 0 0 2024-05-12T04:01:03.839 INFO:tasks.workunit.client.1.smithi033.stdout:4/216: symlink dd/d11/d31/l3a 0 2024-05-12T04:01:03.841 INFO:tasks.workunit.client.1.smithi033.stdout:3/207: truncate db/f31 3470318 0 2024-05-12T04:01:03.841 INFO:tasks.workunit.client.1.smithi033.stdout:0/269: dwrite d6/dc/f46 [0,4194304] 0 2024-05-12T04:01:03.842 INFO:tasks.workunit.client.1.smithi033.stdout:2/176: creat dc/d17/d27/d2f/f41 x:0 0 0 2024-05-12T04:01:03.844 INFO:tasks.workunit.client.1.smithi033.stdout:7/304: rmdir de/df/d17/d1b 39 2024-05-12T04:01:03.846 INFO:tasks.workunit.client.1.smithi033.stdout:7/305: fdatasync de/d2b/d6c/f5a 0 2024-05-12T04:01:03.847 INFO:tasks.workunit.client.1.smithi033.stdout:1/143: dwrite d11/f30 [0,4194304] 0 2024-05-12T04:01:03.848 INFO:tasks.workunit.client.1.smithi033.stdout:7/306: write de/df/f29 [196661,6336] 0 2024-05-12T04:01:03.851 INFO:tasks.workunit.client.1.smithi033.stdout:3/208: dread db/d17/f1c [4194304,4194304] 0 2024-05-12T04:01:03.854 INFO:tasks.workunit.client.1.smithi033.stdout:9/380: rmdir d2/d4/d2b/d33/d7e 0 2024-05-12T04:01:03.861 INFO:tasks.workunit.client.1.smithi033.stdout:7/307: mknod de/d20/d64/c73 0 2024-05-12T04:01:03.865 INFO:tasks.workunit.client.1.smithi033.stdout:5/127: dread d3/fc [0,4194304] 0 2024-05-12T04:01:03.865 INFO:tasks.workunit.client.1.smithi033.stdout:9/381: unlink d2/d4/d2b/d49/f7f 0 2024-05-12T04:01:03.866 INFO:tasks.workunit.client.1.smithi033.stdout:8/128: getdents d1/d4/d1d 0 2024-05-12T04:01:03.867 INFO:tasks.workunit.client.1.smithi033.stdout:2/177: mknod dc/d3b/c42 0 2024-05-12T04:01:03.868 INFO:tasks.workunit.client.1.smithi033.stdout:9/382: write d2/d4/f15 [1343356,105038] 0 2024-05-12T04:01:03.870 INFO:tasks.workunit.client.1.smithi033.stdout:9/383: write d2/d4/d2b/d78/f8c [1296724,22396] 0 2024-05-12T04:01:03.870 INFO:tasks.workunit.client.1.smithi033.stdout:1/144: rename d11/d17/f24 to d11/d1c/f33 0 2024-05-12T04:01:03.870 INFO:tasks.workunit.client.1.smithi033.stdout:2/178: truncate dc/d17/d27/d2f/f38 130438 0 2024-05-12T04:01:03.873 INFO:tasks.workunit.client.1.smithi033.stdout:9/384: readlink d2/d4/d2b/d6d/d3a/l57 0 2024-05-12T04:01:03.873 INFO:tasks.workunit.client.1.smithi033.stdout:4/217: dread dd/d11/f1d [0,4194304] 0 2024-05-12T04:01:03.874 INFO:tasks.workunit.client.1.smithi033.stdout:8/129: write d1/f13 [3514608,74339] 0 2024-05-12T04:01:03.876 INFO:tasks.workunit.client.1.smithi033.stdout:3/209: link db/d17/c19 db/d42/c4b 0 2024-05-12T04:01:03.878 INFO:tasks.workunit.client.1.smithi033.stdout:1/145: write d11/f2d [632483,44734] 0 2024-05-12T04:01:03.880 INFO:tasks.workunit.client.1.smithi033.stdout:2/179: creat dc/d17/d32/f43 x:0 0 0 2024-05-12T04:01:03.882 INFO:tasks.workunit.client.1.smithi033.stdout:3/210: write db/d17/d25/f40 [2038466,26778] 0 2024-05-12T04:01:03.885 INFO:tasks.workunit.client.1.smithi033.stdout:8/130: dread d1/d4/d1d/f26 [4194304,4194304] 0 2024-05-12T04:01:03.886 INFO:tasks.workunit.client.1.smithi033.stdout:9/385: write d2/d4/d29/d3c/d52/f86 [433523,113902] 0 2024-05-12T04:01:03.887 INFO:tasks.workunit.client.1.smithi033.stdout:3/211: fsync db/d17/d25/f40 0 2024-05-12T04:01:03.887 INFO:tasks.workunit.client.1.smithi033.stdout:5/128: rename d3/d17 to d3/d27 0 2024-05-12T04:01:03.887 INFO:tasks.workunit.client.1.smithi033.stdout:4/218: dwrite dd/d11/f19 [0,4194304] 0 2024-05-12T04:01:03.888 INFO:tasks.workunit.client.1.smithi033.stdout:8/131: chown d1/d16/d20 6676 1 2024-05-12T04:01:03.889 INFO:tasks.workunit.client.1.smithi033.stdout:9/386: readlink d2/d4/d2b/d49/l6f 0 2024-05-12T04:01:03.889 INFO:tasks.workunit.client.1.smithi033.stdout:4/219: chown dd/d11/f19 4041607 1 2024-05-12T04:01:03.890 INFO:tasks.workunit.client.1.smithi033.stdout:1/146: rename d11/d17/f23 to d11/d1c/f34 0 2024-05-12T04:01:03.890 INFO:tasks.workunit.client.1.smithi033.stdout:9/387: chown d2/d70 48 1 2024-05-12T04:01:03.892 INFO:tasks.workunit.client.1.smithi033.stdout:9/388: chown d2/dd 30843 1 2024-05-12T04:01:03.892 INFO:tasks.workunit.client.1.smithi033.stdout:2/180: mknod dc/d3b/c44 0 2024-05-12T04:01:03.897 INFO:tasks.workunit.client.1.smithi033.stdout:1/147: dread d11/f2d [0,4194304] 0 2024-05-12T04:01:03.898 INFO:tasks.workunit.client.1.smithi033.stdout:5/129: creat d3/d27/f28 x:0 0 0 2024-05-12T04:01:03.899 INFO:tasks.workunit.client.1.smithi033.stdout:3/212: dread db/d17/d25/d32/d3c/d44/f4a [0,4194304] 0 2024-05-12T04:01:03.899 INFO:tasks.workunit.client.1.smithi033.stdout:8/132: dwrite d1/d4/d1d/f1e [0,4194304] 0 2024-05-12T04:01:03.900 INFO:tasks.workunit.client.1.smithi033.stdout:5/130: fdatasync d3/d12/f1f 0 2024-05-12T04:01:03.902 INFO:tasks.workunit.client.1.smithi033.stdout:3/213: chown db/l2a 5730 1 2024-05-12T04:01:03.902 INFO:tasks.workunit.client.1.smithi033.stdout:7/308: truncate de/df/d17/f34 2754223 0 2024-05-12T04:01:03.907 INFO:tasks.workunit.client.1.smithi033.stdout:3/214: dread db/d17/f1b [0,4194304] 0 2024-05-12T04:01:03.910 INFO:tasks.workunit.client.1.smithi033.stdout:8/133: dwrite d1/d4/f1c [0,4194304] 0 2024-05-12T04:01:03.912 INFO:tasks.workunit.client.1.smithi033.stdout:5/131: rename d3/d12/d13 to d3/d12/d29 0 2024-05-12T04:01:03.914 INFO:tasks.workunit.client.1.smithi033.stdout:8/134: write d1/d4/fd [1188665,119831] 0 2024-05-12T04:01:03.915 INFO:tasks.workunit.client.1.smithi033.stdout:2/181: unlink dc/f10 0 2024-05-12T04:01:03.916 INFO:tasks.workunit.client.1.smithi033.stdout:9/389: symlink d2/dd/l91 0 2024-05-12T04:01:03.918 INFO:tasks.workunit.client.1.smithi033.stdout:4/220: creat dd/f3b x:0 0 0 2024-05-12T04:01:03.922 INFO:tasks.workunit.client.1.smithi033.stdout:7/309: creat de/d2f/d70/f74 x:0 0 0 2024-05-12T04:01:03.922 INFO:tasks.workunit.client.1.smithi033.stdout:8/135: write d1/f13 [3133697,105521] 0 2024-05-12T04:01:03.922 INFO:tasks.workunit.client.1.smithi033.stdout:2/182: chown dc/l11 9 1 2024-05-12T04:01:03.922 INFO:tasks.workunit.client.1.smithi033.stdout:3/215: dwrite db/d17/d25/f39 [0,4194304] 0 2024-05-12T04:01:03.925 INFO:tasks.workunit.client.1.smithi033.stdout:2/183: read dc/d17/d27/d2f/f38 [95938,64172] 0 2024-05-12T04:01:03.926 INFO:tasks.workunit.client.1.smithi033.stdout:5/132: dread d3/d15/d16/f1a [0,4194304] 0 2024-05-12T04:01:03.927 INFO:tasks.workunit.client.1.smithi033.stdout:4/221: mknod dd/c3c 0 2024-05-12T04:01:03.928 INFO:tasks.workunit.client.1.smithi033.stdout:5/133: chown d3/d12/f14 1222755 1 2024-05-12T04:01:03.929 INFO:tasks.workunit.client.1.smithi033.stdout:5/134: readlink d3/l8 0 2024-05-12T04:01:03.929 INFO:tasks.workunit.client.1.smithi033.stdout:9/390: mknod d2/d4/d29/d3c/d52/c92 0 2024-05-12T04:01:03.930 INFO:tasks.workunit.client.1.smithi033.stdout:8/136: readlink d1/d16/l1a 0 2024-05-12T04:01:03.931 INFO:tasks.workunit.client.1.smithi033.stdout:5/135: fsync d3/fb 0 2024-05-12T04:01:03.932 INFO:tasks.workunit.client.1.smithi033.stdout:7/310: mkdir de/df/d60/d75 0 2024-05-12T04:01:03.934 INFO:tasks.workunit.client.1.smithi033.stdout:2/184: symlink dc/d3b/l45 0 2024-05-12T04:01:03.935 INFO:tasks.workunit.client.1.smithi033.stdout:9/391: chown d2/d4/d2b/d6d/d3a/f45 91 1 2024-05-12T04:01:03.935 INFO:tasks.workunit.client.1.smithi033.stdout:3/216: dwrite f6 [0,4194304] 0 2024-05-12T04:01:03.936 INFO:tasks.workunit.client.1.smithi033.stdout:9/392: stat d2/dd/f26 0 2024-05-12T04:01:03.937 INFO:tasks.workunit.client.1.smithi033.stdout:2/185: write f7 [6343741,29336] 0 2024-05-12T04:01:03.938 INFO:tasks.workunit.client.1.smithi033.stdout:9/393: dread - d2/f8e zero size 2024-05-12T04:01:03.940 INFO:tasks.workunit.client.1.smithi033.stdout:9/394: fdatasync d2/d4/d29/d3c/d52/f86 0 2024-05-12T04:01:03.943 INFO:tasks.workunit.client.1.smithi033.stdout:7/311: rename de/d2f/f4e to de/df/d60/f76 0 2024-05-12T04:01:03.944 INFO:tasks.workunit.client.1.smithi033.stdout:2/186: dread dc/d13/f1d [0,4194304] 0 2024-05-12T04:01:03.945 INFO:tasks.workunit.client.1.smithi033.stdout:3/217: write db/d17/f1c [9374543,90004] 0 2024-05-12T04:01:03.946 INFO:tasks.workunit.client.1.smithi033.stdout:8/137: unlink d1/d16/l17 0 2024-05-12T04:01:03.947 INFO:tasks.workunit.client.1.smithi033.stdout:3/218: chown db/d17/c1d 277946 1 2024-05-12T04:01:03.948 INFO:tasks.workunit.client.1.smithi033.stdout:5/136: symlink d3/d12/d25/d26/l2a 0 2024-05-12T04:01:03.948 INFO:tasks.workunit.client.1.smithi033.stdout:9/395: dread d2/d4/d29/f2a [0,4194304] 0 2024-05-12T04:01:03.950 INFO:tasks.workunit.client.1.smithi033.stdout:5/137: write d3/ff [1225821,118280] 0 2024-05-12T04:01:03.950 INFO:tasks.workunit.client.1.smithi033.stdout:9/396: chown d2/d4/d2b/d49/d73/l75 43 1 2024-05-12T04:01:03.951 INFO:tasks.workunit.client.1.smithi033.stdout:4/222: creat dd/f3d x:0 0 0 2024-05-12T04:01:03.952 INFO:tasks.workunit.client.1.smithi033.stdout:8/138: dread d1/d4/f1c [0,4194304] 0 2024-05-12T04:01:03.953 INFO:tasks.workunit.client.1.smithi033.stdout:8/139: write d1/f2 [2700107,116144] 0 2024-05-12T04:01:03.957 INFO:tasks.workunit.client.1.smithi033.stdout:2/187: symlink dc/d17/d27/d2f/l46 0 2024-05-12T04:01:03.957 INFO:tasks.workunit.client.1.smithi033.stdout:3/219: symlink db/d17/d25/l4c 0 2024-05-12T04:01:03.958 INFO:tasks.workunit.client.1.smithi033.stdout:2/188: truncate dc/d17/d27/d2f/f41 97337 0 2024-05-12T04:01:03.961 INFO:tasks.workunit.client.1.smithi033.stdout:9/397: rmdir d2/dd/d39 39 2024-05-12T04:01:03.966 INFO:tasks.workunit.client.1.smithi033.stdout:8/140: mkdir d1/d16/d29 0 2024-05-12T04:01:03.969 INFO:tasks.workunit.client.1.smithi033.stdout:2/189: write dc/d13/d1f/f24 [1979930,84586] 0 2024-05-12T04:01:03.969 INFO:tasks.workunit.client.1.smithi033.stdout:7/312: link de/d20/c3a de/d2b/d6c/c77 0 2024-05-12T04:01:03.971 INFO:tasks.workunit.client.1.smithi033.stdout:9/398: dwrite d2/d4/d2b/d6d/f5c [0,4194304] 0 2024-05-12T04:01:03.971 INFO:tasks.workunit.client.1.smithi033.stdout:8/141: mkdir d1/d16/d27/d2a 0 2024-05-12T04:01:03.972 INFO:tasks.workunit.client.1.smithi033.stdout:4/223: creat dd/f3e x:0 0 0 2024-05-12T04:01:03.973 INFO:tasks.workunit.client.1.smithi033.stdout:9/399: chown d2/f27 6579 1 2024-05-12T04:01:03.973 INFO:tasks.workunit.client.1.smithi033.stdout:8/142: write d1/f12 [1438816,5150] 0 2024-05-12T04:01:03.974 INFO:tasks.workunit.client.1.smithi033.stdout:7/313: unlink de/f1a 0 2024-05-12T04:01:03.975 INFO:tasks.workunit.client.1.smithi033.stdout:9/400: write d2/f8e [319032,111418] 0 2024-05-12T04:01:03.975 INFO:tasks.workunit.client.1.smithi033.stdout:8/143: chown d1/f2 0 1 2024-05-12T04:01:03.981 INFO:tasks.workunit.client.1.smithi033.stdout:7/314: mknod de/df/c78 0 2024-05-12T04:01:03.981 INFO:tasks.workunit.client.1.smithi033.stdout:5/138: write d3/fc [658256,117112] 0 2024-05-12T04:01:03.982 INFO:tasks.workunit.client.1.smithi033.stdout:9/401: creat d2/d4/d2b/d78/f93 x:0 0 0 2024-05-12T04:01:03.983 INFO:tasks.workunit.client.1.smithi033.stdout:8/144: write d1/d4/f1c [440459,27272] 0 2024-05-12T04:01:03.985 INFO:tasks.workunit.client.1.smithi033.stdout:5/139: mkdir d3/d12/d29/d2b 0 2024-05-12T04:01:03.987 INFO:tasks.workunit.client.1.smithi033.stdout:5/140: truncate d3/f1c 1105710 0 2024-05-12T04:01:03.987 INFO:tasks.workunit.client.1.smithi033.stdout:9/402: chown d2/dd/d39/f43 1 1 2024-05-12T04:01:03.987 INFO:tasks.workunit.client.1.smithi033.stdout:7/315: mkdir de/df/d17/d1e/d79 0 2024-05-12T04:01:03.990 INFO:tasks.workunit.client.1.smithi033.stdout:7/316: mkdir de/df/d7a 0 2024-05-12T04:01:03.991 INFO:tasks.workunit.client.1.smithi033.stdout:7/317: fdatasync de/d20/f21 0 2024-05-12T04:01:03.992 INFO:tasks.workunit.client.1.smithi033.stdout:8/145: link d1/d4/d1d/f1e d1/d4/f2b 0 2024-05-12T04:01:03.993 INFO:tasks.workunit.client.1.smithi033.stdout:2/190: write f9 [17808,46869] 0 2024-05-12T04:01:03.993 INFO:tasks.workunit.client.1.smithi033.stdout:8/146: fsync d1/d4/fd 0 2024-05-12T04:01:03.994 INFO:tasks.workunit.client.1.smithi033.stdout:4/224: write fc [1710099,55707] 0 2024-05-12T04:01:03.995 INFO:tasks.workunit.client.1.smithi033.stdout:7/318: mknod de/df/d60/c7b 0 2024-05-12T04:01:03.995 INFO:tasks.workunit.client.1.smithi033.stdout:4/225: chown dd/f35 284350 1 2024-05-12T04:01:03.997 INFO:tasks.workunit.client.1.smithi033.stdout:5/141: creat d3/f2c x:0 0 0 2024-05-12T04:01:03.998 INFO:tasks.workunit.client.1.smithi033.stdout:5/142: dread - d3/d27/f28 zero size 2024-05-12T04:01:03.999 INFO:tasks.workunit.client.1.smithi033.stdout:8/147: dread d1/f13 [0,4194304] 0 2024-05-12T04:01:03.999 INFO:tasks.workunit.client.1.smithi033.stdout:5/143: write d3/f2c [261863,96087] 0 2024-05-12T04:01:04.001 INFO:tasks.workunit.client.1.smithi033.stdout:5/144: read - d3/d27/f22 zero size 2024-05-12T04:01:04.001 INFO:tasks.workunit.client.1.smithi033.stdout:4/226: symlink dd/d11/l3f 0 2024-05-12T04:01:04.001 INFO:tasks.workunit.client.1.smithi033.stdout:2/191: dread f9 [0,4194304] 0 2024-05-12T04:01:04.002 INFO:tasks.workunit.client.1.smithi033.stdout:8/148: rmdir d1/d4/d1d 39 2024-05-12T04:01:04.002 INFO:tasks.workunit.client.1.smithi033.stdout:8/149: readlink d1/l9 0 2024-05-12T04:01:04.003 INFO:tasks.workunit.client.1.smithi033.stdout:7/319: mknod de/df/d17/d1b/d50/c7c 0 2024-05-12T04:01:04.004 INFO:tasks.workunit.client.1.smithi033.stdout:2/192: dread f9 [0,4194304] 0 2024-05-12T04:01:04.004 INFO:tasks.workunit.client.1.smithi033.stdout:5/145: mknod d3/d12/d29/d2b/c2d 0 2024-05-12T04:01:04.005 INFO:tasks.workunit.client.1.smithi033.stdout:4/227: mknod dd/c40 0 2024-05-12T04:01:04.006 INFO:tasks.workunit.client.1.smithi033.stdout:7/320: symlink de/d2f/d42/l7d 0 2024-05-12T04:01:04.008 INFO:tasks.workunit.client.1.smithi033.stdout:4/228: truncate dd/f34 69607 0 2024-05-12T04:01:04.010 INFO:tasks.workunit.client.1.smithi033.stdout:2/193: truncate dc/d17/d27/d2f/f41 451583 0 2024-05-12T04:01:04.012 INFO:tasks.workunit.client.1.smithi033.stdout:7/321: read de/d2b/d6c/f39 [126683,105831] 0 2024-05-12T04:01:04.012 INFO:tasks.workunit.client.1.smithi033.stdout:8/150: dwrite d1/f2 [0,4194304] 0 2024-05-12T04:01:04.012 INFO:tasks.workunit.client.1.smithi033.stdout:4/229: mknod dd/d11/d15/c41 0 2024-05-12T04:01:04.015 INFO:tasks.workunit.client.1.smithi033.stdout:4/230: fsync dd/f36 0 2024-05-12T04:01:04.016 INFO:tasks.workunit.client.1.smithi033.stdout:5/146: dwrite d3/d27/f28 [0,4194304] 0 2024-05-12T04:01:04.018 INFO:tasks.workunit.client.1.smithi033.stdout:8/151: unlink d1/cb 0 2024-05-12T04:01:04.019 INFO:tasks.workunit.client.1.smithi033.stdout:5/147: stat d3/f7 0 2024-05-12T04:01:04.022 INFO:tasks.workunit.client.1.smithi033.stdout:2/194: dread dc/d13/d1f/f24 [0,4194304] 0 2024-05-12T04:01:04.023 INFO:tasks.workunit.client.1.smithi033.stdout:4/231: mkdir dd/d11/d15/d42 0 2024-05-12T04:01:04.024 INFO:tasks.workunit.client.1.smithi033.stdout:2/195: chown dc/d17/f23 348 1 2024-05-12T04:01:04.025 INFO:tasks.workunit.client.1.smithi033.stdout:2/196: write dc/d13/f15 [1665082,74668] 0 2024-05-12T04:01:04.027 INFO:tasks.workunit.client.1.smithi033.stdout:4/232: write dd/d11/f1d [2283526,98611] 0 2024-05-12T04:01:04.027 INFO:tasks.workunit.client.1.smithi033.stdout:7/322: link de/df/d17/c63 de/df/d60/c7e 0 2024-05-12T04:01:04.028 INFO:tasks.workunit.client.1.smithi033.stdout:5/148: dread d3/d12/f1f [0,4194304] 0 2024-05-12T04:01:04.029 INFO:tasks.workunit.client.1.smithi033.stdout:5/149: stat d3/d12/l19 0 2024-05-12T04:01:04.030 INFO:tasks.workunit.client.1.smithi033.stdout:7/323: truncate de/d2b/f2c 941951 0 2024-05-12T04:01:04.031 INFO:tasks.workunit.client.1.smithi033.stdout:2/197: mkdir dc/d17/d27/d47 0 2024-05-12T04:01:04.032 INFO:tasks.workunit.client.1.smithi033.stdout:7/324: fdatasync de/df/d17/d1e/f23 0 2024-05-12T04:01:04.033 INFO:tasks.workunit.client.1.smithi033.stdout:6/168: sync 2024-05-12T04:01:04.033 INFO:tasks.workunit.client.1.smithi033.stdout:5/150: mkdir d3/d12/d25/d26/d2e 0 2024-05-12T04:01:04.034 INFO:tasks.workunit.client.1.smithi033.stdout:7/325: dread - de/df/d17/d1e/f4c zero size 2024-05-12T04:01:04.036 INFO:tasks.workunit.client.1.smithi033.stdout:4/233: rename dd/c16 to dd/d11/d31/d38/c43 0 2024-05-12T04:01:04.039 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:03 smithi032 ceph-mon[40008]: pgmap v30: 65 pgs: 65 active+clean; 395 MiB data, 2.9 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 24 MiB/s wr, 112 op/s 2024-05-12T04:01:04.040 INFO:tasks.workunit.client.1.smithi033.stdout:6/169: symlink d4/l36 0 2024-05-12T04:01:04.043 INFO:tasks.workunit.client.1.smithi033.stdout:6/170: mknod d4/c37 0 2024-05-12T04:01:04.043 INFO:tasks.workunit.client.1.smithi033.stdout:6/171: stat d4/d1a/f23 0 2024-05-12T04:01:04.043 INFO:tasks.workunit.client.1.smithi033.stdout:7/326: getdents de/df/d7a 0 2024-05-12T04:01:04.046 INFO:tasks.workunit.client.1.smithi033.stdout:7/327: dread - de/df/d17/d1e/f23 zero size 2024-05-12T04:01:04.047 INFO:tasks.workunit.client.1.smithi033.stdout:7/328: mkdir de/d20/d38/d7f 0 2024-05-12T04:01:04.055 INFO:tasks.workunit.client.1.smithi033.stdout:5/151: dread d3/fb [0,4194304] 0 2024-05-12T04:01:04.056 INFO:tasks.workunit.client.1.smithi033.stdout:2/198: dwrite dc/d17/d27/d2f/f38 [0,4194304] 0 2024-05-12T04:01:04.058 INFO:tasks.workunit.client.1.smithi033.stdout:7/329: truncate de/df/d17/f51 4001179 0 2024-05-12T04:01:04.064 INFO:tasks.workunit.client.1.smithi033.stdout:2/199: rmdir dc/d17/d27/d2f 39 2024-05-12T04:01:04.067 INFO:tasks.workunit.client.1.smithi033.stdout:2/200: dread f3 [0,4194304] 0 2024-05-12T04:01:04.068 INFO:tasks.workunit.client.1.smithi033.stdout:2/201: read f3 [183075,30082] 0 2024-05-12T04:01:04.073 INFO:tasks.workunit.client.1.smithi033.stdout:7/330: dwrite de/d2b/f2c [0,4194304] 0 2024-05-12T04:01:04.074 INFO:tasks.workunit.client.1.smithi033.stdout:7/331: mkdir de/d80 0 2024-05-12T04:01:04.076 INFO:tasks.workunit.client.1.smithi033.stdout:2/202: dread dc/d13/d1c/f21 [0,4194304] 0 2024-05-12T04:01:04.078 INFO:tasks.workunit.client.1.smithi033.stdout:2/203: mknod dc/d13/d1f/c48 0 2024-05-12T04:01:04.079 INFO:tasks.workunit.client.1.smithi033.stdout:2/204: chown dc/d17/d27 1939 1 2024-05-12T04:01:04.079 INFO:tasks.workunit.client.1.smithi033.stdout:7/332: symlink de/d80/l81 0 2024-05-12T04:01:04.083 INFO:tasks.workunit.client.1.smithi033.stdout:2/205: mkdir dc/d17/d27/d2f/d49 0 2024-05-12T04:01:04.083 INFO:tasks.workunit.client.1.smithi033.stdout:7/333: dread de/d2b/f2c [0,4194304] 0 2024-05-12T04:01:04.086 INFO:tasks.workunit.client.1.smithi033.stdout:2/206: link f7 dc/d3b/f4a 0 2024-05-12T04:01:04.087 INFO:tasks.workunit.client.1.smithi033.stdout:7/334: mknod de/df/d17/d1b/d50/c82 0 2024-05-12T04:01:04.088 INFO:tasks.workunit.client.1.smithi033.stdout:7/335: fsync de/d20/f21 0 2024-05-12T04:01:04.089 INFO:tasks.workunit.client.1.smithi033.stdout:2/207: dread dc/d13/f1d [0,4194304] 0 2024-05-12T04:01:04.091 INFO:tasks.workunit.client.1.smithi033.stdout:2/208: write dc/d17/d27/d2f/f41 [826688,97630] 0 2024-05-12T04:01:04.091 INFO:tasks.workunit.client.1.smithi033.stdout:7/336: creat de/d20/f83 x:0 0 0 2024-05-12T04:01:04.092 INFO:tasks.workunit.client.1.smithi033.stdout:7/337: write de/df/d17/d1e/d24/f57 [8498274,87978] 0 2024-05-12T04:01:04.093 INFO:tasks.workunit.client.1.smithi033.stdout:2/209: symlink dc/d3b/l4b 0 2024-05-12T04:01:04.095 INFO:tasks.workunit.client.1.smithi033.stdout:2/210: mknod dc/d17/d32/c4c 0 2024-05-12T04:01:04.104 INFO:tasks.workunit.client.1.smithi033.stdout:7/338: truncate de/df/d17/d1b/d46/f6d 396848 0 2024-05-12T04:01:04.105 INFO:tasks.workunit.client.1.smithi033.stdout:7/339: readlink de/d2b/d6c/l5b 0 2024-05-12T04:01:04.110 INFO:tasks.workunit.client.1.smithi033.stdout:2/211: dwrite f6 [0,4194304] 0 2024-05-12T04:01:04.110 INFO:tasks.workunit.client.1.smithi033.stdout:7/340: getdents de/df/d17/d1e 0 2024-05-12T04:01:04.112 INFO:tasks.workunit.client.1.smithi033.stdout:7/341: chown de/df/d17/d1b/d36/c3e 10608 1 2024-05-12T04:01:04.114 INFO:tasks.workunit.client.1.smithi033.stdout:7/342: symlink de/d80/l84 0 2024-05-12T04:01:04.115 INFO:tasks.workunit.client.1.smithi033.stdout:7/343: stat de/d2f/d70/f74 0 2024-05-12T04:01:04.116 INFO:tasks.workunit.client.1.smithi033.stdout:2/212: creat dc/d17/f4d x:0 0 0 2024-05-12T04:01:04.118 INFO:tasks.workunit.client.1.smithi033.stdout:2/213: write dc/d17/d27/d2f/f38 [4597575,111554] 0 2024-05-12T04:01:04.119 INFO:tasks.workunit.client.1.smithi033.stdout:7/344: rename de/d2b/d6c/l5b to de/d2f/d70/l85 0 2024-05-12T04:01:04.122 INFO:tasks.workunit.client.1.smithi033.stdout:2/214: truncate dc/d3b/f4a 2203496 0 2024-05-12T04:01:04.122 INFO:tasks.workunit.client.1.smithi033.stdout:7/345: dread de/d20/d38/f61 [0,4194304] 0 2024-05-12T04:01:04.123 INFO:tasks.workunit.client.1.smithi033.stdout:7/346: chown de/d2b/d6c/f72 3 1 2024-05-12T04:01:04.124 INFO:tasks.workunit.client.1.smithi033.stdout:2/215: symlink dc/d17/d3a/l4e 0 2024-05-12T04:01:04.124 INFO:tasks.workunit.client.1.smithi033.stdout:7/347: chown de/d20/d64/c73 939097223 1 2024-05-12T04:01:04.125 INFO:tasks.workunit.client.1.smithi033.stdout:7/348: read de/d2b/f2c [3514007,39478] 0 2024-05-12T04:01:04.125 INFO:tasks.workunit.client.1.smithi033.stdout:2/216: fdatasync f9 0 2024-05-12T04:01:04.125 INFO:tasks.workunit.client.1.smithi033.stdout:7/349: chown de/d20 10784 1 2024-05-12T04:01:04.127 INFO:tasks.workunit.client.1.smithi033.stdout:2/217: creat dc/d13/d1c/f4f x:0 0 0 2024-05-12T04:01:04.127 INFO:tasks.workunit.client.1.smithi033.stdout:7/350: getdents de/df/d60/d75 0 2024-05-12T04:01:04.128 INFO:tasks.workunit.client.1.smithi033.stdout:2/218: write dc/f31 [4851731,3398] 0 2024-05-12T04:01:04.128 INFO:tasks.workunit.client.1.smithi033.stdout:7/351: dread - de/df/d17/d1e/f23 zero size 2024-05-12T04:01:04.134 INFO:tasks.workunit.client.1.smithi033.stdout:7/352: dwrite de/d2b/d6c/f5a [0,4194304] 0 2024-05-12T04:01:04.139 INFO:tasks.workunit.client.1.smithi033.stdout:7/353: fdatasync de/df/d17/f51 0 2024-05-12T04:01:04.142 INFO:tasks.workunit.client.1.smithi033.stdout:7/354: creat de/df/d65/f86 x:0 0 0 2024-05-12T04:01:04.149 INFO:tasks.workunit.client.1.smithi033.stdout:7/355: dwrite de/df/d65/f86 [0,4194304] 0 2024-05-12T04:01:04.154 INFO:tasks.workunit.client.1.smithi033.stdout:7/356: dread de/d2b/f2c [0,4194304] 0 2024-05-12T04:01:04.158 INFO:tasks.workunit.client.1.smithi033.stdout:7/357: unlink de/df/d17/d1b/d46/f6d 0 2024-05-12T04:01:04.160 INFO:tasks.workunit.client.1.smithi033.stdout:7/358: write de/d2b/d6c/f39 [1532426,14277] 0 2024-05-12T04:01:04.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:03 smithi033 ceph-mon[49525]: pgmap v30: 65 pgs: 65 active+clean; 395 MiB data, 2.9 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 24 MiB/s wr, 112 op/s 2024-05-12T04:01:04.163 INFO:tasks.workunit.client.1.smithi033.stdout:7/359: mknod de/df/d17/d1e/d24/c87 0 2024-05-12T04:01:04.164 INFO:tasks.workunit.client.1.smithi033.stdout:7/360: read de/df/d17/d1b/d46/f53 [52326,82804] 0 2024-05-12T04:01:04.168 INFO:tasks.workunit.client.1.smithi033.stdout:7/361: unlink de/df/d17/d1e/d24/l2e 0 2024-05-12T04:01:04.168 INFO:tasks.workunit.client.1.smithi033.stdout:7/362: chown de/df/c59 3241 1 2024-05-12T04:01:04.171 INFO:tasks.workunit.client.1.smithi033.stdout:7/363: creat de/df/d17/f88 x:0 0 0 2024-05-12T04:01:04.172 INFO:tasks.workunit.client.1.smithi033.stdout:7/364: dread - de/df/d17/f6a zero size 2024-05-12T04:01:04.175 INFO:tasks.workunit.client.1.smithi033.stdout:0/270: sync 2024-05-12T04:01:04.177 INFO:tasks.workunit.client.1.smithi033.stdout:7/365: dread de/df/f48 [0,4194304] 0 2024-05-12T04:01:04.180 INFO:tasks.workunit.client.1.smithi033.stdout:7/366: creat de/df/d17/d1b/f89 x:0 0 0 2024-05-12T04:01:04.183 INFO:tasks.workunit.client.1.smithi033.stdout:7/367: creat de/df/d17/d1b/d36/f8a x:0 0 0 2024-05-12T04:01:04.183 INFO:tasks.workunit.client.1.smithi033.stdout:0/271: dread d6/dc/f16 [0,4194304] 0 2024-05-12T04:01:04.184 INFO:tasks.workunit.client.1.smithi033.stdout:7/368: write de/df/d17/d1e/d24/f57 [5728419,44089] 0 2024-05-12T04:01:04.185 INFO:tasks.workunit.client.1.smithi033.stdout:7/369: dread - de/df/d17/d1e/d24/f6f zero size 2024-05-12T04:01:04.188 INFO:tasks.workunit.client.1.smithi033.stdout:7/370: creat de/df/d7a/f8b x:0 0 0 2024-05-12T04:01:04.190 INFO:tasks.workunit.client.1.smithi033.stdout:7/371: symlink de/df/d7a/l8c 0 2024-05-12T04:01:04.193 INFO:tasks.workunit.client.1.smithi033.stdout:7/372: rename de/df/d17/d1b/d50/l55 to de/df/d17/d1e/l8d 0 2024-05-12T04:01:04.194 INFO:tasks.workunit.client.1.smithi033.stdout:0/272: dread d6/d11/f28 [0,4194304] 0 2024-05-12T04:01:04.196 INFO:tasks.workunit.client.1.smithi033.stdout:7/373: readlink de/d20/l35 0 2024-05-12T04:01:04.196 INFO:tasks.workunit.client.1.smithi033.stdout:0/273: write d6/dc/f56 [341094,99960] 0 2024-05-12T04:01:04.197 INFO:tasks.workunit.client.1.smithi033.stdout:7/374: symlink de/df/d60/l8e 0 2024-05-12T04:01:04.197 INFO:tasks.workunit.client.1.smithi033.stdout:0/274: stat d6/db/f19 0 2024-05-12T04:01:04.198 INFO:tasks.workunit.client.1.smithi033.stdout:7/375: chown de/df/d17/d1b/d50/c82 2057337 1 2024-05-12T04:01:04.198 INFO:tasks.workunit.client.1.smithi033.stdout:0/275: readlink d6/db/d2c/l52 0 2024-05-12T04:01:04.201 INFO:tasks.workunit.client.1.smithi033.stdout:7/376: unlink de/d2f/d42/l54 0 2024-05-12T04:01:04.204 INFO:tasks.workunit.client.1.smithi033.stdout:0/276: unlink d6/db/f13 0 2024-05-12T04:01:04.207 INFO:tasks.workunit.client.1.smithi033.stdout:7/377: creat de/df/d60/d75/f8f x:0 0 0 2024-05-12T04:01:04.208 INFO:tasks.workunit.client.1.smithi033.stdout:0/277: creat d6/dc/d58/f59 x:0 0 0 2024-05-12T04:01:04.211 INFO:tasks.workunit.client.1.smithi033.stdout:0/278: dread d6/d11/f1c [0,4194304] 0 2024-05-12T04:01:04.213 INFO:tasks.workunit.client.1.smithi033.stdout:0/279: symlink d6/d11/l5a 0 2024-05-12T04:01:04.214 INFO:tasks.workunit.client.1.smithi033.stdout:0/280: chown d6/dc 860260530 1 2024-05-12T04:01:04.219 INFO:tasks.workunit.client.0.smithi032.stdout:3/71: sync 2024-05-12T04:01:04.219 INFO:tasks.workunit.client.0.smithi032.stdout:9/17: sync 2024-05-12T04:01:04.219 INFO:tasks.workunit.client.0.smithi032.stdout:4/8: sync 2024-05-12T04:01:04.219 INFO:tasks.workunit.client.0.smithi032.stdout:6/30: sync 2024-05-12T04:01:04.219 INFO:tasks.workunit.client.0.smithi032.stdout:0/91: sync 2024-05-12T04:01:04.219 INFO:tasks.workunit.client.0.smithi032.stdout:5/61: sync 2024-05-12T04:01:04.220 INFO:tasks.workunit.client.0.smithi032.stdout:1/27: sync 2024-05-12T04:01:04.220 INFO:tasks.workunit.client.0.smithi032.stdout:6/31: chown f6 34 1 2024-05-12T04:01:04.221 INFO:tasks.workunit.client.0.smithi032.stdout:9/18: readlink d0/l2 0 2024-05-12T04:01:04.221 INFO:tasks.workunit.client.0.smithi032.stdout:9/19: truncate - no filename 2024-05-12T04:01:04.221 INFO:tasks.workunit.client.0.smithi032.stdout:9/20: chown d0 87 1 2024-05-12T04:01:04.226 INFO:tasks.workunit.client.0.smithi032.stdout:1/28: mknod d1/cb 0 2024-05-12T04:01:04.226 INFO:tasks.workunit.client.0.smithi032.stdout:1/29: chown d1/l2 1223 1 2024-05-12T04:01:04.227 INFO:tasks.workunit.client.0.smithi032.stdout:4/9: fsync d0/f2 0 2024-05-12T04:01:04.227 INFO:tasks.workunit.client.0.smithi032.stdout:4/10: chown d0 65813199 1 2024-05-12T04:01:04.230 INFO:tasks.workunit.client.0.smithi032.stdout:9/21: mkdir d0/d3 0 2024-05-12T04:01:04.235 INFO:tasks.workunit.client.0.smithi032.stdout:9/22: rmdir d0 39 2024-05-12T04:01:04.238 INFO:tasks.workunit.client.0.smithi032.stdout:9/23: stat d0/d3 0 2024-05-12T04:01:04.238 INFO:tasks.workunit.client.0.smithi032.stdout:9/24: write - no filename 2024-05-12T04:01:04.240 INFO:tasks.workunit.client.0.smithi032.stdout:9/25: creat d0/f4 x:0 0 0 2024-05-12T04:01:04.242 INFO:tasks.workunit.client.0.smithi032.stdout:9/26: mknod d0/d3/c5 0 2024-05-12T04:01:04.243 INFO:tasks.workunit.client.0.smithi032.stdout:9/27: chown d0 16174954 1 2024-05-12T04:01:04.245 INFO:tasks.workunit.client.0.smithi032.stdout:4/11: write d0/f2 [571670,11714] 0 2024-05-12T04:01:04.246 INFO:tasks.workunit.client.0.smithi032.stdout:9/28: stat d0/l2 0 2024-05-12T04:01:04.246 INFO:tasks.workunit.client.0.smithi032.stdout:1/30: write d1/f3 [3677606,110965] 0 2024-05-12T04:01:04.254 INFO:tasks.workunit.client.0.smithi032.stdout:9/29: mkdir d0/d3/d6 0 2024-05-12T04:01:04.254 INFO:tasks.workunit.client.0.smithi032.stdout:4/12: chown d0/f2 3342 1 2024-05-12T04:01:04.258 INFO:tasks.workunit.client.0.smithi032.stdout:4/13: fdatasync d0/f2 0 2024-05-12T04:01:04.260 INFO:tasks.workunit.client.0.smithi032.stdout:0/92: dwrite f5 [4194304,4194304] 0 2024-05-12T04:01:04.266 INFO:tasks.workunit.client.0.smithi032.stdout:9/30: mkdir d0/d3/d6/d7 0 2024-05-12T04:01:04.267 INFO:tasks.workunit.client.0.smithi032.stdout:6/32: dwrite f6 [0,4194304] 0 2024-05-12T04:01:04.270 INFO:tasks.workunit.client.0.smithi032.stdout:5/62: dwrite f2 [0,4194304] 0 2024-05-12T04:01:04.271 INFO:tasks.workunit.client.0.smithi032.stdout:4/14: write d0/f2 [1602915,102232] 0 2024-05-12T04:01:04.274 INFO:tasks.workunit.client.0.smithi032.stdout:3/72: dwrite f14 [0,4194304] 0 2024-05-12T04:01:04.276 INFO:tasks.workunit.client.0.smithi032.stdout:6/33: mkdir d7 0 2024-05-12T04:01:04.278 INFO:tasks.workunit.client.0.smithi032.stdout:9/31: truncate d0/f4 855590 0 2024-05-12T04:01:04.278 INFO:tasks.workunit.client.0.smithi032.stdout:7/13: sync 2024-05-12T04:01:04.278 INFO:tasks.workunit.client.0.smithi032.stdout:2/39: sync 2024-05-12T04:01:04.279 INFO:tasks.workunit.client.0.smithi032.stdout:8/19: sync 2024-05-12T04:01:04.280 INFO:tasks.workunit.client.0.smithi032.stdout:1/31: dwrite d1/f3 [4194304,4194304] 0 2024-05-12T04:01:04.280 INFO:tasks.workunit.client.0.smithi032.stdout:4/15: mkdir d0/d3 0 2024-05-12T04:01:04.282 INFO:tasks.workunit.client.0.smithi032.stdout:3/73: creat d15/f19 x:0 0 0 2024-05-12T04:01:04.283 INFO:tasks.workunit.client.0.smithi032.stdout:6/34: dread f3 [0,4194304] 0 2024-05-12T04:01:04.284 INFO:tasks.workunit.client.0.smithi032.stdout:9/32: dread d0/f4 [0,4194304] 0 2024-05-12T04:01:04.284 INFO:tasks.workunit.client.0.smithi032.stdout:0/93: rename df/d10/f19 to df/f25 0 2024-05-12T04:01:04.285 INFO:tasks.workunit.client.0.smithi032.stdout:9/33: write d0/f4 [1799587,47831] 0 2024-05-12T04:01:04.286 INFO:tasks.workunit.client.0.smithi032.stdout:9/34: chown d0 1632 1 2024-05-12T04:01:04.291 INFO:tasks.workunit.client.0.smithi032.stdout:8/20: chown d0/f2 0 1 2024-05-12T04:01:04.295 INFO:tasks.workunit.client.0.smithi032.stdout:3/74: creat d15/f1a x:0 0 0 2024-05-12T04:01:04.301 INFO:tasks.workunit.client.0.smithi032.stdout:1/32: fdatasync d1/f3 0 2024-05-12T04:01:04.302 INFO:tasks.workunit.client.0.smithi032.stdout:9/35: read d0/f4 [64770,77907] 0 2024-05-12T04:01:04.302 INFO:tasks.workunit.client.0.smithi032.stdout:8/21: creat d0/f5 x:0 0 0 2024-05-12T04:01:04.303 INFO:tasks.workunit.client.0.smithi032.stdout:1/33: stat d1/f6 0 2024-05-12T04:01:04.304 INFO:tasks.workunit.client.0.smithi032.stdout:5/63: dwrite d4/d7/f11 [0,4194304] 0 2024-05-12T04:01:04.304 INFO:tasks.workunit.client.0.smithi032.stdout:8/22: chown d0/f4 1925856 1 2024-05-12T04:01:04.305 INFO:tasks.workunit.client.0.smithi032.stdout:8/23: readlink - no filename 2024-05-12T04:01:04.305 INFO:tasks.workunit.client.0.smithi032.stdout:0/94: symlink df/l26 0 2024-05-12T04:01:04.305 INFO:tasks.workunit.client.0.smithi032.stdout:9/36: chown d0/d3/c5 384118 1 2024-05-12T04:01:04.306 INFO:tasks.workunit.client.0.smithi032.stdout:2/40: write d1/fc [799520,14369] 0 2024-05-12T04:01:04.307 INFO:tasks.workunit.client.0.smithi032.stdout:3/75: write fd [224579,115098] 0 2024-05-12T04:01:04.311 INFO:tasks.workunit.client.0.smithi032.stdout:6/35: truncate f6 1379560 0 2024-05-12T04:01:04.313 INFO:tasks.workunit.client.0.smithi032.stdout:1/34: unlink d1/cb 0 2024-05-12T04:01:04.318 INFO:tasks.workunit.client.0.smithi032.stdout:3/76: dwrite fd [0,4194304] 0 2024-05-12T04:01:04.318 INFO:tasks.workunit.client.0.smithi032.stdout:9/37: rmdir d0/d3 39 2024-05-12T04:01:04.318 INFO:tasks.workunit.client.0.smithi032.stdout:0/95: dread - df/f25 zero size 2024-05-12T04:01:04.318 INFO:tasks.workunit.client.0.smithi032.stdout:1/35: chown d1/d7/ca 1 1 2024-05-12T04:01:04.319 INFO:tasks.workunit.client.0.smithi032.stdout:8/24: truncate d0/f4 1014040 0 2024-05-12T04:01:04.319 INFO:tasks.workunit.client.0.smithi032.stdout:1/36: chown d1/f3 50535 1 2024-05-12T04:01:04.320 INFO:tasks.workunit.client.0.smithi032.stdout:7/14: link f2 f5 0 2024-05-12T04:01:04.320 INFO:tasks.workunit.client.0.smithi032.stdout:7/15: readlink l4 0 2024-05-12T04:01:04.322 INFO:tasks.workunit.client.0.smithi032.stdout:5/64: mknod d4/db/c16 0 2024-05-12T04:01:04.323 INFO:tasks.workunit.client.0.smithi032.stdout:5/65: rename d4 to d4/d7/d14/d17 22 2024-05-12T04:01:04.326 INFO:tasks.workunit.client.0.smithi032.stdout:8/25: mknod d0/c6 0 2024-05-12T04:01:04.332 INFO:tasks.workunit.client.1.smithi033.stdout:3/220: sync 2024-05-12T04:01:04.332 INFO:tasks.workunit.client.1.smithi033.stdout:9/403: sync 2024-05-12T04:01:04.332 INFO:tasks.workunit.client.1.smithi033.stdout:1/148: sync 2024-05-12T04:01:04.332 INFO:tasks.workunit.client.1.smithi033.stdout:8/152: sync 2024-05-12T04:01:04.334 INFO:tasks.workunit.client.1.smithi033.stdout:1/149: chown cb 13979 1 2024-05-12T04:01:04.334 INFO:tasks.workunit.client.0.smithi032.stdout:3/77: write f3 [4811035,68023] 0 2024-05-12T04:01:04.335 INFO:tasks.workunit.client.1.smithi033.stdout:1/150: dread - d11/d1c/f32 zero size 2024-05-12T04:01:04.336 INFO:tasks.workunit.client.1.smithi033.stdout:9/404: fdatasync d2/dd/d39/f59 0 2024-05-12T04:01:04.340 INFO:tasks.workunit.client.1.smithi033.stdout:8/153: dwrite d1/d4/f1c [0,4194304] 0 2024-05-12T04:01:04.342 INFO:tasks.workunit.client.0.smithi032.stdout:7/16: write f1 [544867,54096] 0 2024-05-12T04:01:04.343 INFO:tasks.workunit.client.1.smithi033.stdout:1/151: creat d11/d17/f35 x:0 0 0 2024-05-12T04:01:04.344 INFO:tasks.workunit.client.0.smithi032.stdout:0/96: dwrite fa [4194304,4194304] 0 2024-05-12T04:01:04.345 INFO:tasks.workunit.client.1.smithi033.stdout:9/405: mkdir d2/d70/d7b/d94 0 2024-05-12T04:01:04.346 INFO:tasks.workunit.client.1.smithi033.stdout:1/152: read d11/f19 [551460,123221] 0 2024-05-12T04:01:04.347 INFO:tasks.workunit.client.1.smithi033.stdout:8/154: creat d1/d16/d20/f2c x:0 0 0 2024-05-12T04:01:04.347 INFO:tasks.workunit.client.0.smithi032.stdout:0/97: chown df/d22 10439 1 2024-05-12T04:01:04.347 INFO:tasks.workunit.client.0.smithi032.stdout:9/38: truncate d0/f4 1414368 0 2024-05-12T04:01:04.349 INFO:tasks.workunit.client.1.smithi033.stdout:1/153: write d11/d17/f18 [729280,63052] 0 2024-05-12T04:01:04.355 INFO:tasks.workunit.client.1.smithi033.stdout:8/155: creat d1/d16/d27/f2d x:0 0 0 2024-05-12T04:01:04.356 INFO:tasks.workunit.client.1.smithi033.stdout:1/154: write d11/d17/f2c [1078703,19748] 0 2024-05-12T04:01:04.356 INFO:tasks.workunit.client.0.smithi032.stdout:1/37: dwrite d1/f6 [0,4194304] 0 2024-05-12T04:01:04.357 INFO:tasks.workunit.client.0.smithi032.stdout:8/26: chown d0/f4 5612062 1 2024-05-12T04:01:04.358 INFO:tasks.workunit.client.0.smithi032.stdout:7/17: creat f6 x:0 0 0 2024-05-12T04:01:04.359 INFO:tasks.workunit.client.1.smithi033.stdout:8/156: chown d1/d16/d20/c23 733156 1 2024-05-12T04:01:04.359 INFO:tasks.workunit.client.1.smithi033.stdout:1/155: write d11/f30 [2462263,19583] 0 2024-05-12T04:01:04.359 INFO:tasks.workunit.client.1.smithi033.stdout:3/221: dwrite db/d17/d25/d32/d3c/d44/f4a [0,4194304] 0 2024-05-12T04:01:04.361 INFO:tasks.workunit.client.0.smithi032.stdout:5/66: dwrite d4/db/ff [0,4194304] 0 2024-05-12T04:01:04.361 INFO:tasks.workunit.client.0.smithi032.stdout:8/27: chown d0/f3 224 1 2024-05-12T04:01:04.363 INFO:tasks.workunit.client.0.smithi032.stdout:5/67: write f2 [2034107,8869] 0 2024-05-12T04:01:04.364 INFO:tasks.workunit.client.1.smithi033.stdout:4/234: sync 2024-05-12T04:01:04.364 INFO:tasks.workunit.client.1.smithi033.stdout:2/219: sync 2024-05-12T04:01:04.364 INFO:tasks.workunit.client.1.smithi033.stdout:6/172: sync 2024-05-12T04:01:04.364 INFO:tasks.workunit.client.1.smithi033.stdout:5/152: sync 2024-05-12T04:01:04.365 INFO:tasks.workunit.client.0.smithi032.stdout:6/36: dwrite f6 [0,4194304] 0 2024-05-12T04:01:04.365 INFO:tasks.workunit.client.1.smithi033.stdout:9/406: truncate d2/dd/f26 673703 0 2024-05-12T04:01:04.367 INFO:tasks.workunit.client.0.smithi032.stdout:2/41: link d1/ld d1/l10 0 2024-05-12T04:01:04.369 INFO:tasks.workunit.client.0.smithi032.stdout:6/37: write f6 [940231,108567] 0 2024-05-12T04:01:04.369 INFO:tasks.workunit.client.1.smithi033.stdout:9/407: chown d2/d4/d2b/d6d/d3a/f5a 32027 1 2024-05-12T04:01:04.370 INFO:tasks.workunit.client.1.smithi033.stdout:2/220: truncate dc/d17/d27/d2f/f41 1888785 0 2024-05-12T04:01:04.371 INFO:tasks.workunit.client.0.smithi032.stdout:7/18: creat f7 x:0 0 0 2024-05-12T04:01:04.371 INFO:tasks.workunit.client.0.smithi032.stdout:7/19: rmdir - no directory 2024-05-12T04:01:04.373 INFO:tasks.workunit.client.0.smithi032.stdout:7/20: write f6 [551707,70014] 0 2024-05-12T04:01:04.374 INFO:tasks.workunit.client.0.smithi032.stdout:7/21: rmdir - no directory 2024-05-12T04:01:04.377 INFO:tasks.workunit.client.1.smithi033.stdout:5/153: symlink d3/d15/d16/l2f 0 2024-05-12T04:01:04.379 INFO:tasks.workunit.client.0.smithi032.stdout:1/38: mknod d1/d7/cc 0 2024-05-12T04:01:04.381 INFO:tasks.workunit.client.0.smithi032.stdout:1/39: write d1/f3 [4511250,69011] 0 2024-05-12T04:01:04.382 INFO:tasks.workunit.client.0.smithi032.stdout:3/78: write f7 [267374,87594] 0 2024-05-12T04:01:04.383 INFO:tasks.workunit.client.1.smithi033.stdout:3/222: mkdir db/d17/d4d 0 2024-05-12T04:01:04.385 INFO:tasks.workunit.client.1.smithi033.stdout:3/223: fsync db/f22 0 2024-05-12T04:01:04.388 INFO:tasks.workunit.client.0.smithi032.stdout:3/79: chown d15/f17 674 1 2024-05-12T04:01:04.388 INFO:tasks.workunit.client.1.smithi033.stdout:1/156: dwrite d11/f19 [0,4194304] 0 2024-05-12T04:01:04.390 INFO:tasks.workunit.client.0.smithi032.stdout:7/22: dwrite f6 [0,4194304] 0 2024-05-12T04:01:04.391 INFO:tasks.workunit.client.0.smithi032.stdout:0/98: symlink df/d22/l27 0 2024-05-12T04:01:04.392 INFO:tasks.workunit.client.1.smithi033.stdout:1/157: write d11/d17/d27/f31 [964502,104030] 0 2024-05-12T04:01:04.393 INFO:tasks.workunit.client.1.smithi033.stdout:9/408: fdatasync d2/dd/f13 0 2024-05-12T04:01:04.395 INFO:tasks.workunit.client.1.smithi033.stdout:5/154: dwrite d3/d15/d16/f20 [4194304,4194304] 0 2024-05-12T04:01:04.395 INFO:tasks.workunit.client.1.smithi033.stdout:1/158: chown d11/f19 56319 1 2024-05-12T04:01:04.395 INFO:tasks.workunit.client.0.smithi032.stdout:7/23: dread f2 [0,4194304] 0 2024-05-12T04:01:04.395 INFO:tasks.workunit.client.1.smithi033.stdout:9/409: chown d2/c64 3 1 2024-05-12T04:01:04.408 INFO:tasks.workunit.client.0.smithi032.stdout:1/40: mkdir d1/dd 0 2024-05-12T04:01:04.409 INFO:tasks.workunit.client.1.smithi033.stdout:4/235: dread dd/f1b [0,4194304] 0 2024-05-12T04:01:04.409 INFO:tasks.workunit.client.1.smithi033.stdout:8/157: dread d1/d4/fd [0,4194304] 0 2024-05-12T04:01:04.411 INFO:tasks.workunit.client.0.smithi032.stdout:3/80: mkdir d15/d1b 0 2024-05-12T04:01:04.412 INFO:tasks.workunit.client.1.smithi033.stdout:5/155: creat d3/d15/f30 x:0 0 0 2024-05-12T04:01:04.413 INFO:tasks.workunit.client.0.smithi032.stdout:9/39: write d0/f4 [649498,51310] 0 2024-05-12T04:01:04.413 INFO:tasks.workunit.client.1.smithi033.stdout:1/159: fdatasync d11/d17/f1d 0 2024-05-12T04:01:04.414 INFO:tasks.workunit.client.0.smithi032.stdout:9/40: write d0/f4 [1858123,60726] 0 2024-05-12T04:01:04.414 INFO:tasks.workunit.client.0.smithi032.stdout:0/99: read fb [215621,41221] 0 2024-05-12T04:01:04.417 INFO:tasks.workunit.client.1.smithi033.stdout:9/410: rename d2/f4f to d2/d4/d2b/d6d/d4c/f95 0 2024-05-12T04:01:04.417 INFO:tasks.workunit.client.0.smithi032.stdout:9/41: fsync d0/f4 0 2024-05-12T04:01:04.419 INFO:tasks.workunit.client.0.smithi032.stdout:0/100: dread df/d10/f13 [0,4194304] 0 2024-05-12T04:01:04.420 INFO:tasks.workunit.client.1.smithi033.stdout:5/156: dwrite d3/d15/d16/f20 [8388608,4194304] 0 2024-05-12T04:01:04.425 INFO:tasks.workunit.client.0.smithi032.stdout:9/42: dwrite d0/f4 [0,4194304] 0 2024-05-12T04:01:04.427 INFO:tasks.workunit.client.0.smithi032.stdout:1/41: mknod d1/d7/ce 0 2024-05-12T04:01:04.428 INFO:tasks.workunit.client.1.smithi033.stdout:5/157: dwrite d3/f7 [0,4194304] 0 2024-05-12T04:01:04.430 INFO:tasks.workunit.client.1.smithi033.stdout:5/158: write d3/d27/f22 [610685,21181] 0 2024-05-12T04:01:04.430 INFO:tasks.workunit.client.1.smithi033.stdout:6/173: write d4/d5/f33 [925902,56873] 0 2024-05-12T04:01:04.433 INFO:tasks.workunit.client.1.smithi033.stdout:9/411: creat d2/d4/d29/d3c/f96 x:0 0 0 2024-05-12T04:01:04.434 INFO:tasks.workunit.client.0.smithi032.stdout:3/81: creat d15/f1c x:0 0 0 2024-05-12T04:01:04.435 INFO:tasks.workunit.client.0.smithi032.stdout:3/82: fsync fd 0 2024-05-12T04:01:04.436 INFO:tasks.workunit.client.1.smithi033.stdout:8/158: mkdir d1/d4/d2e 0 2024-05-12T04:01:04.437 INFO:tasks.workunit.client.1.smithi033.stdout:8/159: readlink d1/l9 0 2024-05-12T04:01:04.437 INFO:tasks.workunit.client.0.smithi032.stdout:6/38: dwrite f3 [0,4194304] 0 2024-05-12T04:01:04.438 INFO:tasks.workunit.client.1.smithi033.stdout:2/221: truncate dc/f31 3173562 0 2024-05-12T04:01:04.438 INFO:tasks.workunit.client.1.smithi033.stdout:9/412: write d2/d4/d2b/d6d/d3a/f65 [468322,30367] 0 2024-05-12T04:01:04.443 INFO:tasks.workunit.client.1.smithi033.stdout:1/160: dread d11/d17/d27/f2b [0,4194304] 0 2024-05-12T04:01:04.444 INFO:tasks.workunit.client.1.smithi033.stdout:6/174: dwrite d4/f16 [4194304,4194304] 0 2024-05-12T04:01:04.447 INFO:tasks.workunit.client.1.smithi033.stdout:4/236: link fa dd/d11/d31/d38/f44 0 2024-05-12T04:01:04.448 INFO:tasks.workunit.client.1.smithi033.stdout:4/237: dread - dd/f3b zero size 2024-05-12T04:01:04.456 INFO:tasks.workunit.client.1.smithi033.stdout:8/160: creat d1/d16/d20/f2f x:0 0 0 2024-05-12T04:01:04.456 INFO:tasks.workunit.client.1.smithi033.stdout:2/222: rmdir dc/d17/d3a 39 2024-05-12T04:01:04.458 INFO:tasks.workunit.client.1.smithi033.stdout:2/223: write dc/d13/d1c/f25 [1839156,87468] 0 2024-05-12T04:01:04.459 INFO:tasks.workunit.client.1.smithi033.stdout:9/413: creat d2/d4/d2b/d6d/d2f/d5f/f97 x:0 0 0 2024-05-12T04:01:04.461 INFO:tasks.workunit.client.0.smithi032.stdout:8/28: getdents d0 0 2024-05-12T04:01:04.461 INFO:tasks.workunit.client.1.smithi033.stdout:8/161: dread d1/f13 [0,4194304] 0 2024-05-12T04:01:04.462 INFO:tasks.workunit.client.1.smithi033.stdout:6/175: symlink d4/l38 0 2024-05-12T04:01:04.462 INFO:tasks.workunit.client.0.smithi032.stdout:2/42: dwrite d1/f4 [0,4194304] 0 2024-05-12T04:01:04.464 INFO:tasks.workunit.client.0.smithi032.stdout:8/29: chown d0/f2 5454 1 2024-05-12T04:01:04.466 INFO:tasks.workunit.client.0.smithi032.stdout:0/101: dwrite f9 [0,4194304] 0 2024-05-12T04:01:04.467 INFO:tasks.workunit.client.1.smithi033.stdout:5/159: link d3/f10 d3/d12/d25/d26/f31 0 2024-05-12T04:01:04.469 INFO:tasks.workunit.client.1.smithi033.stdout:5/160: stat d3/d15 0 2024-05-12T04:01:04.474 INFO:tasks.workunit.client.1.smithi033.stdout:5/161: chown d3/d15/d16/f1a 17330 1 2024-05-12T04:01:04.474 INFO:tasks.workunit.client.1.smithi033.stdout:6/176: dwrite d4/d5/f9 [0,4194304] 0 2024-05-12T04:01:04.476 INFO:tasks.workunit.client.1.smithi033.stdout:2/224: rmdir dc 39 2024-05-12T04:01:04.477 INFO:tasks.workunit.client.0.smithi032.stdout:0/102: dwrite df/d10/f13 [0,4194304] 0 2024-05-12T04:01:04.477 INFO:tasks.workunit.client.1.smithi033.stdout:4/238: dwrite dd/d11/d31/d38/f44 [0,4194304] 0 2024-05-12T04:01:04.482 INFO:tasks.workunit.client.0.smithi032.stdout:5/68: getdents d4/d7 0 2024-05-12T04:01:04.482 INFO:tasks.workunit.client.1.smithi033.stdout:4/239: dread dd/f29 [0,4194304] 0 2024-05-12T04:01:04.483 INFO:tasks.workunit.client.1.smithi033.stdout:9/414: mknod d2/d4/d29/d32/c98 0 2024-05-12T04:01:04.485 INFO:tasks.workunit.client.1.smithi033.stdout:4/240: dread - dd/d11/d15/f2d zero size 2024-05-12T04:01:04.489 INFO:tasks.workunit.client.0.smithi032.stdout:1/42: mknod d1/dd/cf 0 2024-05-12T04:01:04.489 INFO:tasks.workunit.client.1.smithi033.stdout:4/241: write dd/f35 [903409,129602] 0 2024-05-12T04:01:04.489 INFO:tasks.workunit.client.1.smithi033.stdout:2/225: dwrite f6 [0,4194304] 0 2024-05-12T04:01:04.491 INFO:tasks.workunit.client.0.smithi032.stdout:0/103: dwrite df/d10/f13 [4194304,4194304] 0 2024-05-12T04:01:04.493 INFO:tasks.workunit.client.0.smithi032.stdout:9/43: rename d0/f4 to d0/d3/f8 0 2024-05-12T04:01:04.495 INFO:tasks.workunit.client.1.smithi033.stdout:7/378: sync 2024-05-12T04:01:04.497 INFO:tasks.workunit.client.1.smithi033.stdout:7/379: dread - de/df/d17/d1e/f23 zero size 2024-05-12T04:01:04.497 INFO:tasks.workunit.client.1.smithi033.stdout:9/415: dwrite d2/d4/d2b/d6d/d3a/f45 [0,4194304] 0 2024-05-12T04:01:04.498 INFO:tasks.workunit.client.1.smithi033.stdout:5/162: mknod d3/d12/d29/d2b/c32 0 2024-05-12T04:01:04.499 INFO:tasks.workunit.client.0.smithi032.stdout:8/30: mkdir d0/d7 0 2024-05-12T04:01:04.500 INFO:tasks.workunit.client.0.smithi032.stdout:2/43: dread - d1/fe zero size 2024-05-12T04:01:04.502 INFO:tasks.workunit.client.1.smithi033.stdout:5/163: fsync d3/f1c 0 2024-05-12T04:01:04.502 INFO:tasks.workunit.client.1.smithi033.stdout:0/281: sync 2024-05-12T04:01:04.502 INFO:tasks.workunit.client.0.smithi032.stdout:3/83: write f6 [2816424,55958] 0 2024-05-12T04:01:04.503 INFO:tasks.workunit.client.0.smithi032.stdout:9/44: dread d0/d3/f8 [0,4194304] 0 2024-05-12T04:01:04.503 INFO:tasks.workunit.client.0.smithi032.stdout:3/84: readlink l4 0 2024-05-12T04:01:04.506 INFO:tasks.workunit.client.1.smithi033.stdout:9/416: chown d2/d4/d2b/f4b 609519 1 2024-05-12T04:01:04.506 INFO:tasks.workunit.client.1.smithi033.stdout:0/282: readlink d6/dc/d39/l3d 0 2024-05-12T04:01:04.508 INFO:tasks.workunit.client.0.smithi032.stdout:7/24: getdents . 0 2024-05-12T04:01:04.511 INFO:tasks.workunit.client.0.smithi032.stdout:5/69: truncate f1 3145619 0 2024-05-12T04:01:04.517 INFO:tasks.workunit.client.0.smithi032.stdout:2/44: chown d1/l10 2007265 1 2024-05-12T04:01:04.518 INFO:tasks.workunit.client.1.smithi033.stdout:4/242: creat dd/d11/d31/f45 x:0 0 0 2024-05-12T04:01:04.520 INFO:tasks.workunit.client.1.smithi033.stdout:7/380: rename de/df/d17/d1b/c52 to de/d80/c90 0 2024-05-12T04:01:04.520 INFO:tasks.workunit.client.1.smithi033.stdout:2/226: dwrite dc/d13/d1c/f4f [0,4194304] 0 2024-05-12T04:01:04.520 INFO:tasks.workunit.client.1.smithi033.stdout:4/243: write dd/f13 [2900583,34757] 0 2024-05-12T04:01:04.522 INFO:tasks.workunit.client.1.smithi033.stdout:1/161: getdents d11 0 2024-05-12T04:01:04.522 INFO:tasks.workunit.client.1.smithi033.stdout:2/227: dread - dc/d17/f23 zero size 2024-05-12T04:01:04.523 INFO:tasks.workunit.client.1.smithi033.stdout:4/244: dread - dd/d11/d31/f45 zero size 2024-05-12T04:01:04.524 INFO:tasks.workunit.client.0.smithi032.stdout:7/25: mknod c8 0 2024-05-12T04:01:04.524 INFO:tasks.workunit.client.1.smithi033.stdout:4/245: read - dd/f36 zero size 2024-05-12T04:01:04.525 INFO:tasks.workunit.client.1.smithi033.stdout:4/246: fdatasync dd/f3b 0 2024-05-12T04:01:04.527 INFO:tasks.workunit.client.1.smithi033.stdout:0/283: symlink d6/d11/l5b 0 2024-05-12T04:01:04.530 INFO:tasks.workunit.client.0.smithi032.stdout:8/31: getdents d0/d7 0 2024-05-12T04:01:04.533 INFO:tasks.workunit.client.0.smithi032.stdout:9/45: symlink d0/d3/d6/d7/l9 0 2024-05-12T04:01:04.535 INFO:tasks.workunit.client.0.smithi032.stdout:2/45: dread d1/fa [0,4194304] 0 2024-05-12T04:01:04.535 INFO:tasks.workunit.client.1.smithi033.stdout:1/162: mkdir d11/d17/d27/d36 0 2024-05-12T04:01:04.537 INFO:tasks.workunit.client.1.smithi033.stdout:9/417: symlink d2/d4/d2b/l99 0 2024-05-12T04:01:04.538 INFO:tasks.workunit.client.0.smithi032.stdout:6/39: link l1 d7/l8 0 2024-05-12T04:01:04.539 INFO:tasks.workunit.client.0.smithi032.stdout:6/40: write f3 [4344686,127970] 0 2024-05-12T04:01:04.540 INFO:tasks.workunit.client.1.smithi033.stdout:2/228: symlink dc/d17/d27/d2f/l50 0 2024-05-12T04:01:04.541 INFO:tasks.workunit.client.0.smithi032.stdout:7/26: rename f7 to f9 0 2024-05-12T04:01:04.545 INFO:tasks.workunit.client.1.smithi033.stdout:4/247: fdatasync f3 0 2024-05-12T04:01:04.545 INFO:tasks.workunit.client.1.smithi033.stdout:5/164: dread d3/ff [0,4194304] 0 2024-05-12T04:01:04.545 INFO:tasks.workunit.client.1.smithi033.stdout:5/165: write d3/d12/f1f [1186489,92132] 0 2024-05-12T04:01:04.546 INFO:tasks.workunit.client.0.smithi032.stdout:5/70: chown f1 672966276 1 2024-05-12T04:01:04.546 INFO:tasks.workunit.client.0.smithi032.stdout:4/16: sync 2024-05-12T04:01:04.548 INFO:tasks.workunit.client.1.smithi033.stdout:8/162: getdents d1 0 2024-05-12T04:01:04.548 INFO:tasks.workunit.client.0.smithi032.stdout:9/46: mkdir d0/d3/d6/d7/da 0 2024-05-12T04:01:04.549 INFO:tasks.workunit.client.0.smithi032.stdout:2/46: rename d1 to d1/d11 22 2024-05-12T04:01:04.549 INFO:tasks.workunit.client.1.smithi033.stdout:8/163: write d1/f2 [4602315,15547] 0 2024-05-12T04:01:04.550 INFO:tasks.workunit.client.1.smithi033.stdout:1/163: rmdir d11/d17/d27 39 2024-05-12T04:01:04.552 INFO:tasks.workunit.client.0.smithi032.stdout:9/47: dread d0/d3/f8 [0,4194304] 0 2024-05-12T04:01:04.552 INFO:tasks.workunit.client.1.smithi033.stdout:4/248: dwrite dd/d11/f21 [0,4194304] 0 2024-05-12T04:01:04.558 INFO:tasks.workunit.client.1.smithi033.stdout:2/229: readlink dc/l12 0 2024-05-12T04:01:04.558 INFO:tasks.workunit.client.1.smithi033.stdout:4/249: truncate dd/f3d 162707 0 2024-05-12T04:01:04.559 INFO:tasks.workunit.client.1.smithi033.stdout:2/230: dread - dc/d17/f23 zero size 2024-05-12T04:01:04.561 INFO:tasks.workunit.client.1.smithi033.stdout:2/231: chown dc/ce 11412 1 2024-05-12T04:01:04.562 INFO:tasks.workunit.client.1.smithi033.stdout:9/418: dwrite d2/d4/d2b/d6d/d3a/f5a [0,4194304] 0 2024-05-12T04:01:04.562 INFO:tasks.workunit.client.0.smithi032.stdout:7/27: creat fa x:0 0 0 2024-05-12T04:01:04.562 INFO:tasks.workunit.client.0.smithi032.stdout:7/28: stat f5 0 2024-05-12T04:01:04.564 INFO:tasks.workunit.client.0.smithi032.stdout:5/71: chown d4/c9 227493 1 2024-05-12T04:01:04.567 INFO:tasks.workunit.client.1.smithi033.stdout:2/232: dread dc/d17/d27/d2f/f41 [0,4194304] 0 2024-05-12T04:01:04.568 INFO:tasks.workunit.client.1.smithi033.stdout:5/166: symlink d3/d12/d29/l33 0 2024-05-12T04:01:04.568 INFO:tasks.workunit.client.1.smithi033.stdout:2/233: chown dc/d13/f15 13 1 2024-05-12T04:01:04.568 INFO:tasks.workunit.client.0.smithi032.stdout:1/43: truncate d1/f6 2538136 0 2024-05-12T04:01:04.574 INFO:tasks.workunit.client.0.smithi032.stdout:3/85: truncate f6 1538780 0 2024-05-12T04:01:04.580 INFO:tasks.workunit.client.0.smithi032.stdout:9/48: dwrite d0/d3/f8 [0,4194304] 0 2024-05-12T04:01:04.580 INFO:tasks.workunit.client.0.smithi032.stdout:9/49: stat d0/d3 0 2024-05-12T04:01:04.581 INFO:tasks.workunit.client.0.smithi032.stdout:7/29: creat fb x:0 0 0 2024-05-12T04:01:04.582 INFO:tasks.workunit.client.0.smithi032.stdout:8/32: write d0/f4 [58668,28449] 0 2024-05-12T04:01:04.584 INFO:tasks.workunit.client.0.smithi032.stdout:5/72: symlink d4/db/l18 0 2024-05-12T04:01:04.586 INFO:tasks.workunit.client.0.smithi032.stdout:9/50: dread d0/d3/f8 [0,4194304] 0 2024-05-12T04:01:04.586 INFO:tasks.workunit.client.1.smithi033.stdout:9/419: creat d2/d4/d29/d3c/f9a x:0 0 0 2024-05-12T04:01:04.588 INFO:tasks.workunit.client.0.smithi032.stdout:4/17: creat d0/d3/f4 x:0 0 0 2024-05-12T04:01:04.588 INFO:tasks.workunit.client.1.smithi033.stdout:9/420: truncate d2/d4/d29/d3c/d52/f86 1374469 0 2024-05-12T04:01:04.589 INFO:tasks.workunit.client.0.smithi032.stdout:4/18: write d0/d3/f4 [722505,47662] 0 2024-05-12T04:01:04.590 INFO:tasks.workunit.client.0.smithi032.stdout:4/19: stat d0/d3/f4 0 2024-05-12T04:01:04.594 INFO:tasks.workunit.client.1.smithi033.stdout:5/167: fdatasync d3/f10 0 2024-05-12T04:01:04.594 INFO:tasks.workunit.client.1.smithi033.stdout:3/224: sync 2024-05-12T04:01:04.596 INFO:tasks.workunit.client.0.smithi032.stdout:1/44: mknod d1/d7/c10 0 2024-05-12T04:01:04.599 INFO:tasks.workunit.client.0.smithi032.stdout:2/47: write d1/fa [2898819,99378] 0 2024-05-12T04:01:04.599 INFO:tasks.workunit.client.1.smithi033.stdout:7/381: getdents de/d2b/d6c 0 2024-05-12T04:01:04.603 INFO:tasks.workunit.client.0.smithi032.stdout:6/41: truncate f3 3061315 0 2024-05-12T04:01:04.604 INFO:tasks.workunit.client.1.smithi033.stdout:8/164: creat d1/d16/d29/f30 x:0 0 0 2024-05-12T04:01:04.605 INFO:tasks.workunit.client.0.smithi032.stdout:4/20: dwrite d0/f2 [0,4194304] 0 2024-05-12T04:01:04.606 INFO:tasks.workunit.client.0.smithi032.stdout:7/30: fdatasync f5 0 2024-05-12T04:01:04.607 INFO:tasks.workunit.client.1.smithi033.stdout:1/164: creat d11/d17/d27/f37 x:0 0 0 2024-05-12T04:01:04.608 INFO:tasks.workunit.client.1.smithi033.stdout:1/165: chown d11/d17 25442551 1 2024-05-12T04:01:04.609 INFO:tasks.workunit.client.0.smithi032.stdout:3/86: dwrite f6 [0,4194304] 0 2024-05-12T04:01:04.609 INFO:tasks.workunit.client.0.smithi032.stdout:8/33: mknod d0/c8 0 2024-05-12T04:01:04.611 INFO:tasks.workunit.client.0.smithi032.stdout:8/34: stat d0/f5 0 2024-05-12T04:01:04.611 INFO:tasks.workunit.client.0.smithi032.stdout:8/35: readlink - no filename 2024-05-12T04:01:04.613 INFO:tasks.workunit.client.0.smithi032.stdout:5/73: unlink f2 0 2024-05-12T04:01:04.615 INFO:tasks.workunit.client.0.smithi032.stdout:3/87: truncate d15/f1a 657398 0 2024-05-12T04:01:04.619 INFO:tasks.workunit.client.1.smithi033.stdout:0/284: link d6/d11/c49 d6/d11/d57/c5c 0 2024-05-12T04:01:04.620 INFO:tasks.workunit.client.1.smithi033.stdout:0/285: dread - d6/dc/d58/f59 zero size 2024-05-12T04:01:04.621 INFO:tasks.workunit.client.1.smithi033.stdout:2/234: write dc/d13/f1d [1617101,7689] 0 2024-05-12T04:01:04.624 INFO:tasks.workunit.client.0.smithi032.stdout:1/45: creat d1/d7/f11 x:0 0 0 2024-05-12T04:01:04.625 INFO:tasks.workunit.client.1.smithi033.stdout:5/168: mkdir d3/d12/d29/d34 0 2024-05-12T04:01:04.626 INFO:tasks.workunit.client.1.smithi033.stdout:5/169: read d3/f2c [357111,106978] 0 2024-05-12T04:01:04.628 INFO:tasks.workunit.client.1.smithi033.stdout:3/225: symlink db/d17/d3a/l4e 0 2024-05-12T04:01:04.628 INFO:tasks.workunit.client.0.smithi032.stdout:3/88: dwrite f7 [0,4194304] 0 2024-05-12T04:01:04.628 INFO:tasks.workunit.client.1.smithi033.stdout:3/226: stat db/d42 0 2024-05-12T04:01:04.634 INFO:tasks.workunit.client.0.smithi032.stdout:1/46: dwrite d1/f3 [4194304,4194304] 0 2024-05-12T04:01:04.635 INFO:tasks.workunit.client.1.smithi033.stdout:1/166: truncate fd 245527 0 2024-05-12T04:01:04.639 INFO:tasks.workunit.client.0.smithi032.stdout:4/21: unlink d0/f2 0 2024-05-12T04:01:04.642 INFO:tasks.workunit.client.1.smithi033.stdout:1/167: dwrite d11/d17/f18 [0,4194304] 0 2024-05-12T04:01:04.643 INFO:tasks.workunit.client.0.smithi032.stdout:7/31: dwrite f5 [0,4194304] 0 2024-05-12T04:01:04.645 INFO:tasks.workunit.client.1.smithi033.stdout:4/250: creat dd/d11/f46 x:0 0 0 2024-05-12T04:01:04.647 INFO:tasks.workunit.client.0.smithi032.stdout:8/36: mkdir d0/d9 0 2024-05-12T04:01:04.649 INFO:tasks.workunit.client.0.smithi032.stdout:8/37: chown d0/c8 19 1 2024-05-12T04:01:04.653 INFO:tasks.workunit.client.1.smithi033.stdout:1/168: dwrite f8 [4194304,4194304] 0 2024-05-12T04:01:04.655 INFO:tasks.workunit.client.1.smithi033.stdout:5/170: mknod d3/d12/c35 0 2024-05-12T04:01:04.657 INFO:tasks.workunit.client.1.smithi033.stdout:3/227: mknod db/d17/c4f 0 2024-05-12T04:01:04.658 INFO:tasks.workunit.client.0.smithi032.stdout:7/32: dwrite f1 [0,4194304] 0 2024-05-12T04:01:04.659 INFO:tasks.workunit.client.0.smithi032.stdout:3/89: unlink f6 0 2024-05-12T04:01:04.660 INFO:tasks.workunit.client.1.smithi033.stdout:7/382: creat de/df/d17/d1e/d79/f91 x:0 0 0 2024-05-12T04:01:04.661 INFO:tasks.workunit.client.1.smithi033.stdout:7/383: write de/df/d17/f88 [189374,53453] 0 2024-05-12T04:01:04.662 INFO:tasks.workunit.client.1.smithi033.stdout:8/165: creat d1/d4/d2e/f31 x:0 0 0 2024-05-12T04:01:04.664 INFO:tasks.workunit.client.1.smithi033.stdout:7/384: truncate de/df/d60/d75/f8f 128215 0 2024-05-12T04:01:04.671 INFO:tasks.workunit.client.1.smithi033.stdout:7/385: dwrite de/d2b/d6c/f72 [0,4194304] 0 2024-05-12T04:01:04.673 INFO:tasks.workunit.client.1.smithi033.stdout:2/235: creat dc/d17/d27/d2f/d49/f51 x:0 0 0 2024-05-12T04:01:04.673 INFO:tasks.workunit.client.1.smithi033.stdout:2/236: stat dc/d3b/c42 0 2024-05-12T04:01:04.674 INFO:tasks.workunit.client.0.smithi032.stdout:5/74: link d4/db/ff d4/d7/f19 0 2024-05-12T04:01:04.678 INFO:tasks.workunit.client.1.smithi033.stdout:6/177: sync 2024-05-12T04:01:04.679 INFO:tasks.workunit.client.1.smithi033.stdout:5/171: rename d3/ff to d3/d12/d25/d26/d2e/f36 0 2024-05-12T04:01:04.680 INFO:tasks.workunit.client.1.smithi033.stdout:5/172: write d3/d15/f30 [876243,58102] 0 2024-05-12T04:01:04.681 INFO:tasks.workunit.client.1.smithi033.stdout:5/173: fsync d3/fc 0 2024-05-12T04:01:04.682 INFO:tasks.workunit.client.1.smithi033.stdout:5/174: write d3/fb [4135031,15191] 0 2024-05-12T04:01:04.683 INFO:tasks.workunit.client.1.smithi033.stdout:8/166: mknod d1/d4/d2e/c32 0 2024-05-12T04:01:04.683 INFO:tasks.workunit.client.0.smithi032.stdout:2/48: creat d1/f12 x:0 0 0 2024-05-12T04:01:04.683 INFO:tasks.workunit.client.0.smithi032.stdout:7/33: creat fc x:0 0 0 2024-05-12T04:01:04.684 INFO:tasks.workunit.client.0.smithi032.stdout:7/34: readlink l4 0 2024-05-12T04:01:04.685 INFO:tasks.workunit.client.0.smithi032.stdout:7/35: chown fb 87 1 2024-05-12T04:01:04.685 INFO:tasks.workunit.client.1.smithi033.stdout:9/421: getdents d2/d4/d2b/d78 0 2024-05-12T04:01:04.689 INFO:tasks.workunit.client.0.smithi032.stdout:6/42: dread f3 [0,4194304] 0 2024-05-12T04:01:04.690 INFO:tasks.workunit.client.0.smithi032.stdout:7/36: dread f1 [0,4194304] 0 2024-05-12T04:01:04.691 INFO:tasks.workunit.client.0.smithi032.stdout:7/37: truncate fc 438680 0 2024-05-12T04:01:04.691 INFO:tasks.workunit.client.1.smithi033.stdout:9/422: dread d2/dd/d39/f43 [0,4194304] 0 2024-05-12T04:01:04.692 INFO:tasks.workunit.client.0.smithi032.stdout:9/51: link d0/d3/d6/d7/l9 d0/lb 0 2024-05-12T04:01:04.694 INFO:tasks.workunit.client.1.smithi033.stdout:1/169: getdents d11/d17/d27/d36 0 2024-05-12T04:01:04.694 INFO:tasks.workunit.client.0.smithi032.stdout:6/43: dread f6 [0,4194304] 0 2024-05-12T04:01:04.694 INFO:tasks.workunit.client.1.smithi033.stdout:7/386: dwrite de/d2b/f2c [0,4194304] 0 2024-05-12T04:01:04.695 INFO:tasks.workunit.client.1.smithi033.stdout:9/423: truncate d2/d4/d29/d3c/d52/f86 1924381 0 2024-05-12T04:01:04.699 INFO:tasks.workunit.client.0.smithi032.stdout:3/90: symlink d15/d1b/l1d 0 2024-05-12T04:01:04.700 INFO:tasks.workunit.client.0.smithi032.stdout:3/91: fsync f7 0 2024-05-12T04:01:04.706 INFO:tasks.workunit.client.0.smithi032.stdout:7/38: mkdir dd 0 2024-05-12T04:01:04.706 INFO:tasks.workunit.client.0.smithi032.stdout:7/39: dread - fb zero size 2024-05-12T04:01:04.709 INFO:tasks.workunit.client.1.smithi033.stdout:8/167: creat d1/d16/d20/f33 x:0 0 0 2024-05-12T04:01:04.714 INFO:tasks.workunit.client.1.smithi033.stdout:5/175: fsync d3/d15/d16/f1a 0 2024-05-12T04:01:04.715 INFO:tasks.workunit.client.0.smithi032.stdout:9/52: dwrite d0/d3/f8 [0,4194304] 0 2024-05-12T04:01:04.715 INFO:tasks.workunit.client.1.smithi033.stdout:4/251: dwrite f3 [4194304,4194304] 0 2024-05-12T04:01:04.715 INFO:tasks.workunit.client.1.smithi033.stdout:5/176: dread - d3/d12/f1d zero size 2024-05-12T04:01:04.716 INFO:tasks.workunit.client.0.smithi032.stdout:0/104: sync 2024-05-12T04:01:04.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:04.715+0000 7fcdd7214700 1 -- 172.21.15.32:0/1998514893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdd00fbcc0 msgr2=0x7fcdd00fc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:01:04.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:04.715+0000 7fcdd7214700 1 --2- 172.21.15.32:0/1998514893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdd00fbcc0 0x7fcdd00fc0d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4004940 tx=0x7fcdc4009e30 comp rx=0 tx=0).stop 2024-05-12T04:01:04.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:04.715+0000 7fcdd7214700 1 -- 172.21.15.32:0/1998514893 shutdown_connections 2024-05-12T04:01:04.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:04.715+0000 7fcdd7214700 1 --2- 172.21.15.32:0/1998514893 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdd00fcf70 0x7fcdd00fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:04.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:04.715+0000 7fcdd7214700 1 --2- 172.21.15.32:0/1998514893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdd00fbcc0 0x7fcdd00fc0d0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:04.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:04.715+0000 7fcdd7214700 1 -- 172.21.15.32:0/1998514893 >> 172.21.15.32:0/1998514893 conn(0x7fcdd00f7420 msgr2=0x7fcdd00f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:01:04.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:04.715+0000 7fcdd7214700 1 -- 172.21.15.32:0/1998514893 shutdown_connections 2024-05-12T04:01:04.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:04.715+0000 7fcdd7214700 1 -- 172.21.15.32:0/1998514893 wait complete. 2024-05-12T04:01:04.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:04.716+0000 7fcdd7214700 1 Processor -- start 2024-05-12T04:01:04.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:04.716+0000 7fcdd7214700 1 -- start start 2024-05-12T04:01:04.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:04.716+0000 7fcdd7214700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdd00fbcc0 0x7fcdd00665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:01:04.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:04.716+0000 7fcdd7214700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdd00fcf70 0x7fcdd006ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:01:04.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:04.716+0000 7fcdd7214700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd0066c10 con 0x7fcdd00fbcc0 2024-05-12T04:01:04.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:04.716+0000 7fcdd7214700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd0066d50 con 0x7fcdd00fcf70 2024-05-12T04:01:04.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:04.717+0000 7fcdd4fb0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdd00fbcc0 0x7fcdd00665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:01:04.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:04.717+0000 7fcdd4fb0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdd00fbcc0 0x7fcdd00665c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52962/0 (socket says 172.21.15.32:52962) 2024-05-12T04:01:04.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:04.717+0000 7fcdd4fb0700 1 -- 172.21.15.32:0/125333267 learned_addr learned my addr 172.21.15.32:0/125333267 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:01:04.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:04.717+0000 7fcdcffff700 1 --2- 172.21.15.32:0/125333267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdd00fcf70 0x7fcdd006ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:01:04.719 INFO:tasks.workunit.client.1.smithi033.stdout:5/177: write d3/fb [2156351,63541] 0 2024-05-12T04:01:04.719 INFO:tasks.workunit.client.1.smithi033.stdout:0/286: getdents d6/dc/d39 0 2024-05-12T04:01:04.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:04.717+0000 7fcdd4fb0700 1 -- 172.21.15.32:0/125333267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdd00fcf70 msgr2=0x7fcdd006ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:01:04.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:04.717+0000 7fcdd4fb0700 1 --2- 172.21.15.32:0/125333267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdd00fcf70 0x7fcdd006ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:04.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:04.717+0000 7fcdd4fb0700 1 -- 172.21.15.32:0/125333267 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc4005040 con 0x7fcdd00fbcc0 2024-05-12T04:01:04.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:04.718+0000 7fcdd4fb0700 1 --2- 172.21.15.32:0/125333267 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdd00fbcc0 0x7fcdd00665c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fcdc40043d0 tx=0x7fcdc4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:01:04.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:04.718+0000 7fcdcd7fa700 1 -- 172.21.15.32:0/125333267 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdc4020070 con 0x7fcdd00fbcc0 2024-05-12T04:01:04.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:04.718+0000 7fcdcd7fa700 1 -- 172.21.15.32:0/125333267 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcdc401c400 con 0x7fcdd00fbcc0 2024-05-12T04:01:04.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:04.718+0000 7fcdcd7fa700 1 -- 172.21.15.32:0/125333267 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdc400e070 con 0x7fcdd00fbcc0 2024-05-12T04:01:04.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:04.718+0000 7fcdd7214700 1 -- 172.21.15.32:0/125333267 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdd006f1d0 con 0x7fcdd00fbcc0 2024-05-12T04:01:04.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:04.718+0000 7fcdd7214700 1 -- 172.21.15.32:0/125333267 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdd006f790 con 0x7fcdd00fbcc0 2024-05-12T04:01:04.721 INFO:tasks.workunit.client.1.smithi033.stdout:0/287: chown d6/d11/f2f 1 1 2024-05-12T04:01:04.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:04.720+0000 7fcdd7214700 1 -- 172.21.15.32:0/125333267 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdd005f5c0 con 0x7fcdd00fbcc0 2024-05-12T04:01:04.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:04.721+0000 7fcdcd7fa700 1 -- 172.21.15.32:0/125333267 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 25) v1 ==== 88772+0+0 (secure 0 0 0) 0x7fcdc401ca00 con 0x7fcdd00fbcc0 2024-05-12T04:01:04.724 INFO:tasks.workunit.client.0.smithi032.stdout:9/53: stat d0/d3/d6/d7 0 2024-05-12T04:01:04.725 INFO:tasks.workunit.client.1.smithi033.stdout:2/237: mknod dc/d17/c52 0 2024-05-12T04:01:04.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:04.721+0000 7fcdcd7fa700 1 --2- 172.21.15.32:0/125333267 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7fcdc006af10 0x7fcdc006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:01:04.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:04.722+0000 7fcdcd7fa700 1 -- 172.21.15.32:0/125333267 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fcdc408a880 con 0x7fcdd00fbcc0 2024-05-12T04:01:04.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:04.722+0000 7fcdcffff700 1 --2- 172.21.15.32:0/125333267 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7fcdc006af10 0x7fcdc006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:01:04.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:04.725+0000 7fcdcffff700 1 --2- 172.21.15.32:0/125333267 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7fcdc006af10 0x7fcdc006d3c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc009b80 tx=0x7fcdbc00b040 comp rx=0 tx=0).ready entity=mgr.24389 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:01:04.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:04.730+0000 7fcdcd7fa700 1 -- 172.21.15.32:0/125333267 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181777 (secure 0 0 0) 0x7fcdc4054d90 con 0x7fcdd00fbcc0 2024-05-12T04:01:04.730 INFO:tasks.workunit.client.0.smithi032.stdout:3/92: mknod d15/d1b/c1e 0 2024-05-12T04:01:04.730 INFO:tasks.workunit.client.1.smithi033.stdout:7/387: mknod de/d20/d64/c92 0 2024-05-12T04:01:04.731 INFO:tasks.workunit.client.0.smithi032.stdout:8/38: write d0/f3 [438337,116978] 0 2024-05-12T04:01:04.731 INFO:tasks.workunit.client.1.smithi033.stdout:7/388: chown de/df/d17/d1b/l2a 12459195 1 2024-05-12T04:01:04.732 INFO:tasks.workunit.client.0.smithi032.stdout:2/49: rmdir d1 39 2024-05-12T04:01:04.734 INFO:tasks.workunit.client.1.smithi033.stdout:9/424: rmdir d2/d4/d29/d32 39 2024-05-12T04:01:04.736 INFO:tasks.workunit.client.0.smithi032.stdout:8/39: dread d0/f4 [0,4194304] 0 2024-05-12T04:01:04.737 INFO:tasks.workunit.client.0.smithi032.stdout:8/40: write d0/f5 [416395,10496] 0 2024-05-12T04:01:04.739 INFO:tasks.workunit.client.0.smithi032.stdout:8/41: chown d0/f4 16173236 1 2024-05-12T04:01:04.741 INFO:tasks.workunit.client.1.smithi033.stdout:3/228: truncate db/d17/d25/d32/f46 2905878 0 2024-05-12T04:01:04.743 INFO:tasks.workunit.client.1.smithi033.stdout:3/229: dread db/d17/f1b [0,4194304] 0 2024-05-12T04:01:04.743 INFO:tasks.workunit.client.0.smithi032.stdout:9/54: creat d0/d3/d6/d7/fc x:0 0 0 2024-05-12T04:01:04.744 INFO:tasks.workunit.client.0.smithi032.stdout:3/93: mknod d15/c1f 0 2024-05-12T04:01:04.746 INFO:tasks.workunit.client.0.smithi032.stdout:7/40: creat dd/fe x:0 0 0 2024-05-12T04:01:04.747 INFO:tasks.workunit.client.1.smithi033.stdout:5/178: symlink d3/d12/d29/l37 0 2024-05-12T04:01:04.749 INFO:tasks.workunit.client.1.smithi033.stdout:1/170: mkdir d11/d17/d38 0 2024-05-12T04:01:04.749 INFO:tasks.workunit.client.1.smithi033.stdout:1/171: stat lc 0 2024-05-12T04:01:04.750 INFO:tasks.workunit.client.1.smithi033.stdout:5/179: dread d3/f9 [4194304,4194304] 0 2024-05-12T04:01:04.751 INFO:tasks.workunit.client.1.smithi033.stdout:1/172: write d11/d17/f35 [790579,122206] 0 2024-05-12T04:01:04.752 INFO:tasks.workunit.client.0.smithi032.stdout:5/75: truncate d4/db/ff 5010857 0 2024-05-12T04:01:04.753 INFO:tasks.workunit.client.0.smithi032.stdout:1/47: dwrite d1/f6 [0,4194304] 0 2024-05-12T04:01:04.755 INFO:tasks.workunit.client.1.smithi033.stdout:6/178: truncate d4/d5/f27 4514916 0 2024-05-12T04:01:04.755 INFO:tasks.workunit.client.1.smithi033.stdout:9/425: rmdir d2 39 2024-05-12T04:01:04.755 INFO:tasks.workunit.client.1.smithi033.stdout:6/179: dread - d4/d5/f20 zero size 2024-05-12T04:01:04.755 INFO:tasks.workunit.client.0.smithi032.stdout:1/48: write d1/d7/f11 [458705,29453] 0 2024-05-12T04:01:04.758 INFO:tasks.workunit.client.0.smithi032.stdout:0/105: mkdir df/d28 0 2024-05-12T04:01:04.759 INFO:tasks.workunit.client.0.smithi032.stdout:0/106: read f5 [1424168,30015] 0 2024-05-12T04:01:04.761 INFO:tasks.workunit.client.0.smithi032.stdout:1/49: read d1/d7/f11 [101630,5834] 0 2024-05-12T04:01:04.761 INFO:tasks.workunit.client.1.smithi033.stdout:4/252: mknod dd/c47 0 2024-05-12T04:01:04.762 INFO:tasks.workunit.client.0.smithi032.stdout:2/50: dread d1/f4 [0,4194304] 0 2024-05-12T04:01:04.764 INFO:tasks.workunit.client.1.smithi033.stdout:3/230: mknod db/d17/d25/c50 0 2024-05-12T04:01:04.765 INFO:tasks.workunit.client.1.smithi033.stdout:3/231: write db/d17/d25/f40 [2721507,51716] 0 2024-05-12T04:01:04.765 INFO:tasks.workunit.client.0.smithi032.stdout:3/94: mkdir d15/d20 0 2024-05-12T04:01:04.768 INFO:tasks.workunit.client.0.smithi032.stdout:7/41: creat dd/ff x:0 0 0 2024-05-12T04:01:04.770 INFO:tasks.workunit.client.1.smithi033.stdout:5/180: creat d3/d12/d29/f38 x:0 0 0 2024-05-12T04:01:04.771 INFO:tasks.workunit.client.1.smithi033.stdout:5/181: readlink d3/d15/d16/l2f 0 2024-05-12T04:01:04.773 INFO:tasks.workunit.client.1.smithi033.stdout:1/173: mknod d11/d1c/c39 0 2024-05-12T04:01:04.773 INFO:tasks.workunit.client.1.smithi033.stdout:5/182: write d3/fb [1348231,123778] 0 2024-05-12T04:01:04.780 INFO:tasks.workunit.client.0.smithi032.stdout:8/42: creat d0/d9/fa x:0 0 0 2024-05-12T04:01:04.781 INFO:tasks.workunit.client.0.smithi032.stdout:8/43: rename d0 to d0/d9/db 22 2024-05-12T04:01:04.782 INFO:tasks.workunit.client.1.smithi033.stdout:5/183: dwrite d3/d12/f1d [0,4194304] 0 2024-05-12T04:01:04.783 INFO:tasks.workunit.client.0.smithi032.stdout:0/107: creat df/d10/d1f/f29 x:0 0 0 2024-05-12T04:01:04.784 INFO:tasks.workunit.client.1.smithi033.stdout:9/426: dread d2/d4/d2b/d6d/f5c [0,4194304] 0 2024-05-12T04:01:04.785 INFO:tasks.workunit.client.1.smithi033.stdout:9/427: write d2/d4/d2b/d6d/d3a/f45 [5191773,122180] 0 2024-05-12T04:01:04.786 INFO:tasks.workunit.client.0.smithi032.stdout:1/50: mknod d1/d7/c12 0 2024-05-12T04:01:04.789 INFO:tasks.workunit.client.0.smithi032.stdout:9/55: creat d0/d3/d6/d7/da/fd x:0 0 0 2024-05-12T04:01:04.790 INFO:tasks.workunit.client.1.smithi033.stdout:7/389: dwrite de/df/d17/f34 [0,4194304] 0 2024-05-12T04:01:04.793 INFO:tasks.workunit.client.0.smithi032.stdout:3/95: unlink f7 0 2024-05-12T04:01:04.795 INFO:tasks.workunit.client.1.smithi033.stdout:8/168: rename d1/d16/c21 to d1/d4/c34 0 2024-05-12T04:01:04.799 INFO:tasks.workunit.client.0.smithi032.stdout:7/42: symlink dd/l10 0 2024-05-12T04:01:04.802 INFO:tasks.workunit.client.1.smithi033.stdout:8/169: dwrite d1/d4/fd [0,4194304] 0 2024-05-12T04:01:04.803 INFO:tasks.workunit.client.1.smithi033.stdout:8/170: dread - d1/d16/d20/f2f zero size 2024-05-12T04:01:04.804 INFO:tasks.workunit.client.0.smithi032.stdout:8/44: rename d0/f4 to d0/d9/fc 0 2024-05-12T04:01:04.805 INFO:tasks.workunit.client.0.smithi032.stdout:8/45: fsync d0/f5 0 2024-05-12T04:01:04.808 INFO:tasks.workunit.client.0.smithi032.stdout:1/51: creat d1/f13 x:0 0 0 2024-05-12T04:01:04.809 INFO:tasks.workunit.client.0.smithi032.stdout:1/52: fdatasync d1/f6 0 2024-05-12T04:01:04.809 INFO:tasks.workunit.client.0.smithi032.stdout:2/51: mkdir d1/d13 0 2024-05-12T04:01:04.811 INFO:tasks.workunit.client.1.smithi033.stdout:7/390: creat de/d20/d64/f93 x:0 0 0 2024-05-12T04:01:04.813 INFO:tasks.workunit.client.0.smithi032.stdout:9/56: symlink d0/d3/d6/d7/da/le 0 2024-05-12T04:01:04.814 INFO:tasks.workunit.client.0.smithi032.stdout:9/57: chown d0/d3/d6/d7/fc 31411485 1 2024-05-12T04:01:04.816 INFO:tasks.workunit.client.0.smithi032.stdout:3/96: creat d15/d1b/f21 x:0 0 0 2024-05-12T04:01:04.817 INFO:tasks.workunit.client.0.smithi032.stdout:3/97: write d15/f1a [119080,68263] 0 2024-05-12T04:01:04.820 INFO:tasks.workunit.client.0.smithi032.stdout:5/76: mkdir d4/d7/d14/d1a 0 2024-05-12T04:01:04.820 INFO:tasks.workunit.client.1.smithi033.stdout:2/238: sync 2024-05-12T04:01:04.821 INFO:tasks.workunit.client.0.smithi032.stdout:7/43: fsync f1 0 2024-05-12T04:01:04.822 INFO:tasks.workunit.client.0.smithi032.stdout:5/77: readlink d4/db/l18 0 2024-05-12T04:01:04.825 INFO:tasks.workunit.client.1.smithi033.stdout:9/428: mkdir d2/d4/d29/d32/d9b 0 2024-05-12T04:01:04.827 INFO:tasks.workunit.client.0.smithi032.stdout:3/98: dwrite d15/d1b/f21 [0,4194304] 0 2024-05-12T04:01:04.829 INFO:tasks.workunit.client.1.smithi033.stdout:1/174: link d11/c2f d11/d17/d27/d36/c3a 0 2024-05-12T04:01:04.829 INFO:tasks.workunit.client.1.smithi033.stdout:1/175: readlink d11/d1c/l1f 0 2024-05-12T04:01:04.832 INFO:tasks.workunit.client.1.smithi033.stdout:1/176: dread f8 [4194304,4194304] 0 2024-05-12T04:01:04.833 INFO:tasks.workunit.client.0.smithi032.stdout:0/108: mknod df/c2a 0 2024-05-12T04:01:04.834 INFO:tasks.workunit.client.1.smithi033.stdout:7/391: creat de/d2f/d70/f94 x:0 0 0 2024-05-12T04:01:04.835 INFO:tasks.workunit.client.0.smithi032.stdout:7/44: dwrite fa [0,4194304] 0 2024-05-12T04:01:04.835 INFO:tasks.workunit.client.1.smithi033.stdout:7/392: chown de/df/d17/d1b/l2a 342 1 2024-05-12T04:01:04.841 INFO:tasks.workunit.client.1.smithi033.stdout:4/253: getdents dd 0 2024-05-12T04:01:04.843 INFO:tasks.workunit.client.1.smithi033.stdout:4/254: write dd/d11/f19 [409970,49915] 0 2024-05-12T04:01:04.843 INFO:tasks.workunit.client.1.smithi033.stdout:7/393: dwrite de/d2b/d6c/f3d [0,4194304] 0 2024-05-12T04:01:04.844 INFO:tasks.workunit.client.0.smithi032.stdout:9/58: rename d0/d3/f8 to d0/d3/ff 0 2024-05-12T04:01:04.845 INFO:tasks.workunit.client.1.smithi033.stdout:6/180: sync 2024-05-12T04:01:04.846 INFO:tasks.workunit.client.1.smithi033.stdout:7/394: write de/df/d17/f6a [944154,75261] 0 2024-05-12T04:01:04.846 INFO:tasks.workunit.client.1.smithi033.stdout:2/239: mknod dc/d13/d1c/c53 0 2024-05-12T04:01:04.847 INFO:tasks.workunit.client.1.smithi033.stdout:6/181: truncate d4/d1a/f23 495140 0 2024-05-12T04:01:04.848 INFO:tasks.workunit.client.1.smithi033.stdout:7/395: read de/df/d17/f6a [683413,53577] 0 2024-05-12T04:01:04.849 INFO:tasks.workunit.client.1.smithi033.stdout:5/184: rmdir d3/d12/d29/d23 0 2024-05-12T04:01:04.850 INFO:tasks.workunit.client.1.smithi033.stdout:7/396: write de/df/d17/d1b/d36/f8a [147421,72933] 0 2024-05-12T04:01:04.853 INFO:tasks.workunit.client.1.smithi033.stdout:6/182: dread d4/d5/f10 [0,4194304] 0 2024-05-12T04:01:04.859 INFO:tasks.workunit.client.0.smithi032.stdout:8/46: fsync d0/d9/fc 0 2024-05-12T04:01:04.859 INFO:tasks.workunit.client.0.smithi032.stdout:8/47: truncate d0/f5 765571 0 2024-05-12T04:01:04.860 INFO:tasks.workunit.client.0.smithi032.stdout:8/48: rename d0/d7 to d0/d7/dd 22 2024-05-12T04:01:04.861 INFO:tasks.workunit.client.1.smithi033.stdout:0/288: rename d6/dc/f53 to d6/db/f5d 0 2024-05-12T04:01:04.865 INFO:tasks.workunit.client.1.smithi033.stdout:0/289: dread d6/dc/f3c [0,4194304] 0 2024-05-12T04:01:04.866 INFO:tasks.workunit.client.0.smithi032.stdout:7/45: creat dd/f11 x:0 0 0 2024-05-12T04:01:04.869 INFO:tasks.workunit.client.0.smithi032.stdout:2/52: creat d1/d13/f14 x:0 0 0 2024-05-12T04:01:04.870 INFO:tasks.workunit.client.1.smithi033.stdout:2/240: chown dc/d13/d1c/l20 106467425 1 2024-05-12T04:01:04.873 INFO:tasks.workunit.client.0.smithi032.stdout:7/46: dwrite f6 [0,4194304] 0 2024-05-12T04:01:04.876 INFO:tasks.workunit.client.1.smithi033.stdout:5/185: mknod d3/d27/c39 0 2024-05-12T04:01:04.876 INFO:tasks.workunit.client.1.smithi033.stdout:5/186: readlink d3/d12/d29/l37 0 2024-05-12T04:01:04.877 INFO:tasks.workunit.client.1.smithi033.stdout:7/397: creat de/df/d17/d1e/d24/f95 x:0 0 0 2024-05-12T04:01:04.878 INFO:tasks.workunit.client.1.smithi033.stdout:7/398: stat de/df/d17/d1e 0 2024-05-12T04:01:04.880 INFO:tasks.workunit.client.0.smithi032.stdout:9/59: dwrite d0/d3/ff [0,4194304] 0 2024-05-12T04:01:04.881 INFO:tasks.workunit.client.1.smithi033.stdout:6/183: symlink d4/d5/l39 0 2024-05-12T04:01:04.881 INFO:tasks.workunit.client.0.smithi032.stdout:8/49: creat d0/d9/fe x:0 0 0 2024-05-12T04:01:04.884 INFO:tasks.workunit.client.1.smithi033.stdout:1/177: symlink d11/d17/d38/l3b 0 2024-05-12T04:01:04.885 INFO:tasks.workunit.client.0.smithi032.stdout:2/53: rename d1/d13/f14 to d1/d13/f15 0 2024-05-12T04:01:04.886 INFO:tasks.workunit.client.0.smithi032.stdout:2/54: dread - d1/fe zero size 2024-05-12T04:01:04.887 INFO:tasks.workunit.client.0.smithi032.stdout:7/47: rename dd/f11 to dd/f12 0 2024-05-12T04:01:04.889 INFO:tasks.workunit.client.1.smithi033.stdout:3/232: rename db/d17/d25/c27 to db/d17/d25/d32/d3c/d44/c51 0 2024-05-12T04:01:04.891 INFO:tasks.workunit.client.0.smithi032.stdout:5/78: rmdir d4/d7/d14/d1a 0 2024-05-12T04:01:04.898 INFO:tasks.workunit.client.1.smithi033.stdout:0/290: write d6/db/d2c/f3e [1135782,61325] 0 2024-05-12T04:01:04.900 INFO:tasks.workunit.client.0.smithi032.stdout:8/50: unlink d0/f3 0 2024-05-12T04:01:04.901 INFO:tasks.workunit.client.0.smithi032.stdout:8/51: dread - d0/d9/fa zero size 2024-05-12T04:01:04.902 INFO:tasks.workunit.client.0.smithi032.stdout:9/60: mkdir d0/d3/d6/d7/da/d10 0 2024-05-12T04:01:04.906 INFO:tasks.workunit.client.1.smithi033.stdout:9/429: dwrite d2/dd/d39/f54 [0,4194304] 0 2024-05-12T04:01:04.906 INFO:tasks.workunit.client.1.smithi033.stdout:0/291: dwrite d6/f26 [0,4194304] 0 2024-05-12T04:01:04.908 INFO:tasks.workunit.client.1.smithi033.stdout:9/430: write d2/d4/d2b/d6d/d3a/f5a [3573702,45919] 0 2024-05-12T04:01:04.908 INFO:tasks.workunit.client.1.smithi033.stdout:7/399: unlink de/df/c59 0 2024-05-12T04:01:04.909 INFO:tasks.workunit.client.1.smithi033.stdout:7/400: readlink de/d2f/d42/l58 0 2024-05-12T04:01:04.911 INFO:tasks.workunit.client.0.smithi032.stdout:2/55: mknod d1/d13/c16 0 2024-05-12T04:01:04.911 INFO:tasks.workunit.client.1.smithi033.stdout:7/401: chown de/df/l62 2 1 2024-05-12T04:01:04.917 INFO:tasks.workunit.client.1.smithi033.stdout:6/184: symlink d4/d5/l3a 0 2024-05-12T04:01:04.918 INFO:tasks.workunit.client.1.smithi033.stdout:4/255: truncate dd/d11/d15/f1e 1417244 0 2024-05-12T04:01:04.923 INFO:tasks.workunit.client.0.smithi032.stdout:0/109: getdents df 0 2024-05-12T04:01:04.923 INFO:tasks.workunit.client.1.smithi033.stdout:6/185: dwrite d4/f1f [0,4194304] 0 2024-05-12T04:01:04.925 INFO:tasks.workunit.client.1.smithi033.stdout:4/256: dread dd/d11/d31/d38/f44 [0,4194304] 0 2024-05-12T04:01:04.925 INFO:tasks.workunit.client.1.smithi033.stdout:8/171: rename d1/c15 to d1/d16/d29/c35 0 2024-05-12T04:01:04.932 INFO:tasks.workunit.client.0.smithi032.stdout:9/61: symlink d0/d3/d6/d7/da/l11 0 2024-05-12T04:01:04.933 INFO:tasks.workunit.client.1.smithi033.stdout:8/172: dwrite d1/d16/d29/f30 [0,4194304] 0 2024-05-12T04:01:04.933 INFO:tasks.workunit.client.0.smithi032.stdout:2/56: readlink d1/ld 0 2024-05-12T04:01:04.937 INFO:tasks.workunit.client.0.smithi032.stdout:0/110: creat df/f2b x:0 0 0 2024-05-12T04:01:04.938 INFO:tasks.workunit.client.0.smithi032.stdout:7/48: dwrite f1 [0,4194304] 0 2024-05-12T04:01:04.940 INFO:tasks.workunit.client.0.smithi032.stdout:7/49: read - f9 zero size 2024-05-12T04:01:04.942 INFO:tasks.workunit.client.1.smithi033.stdout:0/292: creat d6/dc/f5e x:0 0 0 2024-05-12T04:01:04.944 INFO:tasks.workunit.client.0.smithi032.stdout:8/52: link d0/f5 d0/d7/ff 0 2024-05-12T04:01:04.946 INFO:tasks.workunit.client.0.smithi032.stdout:8/53: chown d0/d9/fe 222791648 1 2024-05-12T04:01:04.946 INFO:tasks.workunit.client.0.smithi032.stdout:2/57: write d1/f4 [866780,108356] 0 2024-05-12T04:01:04.947 INFO:tasks.workunit.client.0.smithi032.stdout:8/54: dread d0/f2 [0,4194304] 0 2024-05-12T04:01:04.950 INFO:tasks.workunit.client.1.smithi033.stdout:9/431: mknod d2/d4/d2b/d78/d76/c9c 0 2024-05-12T04:01:04.954 INFO:tasks.workunit.client.1.smithi033.stdout:2/241: truncate dc/d13/d1c/f25 1876411 0 2024-05-12T04:01:04.959 INFO:tasks.workunit.client.0.smithi032.stdout:2/58: chown d1/f7 0 1 2024-05-12T04:01:04.961 INFO:tasks.workunit.client.1.smithi033.stdout:1/178: creat d11/d17/d27/d36/f3c x:0 0 0 2024-05-12T04:01:04.962 INFO:tasks.workunit.client.1.smithi033.stdout:1/179: truncate d11/d17/d27/f37 365542 0 2024-05-12T04:01:04.963 INFO:tasks.workunit.client.1.smithi033.stdout:5/187: rename d3/d15/d16/l2f to d3/d12/d29/d2b/l3a 0 2024-05-12T04:01:04.963 INFO:tasks.workunit.client.1.smithi033.stdout:6/186: rename d4/d5 to d4/d5/d3b 22 2024-05-12T04:01:04.964 INFO:tasks.workunit.client.0.smithi032.stdout:5/79: getdents d4/d7/d14 0 2024-05-12T04:01:04.964 INFO:tasks.workunit.client.1.smithi033.stdout:4/257: rename dd/d11 to dd/d11/d31/d48 22 2024-05-12T04:01:04.965 INFO:tasks.workunit.client.1.smithi033.stdout:4/258: write dd/f3d [255153,14455] 0 2024-05-12T04:01:04.965 INFO:tasks.workunit.client.1.smithi033.stdout:4/259: dread - dd/f3b zero size 2024-05-12T04:01:04.966 INFO:tasks.workunit.client.1.smithi033.stdout:4/260: stat dd/d11/d15/d42 0 2024-05-12T04:01:04.968 INFO:tasks.workunit.client.0.smithi032.stdout:8/55: dread d0/d9/fc [0,4194304] 0 2024-05-12T04:01:04.971 INFO:tasks.workunit.client.0.smithi032.stdout:8/56: truncate d0/d9/fe 244634 0 2024-05-12T04:01:04.976 INFO:tasks.workunit.client.1.smithi033.stdout:0/293: creat d6/dc/d58/f5f x:0 0 0 2024-05-12T04:01:04.977 INFO:tasks.workunit.client.1.smithi033.stdout:0/294: read - d6/dc/d39/f4f zero size 2024-05-12T04:01:04.977 INFO:tasks.workunit.client.0.smithi032.stdout:5/80: rmdir d4/db 39 2024-05-12T04:01:04.979 INFO:tasks.workunit.client.1.smithi033.stdout:2/242: mknod dc/d13/c54 0 2024-05-12T04:01:04.982 INFO:tasks.workunit.client.0.smithi032.stdout:5/81: dread d4/d7/f11 [0,4194304] 0 2024-05-12T04:01:04.983 INFO:tasks.workunit.client.0.smithi032.stdout:8/57: rename d0/d9/fc to d0/d9/f10 0 2024-05-12T04:01:04.985 INFO:tasks.workunit.client.0.smithi032.stdout:0/111: mknod df/d22/c2c 0 2024-05-12T04:01:04.988 INFO:tasks.workunit.client.1.smithi033.stdout:3/233: truncate db/d17/d25/d32/d3c/d44/f4a 1397398 0 2024-05-12T04:01:04.990 INFO:tasks.workunit.client.1.smithi033.stdout:6/187: mkdir d4/d1a/d3c 0 2024-05-12T04:01:04.992 INFO:tasks.workunit.client.0.smithi032.stdout:7/50: getdents dd 0 2024-05-12T04:01:04.992 INFO:tasks.workunit.client.0.smithi032.stdout:7/51: chown dd/ff 1 1 2024-05-12T04:01:04.994 INFO:tasks.workunit.client.1.smithi033.stdout:4/261: creat dd/d11/d15/f49 x:0 0 0 2024-05-12T04:01:04.996 INFO:tasks.workunit.client.1.smithi033.stdout:7/402: creat de/d20/f96 x:0 0 0 2024-05-12T04:01:04.997 INFO:tasks.workunit.client.0.smithi032.stdout:0/112: dread fd [0,4194304] 0 2024-05-12T04:01:04.998 INFO:tasks.workunit.client.1.smithi033.stdout:8/173: mkdir d1/d4/d1d/d36 0 2024-05-12T04:01:04.998 INFO:tasks.workunit.client.0.smithi032.stdout:0/113: write df/f2b [472917,47292] 0 2024-05-12T04:01:04.999 INFO:tasks.workunit.client.0.smithi032.stdout:2/59: rename d1/l10 to d1/l17 0 2024-05-12T04:01:05.001 INFO:tasks.workunit.client.1.smithi033.stdout:0/295: creat d6/d11/d57/f60 x:0 0 0 2024-05-12T04:01:05.006 INFO:tasks.workunit.client.0.smithi032.stdout:7/52: creat dd/f13 x:0 0 0 2024-05-12T04:01:05.009 INFO:tasks.workunit.client.1.smithi033.stdout:1/180: link d11/d17/f1d d11/d17/d27/d36/f3d 0 2024-05-12T04:01:05.012 INFO:tasks.workunit.client.0.smithi032.stdout:8/58: creat d0/d7/f11 x:0 0 0 2024-05-12T04:01:05.013 INFO:tasks.workunit.client.0.smithi032.stdout:8/59: write d0/d7/f11 [256767,111041] 0 2024-05-12T04:01:05.013 INFO:tasks.workunit.client.0.smithi032.stdout:8/60: dread - d0/d9/fa zero size 2024-05-12T04:01:05.018 INFO:tasks.workunit.client.1.smithi033.stdout:4/262: symlink dd/d11/d31/l4a 0 2024-05-12T04:01:05.020 INFO:tasks.workunit.client.1.smithi033.stdout:7/403: symlink de/d80/l97 0 2024-05-12T04:01:05.022 INFO:tasks.workunit.client.1.smithi033.stdout:1/181: dread d11/d17/d27/d36/f3d [0,4194304] 0 2024-05-12T04:01:05.023 INFO:tasks.workunit.client.1.smithi033.stdout:5/188: creat d3/f3b x:0 0 0 2024-05-12T04:01:05.023 INFO:tasks.workunit.client.0.smithi032.stdout:2/60: creat d1/d13/f18 x:0 0 0 2024-05-12T04:01:05.024 INFO:tasks.workunit.client.0.smithi032.stdout:7/53: rename dd/ff to dd/f14 0 2024-05-12T04:01:05.025 INFO:tasks.workunit.client.1.smithi033.stdout:6/188: mknod d4/d1a/d3c/c3d 0 2024-05-12T04:01:05.027 INFO:tasks.workunit.client.1.smithi033.stdout:1/182: dread d11/d17/d27/d36/f3d [0,4194304] 0 2024-05-12T04:01:05.028 INFO:tasks.workunit.client.1.smithi033.stdout:1/183: write d11/d17/d27/f37 [329109,4450] 0 2024-05-12T04:01:05.029 INFO:tasks.workunit.client.1.smithi033.stdout:1/184: fsync d11/f30 0 2024-05-12T04:01:05.029 INFO:tasks.workunit.client.1.smithi033.stdout:9/432: sync 2024-05-12T04:01:05.029 INFO:tasks.workunit.client.1.smithi033.stdout:8/174: mknod d1/d16/d27/d2a/c37 0 2024-05-12T04:01:05.030 INFO:tasks.workunit.client.0.smithi032.stdout:0/114: dwrite f5 [4194304,4194304] 0 2024-05-12T04:01:05.031 INFO:tasks.workunit.client.1.smithi033.stdout:8/175: dread - d1/d4/d2e/f31 zero size 2024-05-12T04:01:05.031 INFO:tasks.workunit.client.1.smithi033.stdout:9/433: stat d2/d4/d2b/d49/l6f 0 2024-05-12T04:01:05.032 INFO:tasks.workunit.client.0.smithi032.stdout:5/82: getdents d4 0 2024-05-12T04:01:05.032 INFO:tasks.workunit.client.0.smithi032.stdout:5/83: chown d4/db/f13 23 1 2024-05-12T04:01:05.033 INFO:tasks.workunit.client.0.smithi032.stdout:5/84: chown d4/db/f13 174283164 1 2024-05-12T04:01:05.034 INFO:tasks.workunit.client.0.smithi032.stdout:8/61: creat d0/d9/f12 x:0 0 0 2024-05-12T04:01:05.037 INFO:tasks.workunit.client.1.smithi033.stdout:2/243: rename dc/d13/d1c/c28 to dc/d17/d27/c55 0 2024-05-12T04:01:05.038 INFO:tasks.workunit.client.1.smithi033.stdout:2/244: chown dc/d13/d1f/f24 25 1 2024-05-12T04:01:05.039 INFO:tasks.workunit.client.0.smithi032.stdout:7/54: symlink dd/l15 0 2024-05-12T04:01:05.040 INFO:tasks.workunit.client.0.smithi032.stdout:7/55: write dd/fe [951038,6593] 0 2024-05-12T04:01:05.041 INFO:tasks.workunit.client.1.smithi033.stdout:7/404: creat de/d80/f98 x:0 0 0 2024-05-12T04:01:05.046 INFO:tasks.workunit.client.0.smithi032.stdout:0/115: stat df/d10/c1e 0 2024-05-12T04:01:05.047 INFO:tasks.workunit.client.0.smithi032.stdout:0/116: write fa [8977302,89728] 0 2024-05-12T04:01:05.047 INFO:tasks.workunit.client.1.smithi033.stdout:5/189: symlink d3/d15/d16/l3c 0 2024-05-12T04:01:05.049 INFO:tasks.workunit.client.0.smithi032.stdout:5/85: mkdir d4/db/d1b 0 2024-05-12T04:01:05.051 INFO:tasks.workunit.client.0.smithi032.stdout:8/62: fdatasync d0/f5 0 2024-05-12T04:01:05.052 INFO:tasks.workunit.client.1.smithi033.stdout:3/234: dwrite db/d17/d25/d32/f46 [0,4194304] 0 2024-05-12T04:01:05.053 INFO:tasks.workunit.client.0.smithi032.stdout:7/56: symlink dd/l16 0 2024-05-12T04:01:05.056 INFO:tasks.workunit.client.1.smithi033.stdout:4/263: link fa dd/f4b 0 2024-05-12T04:01:05.058 INFO:tasks.workunit.client.0.smithi032.stdout:2/61: write d1/f6 [894776,928] 0 2024-05-12T04:01:05.060 INFO:tasks.workunit.client.0.smithi032.stdout:2/62: write d1/f6 [350663,43403] 0 2024-05-12T04:01:05.062 INFO:tasks.workunit.client.0.smithi032.stdout:0/117: creat df/d10/d1f/f2d x:0 0 0 2024-05-12T04:01:05.063 INFO:tasks.workunit.client.1.smithi033.stdout:6/189: write d4/d5/f10 [1084476,81331] 0 2024-05-12T04:01:05.066 INFO:tasks.workunit.client.1.smithi033.stdout:9/434: mknod d2/d70/c9d 0 2024-05-12T04:01:05.067 INFO:tasks.workunit.client.0.smithi032.stdout:8/63: mkdir d0/d9/d13 0 2024-05-12T04:01:05.069 INFO:tasks.workunit.client.1.smithi033.stdout:0/296: creat d6/f61 x:0 0 0 2024-05-12T04:01:05.069 INFO:tasks.workunit.client.1.smithi033.stdout:0/297: chown d6/dc/f56 1775260 1 2024-05-12T04:01:05.070 INFO:tasks.workunit.client.0.smithi032.stdout:7/57: symlink dd/l17 0 2024-05-12T04:01:05.073 INFO:tasks.workunit.client.1.smithi033.stdout:7/405: creat de/df/d17/d1e/d24/f99 x:0 0 0 2024-05-12T04:01:05.073 INFO:tasks.workunit.client.1.smithi033.stdout:7/406: chown de 142508 1 2024-05-12T04:01:05.074 INFO:tasks.workunit.client.1.smithi033.stdout:5/190: creat d3/d12/d29/f3d x:0 0 0 2024-05-12T04:01:05.076 INFO:tasks.workunit.client.0.smithi032.stdout:0/118: readlink df/l26 0 2024-05-12T04:01:05.078 INFO:tasks.workunit.client.1.smithi033.stdout:3/235: rename db/f22 to db/d17/d25/f52 0 2024-05-12T04:01:05.078 INFO:tasks.workunit.client.0.smithi032.stdout:0/119: write df/d10/d1f/f2d [344495,91658] 0 2024-05-12T04:01:05.080 INFO:tasks.workunit.client.0.smithi032.stdout:5/86: dwrite d4/f12 [4194304,4194304] 0 2024-05-12T04:01:05.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.081+0000 7fcdd7214700 1 -- 172.21.15.32:0/125333267 --> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcdd0104800 con 0x7fcdc006af10 2024-05-12T04:01:05.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.083+0000 7fcdcd7fa700 1 -- 172.21.15.32:0/125333267 <== mgr.24389 v2:172.21.15.33:6828/1598726962 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7fcdd0104800 con 0x7fcdc006af10 2024-05-12T04:01:05.085 INFO:tasks.workunit.client.1.smithi033.stdout:2/245: dwrite dc/d17/d27/d2f/f41 [0,4194304] 0 2024-05-12T04:01:05.085 INFO:tasks.workunit.client.1.smithi033.stdout:6/190: creat d4/d1a/f3e x:0 0 0 2024-05-12T04:01:05.087 INFO:tasks.workunit.client.0.smithi032.stdout:2/63: link d1/d13/f18 d1/d13/f19 0 2024-05-12T04:01:05.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.088+0000 7fcdd7214700 1 -- 172.21.15.32:0/125333267 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7fcdc006af10 msgr2=0x7fcdc006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:01:05.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.088+0000 7fcdd7214700 1 --2- 172.21.15.32:0/125333267 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7fcdc006af10 0x7fcdc006d3c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc009b80 tx=0x7fcdbc00b040 comp rx=0 tx=0).stop 2024-05-12T04:01:05.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.088+0000 7fcdd7214700 1 -- 172.21.15.32:0/125333267 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdd00fbcc0 msgr2=0x7fcdd00665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:01:05.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.088+0000 7fcdd7214700 1 --2- 172.21.15.32:0/125333267 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdd00fbcc0 0x7fcdd00665c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fcdc40043d0 tx=0x7fcdc4009e30 comp rx=0 tx=0).stop 2024-05-12T04:01:05.089 INFO:tasks.workunit.client.0.smithi032.stdout:0/120: rmdir df 39 2024-05-12T04:01:05.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.089+0000 7fcdd7214700 1 -- 172.21.15.32:0/125333267 shutdown_connections 2024-05-12T04:01:05.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.089+0000 7fcdd7214700 1 --2- 172.21.15.32:0/125333267 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7fcdc006af10 0x7fcdc006d3c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:05.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.089+0000 7fcdd7214700 1 --2- 172.21.15.32:0/125333267 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdd00fbcc0 0x7fcdd00665c0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:05.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.089+0000 7fcdd7214700 1 --2- 172.21.15.32:0/125333267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdd00fcf70 0x7fcdd006ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:05.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.089+0000 7fcdd7214700 1 -- 172.21.15.32:0/125333267 >> 172.21.15.32:0/125333267 conn(0x7fcdd00f7420 msgr2=0x7fcdd0100220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:01:05.090 INFO:tasks.workunit.client.0.smithi032.stdout:8/64: dread d0/d7/ff [0,4194304] 0 2024-05-12T04:01:05.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.089+0000 7fcdd7214700 1 -- 172.21.15.32:0/125333267 shutdown_connections 2024-05-12T04:01:05.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.089+0000 7fcdd7214700 1 -- 172.21.15.32:0/125333267 wait complete. 2024-05-12T04:01:05.090 INFO:tasks.workunit.client.1.smithi033.stdout:9/435: fdatasync d2/f27 0 2024-05-12T04:01:05.091 INFO:tasks.workunit.client.0.smithi032.stdout:5/87: mknod d4/d7/c1c 0 2024-05-12T04:01:05.091 INFO:tasks.workunit.client.1.smithi033.stdout:0/298: truncate d6/db/f19 2599793 0 2024-05-12T04:01:05.093 INFO:tasks.workunit.client.0.smithi032.stdout:2/64: stat d1/f3 0 2024-05-12T04:01:05.094 INFO:tasks.workunit.client.1.smithi033.stdout:2/246: dread dc/d13/d1c/f4f [0,4194304] 0 2024-05-12T04:01:05.094 INFO:tasks.workunit.client.1.smithi033.stdout:6/191: dwrite d4/f29 [0,4194304] 0 2024-05-12T04:01:05.096 INFO:tasks.workunit.client.0.smithi032.stdout:7/58: dwrite dd/f14 [0,4194304] 0 2024-05-12T04:01:05.096 INFO:tasks.workunit.client.1.smithi033.stdout:7/407: mkdir de/df/d9a 0 2024-05-12T04:01:05.097 INFO:tasks.workunit.client.0.smithi032.stdout:8/65: rename d0 to d0/d9/d14 22 2024-05-12T04:01:05.099 INFO:tasks.workunit.client.1.smithi033.stdout:3/236: mkdir db/d17/d25/d32/d53 0 2024-05-12T04:01:05.100 INFO:tasks.workunit.client.0.smithi032.stdout:2/65: readlink d1/ld 0 2024-05-12T04:01:05.101 INFO:tasks.workunit.client.1.smithi033.stdout:1/185: creat d11/d17/f3e x:0 0 0 2024-05-12T04:01:05.102 INFO:tasks.workunit.client.0.smithi032.stdout:7/59: symlink dd/l18 0 2024-05-12T04:01:05.106 INFO:tasks.workunit.client.0.smithi032.stdout:5/88: rename d4/d7/f11 to d4/d7/d14/f1d 0 2024-05-12T04:01:05.107 INFO:tasks.workunit.client.1.smithi033.stdout:1/186: truncate d11/d1c/f33 143308 0 2024-05-12T04:01:05.108 INFO:tasks.workunit.client.1.smithi033.stdout:7/408: dwrite de/df/d17/f88 [0,4194304] 0 2024-05-12T04:01:05.112 INFO:teuthology.orchestra.run.smithi032.stdout:true 2024-05-12T04:01:05.112 INFO:tasks.workunit.client.0.smithi032.stdout:0/121: creat df/d10/d1f/f2e x:0 0 0 2024-05-12T04:01:05.115 INFO:tasks.workunit.client.0.smithi032.stdout:0/122: dread - df/d10/d1f/f2e zero size 2024-05-12T04:01:05.117 INFO:tasks.workunit.client.1.smithi033.stdout:9/436: creat d2/d4/f9e x:0 0 0 2024-05-12T04:01:05.118 INFO:tasks.workunit.client.0.smithi032.stdout:0/123: write df/d10/f13 [2932030,40999] 0 2024-05-12T04:01:05.119 INFO:tasks.workunit.client.0.smithi032.stdout:7/60: mkdir dd/d19 0 2024-05-12T04:01:05.121 INFO:tasks.workunit.client.1.smithi033.stdout:0/299: creat d6/db/d2c/f62 x:0 0 0 2024-05-12T04:01:05.122 INFO:tasks.workunit.client.0.smithi032.stdout:0/124: dread f5 [0,4194304] 0 2024-05-12T04:01:05.124 INFO:tasks.workunit.client.0.smithi032.stdout:5/89: creat d4/f1e x:0 0 0 2024-05-12T04:01:05.125 INFO:tasks.workunit.client.0.smithi032.stdout:0/125: truncate df/d10/d1f/f2e 837464 0 2024-05-12T04:01:05.127 INFO:tasks.workunit.client.1.smithi033.stdout:6/192: symlink d4/d1a/l3f 0 2024-05-12T04:01:05.129 INFO:tasks.workunit.client.0.smithi032.stdout:2/66: mknod d1/c1a 0 2024-05-12T04:01:05.132 INFO:tasks.workunit.client.0.smithi032.stdout:0/126: dwrite f9 [0,4194304] 0 2024-05-12T04:01:05.134 INFO:tasks.workunit.client.0.smithi032.stdout:0/127: chown df/d10/c11 2360706 1 2024-05-12T04:01:05.139 INFO:tasks.workunit.client.0.smithi032.stdout:7/61: rename f1 to dd/f1a 0 2024-05-12T04:01:05.142 INFO:tasks.workunit.client.0.smithi032.stdout:8/66: dwrite d0/f2 [0,4194304] 0 2024-05-12T04:01:05.144 INFO:tasks.workunit.client.1.smithi033.stdout:1/187: dread fd [0,4194304] 0 2024-05-12T04:01:05.145 INFO:tasks.workunit.client.1.smithi033.stdout:7/409: stat de/df/d65/l6b 0 2024-05-12T04:01:05.148 INFO:tasks.workunit.client.1.smithi033.stdout:8/176: getdents d1/d16/d20 0 2024-05-12T04:01:05.149 INFO:tasks.workunit.client.1.smithi033.stdout:7/410: dread - de/df/d17/d1e/f4c zero size 2024-05-12T04:01:05.149 INFO:tasks.workunit.client.1.smithi033.stdout:9/437: creat d2/d70/d7b/f9f x:0 0 0 2024-05-12T04:01:05.151 INFO:tasks.workunit.client.1.smithi033.stdout:0/300: rename d6/dc/d58/f5f to d6/d11/d57/f63 0 2024-05-12T04:01:05.152 INFO:tasks.workunit.client.1.smithi033.stdout:6/193: rename d4/d1a/d3c to d4/d1a/d3c/d40 22 2024-05-12T04:01:05.153 INFO:tasks.workunit.client.1.smithi033.stdout:3/237: dwrite db/d17/f2c [0,4194304] 0 2024-05-12T04:01:05.153 INFO:tasks.workunit.client.1.smithi033.stdout:2/247: creat dc/d17/d3a/f56 x:0 0 0 2024-05-12T04:01:05.156 INFO:tasks.workunit.client.0.smithi032.stdout:2/67: rename d1/f6 to d1/d13/f1b 0 2024-05-12T04:01:05.157 INFO:tasks.workunit.client.0.smithi032.stdout:2/68: rename d1 to d1/d13/d1c 22 2024-05-12T04:01:05.160 INFO:tasks.workunit.client.1.smithi033.stdout:4/264: creat dd/d11/d15/d42/f4c x:0 0 0 2024-05-12T04:01:05.161 INFO:tasks.workunit.client.0.smithi032.stdout:8/67: creat d0/d7/f15 x:0 0 0 2024-05-12T04:01:05.163 INFO:tasks.workunit.client.1.smithi033.stdout:1/188: write d11/d1c/f34 [3559118,54445] 0 2024-05-12T04:01:05.165 INFO:tasks.workunit.client.0.smithi032.stdout:2/69: rename d1/d13/f15 to d1/d13/f1d 0 2024-05-12T04:01:05.165 INFO:tasks.workunit.client.0.smithi032.stdout:2/70: fdatasync d1/f12 0 2024-05-12T04:01:05.166 INFO:tasks.workunit.client.0.smithi032.stdout:7/62: rmdir dd 39 2024-05-12T04:01:05.169 INFO:tasks.workunit.client.0.smithi032.stdout:0/128: write fb [1021062,101684] 0 2024-05-12T04:01:05.169 INFO:tasks.workunit.client.1.smithi033.stdout:8/177: symlink d1/d16/d20/l38 0 2024-05-12T04:01:05.171 INFO:tasks.workunit.client.0.smithi032.stdout:8/68: unlink d0/c8 0 2024-05-12T04:01:05.174 INFO:tasks.workunit.client.0.smithi032.stdout:0/129: dread df/d10/f13 [0,4194304] 0 2024-05-12T04:01:05.176 INFO:tasks.workunit.client.0.smithi032.stdout:0/130: dread df/d10/d1f/f2e [0,4194304] 0 2024-05-12T04:01:05.177 INFO:tasks.workunit.client.1.smithi033.stdout:0/301: unlink d6/db/f5d 0 2024-05-12T04:01:05.180 INFO:tasks.workunit.client.0.smithi032.stdout:5/90: symlink d4/db/l1f 0 2024-05-12T04:01:05.181 INFO:tasks.workunit.client.0.smithi032.stdout:6/44: sync 2024-05-12T04:01:05.181 INFO:tasks.workunit.client.0.smithi032.stdout:4/22: sync 2024-05-12T04:01:05.181 INFO:tasks.workunit.client.0.smithi032.stdout:4/23: chown d0 3 1 2024-05-12T04:01:05.182 INFO:tasks.workunit.client.0.smithi032.stdout:6/45: chown f3 19382 1 2024-05-12T04:01:05.184 INFO:tasks.workunit.client.0.smithi032.stdout:4/24: read d0/d3/f4 [476298,86245] 0 2024-05-12T04:01:05.185 INFO:tasks.workunit.client.1.smithi033.stdout:6/194: rename d4/d5/f33 to d4/d5/f41 0 2024-05-12T04:01:05.185 INFO:tasks.workunit.client.1.smithi033.stdout:6/195: chown d4/d1a 3900 1 2024-05-12T04:01:05.186 INFO:tasks.workunit.client.0.smithi032.stdout:0/131: dwrite f9 [0,4194304] 0 2024-05-12T04:01:05.191 INFO:tasks.workunit.client.1.smithi033.stdout:5/191: getdents d3/d12/d25/d26 0 2024-05-12T04:01:05.192 INFO:tasks.workunit.client.1.smithi033.stdout:5/192: fsync d3/d12/f1f 0 2024-05-12T04:01:05.193 INFO:tasks.workunit.client.0.smithi032.stdout:4/25: dwrite d0/d3/f4 [0,4194304] 0 2024-05-12T04:01:05.194 INFO:tasks.workunit.client.0.smithi032.stdout:7/63: fdatasync dd/f12 0 2024-05-12T04:01:05.194 INFO:tasks.workunit.client.0.smithi032.stdout:8/69: rmdir d0 39 2024-05-12T04:01:05.195 INFO:tasks.workunit.client.0.smithi032.stdout:7/64: write dd/f12 [245142,92448] 0 2024-05-12T04:01:05.199 INFO:tasks.workunit.client.1.smithi033.stdout:9/438: rename d2/d4/d2b/d6d/d4c/f83 to d2/d4/d29/d32/d9b/fa0 0 2024-05-12T04:01:05.205 INFO:tasks.workunit.client.1.smithi033.stdout:7/411: mknod de/df/d17/c9b 0 2024-05-12T04:01:05.206 INFO:tasks.workunit.client.1.smithi033.stdout:1/189: dread f8 [4194304,4194304] 0 2024-05-12T04:01:05.206 INFO:tasks.workunit.client.1.smithi033.stdout:1/190: fdatasync d11/d17/d27/f31 0 2024-05-12T04:01:05.208 INFO:tasks.workunit.client.0.smithi032.stdout:0/132: creat df/d22/f2f x:0 0 0 2024-05-12T04:01:05.210 INFO:tasks.workunit.client.0.smithi032.stdout:6/46: dwrite f3 [0,4194304] 0 2024-05-12T04:01:05.210 INFO:tasks.workunit.client.0.smithi032.stdout:0/133: chown df/d10/c1e 23 1 2024-05-12T04:01:05.211 INFO:tasks.workunit.client.0.smithi032.stdout:2/71: creat d1/f1e x:0 0 0 2024-05-12T04:01:05.212 INFO:tasks.workunit.client.0.smithi032.stdout:2/72: read d1/f7 [1219465,121460] 0 2024-05-12T04:01:05.212 INFO:tasks.workunit.client.1.smithi033.stdout:6/196: creat d4/d1a/f42 x:0 0 0 2024-05-12T04:01:05.212 INFO:tasks.workunit.client.1.smithi033.stdout:6/197: write d4/f16 [5054040,25037] 0 2024-05-12T04:01:05.213 INFO:tasks.workunit.client.1.smithi033.stdout:2/248: symlink dc/d17/d27/d47/l57 0 2024-05-12T04:01:05.215 INFO:tasks.workunit.client.0.smithi032.stdout:8/70: chown d0/d9/fa 99 1 2024-05-12T04:01:05.217 INFO:tasks.workunit.client.0.smithi032.stdout:4/26: symlink d0/l5 0 2024-05-12T04:01:05.218 INFO:tasks.workunit.client.0.smithi032.stdout:7/65: mkdir dd/d1b 0 2024-05-12T04:01:05.221 INFO:tasks.workunit.client.0.smithi032.stdout:2/73: dwrite d1/f12 [0,4194304] 0 2024-05-12T04:01:05.222 INFO:tasks.workunit.client.1.smithi033.stdout:9/439: symlink d2/dd/d39/la1 0 2024-05-12T04:01:05.228 INFO:tasks.workunit.client.0.smithi032.stdout:0/134: truncate f5 4302821 0 2024-05-12T04:01:05.228 INFO:tasks.workunit.client.1.smithi033.stdout:7/412: write de/f26 [3314822,81421] 0 2024-05-12T04:01:05.230 INFO:tasks.workunit.client.1.smithi033.stdout:3/238: creat db/f54 x:0 0 0 2024-05-12T04:01:05.233 INFO:tasks.workunit.client.1.smithi033.stdout:1/191: mkdir d11/d17/d27/d36/d3f 0 2024-05-12T04:01:05.233 INFO:tasks.workunit.client.1.smithi033.stdout:9/440: dwrite d2/d4/d29/d3c/f96 [0,4194304] 0 2024-05-12T04:01:05.234 INFO:tasks.workunit.client.0.smithi032.stdout:4/27: symlink d0/d3/l6 0 2024-05-12T04:01:05.238 INFO:tasks.workunit.client.0.smithi032.stdout:7/66: dwrite f9 [0,4194304] 0 2024-05-12T04:01:05.241 INFO:tasks.workunit.client.1.smithi033.stdout:6/198: mkdir d4/d43 0 2024-05-12T04:01:05.241 INFO:tasks.workunit.client.1.smithi033.stdout:6/199: stat d4/d1a/c28 0 2024-05-12T04:01:05.244 INFO:tasks.workunit.client.0.smithi032.stdout:6/47: mkdir d7/d9 0 2024-05-12T04:01:05.245 INFO:tasks.workunit.client.0.smithi032.stdout:4/28: dread d0/d3/f4 [0,4194304] 0 2024-05-12T04:01:05.245 INFO:tasks.workunit.client.0.smithi032.stdout:4/29: stat d0/d3 0 2024-05-12T04:01:05.246 INFO:tasks.workunit.client.0.smithi032.stdout:2/74: creat d1/d13/f1f x:0 0 0 2024-05-12T04:01:05.247 INFO:tasks.workunit.client.0.smithi032.stdout:4/30: truncate d0/d3/f4 4397288 0 2024-05-12T04:01:05.254 INFO:tasks.workunit.client.0.smithi032.stdout:5/91: link d4/db/f13 d4/f20 0 2024-05-12T04:01:05.256 INFO:tasks.workunit.client.1.smithi033.stdout:3/239: creat db/d17/d25/d32/d3c/d44/f55 x:0 0 0 2024-05-12T04:01:05.256 INFO:tasks.workunit.client.1.smithi033.stdout:8/178: truncate d1/f13 1595770 0 2024-05-12T04:01:05.261 INFO:tasks.workunit.client.0.smithi032.stdout:6/48: creat d7/fa x:0 0 0 2024-05-12T04:01:05.263 INFO:tasks.workunit.client.0.smithi032.stdout:6/49: write d7/fa [232194,104865] 0 2024-05-12T04:01:05.264 INFO:tasks.workunit.client.0.smithi032.stdout:2/75: fsync d1/fe 0 2024-05-12T04:01:05.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.264+0000 7f6398e4d700 1 -- 172.21.15.32:0/3230853555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63940fbce0 msgr2=0x7f63940fc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:01:05.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.264+0000 7f6398e4d700 1 --2- 172.21.15.32:0/3230853555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63940fbce0 0x7f63940fc0f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f6384004940 tx=0x7f6384009e30 comp rx=0 tx=0).stop 2024-05-12T04:01:05.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.264+0000 7f6398e4d700 1 -- 172.21.15.32:0/3230853555 shutdown_connections 2024-05-12T04:01:05.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.264+0000 7f6398e4d700 1 --2- 172.21.15.32:0/3230853555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63940fcf90 0x7f63940fd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:05.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.264+0000 7f6398e4d700 1 --2- 172.21.15.32:0/3230853555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63940fbce0 0x7f63940fc0f0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:05.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.264+0000 7f6398e4d700 1 -- 172.21.15.32:0/3230853555 >> 172.21.15.32:0/3230853555 conn(0x7f63940f7440 msgr2=0x7f63940f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:01:05.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.264+0000 7f6398e4d700 1 -- 172.21.15.32:0/3230853555 shutdown_connections 2024-05-12T04:01:05.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.264+0000 7f6398e4d700 1 -- 172.21.15.32:0/3230853555 wait complete. 2024-05-12T04:01:05.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.265+0000 7f6398e4d700 1 Processor -- start 2024-05-12T04:01:05.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.265+0000 7f6398e4d700 1 -- start start 2024-05-12T04:01:05.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.265+0000 7f6398e4d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63940fbce0 0x7f639410b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:01:05.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.265+0000 7f6398e4d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63940fcf90 0x7f639410bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:01:05.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.265+0000 7f6398e4d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f639410c1c0 con 0x7f63940fcf90 2024-05-12T04:01:05.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.265+0000 7f6398e4d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f639410ca10 con 0x7f63940fbce0 2024-05-12T04:01:05.267 INFO:tasks.workunit.client.0.smithi032.stdout:6/50: fdatasync d7/fa 0 2024-05-12T04:01:05.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.266+0000 7f639259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63940fbce0 0x7f639410b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:01:05.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.266+0000 7f6391d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63940fcf90 0x7f639410bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:01:05.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.266+0000 7f639259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63940fbce0 0x7f639410b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.32:41556/0 (socket says 172.21.15.32:41556) 2024-05-12T04:01:05.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.266+0000 7f639259c700 1 -- 172.21.15.32:0/2794001534 learned_addr learned my addr 172.21.15.32:0/2794001534 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:01:05.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.266+0000 7f639259c700 1 -- 172.21.15.32:0/2794001534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63940fcf90 msgr2=0x7f639410bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:01:05.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.266+0000 7f639259c700 1 --2- 172.21.15.32:0/2794001534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63940fcf90 0x7f639410bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:05.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.266+0000 7f639259c700 1 -- 172.21.15.32:0/2794001534 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6388005040 con 0x7f63940fbce0 2024-05-12T04:01:05.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.266+0000 7f639259c700 1 --2- 172.21.15.32:0/2794001534 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63940fbce0 0x7f639410b680 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f638400b0d0 tx=0x7f6384009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:01:05.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.266+0000 7f6382ffd700 1 -- 172.21.15.32:0/2794001534 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f638400e070 con 0x7f63940fbce0 2024-05-12T04:01:05.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.266+0000 7f6382ffd700 1 -- 172.21.15.32:0/2794001534 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f638400b410 con 0x7f63940fbce0 2024-05-12T04:01:05.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.266+0000 7f6398e4d700 1 -- 172.21.15.32:0/2794001534 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6384005040 con 0x7f63940fbce0 2024-05-12T04:01:05.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.267+0000 7f6382ffd700 1 -- 172.21.15.32:0/2794001534 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6384012600 con 0x7f63940fbce0 2024-05-12T04:01:05.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.267+0000 7f6398e4d700 1 -- 172.21.15.32:0/2794001534 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f639410d040 con 0x7f63940fbce0 2024-05-12T04:01:05.269 INFO:tasks.workunit.client.1.smithi033.stdout:6/200: unlink d4/d5/c22 0 2024-05-12T04:01:05.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.268+0000 7f6398e4d700 1 -- 172.21.15.32:0/2794001534 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f639405f5c0 con 0x7f63940fbce0 2024-05-12T04:01:05.269 INFO:tasks.workunit.client.0.smithi032.stdout:4/31: symlink d0/l7 0 2024-05-12T04:01:05.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.268+0000 7f6382ffd700 1 -- 172.21.15.32:0/2794001534 <== mon.1 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) v1 ==== 88772+0+0 (secure 0 0 0) 0x7f6384013a50 con 0x7f63940fbce0 2024-05-12T04:01:05.270 INFO:tasks.workunit.client.0.smithi032.stdout:6/51: dread d7/fa [0,4194304] 0 2024-05-12T04:01:05.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.269+0000 7f6382ffd700 1 --2- 172.21.15.32:0/2794001534 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f637c06aec0 0x7f637c06d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:01:05.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.269+0000 7f6382ffd700 1 -- 172.21.15.32:0/2794001534 <== mon.1 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f638408a440 con 0x7f63940fbce0 2024-05-12T04:01:05.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.269+0000 7f6391d9b700 1 --2- 172.21.15.32:0/2794001534 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f637c06aec0 0x7f637c06d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:01:05.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.270+0000 7f6391d9b700 1 --2- 172.21.15.32:0/2794001534 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f637c06aec0 0x7f637c06d370 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6388009fd0 tx=0x7f6388009e30 comp rx=0 tx=0).ready entity=mgr.24389 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:01:05.271 INFO:tasks.workunit.client.0.smithi032.stdout:6/52: chown d7/fa 1986108 1 2024-05-12T04:01:05.272 INFO:tasks.workunit.client.1.smithi033.stdout:5/193: rmdir d3/d12/d29/d34 0 2024-05-12T04:01:05.272 INFO:tasks.workunit.client.1.smithi033.stdout:6/201: dwrite d4/d5/f9 [0,4194304] 0 2024-05-12T04:01:05.273 INFO:tasks.workunit.client.0.smithi032.stdout:0/135: mkdir df/d30 0 2024-05-12T04:01:05.273 INFO:tasks.workunit.client.1.smithi033.stdout:5/194: dread - d3/d12/d29/f38 zero size 2024-05-12T04:01:05.275 INFO:tasks.workunit.client.1.smithi033.stdout:4/265: getdents dd/d11/d15 0 2024-05-12T04:01:05.275 INFO:tasks.workunit.client.1.smithi033.stdout:4/266: stat dd/d11/c2c 0 2024-05-12T04:01:05.276 INFO:tasks.workunit.client.1.smithi033.stdout:4/267: stat dd/d11/f46 0 2024-05-12T04:01:05.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.277+0000 7f6382ffd700 1 -- 172.21.15.32:0/2794001534 <== mon.1 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181777 (secure 0 0 0) 0x7f63840584f0 con 0x7f63940fbce0 2024-05-12T04:01:05.280 INFO:tasks.workunit.client.0.smithi032.stdout:7/67: creat dd/d19/f1c x:0 0 0 2024-05-12T04:01:05.285 INFO:tasks.workunit.client.0.smithi032.stdout:8/71: dwrite d0/d9/f10 [0,4194304] 0 2024-05-12T04:01:05.290 INFO:tasks.workunit.client.1.smithi033.stdout:9/441: symlink d2/dd/d79/la2 0 2024-05-12T04:01:05.295 INFO:tasks.workunit.client.0.smithi032.stdout:6/53: fdatasync f6 0 2024-05-12T04:01:05.299 INFO:tasks.workunit.client.1.smithi033.stdout:6/202: unlink d4/d5/c17 0 2024-05-12T04:01:05.299 INFO:tasks.workunit.client.1.smithi033.stdout:2/249: dwrite f7 [0,4194304] 0 2024-05-12T04:01:05.300 INFO:tasks.workunit.client.0.smithi032.stdout:4/32: dwrite d0/d3/f4 [0,4194304] 0 2024-05-12T04:01:05.302 INFO:tasks.workunit.client.0.smithi032.stdout:5/92: fsync d4/fa 0 2024-05-12T04:01:05.304 INFO:tasks.workunit.client.0.smithi032.stdout:4/33: read d0/d3/f4 [812851,118031] 0 2024-05-12T04:01:05.306 INFO:tasks.workunit.client.0.smithi032.stdout:7/68: creat dd/f1d x:0 0 0 2024-05-12T04:01:05.309 INFO:tasks.workunit.client.0.smithi032.stdout:4/34: dread d0/d3/f4 [0,4194304] 0 2024-05-12T04:01:05.310 INFO:tasks.workunit.client.1.smithi033.stdout:7/413: link de/df/d60/c5e de/df/d17/d1e/d24/c9c 0 2024-05-12T04:01:05.311 INFO:tasks.workunit.client.0.smithi032.stdout:8/72: mknod d0/d7/c16 0 2024-05-12T04:01:05.312 INFO:tasks.workunit.client.0.smithi032.stdout:4/35: write d0/d3/f4 [3124541,40121] 0 2024-05-12T04:01:05.312 INFO:tasks.workunit.client.0.smithi032.stdout:8/73: rename d0 to d0/d9/d17 22 2024-05-12T04:01:05.313 INFO:tasks.workunit.client.0.smithi032.stdout:4/36: write d0/d3/f4 [946196,18869] 0 2024-05-12T04:01:05.314 INFO:tasks.workunit.client.1.smithi033.stdout:3/240: rename db/f31 to db/d17/d25/d32/d3c/f56 0 2024-05-12T04:01:05.314 INFO:tasks.workunit.client.1.smithi033.stdout:7/414: dread de/d20/f21 [0,4194304] 0 2024-05-12T04:01:05.314 INFO:tasks.workunit.client.0.smithi032.stdout:4/37: readlink d0/d3/l6 0 2024-05-12T04:01:05.315 INFO:tasks.workunit.client.0.smithi032.stdout:4/38: chown d0/l5 8 1 2024-05-12T04:01:05.320 INFO:tasks.workunit.client.0.smithi032.stdout:4/39: fdatasync d0/d3/f4 0 2024-05-12T04:01:05.321 INFO:tasks.workunit.client.0.smithi032.stdout:4/40: chown d0/d3/l6 177341 1 2024-05-12T04:01:05.323 INFO:tasks.workunit.client.0.smithi032.stdout:6/54: write f6 [4817725,112821] 0 2024-05-12T04:01:05.324 INFO:tasks.workunit.client.0.smithi032.stdout:1/53: sync 2024-05-12T04:01:05.324 INFO:tasks.workunit.client.1.smithi033.stdout:2/250: creat dc/d17/d27/d47/f58 x:0 0 0 2024-05-12T04:01:05.327 INFO:tasks.workunit.client.0.smithi032.stdout:0/136: write df/d22/f2f [958628,26884] 0 2024-05-12T04:01:05.329 INFO:tasks.workunit.client.1.smithi033.stdout:7/415: dread de/df/d17/d1b/d36/f4a [0,4194304] 0 2024-05-12T04:01:05.331 INFO:tasks.workunit.client.1.smithi033.stdout:4/268: dwrite dd/f4b [4194304,4194304] 0 2024-05-12T04:01:05.334 INFO:tasks.workunit.client.0.smithi032.stdout:2/76: truncate d1/f4 845920 0 2024-05-12T04:01:05.334 INFO:tasks.workunit.client.0.smithi032.stdout:1/54: dwrite d1/f6 [0,4194304] 0 2024-05-12T04:01:05.336 INFO:tasks.workunit.client.0.smithi032.stdout:2/77: dread - d1/d13/f18 zero size 2024-05-12T04:01:05.340 INFO:tasks.workunit.client.1.smithi033.stdout:0/302: sync 2024-05-12T04:01:05.346 INFO:tasks.workunit.client.0.smithi032.stdout:2/78: dwrite d1/f12 [4194304,4194304] 0 2024-05-12T04:01:05.348 INFO:tasks.workunit.client.1.smithi033.stdout:5/195: truncate d3/d12/d25/d26/d2e/f36 2606512 0 2024-05-12T04:01:05.348 INFO:tasks.workunit.client.1.smithi033.stdout:8/179: creat d1/d4/f39 x:0 0 0 2024-05-12T04:01:05.348 INFO:tasks.workunit.client.1.smithi033.stdout:3/241: rmdir db/d17/d25/d32/d3c/d44 39 2024-05-12T04:01:05.353 INFO:tasks.workunit.client.0.smithi032.stdout:4/41: rmdir d0 39 2024-05-12T04:01:05.356 INFO:tasks.workunit.client.1.smithi033.stdout:1/192: link d11/c2f d11/d17/d27/d36/c40 0 2024-05-12T04:01:05.356 INFO:tasks.workunit.client.1.smithi033.stdout:1/193: chown d11 11815442 1 2024-05-12T04:01:05.356 INFO:tasks.workunit.client.1.smithi033.stdout:8/180: dwrite d1/d4/fd [4194304,4194304] 0 2024-05-12T04:01:05.358 INFO:tasks.workunit.client.1.smithi033.stdout:1/194: write d11/d17/d27/d36/f3c [858896,78142] 0 2024-05-12T04:01:05.360 INFO:tasks.workunit.client.0.smithi032.stdout:0/137: stat df/d10/d1f/f2e 0 2024-05-12T04:01:05.362 INFO:tasks.workunit.client.1.smithi033.stdout:6/203: getdents d4/d43 0 2024-05-12T04:01:05.363 INFO:tasks.workunit.client.1.smithi033.stdout:6/204: write d4/f16 [5404556,3414] 0 2024-05-12T04:01:05.366 INFO:tasks.workunit.client.0.smithi032.stdout:3/99: sync 2024-05-12T04:01:05.367 INFO:tasks.workunit.client.0.smithi032.stdout:9/62: sync 2024-05-12T04:01:05.367 INFO:tasks.workunit.client.0.smithi032.stdout:7/69: mkdir dd/d1b/d1e 0 2024-05-12T04:01:05.368 INFO:tasks.workunit.client.1.smithi033.stdout:6/205: write d4/d5/f35 [980338,27342] 0 2024-05-12T04:01:05.369 INFO:tasks.workunit.client.0.smithi032.stdout:7/70: write f6 [3852101,44745] 0 2024-05-12T04:01:05.369 INFO:tasks.workunit.client.0.smithi032.stdout:9/63: chown d0/d3/c5 0 1 2024-05-12T04:01:05.370 INFO:tasks.workunit.client.1.smithi033.stdout:7/416: rename de/d80/l97 to de/l9d 0 2024-05-12T04:01:05.371 INFO:tasks.workunit.client.0.smithi032.stdout:5/93: write d4/fd [4382551,63090] 0 2024-05-12T04:01:05.372 INFO:tasks.workunit.client.0.smithi032.stdout:1/55: mknod d1/d7/c14 0 2024-05-12T04:01:05.375 INFO:tasks.workunit.client.1.smithi033.stdout:2/251: dwrite dc/d13/d1c/f21 [0,4194304] 0 2024-05-12T04:01:05.375 INFO:tasks.workunit.client.1.smithi033.stdout:0/303: creat d6/dc/d39/f64 x:0 0 0 2024-05-12T04:01:05.378 INFO:tasks.workunit.client.1.smithi033.stdout:0/304: write d6/dc/d39/f40 [644432,72699] 0 2024-05-12T04:01:05.379 INFO:tasks.workunit.client.1.smithi033.stdout:6/206: dwrite d4/d1a/f23 [0,4194304] 0 2024-05-12T04:01:05.380 INFO:tasks.workunit.client.1.smithi033.stdout:6/207: write d4/f2f [577877,38400] 0 2024-05-12T04:01:05.380 INFO:tasks.workunit.client.0.smithi032.stdout:8/74: getdents d0/d9/d13 0 2024-05-12T04:01:05.382 INFO:tasks.workunit.client.0.smithi032.stdout:8/75: dread - d0/d9/f12 zero size 2024-05-12T04:01:05.383 INFO:tasks.workunit.client.1.smithi033.stdout:5/196: creat d3/d12/d25/d26/d2e/f3e x:0 0 0 2024-05-12T04:01:05.390 INFO:tasks.workunit.client.0.smithi032.stdout:2/79: creat d1/d13/f20 x:0 0 0 2024-05-12T04:01:05.391 INFO:tasks.workunit.client.1.smithi033.stdout:8/181: read d1/f13 [198165,84153] 0 2024-05-12T04:01:05.392 INFO:tasks.workunit.client.0.smithi032.stdout:6/55: mknod d7/d9/cb 0 2024-05-12T04:01:05.392 INFO:tasks.workunit.client.0.smithi032.stdout:6/56: stat d7 0 2024-05-12T04:01:05.392 INFO:tasks.workunit.client.0.smithi032.stdout:6/57: fdatasync f3 0 2024-05-12T04:01:05.393 INFO:tasks.workunit.client.0.smithi032.stdout:6/58: fsync d7/fa 0 2024-05-12T04:01:05.395 INFO:tasks.workunit.client.0.smithi032.stdout:5/94: chown d4/c9 14 1 2024-05-12T04:01:05.398 INFO:tasks.workunit.client.1.smithi033.stdout:4/269: rename dd/d11/d15/c20 to dd/d11/d31/d38/c4d 0 2024-05-12T04:01:05.398 INFO:tasks.workunit.client.1.smithi033.stdout:8/182: dwrite d1/d16/d20/f33 [0,4194304] 0 2024-05-12T04:01:05.398 INFO:tasks.workunit.client.1.smithi033.stdout:4/270: readlink dd/d11/l3f 0 2024-05-12T04:01:05.400 INFO:tasks.workunit.client.1.smithi033.stdout:4/271: write dd/d11/f46 [918351,49748] 0 2024-05-12T04:01:05.403 INFO:tasks.workunit.client.1.smithi033.stdout:7/417: creat de/d20/d38/f9e x:0 0 0 2024-05-12T04:01:05.404 INFO:tasks.workunit.client.1.smithi033.stdout:7/418: fsync de/df/d60/d75/f8f 0 2024-05-12T04:01:05.405 INFO:tasks.workunit.client.1.smithi033.stdout:7/419: dread - de/df/d17/d1e/f23 zero size 2024-05-12T04:01:05.405 INFO:tasks.workunit.client.1.smithi033.stdout:7/420: dread - de/df/d60/f76 zero size 2024-05-12T04:01:05.408 INFO:tasks.workunit.client.1.smithi033.stdout:2/252: chown l4 296 1 2024-05-12T04:01:05.409 INFO:tasks.workunit.client.1.smithi033.stdout:2/253: write dc/d17/d32/f43 [541886,44170] 0 2024-05-12T04:01:05.413 INFO:tasks.workunit.client.1.smithi033.stdout:0/305: symlink d6/dc/d58/l65 0 2024-05-12T04:01:05.414 INFO:tasks.workunit.client.1.smithi033.stdout:0/306: write d6/f26 [4163151,126773] 0 2024-05-12T04:01:05.415 INFO:tasks.workunit.client.1.smithi033.stdout:6/208: symlink d4/l44 0 2024-05-12T04:01:05.416 INFO:tasks.workunit.client.1.smithi033.stdout:6/209: stat d4/d5/f35 0 2024-05-12T04:01:05.418 INFO:tasks.workunit.client.1.smithi033.stdout:5/197: creat d3/d12/f3f x:0 0 0 2024-05-12T04:01:05.419 INFO:tasks.workunit.client.1.smithi033.stdout:5/198: write d3/d12/f1f [3669034,58375] 0 2024-05-12T04:01:05.425 INFO:tasks.workunit.client.1.smithi033.stdout:4/272: rmdir dd/d11/d15 39 2024-05-12T04:01:05.426 INFO:tasks.workunit.client.1.smithi033.stdout:7/421: fdatasync de/f12 0 2024-05-12T04:01:05.428 INFO:tasks.workunit.client.1.smithi033.stdout:2/254: mkdir dc/d17/d27/d2f/d49/d59 0 2024-05-12T04:01:05.428 INFO:tasks.workunit.client.1.smithi033.stdout:7/422: write de/d20/f96 [275738,125685] 0 2024-05-12T04:01:05.429 INFO:tasks.workunit.client.1.smithi033.stdout:7/423: readlink de/df/l62 0 2024-05-12T04:01:05.431 INFO:tasks.workunit.client.1.smithi033.stdout:0/307: mkdir d6/dc/d66 0 2024-05-12T04:01:05.432 INFO:tasks.workunit.client.1.smithi033.stdout:3/242: dwrite db/d17/d25/d32/d3c/f56 [0,4194304] 0 2024-05-12T04:01:05.432 INFO:tasks.workunit.client.1.smithi033.stdout:6/210: mknod d4/d5/c45 0 2024-05-12T04:01:05.435 INFO:tasks.workunit.client.1.smithi033.stdout:9/442: link d2/d4/d2b/c6e d2/d4/d2b/ca3 0 2024-05-12T04:01:05.435 INFO:tasks.workunit.client.1.smithi033.stdout:9/443: stat d2/d4/c22 0 2024-05-12T04:01:05.436 INFO:tasks.workunit.client.1.smithi033.stdout:8/183: rename d1/d16/d29/c35 to d1/d4/d1d/d36/c3a 0 2024-05-12T04:01:05.439 INFO:tasks.workunit.client.1.smithi033.stdout:9/444: chown d2/d4/d2b/d33/l4a 1948915 1 2024-05-12T04:01:05.440 INFO:tasks.workunit.client.1.smithi033.stdout:1/195: getdents d11/d17 0 2024-05-12T04:01:05.441 INFO:tasks.workunit.client.1.smithi033.stdout:0/308: dwrite f4 [0,4194304] 0 2024-05-12T04:01:05.441 INFO:tasks.workunit.client.1.smithi033.stdout:8/184: write d1/d16/d20/f33 [2787445,131023] 0 2024-05-12T04:01:05.442 INFO:tasks.workunit.client.1.smithi033.stdout:6/211: rename d4/l6 to d4/d43/l46 0 2024-05-12T04:01:05.443 INFO:tasks.workunit.client.1.smithi033.stdout:0/309: chown d6/dc/d58 20 1 2024-05-12T04:01:05.444 INFO:tasks.workunit.client.1.smithi033.stdout:5/199: creat d3/f40 x:0 0 0 2024-05-12T04:01:05.445 INFO:tasks.workunit.client.1.smithi033.stdout:5/200: readlink d3/d12/d29/l37 0 2024-05-12T04:01:05.447 INFO:tasks.workunit.client.1.smithi033.stdout:9/445: write d2/d4/d29/f2a [861105,129087] 0 2024-05-12T04:01:05.448 INFO:tasks.workunit.client.1.smithi033.stdout:1/196: creat d11/d17/d27/d36/f41 x:0 0 0 2024-05-12T04:01:05.449 INFO:tasks.workunit.client.1.smithi033.stdout:4/273: rename dd/d11/d15/f1e to dd/d11/d15/d42/f4e 0 2024-05-12T04:01:05.451 INFO:tasks.workunit.client.1.smithi033.stdout:3/243: fsync db/f20 0 2024-05-12T04:01:05.451 INFO:tasks.workunit.client.1.smithi033.stdout:6/212: creat d4/d1a/f47 x:0 0 0 2024-05-12T04:01:05.451 INFO:tasks.workunit.client.1.smithi033.stdout:5/201: mkdir d3/d15/d16/d41 0 2024-05-12T04:01:05.453 INFO:tasks.workunit.client.1.smithi033.stdout:8/185: symlink d1/d16/d29/l3b 0 2024-05-12T04:01:05.456 INFO:tasks.workunit.client.1.smithi033.stdout:8/186: readlink d1/d16/d20/l38 0 2024-05-12T04:01:05.456 INFO:tasks.workunit.client.1.smithi033.stdout:0/310: dwrite d6/f26 [0,4194304] 0 2024-05-12T04:01:05.457 INFO:tasks.workunit.client.1.smithi033.stdout:3/244: chown db/d17/c19 0 1 2024-05-12T04:01:05.458 INFO:tasks.workunit.client.1.smithi033.stdout:5/202: dread d3/d15/f30 [0,4194304] 0 2024-05-12T04:01:05.461 INFO:tasks.workunit.client.1.smithi033.stdout:3/245: fsync db/d17/d25/d32/f46 0 2024-05-12T04:01:05.462 INFO:tasks.workunit.client.0.smithi032.stdout:1/56: mknod d1/c15 0 2024-05-12T04:01:05.464 INFO:tasks.workunit.client.0.smithi032.stdout:8/76: unlink d0/c6 0 2024-05-12T04:01:05.465 INFO:tasks.workunit.client.1.smithi033.stdout:9/446: dwrite d2/d4/d2b/d6d/d4c/f95 [0,4194304] 0 2024-05-12T04:01:05.465 INFO:tasks.workunit.client.1.smithi033.stdout:3/246: chown db/d17/d25/c28 172309 1 2024-05-12T04:01:05.466 INFO:tasks.workunit.client.0.smithi032.stdout:2/80: fsync d1/d13/f1b 0 2024-05-12T04:01:05.466 INFO:tasks.workunit.client.1.smithi033.stdout:8/187: rename d1/d16/d27/d2a to d1/d4/d1d/d36/d3c 0 2024-05-12T04:01:05.467 INFO:tasks.workunit.client.0.smithi032.stdout:4/42: creat d0/d3/f8 x:0 0 0 2024-05-12T04:01:05.467 INFO:tasks.workunit.client.1.smithi033.stdout:6/213: link d4/d43/l46 d4/d1a/l48 0 2024-05-12T04:01:05.468 INFO:tasks.workunit.client.0.smithi032.stdout:6/59: creat d7/fc x:0 0 0 2024-05-12T04:01:05.468 INFO:tasks.workunit.client.0.smithi032.stdout:8/77: dread d0/d7/ff [0,4194304] 0 2024-05-12T04:01:05.468 INFO:tasks.workunit.client.1.smithi033.stdout:1/197: dwrite f8 [4194304,4194304] 0 2024-05-12T04:01:05.469 INFO:tasks.workunit.client.0.smithi032.stdout:2/81: dread - d1/f1e zero size 2024-05-12T04:01:05.470 INFO:tasks.workunit.client.0.smithi032.stdout:3/100: mkdir d15/d20/d22 0 2024-05-12T04:01:05.472 INFO:tasks.workunit.client.0.smithi032.stdout:0/138: symlink df/d28/l31 0 2024-05-12T04:01:05.472 INFO:tasks.workunit.client.0.smithi032.stdout:1/57: mkdir d1/d16 0 2024-05-12T04:01:05.473 INFO:tasks.workunit.client.0.smithi032.stdout:2/82: truncate d1/f1e 743644 0 2024-05-12T04:01:05.473 INFO:tasks.workunit.client.0.smithi032.stdout:3/101: fsync d15/d1b/f21 0 2024-05-12T04:01:05.475 INFO:tasks.workunit.client.0.smithi032.stdout:3/102: dread - d15/f1c zero size 2024-05-12T04:01:05.476 INFO:tasks.workunit.client.0.smithi032.stdout:0/139: write df/d22/f2f [1605548,120768] 0 2024-05-12T04:01:05.480 INFO:tasks.workunit.client.0.smithi032.stdout:6/60: mknod d7/cd 0 2024-05-12T04:01:05.483 INFO:tasks.workunit.client.1.smithi033.stdout:8/188: rmdir d1/d4/d1d/d36 39 2024-05-12T04:01:05.483 INFO:tasks.workunit.client.1.smithi033.stdout:3/247: mknod db/d17/d3e/c57 0 2024-05-12T04:01:05.484 INFO:tasks.workunit.client.0.smithi032.stdout:5/95: dwrite d4/db/ff [4194304,4194304] 0 2024-05-12T04:01:05.485 INFO:tasks.workunit.client.1.smithi033.stdout:9/447: symlink d2/dd/d39/la4 0 2024-05-12T04:01:05.485 INFO:tasks.workunit.client.1.smithi033.stdout:5/203: dwrite d3/fb [4194304,4194304] 0 2024-05-12T04:01:05.485 INFO:tasks.workunit.client.1.smithi033.stdout:4/274: truncate dd/d11/f21 2530697 0 2024-05-12T04:01:05.485 INFO:tasks.workunit.client.1.smithi033.stdout:6/214: write d4/d1a/f42 [832942,106383] 0 2024-05-12T04:01:05.489 INFO:tasks.workunit.client.1.smithi033.stdout:1/198: unlink d11/d17/d27/d36/c40 0 2024-05-12T04:01:05.492 INFO:tasks.workunit.client.0.smithi032.stdout:1/58: dwrite d1/f6 [4194304,4194304] 0 2024-05-12T04:01:05.492 INFO:tasks.workunit.client.0.smithi032.stdout:4/43: write d0/d3/f4 [1273412,85888] 0 2024-05-12T04:01:05.493 INFO:tasks.workunit.client.1.smithi033.stdout:8/189: dwrite d1/d16/d20/f2f [0,4194304] 0 2024-05-12T04:01:05.494 INFO:tasks.workunit.client.0.smithi032.stdout:4/44: rename d0/d3 to d0/d3/d9 22 2024-05-12T04:01:05.496 INFO:tasks.workunit.client.0.smithi032.stdout:3/103: rmdir d15 39 2024-05-12T04:01:05.502 INFO:tasks.workunit.client.1.smithi033.stdout:9/448: unlink d2/dd/c74 0 2024-05-12T04:01:05.505 INFO:tasks.workunit.client.0.smithi032.stdout:8/78: symlink d0/d9/d13/l18 0 2024-05-12T04:01:05.510 INFO:tasks.workunit.client.0.smithi032.stdout:6/61: mkdir d7/de 0 2024-05-12T04:01:05.511 INFO:tasks.workunit.client.1.smithi033.stdout:6/215: unlink d4/d1a/l3f 0 2024-05-12T04:01:05.512 INFO:tasks.workunit.client.0.smithi032.stdout:6/62: write d7/fc [110823,34271] 0 2024-05-12T04:01:05.512 INFO:tasks.workunit.client.1.smithi033.stdout:5/204: creat d3/d15/d16/f42 x:0 0 0 2024-05-12T04:01:05.513 INFO:tasks.workunit.client.1.smithi033.stdout:5/205: readlink d3/d12/d29/l37 0 2024-05-12T04:01:05.515 INFO:tasks.workunit.client.1.smithi033.stdout:5/206: truncate d3/d12/d25/d26/d2e/f3e 12 0 2024-05-12T04:01:05.515 INFO:tasks.workunit.client.1.smithi033.stdout:1/199: mkdir d11/d1c/d42 0 2024-05-12T04:01:05.515 INFO:tasks.workunit.client.1.smithi033.stdout:5/207: chown d3/f1c 26000 1 2024-05-12T04:01:05.517 INFO:tasks.workunit.client.1.smithi033.stdout:6/216: dread d4/d1a/f42 [0,4194304] 0 2024-05-12T04:01:05.517 INFO:tasks.workunit.client.0.smithi032.stdout:5/96: dread - d4/db/f13 zero size 2024-05-12T04:01:05.519 INFO:tasks.workunit.client.1.smithi033.stdout:1/200: dread d11/d17/d27/d36/f3c [0,4194304] 0 2024-05-12T04:01:05.520 INFO:tasks.workunit.client.0.smithi032.stdout:1/59: mknod d1/c17 0 2024-05-12T04:01:05.521 INFO:tasks.workunit.client.1.smithi033.stdout:1/201: chown d11/d17/d38/l3b 76 1 2024-05-12T04:01:05.521 INFO:tasks.workunit.client.0.smithi032.stdout:4/45: rmdir d0/d3 39 2024-05-12T04:01:05.521 INFO:tasks.workunit.client.0.smithi032.stdout:1/60: write d1/f13 [310450,115257] 0 2024-05-12T04:01:05.522 INFO:tasks.workunit.client.1.smithi033.stdout:1/202: chown d11/d17/f1d 59 1 2024-05-12T04:01:05.523 INFO:tasks.workunit.client.1.smithi033.stdout:8/190: mkdir d1/d16/d29/d3d 0 2024-05-12T04:01:05.524 INFO:tasks.workunit.client.1.smithi033.stdout:1/203: write d11/f19 [522991,109163] 0 2024-05-12T04:01:05.524 INFO:tasks.workunit.client.1.smithi033.stdout:6/217: dwrite d4/d1a/f42 [0,4194304] 0 2024-05-12T04:01:05.525 INFO:tasks.workunit.client.1.smithi033.stdout:8/191: fdatasync d1/d16/d29/f30 0 2024-05-12T04:01:05.526 INFO:tasks.workunit.client.1.smithi033.stdout:6/218: fdatasync d4/d1a/f47 0 2024-05-12T04:01:05.527 INFO:tasks.workunit.client.1.smithi033.stdout:9/449: symlink d2/dd/d39/d72/la5 0 2024-05-12T04:01:05.528 INFO:tasks.workunit.client.0.smithi032.stdout:8/79: symlink d0/d9/d13/l19 0 2024-05-12T04:01:05.529 INFO:tasks.workunit.client.0.smithi032.stdout:3/104: dread d15/f1a [0,4194304] 0 2024-05-12T04:01:05.531 INFO:tasks.workunit.client.0.smithi032.stdout:0/140: mkdir df/d30/d32 0 2024-05-12T04:01:05.531 INFO:tasks.workunit.client.0.smithi032.stdout:0/141: chown df/d28 469 1 2024-05-12T04:01:05.533 INFO:tasks.workunit.client.1.smithi033.stdout:1/204: dwrite d11/d17/d27/f31 [0,4194304] 0 2024-05-12T04:01:05.533 INFO:tasks.workunit.client.1.smithi033.stdout:1/205: fdatasync d11/f30 0 2024-05-12T04:01:05.534 INFO:tasks.workunit.client.0.smithi032.stdout:6/63: readlink l1 0 2024-05-12T04:01:05.534 INFO:tasks.workunit.client.0.smithi032.stdout:0/142: chown df/d10/d1f/l23 150001312 1 2024-05-12T04:01:05.535 INFO:tasks.workunit.client.1.smithi033.stdout:1/206: dread - d11/d17/d27/d36/f41 zero size 2024-05-12T04:01:05.536 INFO:tasks.workunit.client.0.smithi032.stdout:0/143: chown df/d10/l15 13088221 1 2024-05-12T04:01:05.536 INFO:tasks.workunit.client.0.smithi032.stdout:6/64: dread d7/fa [0,4194304] 0 2024-05-12T04:01:05.537 INFO:tasks.workunit.client.0.smithi032.stdout:2/83: creat d1/f21 x:0 0 0 2024-05-12T04:01:05.537 INFO:tasks.workunit.client.1.smithi033.stdout:5/208: creat d3/d12/f43 x:0 0 0 2024-05-12T04:01:05.537 INFO:tasks.workunit.client.0.smithi032.stdout:6/65: readlink d7/l8 0 2024-05-12T04:01:05.541 INFO:tasks.workunit.client.1.smithi033.stdout:4/275: creat dd/f4f x:0 0 0 2024-05-12T04:01:05.543 INFO:tasks.workunit.client.0.smithi032.stdout:4/46: write d0/d3/f8 [351556,16247] 0 2024-05-12T04:01:05.547 INFO:tasks.workunit.client.0.smithi032.stdout:0/144: fsync df/d10/f1b 0 2024-05-12T04:01:05.549 INFO:tasks.workunit.client.1.smithi033.stdout:6/219: symlink d4/d1a/d3c/l49 0 2024-05-12T04:01:05.550 INFO:tasks.workunit.client.1.smithi033.stdout:9/450: dread d2/d4/f2c [0,4194304] 0 2024-05-12T04:01:05.556 INFO:tasks.workunit.client.0.smithi032.stdout:0/145: dwrite f9 [4194304,4194304] 0 2024-05-12T04:01:05.557 INFO:tasks.workunit.client.0.smithi032.stdout:4/47: unlink d0/l7 0 2024-05-12T04:01:05.558 INFO:tasks.workunit.client.0.smithi032.stdout:4/48: fsync d0/d3/f8 0 2024-05-12T04:01:05.558 INFO:tasks.workunit.client.1.smithi033.stdout:1/207: mkdir d11/d1c/d43 0 2024-05-12T04:01:05.559 INFO:tasks.workunit.client.0.smithi032.stdout:3/105: getdents d15/d20/d22 0 2024-05-12T04:01:05.560 INFO:tasks.workunit.client.1.smithi033.stdout:1/208: truncate d11/d17/d27/d36/f41 407607 0 2024-05-12T04:01:05.560 INFO:tasks.workunit.client.0.smithi032.stdout:4/49: write d0/d3/f8 [240081,121810] 0 2024-05-12T04:01:05.561 INFO:tasks.workunit.client.0.smithi032.stdout:4/50: readlink d0/l5 0 2024-05-12T04:01:05.569 INFO:tasks.workunit.client.0.smithi032.stdout:4/51: dwrite d0/d3/f4 [0,4194304] 0 2024-05-12T04:01:05.574 INFO:tasks.workunit.client.1.smithi033.stdout:3/248: getdents db/d17/d25/d32/d3c 0 2024-05-12T04:01:05.575 INFO:tasks.workunit.client.0.smithi032.stdout:5/97: truncate d4/db/ff 7119105 0 2024-05-12T04:01:05.575 INFO:tasks.workunit.client.1.smithi033.stdout:3/249: write f6 [4493331,53045] 0 2024-05-12T04:01:05.576 INFO:tasks.workunit.client.1.smithi033.stdout:8/192: dwrite d1/d4/d1d/f26 [0,4194304] 0 2024-05-12T04:01:05.578 INFO:tasks.workunit.client.0.smithi032.stdout:0/146: creat df/d10/f33 x:0 0 0 2024-05-12T04:01:05.580 INFO:tasks.workunit.client.1.smithi033.stdout:1/209: creat d11/d17/d27/f44 x:0 0 0 2024-05-12T04:01:05.580 INFO:tasks.workunit.client.0.smithi032.stdout:8/80: link d0/d7/c16 d0/d7/c1a 0 2024-05-12T04:01:05.581 INFO:tasks.workunit.client.0.smithi032.stdout:8/81: readlink d0/d9/d13/l18 0 2024-05-12T04:01:05.581 INFO:tasks.workunit.client.1.smithi033.stdout:1/210: stat d11/c12 0 2024-05-12T04:01:05.584 INFO:tasks.workunit.client.0.smithi032.stdout:8/82: write d0/d9/fa [794764,5825] 0 2024-05-12T04:01:05.585 INFO:tasks.workunit.client.0.smithi032.stdout:6/66: link l1 d7/lf 0 2024-05-12T04:01:05.586 INFO:tasks.workunit.client.1.smithi033.stdout:9/451: dwrite d2/fa [0,4194304] 0 2024-05-12T04:01:05.586 INFO:tasks.workunit.client.1.smithi033.stdout:3/250: dwrite db/d17/d25/f40 [0,4194304] 0 2024-05-12T04:01:05.589 INFO:tasks.workunit.client.1.smithi033.stdout:8/193: mkdir d1/d4/d2e/d3e 0 2024-05-12T04:01:05.590 INFO:tasks.workunit.client.1.smithi033.stdout:4/276: link dd/d11/d31/l4a dd/d11/d31/l50 0 2024-05-12T04:01:05.598 INFO:tasks.workunit.client.1.smithi033.stdout:3/251: dwrite db/d17/d25/d32/d3c/d44/f55 [0,4194304] 0 2024-05-12T04:01:05.598 INFO:tasks.workunit.client.1.smithi033.stdout:9/452: symlink d2/dd/d39/la6 0 2024-05-12T04:01:05.599 INFO:tasks.workunit.client.0.smithi032.stdout:8/83: fdatasync d0/f5 0 2024-05-12T04:01:05.600 INFO:tasks.workunit.client.1.smithi033.stdout:3/252: write db/d17/d25/d32/f46 [1464330,61251] 0 2024-05-12T04:01:05.602 INFO:tasks.workunit.client.0.smithi032.stdout:6/67: symlink d7/l10 0 2024-05-12T04:01:05.602 INFO:tasks.workunit.client.1.smithi033.stdout:1/211: creat d11/d1c/d43/f45 x:0 0 0 2024-05-12T04:01:05.608 INFO:tasks.workunit.client.1.smithi033.stdout:4/277: dread dd/d11/f21 [0,4194304] 0 2024-05-12T04:01:05.608 INFO:tasks.workunit.client.0.smithi032.stdout:0/147: creat df/d30/d32/f34 x:0 0 0 2024-05-12T04:01:05.611 INFO:tasks.workunit.client.0.smithi032.stdout:4/52: creat d0/fa x:0 0 0 2024-05-12T04:01:05.616 INFO:tasks.workunit.client.1.smithi033.stdout:6/220: getdents d4 0 2024-05-12T04:01:05.616 INFO:tasks.workunit.client.0.smithi032.stdout:8/84: mkdir d0/d9/d1b 0 2024-05-12T04:01:05.617 INFO:tasks.workunit.client.0.smithi032.stdout:8/85: write d0/d9/f12 [863537,79219] 0 2024-05-12T04:01:05.618 INFO:tasks.workunit.client.1.smithi033.stdout:6/221: truncate d4/d5/f2d 330484 0 2024-05-12T04:01:05.618 INFO:tasks.workunit.client.1.smithi033.stdout:9/453: rmdir d2/d4/d2b/d33 39 2024-05-12T04:01:05.618 INFO:tasks.workunit.client.1.smithi033.stdout:6/222: chown d4/d43 7118424 1 2024-05-12T04:01:05.619 INFO:tasks.workunit.client.0.smithi032.stdout:6/68: symlink d7/d9/l11 0 2024-05-12T04:01:05.620 INFO:tasks.workunit.client.0.smithi032.stdout:3/106: truncate f3 1205011 0 2024-05-12T04:01:05.621 INFO:tasks.workunit.client.0.smithi032.stdout:6/69: dread d7/fc [0,4194304] 0 2024-05-12T04:01:05.624 INFO:tasks.workunit.client.1.smithi033.stdout:9/454: dwrite d2/d70/d7b/f8a [0,4194304] 0 2024-05-12T04:01:05.625 INFO:tasks.workunit.client.1.smithi033.stdout:3/253: creat db/d17/d25/f58 x:0 0 0 2024-05-12T04:01:05.626 INFO:tasks.workunit.client.1.smithi033.stdout:1/212: rmdir d11/d17 39 2024-05-12T04:01:05.627 INFO:tasks.workunit.client.0.smithi032.stdout:8/86: rmdir d0/d7 39 2024-05-12T04:01:05.631 INFO:tasks.workunit.client.1.smithi033.stdout:4/278: mknod dd/d11/d31/c51 0 2024-05-12T04:01:05.632 INFO:tasks.workunit.client.0.smithi032.stdout:3/107: rmdir d15/d1b 39 2024-05-12T04:01:05.633 INFO:tasks.workunit.client.1.smithi033.stdout:2/255: sync 2024-05-12T04:01:05.634 INFO:tasks.workunit.client.1.smithi033.stdout:6/223: creat d4/d1a/f4a x:0 0 0 2024-05-12T04:01:05.634 INFO:tasks.workunit.client.0.smithi032.stdout:5/98: dwrite d4/db/ff [4194304,4194304] 0 2024-05-12T04:01:05.636 INFO:tasks.workunit.client.1.smithi033.stdout:2/256: dread - dc/d17/d27/d2f/d49/f51 zero size 2024-05-12T04:01:05.638 INFO:tasks.workunit.client.1.smithi033.stdout:4/279: dwrite dd/d11/f14 [0,4194304] 0 2024-05-12T04:01:05.640 INFO:tasks.workunit.client.0.smithi032.stdout:0/148: truncate f5 1223706 0 2024-05-12T04:01:05.643 INFO:tasks.workunit.client.0.smithi032.stdout:8/87: creat d0/d9/f1c x:0 0 0 2024-05-12T04:01:05.644 INFO:tasks.workunit.client.1.smithi033.stdout:2/257: dwrite dc/d17/f4d [0,4194304] 0 2024-05-12T04:01:05.645 INFO:tasks.workunit.client.1.smithi033.stdout:2/258: read dc/d17/f4d [1143772,28651] 0 2024-05-12T04:01:05.647 INFO:tasks.workunit.client.1.smithi033.stdout:2/259: dread - dc/d17/f23 zero size 2024-05-12T04:01:05.649 INFO:tasks.workunit.client.0.smithi032.stdout:6/70: mkdir d7/de/d12 0 2024-05-12T04:01:05.654 INFO:tasks.workunit.client.0.smithi032.stdout:0/149: symlink df/d30/d32/l35 0 2024-05-12T04:01:05.655 INFO:tasks.workunit.client.1.smithi033.stdout:4/280: rmdir dd/d11/d15/d42 39 2024-05-12T04:01:05.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.656+0000 7f6398e4d700 1 -- 172.21.15.32:0/2794001534 --> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f639410d3c0 con 0x7f637c06aec0 2024-05-12T04:01:05.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.657+0000 7f6382ffd700 1 -- 172.21.15.32:0/2794001534 <== mgr.24389 v2:172.21.15.33:6828/1598726962 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7f639410d3c0 con 0x7f637c06aec0 2024-05-12T04:01:05.659 INFO:tasks.workunit.client.0.smithi032.stdout:3/108: creat d15/d20/d22/f23 x:0 0 0 2024-05-12T04:01:05.660 INFO:tasks.workunit.client.0.smithi032.stdout:8/88: dread d0/d7/f11 [0,4194304] 0 2024-05-12T04:01:05.660 INFO:tasks.workunit.client.1.smithi033.stdout:9/455: mknod d2/dd/ca7 0 2024-05-12T04:01:05.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.661+0000 7f6398e4d700 1 -- 172.21.15.32:0/2794001534 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f637c06aec0 msgr2=0x7f637c06d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:01:05.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.661+0000 7f6398e4d700 1 --2- 172.21.15.32:0/2794001534 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f637c06aec0 0x7f637c06d370 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6388009fd0 tx=0x7f6388009e30 comp rx=0 tx=0).stop 2024-05-12T04:01:05.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.661+0000 7f6398e4d700 1 -- 172.21.15.32:0/2794001534 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63940fbce0 msgr2=0x7f639410b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:01:05.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.661+0000 7f6398e4d700 1 --2- 172.21.15.32:0/2794001534 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63940fbce0 0x7f639410b680 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f638400b0d0 tx=0x7f6384009e30 comp rx=0 tx=0).stop 2024-05-12T04:01:05.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.661+0000 7f6398e4d700 1 -- 172.21.15.32:0/2794001534 shutdown_connections 2024-05-12T04:01:05.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.661+0000 7f6398e4d700 1 --2- 172.21.15.32:0/2794001534 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f637c06aec0 0x7f637c06d370 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:05.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.661+0000 7f6398e4d700 1 --2- 172.21.15.32:0/2794001534 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63940fbce0 0x7f639410b680 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:05.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.661+0000 7f6398e4d700 1 --2- 172.21.15.32:0/2794001534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63940fcf90 0x7f639410bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:05.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.661+0000 7f6398e4d700 1 -- 172.21.15.32:0/2794001534 >> 172.21.15.32:0/2794001534 conn(0x7f63940f7440 msgr2=0x7f63940f8c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:01:05.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.661+0000 7f6398e4d700 1 -- 172.21.15.32:0/2794001534 shutdown_connections 2024-05-12T04:01:05.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.661+0000 7f6398e4d700 1 -- 172.21.15.32:0/2794001534 wait complete. 2024-05-12T04:01:05.662 INFO:tasks.workunit.client.1.smithi033.stdout:3/254: mkdir db/d59 0 2024-05-12T04:01:05.662 INFO:tasks.workunit.client.0.smithi032.stdout:5/99: mkdir d4/d7/d21 0 2024-05-12T04:01:05.664 INFO:tasks.workunit.client.0.smithi032.stdout:5/100: readlink d4/db/l18 0 2024-05-12T04:01:05.668 INFO:tasks.workunit.client.1.smithi033.stdout:8/194: getdents d1/d4/d1d 0 2024-05-12T04:01:05.668 INFO:tasks.workunit.client.1.smithi033.stdout:9/456: dwrite d2/d4/d2b/d6d/d7a/f90 [0,4194304] 0 2024-05-12T04:01:05.669 INFO:tasks.workunit.client.0.smithi032.stdout:0/150: mkdir df/d30/d36 0 2024-05-12T04:01:05.670 INFO:tasks.workunit.client.1.smithi033.stdout:3/255: dwrite db/d17/f2c [0,4194304] 0 2024-05-12T04:01:05.672 INFO:tasks.workunit.client.1.smithi033.stdout:1/213: rename d11/d1c/d42 to d11/d17/d27/d36/d46 0 2024-05-12T04:01:05.675 INFO:tasks.workunit.client.1.smithi033.stdout:2/260: mknod dc/d17/d3a/c5a 0 2024-05-12T04:01:05.684 INFO:tasks.workunit.client.0.smithi032.stdout:6/71: truncate f3 1910598 0 2024-05-12T04:01:05.685 INFO:tasks.workunit.client.1.smithi033.stdout:4/281: dread - dd/d11/d15/d42/f4c zero size 2024-05-12T04:01:05.685 INFO:tasks.workunit.client.1.smithi033.stdout:4/282: fsync dd/f4b 0 2024-05-12T04:01:05.690 INFO:tasks.workunit.client.0.smithi032.stdout:3/109: mkdir d15/d24 0 2024-05-12T04:01:05.693 INFO:tasks.workunit.client.0.smithi032.stdout:6/72: symlink d7/l13 0 2024-05-12T04:01:05.694 INFO:tasks.workunit.client.1.smithi033.stdout:6/224: truncate d4/f1f 1427183 0 2024-05-12T04:01:05.698 INFO:tasks.workunit.client.1.smithi033.stdout:9/457: mkdir d2/d4/d2b/d49/d73/da8 0 2024-05-12T04:01:05.698 INFO:tasks.workunit.client.1.smithi033.stdout:9/458: read d2/d70/d7b/f8a [1413888,16466] 0 2024-05-12T04:01:05.698 INFO:tasks.workunit.client.1.smithi033.stdout:3/256: creat db/d17/d25/d32/d3c/f5a x:0 0 0 2024-05-12T04:01:05.698 INFO:tasks.workunit.client.1.smithi033.stdout:3/257: chown db/d42 18811 1 2024-05-12T04:01:05.699 INFO:tasks.workunit.client.1.smithi033.stdout:3/258: stat db/d17/d25/d32/f46 0 2024-05-12T04:01:05.699 INFO:tasks.workunit.client.1.smithi033.stdout:1/214: mknod d11/d17/d38/c47 0 2024-05-12T04:01:05.700 INFO:tasks.workunit.client.1.smithi033.stdout:1/215: chown d11/l26 7 1 2024-05-12T04:01:05.700 INFO:tasks.workunit.client.1.smithi033.stdout:1/216: dread - d11/d1c/f32 zero size 2024-05-12T04:01:05.702 INFO:tasks.workunit.client.0.smithi032.stdout:6/73: creat d7/de/f14 x:0 0 0 2024-05-12T04:01:05.703 INFO:tasks.workunit.client.0.smithi032.stdout:6/74: write d7/fa [75906,127175] 0 2024-05-12T04:01:05.705 INFO:tasks.workunit.client.0.smithi032.stdout:8/89: rename d0/d9/fa to d0/f1d 0 2024-05-12T04:01:05.706 INFO:tasks.workunit.client.1.smithi033.stdout:4/283: symlink dd/d11/d31/d38/l52 0 2024-05-12T04:01:05.710 INFO:tasks.workunit.client.1.smithi033.stdout:6/225: rename d4/d5/lb to d4/d1a/d3c/l4b 0 2024-05-12T04:01:05.710 INFO:tasks.workunit.client.1.smithi033.stdout:6/226: chown d4/d43 783802550 1 2024-05-12T04:01:05.711 INFO:tasks.workunit.client.0.smithi032.stdout:6/75: dwrite f6 [0,4194304] 0 2024-05-12T04:01:05.712 INFO:tasks.workunit.client.0.smithi032.stdout:3/110: creat d15/d1b/f25 x:0 0 0 2024-05-12T04:01:05.714 INFO:tasks.workunit.client.0.smithi032.stdout:3/111: dread - d15/d20/d22/f23 zero size 2024-05-12T04:01:05.716 INFO:tasks.workunit.client.1.smithi033.stdout:9/459: mknod d2/dd/d79/ca9 0 2024-05-12T04:01:05.726 INFO:tasks.workunit.client.0.smithi032.stdout:0/151: getdents df/d30/d32 0 2024-05-12T04:01:05.727 INFO:tasks.workunit.client.0.smithi032.stdout:0/152: write fb [960861,103369] 0 2024-05-12T04:01:05.729 INFO:tasks.workunit.client.1.smithi033.stdout:7/424: sync 2024-05-12T04:01:05.730 INFO:tasks.workunit.client.0.smithi032.stdout:8/90: truncate d0/d7/f11 818505 0 2024-05-12T04:01:05.736 INFO:tasks.workunit.client.0.smithi032.stdout:6/76: creat d7/d9/f15 x:0 0 0 2024-05-12T04:01:05.737 INFO:tasks.workunit.client.1.smithi033.stdout:6/227: unlink d4/l36 0 2024-05-12T04:01:05.739 INFO:tasks.workunit.client.0.smithi032.stdout:3/112: mkdir d15/d20/d26 0 2024-05-12T04:01:05.739 INFO:tasks.workunit.client.0.smithi032.stdout:6/77: dread - d7/de/f14 zero size 2024-05-12T04:01:05.741 INFO:tasks.workunit.client.0.smithi032.stdout:6/78: write d7/de/f14 [478388,99961] 0 2024-05-12T04:01:05.742 INFO:tasks.workunit.client.0.smithi032.stdout:6/79: dread d7/fa [0,4194304] 0 2024-05-12T04:01:05.743 INFO:tasks.workunit.client.1.smithi033.stdout:9/460: rename d2/dd/d79/la2 to d2/dd/d39/laa 0 2024-05-12T04:01:05.747 INFO:tasks.workunit.client.0.smithi032.stdout:0/153: symlink df/d10/d1f/l37 0 2024-05-12T04:01:05.749 INFO:tasks.workunit.client.1.smithi033.stdout:3/259: truncate db/d17/f36 1239502 0 2024-05-12T04:01:05.750 INFO:tasks.workunit.client.1.smithi033.stdout:1/217: dwrite fd [0,4194304] 0 2024-05-12T04:01:05.752 INFO:tasks.workunit.client.1.smithi033.stdout:2/261: dwrite dc/d13/d1c/f25 [0,4194304] 0 2024-05-12T04:01:05.753 INFO:tasks.workunit.client.0.smithi032.stdout:3/113: rmdir d15/d1b 39 2024-05-12T04:01:05.755 INFO:tasks.workunit.client.1.smithi033.stdout:2/262: write dc/d13/f1d [1880479,16138] 0 2024-05-12T04:01:05.757 INFO:tasks.workunit.client.0.smithi032.stdout:5/101: rename d4/db/ff to d4/f22 0 2024-05-12T04:01:05.759 INFO:tasks.workunit.client.1.smithi033.stdout:4/284: mknod dd/d11/c53 0 2024-05-12T04:01:05.760 INFO:tasks.workunit.client.1.smithi033.stdout:2/263: read dc/d13/f15 [756144,15418] 0 2024-05-12T04:01:05.761 INFO:tasks.workunit.client.1.smithi033.stdout:2/264: fsync f6 0 2024-05-12T04:01:05.761 INFO:tasks.workunit.client.1.smithi033.stdout:1/218: dwrite d11/d17/f35 [0,4194304] 0 2024-05-12T04:01:05.762 INFO:tasks.workunit.client.1.smithi033.stdout:1/219: chown d11/d17/f1d 4008 1 2024-05-12T04:01:05.764 INFO:tasks.workunit.client.1.smithi033.stdout:8/195: getdents d1/d16/d27 0 2024-05-12T04:01:05.768 INFO:tasks.workunit.client.1.smithi033.stdout:1/220: dread d11/f13 [4194304,4194304] 0 2024-05-12T04:01:05.768 INFO:tasks.workunit.client.0.smithi032.stdout:0/154: creat df/d10/d1f/f38 x:0 0 0 2024-05-12T04:01:05.777 INFO:tasks.workunit.client.1.smithi033.stdout:9/461: symlink d2/d4/d2b/d6d/lab 0 2024-05-12T04:01:05.778 INFO:tasks.workunit.client.1.smithi033.stdout:9/462: read d2/fa [2340072,91997] 0 2024-05-12T04:01:05.781 INFO:tasks.workunit.client.0.smithi032.stdout:3/114: write d15/d1b/f21 [2355982,110941] 0 2024-05-12T04:01:05.781 INFO:tasks.workunit.client.0.smithi032.stdout:4/53: sync 2024-05-12T04:01:05.781 INFO:tasks.workunit.client.0.smithi032.stdout:2/84: sync 2024-05-12T04:01:05.782 INFO:tasks.workunit.client.0.smithi032.stdout:1/61: sync 2024-05-12T04:01:05.782 INFO:tasks.workunit.client.0.smithi032.stdout:7/71: sync 2024-05-12T04:01:05.782 INFO:tasks.workunit.client.0.smithi032.stdout:9/64: sync 2024-05-12T04:01:05.786 INFO:tasks.workunit.client.0.smithi032.stdout:4/54: readlink d0/l5 0 2024-05-12T04:01:05.788 INFO:tasks.workunit.client.0.smithi032.stdout:8/91: truncate d0/d9/f10 2536880 0 2024-05-12T04:01:05.788 INFO:tasks.workunit.client.0.smithi032.stdout:9/65: chown d0/d3/d6/d7 6 1 2024-05-12T04:01:05.793 INFO:tasks.workunit.client.0.smithi032.stdout:0/155: creat df/d22/f39 x:0 0 0 2024-05-12T04:01:05.795 INFO:tasks.workunit.client.0.smithi032.stdout:3/115: dread fd [0,4194304] 0 2024-05-12T04:01:05.796 INFO:tasks.workunit.client.0.smithi032.stdout:7/72: dwrite dd/fe [0,4194304] 0 2024-05-12T04:01:05.799 INFO:tasks.workunit.client.0.smithi032.stdout:5/102: truncate d4/f6 3654609 0 2024-05-12T04:01:05.800 INFO:tasks.workunit.client.1.smithi033.stdout:2/265: rmdir dc/d17/d27 39 2024-05-12T04:01:05.801 INFO:tasks.workunit.client.0.smithi032.stdout:4/55: creat d0/d3/fb x:0 0 0 2024-05-12T04:01:05.806 INFO:tasks.workunit.client.1.smithi033.stdout:1/221: creat d11/d17/d27/f48 x:0 0 0 2024-05-12T04:01:05.807 INFO:tasks.workunit.client.0.smithi032.stdout:1/62: mkdir d1/d16/d18 0 2024-05-12T04:01:05.814 INFO:tasks.workunit.client.1.smithi033.stdout:9/463: symlink d2/d4/lac 0 2024-05-12T04:01:05.814 INFO:tasks.workunit.client.0.smithi032.stdout:3/116: mknod d15/d20/c27 0 2024-05-12T04:01:05.816 INFO:tasks.workunit.client.0.smithi032.stdout:3/117: dread d15/f18 [0,4194304] 0 2024-05-12T04:01:05.816 INFO:tasks.workunit.client.1.smithi033.stdout:3/260: creat db/d59/f5b x:0 0 0 2024-05-12T04:01:05.818 INFO:tasks.workunit.client.0.smithi032.stdout:7/73: mknod dd/c1f 0 2024-05-12T04:01:05.818 INFO:tasks.workunit.client.1.smithi033.stdout:3/261: write db/d17/d25/f58 [705451,12580] 0 2024-05-12T04:01:05.819 INFO:tasks.workunit.client.0.smithi032.stdout:4/56: unlink d0/c1 0 2024-05-12T04:01:05.821 INFO:tasks.workunit.client.0.smithi032.stdout:4/57: truncate d0/d3/fb 942845 0 2024-05-12T04:01:05.822 INFO:tasks.workunit.client.0.smithi032.stdout:4/58: chown d0/d3/f4 15274189 1 2024-05-12T04:01:05.822 INFO:tasks.workunit.client.1.smithi033.stdout:9/464: dwrite d2/d4/d29/d3c/f9a [0,4194304] 0 2024-05-12T04:01:05.822 INFO:tasks.workunit.client.1.smithi033.stdout:0/311: sync 2024-05-12T04:01:05.822 INFO:tasks.workunit.client.1.smithi033.stdout:5/209: sync 2024-05-12T04:01:05.822 INFO:tasks.workunit.client.0.smithi032.stdout:9/66: creat d0/d3/d6/d7/da/d10/f12 x:0 0 0 2024-05-12T04:01:05.823 INFO:tasks.workunit.client.1.smithi033.stdout:4/285: mknod dd/c54 0 2024-05-12T04:01:05.826 INFO:tasks.workunit.client.1.smithi033.stdout:2/266: symlink dc/d3b/l5b 0 2024-05-12T04:01:05.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.827+0000 7f94c23ec700 1 -- 172.21.15.32:0/1446976094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94bc0fbcb0 msgr2=0x7f94bc0fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:01:05.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.827+0000 7f94c23ec700 1 --2- 172.21.15.32:0/1446976094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94bc0fbcb0 0x7f94bc0fc0c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f94ac004940 tx=0x7f94ac009e30 comp rx=0 tx=0).stop 2024-05-12T04:01:05.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.827+0000 7f94c23ec700 1 -- 172.21.15.32:0/1446976094 shutdown_connections 2024-05-12T04:01:05.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.827+0000 7f94c23ec700 1 --2- 172.21.15.32:0/1446976094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94bc0fcf60 0x7f94bc0fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:05.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.827+0000 7f94c23ec700 1 --2- 172.21.15.32:0/1446976094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94bc0fbcb0 0x7f94bc0fc0c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:05.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.827+0000 7f94c23ec700 1 -- 172.21.15.32:0/1446976094 >> 172.21.15.32:0/1446976094 conn(0x7f94bc0f7430 msgr2=0x7f94bc0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:01:05.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.828+0000 7f94c23ec700 1 -- 172.21.15.32:0/1446976094 shutdown_connections 2024-05-12T04:01:05.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.828+0000 7f94c23ec700 1 -- 172.21.15.32:0/1446976094 wait complete. 2024-05-12T04:01:05.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.828+0000 7f94c23ec700 1 Processor -- start 2024-05-12T04:01:05.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.829+0000 7f94c23ec700 1 -- start start 2024-05-12T04:01:05.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.829+0000 7f94c23ec700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94bc0fbcb0 0x7f94bc10c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:01:05.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.829+0000 7f94c23ec700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94bc0fcf60 0x7f94bc10cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:01:05.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.829+0000 7f94c23ec700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94bc10d3a0 con 0x7f94bc0fcf60 2024-05-12T04:01:05.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.829+0000 7f94c23ec700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94bc10db90 con 0x7f94bc0fbcb0 2024-05-12T04:01:05.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.829+0000 7f94bbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94bc0fbcb0 0x7f94bc10c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:01:05.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.829+0000 7f94bbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94bc0fbcb0 0x7f94bc10c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.32:41568/0 (socket says 172.21.15.32:41568) 2024-05-12T04:01:05.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.829+0000 7f94bbfff700 1 -- 172.21.15.32:0/2158710398 learned_addr learned my addr 172.21.15.32:0/2158710398 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:01:05.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.829+0000 7f94bb7fe700 1 --2- 172.21.15.32:0/2158710398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94bc0fcf60 0x7f94bc10cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:01:05.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.830+0000 7f94bbfff700 1 -- 172.21.15.32:0/2158710398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94bc0fcf60 msgr2=0x7f94bc10cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:01:05.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.830+0000 7f94bbfff700 1 --2- 172.21.15.32:0/2158710398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94bc0fcf60 0x7f94bc10cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:05.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.830+0000 7f94bbfff700 1 -- 172.21.15.32:0/2158710398 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94ac005040 con 0x7f94bc0fbcb0 2024-05-12T04:01:05.831 INFO:tasks.workunit.client.1.smithi033.stdout:5/210: dwrite d3/f3b [0,4194304] 0 2024-05-12T04:01:05.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.830+0000 7f94bbfff700 1 --2- 172.21.15.32:0/2158710398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94bc0fbcb0 0x7f94bc10c7d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f94ac004f90 tx=0x7f94ac009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:01:05.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.830+0000 7f94b8ff9700 1 -- 172.21.15.32:0/2158710398 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94ac00e070 con 0x7f94bc0fbcb0 2024-05-12T04:01:05.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.830+0000 7f94b8ff9700 1 -- 172.21.15.32:0/2158710398 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f94ac013420 con 0x7f94bc0fbcb0 2024-05-12T04:01:05.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.830+0000 7f94b8ff9700 1 -- 172.21.15.32:0/2158710398 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94ac01c640 con 0x7f94bc0fbcb0 2024-05-12T04:01:05.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.831+0000 7f94c23ec700 1 -- 172.21.15.32:0/2158710398 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94bc10de10 con 0x7f94bc0fbcb0 2024-05-12T04:01:05.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.831+0000 7f94c23ec700 1 -- 172.21.15.32:0/2158710398 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94bc10e270 con 0x7f94bc0fbcb0 2024-05-12T04:01:05.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.832+0000 7f94c23ec700 1 -- 172.21.15.32:0/2158710398 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94bc05f5c0 con 0x7f94bc0fbcb0 2024-05-12T04:01:05.833 INFO:tasks.workunit.client.1.smithi033.stdout:4/286: dwrite fa [0,4194304] 0 2024-05-12T04:01:05.834 INFO:tasks.workunit.client.1.smithi033.stdout:4/287: write dd/d11/f19 [3258096,77232] 0 2024-05-12T04:01:05.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.835+0000 7f94b8ff9700 1 -- 172.21.15.32:0/2158710398 <== mon.1 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) v1 ==== 88772+0+0 (secure 0 0 0) 0x7f94ac013590 con 0x7f94bc0fbcb0 2024-05-12T04:01:05.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.835+0000 7f94b8ff9700 1 --2- 172.21.15.32:0/2158710398 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f94a406aec0 0x7f94a406d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:01:05.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.836+0000 7f94b8ff9700 1 -- 172.21.15.32:0/2158710398 <== mon.1 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f94ac08a7f0 con 0x7f94bc0fbcb0 2024-05-12T04:01:05.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.836+0000 7f94bb7fe700 1 --2- 172.21.15.32:0/2158710398 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f94a406aec0 0x7f94a406d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:01:05.837 INFO:tasks.workunit.client.0.smithi032.stdout:2/85: rename d1/d13/f1d to d1/f22 0 2024-05-12T04:01:05.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.840+0000 7f94bb7fe700 1 --2- 172.21.15.32:0/2158710398 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f94a406aec0 0x7f94a406d370 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f94b0009b60 tx=0x7f94b00091a0 comp rx=0 tx=0).ready entity=mgr.24389 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:01:05.841 INFO:tasks.workunit.client.0.smithi032.stdout:5/103: truncate f1 582236 0 2024-05-12T04:01:05.842 INFO:tasks.workunit.client.0.smithi032.stdout:0/156: write df/d10/d1f/f2e [1374678,38276] 0 2024-05-12T04:01:05.843 INFO:tasks.workunit.client.1.smithi033.stdout:1/222: mknod d11/d17/d27/c49 0 2024-05-12T04:01:05.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:05.842+0000 7f94b8ff9700 1 -- 172.21.15.32:0/2158710398 <== mon.1 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181777 (secure 0 0 0) 0x7f94ac058690 con 0x7f94bc0fbcb0 2024-05-12T04:01:05.843 INFO:tasks.workunit.client.0.smithi032.stdout:0/157: chown df/d10/c1e 34050 1 2024-05-12T04:01:05.845 INFO:tasks.workunit.client.0.smithi032.stdout:3/118: symlink d15/d1b/l28 0 2024-05-12T04:01:05.850 INFO:tasks.workunit.client.1.smithi033.stdout:1/223: dwrite d11/d17/d27/f44 [0,4194304] 0 2024-05-12T04:01:05.851 INFO:tasks.workunit.client.0.smithi032.stdout:4/59: creat d0/d3/fc x:0 0 0 2024-05-12T04:01:05.851 INFO:tasks.workunit.client.1.smithi033.stdout:1/224: readlink d11/d1c/l1f 0 2024-05-12T04:01:05.853 INFO:tasks.workunit.client.1.smithi033.stdout:6/228: dwrite d4/f1f [0,4194304] 0 2024-05-12T04:01:05.854 INFO:tasks.workunit.client.0.smithi032.stdout:9/67: mkdir d0/d3/d6/d7/da/d10/d13 0 2024-05-12T04:01:05.855 INFO:tasks.workunit.client.0.smithi032.stdout:0/158: dwrite df/d10/d1f/f2d [0,4194304] 0 2024-05-12T04:01:05.859 INFO:tasks.workunit.client.1.smithi033.stdout:3/262: mknod db/d59/c5c 0 2024-05-12T04:01:05.860 INFO:tasks.workunit.client.0.smithi032.stdout:9/68: write d0/d3/d6/d7/da/d10/f12 [244475,126330] 0 2024-05-12T04:01:05.863 INFO:tasks.workunit.client.0.smithi032.stdout:2/86: unlink d1/fe 0 2024-05-12T04:01:05.864 INFO:tasks.workunit.client.0.smithi032.stdout:4/60: dwrite d0/d3/f4 [0,4194304] 0 2024-05-12T04:01:05.869 INFO:tasks.workunit.client.0.smithi032.stdout:4/61: dread - d0/fa zero size 2024-05-12T04:01:05.869 INFO:tasks.workunit.client.0.smithi032.stdout:4/62: write d0/d3/fb [1488678,58021] 0 2024-05-12T04:01:05.870 INFO:tasks.workunit.client.1.smithi033.stdout:9/465: creat d2/d4/d2b/d6d/fad x:0 0 0 2024-05-12T04:01:05.872 INFO:tasks.workunit.client.1.smithi033.stdout:9/466: truncate d2/d4/d2b/d6d/d2f/d5f/f97 669817 0 2024-05-12T04:01:05.873 INFO:tasks.workunit.client.1.smithi033.stdout:9/467: dread d2/dd/d39/f43 [0,4194304] 0 2024-05-12T04:01:05.875 INFO:tasks.workunit.client.0.smithi032.stdout:5/104: dread d4/f12 [0,4194304] 0 2024-05-12T04:01:05.880 INFO:tasks.workunit.client.1.smithi033.stdout:2/267: creat dc/d17/d32/f5c x:0 0 0 2024-05-12T04:01:05.883 INFO:tasks.workunit.client.1.smithi033.stdout:5/211: creat d3/d15/d16/f44 x:0 0 0 2024-05-12T04:01:05.884 INFO:tasks.workunit.client.0.smithi032.stdout:3/119: rename d15/f1a to d15/d20/d26/f29 0 2024-05-12T04:01:05.884 INFO:tasks.workunit.client.0.smithi032.stdout:1/63: write d1/d7/f11 [664402,41235] 0 2024-05-12T04:01:05.885 INFO:tasks.workunit.client.0.smithi032.stdout:4/63: dread d0/d3/f4 [0,4194304] 0 2024-05-12T04:01:05.886 INFO:tasks.workunit.client.1.smithi033.stdout:4/288: creat dd/d11/d15/d42/f55 x:0 0 0 2024-05-12T04:01:05.886 INFO:tasks.workunit.client.0.smithi032.stdout:7/74: creat dd/d1b/d1e/f20 x:0 0 0 2024-05-12T04:01:05.886 INFO:tasks.workunit.client.0.smithi032.stdout:0/159: mkdir df/d28/d3a 0 2024-05-12T04:01:05.886 INFO:tasks.workunit.client.1.smithi033.stdout:4/289: dread - dd/f4f zero size 2024-05-12T04:01:05.888 INFO:tasks.workunit.client.1.smithi033.stdout:7/425: sync 2024-05-12T04:01:05.889 INFO:tasks.workunit.client.0.smithi032.stdout:1/64: read d1/f13 [289737,53337] 0 2024-05-12T04:01:05.890 INFO:tasks.workunit.client.0.smithi032.stdout:4/64: dread d0/d3/f8 [0,4194304] 0 2024-05-12T04:01:05.891 INFO:tasks.workunit.client.0.smithi032.stdout:1/65: write d1/f3 [6294468,24561] 0 2024-05-12T04:01:05.891 INFO:tasks.workunit.client.1.smithi033.stdout:7/426: write de/df/d17/d1e/d24/f95 [660311,124917] 0 2024-05-12T04:01:05.892 INFO:tasks.workunit.client.0.smithi032.stdout:9/69: creat d0/d3/d6/d7/f14 x:0 0 0 2024-05-12T04:01:05.894 INFO:tasks.workunit.client.1.smithi033.stdout:4/290: dwrite dd/f4b [0,4194304] 0 2024-05-12T04:01:05.895 INFO:tasks.workunit.client.1.smithi033.stdout:4/291: chown dd/d11/d15/d42 833 1 2024-05-12T04:01:05.896 INFO:tasks.workunit.client.0.smithi032.stdout:2/87: truncate d1/fa 864186 0 2024-05-12T04:01:05.896 INFO:tasks.workunit.client.1.smithi033.stdout:4/292: write dd/d11/d15/f2d [32167,61407] 0 2024-05-12T04:01:05.898 INFO:tasks.workunit.client.0.smithi032.stdout:9/70: chown d0/d3/d6/d7/f14 24649301 1 2024-05-12T04:01:05.899 INFO:tasks.workunit.client.0.smithi032.stdout:2/88: read - d1/d13/f18 zero size 2024-05-12T04:01:05.900 INFO:tasks.workunit.client.1.smithi033.stdout:9/468: dread d2/d4/d2b/d6d/d3a/f5a [0,4194304] 0 2024-05-12T04:01:05.902 INFO:tasks.workunit.client.1.smithi033.stdout:6/229: write d4/d1a/f42 [4779027,108023] 0 2024-05-12T04:01:05.904 INFO:tasks.workunit.client.1.smithi033.stdout:3/263: creat db/d17/d25/d32/f5d x:0 0 0 2024-05-12T04:01:05.904 INFO:tasks.workunit.client.0.smithi032.stdout:4/65: dwrite d0/d3/f4 [4194304,4194304] 0 2024-05-12T04:01:05.905 INFO:tasks.workunit.client.0.smithi032.stdout:4/66: fsync d0/d3/fc 0 2024-05-12T04:01:05.907 INFO:tasks.workunit.client.1.smithi033.stdout:5/212: dread d3/d15/d16/f20 [8388608,4194304] 0 2024-05-12T04:01:05.907 INFO:tasks.workunit.client.1.smithi033.stdout:0/312: creat d6/dc/d66/f67 x:0 0 0 2024-05-12T04:01:05.907 INFO:tasks.workunit.client.1.smithi033.stdout:4/293: dwrite dd/d11/d15/d42/f55 [0,4194304] 0 2024-05-12T04:01:05.912 INFO:tasks.workunit.client.1.smithi033.stdout:6/230: dread d4/f1f [0,4194304] 0 2024-05-12T04:01:05.915 INFO:tasks.workunit.client.0.smithi032.stdout:7/75: symlink dd/d1b/l21 0 2024-05-12T04:01:05.917 INFO:tasks.workunit.client.1.smithi033.stdout:3/264: dread db/d17/d25/f58 [0,4194304] 0 2024-05-12T04:01:05.920 INFO:tasks.workunit.client.1.smithi033.stdout:4/294: dwrite dd/d11/d31/d38/f44 [0,4194304] 0 2024-05-12T04:01:05.920 INFO:tasks.workunit.client.1.smithi033.stdout:8/196: getdents d1/d4/d2e 0 2024-05-12T04:01:05.921 INFO:tasks.workunit.client.0.smithi032.stdout:7/76: dread dd/fe [0,4194304] 0 2024-05-12T04:01:05.922 INFO:tasks.workunit.client.1.smithi033.stdout:7/427: fdatasync de/df/d17/f6a 0 2024-05-12T04:01:05.922 INFO:tasks.workunit.client.1.smithi033.stdout:2/268: read dc/d17/d27/d2f/f38 [1220761,112360] 0 2024-05-12T04:01:05.924 INFO:tasks.workunit.client.1.smithi033.stdout:1/225: mkdir d11/d17/d27/d36/d3f/d4a 0 2024-05-12T04:01:05.924 INFO:tasks.workunit.client.0.smithi032.stdout:0/160: mkdir df/d10/d3b 0 2024-05-12T04:01:05.928 INFO:tasks.workunit.client.1.smithi033.stdout:2/269: write dc/d13/d1c/f21 [464981,113880] 0 2024-05-12T04:01:05.930 INFO:tasks.workunit.client.1.smithi033.stdout:9/469: creat d2/dd/d79/fae x:0 0 0 2024-05-12T04:01:05.933 INFO:tasks.workunit.client.1.smithi033.stdout:4/295: dwrite dd/d11/f2f [0,4194304] 0 2024-05-12T04:01:05.936 INFO:tasks.workunit.client.1.smithi033.stdout:8/197: dwrite d1/f2 [4194304,4194304] 0 2024-05-12T04:01:05.936 INFO:tasks.workunit.client.1.smithi033.stdout:4/296: write dd/f4f [670005,96074] 0 2024-05-12T04:01:05.937 INFO:tasks.workunit.client.1.smithi033.stdout:8/198: write d1/d16/f19 [400490,76253] 0 2024-05-12T04:01:05.938 INFO:tasks.workunit.client.0.smithi032.stdout:5/105: creat d4/db/d1b/f23 x:0 0 0 2024-05-12T04:01:05.939 INFO:tasks.workunit.client.1.smithi033.stdout:5/213: mkdir d3/d15/d16/d45 0 2024-05-12T04:01:05.942 INFO:tasks.workunit.client.1.smithi033.stdout:0/313: symlink d6/dc/d39/l68 0 2024-05-12T04:01:05.947 INFO:tasks.workunit.client.1.smithi033.stdout:0/314: dread d6/dc/f1d [0,4194304] 0 2024-05-12T04:01:05.948 INFO:tasks.workunit.client.0.smithi032.stdout:3/120: mkdir d15/d24/d2a 0 2024-05-12T04:01:05.950 INFO:tasks.workunit.client.0.smithi032.stdout:3/121: write d15/d1b/f21 [4073644,37926] 0 2024-05-12T04:01:05.952 INFO:tasks.workunit.client.0.smithi032.stdout:3/122: rename d15/d20 to d15/d20/d26/d2b 22 2024-05-12T04:01:05.953 INFO:tasks.workunit.client.0.smithi032.stdout:3/123: dread - d15/f19 zero size 2024-05-12T04:01:05.953 INFO:tasks.workunit.client.1.smithi033.stdout:0/315: dwrite d6/db/d2c/f3e [0,4194304] 0 2024-05-12T04:01:05.957 INFO:tasks.workunit.client.1.smithi033.stdout:6/231: mknod d4/d1a/c4c 0 2024-05-12T04:01:05.964 INFO:tasks.workunit.client.0.smithi032.stdout:3/124: dwrite d15/d1b/f21 [0,4194304] 0 2024-05-12T04:01:05.965 INFO:tasks.workunit.client.0.smithi032.stdout:0/161: creat df/d10/f3c x:0 0 0 2024-05-12T04:01:05.965 INFO:tasks.workunit.client.0.smithi032.stdout:3/125: fdatasync fd 0 2024-05-12T04:01:05.974 INFO:tasks.workunit.client.0.smithi032.stdout:2/89: write d1/f3 [189422,102547] 0 2024-05-12T04:01:05.974 INFO:tasks.workunit.client.1.smithi033.stdout:1/226: fdatasync d11/d17/d27/d36/f3d 0 2024-05-12T04:01:05.979 INFO:tasks.workunit.client.1.smithi033.stdout:7/428: rename de/df/d17/d1e/d24/f99 to de/df/d7a/f9f 0 2024-05-12T04:01:05.980 INFO:tasks.workunit.client.1.smithi033.stdout:8/199: rename d1/d16/d27 to d1/d16/d27/d28/d3f 22 2024-05-12T04:01:05.980 INFO:tasks.workunit.client.1.smithi033.stdout:6/232: rename d4/d1a to d4/d1a/d4d 22 2024-05-12T04:01:05.981 INFO:tasks.workunit.client.1.smithi033.stdout:6/233: chown d4/c37 937000346 1 2024-05-12T04:01:05.984 INFO:tasks.workunit.client.0.smithi032.stdout:6/80: sync 2024-05-12T04:01:05.984 INFO:tasks.workunit.client.0.smithi032.stdout:8/92: sync 2024-05-12T04:01:05.984 INFO:tasks.workunit.client.1.smithi033.stdout:2/270: creat dc/d17/d27/d47/f5d x:0 0 0 2024-05-12T04:01:05.985 INFO:tasks.workunit.client.0.smithi032.stdout:8/93: stat d0/d7/f15 0 2024-05-12T04:01:05.987 INFO:tasks.workunit.client.0.smithi032.stdout:9/71: link d0/d3/d6/d7/da/fd d0/d3/d6/d7/f15 0 2024-05-12T04:01:05.988 INFO:tasks.workunit.client.1.smithi033.stdout:9/470: rmdir d2/d4/d2b/d78 39 2024-05-12T04:01:05.992 INFO:tasks.workunit.client.0.smithi032.stdout:6/81: dwrite d7/d9/f15 [0,4194304] 0 2024-05-12T04:01:05.996 INFO:tasks.workunit.client.1.smithi033.stdout:4/297: mknod dd/d11/d31/d38/c56 0 2024-05-12T04:01:06.001 INFO:tasks.workunit.client.1.smithi033.stdout:4/298: dread dd/d11/d31/d38/f44 [4194304,4194304] 0 2024-05-12T04:01:06.002 INFO:tasks.workunit.client.0.smithi032.stdout:3/126: read f3 [272942,41080] 0 2024-05-12T04:01:06.005 INFO:tasks.workunit.client.1.smithi033.stdout:0/316: symlink d6/dc/d39/l69 0 2024-05-12T04:01:06.008 INFO:tasks.workunit.client.0.smithi032.stdout:1/66: truncate d1/f6 7026020 0 2024-05-12T04:01:06.011 INFO:tasks.workunit.client.0.smithi032.stdout:8/94: chown d0/d7/c1a 108 1 2024-05-12T04:01:06.012 INFO:tasks.workunit.client.0.smithi032.stdout:8/95: write d0/d9/f12 [1901875,76435] 0 2024-05-12T04:01:06.013 INFO:tasks.workunit.client.1.smithi033.stdout:1/227: mknod d11/d17/d27/d36/c4b 0 2024-05-12T04:01:06.014 INFO:tasks.workunit.client.1.smithi033.stdout:1/228: write d11/d17/f1d [2014172,69137] 0 2024-05-12T04:01:06.014 INFO:tasks.workunit.client.1.smithi033.stdout:1/229: chown c10 19897 1 2024-05-12T04:01:06.016 INFO:tasks.workunit.client.0.smithi032.stdout:9/72: chown d0/d3/d6/d7/l9 387154253 1 2024-05-12T04:01:06.018 INFO:tasks.workunit.client.1.smithi033.stdout:7/429: mkdir de/d2b/d6c/da0 0 2024-05-12T04:01:06.021 INFO:tasks.workunit.client.0.smithi032.stdout:3/127: fdatasync f12 0 2024-05-12T04:01:06.022 INFO:tasks.workunit.client.0.smithi032.stdout:4/67: truncate d0/d3/f8 45186 0 2024-05-12T04:01:06.026 INFO:tasks.workunit.client.1.smithi033.stdout:9/471: creat d2/dd/d39/d72/faf x:0 0 0 2024-05-12T04:01:06.027 INFO:tasks.workunit.client.1.smithi033.stdout:9/472: write d2/d70/d7b/f9f [1048468,6855] 0 2024-05-12T04:01:06.029 INFO:tasks.workunit.client.0.smithi032.stdout:2/90: mknod d1/c23 0 2024-05-12T04:01:06.030 INFO:tasks.workunit.client.0.smithi032.stdout:7/77: truncate f9 1177816 0 2024-05-12T04:01:06.032 INFO:tasks.workunit.client.0.smithi032.stdout:3/128: dwrite d15/d1b/f25 [0,4194304] 0 2024-05-12T04:01:06.034 INFO:tasks.workunit.client.1.smithi033.stdout:2/271: dread dc/d13/f15 [0,4194304] 0 2024-05-12T04:01:06.037 INFO:tasks.workunit.client.1.smithi033.stdout:4/299: mkdir dd/d11/d31/d38/d57 0 2024-05-12T04:01:06.041 INFO:tasks.workunit.client.1.smithi033.stdout:2/272: dwrite f9 [0,4194304] 0 2024-05-12T04:01:06.041 INFO:tasks.workunit.client.1.smithi033.stdout:0/317: mkdir d6/dc/d58/d6a 0 2024-05-12T04:01:06.042 INFO:tasks.workunit.client.1.smithi033.stdout:0/318: readlink d6/db/l30 0 2024-05-12T04:01:06.044 INFO:tasks.workunit.client.1.smithi033.stdout:3/265: creat db/f5e x:0 0 0 2024-05-12T04:01:06.044 INFO:tasks.workunit.client.0.smithi032.stdout:7/78: dwrite f2 [0,4194304] 0 2024-05-12T04:01:06.046 INFO:tasks.workunit.client.1.smithi033.stdout:1/230: mkdir d11/d17/d27/d36/d46/d4c 0 2024-05-12T04:01:06.048 INFO:tasks.workunit.client.1.smithi033.stdout:2/273: dwrite dc/d3b/f4a [0,4194304] 0 2024-05-12T04:01:06.049 INFO:tasks.workunit.client.0.smithi032.stdout:6/82: creat d7/de/d12/f16 x:0 0 0 2024-05-12T04:01:06.050 INFO:tasks.workunit.client.0.smithi032.stdout:3/129: dwrite d15/d1b/f21 [4194304,4194304] 0 2024-05-12T04:01:06.052 INFO:tasks.workunit.client.1.smithi033.stdout:7/430: rmdir de/d80 39 2024-05-12T04:01:06.057 INFO:tasks.workunit.client.0.smithi032.stdout:5/106: dwrite d4/f6 [0,4194304] 0 2024-05-12T04:01:06.057 INFO:tasks.workunit.client.0.smithi032.stdout:0/162: truncate df/d10/d1f/f2d 3116558 0 2024-05-12T04:01:06.068 INFO:tasks.workunit.client.1.smithi033.stdout:7/431: dwrite de/df/d7a/f8b [0,4194304] 0 2024-05-12T04:01:06.068 INFO:tasks.workunit.client.1.smithi033.stdout:5/214: write d3/d12/d25/d26/d2e/f36 [660814,102475] 0 2024-05-12T04:01:06.071 INFO:tasks.workunit.client.0.smithi032.stdout:1/67: symlink d1/d16/d18/l19 0 2024-05-12T04:01:06.071 INFO:tasks.workunit.client.0.smithi032.stdout:1/68: stat d1/d7/f11 0 2024-05-12T04:01:06.074 INFO:tasks.workunit.client.0.smithi032.stdout:6/83: dread d7/de/f14 [0,4194304] 0 2024-05-12T04:01:06.075 INFO:tasks.workunit.client.0.smithi032.stdout:1/69: write d1/d7/f11 [1640884,29899] 0 2024-05-12T04:01:06.077 INFO:tasks.workunit.client.1.smithi033.stdout:1/231: dread d11/f21 [0,4194304] 0 2024-05-12T04:01:06.077 INFO:tasks.workunit.client.0.smithi032.stdout:8/96: mknod d0/c1e 0 2024-05-12T04:01:06.077 INFO:tasks.workunit.client.1.smithi033.stdout:2/274: dread dc/d13/d1f/f24 [0,4194304] 0 2024-05-12T04:01:06.078 INFO:tasks.workunit.client.1.smithi033.stdout:1/232: read - d11/d1c/f32 zero size 2024-05-12T04:01:06.082 INFO:tasks.workunit.client.0.smithi032.stdout:8/97: dread d0/d7/ff [0,4194304] 0 2024-05-12T04:01:06.083 INFO:tasks.workunit.client.0.smithi032.stdout:7/79: rmdir dd 39 2024-05-12T04:01:06.085 INFO:tasks.workunit.client.0.smithi032.stdout:8/98: readlink d0/d9/d13/l19 0 2024-05-12T04:01:06.085 INFO:tasks.workunit.client.0.smithi032.stdout:8/99: stat d0/c1e 0 2024-05-12T04:01:06.086 INFO:tasks.workunit.client.0.smithi032.stdout:9/73: mknod d0/c16 0 2024-05-12T04:01:06.087 INFO:tasks.workunit.client.0.smithi032.stdout:1/70: dwrite d1/f3 [4194304,4194304] 0 2024-05-12T04:01:06.093 INFO:tasks.workunit.client.0.smithi032.stdout:3/130: creat d15/d20/d22/f2c x:0 0 0 2024-05-12T04:01:06.098 INFO:tasks.workunit.client.1.smithi033.stdout:9/473: symlink d2/d4/d2b/d78/lb0 0 2024-05-12T04:01:06.099 INFO:tasks.workunit.client.1.smithi033.stdout:9/474: stat d2/d4/d29/d32/d9b/fa0 0 2024-05-12T04:01:06.100 INFO:tasks.workunit.client.1.smithi033.stdout:9/475: write d2/dd/d39/f67 [1471678,77419] 0 2024-05-12T04:01:06.100 INFO:tasks.workunit.client.0.smithi032.stdout:0/163: readlink l7 0 2024-05-12T04:01:06.105 INFO:tasks.workunit.client.0.smithi032.stdout:0/164: write df/d22/f2f [2289331,108706] 0 2024-05-12T04:01:06.105 INFO:tasks.workunit.client.0.smithi032.stdout:2/91: fsync d1/fa 0 2024-05-12T04:01:06.106 INFO:tasks.workunit.client.1.smithi033.stdout:9/476: truncate d2/d4/d2b/d6d/d2f/d5f/f61 297254 0 2024-05-12T04:01:06.106 INFO:tasks.workunit.client.1.smithi033.stdout:7/432: dwrite de/df/d17/d1b/d36/f4a [0,4194304] 0 2024-05-12T04:01:06.106 INFO:tasks.workunit.client.1.smithi033.stdout:9/477: chown d2/d4/d2b/d6d/d7a/f81 911655 1 2024-05-12T04:01:06.107 INFO:tasks.workunit.client.0.smithi032.stdout:8/100: dwrite d0/d9/fe [0,4194304] 0 2024-05-12T04:01:06.107 INFO:tasks.workunit.client.1.smithi033.stdout:4/300: creat dd/d11/d31/d38/d57/f58 x:0 0 0 2024-05-12T04:01:06.108 INFO:tasks.workunit.client.1.smithi033.stdout:4/301: dread - dd/d11/d15/f49 zero size 2024-05-12T04:01:06.110 INFO:tasks.workunit.client.1.smithi033.stdout:2/275: mkdir dc/d17/d27/d47/d5e 0 2024-05-12T04:01:06.115 INFO:tasks.workunit.client.1.smithi033.stdout:1/233: creat d11/d1c/d43/f4d x:0 0 0 2024-05-12T04:01:06.121 INFO:tasks.workunit.client.1.smithi033.stdout:8/200: sync 2024-05-12T04:01:06.122 INFO:tasks.workunit.client.0.smithi032.stdout:0/165: dwrite f9 [4194304,4194304] 0 2024-05-12T04:01:06.122 INFO:tasks.workunit.client.1.smithi033.stdout:9/478: dread d2/dd/f11 [0,4194304] 0 2024-05-12T04:01:06.123 INFO:tasks.workunit.client.1.smithi033.stdout:6/234: getdents d4/d1a 0 2024-05-12T04:01:06.127 INFO:tasks.workunit.client.1.smithi033.stdout:7/433: dread de/df/d17/d1e/d24/f4f [0,4194304] 0 2024-05-12T04:01:06.130 INFO:tasks.workunit.client.0.smithi032.stdout:9/74: mkdir d0/d3/d6/d7/da/d17 0 2024-05-12T04:01:06.133 INFO:tasks.workunit.client.0.smithi032.stdout:1/71: creat d1/d16/d18/f1a x:0 0 0 2024-05-12T04:01:06.133 INFO:tasks.workunit.client.0.smithi032.stdout:9/75: read - d0/d3/d6/d7/f14 zero size 2024-05-12T04:01:06.134 INFO:tasks.workunit.client.0.smithi032.stdout:9/76: chown d0/lb 7618238 1 2024-05-12T04:01:06.135 INFO:tasks.workunit.client.1.smithi033.stdout:2/276: mknod dc/d13/d1c/c5f 0 2024-05-12T04:01:06.136 INFO:tasks.workunit.client.0.smithi032.stdout:3/131: mknod d15/d20/d26/c2d 0 2024-05-12T04:01:06.138 INFO:tasks.workunit.client.1.smithi033.stdout:1/234: creat d11/d17/d27/d36/d3f/f4e x:0 0 0 2024-05-12T04:01:06.138 INFO:tasks.workunit.client.1.smithi033.stdout:1/235: fsync d11/d1c/d43/f45 0 2024-05-12T04:01:06.140 INFO:tasks.workunit.client.1.smithi033.stdout:0/319: creat d6/f6b x:0 0 0 2024-05-12T04:01:06.140 INFO:tasks.workunit.client.1.smithi033.stdout:0/320: write d6/db/d2c/f31 [1946220,77062] 0 2024-05-12T04:01:06.142 INFO:tasks.workunit.client.1.smithi033.stdout:3/266: rmdir db/d17/d25/d32/d49 0 2024-05-12T04:01:06.144 INFO:tasks.workunit.client.1.smithi033.stdout:8/201: creat d1/d16/d29/f40 x:0 0 0 2024-05-12T04:01:06.145 INFO:tasks.workunit.client.1.smithi033.stdout:5/215: truncate d3/d15/f30 836326 0 2024-05-12T04:01:06.146 INFO:tasks.workunit.client.1.smithi033.stdout:9/479: creat d2/dd/d39/d72/fb1 x:0 0 0 2024-05-12T04:01:06.147 INFO:tasks.workunit.client.1.smithi033.stdout:9/480: write d2/d4/f28 [1676042,96359] 0 2024-05-12T04:01:06.148 INFO:tasks.workunit.client.1.smithi033.stdout:6/235: mkdir d4/d1a/d4e 0 2024-05-12T04:01:06.149 INFO:tasks.workunit.client.0.smithi032.stdout:8/101: creat d0/d9/d13/f1f x:0 0 0 2024-05-12T04:01:06.151 INFO:tasks.workunit.client.1.smithi033.stdout:6/236: write d4/d1a/f3e [8210,78399] 0 2024-05-12T04:01:06.153 INFO:tasks.workunit.client.0.smithi032.stdout:6/84: link d7/d9/f15 d7/d9/f17 0 2024-05-12T04:01:06.153 INFO:tasks.workunit.client.0.smithi032.stdout:6/85: readlink l5 0 2024-05-12T04:01:06.154 INFO:tasks.workunit.client.1.smithi033.stdout:5/216: dwrite d3/d27/f22 [0,4194304] 0 2024-05-12T04:01:06.155 INFO:tasks.workunit.client.0.smithi032.stdout:7/80: creat dd/f22 x:0 0 0 2024-05-12T04:01:06.155 INFO:tasks.workunit.client.0.smithi032.stdout:7/81: chown fb 500284538 1 2024-05-12T04:01:06.157 INFO:tasks.workunit.client.0.smithi032.stdout:0/166: mkdir df/d10/d3d 0 2024-05-12T04:01:06.159 INFO:tasks.workunit.client.1.smithi033.stdout:4/302: unlink dd/f3e 0 2024-05-12T04:01:06.159 INFO:tasks.workunit.client.1.smithi033.stdout:1/236: rmdir d11/d17/d38 39 2024-05-12T04:01:06.161 INFO:tasks.workunit.client.0.smithi032.stdout:9/77: mknod d0/d3/d6/d7/da/c18 0 2024-05-12T04:01:06.162 INFO:tasks.workunit.client.1.smithi033.stdout:0/321: symlink d6/d11/l6c 0 2024-05-12T04:01:06.163 INFO:tasks.workunit.client.0.smithi032.stdout:3/132: dread - d15/f17 zero size 2024-05-12T04:01:06.164 INFO:tasks.workunit.client.1.smithi033.stdout:8/202: rename d1/d16/d20/c24 to d1/d16/c41 0 2024-05-12T04:01:06.165 INFO:tasks.workunit.client.0.smithi032.stdout:3/133: read - d15/f19 zero size 2024-05-12T04:01:06.170 INFO:tasks.workunit.client.0.smithi032.stdout:0/167: dwrite df/d10/f13 [8388608,4194304] 0 2024-05-12T04:01:06.170 INFO:tasks.workunit.client.0.smithi032.stdout:3/134: write d15/d20/d22/f23 [305953,393] 0 2024-05-12T04:01:06.170 INFO:tasks.workunit.client.0.smithi032.stdout:8/102: creat d0/d9/d13/f20 x:0 0 0 2024-05-12T04:01:06.170 INFO:tasks.workunit.client.1.smithi033.stdout:9/481: symlink d2/d4/d2b/d78/lb2 0 2024-05-12T04:01:06.171 INFO:tasks.workunit.client.0.smithi032.stdout:0/168: fdatasync fa 0 2024-05-12T04:01:06.171 INFO:tasks.workunit.client.0.smithi032.stdout:3/135: chown d15/c1f 517882590 1 2024-05-12T04:01:06.172 INFO:tasks.workunit.client.0.smithi032.stdout:8/103: dread - d0/d7/f15 zero size 2024-05-12T04:01:06.172 INFO:tasks.workunit.client.0.smithi032.stdout:9/78: dread d0/d3/ff [0,4194304] 0 2024-05-12T04:01:06.172 INFO:tasks.workunit.client.0.smithi032.stdout:6/86: creat d7/de/d12/f18 x:0 0 0 2024-05-12T04:01:06.172 INFO:tasks.workunit.client.0.smithi032.stdout:3/136: chown l11 6778 1 2024-05-12T04:01:06.173 INFO:tasks.workunit.client.1.smithi033.stdout:9/482: read d2/d4/d2b/d6d/f69 [896015,59787] 0 2024-05-12T04:01:06.174 INFO:tasks.workunit.client.0.smithi032.stdout:9/79: chown d0/d3/c5 160881 1 2024-05-12T04:01:06.174 INFO:tasks.workunit.client.1.smithi033.stdout:9/483: dread d2/d4/d2b/d6d/d2f/d5f/f61 [0,4194304] 0 2024-05-12T04:01:06.178 INFO:tasks.workunit.client.1.smithi033.stdout:5/217: creat d3/d12/d29/d2b/f46 x:0 0 0 2024-05-12T04:01:06.185 INFO:tasks.workunit.client.0.smithi032.stdout:5/107: getdents d4/d7/d14 0 2024-05-12T04:01:06.187 INFO:tasks.workunit.client.0.smithi032.stdout:7/82: dwrite dd/f1a [4194304,4194304] 0 2024-05-12T04:01:06.189 INFO:tasks.workunit.client.1.smithi033.stdout:4/303: dread dd/f4b [0,4194304] 0 2024-05-12T04:01:06.189 INFO:tasks.workunit.client.0.smithi032.stdout:7/83: stat dd/d19/f1c 0 2024-05-12T04:01:06.191 INFO:tasks.workunit.client.0.smithi032.stdout:2/92: rename d1/f22 to d1/f24 0 2024-05-12T04:01:06.195 INFO:tasks.workunit.client.0.smithi032.stdout:4/68: sync 2024-05-12T04:01:06.197 INFO:tasks.workunit.client.0.smithi032.stdout:9/80: dwrite d0/d3/ff [4194304,4194304] 0 2024-05-12T04:01:06.199 INFO:tasks.workunit.client.0.smithi032.stdout:5/108: dwrite d4/fa [0,4194304] 0 2024-05-12T04:01:06.199 INFO:tasks.workunit.client.0.smithi032.stdout:4/69: chown d0/d3/fb 688 1 2024-05-12T04:01:06.200 INFO:tasks.workunit.client.0.smithi032.stdout:0/169: readlink df/l1d 0 2024-05-12T04:01:06.200 INFO:tasks.workunit.client.0.smithi032.stdout:4/70: read - d0/fa zero size 2024-05-12T04:01:06.201 INFO:tasks.workunit.client.0.smithi032.stdout:0/170: dread - df/d22/f39 zero size 2024-05-12T04:01:06.208 INFO:tasks.workunit.client.0.smithi032.stdout:8/104: mkdir d0/d9/d13/d21 0 2024-05-12T04:01:06.210 INFO:tasks.workunit.client.0.smithi032.stdout:3/137: creat d15/d20/d22/f2e x:0 0 0 2024-05-12T04:01:06.211 INFO:tasks.workunit.client.1.smithi033.stdout:6/237: mkdir d4/d1a/d4e/d4f 0 2024-05-12T04:01:06.218 INFO:tasks.workunit.client.1.smithi033.stdout:2/277: creat dc/d17/d27/f60 x:0 0 0 2024-05-12T04:01:06.219 INFO:tasks.workunit.client.1.smithi033.stdout:2/278: write dc/d17/f23 [926992,11247] 0 2024-05-12T04:01:06.221 INFO:tasks.workunit.client.1.smithi033.stdout:4/304: fdatasync dd/d11/f21 0 2024-05-12T04:01:06.222 INFO:tasks.workunit.client.1.smithi033.stdout:4/305: write fc [647811,102301] 0 2024-05-12T04:01:06.223 INFO:tasks.workunit.client.0.smithi032.stdout:7/84: mknod dd/d1b/c23 0 2024-05-12T04:01:06.223 INFO:tasks.workunit.client.0.smithi032.stdout:7/85: fsync dd/f22 0 2024-05-12T04:01:06.225 INFO:tasks.workunit.client.0.smithi032.stdout:2/93: mkdir d1/d13/d25 0 2024-05-12T04:01:06.225 INFO:tasks.workunit.client.0.smithi032.stdout:1/72: write d1/f6 [2715955,77574] 0 2024-05-12T04:01:06.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:06.229+0000 7f94c23ec700 1 -- 172.21.15.32:0/2158710398 --> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f94bc10e5c0 con 0x7f94a406aec0 2024-05-12T04:01:06.230 INFO:tasks.workunit.client.0.smithi032.stdout:9/81: unlink d0/l2 0 2024-05-12T04:01:06.232 INFO:tasks.workunit.client.1.smithi033.stdout:7/434: rename de/f25 to de/fa1 0 2024-05-12T04:01:06.233 INFO:tasks.workunit.client.0.smithi032.stdout:9/82: readlink d0/d3/d6/d7/da/l11 0 2024-05-12T04:01:06.233 INFO:tasks.workunit.client.1.smithi033.stdout:8/203: write d1/d4/f2b [3113613,116831] 0 2024-05-12T04:01:06.233 INFO:tasks.workunit.client.1.smithi033.stdout:7/435: dread - de/df/d17/d1e/d79/f91 zero size 2024-05-12T04:01:06.234 INFO:tasks.workunit.client.0.smithi032.stdout:1/73: dwrite d1/f6 [4194304,4194304] 0 2024-05-12T04:01:06.235 INFO:tasks.workunit.client.0.smithi032.stdout:0/171: mknod df/d30/c3e 0 2024-05-12T04:01:06.235 INFO:tasks.workunit.client.0.smithi032.stdout:6/87: write d7/fc [420449,36512] 0 2024-05-12T04:01:06.237 INFO:tasks.workunit.client.0.smithi032.stdout:1/74: write d1/d7/f11 [2628587,119288] 0 2024-05-12T04:01:06.240 INFO:tasks.workunit.client.1.smithi033.stdout:9/484: symlink d2/d4/d29/lb3 0 2024-05-12T04:01:06.241 INFO:tasks.workunit.client.0.smithi032.stdout:0/172: truncate df/d30/d32/f34 74561 0 2024-05-12T04:01:06.241 INFO:tasks.workunit.client.0.smithi032.stdout:3/138: chown d15/d20/d26/f29 5 1 2024-05-12T04:01:06.242 INFO:tasks.workunit.client.1.smithi033.stdout:9/485: fdatasync d2/d4/d2b/d6d/d3a/f65 0 2024-05-12T04:01:06.244 INFO:tasks.workunit.client.0.smithi032.stdout:3/139: chown d15/d1b/f21 94 1 2024-05-12T04:01:06.244 INFO:tasks.workunit.client.0.smithi032.stdout:3/140: write fd [3163001,79093] 0 2024-05-12T04:01:06.245 INFO:tasks.workunit.client.0.smithi032.stdout:3/141: dread d15/f18 [0,4194304] 0 2024-05-12T04:01:06.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:06.247+0000 7f94b8ff9700 1 -- 172.21.15.32:0/2158710398 <== mgr.24389 v2:172.21.15.33:6828/1598726962 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7f94bc10e5c0 con 0x7f94a406aec0 2024-05-12T04:01:06.247 INFO:teuthology.orchestra.run.smithi032.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-12T04:01:06.248 INFO:teuthology.orchestra.run.smithi032.stdout:alertmanager.smithi032 smithi032 *:9093,9094 running (7m) 48s ago 9m 27.8M - 0.25.0 c8568f914cd2 5c80dfe17d2e 2024-05-12T04:01:06.248 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi032 smithi032 running (9m) 48s ago 9m 6651k - 17.2.7-939-gc190bdee 284803f7d9cf a478874e9dfa 2024-05-12T04:01:06.248 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi033 smithi033 running (8m) 48s ago 8m 6651k - 17.2.7-939-gc190bdee 284803f7d9cf 34c62d004c7e 2024-05-12T04:01:06.248 INFO:teuthology.orchestra.run.smithi032.stdout:grafana.smithi032 smithi032 *:3000 running (7m) 48s ago 8m 87.5M - 9.4.7 954c08fa6188 ac9076563478 2024-05-12T04:01:06.248 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.ialtqb smithi032 running (3m) 48s ago 3m 11.6M - 17.2.7-939-gc190bdee 284803f7d9cf 0683abf00e3e 2024-05-12T04:01:06.248 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.qpsawh smithi032 running (3m) 48s ago 3m 104M - 17.2.7-939-gc190bdee 284803f7d9cf 95a7db5c4fcd 2024-05-12T04:01:06.249 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi033.hglwml smithi033 running (3m) 48s ago 3m 11.6M - 17.2.7-939-gc190bdee 284803f7d9cf c97902b05025 2024-05-12T04:01:06.249 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi033.jgbjhk smithi033 running (3m) 48s ago 3m 82.6M - 17.2.7-939-gc190bdee 284803f7d9cf af97052cb97a 2024-05-12T04:01:06.249 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi032.ukatkq smithi032 *:9283 running (10m) 48s ago 10m 158M - 17.2.7-939-gc190bdee 284803f7d9cf d9763dc47404 2024-05-12T04:01:06.249 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi033.pcvxwy smithi033 *:8443,9283 running (67s) 48s ago 8m 484M - 18.2.2-1272-g58249552 ddcb0a9d6fde b0bff3ab143f 2024-05-12T04:01:06.249 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi032 smithi032 running (10m) 48s ago 10m 49.9M 2048M 17.2.7-939-gc190bdee 284803f7d9cf d7fea430c6e2 2024-05-12T04:01:06.249 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi033 smithi033 running (8m) 48s ago 8m 40.1M 2048M 17.2.7-939-gc190bdee 284803f7d9cf 707654d85748 2024-05-12T04:01:06.249 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi032 smithi032 *:9100 starting - - - - 2024-05-12T04:01:06.250 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi033 smithi033 *:9100 running (8m) 48s ago 8m 17.4M - 1.5.0 0da6a335fe13 1144caed00e9 2024-05-12T04:01:06.250 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0 smithi032 running (7m) 48s ago 7m 66.2M 1570M 17.2.7-939-gc190bdee 284803f7d9cf 75c78a47f8e5 2024-05-12T04:01:06.250 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1 smithi032 running (6m) 48s ago 6m 67.5M 1570M 17.2.7-939-gc190bdee 284803f7d9cf 87cfd247d8d6 2024-05-12T04:01:06.250 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2 smithi032 running (6m) 48s ago 6m 57.7M 1570M 17.2.7-939-gc190bdee 284803f7d9cf 7aef7affb8ba 2024-05-12T04:01:06.250 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3 smithi033 running (5m) 48s ago 5m 77.0M 2594M 17.2.7-939-gc190bdee 284803f7d9cf 3a92b5e3348e 2024-05-12T04:01:06.251 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4 smithi033 running (5m) 48s ago 5m 63.5M 2594M 17.2.7-939-gc190bdee 284803f7d9cf 0896268b4591 2024-05-12T04:01:06.251 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5 smithi033 running (4m) 48s ago 4m 64.4M 2594M 17.2.7-939-gc190bdee 284803f7d9cf 57e65a64c821 2024-05-12T04:01:06.251 INFO:teuthology.orchestra.run.smithi032.stdout:prometheus.smithi032 smithi032 *:9095 running (7m) 48s ago 8m 57.7M - 2.43.0 a07b618ecd1d 0b3ce3323a49 2024-05-12T04:01:06.252 INFO:tasks.workunit.client.0.smithi032.stdout:9/83: mknod d0/d3/d6/d7/da/c19 0 2024-05-12T04:01:06.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:06.250+0000 7f94c23ec700 1 -- 172.21.15.32:0/2158710398 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f94a406aec0 msgr2=0x7f94a406d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:01:06.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:06.250+0000 7f94c23ec700 1 --2- 172.21.15.32:0/2158710398 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f94a406aec0 0x7f94a406d370 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f94b0009b60 tx=0x7f94b00091a0 comp rx=0 tx=0).stop 2024-05-12T04:01:06.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:06.250+0000 7f94c23ec700 1 -- 172.21.15.32:0/2158710398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94bc0fbcb0 msgr2=0x7f94bc10c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:01:06.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:06.250+0000 7f94c23ec700 1 --2- 172.21.15.32:0/2158710398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94bc0fbcb0 0x7f94bc10c7d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f94ac004f90 tx=0x7f94ac009e30 comp rx=0 tx=0).stop 2024-05-12T04:01:06.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:06.250+0000 7f94c23ec700 1 -- 172.21.15.32:0/2158710398 shutdown_connections 2024-05-12T04:01:06.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:06.250+0000 7f94c23ec700 1 --2- 172.21.15.32:0/2158710398 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f94a406aec0 0x7f94a406d370 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:06.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:06.250+0000 7f94c23ec700 1 --2- 172.21.15.32:0/2158710398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94bc0fbcb0 0x7f94bc10c7d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:06.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:06.250+0000 7f94c23ec700 1 --2- 172.21.15.32:0/2158710398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94bc0fcf60 0x7f94bc10cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:06.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:06.250+0000 7f94c23ec700 1 -- 172.21.15.32:0/2158710398 >> 172.21.15.32:0/2158710398 conn(0x7f94bc0f7430 msgr2=0x7f94bc0f8b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:01:06.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:06.250+0000 7f94c23ec700 1 -- 172.21.15.32:0/2158710398 shutdown_connections 2024-05-12T04:01:06.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:06.250+0000 7f94c23ec700 1 -- 172.21.15.32:0/2158710398 wait complete. 2024-05-12T04:01:06.255 INFO:tasks.workunit.client.1.smithi033.stdout:1/237: creat d11/f4f x:0 0 0 2024-05-12T04:01:06.255 INFO:tasks.workunit.client.0.smithi032.stdout:3/142: dwrite f14 [0,4194304] 0 2024-05-12T04:01:06.256 INFO:tasks.workunit.client.1.smithi033.stdout:4/306: dread dd/f3d [0,4194304] 0 2024-05-12T04:01:06.257 INFO:tasks.workunit.client.1.smithi033.stdout:7/436: creat de/d20/d38/fa2 x:0 0 0 2024-05-12T04:01:06.258 INFO:tasks.workunit.client.0.smithi032.stdout:1/75: creat d1/d16/d18/f1b x:0 0 0 2024-05-12T04:01:06.259 INFO:tasks.workunit.client.1.smithi033.stdout:6/238: creat d4/d1a/d4e/d4f/f50 x:0 0 0 2024-05-12T04:01:06.259 INFO:tasks.workunit.client.1.smithi033.stdout:3/267: sync 2024-05-12T04:01:06.260 INFO:tasks.workunit.client.0.smithi032.stdout:0/173: chown df/d10/d1f/f2d 1 1 2024-05-12T04:01:06.260 INFO:tasks.workunit.client.1.smithi033.stdout:6/239: dread - d4/d1a/d4e/d4f/f50 zero size 2024-05-12T04:01:06.261 INFO:tasks.workunit.client.1.smithi033.stdout:6/240: readlink d4/l38 0 2024-05-12T04:01:06.268 INFO:tasks.workunit.client.1.smithi033.stdout:6/241: dwrite d4/d1a/f47 [0,4194304] 0 2024-05-12T04:01:06.268 INFO:tasks.workunit.client.0.smithi032.stdout:9/84: creat d0/d3/d6/d7/da/f1a x:0 0 0 2024-05-12T04:01:06.269 INFO:tasks.workunit.client.0.smithi032.stdout:1/76: dwrite d1/d16/d18/f1a [0,4194304] 0 2024-05-12T04:01:06.269 INFO:tasks.workunit.client.1.smithi033.stdout:2/279: symlink dc/d17/d27/d47/d5e/l61 0 2024-05-12T04:01:06.269 INFO:tasks.workunit.client.1.smithi033.stdout:6/242: dread - d4/d1a/f32 zero size 2024-05-12T04:01:06.270 INFO:tasks.workunit.client.0.smithi032.stdout:9/85: dread - d0/d3/d6/d7/f14 zero size 2024-05-12T04:01:06.271 INFO:tasks.workunit.client.0.smithi032.stdout:8/105: rename d0/d7/ff to d0/f22 0 2024-05-12T04:01:06.271 INFO:tasks.workunit.client.0.smithi032.stdout:1/77: read - d1/d16/d18/f1b zero size 2024-05-12T04:01:06.274 INFO:tasks.workunit.client.1.smithi033.stdout:6/243: dread d4/f2f [0,4194304] 0 2024-05-12T04:01:06.278 INFO:tasks.workunit.client.0.smithi032.stdout:6/88: link d7/fc d7/de/f19 0 2024-05-12T04:01:06.278 INFO:tasks.workunit.client.0.smithi032.stdout:9/86: write d0/d3/d6/d7/da/f1a [512748,38394] 0 2024-05-12T04:01:06.278 INFO:tasks.workunit.client.0.smithi032.stdout:8/106: write d0/d9/fe [2105112,120862] 0 2024-05-12T04:01:06.279 INFO:tasks.workunit.client.0.smithi032.stdout:3/143: truncate f3 1739274 0 2024-05-12T04:01:06.280 INFO:tasks.workunit.client.1.smithi033.stdout:0/322: sync 2024-05-12T04:01:06.283 INFO:tasks.workunit.client.1.smithi033.stdout:6/244: dwrite d4/f16 [4194304,4194304] 0 2024-05-12T04:01:06.283 INFO:tasks.workunit.client.1.smithi033.stdout:0/323: truncate d6/dc/f56 932425 0 2024-05-12T04:01:06.285 INFO:tasks.workunit.client.1.smithi033.stdout:0/324: write f4 [3753485,20410] 0 2024-05-12T04:01:06.285 INFO:tasks.workunit.client.1.smithi033.stdout:6/245: truncate d4/d5/f35 1484734 0 2024-05-12T04:01:06.287 INFO:tasks.workunit.client.1.smithi033.stdout:8/204: symlink d1/d16/d27/d28/l42 0 2024-05-12T04:01:06.288 INFO:tasks.workunit.client.1.smithi033.stdout:9/486: write d2/d70/f87 [719745,95528] 0 2024-05-12T04:01:06.288 INFO:tasks.workunit.client.1.smithi033.stdout:4/307: mkdir dd/d11/d15/d42/d59 0 2024-05-12T04:01:06.290 INFO:tasks.workunit.client.1.smithi033.stdout:9/487: dread - d2/d4/d2b/d6d/fad zero size 2024-05-12T04:01:06.291 INFO:tasks.workunit.client.1.smithi033.stdout:3/268: rename db/d17/c19 to db/d17/d3e/c5f 0 2024-05-12T04:01:06.291 INFO:tasks.workunit.client.1.smithi033.stdout:3/269: fdatasync f6 0 2024-05-12T04:01:06.292 INFO:tasks.workunit.client.0.smithi032.stdout:4/71: getdents d0 0 2024-05-12T04:01:06.294 INFO:tasks.workunit.client.0.smithi032.stdout:1/78: mknod d1/c1c 0 2024-05-12T04:01:06.294 INFO:tasks.workunit.client.0.smithi032.stdout:4/72: write d0/d3/f4 [1291982,42293] 0 2024-05-12T04:01:06.296 INFO:tasks.workunit.client.0.smithi032.stdout:7/86: rename dd/l18 to dd/d1b/l24 0 2024-05-12T04:01:06.298 INFO:tasks.workunit.client.0.smithi032.stdout:7/87: dread fc [0,4194304] 0 2024-05-12T04:01:06.298 INFO:tasks.workunit.client.1.smithi033.stdout:2/280: write f3 [1288140,25735] 0 2024-05-12T04:01:06.300 INFO:tasks.workunit.client.0.smithi032.stdout:6/89: mkdir d7/de/d12/d1a 0 2024-05-12T04:01:06.301 INFO:tasks.workunit.client.0.smithi032.stdout:7/88: read dd/f12 [124086,98526] 0 2024-05-12T04:01:06.301 INFO:tasks.workunit.client.0.smithi032.stdout:7/89: chown fa 77 1 2024-05-12T04:01:06.302 INFO:tasks.workunit.client.1.smithi033.stdout:5/218: dwrite d3/d15/f30 [0,4194304] 0 2024-05-12T04:01:06.303 INFO:tasks.workunit.client.0.smithi032.stdout:9/87: mkdir d0/d3/d1b 0 2024-05-12T04:01:06.311 INFO:tasks.workunit.client.0.smithi032.stdout:3/144: creat d15/d1b/f2f x:0 0 0 2024-05-12T04:01:06.313 INFO:tasks.workunit.client.0.smithi032.stdout:2/94: truncate d1/fc 95506 0 2024-05-12T04:01:06.314 INFO:tasks.workunit.client.1.smithi033.stdout:8/205: dread d1/d16/d29/f30 [0,4194304] 0 2024-05-12T04:01:06.314 INFO:tasks.workunit.client.1.smithi033.stdout:0/325: unlink d6/d11/c3b 0 2024-05-12T04:01:06.316 INFO:tasks.workunit.client.0.smithi032.stdout:0/174: truncate fa 7072072 0 2024-05-12T04:01:06.317 INFO:tasks.workunit.client.1.smithi033.stdout:6/246: stat d4/d1a/d3c/l4b 0 2024-05-12T04:01:06.322 INFO:tasks.workunit.client.0.smithi032.stdout:1/79: unlink d1/l2 0 2024-05-12T04:01:06.324 INFO:tasks.workunit.client.1.smithi033.stdout:7/437: unlink de/df/l19 0 2024-05-12T04:01:06.324 INFO:tasks.workunit.client.1.smithi033.stdout:0/326: dread d6/d11/f1c [0,4194304] 0 2024-05-12T04:01:06.329 INFO:tasks.workunit.client.1.smithi033.stdout:9/488: fsync d2/fa 0 2024-05-12T04:01:06.330 INFO:tasks.workunit.client.1.smithi033.stdout:7/438: dwrite de/df/d17/d1b/f89 [0,4194304] 0 2024-05-12T04:01:06.331 INFO:tasks.workunit.client.0.smithi032.stdout:1/80: dwrite d1/d7/f11 [0,4194304] 0 2024-05-12T04:01:06.333 INFO:tasks.workunit.client.0.smithi032.stdout:1/81: write d1/f6 [6565557,46191] 0 2024-05-12T04:01:06.335 INFO:tasks.workunit.client.1.smithi033.stdout:3/270: symlink db/d17/d3e/l60 0 2024-05-12T04:01:06.335 INFO:tasks.workunit.client.1.smithi033.stdout:3/271: dread - db/f5e zero size 2024-05-12T04:01:06.340 INFO:tasks.workunit.client.0.smithi032.stdout:7/90: creat dd/f25 x:0 0 0 2024-05-12T04:01:06.342 INFO:tasks.workunit.client.0.smithi032.stdout:7/91: read - dd/d1b/d1e/f20 zero size 2024-05-12T04:01:06.344 INFO:tasks.workunit.client.0.smithi032.stdout:9/88: creat d0/d3/d6/d7/da/f1c x:0 0 0 2024-05-12T04:01:06.349 INFO:tasks.workunit.client.0.smithi032.stdout:8/107: symlink d0/l23 0 2024-05-12T04:01:06.349 INFO:tasks.workunit.client.0.smithi032.stdout:8/108: stat d0/d9/d13/l19 0 2024-05-12T04:01:06.350 INFO:tasks.workunit.client.1.smithi033.stdout:2/281: creat dc/d17/d27/d2f/f62 x:0 0 0 2024-05-12T04:01:06.351 INFO:tasks.workunit.client.0.smithi032.stdout:8/109: truncate d0/d7/f15 34621 0 2024-05-12T04:01:06.352 INFO:tasks.workunit.client.0.smithi032.stdout:8/110: dread d0/d7/f15 [0,4194304] 0 2024-05-12T04:01:06.352 INFO:tasks.workunit.client.0.smithi032.stdout:3/145: symlink d15/d20/d26/l30 0 2024-05-12T04:01:06.354 INFO:tasks.workunit.client.1.smithi033.stdout:8/206: rename d1/d16/d29/f40 to d1/d16/d20/f43 0 2024-05-12T04:01:06.356 INFO:tasks.workunit.client.0.smithi032.stdout:2/95: unlink d1/d13/f1b 0 2024-05-12T04:01:06.359 INFO:tasks.workunit.client.1.smithi033.stdout:1/238: creat d11/d17/d27/d36/d3f/f50 x:0 0 0 2024-05-12T04:01:06.360 INFO:tasks.workunit.client.1.smithi033.stdout:8/207: dwrite d1/d16/d20/f2f [0,4194304] 0 2024-05-12T04:01:06.360 INFO:tasks.workunit.client.0.smithi032.stdout:3/146: dwrite d15/f1c [0,4194304] 0 2024-05-12T04:01:06.362 INFO:tasks.workunit.client.1.smithi033.stdout:0/327: unlink d6/dc/f3c 0 2024-05-12T04:01:06.370 INFO:tasks.workunit.client.0.smithi032.stdout:4/73: unlink d0/d3/f8 0 2024-05-12T04:01:06.376 INFO:tasks.workunit.client.1.smithi033.stdout:9/489: unlink d2/dd/d39/d72/fb1 0 2024-05-12T04:01:06.377 INFO:tasks.workunit.client.1.smithi033.stdout:9/490: write d2/d4/f1f [895715,61420] 0 2024-05-12T04:01:06.383 INFO:tasks.workunit.client.1.smithi033.stdout:9/491: dwrite d2/d4/d2b/d6d/d7a/f81 [0,4194304] 0 2024-05-12T04:01:06.385 INFO:tasks.workunit.client.0.smithi032.stdout:1/82: rename d1/d7/c14 to d1/d16/d18/c1d 0 2024-05-12T04:01:06.386 INFO:tasks.workunit.client.0.smithi032.stdout:4/74: dwrite d0/d3/fc [0,4194304] 0 2024-05-12T04:01:06.387 INFO:tasks.workunit.client.0.smithi032.stdout:5/109: sync 2024-05-12T04:01:06.388 INFO:tasks.workunit.client.0.smithi032.stdout:4/75: dread - d0/fa zero size 2024-05-12T04:01:06.389 INFO:tasks.workunit.client.1.smithi033.stdout:9/492: dread d2/d4/d29/d3c/d52/f86 [0,4194304] 0 2024-05-12T04:01:06.392 INFO:tasks.workunit.client.1.smithi033.stdout:5/219: getdents d3/d15/d16/d41 0 2024-05-12T04:01:06.397 INFO:tasks.workunit.client.0.smithi032.stdout:7/92: stat dd/d1b/l24 0 2024-05-12T04:01:06.398 INFO:tasks.workunit.client.0.smithi032.stdout:7/93: readlink dd/l16 0 2024-05-12T04:01:06.400 INFO:tasks.workunit.client.0.smithi032.stdout:6/90: truncate f6 1036497 0 2024-05-12T04:01:06.401 INFO:tasks.workunit.client.0.smithi032.stdout:7/94: fdatasync dd/f1a 0 2024-05-12T04:01:06.403 INFO:tasks.workunit.client.1.smithi033.stdout:3/272: write db/d17/d25/f58 [610568,63303] 0 2024-05-12T04:01:06.406 INFO:tasks.workunit.client.1.smithi033.stdout:1/239: creat d11/d17/d27/d36/d46/f51 x:0 0 0 2024-05-12T04:01:06.406 INFO:tasks.workunit.client.1.smithi033.stdout:1/240: chown d11/f19 7439326 1 2024-05-12T04:01:06.407 INFO:tasks.workunit.client.1.smithi033.stdout:1/241: chown d11/f19 189437680 1 2024-05-12T04:01:06.407 INFO:tasks.workunit.client.0.smithi032.stdout:4/76: dwrite d0/fa [0,4194304] 0 2024-05-12T04:01:06.408 INFO:tasks.workunit.client.0.smithi032.stdout:1/83: dwrite d1/f3 [0,4194304] 0 2024-05-12T04:01:06.408 INFO:tasks.workunit.client.1.smithi033.stdout:8/208: mknod d1/d4/d1d/c44 0 2024-05-12T04:01:06.412 INFO:tasks.workunit.client.1.smithi033.stdout:0/328: mknod d6/dc/c6d 0 2024-05-12T04:01:06.413 INFO:tasks.workunit.client.0.smithi032.stdout:9/89: write d0/d3/d6/d7/da/fd [858154,114091] 0 2024-05-12T04:01:06.414 INFO:tasks.workunit.client.1.smithi033.stdout:8/209: dwrite d1/d16/f19 [0,4194304] 0 2024-05-12T04:01:06.415 INFO:tasks.workunit.client.0.smithi032.stdout:4/77: write d0/fa [4158765,4543] 0 2024-05-12T04:01:06.417 INFO:tasks.workunit.client.0.smithi032.stdout:9/90: write d0/d3/d6/d7/da/f1a [267784,81137] 0 2024-05-12T04:01:06.421 INFO:tasks.workunit.client.0.smithi032.stdout:2/96: write d1/f24 [899155,43728] 0 2024-05-12T04:01:06.429 INFO:tasks.workunit.client.1.smithi033.stdout:7/439: link de/f26 de/d2f/d70/fa3 0 2024-05-12T04:01:06.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:06.430+0000 7fa94f2ad700 1 -- 172.21.15.32:0/23418737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9480fbce0 msgr2=0x7fa9480fc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:01:06.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:06.430+0000 7fa94f2ad700 1 --2- 172.21.15.32:0/23418737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9480fbce0 0x7fa9480fc0f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa938004940 tx=0x7fa938009e30 comp rx=0 tx=0).stop 2024-05-12T04:01:06.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:06.430+0000 7fa94f2ad700 1 -- 172.21.15.32:0/23418737 shutdown_connections 2024-05-12T04:01:06.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:06.430+0000 7fa94f2ad700 1 --2- 172.21.15.32:0/23418737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9480fcf90 0x7fa9480fd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:06.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:06.430+0000 7fa94f2ad700 1 --2- 172.21.15.32:0/23418737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9480fbce0 0x7fa9480fc0f0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:06.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:06.430+0000 7fa94f2ad700 1 -- 172.21.15.32:0/23418737 >> 172.21.15.32:0/23418737 conn(0x7fa9480f7440 msgr2=0x7fa9480f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:01:06.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:06.430+0000 7fa94f2ad700 1 -- 172.21.15.32:0/23418737 shutdown_connections 2024-05-12T04:01:06.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:06.430+0000 7fa94f2ad700 1 -- 172.21.15.32:0/23418737 wait complete. 2024-05-12T04:01:06.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:06.431+0000 7fa94f2ad700 1 Processor -- start 2024-05-12T04:01:06.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:06.431+0000 7fa94f2ad700 1 -- start start 2024-05-12T04:01:06.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:06.431+0000 7fa94f2ad700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9480fbce0 0x7fa948113f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:01:06.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:06.431+0000 7fa94f2ad700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9480fcf90 0x7fa948114440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:01:06.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:06.431+0000 7fa94f2ad700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa948114a40 con 0x7fa9480fcf90 2024-05-12T04:01:06.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:06.431+0000 7fa94f2ad700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa948115290 con 0x7fa9480fbce0 2024-05-12T04:01:06.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:06.431+0000 7fa94c848700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9480fcf90 0x7fa948114440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:01:06.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:06.431+0000 7fa94c848700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9480fcf90 0x7fa948114440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53026/0 (socket says 172.21.15.32:53026) 2024-05-12T04:01:06.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:06.431+0000 7fa94c848700 1 -- 172.21.15.32:0/1059603998 learned_addr learned my addr 172.21.15.32:0/1059603998 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:01:06.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:06.431+0000 7fa94d049700 1 --2- 172.21.15.32:0/1059603998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9480fbce0 0x7fa948113f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:01:06.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:06.432+0000 7fa94c848700 1 -- 172.21.15.32:0/1059603998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9480fbce0 msgr2=0x7fa948113f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:01:06.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:06.432+0000 7fa94c848700 1 --2- 172.21.15.32:0/1059603998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9480fbce0 0x7fa948113f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:06.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:06.432+0000 7fa94c848700 1 -- 172.21.15.32:0/1059603998 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa938005040 con 0x7fa9480fcf90 2024-05-12T04:01:06.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:06.432+0000 7fa94c848700 1 --2- 172.21.15.32:0/1059603998 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9480fcf90 0x7fa948114440 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa94400c8c0 tx=0x7fa9440044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:01:06.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:06.432+0000 7fa93dffb700 1 -- 172.21.15.32:0/1059603998 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa94400a760 con 0x7fa9480fcf90 2024-05-12T04:01:06.434 INFO:tasks.workunit.client.0.smithi032.stdout:5/110: write d4/f12 [3876027,21498] 0 2024-05-12T04:01:06.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:06.433+0000 7fa93dffb700 1 -- 172.21.15.32:0/1059603998 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa94400ad70 con 0x7fa9480fcf90 2024-05-12T04:01:06.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:06.433+0000 7fa93dffb700 1 -- 172.21.15.32:0/1059603998 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa9440113a0 con 0x7fa9480fcf90 2024-05-12T04:01:06.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:06.433+0000 7fa94f2ad700 1 -- 172.21.15.32:0/1059603998 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9481154f0 con 0x7fa9480fcf90 2024-05-12T04:01:06.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:06.433+0000 7fa94f2ad700 1 -- 172.21.15.32:0/1059603998 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa948115ab0 con 0x7fa9480fcf90 2024-05-12T04:01:06.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:06.434+0000 7fa94f2ad700 1 -- 172.21.15.32:0/1059603998 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa94804bed0 con 0x7fa9480fcf90 2024-05-12T04:01:06.437 INFO:tasks.workunit.client.1.smithi033.stdout:5/220: mknod d3/d12/d25/d26/c47 0 2024-05-12T04:01:06.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:06.438+0000 7fa93dffb700 1 -- 172.21.15.32:0/1059603998 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 25) v1 ==== 88772+0+0 (secure 0 0 0) 0x7fa944012400 con 0x7fa9480fcf90 2024-05-12T04:01:06.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:06.438+0000 7fa93dffb700 1 --2- 172.21.15.32:0/1059603998 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7fa93406af90 0x7fa93406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:01:06.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:06.438+0000 7fa93dffb700 1 -- 172.21.15.32:0/1059603998 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fa9440898f0 con 0x7fa9480fcf90 2024-05-12T04:01:06.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:06.439+0000 7fa94d049700 1 --2- 172.21.15.32:0/1059603998 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7fa93406af90 0x7fa93406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:01:06.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:06.439+0000 7fa94d049700 1 --2- 172.21.15.32:0/1059603998 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7fa93406af90 0x7fa93406d440 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa93800b900 tx=0x7fa938009e30 comp rx=0 tx=0).ready entity=mgr.24389 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:01:06.440 INFO:tasks.workunit.client.0.smithi032.stdout:6/91: creat d7/de/f1b x:0 0 0 2024-05-12T04:01:06.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:06.443+0000 7fa93dffb700 1 -- 172.21.15.32:0/1059603998 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181777 (secure 0 0 0) 0x7fa9440578e0 con 0x7fa9480fcf90 2024-05-12T04:01:06.446 INFO:tasks.workunit.client.1.smithi033.stdout:1/242: creat d11/d17/d27/f52 x:0 0 0 2024-05-12T04:01:06.452 INFO:tasks.workunit.client.1.smithi033.stdout:0/329: rename d6/db/d1f to d6/dc/d6e 0 2024-05-12T04:01:06.453 INFO:tasks.workunit.client.1.smithi033.stdout:0/330: readlink d6/l8 0 2024-05-12T04:01:06.454 INFO:tasks.workunit.client.0.smithi032.stdout:1/84: mkdir d1/d1e 0 2024-05-12T04:01:06.456 INFO:tasks.workunit.client.1.smithi033.stdout:4/308: getdents dd/d11/d15 0 2024-05-12T04:01:06.458 INFO:tasks.workunit.client.1.smithi033.stdout:7/440: stat de/d80 0 2024-05-12T04:01:06.460 INFO:tasks.workunit.client.0.smithi032.stdout:1/85: chown d1/d7 27941 1 2024-05-12T04:01:06.461 INFO:tasks.workunit.client.0.smithi032.stdout:1/86: chown d1/d16/d18 1098218 1 2024-05-12T04:01:06.461 INFO:tasks.workunit.client.0.smithi032.stdout:4/78: creat d0/d3/fd x:0 0 0 2024-05-12T04:01:06.463 INFO:tasks.workunit.client.1.smithi033.stdout:4/309: dwrite dd/d11/d15/d42/f4c [0,4194304] 0 2024-05-12T04:01:06.463 INFO:tasks.workunit.client.1.smithi033.stdout:2/282: creat dc/d17/f63 x:0 0 0 2024-05-12T04:01:06.465 INFO:tasks.workunit.client.0.smithi032.stdout:3/147: symlink d15/d24/d2a/l31 0 2024-05-12T04:01:06.466 INFO:tasks.workunit.client.1.smithi033.stdout:9/493: dwrite d2/dd/f11 [4194304,4194304] 0 2024-05-12T04:01:06.468 INFO:tasks.workunit.client.1.smithi033.stdout:6/247: getdents d4 0 2024-05-12T04:01:06.469 INFO:tasks.workunit.client.1.smithi033.stdout:9/494: write d2/d4/f28 [3956357,64577] 0 2024-05-12T04:01:06.469 INFO:tasks.workunit.client.0.smithi032.stdout:3/148: dread d15/f18 [0,4194304] 0 2024-05-12T04:01:06.473 INFO:tasks.workunit.client.1.smithi033.stdout:6/248: dread d4/d5/f35 [0,4194304] 0 2024-05-12T04:01:06.473 INFO:tasks.workunit.client.0.smithi032.stdout:3/149: chown d15/d20/d22/f23 60329 1 2024-05-12T04:01:06.473 INFO:tasks.workunit.client.0.smithi032.stdout:6/92: rename d7/l13 to d7/l1c 0 2024-05-12T04:01:06.477 INFO:tasks.workunit.client.1.smithi033.stdout:5/221: rename d3/d12/f1d to d3/d12/d29/f48 0 2024-05-12T04:01:06.478 INFO:tasks.workunit.client.1.smithi033.stdout:0/331: unlink d6/d11/l5a 0 2024-05-12T04:01:06.485 INFO:tasks.workunit.client.0.smithi032.stdout:4/79: symlink d0/d3/le 0 2024-05-12T04:01:06.487 INFO:tasks.workunit.client.1.smithi033.stdout:9/495: creat d2/d4/d2b/d49/d73/fb4 x:0 0 0 2024-05-12T04:01:06.488 INFO:tasks.workunit.client.1.smithi033.stdout:9/496: chown d2/d4/d2b/d6d/d4c 1442167 1 2024-05-12T04:01:06.489 INFO:tasks.workunit.client.0.smithi032.stdout:7/95: dwrite dd/f12 [0,4194304] 0 2024-05-12T04:01:06.490 INFO:tasks.workunit.client.1.smithi033.stdout:3/273: read db/d17/d25/d32/d3c/d44/f4a [375621,88673] 0 2024-05-12T04:01:06.490 INFO:tasks.workunit.client.1.smithi033.stdout:6/249: creat d4/f51 x:0 0 0 2024-05-12T04:01:06.492 INFO:tasks.workunit.client.0.smithi032.stdout:0/175: getdents df/d28 0 2024-05-12T04:01:06.493 INFO:tasks.workunit.client.0.smithi032.stdout:6/93: mknod d7/c1d 0 2024-05-12T04:01:06.495 INFO:tasks.workunit.client.0.smithi032.stdout:8/111: dwrite d0/f5 [0,4194304] 0 2024-05-12T04:01:06.497 INFO:tasks.workunit.client.1.smithi033.stdout:1/243: symlink d11/d17/d27/d36/d3f/d4a/l53 0 2024-05-12T04:01:06.501 INFO:tasks.workunit.client.0.smithi032.stdout:3/150: dwrite d15/d1b/f25 [4194304,4194304] 0 2024-05-12T04:01:06.503 INFO:tasks.workunit.client.0.smithi032.stdout:3/151: stat d15/f1c 0 2024-05-12T04:01:06.503 INFO:tasks.workunit.client.0.smithi032.stdout:4/80: dread d0/d3/fc [0,4194304] 0 2024-05-12T04:01:06.504 INFO:tasks.workunit.client.1.smithi033.stdout:7/441: rename de/df/d17/c68 to de/df/d60/d75/ca4 0 2024-05-12T04:01:06.504 INFO:tasks.workunit.client.1.smithi033.stdout:4/310: rename dd/d11/d15 to dd/d11/d15/d5a 22 2024-05-12T04:01:06.505 INFO:tasks.workunit.client.0.smithi032.stdout:4/81: stat d0/d3/fb 0 2024-05-12T04:01:06.506 INFO:tasks.workunit.client.0.smithi032.stdout:3/152: chown d15/d20/d22/f23 15491562 1 2024-05-12T04:01:06.509 INFO:tasks.workunit.client.0.smithi032.stdout:3/153: truncate d15/d20/d22/f23 646462 0 2024-05-12T04:01:06.509 INFO:tasks.workunit.client.0.smithi032.stdout:8/112: dread d0/f22 [0,4194304] 0 2024-05-12T04:01:06.513 INFO:tasks.workunit.client.0.smithi032.stdout:8/113: chown d0 122977 1 2024-05-12T04:01:06.514 INFO:tasks.workunit.client.0.smithi032.stdout:8/114: chown d0/d9 0 1 2024-05-12T04:01:06.514 INFO:tasks.workunit.client.0.smithi032.stdout:7/96: mknod dd/d19/c26 0 2024-05-12T04:01:06.514 INFO:tasks.workunit.client.1.smithi033.stdout:0/332: creat d6/d11/f6f x:0 0 0 2024-05-12T04:01:06.514 INFO:tasks.workunit.client.1.smithi033.stdout:0/333: write d6/dc/f18 [3598371,44271] 0 2024-05-12T04:01:06.515 INFO:tasks.workunit.client.1.smithi033.stdout:0/334: dread - d6/dc/f54 zero size 2024-05-12T04:01:06.516 INFO:tasks.workunit.client.0.smithi032.stdout:6/94: mknod d7/c1e 0 2024-05-12T04:01:06.518 INFO:tasks.workunit.client.1.smithi033.stdout:3/274: mknod db/d17/d25/d32/c61 0 2024-05-12T04:01:06.518 INFO:tasks.workunit.client.1.smithi033.stdout:3/275: dread - db/f5e zero size 2024-05-12T04:01:06.518 INFO:tasks.workunit.client.1.smithi033.stdout:3/276: readlink l5 0 2024-05-12T04:01:06.519 INFO:tasks.workunit.client.1.smithi033.stdout:3/277: chown db/l2a 1570961 1 2024-05-12T04:01:06.520 INFO:tasks.workunit.client.0.smithi032.stdout:0/176: rename df/d10/d1f/l23 to df/d30/l3f 0 2024-05-12T04:01:06.521 INFO:tasks.workunit.client.0.smithi032.stdout:8/115: dwrite d0/d9/f1c [0,4194304] 0 2024-05-12T04:01:06.523 INFO:tasks.workunit.client.1.smithi033.stdout:6/250: mknod d4/d1a/d4e/c52 0 2024-05-12T04:01:06.527 INFO:tasks.workunit.client.0.smithi032.stdout:1/87: link d1/c15 d1/d16/c1f 0 2024-05-12T04:01:06.527 INFO:tasks.workunit.client.0.smithi032.stdout:8/116: dread d0/f5 [0,4194304] 0 2024-05-12T04:01:06.528 INFO:tasks.workunit.client.1.smithi033.stdout:1/244: creat d11/d1c/f54 x:0 0 0 2024-05-12T04:01:06.529 INFO:tasks.workunit.client.1.smithi033.stdout:2/283: rename dc/d13/d1f/c22 to dc/d3b/c64 0 2024-05-12T04:01:06.529 INFO:tasks.workunit.client.0.smithi032.stdout:2/97: truncate d1/f12 2949357 0 2024-05-12T04:01:06.531 INFO:tasks.workunit.client.1.smithi033.stdout:2/284: truncate dc/d17/d27/d2f/f62 532845 0 2024-05-12T04:01:06.533 INFO:tasks.workunit.client.0.smithi032.stdout:2/98: dread d1/f3 [0,4194304] 0 2024-05-12T04:01:06.536 INFO:tasks.workunit.client.0.smithi032.stdout:3/154: dread f12 [0,4194304] 0 2024-05-12T04:01:06.537 INFO:tasks.workunit.client.0.smithi032.stdout:3/155: chown d15/d20/d22/f2c 183914 1 2024-05-12T04:01:06.539 INFO:tasks.workunit.client.0.smithi032.stdout:3/156: chown d15/d1b 6216328 1 2024-05-12T04:01:06.539 INFO:tasks.workunit.client.0.smithi032.stdout:7/97: mknod dd/c27 0 2024-05-12T04:01:06.546 INFO:tasks.workunit.client.1.smithi033.stdout:0/335: rmdir d6/dc/d58 39 2024-05-12T04:01:06.547 INFO:tasks.workunit.client.1.smithi033.stdout:0/336: dread - d6/db/d2c/f62 zero size 2024-05-12T04:01:06.548 INFO:tasks.workunit.client.1.smithi033.stdout:0/337: write f2 [1519425,27899] 0 2024-05-12T04:01:06.552 INFO:tasks.workunit.client.1.smithi033.stdout:6/251: creat d4/d1a/d4e/f53 x:0 0 0 2024-05-12T04:01:06.553 INFO:tasks.workunit.client.1.smithi033.stdout:6/252: stat d4/f30 0 2024-05-12T04:01:06.555 INFO:tasks.workunit.client.1.smithi033.stdout:1/245: stat d11/c2f 0 2024-05-12T04:01:06.555 INFO:tasks.workunit.client.1.smithi033.stdout:1/246: stat d11/f30 0 2024-05-12T04:01:06.558 INFO:tasks.workunit.client.1.smithi033.stdout:9/497: rename d2/d4/d29/d32/d9b to d2/d4/d2b/d6d/db5 0 2024-05-12T04:01:06.562 INFO:tasks.workunit.client.1.smithi033.stdout:2/285: mkdir dc/d13/d65 0 2024-05-12T04:01:06.568 INFO:tasks.workunit.client.0.smithi032.stdout:4/82: unlink d0/d3/fc 0 2024-05-12T04:01:06.568 INFO:tasks.workunit.client.0.smithi032.stdout:4/83: fsync d0/d3/f4 0 2024-05-12T04:01:06.568 INFO:tasks.workunit.client.1.smithi033.stdout:6/253: dread d4/d1a/f26 [4194304,4194304] 0 2024-05-12T04:01:06.568 INFO:tasks.workunit.client.1.smithi033.stdout:5/222: truncate d3/fb 1246684 0 2024-05-12T04:01:06.568 INFO:tasks.workunit.client.1.smithi033.stdout:4/311: mkdir dd/d11/d5b 0 2024-05-12T04:01:06.572 INFO:tasks.workunit.client.0.smithi032.stdout:3/157: mknod d15/d20/c32 0 2024-05-12T04:01:06.574 INFO:tasks.workunit.client.0.smithi032.stdout:7/98: rename dd/f14 to dd/d1b/d1e/f28 0 2024-05-12T04:01:06.580 INFO:tasks.workunit.client.0.smithi032.stdout:6/95: dwrite d7/fc [0,4194304] 0 2024-05-12T04:01:06.582 INFO:tasks.workunit.client.1.smithi033.stdout:8/210: sync 2024-05-12T04:01:06.583 INFO:tasks.workunit.client.1.smithi033.stdout:8/211: chown d1/d16/c1f 0 1 2024-05-12T04:01:06.584 INFO:tasks.workunit.client.0.smithi032.stdout:7/99: dread dd/f1a [0,4194304] 0 2024-05-12T04:01:06.585 INFO:tasks.workunit.client.0.smithi032.stdout:7/100: stat dd/l17 0 2024-05-12T04:01:06.587 INFO:tasks.workunit.client.0.smithi032.stdout:7/101: chown dd/l10 18931 1 2024-05-12T04:01:06.593 INFO:tasks.workunit.client.1.smithi033.stdout:7/442: truncate f3 3626919 0 2024-05-12T04:01:06.595 INFO:tasks.workunit.client.1.smithi033.stdout:1/247: mknod d11/d1c/c55 0 2024-05-12T04:01:06.597 INFO:tasks.workunit.client.1.smithi033.stdout:9/498: mknod d2/d4/d2b/d49/d73/cb6 0 2024-05-12T04:01:06.600 INFO:tasks.workunit.client.1.smithi033.stdout:2/286: mknod dc/d13/c66 0 2024-05-12T04:01:06.601 INFO:tasks.workunit.client.0.smithi032.stdout:0/177: truncate df/d10/f13 8638985 0 2024-05-12T04:01:06.601 INFO:tasks.workunit.client.0.smithi032.stdout:0/178: dread - df/d10/f33 zero size 2024-05-12T04:01:06.602 INFO:tasks.workunit.client.0.smithi032.stdout:0/179: dread - df/d10/f33 zero size 2024-05-12T04:01:06.604 INFO:tasks.workunit.client.1.smithi033.stdout:6/254: rename d4/d5/c15 to d4/d1a/d3c/c54 0 2024-05-12T04:01:06.605 INFO:tasks.workunit.client.1.smithi033.stdout:2/287: dwrite dc/d13/d1f/f24 [0,4194304] 0 2024-05-12T04:01:06.608 INFO:tasks.workunit.client.0.smithi032.stdout:1/88: truncate d1/d7/f11 837971 0 2024-05-12T04:01:06.614 INFO:tasks.workunit.client.1.smithi033.stdout:6/255: dwrite d4/f30 [0,4194304] 0 2024-05-12T04:01:06.614 INFO:tasks.workunit.client.0.smithi032.stdout:4/84: unlink d0/l5 0 2024-05-12T04:01:06.614 INFO:tasks.workunit.client.0.smithi032.stdout:0/180: dwrite df/d10/d1f/f38 [0,4194304] 0 2024-05-12T04:01:06.615 INFO:tasks.workunit.client.1.smithi033.stdout:7/443: dread de/df/d17/f51 [0,4194304] 0 2024-05-12T04:01:06.615 INFO:tasks.workunit.client.1.smithi033.stdout:6/256: chown d4/d5/l31 65758510 1 2024-05-12T04:01:06.616 INFO:tasks.workunit.client.0.smithi032.stdout:8/117: truncate d0/d9/f1c 1015906 0 2024-05-12T04:01:06.616 INFO:tasks.workunit.client.1.smithi033.stdout:7/444: read - de/df/d17/d1e/f4c zero size 2024-05-12T04:01:06.617 INFO:tasks.workunit.client.1.smithi033.stdout:7/445: stat de/d20 0 2024-05-12T04:01:06.617 INFO:tasks.workunit.client.1.smithi033.stdout:6/257: truncate d4/d5/f20 195211 0 2024-05-12T04:01:06.618 INFO:tasks.workunit.client.0.smithi032.stdout:2/99: mknod d1/d13/d25/c26 0 2024-05-12T04:01:06.625 INFO:tasks.workunit.client.1.smithi033.stdout:1/248: creat d11/d17/d27/f56 x:0 0 0 2024-05-12T04:01:06.625 INFO:tasks.workunit.client.1.smithi033.stdout:4/312: write dd/f4b [4263326,65336] 0 2024-05-12T04:01:06.626 INFO:tasks.workunit.client.1.smithi033.stdout:1/249: stat d11/d1c/c1b 0 2024-05-12T04:01:06.629 INFO:tasks.workunit.client.1.smithi033.stdout:9/499: mkdir d2/d4/d29/d32/db7 0 2024-05-12T04:01:06.630 INFO:tasks.workunit.client.1.smithi033.stdout:9/500: write d2/d70/d7b/f9f [562927,117477] 0 2024-05-12T04:01:06.639 INFO:tasks.workunit.client.1.smithi033.stdout:2/288: mkdir dc/d17/d32/d67 0 2024-05-12T04:01:06.646 INFO:tasks.workunit.client.0.smithi032.stdout:8/118: fdatasync d0/f22 0 2024-05-12T04:01:06.647 INFO:tasks.workunit.client.1.smithi033.stdout:3/278: getdents db/d17/d25/d32/d3c/d44 0 2024-05-12T04:01:06.648 INFO:tasks.workunit.client.1.smithi033.stdout:3/279: write f6 [2881809,101912] 0 2024-05-12T04:01:06.650 INFO:tasks.workunit.client.0.smithi032.stdout:3/158: mkdir d15/d33 0 2024-05-12T04:01:06.651 INFO:tasks.workunit.client.1.smithi033.stdout:6/258: unlink d4/d5/f35 0 2024-05-12T04:01:06.652 INFO:tasks.workunit.client.1.smithi033.stdout:6/259: write d4/f51 [693017,117684] 0 2024-05-12T04:01:06.652 INFO:tasks.workunit.client.1.smithi033.stdout:6/260: readlink d4/d1a/l1b 0 2024-05-12T04:01:06.653 INFO:tasks.workunit.client.1.smithi033.stdout:6/261: chown d4/f1f 587491 1 2024-05-12T04:01:06.653 INFO:tasks.workunit.client.1.smithi033.stdout:7/446: mknod de/df/d60/d75/ca5 0 2024-05-12T04:01:06.654 INFO:tasks.workunit.client.0.smithi032.stdout:6/96: write d7/d9/f17 [706413,21607] 0 2024-05-12T04:01:06.654 INFO:tasks.workunit.client.1.smithi033.stdout:7/447: dread - de/df/d7a/f9f zero size 2024-05-12T04:01:06.654 INFO:tasks.workunit.client.1.smithi033.stdout:7/448: chown c9 41 1 2024-05-12T04:01:06.655 INFO:tasks.workunit.client.1.smithi033.stdout:7/449: write de/df/d7a/f8b [4263703,6563] 0 2024-05-12T04:01:06.656 INFO:tasks.workunit.client.1.smithi033.stdout:7/450: stat de/d2f/d70/f94 0 2024-05-12T04:01:06.657 INFO:tasks.workunit.client.1.smithi033.stdout:7/451: write de/d2b/d6c/f72 [1725359,10344] 0 2024-05-12T04:01:06.658 INFO:tasks.workunit.client.0.smithi032.stdout:8/119: dwrite d0/f22 [0,4194304] 0 2024-05-12T04:01:06.661 INFO:tasks.workunit.client.0.smithi032.stdout:3/159: dread d15/f1c [0,4194304] 0 2024-05-12T04:01:06.662 INFO:tasks.workunit.client.1.smithi033.stdout:1/250: unlink d11/d17/d27/d36/d3f/f4e 0 2024-05-12T04:01:06.668 INFO:tasks.workunit.client.1.smithi033.stdout:9/501: symlink d2/dd/d39/lb8 0 2024-05-12T04:01:06.670 INFO:tasks.workunit.client.0.smithi032.stdout:0/181: mknod df/c40 0 2024-05-12T04:01:06.670 INFO:tasks.workunit.client.1.smithi033.stdout:8/212: truncate d1/d16/f19 3885798 0 2024-05-12T04:01:06.674 INFO:tasks.workunit.client.1.smithi033.stdout:0/338: getdents d6/db 0 2024-05-12T04:01:06.675 INFO:tasks.workunit.client.0.smithi032.stdout:4/85: dwrite d0/d3/fb [0,4194304] 0 2024-05-12T04:01:06.681 INFO:tasks.workunit.client.1.smithi033.stdout:3/280: dread db/d17/f36 [0,4194304] 0 2024-05-12T04:01:06.682 INFO:tasks.workunit.client.1.smithi033.stdout:3/281: chown db/d17/d25/d32/d3c 14344 1 2024-05-12T04:01:06.683 INFO:tasks.workunit.client.1.smithi033.stdout:3/282: truncate db/d59/f5b 483059 0 2024-05-12T04:01:06.683 INFO:tasks.workunit.client.1.smithi033.stdout:3/283: readlink db/d17/d3a/l4e 0 2024-05-12T04:01:06.685 INFO:tasks.workunit.client.0.smithi032.stdout:8/120: rmdir d0/d9/d13 39 2024-05-12T04:01:06.692 INFO:tasks.workunit.client.1.smithi033.stdout:5/223: sync 2024-05-12T04:01:06.693 INFO:tasks.workunit.client.1.smithi033.stdout:1/251: readlink d11/d17/d38/l3b 0 2024-05-12T04:01:06.693 INFO:tasks.workunit.client.1.smithi033.stdout:5/224: readlink d3/d15/d16/l3c 0 2024-05-12T04:01:06.694 INFO:tasks.workunit.client.1.smithi033.stdout:9/502: creat d2/d4/d2b/d49/fb9 x:0 0 0 2024-05-12T04:01:06.694 INFO:tasks.workunit.client.1.smithi033.stdout:5/225: chown d3/d12/d29/d2b/c32 8451 1 2024-05-12T04:01:06.695 INFO:tasks.workunit.client.0.smithi032.stdout:9/91: sync 2024-05-12T04:01:06.695 INFO:tasks.workunit.client.1.smithi033.stdout:5/226: write d3/fc [2073039,44745] 0 2024-05-12T04:01:06.696 INFO:tasks.workunit.client.0.smithi032.stdout:9/92: stat d0/d3/d6/d7/da/d10/f12 0 2024-05-12T04:01:06.696 INFO:tasks.workunit.client.0.smithi032.stdout:2/100: rename d1/d13/f20 to d1/f27 0 2024-05-12T04:01:06.696 INFO:tasks.workunit.client.1.smithi033.stdout:5/227: chown d3/d12/l19 9265 1 2024-05-12T04:01:06.697 INFO:tasks.workunit.client.1.smithi033.stdout:8/213: mknod d1/d16/d20/c45 0 2024-05-12T04:01:06.698 INFO:tasks.workunit.client.0.smithi032.stdout:6/97: symlink d7/de/d12/d1a/l1f 0 2024-05-12T04:01:06.699 INFO:tasks.workunit.client.1.smithi033.stdout:2/289: mkdir dc/d17/d27/d68 0 2024-05-12T04:01:06.701 INFO:tasks.workunit.client.1.smithi033.stdout:0/339: mknod d6/dc/d39/c70 0 2024-05-12T04:01:06.703 INFO:tasks.workunit.client.0.smithi032.stdout:1/89: write d1/d7/f11 [112534,93762] 0 2024-05-12T04:01:06.703 INFO:tasks.workunit.client.1.smithi033.stdout:3/284: write db/d17/d25/f52 [7051787,99478] 0 2024-05-12T04:01:06.707 INFO:tasks.workunit.client.1.smithi033.stdout:7/452: mkdir de/d2b/d6c/da0/da6 0 2024-05-12T04:01:06.711 INFO:tasks.workunit.client.1.smithi033.stdout:1/252: fsync d11/d17/f1d 0 2024-05-12T04:01:06.712 INFO:tasks.workunit.client.1.smithi033.stdout:1/253: stat d11/d17/d27/d36/d46/d4c 0 2024-05-12T04:01:06.714 INFO:tasks.workunit.client.0.smithi032.stdout:1/90: dwrite d1/f3 [0,4194304] 0 2024-05-12T04:01:06.714 INFO:tasks.workunit.client.1.smithi033.stdout:9/503: creat d2/d4/d29/d3c/d52/fba x:0 0 0 2024-05-12T04:01:06.718 INFO:tasks.workunit.client.1.smithi033.stdout:9/504: write d2/d4/d29/d3c/f9a [2571498,107422] 0 2024-05-12T04:01:06.718 INFO:tasks.workunit.client.1.smithi033.stdout:9/505: write d2/dd/d79/fae [575448,44691] 0 2024-05-12T04:01:06.718 INFO:tasks.workunit.client.0.smithi032.stdout:3/160: truncate d15/d1b/f25 7139041 0 2024-05-12T04:01:06.719 INFO:tasks.workunit.client.0.smithi032.stdout:4/86: dwrite d0/d3/fb [4194304,4194304] 0 2024-05-12T04:01:06.719 INFO:tasks.workunit.client.1.smithi033.stdout:5/228: rename d3/f1c to d3/d12/d25/d26/d2e/f49 0 2024-05-12T04:01:06.720 INFO:tasks.workunit.client.0.smithi032.stdout:0/182: write df/d10/d1f/f2d [610237,100018] 0 2024-05-12T04:01:06.722 INFO:tasks.workunit.client.0.smithi032.stdout:2/101: rename d1/f27 to d1/d13/d25/f28 0 2024-05-12T04:01:06.724 INFO:tasks.workunit.client.1.smithi033.stdout:2/290: creat dc/d13/d1f/f69 x:0 0 0 2024-05-12T04:01:06.725 INFO:tasks.workunit.client.1.smithi033.stdout:2/291: write f6 [4341757,36037] 0 2024-05-12T04:01:06.728 INFO:tasks.workunit.client.0.smithi032.stdout:6/98: dread f3 [0,4194304] 0 2024-05-12T04:01:06.730 INFO:tasks.workunit.client.0.smithi032.stdout:8/121: creat d0/d9/d13/f24 x:0 0 0 2024-05-12T04:01:06.735 INFO:tasks.workunit.client.0.smithi032.stdout:4/87: stat d0/fa 0 2024-05-12T04:01:06.736 INFO:tasks.workunit.client.1.smithi033.stdout:7/453: dwrite de/d2b/d6c/f3d [0,4194304] 0 2024-05-12T04:01:06.737 INFO:tasks.workunit.client.0.smithi032.stdout:5/111: sync 2024-05-12T04:01:06.737 INFO:tasks.workunit.client.1.smithi033.stdout:9/506: symlink d2/d4/d29/d3c/lbb 0 2024-05-12T04:01:06.738 INFO:tasks.workunit.client.1.smithi033.stdout:9/507: chown d2/dd/d79/fae 209325 1 2024-05-12T04:01:06.739 INFO:tasks.workunit.client.0.smithi032.stdout:7/102: sync 2024-05-12T04:01:06.741 INFO:tasks.workunit.client.1.smithi033.stdout:6/262: link d4/d5/l34 d4/d1a/d4e/l55 0 2024-05-12T04:01:06.741 INFO:tasks.workunit.client.0.smithi032.stdout:9/93: creat d0/d3/d6/d7/da/d10/d13/f1d x:0 0 0 2024-05-12T04:01:06.743 INFO:tasks.workunit.client.0.smithi032.stdout:6/99: mkdir d7/de/d12/d1a/d20 0 2024-05-12T04:01:06.744 INFO:tasks.workunit.client.1.smithi033.stdout:2/292: dwrite dc/d13/d1c/f4f [0,4194304] 0 2024-05-12T04:01:06.744 INFO:tasks.workunit.client.1.smithi033.stdout:0/340: link d6/d11/d57/f60 d6/d11/d57/f71 0 2024-05-12T04:01:06.746 INFO:tasks.workunit.client.0.smithi032.stdout:6/100: truncate d7/de/d12/f16 556360 0 2024-05-12T04:01:06.748 INFO:tasks.workunit.client.1.smithi033.stdout:6/263: dwrite d4/f16 [0,4194304] 0 2024-05-12T04:01:06.750 INFO:tasks.workunit.client.1.smithi033.stdout:6/264: write d4/f51 [967005,28387] 0 2024-05-12T04:01:06.754 INFO:tasks.workunit.client.1.smithi033.stdout:2/293: dwrite dc/d13/d1c/f25 [0,4194304] 0 2024-05-12T04:01:06.756 INFO:tasks.workunit.client.0.smithi032.stdout:9/94: dwrite d0/d3/d6/d7/da/fd [0,4194304] 0 2024-05-12T04:01:06.756 INFO:tasks.workunit.client.1.smithi033.stdout:2/294: chown dc/d3b/l45 1938 1 2024-05-12T04:01:06.757 INFO:tasks.workunit.client.1.smithi033.stdout:2/295: chown dc/d17/d27/c2b 9 1 2024-05-12T04:01:06.760 INFO:tasks.workunit.client.1.smithi033.stdout:7/454: write de/df/d60/f76 [130799,115214] 0 2024-05-12T04:01:06.762 INFO:tasks.workunit.client.0.smithi032.stdout:0/183: creat df/d30/d36/f41 x:0 0 0 2024-05-12T04:01:06.766 INFO:tasks.workunit.client.1.smithi033.stdout:4/313: sync 2024-05-12T04:01:06.767 INFO:tasks.workunit.client.1.smithi033.stdout:4/314: write dd/f4b [5845796,119161] 0 2024-05-12T04:01:06.768 INFO:tasks.workunit.client.1.smithi033.stdout:0/341: rmdir d6/dc/d39 39 2024-05-12T04:01:06.769 INFO:tasks.workunit.client.0.smithi032.stdout:4/88: dwrite d0/d3/fb [0,4194304] 0 2024-05-12T04:01:06.769 INFO:tasks.workunit.client.1.smithi033.stdout:0/342: readlink d6/db/l30 0 2024-05-12T04:01:06.770 INFO:tasks.workunit.client.1.smithi033.stdout:7/455: dwrite de/df/f29 [0,4194304] 0 2024-05-12T04:01:06.770 INFO:tasks.workunit.client.1.smithi033.stdout:6/265: dread d4/d5/f10 [0,4194304] 0 2024-05-12T04:01:06.772 INFO:tasks.workunit.client.1.smithi033.stdout:3/285: link db/l29 db/d17/d4d/l62 0 2024-05-12T04:01:06.772 INFO:tasks.workunit.client.1.smithi033.stdout:3/286: chown db/d17/c1d 26796281 1 2024-05-12T04:01:06.777 INFO:tasks.workunit.client.0.smithi032.stdout:5/112: symlink d4/d7/l24 0 2024-05-12T04:01:06.780 INFO:tasks.workunit.client.0.smithi032.stdout:2/102: truncate d1/fc 942592 0 2024-05-12T04:01:06.782 INFO:tasks.workunit.client.0.smithi032.stdout:5/113: rename d4 to d4/d7/d14/d25 22 2024-05-12T04:01:06.786 INFO:tasks.workunit.client.1.smithi033.stdout:8/214: rename d1/d4/d2e/d3e to d1/d4/d46 0 2024-05-12T04:01:06.787 INFO:tasks.workunit.client.1.smithi033.stdout:2/296: creat dc/d17/d27/d2f/f6a x:0 0 0 2024-05-12T04:01:06.787 INFO:tasks.workunit.client.1.smithi033.stdout:7/456: fdatasync de/d2b/d6c/f3d 0 2024-05-12T04:01:06.789 INFO:tasks.workunit.client.1.smithi033.stdout:2/297: write dc/d17/d27/d2f/f6a [501298,129987] 0 2024-05-12T04:01:06.789 INFO:tasks.workunit.client.0.smithi032.stdout:7/103: write dd/d1b/d1e/f28 [5187241,78205] 0 2024-05-12T04:01:06.790 INFO:tasks.workunit.client.0.smithi032.stdout:3/161: dwrite d15/f17 [0,4194304] 0 2024-05-12T04:01:06.793 INFO:tasks.workunit.client.1.smithi033.stdout:6/266: creat d4/f56 x:0 0 0 2024-05-12T04:01:06.794 INFO:tasks.workunit.client.1.smithi033.stdout:6/267: fdatasync d4/d1a/f42 0 2024-05-12T04:01:06.794 INFO:tasks.workunit.client.0.smithi032.stdout:1/91: rmdir d1/d1e 0 2024-05-12T04:01:06.795 INFO:tasks.workunit.client.1.smithi033.stdout:6/268: write d4/d1a/f42 [789019,56553] 0 2024-05-12T04:01:06.796 INFO:tasks.workunit.client.1.smithi033.stdout:6/269: write d4/d1a/f47 [2818374,105698] 0 2024-05-12T04:01:06.799 INFO:tasks.workunit.client.1.smithi033.stdout:3/287: creat db/d17/d3a/f63 x:0 0 0 2024-05-12T04:01:06.799 INFO:tasks.workunit.client.0.smithi032.stdout:3/162: fsync d15/d20/d22/f23 0 2024-05-12T04:01:06.799 INFO:tasks.workunit.client.0.smithi032.stdout:7/104: chown dd/c27 485 1 2024-05-12T04:01:06.800 INFO:tasks.workunit.client.0.smithi032.stdout:9/95: link d0/d3/ff d0/d3/d6/d7/da/d17/f1e 0 2024-05-12T04:01:06.802 INFO:tasks.workunit.client.1.smithi033.stdout:1/254: getdents d11/d1c 0 2024-05-12T04:01:06.802 INFO:tasks.workunit.client.0.smithi032.stdout:0/184: fsync df/d30/d36/f41 0 2024-05-12T04:01:06.804 INFO:tasks.workunit.client.1.smithi033.stdout:1/255: chown d11/d17/d27/d36/d3f/d4a/l53 128 1 2024-05-12T04:01:06.805 INFO:tasks.workunit.client.1.smithi033.stdout:6/270: dwrite d4/f29 [0,4194304] 0 2024-05-12T04:01:06.805 INFO:tasks.workunit.client.0.smithi032.stdout:1/92: symlink d1/d7/l20 0 2024-05-12T04:01:06.806 INFO:tasks.workunit.client.1.smithi033.stdout:6/271: write d4/d1a/f42 [1389407,28705] 0 2024-05-12T04:01:06.808 INFO:tasks.workunit.client.1.smithi033.stdout:9/508: creat d2/dd/fbc x:0 0 0 2024-05-12T04:01:06.808 INFO:tasks.workunit.client.0.smithi032.stdout:4/89: dwrite d0/fa [0,4194304] 0 2024-05-12T04:01:06.809 INFO:tasks.workunit.client.1.smithi033.stdout:6/272: dread - d4/d1a/d4e/d4f/f50 zero size 2024-05-12T04:01:06.810 INFO:tasks.workunit.client.0.smithi032.stdout:1/93: fdatasync d1/d16/d18/f1a 0 2024-05-12T04:01:06.810 INFO:tasks.workunit.client.0.smithi032.stdout:8/122: getdents d0/d7 0 2024-05-12T04:01:06.812 INFO:tasks.workunit.client.0.smithi032.stdout:4/90: write d0/fa [3254101,45224] 0 2024-05-12T04:01:06.813 INFO:tasks.workunit.client.0.smithi032.stdout:1/94: chown d1/d7/c8 885175 1 2024-05-12T04:01:06.815 INFO:tasks.workunit.client.0.smithi032.stdout:1/95: chown d1/d7/l20 4041 1 2024-05-12T04:01:06.815 INFO:tasks.workunit.client.1.smithi033.stdout:7/457: creat de/df/d60/d75/fa7 x:0 0 0 2024-05-12T04:01:06.816 INFO:tasks.workunit.client.0.smithi032.stdout:1/96: chown d1/d7/ce 87558 1 2024-05-12T04:01:06.817 INFO:tasks.workunit.client.1.smithi033.stdout:6/273: dwrite d4/d1a/f47 [0,4194304] 0 2024-05-12T04:01:06.817 INFO:tasks.workunit.client.1.smithi033.stdout:4/315: symlink dd/l5c 0 2024-05-12T04:01:06.818 INFO:tasks.workunit.client.1.smithi033.stdout:7/458: write de/df/d17/d1b/f89 [3974550,64017] 0 2024-05-12T04:01:06.821 INFO:tasks.workunit.client.1.smithi033.stdout:0/343: rename d6/d11/d57/c37 to d6/dc/d58/d6a/c72 0 2024-05-12T04:01:06.821 INFO:tasks.workunit.client.0.smithi032.stdout:9/96: rmdir d0/d3/d6/d7/da/d17 39 2024-05-12T04:01:06.830 INFO:tasks.workunit.client.0.smithi032.stdout:7/105: rename dd/d1b/l21 to dd/d1b/d1e/l29 0 2024-05-12T04:01:06.831 INFO:tasks.workunit.client.1.smithi033.stdout:3/288: creat db/d17/f64 x:0 0 0 2024-05-12T04:01:06.831 INFO:tasks.workunit.client.0.smithi032.stdout:6/101: truncate d7/d9/f17 1036936 0 2024-05-12T04:01:06.833 INFO:tasks.workunit.client.0.smithi032.stdout:5/114: write d4/f20 [1036971,65873] 0 2024-05-12T04:01:06.838 INFO:tasks.workunit.client.1.smithi033.stdout:1/256: dread d11/d17/f35 [0,4194304] 0 2024-05-12T04:01:06.840 INFO:tasks.workunit.client.1.smithi033.stdout:5/229: sync 2024-05-12T04:01:06.840 INFO:tasks.workunit.client.1.smithi033.stdout:5/230: readlink d3/d12/d25/d26/l2a 0 2024-05-12T04:01:06.841 INFO:tasks.workunit.client.1.smithi033.stdout:5/231: chown d3/d15/d16/f20 111 1 2024-05-12T04:01:06.841 INFO:tasks.workunit.client.0.smithi032.stdout:4/91: mkdir d0/d3/df 0 2024-05-12T04:01:06.842 INFO:tasks.workunit.client.1.smithi033.stdout:2/298: write dc/d13/f1b [526479,32353] 0 2024-05-12T04:01:06.843 INFO:tasks.workunit.client.0.smithi032.stdout:2/103: write d1/f12 [1434101,24739] 0 2024-05-12T04:01:06.843 INFO:tasks.workunit.client.0.smithi032.stdout:0/185: mknod df/d28/c42 0 2024-05-12T04:01:06.845 INFO:tasks.workunit.client.0.smithi032.stdout:4/92: chown d0/d3 55 1 2024-05-12T04:01:06.852 INFO:tasks.workunit.client.0.smithi032.stdout:6/102: mknod d7/c21 0 2024-05-12T04:01:06.853 INFO:tasks.workunit.client.0.smithi032.stdout:7/106: creat dd/d19/f2a x:0 0 0 2024-05-12T04:01:06.854 INFO:tasks.workunit.client.1.smithi033.stdout:4/316: write dd/f36 [486730,82471] 0 2024-05-12T04:01:06.862 INFO:tasks.workunit.client.0.smithi032.stdout:5/115: unlink d4/d7/f19 0 2024-05-12T04:01:06.862 INFO:tasks.workunit.client.0.smithi032.stdout:6/103: write d7/fc [3543723,119300] 0 2024-05-12T04:01:06.863 INFO:tasks.workunit.client.0.smithi032.stdout:8/123: mkdir d0/d9/d13/d21/d25 0 2024-05-12T04:01:06.865 INFO:tasks.workunit.client.1.smithi033.stdout:7/459: unlink de/d20/d64/c92 0 2024-05-12T04:01:06.865 INFO:tasks.workunit.client.0.smithi032.stdout:0/186: dwrite df/d10/d1f/f2d [0,4194304] 0 2024-05-12T04:01:06.867 INFO:tasks.workunit.client.1.smithi033.stdout:6/274: mkdir d4/d43/d57 0 2024-05-12T04:01:06.871 INFO:tasks.workunit.client.0.smithi032.stdout:0/187: truncate df/d10/d1f/f38 4764053 0 2024-05-12T04:01:06.871 INFO:tasks.workunit.client.0.smithi032.stdout:4/93: rename d0/d3/l6 to d0/d3/df/l10 0 2024-05-12T04:01:06.871 INFO:tasks.workunit.client.0.smithi032.stdout:7/107: dwrite dd/f13 [0,4194304] 0 2024-05-12T04:01:06.872 INFO:tasks.workunit.client.1.smithi033.stdout:3/289: mkdir db/d17/d3e/d65 0 2024-05-12T04:01:06.878 INFO:tasks.workunit.client.1.smithi033.stdout:1/257: rmdir d11/d17/d27/d36/d3f 39 2024-05-12T04:01:06.881 INFO:tasks.workunit.client.1.smithi033.stdout:3/290: dwrite db/d17/d25/d32/f5d [0,4194304] 0 2024-05-12T04:01:06.882 INFO:tasks.workunit.client.1.smithi033.stdout:8/215: rmdir d1/d16/d29/d3d 0 2024-05-12T04:01:06.887 INFO:tasks.workunit.client.1.smithi033.stdout:5/232: rename d3/f10 to d3/d12/d25/f4a 0 2024-05-12T04:01:06.896 INFO:tasks.workunit.client.1.smithi033.stdout:9/509: mknod d2/d4/d2b/d33/cbd 0 2024-05-12T04:01:06.897 INFO:tasks.workunit.client.0.smithi032.stdout:8/124: mkdir d0/d9/d13/d26 0 2024-05-12T04:01:06.897 INFO:tasks.workunit.client.0.smithi032.stdout:3/163: dwrite d15/d1b/f25 [0,4194304] 0 2024-05-12T04:01:06.899 INFO:tasks.workunit.client.0.smithi032.stdout:5/116: mkdir d4/d7/d14/d26 0 2024-05-12T04:01:06.899 INFO:tasks.workunit.client.0.smithi032.stdout:5/117: chown d4/db 22610898 1 2024-05-12T04:01:06.900 INFO:tasks.workunit.client.1.smithi033.stdout:4/317: mknod dd/d11/d31/d38/d57/c5d 0 2024-05-12T04:01:06.902 INFO:tasks.workunit.client.0.smithi032.stdout:6/104: creat d7/de/d12/f22 x:0 0 0 2024-05-12T04:01:06.905 INFO:tasks.workunit.client.1.smithi033.stdout:7/460: creat de/d2b/d6c/da0/fa8 x:0 0 0 2024-05-12T04:01:06.906 INFO:tasks.workunit.client.1.smithi033.stdout:7/461: truncate de/d80/f98 951410 0 2024-05-12T04:01:06.907 INFO:tasks.workunit.client.1.smithi033.stdout:7/462: stat de/df/f48 0 2024-05-12T04:01:06.907 INFO:tasks.workunit.client.1.smithi033.stdout:7/463: write de/df/d17/f34 [414274,65629] 0 2024-05-12T04:01:06.908 INFO:tasks.workunit.client.1.smithi033.stdout:6/275: symlink d4/d1a/l58 0 2024-05-12T04:01:06.909 INFO:tasks.workunit.client.0.smithi032.stdout:0/188: creat df/d30/d36/f43 x:0 0 0 2024-05-12T04:01:06.915 INFO:tasks.workunit.client.0.smithi032.stdout:4/94: chown d0/d3/df/l10 0 1 2024-05-12T04:01:06.915 INFO:tasks.workunit.client.0.smithi032.stdout:6/105: dwrite d7/de/f19 [0,4194304] 0 2024-05-12T04:01:06.916 INFO:tasks.workunit.client.0.smithi032.stdout:7/108: mknod dd/d19/c2b 0 2024-05-12T04:01:06.922 INFO:tasks.workunit.client.0.smithi032.stdout:7/109: write dd/f12 [211307,107587] 0 2024-05-12T04:01:06.922 INFO:tasks.workunit.client.1.smithi033.stdout:1/258: creat d11/d17/d27/f57 x:0 0 0 2024-05-12T04:01:06.922 INFO:tasks.workunit.client.1.smithi033.stdout:8/216: fsync d1/f13 0 2024-05-12T04:01:06.923 INFO:tasks.workunit.client.1.smithi033.stdout:8/217: readlink d1/l9 0 2024-05-12T04:01:06.923 INFO:tasks.workunit.client.0.smithi032.stdout:3/164: creat d15/d1b/f34 x:0 0 0 2024-05-12T04:01:06.927 INFO:tasks.workunit.client.1.smithi033.stdout:9/510: creat d2/d70/d7b/fbe x:0 0 0 2024-05-12T04:01:06.932 INFO:tasks.workunit.client.0.smithi032.stdout:2/104: dwrite d1/fc [0,4194304] 0 2024-05-12T04:01:06.933 INFO:tasks.workunit.client.0.smithi032.stdout:7/110: dread f2 [0,4194304] 0 2024-05-12T04:01:06.933 INFO:tasks.workunit.client.0.smithi032.stdout:0/189: write df/d22/f39 [616352,9644] 0 2024-05-12T04:01:06.935 INFO:tasks.workunit.client.1.smithi033.stdout:6/276: mknod d4/d5/c59 0 2024-05-12T04:01:06.936 INFO:tasks.workunit.client.0.smithi032.stdout:7/111: chown dd/f25 12340 1 2024-05-12T04:01:06.936 INFO:tasks.workunit.client.0.smithi032.stdout:6/106: creat d7/de/d12/f23 x:0 0 0 2024-05-12T04:01:06.936 INFO:tasks.workunit.client.1.smithi033.stdout:6/277: chown d4/d1a/d4e 35571 1 2024-05-12T04:01:06.936 INFO:tasks.workunit.client.1.smithi033.stdout:6/278: chown d4/d43/d57 657 1 2024-05-12T04:01:06.938 INFO:tasks.workunit.client.0.smithi032.stdout:6/107: chown d7/de/d12/f22 359165 1 2024-05-12T04:01:06.941 INFO:tasks.workunit.client.1.smithi033.stdout:0/344: getdents d6/db 0 2024-05-12T04:01:06.942 INFO:tasks.workunit.client.0.smithi032.stdout:8/125: mkdir d0/d9/d1b/d27 0 2024-05-12T04:01:06.943 INFO:tasks.workunit.client.1.smithi033.stdout:9/511: dread d2/d4/f2c [0,4194304] 0 2024-05-12T04:01:06.944 INFO:tasks.workunit.client.1.smithi033.stdout:3/291: link db/d17/d3e/f48 db/d17/d3a/f66 0 2024-05-12T04:01:06.945 INFO:tasks.workunit.client.1.smithi033.stdout:3/292: write f6 [5396704,84951] 0 2024-05-12T04:01:06.946 INFO:tasks.workunit.client.1.smithi033.stdout:3/293: chown db/f54 213 1 2024-05-12T04:01:06.948 INFO:tasks.workunit.client.1.smithi033.stdout:1/259: mkdir d11/d1c/d58 0 2024-05-12T04:01:06.948 INFO:tasks.workunit.client.1.smithi033.stdout:7/464: dread de/df/d17/f6a [0,4194304] 0 2024-05-12T04:01:06.949 INFO:tasks.workunit.client.1.smithi033.stdout:1/260: write d11/d17/d27/f52 [353037,101806] 0 2024-05-12T04:01:06.950 INFO:tasks.workunit.client.1.smithi033.stdout:1/261: readlink d11/l26 0 2024-05-12T04:01:06.950 INFO:tasks.workunit.client.1.smithi033.stdout:0/345: dread d6/d11/f2f [0,4194304] 0 2024-05-12T04:01:06.953 INFO:tasks.workunit.client.0.smithi032.stdout:5/118: symlink d4/l27 0 2024-05-12T04:01:06.954 INFO:tasks.workunit.client.0.smithi032.stdout:3/165: rename l11 to d15/d20/l35 0 2024-05-12T04:01:06.954 INFO:tasks.workunit.client.0.smithi032.stdout:6/108: dwrite d7/de/d12/f16 [0,4194304] 0 2024-05-12T04:01:06.954 INFO:tasks.workunit.client.1.smithi033.stdout:9/512: dread d2/d4/d2b/d78/f8c [0,4194304] 0 2024-05-12T04:01:06.960 INFO:tasks.workunit.client.1.smithi033.stdout:9/513: dwrite d2/d4/d29/d3c/d52/f5e [0,4194304] 0 2024-05-12T04:01:06.969 INFO:tasks.workunit.client.1.smithi033.stdout:4/318: symlink dd/d11/d15/d42/d59/l5e 0 2024-05-12T04:01:06.971 INFO:tasks.workunit.client.0.smithi032.stdout:7/112: symlink dd/l2c 0 2024-05-12T04:01:06.973 INFO:tasks.workunit.client.0.smithi032.stdout:7/113: write dd/f12 [5049847,96594] 0 2024-05-12T04:01:06.980 INFO:tasks.workunit.client.1.smithi033.stdout:3/294: unlink db/d17/c1d 0 2024-05-12T04:01:06.981 INFO:tasks.workunit.client.0.smithi032.stdout:2/105: write d1/ff [778188,122299] 0 2024-05-12T04:01:06.982 INFO:tasks.workunit.client.1.smithi033.stdout:3/295: chown db/d17/d25/d32/d3c/d44 2069700863 1 2024-05-12T04:01:06.982 INFO:tasks.workunit.client.0.smithi032.stdout:2/106: dread - d1/f21 zero size 2024-05-12T04:01:06.984 INFO:tasks.workunit.client.1.smithi033.stdout:3/296: fsync db/d17/d3a/f63 0 2024-05-12T04:01:06.985 INFO:tasks.workunit.client.0.smithi032.stdout:5/119: mknod d4/d7/d14/c28 0 2024-05-12T04:01:06.987 INFO:tasks.workunit.client.0.smithi032.stdout:5/120: chown d4/d7/l24 2376 1 2024-05-12T04:01:06.987 INFO:tasks.workunit.client.1.smithi033.stdout:1/262: write d11/d17/d27/d36/d3f/f50 [1045265,57041] 0 2024-05-12T04:01:06.989 INFO:tasks.workunit.client.0.smithi032.stdout:6/109: creat d7/de/d12/d1a/f24 x:0 0 0 2024-05-12T04:01:06.990 INFO:tasks.workunit.client.1.smithi033.stdout:0/346: mkdir d6/db/d2c/d73 0 2024-05-12T04:01:06.990 INFO:tasks.workunit.client.0.smithi032.stdout:5/121: write d4/db/d1b/f23 [867309,22483] 0 2024-05-12T04:01:06.991 INFO:tasks.workunit.client.0.smithi032.stdout:5/122: chown d4/db/l18 184566279 1 2024-05-12T04:01:06.993 INFO:tasks.workunit.client.0.smithi032.stdout:4/95: creat d0/f11 x:0 0 0 2024-05-12T04:01:07.001 INFO:tasks.workunit.client.1.smithi033.stdout:2/299: link dc/d17/d27/c55 dc/d17/d32/d67/c6b 0 2024-05-12T04:01:07.002 INFO:tasks.workunit.client.0.smithi032.stdout:8/126: mkdir d0/d9/d13/d26/d28 0 2024-05-12T04:01:07.003 INFO:tasks.workunit.client.1.smithi033.stdout:4/319: creat dd/d11/d15/d42/f5f x:0 0 0 2024-05-12T04:01:07.006 INFO:tasks.workunit.client.1.smithi033.stdout:2/300: dwrite dc/d13/f1d [0,4194304] 0 2024-05-12T04:01:07.007 INFO:tasks.workunit.client.0.smithi032.stdout:3/166: symlink d15/d33/l36 0 2024-05-12T04:01:07.008 INFO:tasks.workunit.client.0.smithi032.stdout:3/167: dread d15/f18 [0,4194304] 0 2024-05-12T04:01:07.010 INFO:tasks.workunit.client.0.smithi032.stdout:6/110: mkdir d7/de/d12/d25 0 2024-05-12T04:01:07.012 INFO:tasks.workunit.client.1.smithi033.stdout:7/465: mkdir de/d20/da9 0 2024-05-12T04:01:07.012 INFO:tasks.workunit.client.1.smithi033.stdout:8/218: truncate d1/f12 383238 0 2024-05-12T04:01:07.015 INFO:tasks.workunit.client.1.smithi033.stdout:3/297: mkdir db/d17/d3a/d67 0 2024-05-12T04:01:07.015 INFO:tasks.workunit.client.1.smithi033.stdout:3/298: fsync db/d17/d25/f39 0 2024-05-12T04:01:07.017 INFO:tasks.workunit.client.0.smithi032.stdout:3/168: dwrite d15/d1b/f34 [0,4194304] 0 2024-05-12T04:01:07.019 INFO:tasks.workunit.client.0.smithi032.stdout:0/190: link df/d10/c21 df/d28/c44 0 2024-05-12T04:01:07.022 INFO:tasks.workunit.client.1.smithi033.stdout:5/233: getdents d3/d27 0 2024-05-12T04:01:07.025 INFO:tasks.workunit.client.0.smithi032.stdout:8/127: creat d0/d9/d13/f29 x:0 0 0 2024-05-12T04:01:07.027 INFO:tasks.workunit.client.1.smithi033.stdout:8/219: dread d1/f13 [0,4194304] 0 2024-05-12T04:01:07.027 INFO:tasks.workunit.client.1.smithi033.stdout:0/347: dwrite d6/d11/f33 [4194304,4194304] 0 2024-05-12T04:01:07.029 INFO:tasks.workunit.client.0.smithi032.stdout:6/111: creat d7/de/f26 x:0 0 0 2024-05-12T04:01:07.033 INFO:tasks.workunit.client.1.smithi033.stdout:6/279: link d4/d5/l31 d4/d1a/d4e/d4f/l5a 0 2024-05-12T04:01:07.035 INFO:tasks.workunit.client.0.smithi032.stdout:5/123: symlink d4/d7/d14/d26/l29 0 2024-05-12T04:01:07.035 INFO:tasks.workunit.client.0.smithi032.stdout:8/128: write d0/d9/d13/f29 [655386,5392] 0 2024-05-12T04:01:07.035 INFO:tasks.workunit.client.0.smithi032.stdout:4/96: dwrite d0/d3/fb [0,4194304] 0 2024-05-12T04:01:07.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.041+0000 7fa94f2ad700 1 -- 172.21.15.32:0/1059603998 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa9480ff560 con 0x7fa9480fcf90 2024-05-12T04:01:07.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.044+0000 7fa93dffb700 1 -- 172.21.15.32:0/1059603998 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+850 (secure 0 0 0) 0x7fa944057470 con 0x7fa9480fcf90 2024-05-12T04:01:07.047 INFO:tasks.workunit.client.0.smithi032.stdout:0/191: dwrite df/d10/f3c [0,4194304] 0 2024-05-12T04:01:07.048 INFO:tasks.workunit.client.0.smithi032.stdout:2/107: dwrite d1/f3 [0,4194304] 0 2024-05-12T04:01:07.048 INFO:tasks.workunit.client.0.smithi032.stdout:3/169: mknod d15/d33/c37 0 2024-05-12T04:01:07.051 INFO:tasks.workunit.client.0.smithi032.stdout:3/170: readlink d15/d1b/l1d 0 2024-05-12T04:01:07.053 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-05-12T04:01:07.053 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": { 2024-05-12T04:01:07.054 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 2 2024-05-12T04:01:07.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:01:07.054 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": { 2024-05-12T04:01:07.054 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 1, 2024-05-12T04:01:07.054 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 1 2024-05-12T04:01:07.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:01:07.054 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": { 2024-05-12T04:01:07.054 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 6 2024-05-12T04:01:07.054 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:01:07.054 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": { 2024-05-12T04:01:07.055 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 4 2024-05-12T04:01:07.055 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:01:07.055 INFO:teuthology.orchestra.run.smithi032.stdout: "overall": { 2024-05-12T04:01:07.055 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 13, 2024-05-12T04:01:07.055 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 1 2024-05-12T04:01:07.055 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-05-12T04:01:07.055 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-05-12T04:01:07.055 INFO:tasks.workunit.client.1.smithi033.stdout:4/320: rename dd/d11/d31/l37 to dd/d11/d15/d42/d59/l60 0 2024-05-12T04:01:07.056 INFO:tasks.workunit.client.0.smithi032.stdout:2/108: truncate d1/f1e 1321959 0 2024-05-12T04:01:07.056 INFO:tasks.workunit.client.1.smithi033.stdout:6/280: mknod d4/d1a/d4e/d4f/c5b 0 2024-05-12T04:01:07.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.057+0000 7fa94f2ad700 1 -- 172.21.15.32:0/1059603998 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7fa93406af90 msgr2=0x7fa93406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:01:07.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.057+0000 7fa94f2ad700 1 --2- 172.21.15.32:0/1059603998 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7fa93406af90 0x7fa93406d440 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa93800b900 tx=0x7fa938009e30 comp rx=0 tx=0).stop 2024-05-12T04:01:07.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.057+0000 7fa94f2ad700 1 -- 172.21.15.32:0/1059603998 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9480fcf90 msgr2=0x7fa948114440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:01:07.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.057+0000 7fa94f2ad700 1 --2- 172.21.15.32:0/1059603998 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9480fcf90 0x7fa948114440 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa94400c8c0 tx=0x7fa9440044c0 comp rx=0 tx=0).stop 2024-05-12T04:01:07.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.058+0000 7fa94f2ad700 1 -- 172.21.15.32:0/1059603998 shutdown_connections 2024-05-12T04:01:07.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.058+0000 7fa94f2ad700 1 --2- 172.21.15.32:0/1059603998 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7fa93406af90 0x7fa93406d440 secure :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa93800b900 tx=0x7fa938009e30 comp rx=0 tx=0).stop 2024-05-12T04:01:07.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.058+0000 7fa94f2ad700 1 --2- 172.21.15.32:0/1059603998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9480fbce0 0x7fa948113f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:07.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.058+0000 7fa94f2ad700 1 --2- 172.21.15.32:0/1059603998 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9480fcf90 0x7fa948114440 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:07.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.058+0000 7fa94f2ad700 1 -- 172.21.15.32:0/1059603998 >> 172.21.15.32:0/1059603998 conn(0x7fa9480f7440 msgr2=0x7fa948106960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:01:07.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.058+0000 7fa94f2ad700 1 -- 172.21.15.32:0/1059603998 shutdown_connections 2024-05-12T04:01:07.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.058+0000 7fa94f2ad700 1 -- 172.21.15.32:0/1059603998 wait complete. 2024-05-12T04:01:07.059 INFO:tasks.workunit.client.0.smithi032.stdout:3/171: dread - d15/d20/d22/f2e zero size 2024-05-12T04:01:07.060 INFO:tasks.workunit.client.0.smithi032.stdout:7/114: dwrite f9 [0,4194304] 0 2024-05-12T04:01:07.061 INFO:tasks.workunit.client.1.smithi033.stdout:4/321: dwrite dd/d11/f2f [0,4194304] 0 2024-05-12T04:01:07.062 INFO:tasks.workunit.client.1.smithi033.stdout:4/322: chown dd/d11/f1d 13 1 2024-05-12T04:01:07.067 INFO:tasks.workunit.client.0.smithi032.stdout:8/129: rename d0/f1d to d0/d9/d13/d21/f2a 0 2024-05-12T04:01:07.067 INFO:tasks.workunit.client.1.smithi033.stdout:1/263: rmdir d11/d1c/d58 0 2024-05-12T04:01:07.068 INFO:tasks.workunit.client.1.smithi033.stdout:9/514: getdents d2/d4/d2b/d6d/d2f 0 2024-05-12T04:01:07.072 INFO:tasks.workunit.client.0.smithi032.stdout:4/97: dwrite d0/fa [0,4194304] 0 2024-05-12T04:01:07.072 INFO:tasks.workunit.client.0.smithi032.stdout:8/130: chown d0/d9/f12 342986 1 2024-05-12T04:01:07.075 INFO:tasks.workunit.client.0.smithi032.stdout:1/97: sync 2024-05-12T04:01:07.075 INFO:tasks.workunit.client.0.smithi032.stdout:4/98: chown d0/d3/df/l10 6260 1 2024-05-12T04:01:07.079 INFO:tasks.workunit.client.0.smithi032.stdout:4/99: chown d0/d3/f4 1738 1 2024-05-12T04:01:07.080 INFO:tasks.workunit.client.0.smithi032.stdout:8/131: dread d0/d7/f11 [0,4194304] 0 2024-05-12T04:01:07.083 INFO:tasks.workunit.client.1.smithi033.stdout:5/234: mkdir d3/d15/d16/d45/d4b 0 2024-05-12T04:01:07.084 INFO:tasks.workunit.client.1.smithi033.stdout:3/299: write db/d17/d3a/f66 [326477,97259] 0 2024-05-12T04:01:07.089 INFO:tasks.workunit.client.0.smithi032.stdout:3/172: creat d15/d33/f38 x:0 0 0 2024-05-12T04:01:07.089 INFO:tasks.workunit.client.1.smithi033.stdout:6/281: creat d4/d5/f5c x:0 0 0 2024-05-12T04:01:07.092 INFO:tasks.workunit.client.0.smithi032.stdout:8/132: dread d0/d7/f11 [0,4194304] 0 2024-05-12T04:01:07.093 INFO:tasks.workunit.client.0.smithi032.stdout:7/115: creat dd/d1b/f2d x:0 0 0 2024-05-12T04:01:07.094 INFO:tasks.workunit.client.1.smithi033.stdout:1/264: mkdir d11/d1c/d59 0 2024-05-12T04:01:07.095 INFO:tasks.workunit.client.1.smithi033.stdout:3/300: dread db/d17/d3a/f3b [0,4194304] 0 2024-05-12T04:01:07.096 INFO:tasks.workunit.client.1.smithi033.stdout:1/265: write d11/d17/d27/d36/f3d [1682183,127277] 0 2024-05-12T04:01:07.097 INFO:tasks.workunit.client.0.smithi032.stdout:3/173: dread d15/d20/d26/f29 [0,4194304] 0 2024-05-12T04:01:07.097 INFO:tasks.workunit.client.0.smithi032.stdout:4/100: dwrite d0/fa [0,4194304] 0 2024-05-12T04:01:07.100 INFO:tasks.workunit.client.1.smithi033.stdout:2/301: rename dc/d13/d39/c3d to dc/d13/d65/c6c 0 2024-05-12T04:01:07.101 INFO:tasks.workunit.client.1.smithi033.stdout:0/348: truncate d6/d11/f2f 923515 0 2024-05-12T04:01:07.102 INFO:tasks.workunit.client.0.smithi032.stdout:4/101: dread - d0/f11 zero size 2024-05-12T04:01:07.103 INFO:tasks.workunit.client.1.smithi033.stdout:3/301: dwrite db/d17/d25/d32/d3c/f5a [0,4194304] 0 2024-05-12T04:01:07.104 INFO:tasks.workunit.client.1.smithi033.stdout:8/220: mkdir d1/d4/d1d/d36/d3c/d47 0 2024-05-12T04:01:07.105 INFO:tasks.workunit.client.0.smithi032.stdout:6/112: dwrite d7/d9/f17 [0,4194304] 0 2024-05-12T04:01:07.106 INFO:tasks.workunit.client.1.smithi033.stdout:6/282: dread d4/d5/f41 [0,4194304] 0 2024-05-12T04:01:07.106 INFO:tasks.workunit.client.1.smithi033.stdout:6/283: fdatasync d4/d5/f9 0 2024-05-12T04:01:07.107 INFO:tasks.workunit.client.1.smithi033.stdout:6/284: chown d4/d1a/d3c 14066125 1 2024-05-12T04:01:07.108 INFO:tasks.workunit.client.1.smithi033.stdout:4/323: mknod dd/d11/d5b/c61 0 2024-05-12T04:01:07.110 INFO:tasks.workunit.client.1.smithi033.stdout:1/266: write d11/d17/d27/f2b [347406,118439] 0 2024-05-12T04:01:07.110 INFO:tasks.workunit.client.0.smithi032.stdout:6/113: dread - d7/de/f26 zero size 2024-05-12T04:01:07.111 INFO:tasks.workunit.client.1.smithi033.stdout:9/515: creat d2/d4/d29/d32/db7/fbf x:0 0 0 2024-05-12T04:01:07.113 INFO:tasks.workunit.client.1.smithi033.stdout:9/516: write d2/d4/d2b/d78/f88 [60389,106958] 0 2024-05-12T04:01:07.113 INFO:tasks.workunit.client.0.smithi032.stdout:5/124: dwrite f1 [0,4194304] 0 2024-05-12T04:01:07.117 INFO:tasks.workunit.client.1.smithi033.stdout:7/466: rename c9 to de/df/d17/d1b/d36/caa 0 2024-05-12T04:01:07.118 INFO:tasks.workunit.client.1.smithi033.stdout:7/467: dread - de/d20/d38/f9e zero size 2024-05-12T04:01:07.121 INFO:tasks.workunit.client.1.smithi033.stdout:0/349: chown d6/dc/d39/l69 13 1 2024-05-12T04:01:07.123 INFO:tasks.workunit.client.0.smithi032.stdout:7/116: dwrite dd/d1b/d1e/f20 [0,4194304] 0 2024-05-12T04:01:07.127 INFO:tasks.workunit.client.1.smithi033.stdout:8/221: creat d1/d4/d2e/f48 x:0 0 0 2024-05-12T04:01:07.128 INFO:tasks.workunit.client.0.smithi032.stdout:6/114: dwrite d7/d9/f15 [0,4194304] 0 2024-05-12T04:01:07.131 INFO:tasks.workunit.client.0.smithi032.stdout:6/115: read - d7/de/f1b zero size 2024-05-12T04:01:07.132 INFO:tasks.workunit.client.0.smithi032.stdout:6/116: dread - d7/de/f1b zero size 2024-05-12T04:01:07.135 INFO:tasks.workunit.client.0.smithi032.stdout:4/102: mkdir d0/d3/d12 0 2024-05-12T04:01:07.136 INFO:tasks.workunit.client.0.smithi032.stdout:2/109: creat d1/f29 x:0 0 0 2024-05-12T04:01:07.138 INFO:tasks.workunit.client.0.smithi032.stdout:0/192: getdents df 0 2024-05-12T04:01:07.138 INFO:tasks.workunit.client.0.smithi032.stdout:2/110: chown d1/f1e 27727342 1 2024-05-12T04:01:07.139 INFO:tasks.workunit.client.0.smithi032.stdout:6/117: write d7/de/f1b [595072,65193] 0 2024-05-12T04:01:07.140 INFO:tasks.workunit.client.1.smithi033.stdout:1/267: mknod d11/d17/d27/d36/d3f/c5a 0 2024-05-12T04:01:07.140 INFO:tasks.workunit.client.0.smithi032.stdout:1/98: link d1/d7/cc d1/d16/c21 0 2024-05-12T04:01:07.141 INFO:tasks.workunit.client.0.smithi032.stdout:1/99: stat d1/d7/f11 0 2024-05-12T04:01:07.142 INFO:tasks.workunit.client.0.smithi032.stdout:7/117: dread dd/f13 [0,4194304] 0 2024-05-12T04:01:07.144 INFO:tasks.workunit.client.0.smithi032.stdout:1/100: fsync d1/d7/f11 0 2024-05-12T04:01:07.147 INFO:tasks.workunit.client.1.smithi033.stdout:2/302: rename dc/d13/f1d to dc/d17/d3a/f6d 0 2024-05-12T04:01:07.148 INFO:tasks.workunit.client.1.smithi033.stdout:2/303: write dc/d17/d27/d2f/f62 [1468005,108323] 0 2024-05-12T04:01:07.149 INFO:tasks.workunit.client.1.smithi033.stdout:7/468: mknod de/df/d7a/cab 0 2024-05-12T04:01:07.153 INFO:tasks.workunit.client.0.smithi032.stdout:8/133: rename d0/d9/f10 to d0/d9/d13/d26/d28/f2b 0 2024-05-12T04:01:07.153 INFO:tasks.workunit.client.0.smithi032.stdout:0/193: dread fb [0,4194304] 0 2024-05-12T04:01:07.154 INFO:tasks.workunit.client.1.smithi033.stdout:3/302: symlink db/d17/d25/d32/d53/l68 0 2024-05-12T04:01:07.158 INFO:tasks.workunit.client.0.smithi032.stdout:5/125: creat d4/d7/d21/f2a x:0 0 0 2024-05-12T04:01:07.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:06 smithi033 ceph-mon[49525]: from='client.14698 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:01:07.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:06 smithi033 ceph-mon[49525]: from='client.24461 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:01:07.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:06 smithi033 ceph-mon[49525]: pgmap v31: 65 pgs: 65 active+clean; 573 MiB data, 3.4 GiB used, 533 GiB / 536 GiB avail; 4.8 MiB/s rd, 53 MiB/s wr, 142 op/s 2024-05-12T04:01:07.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:06 smithi033 ceph-mon[49525]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:01:07.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:06 smithi033 ceph-mon[49525]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:01:07.162 INFO:tasks.workunit.client.1.smithi033.stdout:6/285: mknod d4/d43/d57/c5d 0 2024-05-12T04:01:07.165 INFO:tasks.workunit.client.1.smithi033.stdout:1/268: write d11/d17/f2e [13804,93897] 0 2024-05-12T04:01:07.165 INFO:tasks.workunit.client.0.smithi032.stdout:8/134: dread d0/f2 [0,4194304] 0 2024-05-12T04:01:07.166 INFO:tasks.workunit.client.0.smithi032.stdout:8/135: readlink d0/l23 0 2024-05-12T04:01:07.167 INFO:tasks.workunit.client.1.smithi033.stdout:9/517: mknod d2/d4/d2b/cc0 0 2024-05-12T04:01:07.168 INFO:tasks.workunit.client.0.smithi032.stdout:6/118: creat d7/de/d12/f27 x:0 0 0 2024-05-12T04:01:07.168 INFO:tasks.workunit.client.1.smithi033.stdout:6/286: dwrite d4/f29 [0,4194304] 0 2024-05-12T04:01:07.173 INFO:tasks.workunit.client.0.smithi032.stdout:7/118: symlink dd/l2e 0 2024-05-12T04:01:07.174 INFO:tasks.workunit.client.0.smithi032.stdout:7/119: write dd/f1d [184895,100441] 0 2024-05-12T04:01:07.177 INFO:tasks.workunit.client.0.smithi032.stdout:3/174: truncate d15/f17 3868835 0 2024-05-12T04:01:07.178 INFO:tasks.workunit.client.0.smithi032.stdout:1/101: mknod d1/d16/c22 0 2024-05-12T04:01:07.180 INFO:tasks.workunit.client.1.smithi033.stdout:2/304: mkdir dc/d13/d1f/d6e 0 2024-05-12T04:01:07.185 INFO:tasks.workunit.client.1.smithi033.stdout:3/303: symlink db/d17/d25/d32/d53/l69 0 2024-05-12T04:01:07.185 INFO:tasks.workunit.client.0.smithi032.stdout:3/175: dwrite f14 [4194304,4194304] 0 2024-05-12T04:01:07.187 INFO:tasks.workunit.client.0.smithi032.stdout:3/176: chown d15/d20/d22/f2c 7666077 1 2024-05-12T04:01:07.189 INFO:tasks.workunit.client.1.smithi033.stdout:5/235: getdents d3/d15/d16 0 2024-05-12T04:01:07.193 INFO:tasks.workunit.client.0.smithi032.stdout:5/126: mkdir d4/db/d2b 0 2024-05-12T04:01:07.194 INFO:tasks.workunit.client.0.smithi032.stdout:5/127: write d4/f1e [311377,80092] 0 2024-05-12T04:01:07.202 INFO:tasks.workunit.client.0.smithi032.stdout:2/111: fdatasync d1/f4 0 2024-05-12T04:01:07.202 INFO:tasks.workunit.client.1.smithi033.stdout:6/287: creat d4/d1a/d4e/d4f/f5e x:0 0 0 2024-05-12T04:01:07.203 INFO:tasks.workunit.client.1.smithi033.stdout:6/288: write d4/f51 [145475,32175] 0 2024-05-12T04:01:07.209 INFO:tasks.workunit.client.0.smithi032.stdout:1/102: rename d1/d7/c8 to d1/dd/c23 0 2024-05-12T04:01:07.210 INFO:tasks.workunit.client.1.smithi033.stdout:7/469: getdents de/d20/da9 0 2024-05-12T04:01:07.211 INFO:tasks.workunit.client.1.smithi033.stdout:8/222: dwrite d1/d4/d1d/f26 [0,4194304] 0 2024-05-12T04:01:07.211 INFO:tasks.workunit.client.1.smithi033.stdout:7/470: chown de/d20/f83 37 1 2024-05-12T04:01:07.212 INFO:tasks.workunit.client.1.smithi033.stdout:3/304: symlink db/d17/d25/l6a 0 2024-05-12T04:01:07.213 INFO:tasks.workunit.client.1.smithi033.stdout:5/236: symlink d3/d15/d16/l4c 0 2024-05-12T04:01:07.217 INFO:tasks.workunit.client.1.smithi033.stdout:3/305: dread db/d17/d25/d32/d3c/f5a [0,4194304] 0 2024-05-12T04:01:07.220 INFO:tasks.workunit.client.0.smithi032.stdout:4/103: creat d0/f13 x:0 0 0 2024-05-12T04:01:07.220 INFO:tasks.workunit.client.0.smithi032.stdout:3/177: read f3 [1557243,32056] 0 2024-05-12T04:01:07.221 INFO:tasks.workunit.client.1.smithi033.stdout:4/324: rename dd/d11/d15/d42/f5f to dd/f62 0 2024-05-12T04:01:07.224 INFO:tasks.workunit.client.1.smithi033.stdout:6/289: truncate d4/d1a/f26 6947057 0 2024-05-12T04:01:07.225 INFO:tasks.workunit.client.1.smithi033.stdout:2/305: mknod dc/d13/d1f/d6e/c6f 0 2024-05-12T04:01:07.227 INFO:tasks.workunit.client.0.smithi032.stdout:8/136: creat d0/d9/d13/d21/d25/f2c x:0 0 0 2024-05-12T04:01:07.230 INFO:tasks.workunit.client.0.smithi032.stdout:3/178: dwrite d15/d1b/f25 [4194304,4194304] 0 2024-05-12T04:01:07.230 INFO:tasks.workunit.client.0.smithi032.stdout:2/112: dread d1/f7 [0,4194304] 0 2024-05-12T04:01:07.230 INFO:tasks.workunit.client.0.smithi032.stdout:5/128: dread d4/f22 [4194304,4194304] 0 2024-05-12T04:01:07.232 INFO:tasks.workunit.client.0.smithi032.stdout:0/194: truncate fb 689049 0 2024-05-12T04:01:07.233 INFO:tasks.workunit.client.0.smithi032.stdout:0/195: chown l7 8186374 1 2024-05-12T04:01:07.233 INFO:tasks.workunit.client.0.smithi032.stdout:5/129: readlink d4/db/l18 0 2024-05-12T04:01:07.242 INFO:tasks.workunit.client.1.smithi033.stdout:3/306: unlink db/d17/f1c 0 2024-05-12T04:01:07.243 INFO:tasks.workunit.client.1.smithi033.stdout:3/307: chown db/d17/d25/d32/d3c/d44 9 1 2024-05-12T04:01:07.250 INFO:tasks.workunit.client.1.smithi033.stdout:0/350: rename d6/db/l50 to d6/dc/d66/l74 0 2024-05-12T04:01:07.250 INFO:tasks.workunit.client.1.smithi033.stdout:0/351: fsync d6/dc/d66/f67 0 2024-05-12T04:01:07.252 INFO:tasks.workunit.client.1.smithi033.stdout:4/325: creat dd/d11/d15/d42/f63 x:0 0 0 2024-05-12T04:01:07.253 INFO:tasks.workunit.client.1.smithi033.stdout:4/326: readlink dd/d11/d31/l3a 0 2024-05-12T04:01:07.253 INFO:tasks.workunit.client.1.smithi033.stdout:1/269: truncate d11/d17/f20 3080651 0 2024-05-12T04:01:07.254 INFO:tasks.workunit.client.0.smithi032.stdout:9/97: sync 2024-05-12T04:01:07.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.255+0000 7f8de12dc700 1 -- 172.21.15.32:0/122518425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ddc0fcf60 msgr2=0x7f8ddc0fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:01:07.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.255+0000 7f8de12dc700 1 --2- 172.21.15.32:0/122518425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ddc0fcf60 0x7f8ddc0fd3d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8dd0004940 tx=0x7f8dd0009e30 comp rx=0 tx=0).stop 2024-05-12T04:01:07.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.255+0000 7f8de12dc700 1 -- 172.21.15.32:0/122518425 shutdown_connections 2024-05-12T04:01:07.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.255+0000 7f8de12dc700 1 --2- 172.21.15.32:0/122518425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ddc0fcf60 0x7f8ddc0fd3d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:07.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.255+0000 7f8de12dc700 1 --2- 172.21.15.32:0/122518425 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ddc0fbcb0 0x7f8ddc0fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:07.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.255+0000 7f8de12dc700 1 -- 172.21.15.32:0/122518425 >> 172.21.15.32:0/122518425 conn(0x7f8ddc0f7430 msgr2=0x7f8ddc0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:01:07.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.255+0000 7f8de12dc700 1 -- 172.21.15.32:0/122518425 shutdown_connections 2024-05-12T04:01:07.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.255+0000 7f8de12dc700 1 -- 172.21.15.32:0/122518425 wait complete. 2024-05-12T04:01:07.256 INFO:tasks.workunit.client.0.smithi032.stdout:2/113: creat d1/d13/d25/f2a x:0 0 0 2024-05-12T04:01:07.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.256+0000 7f8de12dc700 1 Processor -- start 2024-05-12T04:01:07.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.256+0000 7f8de12dc700 1 -- start start 2024-05-12T04:01:07.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.256+0000 7f8de12dc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ddc0fbcb0 0x7f8ddc10c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:01:07.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.256+0000 7f8de12dc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ddc0fcf60 0x7f8ddc10ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:01:07.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.256+0000 7f8de12dc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ddc10d2c0 con 0x7f8ddc0fcf60 2024-05-12T04:01:07.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.256+0000 7f8de12dc700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ddc10db10 con 0x7f8ddc0fbcb0 2024-05-12T04:01:07.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.256+0000 7f8ddaffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ddc0fbcb0 0x7f8ddc10c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:01:07.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.256+0000 7f8dda7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ddc0fcf60 0x7f8ddc10ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:01:07.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.256+0000 7f8dda7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ddc0fcf60 0x7f8ddc10ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53046/0 (socket says 172.21.15.32:53046) 2024-05-12T04:01:07.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.256+0000 7f8ddaffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ddc0fbcb0 0x7f8ddc10c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.32:41614/0 (socket says 172.21.15.32:41614) 2024-05-12T04:01:07.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.256+0000 7f8dda7fc700 1 -- 172.21.15.32:0/2665773876 learned_addr learned my addr 172.21.15.32:0/2665773876 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:01:07.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.257+0000 7f8dda7fc700 1 -- 172.21.15.32:0/2665773876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ddc0fbcb0 msgr2=0x7f8ddc10c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:01:07.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.257+0000 7f8dda7fc700 1 --2- 172.21.15.32:0/2665773876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ddc0fbcb0 0x7f8ddc10c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:07.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.257+0000 7f8dda7fc700 1 -- 172.21.15.32:0/2665773876 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dcc005040 con 0x7f8ddc0fcf60 2024-05-12T04:01:07.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.257+0000 7f8dda7fc700 1 --2- 172.21.15.32:0/2665773876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ddc0fcf60 0x7f8ddc10ccc0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8dd00137b0 tx=0x7f8dd0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:01:07.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.257+0000 7f8dc37fe700 1 -- 172.21.15.32:0/2665773876 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8dd0021070 con 0x7f8ddc0fcf60 2024-05-12T04:01:07.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.257+0000 7f8de12dc700 1 -- 172.21.15.32:0/2665773876 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dd0005040 con 0x7f8ddc0fcf60 2024-05-12T04:01:07.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.257+0000 7f8de12dc700 1 -- 172.21.15.32:0/2665773876 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ddc10e0e0 con 0x7f8ddc0fcf60 2024-05-12T04:01:07.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.257+0000 7f8dc37fe700 1 -- 172.21.15.32:0/2665773876 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8dd001c400 con 0x7f8ddc0fcf60 2024-05-12T04:01:07.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.257+0000 7f8dc37fe700 1 -- 172.21.15.32:0/2665773876 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8dd000e070 con 0x7f8ddc0fcf60 2024-05-12T04:01:07.259 INFO:tasks.workunit.client.1.smithi033.stdout:1/270: dwrite d11/d17/d27/f56 [0,4194304] 0 2024-05-12T04:01:07.260 INFO:tasks.workunit.client.0.smithi032.stdout:3/179: symlink d15/d24/d2a/l39 0 2024-05-12T04:01:07.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.259+0000 7f8dc37fe700 1 -- 172.21.15.32:0/2665773876 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 25) v1 ==== 88772+0+0 (secure 0 0 0) 0x7f8dd000b760 con 0x7f8ddc0fcf60 2024-05-12T04:01:07.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.260+0000 7f8dc37fe700 1 --2- 172.21.15.32:0/2665773876 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f8dc406aec0 0x7f8dc406d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:01:07.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.260+0000 7f8de12dc700 1 -- 172.21.15.32:0/2665773876 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dc80052e0 con 0x7f8ddc0fcf60 2024-05-12T04:01:07.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.260+0000 7f8dc37fe700 1 -- 172.21.15.32:0/2665773876 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f8dd008a8b0 con 0x7f8ddc0fcf60 2024-05-12T04:01:07.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.260+0000 7f8ddaffd700 1 --2- 172.21.15.32:0/2665773876 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f8dc406aec0 0x7f8dc406d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:01:07.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.261+0000 7f8ddaffd700 1 --2- 172.21.15.32:0/2665773876 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f8dc406aec0 0x7f8dc406d370 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc009fd0 tx=0x7f8dcc00b040 comp rx=0 tx=0).ready entity=mgr.24389 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:01:07.261 INFO:tasks.workunit.client.1.smithi033.stdout:2/306: unlink dc/d3b/f4a 0 2024-05-12T04:01:07.262 INFO:tasks.workunit.client.1.smithi033.stdout:2/307: chown dc/d13/d1f/l2d 0 1 2024-05-12T04:01:07.264 INFO:tasks.workunit.client.0.smithi032.stdout:5/130: mkdir d4/d7/d21/d2c 0 2024-05-12T04:01:07.265 INFO:tasks.workunit.client.0.smithi032.stdout:4/104: symlink d0/l14 0 2024-05-12T04:01:07.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.266+0000 7f8dc37fe700 1 -- 172.21.15.32:0/2665773876 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181777 (secure 0 0 0) 0x7f8dd0058850 con 0x7f8ddc0fcf60 2024-05-12T04:01:07.269 INFO:tasks.workunit.client.0.smithi032.stdout:4/105: write d0/d3/fd [725832,38845] 0 2024-05-12T04:01:07.270 INFO:tasks.workunit.client.1.smithi033.stdout:5/237: truncate d3/d12/d25/d26/f31 1118614 0 2024-05-12T04:01:07.271 INFO:tasks.workunit.client.0.smithi032.stdout:4/106: chown d0/f13 1 1 2024-05-12T04:01:07.272 INFO:tasks.workunit.client.0.smithi032.stdout:6/119: link f6 d7/de/d12/f28 0 2024-05-12T04:01:07.273 INFO:tasks.workunit.client.0.smithi032.stdout:4/107: write d0/f11 [625947,5886] 0 2024-05-12T04:01:07.274 INFO:tasks.workunit.client.0.smithi032.stdout:5/131: dread f1 [0,4194304] 0 2024-05-12T04:01:07.275 INFO:tasks.workunit.client.1.smithi033.stdout:9/518: rename d2/d4/d29/d32/c98 to d2/dd/d39/d72/cc1 0 2024-05-12T04:01:07.276 INFO:tasks.workunit.client.0.smithi032.stdout:5/132: chown d4/db/d2b 6612431 1 2024-05-12T04:01:07.276 INFO:tasks.workunit.client.0.smithi032.stdout:2/114: creat d1/d13/f2b x:0 0 0 2024-05-12T04:01:07.278 INFO:tasks.workunit.client.1.smithi033.stdout:0/352: write d6/d11/d57/f63 [170253,74229] 0 2024-05-12T04:01:07.281 INFO:tasks.workunit.client.0.smithi032.stdout:3/180: rmdir d15/d1b 39 2024-05-12T04:01:07.286 INFO:tasks.workunit.client.0.smithi032.stdout:5/133: dwrite d4/d7/d21/f2a [0,4194304] 0 2024-05-12T04:01:07.288 INFO:tasks.workunit.client.1.smithi033.stdout:6/290: unlink d4/d1a/d4e/l55 0 2024-05-12T04:01:07.288 INFO:tasks.workunit.client.1.smithi033.stdout:6/291: chown d4/f30 926132 1 2024-05-12T04:01:07.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:06 smithi032 ceph-mon[40008]: from='client.14698 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:01:07.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:06 smithi032 ceph-mon[40008]: from='client.24461 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:01:07.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:06 smithi032 ceph-mon[40008]: pgmap v31: 65 pgs: 65 active+clean; 573 MiB data, 3.4 GiB used, 533 GiB / 536 GiB avail; 4.8 MiB/s rd, 53 MiB/s wr, 142 op/s 2024-05-12T04:01:07.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:06 smithi032 ceph-mon[40008]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:01:07.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:06 smithi032 ceph-mon[40008]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:01:07.290 INFO:tasks.workunit.client.1.smithi033.stdout:6/292: write d4/d1a/d4e/d4f/f50 [140408,110671] 0 2024-05-12T04:01:07.293 INFO:tasks.workunit.client.0.smithi032.stdout:0/196: symlink df/d10/d3b/l45 0 2024-05-12T04:01:07.293 INFO:tasks.workunit.client.0.smithi032.stdout:8/137: creat d0/d9/f2d x:0 0 0 2024-05-12T04:01:07.294 INFO:tasks.workunit.client.0.smithi032.stdout:0/197: fdatasync df/d10/f3c 0 2024-05-12T04:01:07.294 INFO:tasks.workunit.client.1.smithi033.stdout:8/223: truncate d1/d16/f19 2807359 0 2024-05-12T04:01:07.294 INFO:tasks.workunit.client.0.smithi032.stdout:8/138: write d0/d9/f12 [1681824,129245] 0 2024-05-12T04:01:07.295 INFO:tasks.workunit.client.0.smithi032.stdout:1/103: getdents d1/d16/d18 0 2024-05-12T04:01:07.299 INFO:tasks.workunit.client.0.smithi032.stdout:8/139: write d0/d9/fe [4705847,29444] 0 2024-05-12T04:01:07.302 INFO:tasks.workunit.client.0.smithi032.stdout:9/98: symlink d0/d3/d6/d7/da/l1f 0 2024-05-12T04:01:07.306 INFO:tasks.workunit.client.0.smithi032.stdout:2/115: fsync d1/f4 0 2024-05-12T04:01:07.308 INFO:tasks.workunit.client.1.smithi033.stdout:9/519: mknod d2/d4/d2b/d78/cc2 0 2024-05-12T04:01:07.312 INFO:tasks.workunit.client.0.smithi032.stdout:3/181: chown d15/d1b/f21 219214883 1 2024-05-12T04:01:07.315 INFO:tasks.workunit.client.1.smithi033.stdout:6/293: mknod d4/d5/c5f 0 2024-05-12T04:01:07.317 INFO:tasks.workunit.client.1.smithi033.stdout:6/294: dread d4/d5/f41 [0,4194304] 0 2024-05-12T04:01:07.317 INFO:tasks.workunit.client.0.smithi032.stdout:5/134: symlink d4/d7/l2d 0 2024-05-12T04:01:07.323 INFO:tasks.workunit.client.1.smithi033.stdout:7/471: getdents de/df/d17/d1b/d36 0 2024-05-12T04:01:07.325 INFO:tasks.workunit.client.0.smithi032.stdout:1/104: mkdir d1/dd/d24 0 2024-05-12T04:01:07.327 INFO:tasks.workunit.client.0.smithi032.stdout:6/120: creat d7/de/d12/d25/f29 x:0 0 0 2024-05-12T04:01:07.328 INFO:tasks.workunit.client.1.smithi033.stdout:5/238: symlink d3/l4d 0 2024-05-12T04:01:07.328 INFO:tasks.workunit.client.1.smithi033.stdout:5/239: stat d3/d12/d25/d26/l2a 0 2024-05-12T04:01:07.329 INFO:tasks.workunit.client.0.smithi032.stdout:0/198: mknod df/d10/d1f/c46 0 2024-05-12T04:01:07.331 INFO:tasks.workunit.client.0.smithi032.stdout:8/140: write d0/d9/d13/d21/f2a [690028,11965] 0 2024-05-12T04:01:07.334 INFO:tasks.workunit.client.0.smithi032.stdout:9/99: rename d0/d3/d6/d7/da/fd to d0/d3/d6/d7/da/d17/f20 0 2024-05-12T04:01:07.335 INFO:tasks.workunit.client.0.smithi032.stdout:9/100: readlink d0/d3/d6/d7/da/l1f 0 2024-05-12T04:01:07.337 INFO:tasks.workunit.client.1.smithi033.stdout:6/295: creat d4/d43/f60 x:0 0 0 2024-05-12T04:01:07.338 INFO:tasks.workunit.client.0.smithi032.stdout:2/116: mkdir d1/d13/d25/d2c 0 2024-05-12T04:01:07.338 INFO:tasks.workunit.client.1.smithi033.stdout:1/271: link d11/d1c/c1b d11/d17/d27/d36/d46/d4c/c5b 0 2024-05-12T04:01:07.340 INFO:tasks.workunit.client.1.smithi033.stdout:1/272: write d11/d1c/d43/f45 [521811,79895] 0 2024-05-12T04:01:07.340 INFO:tasks.workunit.client.0.smithi032.stdout:2/117: dread d1/f7 [0,4194304] 0 2024-05-12T04:01:07.341 INFO:tasks.workunit.client.1.smithi033.stdout:1/273: write d11/d17/d27/d36/d3f/f50 [1787967,21179] 0 2024-05-12T04:01:07.341 INFO:tasks.workunit.client.1.smithi033.stdout:8/224: mknod d1/d4/d1d/d36/d3c/d47/c49 0 2024-05-12T04:01:07.346 INFO:tasks.workunit.client.1.smithi033.stdout:4/327: truncate dd/f1b 88117 0 2024-05-12T04:01:07.346 INFO:tasks.workunit.client.0.smithi032.stdout:2/118: dread d1/fc [0,4194304] 0 2024-05-12T04:01:07.347 INFO:tasks.workunit.client.0.smithi032.stdout:3/182: dwrite d15/f1c [0,4194304] 0 2024-05-12T04:01:07.347 INFO:tasks.workunit.client.0.smithi032.stdout:1/105: readlink l0 0 2024-05-12T04:01:07.348 INFO:tasks.workunit.client.0.smithi032.stdout:2/119: dread - d1/d13/d25/f2a zero size 2024-05-12T04:01:07.349 INFO:tasks.workunit.client.0.smithi032.stdout:6/121: creat d7/de/d12/d25/f2a x:0 0 0 2024-05-12T04:01:07.351 INFO:tasks.workunit.client.0.smithi032.stdout:1/106: truncate d1/d7/f11 1635740 0 2024-05-12T04:01:07.351 INFO:tasks.workunit.client.0.smithi032.stdout:1/107: chown d1/c1c 8180502 1 2024-05-12T04:01:07.352 INFO:tasks.workunit.client.1.smithi033.stdout:5/240: rename d3/d12/d29/l37 to d3/d12/d29/d2b/l4e 0 2024-05-12T04:01:07.353 INFO:tasks.workunit.client.0.smithi032.stdout:0/199: mkdir df/d30/d36/d47 0 2024-05-12T04:01:07.354 INFO:tasks.workunit.client.0.smithi032.stdout:0/200: chown df/d10/d1f/f38 135251 1 2024-05-12T04:01:07.355 INFO:tasks.workunit.client.1.smithi033.stdout:9/520: link d2/d4/d2b/d6d/d4c/f95 d2/d4/d29/d32/fc3 0 2024-05-12T04:01:07.355 INFO:tasks.workunit.client.1.smithi033.stdout:4/328: dread dd/d11/d15/f2d [0,4194304] 0 2024-05-12T04:01:07.358 INFO:tasks.workunit.client.0.smithi032.stdout:4/108: link d0/d3/le d0/d3/df/l15 0 2024-05-12T04:01:07.359 INFO:tasks.workunit.client.1.smithi033.stdout:5/241: dwrite d3/d15/d16/f44 [0,4194304] 0 2024-05-12T04:01:07.363 INFO:tasks.workunit.client.0.smithi032.stdout:0/201: dwrite df/d30/d36/f41 [0,4194304] 0 2024-05-12T04:01:07.364 INFO:tasks.workunit.client.0.smithi032.stdout:0/202: chown df/d10/d1f/c46 1200834 1 2024-05-12T04:01:07.365 INFO:tasks.workunit.client.1.smithi033.stdout:1/274: mknod d11/d1c/d43/c5c 0 2024-05-12T04:01:07.365 INFO:tasks.workunit.client.1.smithi033.stdout:1/275: dread - d11/d17/d27/d36/d46/f51 zero size 2024-05-12T04:01:07.366 INFO:tasks.workunit.client.1.smithi033.stdout:8/225: chown d1/d4/d1d/f1e 14 1 2024-05-12T04:01:07.367 INFO:tasks.workunit.client.1.smithi033.stdout:2/308: link dc/d17/d27/d47/l57 dc/d17/d27/l70 0 2024-05-12T04:01:07.368 INFO:tasks.workunit.client.0.smithi032.stdout:5/135: getdents d4/db/d2b 0 2024-05-12T04:01:07.371 INFO:tasks.workunit.client.0.smithi032.stdout:3/183: rmdir d15/d20 39 2024-05-12T04:01:07.377 INFO:tasks.workunit.client.0.smithi032.stdout:6/122: rename d7/de/f19 to d7/f2b 0 2024-05-12T04:01:07.377 INFO:tasks.workunit.client.0.smithi032.stdout:5/136: dwrite d4/d7/d21/f2a [0,4194304] 0 2024-05-12T04:01:07.379 INFO:tasks.workunit.client.1.smithi033.stdout:4/329: symlink dd/d11/d15/d42/d59/l64 0 2024-05-12T04:01:07.385 INFO:tasks.workunit.client.1.smithi033.stdout:0/353: link d6/dc/l4b d6/dc/d66/l75 0 2024-05-12T04:01:07.386 INFO:tasks.workunit.client.0.smithi032.stdout:6/123: dread d7/fc [0,4194304] 0 2024-05-12T04:01:07.386 INFO:tasks.workunit.client.0.smithi032.stdout:8/141: fsync d0/d9/d13/d26/d28/f2b 0 2024-05-12T04:01:07.387 INFO:tasks.workunit.client.1.smithi033.stdout:5/242: readlink d3/d12/d29/d2b/l3a 0 2024-05-12T04:01:07.388 INFO:tasks.workunit.client.0.smithi032.stdout:6/124: chown d7/de/f26 31 1 2024-05-12T04:01:07.388 INFO:tasks.workunit.client.0.smithi032.stdout:8/142: write d0/d9/f12 [170109,7014] 0 2024-05-12T04:01:07.391 INFO:tasks.workunit.client.0.smithi032.stdout:4/109: mknod d0/d3/df/c16 0 2024-05-12T04:01:07.393 INFO:tasks.workunit.client.1.smithi033.stdout:1/276: creat d11/d17/d38/f5d x:0 0 0 2024-05-12T04:01:07.394 INFO:tasks.workunit.client.1.smithi033.stdout:6/296: write d4/f2f [537099,98543] 0 2024-05-12T04:01:07.397 INFO:tasks.workunit.client.1.smithi033.stdout:8/226: unlink d1/d4/d2e/c32 0 2024-05-12T04:01:07.398 INFO:tasks.workunit.client.0.smithi032.stdout:0/203: mkdir df/d10/d48 0 2024-05-12T04:01:07.398 INFO:tasks.workunit.client.0.smithi032.stdout:0/204: dread - df/d10/f33 zero size 2024-05-12T04:01:07.398 INFO:tasks.workunit.client.1.smithi033.stdout:2/309: symlink dc/d3b/l71 0 2024-05-12T04:01:07.400 INFO:tasks.workunit.client.1.smithi033.stdout:2/310: stat dc/d13/d1c/c3c 0 2024-05-12T04:01:07.403 INFO:tasks.workunit.client.0.smithi032.stdout:3/184: chown d15/d20/c32 1878 1 2024-05-12T04:01:07.405 INFO:tasks.workunit.client.0.smithi032.stdout:2/120: mknod d1/c2d 0 2024-05-12T04:01:07.407 INFO:tasks.workunit.client.0.smithi032.stdout:1/108: write d1/f13 [1014346,18551] 0 2024-05-12T04:01:07.410 INFO:tasks.workunit.client.1.smithi033.stdout:9/521: mknod d2/d70/d7b/d94/cc4 0 2024-05-12T04:01:07.412 INFO:tasks.workunit.client.0.smithi032.stdout:5/137: rename d4/d7/lc to d4/db/d2b/l2e 0 2024-05-12T04:01:07.412 INFO:tasks.workunit.client.0.smithi032.stdout:5/138: chown d4/fd 399422 1 2024-05-12T04:01:07.413 INFO:tasks.workunit.client.0.smithi032.stdout:5/139: chown d4/f1e 643 1 2024-05-12T04:01:07.414 INFO:tasks.workunit.client.0.smithi032.stdout:5/140: fdatasync d4/f1e 0 2024-05-12T04:01:07.415 INFO:tasks.workunit.client.1.smithi033.stdout:0/354: write d6/d11/f28 [1543376,11785] 0 2024-05-12T04:01:07.416 INFO:tasks.workunit.client.1.smithi033.stdout:5/243: mkdir d3/d12/d29/d2b/d4f 0 2024-05-12T04:01:07.417 INFO:tasks.workunit.client.0.smithi032.stdout:6/125: readlink d7/l8 0 2024-05-12T04:01:07.419 INFO:tasks.workunit.client.1.smithi033.stdout:1/277: creat d11/d1c/d43/f5e x:0 0 0 2024-05-12T04:01:07.419 INFO:tasks.workunit.client.1.smithi033.stdout:3/308: sync 2024-05-12T04:01:07.420 INFO:tasks.workunit.client.1.smithi033.stdout:6/297: fsync d4/f1f 0 2024-05-12T04:01:07.424 INFO:tasks.workunit.client.1.smithi033.stdout:2/311: fsync dc/d17/d27/d2f/f38 0 2024-05-12T04:01:07.427 INFO:tasks.workunit.client.0.smithi032.stdout:6/126: dwrite d7/de/d12/d1a/f24 [0,4194304] 0 2024-05-12T04:01:07.428 INFO:tasks.workunit.client.1.smithi033.stdout:9/522: mknod d2/d4/d2b/d6d/db5/cc5 0 2024-05-12T04:01:07.429 INFO:tasks.workunit.client.1.smithi033.stdout:9/523: fdatasync d2/dd/d39/f54 0 2024-05-12T04:01:07.430 INFO:tasks.workunit.client.0.smithi032.stdout:0/205: fdatasync fd 0 2024-05-12T04:01:07.431 INFO:tasks.workunit.client.0.smithi032.stdout:3/185: mknod d15/d24/d2a/c3a 0 2024-05-12T04:01:07.433 INFO:tasks.workunit.client.0.smithi032.stdout:3/186: rename d15 to d15/d20/d22/d3b 22 2024-05-12T04:01:07.433 INFO:tasks.workunit.client.1.smithi033.stdout:0/355: creat d6/dc/d39/f76 x:0 0 0 2024-05-12T04:01:07.434 INFO:tasks.workunit.client.0.smithi032.stdout:1/109: mkdir d1/d25 0 2024-05-12T04:01:07.434 INFO:tasks.workunit.client.1.smithi033.stdout:0/356: dread - d6/d11/d57/f60 zero size 2024-05-12T04:01:07.435 INFO:tasks.workunit.client.1.smithi033.stdout:5/244: write d3/f9 [10157847,14509] 0 2024-05-12T04:01:07.436 INFO:tasks.workunit.client.1.smithi033.stdout:5/245: dread - d3/f1e zero size 2024-05-12T04:01:07.437 INFO:tasks.workunit.client.1.smithi033.stdout:1/278: creat d11/d17/d38/f5f x:0 0 0 2024-05-12T04:01:07.440 INFO:tasks.workunit.client.0.smithi032.stdout:5/141: fsync f1 0 2024-05-12T04:01:07.441 INFO:tasks.workunit.client.0.smithi032.stdout:5/142: write d4/f20 [1730028,23596] 0 2024-05-12T04:01:07.443 INFO:tasks.workunit.client.1.smithi033.stdout:6/298: rmdir d4 39 2024-05-12T04:01:07.444 INFO:tasks.workunit.client.0.smithi032.stdout:8/143: symlink d0/l2e 0 2024-05-12T04:01:07.446 INFO:tasks.workunit.client.1.smithi033.stdout:8/227: unlink d1/d4/c34 0 2024-05-12T04:01:07.448 INFO:tasks.workunit.client.0.smithi032.stdout:4/110: symlink d0/l17 0 2024-05-12T04:01:07.452 INFO:tasks.workunit.client.0.smithi032.stdout:9/101: getdents d0 0 2024-05-12T04:01:07.454 INFO:tasks.workunit.client.0.smithi032.stdout:2/121: write d1/f4 [304970,105123] 0 2024-05-12T04:01:07.454 INFO:tasks.workunit.client.0.smithi032.stdout:9/102: write d0/d3/d6/d7/f14 [335389,25517] 0 2024-05-12T04:01:07.456 INFO:tasks.workunit.client.0.smithi032.stdout:1/110: dread d1/f6 [0,4194304] 0 2024-05-12T04:01:07.459 INFO:tasks.workunit.client.0.smithi032.stdout:6/127: chown d7/l1c 952424 1 2024-05-12T04:01:07.460 INFO:tasks.workunit.client.1.smithi033.stdout:9/524: creat d2/d4/d29/d32/db7/fc6 x:0 0 0 2024-05-12T04:01:07.461 INFO:tasks.workunit.client.1.smithi033.stdout:9/525: write d2/dd/f11 [1675417,52942] 0 2024-05-12T04:01:07.461 INFO:tasks.workunit.client.0.smithi032.stdout:3/187: readlink d15/d20/l35 0 2024-05-12T04:01:07.465 INFO:tasks.workunit.client.0.smithi032.stdout:3/188: read d15/d20/d22/f23 [617757,56203] 0 2024-05-12T04:01:07.466 INFO:tasks.workunit.client.0.smithi032.stdout:2/122: dwrite d1/f21 [0,4194304] 0 2024-05-12T04:01:07.468 INFO:tasks.workunit.client.1.smithi033.stdout:0/357: mkdir d6/d11/d57/d77 0 2024-05-12T04:01:07.468 INFO:tasks.workunit.client.1.smithi033.stdout:0/358: dread - d6/f61 zero size 2024-05-12T04:01:07.470 INFO:tasks.workunit.client.0.smithi032.stdout:5/143: creat d4/db/f2f x:0 0 0 2024-05-12T04:01:07.471 INFO:tasks.workunit.client.1.smithi033.stdout:1/279: mkdir d11/d1c/d43/d60 0 2024-05-12T04:01:07.472 INFO:tasks.workunit.client.0.smithi032.stdout:5/144: chown d4/db/d2b 13278965 1 2024-05-12T04:01:07.474 INFO:tasks.workunit.client.0.smithi032.stdout:1/111: creat d1/f26 x:0 0 0 2024-05-12T04:01:07.474 INFO:tasks.workunit.client.0.smithi032.stdout:1/112: chown d1/d16 12 1 2024-05-12T04:01:07.475 INFO:tasks.workunit.client.0.smithi032.stdout:1/113: fdatasync d1/d16/d18/f1a 0 2024-05-12T04:01:07.476 INFO:tasks.workunit.client.0.smithi032.stdout:1/114: chown d1/f3 0 1 2024-05-12T04:01:07.477 INFO:tasks.workunit.client.0.smithi032.stdout:9/103: rename d0/c16 to d0/d3/d6/d7/da/d17/c21 0 2024-05-12T04:01:07.477 INFO:tasks.workunit.client.0.smithi032.stdout:1/115: chown d1/dd/d24 11 1 2024-05-12T04:01:07.479 INFO:tasks.workunit.client.1.smithi033.stdout:2/312: mknod dc/d17/d27/d68/c72 0 2024-05-12T04:01:07.480 INFO:tasks.workunit.client.1.smithi033.stdout:2/313: write dc/d17/f63 [71970,121482] 0 2024-05-12T04:01:07.480 INFO:tasks.workunit.client.0.smithi032.stdout:6/128: creat d7/de/d12/f2c x:0 0 0 2024-05-12T04:01:07.481 INFO:tasks.workunit.client.1.smithi033.stdout:4/330: getdents dd/d11/d15/d42/d59 0 2024-05-12T04:01:07.481 INFO:tasks.workunit.client.1.smithi033.stdout:2/314: write f3 [2638350,79302] 0 2024-05-12T04:01:07.482 INFO:tasks.workunit.client.1.smithi033.stdout:2/315: readlink dc/d3b/l45 0 2024-05-12T04:01:07.483 INFO:tasks.workunit.client.1.smithi033.stdout:2/316: dread - dc/d13/d1f/f69 zero size 2024-05-12T04:01:07.484 INFO:tasks.workunit.client.1.smithi033.stdout:2/317: write dc/f3f [693142,31980] 0 2024-05-12T04:01:07.486 INFO:tasks.workunit.client.1.smithi033.stdout:9/526: unlink d2/d4/d2b/d6d/l3f 0 2024-05-12T04:01:07.486 INFO:tasks.workunit.client.1.smithi033.stdout:2/318: dread dc/d17/f63 [0,4194304] 0 2024-05-12T04:01:07.487 INFO:tasks.workunit.client.1.smithi033.stdout:2/319: chown dc/d13/d1f/f24 0 1 2024-05-12T04:01:07.489 INFO:tasks.workunit.client.0.smithi032.stdout:2/123: rmdir d1/d13 39 2024-05-12T04:01:07.491 INFO:tasks.workunit.client.1.smithi033.stdout:0/359: creat d6/dc/d58/f78 x:0 0 0 2024-05-12T04:01:07.492 INFO:tasks.workunit.client.1.smithi033.stdout:0/360: stat d6/dc/l1e 0 2024-05-12T04:01:07.493 INFO:tasks.workunit.client.1.smithi033.stdout:2/320: dwrite dc/d17/d27/d2f/f62 [0,4194304] 0 2024-05-12T04:01:07.493 INFO:tasks.workunit.client.1.smithi033.stdout:5/246: truncate d3/d15/d16/f1a 777773 0 2024-05-12T04:01:07.494 INFO:tasks.workunit.client.1.smithi033.stdout:5/247: stat d3/d15/d16/d41 0 2024-05-12T04:01:07.494 INFO:tasks.workunit.client.0.smithi032.stdout:4/111: symlink d0/l18 0 2024-05-12T04:01:07.498 INFO:tasks.workunit.client.1.smithi033.stdout:7/472: sync 2024-05-12T04:01:07.499 INFO:tasks.workunit.client.1.smithi033.stdout:7/473: chown de/df/d17/d1e/d24/f4f 750 1 2024-05-12T04:01:07.501 INFO:tasks.workunit.client.0.smithi032.stdout:4/112: dwrite d0/d3/f4 [0,4194304] 0 2024-05-12T04:01:07.504 INFO:tasks.workunit.client.1.smithi033.stdout:6/299: symlink d4/d1a/l61 0 2024-05-12T04:01:07.505 INFO:tasks.workunit.client.0.smithi032.stdout:5/145: rename d4/d7/c1c to d4/d7/d14/c30 0 2024-05-12T04:01:07.508 INFO:tasks.workunit.client.0.smithi032.stdout:6/129: symlink d7/de/d12/d1a/l2d 0 2024-05-12T04:01:07.512 INFO:tasks.workunit.client.1.smithi033.stdout:4/331: mknod dd/d11/d15/c65 0 2024-05-12T04:01:07.517 INFO:tasks.workunit.client.0.smithi032.stdout:8/144: creat d0/f2f x:0 0 0 2024-05-12T04:01:07.518 INFO:tasks.workunit.client.1.smithi033.stdout:7/474: dread de/d2b/f2c [0,4194304] 0 2024-05-12T04:01:07.519 INFO:tasks.workunit.client.1.smithi033.stdout:7/475: chown de/d80/l81 52 1 2024-05-12T04:01:07.520 INFO:tasks.workunit.client.1.smithi033.stdout:7/476: write de/d80/f98 [1641622,6975] 0 2024-05-12T04:01:07.521 INFO:tasks.workunit.client.0.smithi032.stdout:3/189: write d15/f17 [858076,96532] 0 2024-05-12T04:01:07.522 INFO:tasks.workunit.client.0.smithi032.stdout:4/113: mknod d0/d3/c19 0 2024-05-12T04:01:07.523 INFO:tasks.workunit.client.0.smithi032.stdout:2/124: dwrite d1/d13/f1f [0,4194304] 0 2024-05-12T04:01:07.525 INFO:tasks.workunit.client.1.smithi033.stdout:0/361: fdatasync d6/f23 0 2024-05-12T04:01:07.526 INFO:tasks.workunit.client.0.smithi032.stdout:4/114: write d0/d3/fb [2551523,44491] 0 2024-05-12T04:01:07.527 INFO:tasks.workunit.client.1.smithi033.stdout:2/321: symlink dc/d17/d27/d47/l73 0 2024-05-12T04:01:07.527 INFO:tasks.workunit.client.1.smithi033.stdout:2/322: fsync dc/d13/d1f/f69 0 2024-05-12T04:01:07.528 INFO:tasks.workunit.client.1.smithi033.stdout:2/323: dread - dc/d17/d27/d2f/d49/f51 zero size 2024-05-12T04:01:07.529 INFO:tasks.workunit.client.0.smithi032.stdout:1/116: mknod d1/d25/c27 0 2024-05-12T04:01:07.529 INFO:tasks.workunit.client.0.smithi032.stdout:6/130: mkdir d7/de/d2e 0 2024-05-12T04:01:07.530 INFO:tasks.workunit.client.1.smithi033.stdout:1/280: truncate d11/d1c/f34 3363487 0 2024-05-12T04:01:07.533 INFO:tasks.workunit.client.1.smithi033.stdout:3/309: getdents db/d17/d3e 0 2024-05-12T04:01:07.534 INFO:tasks.workunit.client.0.smithi032.stdout:8/145: symlink d0/d7/l30 0 2024-05-12T04:01:07.537 INFO:tasks.workunit.client.0.smithi032.stdout:9/104: symlink d0/d3/d6/d7/l22 0 2024-05-12T04:01:07.538 INFO:tasks.workunit.client.0.smithi032.stdout:1/117: dwrite d1/f13 [0,4194304] 0 2024-05-12T04:01:07.539 INFO:tasks.workunit.client.0.smithi032.stdout:9/105: stat d0/d3/d6/d7/da/l1f 0 2024-05-12T04:01:07.540 INFO:tasks.workunit.client.0.smithi032.stdout:5/146: write d4/f22 [1633447,116957] 0 2024-05-12T04:01:07.542 INFO:tasks.workunit.client.0.smithi032.stdout:5/147: dread - d4/db/f2f zero size 2024-05-12T04:01:07.548 INFO:tasks.workunit.client.0.smithi032.stdout:9/106: dread d0/d3/d6/d7/da/f1a [0,4194304] 0 2024-05-12T04:01:07.549 INFO:tasks.workunit.client.0.smithi032.stdout:2/125: mkdir d1/d13/d25/d2e 0 2024-05-12T04:01:07.550 INFO:tasks.workunit.client.0.smithi032.stdout:5/148: dread d4/fa [0,4194304] 0 2024-05-12T04:01:07.551 INFO:tasks.workunit.client.0.smithi032.stdout:5/149: rename d4/d7/d14 to d4/d7/d14/d31 22 2024-05-12T04:01:07.552 INFO:tasks.workunit.client.0.smithi032.stdout:5/150: chown d4/db/d2b 45702 1 2024-05-12T04:01:07.553 INFO:tasks.workunit.client.1.smithi033.stdout:6/300: readlink d4/d5/l34 0 2024-05-12T04:01:07.553 INFO:tasks.workunit.client.1.smithi033.stdout:9/527: mkdir d2/d4/d2b/dc7 0 2024-05-12T04:01:07.555 INFO:tasks.workunit.client.1.smithi033.stdout:3/310: dwrite db/f54 [0,4194304] 0 2024-05-12T04:01:07.562 INFO:tasks.workunit.client.0.smithi032.stdout:1/118: dread d1/f6 [4194304,4194304] 0 2024-05-12T04:01:07.564 INFO:tasks.workunit.client.0.smithi032.stdout:1/119: fsync d1/f13 0 2024-05-12T04:01:07.567 INFO:tasks.workunit.client.0.smithi032.stdout:4/115: symlink d0/l1a 0 2024-05-12T04:01:07.569 INFO:tasks.workunit.client.0.smithi032.stdout:6/131: write d7/de/f14 [579371,9992] 0 2024-05-12T04:01:07.571 INFO:tasks.workunit.client.1.smithi033.stdout:8/228: getdents d1/d16/d29 0 2024-05-12T04:01:07.572 INFO:tasks.workunit.client.0.smithi032.stdout:3/190: dwrite f12 [0,4194304] 0 2024-05-12T04:01:07.574 INFO:tasks.workunit.client.1.smithi033.stdout:4/332: rename dd/d11/d15 to dd/d11/d31/d66 0 2024-05-12T04:01:07.575 INFO:tasks.workunit.client.1.smithi033.stdout:9/528: rmdir d2/dd/d39 39 2024-05-12T04:01:07.575 INFO:tasks.workunit.client.0.smithi032.stdout:9/107: fdatasync d0/d3/d6/d7/da/d17/f20 0 2024-05-12T04:01:07.576 INFO:tasks.workunit.client.1.smithi033.stdout:9/529: chown d2/d4/d2b/d6d/d4c/f95 10760 1 2024-05-12T04:01:07.577 INFO:tasks.workunit.client.0.smithi032.stdout:5/151: rmdir d4/d7/d21 39 2024-05-12T04:01:07.579 INFO:tasks.workunit.client.0.smithi032.stdout:9/108: truncate d0/d3/d6/d7/da/d10/f12 1286244 0 2024-05-12T04:01:07.580 INFO:tasks.workunit.client.1.smithi033.stdout:0/362: symlink d6/db/d2c/d73/l79 0 2024-05-12T04:01:07.581 INFO:tasks.workunit.client.1.smithi033.stdout:0/363: fsync d6/dc/f18 0 2024-05-12T04:01:07.581 INFO:tasks.workunit.client.1.smithi033.stdout:3/311: rmdir db/d42 39 2024-05-12T04:01:07.583 INFO:tasks.workunit.client.1.smithi033.stdout:5/248: link d3/d27/l1b d3/d12/d29/l50 0 2024-05-12T04:01:07.586 INFO:tasks.workunit.client.0.smithi032.stdout:1/120: creat d1/d16/f28 x:0 0 0 2024-05-12T04:01:07.586 INFO:tasks.workunit.client.0.smithi032.stdout:2/126: dwrite d1/fc [0,4194304] 0 2024-05-12T04:01:07.646 INFO:tasks.workunit.client.1.smithi033.stdout:7/477: sync 2024-05-12T04:01:07.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.651+0000 7f8de12dc700 1 -- 172.21.15.32:0/2665773876 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f8dc8005860 con 0x7f8ddc0fcf60 2024-05-12T04:01:07.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.651+0000 7f8dc37fe700 1 -- 172.21.15.32:0/2665773876 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1828 (secure 0 0 0) 0x7f8dd00583e0 con 0x7f8ddc0fcf60 2024-05-12T04:01:07.651 INFO:teuthology.orchestra.run.smithi032.stdout:e18 2024-05-12T04:01:07.652 INFO:teuthology.orchestra.run.smithi032.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-12T04:01:07.652 INFO:teuthology.orchestra.run.smithi032.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-12T04:01:07.652 INFO:teuthology.orchestra.run.smithi032.stdout:legacy client fscid: 1 2024-05-12T04:01:07.652 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:01:07.652 INFO:teuthology.orchestra.run.smithi032.stdout:Filesystem 'cephfs' (1) 2024-05-12T04:01:07.652 INFO:teuthology.orchestra.run.smithi032.stdout:fs_name cephfs 2024-05-12T04:01:07.652 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 18 2024-05-12T04:01:07.652 INFO:teuthology.orchestra.run.smithi032.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-12T04:01:07.652 INFO:teuthology.orchestra.run.smithi032.stdout:created 2024-05-12T03:57:48.904614+0000 2024-05-12T04:01:07.652 INFO:teuthology.orchestra.run.smithi032.stdout:modified 2024-05-12T03:58:06.821591+0000 2024-05-12T04:01:07.652 INFO:teuthology.orchestra.run.smithi032.stdout:tableserver 0 2024-05-12T04:01:07.652 INFO:teuthology.orchestra.run.smithi032.stdout:root 0 2024-05-12T04:01:07.652 INFO:teuthology.orchestra.run.smithi032.stdout:session_timeout 60 2024-05-12T04:01:07.652 INFO:teuthology.orchestra.run.smithi032.stdout:session_autoclose 300 2024-05-12T04:01:07.652 INFO:teuthology.orchestra.run.smithi032.stdout:max_file_size 1099511627776 2024-05-12T04:01:07.653 INFO:teuthology.orchestra.run.smithi032.stdout:required_client_features {} 2024-05-12T04:01:07.653 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure 0 2024-05-12T04:01:07.653 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure_osd_epoch 46 2024-05-12T04:01:07.653 INFO:teuthology.orchestra.run.smithi032.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-12T04:01:07.653 INFO:teuthology.orchestra.run.smithi032.stdout:max_mds 1 2024-05-12T04:01:07.653 INFO:teuthology.orchestra.run.smithi032.stdout:in 0 2024-05-12T04:01:07.653 INFO:teuthology.orchestra.run.smithi032.stdout:up {0=14474} 2024-05-12T04:01:07.653 INFO:teuthology.orchestra.run.smithi032.stdout:failed 2024-05-12T04:01:07.653 INFO:teuthology.orchestra.run.smithi032.stdout:damaged 2024-05-12T04:01:07.653 INFO:teuthology.orchestra.run.smithi032.stdout:stopped 2024-05-12T04:01:07.653 INFO:teuthology.orchestra.run.smithi032.stdout:data_pools [3] 2024-05-12T04:01:07.653 INFO:teuthology.orchestra.run.smithi032.stdout:metadata_pool 2 2024-05-12T04:01:07.653 INFO:teuthology.orchestra.run.smithi032.stdout:inline_data enabled 2024-05-12T04:01:07.653 INFO:teuthology.orchestra.run.smithi032.stdout:balancer 2024-05-12T04:01:07.653 INFO:teuthology.orchestra.run.smithi032.stdout:standby_count_wanted 1 2024-05-12T04:01:07.654 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.qpsawh{0:14474} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.32:6828/3100155296,v1:172.21.15.32:6829/3100155296] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:01:07.654 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi033.jgbjhk{0:24267} state up:standby-replay seq 3 join_fscid=1 addr [v2:172.21.15.33:6824/55417471,v1:172.21.15.33:6825/55417471] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:01:07.654 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:01:07.654 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:01:07.654 INFO:teuthology.orchestra.run.smithi032.stdout:Standby daemons: 2024-05-12T04:01:07.654 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:01:07.654 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi033.hglwml{-1:24283} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/433858238,v1:172.21.15.33:6827/433858238] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:01:07.654 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.ialtqb{-1:24287} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.32:6826/3703132329,v1:172.21.15.32:6827/3703132329] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:01:07.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.653+0000 7f8de12dc700 1 -- 172.21.15.32:0/2665773876 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f8dc406aec0 msgr2=0x7f8dc406d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:01:07.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.653+0000 7f8de12dc700 1 --2- 172.21.15.32:0/2665773876 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f8dc406aec0 0x7f8dc406d370 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc009fd0 tx=0x7f8dcc00b040 comp rx=0 tx=0).stop 2024-05-12T04:01:07.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.653+0000 7f8de12dc700 1 -- 172.21.15.32:0/2665773876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ddc0fcf60 msgr2=0x7f8ddc10ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:01:07.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.653+0000 7f8de12dc700 1 --2- 172.21.15.32:0/2665773876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ddc0fcf60 0x7f8ddc10ccc0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8dd00137b0 tx=0x7f8dd0009e30 comp rx=0 tx=0).stop 2024-05-12T04:01:07.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.653+0000 7f8de12dc700 1 -- 172.21.15.32:0/2665773876 shutdown_connections 2024-05-12T04:01:07.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.653+0000 7f8de12dc700 1 --2- 172.21.15.32:0/2665773876 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f8dc406aec0 0x7f8dc406d370 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:07.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.653+0000 7f8de12dc700 1 --2- 172.21.15.32:0/2665773876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ddc0fbcb0 0x7f8ddc10c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:07.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.653+0000 7f8de12dc700 1 --2- 172.21.15.32:0/2665773876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ddc0fcf60 0x7f8ddc10ccc0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:07.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.653+0000 7f8de12dc700 1 -- 172.21.15.32:0/2665773876 >> 172.21.15.32:0/2665773876 conn(0x7f8ddc0f7430 msgr2=0x7f8ddc0f8b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:01:07.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.654+0000 7f8de12dc700 1 -- 172.21.15.32:0/2665773876 shutdown_connections 2024-05-12T04:01:07.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.654+0000 7f8de12dc700 1 -- 172.21.15.32:0/2665773876 wait complete. 2024-05-12T04:01:07.656 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 18 2024-05-12T04:01:07.681 INFO:tasks.workunit.client.1.smithi033.stdout:1/281: rename d11/d17/d27/f56 to d11/d17/d27/d36/d3f/f61 0 2024-05-12T04:01:07.685 INFO:tasks.workunit.client.0.smithi032.stdout:4/116: rmdir d0/d3 39 2024-05-12T04:01:07.688 INFO:tasks.workunit.client.1.smithi033.stdout:9/530: creat d2/d4/d2b/d78/fc8 x:0 0 0 2024-05-12T04:01:07.693 INFO:tasks.workunit.client.1.smithi033.stdout:0/364: rmdir d6/dc/d58/d6a 39 2024-05-12T04:01:07.694 INFO:tasks.workunit.client.0.smithi032.stdout:6/132: dwrite d7/f2b [0,4194304] 0 2024-05-12T04:01:07.696 INFO:tasks.workunit.client.0.smithi032.stdout:5/152: unlink d4/d7/d14/d26/l29 0 2024-05-12T04:01:07.696 INFO:tasks.workunit.client.0.smithi032.stdout:6/133: fsync d7/de/d12/f16 0 2024-05-12T04:01:07.697 INFO:tasks.workunit.client.0.smithi032.stdout:5/153: read d4/f22 [4715230,111856] 0 2024-05-12T04:01:07.697 INFO:tasks.workunit.client.1.smithi033.stdout:3/312: symlink db/d17/d25/l6b 0 2024-05-12T04:01:07.698 INFO:tasks.workunit.client.1.smithi033.stdout:3/313: dread - db/d17/f64 zero size 2024-05-12T04:01:07.700 INFO:tasks.workunit.client.0.smithi032.stdout:6/134: chown d7/de/d12/f22 2332 1 2024-05-12T04:01:07.700 INFO:tasks.workunit.client.0.smithi032.stdout:5/154: dread - d4/db/f2f zero size 2024-05-12T04:01:07.700 INFO:tasks.workunit.client.0.smithi032.stdout:8/146: creat d0/d9/f31 x:0 0 0 2024-05-12T04:01:07.701 INFO:tasks.workunit.client.0.smithi032.stdout:6/135: read - d7/de/d12/f18 zero size 2024-05-12T04:01:07.703 INFO:tasks.workunit.client.0.smithi032.stdout:6/136: chown d7/cd 46833 1 2024-05-12T04:01:07.704 INFO:tasks.workunit.client.0.smithi032.stdout:2/127: creat d1/d13/f2f x:0 0 0 2024-05-12T04:01:07.704 INFO:tasks.workunit.client.1.smithi033.stdout:3/314: dwrite db/d17/d25/d32/f46 [0,4194304] 0 2024-05-12T04:01:07.705 INFO:tasks.workunit.client.1.smithi033.stdout:7/478: mknod de/df/d17/d1b/cac 0 2024-05-12T04:01:07.713 INFO:tasks.workunit.client.1.smithi033.stdout:6/301: rename d4/f1f to d4/d43/f62 0 2024-05-12T04:01:07.713 INFO:tasks.workunit.client.0.smithi032.stdout:3/191: dwrite d15/d20/d26/f29 [0,4194304] 0 2024-05-12T04:01:07.714 INFO:tasks.workunit.client.0.smithi032.stdout:3/192: chown d15/d1b/f2f 46862978 1 2024-05-12T04:01:07.716 INFO:tasks.workunit.client.1.smithi033.stdout:8/229: truncate d1/d16/d20/f33 2920397 0 2024-05-12T04:01:07.720 INFO:tasks.workunit.client.0.smithi032.stdout:1/121: mknod d1/dd/d24/c29 0 2024-05-12T04:01:07.720 INFO:tasks.workunit.client.0.smithi032.stdout:1/122: fdatasync d1/f26 0 2024-05-12T04:01:07.721 INFO:tasks.workunit.client.1.smithi033.stdout:6/302: dwrite d4/f56 [0,4194304] 0 2024-05-12T04:01:07.724 INFO:tasks.workunit.client.1.smithi033.stdout:4/333: read dd/d11/d31/d66/d42/f4e [1317855,113001] 0 2024-05-12T04:01:07.724 INFO:tasks.workunit.client.1.smithi033.stdout:9/531: write d2/d70/d7b/f8a [3163691,19657] 0 2024-05-12T04:01:07.726 INFO:tasks.workunit.client.0.smithi032.stdout:2/128: creat d1/d13/f30 x:0 0 0 2024-05-12T04:01:07.727 INFO:tasks.workunit.client.1.smithi033.stdout:0/365: creat d6/db/d2c/d73/f7a x:0 0 0 2024-05-12T04:01:07.728 INFO:tasks.workunit.client.1.smithi033.stdout:0/366: write d6/dc/f18 [439474,109509] 0 2024-05-12T04:01:07.729 INFO:tasks.workunit.client.0.smithi032.stdout:6/137: creat d7/d9/f2f x:0 0 0 2024-05-12T04:01:07.729 INFO:tasks.workunit.client.1.smithi033.stdout:0/367: readlink d6/dc/d58/l65 0 2024-05-12T04:01:07.733 INFO:tasks.workunit.client.0.smithi032.stdout:1/123: dwrite d1/f26 [0,4194304] 0 2024-05-12T04:01:07.733 INFO:tasks.workunit.client.0.smithi032.stdout:1/124: chown d1/d16/c22 219 1 2024-05-12T04:01:07.733 INFO:tasks.workunit.client.0.smithi032.stdout:1/125: stat d1/f13 0 2024-05-12T04:01:07.735 INFO:tasks.workunit.client.0.smithi032.stdout:5/155: write d4/d7/d14/f1d [5141641,63776] 0 2024-05-12T04:01:07.736 INFO:tasks.workunit.client.1.smithi033.stdout:2/324: sync 2024-05-12T04:01:07.736 INFO:tasks.workunit.client.0.smithi032.stdout:5/156: fsync d4/f20 0 2024-05-12T04:01:07.736 INFO:tasks.workunit.client.1.smithi033.stdout:7/479: creat de/df/d17/d1e/d24/fad x:0 0 0 2024-05-12T04:01:07.746 INFO:tasks.workunit.client.0.smithi032.stdout:8/147: symlink d0/l32 0 2024-05-12T04:01:07.746 INFO:tasks.workunit.client.0.smithi032.stdout:8/148: write d0/d9/f12 [1038781,114443] 0 2024-05-12T04:01:07.746 INFO:tasks.workunit.client.0.smithi032.stdout:8/149: write d0/f22 [640902,23805] 0 2024-05-12T04:01:07.746 INFO:tasks.workunit.client.0.smithi032.stdout:8/150: write d0/d9/d13/f20 [926913,39885] 0 2024-05-12T04:01:07.747 INFO:tasks.workunit.client.1.smithi033.stdout:8/230: symlink d1/d16/l4a 0 2024-05-12T04:01:07.747 INFO:tasks.workunit.client.0.smithi032.stdout:6/138: symlink d7/l30 0 2024-05-12T04:01:07.750 INFO:tasks.workunit.client.0.smithi032.stdout:1/126: mknod d1/c2a 0 2024-05-12T04:01:07.751 INFO:tasks.workunit.client.0.smithi032.stdout:6/139: dread d7/f2b [0,4194304] 0 2024-05-12T04:01:07.754 INFO:tasks.workunit.client.0.smithi032.stdout:5/157: creat d4/db/d2b/f32 x:0 0 0 2024-05-12T04:01:07.755 INFO:tasks.workunit.client.1.smithi033.stdout:9/532: creat d2/d4/d2b/d78/d76/fc9 x:0 0 0 2024-05-12T04:01:07.756 INFO:tasks.workunit.client.0.smithi032.stdout:5/158: write d4/db/d1b/f23 [439318,111988] 0 2024-05-12T04:01:07.757 INFO:tasks.workunit.client.0.smithi032.stdout:3/193: link d15/f19 d15/d20/d26/f3c 0 2024-05-12T04:01:07.757 INFO:tasks.workunit.client.0.smithi032.stdout:1/127: dwrite d1/f3 [4194304,4194304] 0 2024-05-12T04:01:07.761 INFO:tasks.workunit.client.0.smithi032.stdout:1/128: dread - d1/d16/f28 zero size 2024-05-12T04:01:07.761 INFO:tasks.workunit.client.1.smithi033.stdout:7/480: mknod de/df/d17/d1e/cae 0 2024-05-12T04:01:07.762 INFO:tasks.workunit.client.0.smithi032.stdout:1/129: dread - d1/d16/f28 zero size 2024-05-12T04:01:07.765 INFO:tasks.workunit.client.0.smithi032.stdout:1/130: dread - d1/d16/f28 zero size 2024-05-12T04:01:07.765 INFO:tasks.workunit.client.0.smithi032.stdout:3/194: dread f3 [0,4194304] 0 2024-05-12T04:01:07.767 INFO:tasks.workunit.client.1.smithi033.stdout:1/282: rmdir d11/d1c/d43/d60 0 2024-05-12T04:01:07.768 INFO:tasks.workunit.client.0.smithi032.stdout:1/131: chown d1/d7/c12 0 1 2024-05-12T04:01:07.768 INFO:tasks.workunit.client.0.smithi032.stdout:6/140: symlink d7/l31 0 2024-05-12T04:01:07.774 INFO:tasks.workunit.client.1.smithi033.stdout:4/334: mknod dd/d11/c67 0 2024-05-12T04:01:07.774 INFO:tasks.workunit.client.0.smithi032.stdout:5/159: mkdir d4/d7/d14/d26/d33 0 2024-05-12T04:01:07.774 INFO:tasks.workunit.client.0.smithi032.stdout:2/129: creat d1/f31 x:0 0 0 2024-05-12T04:01:07.775 INFO:tasks.workunit.client.0.smithi032.stdout:5/160: write d4/db/d1b/f23 [981320,86684] 0 2024-05-12T04:01:07.776 INFO:tasks.workunit.client.1.smithi033.stdout:5/249: getdents d3/d12/d25 0 2024-05-12T04:01:07.776 INFO:tasks.workunit.client.0.smithi032.stdout:5/161: readlink d4/l27 0 2024-05-12T04:01:07.777 INFO:tasks.workunit.client.1.smithi033.stdout:9/533: dread d2/d4/d2b/d6d/d4c/f95 [0,4194304] 0 2024-05-12T04:01:07.778 INFO:tasks.workunit.client.0.smithi032.stdout:3/195: creat d15/d24/f3d x:0 0 0 2024-05-12T04:01:07.778 INFO:tasks.workunit.client.1.smithi033.stdout:2/325: mkdir dc/d17/d74 0 2024-05-12T04:01:07.779 INFO:tasks.workunit.client.1.smithi033.stdout:2/326: chown dc/d17/c52 62 1 2024-05-12T04:01:07.779 INFO:tasks.workunit.client.0.smithi032.stdout:8/151: dwrite d0/f2 [4194304,4194304] 0 2024-05-12T04:01:07.781 INFO:tasks.workunit.client.1.smithi033.stdout:7/481: stat de/d20/d38/c3b 0 2024-05-12T04:01:07.782 INFO:tasks.workunit.client.0.smithi032.stdout:5/162: write d4/f1e [1227518,49228] 0 2024-05-12T04:01:07.782 INFO:tasks.workunit.client.0.smithi032.stdout:6/141: truncate f3 1581138 0 2024-05-12T04:01:07.784 INFO:tasks.workunit.client.0.smithi032.stdout:1/132: dread d1/f26 [0,4194304] 0 2024-05-12T04:01:07.785 INFO:tasks.workunit.client.1.smithi033.stdout:3/315: rename l5 to db/d17/d25/d32/d3c/l6c 0 2024-05-12T04:01:07.788 INFO:tasks.workunit.client.1.smithi033.stdout:1/283: unlink l7 0 2024-05-12T04:01:07.791 INFO:tasks.workunit.client.1.smithi033.stdout:4/335: fsync dd/f32 0 2024-05-12T04:01:07.795 INFO:tasks.workunit.client.1.smithi033.stdout:2/327: write dc/d17/f4d [3805820,30463] 0 2024-05-12T04:01:07.796 INFO:tasks.workunit.client.1.smithi033.stdout:2/328: fsync f3 0 2024-05-12T04:01:07.797 INFO:tasks.workunit.client.0.smithi032.stdout:3/196: rename d15/d20/d22/f2c to d15/d24/f3e 0 2024-05-12T04:01:07.799 INFO:tasks.workunit.client.1.smithi033.stdout:8/231: rename d1/d16/d20/f2c to d1/d4/d2e/f4b 0 2024-05-12T04:01:07.805 INFO:tasks.workunit.client.1.smithi033.stdout:6/303: truncate d4/f51 417272 0 2024-05-12T04:01:07.806 INFO:tasks.workunit.client.1.smithi033.stdout:6/304: write d4/d1a/f32 [668935,15632] 0 2024-05-12T04:01:07.806 INFO:tasks.workunit.client.0.smithi032.stdout:8/152: truncate d0/d9/f1c 913050 0 2024-05-12T04:01:07.808 INFO:tasks.workunit.client.1.smithi033.stdout:6/305: read d4/d1a/d4e/d4f/f50 [62799,106140] 0 2024-05-12T04:01:07.809 INFO:tasks.workunit.client.1.smithi033.stdout:0/368: rename d6/dc/l1e to d6/dc/d58/d6a/l7b 0 2024-05-12T04:01:07.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.811+0000 7f1365c91700 1 -- 172.21.15.32:0/2418444895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13600fbcc0 msgr2=0x7f13600fc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:01:07.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.811+0000 7f1365c91700 1 --2- 172.21.15.32:0/2418444895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13600fbcc0 0x7f13600fc0d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1350004940 tx=0x7f1350009e30 comp rx=0 tx=0).stop 2024-05-12T04:01:07.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.811+0000 7f1365c91700 1 -- 172.21.15.32:0/2418444895 shutdown_connections 2024-05-12T04:01:07.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.811+0000 7f1365c91700 1 --2- 172.21.15.32:0/2418444895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13600fcf70 0x7f13600fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:07.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.811+0000 7f1365c91700 1 --2- 172.21.15.32:0/2418444895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13600fbcc0 0x7f13600fc0d0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:07.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.811+0000 7f1365c91700 1 -- 172.21.15.32:0/2418444895 >> 172.21.15.32:0/2418444895 conn(0x7f13600f7440 msgr2=0x7f13600f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:01:07.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.811+0000 7f1365c91700 1 -- 172.21.15.32:0/2418444895 shutdown_connections 2024-05-12T04:01:07.812 INFO:tasks.workunit.client.0.smithi032.stdout:3/197: mkdir d15/d33/d3f 0 2024-05-12T04:01:07.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.811+0000 7f1365c91700 1 -- 172.21.15.32:0/2418444895 wait complete. 2024-05-12T04:01:07.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.812+0000 7f1365c91700 1 Processor -- start 2024-05-12T04:01:07.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.812+0000 7f1365c91700 1 -- start start 2024-05-12T04:01:07.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.812+0000 7f1365c91700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13600fbcc0 0x7f136010b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:01:07.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.812+0000 7f1365c91700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13600fcf70 0x7f136010bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:01:07.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.812+0000 7f1365c91700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136010c1d0 con 0x7f13600fcf70 2024-05-12T04:01:07.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.812+0000 7f1365c91700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136010c340 con 0x7f13600fbcc0 2024-05-12T04:01:07.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.812+0000 7f135effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13600fcf70 0x7f136010bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:01:07.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.813+0000 7f135effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13600fcf70 0x7f136010bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53064/0 (socket says 172.21.15.32:53064) 2024-05-12T04:01:07.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.813+0000 7f135effd700 1 -- 172.21.15.32:0/2308517047 learned_addr learned my addr 172.21.15.32:0/2308517047 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:01:07.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.813+0000 7f135f7fe700 1 --2- 172.21.15.32:0/2308517047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13600fbcc0 0x7f136010b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:01:07.814 INFO:tasks.workunit.client.0.smithi032.stdout:3/198: write d15/f1c [3526130,30204] 0 2024-05-12T04:01:07.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.813+0000 7f135effd700 1 -- 172.21.15.32:0/2308517047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13600fbcc0 msgr2=0x7f136010b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:01:07.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.813+0000 7f135effd700 1 --2- 172.21.15.32:0/2308517047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13600fbcc0 0x7f136010b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:07.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.813+0000 7f135effd700 1 -- 172.21.15.32:0/2308517047 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1354005040 con 0x7f13600fcf70 2024-05-12T04:01:07.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.813+0000 7f135effd700 1 --2- 172.21.15.32:0/2308517047 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13600fcf70 0x7f136010bbd0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f13540096d0 tx=0x7f1354009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:01:07.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.813+0000 7f1347fff700 1 -- 172.21.15.32:0/2308517047 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f135400e840 con 0x7f13600fcf70 2024-05-12T04:01:07.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.814+0000 7f1347fff700 1 -- 172.21.15.32:0/2308517047 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f135400f040 con 0x7f13600fcf70 2024-05-12T04:01:07.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.814+0000 7f1347fff700 1 -- 172.21.15.32:0/2308517047 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1354013d30 con 0x7f13600fcf70 2024-05-12T04:01:07.815 INFO:tasks.workunit.client.0.smithi032.stdout:5/163: mkdir d4/d34 0 2024-05-12T04:01:07.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.814+0000 7f1365c91700 1 -- 172.21.15.32:0/2308517047 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1350005040 con 0x7f13600fcf70 2024-05-12T04:01:07.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.815+0000 7f1365c91700 1 -- 172.21.15.32:0/2308517047 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13601110f0 con 0x7f13600fcf70 2024-05-12T04:01:07.816 INFO:tasks.workunit.client.0.smithi032.stdout:5/164: readlink d4/db/l18 0 2024-05-12T04:01:07.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.816+0000 7f1365c91700 1 -- 172.21.15.32:0/2308517047 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1360104e80 con 0x7f13600fcf70 2024-05-12T04:01:07.817 INFO:tasks.workunit.client.0.smithi032.stdout:5/165: write d4/f1e [1670396,39778] 0 2024-05-12T04:01:07.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.817+0000 7f1347fff700 1 -- 172.21.15.32:0/2308517047 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 25) v1 ==== 88772+0+0 (secure 0 0 0) 0x7f135400e9a0 con 0x7f13600fcf70 2024-05-12T04:01:07.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.817+0000 7f1347fff700 1 --2- 172.21.15.32:0/2308517047 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f134806aed0 0x7f134806d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:01:07.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.817+0000 7f1347fff700 1 -- 172.21.15.32:0/2308517047 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f13540898e0 con 0x7f13600fcf70 2024-05-12T04:01:07.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.818+0000 7f135f7fe700 1 --2- 172.21.15.32:0/2308517047 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f134806aed0 0x7f134806d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:01:07.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.818+0000 7f135f7fe700 1 --2- 172.21.15.32:0/2308517047 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f134806aed0 0x7f134806d380 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f13500045a0 tx=0x7f1350009e30 comp rx=0 tx=0).ready entity=mgr.24389 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:01:07.820 INFO:tasks.workunit.client.1.smithi033.stdout:6/306: dwrite d4/d43/f62 [0,4194304] 0 2024-05-12T04:01:07.821 INFO:tasks.workunit.client.1.smithi033.stdout:6/307: readlink d4/l19 0 2024-05-12T04:01:07.822 INFO:tasks.workunit.client.1.smithi033.stdout:5/250: rename d3/d12/d25/d26/d2e/f36 to d3/d15/d16/f51 0 2024-05-12T04:01:07.824 INFO:tasks.workunit.client.1.smithi033.stdout:5/251: write d3/d27/f28 [1185396,75909] 0 2024-05-12T04:01:07.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:07.823+0000 7f1347fff700 1 -- 172.21.15.32:0/2308517047 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181777 (secure 0 0 0) 0x7f1354057880 con 0x7f13600fcf70 2024-05-12T04:01:07.825 INFO:tasks.workunit.client.0.smithi032.stdout:6/142: truncate d7/de/d12/f16 3998444 0 2024-05-12T04:01:07.825 INFO:tasks.workunit.client.0.smithi032.stdout:6/143: stat d7/fa 0 2024-05-12T04:01:07.825 INFO:tasks.workunit.client.1.smithi033.stdout:5/252: write d3/d15/d16/f42 [574582,63260] 0 2024-05-12T04:01:07.827 INFO:tasks.workunit.client.1.smithi033.stdout:6/308: creat d4/d1a/d4e/d4f/f63 x:0 0 0 2024-05-12T04:01:07.832 INFO:tasks.workunit.client.1.smithi033.stdout:5/253: read d3/f3b [937791,70688] 0 2024-05-12T04:01:07.833 INFO:tasks.workunit.client.1.smithi033.stdout:5/254: chown d3/d12/d29/d2b/c2d 7 1 2024-05-12T04:01:07.835 INFO:tasks.workunit.client.1.smithi033.stdout:2/329: write dc/d17/f63 [1126734,11744] 0 2024-05-12T04:01:07.837 INFO:tasks.workunit.client.1.smithi033.stdout:1/284: rename d11/d17/d27/d36/c4b to d11/d1c/d43/c62 0 2024-05-12T04:01:07.838 INFO:tasks.workunit.client.0.smithi032.stdout:3/199: mkdir d15/d24/d40 0 2024-05-12T04:01:07.846 INFO:tasks.workunit.client.0.smithi032.stdout:5/166: mknod d4/db/c35 0 2024-05-12T04:01:07.849 INFO:tasks.workunit.client.1.smithi033.stdout:6/309: symlink d4/d43/d57/l64 0 2024-05-12T04:01:07.849 INFO:tasks.workunit.client.1.smithi033.stdout:6/310: chown d4/d1a/d4e/c52 26563 1 2024-05-12T04:01:07.849 INFO:tasks.workunit.client.0.smithi032.stdout:6/144: creat d7/f32 x:0 0 0 2024-05-12T04:01:07.860 INFO:tasks.workunit.client.1.smithi033.stdout:2/330: unlink ca 0 2024-05-12T04:01:07.863 INFO:tasks.workunit.client.0.smithi032.stdout:8/153: write d0/d7/f15 [807211,90472] 0 2024-05-12T04:01:07.863 INFO:tasks.workunit.client.0.smithi032.stdout:5/167: symlink d4/db/l36 0 2024-05-12T04:01:07.864 INFO:tasks.workunit.client.0.smithi032.stdout:8/154: readlink d0/d9/d13/l18 0 2024-05-12T04:01:07.864 INFO:tasks.workunit.client.1.smithi033.stdout:2/331: dread dc/d17/f63 [0,4194304] 0 2024-05-12T04:01:07.867 INFO:tasks.workunit.client.1.smithi033.stdout:4/336: rename dd/f3d to dd/d11/d31/d38/f68 0 2024-05-12T04:01:07.868 INFO:tasks.workunit.client.0.smithi032.stdout:6/145: mkdir d7/d33 0 2024-05-12T04:01:07.870 INFO:tasks.workunit.client.1.smithi033.stdout:1/285: dwrite d11/d17/d27/f2b [0,4194304] 0 2024-05-12T04:01:07.870 INFO:tasks.workunit.client.1.smithi033.stdout:7/482: sync 2024-05-12T04:01:07.870 INFO:tasks.workunit.client.1.smithi033.stdout:9/534: sync 2024-05-12T04:01:07.870 INFO:tasks.workunit.client.1.smithi033.stdout:3/316: sync 2024-05-12T04:01:07.871 INFO:tasks.workunit.client.0.smithi032.stdout:1/133: getdents d1/d7 0 2024-05-12T04:01:07.875 INFO:tasks.workunit.client.1.smithi033.stdout:6/311: creat d4/d1a/d3c/f65 x:0 0 0 2024-05-12T04:01:07.875 INFO:tasks.workunit.client.1.smithi033.stdout:3/317: write db/d17/d3e/f48 [1131461,109724] 0 2024-05-12T04:01:07.877 INFO:tasks.workunit.client.1.smithi033.stdout:2/332: dwrite dc/d17/d27/d47/f5d [0,4194304] 0 2024-05-12T04:01:07.881 INFO:tasks.workunit.client.1.smithi033.stdout:1/286: dwrite d11/d17/d27/f31 [0,4194304] 0 2024-05-12T04:01:07.881 INFO:tasks.workunit.client.0.smithi032.stdout:0/206: sync 2024-05-12T04:01:07.882 INFO:tasks.workunit.client.0.smithi032.stdout:7/120: sync 2024-05-12T04:01:07.882 INFO:tasks.workunit.client.0.smithi032.stdout:7/121: write dd/d1b/f2d [111691,101476] 0 2024-05-12T04:01:07.882 INFO:tasks.workunit.client.1.smithi033.stdout:1/287: dread - d11/d1c/f32 zero size 2024-05-12T04:01:07.887 INFO:tasks.workunit.client.1.smithi033.stdout:8/232: rename d1/c25 to d1/d4/d2e/c4c 0 2024-05-12T04:01:07.894 INFO:tasks.workunit.client.0.smithi032.stdout:8/155: unlink d0/d9/d13/f1f 0 2024-05-12T04:01:07.899 INFO:tasks.workunit.client.0.smithi032.stdout:8/156: dread d0/d9/d13/f29 [0,4194304] 0 2024-05-12T04:01:07.900 INFO:tasks.workunit.client.0.smithi032.stdout:6/146: unlink d7/de/d12/d1a/l1f 0 2024-05-12T04:01:07.909 INFO:tasks.workunit.client.0.smithi032.stdout:3/200: truncate d15/d1b/f34 2761359 0 2024-05-12T04:01:07.909 INFO:tasks.workunit.client.0.smithi032.stdout:3/201: stat ca 0 2024-05-12T04:01:07.911 INFO:tasks.workunit.client.0.smithi032.stdout:5/168: write d4/fa [1088486,59274] 0 2024-05-12T04:01:07.912 INFO:tasks.workunit.client.0.smithi032.stdout:5/169: readlink d4/d7/l24 0 2024-05-12T04:01:07.914 INFO:tasks.workunit.client.0.smithi032.stdout:5/170: write d4/f20 [237150,14042] 0 2024-05-12T04:01:07.915 INFO:tasks.workunit.client.0.smithi032.stdout:3/202: dread d15/d20/d26/f29 [0,4194304] 0 2024-05-12T04:01:07.915 INFO:tasks.workunit.client.0.smithi032.stdout:5/171: chown d4/db/f2f 6 1 2024-05-12T04:01:07.917 INFO:tasks.workunit.client.0.smithi032.stdout:7/122: truncate f5 341896 0 2024-05-12T04:01:07.919 INFO:tasks.workunit.client.1.smithi033.stdout:6/312: creat d4/d1a/d4e/f66 x:0 0 0 2024-05-12T04:01:07.921 INFO:tasks.workunit.client.1.smithi033.stdout:2/333: creat dc/d17/d27/d2f/f75 x:0 0 0 2024-05-12T04:01:07.923 INFO:tasks.workunit.client.0.smithi032.stdout:8/157: mkdir d0/d9/d1b/d33 0 2024-05-12T04:01:07.924 INFO:tasks.workunit.client.1.smithi033.stdout:4/337: mkdir dd/d69 0 2024-05-12T04:01:07.925 INFO:tasks.workunit.client.0.smithi032.stdout:8/158: write d0/f2f [968498,76980] 0 2024-05-12T04:01:07.926 INFO:tasks.workunit.client.1.smithi033.stdout:4/338: read dd/d11/d31/d66/d42/f4e [643287,56514] 0 2024-05-12T04:01:07.926 INFO:tasks.workunit.client.0.smithi032.stdout:1/134: dwrite d1/f6 [8388608,4194304] 0 2024-05-12T04:01:07.928 INFO:tasks.workunit.client.0.smithi032.stdout:6/147: rename d7/d9/f2f to d7/de/d12/d1a/d20/f34 0 2024-05-12T04:01:07.932 INFO:tasks.workunit.client.1.smithi033.stdout:8/233: mkdir d1/d16/d27/d4d 0 2024-05-12T04:01:07.932 INFO:tasks.workunit.client.1.smithi033.stdout:8/234: chown d1/d4/cc 42607 1 2024-05-12T04:01:07.932 INFO:tasks.workunit.client.0.smithi032.stdout:0/207: mknod df/d10/d3d/c49 0 2024-05-12T04:01:07.938 INFO:tasks.workunit.client.0.smithi032.stdout:5/172: dread d4/fd [0,4194304] 0 2024-05-12T04:01:07.939 INFO:tasks.workunit.client.1.smithi033.stdout:8/235: dwrite d1/d16/d20/f2f [0,4194304] 0 2024-05-12T04:01:07.942 INFO:tasks.workunit.client.0.smithi032.stdout:8/159: dread d0/d7/f15 [0,4194304] 0 2024-05-12T04:01:07.945 INFO:tasks.workunit.client.0.smithi032.stdout:1/135: dwrite d1/d16/d18/f1b [0,4194304] 0 2024-05-12T04:01:07.947 INFO:tasks.workunit.client.1.smithi033.stdout:3/318: mkdir db/d17/d3a/d67/d6d 0 2024-05-12T04:01:07.950 INFO:tasks.workunit.client.0.smithi032.stdout:8/160: dread d0/d9/d13/f20 [0,4194304] 0 2024-05-12T04:01:07.952 INFO:tasks.workunit.client.1.smithi033.stdout:6/313: mkdir d4/d1a/d67 0 2024-05-12T04:01:07.953 INFO:tasks.workunit.client.0.smithi032.stdout:8/161: fsync d0/d7/f15 0 2024-05-12T04:01:07.958 INFO:tasks.workunit.client.1.smithi033.stdout:1/288: mknod d11/d17/d27/d36/d46/d4c/c63 0 2024-05-12T04:01:07.970 INFO:tasks.workunit.client.1.smithi033.stdout:9/535: rmdir d2/d4/d2b/d49/d73/da8 0 2024-05-12T04:01:07.971 INFO:tasks.workunit.client.1.smithi033.stdout:9/536: write d2/d4/f15 [9340395,17760] 0 2024-05-12T04:01:07.973 INFO:tasks.workunit.client.1.smithi033.stdout:8/236: mknod d1/d4/d1d/d36/c4e 0 2024-05-12T04:01:07.974 INFO:tasks.workunit.client.1.smithi033.stdout:5/255: link d3/d15/d16/f1a d3/f52 0 2024-05-12T04:01:07.979 INFO:tasks.workunit.client.1.smithi033.stdout:6/314: fsync d4/d1a/d4e/d4f/f50 0 2024-05-12T04:01:07.980 INFO:tasks.workunit.client.1.smithi033.stdout:6/315: write d4/f2f [1607662,130868] 0 2024-05-12T04:01:07.984 INFO:tasks.workunit.client.1.smithi033.stdout:1/289: write fd [3167637,23692] 0 2024-05-12T04:01:07.985 INFO:tasks.workunit.client.0.smithi032.stdout:5/173: dwrite d4/fd [0,4194304] 0 2024-05-12T04:01:07.986 INFO:tasks.workunit.client.0.smithi032.stdout:1/136: creat d1/d16/d18/f2b x:0 0 0 2024-05-12T04:01:07.987 INFO:tasks.workunit.client.0.smithi032.stdout:1/137: dread - d1/d16/f28 zero size 2024-05-12T04:01:07.992 INFO:tasks.workunit.client.1.smithi033.stdout:0/369: rename d6/dc/d39/f4f to d6/f7c 0 2024-05-12T04:01:07.996 INFO:tasks.workunit.client.1.smithi033.stdout:9/537: mkdir d2/d4/d2b/d6d/d3a/dca 0 2024-05-12T04:01:07.997 INFO:tasks.workunit.client.1.smithi033.stdout:8/237: creat d1/d4/d46/f4f x:0 0 0 2024-05-12T04:01:08.005 INFO:tasks.workunit.client.0.smithi032.stdout:6/148: dwrite f6 [0,4194304] 0 2024-05-12T04:01:08.006 INFO:tasks.workunit.client.1.smithi033.stdout:4/339: creat dd/d11/d31/d66/f6a x:0 0 0 2024-05-12T04:01:08.013 INFO:tasks.workunit.client.0.smithi032.stdout:1/138: rename d1/f6 to d1/d16/f2c 0 2024-05-12T04:01:08.014 INFO:tasks.workunit.client.0.smithi032.stdout:1/139: write d1/f13 [3392265,124217] 0 2024-05-12T04:01:08.016 INFO:tasks.workunit.client.0.smithi032.stdout:1/140: chown d1/d7/ce 0 1 2024-05-12T04:01:08.016 INFO:tasks.workunit.client.1.smithi033.stdout:1/290: rename d11/f21 to d11/d1c/f64 0 2024-05-12T04:01:08.017 INFO:tasks.workunit.client.1.smithi033.stdout:1/291: readlink d11/d1c/l1f 0 2024-05-12T04:01:08.018 INFO:tasks.workunit.client.0.smithi032.stdout:0/208: rmdir df/d30/d36/d47 0 2024-05-12T04:01:08.020 INFO:tasks.workunit.client.1.smithi033.stdout:0/370: symlink d6/dc/d58/d6a/l7d 0 2024-05-12T04:01:08.021 INFO:tasks.workunit.client.0.smithi032.stdout:3/203: getdents d15/d20/d22 0 2024-05-12T04:01:08.021 INFO:tasks.workunit.client.1.smithi033.stdout:0/371: read d6/d11/f47 [512688,25961] 0 2024-05-12T04:01:08.022 INFO:tasks.workunit.client.0.smithi032.stdout:3/204: truncate d15/d24/f3d 692931 0 2024-05-12T04:01:08.023 INFO:tasks.workunit.client.1.smithi033.stdout:9/538: creat d2/d4/d2b/d6d/db5/fcb x:0 0 0 2024-05-12T04:01:08.026 INFO:tasks.workunit.client.1.smithi033.stdout:5/256: dwrite d3/f2c [0,4194304] 0 2024-05-12T04:01:08.030 INFO:tasks.workunit.client.1.smithi033.stdout:6/316: mkdir d4/d1a/d67/d68 0 2024-05-12T04:01:08.031 INFO:tasks.workunit.client.0.smithi032.stdout:3/205: dwrite d15/d20/d22/f2e [0,4194304] 0 2024-05-12T04:01:08.034 INFO:tasks.workunit.client.1.smithi033.stdout:7/483: sync 2024-05-12T04:01:08.037 INFO:tasks.workunit.client.0.smithi032.stdout:8/162: dwrite d0/d9/f1c [0,4194304] 0 2024-05-12T04:01:08.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:07 smithi032 ceph-mon[40008]: from='client.24463 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:01:08.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:07 smithi032 ceph-mon[40008]: Reconfiguring grafana.smithi032 (dependencies changed)... 2024-05-12T04:01:08.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:07 smithi032 ceph-mon[40008]: Reconfiguring daemon grafana.smithi032 on smithi032 2024-05-12T04:01:08.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:07 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/1059603998' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:01:08.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:07 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/2665773876' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-12T04:01:08.041 INFO:tasks.workunit.client.0.smithi032.stdout:8/163: write d0/d9/d13/d21/d25/f2c [892052,111744] 0 2024-05-12T04:01:08.045 INFO:tasks.workunit.client.0.smithi032.stdout:6/149: rename d7/de/d12/f27 to d7/d33/f35 0 2024-05-12T04:01:08.045 INFO:tasks.workunit.client.0.smithi032.stdout:7/123: write f5 [561376,116494] 0 2024-05-12T04:01:08.048 INFO:tasks.workunit.client.0.smithi032.stdout:1/141: symlink d1/dd/d24/l2d 0 2024-05-12T04:01:08.049 INFO:tasks.workunit.client.1.smithi033.stdout:8/238: rename d1/d4/f2b to d1/d4/d1d/d36/f50 0 2024-05-12T04:01:08.050 INFO:tasks.workunit.client.0.smithi032.stdout:1/142: write d1/f3 [2502839,75164] 0 2024-05-12T04:01:08.051 INFO:tasks.workunit.client.1.smithi033.stdout:8/239: dread - d1/d16/d20/f43 zero size 2024-05-12T04:01:08.053 INFO:tasks.workunit.client.0.smithi032.stdout:8/164: dread d0/d9/d13/d21/d25/f2c [0,4194304] 0 2024-05-12T04:01:08.054 INFO:tasks.workunit.client.0.smithi032.stdout:8/165: dread - d0/d9/f2d zero size 2024-05-12T04:01:08.054 INFO:tasks.workunit.client.1.smithi033.stdout:1/292: symlink d11/d17/d27/d36/d3f/l65 0 2024-05-12T04:01:08.059 INFO:tasks.workunit.client.1.smithi033.stdout:4/340: dwrite dd/d11/f21 [0,4194304] 0 2024-05-12T04:01:08.060 INFO:tasks.workunit.client.1.smithi033.stdout:0/372: unlink d6/d11/f6f 0 2024-05-12T04:01:08.061 INFO:tasks.workunit.client.0.smithi032.stdout:8/166: dwrite d0/d9/d13/d21/f2a [0,4194304] 0 2024-05-12T04:01:08.066 INFO:tasks.workunit.client.1.smithi033.stdout:5/257: fdatasync d3/f3b 0 2024-05-12T04:01:08.068 INFO:tasks.workunit.client.1.smithi033.stdout:4/341: dwrite dd/d11/d31/f45 [0,4194304] 0 2024-05-12T04:01:08.068 INFO:tasks.workunit.client.1.smithi033.stdout:5/258: stat d3/d27 0 2024-05-12T04:01:08.069 INFO:tasks.workunit.client.0.smithi032.stdout:3/206: unlink d15/f18 0 2024-05-12T04:01:08.069 INFO:tasks.workunit.client.1.smithi033.stdout:6/317: symlink d4/l69 0 2024-05-12T04:01:08.073 INFO:tasks.workunit.client.0.smithi032.stdout:0/209: rename df/d10/d1f to df/d10/d3d/d4a 0 2024-05-12T04:01:08.078 INFO:tasks.workunit.client.1.smithi033.stdout:2/334: rename dc/d17/d27/d47/l73 to dc/d3b/l76 0 2024-05-12T04:01:08.080 INFO:tasks.workunit.client.1.smithi033.stdout:8/240: read d1/d4/d1d/f26 [1368701,12738] 0 2024-05-12T04:01:08.081 INFO:tasks.workunit.client.1.smithi033.stdout:8/241: chown d1/c11 0 1 2024-05-12T04:01:08.083 INFO:tasks.workunit.client.1.smithi033.stdout:1/293: creat d11/d17/d27/d36/d46/f66 x:0 0 0 2024-05-12T04:01:08.084 INFO:tasks.workunit.client.0.smithi032.stdout:1/143: symlink d1/dd/l2e 0 2024-05-12T04:01:08.084 INFO:tasks.workunit.client.0.smithi032.stdout:1/144: chown d1/f26 3438 1 2024-05-12T04:01:08.088 INFO:tasks.workunit.client.0.smithi032.stdout:5/174: rmdir d4/d7/d21/d2c 0 2024-05-12T04:01:08.088 INFO:tasks.workunit.client.1.smithi033.stdout:8/242: dwrite d1/d4/d46/f4f [0,4194304] 0 2024-05-12T04:01:08.089 INFO:tasks.workunit.client.1.smithi033.stdout:8/243: stat d1/d16/l1a 0 2024-05-12T04:01:08.090 INFO:tasks.workunit.client.1.smithi033.stdout:0/373: creat d6/dc/d58/d6a/f7e x:0 0 0 2024-05-12T04:01:08.091 INFO:tasks.workunit.client.0.smithi032.stdout:8/167: dwrite d0/d9/f1c [4194304,4194304] 0 2024-05-12T04:01:08.093 INFO:tasks.workunit.client.1.smithi033.stdout:8/244: dread d1/d4/d1d/f26 [0,4194304] 0 2024-05-12T04:01:08.094 INFO:tasks.workunit.client.1.smithi033.stdout:4/342: readlink dd/d11/d31/l50 0 2024-05-12T04:01:08.095 INFO:tasks.workunit.client.1.smithi033.stdout:3/319: sync 2024-05-12T04:01:08.096 INFO:tasks.workunit.client.0.smithi032.stdout:3/207: mknod d15/d33/c41 0 2024-05-12T04:01:08.096 INFO:tasks.workunit.client.1.smithi033.stdout:3/320: chown db/d17/d25/l6a 1751 1 2024-05-12T04:01:08.099 INFO:tasks.workunit.client.0.smithi032.stdout:6/150: rename d7/de/d12/d1a/f24 to d7/de/d2e/f36 0 2024-05-12T04:01:08.106 INFO:tasks.workunit.client.1.smithi033.stdout:6/318: rename d4/d1a/f32 to d4/d1a/d67/d68/f6a 0 2024-05-12T04:01:08.107 INFO:tasks.workunit.client.1.smithi033.stdout:5/259: dwrite d3/d12/d25/d26/d2e/f49 [0,4194304] 0 2024-05-12T04:01:08.109 INFO:tasks.workunit.client.1.smithi033.stdout:2/335: creat dc/d17/d27/d47/d5e/f77 x:0 0 0 2024-05-12T04:01:08.110 INFO:tasks.workunit.client.1.smithi033.stdout:2/336: read dc/d17/d27/d2f/f62 [1832376,101128] 0 2024-05-12T04:01:08.112 INFO:tasks.workunit.client.1.smithi033.stdout:2/337: dread - dc/d17/d27/d47/d5e/f77 zero size 2024-05-12T04:01:08.115 INFO:tasks.workunit.client.0.smithi032.stdout:8/168: read d0/d9/d13/f29 [441394,104993] 0 2024-05-12T04:01:08.119 INFO:tasks.workunit.client.0.smithi032.stdout:7/124: truncate f6 405926 0 2024-05-12T04:01:08.120 INFO:tasks.workunit.client.0.smithi032.stdout:7/125: write fb [835582,57322] 0 2024-05-12T04:01:08.121 INFO:tasks.workunit.client.0.smithi032.stdout:7/126: fdatasync dd/d1b/d1e/f28 0 2024-05-12T04:01:08.122 INFO:tasks.workunit.client.0.smithi032.stdout:0/210: rename df/d22/l27 to df/d28/d3a/l4b 0 2024-05-12T04:01:08.136 INFO:tasks.workunit.client.0.smithi032.stdout:3/208: mknod d15/c42 0 2024-05-12T04:01:08.137 INFO:tasks.workunit.client.0.smithi032.stdout:3/209: stat d15/f17 0 2024-05-12T04:01:08.139 INFO:tasks.workunit.client.0.smithi032.stdout:8/169: write d0/d7/f11 [1467243,45995] 0 2024-05-12T04:01:08.140 INFO:tasks.workunit.client.0.smithi032.stdout:6/151: dwrite d7/f2b [0,4194304] 0 2024-05-12T04:01:08.141 INFO:tasks.workunit.client.0.smithi032.stdout:8/170: read d0/d9/f1c [7392766,31976] 0 2024-05-12T04:01:08.142 INFO:tasks.workunit.client.0.smithi032.stdout:4/117: sync 2024-05-12T04:01:08.142 INFO:tasks.workunit.client.0.smithi032.stdout:9/109: sync 2024-05-12T04:01:08.143 INFO:tasks.workunit.client.1.smithi033.stdout:8/245: rename d1/d4/d2e to d1/d16/d27/d4d/d51 0 2024-05-12T04:01:08.147 INFO:tasks.workunit.client.1.smithi033.stdout:6/319: symlink d4/d43/d57/l6b 0 2024-05-12T04:01:08.148 INFO:tasks.workunit.client.0.smithi032.stdout:6/152: fsync d7/de/d12/f18 0 2024-05-12T04:01:08.149 INFO:tasks.workunit.client.0.smithi032.stdout:3/210: dwrite d15/d20/d22/f2e [4194304,4194304] 0 2024-05-12T04:01:08.150 INFO:tasks.workunit.client.1.smithi033.stdout:5/260: mknod d3/d12/d25/c53 0 2024-05-12T04:01:08.151 INFO:tasks.workunit.client.0.smithi032.stdout:7/127: write dd/fe [214337,121428] 0 2024-05-12T04:01:08.152 INFO:tasks.workunit.client.0.smithi032.stdout:5/175: rename d4/c9 to d4/d34/c37 0 2024-05-12T04:01:08.153 INFO:tasks.workunit.client.0.smithi032.stdout:5/176: readlink d4/d7/l2d 0 2024-05-12T04:01:08.158 INFO:tasks.workunit.client.1.smithi033.stdout:2/338: write dc/d13/d1c/f21 [210962,56874] 0 2024-05-12T04:01:08.158 INFO:tasks.workunit.client.0.smithi032.stdout:5/177: write d4/f20 [909331,26324] 0 2024-05-12T04:01:08.158 INFO:tasks.workunit.client.0.smithi032.stdout:0/211: rmdir df/d10/d3d/d4a 39 2024-05-12T04:01:08.160 INFO:tasks.workunit.client.0.smithi032.stdout:0/212: dread df/d10/f3c [0,4194304] 0 2024-05-12T04:01:08.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:07 smithi033 ceph-mon[49525]: from='client.24463 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:01:08.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:07 smithi033 ceph-mon[49525]: Reconfiguring grafana.smithi032 (dependencies changed)... 2024-05-12T04:01:08.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:07 smithi033 ceph-mon[49525]: Reconfiguring daemon grafana.smithi032 on smithi032 2024-05-12T04:01:08.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:07 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/1059603998' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:01:08.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:07 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/2665773876' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-12T04:01:08.162 INFO:tasks.workunit.client.1.smithi033.stdout:7/484: getdents de/d2b/d6c/da0 0 2024-05-12T04:01:08.163 INFO:tasks.workunit.client.1.smithi033.stdout:3/321: mknod db/c6e 0 2024-05-12T04:01:08.164 INFO:tasks.workunit.client.1.smithi033.stdout:3/322: write db/f54 [4716501,17820] 0 2024-05-12T04:01:08.168 INFO:tasks.workunit.client.1.smithi033.stdout:8/246: symlink d1/d16/d27/l52 0 2024-05-12T04:01:08.170 INFO:tasks.workunit.client.0.smithi032.stdout:8/171: mknod d0/d9/d13/d21/c34 0 2024-05-12T04:01:08.170 INFO:tasks.workunit.client.1.smithi033.stdout:3/323: dwrite db/d17/d25/f52 [0,4194304] 0 2024-05-12T04:01:08.171 INFO:tasks.workunit.client.0.smithi032.stdout:8/172: chown d0/d9/d13/l19 38096989 1 2024-05-12T04:01:08.171 INFO:tasks.workunit.client.1.smithi033.stdout:3/324: readlink db/l11 0 2024-05-12T04:01:08.171 INFO:tasks.workunit.client.1.smithi033.stdout:3/325: chown f6 63600 1 2024-05-12T04:01:08.172 INFO:tasks.workunit.client.0.smithi032.stdout:8/173: dread - d0/d9/d13/f24 zero size 2024-05-12T04:01:08.172 INFO:tasks.workunit.client.1.smithi033.stdout:6/320: mkdir d4/d1a/d6c 0 2024-05-12T04:01:08.172 INFO:tasks.workunit.client.0.smithi032.stdout:8/174: chown d0/d7 1865074858 1 2024-05-12T04:01:08.173 INFO:tasks.workunit.client.0.smithi032.stdout:9/110: symlink d0/d3/d6/d7/da/d10/l23 0 2024-05-12T04:01:08.174 INFO:tasks.workunit.client.1.smithi033.stdout:0/374: write d6/d11/f2f [1958325,112850] 0 2024-05-12T04:01:08.174 INFO:tasks.workunit.client.1.smithi033.stdout:3/326: read db/d17/d3e/f48 [345057,129345] 0 2024-05-12T04:01:08.175 INFO:tasks.workunit.client.1.smithi033.stdout:9/539: sync 2024-05-12T04:01:08.175 INFO:tasks.workunit.client.1.smithi033.stdout:3/327: chown db/d17/d3e/c57 0 1 2024-05-12T04:01:08.175 INFO:tasks.workunit.client.1.smithi033.stdout:5/261: mkdir d3/d12/d25/d26/d2e/d54 0 2024-05-12T04:01:08.176 INFO:tasks.workunit.client.1.smithi033.stdout:3/328: stat db/d17/d3e 0 2024-05-12T04:01:08.176 INFO:tasks.workunit.client.0.smithi032.stdout:9/111: write d0/d3/d6/d7/da/d10/f12 [1540710,92780] 0 2024-05-12T04:01:08.181 INFO:tasks.workunit.client.0.smithi032.stdout:3/211: mkdir d15/d1b/d43 0 2024-05-12T04:01:08.181 INFO:tasks.workunit.client.0.smithi032.stdout:7/128: symlink dd/d1b/d1e/l2f 0 2024-05-12T04:01:08.183 INFO:tasks.workunit.client.1.smithi033.stdout:2/339: rmdir dc/d3b 39 2024-05-12T04:01:08.187 INFO:tasks.workunit.client.0.smithi032.stdout:5/178: mkdir d4/d34/d38 0 2024-05-12T04:01:08.187 INFO:tasks.workunit.client.0.smithi032.stdout:1/145: link d1/d7/cc d1/dd/d24/c2f 0 2024-05-12T04:01:08.187 INFO:tasks.workunit.client.1.smithi033.stdout:2/340: dread dc/d13/f1b [0,4194304] 0 2024-05-12T04:01:08.187 INFO:tasks.workunit.client.0.smithi032.stdout:0/213: unlink c3 0 2024-05-12T04:01:08.188 INFO:tasks.workunit.client.0.smithi032.stdout:1/146: chown d1/d16/d18/f1b 0 1 2024-05-12T04:01:08.190 INFO:tasks.workunit.client.0.smithi032.stdout:4/118: mknod d0/d3/df/c1b 0 2024-05-12T04:01:08.191 INFO:tasks.workunit.client.0.smithi032.stdout:8/175: mkdir d0/d9/d13/d21/d35 0 2024-05-12T04:01:08.193 INFO:tasks.workunit.client.1.smithi033.stdout:8/247: write d1/d4/d1d/d36/f50 [379369,100387] 0 2024-05-12T04:01:08.196 INFO:tasks.workunit.client.0.smithi032.stdout:5/179: dread d4/f20 [0,4194304] 0 2024-05-12T04:01:08.199 INFO:tasks.workunit.client.0.smithi032.stdout:4/119: dwrite d0/fa [0,4194304] 0 2024-05-12T04:01:08.199 INFO:tasks.workunit.client.1.smithi033.stdout:6/321: creat d4/d1a/d4e/f6d x:0 0 0 2024-05-12T04:01:08.200 INFO:tasks.workunit.client.1.smithi033.stdout:6/322: truncate d4/d5/f5c 102839 0 2024-05-12T04:01:08.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:08.200+0000 7f1365c91700 1 -- 172.21.15.32:0/2308517047 --> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1360101490 con 0x7f134806aed0 2024-05-12T04:01:08.200 INFO:tasks.workunit.client.0.smithi032.stdout:4/120: write d0/f11 [838499,91214] 0 2024-05-12T04:01:08.202 INFO:tasks.workunit.client.1.smithi033.stdout:0/375: creat d6/dc/d58/d6a/f7f x:0 0 0 2024-05-12T04:01:08.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:08.202+0000 7f1347fff700 1 -- 172.21.15.32:0/2308517047 <== mgr.24389 v2:172.21.15.33:6828/1598726962 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7f1360101490 con 0x7f134806aed0 2024-05-12T04:01:08.203 INFO:tasks.workunit.client.1.smithi033.stdout:0/376: write f4 [1503631,118798] 0 2024-05-12T04:01:08.204 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-05-12T04:01:08.204 INFO:teuthology.orchestra.run.smithi032.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:19c9734256aa43a60e203feb242f50bf8b7f58666d520536af43cc4fee96173a", 2024-05-12T04:01:08.204 INFO:teuthology.orchestra.run.smithi032.stdout: "in_progress": true, 2024-05-12T04:01:08.204 INFO:teuthology.orchestra.run.smithi032.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-12T04:01:08.204 INFO:teuthology.orchestra.run.smithi032.stdout: "services_complete": [], 2024-05-12T04:01:08.204 INFO:teuthology.orchestra.run.smithi032.stdout: "progress": "1/21 daemons upgraded", 2024-05-12T04:01:08.204 INFO:teuthology.orchestra.run.smithi032.stdout: "message": "", 2024-05-12T04:01:08.204 INFO:teuthology.orchestra.run.smithi032.stdout: "is_paused": false 2024-05-12T04:01:08.204 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-05-12T04:01:08.205 INFO:tasks.workunit.client.1.smithi033.stdout:0/377: readlink d6/d11/l6c 0 2024-05-12T04:01:08.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:08.206+0000 7f1365c91700 1 -- 172.21.15.32:0/2308517047 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f134806aed0 msgr2=0x7f134806d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:01:08.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:08.206+0000 7f1365c91700 1 --2- 172.21.15.32:0/2308517047 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f134806aed0 0x7f134806d380 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f13500045a0 tx=0x7f1350009e30 comp rx=0 tx=0).stop 2024-05-12T04:01:08.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:08.206+0000 7f1365c91700 1 -- 172.21.15.32:0/2308517047 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13600fcf70 msgr2=0x7f136010bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:01:08.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:08.206+0000 7f1365c91700 1 --2- 172.21.15.32:0/2308517047 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13600fcf70 0x7f136010bbd0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f13540096d0 tx=0x7f1354009e30 comp rx=0 tx=0).stop 2024-05-12T04:01:08.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:08.207+0000 7f1365c91700 1 -- 172.21.15.32:0/2308517047 shutdown_connections 2024-05-12T04:01:08.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:08.207+0000 7f1365c91700 1 --2- 172.21.15.32:0/2308517047 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f134806aed0 0x7f134806d380 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:08.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:08.207+0000 7f1365c91700 1 --2- 172.21.15.32:0/2308517047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13600fbcc0 0x7f136010b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:08.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:08.207+0000 7f1365c91700 1 --2- 172.21.15.32:0/2308517047 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13600fcf70 0x7f136010bbd0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:08.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:08.207+0000 7f1365c91700 1 -- 172.21.15.32:0/2308517047 >> 172.21.15.32:0/2308517047 conn(0x7f13600f7440 msgr2=0x7f13600669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:01:08.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:08.207+0000 7f1365c91700 1 -- 172.21.15.32:0/2308517047 shutdown_connections 2024-05-12T04:01:08.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:08.207+0000 7f1365c91700 1 -- 172.21.15.32:0/2308517047 wait complete. 2024-05-12T04:01:08.209 INFO:tasks.workunit.client.0.smithi032.stdout:0/214: chown df/d10/d3d/d4a/f38 250 1 2024-05-12T04:01:08.211 INFO:tasks.workunit.client.1.smithi033.stdout:1/294: getdents d11/d1c 0 2024-05-12T04:01:08.212 INFO:tasks.workunit.client.0.smithi032.stdout:1/147: rename d1/d16 to d1/d7/d30 0 2024-05-12T04:01:08.217 INFO:tasks.workunit.client.0.smithi032.stdout:8/176: creat d0/d9/d13/d21/d25/f36 x:0 0 0 2024-05-12T04:01:08.218 INFO:tasks.workunit.client.1.smithi033.stdout:7/485: creat de/d20/da9/faf x:0 0 0 2024-05-12T04:01:08.219 INFO:tasks.workunit.client.0.smithi032.stdout:9/112: creat d0/d3/d1b/f24 x:0 0 0 2024-05-12T04:01:08.221 INFO:tasks.workunit.client.0.smithi032.stdout:9/113: truncate d0/d3/d6/d7/f14 584078 0 2024-05-12T04:01:08.223 INFO:tasks.workunit.client.0.smithi032.stdout:5/180: unlink d4/db/f2f 0 2024-05-12T04:01:08.224 INFO:tasks.workunit.client.1.smithi033.stdout:7/486: dwrite de/df/f48 [0,4194304] 0 2024-05-12T04:01:08.224 INFO:tasks.workunit.client.1.smithi033.stdout:8/248: symlink d1/d16/d27/d4d/l53 0 2024-05-12T04:01:08.226 INFO:tasks.workunit.client.1.smithi033.stdout:6/323: mkdir d4/d6e 0 2024-05-12T04:01:08.232 INFO:tasks.workunit.client.1.smithi033.stdout:0/378: mkdir d6/dc/d58/d6a/d80 0 2024-05-12T04:01:08.232 INFO:tasks.workunit.client.0.smithi032.stdout:1/148: rmdir d1/d25 39 2024-05-12T04:01:08.233 INFO:tasks.workunit.client.0.smithi032.stdout:1/149: fsync d1/d7/f11 0 2024-05-12T04:01:08.233 INFO:tasks.workunit.client.1.smithi033.stdout:9/540: symlink d2/d4/d2b/d6d/d3a/dca/lcc 0 2024-05-12T04:01:08.235 INFO:tasks.workunit.client.1.smithi033.stdout:9/541: write d2/d4/d2b/d78/fc8 [139802,108856] 0 2024-05-12T04:01:08.236 INFO:tasks.workunit.client.1.smithi033.stdout:3/329: creat db/d17/d25/d32/d3c/d3d/f6f x:0 0 0 2024-05-12T04:01:08.239 INFO:tasks.workunit.client.1.smithi033.stdout:1/295: write d11/d17/f18 [1357038,117861] 0 2024-05-12T04:01:08.240 INFO:tasks.workunit.client.0.smithi032.stdout:9/114: creat d0/d3/f25 x:0 0 0 2024-05-12T04:01:08.240 INFO:tasks.workunit.client.1.smithi033.stdout:1/296: fdatasync d11/d17/f1d 0 2024-05-12T04:01:08.243 INFO:tasks.workunit.client.0.smithi032.stdout:3/212: creat d15/f44 x:0 0 0 2024-05-12T04:01:08.244 INFO:tasks.workunit.client.1.smithi033.stdout:4/343: link dd/d11/d31/d38/c43 dd/d11/d31/d66/c6b 0 2024-05-12T04:01:08.247 INFO:tasks.workunit.client.0.smithi032.stdout:0/215: rename df/d10/c14 to df/d28/c4c 0 2024-05-12T04:01:08.249 INFO:tasks.workunit.client.0.smithi032.stdout:0/216: fsync df/d22/f39 0 2024-05-12T04:01:08.249 INFO:tasks.workunit.client.0.smithi032.stdout:9/115: dwrite d0/d3/d6/d7/da/d10/d13/f1d [0,4194304] 0 2024-05-12T04:01:08.249 INFO:tasks.workunit.client.1.smithi033.stdout:2/341: mknod dc/d17/c78 0 2024-05-12T04:01:08.250 INFO:tasks.workunit.client.0.smithi032.stdout:9/116: fsync d0/d3/d6/d7/f15 0 2024-05-12T04:01:08.254 INFO:tasks.workunit.client.0.smithi032.stdout:8/177: mkdir d0/d9/d13/d21/d35/d37 0 2024-05-12T04:01:08.255 INFO:tasks.workunit.client.0.smithi032.stdout:8/178: write d0/d9/f2d [367186,44056] 0 2024-05-12T04:01:08.256 INFO:tasks.workunit.client.0.smithi032.stdout:7/129: truncate dd/f13 1323726 0 2024-05-12T04:01:08.258 INFO:tasks.workunit.client.1.smithi033.stdout:8/249: symlink d1/d16/d27/l54 0 2024-05-12T04:01:08.259 INFO:tasks.workunit.client.0.smithi032.stdout:3/213: fsync d15/d24/f3e 0 2024-05-12T04:01:08.260 INFO:tasks.workunit.client.1.smithi033.stdout:6/324: mkdir d4/d1a/d67/d6f 0 2024-05-12T04:01:08.265 INFO:tasks.workunit.client.0.smithi032.stdout:5/181: rename d4/db to d4/d7/d14/d39 0 2024-05-12T04:01:08.266 INFO:tasks.workunit.client.1.smithi033.stdout:5/262: dwrite d3/d12/d25/d26/f31 [0,4194304] 0 2024-05-12T04:01:08.266 INFO:tasks.workunit.client.0.smithi032.stdout:5/182: dread - d4/d7/d14/d39/d2b/f32 zero size 2024-05-12T04:01:08.267 INFO:tasks.workunit.client.1.smithi033.stdout:8/250: dwrite d1/d16/d27/f2d [0,4194304] 0 2024-05-12T04:01:08.268 INFO:tasks.workunit.client.0.smithi032.stdout:5/183: write d4/f6 [898629,35379] 0 2024-05-12T04:01:08.268 INFO:tasks.workunit.client.1.smithi033.stdout:5/263: fdatasync d3/d12/d29/f3d 0 2024-05-12T04:01:08.269 INFO:tasks.workunit.client.1.smithi033.stdout:6/325: dread d4/f56 [0,4194304] 0 2024-05-12T04:01:08.269 INFO:tasks.workunit.client.1.smithi033.stdout:5/264: chown d3/fc 2565 1 2024-05-12T04:01:08.270 INFO:tasks.workunit.client.1.smithi033.stdout:5/265: dread - d3/f40 zero size 2024-05-12T04:01:08.271 INFO:tasks.workunit.client.1.smithi033.stdout:9/542: unlink d2/d4/d29/d3c/f96 0 2024-05-12T04:01:08.278 INFO:tasks.workunit.client.1.smithi033.stdout:3/330: chown db/d17/d25/d32/d3c/l45 15287 1 2024-05-12T04:01:08.279 INFO:tasks.workunit.client.0.smithi032.stdout:8/179: symlink d0/d7/l38 0 2024-05-12T04:01:08.289 INFO:tasks.workunit.client.0.smithi032.stdout:8/180: dwrite d0/f22 [4194304,4194304] 0 2024-05-12T04:01:08.291 INFO:tasks.workunit.client.1.smithi033.stdout:4/344: chown dd/d11/d31/d66/d42 43904 1 2024-05-12T04:01:08.293 INFO:tasks.workunit.client.1.smithi033.stdout:1/297: dread d11/d17/f2c [0,4194304] 0 2024-05-12T04:01:08.296 INFO:tasks.workunit.client.1.smithi033.stdout:7/487: mkdir de/d20/db0 0 2024-05-12T04:01:08.297 INFO:tasks.workunit.client.1.smithi033.stdout:1/298: read d11/d17/d27/d36/f3d [129011,128002] 0 2024-05-12T04:01:08.300 INFO:tasks.workunit.client.0.smithi032.stdout:1/150: link d1/d7/d30/d18/c1d d1/d7/d30/d18/c31 0 2024-05-12T04:01:08.301 INFO:tasks.workunit.client.1.smithi033.stdout:0/379: dwrite d6/dc/f1d [0,4194304] 0 2024-05-12T04:01:08.309 INFO:tasks.workunit.client.0.smithi032.stdout:5/184: symlink d4/d7/l3a 0 2024-05-12T04:01:08.313 INFO:tasks.workunit.client.1.smithi033.stdout:8/251: unlink d1/d16/d27/l54 0 2024-05-12T04:01:08.314 INFO:tasks.workunit.client.0.smithi032.stdout:0/217: dwrite fa [4194304,4194304] 0 2024-05-12T04:01:08.315 INFO:tasks.workunit.client.0.smithi032.stdout:0/218: write df/d30/d36/f41 [871938,78430] 0 2024-05-12T04:01:08.317 INFO:tasks.workunit.client.1.smithi033.stdout:6/326: creat d4/d43/f70 x:0 0 0 2024-05-12T04:01:08.321 INFO:tasks.workunit.client.1.smithi033.stdout:5/266: mknod d3/d27/c55 0 2024-05-12T04:01:08.322 INFO:tasks.workunit.client.0.smithi032.stdout:7/130: read dd/f13 [941876,68254] 0 2024-05-12T04:01:08.324 INFO:tasks.workunit.client.1.smithi033.stdout:8/252: dwrite d1/d16/d27/d4d/d51/f31 [0,4194304] 0 2024-05-12T04:01:08.325 INFO:tasks.workunit.client.1.smithi033.stdout:3/331: unlink db/d17/d25/l6a 0 2024-05-12T04:01:08.325 INFO:tasks.workunit.client.1.smithi033.stdout:5/267: truncate d3/d12/f43 622460 0 2024-05-12T04:01:08.327 INFO:tasks.workunit.client.0.smithi032.stdout:8/181: dwrite d0/d9/d13/d21/d25/f2c [0,4194304] 0 2024-05-12T04:01:08.327 INFO:tasks.workunit.client.0.smithi032.stdout:5/185: fdatasync d4/f22 0 2024-05-12T04:01:08.328 INFO:tasks.workunit.client.1.smithi033.stdout:6/327: dwrite d4/d5/f9 [0,4194304] 0 2024-05-12T04:01:08.328 INFO:tasks.workunit.client.1.smithi033.stdout:3/332: chown db/d17/d25/c37 28510218 1 2024-05-12T04:01:08.328 INFO:tasks.workunit.client.0.smithi032.stdout:8/182: read d0/d9/f2d [324051,63879] 0 2024-05-12T04:01:08.329 INFO:tasks.workunit.client.1.smithi033.stdout:4/345: creat dd/d11/d31/f6c x:0 0 0 2024-05-12T04:01:08.332 INFO:tasks.workunit.client.1.smithi033.stdout:2/342: unlink dc/d3b/c42 0 2024-05-12T04:01:08.336 INFO:tasks.workunit.client.0.smithi032.stdout:0/219: mkdir df/d30/d4d 0 2024-05-12T04:01:08.336 INFO:tasks.workunit.client.0.smithi032.stdout:0/220: chown df/c40 97572 1 2024-05-12T04:01:08.338 INFO:tasks.workunit.client.1.smithi033.stdout:5/268: dwrite d3/f40 [0,4194304] 0 2024-05-12T04:01:08.341 INFO:tasks.workunit.client.0.smithi032.stdout:7/131: fdatasync dd/f13 0 2024-05-12T04:01:08.348 INFO:tasks.workunit.client.1.smithi033.stdout:1/299: rename d11/d17/d27/f37 to d11/d1c/d59/f67 0 2024-05-12T04:01:08.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:08.366+0000 7f30dccfe700 1 -- 172.21.15.32:0/3449192435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30d80fdfd0 msgr2=0x7f30d80fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:01:08.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:08.366+0000 7f30dccfe700 1 --2- 172.21.15.32:0/3449192435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30d80fdfd0 0x7f30d80fe440 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f30cc004940 tx=0x7f30cc009e30 comp rx=0 tx=0).stop 2024-05-12T04:01:08.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:08.366+0000 7f30dccfe700 1 -- 172.21.15.32:0/3449192435 shutdown_connections 2024-05-12T04:01:08.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:08.366+0000 7f30dccfe700 1 --2- 172.21.15.32:0/3449192435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30d80fdfd0 0x7f30d80fe440 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:08.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:08.366+0000 7f30dccfe700 1 --2- 172.21.15.32:0/3449192435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30d80fce60 0x7f30d80fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:08.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:08.366+0000 7f30dccfe700 1 -- 172.21.15.32:0/3449192435 >> 172.21.15.32:0/3449192435 conn(0x7f30d80f8540 msgr2=0x7f30d80fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:01:08.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:08.366+0000 7f30dccfe700 1 -- 172.21.15.32:0/3449192435 shutdown_connections 2024-05-12T04:01:08.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:08.366+0000 7f30dccfe700 1 -- 172.21.15.32:0/3449192435 wait complete. 2024-05-12T04:01:08.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:08.366+0000 7f30dccfe700 1 Processor -- start 2024-05-12T04:01:08.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:08.366+0000 7f30dccfe700 1 -- start start 2024-05-12T04:01:08.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:08.367+0000 7f30dccfe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30d80fce60 0x7f30d810d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:01:08.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:08.367+0000 7f30dccfe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30d80fdfd0 0x7f30d810dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:01:08.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:08.367+0000 7f30dccfe700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d810e390 con 0x7f30d80fdfd0 2024-05-12T04:01:08.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:08.367+0000 7f30dccfe700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d810e500 con 0x7f30d80fce60 2024-05-12T04:01:08.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:08.367+0000 7f30d5d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30d80fdfd0 0x7f30d810dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:01:08.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:08.367+0000 7f30d659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30d80fce60 0x7f30d810d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:01:08.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:08.367+0000 7f30d5d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30d80fdfd0 0x7f30d810dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53074/0 (socket says 172.21.15.32:53074) 2024-05-12T04:01:08.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:08.367+0000 7f30d5d9b700 1 -- 172.21.15.32:0/1919915313 learned_addr learned my addr 172.21.15.32:0/1919915313 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:01:08.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:08.367+0000 7f30d5d9b700 1 -- 172.21.15.32:0/1919915313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30d80fce60 msgr2=0x7f30d810d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:01:08.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:08.367+0000 7f30d5d9b700 1 --2- 172.21.15.32:0/1919915313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30d80fce60 0x7f30d810d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:08.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:08.367+0000 7f30d5d9b700 1 -- 172.21.15.32:0/1919915313 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30cc005040 con 0x7f30d80fdfd0 2024-05-12T04:01:08.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:08.367+0000 7f30d5d9b700 1 --2- 172.21.15.32:0/1919915313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30d80fdfd0 0x7f30d810dd90 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f30cc00b3b0 tx=0x7f30cc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:01:08.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:08.367+0000 7f30c6ffd700 1 -- 172.21.15.32:0/1919915313 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f30cc00e040 con 0x7f30d80fdfd0 2024-05-12T04:01:08.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:08.367+0000 7f30dccfe700 1 -- 172.21.15.32:0/1919915313 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30d8070190 con 0x7f30d80fdfd0 2024-05-12T04:01:08.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:08.367+0000 7f30c6ffd700 1 -- 172.21.15.32:0/1919915313 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f30cc00bc50 con 0x7f30d80fdfd0 2024-05-12T04:01:08.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:08.367+0000 7f30c6ffd700 1 -- 172.21.15.32:0/1919915313 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f30cc01c630 con 0x7f30d80fdfd0 2024-05-12T04:01:08.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:08.367+0000 7f30dccfe700 1 -- 172.21.15.32:0/1919915313 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30d8070750 con 0x7f30d80fdfd0 2024-05-12T04:01:08.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:08.368+0000 7f30dccfe700 1 -- 172.21.15.32:0/1919915313 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30d804c710 con 0x7f30d80fdfd0 2024-05-12T04:01:08.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:08.369+0000 7f30c6ffd700 1 -- 172.21.15.32:0/1919915313 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 25) v1 ==== 88772+0+0 (secure 0 0 0) 0x7f30cc013420 con 0x7f30d80fdfd0 2024-05-12T04:01:08.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:08.369+0000 7f30c6ffd700 1 --2- 172.21.15.32:0/1919915313 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f30c006aed0 0x7f30c006d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:01:08.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:08.369+0000 7f30c6ffd700 1 -- 172.21.15.32:0/1919915313 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f30cc08a490 con 0x7f30d80fdfd0 2024-05-12T04:01:08.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:08.370+0000 7f30d659c700 1 --2- 172.21.15.32:0/1919915313 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f30c006aed0 0x7f30c006d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:01:08.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:08.370+0000 7f30d659c700 1 --2- 172.21.15.32:0/1919915313 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f30c006aed0 0x7f30c006d380 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f30c80041d0 tx=0x7f30c8010040 comp rx=0 tx=0).ready entity=mgr.24389 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:01:08.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:08.374+0000 7f30c6ffd700 1 -- 172.21.15.32:0/1919915313 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181777 (secure 0 0 0) 0x7f30cc058490 con 0x7f30d80fdfd0 2024-05-12T04:01:08.425 INFO:tasks.workunit.client.0.smithi032.stdout:2/130: sync 2024-05-12T04:01:08.462 INFO:tasks.workunit.client.0.smithi032.stdout:5/186: rename d4/d7/d21 to d4/d34/d38/d3b 0 2024-05-12T04:01:08.464 INFO:tasks.workunit.client.0.smithi032.stdout:1/151: link d1/dd/d24/c29 d1/d7/c32 0 2024-05-12T04:01:08.467 INFO:tasks.workunit.client.1.smithi033.stdout:8/253: creat d1/d4/d1d/d36/d3c/f55 x:0 0 0 2024-05-12T04:01:08.469 INFO:tasks.workunit.client.0.smithi032.stdout:5/187: dread d4/f1e [0,4194304] 0 2024-05-12T04:01:08.472 INFO:tasks.workunit.client.0.smithi032.stdout:0/221: dread f5 [0,4194304] 0 2024-05-12T04:01:08.475 INFO:tasks.workunit.client.1.smithi033.stdout:4/346: write dd/f34 [1112890,71940] 0 2024-05-12T04:01:08.476 INFO:tasks.workunit.client.1.smithi033.stdout:6/328: creat d4/d1a/d4e/f71 x:0 0 0 2024-05-12T04:01:08.478 INFO:tasks.workunit.client.0.smithi032.stdout:8/183: rename d0/d9/d13/d21/d25/f36 to d0/d9/d13/d21/d35/d37/f39 0 2024-05-12T04:01:08.480 INFO:tasks.workunit.client.1.smithi033.stdout:5/269: rmdir d3/d27 39 2024-05-12T04:01:08.483 INFO:tasks.workunit.client.0.smithi032.stdout:5/188: mkdir d4/d7/d14/d26/d3c 0 2024-05-12T04:01:08.486 INFO:tasks.workunit.client.0.smithi032.stdout:2/131: link d1/f21 d1/d13/d25/d2e/f32 0 2024-05-12T04:01:08.487 INFO:tasks.workunit.client.0.smithi032.stdout:5/189: dread d4/fd [0,4194304] 0 2024-05-12T04:01:08.490 INFO:tasks.workunit.client.1.smithi033.stdout:9/543: rename d2/dd/fbc to d2/dd/d39/d72/d82/fcd 0 2024-05-12T04:01:08.491 INFO:tasks.workunit.client.1.smithi033.stdout:1/300: truncate d11/f2d 793151 0 2024-05-12T04:01:08.492 INFO:tasks.workunit.client.0.smithi032.stdout:8/184: unlink d0/d7/l30 0 2024-05-12T04:01:08.493 INFO:tasks.workunit.client.1.smithi033.stdout:1/301: write d11/d17/d27/d36/d46/f51 [959217,59339] 0 2024-05-12T04:01:08.493 INFO:tasks.workunit.client.1.smithi033.stdout:1/302: write d11/d1c/d43/f5e [762079,26431] 0 2024-05-12T04:01:08.495 INFO:tasks.workunit.client.0.smithi032.stdout:7/132: rename dd/l10 to dd/d19/l30 0 2024-05-12T04:01:08.500 INFO:tasks.workunit.client.0.smithi032.stdout:5/190: mkdir d4/d34/d3d 0 2024-05-12T04:01:08.503 INFO:tasks.workunit.client.0.smithi032.stdout:8/185: unlink d0/f22 0 2024-05-12T04:01:08.505 INFO:tasks.workunit.client.0.smithi032.stdout:0/222: rename df/d28/d3a to df/d30/d4e 0 2024-05-12T04:01:08.507 INFO:tasks.workunit.client.0.smithi032.stdout:5/191: dwrite d4/fa [0,4194304] 0 2024-05-12T04:01:08.508 INFO:tasks.workunit.client.0.smithi032.stdout:0/223: fdatasync df/d10/d3d/d4a/f2d 0 2024-05-12T04:01:08.508 INFO:tasks.workunit.client.0.smithi032.stdout:7/133: symlink dd/d1b/l31 0 2024-05-12T04:01:08.509 INFO:tasks.workunit.client.0.smithi032.stdout:1/152: link d1/dd/d24/c29 d1/dd/c33 0 2024-05-12T04:01:08.511 INFO:tasks.workunit.client.1.smithi033.stdout:0/380: unlink d6/dc/d66/l75 0 2024-05-12T04:01:08.511 INFO:tasks.workunit.client.1.smithi033.stdout:0/381: fdatasync d6/f26 0 2024-05-12T04:01:08.515 INFO:tasks.workunit.client.0.smithi032.stdout:2/132: link d1/f3 d1/d13/d25/f33 0 2024-05-12T04:01:08.516 INFO:tasks.workunit.client.1.smithi033.stdout:3/333: mknod db/c70 0 2024-05-12T04:01:08.517 INFO:tasks.workunit.client.1.smithi033.stdout:3/334: fdatasync db/d17/d25/d32/f46 0 2024-05-12T04:01:08.519 INFO:tasks.workunit.client.0.smithi032.stdout:7/134: dwrite f5 [0,4194304] 0 2024-05-12T04:01:08.522 INFO:tasks.workunit.client.1.smithi033.stdout:2/343: creat dc/d17/d27/d2f/d49/d59/f79 x:0 0 0 2024-05-12T04:01:08.523 INFO:tasks.workunit.client.0.smithi032.stdout:5/192: dread d4/f6 [0,4194304] 0 2024-05-12T04:01:08.527 INFO:tasks.workunit.client.1.smithi033.stdout:7/488: link de/d2b/d6c/c40 de/d2f/d70/cb1 0 2024-05-12T04:01:08.530 INFO:tasks.workunit.client.0.smithi032.stdout:7/135: unlink dd/d19/c2b 0 2024-05-12T04:01:08.533 INFO:tasks.workunit.client.1.smithi033.stdout:7/489: dread de/d80/f98 [0,4194304] 0 2024-05-12T04:01:08.537 INFO:tasks.workunit.client.0.smithi032.stdout:5/193: chown d4/d7/d14/d39/d2b/l2e 1275955279 1 2024-05-12T04:01:08.539 INFO:tasks.workunit.client.0.smithi032.stdout:0/224: mknod df/d30/d4d/c4f 0 2024-05-12T04:01:08.541 INFO:tasks.workunit.client.0.smithi032.stdout:7/136: mknod dd/d1b/d1e/c32 0 2024-05-12T04:01:08.544 INFO:tasks.workunit.client.0.smithi032.stdout:5/194: dwrite d4/fa [0,4194304] 0 2024-05-12T04:01:08.545 INFO:tasks.workunit.client.0.smithi032.stdout:7/137: dread dd/f1d [0,4194304] 0 2024-05-12T04:01:08.546 INFO:tasks.workunit.client.0.smithi032.stdout:5/195: readlink d4/d7/d14/d39/d2b/l2e 0 2024-05-12T04:01:08.546 INFO:tasks.workunit.client.0.smithi032.stdout:0/225: readlink df/d30/l3f 0 2024-05-12T04:01:08.550 INFO:tasks.workunit.client.0.smithi032.stdout:7/138: read dd/f12 [1617215,27814] 0 2024-05-12T04:01:08.553 INFO:tasks.workunit.client.0.smithi032.stdout:5/196: creat d4/d34/f3e x:0 0 0 2024-05-12T04:01:08.557 INFO:tasks.workunit.client.0.smithi032.stdout:0/226: symlink df/d30/d4e/l50 0 2024-05-12T04:01:08.558 INFO:tasks.workunit.client.1.smithi033.stdout:2/344: rename dc/d17/d3a/c5a to dc/d17/d3a/c7a 0 2024-05-12T04:01:08.558 INFO:tasks.workunit.client.1.smithi033.stdout:6/329: truncate d4/d1a/f26 149802 0 2024-05-12T04:01:08.558 INFO:tasks.workunit.client.0.smithi032.stdout:0/227: chown df/d30/d4d 1 1 2024-05-12T04:01:08.561 INFO:tasks.workunit.client.1.smithi033.stdout:2/345: read - dc/d17/d27/d2f/d49/f51 zero size 2024-05-12T04:01:08.562 INFO:tasks.workunit.client.1.smithi033.stdout:5/270: link d3/d12/f43 d3/d15/d16/d45/d4b/f56 0 2024-05-12T04:01:08.564 INFO:tasks.workunit.client.0.smithi032.stdout:7/139: dread fa [0,4194304] 0 2024-05-12T04:01:08.564 INFO:tasks.workunit.client.1.smithi033.stdout:6/330: dwrite d4/d1a/d4e/f66 [0,4194304] 0 2024-05-12T04:01:08.565 INFO:tasks.workunit.client.0.smithi032.stdout:7/140: write fb [547958,8155] 0 2024-05-12T04:01:08.565 INFO:tasks.workunit.client.1.smithi033.stdout:6/331: fdatasync d4/d1a/f4a 0 2024-05-12T04:01:08.565 INFO:tasks.workunit.client.0.smithi032.stdout:7/141: truncate dd/f25 1002063 0 2024-05-12T04:01:08.566 INFO:tasks.workunit.client.1.smithi033.stdout:6/332: chown d4/d43/f62 1607 1 2024-05-12T04:01:08.567 INFO:tasks.workunit.client.1.smithi033.stdout:6/333: read - d4/d1a/d4e/d4f/f63 zero size 2024-05-12T04:01:08.570 INFO:tasks.workunit.client.1.smithi033.stdout:7/490: mknod de/d80/cb2 0 2024-05-12T04:01:08.573 INFO:tasks.workunit.client.0.smithi032.stdout:7/142: dwrite dd/d1b/d1e/f20 [0,4194304] 0 2024-05-12T04:01:08.575 INFO:tasks.workunit.client.0.smithi032.stdout:0/228: unlink df/d22/c2c 0 2024-05-12T04:01:08.577 INFO:tasks.workunit.client.0.smithi032.stdout:5/197: truncate d4/f6 3804030 0 2024-05-12T04:01:08.578 INFO:tasks.workunit.client.1.smithi033.stdout:8/254: link d1/d16/d20/l38 d1/d4/d1d/d36/d3c/l56 0 2024-05-12T04:01:08.583 INFO:tasks.workunit.client.1.smithi033.stdout:9/544: dwrite d2/d4/d2b/d6d/d3a/f5a [4194304,4194304] 0 2024-05-12T04:01:08.584 INFO:tasks.workunit.client.1.smithi033.stdout:3/335: rename l7 to db/d17/d25/l71 0 2024-05-12T04:01:08.585 INFO:tasks.workunit.client.1.smithi033.stdout:9/545: chown d2/d4/d2b/d78/f8c 40 1 2024-05-12T04:01:08.586 INFO:tasks.workunit.client.0.smithi032.stdout:0/229: creat df/f51 x:0 0 0 2024-05-12T04:01:08.587 INFO:tasks.workunit.client.1.smithi033.stdout:4/347: creat dd/d11/f6d x:0 0 0 2024-05-12T04:01:08.589 INFO:tasks.workunit.client.0.smithi032.stdout:5/198: rename d4/d7/d14/d39/l18 to d4/d7/d14/d26/d3c/l3f 0 2024-05-12T04:01:08.590 INFO:tasks.workunit.client.1.smithi033.stdout:2/346: symlink dc/d13/d65/l7b 0 2024-05-12T04:01:08.603 INFO:tasks.workunit.client.1.smithi033.stdout:6/334: creat d4/d1a/d67/d68/f72 x:0 0 0 2024-05-12T04:01:08.606 INFO:tasks.workunit.client.1.smithi033.stdout:7/491: unlink de/d20/d38/l4d 0 2024-05-12T04:01:08.608 INFO:tasks.workunit.client.0.smithi032.stdout:7/143: getdents dd/d1b 0 2024-05-12T04:01:08.609 INFO:tasks.workunit.client.1.smithi033.stdout:8/255: write d1/d16/d20/f43 [141934,12421] 0 2024-05-12T04:01:08.612 INFO:tasks.workunit.client.1.smithi033.stdout:0/382: creat d6/dc/d58/f81 x:0 0 0 2024-05-12T04:01:08.616 INFO:tasks.workunit.client.0.smithi032.stdout:7/144: rename fc to dd/d19/f33 0 2024-05-12T04:01:08.617 INFO:tasks.workunit.client.1.smithi033.stdout:0/383: dwrite d6/f26 [4194304,4194304] 0 2024-05-12T04:01:08.617 INFO:tasks.workunit.client.1.smithi033.stdout:3/336: fsync db/d17/d3e/f48 0 2024-05-12T04:01:08.620 INFO:tasks.workunit.client.1.smithi033.stdout:9/546: symlink d2/d4/d2b/d6d/d3a/lce 0 2024-05-12T04:01:08.621 INFO:tasks.workunit.client.1.smithi033.stdout:9/547: chown d2/d4/d29/d3c/d52/fba 34 1 2024-05-12T04:01:08.622 INFO:tasks.workunit.client.0.smithi032.stdout:7/145: creat dd/d19/f34 x:0 0 0 2024-05-12T04:01:08.624 INFO:tasks.workunit.client.0.smithi032.stdout:7/146: truncate dd/d19/f2a 878992 0 2024-05-12T04:01:08.625 INFO:tasks.workunit.client.0.smithi032.stdout:7/147: fdatasync dd/f13 0 2024-05-12T04:01:08.626 INFO:tasks.workunit.client.1.smithi033.stdout:2/347: mknod dc/d17/d27/d2f/d49/c7c 0 2024-05-12T04:01:08.627 INFO:tasks.workunit.client.1.smithi033.stdout:5/271: write d3/f52 [1125834,56966] 0 2024-05-12T04:01:08.629 INFO:tasks.workunit.client.0.smithi032.stdout:5/199: write d4/f1e [145220,1413] 0 2024-05-12T04:01:08.630 INFO:tasks.workunit.client.0.smithi032.stdout:7/148: mknod dd/d1b/c35 0 2024-05-12T04:01:08.631 INFO:tasks.workunit.client.0.smithi032.stdout:7/149: stat dd/d1b/c23 0 2024-05-12T04:01:08.632 INFO:tasks.workunit.client.1.smithi033.stdout:3/337: dread db/d17/d25/f39 [0,4194304] 0 2024-05-12T04:01:08.632 INFO:tasks.workunit.client.0.smithi032.stdout:7/150: truncate dd/f25 1704638 0 2024-05-12T04:01:08.632 INFO:tasks.workunit.client.1.smithi033.stdout:6/335: creat d4/d5/f73 x:0 0 0 2024-05-12T04:01:08.633 INFO:tasks.workunit.client.0.smithi032.stdout:0/230: creat df/d10/d48/f52 x:0 0 0 2024-05-12T04:01:08.634 INFO:tasks.workunit.client.0.smithi032.stdout:0/231: fsync df/d30/d36/f43 0 2024-05-12T04:01:08.635 INFO:tasks.workunit.client.1.smithi033.stdout:7/492: rmdir de/df/d65 39 2024-05-12T04:01:08.637 INFO:tasks.workunit.client.0.smithi032.stdout:5/200: mkdir d4/d7/d14/d26/d40 0 2024-05-12T04:01:08.638 INFO:tasks.workunit.client.1.smithi033.stdout:1/303: getdents d11/d1c/d59 0 2024-05-12T04:01:08.639 INFO:tasks.workunit.client.1.smithi033.stdout:1/304: write d11/d1c/f54 [621306,114636] 0 2024-05-12T04:01:08.643 INFO:tasks.workunit.client.0.smithi032.stdout:0/232: chown df/d22/f2f 35994 1 2024-05-12T04:01:08.645 INFO:tasks.workunit.client.1.smithi033.stdout:0/384: mkdir d6/db/d2c/d82 0 2024-05-12T04:01:08.645 INFO:tasks.workunit.client.1.smithi033.stdout:0/385: chown f4 154193 1 2024-05-12T04:01:08.646 INFO:tasks.workunit.client.1.smithi033.stdout:7/493: dread de/df/d17/f88 [0,4194304] 0 2024-05-12T04:01:08.647 INFO:tasks.workunit.client.1.smithi033.stdout:7/494: fsync de/df/f48 0 2024-05-12T04:01:08.648 INFO:tasks.workunit.client.1.smithi033.stdout:9/548: unlink d2/dd/c1a 0 2024-05-12T04:01:08.652 INFO:tasks.workunit.client.1.smithi033.stdout:0/386: dwrite d6/db/d2c/f31 [0,4194304] 0 2024-05-12T04:01:08.652 INFO:tasks.workunit.client.0.smithi032.stdout:7/151: link fb dd/f36 0 2024-05-12T04:01:08.652 INFO:tasks.workunit.client.0.smithi032.stdout:7/152: write f2 [582403,127836] 0 2024-05-12T04:01:08.653 INFO:tasks.workunit.client.1.smithi033.stdout:0/387: write d6/db/d2c/f31 [2683886,95903] 0 2024-05-12T04:01:08.656 INFO:tasks.workunit.client.1.smithi033.stdout:4/348: mknod dd/d11/d31/d66/c6e 0 2024-05-12T04:01:08.659 INFO:tasks.workunit.client.1.smithi033.stdout:2/348: mknod dc/d17/d32/c7d 0 2024-05-12T04:01:08.660 INFO:tasks.workunit.client.0.smithi032.stdout:5/201: mknod d4/d7/c41 0 2024-05-12T04:01:08.661 INFO:tasks.workunit.client.0.smithi032.stdout:0/233: readlink df/l24 0 2024-05-12T04:01:08.663 INFO:tasks.workunit.client.1.smithi033.stdout:8/256: write d1/d4/d1d/f26 [2341819,22301] 0 2024-05-12T04:01:08.665 INFO:tasks.workunit.client.1.smithi033.stdout:2/349: dwrite dc/d13/d1c/f4f [0,4194304] 0 2024-05-12T04:01:08.665 INFO:tasks.workunit.client.0.smithi032.stdout:7/153: mkdir dd/d19/d37 0 2024-05-12T04:01:08.669 INFO:tasks.workunit.client.1.smithi033.stdout:6/336: unlink d4/d5/f9 0 2024-05-12T04:01:08.670 INFO:tasks.workunit.client.1.smithi033.stdout:3/338: creat db/d17/d25/d32/d3c/d3d/f72 x:0 0 0 2024-05-12T04:01:08.673 INFO:tasks.workunit.client.1.smithi033.stdout:2/350: dwrite dc/d13/d1c/f4f [4194304,4194304] 0 2024-05-12T04:01:08.676 INFO:tasks.workunit.client.0.smithi032.stdout:7/154: creat dd/d1b/d1e/f38 x:0 0 0 2024-05-12T04:01:08.679 INFO:tasks.workunit.client.0.smithi032.stdout:6/153: sync 2024-05-12T04:01:08.679 INFO:tasks.workunit.client.0.smithi032.stdout:6/154: chown d7/de/d12/d1a 344 1 2024-05-12T04:01:08.680 INFO:tasks.workunit.client.1.smithi033.stdout:6/337: dread d4/f2f [0,4194304] 0 2024-05-12T04:01:08.681 INFO:tasks.workunit.client.1.smithi033.stdout:6/338: dread - d4/d1a/d4e/f6d zero size 2024-05-12T04:01:08.682 INFO:tasks.workunit.client.1.smithi033.stdout:6/339: write d4/d1a/f47 [4036037,59947] 0 2024-05-12T04:01:08.727 INFO:tasks.workunit.client.0.smithi032.stdout:5/202: symlink d4/d7/d14/d26/d33/l42 0 2024-05-12T04:01:08.738 INFO:tasks.workunit.client.0.smithi032.stdout:6/155: unlink d7/d9/f17 0 2024-05-12T04:01:08.744 INFO:tasks.workunit.client.1.smithi033.stdout:9/549: truncate d2/d4/d2b/d6d/d2f/f38 2021899 0 2024-05-12T04:01:08.747 INFO:tasks.workunit.client.1.smithi033.stdout:9/550: dread d2/d4/d2b/d6d/d3a/f5a [4194304,4194304] 0 2024-05-12T04:01:08.753 INFO:tasks.workunit.client.1.smithi033.stdout:4/349: chown dd/f2a 987 1 2024-05-12T04:01:08.757 INFO:tasks.workunit.client.1.smithi033.stdout:5/272: mkdir d3/d12/d25/d26/d2e/d54/d57 0 2024-05-12T04:01:08.758 INFO:tasks.workunit.client.1.smithi033.stdout:9/551: dread d2/d4/d2b/d6d/d3a/f5a [0,4194304] 0 2024-05-12T04:01:08.761 INFO:tasks.workunit.client.1.smithi033.stdout:8/257: creat d1/d16/d27/d28/f57 x:0 0 0 2024-05-12T04:01:08.762 INFO:tasks.workunit.client.1.smithi033.stdout:9/552: dread d2/d4/d2b/d6d/d4c/f95 [0,4194304] 0 2024-05-12T04:01:08.766 INFO:tasks.workunit.client.0.smithi032.stdout:5/203: getdents d4/d7/d14/d26 0 2024-05-12T04:01:08.768 INFO:tasks.workunit.client.1.smithi033.stdout:8/258: dwrite d1/d4/d1d/f1e [0,4194304] 0 2024-05-12T04:01:08.769 INFO:tasks.workunit.client.0.smithi032.stdout:6/156: getdents d7/de/d12 0 2024-05-12T04:01:08.769 INFO:tasks.workunit.client.1.smithi033.stdout:9/553: dwrite d2/d4/d29/d32/db7/fbf [0,4194304] 0 2024-05-12T04:01:08.769 INFO:tasks.workunit.client.0.smithi032.stdout:6/157: chown d7/de/d12/f28 7579 1 2024-05-12T04:01:08.771 INFO:tasks.workunit.client.0.smithi032.stdout:6/158: write d7/f32 [274792,115321] 0 2024-05-12T04:01:08.776 INFO:tasks.workunit.client.1.smithi033.stdout:8/259: dwrite d1/d4/fd [4194304,4194304] 0 2024-05-12T04:01:08.777 INFO:tasks.workunit.client.1.smithi033.stdout:2/351: mknod dc/d17/d27/d2f/d49/d59/c7e 0 2024-05-12T04:01:08.778 INFO:tasks.workunit.client.1.smithi033.stdout:2/352: chown dc/d17/d27/f60 8415 1 2024-05-12T04:01:08.781 INFO:tasks.workunit.client.1.smithi033.stdout:2/353: chown dc/d17/d27/d2f/d49/c7c 444 1 2024-05-12T04:01:08.782 INFO:tasks.workunit.client.1.smithi033.stdout:2/354: fsync dc/d17/d27/d2f/f75 0 2024-05-12T04:01:08.782 INFO:tasks.workunit.client.1.smithi033.stdout:2/355: chown dc/d17/d27/d2f/d49 1 1 2024-05-12T04:01:08.783 INFO:tasks.workunit.client.0.smithi032.stdout:6/159: mknod d7/de/d12/c37 0 2024-05-12T04:01:08.786 INFO:tasks.workunit.client.1.smithi033.stdout:6/340: symlink d4/d1a/l74 0 2024-05-12T04:01:08.787 INFO:tasks.workunit.client.1.smithi033.stdout:6/341: dread - d4/d43/f60 zero size 2024-05-12T04:01:08.788 INFO:tasks.workunit.client.1.smithi033.stdout:2/356: dwrite dc/d13/d1c/f21 [0,4194304] 0 2024-05-12T04:01:08.792 INFO:tasks.workunit.client.0.smithi032.stdout:6/160: dwrite d7/de/f1b [0,4194304] 0 2024-05-12T04:01:08.801 INFO:tasks.workunit.client.0.smithi032.stdout:6/161: unlink d7/d33/f35 0 2024-05-12T04:01:08.803 INFO:tasks.workunit.client.0.smithi032.stdout:5/204: link d4/d7/l24 d4/d7/l43 0 2024-05-12T04:01:08.804 INFO:tasks.workunit.client.1.smithi033.stdout:5/273: rmdir d3/d15/d16/d45/d4b 39 2024-05-12T04:01:08.804 INFO:tasks.workunit.client.0.smithi032.stdout:5/205: chown d4/d7 6704252 1 2024-05-12T04:01:08.806 INFO:tasks.workunit.client.0.smithi032.stdout:6/162: creat d7/f38 x:0 0 0 2024-05-12T04:01:08.812 INFO:tasks.workunit.client.0.smithi032.stdout:6/163: unlink d7/l8 0 2024-05-12T04:01:08.813 INFO:tasks.workunit.client.0.smithi032.stdout:5/206: dread d4/f6 [0,4194304] 0 2024-05-12T04:01:08.814 INFO:tasks.workunit.client.0.smithi032.stdout:6/164: write d7/f38 [312668,19310] 0 2024-05-12T04:01:08.816 INFO:tasks.workunit.client.0.smithi032.stdout:5/207: write d4/f1e [2486303,5236] 0 2024-05-12T04:01:08.816 INFO:tasks.workunit.client.0.smithi032.stdout:5/208: chown d4/d7/d14 209 1 2024-05-12T04:01:08.817 INFO:tasks.workunit.client.1.smithi033.stdout:8/260: unlink d1/d4/d1d/f1e 0 2024-05-12T04:01:08.818 INFO:tasks.workunit.client.1.smithi033.stdout:8/261: chown d1/d4/d1d/d36/c4e 924 1 2024-05-12T04:01:08.820 INFO:tasks.workunit.client.1.smithi033.stdout:8/262: truncate d1/d16/f19 3787189 0 2024-05-12T04:01:08.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:08.820+0000 7f30dccfe700 1 -- 172.21.15.32:0/1919915313 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f30d8105780 con 0x7f30d80fdfd0 2024-05-12T04:01:08.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:08.820+0000 7f30c6ffd700 1 -- 172.21.15.32:0/1919915313 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f30cc058020 con 0x7f30d80fdfd0 2024-05-12T04:01:08.820 INFO:teuthology.orchestra.run.smithi032.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-12T04:01:08.820 INFO:teuthology.orchestra.run.smithi032.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-12T04:01:08.821 INFO:teuthology.orchestra.run.smithi032.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-12T04:01:08.821 INFO:tasks.workunit.client.1.smithi033.stdout:8/263: read - d1/d4/f39 zero size 2024-05-12T04:01:08.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:08.823+0000 7f30dccfe700 1 -- 172.21.15.32:0/1919915313 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f30c006aed0 msgr2=0x7f30c006d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:01:08.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:08.823+0000 7f30dccfe700 1 --2- 172.21.15.32:0/1919915313 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f30c006aed0 0x7f30c006d380 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f30c80041d0 tx=0x7f30c8010040 comp rx=0 tx=0).stop 2024-05-12T04:01:08.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:08.823+0000 7f30dccfe700 1 -- 172.21.15.32:0/1919915313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30d80fdfd0 msgr2=0x7f30d810dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:01:08.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:08.823+0000 7f30dccfe700 1 --2- 172.21.15.32:0/1919915313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30d80fdfd0 0x7f30d810dd90 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f30cc00b3b0 tx=0x7f30cc009e30 comp rx=0 tx=0).stop 2024-05-12T04:01:08.823 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:08.823+0000 7f30dccfe700 1 -- 172.21.15.32:0/1919915313 shutdown_connections 2024-05-12T04:01:08.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:08.823+0000 7f30dccfe700 1 --2- 172.21.15.32:0/1919915313 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f30c006aed0 0x7f30c006d380 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:08.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:08.823+0000 7f30dccfe700 1 --2- 172.21.15.32:0/1919915313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30d80fce60 0x7f30d810d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:08.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:08.823+0000 7f30dccfe700 1 --2- 172.21.15.32:0/1919915313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30d80fdfd0 0x7f30d810dd90 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:08.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:08.823+0000 7f30dccfe700 1 -- 172.21.15.32:0/1919915313 >> 172.21.15.32:0/1919915313 conn(0x7f30d80f8540 msgr2=0x7f30d80f9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:01:08.824 INFO:tasks.workunit.client.0.smithi032.stdout:5/209: creat d4/d7/d14/d39/d1b/f44 x:0 0 0 2024-05-12T04:01:08.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:08.823+0000 7f30dccfe700 1 -- 172.21.15.32:0/1919915313 shutdown_connections 2024-05-12T04:01:08.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:08.823+0000 7f30dccfe700 1 -- 172.21.15.32:0/1919915313 wait complete. 2024-05-12T04:01:08.825 INFO:tasks.workunit.client.1.smithi033.stdout:6/342: rmdir d4/d1a/d3c 39 2024-05-12T04:01:08.826 INFO:tasks.workunit.client.1.smithi033.stdout:6/343: dread - d4/d1a/d67/d68/f72 zero size 2024-05-12T04:01:08.828 INFO:tasks.workunit.client.0.smithi032.stdout:0/234: truncate df/d22/f2f 100511 0 2024-05-12T04:01:08.828 INFO:tasks.workunit.client.0.smithi032.stdout:5/210: creat d4/d34/d38/f45 x:0 0 0 2024-05-12T04:01:08.829 INFO:tasks.workunit.client.0.smithi032.stdout:6/165: dread d7/de/d12/f16 [0,4194304] 0 2024-05-12T04:01:08.830 INFO:tasks.workunit.client.0.smithi032.stdout:5/211: stat d4/d7/d14 0 2024-05-12T04:01:08.834 INFO:tasks.workunit.client.1.smithi033.stdout:4/350: fdatasync dd/d11/d31/d66/f2d 0 2024-05-12T04:01:08.838 INFO:tasks.workunit.client.0.smithi032.stdout:6/166: mknod d7/de/d2e/c39 0 2024-05-12T04:01:08.842 INFO:tasks.workunit.client.0.smithi032.stdout:5/212: truncate d4/f22 1764850 0 2024-05-12T04:01:08.842 INFO:tasks.workunit.client.0.smithi032.stdout:0/235: dwrite df/d10/d48/f52 [0,4194304] 0 2024-05-12T04:01:08.842 INFO:tasks.workunit.client.1.smithi033.stdout:3/339: sync 2024-05-12T04:01:08.843 INFO:tasks.workunit.client.1.smithi033.stdout:3/340: dread - db/d17/d3a/f63 zero size 2024-05-12T04:01:08.845 INFO:tasks.workunit.client.0.smithi032.stdout:0/236: write df/d30/d36/f43 [69357,193] 0 2024-05-12T04:01:08.846 INFO:tasks.workunit.client.0.smithi032.stdout:7/155: dwrite dd/f36 [0,4194304] 0 2024-05-12T04:01:08.852 INFO:tasks.workunit.client.1.smithi033.stdout:3/341: dwrite db/d17/d25/d32/d3c/d3d/f72 [0,4194304] 0 2024-05-12T04:01:08.853 INFO:tasks.workunit.client.0.smithi032.stdout:6/167: creat d7/de/d2e/f3a x:0 0 0 2024-05-12T04:01:08.853 INFO:tasks.workunit.client.0.smithi032.stdout:6/168: readlink d7/de/d12/d1a/l2d 0 2024-05-12T04:01:08.854 INFO:tasks.workunit.client.0.smithi032.stdout:6/169: chown d7/de/d12/f16 187038316 1 2024-05-12T04:01:08.854 INFO:tasks.workunit.client.1.smithi033.stdout:8/264: rmdir d1/d4/d1d/d36 39 2024-05-12T04:01:08.855 INFO:tasks.workunit.client.1.smithi033.stdout:0/388: unlink d6/db/fd 0 2024-05-12T04:01:08.861 INFO:tasks.workunit.client.0.smithi032.stdout:7/156: unlink dd/c27 0 2024-05-12T04:01:08.862 INFO:tasks.workunit.client.1.smithi033.stdout:7/495: link de/l9d de/d2b/lb3 0 2024-05-12T04:01:08.864 INFO:tasks.workunit.client.0.smithi032.stdout:0/237: mknod df/d30/d36/c53 0 2024-05-12T04:01:08.868 INFO:tasks.workunit.client.0.smithi032.stdout:5/213: symlink d4/d34/d3d/l46 0 2024-05-12T04:01:08.875 INFO:tasks.workunit.client.1.smithi033.stdout:9/554: symlink d2/d4/d29/lcf 0 2024-05-12T04:01:08.876 INFO:tasks.workunit.client.1.smithi033.stdout:9/555: read d2/d70/f87 [432260,26310] 0 2024-05-12T04:01:08.876 INFO:tasks.workunit.client.0.smithi032.stdout:5/214: mkdir d4/d7/d14/d26/d47 0 2024-05-12T04:01:08.878 INFO:tasks.workunit.client.0.smithi032.stdout:7/157: creat dd/d19/d37/f39 x:0 0 0 2024-05-12T04:01:08.880 INFO:tasks.workunit.client.0.smithi032.stdout:0/238: creat df/d28/f54 x:0 0 0 2024-05-12T04:01:08.884 INFO:tasks.workunit.client.1.smithi033.stdout:0/389: fsync d6/d11/d57/f71 0 2024-05-12T04:01:08.885 INFO:tasks.workunit.client.1.smithi033.stdout:0/390: truncate d6/dc/d58/f78 885892 0 2024-05-12T04:01:08.886 INFO:tasks.workunit.client.1.smithi033.stdout:0/391: readlink d6/db/l30 0 2024-05-12T04:01:08.887 INFO:tasks.workunit.client.1.smithi033.stdout:6/344: creat d4/d1a/d3c/f75 x:0 0 0 2024-05-12T04:01:08.888 INFO:tasks.workunit.client.0.smithi032.stdout:0/239: mkdir df/d30/d4e/d55 0 2024-05-12T04:01:08.890 INFO:tasks.workunit.client.1.smithi033.stdout:1/305: dwrite d11/d1c/f34 [0,4194304] 0 2024-05-12T04:01:08.891 INFO:tasks.workunit.client.1.smithi033.stdout:9/556: dread d2/d4/d2b/d6d/d7a/f90 [0,4194304] 0 2024-05-12T04:01:08.892 INFO:tasks.workunit.client.1.smithi033.stdout:4/351: symlink dd/l6f 0 2024-05-12T04:01:08.894 INFO:tasks.workunit.client.1.smithi033.stdout:4/352: read dd/d11/f19 [356341,21961] 0 2024-05-12T04:01:08.894 INFO:tasks.workunit.client.0.smithi032.stdout:0/240: truncate f5 610564 0 2024-05-12T04:01:08.897 INFO:tasks.workunit.client.1.smithi033.stdout:9/557: dwrite d2/dd/f11 [4194304,4194304] 0 2024-05-12T04:01:08.901 INFO:tasks.workunit.client.1.smithi033.stdout:3/342: symlink db/l73 0 2024-05-12T04:01:08.904 INFO:tasks.workunit.client.1.smithi033.stdout:9/558: dread d2/d4/f1f [0,4194304] 0 2024-05-12T04:01:08.904 INFO:tasks.workunit.client.0.smithi032.stdout:0/241: rmdir df/d30/d32 39 2024-05-12T04:01:08.914 INFO:tasks.workunit.client.1.smithi033.stdout:1/306: mkdir d11/d1c/d59/d68 0 2024-05-12T04:01:08.914 INFO:tasks.workunit.client.1.smithi033.stdout:1/307: fdatasync d11/f30 0 2024-05-12T04:01:08.916 INFO:tasks.workunit.client.0.smithi032.stdout:5/215: getdents d4/d7/d14/d39/d1b 0 2024-05-12T04:01:08.921 INFO:tasks.workunit.client.0.smithi032.stdout:0/242: creat df/d10/d3d/f56 x:0 0 0 2024-05-12T04:01:08.924 INFO:tasks.workunit.client.0.smithi032.stdout:5/216: creat d4/d34/d38/d3b/f48 x:0 0 0 2024-05-12T04:01:08.926 INFO:tasks.workunit.client.0.smithi032.stdout:0/243: rename df/d10/c11 to df/d22/c57 0 2024-05-12T04:01:08.930 INFO:tasks.workunit.client.1.smithi033.stdout:3/343: rmdir db 39 2024-05-12T04:01:08.931 INFO:tasks.workunit.client.0.smithi032.stdout:0/244: mkdir df/d22/d58 0 2024-05-12T04:01:08.933 INFO:tasks.workunit.client.1.smithi033.stdout:8/265: creat d1/d4/d1d/f58 x:0 0 0 2024-05-12T04:01:08.934 INFO:tasks.workunit.client.1.smithi033.stdout:8/266: fdatasync d1/d16/d20/f43 0 2024-05-12T04:01:08.934 INFO:tasks.workunit.client.1.smithi033.stdout:8/267: chown d1/d16/d29/l3b 96 1 2024-05-12T04:01:08.935 INFO:tasks.workunit.client.1.smithi033.stdout:2/357: write dc/f31 [43430,117012] 0 2024-05-12T04:01:08.938 INFO:tasks.workunit.client.0.smithi032.stdout:5/217: creat d4/f49 x:0 0 0 2024-05-12T04:01:08.938 INFO:tasks.workunit.client.0.smithi032.stdout:0/245: dwrite df/d30/d36/f43 [0,4194304] 0 2024-05-12T04:01:08.939 INFO:tasks.workunit.client.0.smithi032.stdout:0/246: chown df/f2b 1767386717 1 2024-05-12T04:01:08.940 INFO:tasks.workunit.client.0.smithi032.stdout:5/218: readlink d4/d7/d14/d26/d33/l42 0 2024-05-12T04:01:08.947 INFO:tasks.workunit.client.1.smithi033.stdout:1/308: unlink d11/d17/f2c 0 2024-05-12T04:01:08.951 INFO:tasks.workunit.client.1.smithi033.stdout:5/274: truncate d3/f7 748314 0 2024-05-12T04:01:08.952 INFO:tasks.workunit.client.0.smithi032.stdout:6/170: dwrite d7/fa [0,4194304] 0 2024-05-12T04:01:08.955 INFO:tasks.workunit.client.0.smithi032.stdout:6/171: read d7/de/f14 [356958,99256] 0 2024-05-12T04:01:08.957 INFO:tasks.workunit.client.1.smithi033.stdout:3/344: read - db/d17/f64 zero size 2024-05-12T04:01:08.958 INFO:tasks.workunit.client.0.smithi032.stdout:7/158: getdents dd/d19/d37 0 2024-05-12T04:01:08.959 INFO:tasks.workunit.client.0.smithi032.stdout:7/159: chown dd/d1b/d1e/l2f 883204 1 2024-05-12T04:01:08.960 INFO:tasks.workunit.client.1.smithi033.stdout:0/392: sync 2024-05-12T04:01:08.960 INFO:tasks.workunit.client.1.smithi033.stdout:4/353: sync 2024-05-12T04:01:08.961 INFO:tasks.workunit.client.0.smithi032.stdout:4/121: sync 2024-05-12T04:01:08.961 INFO:tasks.workunit.client.0.smithi032.stdout:2/133: sync 2024-05-12T04:01:08.961 INFO:tasks.workunit.client.0.smithi032.stdout:8/186: sync 2024-05-12T04:01:08.961 INFO:tasks.workunit.client.0.smithi032.stdout:9/117: sync 2024-05-12T04:01:08.961 INFO:tasks.workunit.client.0.smithi032.stdout:3/214: sync 2024-05-12T04:01:08.961 INFO:tasks.workunit.client.0.smithi032.stdout:1/153: sync 2024-05-12T04:01:08.962 INFO:tasks.workunit.client.1.smithi033.stdout:0/393: write d6/f6b [13892,128833] 0 2024-05-12T04:01:08.965 INFO:tasks.workunit.client.0.smithi032.stdout:3/215: truncate f14 9145633 0 2024-05-12T04:01:08.965 INFO:tasks.workunit.client.0.smithi032.stdout:6/172: creat d7/de/f3b x:0 0 0 2024-05-12T04:01:08.966 INFO:tasks.workunit.client.0.smithi032.stdout:1/154: write d1/f3 [6123330,53857] 0 2024-05-12T04:01:08.972 INFO:tasks.workunit.client.1.smithi033.stdout:7/496: link de/df/d65/l6b de/d2b/d6c/da0/lb4 0 2024-05-12T04:01:08.972 INFO:tasks.workunit.client.0.smithi032.stdout:7/160: write fa [4975497,8160] 0 2024-05-12T04:01:08.972 INFO:tasks.workunit.client.0.smithi032.stdout:1/155: fsync d1/f3 0 2024-05-12T04:01:08.972 INFO:tasks.workunit.client.1.smithi033.stdout:7/497: stat de/df/d17/d1e/d79/f91 0 2024-05-12T04:01:08.973 INFO:tasks.workunit.client.1.smithi033.stdout:7/498: write de/df/d17/d1b/d36/f8a [1203101,31569] 0 2024-05-12T04:01:08.976 INFO:tasks.workunit.client.0.smithi032.stdout:7/161: dread dd/d1b/f2d [0,4194304] 0 2024-05-12T04:01:08.976 INFO:tasks.workunit.client.0.smithi032.stdout:2/134: dwrite d1/d13/d25/f2a [0,4194304] 0 2024-05-12T04:01:08.977 INFO:tasks.workunit.client.1.smithi033.stdout:5/275: unlink d3/d12/d29/d2b/c32 0 2024-05-12T04:01:08.977 INFO:tasks.workunit.client.1.smithi033.stdout:7/499: dread de/df/d17/f88 [0,4194304] 0 2024-05-12T04:01:08.980 INFO:tasks.workunit.client.0.smithi032.stdout:4/122: unlink d0/d3/f4 0 2024-05-12T04:01:08.981 INFO:tasks.workunit.client.0.smithi032.stdout:4/123: read - d0/f13 zero size 2024-05-12T04:01:08.982 INFO:tasks.workunit.client.1.smithi033.stdout:7/500: dread de/df/f48 [0,4194304] 0 2024-05-12T04:01:08.983 INFO:tasks.workunit.client.1.smithi033.stdout:5/276: dwrite d3/d12/f14 [0,4194304] 0 2024-05-12T04:01:08.984 INFO:tasks.workunit.client.1.smithi033.stdout:7/501: dread - de/df/d17/d1e/d79/f91 zero size 2024-05-12T04:01:08.984 INFO:tasks.workunit.client.0.smithi032.stdout:7/162: dwrite dd/f36 [0,4194304] 0 2024-05-12T04:01:08.985 INFO:tasks.workunit.client.1.smithi033.stdout:3/345: creat db/d59/f74 x:0 0 0 2024-05-12T04:01:08.985 INFO:tasks.workunit.client.1.smithi033.stdout:7/502: readlink de/d2f/d42/l58 0 2024-05-12T04:01:08.991 INFO:tasks.workunit.client.1.smithi033.stdout:7/503: dread de/f12 [0,4194304] 0 2024-05-12T04:01:08.991 INFO:tasks.workunit.client.0.smithi032.stdout:0/247: getdents df/d10/d3d 0 2024-05-12T04:01:08.993 INFO:tasks.workunit.client.1.smithi033.stdout:5/277: dread d3/d15/d16/f1a [0,4194304] 0 2024-05-12T04:01:08.996 INFO:tasks.workunit.client.0.smithi032.stdout:5/219: rename d4/d34 to d4/d7/d14/d26/d33/d4a 0 2024-05-12T04:01:08.999 INFO:tasks.workunit.client.1.smithi033.stdout:8/268: truncate d1/f12 1397422 0 2024-05-12T04:01:09.003 INFO:tasks.workunit.client.1.smithi033.stdout:2/358: mknod dc/d17/d32/d67/c7f 0 2024-05-12T04:01:09.007 INFO:tasks.workunit.client.1.smithi033.stdout:6/345: truncate d4/d1a/d4e/f66 1358363 0 2024-05-12T04:01:09.009 INFO:tasks.workunit.client.1.smithi033.stdout:6/346: dread - d4/d1a/d4e/d4f/f63 zero size 2024-05-12T04:01:09.010 INFO:tasks.workunit.client.1.smithi033.stdout:6/347: write d4/d1a/f42 [3860334,42703] 0 2024-05-12T04:01:09.011 INFO:tasks.workunit.client.1.smithi033.stdout:6/348: write d4/d43/f70 [682521,43521] 0 2024-05-12T04:01:09.017 INFO:tasks.workunit.client.1.smithi033.stdout:3/346: mknod db/d17/d25/d32/d53/c75 0 2024-05-12T04:01:09.023 INFO:tasks.workunit.client.1.smithi033.stdout:9/559: getdents d2 0 2024-05-12T04:01:09.024 INFO:tasks.workunit.client.0.smithi032.stdout:4/124: rename d0/d3/fb to d0/d3/f1c 0 2024-05-12T04:01:09.026 INFO:tasks.workunit.client.1.smithi033.stdout:7/504: mkdir de/d2b/d6c/db5 0 2024-05-12T04:01:09.027 INFO:tasks.workunit.client.1.smithi033.stdout:7/505: write de/d20/f21 [4780819,74633] 0 2024-05-12T04:01:09.029 INFO:tasks.workunit.client.1.smithi033.stdout:1/309: sync 2024-05-12T04:01:09.031 INFO:tasks.workunit.client.1.smithi033.stdout:8/269: unlink d1/d16/d27/d4d/d51/f48 0 2024-05-12T04:01:09.033 INFO:tasks.workunit.client.1.smithi033.stdout:8/270: write d1/d4/d46/f4f [4107680,55354] 0 2024-05-12T04:01:09.034 INFO:tasks.workunit.client.1.smithi033.stdout:2/359: creat dc/d17/d27/d2f/d49/d59/f80 x:0 0 0 2024-05-12T04:01:09.036 INFO:tasks.workunit.client.1.smithi033.stdout:1/310: dread d11/d17/d27/d36/d3f/f50 [0,4194304] 0 2024-05-12T04:01:09.039 INFO:tasks.workunit.client.1.smithi033.stdout:6/349: creat d4/d5/f76 x:0 0 0 2024-05-12T04:01:09.042 INFO:tasks.workunit.client.0.smithi032.stdout:7/163: dread dd/f12 [0,4194304] 0 2024-05-12T04:01:09.042 INFO:tasks.workunit.client.0.smithi032.stdout:5/220: mknod d4/d7/d14/d26/d33/d4a/d38/d3b/c4b 0 2024-05-12T04:01:09.045 INFO:tasks.workunit.client.0.smithi032.stdout:5/221: truncate d4/d7/d14/d26/d33/d4a/f3e 604056 0 2024-05-12T04:01:09.045 INFO:tasks.workunit.client.0.smithi032.stdout:9/118: write d0/d3/d6/d7/da/d17/f1e [2277156,118976] 0 2024-05-12T04:01:09.047 INFO:tasks.workunit.client.0.smithi032.stdout:1/156: creat d1/d25/f34 x:0 0 0 2024-05-12T04:01:09.047 INFO:tasks.workunit.client.1.smithi033.stdout:3/347: dread db/d17/d3a/f66 [0,4194304] 0 2024-05-12T04:01:09.048 INFO:tasks.workunit.client.1.smithi033.stdout:9/560: symlink d2/dd/d39/ld0 0 2024-05-12T04:01:09.051 INFO:tasks.workunit.client.1.smithi033.stdout:7/506: truncate de/d2b/f2c 3800107 0 2024-05-12T04:01:09.053 INFO:tasks.workunit.client.1.smithi033.stdout:7/507: fsync de/d2f/d70/f94 0 2024-05-12T04:01:09.053 INFO:tasks.workunit.client.1.smithi033.stdout:0/394: dwrite d6/db/f19 [0,4194304] 0 2024-05-12T04:01:09.055 INFO:tasks.workunit.client.1.smithi033.stdout:7/508: truncate de/d2f/d70/f74 859516 0 2024-05-12T04:01:09.056 INFO:tasks.workunit.client.1.smithi033.stdout:7/509: write de/df/f48 [2415071,116865] 0 2024-05-12T04:01:09.057 INFO:tasks.workunit.client.1.smithi033.stdout:8/271: rmdir d1/d16/d27/d28 39 2024-05-12T04:01:09.058 INFO:tasks.workunit.client.1.smithi033.stdout:8/272: write d1/d16/d20/f43 [56191,111878] 0 2024-05-12T04:01:09.060 INFO:tasks.workunit.client.1.smithi033.stdout:8/273: truncate d1/d16/d27/f2d 4417449 0 2024-05-12T04:01:09.062 INFO:tasks.workunit.client.0.smithi032.stdout:3/216: creat d15/f45 x:0 0 0 2024-05-12T04:01:09.063 INFO:tasks.workunit.client.0.smithi032.stdout:3/217: chown d15/d24/f3e 30561 1 2024-05-12T04:01:09.064 INFO:tasks.workunit.client.0.smithi032.stdout:3/218: chown d15/d33/c41 16299 1 2024-05-12T04:01:09.065 INFO:tasks.workunit.client.0.smithi032.stdout:7/164: creat dd/d19/d37/f3a x:0 0 0 2024-05-12T04:01:09.066 INFO:tasks.workunit.client.1.smithi033.stdout:6/350: mknod d4/d1a/d67/d68/c77 0 2024-05-12T04:01:09.070 INFO:tasks.workunit.client.1.smithi033.stdout:4/354: rename dd/c40 to dd/d11/c70 0 2024-05-12T04:01:09.071 INFO:tasks.workunit.client.0.smithi032.stdout:5/222: dread d4/f20 [0,4194304] 0 2024-05-12T04:01:09.071 INFO:tasks.workunit.client.0.smithi032.stdout:3/219: dread d15/d20/d22/f2e [4194304,4194304] 0 2024-05-12T04:01:09.073 INFO:tasks.workunit.client.0.smithi032.stdout:7/165: dwrite fa [0,4194304] 0 2024-05-12T04:01:09.073 INFO:tasks.workunit.client.1.smithi033.stdout:3/348: mkdir db/d17/d3a/d67/d76 0 2024-05-12T04:01:09.076 INFO:tasks.workunit.client.0.smithi032.stdout:1/157: mkdir d1/d7/d30/d35 0 2024-05-12T04:01:09.079 INFO:tasks.workunit.client.0.smithi032.stdout:4/125: mkdir d0/d3/d12/d1d 0 2024-05-12T04:01:09.081 INFO:tasks.workunit.client.1.smithi033.stdout:9/561: symlink d2/d4/d2b/d78/d76/ld1 0 2024-05-12T04:01:09.083 INFO:tasks.workunit.client.0.smithi032.stdout:2/135: rename d1/d13/d25/d2e to d1/d34 0 2024-05-12T04:01:09.084 INFO:tasks.workunit.client.0.smithi032.stdout:0/248: write fd [501087,100556] 0 2024-05-12T04:01:09.086 INFO:tasks.workunit.client.0.smithi032.stdout:0/249: chown df/d10/d3d/d4a/f29 51647 1 2024-05-12T04:01:09.088 INFO:tasks.workunit.client.0.smithi032.stdout:8/187: truncate d0/f2 6889014 0 2024-05-12T04:01:09.090 INFO:tasks.workunit.client.1.smithi033.stdout:0/395: truncate d6/dc/d6e/f29 1066977 0 2024-05-12T04:01:09.095 INFO:tasks.workunit.client.0.smithi032.stdout:9/119: symlink d0/d3/d6/d7/da/l26 0 2024-05-12T04:01:09.096 INFO:tasks.workunit.client.1.smithi033.stdout:9/562: dread d2/dd/f13 [0,4194304] 0 2024-05-12T04:01:09.097 INFO:tasks.workunit.client.1.smithi033.stdout:1/311: truncate f8 6079185 0 2024-05-12T04:01:09.099 INFO:tasks.workunit.client.0.smithi032.stdout:7/166: mknod dd/c3b 0 2024-05-12T04:01:09.100 INFO:tasks.workunit.client.0.smithi032.stdout:5/223: dread d4/fd [0,4194304] 0 2024-05-12T04:01:09.102 INFO:tasks.workunit.client.1.smithi033.stdout:9/563: dwrite d2/d4/d2b/d78/f93 [0,4194304] 0 2024-05-12T04:01:09.104 INFO:tasks.workunit.client.1.smithi033.stdout:2/360: rename dc/d17/d27/d2f/f62 to dc/d13/d1f/d6e/f81 0 2024-05-12T04:01:09.105 INFO:tasks.workunit.client.0.smithi032.stdout:5/224: dread d4/f1e [0,4194304] 0 2024-05-12T04:01:09.106 INFO:tasks.workunit.client.0.smithi032.stdout:4/126: symlink d0/d3/l1e 0 2024-05-12T04:01:09.107 INFO:tasks.workunit.client.1.smithi033.stdout:4/355: truncate dd/f2a 123961 0 2024-05-12T04:01:09.112 INFO:tasks.workunit.client.1.smithi033.stdout:3/349: chown db/d17/d25/d32/d3c/d44/f55 745818 1 2024-05-12T04:01:09.113 INFO:tasks.workunit.client.0.smithi032.stdout:3/220: rename d15/f17 to d15/d33/f46 0 2024-05-12T04:01:09.114 INFO:tasks.workunit.client.0.smithi032.stdout:3/221: readlink d15/d20/d26/l30 0 2024-05-12T04:01:09.116 INFO:tasks.workunit.client.1.smithi033.stdout:5/278: getdents d3/d12 0 2024-05-12T04:01:09.117 INFO:tasks.workunit.client.1.smithi033.stdout:5/279: dread d3/d12/d25/d26/d2e/f3e [0,4194304] 0 2024-05-12T04:01:09.118 INFO:tasks.workunit.client.1.smithi033.stdout:5/280: readlink d3/l4d 0 2024-05-12T04:01:09.119 INFO:tasks.workunit.client.0.smithi032.stdout:0/250: creat df/d28/f59 x:0 0 0 2024-05-12T04:01:09.122 INFO:tasks.workunit.client.0.smithi032.stdout:9/120: rmdir d0 39 2024-05-12T04:01:09.124 INFO:tasks.workunit.client.1.smithi033.stdout:1/312: mkdir d11/d17/d27/d36/d3f/d69 0 2024-05-12T04:01:09.126 INFO:tasks.workunit.client.1.smithi033.stdout:6/351: getdents d4/d1a/d6c 0 2024-05-12T04:01:09.127 INFO:tasks.workunit.client.0.smithi032.stdout:7/167: symlink dd/d19/l3c 0 2024-05-12T04:01:09.130 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:08 smithi032 ceph-mon[40008]: pgmap v32: 65 pgs: 65 active+clean; 970 MiB data, 4.4 GiB used, 532 GiB / 536 GiB avail; 13 MiB/s rd, 111 MiB/s wr, 237 op/s 2024-05-12T04:01:09.130 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:08 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/1919915313' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-12T04:01:09.132 INFO:tasks.workunit.client.1.smithi033.stdout:7/510: write de/df/d17/f51 [1830466,54018] 0 2024-05-12T04:01:09.134 INFO:tasks.workunit.client.0.smithi032.stdout:5/225: mkdir d4/d7/d14/d26/d33/d4a/d38/d3b/d4c 0 2024-05-12T04:01:09.137 INFO:tasks.workunit.client.1.smithi033.stdout:2/361: rename dc/d17/d27/d2f/l46 to dc/d17/d27/d47/d5e/l82 0 2024-05-12T04:01:09.140 INFO:tasks.workunit.client.1.smithi033.stdout:7/511: dwrite de/df/d17/d1e/d79/f91 [0,4194304] 0 2024-05-12T04:01:09.144 INFO:tasks.workunit.client.1.smithi033.stdout:6/352: dread d4/d1a/f23 [0,4194304] 0 2024-05-12T04:01:09.146 INFO:tasks.workunit.client.1.smithi033.stdout:2/362: dwrite dc/d17/d27/d47/f58 [0,4194304] 0 2024-05-12T04:01:09.146 INFO:tasks.workunit.client.1.smithi033.stdout:6/353: write d4/d1a/f47 [2280189,17801] 0 2024-05-12T04:01:09.147 INFO:tasks.workunit.client.1.smithi033.stdout:6/354: chown d4/l38 3 1 2024-05-12T04:01:09.148 INFO:tasks.workunit.client.0.smithi032.stdout:8/188: creat d0/d9/d1b/d33/f3a x:0 0 0 2024-05-12T04:01:09.151 INFO:tasks.workunit.client.0.smithi032.stdout:2/136: write d1/fa [908429,79472] 0 2024-05-12T04:01:09.153 INFO:tasks.workunit.client.1.smithi033.stdout:5/281: unlink d3/d12/d29/f48 0 2024-05-12T04:01:09.154 INFO:tasks.workunit.client.1.smithi033.stdout:5/282: read d3/d12/d25/d26/f31 [2921500,48168] 0 2024-05-12T04:01:09.156 INFO:tasks.workunit.client.1.smithi033.stdout:0/396: write d6/dc/f27 [1279565,89129] 0 2024-05-12T04:01:09.156 INFO:tasks.workunit.client.0.smithi032.stdout:7/168: symlink dd/l3d 0 2024-05-12T04:01:09.160 INFO:tasks.workunit.client.1.smithi033.stdout:1/313: mknod d11/d17/d27/d36/d46/c6a 0 2024-05-12T04:01:09.160 INFO:tasks.workunit.client.0.smithi032.stdout:7/169: write dd/d19/d37/f3a [776483,65881] 0 2024-05-12T04:01:09.160 INFO:tasks.workunit.client.0.smithi032.stdout:2/137: dread d1/f24 [0,4194304] 0 2024-05-12T04:01:09.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:08 smithi033 ceph-mon[49525]: pgmap v32: 65 pgs: 65 active+clean; 970 MiB data, 4.4 GiB used, 532 GiB / 536 GiB avail; 13 MiB/s rd, 111 MiB/s wr, 237 op/s 2024-05-12T04:01:09.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:08 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/1919915313' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-12T04:01:09.166 INFO:tasks.workunit.client.0.smithi032.stdout:2/138: dread d1/ff [0,4194304] 0 2024-05-12T04:01:09.167 INFO:tasks.workunit.client.0.smithi032.stdout:2/139: read d1/f3 [272139,61271] 0 2024-05-12T04:01:09.168 INFO:tasks.workunit.client.0.smithi032.stdout:3/222: link d15/d1b/f25 d15/d20/d22/f47 0 2024-05-12T04:01:09.168 INFO:tasks.workunit.client.0.smithi032.stdout:2/140: chown d1/d13/d25/f28 55573098 1 2024-05-12T04:01:09.172 INFO:tasks.workunit.client.0.smithi032.stdout:9/121: mkdir d0/d3/d6/d7/d27 0 2024-05-12T04:01:09.176 INFO:tasks.workunit.client.0.smithi032.stdout:1/158: getdents d1/d7/d30 0 2024-05-12T04:01:09.181 INFO:tasks.workunit.client.0.smithi032.stdout:7/170: creat dd/d19/f3e x:0 0 0 2024-05-12T04:01:09.184 INFO:tasks.workunit.client.0.smithi032.stdout:5/226: symlink d4/d7/d14/d26/d33/d4a/d38/d3b/d4c/l4d 0 2024-05-12T04:01:09.189 INFO:tasks.workunit.client.1.smithi033.stdout:2/363: unlink dc/d17/d27/d47/f5d 0 2024-05-12T04:01:09.192 INFO:tasks.workunit.client.1.smithi033.stdout:6/355: rename d4/d1a/f47 to d4/f78 0 2024-05-12T04:01:09.196 INFO:tasks.workunit.client.1.smithi033.stdout:5/283: mknod d3/d12/d25/d26/d2e/d54/c58 0 2024-05-12T04:01:09.197 INFO:tasks.workunit.client.0.smithi032.stdout:3/223: read d15/d1b/f34 [1725274,102375] 0 2024-05-12T04:01:09.197 INFO:tasks.workunit.client.1.smithi033.stdout:5/284: chown d3/d15/d16/d45/d4b 24300 1 2024-05-12T04:01:09.198 INFO:tasks.workunit.client.0.smithi032.stdout:3/224: truncate d15/f19 762357 0 2024-05-12T04:01:09.199 INFO:tasks.workunit.client.1.smithi033.stdout:8/274: getdents d1/d4/d1d 0 2024-05-12T04:01:09.206 INFO:tasks.workunit.client.1.smithi033.stdout:4/356: link l7 dd/d11/d5b/l71 0 2024-05-12T04:01:09.207 INFO:tasks.workunit.client.0.smithi032.stdout:5/227: creat d4/d7/d14/d26/d33/d4a/f4e x:0 0 0 2024-05-12T04:01:09.208 INFO:tasks.workunit.client.0.smithi032.stdout:4/127: getdents d0/d3 0 2024-05-12T04:01:09.209 INFO:tasks.workunit.client.1.smithi033.stdout:8/275: dread d1/d4/fd [0,4194304] 0 2024-05-12T04:01:09.210 INFO:tasks.workunit.client.0.smithi032.stdout:2/141: symlink d1/l35 0 2024-05-12T04:01:09.210 INFO:tasks.workunit.client.0.smithi032.stdout:9/122: dwrite d0/d3/d6/d7/da/f1a [0,4194304] 0 2024-05-12T04:01:09.211 INFO:tasks.workunit.client.0.smithi032.stdout:4/128: write d0/d3/fd [338224,34492] 0 2024-05-12T04:01:09.213 INFO:tasks.workunit.client.0.smithi032.stdout:9/123: readlink d0/d3/d6/d7/l22 0 2024-05-12T04:01:09.213 INFO:tasks.workunit.client.1.smithi033.stdout:2/364: rmdir dc/d17/d32/d67 39 2024-05-12T04:01:09.215 INFO:tasks.workunit.client.1.smithi033.stdout:2/365: write dc/d17/d27/d2f/d49/d59/f79 [660050,62052] 0 2024-05-12T04:01:09.216 INFO:tasks.workunit.client.1.smithi033.stdout:6/356: mknod d4/d1a/d3c/c79 0 2024-05-12T04:01:09.219 INFO:tasks.workunit.client.0.smithi032.stdout:3/225: dwrite d15/d20/d26/f29 [4194304,4194304] 0 2024-05-12T04:01:09.221 INFO:tasks.workunit.client.0.smithi032.stdout:7/171: truncate dd/d1b/d1e/f28 4660230 0 2024-05-12T04:01:09.222 INFO:tasks.workunit.client.0.smithi032.stdout:7/172: read dd/f36 [1436787,12589] 0 2024-05-12T04:01:09.227 INFO:tasks.workunit.client.0.smithi032.stdout:8/189: getdents d0/d9/d13/d21/d25 0 2024-05-12T04:01:09.229 INFO:tasks.workunit.client.1.smithi033.stdout:9/564: getdents d2/d70/d7b 0 2024-05-12T04:01:09.233 INFO:tasks.workunit.client.1.smithi033.stdout:3/350: getdents db/d17/d25/d32 0 2024-05-12T04:01:09.234 INFO:tasks.workunit.client.1.smithi033.stdout:8/276: stat d1/d16/c41 0 2024-05-12T04:01:09.236 INFO:tasks.workunit.client.1.smithi033.stdout:9/565: dread d2/d70/d7b/f9f [0,4194304] 0 2024-05-12T04:01:09.237 INFO:tasks.workunit.client.1.smithi033.stdout:2/366: creat dc/d13/d65/f83 x:0 0 0 2024-05-12T04:01:09.237 INFO:tasks.workunit.client.1.smithi033.stdout:8/277: write d1/d4/d1d/f26 [1813248,118204] 0 2024-05-12T04:01:09.240 INFO:tasks.workunit.client.0.smithi032.stdout:5/228: write d4/f6 [1095625,78246] 0 2024-05-12T04:01:09.241 INFO:tasks.workunit.client.1.smithi033.stdout:8/278: read d1/f13 [462638,52907] 0 2024-05-12T04:01:09.241 INFO:tasks.workunit.client.1.smithi033.stdout:3/351: dwrite f6 [4194304,4194304] 0 2024-05-12T04:01:09.242 INFO:tasks.workunit.client.1.smithi033.stdout:2/367: write dc/d17/d27/d2f/d49/d59/f80 [264614,53773] 0 2024-05-12T04:01:09.242 INFO:tasks.workunit.client.0.smithi032.stdout:8/190: creat d0/d9/d1b/d33/f3b x:0 0 0 2024-05-12T04:01:09.245 INFO:tasks.workunit.client.1.smithi033.stdout:8/279: write d1/d16/d27/d4d/d51/f31 [4230270,83666] 0 2024-05-12T04:01:09.248 INFO:tasks.workunit.client.1.smithi033.stdout:1/314: link d11/f4f d11/f6b 0 2024-05-12T04:01:09.249 INFO:tasks.workunit.client.0.smithi032.stdout:3/226: creat d15/d24/d40/f48 x:0 0 0 2024-05-12T04:01:09.249 INFO:tasks.workunit.client.1.smithi033.stdout:1/315: write d11/d1c/d43/f5e [1503835,78823] 0 2024-05-12T04:01:09.254 INFO:tasks.workunit.client.1.smithi033.stdout:7/512: getdents de/d20/da9 0 2024-05-12T04:01:09.256 INFO:tasks.workunit.client.0.smithi032.stdout:5/229: rmdir d4 39 2024-05-12T04:01:09.259 INFO:tasks.workunit.client.0.smithi032.stdout:9/124: creat d0/f28 x:0 0 0 2024-05-12T04:01:09.262 INFO:tasks.workunit.client.0.smithi032.stdout:8/191: creat d0/d9/d13/d21/d35/f3c x:0 0 0 2024-05-12T04:01:09.263 INFO:tasks.workunit.client.1.smithi033.stdout:5/285: rmdir d3/d15/d16/d41 0 2024-05-12T04:01:09.265 INFO:tasks.workunit.client.0.smithi032.stdout:4/129: link d0/d3/df/c1b d0/d3/d12/c1f 0 2024-05-12T04:01:09.265 INFO:tasks.workunit.client.0.smithi032.stdout:4/130: chown d0/fa 23 1 2024-05-12T04:01:09.267 INFO:tasks.workunit.client.1.smithi033.stdout:8/280: dread d1/f2 [0,4194304] 0 2024-05-12T04:01:09.268 INFO:tasks.workunit.client.1.smithi033.stdout:3/352: dread db/d17/d25/f40 [0,4194304] 0 2024-05-12T04:01:09.269 INFO:tasks.workunit.client.0.smithi032.stdout:3/227: rename d15/d20/d26/l30 to d15/d24/l49 0 2024-05-12T04:01:09.273 INFO:tasks.workunit.client.1.smithi033.stdout:9/566: write d2/d4/d2b/d6d/d3a/f5a [1201258,68663] 0 2024-05-12T04:01:09.273 INFO:tasks.workunit.client.1.smithi033.stdout:0/397: getdents d6/dc/d39 0 2024-05-12T04:01:09.277 INFO:tasks.workunit.client.1.smithi033.stdout:0/398: write d6/dc/f1d [4820997,111980] 0 2024-05-12T04:01:09.278 INFO:tasks.workunit.client.0.smithi032.stdout:9/125: creat d0/d3/d6/f29 x:0 0 0 2024-05-12T04:01:09.278 INFO:tasks.workunit.client.1.smithi033.stdout:9/567: fdatasync d2/d4/d29/d32/db7/fc6 0 2024-05-12T04:01:09.278 INFO:tasks.workunit.client.1.smithi033.stdout:8/281: dwrite d1/d4/d1d/f26 [0,4194304] 0 2024-05-12T04:01:09.279 INFO:tasks.workunit.client.1.smithi033.stdout:6/357: dwrite d4/d5/f27 [4194304,4194304] 0 2024-05-12T04:01:09.282 INFO:tasks.workunit.client.1.smithi033.stdout:8/282: readlink d1/d16/d29/l3b 0 2024-05-12T04:01:09.284 INFO:tasks.workunit.client.1.smithi033.stdout:4/357: link dd/f35 dd/d11/d31/f72 0 2024-05-12T04:01:09.285 INFO:tasks.workunit.client.1.smithi033.stdout:7/513: unlink de/df/l49 0 2024-05-12T04:01:09.286 INFO:tasks.workunit.client.0.smithi032.stdout:7/173: getdents dd/d1b 0 2024-05-12T04:01:09.288 INFO:tasks.workunit.client.0.smithi032.stdout:5/230: creat d4/d7/d14/d26/d33/d4a/f4f x:0 0 0 2024-05-12T04:01:09.289 INFO:tasks.workunit.client.0.smithi032.stdout:9/126: mkdir d0/d3/d6/d7/d2a 0 2024-05-12T04:01:09.290 INFO:tasks.workunit.client.0.smithi032.stdout:9/127: stat d0/d3 0 2024-05-12T04:01:09.290 INFO:tasks.workunit.client.1.smithi033.stdout:0/399: dwrite d6/dc/d58/f59 [0,4194304] 0 2024-05-12T04:01:09.290 INFO:tasks.workunit.client.1.smithi033.stdout:9/568: dread d2/dd/d39/f43 [0,4194304] 0 2024-05-12T04:01:09.291 INFO:tasks.workunit.client.1.smithi033.stdout:5/286: creat d3/d12/d29/f59 x:0 0 0 2024-05-12T04:01:09.293 INFO:tasks.workunit.client.1.smithi033.stdout:5/287: chown d3/ce 1763 1 2024-05-12T04:01:09.295 INFO:tasks.workunit.client.0.smithi032.stdout:3/228: symlink d15/d1b/d43/l4a 0 2024-05-12T04:01:09.295 INFO:tasks.workunit.client.1.smithi033.stdout:5/288: stat d3/d15/d16/f51 0 2024-05-12T04:01:09.295 INFO:tasks.workunit.client.0.smithi032.stdout:3/229: dread - d15/d24/f3e zero size 2024-05-12T04:01:09.296 INFO:tasks.workunit.client.1.smithi033.stdout:3/353: rmdir db/d17/d25/d32/d3c/d3d 39 2024-05-12T04:01:09.297 INFO:tasks.workunit.client.1.smithi033.stdout:7/514: dwrite de/d2f/d70/f94 [0,4194304] 0 2024-05-12T04:01:09.297 INFO:tasks.workunit.client.0.smithi032.stdout:7/174: mknod dd/d1b/c3f 0 2024-05-12T04:01:09.299 INFO:tasks.workunit.client.1.smithi033.stdout:2/368: mkdir dc/d17/d32/d84 0 2024-05-12T04:01:09.302 INFO:tasks.workunit.client.1.smithi033.stdout:7/515: dread de/d2f/d70/f94 [0,4194304] 0 2024-05-12T04:01:09.305 INFO:tasks.workunit.client.0.smithi032.stdout:3/230: dread f12 [0,4194304] 0 2024-05-12T04:01:09.307 INFO:tasks.workunit.client.1.smithi033.stdout:4/358: chown dd/d11/d31/d66/l23 0 1 2024-05-12T04:01:09.308 INFO:tasks.workunit.client.0.smithi032.stdout:3/231: chown d15/d33/f46 331148638 1 2024-05-12T04:01:09.308 INFO:tasks.workunit.client.0.smithi032.stdout:7/175: dwrite dd/d19/f3e [0,4194304] 0 2024-05-12T04:01:09.314 INFO:tasks.workunit.client.1.smithi033.stdout:0/400: unlink f2 0 2024-05-12T04:01:09.315 INFO:tasks.workunit.client.0.smithi032.stdout:9/128: truncate d0/d3/d6/d7/f15 471621 0 2024-05-12T04:01:09.315 INFO:tasks.workunit.client.0.smithi032.stdout:4/131: getdents d0/d3 0 2024-05-12T04:01:09.320 INFO:tasks.workunit.client.1.smithi033.stdout:9/569: mkdir d2/d4/d2b/d33/dd2 0 2024-05-12T04:01:09.321 INFO:tasks.workunit.client.1.smithi033.stdout:0/401: dwrite d6/dc/f1d [4194304,4194304] 0 2024-05-12T04:01:09.322 INFO:tasks.workunit.client.1.smithi033.stdout:5/289: mkdir d3/d15/d5a 0 2024-05-12T04:01:09.324 INFO:tasks.workunit.client.0.smithi032.stdout:3/232: rename d15/d33/c37 to d15/d33/c4b 0 2024-05-12T04:01:09.326 INFO:tasks.workunit.client.1.smithi033.stdout:2/369: dread dc/d17/d27/d2f/f41 [0,4194304] 0 2024-05-12T04:01:09.327 INFO:tasks.workunit.client.0.smithi032.stdout:8/192: dwrite d0/f5 [4194304,4194304] 0 2024-05-12T04:01:09.327 INFO:tasks.workunit.client.1.smithi033.stdout:2/370: stat dc/d17/d3a/l4e 0 2024-05-12T04:01:09.329 INFO:tasks.workunit.client.0.smithi032.stdout:7/176: rename dd/d1b/l24 to dd/d1b/l40 0 2024-05-12T04:01:09.332 INFO:tasks.workunit.client.0.smithi032.stdout:6/173: sync 2024-05-12T04:01:09.335 INFO:tasks.workunit.client.0.smithi032.stdout:4/132: rename d0/l18 to d0/d3/l20 0 2024-05-12T04:01:09.341 INFO:tasks.workunit.client.0.smithi032.stdout:7/177: creat dd/d19/d37/f41 x:0 0 0 2024-05-12T04:01:09.343 INFO:tasks.workunit.client.0.smithi032.stdout:7/178: write dd/d19/f2a [761277,106493] 0 2024-05-12T04:01:09.343 INFO:tasks.workunit.client.0.smithi032.stdout:8/193: dread d0/d7/f11 [0,4194304] 0 2024-05-12T04:01:09.344 INFO:tasks.workunit.client.0.smithi032.stdout:9/129: mknod d0/d3/d6/d7/da/c2b 0 2024-05-12T04:01:09.347 INFO:tasks.workunit.client.0.smithi032.stdout:6/174: mknod d7/d9/c3c 0 2024-05-12T04:01:09.347 INFO:tasks.workunit.client.0.smithi032.stdout:7/179: dread dd/f13 [0,4194304] 0 2024-05-12T04:01:09.348 INFO:tasks.workunit.client.1.smithi033.stdout:4/359: readlink dd/l1f 0 2024-05-12T04:01:09.349 INFO:tasks.workunit.client.1.smithi033.stdout:4/360: chown dd/d11/c67 142 1 2024-05-12T04:01:09.350 INFO:tasks.workunit.client.0.smithi032.stdout:7/180: write f9 [1216001,22291] 0 2024-05-12T04:01:09.351 INFO:tasks.workunit.client.1.smithi033.stdout:4/361: dread - dd/d11/d31/f6c zero size 2024-05-12T04:01:09.352 INFO:tasks.workunit.client.1.smithi033.stdout:3/354: dwrite db/d17/d25/d32/d3c/d44/f4a [0,4194304] 0 2024-05-12T04:01:09.352 INFO:tasks.workunit.client.0.smithi032.stdout:4/133: mknod d0/d3/df/c21 0 2024-05-12T04:01:09.354 INFO:tasks.workunit.client.1.smithi033.stdout:3/355: truncate db/d59/f5b 1091324 0 2024-05-12T04:01:09.354 INFO:tasks.workunit.client.0.smithi032.stdout:9/130: dwrite d0/f28 [0,4194304] 0 2024-05-12T04:01:09.355 INFO:tasks.workunit.client.0.smithi032.stdout:4/134: read - d0/f13 zero size 2024-05-12T04:01:09.356 INFO:tasks.workunit.client.1.smithi033.stdout:6/358: truncate d4/d1a/f42 4326930 0 2024-05-12T04:01:09.358 INFO:tasks.workunit.client.1.smithi033.stdout:9/570: mknod d2/d70/d7b/cd3 0 2024-05-12T04:01:09.358 INFO:tasks.workunit.client.1.smithi033.stdout:4/362: dwrite dd/d11/d31/d66/d42/f4c [0,4194304] 0 2024-05-12T04:01:09.362 INFO:tasks.workunit.client.0.smithi032.stdout:3/233: dread d15/d33/f46 [0,4194304] 0 2024-05-12T04:01:09.363 INFO:tasks.workunit.client.0.smithi032.stdout:8/194: rename d0/d9/d13/d21/c34 to d0/d7/c3d 0 2024-05-12T04:01:09.364 INFO:tasks.workunit.client.0.smithi032.stdout:3/234: chown d15/d1b/d43 59826006 1 2024-05-12T04:01:09.365 INFO:tasks.workunit.client.1.smithi033.stdout:0/402: creat d6/db/d2c/f83 x:0 0 0 2024-05-12T04:01:09.366 INFO:tasks.workunit.client.1.smithi033.stdout:5/290: unlink d3/d15/d16/f1a 0 2024-05-12T04:01:09.367 INFO:tasks.workunit.client.0.smithi032.stdout:4/135: dwrite d0/f13 [0,4194304] 0 2024-05-12T04:01:09.372 INFO:tasks.workunit.client.1.smithi033.stdout:0/403: dwrite d6/dc/f1d [0,4194304] 0 2024-05-12T04:01:09.372 INFO:tasks.workunit.client.0.smithi032.stdout:6/175: creat d7/de/d12/d1a/d20/f3d x:0 0 0 2024-05-12T04:01:09.373 INFO:tasks.workunit.client.1.smithi033.stdout:0/404: chown d6/db 24 1 2024-05-12T04:01:09.377 INFO:tasks.workunit.client.1.smithi033.stdout:0/405: chown d6/f23 4 1 2024-05-12T04:01:09.377 INFO:tasks.workunit.client.1.smithi033.stdout:5/291: dread d3/d12/d25/f4a [0,4194304] 0 2024-05-12T04:01:09.378 INFO:tasks.workunit.client.0.smithi032.stdout:4/136: dread d0/d3/fd [0,4194304] 0 2024-05-12T04:01:09.378 INFO:tasks.workunit.client.0.smithi032.stdout:9/131: creat d0/d3/d6/d7/f2c x:0 0 0 2024-05-12T04:01:09.378 INFO:tasks.workunit.client.0.smithi032.stdout:8/195: dread d0/d9/d13/d21/d25/f2c [0,4194304] 0 2024-05-12T04:01:09.380 INFO:tasks.workunit.client.0.smithi032.stdout:9/132: fdatasync d0/d3/d1b/f24 0 2024-05-12T04:01:09.381 INFO:tasks.workunit.client.0.smithi032.stdout:6/176: dread d7/fa [0,4194304] 0 2024-05-12T04:01:09.381 INFO:tasks.workunit.client.0.smithi032.stdout:3/235: creat d15/d33/f4c x:0 0 0 2024-05-12T04:01:09.382 INFO:tasks.workunit.client.0.smithi032.stdout:6/177: dread - d7/de/f3b zero size 2024-05-12T04:01:09.384 INFO:tasks.workunit.client.1.smithi033.stdout:9/571: fsync d2/dd/d39/d72/d82/fcd 0 2024-05-12T04:01:09.385 INFO:tasks.workunit.client.1.smithi033.stdout:9/572: stat d2/d4/d2b/d6d/d2f/l53 0 2024-05-12T04:01:09.387 INFO:tasks.workunit.client.0.smithi032.stdout:9/133: truncate d0/d3/d6/f29 248269 0 2024-05-12T04:01:09.392 INFO:tasks.workunit.client.1.smithi033.stdout:8/283: rename d1/f12 to d1/d4/d1d/d36/f59 0 2024-05-12T04:01:09.394 INFO:tasks.workunit.client.0.smithi032.stdout:8/196: mkdir d0/d9/d13/d26/d28/d3e 0 2024-05-12T04:01:09.395 INFO:tasks.workunit.client.0.smithi032.stdout:4/137: dwrite d0/d3/f1c [4194304,4194304] 0 2024-05-12T04:01:09.396 INFO:tasks.workunit.client.0.smithi032.stdout:8/197: chown d0/d9/d13/d21/f2a 8056 1 2024-05-12T04:01:09.404 INFO:tasks.workunit.client.0.smithi032.stdout:6/178: rename d7/fc to d7/de/d12/d25/f3e 0 2024-05-12T04:01:09.405 INFO:tasks.workunit.client.0.smithi032.stdout:0/251: sync 2024-05-12T04:01:09.439 INFO:tasks.workunit.client.1.smithi033.stdout:1/316: sync 2024-05-12T04:01:09.513 INFO:tasks.workunit.client.1.smithi033.stdout:2/371: sync 2024-05-12T04:01:09.513 INFO:tasks.workunit.client.1.smithi033.stdout:6/359: sync 2024-05-12T04:01:09.514 INFO:tasks.workunit.client.1.smithi033.stdout:6/360: chown d4/d43/d57/l6b 195708 1 2024-05-12T04:01:09.514 INFO:tasks.workunit.client.1.smithi033.stdout:6/361: chown d4/d1a 6315 1 2024-05-12T04:01:09.515 INFO:tasks.workunit.client.1.smithi033.stdout:6/362: chown d4/d5/f10 119 1 2024-05-12T04:01:09.536 INFO:tasks.workunit.client.0.smithi032.stdout:6/179: rename d7/l31 to d7/l3f 0 2024-05-12T04:01:09.536 INFO:tasks.workunit.client.0.smithi032.stdout:6/180: chown d7/l10 208877 1 2024-05-12T04:01:09.540 INFO:tasks.workunit.client.0.smithi032.stdout:6/181: dread d7/de/d2e/f36 [0,4194304] 0 2024-05-12T04:01:09.540 INFO:tasks.workunit.client.0.smithi032.stdout:7/181: getdents dd/d19 0 2024-05-12T04:01:09.541 INFO:tasks.workunit.client.0.smithi032.stdout:6/182: write d7/de/d12/f23 [396193,48315] 0 2024-05-12T04:01:09.545 INFO:tasks.workunit.client.0.smithi032.stdout:3/236: creat d15/f4d x:0 0 0 2024-05-12T04:01:09.548 INFO:tasks.workunit.client.0.smithi032.stdout:7/182: creat dd/f42 x:0 0 0 2024-05-12T04:01:09.549 INFO:tasks.workunit.client.0.smithi032.stdout:8/198: write d0/d7/f15 [530574,68456] 0 2024-05-12T04:01:09.549 INFO:tasks.workunit.client.0.smithi032.stdout:7/183: readlink dd/l2e 0 2024-05-12T04:01:09.551 INFO:tasks.workunit.client.0.smithi032.stdout:6/183: chown d7/d9/f15 0 1 2024-05-12T04:01:09.552 INFO:tasks.workunit.client.0.smithi032.stdout:8/199: truncate d0/d9/d13/d21/d35/f3c 819328 0 2024-05-12T04:01:09.552 INFO:tasks.workunit.client.0.smithi032.stdout:0/252: link df/d30/d4d/c4f df/d10/d3b/c5a 0 2024-05-12T04:01:09.553 INFO:tasks.workunit.client.0.smithi032.stdout:6/184: write d7/de/f3b [779893,53917] 0 2024-05-12T04:01:09.554 INFO:tasks.workunit.client.0.smithi032.stdout:8/200: chown d0/f5 446095 1 2024-05-12T04:01:09.556 INFO:tasks.workunit.client.0.smithi032.stdout:7/184: dread dd/d1b/d1e/f20 [0,4194304] 0 2024-05-12T04:01:09.556 INFO:tasks.workunit.client.1.smithi033.stdout:5/292: creat d3/d12/d25/d26/d2e/f5b x:0 0 0 2024-05-12T04:01:09.559 INFO:tasks.workunit.client.0.smithi032.stdout:6/185: unlink d7/c21 0 2024-05-12T04:01:09.560 INFO:tasks.workunit.client.1.smithi033.stdout:7/516: getdents de/df/d60 0 2024-05-12T04:01:09.561 INFO:tasks.workunit.client.1.smithi033.stdout:7/517: write de/df/d17/f34 [1782689,116587] 0 2024-05-12T04:01:09.562 INFO:tasks.workunit.client.1.smithi033.stdout:9/573: rename d2/d4/ff to d2/d4/d29/d32/db7/fd4 0 2024-05-12T04:01:09.563 INFO:tasks.workunit.client.0.smithi032.stdout:3/237: creat d15/f4e x:0 0 0 2024-05-12T04:01:09.564 INFO:tasks.workunit.client.1.smithi033.stdout:9/574: chown d2/d4/d29/d3c/d52/fba 577307 1 2024-05-12T04:01:09.564 INFO:tasks.workunit.client.1.smithi033.stdout:9/575: stat d2/dd/d79/fae 0 2024-05-12T04:01:09.564 INFO:tasks.workunit.client.1.smithi033.stdout:8/284: fsync d1/d16/d27/d4d/d51/f4b 0 2024-05-12T04:01:09.568 INFO:tasks.workunit.client.0.smithi032.stdout:0/253: unlink df/l26 0 2024-05-12T04:01:09.569 INFO:tasks.workunit.client.1.smithi033.stdout:9/576: dwrite d2/d4/d2b/d78/f93 [0,4194304] 0 2024-05-12T04:01:09.570 INFO:tasks.workunit.client.1.smithi033.stdout:9/577: stat d2/dd/d39/d72 0 2024-05-12T04:01:09.571 INFO:tasks.workunit.client.0.smithi032.stdout:8/201: dwrite d0/d9/d13/d21/d35/d37/f39 [0,4194304] 0 2024-05-12T04:01:09.572 INFO:tasks.workunit.client.0.smithi032.stdout:7/185: unlink dd/d19/l3c 0 2024-05-12T04:01:09.574 INFO:tasks.workunit.client.0.smithi032.stdout:8/202: write d0/d9/d1b/d33/f3a [1028781,43212] 0 2024-05-12T04:01:09.575 INFO:tasks.workunit.client.0.smithi032.stdout:8/203: chown d0/d9/d13/f20 413 1 2024-05-12T04:01:09.582 INFO:tasks.workunit.client.0.smithi032.stdout:6/186: mknod d7/de/d12/d25/c40 0 2024-05-12T04:01:09.583 INFO:tasks.workunit.client.0.smithi032.stdout:3/238: mkdir d15/d20/d22/d4f 0 2024-05-12T04:01:09.584 INFO:tasks.workunit.client.1.smithi033.stdout:2/372: rmdir dc/d13/d39 39 2024-05-12T04:01:09.587 INFO:tasks.workunit.client.0.smithi032.stdout:0/254: mknod df/d30/d36/c5b 0 2024-05-12T04:01:09.589 INFO:tasks.workunit.client.1.smithi033.stdout:3/356: creat db/f77 x:0 0 0 2024-05-12T04:01:09.589 INFO:tasks.workunit.client.1.smithi033.stdout:4/363: write dd/d11/d31/d66/f2e [459333,87469] 0 2024-05-12T04:01:09.589 INFO:tasks.workunit.client.0.smithi032.stdout:7/186: mkdir dd/d1b/d43 0 2024-05-12T04:01:09.590 INFO:tasks.workunit.client.1.smithi033.stdout:3/357: write db/d17/f64 [885679,35046] 0 2024-05-12T04:01:09.591 INFO:tasks.workunit.client.0.smithi032.stdout:7/187: write dd/fe [2373525,42246] 0 2024-05-12T04:01:09.594 INFO:tasks.workunit.client.0.smithi032.stdout:8/204: mknod d0/d9/d13/d21/d35/c3f 0 2024-05-12T04:01:09.596 INFO:tasks.workunit.client.0.smithi032.stdout:6/187: creat d7/de/d12/f41 x:0 0 0 2024-05-12T04:01:09.600 INFO:tasks.workunit.client.0.smithi032.stdout:3/239: symlink d15/d24/d40/l50 0 2024-05-12T04:01:09.601 INFO:tasks.workunit.client.1.smithi033.stdout:7/518: mknod de/d2f/cb6 0 2024-05-12T04:01:09.603 INFO:tasks.workunit.client.1.smithi033.stdout:6/363: rename d4/c1d to d4/d1a/d67/c7a 0 2024-05-12T04:01:09.605 INFO:tasks.workunit.client.1.smithi033.stdout:3/358: dread db/f20 [0,4194304] 0 2024-05-12T04:01:09.608 INFO:tasks.workunit.client.0.smithi032.stdout:8/205: creat d0/d9/d13/d21/d35/d37/f40 x:0 0 0 2024-05-12T04:01:09.610 INFO:tasks.workunit.client.1.smithi033.stdout:0/406: rmdir d6/d11/d57/d77 0 2024-05-12T04:01:09.611 INFO:tasks.workunit.client.0.smithi032.stdout:6/188: creat d7/de/d12/d1a/d20/f42 x:0 0 0 2024-05-12T04:01:09.612 INFO:tasks.workunit.client.0.smithi032.stdout:6/189: dread - d7/de/d12/d1a/d20/f34 zero size 2024-05-12T04:01:09.614 INFO:tasks.workunit.client.0.smithi032.stdout:3/240: creat d15/d24/f51 x:0 0 0 2024-05-12T04:01:09.614 INFO:tasks.workunit.client.0.smithi032.stdout:1/159: sync 2024-05-12T04:01:09.615 INFO:tasks.workunit.client.0.smithi032.stdout:3/241: dread - d15/d33/f4c zero size 2024-05-12T04:01:09.615 INFO:tasks.workunit.client.0.smithi032.stdout:1/160: fsync d1/d25/f34 0 2024-05-12T04:01:09.615 INFO:tasks.workunit.client.1.smithi033.stdout:1/317: symlink d11/d1c/d59/d68/l6c 0 2024-05-12T04:01:09.617 INFO:tasks.workunit.client.1.smithi033.stdout:2/373: creat dc/d17/d27/d2f/d49/f85 x:0 0 0 2024-05-12T04:01:09.619 INFO:tasks.workunit.client.0.smithi032.stdout:8/206: rename d0/d9/d13/d21/d35/c3f to d0/d9/d13/c41 0 2024-05-12T04:01:09.621 INFO:tasks.workunit.client.0.smithi032.stdout:6/190: unlink d7/d9/f15 0 2024-05-12T04:01:09.621 INFO:tasks.workunit.client.1.smithi033.stdout:1/318: dwrite d11/d1c/d43/f45 [0,4194304] 0 2024-05-12T04:01:09.622 INFO:tasks.workunit.client.0.smithi032.stdout:6/191: write d7/de/d12/d25/f29 [403380,41882] 0 2024-05-12T04:01:09.624 INFO:tasks.workunit.client.0.smithi032.stdout:6/192: chown d7/de/f14 1 1 2024-05-12T04:01:09.625 INFO:tasks.workunit.client.1.smithi033.stdout:7/519: rename de/d20/db0 to de/df/d9a/db7 0 2024-05-12T04:01:09.627 INFO:tasks.workunit.client.1.smithi033.stdout:1/319: dwrite d11/d17/f2e [0,4194304] 0 2024-05-12T04:01:09.627 INFO:tasks.workunit.client.1.smithi033.stdout:7/520: chown de/l32 562 1 2024-05-12T04:01:09.628 INFO:tasks.workunit.client.1.smithi033.stdout:1/320: stat d11/c2f 0 2024-05-12T04:01:09.629 INFO:tasks.workunit.client.1.smithi033.stdout:8/285: rmdir d1/d4/d1d/d36/d3c 39 2024-05-12T04:01:09.630 INFO:tasks.workunit.client.0.smithi032.stdout:7/188: link dd/d19/c26 dd/d19/d37/c44 0 2024-05-12T04:01:09.635 INFO:tasks.workunit.client.0.smithi032.stdout:6/193: write d7/f2b [3128191,53962] 0 2024-05-12T04:01:09.635 INFO:tasks.workunit.client.1.smithi033.stdout:6/364: dwrite d4/d5/f41 [0,4194304] 0 2024-05-12T04:01:09.636 INFO:tasks.workunit.client.0.smithi032.stdout:6/194: dread - d7/de/d12/f41 zero size 2024-05-12T04:01:09.636 INFO:tasks.workunit.client.1.smithi033.stdout:4/364: mknod dd/c73 0 2024-05-12T04:01:09.638 INFO:tasks.workunit.client.0.smithi032.stdout:6/195: fdatasync d7/de/d12/d1a/d20/f3d 0 2024-05-12T04:01:09.638 INFO:tasks.workunit.client.1.smithi033.stdout:4/365: readlink dd/d11/d31/d38/l52 0 2024-05-12T04:01:09.638 INFO:tasks.workunit.client.1.smithi033.stdout:7/521: dwrite de/df/d7a/f8b [0,4194304] 0 2024-05-12T04:01:09.641 INFO:tasks.workunit.client.1.smithi033.stdout:2/374: unlink dc/d17/d27/d47/d5e/l61 0 2024-05-12T04:01:09.641 INFO:tasks.workunit.client.1.smithi033.stdout:7/522: read de/d20/f21 [685788,77934] 0 2024-05-12T04:01:09.642 INFO:tasks.workunit.client.0.smithi032.stdout:3/242: link d15/d33/f38 d15/d20/d26/f52 0 2024-05-12T04:01:09.643 INFO:tasks.workunit.client.1.smithi033.stdout:7/523: read de/d2f/d70/f74 [318814,54440] 0 2024-05-12T04:01:09.643 INFO:tasks.workunit.client.1.smithi033.stdout:7/524: stat de 0 2024-05-12T04:01:09.644 INFO:tasks.workunit.client.1.smithi033.stdout:5/293: mkdir d3/d5c 0 2024-05-12T04:01:09.645 INFO:tasks.workunit.client.0.smithi032.stdout:7/189: mkdir dd/d1b/d1e/d45 0 2024-05-12T04:01:09.646 INFO:tasks.workunit.client.0.smithi032.stdout:7/190: chown f9 1604792 1 2024-05-12T04:01:09.652 INFO:tasks.workunit.client.0.smithi032.stdout:1/161: creat d1/f36 x:0 0 0 2024-05-12T04:01:09.653 INFO:tasks.workunit.client.0.smithi032.stdout:1/162: chown d1/d7 249 1 2024-05-12T04:01:09.654 INFO:tasks.workunit.client.0.smithi032.stdout:6/196: rename d7/l10 to d7/de/d12/d1a/d20/l43 0 2024-05-12T04:01:09.655 INFO:tasks.workunit.client.1.smithi033.stdout:1/321: write d11/d1c/f64 [1622212,14725] 0 2024-05-12T04:01:09.656 INFO:tasks.workunit.client.0.smithi032.stdout:6/197: write d7/de/d12/f18 [861163,26169] 0 2024-05-12T04:01:09.658 INFO:tasks.workunit.client.1.smithi033.stdout:0/407: creat d6/dc/d58/d6a/d80/f84 x:0 0 0 2024-05-12T04:01:09.660 INFO:tasks.workunit.client.1.smithi033.stdout:8/286: symlink d1/d16/d27/d4d/d51/l5a 0 2024-05-12T04:01:09.661 INFO:tasks.workunit.client.0.smithi032.stdout:6/198: read - d7/de/d12/d1a/d20/f34 zero size 2024-05-12T04:01:09.662 INFO:tasks.workunit.client.0.smithi032.stdout:0/255: dwrite df/d10/f13 [8388608,4194304] 0 2024-05-12T04:01:09.662 INFO:tasks.workunit.client.0.smithi032.stdout:1/163: stat d1/dd/c33 0 2024-05-12T04:01:09.664 INFO:tasks.workunit.client.0.smithi032.stdout:6/199: dread - d7/de/d12/d1a/d20/f42 zero size 2024-05-12T04:01:09.666 INFO:tasks.workunit.client.0.smithi032.stdout:8/207: getdents d0/d7 0 2024-05-12T04:01:09.666 INFO:tasks.workunit.client.0.smithi032.stdout:8/208: chown d0/d9/fe 118 1 2024-05-12T04:01:09.671 INFO:tasks.workunit.client.0.smithi032.stdout:0/256: creat df/d10/d3d/f5c x:0 0 0 2024-05-12T04:01:09.672 INFO:tasks.workunit.client.0.smithi032.stdout:8/209: dread d0/d9/d13/d21/f2a [0,4194304] 0 2024-05-12T04:01:09.674 INFO:tasks.workunit.client.0.smithi032.stdout:1/164: symlink d1/d7/d30/d18/l37 0 2024-05-12T04:01:09.676 INFO:tasks.workunit.client.0.smithi032.stdout:3/243: dread d15/d20/d22/f47 [0,4194304] 0 2024-05-12T04:01:09.677 INFO:tasks.workunit.client.1.smithi033.stdout:6/365: rename d4/d43/f62 to d4/d1a/d67/d68/f7b 0 2024-05-12T04:01:09.681 INFO:tasks.workunit.client.1.smithi033.stdout:2/375: chown dc/d17/d3a/c7a 68207 1 2024-05-12T04:01:09.683 INFO:tasks.workunit.client.0.smithi032.stdout:0/257: dwrite df/f2b [0,4194304] 0 2024-05-12T04:01:09.684 INFO:tasks.workunit.client.1.smithi033.stdout:6/366: dwrite d4/d5/f27 [4194304,4194304] 0 2024-05-12T04:01:09.685 INFO:tasks.workunit.client.1.smithi033.stdout:6/367: read - d4/d1a/d3c/f75 zero size 2024-05-12T04:01:09.686 INFO:tasks.workunit.client.0.smithi032.stdout:8/210: dwrite d0/d9/fe [0,4194304] 0 2024-05-12T04:01:09.688 INFO:tasks.workunit.client.1.smithi033.stdout:7/525: symlink de/df/d60/lb8 0 2024-05-12T04:01:09.692 INFO:tasks.workunit.client.0.smithi032.stdout:7/191: dwrite dd/d1b/f2d [0,4194304] 0 2024-05-12T04:01:09.698 INFO:tasks.workunit.client.1.smithi033.stdout:9/578: getdents d2/d4/d2b/d6d/d7a 0 2024-05-12T04:01:09.699 INFO:tasks.workunit.client.0.smithi032.stdout:7/192: write dd/d19/d37/f39 [53601,129947] 0 2024-05-12T04:01:09.701 INFO:tasks.workunit.client.0.smithi032.stdout:1/165: creat d1/d7/d30/d35/f38 x:0 0 0 2024-05-12T04:01:09.701 INFO:tasks.workunit.client.0.smithi032.stdout:8/211: rmdir d0/d9/d13/d26/d28 39 2024-05-12T04:01:09.706 INFO:tasks.workunit.client.0.smithi032.stdout:0/258: mknod df/d22/d58/c5d 0 2024-05-12T04:01:09.707 INFO:tasks.workunit.client.0.smithi032.stdout:8/212: dread d0/d9/d1b/d33/f3a [0,4194304] 0 2024-05-12T04:01:09.711 INFO:tasks.workunit.client.0.smithi032.stdout:1/166: dwrite d1/d7/d30/d18/f1b [0,4194304] 0 2024-05-12T04:01:09.712 INFO:tasks.workunit.client.0.smithi032.stdout:7/193: link dd/d1b/d1e/f20 dd/d19/d37/f46 0 2024-05-12T04:01:09.712 INFO:tasks.workunit.client.0.smithi032.stdout:1/167: chown d1/c1c 176046 1 2024-05-12T04:01:09.713 INFO:tasks.workunit.client.1.smithi033.stdout:5/294: readlink d3/d12/d29/l50 0 2024-05-12T04:01:09.713 INFO:tasks.workunit.client.0.smithi032.stdout:7/194: chown dd/l17 279 1 2024-05-12T04:01:09.713 INFO:tasks.workunit.client.0.smithi032.stdout:7/195: fdatasync dd/f42 0 2024-05-12T04:01:09.713 INFO:tasks.workunit.client.0.smithi032.stdout:1/168: write d1/d7/d30/d18/f2b [324065,118300] 0 2024-05-12T04:01:09.714 INFO:tasks.workunit.client.1.smithi033.stdout:7/526: truncate de/df/d17/d1e/d24/f6f 817709 0 2024-05-12T04:01:09.718 INFO:tasks.workunit.client.0.smithi032.stdout:1/169: mknod d1/d7/d30/d35/c39 0 2024-05-12T04:01:09.718 INFO:tasks.workunit.client.0.smithi032.stdout:8/213: write d0/d9/d13/d21/f2a [3513684,87692] 0 2024-05-12T04:01:09.720 INFO:tasks.workunit.client.1.smithi033.stdout:9/579: write d2/d4/f2c [2762140,64722] 0 2024-05-12T04:01:09.720 INFO:tasks.workunit.client.1.smithi033.stdout:9/580: fsync d2/dd/f13 0 2024-05-12T04:01:09.721 INFO:tasks.workunit.client.0.smithi032.stdout:1/170: read - d1/d25/f34 zero size 2024-05-12T04:01:09.722 INFO:tasks.workunit.client.0.smithi032.stdout:0/259: rename df/d30/d36/f43 to df/d30/d32/f5e 0 2024-05-12T04:01:09.723 INFO:tasks.workunit.client.0.smithi032.stdout:7/196: dwrite dd/f13 [0,4194304] 0 2024-05-12T04:01:09.724 INFO:tasks.workunit.client.1.smithi033.stdout:4/366: rename dd/l1f to dd/l74 0 2024-05-12T04:01:09.724 INFO:tasks.workunit.client.0.smithi032.stdout:8/214: write d0/d9/d13/d21/d35/f3c [410932,74271] 0 2024-05-12T04:01:09.728 INFO:tasks.workunit.client.0.smithi032.stdout:8/215: dread d0/d9/d13/f29 [0,4194304] 0 2024-05-12T04:01:09.732 INFO:tasks.workunit.client.0.smithi032.stdout:1/171: dread d1/d7/f11 [0,4194304] 0 2024-05-12T04:01:09.733 INFO:tasks.workunit.client.1.smithi033.stdout:3/359: sync 2024-05-12T04:01:09.734 INFO:tasks.workunit.client.1.smithi033.stdout:5/295: fdatasync d3/d15/d16/f20 0 2024-05-12T04:01:09.736 INFO:tasks.workunit.client.1.smithi033.stdout:8/287: dwrite d1/d4/fd [4194304,4194304] 0 2024-05-12T04:01:09.737 INFO:tasks.workunit.client.1.smithi033.stdout:1/322: creat d11/d17/f6d x:0 0 0 2024-05-12T04:01:09.737 INFO:tasks.workunit.client.1.smithi033.stdout:5/296: truncate d3/d12/d29/f3d 367972 0 2024-05-12T04:01:09.737 INFO:tasks.workunit.client.1.smithi033.stdout:8/288: dread - d1/d4/f39 zero size 2024-05-12T04:01:09.739 INFO:tasks.workunit.client.1.smithi033.stdout:8/289: chown d1/d4/cc 308 1 2024-05-12T04:01:09.741 INFO:tasks.workunit.client.0.smithi032.stdout:8/216: creat d0/d9/d13/d21/d25/f42 x:0 0 0 2024-05-12T04:01:09.746 INFO:tasks.workunit.client.0.smithi032.stdout:1/172: symlink d1/dd/l3a 0 2024-05-12T04:01:09.746 INFO:tasks.workunit.client.1.smithi033.stdout:2/376: link dc/d17/d27/d2f/d49/d59/c7e dc/d13/d1f/d6e/c86 0 2024-05-12T04:01:09.748 INFO:tasks.workunit.client.0.smithi032.stdout:8/217: symlink d0/d9/d13/d21/l43 0 2024-05-12T04:01:09.749 INFO:tasks.workunit.client.0.smithi032.stdout:1/173: dread d1/d7/f11 [0,4194304] 0 2024-05-12T04:01:09.750 INFO:tasks.workunit.client.0.smithi032.stdout:7/197: link dd/l17 dd/d1b/l47 0 2024-05-12T04:01:09.750 INFO:tasks.workunit.client.0.smithi032.stdout:1/174: dread - d1/d7/d30/d35/f38 zero size 2024-05-12T04:01:09.751 INFO:tasks.workunit.client.1.smithi033.stdout:2/377: read dc/f31 [1330979,116656] 0 2024-05-12T04:01:09.751 INFO:tasks.workunit.client.1.smithi033.stdout:2/378: chown c0 12 1 2024-05-12T04:01:09.753 INFO:tasks.workunit.client.1.smithi033.stdout:2/379: truncate dc/d17/d27/d2f/d49/f51 1039477 0 2024-05-12T04:01:09.754 INFO:tasks.workunit.client.1.smithi033.stdout:2/380: dread - dc/d17/d27/d2f/f75 zero size 2024-05-12T04:01:09.755 INFO:tasks.workunit.client.1.smithi033.stdout:6/368: truncate d4/f56 2925067 0 2024-05-12T04:01:09.755 INFO:tasks.workunit.client.1.smithi033.stdout:9/581: dread d2/d4/d2b/d6d/f5c [0,4194304] 0 2024-05-12T04:01:09.756 INFO:tasks.workunit.client.1.smithi033.stdout:9/582: read d2/dd/d39/f43 [150645,97471] 0 2024-05-12T04:01:09.757 INFO:tasks.workunit.client.1.smithi033.stdout:9/583: chown d2/d4/d2b/d78/lb2 35346 1 2024-05-12T04:01:09.757 INFO:tasks.workunit.client.1.smithi033.stdout:1/323: fdatasync d11/d17/d27/d36/d3f/f50 0 2024-05-12T04:01:09.759 INFO:tasks.workunit.client.1.smithi033.stdout:1/324: write d11/d1c/f32 [733921,119606] 0 2024-05-12T04:01:09.759 INFO:tasks.workunit.client.1.smithi033.stdout:9/584: write d2/d4/d2b/d6d/d3a/f65 [1505160,101518] 0 2024-05-12T04:01:09.761 INFO:tasks.workunit.client.1.smithi033.stdout:0/408: link d6/f23 d6/dc/d58/f85 0 2024-05-12T04:01:09.761 INFO:tasks.workunit.client.1.smithi033.stdout:9/585: chown d2/d4/d29/f2a 1571 1 2024-05-12T04:01:09.761 INFO:tasks.workunit.client.0.smithi032.stdout:7/198: mknod dd/d1b/d1e/d45/c48 0 2024-05-12T04:01:09.762 INFO:tasks.workunit.client.1.smithi033.stdout:5/297: mknod d3/d12/d25/d26/d2e/d54/c5d 0 2024-05-12T04:01:09.763 INFO:tasks.workunit.client.1.smithi033.stdout:5/298: chown d3/d12/d29 11 1 2024-05-12T04:01:09.765 INFO:tasks.workunit.client.0.smithi032.stdout:7/199: creat dd/d1b/f49 x:0 0 0 2024-05-12T04:01:09.766 INFO:tasks.workunit.client.1.smithi033.stdout:9/586: dread d2/d4/d29/d32/db7/fbf [0,4194304] 0 2024-05-12T04:01:09.766 INFO:tasks.workunit.client.1.smithi033.stdout:4/367: mkdir dd/d11/d75 0 2024-05-12T04:01:09.766 INFO:tasks.workunit.client.0.smithi032.stdout:8/218: write d0/d9/d13/f29 [440993,34124] 0 2024-05-12T04:01:09.769 INFO:tasks.workunit.client.0.smithi032.stdout:7/200: symlink dd/d19/d37/l4a 0 2024-05-12T04:01:09.771 INFO:tasks.workunit.client.0.smithi032.stdout:1/175: dread d1/f3 [0,4194304] 0 2024-05-12T04:01:09.772 INFO:tasks.workunit.client.1.smithi033.stdout:7/527: creat de/fb9 x:0 0 0 2024-05-12T04:01:09.773 INFO:tasks.workunit.client.1.smithi033.stdout:2/381: readlink dc/d13/d39/l3e 0 2024-05-12T04:01:09.775 INFO:tasks.workunit.client.0.smithi032.stdout:7/201: unlink dd/d19/d37/f3a 0 2024-05-12T04:01:09.776 INFO:tasks.workunit.client.0.smithi032.stdout:8/219: dwrite d0/d9/d13/f24 [0,4194304] 0 2024-05-12T04:01:09.777 INFO:tasks.workunit.client.1.smithi033.stdout:3/360: creat db/d17/d3a/d67/d76/f78 x:0 0 0 2024-05-12T04:01:09.778 INFO:tasks.workunit.client.0.smithi032.stdout:1/176: rmdir d1/dd/d24 39 2024-05-12T04:01:09.779 INFO:tasks.workunit.client.0.smithi032.stdout:1/177: write d1/f36 [262296,30110] 0 2024-05-12T04:01:09.779 INFO:tasks.workunit.client.0.smithi032.stdout:8/220: dread d0/d7/f11 [0,4194304] 0 2024-05-12T04:01:09.789 INFO:tasks.workunit.client.1.smithi033.stdout:8/290: symlink d1/d16/d27/d28/l5b 0 2024-05-12T04:01:09.793 INFO:tasks.workunit.client.0.smithi032.stdout:8/221: dwrite d0/d7/f15 [0,4194304] 0 2024-05-12T04:01:09.793 INFO:tasks.workunit.client.1.smithi033.stdout:5/299: mknod d3/d12/d29/d2b/c5e 0 2024-05-12T04:01:09.794 INFO:tasks.workunit.client.1.smithi033.stdout:5/300: truncate d3/d12/d29/f3d 636754 0 2024-05-12T04:01:09.794 INFO:tasks.workunit.client.1.smithi033.stdout:5/301: fsync d3/d15/f30 0 2024-05-12T04:01:09.795 INFO:tasks.workunit.client.1.smithi033.stdout:5/302: stat d3/d12/d25/f4a 0 2024-05-12T04:01:09.796 INFO:tasks.workunit.client.1.smithi033.stdout:4/368: write dd/d11/d31/d66/f49 [709479,121465] 0 2024-05-12T04:01:09.800 INFO:tasks.workunit.client.1.smithi033.stdout:9/587: creat d2/d4/d2b/d6d/d2f/d5f/fd5 x:0 0 0 2024-05-12T04:01:09.801 INFO:tasks.workunit.client.0.smithi032.stdout:1/178: truncate d1/d7/d30/d18/f1a 3866261 0 2024-05-12T04:01:09.802 INFO:tasks.workunit.client.1.smithi033.stdout:0/409: dwrite d6/dc/f56 [0,4194304] 0 2024-05-12T04:01:09.803 INFO:tasks.workunit.client.0.smithi032.stdout:7/202: write dd/d1b/d1e/f28 [5147179,7243] 0 2024-05-12T04:01:09.809 INFO:tasks.workunit.client.1.smithi033.stdout:0/410: dwrite d6/f6b [0,4194304] 0 2024-05-12T04:01:09.812 INFO:tasks.workunit.client.0.smithi032.stdout:7/203: dread dd/d19/d37/f46 [0,4194304] 0 2024-05-12T04:01:09.814 INFO:tasks.workunit.client.0.smithi032.stdout:7/204: truncate dd/f22 719480 0 2024-05-12T04:01:09.815 INFO:tasks.workunit.client.1.smithi033.stdout:1/325: link d11/d1c/d43/f45 d11/d1c/d59/d68/f6e 0 2024-05-12T04:01:09.815 INFO:tasks.workunit.client.0.smithi032.stdout:7/205: write dd/f42 [250469,72411] 0 2024-05-12T04:01:09.817 INFO:tasks.workunit.client.1.smithi033.stdout:1/326: chown d11/d17/d27/d36/d46/f66 383660 1 2024-05-12T04:01:09.818 INFO:tasks.workunit.client.1.smithi033.stdout:1/327: readlink d11/l1a 0 2024-05-12T04:01:09.819 INFO:tasks.workunit.client.1.smithi033.stdout:8/291: symlink d1/d16/d29/l5c 0 2024-05-12T04:01:09.820 INFO:tasks.workunit.client.1.smithi033.stdout:1/328: fsync d11/d17/d27/f2b 0 2024-05-12T04:01:09.821 INFO:tasks.workunit.client.1.smithi033.stdout:7/528: dwrite de/df/d17/d1b/d46/f53 [0,4194304] 0 2024-05-12T04:01:09.824 INFO:tasks.workunit.client.1.smithi033.stdout:1/329: write d11/d17/d27/d36/d46/f51 [1748960,96714] 0 2024-05-12T04:01:09.826 INFO:tasks.workunit.client.0.smithi032.stdout:7/206: creat dd/d1b/f4b x:0 0 0 2024-05-12T04:01:09.827 INFO:tasks.workunit.client.0.smithi032.stdout:1/179: link d1/d7/c32 d1/d7/c3b 0 2024-05-12T04:01:09.828 INFO:tasks.workunit.client.0.smithi032.stdout:8/222: dwrite d0/d9/d13/d26/d28/f2b [0,4194304] 0 2024-05-12T04:01:09.829 INFO:tasks.workunit.client.0.smithi032.stdout:7/207: write dd/f42 [907438,130465] 0 2024-05-12T04:01:09.832 INFO:tasks.workunit.client.1.smithi033.stdout:7/529: dwrite de/d20/da9/faf [0,4194304] 0 2024-05-12T04:01:09.832 INFO:tasks.workunit.client.1.smithi033.stdout:1/330: truncate d11/d17/d27/d36/f41 1396480 0 2024-05-12T04:01:09.833 INFO:tasks.workunit.client.0.smithi032.stdout:1/180: write d1/d7/d30/d18/f1b [2622649,70561] 0 2024-05-12T04:01:09.833 INFO:tasks.workunit.client.1.smithi033.stdout:9/588: creat d2/d4/d2b/d49/d73/fd6 x:0 0 0 2024-05-12T04:01:09.839 INFO:tasks.workunit.client.1.smithi033.stdout:2/382: fsync f7 0 2024-05-12T04:01:09.839 INFO:tasks.workunit.client.1.smithi033.stdout:7/530: truncate de/df/d17/d1e/d24/fad 774171 0 2024-05-12T04:01:09.839 INFO:tasks.workunit.client.0.smithi032.stdout:7/208: dread dd/f12 [0,4194304] 0 2024-05-12T04:01:09.841 INFO:tasks.workunit.client.1.smithi033.stdout:0/411: unlink d6/d11/f28 0 2024-05-12T04:01:09.842 INFO:tasks.workunit.client.0.smithi032.stdout:8/223: dwrite d0/d9/d13/d21/d25/f42 [0,4194304] 0 2024-05-12T04:01:09.843 INFO:tasks.workunit.client.1.smithi033.stdout:0/412: read d6/db/f19 [3468175,32273] 0 2024-05-12T04:01:09.846 INFO:tasks.workunit.client.0.smithi032.stdout:7/209: fdatasync dd/f12 0 2024-05-12T04:01:09.850 INFO:tasks.workunit.client.0.smithi032.stdout:1/181: link d1/d7/c12 d1/d7/d30/d35/c3c 0 2024-05-12T04:01:09.852 INFO:tasks.workunit.client.1.smithi033.stdout:4/369: mknod dd/c76 0 2024-05-12T04:01:09.855 INFO:tasks.workunit.client.0.smithi032.stdout:7/210: rename dd/f25 to dd/d19/f4c 0 2024-05-12T04:01:09.856 INFO:tasks.workunit.client.1.smithi033.stdout:1/331: creat d11/d1c/d43/f6f x:0 0 0 2024-05-12T04:01:09.858 INFO:tasks.workunit.client.0.smithi032.stdout:8/224: dwrite d0/d9/d1b/d33/f3a [0,4194304] 0 2024-05-12T04:01:09.860 INFO:tasks.workunit.client.1.smithi033.stdout:7/531: fsync de/df/d17/f88 0 2024-05-12T04:01:09.861 INFO:tasks.workunit.client.0.smithi032.stdout:7/211: readlink dd/l17 0 2024-05-12T04:01:09.863 INFO:tasks.workunit.client.0.smithi032.stdout:1/182: symlink d1/dd/d24/l3d 0 2024-05-12T04:01:09.865 INFO:tasks.workunit.client.1.smithi033.stdout:6/369: sync 2024-05-12T04:01:09.865 INFO:tasks.workunit.client.1.smithi033.stdout:2/383: chown dc/d17/d27/d2f/d49/d59/c7e 306186 1 2024-05-12T04:01:09.866 INFO:tasks.workunit.client.1.smithi033.stdout:6/370: read - d4/d1a/d4e/d4f/f5e zero size 2024-05-12T04:01:09.868 INFO:tasks.workunit.client.0.smithi032.stdout:7/212: read f6 [37310,86942] 0 2024-05-12T04:01:09.870 INFO:tasks.workunit.client.0.smithi032.stdout:8/225: dwrite d0/d9/d1b/d33/f3a [0,4194304] 0 2024-05-12T04:01:09.872 INFO:tasks.workunit.client.0.smithi032.stdout:1/183: mknod d1/d7/d30/d18/c3e 0 2024-05-12T04:01:09.875 INFO:tasks.workunit.client.1.smithi033.stdout:8/292: rename d1/d4/d1d/d36/d3c/d47/c49 to d1/c5d 0 2024-05-12T04:01:09.882 INFO:tasks.workunit.client.0.smithi032.stdout:7/213: creat dd/d19/f4d x:0 0 0 2024-05-12T04:01:09.885 INFO:tasks.workunit.client.1.smithi033.stdout:3/361: sync 2024-05-12T04:01:09.886 INFO:tasks.workunit.client.0.smithi032.stdout:8/226: unlink d0/d9/d13/l18 0 2024-05-12T04:01:09.887 INFO:tasks.workunit.client.1.smithi033.stdout:1/332: mknod d11/d17/d38/c70 0 2024-05-12T04:01:09.892 INFO:tasks.workunit.client.1.smithi033.stdout:5/303: dwrite d3/f7 [0,4194304] 0 2024-05-12T04:01:09.893 INFO:tasks.workunit.client.1.smithi033.stdout:6/371: unlink d4/f30 0 2024-05-12T04:01:09.899 INFO:tasks.workunit.client.1.smithi033.stdout:6/372: dwrite d4/d43/f70 [0,4194304] 0 2024-05-12T04:01:09.904 INFO:tasks.workunit.client.0.smithi032.stdout:7/214: link l4 dd/d1b/d1e/l4e 0 2024-05-12T04:01:09.907 INFO:tasks.workunit.client.1.smithi033.stdout:3/362: truncate db/d17/d3e/f48 1005889 0 2024-05-12T04:01:09.908 INFO:tasks.workunit.client.1.smithi033.stdout:9/589: creat d2/d4/d2b/fd7 x:0 0 0 2024-05-12T04:01:09.908 INFO:tasks.workunit.client.0.smithi032.stdout:1/184: truncate d1/f36 52136 0 2024-05-12T04:01:09.910 INFO:tasks.workunit.client.1.smithi033.stdout:9/590: read d2/d4/d29/d3c/f9a [3975816,24407] 0 2024-05-12T04:01:09.910 INFO:tasks.workunit.client.0.smithi032.stdout:7/215: rename f9 to dd/d19/d37/f4f 0 2024-05-12T04:01:09.916 INFO:tasks.workunit.client.0.smithi032.stdout:8/227: truncate d0/f2f 115821 0 2024-05-12T04:01:09.917 INFO:tasks.workunit.client.0.smithi032.stdout:1/185: stat d1/d7/d30/d35/c3c 0 2024-05-12T04:01:09.918 INFO:tasks.workunit.client.0.smithi032.stdout:7/216: truncate dd/d19/d37/f46 1554578 0 2024-05-12T04:01:09.922 INFO:tasks.workunit.client.1.smithi033.stdout:5/304: readlink d3/l4d 0 2024-05-12T04:01:09.923 INFO:tasks.workunit.client.0.smithi032.stdout:1/186: mknod d1/d7/c3f 0 2024-05-12T04:01:09.929 INFO:tasks.workunit.client.0.smithi032.stdout:8/228: symlink d0/l44 0 2024-05-12T04:01:09.930 INFO:tasks.workunit.client.0.smithi032.stdout:8/229: write d0/d9/f12 [1862864,4714] 0 2024-05-12T04:01:09.934 INFO:tasks.workunit.client.1.smithi033.stdout:0/413: link d6/dc/d58/f78 d6/dc/d58/f86 0 2024-05-12T04:01:09.934 INFO:tasks.workunit.client.1.smithi033.stdout:7/532: write f3 [949814,25330] 0 2024-05-12T04:01:09.935 INFO:tasks.workunit.client.1.smithi033.stdout:7/533: chown de/d20/d64/c66 1 1 2024-05-12T04:01:09.942 INFO:tasks.workunit.client.1.smithi033.stdout:3/363: symlink db/d17/d3a/l79 0 2024-05-12T04:01:09.946 INFO:tasks.workunit.client.1.smithi033.stdout:9/591: creat d2/d4/d2b/d6d/d2f/fd8 x:0 0 0 2024-05-12T04:01:09.947 INFO:tasks.workunit.client.1.smithi033.stdout:9/592: chown d2/d4/d2b/d6d/d3a/dca 110936 1 2024-05-12T04:01:09.947 INFO:tasks.workunit.client.1.smithi033.stdout:0/414: dread f4 [0,4194304] 0 2024-05-12T04:01:09.948 INFO:tasks.workunit.client.1.smithi033.stdout:1/333: link d11/d17/d27/d36/f41 d11/d17/d38/f71 0 2024-05-12T04:01:09.948 INFO:tasks.workunit.client.0.smithi032.stdout:9/134: sync 2024-05-12T04:01:09.948 INFO:tasks.workunit.client.0.smithi032.stdout:5/231: sync 2024-05-12T04:01:09.948 INFO:tasks.workunit.client.0.smithi032.stdout:2/142: sync 2024-05-12T04:01:09.949 INFO:tasks.workunit.client.0.smithi032.stdout:8/230: getdents d0/d9/d1b/d27 0 2024-05-12T04:01:09.949 INFO:tasks.workunit.client.0.smithi032.stdout:8/231: fsync d0/d9/fe 0 2024-05-12T04:01:09.951 INFO:tasks.workunit.client.0.smithi032.stdout:7/217: truncate fa 3514309 0 2024-05-12T04:01:09.952 INFO:tasks.workunit.client.0.smithi032.stdout:7/218: write dd/d19/f1c [579960,27331] 0 2024-05-12T04:01:09.952 INFO:tasks.workunit.client.0.smithi032.stdout:7/219: chown dd/l2e 69880 1 2024-05-12T04:01:09.954 INFO:tasks.workunit.client.1.smithi033.stdout:8/293: write d1/d16/d20/f33 [1951698,44533] 0 2024-05-12T04:01:09.955 INFO:tasks.workunit.client.0.smithi032.stdout:9/135: readlink d0/lb 0 2024-05-12T04:01:09.956 INFO:tasks.workunit.client.0.smithi032.stdout:2/143: mknod d1/d34/c36 0 2024-05-12T04:01:09.958 INFO:tasks.workunit.client.0.smithi032.stdout:5/232: symlink d4/d7/d14/d26/d3c/l50 0 2024-05-12T04:01:09.960 INFO:tasks.workunit.client.1.smithi033.stdout:7/534: creat de/d2f/fba x:0 0 0 2024-05-12T04:01:09.962 INFO:tasks.workunit.client.0.smithi032.stdout:1/187: getdents d1/d7/d30/d18 0 2024-05-12T04:01:09.963 INFO:tasks.workunit.client.1.smithi033.stdout:4/370: link dd/d11/d31/d66/c41 dd/d11/d5b/c77 0 2024-05-12T04:01:09.969 INFO:tasks.workunit.client.1.smithi033.stdout:4/371: dwrite dd/d11/d31/d66/d42/f63 [0,4194304] 0 2024-05-12T04:01:09.971 INFO:tasks.workunit.client.0.smithi032.stdout:9/136: symlink d0/d3/l2d 0 2024-05-12T04:01:09.972 INFO:tasks.workunit.client.1.smithi033.stdout:9/593: symlink d2/d70/d7b/ld9 0 2024-05-12T04:01:09.972 INFO:tasks.workunit.client.0.smithi032.stdout:7/220: dwrite dd/d19/f4c [0,4194304] 0 2024-05-12T04:01:09.974 INFO:tasks.workunit.client.0.smithi032.stdout:9/137: write d0/d3/d6/d7/da/f1a [2625445,102543] 0 2024-05-12T04:01:09.974 INFO:tasks.workunit.client.0.smithi032.stdout:2/144: fsync d1/d34/f32 0 2024-05-12T04:01:09.976 INFO:tasks.workunit.client.1.smithi033.stdout:0/415: creat d6/db/d2c/d73/f87 x:0 0 0 2024-05-12T04:01:09.976 INFO:tasks.workunit.client.0.smithi032.stdout:2/145: fsync d1/d13/f2f 0 2024-05-12T04:01:09.978 INFO:tasks.workunit.client.0.smithi032.stdout:2/146: truncate d1/d13/d25/f28 250224 0 2024-05-12T04:01:09.978 INFO:tasks.workunit.client.0.smithi032.stdout:5/233: rename d4/d7/d14/d39/f13 to d4/d7/d14/d26/d33/d4a/f51 0 2024-05-12T04:01:09.983 INFO:tasks.workunit.client.1.smithi033.stdout:1/334: rename d11/d17/d38/f5f to d11/d1c/d59/d68/f72 0 2024-05-12T04:01:09.988 INFO:tasks.workunit.client.0.smithi032.stdout:5/234: dwrite d4/f6 [0,4194304] 0 2024-05-12T04:01:09.988 INFO:tasks.workunit.client.0.smithi032.stdout:1/188: mknod d1/d7/c40 0 2024-05-12T04:01:09.993 INFO:tasks.workunit.client.0.smithi032.stdout:5/235: chown d4/d7/d14/d26/d33/d4a/f4e 156053 1 2024-05-12T04:01:09.994 INFO:tasks.workunit.client.1.smithi033.stdout:5/305: creat d3/d12/d25/d26/d2e/d54/d57/f5f x:0 0 0 2024-05-12T04:01:09.994 INFO:tasks.workunit.client.0.smithi032.stdout:5/236: write d4/d7/d14/d26/d33/d4a/f4f [566183,90932] 0 2024-05-12T04:01:09.997 INFO:tasks.workunit.client.1.smithi033.stdout:2/384: getdents dc/d13/d1f/d6e 0 2024-05-12T04:01:09.999 INFO:tasks.workunit.client.0.smithi032.stdout:7/221: mknod dd/d1b/d1e/c50 0 2024-05-12T04:01:10.002 INFO:tasks.workunit.client.1.smithi033.stdout:8/294: creat d1/d16/d27/d4d/f5e x:0 0 0 2024-05-12T04:01:10.006 INFO:tasks.workunit.client.0.smithi032.stdout:9/138: read d0/d3/d6/d7/da/d17/f20 [409692,61367] 0 2024-05-12T04:01:10.007 INFO:tasks.workunit.client.0.smithi032.stdout:9/139: readlink d0/d3/l2d 0 2024-05-12T04:01:10.009 INFO:tasks.workunit.client.1.smithi033.stdout:8/295: dwrite d1/d4/d1d/f26 [0,4194304] 0 2024-05-12T04:01:10.018 INFO:tasks.workunit.client.1.smithi033.stdout:4/372: creat dd/d11/d31/d38/f78 x:0 0 0 2024-05-12T04:01:10.018 INFO:tasks.workunit.client.1.smithi033.stdout:4/373: write dd/f32 [13368,11768] 0 2024-05-12T04:01:10.019 INFO:tasks.workunit.client.1.smithi033.stdout:4/374: chown dd/d11/d31/f45 616412 1 2024-05-12T04:01:10.020 INFO:tasks.workunit.client.0.smithi032.stdout:5/237: mknod d4/d7/d14/d26/d33/d4a/d3d/c52 0 2024-05-12T04:01:10.020 INFO:tasks.workunit.client.0.smithi032.stdout:5/238: stat d4/d7/d14/d26/d47 0 2024-05-12T04:01:10.023 INFO:tasks.workunit.client.0.smithi032.stdout:2/147: creat d1/d13/d25/d2c/f37 x:0 0 0 2024-05-12T04:01:10.025 INFO:tasks.workunit.client.1.smithi033.stdout:0/416: rmdir d6/db/d2c/d73 39 2024-05-12T04:01:10.028 INFO:tasks.workunit.client.0.smithi032.stdout:1/189: truncate d1/f36 935237 0 2024-05-12T04:01:10.028 INFO:tasks.workunit.client.0.smithi032.stdout:5/239: dwrite d4/d7/d14/d39/d1b/f44 [0,4194304] 0 2024-05-12T04:01:10.029 INFO:tasks.workunit.client.1.smithi033.stdout:5/306: write d3/d12/d25/d26/f31 [4933197,95803] 0 2024-05-12T04:01:10.029 INFO:tasks.workunit.client.1.smithi033.stdout:0/417: dread d6/d11/d57/f63 [0,4194304] 0 2024-05-12T04:01:10.031 INFO:tasks.workunit.client.1.smithi033.stdout:5/307: rename d3/d15/d16 to d3/d15/d16/d45/d4b/d60 22 2024-05-12T04:01:10.031 INFO:tasks.workunit.client.0.smithi032.stdout:5/240: write d4/d7/d14/d26/d33/d4a/d38/d3b/f48 [68808,33449] 0 2024-05-12T04:01:10.032 INFO:tasks.workunit.client.1.smithi033.stdout:0/418: truncate d6/dc/d58/d6a/d80/f84 372159 0 2024-05-12T04:01:10.032 INFO:tasks.workunit.client.1.smithi033.stdout:2/385: truncate dc/d17/d27/d2f/f38 2547662 0 2024-05-12T04:01:10.032 INFO:tasks.workunit.client.1.smithi033.stdout:5/308: stat d3/d12/d29/l33 0 2024-05-12T04:01:10.033 INFO:tasks.workunit.client.0.smithi032.stdout:7/222: mknod dd/d1b/d43/c51 0 2024-05-12T04:01:10.036 INFO:tasks.workunit.client.0.smithi032.stdout:5/241: write d4/d7/d14/d26/d33/d4a/d38/f45 [1019074,77548] 0 2024-05-12T04:01:10.038 INFO:tasks.workunit.client.1.smithi033.stdout:6/373: link d4/d43/l46 d4/d6e/l7c 0 2024-05-12T04:01:10.039 INFO:tasks.workunit.client.1.smithi033.stdout:5/309: dwrite d3/d15/f30 [0,4194304] 0 2024-05-12T04:01:10.041 INFO:tasks.workunit.client.0.smithi032.stdout:2/148: symlink d1/d13/d25/d2c/l38 0 2024-05-12T04:01:10.044 INFO:tasks.workunit.client.1.smithi033.stdout:5/310: dwrite d3/d15/d16/f20 [8388608,4194304] 0 2024-05-12T04:01:10.044 INFO:tasks.workunit.client.0.smithi032.stdout:7/223: dwrite dd/d19/f1c [0,4194304] 0 2024-05-12T04:01:10.045 INFO:tasks.workunit.client.0.smithi032.stdout:9/140: symlink d0/d3/d6/d7/d27/l2e 0 2024-05-12T04:01:10.046 INFO:tasks.workunit.client.0.smithi032.stdout:9/141: readlink d0/d3/d6/d7/l9 0 2024-05-12T04:01:10.054 INFO:tasks.workunit.client.0.smithi032.stdout:8/232: link d0/f2f d0/d9/d1b/d33/f45 0 2024-05-12T04:01:10.055 INFO:tasks.workunit.client.0.smithi032.stdout:7/224: dread dd/f42 [0,4194304] 0 2024-05-12T04:01:10.060 INFO:tasks.workunit.client.0.smithi032.stdout:9/142: dwrite d0/d3/ff [8388608,4194304] 0 2024-05-12T04:01:10.060 INFO:tasks.workunit.client.1.smithi033.stdout:3/364: link db/d17/c4f db/d17/d25/d32/d53/c7a 0 2024-05-12T04:01:10.060 INFO:tasks.workunit.client.1.smithi033.stdout:5/311: readlink d3/d12/d29/l33 0 2024-05-12T04:01:10.061 INFO:tasks.workunit.client.1.smithi033.stdout:4/375: symlink dd/d11/d31/d66/d42/l79 0 2024-05-12T04:01:10.064 INFO:tasks.workunit.client.0.smithi032.stdout:2/149: mkdir d1/d13/d25/d39 0 2024-05-12T04:01:10.068 INFO:tasks.workunit.client.0.smithi032.stdout:2/150: chown d1/d13/d25/c26 2 1 2024-05-12T04:01:10.068 INFO:tasks.workunit.client.1.smithi033.stdout:4/376: dwrite dd/d11/f46 [0,4194304] 0 2024-05-12T04:01:10.069 INFO:tasks.workunit.client.0.smithi032.stdout:2/151: chown d1/f12 3449 1 2024-05-12T04:01:10.071 INFO:tasks.workunit.client.1.smithi033.stdout:7/535: creat de/fbb x:0 0 0 2024-05-12T04:01:10.071 INFO:tasks.workunit.client.1.smithi033.stdout:4/377: write fa [1870450,72943] 0 2024-05-12T04:01:10.073 INFO:tasks.workunit.client.1.smithi033.stdout:8/296: symlink d1/d16/d27/l5f 0 2024-05-12T04:01:10.073 INFO:tasks.workunit.client.1.smithi033.stdout:4/378: dread - dd/d11/d31/f6c zero size 2024-05-12T04:01:10.074 INFO:tasks.workunit.client.1.smithi033.stdout:7/536: read de/df/d17/f34 [704318,63723] 0 2024-05-12T04:01:10.077 INFO:tasks.workunit.client.1.smithi033.stdout:3/365: creat db/d17/f7b x:0 0 0 2024-05-12T04:01:10.078 INFO:tasks.workunit.client.1.smithi033.stdout:9/594: write d2/d4/d2b/d6d/d2f/f38 [96330,90935] 0 2024-05-12T04:01:10.080 INFO:tasks.workunit.client.0.smithi032.stdout:8/233: mknod d0/d9/d1b/c46 0 2024-05-12T04:01:10.083 INFO:tasks.workunit.client.1.smithi033.stdout:5/312: unlink d3/d12/d25/f4a 0 2024-05-12T04:01:10.085 INFO:tasks.workunit.client.1.smithi033.stdout:1/335: sync 2024-05-12T04:01:10.086 INFO:tasks.workunit.client.1.smithi033.stdout:5/313: dread d3/d15/d16/f42 [0,4194304] 0 2024-05-12T04:01:10.086 INFO:tasks.workunit.client.1.smithi033.stdout:1/336: write d11/d17/f18 [3001106,77673] 0 2024-05-12T04:01:10.087 INFO:tasks.workunit.client.0.smithi032.stdout:8/234: dwrite d0/f5 [4194304,4194304] 0 2024-05-12T04:01:10.089 INFO:tasks.workunit.client.1.smithi033.stdout:1/337: truncate d11/d17/d27/d36/d46/f66 391432 0 2024-05-12T04:01:10.091 INFO:tasks.workunit.client.1.smithi033.stdout:1/338: dread d11/d17/d38/f71 [0,4194304] 0 2024-05-12T04:01:10.091 INFO:tasks.workunit.client.0.smithi032.stdout:1/190: truncate d1/f13 2989533 0 2024-05-12T04:01:10.092 INFO:tasks.workunit.client.1.smithi033.stdout:1/339: stat d11/d1c/f54 0 2024-05-12T04:01:10.092 INFO:tasks.workunit.client.0.smithi032.stdout:1/191: chown d1/f26 381 1 2024-05-12T04:01:10.092 INFO:tasks.workunit.client.1.smithi033.stdout:6/374: symlink d4/d6e/l7d 0 2024-05-12T04:01:10.093 INFO:tasks.workunit.client.0.smithi032.stdout:1/192: write d1/f3 [201002,23015] 0 2024-05-12T04:01:10.103 INFO:tasks.workunit.client.0.smithi032.stdout:9/143: symlink d0/l2f 0 2024-05-12T04:01:10.104 INFO:tasks.workunit.client.0.smithi032.stdout:8/235: dread d0/f5 [0,4194304] 0 2024-05-12T04:01:10.105 INFO:tasks.workunit.client.1.smithi033.stdout:3/366: dread - db/d17/d25/d32/d3c/d3d/f6f zero size 2024-05-12T04:01:10.107 INFO:tasks.workunit.client.0.smithi032.stdout:5/242: dwrite d4/d7/d14/d26/d33/d4a/f51 [0,4194304] 0 2024-05-12T04:01:10.107 INFO:tasks.workunit.client.0.smithi032.stdout:8/236: stat d0/d9/d13/f29 0 2024-05-12T04:01:10.107 INFO:tasks.workunit.client.1.smithi033.stdout:9/595: mkdir d2/d4/d29/d3c/dda 0 2024-05-12T04:01:10.108 INFO:tasks.workunit.client.1.smithi033.stdout:9/596: truncate d2/d4/d29/d3c/d52/fba 349924 0 2024-05-12T04:01:10.114 INFO:tasks.workunit.client.0.smithi032.stdout:1/193: dread d1/f26 [0,4194304] 0 2024-05-12T04:01:10.115 INFO:tasks.workunit.client.0.smithi032.stdout:5/243: fdatasync d4/f1e 0 2024-05-12T04:01:10.117 INFO:tasks.workunit.client.0.smithi032.stdout:2/152: getdents d1/d34 0 2024-05-12T04:01:10.117 INFO:tasks.workunit.client.0.smithi032.stdout:9/144: dwrite d0/d3/f25 [0,4194304] 0 2024-05-12T04:01:10.119 INFO:tasks.workunit.client.1.smithi033.stdout:2/386: sync 2024-05-12T04:01:10.120 INFO:tasks.workunit.client.1.smithi033.stdout:2/387: readlink dc/d3b/l71 0 2024-05-12T04:01:10.122 INFO:tasks.workunit.client.1.smithi033.stdout:5/314: write d3/d27/f28 [965721,53579] 0 2024-05-12T04:01:10.125 INFO:tasks.workunit.client.1.smithi033.stdout:0/419: creat d6/dc/d58/f88 x:0 0 0 2024-05-12T04:01:10.126 INFO:tasks.workunit.client.1.smithi033.stdout:0/420: fdatasync d6/dc/f54 0 2024-05-12T04:01:10.127 INFO:tasks.workunit.client.0.smithi032.stdout:1/194: creat d1/d7/d30/d18/f41 x:0 0 0 2024-05-12T04:01:10.127 INFO:tasks.workunit.client.1.smithi033.stdout:7/537: dwrite de/d2b/d6c/f5a [0,4194304] 0 2024-05-12T04:01:10.128 INFO:tasks.workunit.client.1.smithi033.stdout:6/375: fdatasync d4/d1a/d67/d68/f6a 0 2024-05-12T04:01:10.129 INFO:tasks.workunit.client.1.smithi033.stdout:7/538: dread - de/df/d17/d1e/f23 zero size 2024-05-12T04:01:10.130 INFO:tasks.workunit.client.0.smithi032.stdout:5/244: unlink d4/d7/d14/d39/d1b/f44 0 2024-05-12T04:01:10.130 INFO:tasks.workunit.client.1.smithi033.stdout:7/539: dread - de/d20/d38/f9e zero size 2024-05-12T04:01:10.131 INFO:tasks.workunit.client.1.smithi033.stdout:6/376: write d4/d1a/d67/d68/f6a [1425573,91250] 0 2024-05-12T04:01:10.133 INFO:tasks.workunit.client.0.smithi032.stdout:2/153: dread d1/ff [0,4194304] 0 2024-05-12T04:01:10.135 INFO:tasks.workunit.client.0.smithi032.stdout:9/145: creat d0/d3/d6/f30 x:0 0 0 2024-05-12T04:01:10.144 INFO:tasks.workunit.client.0.smithi032.stdout:1/195: creat d1/d7/d30/d35/f42 x:0 0 0 2024-05-12T04:01:10.144 INFO:tasks.workunit.client.0.smithi032.stdout:1/196: chown d1/d25/c27 20003203 1 2024-05-12T04:01:10.144 INFO:tasks.workunit.client.1.smithi033.stdout:4/379: write dd/d11/d31/d38/f68 [443956,75846] 0 2024-05-12T04:01:10.147 INFO:tasks.workunit.client.1.smithi033.stdout:9/597: creat d2/d4/d2b/d49/d73/fdb x:0 0 0 2024-05-12T04:01:10.148 INFO:tasks.workunit.client.0.smithi032.stdout:5/245: mknod d4/d7/d14/d39/d2b/c53 0 2024-05-12T04:01:10.149 INFO:tasks.workunit.client.1.smithi033.stdout:2/388: dread dc/d17/d3a/f6d [0,4194304] 0 2024-05-12T04:01:10.151 INFO:tasks.workunit.client.0.smithi032.stdout:8/237: truncate d0/d9/f1c 3017492 0 2024-05-12T04:01:10.152 INFO:tasks.workunit.client.1.smithi033.stdout:8/297: truncate d1/f2 7479658 0 2024-05-12T04:01:10.153 INFO:tasks.workunit.client.1.smithi033.stdout:8/298: chown d1/d16/d20/f33 54 1 2024-05-12T04:01:10.154 INFO:tasks.workunit.client.1.smithi033.stdout:8/299: write d1/d16/d27/d4d/d51/f31 [3029019,124857] 0 2024-05-12T04:01:10.154 INFO:tasks.workunit.client.0.smithi032.stdout:2/154: creat d1/d13/d25/f3a x:0 0 0 2024-05-12T04:01:10.156 INFO:tasks.workunit.client.0.smithi032.stdout:5/246: dwrite d4/d7/d14/d26/d33/d4a/f4e [0,4194304] 0 2024-05-12T04:01:10.156 INFO:tasks.workunit.client.0.smithi032.stdout:9/146: creat d0/d3/d6/d7/d27/f31 x:0 0 0 2024-05-12T04:01:10.157 INFO:tasks.workunit.client.0.smithi032.stdout:2/155: write d1/d13/d25/f3a [462671,43027] 0 2024-05-12T04:01:10.160 INFO:tasks.workunit.client.0.smithi032.stdout:8/238: rmdir d0/d7 39 2024-05-12T04:01:10.160 INFO:tasks.workunit.client.0.smithi032.stdout:2/156: fsync d1/d13/d25/f2a 0 2024-05-12T04:01:10.161 INFO:tasks.workunit.client.0.smithi032.stdout:5/247: write d4/d7/d14/d39/d2b/f32 [974325,44184] 0 2024-05-12T04:01:10.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:09 smithi033 ceph-mon[49525]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:01:10.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:09 smithi033 ceph-mon[49525]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:01:10.163 INFO:tasks.workunit.client.1.smithi033.stdout:6/377: mknod d4/d5/c7e 0 2024-05-12T04:01:10.163 INFO:tasks.workunit.client.1.smithi033.stdout:6/378: readlink d4/d6e/l7d 0 2024-05-12T04:01:10.166 INFO:tasks.workunit.client.1.smithi033.stdout:3/367: mkdir db/d17/d25/d32/d3c/d7c 0 2024-05-12T04:01:10.173 INFO:tasks.workunit.client.0.smithi032.stdout:9/147: dwrite d0/d3/d6/d7/da/d10/d13/f1d [0,4194304] 0 2024-05-12T04:01:10.175 INFO:tasks.workunit.client.0.smithi032.stdout:8/239: dwrite d0/d9/d13/f24 [0,4194304] 0 2024-05-12T04:01:10.177 INFO:tasks.workunit.client.0.smithi032.stdout:2/157: symlink d1/d13/d25/l3b 0 2024-05-12T04:01:10.181 INFO:tasks.workunit.client.1.smithi033.stdout:2/389: rename dc/d13/c36 to dc/d13/d39/c87 0 2024-05-12T04:01:10.184 INFO:tasks.workunit.client.1.smithi033.stdout:7/540: write de/d2f/f5c [379773,64406] 0 2024-05-12T04:01:10.185 INFO:tasks.workunit.client.0.smithi032.stdout:9/148: creat d0/d3/f32 x:0 0 0 2024-05-12T04:01:10.186 INFO:tasks.workunit.client.0.smithi032.stdout:8/240: dwrite d0/d9/d13/f24 [0,4194304] 0 2024-05-12T04:01:10.188 INFO:tasks.workunit.client.0.smithi032.stdout:8/241: read - d0/d9/d1b/d33/f3b zero size 2024-05-12T04:01:10.192 INFO:tasks.workunit.client.1.smithi033.stdout:5/315: fsync d3/f3b 0 2024-05-12T04:01:10.193 INFO:tasks.workunit.client.0.smithi032.stdout:9/149: write d0/d3/d6/f30 [526827,7380] 0 2024-05-12T04:01:10.193 INFO:tasks.workunit.client.1.smithi033.stdout:5/316: chown d3/d15/d16/f51 252180 1 2024-05-12T04:01:10.193 INFO:tasks.workunit.client.0.smithi032.stdout:5/248: rename d4/d7/d14/d26/d33/d4a/d38/d3b/c4b to d4/d7/d14/d26/d33/d4a/d38/c54 0 2024-05-12T04:01:10.195 INFO:tasks.workunit.client.0.smithi032.stdout:4/138: sync 2024-05-12T04:01:10.195 INFO:tasks.workunit.client.0.smithi032.stdout:6/200: sync 2024-05-12T04:01:10.197 INFO:tasks.workunit.client.0.smithi032.stdout:9/150: chown d0/d3/d6/d7/da/d17 248 1 2024-05-12T04:01:10.197 INFO:tasks.workunit.client.1.smithi033.stdout:4/380: write dd/d11/f19 [4244484,63921] 0 2024-05-12T04:01:10.199 INFO:tasks.workunit.client.0.smithi032.stdout:6/201: write d7/de/f3b [1391782,65589] 0 2024-05-12T04:01:10.202 INFO:tasks.workunit.client.0.smithi032.stdout:8/242: mknod d0/d9/d13/c47 0 2024-05-12T04:01:10.203 INFO:tasks.workunit.client.1.smithi033.stdout:9/598: mkdir d2/d4/d2b/d33/dd2/ddc 0 2024-05-12T04:01:10.203 INFO:tasks.workunit.client.0.smithi032.stdout:1/197: dwrite d1/f13 [0,4194304] 0 2024-05-12T04:01:10.204 INFO:tasks.workunit.client.0.smithi032.stdout:9/151: dread d0/d3/d6/d7/da/d17/f20 [0,4194304] 0 2024-05-12T04:01:10.205 INFO:tasks.workunit.client.0.smithi032.stdout:1/198: write d1/d7/d30/f28 [816312,66516] 0 2024-05-12T04:01:10.206 INFO:tasks.workunit.client.1.smithi033.stdout:6/379: rename d4/d1a/d4e/f6d to d4/d1a/d6c/f7f 0 2024-05-12T04:01:10.207 INFO:tasks.workunit.client.1.smithi033.stdout:6/380: dread - d4/d5/f73 zero size 2024-05-12T04:01:10.208 INFO:tasks.workunit.client.0.smithi032.stdout:4/139: unlink d0/d3/df/c1b 0 2024-05-12T04:01:10.208 INFO:tasks.workunit.client.0.smithi032.stdout:9/152: write d0/d3/d6/d7/da/f1a [3085585,99878] 0 2024-05-12T04:01:10.209 INFO:tasks.workunit.client.0.smithi032.stdout:4/140: chown d0/l14 0 1 2024-05-12T04:01:10.210 INFO:tasks.workunit.client.1.smithi033.stdout:6/381: write d4/d1a/d3c/f65 [596892,73901] 0 2024-05-12T04:01:10.210 INFO:tasks.workunit.client.1.smithi033.stdout:7/541: mkdir de/df/d65/dbc 0 2024-05-12T04:01:10.211 INFO:tasks.workunit.client.0.smithi032.stdout:2/158: mknod d1/c3c 0 2024-05-12T04:01:10.212 INFO:tasks.workunit.client.1.smithi033.stdout:9/599: dwrite d2/d4/d2b/d78/f88 [0,4194304] 0 2024-05-12T04:01:10.212 INFO:tasks.workunit.client.0.smithi032.stdout:2/159: dread d1/ff [0,4194304] 0 2024-05-12T04:01:10.213 INFO:tasks.workunit.client.1.smithi033.stdout:9/600: stat d2/f27 0 2024-05-12T04:01:10.215 INFO:tasks.workunit.client.0.smithi032.stdout:2/160: write d1/f7 [1399420,63281] 0 2024-05-12T04:01:10.215 INFO:tasks.workunit.client.0.smithi032.stdout:2/161: readlink d1/l35 0 2024-05-12T04:01:10.217 INFO:tasks.workunit.client.0.smithi032.stdout:2/162: chown d1/d13/d25/f33 83 1 2024-05-12T04:01:10.219 INFO:tasks.workunit.client.0.smithi032.stdout:6/202: symlink d7/de/l44 0 2024-05-12T04:01:10.221 INFO:tasks.workunit.client.1.smithi033.stdout:1/340: getdents d11 0 2024-05-12T04:01:10.221 INFO:tasks.workunit.client.1.smithi033.stdout:8/300: write d1/f13 [1852881,13155] 0 2024-05-12T04:01:10.221 INFO:tasks.workunit.client.1.smithi033.stdout:6/382: dwrite d4/d1a/d4e/f71 [0,4194304] 0 2024-05-12T04:01:10.223 INFO:tasks.workunit.client.1.smithi033.stdout:0/421: link d6/db/d2c/d73/f7a d6/dc/d58/d6a/f89 0 2024-05-12T04:01:10.224 INFO:tasks.workunit.client.0.smithi032.stdout:1/199: mknod d1/dd/d24/c43 0 2024-05-12T04:01:10.225 INFO:tasks.workunit.client.0.smithi032.stdout:1/200: rename d1 to d1/d7/d44 22 2024-05-12T04:01:10.226 INFO:tasks.workunit.client.1.smithi033.stdout:3/368: mkdir db/d17/d3e/d65/d7d 0 2024-05-12T04:01:10.229 INFO:tasks.workunit.client.0.smithi032.stdout:4/141: mknod d0/d3/c22 0 2024-05-12T04:01:10.231 INFO:tasks.workunit.client.0.smithi032.stdout:3/244: sync 2024-05-12T04:01:10.231 INFO:tasks.workunit.client.0.smithi032.stdout:0/260: sync 2024-05-12T04:01:10.232 INFO:tasks.workunit.client.0.smithi032.stdout:3/245: write d15/d1b/f21 [147720,90187] 0 2024-05-12T04:01:10.236 INFO:tasks.workunit.client.0.smithi032.stdout:2/163: fsync d1/d34/f32 0 2024-05-12T04:01:10.237 INFO:tasks.workunit.client.1.smithi033.stdout:8/301: dread d1/d4/d46/f4f [0,4194304] 0 2024-05-12T04:01:10.237 INFO:tasks.workunit.client.0.smithi032.stdout:4/142: dread d0/f11 [0,4194304] 0 2024-05-12T04:01:10.238 INFO:tasks.workunit.client.0.smithi032.stdout:4/143: readlink d0/d3/l1e 0 2024-05-12T04:01:10.240 INFO:tasks.workunit.client.0.smithi032.stdout:4/144: write d0/f13 [3242085,131054] 0 2024-05-12T04:01:10.242 INFO:tasks.workunit.client.0.smithi032.stdout:1/201: rename d1/d7/f11 to d1/d7/d30/d35/f45 0 2024-05-12T04:01:10.243 INFO:tasks.workunit.client.1.smithi033.stdout:8/302: dwrite d1/d16/f19 [0,4194304] 0 2024-05-12T04:01:10.243 INFO:tasks.workunit.client.0.smithi032.stdout:1/202: write d1/f13 [744569,43018] 0 2024-05-12T04:01:10.244 INFO:tasks.workunit.client.0.smithi032.stdout:5/249: link d4/d7/d14/d26/d3c/l50 d4/d7/d14/d39/l55 0 2024-05-12T04:01:10.245 INFO:tasks.workunit.client.0.smithi032.stdout:1/203: chown d1/d7/d30/f2c 2 1 2024-05-12T04:01:10.247 INFO:tasks.workunit.client.0.smithi032.stdout:1/204: fsync d1/d7/d30/d18/f1b 0 2024-05-12T04:01:10.252 INFO:tasks.workunit.client.1.smithi033.stdout:6/383: dread d4/f56 [0,4194304] 0 2024-05-12T04:01:10.253 INFO:tasks.workunit.client.1.smithi033.stdout:0/422: creat d6/dc/d39/f8a x:0 0 0 2024-05-12T04:01:10.256 INFO:tasks.workunit.client.1.smithi033.stdout:4/381: mkdir dd/d7a 0 2024-05-12T04:01:10.256 INFO:tasks.workunit.client.1.smithi033.stdout:4/382: write dd/d11/d31/f45 [2769110,48506] 0 2024-05-12T04:01:10.257 INFO:tasks.workunit.client.1.smithi033.stdout:4/383: stat dd/c54 0 2024-05-12T04:01:10.259 INFO:tasks.workunit.client.1.smithi033.stdout:3/369: mknod db/d17/d3a/c7e 0 2024-05-12T04:01:10.261 INFO:tasks.workunit.client.0.smithi032.stdout:0/261: dwrite df/d28/f59 [0,4194304] 0 2024-05-12T04:01:10.262 INFO:tasks.workunit.client.0.smithi032.stdout:0/262: write df/d28/f59 [1056764,96835] 0 2024-05-12T04:01:10.263 INFO:tasks.workunit.client.1.smithi033.stdout:2/390: rename dc/d17/d32/d67 to dc/d3b/d88 0 2024-05-12T04:01:10.263 INFO:tasks.workunit.client.1.smithi033.stdout:2/391: readlink dc/d13/d39/l3e 0 2024-05-12T04:01:10.267 INFO:tasks.workunit.client.0.smithi032.stdout:2/164: mkdir d1/d13/d3d 0 2024-05-12T04:01:10.267 INFO:tasks.workunit.client.0.smithi032.stdout:2/165: dread - d1/d13/f18 zero size 2024-05-12T04:01:10.272 INFO:tasks.workunit.client.1.smithi033.stdout:5/317: creat d3/d12/d25/d26/d2e/f61 x:0 0 0 2024-05-12T04:01:10.272 INFO:tasks.workunit.client.0.smithi032.stdout:8/243: link d0/d9/d13/d21/l43 d0/d9/d13/l48 0 2024-05-12T04:01:10.276 INFO:tasks.workunit.client.0.smithi032.stdout:4/145: symlink d0/d3/l23 0 2024-05-12T04:01:10.278 INFO:tasks.workunit.client.0.smithi032.stdout:4/146: write d0/f13 [3279547,118960] 0 2024-05-12T04:01:10.279 INFO:tasks.workunit.client.1.smithi033.stdout:7/542: dwrite de/df/d17/f6a [0,4194304] 0 2024-05-12T04:01:10.280 INFO:tasks.workunit.client.1.smithi033.stdout:7/543: stat de/df/d17/d1b/d50/c7c 0 2024-05-12T04:01:10.282 INFO:tasks.workunit.client.1.smithi033.stdout:6/384: mknod d4/c80 0 2024-05-12T04:01:10.282 INFO:tasks.workunit.client.0.smithi032.stdout:6/203: dwrite d7/de/f14 [0,4194304] 0 2024-05-12T04:01:10.283 INFO:tasks.workunit.client.0.smithi032.stdout:6/204: chown d7/de/d12/d1a/d20/f34 116 1 2024-05-12T04:01:10.288 INFO:tasks.workunit.client.0.smithi032.stdout:5/250: rename d4/d7/d14/d39/c35 to d4/d7/d14/d26/d47/c56 0 2024-05-12T04:01:10.288 INFO:tasks.workunit.client.1.smithi033.stdout:0/423: symlink d6/dc/d39/l8b 0 2024-05-12T04:01:10.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:09 smithi032 ceph-mon[40008]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:01:10.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:09 smithi032 ceph-mon[40008]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:01:10.294 INFO:tasks.workunit.client.1.smithi033.stdout:4/384: rmdir dd/d11/d31/d66/d42/d59 39 2024-05-12T04:01:10.294 INFO:tasks.workunit.client.0.smithi032.stdout:7/225: sync 2024-05-12T04:01:10.296 INFO:tasks.workunit.client.0.smithi032.stdout:1/205: mkdir d1/d7/d30/d35/d46 0 2024-05-12T04:01:10.299 INFO:tasks.workunit.client.0.smithi032.stdout:4/147: dread d0/fa [0,4194304] 0 2024-05-12T04:01:10.299 INFO:tasks.workunit.client.0.smithi032.stdout:9/153: link d0/d3/d6/f30 d0/d3/d6/d7/da/f33 0 2024-05-12T04:01:10.303 INFO:tasks.workunit.client.1.smithi033.stdout:9/601: creat d2/d4/d2b/fdd x:0 0 0 2024-05-12T04:01:10.306 INFO:tasks.workunit.client.1.smithi033.stdout:5/318: chown d3/c5 1 1 2024-05-12T04:01:10.306 INFO:tasks.workunit.client.0.smithi032.stdout:9/154: dwrite d0/d3/f25 [0,4194304] 0 2024-05-12T04:01:10.307 INFO:tasks.workunit.client.0.smithi032.stdout:3/246: creat d15/d20/d22/d4f/f53 x:0 0 0 2024-05-12T04:01:10.308 INFO:tasks.workunit.client.0.smithi032.stdout:3/247: write d15/d24/f51 [146199,70571] 0 2024-05-12T04:01:10.314 INFO:tasks.workunit.client.0.smithi032.stdout:0/263: write df/d10/d3d/d4a/f38 [5072589,19538] 0 2024-05-12T04:01:10.315 INFO:tasks.workunit.client.0.smithi032.stdout:2/166: mkdir d1/d34/d3e 0 2024-05-12T04:01:10.317 INFO:tasks.workunit.client.1.smithi033.stdout:7/544: mkdir de/d2b/d6c/dbd 0 2024-05-12T04:01:10.319 INFO:tasks.workunit.client.0.smithi032.stdout:8/244: fdatasync d0/d9/d13/f20 0 2024-05-12T04:01:10.323 INFO:tasks.workunit.client.1.smithi033.stdout:6/385: mknod d4/c81 0 2024-05-12T04:01:10.327 INFO:tasks.workunit.client.1.smithi033.stdout:4/385: fsync dd/f3b 0 2024-05-12T04:01:10.329 INFO:tasks.workunit.client.1.smithi033.stdout:8/303: dwrite d1/d4/d1d/d36/f50 [0,4194304] 0 2024-05-12T04:01:10.331 INFO:tasks.workunit.client.1.smithi033.stdout:2/392: creat dc/d17/d32/d84/f89 x:0 0 0 2024-05-12T04:01:10.334 INFO:tasks.workunit.client.0.smithi032.stdout:6/205: rename d7/de/d2e/f3a to d7/de/d12/f45 0 2024-05-12T04:01:10.338 INFO:tasks.workunit.client.1.smithi033.stdout:9/602: dread d2/d4/d2b/f4b [0,4194304] 0 2024-05-12T04:01:10.339 INFO:tasks.workunit.client.1.smithi033.stdout:9/603: chown d2/dd/d39/d72/d82/fcd 36122 1 2024-05-12T04:01:10.341 INFO:tasks.workunit.client.1.smithi033.stdout:1/341: getdents d11/d17/d27 0 2024-05-12T04:01:10.342 INFO:tasks.workunit.client.0.smithi032.stdout:5/251: creat d4/d7/d14/d26/f57 x:0 0 0 2024-05-12T04:01:10.342 INFO:tasks.workunit.client.1.smithi033.stdout:9/604: write d2/d4/d29/d3c/d52/fba [1074391,26046] 0 2024-05-12T04:01:10.344 INFO:tasks.workunit.client.1.smithi033.stdout:2/393: dread f7 [0,4194304] 0 2024-05-12T04:01:10.345 INFO:tasks.workunit.client.0.smithi032.stdout:4/148: read d0/d3/fd [567475,110508] 0 2024-05-12T04:01:10.346 INFO:tasks.workunit.client.1.smithi033.stdout:6/386: creat d4/f82 x:0 0 0 2024-05-12T04:01:10.347 INFO:tasks.workunit.client.0.smithi032.stdout:5/252: dread d4/fd [0,4194304] 0 2024-05-12T04:01:10.348 INFO:tasks.workunit.client.1.smithi033.stdout:5/319: write d3/f3b [1056931,62454] 0 2024-05-12T04:01:10.349 INFO:tasks.workunit.client.0.smithi032.stdout:5/253: fdatasync d4/d7/d14/d39/d1b/f23 0 2024-05-12T04:01:10.349 INFO:tasks.workunit.client.0.smithi032.stdout:7/226: write dd/f12 [4496726,89675] 0 2024-05-12T04:01:10.351 INFO:tasks.workunit.client.0.smithi032.stdout:7/227: write dd/d19/f34 [175843,63445] 0 2024-05-12T04:01:10.352 INFO:tasks.workunit.client.0.smithi032.stdout:0/264: mknod df/d10/d48/c5f 0 2024-05-12T04:01:10.353 INFO:tasks.workunit.client.1.smithi033.stdout:3/370: link db/l11 db/d17/d25/l7f 0 2024-05-12T04:01:10.356 INFO:tasks.workunit.client.0.smithi032.stdout:8/245: creat d0/d9/d13/d26/d28/f49 x:0 0 0 2024-05-12T04:01:10.358 INFO:tasks.workunit.client.1.smithi033.stdout:7/545: mknod de/d2b/d6c/dbd/cbe 0 2024-05-12T04:01:10.359 INFO:tasks.workunit.client.1.smithi033.stdout:9/605: mkdir d2/d4/d2b/d33/dd2/dde 0 2024-05-12T04:01:10.360 INFO:tasks.workunit.client.1.smithi033.stdout:1/342: creat d11/d17/d27/d36/d46/d4c/f73 x:0 0 0 2024-05-12T04:01:10.361 INFO:tasks.workunit.client.0.smithi032.stdout:6/206: mknod d7/de/d12/d1a/d20/c46 0 2024-05-12T04:01:10.361 INFO:tasks.workunit.client.1.smithi033.stdout:1/343: chown d11/d1c/d43/f4d 73714734 1 2024-05-12T04:01:10.361 INFO:tasks.workunit.client.1.smithi033.stdout:2/394: truncate f7 2694592 0 2024-05-12T04:01:10.363 INFO:tasks.workunit.client.1.smithi033.stdout:2/395: write dc/d17/d27/d2f/f75 [1047127,38329] 0 2024-05-12T04:01:10.364 INFO:tasks.workunit.client.0.smithi032.stdout:4/149: mkdir d0/d3/d24 0 2024-05-12T04:01:10.364 INFO:tasks.workunit.client.1.smithi033.stdout:4/386: getdents dd/d69 0 2024-05-12T04:01:10.365 INFO:tasks.workunit.client.1.smithi033.stdout:4/387: readlink dd/d11/d31/l39 0 2024-05-12T04:01:10.365 INFO:tasks.workunit.client.0.smithi032.stdout:4/150: write d0/f13 [4160478,24686] 0 2024-05-12T04:01:10.366 INFO:tasks.workunit.client.0.smithi032.stdout:9/155: mkdir d0/d3/d6/d7/d2a/d34 0 2024-05-12T04:01:10.366 INFO:tasks.workunit.client.0.smithi032.stdout:4/151: chown d0/d3/c19 6639 1 2024-05-12T04:01:10.367 INFO:tasks.workunit.client.0.smithi032.stdout:9/156: chown d0/d3/ff 2941903 1 2024-05-12T04:01:10.367 INFO:tasks.workunit.client.0.smithi032.stdout:4/152: chown d0/d3/f1c 6658 1 2024-05-12T04:01:10.367 INFO:tasks.workunit.client.1.smithi033.stdout:3/371: symlink db/d17/d25/d32/d3c/d44/l80 0 2024-05-12T04:01:10.371 INFO:tasks.workunit.client.0.smithi032.stdout:5/254: fdatasync d4/fd 0 2024-05-12T04:01:10.373 INFO:tasks.workunit.client.0.smithi032.stdout:7/228: symlink dd/d1b/d1e/l52 0 2024-05-12T04:01:10.373 INFO:tasks.workunit.client.0.smithi032.stdout:5/255: fsync d4/d7/d14/f1d 0 2024-05-12T04:01:10.374 INFO:tasks.workunit.client.0.smithi032.stdout:4/153: dwrite d0/f13 [0,4194304] 0 2024-05-12T04:01:10.378 INFO:tasks.workunit.client.1.smithi033.stdout:1/344: write d11/d17/d27/d36/d3f/f61 [439992,52046] 0 2024-05-12T04:01:10.380 INFO:tasks.workunit.client.0.smithi032.stdout:0/265: mkdir df/d30/d32/d60 0 2024-05-12T04:01:10.383 INFO:tasks.workunit.client.0.smithi032.stdout:4/154: dwrite d0/d3/f1c [0,4194304] 0 2024-05-12T04:01:10.388 INFO:tasks.workunit.client.1.smithi033.stdout:0/424: getdents d6 0 2024-05-12T04:01:10.391 INFO:tasks.workunit.client.0.smithi032.stdout:6/207: symlink d7/de/d12/l47 0 2024-05-12T04:01:10.392 INFO:tasks.workunit.client.0.smithi032.stdout:1/206: creat d1/f47 x:0 0 0 2024-05-12T04:01:10.394 INFO:tasks.workunit.client.1.smithi033.stdout:9/606: write d2/d4/d29/d3c/f9a [84542,54868] 0 2024-05-12T04:01:10.395 INFO:tasks.workunit.client.1.smithi033.stdout:4/388: chown dd/d11/d5b/c77 360855 1 2024-05-12T04:01:10.396 INFO:tasks.workunit.client.1.smithi033.stdout:9/607: stat d2/d4/d2b/d6d/d4c 0 2024-05-12T04:01:10.397 INFO:tasks.workunit.client.0.smithi032.stdout:9/157: mknod d0/d3/d6/c35 0 2024-05-12T04:01:10.398 INFO:tasks.workunit.client.1.smithi033.stdout:1/345: creat d11/d17/d27/d36/d46/f74 x:0 0 0 2024-05-12T04:01:10.398 INFO:tasks.workunit.client.0.smithi032.stdout:6/208: dwrite d7/de/d12/f28 [0,4194304] 0 2024-05-12T04:01:10.401 INFO:tasks.workunit.client.1.smithi033.stdout:1/346: chown d11/d17/d27/f2b 211676 1 2024-05-12T04:01:10.401 INFO:tasks.workunit.client.1.smithi033.stdout:6/387: creat d4/d1a/f83 x:0 0 0 2024-05-12T04:01:10.404 INFO:tasks.workunit.client.0.smithi032.stdout:7/229: unlink dd/d19/l30 0 2024-05-12T04:01:10.412 INFO:tasks.workunit.client.0.smithi032.stdout:6/209: dread d7/de/d12/f18 [0,4194304] 0 2024-05-12T04:01:10.413 INFO:tasks.workunit.client.0.smithi032.stdout:0/266: dread df/d30/d36/f41 [0,4194304] 0 2024-05-12T04:01:10.417 INFO:tasks.workunit.client.0.smithi032.stdout:3/248: rename d15/d24/d40/f48 to d15/f54 0 2024-05-12T04:01:10.438 INFO:tasks.workunit.client.1.smithi033.stdout:5/320: rename d3/d5c to d3/d15/d16/d45/d62 0 2024-05-12T04:01:10.442 INFO:tasks.workunit.client.1.smithi033.stdout:0/425: rmdir d6/dc/d58 39 2024-05-12T04:01:10.447 INFO:tasks.workunit.client.1.smithi033.stdout:0/426: dwrite d6/f6b [0,4194304] 0 2024-05-12T04:01:10.448 INFO:tasks.workunit.client.1.smithi033.stdout:0/427: fdatasync d6/db/d2c/f83 0 2024-05-12T04:01:10.450 INFO:tasks.workunit.client.1.smithi033.stdout:8/304: getdents d1/d16 0 2024-05-12T04:01:10.452 INFO:tasks.workunit.client.1.smithi033.stdout:7/546: link de/df/d17/d1b/d50/c5f de/df/d9a/db7/cbf 0 2024-05-12T04:01:10.453 INFO:tasks.workunit.client.1.smithi033.stdout:7/547: fdatasync de/df/d17/f51 0 2024-05-12T04:01:10.457 INFO:tasks.workunit.client.1.smithi033.stdout:8/305: dwrite d1/d4/f39 [0,4194304] 0 2024-05-12T04:01:10.457 INFO:tasks.workunit.client.1.smithi033.stdout:1/347: symlink d11/d1c/d59/d68/l75 0 2024-05-12T04:01:10.458 INFO:tasks.workunit.client.1.smithi033.stdout:1/348: stat d11/d17/f20 0 2024-05-12T04:01:10.459 INFO:tasks.workunit.client.1.smithi033.stdout:6/388: symlink d4/d1a/d4e/d4f/l84 0 2024-05-12T04:01:10.461 INFO:tasks.workunit.client.1.smithi033.stdout:5/321: creat d3/d12/f63 x:0 0 0 2024-05-12T04:01:10.462 INFO:tasks.workunit.client.1.smithi033.stdout:1/349: truncate d11/d17/d27/d36/d46/f74 113905 0 2024-05-12T04:01:10.466 INFO:tasks.workunit.client.1.smithi033.stdout:4/389: symlink dd/d7a/l7b 0 2024-05-12T04:01:10.469 INFO:tasks.workunit.client.1.smithi033.stdout:8/306: creat d1/d4/d1d/d36/d3c/d47/f60 x:0 0 0 2024-05-12T04:01:10.471 INFO:tasks.workunit.client.1.smithi033.stdout:3/372: getdents db/d42 0 2024-05-12T04:01:10.473 INFO:tasks.workunit.client.1.smithi033.stdout:1/350: rename d11/d1c/d59/d68 to d11/d17/d27/d76 0 2024-05-12T04:01:10.476 INFO:tasks.workunit.client.1.smithi033.stdout:9/608: dwrite d2/fa [0,4194304] 0 2024-05-12T04:01:10.477 INFO:tasks.workunit.client.1.smithi033.stdout:9/609: readlink d2/d4/d29/lcf 0 2024-05-12T04:01:10.478 INFO:tasks.workunit.client.1.smithi033.stdout:8/307: symlink d1/d16/d20/l61 0 2024-05-12T04:01:10.478 INFO:tasks.workunit.client.1.smithi033.stdout:1/351: truncate d11/d17/d27/f57 486173 0 2024-05-12T04:01:10.479 INFO:tasks.workunit.client.1.smithi033.stdout:0/428: creat d6/dc/f8c x:0 0 0 2024-05-12T04:01:10.481 INFO:tasks.workunit.client.1.smithi033.stdout:5/322: dwrite d3/f40 [4194304,4194304] 0 2024-05-12T04:01:10.484 INFO:tasks.workunit.client.1.smithi033.stdout:6/389: getdents d4 0 2024-05-12T04:01:10.485 INFO:tasks.workunit.client.1.smithi033.stdout:5/323: chown d3/d12/d29/d2b/c5e 2160028 1 2024-05-12T04:01:10.486 INFO:tasks.workunit.client.1.smithi033.stdout:4/390: rename dd/c3c to dd/d69/c7c 0 2024-05-12T04:01:10.489 INFO:tasks.workunit.client.1.smithi033.stdout:3/373: creat db/f81 x:0 0 0 2024-05-12T04:01:10.492 INFO:tasks.workunit.client.1.smithi033.stdout:1/352: mknod d11/d17/d27/d36/d3f/d4a/c77 0 2024-05-12T04:01:10.494 INFO:tasks.workunit.client.1.smithi033.stdout:9/610: dwrite d2/d4/d29/d3c/f9a [0,4194304] 0 2024-05-12T04:01:10.496 INFO:tasks.workunit.client.1.smithi033.stdout:4/391: truncate dd/d11/d31/d66/f6a 462282 0 2024-05-12T04:01:10.498 INFO:tasks.workunit.client.0.smithi032.stdout:1/207: creat d1/d7/d30/d35/f48 x:0 0 0 2024-05-12T04:01:10.499 INFO:tasks.workunit.client.0.smithi032.stdout:9/158: rmdir d0 39 2024-05-12T04:01:10.500 INFO:tasks.workunit.client.0.smithi032.stdout:5/256: mkdir d4/d7/d58 0 2024-05-12T04:01:10.500 INFO:tasks.workunit.client.0.smithi032.stdout:6/210: dwrite d7/de/d12/d25/f3e [4194304,4194304] 0 2024-05-12T04:01:10.501 INFO:tasks.workunit.client.0.smithi032.stdout:2/167: getdents d1/d13/d25/d2c 0 2024-05-12T04:01:10.502 INFO:tasks.workunit.client.1.smithi033.stdout:6/390: truncate d4/d1a/f26 796622 0 2024-05-12T04:01:10.504 INFO:tasks.workunit.client.0.smithi032.stdout:4/155: truncate d0/f13 1728393 0 2024-05-12T04:01:10.504 INFO:tasks.workunit.client.1.smithi033.stdout:9/611: dread - d2/d70/d7b/fbe zero size 2024-05-12T04:01:10.504 INFO:tasks.workunit.client.1.smithi033.stdout:4/392: creat dd/d69/f7d x:0 0 0 2024-05-12T04:01:10.505 INFO:tasks.workunit.client.0.smithi032.stdout:4/156: stat d0 0 2024-05-12T04:01:10.506 INFO:tasks.workunit.client.1.smithi033.stdout:3/374: truncate db/d17/d3a/d67/d76/f78 793779 0 2024-05-12T04:01:10.508 INFO:tasks.workunit.client.1.smithi033.stdout:5/324: dwrite d3/d15/f30 [0,4194304] 0 2024-05-12T04:01:10.509 INFO:tasks.workunit.client.1.smithi033.stdout:8/308: dread d1/d4/d46/f4f [0,4194304] 0 2024-05-12T04:01:10.509 INFO:tasks.workunit.client.1.smithi033.stdout:6/391: write d4/f16 [6540601,106143] 0 2024-05-12T04:01:10.512 INFO:tasks.workunit.client.0.smithi032.stdout:4/157: write d0/d3/f1c [5010716,22650] 0 2024-05-12T04:01:10.513 INFO:tasks.workunit.client.1.smithi033.stdout:6/392: chown d4/f51 104330 1 2024-05-12T04:01:10.514 INFO:tasks.workunit.client.1.smithi033.stdout:1/353: dwrite d11/d17/d27/f48 [0,4194304] 0 2024-05-12T04:01:10.515 INFO:tasks.workunit.client.0.smithi032.stdout:8/246: dwrite d0/f2 [0,4194304] 0 2024-05-12T04:01:10.515 INFO:tasks.workunit.client.1.smithi033.stdout:0/429: dwrite d6/dc/d39/f40 [0,4194304] 0 2024-05-12T04:01:10.516 INFO:tasks.workunit.client.1.smithi033.stdout:2/396: sync 2024-05-12T04:01:10.517 INFO:tasks.workunit.client.1.smithi033.stdout:0/430: chown d6/db/f1a 23064 1 2024-05-12T04:01:10.519 INFO:tasks.workunit.client.0.smithi032.stdout:1/208: dwrite d1/d7/d30/d18/f1b [0,4194304] 0 2024-05-12T04:01:10.522 INFO:tasks.workunit.client.0.smithi032.stdout:2/168: dwrite d1/f4 [0,4194304] 0 2024-05-12T04:01:10.525 INFO:tasks.workunit.client.0.smithi032.stdout:6/211: dread f3 [0,4194304] 0 2024-05-12T04:01:10.525 INFO:tasks.workunit.client.0.smithi032.stdout:2/169: dread - d1/d13/f19 zero size 2024-05-12T04:01:10.525 INFO:tasks.workunit.client.0.smithi032.stdout:1/209: write d1/d7/d30/d35/f42 [780393,111887] 0 2024-05-12T04:01:10.529 INFO:tasks.workunit.client.0.smithi032.stdout:1/210: dread - d1/d7/d30/d35/f38 zero size 2024-05-12T04:01:10.529 INFO:tasks.workunit.client.0.smithi032.stdout:2/170: write d1/d13/d25/f2a [3469109,25506] 0 2024-05-12T04:01:10.530 INFO:tasks.workunit.client.0.smithi032.stdout:1/211: stat d1/d7/d30/d35/f45 0 2024-05-12T04:01:10.530 INFO:tasks.workunit.client.0.smithi032.stdout:2/171: readlink d1/l35 0 2024-05-12T04:01:10.533 INFO:tasks.workunit.client.0.smithi032.stdout:2/172: write d1/d13/d25/f2a [4101128,115387] 0 2024-05-12T04:01:10.533 INFO:tasks.workunit.client.0.smithi032.stdout:1/212: chown d1/d7/d30/d18/f1a 39337 1 2024-05-12T04:01:10.535 INFO:tasks.workunit.client.0.smithi032.stdout:1/213: write d1/d7/d30/d18/f41 [944658,42431] 0 2024-05-12T04:01:10.536 INFO:tasks.workunit.client.0.smithi032.stdout:1/214: chown d1/dd/cf 100 1 2024-05-12T04:01:10.552 INFO:tasks.workunit.client.1.smithi033.stdout:1/354: dread d11/d17/d27/d36/f3d [0,4194304] 0 2024-05-12T04:01:10.561 INFO:tasks.workunit.client.0.smithi032.stdout:2/173: fdatasync d1/d13/d25/f2a 0 2024-05-12T04:01:10.566 INFO:tasks.workunit.client.0.smithi032.stdout:2/174: dwrite d1/fa [0,4194304] 0 2024-05-12T04:01:10.594 INFO:tasks.workunit.client.0.smithi032.stdout:0/267: truncate df/d30/d36/f41 2270047 0 2024-05-12T04:01:10.595 INFO:tasks.workunit.client.0.smithi032.stdout:9/159: fdatasync d0/d3/d6/f30 0 2024-05-12T04:01:10.596 INFO:tasks.workunit.client.0.smithi032.stdout:0/268: chown df/d30/d36/c53 37 1 2024-05-12T04:01:10.598 INFO:tasks.workunit.client.0.smithi032.stdout:3/249: truncate d15/d33/f46 1068200 0 2024-05-12T04:01:10.605 INFO:tasks.workunit.client.1.smithi033.stdout:6/393: creat d4/d1a/d6c/f85 x:0 0 0 2024-05-12T04:01:10.606 INFO:tasks.workunit.client.1.smithi033.stdout:6/394: readlink d4/d43/d57/l6b 0 2024-05-12T04:01:10.607 INFO:tasks.workunit.client.1.smithi033.stdout:6/395: write d4/d5/f41 [4280424,41368] 0 2024-05-12T04:01:10.611 INFO:tasks.workunit.client.1.smithi033.stdout:0/431: dread - d6/dc/f5e zero size 2024-05-12T04:01:10.612 INFO:tasks.workunit.client.1.smithi033.stdout:5/325: write d3/d12/d25/d26/d2e/f3e [319527,40232] 0 2024-05-12T04:01:10.613 INFO:tasks.workunit.client.1.smithi033.stdout:4/393: dwrite dd/d11/d31/d66/f2d [0,4194304] 0 2024-05-12T04:01:10.614 INFO:tasks.workunit.client.1.smithi033.stdout:1/355: symlink d11/d17/d27/d36/d3f/l78 0 2024-05-12T04:01:10.616 INFO:tasks.workunit.client.1.smithi033.stdout:5/326: chown d3/d12/d25/d26/d2e/d54/d57/f5f 107504296 1 2024-05-12T04:01:10.617 INFO:tasks.workunit.client.0.smithi032.stdout:5/257: creat d4/f59 x:0 0 0 2024-05-12T04:01:10.621 INFO:tasks.workunit.client.1.smithi033.stdout:2/397: dwrite dc/d13/f1b [0,4194304] 0 2024-05-12T04:01:10.622 INFO:tasks.workunit.client.0.smithi032.stdout:5/258: write d4/d7/d14/d26/d33/d4a/f4f [415451,88389] 0 2024-05-12T04:01:10.623 INFO:tasks.workunit.client.0.smithi032.stdout:4/158: mknod d0/c25 0 2024-05-12T04:01:10.624 INFO:tasks.workunit.client.0.smithi032.stdout:8/247: creat d0/d7/f4a x:0 0 0 2024-05-12T04:01:10.626 INFO:tasks.workunit.client.0.smithi032.stdout:8/248: chown d0/d9/d1b/d33/f3a 451422 1 2024-05-12T04:01:10.628 INFO:tasks.workunit.client.1.smithi033.stdout:0/432: creat d6/dc/d39/f8d x:0 0 0 2024-05-12T04:01:10.629 INFO:tasks.workunit.client.0.smithi032.stdout:4/159: dread d0/d3/f1c [4194304,4194304] 0 2024-05-12T04:01:10.631 INFO:tasks.workunit.client.1.smithi033.stdout:9/612: truncate d2/dd/f11 2592565 0 2024-05-12T04:01:10.631 INFO:tasks.workunit.client.1.smithi033.stdout:9/613: readlink d2/l1e 0 2024-05-12T04:01:10.631 INFO:tasks.workunit.client.0.smithi032.stdout:1/215: truncate d1/f36 296210 0 2024-05-12T04:01:10.632 INFO:tasks.workunit.client.1.smithi033.stdout:9/614: write d2/d4/d2b/d78/d76/fc9 [483496,84393] 0 2024-05-12T04:01:10.633 INFO:tasks.workunit.client.1.smithi033.stdout:4/394: mkdir dd/d11/d31/d38/d57/d7e 0 2024-05-12T04:01:10.634 INFO:tasks.workunit.client.0.smithi032.stdout:2/175: mknod d1/d13/d3d/c3f 0 2024-05-12T04:01:10.636 INFO:tasks.workunit.client.0.smithi032.stdout:1/216: dread d1/f26 [0,4194304] 0 2024-05-12T04:01:10.636 INFO:tasks.workunit.client.1.smithi033.stdout:5/327: creat d3/d15/d16/d45/d62/f64 x:0 0 0 2024-05-12T04:01:10.637 INFO:tasks.workunit.client.1.smithi033.stdout:5/328: fdatasync d3/f3b 0 2024-05-12T04:01:10.638 INFO:tasks.workunit.client.0.smithi032.stdout:6/212: truncate d7/de/f14 1709468 0 2024-05-12T04:01:10.638 INFO:tasks.workunit.client.0.smithi032.stdout:6/213: dread - d7/de/d12/f22 zero size 2024-05-12T04:01:10.639 INFO:tasks.workunit.client.1.smithi033.stdout:4/395: dwrite dd/d11/d31/f45 [0,4194304] 0 2024-05-12T04:01:10.640 INFO:tasks.workunit.client.1.smithi033.stdout:2/398: symlink dc/d13/d1f/d6e/l8a 0 2024-05-12T04:01:10.641 INFO:tasks.workunit.client.1.smithi033.stdout:2/399: fsync dc/d17/d27/d47/d5e/f77 0 2024-05-12T04:01:10.642 INFO:tasks.workunit.client.1.smithi033.stdout:2/400: chown dc/d13/d1f/f24 3560367 1 2024-05-12T04:01:10.645 INFO:tasks.workunit.client.1.smithi033.stdout:0/433: rmdir d6/dc 39 2024-05-12T04:01:10.646 INFO:tasks.workunit.client.1.smithi033.stdout:7/548: sync 2024-05-12T04:01:10.647 INFO:tasks.workunit.client.1.smithi033.stdout:1/356: mknod d11/d17/d27/d36/c79 0 2024-05-12T04:01:10.649 INFO:tasks.workunit.client.1.smithi033.stdout:1/357: write d11/d17/d27/d36/d46/f51 [1889128,119310] 0 2024-05-12T04:01:10.652 INFO:tasks.workunit.client.1.smithi033.stdout:9/615: mknod d2/dd/d39/cdf 0 2024-05-12T04:01:10.653 INFO:tasks.workunit.client.1.smithi033.stdout:9/616: chown d2/f8e 1003 1 2024-05-12T04:01:10.657 INFO:tasks.workunit.client.1.smithi033.stdout:1/358: dwrite d11/d17/f20 [0,4194304] 0 2024-05-12T04:01:10.660 INFO:tasks.workunit.client.1.smithi033.stdout:5/329: symlink d3/d15/l65 0 2024-05-12T04:01:10.662 INFO:tasks.workunit.client.0.smithi032.stdout:8/249: dwrite d0/f5 [0,4194304] 0 2024-05-12T04:01:10.664 INFO:tasks.workunit.client.1.smithi033.stdout:4/396: stat dd/c26 0 2024-05-12T04:01:10.665 INFO:tasks.workunit.client.0.smithi032.stdout:1/217: mkdir d1/d7/d49 0 2024-05-12T04:01:10.666 INFO:tasks.workunit.client.1.smithi033.stdout:6/396: truncate d4/d1a/d67/d68/f7b 782149 0 2024-05-12T04:01:10.671 INFO:tasks.workunit.client.1.smithi033.stdout:4/397: dwrite fa [4194304,4194304] 0 2024-05-12T04:01:10.671 INFO:tasks.workunit.client.0.smithi032.stdout:1/218: write d1/d7/d30/f28 [257248,122020] 0 2024-05-12T04:01:10.673 INFO:tasks.workunit.client.0.smithi032.stdout:5/259: write d4/fd [5342507,47606] 0 2024-05-12T04:01:10.675 INFO:tasks.workunit.client.1.smithi033.stdout:7/549: mkdir de/df/d7a/dc0 0 2024-05-12T04:01:10.676 INFO:tasks.workunit.client.1.smithi033.stdout:4/398: dread - dd/d69/f7d zero size 2024-05-12T04:01:10.676 INFO:tasks.workunit.client.0.smithi032.stdout:3/250: creat d15/f55 x:0 0 0 2024-05-12T04:01:10.677 INFO:tasks.workunit.client.1.smithi033.stdout:4/399: write dd/f4b [5016134,29487] 0 2024-05-12T04:01:10.678 INFO:tasks.workunit.client.1.smithi033.stdout:3/375: sync 2024-05-12T04:01:10.678 INFO:tasks.workunit.client.1.smithi033.stdout:8/309: sync 2024-05-12T04:01:10.678 INFO:tasks.workunit.client.1.smithi033.stdout:3/376: dread - db/d59/f74 zero size 2024-05-12T04:01:10.679 INFO:tasks.workunit.client.1.smithi033.stdout:3/377: stat db/d17/d3a/d67/d6d 0 2024-05-12T04:01:10.683 INFO:tasks.workunit.client.1.smithi033.stdout:5/330: dread d3/d12/d25/d26/d2e/f49 [0,4194304] 0 2024-05-12T04:01:10.685 INFO:tasks.workunit.client.1.smithi033.stdout:1/359: creat d11/d17/d27/d36/d3f/d4a/f7a x:0 0 0 2024-05-12T04:01:10.691 INFO:tasks.workunit.client.1.smithi033.stdout:1/360: dwrite fd [0,4194304] 0 2024-05-12T04:01:10.692 INFO:tasks.workunit.client.1.smithi033.stdout:1/361: chown d11/d17/d27/d36/d3f/f50 678 1 2024-05-12T04:01:10.692 INFO:tasks.workunit.client.1.smithi033.stdout:6/397: rmdir d4/d1a/d67 39 2024-05-12T04:01:10.693 INFO:tasks.workunit.client.0.smithi032.stdout:2/176: dwrite d1/f24 [0,4194304] 0 2024-05-12T04:01:10.693 INFO:tasks.workunit.client.0.smithi032.stdout:0/269: getdents df 0 2024-05-12T04:01:10.698 INFO:tasks.workunit.client.0.smithi032.stdout:5/260: rename d4/d7/d14/d26/d40 to d4/d7/d58/d5a 0 2024-05-12T04:01:10.698 INFO:tasks.workunit.client.0.smithi032.stdout:5/261: chown f1 1075982051 1 2024-05-12T04:01:10.700 INFO:tasks.workunit.client.1.smithi033.stdout:0/434: dread - d6/dc/f5e zero size 2024-05-12T04:01:10.702 INFO:tasks.workunit.client.0.smithi032.stdout:4/160: write d0/f13 [455580,98356] 0 2024-05-12T04:01:10.706 INFO:tasks.workunit.client.1.smithi033.stdout:7/550: rename c6 to de/d20/da9/cc1 0 2024-05-12T04:01:10.709 INFO:tasks.workunit.client.0.smithi032.stdout:8/250: mkdir d0/d4b 0 2024-05-12T04:01:10.709 INFO:tasks.workunit.client.1.smithi033.stdout:4/400: readlink dd/d11/d31/d66/l30 0 2024-05-12T04:01:10.710 INFO:tasks.workunit.client.1.smithi033.stdout:4/401: dread - dd/f62 zero size 2024-05-12T04:01:10.710 INFO:tasks.workunit.client.1.smithi033.stdout:8/310: mknod d1/d16/d27/d28/c62 0 2024-05-12T04:01:10.711 INFO:tasks.workunit.client.1.smithi033.stdout:4/402: write dd/d11/d31/d66/f49 [1591971,22309] 0 2024-05-12T04:01:10.714 INFO:tasks.workunit.client.1.smithi033.stdout:9/617: link d2/d4/d2b/fdd d2/d4/d2b/d6d/d4c/fe0 0 2024-05-12T04:01:10.715 INFO:tasks.workunit.client.1.smithi033.stdout:9/618: truncate d2/d4/d2b/d78/f88 4594159 0 2024-05-12T04:01:10.716 INFO:tasks.workunit.client.0.smithi032.stdout:1/219: link d1/d25/f34 d1/d25/f4a 0 2024-05-12T04:01:10.716 INFO:tasks.workunit.client.1.smithi033.stdout:5/331: chown d3/d12/d25/d26/c47 90843373 1 2024-05-12T04:01:10.717 INFO:tasks.workunit.client.0.smithi032.stdout:2/177: creat d1/d13/d25/f40 x:0 0 0 2024-05-12T04:01:10.717 INFO:tasks.workunit.client.1.smithi033.stdout:9/619: write d2/d4/d29/d3c/d52/fba [252123,58333] 0 2024-05-12T04:01:10.722 INFO:tasks.workunit.client.1.smithi033.stdout:5/332: dwrite d3/f40 [0,4194304] 0 2024-05-12T04:01:10.722 INFO:tasks.workunit.client.1.smithi033.stdout:2/401: link dc/f31 dc/d17/d3a/f8b 0 2024-05-12T04:01:10.725 INFO:tasks.workunit.client.1.smithi033.stdout:0/435: dread d6/dc/d58/d6a/d80/f84 [0,4194304] 0 2024-05-12T04:01:10.730 INFO:tasks.workunit.client.0.smithi032.stdout:1/220: fdatasync d1/d7/d30/d18/f1a 0 2024-05-12T04:01:10.733 INFO:tasks.workunit.client.0.smithi032.stdout:0/270: dread fa [4194304,4194304] 0 2024-05-12T04:01:10.735 INFO:tasks.workunit.client.0.smithi032.stdout:1/221: dread d1/d7/d30/d35/f42 [0,4194304] 0 2024-05-12T04:01:10.736 INFO:tasks.workunit.client.0.smithi032.stdout:5/262: mknod d4/c5b 0 2024-05-12T04:01:10.737 INFO:tasks.workunit.client.1.smithi033.stdout:3/378: mknod db/d17/d3e/d65/d7d/c82 0 2024-05-12T04:01:10.738 INFO:tasks.workunit.client.0.smithi032.stdout:2/178: rename d1/d13/d25 to d1/d13/d3d/d41 0 2024-05-12T04:01:10.745 INFO:tasks.workunit.client.1.smithi033.stdout:7/551: dwrite de/d20/f83 [0,4194304] 0 2024-05-12T04:01:10.746 INFO:tasks.workunit.client.0.smithi032.stdout:2/179: dwrite d1/f7 [0,4194304] 0 2024-05-12T04:01:10.747 INFO:tasks.workunit.client.0.smithi032.stdout:2/180: stat d1/l17 0 2024-05-12T04:01:10.750 INFO:tasks.workunit.client.0.smithi032.stdout:7/230: sync 2024-05-12T04:01:10.750 INFO:tasks.workunit.client.1.smithi033.stdout:9/620: symlink d2/d4/d2b/d6d/db5/le1 0 2024-05-12T04:01:10.754 INFO:tasks.workunit.client.1.smithi033.stdout:6/398: link d4/d1a/d3c/f65 d4/d1a/d6c/f86 0 2024-05-12T04:01:10.755 INFO:tasks.workunit.client.1.smithi033.stdout:7/552: dread de/df/d17/d1b/d36/f8a [0,4194304] 0 2024-05-12T04:01:10.755 INFO:tasks.workunit.client.1.smithi033.stdout:6/399: chown d4/d43/d57/l6b 518 1 2024-05-12T04:01:10.757 INFO:tasks.workunit.client.1.smithi033.stdout:7/553: write de/df/d17/d1b/d36/f4a [5170201,84861] 0 2024-05-12T04:01:10.758 INFO:tasks.workunit.client.0.smithi032.stdout:8/251: dwrite d0/d9/f1c [0,4194304] 0 2024-05-12T04:01:10.760 INFO:tasks.workunit.client.1.smithi033.stdout:5/333: rename d3/d12/d29/l33 to d3/d12/d25/d26/d2e/l66 0 2024-05-12T04:01:10.761 INFO:tasks.workunit.client.1.smithi033.stdout:7/554: write de/df/d17/d1e/f23 [287186,91938] 0 2024-05-12T04:01:10.761 INFO:tasks.workunit.client.0.smithi032.stdout:0/271: rename df/l24 to df/d10/d48/l61 0 2024-05-12T04:01:10.763 INFO:tasks.workunit.client.0.smithi032.stdout:3/251: getdents d15/d20/d22/d4f 0 2024-05-12T04:01:10.764 INFO:tasks.workunit.client.0.smithi032.stdout:3/252: chown d15/d20/d26/f52 3 1 2024-05-12T04:01:10.765 INFO:tasks.workunit.client.0.smithi032.stdout:3/253: read d15/d24/f3d [312019,91840] 0 2024-05-12T04:01:10.766 INFO:tasks.workunit.client.0.smithi032.stdout:3/254: dread - d15/d24/f3e zero size 2024-05-12T04:01:10.766 INFO:tasks.workunit.client.0.smithi032.stdout:3/255: dread - d15/d1b/f2f zero size 2024-05-12T04:01:10.767 INFO:tasks.workunit.client.1.smithi033.stdout:2/402: dwrite dc/d17/d3a/f6d [4194304,4194304] 0 2024-05-12T04:01:10.767 INFO:tasks.workunit.client.0.smithi032.stdout:3/256: write d15/f44 [748956,36284] 0 2024-05-12T04:01:10.771 INFO:tasks.workunit.client.1.smithi033.stdout:3/379: mknod db/d17/d3a/c83 0 2024-05-12T04:01:10.772 INFO:tasks.workunit.client.0.smithi032.stdout:7/231: mknod dd/d1b/d1e/c53 0 2024-05-12T04:01:10.773 INFO:tasks.workunit.client.0.smithi032.stdout:1/222: getdents d1/d7/d30/d35/d46 0 2024-05-12T04:01:10.774 INFO:tasks.workunit.client.0.smithi032.stdout:1/223: dread - d1/f47 zero size 2024-05-12T04:01:10.775 INFO:tasks.workunit.client.1.smithi033.stdout:2/403: dwrite f9 [0,4194304] 0 2024-05-12T04:01:10.777 INFO:tasks.workunit.client.1.smithi033.stdout:6/400: unlink d4/d5/c7e 0 2024-05-12T04:01:10.781 INFO:tasks.workunit.client.1.smithi033.stdout:9/621: rename d2/d4/d29/f2a to d2/d4/d2b/d33/fe2 0 2024-05-12T04:01:10.783 INFO:tasks.workunit.client.1.smithi033.stdout:9/622: dread d2/d4/d2b/f4b [0,4194304] 0 2024-05-12T04:01:10.784 INFO:tasks.workunit.client.1.smithi033.stdout:5/334: creat d3/d15/d16/d45/d62/f67 x:0 0 0 2024-05-12T04:01:10.785 INFO:tasks.workunit.client.1.smithi033.stdout:5/335: readlink d3/l4d 0 2024-05-12T04:01:10.785 INFO:tasks.workunit.client.0.smithi032.stdout:3/257: creat d15/d33/f56 x:0 0 0 2024-05-12T04:01:10.794 INFO:tasks.workunit.client.0.smithi032.stdout:8/252: mkdir d0/d9/d1b/d27/d4c 0 2024-05-12T04:01:10.795 INFO:tasks.workunit.client.1.smithi033.stdout:4/403: creat dd/d11/d31/d66/f7f x:0 0 0 2024-05-12T04:01:10.796 INFO:tasks.workunit.client.1.smithi033.stdout:8/311: dwrite d1/f2 [4194304,4194304] 0 2024-05-12T04:01:10.797 INFO:tasks.workunit.client.1.smithi033.stdout:3/380: symlink db/d17/d3a/l84 0 2024-05-12T04:01:10.797 INFO:tasks.workunit.client.0.smithi032.stdout:5/263: truncate d4/d7/d14/d26/d33/d4a/f51 285819 0 2024-05-12T04:01:10.798 INFO:tasks.workunit.client.0.smithi032.stdout:5/264: readlink d4/d7/l2d 0 2024-05-12T04:01:10.806 INFO:tasks.workunit.client.1.smithi033.stdout:6/401: read d4/d1a/d4e/d4f/f50 [50293,110759] 0 2024-05-12T04:01:10.809 INFO:tasks.workunit.client.0.smithi032.stdout:7/232: dread dd/f36 [0,4194304] 0 2024-05-12T04:01:10.810 INFO:tasks.workunit.client.0.smithi032.stdout:8/253: creat d0/d9/d1b/f4d x:0 0 0 2024-05-12T04:01:10.811 INFO:tasks.workunit.client.1.smithi033.stdout:9/623: mkdir d2/d70/d7b/d94/de3 0 2024-05-12T04:01:10.812 INFO:tasks.workunit.client.0.smithi032.stdout:2/181: getdents d1/d34 0 2024-05-12T04:01:10.814 INFO:tasks.workunit.client.1.smithi033.stdout:6/402: dwrite d4/d5/f2d [0,4194304] 0 2024-05-12T04:01:10.814 INFO:tasks.workunit.client.0.smithi032.stdout:5/265: symlink d4/d7/d14/d26/l5c 0 2024-05-12T04:01:10.815 INFO:tasks.workunit.client.1.smithi033.stdout:5/336: write d3/fc [2623525,43297] 0 2024-05-12T04:01:10.816 INFO:tasks.workunit.client.0.smithi032.stdout:0/272: symlink df/d30/d32/d60/l62 0 2024-05-12T04:01:10.818 INFO:tasks.workunit.client.1.smithi033.stdout:4/404: symlink dd/d11/d31/d38/l80 0 2024-05-12T04:01:10.818 INFO:tasks.workunit.client.1.smithi033.stdout:0/436: dwrite d6/d11/d57/f71 [0,4194304] 0 2024-05-12T04:01:10.819 INFO:tasks.workunit.client.0.smithi032.stdout:2/182: dwrite d1/f4 [0,4194304] 0 2024-05-12T04:01:10.821 INFO:tasks.workunit.client.0.smithi032.stdout:2/183: dread - d1/d13/d3d/d41/d2c/f37 zero size 2024-05-12T04:01:10.822 INFO:tasks.workunit.client.0.smithi032.stdout:7/233: rmdir dd/d19 39 2024-05-12T04:01:10.823 INFO:tasks.workunit.client.1.smithi033.stdout:6/403: dread d4/d1a/d6c/f86 [0,4194304] 0 2024-05-12T04:01:10.825 INFO:tasks.workunit.client.1.smithi033.stdout:5/337: dread d3/f40 [4194304,4194304] 0 2024-05-12T04:01:10.826 INFO:tasks.workunit.client.1.smithi033.stdout:8/312: mkdir d1/d16/d29/d63 0 2024-05-12T04:01:10.826 INFO:tasks.workunit.client.0.smithi032.stdout:1/224: truncate d1/f3 8267299 0 2024-05-12T04:01:10.829 INFO:tasks.workunit.client.0.smithi032.stdout:8/254: rmdir d0/d9/d1b/d33 39 2024-05-12T04:01:10.830 INFO:tasks.workunit.client.1.smithi033.stdout:6/404: dread d4/d5/f41 [0,4194304] 0 2024-05-12T04:01:10.832 INFO:tasks.workunit.client.0.smithi032.stdout:8/255: fsync d0/d9/d13/f24 0 2024-05-12T04:01:10.833 INFO:tasks.workunit.client.0.smithi032.stdout:7/234: dwrite dd/d1b/d1e/f38 [0,4194304] 0 2024-05-12T04:01:10.838 INFO:tasks.workunit.client.0.smithi032.stdout:5/266: symlink d4/d7/d14/d39/d1b/l5d 0 2024-05-12T04:01:10.841 INFO:tasks.workunit.client.1.smithi033.stdout:9/624: symlink d2/d4/d2b/d6d/d3a/le4 0 2024-05-12T04:01:10.845 INFO:tasks.workunit.client.1.smithi033.stdout:1/362: sync 2024-05-12T04:01:10.845 INFO:tasks.workunit.client.1.smithi033.stdout:9/625: dread d2/d4/d29/d32/db7/fbf [0,4194304] 0 2024-05-12T04:01:10.848 INFO:tasks.workunit.client.0.smithi032.stdout:5/267: dwrite d4/f59 [0,4194304] 0 2024-05-12T04:01:10.850 INFO:tasks.workunit.client.0.smithi032.stdout:8/256: rmdir d0/d9/d13/d26/d28 39 2024-05-12T04:01:10.850 INFO:tasks.workunit.client.0.smithi032.stdout:8/257: stat d0/d9/d13/d21/d35 0 2024-05-12T04:01:10.852 INFO:tasks.workunit.client.0.smithi032.stdout:2/184: symlink d1/d13/d3d/d41/d39/l42 0 2024-05-12T04:01:10.856 INFO:tasks.workunit.client.0.smithi032.stdout:1/225: creat d1/d7/d30/d35/d46/f4b x:0 0 0 2024-05-12T04:01:10.857 INFO:tasks.workunit.client.0.smithi032.stdout:1/226: fdatasync d1/d7/d30/f28 0 2024-05-12T04:01:10.857 INFO:tasks.workunit.client.1.smithi033.stdout:3/381: write db/d17/f2d [1045644,62580] 0 2024-05-12T04:01:10.863 INFO:tasks.workunit.client.1.smithi033.stdout:3/382: dwrite db/d17/d25/d32/f5d [0,4194304] 0 2024-05-12T04:01:10.863 INFO:tasks.workunit.client.1.smithi033.stdout:8/313: mkdir d1/d4/d1d/d36/d3c/d64 0 2024-05-12T04:01:10.865 INFO:tasks.workunit.client.1.smithi033.stdout:8/314: write d1/d16/d29/f30 [918026,63543] 0 2024-05-12T04:01:10.865 INFO:tasks.workunit.client.0.smithi032.stdout:7/235: creat dd/d19/d37/f54 x:0 0 0 2024-05-12T04:01:10.867 INFO:tasks.workunit.client.1.smithi033.stdout:8/315: truncate d1/d16/d27/d4d/f5e 795556 0 2024-05-12T04:01:10.873 INFO:tasks.workunit.client.0.smithi032.stdout:7/236: dread dd/d19/f34 [0,4194304] 0 2024-05-12T04:01:10.873 INFO:tasks.workunit.client.1.smithi033.stdout:8/316: dwrite d1/d4/fd [4194304,4194304] 0 2024-05-12T04:01:10.874 INFO:tasks.workunit.client.1.smithi033.stdout:8/317: fdatasync d1/d16/d27/d4d/d51/f4b 0 2024-05-12T04:01:10.876 INFO:tasks.workunit.client.1.smithi033.stdout:8/318: write d1/d16/f19 [3551541,68575] 0 2024-05-12T04:01:10.881 INFO:tasks.workunit.client.0.smithi032.stdout:0/273: write df/d10/f1b [1406884,65292] 0 2024-05-12T04:01:10.886 INFO:tasks.workunit.client.1.smithi033.stdout:4/405: rename dd/d11/d31/d66/d42/d59/l64 to dd/d11/d31/d38/l81 0 2024-05-12T04:01:10.886 INFO:tasks.workunit.client.1.smithi033.stdout:4/406: fdatasync dd/d69/f7d 0 2024-05-12T04:01:10.887 INFO:tasks.workunit.client.1.smithi033.stdout:4/407: chown dd/d11/d31/d66/d42/d59/l5e 286 1 2024-05-12T04:01:10.888 INFO:tasks.workunit.client.1.smithi033.stdout:4/408: chown dd/c73 112 1 2024-05-12T04:01:10.889 INFO:tasks.workunit.client.0.smithi032.stdout:7/237: mkdir dd/d1b/d1e/d45/d55 0 2024-05-12T04:01:10.891 INFO:tasks.workunit.client.0.smithi032.stdout:8/258: mkdir d0/d9/d1b/d27/d4c/d4e 0 2024-05-12T04:01:10.893 INFO:tasks.workunit.client.1.smithi033.stdout:9/626: write d2/d4/f1f [3152755,48857] 0 2024-05-12T04:01:10.895 INFO:tasks.workunit.client.1.smithi033.stdout:0/437: write d6/dc/d6e/f29 [187455,14772] 0 2024-05-12T04:01:10.897 INFO:tasks.workunit.client.0.smithi032.stdout:2/185: dwrite d1/f21 [0,4194304] 0 2024-05-12T04:01:10.897 INFO:tasks.workunit.client.0.smithi032.stdout:2/186: chown d1/d34/f32 400169 1 2024-05-12T04:01:10.898 INFO:tasks.workunit.client.0.smithi032.stdout:1/227: rename d1/d7/d30/c1f to d1/d7/d30/d35/c4c 0 2024-05-12T04:01:10.899 INFO:tasks.workunit.client.0.smithi032.stdout:2/187: dread - d1/d13/d3d/d41/f40 zero size 2024-05-12T04:01:10.900 INFO:tasks.workunit.client.0.smithi032.stdout:2/188: chown d1/c1a 218755521 1 2024-05-12T04:01:10.901 INFO:tasks.workunit.client.0.smithi032.stdout:0/274: chown ce 879618473 1 2024-05-12T04:01:10.902 INFO:tasks.workunit.client.1.smithi033.stdout:2/404: getdents dc/d17/d27/d2f 0 2024-05-12T04:01:10.906 INFO:tasks.workunit.client.0.smithi032.stdout:5/268: dwrite d4/f22 [0,4194304] 0 2024-05-12T04:01:10.907 INFO:tasks.workunit.client.1.smithi033.stdout:8/319: unlink d1/d16/d29/f30 0 2024-05-12T04:01:10.909 INFO:tasks.workunit.client.0.smithi032.stdout:7/238: rename c8 to dd/d1b/c56 0 2024-05-12T04:01:10.910 INFO:tasks.workunit.client.0.smithi032.stdout:8/259: rename d0/d9/d13 to d0/d9/d13/d26/d28/d3e/d4f 22 2024-05-12T04:01:10.913 INFO:tasks.workunit.client.0.smithi032.stdout:2/189: unlink d1/ff 0 2024-05-12T04:01:10.917 INFO:tasks.workunit.client.0.smithi032.stdout:5/269: creat d4/d7/d14/d26/f5e x:0 0 0 2024-05-12T04:01:10.919 INFO:tasks.workunit.client.0.smithi032.stdout:7/239: unlink dd/d19/d37/l4a 0 2024-05-12T04:01:10.919 INFO:tasks.workunit.client.0.smithi032.stdout:5/270: fdatasync d4/d7/d14/d39/d1b/f23 0 2024-05-12T04:01:10.919 INFO:tasks.workunit.client.0.smithi032.stdout:8/260: dwrite d0/d9/d13/d21/d35/d37/f40 [0,4194304] 0 2024-05-12T04:01:10.920 INFO:tasks.workunit.client.1.smithi033.stdout:7/555: sync 2024-05-12T04:01:10.920 INFO:tasks.workunit.client.1.smithi033.stdout:0/438: symlink d6/dc/d39/l8e 0 2024-05-12T04:01:10.921 INFO:tasks.workunit.client.0.smithi032.stdout:8/261: read d0/d9/f1c [2487813,10981] 0 2024-05-12T04:01:10.928 INFO:tasks.workunit.client.1.smithi033.stdout:2/405: rename dc/d17/d3a/c7a to dc/d3b/d88/c8c 0 2024-05-12T04:01:10.929 INFO:tasks.workunit.client.1.smithi033.stdout:2/406: readlink dc/d3b/l45 0 2024-05-12T04:01:10.929 INFO:tasks.workunit.client.1.smithi033.stdout:2/407: chown dc/d13/d1f/d6e/c6f 1783645227 1 2024-05-12T04:01:10.930 INFO:tasks.workunit.client.0.smithi032.stdout:2/190: mknod d1/d13/d3d/d41/d39/c43 0 2024-05-12T04:01:10.931 INFO:tasks.workunit.client.0.smithi032.stdout:7/240: dwrite dd/d19/f3e [0,4194304] 0 2024-05-12T04:01:10.931 INFO:tasks.workunit.client.0.smithi032.stdout:2/191: readlink d1/l35 0 2024-05-12T04:01:10.934 INFO:tasks.workunit.client.0.smithi032.stdout:5/271: dwrite d4/d7/d14/d26/d33/d4a/f4f [0,4194304] 0 2024-05-12T04:01:10.941 INFO:tasks.workunit.client.1.smithi033.stdout:8/320: creat d1/d16/d20/f65 x:0 0 0 2024-05-12T04:01:10.941 INFO:tasks.workunit.client.1.smithi033.stdout:8/321: stat d1/d16/d20/f65 0 2024-05-12T04:01:10.944 INFO:tasks.workunit.client.1.smithi033.stdout:4/409: truncate dd/f13 6355502 0 2024-05-12T04:01:10.947 INFO:tasks.workunit.client.0.smithi032.stdout:5/272: dwrite d4/d7/d14/d26/d33/d4a/d38/f45 [0,4194304] 0 2024-05-12T04:01:10.948 INFO:tasks.workunit.client.1.smithi033.stdout:8/322: dwrite d1/d4/d1d/f26 [0,4194304] 0 2024-05-12T04:01:10.951 INFO:tasks.workunit.client.1.smithi033.stdout:5/338: getdents d3/d12/d25/d26/d2e/d54 0 2024-05-12T04:01:10.952 INFO:tasks.workunit.client.0.smithi032.stdout:5/273: fdatasync d4/d7/d14/d26/d33/d4a/f4e 0 2024-05-12T04:01:10.953 INFO:tasks.workunit.client.1.smithi033.stdout:4/410: dwrite dd/f4b [4194304,4194304] 0 2024-05-12T04:01:10.955 INFO:tasks.workunit.client.0.smithi032.stdout:0/275: dwrite f5 [0,4194304] 0 2024-05-12T04:01:10.958 INFO:tasks.workunit.client.0.smithi032.stdout:7/241: stat dd/d19/c26 0 2024-05-12T04:01:10.959 INFO:tasks.workunit.client.0.smithi032.stdout:7/242: fsync f5 0 2024-05-12T04:01:10.959 INFO:tasks.workunit.client.1.smithi033.stdout:7/556: chown de/fa1 654149 1 2024-05-12T04:01:10.960 INFO:tasks.workunit.client.0.smithi032.stdout:7/243: chown dd/d1b/d1e/f38 16615833 1 2024-05-12T04:01:10.960 INFO:tasks.workunit.client.0.smithi032.stdout:2/192: mknod d1/d13/d3d/c44 0 2024-05-12T04:01:10.961 INFO:tasks.workunit.client.0.smithi032.stdout:7/244: chown dd/c3b 1818 1 2024-05-12T04:01:10.962 INFO:tasks.workunit.client.1.smithi033.stdout:4/411: dwrite dd/f4b [0,4194304] 0 2024-05-12T04:01:10.962 INFO:tasks.workunit.client.0.smithi032.stdout:7/245: fsync dd/d19/d37/f39 0 2024-05-12T04:01:10.963 INFO:tasks.workunit.client.1.smithi033.stdout:2/408: write dc/d13/d1f/d6e/f81 [1064040,98911] 0 2024-05-12T04:01:10.965 INFO:tasks.workunit.client.1.smithi033.stdout:1/363: getdents d11/d17 0 2024-05-12T04:01:10.967 INFO:tasks.workunit.client.1.smithi033.stdout:1/364: dread - d11/d17/d27/d36/d3f/d4a/f7a zero size 2024-05-12T04:01:10.968 INFO:tasks.workunit.client.1.smithi033.stdout:9/627: write d2/d4/d2b/d6d/d4c/f95 [4986088,118592] 0 2024-05-12T04:01:10.968 INFO:tasks.workunit.client.0.smithi032.stdout:1/228: link d1/f36 d1/d25/f4d 0 2024-05-12T04:01:10.975 INFO:tasks.workunit.client.1.smithi033.stdout:3/383: getdents db/d17/d4d 0 2024-05-12T04:01:10.976 INFO:tasks.workunit.client.0.smithi032.stdout:1/229: dread d1/d7/d30/d18/f41 [0,4194304] 0 2024-05-12T04:01:10.977 INFO:tasks.workunit.client.1.smithi033.stdout:8/323: truncate d1/d4/f1c 1749624 0 2024-05-12T04:01:10.981 INFO:tasks.workunit.client.0.smithi032.stdout:5/274: mkdir d4/d7/d14/d26/d33/d5f 0 2024-05-12T04:01:10.984 INFO:tasks.workunit.client.1.smithi033.stdout:8/324: dwrite d1/d16/d20/f2f [0,4194304] 0 2024-05-12T04:01:10.987 INFO:tasks.workunit.client.1.smithi033.stdout:0/439: mknod d6/db/d2c/d73/c8f 0 2024-05-12T04:01:10.988 INFO:tasks.workunit.client.1.smithi033.stdout:0/440: dread - d6/dc/d58/f88 zero size 2024-05-12T04:01:10.989 INFO:tasks.workunit.client.0.smithi032.stdout:7/246: dread dd/d19/f34 [0,4194304] 0 2024-05-12T04:01:10.989 INFO:tasks.workunit.client.1.smithi033.stdout:6/405: getdents d4/d1a/d67 0 2024-05-12T04:01:10.992 INFO:tasks.workunit.client.0.smithi032.stdout:3/258: sync 2024-05-12T04:01:10.992 INFO:tasks.workunit.client.0.smithi032.stdout:9/160: sync 2024-05-12T04:01:10.992 INFO:tasks.workunit.client.0.smithi032.stdout:4/161: sync 2024-05-12T04:01:10.992 INFO:tasks.workunit.client.0.smithi032.stdout:6/214: sync 2024-05-12T04:01:10.994 INFO:tasks.workunit.client.0.smithi032.stdout:3/259: chown d15/f1c 27444 1 2024-05-12T04:01:10.996 INFO:tasks.workunit.client.1.smithi033.stdout:6/406: dwrite d4/d1a/d4e/d4f/f63 [0,4194304] 0 2024-05-12T04:01:10.998 INFO:tasks.workunit.client.0.smithi032.stdout:6/215: write d7/de/d12/d1a/d20/f42 [190785,123542] 0 2024-05-12T04:01:11.001 INFO:tasks.workunit.client.0.smithi032.stdout:8/262: dwrite d0/f2f [0,4194304] 0 2024-05-12T04:01:11.003 INFO:tasks.workunit.client.0.smithi032.stdout:8/263: dread - d0/d9/f31 zero size 2024-05-12T04:01:11.003 INFO:tasks.workunit.client.0.smithi032.stdout:3/260: read d15/d24/f51 [87298,129874] 0 2024-05-12T04:01:11.004 INFO:tasks.workunit.client.0.smithi032.stdout:0/276: write df/d10/d3d/d4a/f29 [66142,89207] 0 2024-05-12T04:01:11.006 INFO:tasks.workunit.client.0.smithi032.stdout:8/264: write d0/d9/d1b/f4d [682549,127383] 0 2024-05-12T04:01:11.006 INFO:tasks.workunit.client.0.smithi032.stdout:0/277: readlink df/d10/d3b/l45 0 2024-05-12T04:01:11.009 INFO:tasks.workunit.client.1.smithi033.stdout:9/628: symlink d2/d4/d29/d3c/le5 0 2024-05-12T04:01:11.014 INFO:tasks.workunit.client.1.smithi033.stdout:0/441: dread d6/db/d2c/f3e [0,4194304] 0 2024-05-12T04:01:11.014 INFO:tasks.workunit.client.1.smithi033.stdout:3/384: unlink db/c70 0 2024-05-12T04:01:11.021 INFO:tasks.workunit.client.0.smithi032.stdout:7/247: rename dd/c3b to dd/d1b/d1e/c57 0 2024-05-12T04:01:11.022 INFO:tasks.workunit.client.1.smithi033.stdout:0/442: dwrite d6/dc/d39/f76 [0,4194304] 0 2024-05-12T04:01:11.026 INFO:tasks.workunit.client.1.smithi033.stdout:8/325: rename d1/d16/d20/c45 to d1/d16/d27/d4d/c66 0 2024-05-12T04:01:11.028 INFO:tasks.workunit.client.1.smithi033.stdout:4/412: dwrite dd/d11/d31/f72 [0,4194304] 0 2024-05-12T04:01:11.028 INFO:tasks.workunit.client.1.smithi033.stdout:8/326: truncate d1/d16/d20/f65 318017 0 2024-05-12T04:01:11.029 INFO:tasks.workunit.client.0.smithi032.stdout:4/162: creat d0/d3/df/f26 x:0 0 0 2024-05-12T04:01:11.030 INFO:tasks.workunit.client.1.smithi033.stdout:6/407: fsync d4/d1a/f23 0 2024-05-12T04:01:11.032 INFO:tasks.workunit.client.0.smithi032.stdout:5/275: write d4/d7/d14/d26/d33/d4a/f4f [4912449,72934] 0 2024-05-12T04:01:11.032 INFO:tasks.workunit.client.1.smithi033.stdout:1/365: write d11/d17/f3e [130365,166] 0 2024-05-12T04:01:11.036 INFO:tasks.workunit.client.0.smithi032.stdout:3/261: rmdir d15/d20/d26 39 2024-05-12T04:01:11.037 INFO:tasks.workunit.client.0.smithi032.stdout:1/230: dwrite d1/d25/f4d [0,4194304] 0 2024-05-12T04:01:11.037 INFO:tasks.workunit.client.1.smithi033.stdout:3/385: symlink db/d17/d25/d32/d53/l85 0 2024-05-12T04:01:11.041 INFO:tasks.workunit.client.0.smithi032.stdout:8/265: fsync d0/d9/d1b/d33/f45 0 2024-05-12T04:01:11.041 INFO:tasks.workunit.client.1.smithi033.stdout:5/339: creat d3/d15/d16/d45/f68 x:0 0 0 2024-05-12T04:01:11.042 INFO:tasks.workunit.client.1.smithi033.stdout:5/340: chown d3/d12/d25/d26/d2e/d54/c58 4321 1 2024-05-12T04:01:11.045 INFO:tasks.workunit.client.0.smithi032.stdout:2/193: truncate d1/fa 3546432 0 2024-05-12T04:01:11.047 INFO:tasks.workunit.client.0.smithi032.stdout:2/194: chown d1/d13/f30 14426014 1 2024-05-12T04:01:11.048 INFO:tasks.workunit.client.0.smithi032.stdout:6/216: dwrite d7/de/d12/f45 [0,4194304] 0 2024-05-12T04:01:11.051 INFO:tasks.workunit.client.1.smithi033.stdout:7/557: creat de/fc2 x:0 0 0 2024-05-12T04:01:11.054 INFO:tasks.workunit.client.0.smithi032.stdout:7/248: rmdir dd 39 2024-05-12T04:01:11.056 INFO:tasks.workunit.client.1.smithi033.stdout:0/443: creat d6/d11/d57/f90 x:0 0 0 2024-05-12T04:01:11.056 INFO:tasks.workunit.client.0.smithi032.stdout:7/249: write f2 [4921733,93754] 0 2024-05-12T04:01:11.057 INFO:tasks.workunit.client.0.smithi032.stdout:2/195: dwrite d1/d13/f30 [0,4194304] 0 2024-05-12T04:01:11.059 INFO:tasks.workunit.client.0.smithi032.stdout:4/163: dread d0/f11 [0,4194304] 0 2024-05-12T04:01:11.061 INFO:tasks.workunit.client.0.smithi032.stdout:2/196: write d1/d13/d3d/d41/f40 [221527,44896] 0 2024-05-12T04:01:11.064 INFO:tasks.workunit.client.0.smithi032.stdout:5/276: symlink d4/d7/d14/d26/d33/d4a/d38/d3b/l60 0 2024-05-12T04:01:11.066 INFO:tasks.workunit.client.0.smithi032.stdout:5/277: chown d4/fd 13 1 2024-05-12T04:01:11.067 INFO:tasks.workunit.client.0.smithi032.stdout:1/231: creat d1/d7/d30/d35/f4e x:0 0 0 2024-05-12T04:01:11.067 INFO:tasks.workunit.client.1.smithi033.stdout:8/327: mknod d1/d16/d27/d4d/d51/c67 0 2024-05-12T04:01:11.070 INFO:tasks.workunit.client.1.smithi033.stdout:6/408: creat d4/f87 x:0 0 0 2024-05-12T04:01:11.070 INFO:tasks.workunit.client.1.smithi033.stdout:2/409: creat dc/f8d x:0 0 0 2024-05-12T04:01:11.070 INFO:tasks.workunit.client.0.smithi032.stdout:1/232: chown d1/d7/d30/d18/l19 8723 1 2024-05-12T04:01:11.071 INFO:tasks.workunit.client.1.smithi033.stdout:9/629: creat d2/d4/d2b/dc7/fe6 x:0 0 0 2024-05-12T04:01:11.071 INFO:tasks.workunit.client.1.smithi033.stdout:8/328: fdatasync d1/d16/d27/d28/f57 0 2024-05-12T04:01:11.072 INFO:tasks.workunit.client.0.smithi032.stdout:2/197: dwrite d1/d13/f2f [0,4194304] 0 2024-05-12T04:01:11.074 INFO:tasks.workunit.client.0.smithi032.stdout:1/233: read d1/d7/d30/d35/f45 [1468998,91114] 0 2024-05-12T04:01:11.075 INFO:tasks.workunit.client.0.smithi032.stdout:8/266: creat d0/d9/d13/d21/d25/f50 x:0 0 0 2024-05-12T04:01:11.077 INFO:tasks.workunit.client.1.smithi033.stdout:7/558: creat de/df/d60/fc3 x:0 0 0 2024-05-12T04:01:11.077 INFO:tasks.workunit.client.0.smithi032.stdout:1/234: write d1/d25/f4d [2952646,73343] 0 2024-05-12T04:01:11.078 INFO:tasks.workunit.client.1.smithi033.stdout:2/410: dwrite dc/d13/d1f/f24 [0,4194304] 0 2024-05-12T04:01:11.081 INFO:tasks.workunit.client.0.smithi032.stdout:0/278: truncate df/d10/f3c 981827 0 2024-05-12T04:01:11.083 INFO:tasks.workunit.client.0.smithi032.stdout:6/217: mkdir d7/d9/d48 0 2024-05-12T04:01:11.084 INFO:tasks.workunit.client.0.smithi032.stdout:6/218: chown d7/de/d12/d1a/l2d 127 1 2024-05-12T04:01:11.086 INFO:tasks.workunit.client.0.smithi032.stdout:6/219: write d7/de/d12/f23 [1383588,38229] 0 2024-05-12T04:01:11.088 INFO:tasks.workunit.client.1.smithi033.stdout:6/409: mkdir d4/d1a/d67/d68/d88 0 2024-05-12T04:01:11.088 INFO:tasks.workunit.client.1.smithi033.stdout:7/559: dwrite de/d20/f96 [0,4194304] 0 2024-05-12T04:01:11.090 INFO:tasks.workunit.client.1.smithi033.stdout:7/560: truncate f3 4355524 0 2024-05-12T04:01:11.090 INFO:tasks.workunit.client.1.smithi033.stdout:6/410: chown d4/d1a/d67/c7a 30 1 2024-05-12T04:01:11.091 INFO:tasks.workunit.client.1.smithi033.stdout:9/630: dread d2/d4/f2c [0,4194304] 0 2024-05-12T04:01:11.094 INFO:tasks.workunit.client.1.smithi033.stdout:1/366: link d11/d17/f6d d11/d1c/d59/f7b 0 2024-05-12T04:01:11.101 INFO:tasks.workunit.client.1.smithi033.stdout:0/444: mkdir d6/dc/d58/d91 0 2024-05-12T04:01:11.105 INFO:tasks.workunit.client.1.smithi033.stdout:4/413: rename dd/d11/d31/d66/f2e to dd/d11/d31/d66/d42/f82 0 2024-05-12T04:01:11.106 INFO:tasks.workunit.client.0.smithi032.stdout:5/278: creat d4/d7/d14/d26/d33/f61 x:0 0 0 2024-05-12T04:01:11.107 INFO:tasks.workunit.client.0.smithi032.stdout:2/198: rmdir d1/d13/d3d/d41 39 2024-05-12T04:01:11.108 INFO:tasks.workunit.client.1.smithi033.stdout:8/329: fsync d1/d4/f1c 0 2024-05-12T04:01:11.109 INFO:tasks.workunit.client.1.smithi033.stdout:0/445: dwrite d6/dc/d39/f8a [0,4194304] 0 2024-05-12T04:01:11.110 INFO:tasks.workunit.client.1.smithi033.stdout:8/330: chown d1/d16/f19 1012549779 1 2024-05-12T04:01:11.111 INFO:tasks.workunit.client.1.smithi033.stdout:7/561: stat de/d2b/d6c/c77 0 2024-05-12T04:01:11.113 INFO:tasks.workunit.client.1.smithi033.stdout:7/562: write de/df/d17/f51 [2376638,7920] 0 2024-05-12T04:01:11.114 INFO:tasks.workunit.client.1.smithi033.stdout:9/631: mknod d2/ce7 0 2024-05-12T04:01:11.115 INFO:tasks.workunit.client.1.smithi033.stdout:2/411: write dc/f31 [1588871,27150] 0 2024-05-12T04:01:11.116 INFO:tasks.workunit.client.0.smithi032.stdout:2/199: dread d1/f12 [0,4194304] 0 2024-05-12T04:01:11.121 INFO:tasks.workunit.client.0.smithi032.stdout:4/164: dwrite d0/fa [0,4194304] 0 2024-05-12T04:01:11.122 INFO:tasks.workunit.client.0.smithi032.stdout:0/279: truncate df/d10/d3d/f56 629378 0 2024-05-12T04:01:11.122 INFO:tasks.workunit.client.1.smithi033.stdout:2/412: dwrite dc/d17/d27/d2f/d49/f51 [0,4194304] 0 2024-05-12T04:01:11.124 INFO:tasks.workunit.client.1.smithi033.stdout:2/413: dread - dc/d17/d27/d47/d5e/f77 zero size 2024-05-12T04:01:11.124 INFO:tasks.workunit.client.1.smithi033.stdout:5/341: rename d3/d15/d16/l3c to d3/d27/l69 0 2024-05-12T04:01:11.124 INFO:tasks.workunit.client.0.smithi032.stdout:9/161: getdents d0/d3/d1b 0 2024-05-12T04:01:11.125 INFO:tasks.workunit.client.0.smithi032.stdout:9/162: dread - d0/d3/d1b/f24 zero size 2024-05-12T04:01:11.126 INFO:tasks.workunit.client.1.smithi033.stdout:2/414: write dc/d17/d3a/f8b [464104,88799] 0 2024-05-12T04:01:11.133 INFO:tasks.workunit.client.1.smithi033.stdout:2/415: dwrite dc/d17/d27/d2f/d49/d59/f79 [0,4194304] 0 2024-05-12T04:01:11.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:10 smithi033 ceph-mon[49525]: pgmap v33: 65 pgs: 65 active+clean; 970 MiB data, 4.4 GiB used, 532 GiB / 536 GiB avail; 13 MiB/s rd, 106 MiB/s wr, 175 op/s 2024-05-12T04:01:11.195 INFO:tasks.workunit.client.1.smithi033.stdout:4/414: unlink dd/d11/c24 0 2024-05-12T04:01:11.196 INFO:tasks.workunit.client.1.smithi033.stdout:4/415: chown dd/c26 16964 1 2024-05-12T04:01:11.200 INFO:tasks.workunit.client.1.smithi033.stdout:0/446: read d6/dc/f16 [2417926,112534] 0 2024-05-12T04:01:11.201 INFO:tasks.workunit.client.1.smithi033.stdout:1/367: dwrite d11/d1c/d59/f7b [0,4194304] 0 2024-05-12T04:01:11.203 INFO:tasks.workunit.client.1.smithi033.stdout:7/563: readlink de/df/d17/l33 0 2024-05-12T04:01:11.205 INFO:tasks.workunit.client.1.smithi033.stdout:9/632: fdatasync d2/d4/d2b/d6d/d7a/f90 0 2024-05-12T04:01:11.212 INFO:tasks.workunit.client.0.smithi032.stdout:8/267: symlink d0/l51 0 2024-05-12T04:01:11.212 INFO:tasks.workunit.client.0.smithi032.stdout:8/268: chown d0/d9/f1c 639537 1 2024-05-12T04:01:11.213 INFO:tasks.workunit.client.1.smithi033.stdout:3/386: sync 2024-05-12T04:01:11.213 INFO:tasks.workunit.client.0.smithi032.stdout:1/235: creat d1/d7/d49/f4f x:0 0 0 2024-05-12T04:01:11.214 INFO:tasks.workunit.client.1.smithi033.stdout:3/387: stat db/ce 0 2024-05-12T04:01:11.214 INFO:tasks.workunit.client.0.smithi032.stdout:8/269: write d0/d9/d13/f29 [1435122,68571] 0 2024-05-12T04:01:11.218 INFO:tasks.workunit.client.0.smithi032.stdout:4/165: symlink d0/d3/df/l27 0 2024-05-12T04:01:11.218 INFO:tasks.workunit.client.1.smithi033.stdout:6/411: rename d4/d1a/f3e to d4/d1a/d4e/f89 0 2024-05-12T04:01:11.220 INFO:tasks.workunit.client.0.smithi032.stdout:4/166: truncate d0/d3/df/f26 909129 0 2024-05-12T04:01:11.221 INFO:tasks.workunit.client.0.smithi032.stdout:9/163: unlink d0/d3/d6/d7/da/c19 0 2024-05-12T04:01:11.222 INFO:tasks.workunit.client.0.smithi032.stdout:8/270: dwrite d0/d9/d13/d26/d28/f2b [4194304,4194304] 0 2024-05-12T04:01:11.223 INFO:tasks.workunit.client.1.smithi033.stdout:4/416: mknod dd/d11/d31/d38/c83 0 2024-05-12T04:01:11.223 INFO:tasks.workunit.client.0.smithi032.stdout:9/164: chown d0/d3/d6/d7/d27/f31 20416 1 2024-05-12T04:01:11.228 INFO:tasks.workunit.client.0.smithi032.stdout:6/220: rename d7/l3f to d7/de/d12/l49 0 2024-05-12T04:01:11.230 INFO:tasks.workunit.client.0.smithi032.stdout:7/250: link dd/d19/d37/f54 dd/d1b/d43/f58 0 2024-05-12T04:01:11.232 INFO:tasks.workunit.client.0.smithi032.stdout:3/262: getdents d15/d24 0 2024-05-12T04:01:11.233 INFO:tasks.workunit.client.0.smithi032.stdout:5/279: truncate d4/f20 628591 0 2024-05-12T04:01:11.237 INFO:tasks.workunit.client.1.smithi033.stdout:9/633: symlink d2/d4/d2b/d6d/db5/le8 0 2024-05-12T04:01:11.243 INFO:tasks.workunit.client.1.smithi033.stdout:3/388: creat db/d17/d3e/d65/d7d/f86 x:0 0 0 2024-05-12T04:01:11.244 INFO:tasks.workunit.client.1.smithi033.stdout:3/389: truncate db/d17/d25/f40 4980730 0 2024-05-12T04:01:11.244 INFO:tasks.workunit.client.0.smithi032.stdout:0/280: truncate df/d10/d3d/d4a/f2d 291627 0 2024-05-12T04:01:11.246 INFO:tasks.workunit.client.1.smithi033.stdout:5/342: rename d3/f40 to d3/d27/f6a 0 2024-05-12T04:01:11.247 INFO:tasks.workunit.client.1.smithi033.stdout:5/343: chown d3/d12/d29/d2b/l3a 472 1 2024-05-12T04:01:11.247 INFO:tasks.workunit.client.0.smithi032.stdout:4/167: rename d0/d3/c22 to d0/d3/c28 0 2024-05-12T04:01:11.248 INFO:tasks.workunit.client.1.smithi033.stdout:3/390: dread db/d17/f64 [0,4194304] 0 2024-05-12T04:01:11.249 INFO:tasks.workunit.client.1.smithi033.stdout:3/391: write db/d17/d25/d32/d3c/d3d/f72 [4455708,11041] 0 2024-05-12T04:01:11.250 INFO:tasks.workunit.client.0.smithi032.stdout:4/168: write d0/f13 [2197867,10313] 0 2024-05-12T04:01:11.250 INFO:tasks.workunit.client.1.smithi033.stdout:8/331: truncate d1/d4/d1d/f26 4122918 0 2024-05-12T04:01:11.251 INFO:tasks.workunit.client.0.smithi032.stdout:4/169: readlink d0/d3/df/l10 0 2024-05-12T04:01:11.252 INFO:tasks.workunit.client.0.smithi032.stdout:6/221: fsync d7/de/d12/f16 0 2024-05-12T04:01:11.254 INFO:tasks.workunit.client.1.smithi033.stdout:4/417: mkdir dd/d11/d5b/d84 0 2024-05-12T04:01:11.256 INFO:tasks.workunit.client.0.smithi032.stdout:5/280: symlink d4/d7/d14/d26/d33/d4a/l62 0 2024-05-12T04:01:11.261 INFO:tasks.workunit.client.0.smithi032.stdout:0/281: symlink df/d30/d4d/l63 0 2024-05-12T04:01:11.262 INFO:tasks.workunit.client.0.smithi032.stdout:0/282: truncate fd 702677 0 2024-05-12T04:01:11.263 INFO:tasks.workunit.client.1.smithi033.stdout:9/634: creat d2/d4/d29/d3c/fe9 x:0 0 0 2024-05-12T04:01:11.263 INFO:tasks.workunit.client.1.smithi033.stdout:2/416: truncate dc/d13/d1c/f4f 5489985 0 2024-05-12T04:01:11.274 INFO:tasks.workunit.client.1.smithi033.stdout:8/332: symlink d1/d16/d27/d4d/l68 0 2024-05-12T04:01:11.277 INFO:tasks.workunit.client.0.smithi032.stdout:6/222: unlink d7/de/d12/d25/c40 0 2024-05-12T04:01:11.279 INFO:tasks.workunit.client.1.smithi033.stdout:4/418: creat dd/d7a/f85 x:0 0 0 2024-05-12T04:01:11.279 INFO:tasks.workunit.client.1.smithi033.stdout:8/333: dwrite d1/d4/fd [4194304,4194304] 0 2024-05-12T04:01:11.280 INFO:tasks.workunit.client.0.smithi032.stdout:4/170: write d0/d3/f1c [2605108,45510] 0 2024-05-12T04:01:11.284 INFO:tasks.workunit.client.0.smithi032.stdout:0/283: creat df/d10/f64 x:0 0 0 2024-05-12T04:01:11.285 INFO:tasks.workunit.client.1.smithi033.stdout:3/392: write db/d17/f36 [2187099,11184] 0 2024-05-12T04:01:11.287 INFO:tasks.workunit.client.1.smithi033.stdout:9/635: rename d2/d4/d2b/d78/d76/ld1 to d2/d4/d2b/d6d/d2f/d5f/lea 0 2024-05-12T04:01:11.288 INFO:tasks.workunit.client.1.smithi033.stdout:9/636: read d2/d4/d29/d32/db7/fbf [3583671,47222] 0 2024-05-12T04:01:11.289 INFO:tasks.workunit.client.1.smithi033.stdout:9/637: chown d2/d4/d2b/d49/f56 0 1 2024-05-12T04:01:11.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:10 smithi032 ceph-mon[40008]: pgmap v33: 65 pgs: 65 active+clean; 970 MiB data, 4.4 GiB used, 532 GiB / 536 GiB avail; 13 MiB/s rd, 106 MiB/s wr, 175 op/s 2024-05-12T04:01:11.289 INFO:tasks.workunit.client.0.smithi032.stdout:1/236: rename d1/dd/d24/l2d to d1/dd/l50 0 2024-05-12T04:01:11.291 INFO:tasks.workunit.client.0.smithi032.stdout:1/237: read - d1/d7/d30/d35/f48 zero size 2024-05-12T04:01:11.293 INFO:tasks.workunit.client.0.smithi032.stdout:6/223: creat d7/d9/f4a x:0 0 0 2024-05-12T04:01:11.294 INFO:tasks.workunit.client.0.smithi032.stdout:6/224: chown d7/de/f3b 75971543 1 2024-05-12T04:01:11.295 INFO:tasks.workunit.client.0.smithi032.stdout:5/281: symlink d4/d7/d14/d26/d33/d5f/l63 0 2024-05-12T04:01:11.296 INFO:tasks.workunit.client.0.smithi032.stdout:1/238: dread d1/d7/d30/d18/f1b [0,4194304] 0 2024-05-12T04:01:11.297 INFO:tasks.workunit.client.0.smithi032.stdout:1/239: chown d1/d25/c27 97 1 2024-05-12T04:01:11.298 INFO:tasks.workunit.client.1.smithi033.stdout:0/447: rename d6/dc/l4b to d6/dc/l92 0 2024-05-12T04:01:11.299 INFO:tasks.workunit.client.0.smithi032.stdout:4/171: creat d0/d3/df/f29 x:0 0 0 2024-05-12T04:01:11.301 INFO:tasks.workunit.client.1.smithi033.stdout:0/448: dread d6/dc/d39/f40 [0,4194304] 0 2024-05-12T04:01:11.301 INFO:tasks.workunit.client.1.smithi033.stdout:1/368: getdents d11/d17/d27/d36/d3f 0 2024-05-12T04:01:11.302 INFO:tasks.workunit.client.0.smithi032.stdout:0/284: creat df/d10/d3d/d4a/f65 x:0 0 0 2024-05-12T04:01:11.302 INFO:tasks.workunit.client.0.smithi032.stdout:5/282: dwrite d4/d7/d14/d26/d33/f61 [0,4194304] 0 2024-05-12T04:01:11.304 INFO:tasks.workunit.client.0.smithi032.stdout:5/283: readlink d4/d7/d14/d26/d33/d4a/d38/d3b/d4c/l4d 0 2024-05-12T04:01:11.305 INFO:tasks.workunit.client.0.smithi032.stdout:5/284: readlink d4/d7/d14/d39/l36 0 2024-05-12T04:01:11.306 INFO:tasks.workunit.client.0.smithi032.stdout:5/285: chown d4/d7/l2d 2072225 1 2024-05-12T04:01:11.308 INFO:tasks.workunit.client.1.smithi033.stdout:5/344: dwrite d3/f52 [0,4194304] 0 2024-05-12T04:01:11.364 INFO:tasks.workunit.client.1.smithi033.stdout:7/564: sync 2024-05-12T04:01:11.364 INFO:tasks.workunit.client.1.smithi033.stdout:6/412: sync 2024-05-12T04:01:11.365 INFO:tasks.workunit.client.1.smithi033.stdout:7/565: dread de/df/d60/d75/f8f [0,4194304] 0 2024-05-12T04:01:11.403 INFO:tasks.workunit.client.1.smithi033.stdout:4/419: rename dd/d11/d31/d66/d42/f55 to dd/d11/d5b/f86 0 2024-05-12T04:01:11.404 INFO:tasks.workunit.client.0.smithi032.stdout:3/263: getdents d15/d1b/d43 0 2024-05-12T04:01:11.406 INFO:tasks.workunit.client.0.smithi032.stdout:6/225: creat d7/de/f4b x:0 0 0 2024-05-12T04:01:11.411 INFO:tasks.workunit.client.0.smithi032.stdout:6/226: dwrite d7/d9/f4a [0,4194304] 0 2024-05-12T04:01:11.427 INFO:tasks.workunit.client.1.smithi033.stdout:9/638: creat d2/d4/feb x:0 0 0 2024-05-12T04:01:11.428 INFO:tasks.workunit.client.0.smithi032.stdout:1/240: symlink d1/d7/d49/l51 0 2024-05-12T04:01:11.429 INFO:tasks.workunit.client.1.smithi033.stdout:9/639: read d2/d4/d2b/d78/d76/fc9 [140502,2524] 0 2024-05-12T04:01:11.431 INFO:tasks.workunit.client.1.smithi033.stdout:3/393: write db/d17/d3a/f3b [192090,127351] 0 2024-05-12T04:01:11.433 INFO:tasks.workunit.client.0.smithi032.stdout:9/165: rename d0/d3/d6/d7/da/d17/c21 to d0/d3/d6/d7/d2a/c36 0 2024-05-12T04:01:11.445 INFO:tasks.workunit.client.0.smithi032.stdout:3/264: write d15/d20/d22/f2e [542346,45700] 0 2024-05-12T04:01:11.446 INFO:tasks.workunit.client.1.smithi033.stdout:5/345: mknod d3/d15/d16/d45/d4b/c6b 0 2024-05-12T04:01:11.447 INFO:tasks.workunit.client.0.smithi032.stdout:3/265: truncate d15/d20/d22/f23 803768 0 2024-05-12T04:01:11.447 INFO:tasks.workunit.client.0.smithi032.stdout:3/266: chown d15/d33/c41 214815887 1 2024-05-12T04:01:11.451 INFO:tasks.workunit.client.0.smithi032.stdout:9/166: rename d0/l2f to d0/d3/d6/d7/d2a/d34/l37 0 2024-05-12T04:01:11.452 INFO:tasks.workunit.client.0.smithi032.stdout:6/227: write d7/fa [2148373,42495] 0 2024-05-12T04:01:11.453 INFO:tasks.workunit.client.1.smithi033.stdout:7/566: truncate de/df/d17/f34 800369 0 2024-05-12T04:01:11.455 INFO:tasks.workunit.client.1.smithi033.stdout:6/413: dread d4/d1a/d4e/f66 [0,4194304] 0 2024-05-12T04:01:11.455 INFO:tasks.workunit.client.0.smithi032.stdout:1/241: dwrite d1/d7/d30/d35/f45 [0,4194304] 0 2024-05-12T04:01:11.456 INFO:tasks.workunit.client.1.smithi033.stdout:6/414: write d4/d5/f5c [723327,35030] 0 2024-05-12T04:01:11.458 INFO:tasks.workunit.client.0.smithi032.stdout:6/228: dread f6 [0,4194304] 0 2024-05-12T04:01:11.460 INFO:tasks.workunit.client.0.smithi032.stdout:6/229: write d7/de/f1b [3140432,64108] 0 2024-05-12T04:01:11.464 INFO:tasks.workunit.client.0.smithi032.stdout:5/286: link d4/d7/d14/d26/d33/d4a/d3d/c52 d4/d7/d14/d26/d33/d4a/c64 0 2024-05-12T04:01:11.467 INFO:tasks.workunit.client.0.smithi032.stdout:2/200: sync 2024-05-12T04:01:11.468 INFO:tasks.workunit.client.0.smithi032.stdout:8/271: sync 2024-05-12T04:01:11.468 INFO:tasks.workunit.client.0.smithi032.stdout:7/251: sync 2024-05-12T04:01:11.468 INFO:tasks.workunit.client.0.smithi032.stdout:8/272: read d0/f5 [2776201,96508] 0 2024-05-12T04:01:11.469 INFO:tasks.workunit.client.0.smithi032.stdout:1/242: rmdir d1/d7 39 2024-05-12T04:01:11.481 INFO:tasks.workunit.client.0.smithi032.stdout:6/230: rename d7/de/f3b to d7/de/d12/f4c 0 2024-05-12T04:01:11.483 INFO:tasks.workunit.client.0.smithi032.stdout:5/287: creat d4/d7/d14/d26/d47/f65 x:0 0 0 2024-05-12T04:01:11.492 INFO:tasks.workunit.client.0.smithi032.stdout:8/273: creat d0/d9/d13/d21/d25/f52 x:0 0 0 2024-05-12T04:01:11.492 INFO:tasks.workunit.client.0.smithi032.stdout:3/267: dwrite d15/d20/d26/f3c [0,4194304] 0 2024-05-12T04:01:11.500 INFO:tasks.workunit.client.0.smithi032.stdout:9/167: truncate d0/f28 1506889 0 2024-05-12T04:01:11.504 INFO:tasks.workunit.client.0.smithi032.stdout:7/252: rename dd/d19/f2a to dd/d1b/d43/f59 0 2024-05-12T04:01:11.506 INFO:tasks.workunit.client.0.smithi032.stdout:7/253: write f2 [5265283,101212] 0 2024-05-12T04:01:11.507 INFO:tasks.workunit.client.0.smithi032.stdout:5/288: creat d4/d7/d14/d26/f66 x:0 0 0 2024-05-12T04:01:11.508 INFO:tasks.workunit.client.0.smithi032.stdout:9/168: dwrite d0/d3/d6/d7/da/d17/f1e [4194304,4194304] 0 2024-05-12T04:01:11.509 INFO:tasks.workunit.client.0.smithi032.stdout:5/289: stat d4/d7/d14/d26/d33 0 2024-05-12T04:01:11.514 INFO:tasks.workunit.client.0.smithi032.stdout:2/201: creat d1/d34/d3e/f45 x:0 0 0 2024-05-12T04:01:11.515 INFO:tasks.workunit.client.0.smithi032.stdout:8/274: symlink d0/d9/d13/d21/d35/d37/l53 0 2024-05-12T04:01:11.516 INFO:tasks.workunit.client.0.smithi032.stdout:2/202: dread - d1/f31 zero size 2024-05-12T04:01:11.518 INFO:tasks.workunit.client.0.smithi032.stdout:2/203: chown d1/fc 0 1 2024-05-12T04:01:11.519 INFO:tasks.workunit.client.0.smithi032.stdout:3/268: dread d15/d33/f46 [0,4194304] 0 2024-05-12T04:01:11.521 INFO:tasks.workunit.client.0.smithi032.stdout:5/290: dwrite d4/fd [4194304,4194304] 0 2024-05-12T04:01:11.524 INFO:tasks.workunit.client.0.smithi032.stdout:5/291: read d4/d7/d14/d26/d33/d4a/d38/d3b/f48 [58094,5202] 0 2024-05-12T04:01:11.526 INFO:tasks.workunit.client.0.smithi032.stdout:3/269: dread d15/d20/d22/f23 [0,4194304] 0 2024-05-12T04:01:11.530 INFO:tasks.workunit.client.1.smithi033.stdout:8/334: creat d1/d4/d1d/d36/f69 x:0 0 0 2024-05-12T04:01:11.532 INFO:tasks.workunit.client.1.smithi033.stdout:8/335: read - d1/d4/d1d/d36/d3c/d47/f60 zero size 2024-05-12T04:01:11.533 INFO:tasks.workunit.client.1.smithi033.stdout:8/336: write d1/d16/d27/f2d [4704755,116666] 0 2024-05-12T04:01:11.533 INFO:tasks.workunit.client.0.smithi032.stdout:7/254: creat dd/d19/f5a x:0 0 0 2024-05-12T04:01:11.539 INFO:tasks.workunit.client.0.smithi032.stdout:9/169: creat d0/d3/d1b/f38 x:0 0 0 2024-05-12T04:01:11.541 INFO:tasks.workunit.client.0.smithi032.stdout:8/275: mknod d0/d7/c54 0 2024-05-12T04:01:11.542 INFO:tasks.workunit.client.0.smithi032.stdout:8/276: chown d0/d7/f15 3 1 2024-05-12T04:01:11.542 INFO:tasks.workunit.client.1.smithi033.stdout:2/417: link dc/d13/d1f/l2d dc/d17/d27/l8e 0 2024-05-12T04:01:11.543 INFO:tasks.workunit.client.1.smithi033.stdout:3/394: dread db/d17/f2c [0,4194304] 0 2024-05-12T04:01:11.543 INFO:tasks.workunit.client.1.smithi033.stdout:2/418: write dc/d17/d27/d2f/d49/f85 [25226,159] 0 2024-05-12T04:01:11.544 INFO:tasks.workunit.client.1.smithi033.stdout:5/346: creat d3/d15/d16/d45/d4b/f6c x:0 0 0 2024-05-12T04:01:11.547 INFO:tasks.workunit.client.1.smithi033.stdout:7/567: mkdir de/df/d17/d1e/dc4 0 2024-05-12T04:01:11.548 INFO:tasks.workunit.client.1.smithi033.stdout:7/568: chown de/df/d60/lb8 6604034 1 2024-05-12T04:01:11.549 INFO:tasks.workunit.client.0.smithi032.stdout:2/204: dwrite d1/d13/d3d/d41/f40 [0,4194304] 0 2024-05-12T04:01:11.549 INFO:tasks.workunit.client.1.smithi033.stdout:8/337: dread d1/d16/d20/f33 [0,4194304] 0 2024-05-12T04:01:11.550 INFO:tasks.workunit.client.0.smithi032.stdout:1/243: write d1/d7/d30/d18/f41 [835421,44131] 0 2024-05-12T04:01:11.553 INFO:tasks.workunit.client.1.smithi033.stdout:9/640: write d2/d70/f87 [170061,59225] 0 2024-05-12T04:01:11.554 INFO:tasks.workunit.client.1.smithi033.stdout:9/641: chown d2/d4/d2b/d6d/d3a 29357735 1 2024-05-12T04:01:11.557 INFO:tasks.workunit.client.0.smithi032.stdout:5/292: creat d4/d7/d14/d26/d33/d4a/d38/d3b/f67 x:0 0 0 2024-05-12T04:01:11.557 INFO:tasks.workunit.client.1.smithi033.stdout:9/642: read d2/d4/d2b/d78/f93 [3187415,115529] 0 2024-05-12T04:01:11.558 INFO:tasks.workunit.client.0.smithi032.stdout:5/293: write d4/d7/d14/d26/f57 [80037,3178] 0 2024-05-12T04:01:11.560 INFO:tasks.workunit.client.0.smithi032.stdout:3/270: creat d15/d20/d22/f57 x:0 0 0 2024-05-12T04:01:11.561 INFO:tasks.workunit.client.1.smithi033.stdout:0/449: truncate d6/db/d2c/f31 2710611 0 2024-05-12T04:01:11.564 INFO:tasks.workunit.client.0.smithi032.stdout:3/271: write d15/d20/d22/f2e [8697462,115030] 0 2024-05-12T04:01:11.567 INFO:tasks.workunit.client.0.smithi032.stdout:5/294: dwrite d4/f22 [4194304,4194304] 0 2024-05-12T04:01:11.571 INFO:tasks.workunit.client.1.smithi033.stdout:6/415: write d4/d1a/d4e/d4f/f50 [1222451,8315] 0 2024-05-12T04:01:11.575 INFO:tasks.workunit.client.1.smithi033.stdout:3/395: creat db/d17/d25/d32/d53/f87 x:0 0 0 2024-05-12T04:01:11.577 INFO:tasks.workunit.client.0.smithi032.stdout:2/205: symlink d1/d13/d3d/d41/d2c/l46 0 2024-05-12T04:01:11.578 INFO:tasks.workunit.client.1.smithi033.stdout:2/419: chown dc/d13/d1c/f4f 735091602 1 2024-05-12T04:01:11.579 INFO:tasks.workunit.client.1.smithi033.stdout:2/420: chown dc/d13/d1c/l20 128938670 1 2024-05-12T04:01:11.580 INFO:tasks.workunit.client.0.smithi032.stdout:1/244: rmdir d1/d25 39 2024-05-12T04:01:11.581 INFO:tasks.workunit.client.1.smithi033.stdout:5/347: dread - d3/d12/d29/f38 zero size 2024-05-12T04:01:11.584 INFO:tasks.workunit.client.1.smithi033.stdout:2/421: dread dc/d17/d27/d2f/f75 [0,4194304] 0 2024-05-12T04:01:11.584 INFO:tasks.workunit.client.1.smithi033.stdout:7/569: rmdir de/df/d17/d1b/d36 39 2024-05-12T04:01:11.585 INFO:tasks.workunit.client.0.smithi032.stdout:7/255: dwrite dd/f36 [0,4194304] 0 2024-05-12T04:01:11.587 INFO:tasks.workunit.client.0.smithi032.stdout:1/245: truncate d1/d7/d30/d18/f41 1719092 0 2024-05-12T04:01:11.588 INFO:tasks.workunit.client.0.smithi032.stdout:7/256: fsync dd/d19/f4d 0 2024-05-12T04:01:11.588 INFO:tasks.workunit.client.0.smithi032.stdout:2/206: dwrite d1/d13/f2b [0,4194304] 0 2024-05-12T04:01:11.593 INFO:tasks.workunit.client.1.smithi033.stdout:9/643: rmdir d2/d4/d2b/d78 39 2024-05-12T04:01:11.597 INFO:tasks.workunit.client.0.smithi032.stdout:3/272: creat d15/d20/d22/f58 x:0 0 0 2024-05-12T04:01:11.597 INFO:tasks.workunit.client.1.smithi033.stdout:0/450: creat d6/d11/d57/f93 x:0 0 0 2024-05-12T04:01:11.598 INFO:tasks.workunit.client.0.smithi032.stdout:3/273: chown d15/l16 2871 1 2024-05-12T04:01:11.599 INFO:tasks.workunit.client.0.smithi032.stdout:7/257: dwrite dd/d1b/d1e/f38 [0,4194304] 0 2024-05-12T04:01:11.599 INFO:tasks.workunit.client.0.smithi032.stdout:5/295: chown d4/f20 1389 1 2024-05-12T04:01:11.602 INFO:tasks.workunit.client.0.smithi032.stdout:7/258: truncate dd/d19/f4d 8745 0 2024-05-12T04:01:11.602 INFO:tasks.workunit.client.1.smithi033.stdout:1/369: getdents d11/d17/d27/d36/d3f 0 2024-05-12T04:01:11.603 INFO:tasks.workunit.client.0.smithi032.stdout:1/246: creat d1/d7/d30/f52 x:0 0 0 2024-05-12T04:01:11.603 INFO:tasks.workunit.client.1.smithi033.stdout:1/370: chown d11/d1c/d43/f45 4168157 1 2024-05-12T04:01:11.604 INFO:tasks.workunit.client.1.smithi033.stdout:3/396: mkdir db/d17/d3e/d65/d88 0 2024-05-12T04:01:11.605 INFO:tasks.workunit.client.0.smithi032.stdout:2/207: creat d1/d13/f47 x:0 0 0 2024-05-12T04:01:11.606 INFO:tasks.workunit.client.0.smithi032.stdout:4/172: sync 2024-05-12T04:01:11.606 INFO:tasks.workunit.client.0.smithi032.stdout:0/285: sync 2024-05-12T04:01:11.610 INFO:tasks.workunit.client.1.smithi033.stdout:5/348: symlink d3/d15/d16/d45/d4b/l6d 0 2024-05-12T04:01:11.613 INFO:tasks.workunit.client.0.smithi032.stdout:9/170: rename d0/d3/d6/d7/d2a/d34 to d0/d3/d6/d7/da/d39 0 2024-05-12T04:01:11.614 INFO:tasks.workunit.client.0.smithi032.stdout:9/171: write d0/d3/f32 [142263,59203] 0 2024-05-12T04:01:11.617 INFO:tasks.workunit.client.0.smithi032.stdout:8/277: link d0/d9/d1b/d33/f3a d0/d9/d13/d26/f55 0 2024-05-12T04:01:11.619 INFO:tasks.workunit.client.0.smithi032.stdout:3/274: mknod d15/d24/d2a/c59 0 2024-05-12T04:01:11.619 INFO:tasks.workunit.client.0.smithi032.stdout:3/275: stat d15/f4e 0 2024-05-12T04:01:11.620 INFO:tasks.workunit.client.1.smithi033.stdout:8/338: creat d1/d16/d29/d63/f6a x:0 0 0 2024-05-12T04:01:11.621 INFO:tasks.workunit.client.1.smithi033.stdout:8/339: fdatasync d1/d16/d29/d63/f6a 0 2024-05-12T04:01:11.621 INFO:tasks.workunit.client.0.smithi032.stdout:5/296: mkdir d4/d7/d14/d26/d33/d4a/d38/d3b/d68 0 2024-05-12T04:01:11.622 INFO:tasks.workunit.client.1.smithi033.stdout:8/340: fdatasync d1/d4/d1d/f58 0 2024-05-12T04:01:11.623 INFO:tasks.workunit.client.1.smithi033.stdout:8/341: read - d1/d4/d1d/d36/d3c/d47/f60 zero size 2024-05-12T04:01:11.625 INFO:tasks.workunit.client.0.smithi032.stdout:5/297: dread d4/f22 [0,4194304] 0 2024-05-12T04:01:11.627 INFO:tasks.workunit.client.0.smithi032.stdout:2/208: rename d1/d13/f30 to d1/d13/f48 0 2024-05-12T04:01:11.627 INFO:tasks.workunit.client.0.smithi032.stdout:5/298: dread - d4/d7/d14/d26/f5e zero size 2024-05-12T04:01:11.630 INFO:tasks.workunit.client.1.smithi033.stdout:2/422: write dc/d17/d32/f5c [664927,86983] 0 2024-05-12T04:01:11.630 INFO:tasks.workunit.client.1.smithi033.stdout:9/644: rename d2/dd/lb to d2/d4/d2b/d33/dd2/dde/lec 0 2024-05-12T04:01:11.631 INFO:tasks.workunit.client.0.smithi032.stdout:0/286: creat df/d10/d3b/f66 x:0 0 0 2024-05-12T04:01:11.632 INFO:tasks.workunit.client.1.smithi033.stdout:4/420: getdents dd 0 2024-05-12T04:01:11.632 INFO:tasks.workunit.client.0.smithi032.stdout:0/287: write f5 [1102921,12091] 0 2024-05-12T04:01:11.632 INFO:tasks.workunit.client.0.smithi032.stdout:9/172: rmdir d0/d3/d6 39 2024-05-12T04:01:11.633 INFO:tasks.workunit.client.0.smithi032.stdout:8/278: fdatasync d0/d7/f11 0 2024-05-12T04:01:11.635 INFO:tasks.workunit.client.0.smithi032.stdout:3/276: creat d15/d24/d40/f5a x:0 0 0 2024-05-12T04:01:11.635 INFO:tasks.workunit.client.0.smithi032.stdout:8/279: write d0/d9/d13/f29 [1907972,33564] 0 2024-05-12T04:01:11.636 INFO:tasks.workunit.client.1.smithi033.stdout:3/397: symlink db/d17/d25/d32/d3c/d44/l89 0 2024-05-12T04:01:11.642 INFO:tasks.workunit.client.1.smithi033.stdout:5/349: dwrite d3/f1e [0,4194304] 0 2024-05-12T04:01:11.643 INFO:tasks.workunit.client.0.smithi032.stdout:2/209: unlink d1/d13/f47 0 2024-05-12T04:01:11.649 INFO:tasks.workunit.client.1.smithi033.stdout:8/342: creat d1/f6b x:0 0 0 2024-05-12T04:01:11.653 INFO:tasks.workunit.client.1.smithi033.stdout:8/343: dread d1/d4/d46/f4f [0,4194304] 0 2024-05-12T04:01:11.655 INFO:tasks.workunit.client.1.smithi033.stdout:2/423: mkdir dc/d13/d39/d8f 0 2024-05-12T04:01:11.658 INFO:tasks.workunit.client.1.smithi033.stdout:9/645: creat d2/d4/d2b/d6d/d7a/fed x:0 0 0 2024-05-12T04:01:11.659 INFO:tasks.workunit.client.1.smithi033.stdout:0/451: write d6/f7c [562034,39090] 0 2024-05-12T04:01:11.661 INFO:tasks.workunit.client.1.smithi033.stdout:0/452: chown d6/d11/d57/f21 704841302 1 2024-05-12T04:01:11.661 INFO:tasks.workunit.client.1.smithi033.stdout:7/570: truncate de/df/d17/d1b/f89 4071108 0 2024-05-12T04:01:11.663 INFO:tasks.workunit.client.0.smithi032.stdout:8/280: mkdir d0/d9/d13/d21/d35/d37/d56 0 2024-05-12T04:01:11.663 INFO:tasks.workunit.client.0.smithi032.stdout:8/281: fdatasync d0/d7/f11 0 2024-05-12T04:01:11.664 INFO:tasks.workunit.client.1.smithi033.stdout:1/371: symlink d11/d17/d27/d36/d3f/d69/l7c 0 2024-05-12T04:01:11.667 INFO:tasks.workunit.client.1.smithi033.stdout:5/350: creat d3/d12/d29/d2b/f6e x:0 0 0 2024-05-12T04:01:11.669 INFO:tasks.workunit.client.0.smithi032.stdout:9/173: mknod d0/d3/d6/d7/d27/c3a 0 2024-05-12T04:01:11.671 INFO:tasks.workunit.client.1.smithi033.stdout:2/424: creat dc/d17/d27/d2f/f90 x:0 0 0 2024-05-12T04:01:11.675 INFO:tasks.workunit.client.1.smithi033.stdout:0/453: rmdir d6/dc 39 2024-05-12T04:01:11.676 INFO:tasks.workunit.client.1.smithi033.stdout:3/398: dread db/d17/d25/d32/d3c/f56 [0,4194304] 0 2024-05-12T04:01:11.677 INFO:tasks.workunit.client.0.smithi032.stdout:1/247: getdents d1/dd 0 2024-05-12T04:01:11.677 INFO:tasks.workunit.client.1.smithi033.stdout:6/416: getdents d4/d6e 0 2024-05-12T04:01:11.680 INFO:tasks.workunit.client.0.smithi032.stdout:5/299: rename d4/d7/d14/d26/d33/d4a/f4f to d4/d7/f69 0 2024-05-12T04:01:11.681 INFO:tasks.workunit.client.1.smithi033.stdout:7/571: creat de/df/d17/d1e/d24/fc5 x:0 0 0 2024-05-12T04:01:11.683 INFO:tasks.workunit.client.0.smithi032.stdout:4/173: creat d0/d3/d12/d1d/f2a x:0 0 0 2024-05-12T04:01:11.684 INFO:tasks.workunit.client.1.smithi033.stdout:9/646: write d2/d4/f9e [87096,122074] 0 2024-05-12T04:01:11.685 INFO:tasks.workunit.client.0.smithi032.stdout:9/174: fdatasync d0/d3/d6/d7/f15 0 2024-05-12T04:01:11.686 INFO:tasks.workunit.client.1.smithi033.stdout:5/351: dread - d3/d12/f3f zero size 2024-05-12T04:01:11.687 INFO:tasks.workunit.client.0.smithi032.stdout:1/248: dwrite d1/d7/d30/f28 [0,4194304] 0 2024-05-12T04:01:11.689 INFO:tasks.workunit.client.1.smithi033.stdout:8/344: unlink d1/d16/d27/d4d/d51/c4c 0 2024-05-12T04:01:11.689 INFO:tasks.workunit.client.0.smithi032.stdout:9/175: read d0/d3/d6/d7/f15 [135278,50860] 0 2024-05-12T04:01:11.689 INFO:tasks.workunit.client.0.smithi032.stdout:8/282: creat d0/d9/d13/d26/d28/d3e/f57 x:0 0 0 2024-05-12T04:01:11.691 INFO:tasks.workunit.client.1.smithi033.stdout:9/647: dwrite d2/d4/d29/d3c/d52/f5e [0,4194304] 0 2024-05-12T04:01:11.694 INFO:tasks.workunit.client.1.smithi033.stdout:9/648: readlink d2/d4/d2b/d6d/d3a/lce 0 2024-05-12T04:01:11.696 INFO:tasks.workunit.client.1.smithi033.stdout:3/399: mkdir db/d17/d3a/d8a 0 2024-05-12T04:01:11.697 INFO:tasks.workunit.client.0.smithi032.stdout:4/174: dwrite d0/fa [0,4194304] 0 2024-05-12T04:01:11.697 INFO:tasks.workunit.client.1.smithi033.stdout:3/400: write db/d17/f36 [1503862,29657] 0 2024-05-12T04:01:11.698 INFO:tasks.workunit.client.0.smithi032.stdout:5/300: dread d4/f6 [0,4194304] 0 2024-05-12T04:01:11.699 INFO:tasks.workunit.client.1.smithi033.stdout:8/345: dread d1/f13 [0,4194304] 0 2024-05-12T04:01:11.699 INFO:tasks.workunit.client.0.smithi032.stdout:8/283: truncate d0/d9/d13/d26/d28/d3e/f57 932063 0 2024-05-12T04:01:11.701 INFO:tasks.workunit.client.0.smithi032.stdout:0/288: rename fd to df/d30/d4e/d55/f67 0 2024-05-12T04:01:11.702 INFO:tasks.workunit.client.1.smithi033.stdout:5/352: dread d3/d15/d16/f44 [0,4194304] 0 2024-05-12T04:01:11.703 INFO:tasks.workunit.client.0.smithi032.stdout:8/284: chown d0/d9/d13/c47 107377 1 2024-05-12T04:01:11.709 INFO:tasks.workunit.client.0.smithi032.stdout:9/176: dwrite d0/d3/d6/f29 [0,4194304] 0 2024-05-12T04:01:11.713 INFO:tasks.workunit.client.0.smithi032.stdout:3/277: getdents d15/d33 0 2024-05-12T04:01:11.714 INFO:tasks.workunit.client.0.smithi032.stdout:1/249: creat d1/dd/f53 x:0 0 0 2024-05-12T04:01:11.714 INFO:tasks.workunit.client.0.smithi032.stdout:5/301: truncate d4/d7/d14/d26/d47/f65 480387 0 2024-05-12T04:01:11.714 INFO:tasks.workunit.client.1.smithi033.stdout:0/454: mkdir d6/dc/d58/d6a/d94 0 2024-05-12T04:01:11.717 INFO:tasks.workunit.client.1.smithi033.stdout:3/401: fdatasync db/d17/d25/d32/d3c/d44/f55 0 2024-05-12T04:01:11.720 INFO:tasks.workunit.client.1.smithi033.stdout:8/346: mknod d1/d16/d29/d63/c6c 0 2024-05-12T04:01:11.726 INFO:tasks.workunit.client.1.smithi033.stdout:5/353: unlink d3/d15/d16/d45/d4b/l6d 0 2024-05-12T04:01:11.728 INFO:tasks.workunit.client.0.smithi032.stdout:2/210: rename d1/l17 to d1/d13/l49 0 2024-05-12T04:01:11.733 INFO:tasks.workunit.client.0.smithi032.stdout:3/278: mkdir d15/d24/d2a/d5b 0 2024-05-12T04:01:11.733 INFO:tasks.workunit.client.0.smithi032.stdout:3/279: truncate d15/d33/f4c 633410 0 2024-05-12T04:01:11.736 INFO:tasks.workunit.client.0.smithi032.stdout:1/250: dread - d1/dd/f53 zero size 2024-05-12T04:01:11.737 INFO:tasks.workunit.client.0.smithi032.stdout:1/251: chown d1/d7/d30/d35/f38 53 1 2024-05-12T04:01:11.737 INFO:tasks.workunit.client.1.smithi033.stdout:0/455: symlink d6/d11/l95 0 2024-05-12T04:01:11.738 INFO:tasks.workunit.client.1.smithi033.stdout:0/456: readlink d6/d11/d57/l17 0 2024-05-12T04:01:11.739 INFO:tasks.workunit.client.0.smithi032.stdout:5/302: creat d4/d7/d14/d26/d33/d4a/d38/d3b/d4c/f6a x:0 0 0 2024-05-12T04:01:11.742 INFO:tasks.workunit.client.0.smithi032.stdout:5/303: chown d4/d7/d14/d26/d33/d4a/d38/d3b/d68 49687 1 2024-05-12T04:01:11.742 INFO:tasks.workunit.client.1.smithi033.stdout:3/402: creat db/d17/d25/d32/d3c/d3d/f8b x:0 0 0 2024-05-12T04:01:11.745 INFO:tasks.workunit.client.1.smithi033.stdout:8/347: fsync d1/d4/d46/f4f 0 2024-05-12T04:01:11.746 INFO:tasks.workunit.client.0.smithi032.stdout:0/289: dread df/d22/f2f [0,4194304] 0 2024-05-12T04:01:11.747 INFO:tasks.workunit.client.1.smithi033.stdout:8/348: write d1/d4/d1d/d36/f69 [451866,64764] 0 2024-05-12T04:01:11.747 INFO:tasks.workunit.client.0.smithi032.stdout:4/175: dwrite d0/f11 [0,4194304] 0 2024-05-12T04:01:11.750 INFO:tasks.workunit.client.0.smithi032.stdout:9/177: rename d0/d3/d6/d7/f14 to d0/d3/d6/d7/d2a/f3b 0 2024-05-12T04:01:11.751 INFO:tasks.workunit.client.1.smithi033.stdout:6/417: link d4/d1a/c4c d4/d1a/d4e/d4f/c8a 0 2024-05-12T04:01:11.753 INFO:tasks.workunit.client.0.smithi032.stdout:2/211: mkdir d1/d34/d4a 0 2024-05-12T04:01:11.754 INFO:tasks.workunit.client.0.smithi032.stdout:9/178: dread - d0/d3/d6/d7/f2c zero size 2024-05-12T04:01:11.755 INFO:tasks.workunit.client.0.smithi032.stdout:5/304: dwrite d4/f59 [0,4194304] 0 2024-05-12T04:01:11.757 INFO:tasks.workunit.client.0.smithi032.stdout:8/285: link d0/d9/d13/d21/d35/f3c d0/d9/d13/f58 0 2024-05-12T04:01:11.758 INFO:tasks.workunit.client.0.smithi032.stdout:5/305: chown d4/d7/d14/d26/d33 2 1 2024-05-12T04:01:11.758 INFO:tasks.workunit.client.0.smithi032.stdout:9/179: readlink d0/d3/d6/d7/da/l1f 0 2024-05-12T04:01:11.758 INFO:tasks.workunit.client.0.smithi032.stdout:2/212: chown d1/d13/d3d/d41/f2a 2608 1 2024-05-12T04:01:11.759 INFO:tasks.workunit.client.1.smithi033.stdout:7/572: rename l5 to de/df/lc6 0 2024-05-12T04:01:11.760 INFO:tasks.workunit.client.0.smithi032.stdout:0/290: dwrite df/d10/f13 [8388608,4194304] 0 2024-05-12T04:01:11.761 INFO:tasks.workunit.client.0.smithi032.stdout:3/280: creat d15/d33/f5c x:0 0 0 2024-05-12T04:01:11.762 INFO:tasks.workunit.client.0.smithi032.stdout:9/180: chown d0/d3/d6/d7/da/f33 4454 1 2024-05-12T04:01:11.762 INFO:tasks.workunit.client.0.smithi032.stdout:5/306: readlink d4/d7/d14/d39/d2b/l2e 0 2024-05-12T04:01:11.763 INFO:tasks.workunit.client.1.smithi033.stdout:1/372: getdents d11/d17/d27/d76 0 2024-05-12T04:01:11.764 INFO:tasks.workunit.client.0.smithi032.stdout:9/181: fdatasync d0/d3/ff 0 2024-05-12T04:01:11.764 INFO:tasks.workunit.client.0.smithi032.stdout:1/252: mkdir d1/d7/d30/d35/d54 0 2024-05-12T04:01:11.765 INFO:tasks.workunit.client.1.smithi033.stdout:1/373: write d11/d17/d27/d36/d3f/d4a/f7a [430692,64448] 0 2024-05-12T04:01:11.771 INFO:tasks.workunit.client.1.smithi033.stdout:5/354: mknod d3/d12/d29/d2b/d4f/c6f 0 2024-05-12T04:01:11.774 INFO:tasks.workunit.client.1.smithi033.stdout:9/649: rename d2/c1c to d2/d4/d29/d3c/d52/cee 0 2024-05-12T04:01:11.775 INFO:tasks.workunit.client.1.smithi033.stdout:9/650: fsync d2/d4/d29/d3c/d52/f5e 0 2024-05-12T04:01:11.777 INFO:tasks.workunit.client.0.smithi032.stdout:8/286: mkdir d0/d9/d1b/d27/d59 0 2024-05-12T04:01:11.780 INFO:tasks.workunit.client.0.smithi032.stdout:5/307: readlink d4/d7/l43 0 2024-05-12T04:01:11.781 INFO:tasks.workunit.client.0.smithi032.stdout:5/308: chown d4/d7/d14/d26/d33 1 1 2024-05-12T04:01:11.783 INFO:tasks.workunit.client.0.smithi032.stdout:3/281: write d15/d20/d22/f47 [7714442,123659] 0 2024-05-12T04:01:11.783 INFO:tasks.workunit.client.0.smithi032.stdout:7/259: sync 2024-05-12T04:01:11.783 INFO:tasks.workunit.client.0.smithi032.stdout:6/231: sync 2024-05-12T04:01:11.786 INFO:tasks.workunit.client.0.smithi032.stdout:7/260: write dd/d1b/f2d [3373778,80028] 0 2024-05-12T04:01:11.787 INFO:tasks.workunit.client.0.smithi032.stdout:0/291: symlink df/d30/d36/l68 0 2024-05-12T04:01:11.790 INFO:tasks.workunit.client.0.smithi032.stdout:9/182: unlink d0/d3/d6/f29 0 2024-05-12T04:01:11.791 INFO:tasks.workunit.client.1.smithi033.stdout:1/374: dread d11/f19 [0,4194304] 0 2024-05-12T04:01:11.793 INFO:tasks.workunit.client.0.smithi032.stdout:1/253: stat d1/d7/d30/d18/c1d 0 2024-05-12T04:01:11.793 INFO:tasks.workunit.client.1.smithi033.stdout:4/421: sync 2024-05-12T04:01:11.795 INFO:tasks.workunit.client.0.smithi032.stdout:4/176: write d0/f11 [5094283,102569] 0 2024-05-12T04:01:11.796 INFO:tasks.workunit.client.1.smithi033.stdout:6/418: write d4/d43/f70 [4381393,7562] 0 2024-05-12T04:01:11.797 INFO:tasks.workunit.client.0.smithi032.stdout:8/287: mknod d0/d9/d13/d21/d25/c5a 0 2024-05-12T04:01:11.797 INFO:tasks.workunit.client.0.smithi032.stdout:4/177: truncate d0/d3/d12/d1d/f2a 324897 0 2024-05-12T04:01:11.801 INFO:tasks.workunit.client.0.smithi032.stdout:0/292: dread df/d10/d3d/d4a/f38 [0,4194304] 0 2024-05-12T04:01:11.801 INFO:tasks.workunit.client.1.smithi033.stdout:6/419: dwrite d4/d1a/d67/d68/f6a [0,4194304] 0 2024-05-12T04:01:11.802 INFO:tasks.workunit.client.1.smithi033.stdout:5/355: unlink d3/l8 0 2024-05-12T04:01:11.806 INFO:tasks.workunit.client.1.smithi033.stdout:1/375: mkdir d11/d1c/d7d 0 2024-05-12T04:01:11.806 INFO:tasks.workunit.client.1.smithi033.stdout:1/376: fdatasync d11/d17/f18 0 2024-05-12T04:01:11.808 INFO:tasks.workunit.client.0.smithi032.stdout:3/282: symlink d15/d24/l5d 0 2024-05-12T04:01:11.811 INFO:tasks.workunit.client.1.smithi033.stdout:3/403: truncate f6 1055484 0 2024-05-12T04:01:11.812 INFO:tasks.workunit.client.1.smithi033.stdout:6/420: dread d4/d5/f5c [0,4194304] 0 2024-05-12T04:01:11.812 INFO:tasks.workunit.client.0.smithi032.stdout:6/232: creat d7/de/d12/f4d x:0 0 0 2024-05-12T04:01:11.813 INFO:tasks.workunit.client.0.smithi032.stdout:2/213: dwrite d1/d13/f19 [0,4194304] 0 2024-05-12T04:01:11.814 INFO:tasks.workunit.client.1.smithi033.stdout:1/377: dwrite d11/d1c/f64 [0,4194304] 0 2024-05-12T04:01:11.814 INFO:tasks.workunit.client.1.smithi033.stdout:7/573: link de/d80/cb2 de/df/d17/d1e/d79/cc7 0 2024-05-12T04:01:11.815 INFO:tasks.workunit.client.0.smithi032.stdout:6/233: write d7/de/d12/f4d [442431,49683] 0 2024-05-12T04:01:11.815 INFO:tasks.workunit.client.1.smithi033.stdout:1/378: read - d11/d17/d38/f5d zero size 2024-05-12T04:01:11.815 INFO:tasks.workunit.client.1.smithi033.stdout:7/574: write de/d2f/fba [579570,87303] 0 2024-05-12T04:01:11.816 INFO:tasks.workunit.client.1.smithi033.stdout:8/349: truncate d1/f2 3850974 0 2024-05-12T04:01:11.817 INFO:tasks.workunit.client.1.smithi033.stdout:1/379: chown d11/d17/d27/d36/d3f/d4a/f7a 997 1 2024-05-12T04:01:11.818 INFO:tasks.workunit.client.1.smithi033.stdout:8/350: read - d1/d16/d27/d4d/d51/f4b zero size 2024-05-12T04:01:11.830 INFO:tasks.workunit.client.0.smithi032.stdout:9/183: fsync d0/d3/d6/d7/fc 0 2024-05-12T04:01:11.834 INFO:tasks.workunit.client.1.smithi033.stdout:5/356: mkdir d3/d12/d25/d26/d2e/d54/d57/d70 0 2024-05-12T04:01:11.834 INFO:tasks.workunit.client.0.smithi032.stdout:2/214: dread d1/f3 [0,4194304] 0 2024-05-12T04:01:11.834 INFO:tasks.workunit.client.0.smithi032.stdout:9/184: fsync d0/d3/d6/d7/da/d10/f12 0 2024-05-12T04:01:11.834 INFO:tasks.workunit.client.0.smithi032.stdout:1/254: chown d1/dd/d24/c2f 127548 1 2024-05-12T04:01:11.835 INFO:tasks.workunit.client.0.smithi032.stdout:5/309: unlink d4/d7/d14/c30 0 2024-05-12T04:01:11.839 INFO:tasks.workunit.client.0.smithi032.stdout:4/178: symlink d0/d3/d12/d1d/l2b 0 2024-05-12T04:01:11.840 INFO:tasks.workunit.client.0.smithi032.stdout:4/179: fsync d0/d3/f1c 0 2024-05-12T04:01:11.841 INFO:tasks.workunit.client.1.smithi033.stdout:4/422: dwrite dd/f2b [0,4194304] 0 2024-05-12T04:01:11.842 INFO:tasks.workunit.client.0.smithi032.stdout:4/180: readlink d0/d3/df/l10 0 2024-05-12T04:01:11.844 INFO:tasks.workunit.client.1.smithi033.stdout:6/421: creat d4/d1a/d67/d68/f8b x:0 0 0 2024-05-12T04:01:11.845 INFO:tasks.workunit.client.0.smithi032.stdout:0/293: symlink df/d30/d32/l69 0 2024-05-12T04:01:11.845 INFO:tasks.workunit.client.0.smithi032.stdout:0/294: readlink df/d30/d32/l69 0 2024-05-12T04:01:11.849 INFO:tasks.workunit.client.1.smithi033.stdout:7/575: symlink de/d2b/d6c/lc8 0 2024-05-12T04:01:11.852 INFO:tasks.workunit.client.1.smithi033.stdout:1/380: mknod d11/d17/d27/d76/c7e 0 2024-05-12T04:01:11.854 INFO:tasks.workunit.client.0.smithi032.stdout:7/261: creat dd/d1b/d1e/d45/d55/f5b x:0 0 0 2024-05-12T04:01:11.856 INFO:tasks.workunit.client.1.smithi033.stdout:8/351: unlink d1/d4/d1d/d36/f69 0 2024-05-12T04:01:11.857 INFO:tasks.workunit.client.1.smithi033.stdout:1/381: dread d11/d17/d27/f52 [0,4194304] 0 2024-05-12T04:01:11.858 INFO:tasks.workunit.client.1.smithi033.stdout:9/651: link d2/dd/d39/la6 d2/d4/d2b/d6d/d2f/lef 0 2024-05-12T04:01:11.865 INFO:tasks.workunit.client.0.smithi032.stdout:2/215: symlink d1/d13/d3d/d41/l4b 0 2024-05-12T04:01:11.866 INFO:tasks.workunit.client.0.smithi032.stdout:2/216: write d1/f31 [631969,22426] 0 2024-05-12T04:01:11.869 INFO:tasks.workunit.client.1.smithi033.stdout:4/423: rename dd/d11/d31/d38/d57 to dd/d7a/d87 0 2024-05-12T04:01:11.870 INFO:tasks.workunit.client.0.smithi032.stdout:9/185: creat d0/d3/d6/d7/da/d10/d13/f3c x:0 0 0 2024-05-12T04:01:11.872 INFO:tasks.workunit.client.1.smithi033.stdout:6/422: rmdir d4/d5 39 2024-05-12T04:01:11.874 INFO:tasks.workunit.client.0.smithi032.stdout:1/255: rename d1/d7/ca to d1/d7/d30/d18/c55 0 2024-05-12T04:01:11.875 INFO:tasks.workunit.client.1.smithi033.stdout:3/404: getdents db/d17/d25/d32/d3c/d3d/d47 0 2024-05-12T04:01:11.878 INFO:tasks.workunit.client.1.smithi033.stdout:7/576: write de/df/d7a/f9f [622434,51476] 0 2024-05-12T04:01:11.878 INFO:tasks.workunit.client.0.smithi032.stdout:8/288: creat d0/d9/d13/d21/d35/d37/d56/f5b x:0 0 0 2024-05-12T04:01:11.880 INFO:tasks.workunit.client.0.smithi032.stdout:8/289: write d0/d9/f31 [768529,19191] 0 2024-05-12T04:01:11.881 INFO:tasks.workunit.client.0.smithi032.stdout:3/283: creat d15/d33/d3f/f5e x:0 0 0 2024-05-12T04:01:11.882 INFO:tasks.workunit.client.1.smithi033.stdout:3/405: dwrite db/d17/f7b [0,4194304] 0 2024-05-12T04:01:11.884 INFO:tasks.workunit.client.0.smithi032.stdout:7/262: readlink dd/d1b/d1e/l29 0 2024-05-12T04:01:11.886 INFO:tasks.workunit.client.1.smithi033.stdout:8/352: mknod d1/d16/d29/d63/c6d 0 2024-05-12T04:01:11.887 INFO:tasks.workunit.client.0.smithi032.stdout:6/234: link d7/de/d12/d25/f2a d7/de/d2e/f4e 0 2024-05-12T04:01:11.889 INFO:tasks.workunit.client.1.smithi033.stdout:1/382: creat d11/d1c/d59/f7f x:0 0 0 2024-05-12T04:01:11.889 INFO:tasks.workunit.client.1.smithi033.stdout:1/383: fsync d11/d17/f18 0 2024-05-12T04:01:11.890 INFO:tasks.workunit.client.1.smithi033.stdout:9/652: creat d2/d4/d2b/d49/ff0 x:0 0 0 2024-05-12T04:01:11.892 INFO:tasks.workunit.client.0.smithi032.stdout:2/217: unlink d1/d13/d3d/d41/f33 0 2024-05-12T04:01:11.894 INFO:tasks.workunit.client.1.smithi033.stdout:1/384: dread d11/d17/d27/d36/d3f/d4a/f7a [0,4194304] 0 2024-05-12T04:01:11.894 INFO:tasks.workunit.client.1.smithi033.stdout:7/577: dread de/df/d17/d1e/d79/f91 [0,4194304] 0 2024-05-12T04:01:11.895 INFO:tasks.workunit.client.1.smithi033.stdout:7/578: readlink de/l14 0 2024-05-12T04:01:11.895 INFO:tasks.workunit.client.0.smithi032.stdout:5/310: rename d4/d7/d14/d26/d33/d4a/d38/d3b/l60 to d4/d7/d14/d39/d1b/l6b 0 2024-05-12T04:01:11.897 INFO:tasks.workunit.client.0.smithi032.stdout:8/290: mknod d0/d9/d13/d26/d28/c5c 0 2024-05-12T04:01:11.899 INFO:tasks.workunit.client.0.smithi032.stdout:6/235: mknod d7/de/d12/d1a/c4f 0 2024-05-12T04:01:11.900 INFO:tasks.workunit.client.1.smithi033.stdout:6/423: chown d4/d5 932638216 1 2024-05-12T04:01:11.904 INFO:tasks.workunit.client.1.smithi033.stdout:0/457: sync 2024-05-12T04:01:11.904 INFO:tasks.workunit.client.1.smithi033.stdout:2/425: sync 2024-05-12T04:01:11.909 INFO:tasks.workunit.client.1.smithi033.stdout:2/426: dwrite dc/d13/d1c/f21 [0,4194304] 0 2024-05-12T04:01:11.910 INFO:tasks.workunit.client.1.smithi033.stdout:8/353: symlink d1/d4/d46/l6e 0 2024-05-12T04:01:11.912 INFO:tasks.workunit.client.0.smithi032.stdout:8/291: creat d0/d9/d13/f5d x:0 0 0 2024-05-12T04:01:11.916 INFO:tasks.workunit.client.1.smithi033.stdout:2/427: dwrite dc/d17/d27/d47/d5e/f77 [0,4194304] 0 2024-05-12T04:01:11.918 INFO:tasks.workunit.client.1.smithi033.stdout:5/357: truncate d3/f7 3780521 0 2024-05-12T04:01:11.918 INFO:tasks.workunit.client.0.smithi032.stdout:1/256: creat d1/dd/f56 x:0 0 0 2024-05-12T04:01:11.923 INFO:tasks.workunit.client.0.smithi032.stdout:4/181: rename d0/d3/d12/c1f to d0/d3/c2c 0 2024-05-12T04:01:11.925 INFO:tasks.workunit.client.1.smithi033.stdout:4/424: symlink dd/d11/l88 0 2024-05-12T04:01:11.926 INFO:tasks.workunit.client.0.smithi032.stdout:5/311: mknod d4/d7/c6c 0 2024-05-12T04:01:11.930 INFO:tasks.workunit.client.0.smithi032.stdout:6/236: link d7/de/d12/f22 d7/de/d12/d1a/d20/f50 0 2024-05-12T04:01:11.932 INFO:tasks.workunit.client.0.smithi032.stdout:6/237: chown d7/de/d12/f45 0 1 2024-05-12T04:01:11.933 INFO:tasks.workunit.client.1.smithi033.stdout:8/354: unlink d1/d16/d27/d28/l42 0 2024-05-12T04:01:11.933 INFO:tasks.workunit.client.0.smithi032.stdout:5/312: dwrite d4/d7/d14/d39/d1b/f23 [0,4194304] 0 2024-05-12T04:01:11.935 INFO:tasks.workunit.client.0.smithi032.stdout:0/295: rename df/d10/f33 to df/d22/d58/f6a 0 2024-05-12T04:01:11.944 INFO:tasks.workunit.client.1.smithi033.stdout:4/425: dread dd/d11/f46 [0,4194304] 0 2024-05-12T04:01:11.955 INFO:tasks.workunit.client.0.smithi032.stdout:7/263: getdents dd 0 2024-05-12T04:01:11.958 INFO:tasks.workunit.client.0.smithi032.stdout:8/292: symlink d0/d4b/l5e 0 2024-05-12T04:01:11.960 INFO:tasks.workunit.client.0.smithi032.stdout:1/257: mkdir d1/dd/d24/d57 0 2024-05-12T04:01:11.961 INFO:tasks.workunit.client.1.smithi033.stdout:5/358: readlink d3/d12/d29/d2b/l4e 0 2024-05-12T04:01:11.965 INFO:tasks.workunit.client.0.smithi032.stdout:2/218: getdents d1/d13 0 2024-05-12T04:01:11.966 INFO:tasks.workunit.client.1.smithi033.stdout:7/579: symlink de/d2b/d6c/da0/da6/lc9 0 2024-05-12T04:01:11.967 INFO:tasks.workunit.client.1.smithi033.stdout:7/580: readlink de/df/d17/d1b/l2a 0 2024-05-12T04:01:11.970 INFO:tasks.workunit.client.0.smithi032.stdout:9/186: rename d0/d3/f32 to d0/d3/d6/d7/d27/f3d 0 2024-05-12T04:01:11.975 INFO:tasks.workunit.client.0.smithi032.stdout:0/296: symlink df/d10/d3b/l6b 0 2024-05-12T04:01:11.975 INFO:tasks.workunit.client.0.smithi032.stdout:0/297: stat df/d10/f64 0 2024-05-12T04:01:11.977 INFO:tasks.workunit.client.0.smithi032.stdout:7/264: symlink dd/d19/d37/l5c 0 2024-05-12T04:01:11.978 INFO:tasks.workunit.client.1.smithi033.stdout:0/458: mkdir d6/dc/d96 0 2024-05-12T04:01:11.980 INFO:tasks.workunit.client.1.smithi033.stdout:3/406: creat db/f8c x:0 0 0 2024-05-12T04:01:11.981 INFO:tasks.workunit.client.0.smithi032.stdout:1/258: mknod d1/dd/d24/c58 0 2024-05-12T04:01:11.984 INFO:tasks.workunit.client.1.smithi033.stdout:6/424: truncate d4/d5/f27 2846327 0 2024-05-12T04:01:11.985 INFO:tasks.workunit.client.1.smithi033.stdout:4/426: symlink dd/d11/d31/d66/d42/d59/l89 0 2024-05-12T04:01:11.989 INFO:tasks.workunit.client.0.smithi032.stdout:3/284: rename d15/f44 to d15/d20/d26/f5f 0 2024-05-12T04:01:11.990 INFO:tasks.workunit.client.1.smithi033.stdout:5/359: symlink d3/d12/l71 0 2024-05-12T04:01:11.994 INFO:tasks.workunit.client.0.smithi032.stdout:8/293: mknod d0/d9/d1b/d27/d59/c5f 0 2024-05-12T04:01:11.995 INFO:tasks.workunit.client.0.smithi032.stdout:1/259: dread - d1/d25/f4a zero size 2024-05-12T04:01:11.996 INFO:tasks.workunit.client.0.smithi032.stdout:1/260: read d1/f26 [112174,68877] 0 2024-05-12T04:01:11.997 INFO:tasks.workunit.client.1.smithi033.stdout:4/427: dread dd/d11/d31/d66/f49 [0,4194304] 0 2024-05-12T04:01:12.004 INFO:tasks.workunit.client.0.smithi032.stdout:6/238: rename d7/de/d12/f22 to d7/f51 0 2024-05-12T04:01:12.005 INFO:tasks.workunit.client.0.smithi032.stdout:6/239: read - d7/de/d12/d1a/d20/f50 zero size 2024-05-12T04:01:12.009 INFO:tasks.workunit.client.0.smithi032.stdout:0/298: mkdir df/d10/d3d/d6c 0 2024-05-12T04:01:12.009 INFO:tasks.workunit.client.0.smithi032.stdout:0/299: chown df/d10/f13 13 1 2024-05-12T04:01:12.012 INFO:tasks.workunit.client.1.smithi033.stdout:1/385: rename d11/f13 to d11/d17/d27/d36/f80 0 2024-05-12T04:01:12.022 INFO:tasks.workunit.client.0.smithi032.stdout:5/313: rename d4/fd to d4/d7/d14/d26/d47/f6d 0 2024-05-12T04:01:12.022 INFO:tasks.workunit.client.0.smithi032.stdout:9/187: rename d0/d3/d6/d7 to d0/d3/d6/d7/d3e 22 2024-05-12T04:01:12.024 INFO:tasks.workunit.client.1.smithi033.stdout:6/425: link d4/d43/f70 d4/d1a/d67/f8c 0 2024-05-12T04:01:12.028 INFO:tasks.workunit.client.1.smithi033.stdout:1/386: dread d11/d17/d27/d36/d3f/f61 [0,4194304] 0 2024-05-12T04:01:12.028 INFO:tasks.workunit.client.0.smithi032.stdout:5/314: dread d4/d7/d14/d26/d33/d4a/f51 [0,4194304] 0 2024-05-12T04:01:12.029 INFO:tasks.workunit.client.1.smithi033.stdout:9/653: getdents d2/d4/d2b/d78 0 2024-05-12T04:01:12.031 INFO:tasks.workunit.client.1.smithi033.stdout:7/581: dwrite de/d2f/d70/f74 [0,4194304] 0 2024-05-12T04:01:12.031 INFO:tasks.workunit.client.0.smithi032.stdout:6/240: creat d7/de/d12/f52 x:0 0 0 2024-05-12T04:01:12.031 INFO:tasks.workunit.client.0.smithi032.stdout:5/315: dread - d4/d7/d14/d26/f5e zero size 2024-05-12T04:01:12.033 INFO:tasks.workunit.client.1.smithi033.stdout:0/459: rename d6/db/d2c/c2e to d6/dc/d58/d6a/d80/c97 0 2024-05-12T04:01:12.038 INFO:tasks.workunit.client.1.smithi033.stdout:8/355: dwrite d1/d16/f19 [0,4194304] 0 2024-05-12T04:01:12.038 INFO:tasks.workunit.client.0.smithi032.stdout:2/219: dwrite d1/f7 [0,4194304] 0 2024-05-12T04:01:12.038 INFO:tasks.workunit.client.0.smithi032.stdout:0/300: creat df/d30/d36/f6d x:0 0 0 2024-05-12T04:01:12.038 INFO:tasks.workunit.client.0.smithi032.stdout:7/265: truncate dd/f13 2371815 0 2024-05-12T04:01:12.039 INFO:tasks.workunit.client.0.smithi032.stdout:7/266: readlink dd/d1b/d1e/l52 0 2024-05-12T04:01:12.040 INFO:tasks.workunit.client.0.smithi032.stdout:1/261: mkdir d1/d59 0 2024-05-12T04:01:12.043 INFO:tasks.workunit.client.1.smithi033.stdout:4/428: symlink dd/d11/d31/d66/l8a 0 2024-05-12T04:01:12.043 INFO:tasks.workunit.client.0.smithi032.stdout:7/267: truncate dd/d19/f4d 930979 0 2024-05-12T04:01:12.044 INFO:tasks.workunit.client.0.smithi032.stdout:5/316: dread d4/f22 [0,4194304] 0 2024-05-12T04:01:12.045 INFO:tasks.workunit.client.1.smithi033.stdout:4/429: dread - dd/d11/f6d zero size 2024-05-12T04:01:12.046 INFO:tasks.workunit.client.1.smithi033.stdout:8/356: dwrite d1/d16/d27/d28/f57 [0,4194304] 0 2024-05-12T04:01:12.051 INFO:tasks.workunit.client.1.smithi033.stdout:0/460: dread d6/dc/d58/f59 [0,4194304] 0 2024-05-12T04:01:12.052 INFO:tasks.workunit.client.1.smithi033.stdout:3/407: write db/d17/d3a/f66 [1318182,124086] 0 2024-05-12T04:01:12.052 INFO:tasks.workunit.client.0.smithi032.stdout:5/317: dread d4/d7/d14/d26/d33/d4a/f51 [0,4194304] 0 2024-05-12T04:01:12.052 INFO:tasks.workunit.client.0.smithi032.stdout:5/318: dread - d4/d7/d14/d26/f5e zero size 2024-05-12T04:01:12.052 INFO:tasks.workunit.client.0.smithi032.stdout:9/188: dread d0/d3/f25 [0,4194304] 0 2024-05-12T04:01:12.054 INFO:tasks.workunit.client.0.smithi032.stdout:9/189: dread - d0/d3/d6/d7/da/f1c zero size 2024-05-12T04:01:12.054 INFO:tasks.workunit.client.0.smithi032.stdout:6/241: symlink d7/l53 0 2024-05-12T04:01:12.056 INFO:tasks.workunit.client.1.smithi033.stdout:1/387: mknod d11/d1c/d59/c81 0 2024-05-12T04:01:12.060 INFO:tasks.workunit.client.0.smithi032.stdout:6/242: dread d7/f32 [0,4194304] 0 2024-05-12T04:01:12.060 INFO:tasks.workunit.client.0.smithi032.stdout:2/220: creat d1/d13/f4c x:0 0 0 2024-05-12T04:01:12.061 INFO:tasks.workunit.client.1.smithi033.stdout:0/461: dwrite d6/d11/f2f [0,4194304] 0 2024-05-12T04:01:12.062 INFO:tasks.workunit.client.0.smithi032.stdout:6/243: write d7/de/d12/f4d [1097376,111335] 0 2024-05-12T04:01:12.063 INFO:tasks.workunit.client.1.smithi033.stdout:1/388: dread fd [0,4194304] 0 2024-05-12T04:01:12.064 INFO:tasks.workunit.client.0.smithi032.stdout:0/301: mknod df/d30/d4e/d55/c6e 0 2024-05-12T04:01:12.065 INFO:tasks.workunit.client.1.smithi033.stdout:7/582: fsync de/df/d17/d1e/d24/f57 0 2024-05-12T04:01:12.070 INFO:tasks.workunit.client.0.smithi032.stdout:0/302: dread fa [4194304,4194304] 0 2024-05-12T04:01:12.076 INFO:tasks.workunit.client.0.smithi032.stdout:5/319: mkdir d4/d7/d14/d39/d2b/d6e 0 2024-05-12T04:01:12.078 INFO:tasks.workunit.client.1.smithi033.stdout:4/430: unlink dd/d7a/d87/f58 0 2024-05-12T04:01:12.079 INFO:tasks.workunit.client.0.smithi032.stdout:0/303: dwrite df/d30/d32/f34 [0,4194304] 0 2024-05-12T04:01:12.081 INFO:tasks.workunit.client.1.smithi033.stdout:8/357: mknod d1/d16/d27/d28/c6f 0 2024-05-12T04:01:12.084 INFO:tasks.workunit.client.1.smithi033.stdout:6/426: mkdir d4/d8d 0 2024-05-12T04:01:12.085 INFO:tasks.workunit.client.1.smithi033.stdout:6/427: stat d4/d43/d57 0 2024-05-12T04:01:12.085 INFO:tasks.workunit.client.0.smithi032.stdout:2/221: rmdir d1/d13/d3d/d41/d39 39 2024-05-12T04:01:12.090 INFO:tasks.workunit.client.0.smithi032.stdout:2/222: dread d1/f31 [0,4194304] 0 2024-05-12T04:01:12.090 INFO:tasks.workunit.client.0.smithi032.stdout:6/244: rmdir d7/de/d12 39 2024-05-12T04:01:12.093 INFO:tasks.workunit.client.0.smithi032.stdout:0/304: dread f9 [4194304,4194304] 0 2024-05-12T04:01:12.094 INFO:tasks.workunit.client.0.smithi032.stdout:3/285: rename d15/d20/d26/c2d to d15/d20/c60 0 2024-05-12T04:01:12.094 INFO:tasks.workunit.client.1.smithi033.stdout:9/654: write d2/d4/d2b/d78/f8c [883553,107395] 0 2024-05-12T04:01:12.095 INFO:tasks.workunit.client.0.smithi032.stdout:3/286: chown d15/d24/f51 129152 1 2024-05-12T04:01:12.096 INFO:tasks.workunit.client.0.smithi032.stdout:3/287: chown d15/l16 34 1 2024-05-12T04:01:12.099 INFO:tasks.workunit.client.1.smithi033.stdout:0/462: dread d6/dc/d58/d6a/d80/f84 [0,4194304] 0 2024-05-12T04:01:12.103 INFO:tasks.workunit.client.0.smithi032.stdout:5/320: creat d4/d7/d14/d26/d3c/f6f x:0 0 0 2024-05-12T04:01:12.106 INFO:tasks.workunit.client.1.smithi033.stdout:1/389: rmdir d11/d17/d27/d36 39 2024-05-12T04:01:12.111 INFO:tasks.workunit.client.1.smithi033.stdout:7/583: mknod de/df/d17/d1e/cca 0 2024-05-12T04:01:12.112 INFO:tasks.workunit.client.0.smithi032.stdout:3/288: dread fd [0,4194304] 0 2024-05-12T04:01:12.112 INFO:tasks.workunit.client.1.smithi033.stdout:1/390: dwrite d11/d17/d27/f57 [0,4194304] 0 2024-05-12T04:01:12.116 INFO:tasks.workunit.client.1.smithi033.stdout:4/431: read dd/d11/d31/d66/f49 [1296394,96575] 0 2024-05-12T04:01:12.117 INFO:tasks.workunit.client.1.smithi033.stdout:3/408: write db/d17/f2c [3195226,98567] 0 2024-05-12T04:01:12.120 INFO:tasks.workunit.client.1.smithi033.stdout:8/358: rename d1/d16/d20/c23 to d1/c70 0 2024-05-12T04:01:12.120 INFO:tasks.workunit.client.1.smithi033.stdout:8/359: stat d1/d4/d1d/d36/d3c 0 2024-05-12T04:01:12.121 INFO:tasks.workunit.client.1.smithi033.stdout:8/360: chown d1/d16/d29/d63/c6d 10356 1 2024-05-12T04:01:12.123 INFO:tasks.workunit.client.1.smithi033.stdout:6/428: symlink d4/d1a/d67/d68/l8e 0 2024-05-12T04:01:12.124 INFO:tasks.workunit.client.1.smithi033.stdout:6/429: chown d4/d1a/l1b 133188785 1 2024-05-12T04:01:12.125 INFO:tasks.workunit.client.0.smithi032.stdout:8/294: rename d0/d9/d13/d26/d28/c5c to d0/d9/d13/c60 0 2024-05-12T04:01:12.126 INFO:tasks.workunit.client.1.smithi033.stdout:9/655: fsync d2/dd/d39/d72/faf 0 2024-05-12T04:01:12.131 INFO:tasks.workunit.client.1.smithi033.stdout:0/463: mkdir d6/d11/d98 0 2024-05-12T04:01:12.132 INFO:tasks.workunit.client.1.smithi033.stdout:9/656: dwrite d2/d4/d2b/dc7/fe6 [0,4194304] 0 2024-05-12T04:01:12.132 INFO:tasks.workunit.client.0.smithi032.stdout:5/321: unlink d4/d7/d14/c28 0 2024-05-12T04:01:12.141 INFO:tasks.workunit.client.0.smithi032.stdout:2/223: mkdir d1/d4d 0 2024-05-12T04:01:12.147 INFO:tasks.workunit.client.1.smithi033.stdout:0/464: dread d6/d11/f33 [4194304,4194304] 0 2024-05-12T04:01:12.149 INFO:tasks.workunit.client.0.smithi032.stdout:9/190: rename d0/d3/d6/d7/da/d10 to d0/d3/d6/d7/da/d17/d3f 0 2024-05-12T04:01:12.149 INFO:tasks.workunit.client.0.smithi032.stdout:2/224: dread d1/d13/d3d/d41/f3a [0,4194304] 0 2024-05-12T04:01:12.150 INFO:tasks.workunit.client.1.smithi033.stdout:2/428: sync 2024-05-12T04:01:12.153 INFO:tasks.workunit.client.1.smithi033.stdout:8/361: mknod d1/d16/c71 0 2024-05-12T04:01:12.154 INFO:tasks.workunit.client.0.smithi032.stdout:1/262: link d1/dd/l3a d1/d7/d30/l5a 0 2024-05-12T04:01:12.154 INFO:tasks.workunit.client.1.smithi033.stdout:8/362: dread d1/d16/d20/f65 [0,4194304] 0 2024-05-12T04:01:12.155 INFO:tasks.workunit.client.1.smithi033.stdout:6/430: fdatasync d4/d43/f70 0 2024-05-12T04:01:12.158 INFO:tasks.workunit.client.0.smithi032.stdout:0/305: write df/d10/f3c [1446681,88599] 0 2024-05-12T04:01:12.160 INFO:tasks.workunit.client.0.smithi032.stdout:3/289: getdents d15/d1b/d43 0 2024-05-12T04:01:12.160 INFO:tasks.workunit.client.0.smithi032.stdout:5/322: dread d4/d7/d14/d26/d33/d4a/d38/d3b/f2a [0,4194304] 0 2024-05-12T04:01:12.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:11 smithi033 ceph-mon[49525]: pgmap v34: 65 pgs: 65 active+clean; 1.5 GiB data, 5.9 GiB used, 531 GiB / 536 GiB avail; 34 MiB/s rd, 203 MiB/s wr, 344 op/s 2024-05-12T04:01:12.164 INFO:tasks.workunit.client.0.smithi032.stdout:9/191: dread d0/d3/ff [8388608,4194304] 0 2024-05-12T04:01:12.166 INFO:tasks.workunit.client.1.smithi033.stdout:3/409: dwrite db/d17/d25/d32/d3c/d44/f55 [4194304,4194304] 0 2024-05-12T04:01:12.167 INFO:tasks.workunit.client.1.smithi033.stdout:9/657: mknod d2/d4/d2b/d78/cf1 0 2024-05-12T04:01:12.171 INFO:tasks.workunit.client.0.smithi032.stdout:8/295: rename d0/d7/c16 to d0/d9/d1b/d27/d59/c61 0 2024-05-12T04:01:12.171 INFO:tasks.workunit.client.1.smithi033.stdout:1/391: symlink d11/d17/d27/l82 0 2024-05-12T04:01:12.172 INFO:tasks.workunit.client.1.smithi033.stdout:4/432: getdents dd/d7a/d87/d7e 0 2024-05-12T04:01:12.174 INFO:tasks.workunit.client.1.smithi033.stdout:4/433: chown dd/d11/d31/f6c 24 1 2024-05-12T04:01:12.174 INFO:tasks.workunit.client.0.smithi032.stdout:1/263: creat d1/d25/f5b x:0 0 0 2024-05-12T04:01:12.175 INFO:tasks.workunit.client.1.smithi033.stdout:0/465: creat d6/d11/f99 x:0 0 0 2024-05-12T04:01:12.175 INFO:tasks.workunit.client.0.smithi032.stdout:3/290: dread f14 [4194304,4194304] 0 2024-05-12T04:01:12.175 INFO:tasks.workunit.client.1.smithi033.stdout:4/434: fsync dd/d11/d31/d66/d42/f63 0 2024-05-12T04:01:12.176 INFO:tasks.workunit.client.0.smithi032.stdout:1/264: chown d1/dd/d24/c58 6820 1 2024-05-12T04:01:12.176 INFO:tasks.workunit.client.1.smithi033.stdout:2/429: symlink dc/d3b/d88/l91 0 2024-05-12T04:01:12.176 INFO:tasks.workunit.client.1.smithi033.stdout:0/466: chown d6/dc/f56 12184 1 2024-05-12T04:01:12.176 INFO:tasks.workunit.client.1.smithi033.stdout:1/392: dread d11/d1c/f64 [0,4194304] 0 2024-05-12T04:01:12.177 INFO:tasks.workunit.client.0.smithi032.stdout:0/306: symlink df/d10/d3b/l6f 0 2024-05-12T04:01:12.182 INFO:tasks.workunit.client.0.smithi032.stdout:6/245: getdents d7/d9 0 2024-05-12T04:01:12.182 INFO:tasks.workunit.client.0.smithi032.stdout:6/246: dread - d7/f51 zero size 2024-05-12T04:01:12.185 INFO:tasks.workunit.client.0.smithi032.stdout:9/192: rename d0/d3/d6/d7/l9 to d0/d3/d6/d7/d27/l40 0 2024-05-12T04:01:12.186 INFO:tasks.workunit.client.1.smithi033.stdout:6/431: unlink d4/d1a/d4e/f53 0 2024-05-12T04:01:12.187 INFO:tasks.workunit.client.1.smithi033.stdout:6/432: chown d4/f87 62918 1 2024-05-12T04:01:12.187 INFO:tasks.workunit.client.0.smithi032.stdout:2/225: symlink d1/d34/d4a/l4e 0 2024-05-12T04:01:12.189 INFO:tasks.workunit.client.1.smithi033.stdout:6/433: dread d4/d1a/d4e/f66 [0,4194304] 0 2024-05-12T04:01:12.189 INFO:tasks.workunit.client.0.smithi032.stdout:2/226: read d1/fc [1734556,66682] 0 2024-05-12T04:01:12.190 INFO:tasks.workunit.client.0.smithi032.stdout:8/296: creat d0/d9/d13/d21/d35/d37/f62 x:0 0 0 2024-05-12T04:01:12.191 INFO:tasks.workunit.client.0.smithi032.stdout:8/297: stat d0/d9/d13/d21/d25/f50 0 2024-05-12T04:01:12.191 INFO:tasks.workunit.client.0.smithi032.stdout:3/291: rmdir d15/d20/d26 39 2024-05-12T04:01:12.194 INFO:tasks.workunit.client.0.smithi032.stdout:1/265: chown d1/c15 905 1 2024-05-12T04:01:12.195 INFO:tasks.workunit.client.1.smithi033.stdout:3/410: write db/d17/d25/f40 [3614494,26000] 0 2024-05-12T04:01:12.198 INFO:tasks.workunit.client.1.smithi033.stdout:8/363: dwrite d1/d16/d20/f33 [0,4194304] 0 2024-05-12T04:01:12.201 INFO:tasks.workunit.client.0.smithi032.stdout:9/193: mknod d0/d3/d6/d7/da/d17/d3f/c41 0 2024-05-12T04:01:12.204 INFO:tasks.workunit.client.1.smithi033.stdout:5/360: sync 2024-05-12T04:01:12.207 INFO:tasks.workunit.client.1.smithi033.stdout:1/393: chown d11/d17/d27/d36/d46/f66 66659 1 2024-05-12T04:01:12.207 INFO:tasks.workunit.client.0.smithi032.stdout:3/292: rename f3 to d15/d1b/f61 0 2024-05-12T04:01:12.208 INFO:tasks.workunit.client.1.smithi033.stdout:5/361: dread - d3/d12/d25/d26/d2e/d54/d57/f5f zero size 2024-05-12T04:01:12.208 INFO:tasks.workunit.client.1.smithi033.stdout:9/658: dwrite d2/f24 [4194304,4194304] 0 2024-05-12T04:01:12.210 INFO:tasks.workunit.client.1.smithi033.stdout:2/430: rename dc/d13/d1f/f24 to dc/d13/d1c/f92 0 2024-05-12T04:01:12.210 INFO:tasks.workunit.client.0.smithi032.stdout:8/298: mkdir d0/d9/d13/d21/d35/d37/d56/d63 0 2024-05-12T04:01:12.211 INFO:tasks.workunit.client.1.smithi033.stdout:9/659: write d2/d4/d2b/dc7/fe6 [3861159,19784] 0 2024-05-12T04:01:12.211 INFO:tasks.workunit.client.0.smithi032.stdout:6/247: dread d7/de/d12/f4c [0,4194304] 0 2024-05-12T04:01:12.211 INFO:tasks.workunit.client.1.smithi033.stdout:2/431: write f6 [717644,43779] 0 2024-05-12T04:01:12.214 INFO:tasks.workunit.client.1.smithi033.stdout:2/432: write dc/f8d [255862,129267] 0 2024-05-12T04:01:12.217 INFO:tasks.workunit.client.1.smithi033.stdout:6/434: unlink d4/d1a/f23 0 2024-05-12T04:01:12.218 INFO:tasks.workunit.client.1.smithi033.stdout:6/435: stat d4/d6e/l7d 0 2024-05-12T04:01:12.221 INFO:tasks.workunit.client.1.smithi033.stdout:2/433: dwrite dc/d17/d27/d2f/d49/f51 [0,4194304] 0 2024-05-12T04:01:12.223 INFO:tasks.workunit.client.0.smithi032.stdout:2/227: mknod d1/d13/d3d/c4f 0 2024-05-12T04:01:12.224 INFO:tasks.workunit.client.1.smithi033.stdout:8/364: creat d1/d16/f72 x:0 0 0 2024-05-12T04:01:12.225 INFO:tasks.workunit.client.1.smithi033.stdout:7/584: getdents de/d2f/d42 0 2024-05-12T04:01:12.226 INFO:tasks.workunit.client.1.smithi033.stdout:7/585: fsync de/d2b/d6c/f72 0 2024-05-12T04:01:12.228 INFO:tasks.workunit.client.0.smithi032.stdout:9/194: dwrite d0/d3/f25 [0,4194304] 0 2024-05-12T04:01:12.229 INFO:tasks.workunit.client.1.smithi033.stdout:8/365: dwrite d1/d16/d20/f33 [0,4194304] 0 2024-05-12T04:01:12.230 INFO:tasks.workunit.client.1.smithi033.stdout:0/467: mknod d6/dc/c9a 0 2024-05-12T04:01:12.231 INFO:tasks.workunit.client.0.smithi032.stdout:6/248: mknod d7/c54 0 2024-05-12T04:01:12.234 INFO:tasks.workunit.client.1.smithi033.stdout:1/394: mknod d11/d1c/d59/c83 0 2024-05-12T04:01:12.235 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:11 smithi032 ceph-mon[40008]: pgmap v34: 65 pgs: 65 active+clean; 1.5 GiB data, 5.9 GiB used, 531 GiB / 536 GiB avail; 34 MiB/s rd, 203 MiB/s wr, 344 op/s 2024-05-12T04:01:12.235 INFO:tasks.workunit.client.1.smithi033.stdout:8/366: dread d1/d4/fd [4194304,4194304] 0 2024-05-12T04:01:12.237 INFO:tasks.workunit.client.1.smithi033.stdout:8/367: fsync d1/d4/d1d/d36/f50 0 2024-05-12T04:01:12.238 INFO:tasks.workunit.client.0.smithi032.stdout:8/299: mknod d0/d9/d1b/d27/d4c/d4e/c64 0 2024-05-12T04:01:12.238 INFO:tasks.workunit.client.0.smithi032.stdout:3/293: dwrite f12 [0,4194304] 0 2024-05-12T04:01:12.242 INFO:tasks.workunit.client.0.smithi032.stdout:9/195: unlink d0/d3/d6/d7/da/d17/d3f/l23 0 2024-05-12T04:01:12.246 INFO:tasks.workunit.client.0.smithi032.stdout:6/249: symlink d7/de/d12/d1a/l55 0 2024-05-12T04:01:12.246 INFO:tasks.workunit.client.0.smithi032.stdout:6/250: readlink d7/de/l44 0 2024-05-12T04:01:12.247 INFO:tasks.workunit.client.0.smithi032.stdout:8/300: unlink d0/d7/c54 0 2024-05-12T04:01:12.249 INFO:tasks.workunit.client.0.smithi032.stdout:8/301: stat d0/l32 0 2024-05-12T04:01:12.250 INFO:tasks.workunit.client.1.smithi033.stdout:9/660: unlink d2/dd/d39/la1 0 2024-05-12T04:01:12.251 INFO:tasks.workunit.client.1.smithi033.stdout:9/661: readlink d2/d4/d2b/d6d/d7a/l8b 0 2024-05-12T04:01:12.251 INFO:tasks.workunit.client.0.smithi032.stdout:8/302: chown d0/l51 4 1 2024-05-12T04:01:12.253 INFO:tasks.workunit.client.0.smithi032.stdout:9/196: dwrite d0/d3/d1b/f38 [0,4194304] 0 2024-05-12T04:01:12.254 INFO:tasks.workunit.client.1.smithi033.stdout:6/436: creat d4/d1a/d67/d68/f8f x:0 0 0 2024-05-12T04:01:12.260 INFO:tasks.workunit.client.0.smithi032.stdout:2/228: dread d1/d13/f1f [0,4194304] 0 2024-05-12T04:01:12.261 INFO:tasks.workunit.client.0.smithi032.stdout:2/229: chown d1/c2d 29982094 1 2024-05-12T04:01:12.261 INFO:tasks.workunit.client.1.smithi033.stdout:3/411: link db/d17/f1b db/d59/f8d 0 2024-05-12T04:01:12.263 INFO:tasks.workunit.client.1.smithi033.stdout:3/412: write db/d17/d25/f58 [1686995,90002] 0 2024-05-12T04:01:12.270 INFO:tasks.workunit.client.0.smithi032.stdout:2/230: symlink d1/d34/l50 0 2024-05-12T04:01:12.274 INFO:tasks.workunit.client.0.smithi032.stdout:2/231: mknod d1/d13/d3d/d41/d2c/c51 0 2024-05-12T04:01:12.276 INFO:tasks.workunit.client.1.smithi033.stdout:1/395: mkdir d11/d17/d84 0 2024-05-12T04:01:12.277 INFO:tasks.workunit.client.1.smithi033.stdout:1/396: dread - d11/f4f zero size 2024-05-12T04:01:12.277 INFO:tasks.workunit.client.1.smithi033.stdout:0/468: dwrite d6/d11/f33 [0,4194304] 0 2024-05-12T04:01:12.279 INFO:tasks.workunit.client.0.smithi032.stdout:1/266: truncate d1/f26 3547651 0 2024-05-12T04:01:12.280 INFO:tasks.workunit.client.0.smithi032.stdout:1/267: truncate d1/dd/f53 234584 0 2024-05-12T04:01:12.281 INFO:tasks.workunit.client.1.smithi033.stdout:8/368: mkdir d1/d16/d27/d4d/d73 0 2024-05-12T04:01:12.283 INFO:tasks.workunit.client.1.smithi033.stdout:8/369: dread d1/d16/d27/d4d/f5e [0,4194304] 0 2024-05-12T04:01:12.285 INFO:tasks.workunit.client.1.smithi033.stdout:9/662: symlink d2/d4/d2b/dc7/lf2 0 2024-05-12T04:01:12.285 INFO:tasks.workunit.client.0.smithi032.stdout:2/232: mkdir d1/d13/d3d/d41/d52 0 2024-05-12T04:01:12.288 INFO:tasks.workunit.client.1.smithi033.stdout:6/437: creat d4/d6e/f90 x:0 0 0 2024-05-12T04:01:12.289 INFO:tasks.workunit.client.0.smithi032.stdout:1/268: dwrite d1/d7/d30/d18/f41 [0,4194304] 0 2024-05-12T04:01:12.290 INFO:tasks.workunit.client.1.smithi033.stdout:2/434: link f3 dc/d13/d1f/f93 0 2024-05-12T04:01:12.291 INFO:tasks.workunit.client.1.smithi033.stdout:2/435: truncate dc/d13/d1f/f69 889117 0 2024-05-12T04:01:12.293 INFO:tasks.workunit.client.0.smithi032.stdout:2/233: creat d1/d34/f53 x:0 0 0 2024-05-12T04:01:12.293 INFO:tasks.workunit.client.1.smithi033.stdout:5/362: write d3/d15/d16/d45/d4b/f56 [475883,24567] 0 2024-05-12T04:01:12.294 INFO:tasks.workunit.client.0.smithi032.stdout:2/234: fdatasync d1/d34/d3e/f45 0 2024-05-12T04:01:12.294 INFO:tasks.workunit.client.1.smithi033.stdout:5/363: chown d3/d27 5031816 1 2024-05-12T04:01:12.295 INFO:tasks.workunit.client.0.smithi032.stdout:1/269: dread d1/f36 [0,4194304] 0 2024-05-12T04:01:12.296 INFO:tasks.workunit.client.1.smithi033.stdout:5/364: dread - d3/d15/d16/d45/d4b/f6c zero size 2024-05-12T04:01:12.302 INFO:tasks.workunit.client.0.smithi032.stdout:3/294: write fd [2832242,73148] 0 2024-05-12T04:01:12.306 INFO:tasks.workunit.client.0.smithi032.stdout:3/295: dread d15/d20/d22/f2e [8388608,4194304] 0 2024-05-12T04:01:12.310 INFO:tasks.workunit.client.0.smithi032.stdout:8/303: write d0/f5 [8575765,29708] 0 2024-05-12T04:01:12.311 INFO:tasks.workunit.client.0.smithi032.stdout:3/296: symlink d15/d24/l62 0 2024-05-12T04:01:12.312 INFO:tasks.workunit.client.0.smithi032.stdout:6/251: write d7/de/f14 [539625,53155] 0 2024-05-12T04:01:12.313 INFO:tasks.workunit.client.0.smithi032.stdout:1/270: dwrite d1/d25/f4d [4194304,4194304] 0 2024-05-12T04:01:12.314 INFO:tasks.workunit.client.0.smithi032.stdout:8/304: write d0/f2f [4568331,129429] 0 2024-05-12T04:01:12.315 INFO:tasks.workunit.client.0.smithi032.stdout:3/297: chown d15/d33/f5c 29 1 2024-05-12T04:01:12.323 INFO:tasks.workunit.client.0.smithi032.stdout:6/252: creat d7/de/d12/d1a/f56 x:0 0 0 2024-05-12T04:01:12.324 INFO:tasks.workunit.client.0.smithi032.stdout:9/197: dwrite d0/d3/d6/d7/da/f33 [0,4194304] 0 2024-05-12T04:01:12.327 INFO:tasks.workunit.client.0.smithi032.stdout:3/298: creat d15/d24/d2a/f63 x:0 0 0 2024-05-12T04:01:12.327 INFO:tasks.workunit.client.1.smithi033.stdout:4/435: truncate dd/d11/d31/d66/f6a 392139 0 2024-05-12T04:01:12.328 INFO:tasks.workunit.client.1.smithi033.stdout:2/436: dwrite dc/f31 [0,4194304] 0 2024-05-12T04:01:12.329 INFO:tasks.workunit.client.1.smithi033.stdout:5/365: chown d3/d12/d29/d2b/f46 78036773 1 2024-05-12T04:01:12.329 INFO:tasks.workunit.client.1.smithi033.stdout:0/469: symlink d6/db/d2c/d73/l9b 0 2024-05-12T04:01:12.329 INFO:tasks.workunit.client.0.smithi032.stdout:8/305: creat d0/d9/d13/d21/d35/d37/d56/f65 x:0 0 0 2024-05-12T04:01:12.330 INFO:tasks.workunit.client.1.smithi033.stdout:9/663: rename d2/d4/f15 to d2/d4/d2b/d6d/ff3 0 2024-05-12T04:01:12.330 INFO:tasks.workunit.client.1.smithi033.stdout:2/437: stat dc/d17/f63 0 2024-05-12T04:01:12.330 INFO:tasks.workunit.client.1.smithi033.stdout:5/366: chown d3/d15/d5a 9853 1 2024-05-12T04:01:12.331 INFO:tasks.workunit.client.0.smithi032.stdout:8/306: chown d0/d9/f31 117 1 2024-05-12T04:01:12.331 INFO:tasks.workunit.client.1.smithi033.stdout:3/413: symlink db/d17/d3a/d67/d6d/l8e 0 2024-05-12T04:01:12.332 INFO:tasks.workunit.client.0.smithi032.stdout:8/307: readlink d0/l23 0 2024-05-12T04:01:12.335 INFO:tasks.workunit.client.0.smithi032.stdout:8/308: chown d0/d9/d1b/d33/f3b 247 1 2024-05-12T04:01:12.336 INFO:tasks.workunit.client.1.smithi033.stdout:9/664: chown d2/d4/d2b/d78/d76/c9c 987 1 2024-05-12T04:01:12.337 INFO:tasks.workunit.client.0.smithi032.stdout:8/309: readlink d0/d4b/l5e 0 2024-05-12T04:01:12.337 INFO:tasks.workunit.client.1.smithi033.stdout:2/438: write dc/d13/d1f/d6e/f81 [4400991,98366] 0 2024-05-12T04:01:12.338 INFO:tasks.workunit.client.0.smithi032.stdout:3/299: dwrite d15/d24/d2a/f63 [0,4194304] 0 2024-05-12T04:01:12.340 INFO:tasks.workunit.client.0.smithi032.stdout:3/300: stat d15/d20/d22/d4f 0 2024-05-12T04:01:12.340 INFO:tasks.workunit.client.1.smithi033.stdout:9/665: write d2/d4/d29/d3c/d52/f5e [3052270,58403] 0 2024-05-12T04:01:12.340 INFO:tasks.workunit.client.1.smithi033.stdout:7/586: dwrite de/d80/f98 [0,4194304] 0 2024-05-12T04:01:12.348 INFO:tasks.workunit.client.0.smithi032.stdout:6/253: mkdir d7/de/d57 0 2024-05-12T04:01:12.352 INFO:tasks.workunit.client.1.smithi033.stdout:5/367: mkdir d3/d12/d72 0 2024-05-12T04:01:12.353 INFO:tasks.workunit.client.0.smithi032.stdout:8/310: dwrite d0/d7/f15 [0,4194304] 0 2024-05-12T04:01:12.360 INFO:tasks.workunit.client.1.smithi033.stdout:4/436: dread dd/d11/f19 [0,4194304] 0 2024-05-12T04:01:12.360 INFO:tasks.workunit.client.0.smithi032.stdout:1/271: link d1/d7/d30/l5a d1/d7/d49/l5c 0 2024-05-12T04:01:12.361 INFO:tasks.workunit.client.0.smithi032.stdout:2/235: dwrite d1/f31 [0,4194304] 0 2024-05-12T04:01:12.362 INFO:tasks.workunit.client.1.smithi033.stdout:2/439: readlink dc/l2a 0 2024-05-12T04:01:12.363 INFO:tasks.workunit.client.0.smithi032.stdout:6/254: creat d7/de/d2e/f58 x:0 0 0 2024-05-12T04:01:12.367 INFO:tasks.workunit.client.1.smithi033.stdout:4/437: dwrite dd/f32 [0,4194304] 0 2024-05-12T04:01:12.368 INFO:tasks.workunit.client.0.smithi032.stdout:3/301: rename d15/d33/d3f/f5e to d15/d20/d26/f64 0 2024-05-12T04:01:12.368 INFO:tasks.workunit.client.0.smithi032.stdout:3/302: fsync d15/f4e 0 2024-05-12T04:01:12.369 INFO:tasks.workunit.client.0.smithi032.stdout:3/303: fsync d15/d33/f4c 0 2024-05-12T04:01:12.372 INFO:tasks.workunit.client.1.smithi033.stdout:7/587: rmdir de/d20/da9 39 2024-05-12T04:01:12.376 INFO:tasks.workunit.client.0.smithi032.stdout:3/304: dwrite f12 [0,4194304] 0 2024-05-12T04:01:12.376 INFO:tasks.workunit.client.1.smithi033.stdout:5/368: mkdir d3/d12/d25/d26/d2e/d54/d57/d73 0 2024-05-12T04:01:12.378 INFO:tasks.workunit.client.1.smithi033.stdout:3/414: getdents db/d17/d3e/d65/d88 0 2024-05-12T04:01:12.380 INFO:tasks.workunit.client.0.smithi032.stdout:6/255: creat d7/f59 x:0 0 0 2024-05-12T04:01:12.381 INFO:tasks.workunit.client.0.smithi032.stdout:6/256: chown d7/de/d12/d1a/d20/f42 59157425 1 2024-05-12T04:01:12.382 INFO:tasks.workunit.client.0.smithi032.stdout:2/236: creat d1/d13/d3d/d41/d2c/f54 x:0 0 0 2024-05-12T04:01:12.383 INFO:tasks.workunit.client.1.smithi033.stdout:2/440: dread - dc/d17/d27/f60 zero size 2024-05-12T04:01:12.384 INFO:tasks.workunit.client.1.smithi033.stdout:2/441: chown c5 6 1 2024-05-12T04:01:12.385 INFO:tasks.workunit.client.1.smithi033.stdout:4/438: mknod dd/d11/d31/d38/c8b 0 2024-05-12T04:01:12.390 INFO:tasks.workunit.client.0.smithi032.stdout:8/311: mkdir d0/d9/d1b/d66 0 2024-05-12T04:01:12.391 INFO:tasks.workunit.client.0.smithi032.stdout:8/312: chown d0/d9/d13/d21/d35/d37/d56/f65 2462134 1 2024-05-12T04:01:12.393 INFO:tasks.workunit.client.0.smithi032.stdout:3/305: mkdir d15/d1b/d65 0 2024-05-12T04:01:12.395 INFO:tasks.workunit.client.1.smithi033.stdout:1/397: link d11/d17/d27/d36/d3f/f61 d11/d1c/d43/f85 0 2024-05-12T04:01:12.396 INFO:tasks.workunit.client.1.smithi033.stdout:1/398: read - d11/d17/d27/d76/f72 zero size 2024-05-12T04:01:12.397 INFO:tasks.workunit.client.0.smithi032.stdout:6/257: creat d7/de/d2e/f5a x:0 0 0 2024-05-12T04:01:12.400 INFO:tasks.workunit.client.0.smithi032.stdout:9/198: dwrite d0/f28 [0,4194304] 0 2024-05-12T04:01:12.403 INFO:tasks.workunit.client.0.smithi032.stdout:2/237: rename d1/d34/c36 to d1/d13/c55 0 2024-05-12T04:01:12.405 INFO:tasks.workunit.client.1.smithi033.stdout:9/666: dwrite d2/d4/f16 [0,4194304] 0 2024-05-12T04:01:12.405 INFO:tasks.workunit.client.0.smithi032.stdout:3/306: symlink d15/d1b/d43/l66 0 2024-05-12T04:01:12.407 INFO:tasks.workunit.client.0.smithi032.stdout:3/307: fdatasync d15/d20/d22/d4f/f53 0 2024-05-12T04:01:12.416 INFO:tasks.workunit.client.0.smithi032.stdout:2/238: creat d1/d13/d3d/d41/d2c/f56 x:0 0 0 2024-05-12T04:01:12.418 INFO:tasks.workunit.client.0.smithi032.stdout:2/239: chown d1/d13/d3d/c4f 244 1 2024-05-12T04:01:12.418 INFO:tasks.workunit.client.0.smithi032.stdout:8/313: link d0/c1e d0/d9/d13/d21/d35/d37/d56/d63/c67 0 2024-05-12T04:01:12.419 INFO:tasks.workunit.client.0.smithi032.stdout:2/240: chown d1/f21 73054490 1 2024-05-12T04:01:12.421 INFO:tasks.workunit.client.0.smithi032.stdout:9/199: dwrite d0/d3/d6/d7/d27/f3d [0,4194304] 0 2024-05-12T04:01:12.422 INFO:tasks.workunit.client.0.smithi032.stdout:2/241: truncate d1/d13/f4c 480464 0 2024-05-12T04:01:12.422 INFO:tasks.workunit.client.1.smithi033.stdout:8/370: sync 2024-05-12T04:01:12.423 INFO:tasks.workunit.client.1.smithi033.stdout:6/438: sync 2024-05-12T04:01:12.423 INFO:tasks.workunit.client.0.smithi032.stdout:9/200: write d0/d3/d6/f30 [5026986,87544] 0 2024-05-12T04:01:12.425 INFO:tasks.workunit.client.1.smithi033.stdout:8/371: write d1/d16/d20/f2f [4386238,114454] 0 2024-05-12T04:01:12.428 INFO:tasks.workunit.client.0.smithi032.stdout:3/308: link d15/d20/c32 d15/d33/d3f/c67 0 2024-05-12T04:01:12.433 INFO:tasks.workunit.client.0.smithi032.stdout:9/201: stat d0/d3/d6/d7/da/d17 0 2024-05-12T04:01:12.435 INFO:tasks.workunit.client.0.smithi032.stdout:6/258: link d7/de/d12/d1a/d20/l43 d7/de/d12/d1a/d20/l5b 0 2024-05-12T04:01:12.438 INFO:tasks.workunit.client.1.smithi033.stdout:4/439: dread dd/d11/d31/d66/d42/f63 [0,4194304] 0 2024-05-12T04:01:12.438 INFO:tasks.workunit.client.0.smithi032.stdout:2/242: dwrite d1/d34/d3e/f45 [0,4194304] 0 2024-05-12T04:01:12.439 INFO:tasks.workunit.client.0.smithi032.stdout:1/272: dwrite d1/d7/d30/d35/f42 [0,4194304] 0 2024-05-12T04:01:12.440 INFO:tasks.workunit.client.0.smithi032.stdout:6/259: read - d7/de/d12/f41 zero size 2024-05-12T04:01:12.444 INFO:tasks.workunit.client.1.smithi033.stdout:7/588: dwrite de/d20/d38/f61 [0,4194304] 0 2024-05-12T04:01:12.445 INFO:tasks.workunit.client.0.smithi032.stdout:3/309: read d15/d20/d22/f2e [2506520,1821] 0 2024-05-12T04:01:12.447 INFO:tasks.workunit.client.0.smithi032.stdout:9/202: dwrite d0/f28 [0,4194304] 0 2024-05-12T04:01:12.448 INFO:tasks.workunit.client.1.smithi033.stdout:1/399: creat d11/d1c/d59/f86 x:0 0 0 2024-05-12T04:01:12.448 INFO:tasks.workunit.client.0.smithi032.stdout:3/310: dread - d15/f45 zero size 2024-05-12T04:01:12.450 INFO:tasks.workunit.client.0.smithi032.stdout:3/311: chown d15/f55 24267 1 2024-05-12T04:01:12.454 INFO:tasks.workunit.client.1.smithi033.stdout:9/667: creat d2/d4/d29/d32/db7/ff4 x:0 0 0 2024-05-12T04:01:12.455 INFO:tasks.workunit.client.0.smithi032.stdout:8/314: creat d0/f68 x:0 0 0 2024-05-12T04:01:12.460 INFO:tasks.workunit.client.0.smithi032.stdout:1/273: write d1/d7/d30/f2c [5969212,70668] 0 2024-05-12T04:01:12.461 INFO:tasks.workunit.client.0.smithi032.stdout:4/182: sync 2024-05-12T04:01:12.463 INFO:tasks.workunit.client.0.smithi032.stdout:1/274: write d1/d7/d30/d18/f2b [943928,113927] 0 2024-05-12T04:01:12.465 INFO:tasks.workunit.client.0.smithi032.stdout:6/260: dread f3 [0,4194304] 0 2024-05-12T04:01:12.465 INFO:tasks.workunit.client.0.smithi032.stdout:1/275: fsync d1/d7/d30/d18/f41 0 2024-05-12T04:01:12.465 INFO:tasks.workunit.client.1.smithi033.stdout:2/442: creat dc/d17/d27/f94 x:0 0 0 2024-05-12T04:01:12.471 INFO:tasks.workunit.client.1.smithi033.stdout:7/589: symlink de/df/d17/d1b/d46/lcb 0 2024-05-12T04:01:12.472 INFO:tasks.workunit.client.1.smithi033.stdout:0/470: sync 2024-05-12T04:01:12.472 INFO:tasks.workunit.client.0.smithi032.stdout:2/243: creat d1/d13/d3d/d41/d52/f57 x:0 0 0 2024-05-12T04:01:12.474 INFO:tasks.workunit.client.1.smithi033.stdout:1/400: mkdir d11/d17/d27/d36/d3f/d69/d87 0 2024-05-12T04:01:12.474 INFO:tasks.workunit.client.0.smithi032.stdout:2/244: chown d1/d13/d3d/d41/d2c/l38 23116 1 2024-05-12T04:01:12.476 INFO:tasks.workunit.client.1.smithi033.stdout:7/590: dwrite de/d80/f98 [0,4194304] 0 2024-05-12T04:01:12.477 INFO:tasks.workunit.client.0.smithi032.stdout:3/312: mknod d15/d1b/d65/c68 0 2024-05-12T04:01:12.478 INFO:tasks.workunit.client.0.smithi032.stdout:3/313: chown d15/d1b/d43/l4a 226698695 1 2024-05-12T04:01:12.478 INFO:tasks.workunit.client.1.smithi033.stdout:7/591: fdatasync de/df/d17/d1e/d24/fad 0 2024-05-12T04:01:12.485 INFO:tasks.workunit.client.1.smithi033.stdout:7/592: dwrite de/df/d7a/f9f [0,4194304] 0 2024-05-12T04:01:12.486 INFO:tasks.workunit.client.0.smithi032.stdout:6/261: mkdir d7/d9/d5c 0 2024-05-12T04:01:12.493 INFO:tasks.workunit.client.1.smithi033.stdout:6/439: creat d4/d1a/f91 x:0 0 0 2024-05-12T04:01:12.494 INFO:tasks.workunit.client.0.smithi032.stdout:8/315: dread d0/d9/d1b/d33/f45 [0,4194304] 0 2024-05-12T04:01:12.496 INFO:tasks.workunit.client.0.smithi032.stdout:4/183: creat d0/d3/d24/f2d x:0 0 0 2024-05-12T04:01:12.497 INFO:tasks.workunit.client.1.smithi033.stdout:4/440: dwrite dd/f1b [0,4194304] 0 2024-05-12T04:01:12.498 INFO:tasks.workunit.client.0.smithi032.stdout:4/184: write d0/d3/df/f29 [594695,115742] 0 2024-05-12T04:01:12.500 INFO:tasks.workunit.client.1.smithi033.stdout:8/372: creat d1/d4/d1d/d36/f74 x:0 0 0 2024-05-12T04:01:12.502 INFO:tasks.workunit.client.1.smithi033.stdout:7/593: rmdir de/d2f/d42 39 2024-05-12T04:01:12.504 INFO:tasks.workunit.client.0.smithi032.stdout:6/262: truncate d7/de/d12/f18 1154396 0 2024-05-12T04:01:12.506 INFO:tasks.workunit.client.0.smithi032.stdout:4/185: dwrite d0/d3/f1c [0,4194304] 0 2024-05-12T04:01:12.509 INFO:tasks.workunit.client.1.smithi033.stdout:8/373: dwrite d1/d16/f72 [0,4194304] 0 2024-05-12T04:01:12.509 INFO:tasks.workunit.client.0.smithi032.stdout:4/186: write d0/d3/d12/d1d/f2a [1222305,50464] 0 2024-05-12T04:01:12.511 INFO:tasks.workunit.client.1.smithi033.stdout:9/668: dwrite d2/d4/d2b/d78/f88 [4194304,4194304] 0 2024-05-12T04:01:12.511 INFO:tasks.workunit.client.0.smithi032.stdout:1/276: rename d1/f26 to d1/d7/f5d 0 2024-05-12T04:01:12.515 INFO:tasks.workunit.client.0.smithi032.stdout:2/245: mkdir d1/d13/d3d/d41/d58 0 2024-05-12T04:01:12.516 INFO:tasks.workunit.client.0.smithi032.stdout:3/314: symlink d15/l69 0 2024-05-12T04:01:12.519 INFO:tasks.workunit.client.0.smithi032.stdout:9/203: getdents d0/d3/d6/d7/da/d17/d3f/d13 0 2024-05-12T04:01:12.522 INFO:tasks.workunit.client.1.smithi033.stdout:4/441: dwrite dd/d11/d5b/f86 [4194304,4194304] 0 2024-05-12T04:01:12.523 INFO:tasks.workunit.client.1.smithi033.stdout:4/442: dread - dd/d69/f7d zero size 2024-05-12T04:01:12.526 INFO:tasks.workunit.client.1.smithi033.stdout:8/374: creat d1/d16/f75 x:0 0 0 2024-05-12T04:01:12.527 INFO:tasks.workunit.client.0.smithi032.stdout:7/268: sync 2024-05-12T04:01:12.528 INFO:tasks.workunit.client.0.smithi032.stdout:5/323: sync 2024-05-12T04:01:12.528 INFO:tasks.workunit.client.0.smithi032.stdout:0/307: sync 2024-05-12T04:01:12.528 INFO:tasks.workunit.client.0.smithi032.stdout:4/187: symlink d0/d3/d24/l2e 0 2024-05-12T04:01:12.529 INFO:tasks.workunit.client.0.smithi032.stdout:0/308: chown df/d30/d4e 20 1 2024-05-12T04:01:12.529 INFO:tasks.workunit.client.1.smithi033.stdout:1/401: write f8 [5990902,77258] 0 2024-05-12T04:01:12.530 INFO:tasks.workunit.client.0.smithi032.stdout:1/277: mknod d1/d7/d30/d18/c5e 0 2024-05-12T04:01:12.530 INFO:tasks.workunit.client.0.smithi032.stdout:7/269: write dd/d1b/d1e/d45/d55/f5b [293233,16742] 0 2024-05-12T04:01:12.533 INFO:tasks.workunit.client.0.smithi032.stdout:4/188: write d0/d3/d24/f2d [360124,15350] 0 2024-05-12T04:01:12.535 INFO:tasks.workunit.client.0.smithi032.stdout:4/189: stat d0/d3/d24 0 2024-05-12T04:01:12.535 INFO:tasks.workunit.client.1.smithi033.stdout:6/440: write d4/d5/f10 [435364,23997] 0 2024-05-12T04:01:12.535 INFO:tasks.workunit.client.1.smithi033.stdout:6/441: chown d4/d1a/d4e/d4f 5217 1 2024-05-12T04:01:12.540 INFO:tasks.workunit.client.0.smithi032.stdout:3/315: rmdir d15/d1b 39 2024-05-12T04:01:12.541 INFO:tasks.workunit.client.1.smithi033.stdout:4/443: rename dd/d11/d31/d38/c43 to dd/d11/d5b/c8c 0 2024-05-12T04:01:12.541 INFO:tasks.workunit.client.1.smithi033.stdout:7/594: dread de/df/d17/d1b/d36/f8a [0,4194304] 0 2024-05-12T04:01:12.543 INFO:tasks.workunit.client.0.smithi032.stdout:6/263: dread d7/de/f1b [0,4194304] 0 2024-05-12T04:01:12.544 INFO:tasks.workunit.client.1.smithi033.stdout:8/375: creat d1/d16/d27/d28/f76 x:0 0 0 2024-05-12T04:01:12.547 INFO:tasks.workunit.client.0.smithi032.stdout:4/190: dwrite d0/d3/d24/f2d [0,4194304] 0 2024-05-12T04:01:12.553 INFO:tasks.workunit.client.0.smithi032.stdout:8/316: link d0/d9/f2d d0/d9/d1b/d27/f69 0 2024-05-12T04:01:12.555 INFO:tasks.workunit.client.0.smithi032.stdout:0/309: mkdir df/d10/d48/d70 0 2024-05-12T04:01:12.556 INFO:tasks.workunit.client.0.smithi032.stdout:5/324: chown d4/d7/d14/d26/d33/d4a/c37 9659 1 2024-05-12T04:01:12.557 INFO:tasks.workunit.client.0.smithi032.stdout:1/278: dread d1/d7/d30/d35/f45 [0,4194304] 0 2024-05-12T04:01:12.560 INFO:tasks.workunit.client.0.smithi032.stdout:4/191: dwrite d0/f13 [0,4194304] 0 2024-05-12T04:01:12.561 INFO:tasks.workunit.client.1.smithi033.stdout:8/376: rename d1/d16/d29/d63/f6a to d1/d16/d27/d28/f77 0 2024-05-12T04:01:12.561 INFO:tasks.workunit.client.0.smithi032.stdout:7/270: chown dd/d1b/c56 241089 1 2024-05-12T04:01:12.561 INFO:tasks.workunit.client.1.smithi033.stdout:1/402: write d11/d1c/f33 [419723,9745] 0 2024-05-12T04:01:12.561 INFO:tasks.workunit.client.1.smithi033.stdout:6/442: dread d4/f56 [0,4194304] 0 2024-05-12T04:01:12.563 INFO:tasks.workunit.client.1.smithi033.stdout:3/415: sync 2024-05-12T04:01:12.563 INFO:tasks.workunit.client.1.smithi033.stdout:5/369: sync 2024-05-12T04:01:12.563 INFO:tasks.workunit.client.1.smithi033.stdout:6/443: readlink d4/d43/d57/l64 0 2024-05-12T04:01:12.565 INFO:tasks.workunit.client.1.smithi033.stdout:4/444: creat dd/d11/d75/f8d x:0 0 0 2024-05-12T04:01:12.567 INFO:tasks.workunit.client.1.smithi033.stdout:4/445: write dd/d11/d31/f6c [181594,50107] 0 2024-05-12T04:01:12.569 INFO:tasks.workunit.client.0.smithi032.stdout:4/192: dread d0/d3/df/f26 [0,4194304] 0 2024-05-12T04:01:12.570 INFO:tasks.workunit.client.0.smithi032.stdout:2/246: dwrite d1/d13/d3d/d41/f3a [0,4194304] 0 2024-05-12T04:01:12.570 INFO:tasks.workunit.client.1.smithi033.stdout:8/377: creat d1/d4/d1d/f78 x:0 0 0 2024-05-12T04:01:12.574 INFO:tasks.workunit.client.1.smithi033.stdout:9/669: getdents d2/d4/d29/d32/db7 0 2024-05-12T04:01:12.575 INFO:tasks.workunit.client.0.smithi032.stdout:9/204: dwrite d0/d3/d6/d7/fc [0,4194304] 0 2024-05-12T04:01:12.580 INFO:tasks.workunit.client.1.smithi033.stdout:6/444: mknod d4/d1a/d4e/d4f/c92 0 2024-05-12T04:01:12.582 INFO:tasks.workunit.client.0.smithi032.stdout:1/279: mknod d1/d7/d30/d18/c5f 0 2024-05-12T04:01:12.585 INFO:tasks.workunit.client.0.smithi032.stdout:4/193: dwrite d0/fa [0,4194304] 0 2024-05-12T04:01:12.586 INFO:tasks.workunit.client.1.smithi033.stdout:1/403: symlink d11/d17/d27/d36/d3f/d69/d87/l88 0 2024-05-12T04:01:12.588 INFO:tasks.workunit.client.0.smithi032.stdout:3/316: dwrite d15/d33/f38 [0,4194304] 0 2024-05-12T04:01:12.591 INFO:tasks.workunit.client.1.smithi033.stdout:9/670: unlink d2/d4/d2b/d6d/d4c/f95 0 2024-05-12T04:01:12.591 INFO:tasks.workunit.client.0.smithi032.stdout:6/264: creat d7/d9/d48/f5d x:0 0 0 2024-05-12T04:01:12.591 INFO:tasks.workunit.client.0.smithi032.stdout:3/317: dread - d15/d20/d22/d4f/f53 zero size 2024-05-12T04:01:12.592 INFO:tasks.workunit.client.1.smithi033.stdout:3/416: mknod db/d17/d3a/d8a/c8f 0 2024-05-12T04:01:12.593 INFO:tasks.workunit.client.0.smithi032.stdout:6/265: read d7/de/d12/d1a/d20/f42 [235260,20260] 0 2024-05-12T04:01:12.594 INFO:tasks.workunit.client.1.smithi033.stdout:5/370: mkdir d3/d12/d25/d26/d2e/d54/d57/d73/d74 0 2024-05-12T04:01:12.597 INFO:tasks.workunit.client.0.smithi032.stdout:0/310: symlink df/l71 0 2024-05-12T04:01:12.598 INFO:tasks.workunit.client.1.smithi033.stdout:7/595: getdents de/d2b/d6c/da0 0 2024-05-12T04:01:12.599 INFO:tasks.workunit.client.0.smithi032.stdout:8/317: symlink d0/d9/d13/d21/d35/d37/d56/d63/l6a 0 2024-05-12T04:01:12.601 INFO:tasks.workunit.client.0.smithi032.stdout:2/247: creat d1/d13/d3d/d41/d2c/f59 x:0 0 0 2024-05-12T04:01:12.604 INFO:tasks.workunit.client.1.smithi033.stdout:3/417: unlink db/d59/c5c 0 2024-05-12T04:01:12.605 INFO:tasks.workunit.client.0.smithi032.stdout:1/280: creat d1/d7/d30/f60 x:0 0 0 2024-05-12T04:01:12.610 INFO:tasks.workunit.client.1.smithi033.stdout:6/445: symlink d4/d1a/d67/d68/d88/l93 0 2024-05-12T04:01:12.610 INFO:tasks.workunit.client.1.smithi033.stdout:7/596: mkdir de/df/d17/d1e/dcc 0 2024-05-12T04:01:12.611 INFO:tasks.workunit.client.1.smithi033.stdout:8/378: dread d1/f2 [0,4194304] 0 2024-05-12T04:01:12.611 INFO:tasks.workunit.client.1.smithi033.stdout:7/597: chown de/df/d60/d75/ca5 212 1 2024-05-12T04:01:12.613 INFO:tasks.workunit.client.1.smithi033.stdout:4/446: link dd/d11/d31/d66/l30 dd/d11/d31/d38/l8e 0 2024-05-12T04:01:12.613 INFO:tasks.workunit.client.1.smithi033.stdout:7/598: read de/df/d17/d1e/f23 [128432,5784] 0 2024-05-12T04:01:12.614 INFO:tasks.workunit.client.0.smithi032.stdout:3/318: mkdir d15/d20/d26/d6a 0 2024-05-12T04:01:12.619 INFO:tasks.workunit.client.1.smithi033.stdout:5/371: symlink d3/d12/d25/d26/d2e/d54/d57/d73/d74/l75 0 2024-05-12T04:01:12.620 INFO:tasks.workunit.client.1.smithi033.stdout:6/446: mkdir d4/d1a/d4e/d4f/d94 0 2024-05-12T04:01:12.621 INFO:tasks.workunit.client.0.smithi032.stdout:8/318: symlink d0/d9/d13/d21/d35/d37/d56/d63/l6b 0 2024-05-12T04:01:12.621 INFO:tasks.workunit.client.0.smithi032.stdout:8/319: readlink d0/d7/l38 0 2024-05-12T04:01:12.623 INFO:tasks.workunit.client.0.smithi032.stdout:2/248: symlink d1/d34/d4a/l5a 0 2024-05-12T04:01:12.627 INFO:tasks.workunit.client.1.smithi033.stdout:4/447: chown dd/d11/d31/d66/c18 19 1 2024-05-12T04:01:12.628 INFO:tasks.workunit.client.0.smithi032.stdout:1/281: mkdir d1/d7/d30/d61 0 2024-05-12T04:01:12.629 INFO:tasks.workunit.client.0.smithi032.stdout:4/194: symlink d0/d3/d12/l2f 0 2024-05-12T04:01:12.630 INFO:tasks.workunit.client.0.smithi032.stdout:4/195: read d0/fa [1181243,15574] 0 2024-05-12T04:01:12.635 INFO:tasks.workunit.client.1.smithi033.stdout:6/447: symlink d4/d5/l95 0 2024-05-12T04:01:12.636 INFO:tasks.workunit.client.0.smithi032.stdout:9/205: creat d0/d3/f42 x:0 0 0 2024-05-12T04:01:12.637 INFO:tasks.workunit.client.0.smithi032.stdout:8/320: creat d0/d7/f6c x:0 0 0 2024-05-12T04:01:12.638 INFO:tasks.workunit.client.0.smithi032.stdout:0/311: dread df/d10/d3d/d4a/f2d [0,4194304] 0 2024-05-12T04:01:12.639 INFO:tasks.workunit.client.0.smithi032.stdout:7/271: link fa dd/d1b/d1e/f5d 0 2024-05-12T04:01:12.641 INFO:tasks.workunit.client.1.smithi033.stdout:7/599: mknod de/df/d17/d1e/dc4/ccd 0 2024-05-12T04:01:12.643 INFO:tasks.workunit.client.1.smithi033.stdout:9/671: getdents d2/d4/d2b/d6d/d4c 0 2024-05-12T04:01:12.644 INFO:tasks.workunit.client.0.smithi032.stdout:1/282: dread d1/d7/f5d [0,4194304] 0 2024-05-12T04:01:12.645 INFO:tasks.workunit.client.1.smithi033.stdout:3/418: creat db/f90 x:0 0 0 2024-05-12T04:01:12.645 INFO:tasks.workunit.client.0.smithi032.stdout:1/283: truncate d1/d25/f5b 215607 0 2024-05-12T04:01:12.646 INFO:tasks.workunit.client.0.smithi032.stdout:4/196: truncate d0/d3/df/f26 676653 0 2024-05-12T04:01:12.646 INFO:tasks.workunit.client.1.smithi033.stdout:3/419: readlink db/d17/d25/l7f 0 2024-05-12T04:01:12.647 INFO:tasks.workunit.client.0.smithi032.stdout:3/319: rename d15/d20/d22/f2e to d15/d1b/d65/f6b 0 2024-05-12T04:01:12.649 INFO:tasks.workunit.client.0.smithi032.stdout:2/249: mknod d1/c5b 0 2024-05-12T04:01:12.649 INFO:tasks.workunit.client.0.smithi032.stdout:3/320: write d15/d24/d40/f5a [24520,106825] 0 2024-05-12T04:01:12.649 INFO:tasks.workunit.client.1.smithi033.stdout:0/471: sync 2024-05-12T04:01:12.649 INFO:tasks.workunit.client.1.smithi033.stdout:2/443: sync 2024-05-12T04:01:12.650 INFO:tasks.workunit.client.1.smithi033.stdout:6/448: fdatasync d4/d5/f41 0 2024-05-12T04:01:12.650 INFO:tasks.workunit.client.1.smithi033.stdout:0/472: fsync d6/dc/d39/f8a 0 2024-05-12T04:01:12.651 INFO:tasks.workunit.client.1.smithi033.stdout:2/444: write dc/d13/f1b [705486,59439] 0 2024-05-12T04:01:12.652 INFO:tasks.workunit.client.1.smithi033.stdout:2/445: chown dc/d17/d3a/f56 19 1 2024-05-12T04:01:12.653 INFO:tasks.workunit.client.1.smithi033.stdout:2/446: chown dc/d13/d1c/l20 713 1 2024-05-12T04:01:12.653 INFO:tasks.workunit.client.1.smithi033.stdout:4/448: unlink dd/d11/d31/d66/l30 0 2024-05-12T04:01:12.655 INFO:tasks.workunit.client.1.smithi033.stdout:4/449: read dd/d11/d31/d66/d42/f63 [1927862,91463] 0 2024-05-12T04:01:12.661 INFO:tasks.workunit.client.0.smithi032.stdout:1/284: unlink d1/dd/f56 0 2024-05-12T04:01:12.662 INFO:tasks.workunit.client.1.smithi033.stdout:3/420: rename db/d17/d25/d32/d3c/d7c to db/d17/d3e/d65/d91 0 2024-05-12T04:01:12.666 INFO:tasks.workunit.client.1.smithi033.stdout:5/372: rmdir d3/d12/d72 0 2024-05-12T04:01:12.667 INFO:tasks.workunit.client.0.smithi032.stdout:2/250: readlink d1/ld 0 2024-05-12T04:01:12.669 INFO:tasks.workunit.client.1.smithi033.stdout:3/421: dread db/d17/d25/f58 [0,4194304] 0 2024-05-12T04:01:12.670 INFO:tasks.workunit.client.0.smithi032.stdout:3/321: mknod d15/d33/c6c 0 2024-05-12T04:01:12.675 INFO:tasks.workunit.client.0.smithi032.stdout:3/322: dread d15/d20/d26/f52 [0,4194304] 0 2024-05-12T04:01:12.676 INFO:tasks.workunit.client.0.smithi032.stdout:3/323: write d15/d33/f56 [85412,7575] 0 2024-05-12T04:01:12.677 INFO:tasks.workunit.client.1.smithi033.stdout:6/449: fsync d4/f2f 0 2024-05-12T04:01:12.679 INFO:tasks.workunit.client.1.smithi033.stdout:6/450: write d4/d1a/d3c/f75 [574561,13998] 0 2024-05-12T04:01:12.679 INFO:tasks.workunit.client.0.smithi032.stdout:8/321: link d0/f68 d0/d9/d1b/d27/d59/f6d 0 2024-05-12T04:01:12.681 INFO:tasks.workunit.client.1.smithi033.stdout:1/404: sync 2024-05-12T04:01:12.682 INFO:tasks.workunit.client.1.smithi033.stdout:8/379: creat d1/d4/d1d/f79 x:0 0 0 2024-05-12T04:01:12.683 INFO:tasks.workunit.client.1.smithi033.stdout:7/600: write de/df/d17/d1e/d24/f57 [3411397,113852] 0 2024-05-12T04:01:12.683 INFO:tasks.workunit.client.1.smithi033.stdout:8/380: dread - d1/d16/d27/d28/f77 zero size 2024-05-12T04:01:12.687 INFO:tasks.workunit.client.1.smithi033.stdout:2/447: truncate dc/d17/d27/d2f/f75 2098414 0 2024-05-12T04:01:12.690 INFO:tasks.workunit.client.1.smithi033.stdout:4/450: rmdir dd/d11/d31/d66/d42/d59 39 2024-05-12T04:01:12.690 INFO:tasks.workunit.client.1.smithi033.stdout:8/381: dwrite d1/d16/d27/d4d/d51/f4b [0,4194304] 0 2024-05-12T04:01:12.693 INFO:tasks.workunit.client.1.smithi033.stdout:8/382: dread - d1/d16/d27/d28/f77 zero size 2024-05-12T04:01:12.696 INFO:tasks.workunit.client.0.smithi032.stdout:9/206: link d0/d3/d6/d7/da/c2b d0/d3/d6/d7/da/d39/c43 0 2024-05-12T04:01:12.698 INFO:tasks.workunit.client.1.smithi033.stdout:4/451: dread dd/f2b [0,4194304] 0 2024-05-12T04:01:12.701 INFO:tasks.workunit.client.1.smithi033.stdout:2/448: dwrite dc/d13/d1f/d6e/f81 [0,4194304] 0 2024-05-12T04:01:12.701 INFO:tasks.workunit.client.0.smithi032.stdout:3/324: rmdir d15/d1b/d43 39 2024-05-12T04:01:12.704 INFO:tasks.workunit.client.0.smithi032.stdout:8/322: creat d0/d9/d13/d21/d25/f6e x:0 0 0 2024-05-12T04:01:12.705 INFO:tasks.workunit.client.0.smithi032.stdout:9/207: dwrite d0/d3/d6/d7/da/d17/d3f/f12 [0,4194304] 0 2024-05-12T04:01:12.707 INFO:tasks.workunit.client.1.smithi033.stdout:3/422: rename db/d17/d25/d32/d3c/d44/l89 to db/d17/d25/d32/d3c/d3d/l92 0 2024-05-12T04:01:12.708 INFO:tasks.workunit.client.0.smithi032.stdout:1/285: symlink d1/l62 0 2024-05-12T04:01:12.712 INFO:tasks.workunit.client.1.smithi033.stdout:6/451: rmdir d4/d1a/d4e 39 2024-05-12T04:01:12.715 INFO:tasks.workunit.client.1.smithi033.stdout:0/473: mknod d6/c9c 0 2024-05-12T04:01:12.715 INFO:tasks.workunit.client.0.smithi032.stdout:0/312: link c6 df/d30/d36/c72 0 2024-05-12T04:01:12.717 INFO:tasks.workunit.client.0.smithi032.stdout:3/325: dwrite d15/d20/d26/f29 [0,4194304] 0 2024-05-12T04:01:12.722 INFO:tasks.workunit.client.0.smithi032.stdout:2/251: dwrite d1/f29 [0,4194304] 0 2024-05-12T04:01:12.726 INFO:tasks.workunit.client.0.smithi032.stdout:9/208: symlink d0/d3/d6/d7/da/d17/l44 0 2024-05-12T04:01:12.726 INFO:tasks.workunit.client.0.smithi032.stdout:4/197: write d0/d3/df/f26 [789,31187] 0 2024-05-12T04:01:12.728 INFO:tasks.workunit.client.0.smithi032.stdout:3/326: truncate d15/d24/f3e 229533 0 2024-05-12T04:01:12.728 INFO:tasks.workunit.client.0.smithi032.stdout:1/286: dwrite d1/d7/d30/d35/f4e [0,4194304] 0 2024-05-12T04:01:12.733 INFO:tasks.workunit.client.0.smithi032.stdout:8/323: rename d0/d9/d13/d21/d25/c5a to d0/d4b/c6f 0 2024-05-12T04:01:12.735 INFO:tasks.workunit.client.1.smithi033.stdout:8/383: creat d1/d4/d46/f7a x:0 0 0 2024-05-12T04:01:12.738 INFO:tasks.workunit.client.0.smithi032.stdout:1/287: dread d1/d7/d30/d35/f45 [0,4194304] 0 2024-05-12T04:01:12.739 INFO:tasks.workunit.client.0.smithi032.stdout:0/313: write fa [7864355,62422] 0 2024-05-12T04:01:12.741 INFO:tasks.workunit.client.1.smithi033.stdout:9/672: link d2/d4/d2b/d6d/d3a/f45 d2/d70/d7b/d94/ff5 0 2024-05-12T04:01:12.743 INFO:tasks.workunit.client.1.smithi033.stdout:4/452: fsync fc 0 2024-05-12T04:01:12.745 INFO:tasks.workunit.client.1.smithi033.stdout:4/453: fsync dd/d11/d31/f72 0 2024-05-12T04:01:12.748 INFO:tasks.workunit.client.0.smithi032.stdout:4/198: rmdir d0 39 2024-05-12T04:01:12.752 INFO:tasks.workunit.client.1.smithi033.stdout:5/373: symlink d3/d12/d25/d26/l76 0 2024-05-12T04:01:12.754 INFO:tasks.workunit.client.0.smithi032.stdout:9/209: rename d0/d3/d6/d7/d27/f3d to d0/d3/d6/f45 0 2024-05-12T04:01:12.754 INFO:tasks.workunit.client.1.smithi033.stdout:2/449: write dc/d13/f15 [3522268,61463] 0 2024-05-12T04:01:12.756 INFO:tasks.workunit.client.0.smithi032.stdout:8/324: creat d0/d9/d1b/d27/d4c/f70 x:0 0 0 2024-05-12T04:01:12.757 INFO:tasks.workunit.client.1.smithi033.stdout:5/374: dread d3/d12/d25/d26/d2e/f3e [0,4194304] 0 2024-05-12T04:01:12.757 INFO:tasks.workunit.client.0.smithi032.stdout:1/288: rmdir d1/dd/d24 39 2024-05-12T04:01:12.760 INFO:tasks.workunit.client.0.smithi032.stdout:3/327: symlink d15/d1b/l6d 0 2024-05-12T04:01:12.761 INFO:tasks.workunit.client.1.smithi033.stdout:7/601: creat de/df/d17/d1e/dcc/fce x:0 0 0 2024-05-12T04:01:12.766 INFO:tasks.workunit.client.0.smithi032.stdout:0/314: link df/d10/d3b/f66 df/d30/d4e/f73 0 2024-05-12T04:01:12.768 INFO:tasks.workunit.client.0.smithi032.stdout:3/328: dwrite d15/d20/d22/d4f/f53 [0,4194304] 0 2024-05-12T04:01:12.769 INFO:tasks.workunit.client.0.smithi032.stdout:2/252: creat d1/d13/d3d/d41/f5c x:0 0 0 2024-05-12T04:01:12.771 INFO:tasks.workunit.client.0.smithi032.stdout:4/199: creat d0/d3/df/f30 x:0 0 0 2024-05-12T04:01:12.772 INFO:tasks.workunit.client.1.smithi033.stdout:4/454: fsync dd/f4f 0 2024-05-12T04:01:12.774 INFO:tasks.workunit.client.0.smithi032.stdout:8/325: write d0/f68 [222788,31764] 0 2024-05-12T04:01:12.778 INFO:tasks.workunit.client.0.smithi032.stdout:0/315: mknod df/d10/d48/c74 0 2024-05-12T04:01:12.782 INFO:tasks.workunit.client.0.smithi032.stdout:4/200: dwrite d0/d3/df/f30 [0,4194304] 0 2024-05-12T04:01:12.783 INFO:tasks.workunit.client.0.smithi032.stdout:8/326: rmdir d0/d4b 39 2024-05-12T04:01:12.786 INFO:tasks.workunit.client.0.smithi032.stdout:0/316: dread - df/f25 zero size 2024-05-12T04:01:12.788 INFO:tasks.workunit.client.1.smithi033.stdout:0/474: mkdir d6/dc/d96/d9d 0 2024-05-12T04:01:12.790 INFO:tasks.workunit.client.0.smithi032.stdout:8/327: symlink d0/d9/d13/d21/d35/d37/d56/d63/l71 0 2024-05-12T04:01:12.792 INFO:tasks.workunit.client.1.smithi033.stdout:7/602: rename de/d80/l84 to de/df/d17/d1e/d24/lcf 0 2024-05-12T04:01:12.793 INFO:tasks.workunit.client.0.smithi032.stdout:4/201: fsync d0/d3/fd 0 2024-05-12T04:01:12.797 INFO:tasks.workunit.client.1.smithi033.stdout:3/423: dwrite db/d17/d25/d32/f46 [0,4194304] 0 2024-05-12T04:01:12.798 INFO:tasks.workunit.client.1.smithi033.stdout:0/475: dwrite d6/db/d2c/d73/f87 [0,4194304] 0 2024-05-12T04:01:12.799 INFO:tasks.workunit.client.0.smithi032.stdout:8/328: dwrite d0/d9/f12 [0,4194304] 0 2024-05-12T04:01:12.799 INFO:tasks.workunit.client.1.smithi033.stdout:4/455: write dd/d11/d31/d66/d42/f82 [2124,14701] 0 2024-05-12T04:01:12.800 INFO:tasks.workunit.client.0.smithi032.stdout:8/329: fdatasync d0/d9/d13/d21/d25/f6e 0 2024-05-12T04:01:12.805 INFO:tasks.workunit.client.0.smithi032.stdout:4/202: mkdir d0/d3/d31 0 2024-05-12T04:01:12.807 INFO:tasks.workunit.client.1.smithi033.stdout:2/450: unlink dc/d13/d65/c6c 0 2024-05-12T04:01:12.810 INFO:tasks.workunit.client.0.smithi032.stdout:8/330: symlink d0/d9/d1b/d27/l72 0 2024-05-12T04:01:12.811 INFO:tasks.workunit.client.1.smithi033.stdout:1/405: getdents d11/d17/d27/d36/d3f/d69 0 2024-05-12T04:01:12.815 INFO:tasks.workunit.client.0.smithi032.stdout:2/253: getdents d1/d13/d3d/d41/d39 0 2024-05-12T04:01:12.817 INFO:tasks.workunit.client.0.smithi032.stdout:2/254: truncate d1/d13/d3d/d41/f5c 721282 0 2024-05-12T04:01:12.818 INFO:tasks.workunit.client.0.smithi032.stdout:2/255: fdatasync d1/d13/f4c 0 2024-05-12T04:01:12.819 INFO:tasks.workunit.client.0.smithi032.stdout:8/331: mkdir d0/d9/d1b/d33/d73 0 2024-05-12T04:01:12.819 INFO:tasks.workunit.client.0.smithi032.stdout:0/317: write fb [1217445,27815] 0 2024-05-12T04:01:12.821 INFO:tasks.workunit.client.0.smithi032.stdout:4/203: write d0/fa [1929744,32144] 0 2024-05-12T04:01:12.822 INFO:tasks.workunit.client.0.smithi032.stdout:4/204: fsync d0/d3/f1c 0 2024-05-12T04:01:12.825 INFO:tasks.workunit.client.0.smithi032.stdout:2/256: creat d1/d13/f5d x:0 0 0 2024-05-12T04:01:12.826 INFO:tasks.workunit.client.1.smithi033.stdout:0/476: fdatasync d6/db/f19 0 2024-05-12T04:01:12.827 INFO:tasks.workunit.client.0.smithi032.stdout:0/318: mkdir df/d10/d3b/d75 0 2024-05-12T04:01:12.829 INFO:tasks.workunit.client.1.smithi033.stdout:3/424: rename db/d17/d3a/d67/d6d to db/d17/d3a/d93 0 2024-05-12T04:01:12.830 INFO:tasks.workunit.client.0.smithi032.stdout:8/332: rmdir d0/d9/d1b/d27 39 2024-05-12T04:01:12.832 INFO:tasks.workunit.client.0.smithi032.stdout:2/257: dwrite d1/d13/f19 [0,4194304] 0 2024-05-12T04:01:12.834 INFO:tasks.workunit.client.1.smithi033.stdout:5/375: creat d3/f77 x:0 0 0 2024-05-12T04:01:12.835 INFO:tasks.workunit.client.1.smithi033.stdout:6/452: getdents d4/d1a/d3c 0 2024-05-12T04:01:12.837 INFO:tasks.workunit.client.1.smithi033.stdout:7/603: mkdir de/d2b/d6c/db5/dd0 0 2024-05-12T04:01:12.839 INFO:tasks.workunit.client.1.smithi033.stdout:9/673: getdents d2/d4/d2b/d6d/d3a/dca 0 2024-05-12T04:01:12.845 INFO:tasks.workunit.client.0.smithi032.stdout:0/319: link df/d30/d32/f34 df/d10/d48/d70/f76 0 2024-05-12T04:01:12.848 INFO:tasks.workunit.client.1.smithi033.stdout:1/406: rename d11/d17/d27/d36/d3f/d69/d87/l88 to d11/d17/d27/d36/d3f/d69/l89 0 2024-05-12T04:01:12.849 INFO:tasks.workunit.client.1.smithi033.stdout:3/425: creat db/d17/d3e/d65/d7d/f94 x:0 0 0 2024-05-12T04:01:12.850 INFO:tasks.workunit.client.0.smithi032.stdout:8/333: dread d0/d9/d1b/f4d [0,4194304] 0 2024-05-12T04:01:12.850 INFO:tasks.workunit.client.0.smithi032.stdout:4/205: link d0/l17 d0/d3/l32 0 2024-05-12T04:01:12.851 INFO:tasks.workunit.client.0.smithi032.stdout:8/334: read d0/d9/d1b/d33/f45 [4153300,106153] 0 2024-05-12T04:01:12.852 INFO:tasks.workunit.client.1.smithi033.stdout:4/456: creat dd/d7a/d87/d7e/f8f x:0 0 0 2024-05-12T04:01:12.853 INFO:tasks.workunit.client.0.smithi032.stdout:2/258: mkdir d1/d4d/d5e 0 2024-05-12T04:01:12.853 INFO:tasks.workunit.client.1.smithi033.stdout:2/451: truncate dc/d13/d1f/f93 3767663 0 2024-05-12T04:01:12.858 INFO:tasks.workunit.client.1.smithi033.stdout:6/453: readlink d4/d1a/d4e/d4f/l84 0 2024-05-12T04:01:12.858 INFO:tasks.workunit.client.1.smithi033.stdout:2/452: dread dc/f8d [0,4194304] 0 2024-05-12T04:01:12.859 INFO:tasks.workunit.client.0.smithi032.stdout:4/206: dwrite d0/f13 [0,4194304] 0 2024-05-12T04:01:12.859 INFO:tasks.workunit.client.1.smithi033.stdout:9/674: mkdir d2/d4/d2b/d6d/d2f/df6 0 2024-05-12T04:01:12.859 INFO:tasks.workunit.client.1.smithi033.stdout:0/477: dwrite d6/dc/f5e [0,4194304] 0 2024-05-12T04:01:12.862 INFO:tasks.workunit.client.1.smithi033.stdout:2/453: write dc/d17/d27/f94 [819107,111477] 0 2024-05-12T04:01:12.865 INFO:tasks.workunit.client.0.smithi032.stdout:4/207: dwrite d0/fa [0,4194304] 0 2024-05-12T04:01:12.868 INFO:tasks.workunit.client.1.smithi033.stdout:8/384: sync 2024-05-12T04:01:12.871 INFO:tasks.workunit.client.1.smithi033.stdout:8/385: chown d1/d16/d27/d28/c62 9932323 1 2024-05-12T04:01:12.872 INFO:tasks.workunit.client.0.smithi032.stdout:8/335: truncate d0/d9/d13/f24 3520203 0 2024-05-12T04:01:12.874 INFO:tasks.workunit.client.1.smithi033.stdout:2/454: write dc/d17/d32/d84/f89 [1035393,73931] 0 2024-05-12T04:01:12.874 INFO:tasks.workunit.client.0.smithi032.stdout:8/336: truncate d0/d9/d13/d21/d25/f52 514074 0 2024-05-12T04:01:12.875 INFO:tasks.workunit.client.0.smithi032.stdout:0/320: link df/d30/d4d/l63 df/d10/d3d/d6c/l77 0 2024-05-12T04:01:12.876 INFO:tasks.workunit.client.1.smithi033.stdout:4/457: mkdir dd/d7a/d87/d7e/d90 0 2024-05-12T04:01:12.877 INFO:tasks.workunit.client.1.smithi033.stdout:3/426: dread db/d17/d25/f40 [0,4194304] 0 2024-05-12T04:01:12.877 INFO:tasks.workunit.client.0.smithi032.stdout:0/321: truncate df/f51 93157 0 2024-05-12T04:01:12.878 INFO:tasks.workunit.client.0.smithi032.stdout:2/259: mknod d1/d13/d3d/d41/c5f 0 2024-05-12T04:01:12.880 INFO:tasks.workunit.client.1.smithi033.stdout:9/675: dwrite d2/d70/f87 [0,4194304] 0 2024-05-12T04:01:12.882 INFO:tasks.workunit.client.1.smithi033.stdout:9/676: stat d2/d4/d2b/d6d/d3a/le4 0 2024-05-12T04:01:12.883 INFO:tasks.workunit.client.1.smithi033.stdout:6/454: creat d4/d1a/d4e/f96 x:0 0 0 2024-05-12T04:01:12.884 INFO:tasks.workunit.client.1.smithi033.stdout:9/677: write d2/d4/f1f [2424034,45411] 0 2024-05-12T04:01:12.886 INFO:tasks.workunit.client.1.smithi033.stdout:5/376: dread d3/d27/f6a [0,4194304] 0 2024-05-12T04:01:12.889 INFO:tasks.workunit.client.1.smithi033.stdout:7/604: rename de/d20/da9 to de/d2b/d6c/da0/da6/dd1 0 2024-05-12T04:01:12.890 INFO:tasks.workunit.client.0.smithi032.stdout:0/322: stat df/l1d 0 2024-05-12T04:01:12.895 INFO:tasks.workunit.client.1.smithi033.stdout:0/478: dread d6/dc/d58/f85 [0,4194304] 0 2024-05-12T04:01:12.895 INFO:tasks.workunit.client.1.smithi033.stdout:8/386: rmdir d1/d16/d27/d28 39 2024-05-12T04:01:12.895 INFO:tasks.workunit.client.1.smithi033.stdout:5/377: dwrite d3/d12/f14 [0,4194304] 0 2024-05-12T04:01:12.896 INFO:tasks.workunit.client.0.smithi032.stdout:4/208: mknod d0/d3/d12/c33 0 2024-05-12T04:01:12.896 INFO:tasks.workunit.client.1.smithi033.stdout:5/378: chown d3 52156879 1 2024-05-12T04:01:12.897 INFO:tasks.workunit.client.1.smithi033.stdout:5/379: write d3/f3b [3006478,92642] 0 2024-05-12T04:01:12.902 INFO:tasks.workunit.client.0.smithi032.stdout:8/337: unlink d0/c1e 0 2024-05-12T04:01:12.905 INFO:tasks.workunit.client.1.smithi033.stdout:4/458: mkdir dd/d91 0 2024-05-12T04:01:12.906 INFO:tasks.workunit.client.0.smithi032.stdout:4/209: dwrite d0/d3/df/f29 [0,4194304] 0 2024-05-12T04:01:12.906 INFO:tasks.workunit.client.1.smithi033.stdout:2/455: dread dc/d13/f1b [0,4194304] 0 2024-05-12T04:01:12.907 INFO:tasks.workunit.client.0.smithi032.stdout:0/323: rename df/f25 to df/d10/d3b/d75/f78 0 2024-05-12T04:01:12.909 INFO:tasks.workunit.client.1.smithi033.stdout:3/427: symlink db/d17/d3a/d93/l95 0 2024-05-12T04:01:12.910 INFO:tasks.workunit.client.0.smithi032.stdout:4/210: dread d0/d3/df/f30 [0,4194304] 0 2024-05-12T04:01:12.911 INFO:tasks.workunit.client.0.smithi032.stdout:4/211: write d0/d3/d12/d1d/f2a [1441026,77328] 0 2024-05-12T04:01:12.913 INFO:tasks.workunit.client.1.smithi033.stdout:6/455: fsync d4/d43/f70 0 2024-05-12T04:01:12.914 INFO:tasks.workunit.client.1.smithi033.stdout:6/456: readlink d4/d1a/d3c/l49 0 2024-05-12T04:01:12.915 INFO:tasks.workunit.client.1.smithi033.stdout:6/457: write d4/f87 [287713,68730] 0 2024-05-12T04:01:12.917 INFO:tasks.workunit.client.1.smithi033.stdout:6/458: dread d4/d1a/d4e/f66 [0,4194304] 0 2024-05-12T04:01:12.917 INFO:tasks.workunit.client.1.smithi033.stdout:1/407: rename d11/d17/d27/d36/d3f/l78 to d11/d17/d27/d36/d3f/d69/d87/l8a 0 2024-05-12T04:01:12.923 INFO:tasks.workunit.client.0.smithi032.stdout:2/260: creat d1/d13/d3d/f60 x:0 0 0 2024-05-12T04:01:12.930 INFO:tasks.workunit.client.0.smithi032.stdout:4/212: chown d0/l17 933 1 2024-05-12T04:01:12.932 INFO:tasks.workunit.client.0.smithi032.stdout:8/338: mknod d0/d9/d1b/d27/d4c/c74 0 2024-05-12T04:01:12.935 INFO:tasks.workunit.client.1.smithi033.stdout:2/456: mknod dc/d17/d27/d2f/d49/c95 0 2024-05-12T04:01:12.936 INFO:tasks.workunit.client.0.smithi032.stdout:0/324: unlink df/d28/c42 0 2024-05-12T04:01:12.940 INFO:tasks.workunit.client.0.smithi032.stdout:2/261: rmdir d1/d34 39 2024-05-12T04:01:12.940 INFO:tasks.workunit.client.1.smithi033.stdout:5/380: write d3/d12/d29/f38 [479329,63705] 0 2024-05-12T04:01:12.942 INFO:tasks.workunit.client.1.smithi033.stdout:5/381: readlink d3/d12/d29/d2b/l3a 0 2024-05-12T04:01:12.945 INFO:tasks.workunit.client.1.smithi033.stdout:9/678: rename d2/d4/d2b/d6d/d3a/f45 to d2/d70/d7b/d94/de3/ff7 0 2024-05-12T04:01:12.949 INFO:tasks.workunit.client.0.smithi032.stdout:0/325: stat df/d22 0 2024-05-12T04:01:12.951 INFO:tasks.workunit.client.0.smithi032.stdout:8/339: getdents d0/d9/d1b/d66 0 2024-05-12T04:01:12.952 INFO:tasks.workunit.client.1.smithi033.stdout:3/428: dwrite db/d17/f64 [0,4194304] 0 2024-05-12T04:01:12.953 INFO:tasks.workunit.client.0.smithi032.stdout:0/326: symlink df/d10/d3d/d4a/l79 0 2024-05-12T04:01:12.954 INFO:tasks.workunit.client.1.smithi033.stdout:8/387: unlink d1/d4/d1d/d36/d3c/c37 0 2024-05-12T04:01:12.958 INFO:tasks.workunit.client.1.smithi033.stdout:9/679: dread d2/d4/d29/d3c/f9a [0,4194304] 0 2024-05-12T04:01:12.959 INFO:tasks.workunit.client.0.smithi032.stdout:0/327: rmdir df/d30/d4d 39 2024-05-12T04:01:12.961 INFO:tasks.workunit.client.0.smithi032.stdout:8/340: mkdir d0/d9/d1b/d66/d75 0 2024-05-12T04:01:12.962 INFO:tasks.workunit.client.0.smithi032.stdout:2/262: truncate d1/d13/f2f 1823067 0 2024-05-12T04:01:12.963 INFO:tasks.workunit.client.0.smithi032.stdout:2/263: dread - d1/d13/d3d/f60 zero size 2024-05-12T04:01:12.963 INFO:tasks.workunit.client.1.smithi033.stdout:9/680: dwrite d2/d70/d7b/f8a [0,4194304] 0 2024-05-12T04:01:12.964 INFO:tasks.workunit.client.0.smithi032.stdout:0/328: symlink df/d30/d36/l7a 0 2024-05-12T04:01:12.966 INFO:tasks.workunit.client.1.smithi033.stdout:5/382: write d3/d12/d25/d26/d2e/f49 [4022060,83559] 0 2024-05-12T04:01:12.969 INFO:tasks.workunit.client.1.smithi033.stdout:1/408: write d11/d17/f35 [4437655,93487] 0 2024-05-12T04:01:12.970 INFO:tasks.workunit.client.0.smithi032.stdout:8/341: link d0/d9/d1b/d27/d59/c5f d0/d9/d13/d21/d35/d37/c76 0 2024-05-12T04:01:12.971 INFO:tasks.workunit.client.0.smithi032.stdout:2/264: write d1/d13/d3d/f60 [195,5695] 0 2024-05-12T04:01:12.974 INFO:tasks.workunit.client.0.smithi032.stdout:0/329: rename df/d30/d4e/l4b to df/d30/d4e/d55/l7b 0 2024-05-12T04:01:12.979 INFO:tasks.workunit.client.0.smithi032.stdout:8/342: unlink d0/d7/f15 0 2024-05-12T04:01:12.980 INFO:tasks.workunit.client.1.smithi033.stdout:0/479: creat d6/f9e x:0 0 0 2024-05-12T04:01:12.981 INFO:tasks.workunit.client.0.smithi032.stdout:8/343: dread - d0/d9/d13/d21/d35/d37/f62 zero size 2024-05-12T04:01:12.984 INFO:tasks.workunit.client.1.smithi033.stdout:3/429: mkdir db/d17/d3a/d8a/d96 0 2024-05-12T04:01:12.985 INFO:tasks.workunit.client.0.smithi032.stdout:0/330: symlink df/d30/d4e/d55/l7c 0 2024-05-12T04:01:12.986 INFO:tasks.workunit.client.0.smithi032.stdout:0/331: chown df/d22/f39 8899275 1 2024-05-12T04:01:12.986 INFO:tasks.workunit.client.0.smithi032.stdout:8/344: dread d0/d9/d1b/d27/f69 [0,4194304] 0 2024-05-12T04:01:12.987 INFO:tasks.workunit.client.0.smithi032.stdout:8/345: fdatasync d0/d9/f12 0 2024-05-12T04:01:12.993 INFO:tasks.workunit.client.0.smithi032.stdout:2/265: mknod d1/d34/c61 0 2024-05-12T04:01:12.995 INFO:tasks.workunit.client.0.smithi032.stdout:2/266: write d1/d13/d3d/f60 [460909,96106] 0 2024-05-12T04:01:12.997 INFO:tasks.workunit.client.1.smithi033.stdout:9/681: unlink d2/dd/d39/ld0 0 2024-05-12T04:01:13.000 INFO:tasks.workunit.client.0.smithi032.stdout:8/346: mknod d0/d9/d1b/d27/d59/c77 0 2024-05-12T04:01:13.004 INFO:tasks.workunit.client.1.smithi033.stdout:6/459: link d4/l19 d4/d1a/d4e/d4f/d94/l97 0 2024-05-12T04:01:13.005 INFO:tasks.workunit.client.0.smithi032.stdout:0/332: fsync df/d10/d3b/d75/f78 0 2024-05-12T04:01:13.006 INFO:tasks.workunit.client.1.smithi033.stdout:7/605: getdents de/df/d17/d1b 0 2024-05-12T04:01:13.007 INFO:tasks.workunit.client.0.smithi032.stdout:2/267: write d1/d13/f48 [4744616,25462] 0 2024-05-12T04:01:13.009 INFO:tasks.workunit.client.1.smithi033.stdout:1/409: dwrite d11/d17/f3e [0,4194304] 0 2024-05-12T04:01:13.009 INFO:tasks.workunit.client.1.smithi033.stdout:2/457: truncate dc/d17/d27/d2f/d49/f51 1283291 0 2024-05-12T04:01:13.009 INFO:tasks.workunit.client.1.smithi033.stdout:1/410: readlink d11/d17/d27/d36/d3f/l65 0 2024-05-12T04:01:13.011 INFO:tasks.workunit.client.0.smithi032.stdout:8/347: unlink d0/d9/d13/f20 0 2024-05-12T04:01:13.011 INFO:tasks.workunit.client.0.smithi032.stdout:2/268: dread d1/f7 [0,4194304] 0 2024-05-12T04:01:13.013 INFO:tasks.workunit.client.1.smithi033.stdout:3/430: mkdir db/d17/d3e/d65/d91/d97 0 2024-05-12T04:01:13.015 INFO:tasks.workunit.client.1.smithi033.stdout:8/388: creat d1/d4/d1d/d36/d3c/d64/f7b x:0 0 0 2024-05-12T04:01:13.020 INFO:tasks.workunit.client.1.smithi033.stdout:9/682: creat d2/d4/d2b/d78/d76/ff8 x:0 0 0 2024-05-12T04:01:13.021 INFO:tasks.workunit.client.0.smithi032.stdout:8/348: creat d0/d9/d1b/d33/f78 x:0 0 0 2024-05-12T04:01:13.028 INFO:tasks.workunit.client.0.smithi032.stdout:0/333: dread df/d30/d36/f41 [0,4194304] 0 2024-05-12T04:01:13.029 INFO:tasks.workunit.client.1.smithi033.stdout:6/460: rmdir d4/d6e 39 2024-05-12T04:01:13.029 INFO:tasks.workunit.client.0.smithi032.stdout:0/334: read - df/d30/d36/f6d zero size 2024-05-12T04:01:13.030 INFO:tasks.workunit.client.0.smithi032.stdout:8/349: creat d0/d9/d13/d21/d35/d37/f79 x:0 0 0 2024-05-12T04:01:13.031 INFO:tasks.workunit.client.0.smithi032.stdout:0/335: fsync df/f2b 0 2024-05-12T04:01:13.034 INFO:tasks.workunit.client.0.smithi032.stdout:2/269: link d1/d13/d3d/d41/d2c/f54 d1/d4d/d5e/f62 0 2024-05-12T04:01:13.036 INFO:tasks.workunit.client.1.smithi033.stdout:0/480: symlink d6/dc/d58/d91/l9f 0 2024-05-12T04:01:13.040 INFO:tasks.workunit.client.1.smithi033.stdout:2/458: rename dc/d13/l29 to dc/d17/d32/d84/l96 0 2024-05-12T04:01:13.041 INFO:tasks.workunit.client.0.smithi032.stdout:8/350: mknod d0/d7/c7a 0 2024-05-12T04:01:13.042 INFO:tasks.workunit.client.0.smithi032.stdout:2/270: dwrite d1/d13/d3d/d41/f5c [0,4194304] 0 2024-05-12T04:01:13.044 INFO:tasks.workunit.client.0.smithi032.stdout:8/351: write d0/d9/d13/d21/d25/f6e [278942,35628] 0 2024-05-12T04:01:13.044 INFO:tasks.workunit.client.0.smithi032.stdout:0/336: fdatasync df/d30/d32/f34 0 2024-05-12T04:01:13.047 INFO:tasks.workunit.client.0.smithi032.stdout:8/352: read d0/d9/d13/d21/f2a [928628,120304] 0 2024-05-12T04:01:13.049 INFO:tasks.workunit.client.1.smithi033.stdout:8/389: truncate d1/f2 1258183 0 2024-05-12T04:01:13.052 INFO:tasks.workunit.client.1.smithi033.stdout:7/606: dwrite de/fa1 [0,4194304] 0 2024-05-12T04:01:13.052 INFO:tasks.workunit.client.1.smithi033.stdout:4/459: creat dd/d11/d31/d66/f92 x:0 0 0 2024-05-12T04:01:13.053 INFO:tasks.workunit.client.1.smithi033.stdout:7/607: fsync de/df/d60/d75/f8f 0 2024-05-12T04:01:13.054 INFO:tasks.workunit.client.0.smithi032.stdout:2/271: symlink d1/d34/d3e/l63 0 2024-05-12T04:01:13.056 INFO:tasks.workunit.client.1.smithi033.stdout:9/683: mkdir d2/d4/d2b/d49/d73/df9 0 2024-05-12T04:01:13.064 INFO:tasks.workunit.client.1.smithi033.stdout:6/461: symlink d4/d1a/d67/d68/d88/l98 0 2024-05-12T04:01:13.066 INFO:tasks.workunit.client.0.smithi032.stdout:2/272: symlink d1/d34/d4a/l64 0 2024-05-12T04:01:13.067 INFO:tasks.workunit.client.0.smithi032.stdout:2/273: chown d1/c3c 0 1 2024-05-12T04:01:13.068 INFO:tasks.workunit.client.1.smithi033.stdout:0/481: mknod d6/dc/d58/d91/ca0 0 2024-05-12T04:01:13.068 INFO:tasks.workunit.client.0.smithi032.stdout:2/274: stat d1/d13/d3d/d41/f3a 0 2024-05-12T04:01:13.068 INFO:tasks.workunit.client.1.smithi033.stdout:5/383: truncate d3/f2c 3149910 0 2024-05-12T04:01:13.069 INFO:tasks.workunit.client.1.smithi033.stdout:5/384: readlink d3/d12/d29/d2b/l4e 0 2024-05-12T04:01:13.070 INFO:tasks.workunit.client.0.smithi032.stdout:0/337: creat df/d30/d4d/f7d x:0 0 0 2024-05-12T04:01:13.073 INFO:tasks.workunit.client.0.smithi032.stdout:0/338: dread fa [4194304,4194304] 0 2024-05-12T04:01:13.075 INFO:tasks.workunit.client.0.smithi032.stdout:2/275: symlink d1/d13/d3d/d41/d2c/l65 0 2024-05-12T04:01:13.076 INFO:tasks.workunit.client.0.smithi032.stdout:2/276: dread - d1/d13/d3d/d41/d2c/f54 zero size 2024-05-12T04:01:13.077 INFO:tasks.workunit.client.0.smithi032.stdout:2/277: dread d1/d13/f4c [0,4194304] 0 2024-05-12T04:01:13.080 INFO:tasks.workunit.client.0.smithi032.stdout:0/339: creat df/d30/d4e/f7e x:0 0 0 2024-05-12T04:01:13.089 INFO:tasks.workunit.client.0.smithi032.stdout:8/353: truncate d0/f2 6070913 0 2024-05-12T04:01:13.090 INFO:tasks.workunit.client.1.smithi033.stdout:9/684: mkdir d2/d4/d29/d3c/dfa 0 2024-05-12T04:01:13.090 INFO:tasks.workunit.client.1.smithi033.stdout:9/685: stat d2/d4/d2b/ca3 0 2024-05-12T04:01:13.090 INFO:tasks.workunit.client.1.smithi033.stdout:1/411: truncate d11/d17/d27/f57 1294576 0 2024-05-12T04:01:13.091 INFO:tasks.workunit.client.0.smithi032.stdout:8/354: dread d0/d9/d13/d26/d28/d3e/f57 [0,4194304] 0 2024-05-12T04:01:13.093 INFO:tasks.workunit.client.1.smithi033.stdout:6/462: rmdir d4/d1a/d4e/d4f/d94 39 2024-05-12T04:01:13.094 INFO:tasks.workunit.client.0.smithi032.stdout:5/325: sync 2024-05-12T04:01:13.097 INFO:tasks.workunit.client.1.smithi033.stdout:5/385: mkdir d3/d15/d16/d78 0 2024-05-12T04:01:13.099 INFO:tasks.workunit.client.0.smithi032.stdout:8/355: creat d0/d9/d1b/d27/f7b x:0 0 0 2024-05-12T04:01:13.101 INFO:tasks.workunit.client.0.smithi032.stdout:8/356: write d0/d9/d13/d21/d35/d37/f39 [4624608,77026] 0 2024-05-12T04:01:13.101 INFO:tasks.workunit.client.1.smithi033.stdout:2/459: symlink dc/d17/d74/l97 0 2024-05-12T04:01:13.102 INFO:tasks.workunit.client.1.smithi033.stdout:3/431: link db/d17/d25/d32/d53/c75 db/d17/d4d/c98 0 2024-05-12T04:01:13.103 INFO:tasks.workunit.client.0.smithi032.stdout:5/326: mknod d4/d7/d14/d26/c70 0 2024-05-12T04:01:13.104 INFO:tasks.workunit.client.0.smithi032.stdout:8/357: dread d0/d9/d1b/d27/d59/f6d [0,4194304] 0 2024-05-12T04:01:13.112 INFO:tasks.workunit.client.1.smithi033.stdout:7/608: rename de/df/d60/l3c to de/d2b/d6c/dbd/ld2 0 2024-05-12T04:01:13.114 INFO:tasks.workunit.client.1.smithi033.stdout:7/609: dread - de/df/d60/fc3 zero size 2024-05-12T04:01:13.114 INFO:tasks.workunit.client.1.smithi033.stdout:4/460: dwrite dd/d11/d31/d38/f44 [4194304,4194304] 0 2024-05-12T04:01:13.115 INFO:tasks.workunit.client.0.smithi032.stdout:2/278: dwrite d1/d4d/d5e/f62 [0,4194304] 0 2024-05-12T04:01:13.121 INFO:tasks.workunit.client.0.smithi032.stdout:0/340: dwrite df/d10/f13 [4194304,4194304] 0 2024-05-12T04:01:13.123 INFO:tasks.workunit.client.1.smithi033.stdout:6/463: stat d4/d1a/l48 0 2024-05-12T04:01:13.124 INFO:tasks.workunit.client.1.smithi033.stdout:9/686: dread d2/d4/d2b/d78/d76/fc9 [0,4194304] 0 2024-05-12T04:01:13.132 INFO:tasks.workunit.client.0.smithi032.stdout:2/279: dread d1/f7 [0,4194304] 0 2024-05-12T04:01:13.133 INFO:tasks.workunit.client.1.smithi033.stdout:6/464: dwrite d4/d5/f73 [0,4194304] 0 2024-05-12T04:01:13.134 INFO:tasks.workunit.client.0.smithi032.stdout:8/358: creat d0/d9/d1b/d66/d75/f7c x:0 0 0 2024-05-12T04:01:13.135 INFO:tasks.workunit.client.0.smithi032.stdout:8/359: fdatasync d0/d7/f4a 0 2024-05-12T04:01:13.138 INFO:tasks.workunit.client.1.smithi033.stdout:8/390: dwrite d1/d4/d1d/d36/f59 [0,4194304] 0 2024-05-12T04:01:13.141 INFO:tasks.workunit.client.0.smithi032.stdout:2/280: unlink d1/d13/d3d/d41/l3b 0 2024-05-12T04:01:13.142 INFO:tasks.workunit.client.0.smithi032.stdout:2/281: dread - d1/d13/d3d/d41/d2c/f37 zero size 2024-05-12T04:01:13.143 INFO:tasks.workunit.client.1.smithi033.stdout:0/482: rename d6/dc/d6e/f29 to d6/d11/d98/fa1 0 2024-05-12T04:01:13.144 INFO:tasks.workunit.client.0.smithi032.stdout:5/327: link d4/d7/d14/d26/d33/d4a/d38/d3b/d4c/l4d d4/d7/d14/d26/l71 0 2024-05-12T04:01:13.144 INFO:tasks.workunit.client.0.smithi032.stdout:5/328: fdatasync d4/d7/d14/d26/f5e 0 2024-05-12T04:01:13.147 INFO:tasks.workunit.client.1.smithi033.stdout:7/610: creat de/df/d7a/fd3 x:0 0 0 2024-05-12T04:01:13.148 INFO:tasks.workunit.client.0.smithi032.stdout:5/329: mkdir d4/d7/d14/d26/d33/d72 0 2024-05-12T04:01:13.149 INFO:tasks.workunit.client.1.smithi033.stdout:4/461: dread dd/f36 [0,4194304] 0 2024-05-12T04:01:13.151 INFO:tasks.workunit.client.0.smithi032.stdout:5/330: write d4/d7/d14/d26/f5e [254755,81342] 0 2024-05-12T04:01:13.151 INFO:tasks.workunit.client.1.smithi033.stdout:7/611: dread de/d2f/f5c [0,4194304] 0 2024-05-12T04:01:13.153 INFO:tasks.workunit.client.1.smithi033.stdout:2/460: fdatasync f7 0 2024-05-12T04:01:13.154 INFO:tasks.workunit.client.1.smithi033.stdout:6/465: rmdir d4/d1a/d67 39 2024-05-12T04:01:13.155 INFO:tasks.workunit.client.1.smithi033.stdout:2/461: chown dc/d17/d27/c2b 297 1 2024-05-12T04:01:13.156 INFO:tasks.workunit.client.1.smithi033.stdout:4/462: dwrite dd/d11/d31/d66/d42/f63 [0,4194304] 0 2024-05-12T04:01:13.156 INFO:tasks.workunit.client.1.smithi033.stdout:2/462: write f6 [3772912,39737] 0 2024-05-12T04:01:13.157 INFO:tasks.workunit.client.1.smithi033.stdout:2/463: truncate dc/f31 4609218 0 2024-05-12T04:01:13.158 INFO:tasks.workunit.client.1.smithi033.stdout:3/432: link db/d17/f2d db/d17/d3a/d8a/d96/f99 0 2024-05-12T04:01:13.161 INFO:tasks.workunit.client.1.smithi033.stdout:8/391: unlink d1/d16/f19 0 2024-05-12T04:01:13.162 INFO:tasks.workunit.client.0.smithi032.stdout:5/331: dwrite d4/f22 [4194304,4194304] 0 2024-05-12T04:01:13.166 INFO:tasks.workunit.client.1.smithi033.stdout:3/433: dwrite db/d17/d3a/f63 [0,4194304] 0 2024-05-12T04:01:13.167 INFO:tasks.workunit.client.1.smithi033.stdout:3/434: truncate db/f8c 35992 0 2024-05-12T04:01:13.174 INFO:tasks.workunit.client.0.smithi032.stdout:5/332: mknod d4/d7/d14/d26/d33/d4a/d38/d3b/c73 0 2024-05-12T04:01:13.176 INFO:tasks.workunit.client.1.smithi033.stdout:3/435: dwrite db/d17/f64 [0,4194304] 0 2024-05-12T04:01:13.178 INFO:tasks.workunit.client.0.smithi032.stdout:5/333: creat d4/d7/d14/d26/d33/d4a/d38/f74 x:0 0 0 2024-05-12T04:01:13.180 INFO:tasks.workunit.client.0.smithi032.stdout:5/334: chown d4/d7/d14/d26/d33/d4a/f4e 26903989 1 2024-05-12T04:01:13.180 INFO:tasks.workunit.client.1.smithi033.stdout:9/687: write d2/d4/d2b/fdd [228377,64180] 0 2024-05-12T04:01:13.186 INFO:tasks.workunit.client.0.smithi032.stdout:5/335: symlink d4/d7/d14/d39/d2b/l75 0 2024-05-12T04:01:13.191 INFO:tasks.workunit.client.1.smithi033.stdout:4/463: chown dd/d11/d31/d38/l8e 245061 1 2024-05-12T04:01:13.192 INFO:tasks.workunit.client.0.smithi032.stdout:2/282: dread d1/f4 [0,4194304] 0 2024-05-12T04:01:13.194 INFO:tasks.workunit.client.1.smithi033.stdout:0/483: write d6/d11/d57/f63 [1001449,47502] 0 2024-05-12T04:01:13.194 INFO:tasks.workunit.client.0.smithi032.stdout:5/336: dwrite d4/d7/d14/d26/d47/f65 [0,4194304] 0 2024-05-12T04:01:13.195 INFO:tasks.workunit.client.1.smithi033.stdout:0/484: chown d6/db/d2c/l52 112727 1 2024-05-12T04:01:13.199 INFO:tasks.workunit.client.1.smithi033.stdout:2/464: mkdir dc/d13/d1c/d98 0 2024-05-12T04:01:13.204 INFO:tasks.workunit.client.0.smithi032.stdout:5/337: creat d4/d7/d14/d39/d1b/f76 x:0 0 0 2024-05-12T04:01:13.205 INFO:tasks.workunit.client.1.smithi033.stdout:7/612: write de/d20/f96 [4864205,112474] 0 2024-05-12T04:01:13.211 INFO:tasks.workunit.client.1.smithi033.stdout:1/412: link le d11/d17/d27/d36/d3f/d69/l8b 0 2024-05-12T04:01:13.211 INFO:tasks.workunit.client.1.smithi033.stdout:6/466: dwrite d4/f56 [0,4194304] 0 2024-05-12T04:01:13.215 INFO:tasks.workunit.client.0.smithi032.stdout:5/338: rmdir d4/d7/d14/d26/d33/d4a/d3d 39 2024-05-12T04:01:13.217 INFO:tasks.workunit.client.1.smithi033.stdout:5/386: link d3/d12/d29/d2b/d4f/c6f d3/c79 0 2024-05-12T04:01:13.219 INFO:tasks.workunit.client.1.smithi033.stdout:4/464: stat dd/f2a 0 2024-05-12T04:01:13.222 INFO:tasks.workunit.client.0.smithi032.stdout:2/283: dread d1/d34/f32 [0,4194304] 0 2024-05-12T04:01:13.224 INFO:tasks.workunit.client.0.smithi032.stdout:5/339: link d4/f59 d4/d7/d14/d39/d2b/d6e/f77 0 2024-05-12T04:01:13.228 INFO:tasks.workunit.client.1.smithi033.stdout:8/392: unlink d1/d16/d20/l38 0 2024-05-12T04:01:13.230 INFO:tasks.workunit.client.1.smithi033.stdout:7/613: mkdir de/df/d17/d1e/d79/dd4 0 2024-05-12T04:01:13.230 INFO:tasks.workunit.client.0.smithi032.stdout:2/284: dwrite d1/f31 [0,4194304] 0 2024-05-12T04:01:13.231 INFO:tasks.workunit.client.1.smithi033.stdout:7/614: write de/fa1 [1561701,65230] 0 2024-05-12T04:01:13.232 INFO:tasks.workunit.client.1.smithi033.stdout:1/413: mknod d11/d1c/c8c 0 2024-05-12T04:01:13.234 INFO:tasks.workunit.client.0.smithi032.stdout:5/340: rename d4/d7/f69 to d4/d7/d14/d26/d33/d4a/d38/f78 0 2024-05-12T04:01:13.235 INFO:tasks.workunit.client.0.smithi032.stdout:2/285: dread d1/d13/f1f [0,4194304] 0 2024-05-12T04:01:13.239 INFO:tasks.workunit.client.1.smithi033.stdout:6/467: fdatasync d4/d1a/d67/d68/f8b 0 2024-05-12T04:01:13.242 INFO:tasks.workunit.client.0.smithi032.stdout:2/286: unlink d1/d13/d3d/d41/d2c/l38 0 2024-05-12T04:01:13.244 INFO:tasks.workunit.client.1.smithi033.stdout:5/387: creat d3/d15/d16/d45/d62/f7a x:0 0 0 2024-05-12T04:01:13.248 INFO:tasks.workunit.client.1.smithi033.stdout:9/688: truncate d2/d4/d29/d32/fc3 539961 0 2024-05-12T04:01:13.248 INFO:tasks.workunit.client.1.smithi033.stdout:9/689: chown d2/d4/d2b/d6d/d3a/f65 49 1 2024-05-12T04:01:13.251 INFO:tasks.workunit.client.0.smithi032.stdout:5/341: rename d4/d7/d14/d26/d33/d4a/d3d to d4/d7/d14/d26/d33/d4a/d38/d79 0 2024-05-12T04:01:13.253 INFO:tasks.workunit.client.0.smithi032.stdout:2/287: dread d1/d13/d3d/d41/f2a [0,4194304] 0 2024-05-12T04:01:13.254 INFO:tasks.workunit.client.0.smithi032.stdout:2/288: fsync d1/d13/d3d/d41/d2c/f59 0 2024-05-12T04:01:13.257 INFO:tasks.workunit.client.0.smithi032.stdout:5/342: write d4/d7/d14/d26/d33/d4a/d38/d3b/f2a [307334,119542] 0 2024-05-12T04:01:13.259 INFO:tasks.workunit.client.0.smithi032.stdout:2/289: symlink d1/d4d/l66 0 2024-05-12T04:01:13.260 INFO:tasks.workunit.client.0.smithi032.stdout:2/290: chown d1/f7 21 1 2024-05-12T04:01:13.261 INFO:tasks.workunit.client.1.smithi033.stdout:3/436: link db/d17/d3e/l60 db/d17/d4d/l9a 0 2024-05-12T04:01:13.262 INFO:tasks.workunit.client.0.smithi032.stdout:2/291: fdatasync d1/d34/d3e/f45 0 2024-05-12T04:01:13.262 INFO:tasks.workunit.client.1.smithi033.stdout:6/468: mknod d4/d1a/d6c/c99 0 2024-05-12T04:01:13.265 INFO:tasks.workunit.client.0.smithi032.stdout:6/266: sync 2024-05-12T04:01:13.265 INFO:tasks.workunit.client.0.smithi032.stdout:7/272: sync 2024-05-12T04:01:13.266 INFO:tasks.workunit.client.0.smithi032.stdout:6/267: stat d7/de/d12/f4c 0 2024-05-12T04:01:13.266 INFO:tasks.workunit.client.1.smithi033.stdout:5/388: chown d3/d12/d25/d26/d2e/l66 17462 1 2024-05-12T04:01:13.266 INFO:tasks.workunit.client.1.smithi033.stdout:2/465: truncate dc/d13/f1b 476465 0 2024-05-12T04:01:13.267 INFO:tasks.workunit.client.0.smithi032.stdout:5/343: rename d4/d7/d14/d39/d2b/l2e to d4/d7/d14/d26/d33/d4a/d38/d3b/l7a 0 2024-05-12T04:01:13.267 INFO:tasks.workunit.client.0.smithi032.stdout:5/344: chown d4/d7/d14/d39/d1b/f76 990 1 2024-05-12T04:01:13.271 INFO:tasks.workunit.client.0.smithi032.stdout:5/345: truncate d4/d7/d14/d26/d33/d4a/d38/d3b/f67 703696 0 2024-05-12T04:01:13.272 INFO:tasks.workunit.client.1.smithi033.stdout:5/389: dread d3/fc [0,4194304] 0 2024-05-12T04:01:13.272 INFO:tasks.workunit.client.1.smithi033.stdout:9/690: read d2/dd/f26 [236503,78685] 0 2024-05-12T04:01:13.273 INFO:tasks.workunit.client.1.smithi033.stdout:4/465: fdatasync dd/d11/d31/f33 0 2024-05-12T04:01:13.277 INFO:tasks.workunit.client.1.smithi033.stdout:0/485: rename d6/dc/d6e/l4e to d6/la2 0 2024-05-12T04:01:13.277 INFO:tasks.workunit.client.1.smithi033.stdout:7/615: rename de/df/d7a to de/df/d7a/dc0/dd5 22 2024-05-12T04:01:13.278 INFO:tasks.workunit.client.1.smithi033.stdout:7/616: stat de/df/f29 0 2024-05-12T04:01:13.282 INFO:tasks.workunit.client.0.smithi032.stdout:5/346: creat d4/d7/d58/f7b x:0 0 0 2024-05-12T04:01:13.291 INFO:tasks.workunit.client.1.smithi033.stdout:1/414: write d11/d17/d27/f57 [355324,87646] 0 2024-05-12T04:01:13.293 INFO:tasks.workunit.client.0.smithi032.stdout:7/273: dwrite dd/d19/f33 [0,4194304] 0 2024-05-12T04:01:13.294 INFO:tasks.workunit.client.0.smithi032.stdout:6/268: mknod d7/d9/d5c/c5e 0 2024-05-12T04:01:13.294 INFO:tasks.workunit.client.0.smithi032.stdout:7/274: chown dd/d19 8101873 1 2024-05-12T04:01:13.295 INFO:tasks.workunit.client.0.smithi032.stdout:2/292: truncate d1/f31 1832524 0 2024-05-12T04:01:13.296 INFO:tasks.workunit.client.0.smithi032.stdout:5/347: mknod d4/d7/d14/c7c 0 2024-05-12T04:01:13.297 INFO:tasks.workunit.client.0.smithi032.stdout:7/275: write dd/f22 [935615,82775] 0 2024-05-12T04:01:13.298 INFO:tasks.workunit.client.1.smithi033.stdout:5/390: mkdir d3/d12/d25/d26/d2e/d7b 0 2024-05-12T04:01:13.302 INFO:tasks.workunit.client.1.smithi033.stdout:0/486: readlink d6/dc/d66/l74 0 2024-05-12T04:01:13.303 INFO:tasks.workunit.client.0.smithi032.stdout:2/293: mkdir d1/d34/d3e/d67 0 2024-05-12T04:01:13.304 INFO:tasks.workunit.client.1.smithi033.stdout:7/617: chown de/df/d17/f34 913979 1 2024-05-12T04:01:13.311 INFO:tasks.workunit.client.0.smithi032.stdout:6/269: mknod d7/de/c5f 0 2024-05-12T04:01:13.311 INFO:tasks.workunit.client.1.smithi033.stdout:0/487: dread d6/dc/f16 [0,4194304] 0 2024-05-12T04:01:13.312 INFO:tasks.workunit.client.1.smithi033.stdout:6/469: fsync d4/d1a/f42 0 2024-05-12T04:01:13.314 INFO:tasks.workunit.client.0.smithi032.stdout:2/294: rename d1/d34/c61 to d1/d4d/d5e/c68 0 2024-05-12T04:01:13.316 INFO:tasks.workunit.client.1.smithi033.stdout:9/691: dread d2/dd/d39/f67 [4194304,4194304] 0 2024-05-12T04:01:13.316 INFO:tasks.workunit.client.0.smithi032.stdout:2/295: write d1/d13/d3d/d41/d2c/f37 [1001480,49274] 0 2024-05-12T04:01:13.319 INFO:tasks.workunit.client.0.smithi032.stdout:5/348: unlink d4/d7/d14/d26/d33/d4a/d38/d3b/l7a 0 2024-05-12T04:01:13.322 INFO:tasks.workunit.client.0.smithi032.stdout:9/210: sync 2024-05-12T04:01:13.322 INFO:tasks.workunit.client.0.smithi032.stdout:0/341: sync 2024-05-12T04:01:13.322 INFO:tasks.workunit.client.0.smithi032.stdout:4/213: sync 2024-05-12T04:01:13.322 INFO:tasks.workunit.client.0.smithi032.stdout:3/329: sync 2024-05-12T04:01:13.322 INFO:tasks.workunit.client.0.smithi032.stdout:1/289: sync 2024-05-12T04:01:13.324 INFO:tasks.workunit.client.1.smithi033.stdout:6/470: dread d4/d1a/d4e/d4f/f63 [0,4194304] 0 2024-05-12T04:01:13.324 INFO:tasks.workunit.client.0.smithi032.stdout:0/342: readlink df/d30/d4e/l50 0 2024-05-12T04:01:13.324 INFO:tasks.workunit.client.0.smithi032.stdout:3/330: readlink d15/d20/l35 0 2024-05-12T04:01:13.325 INFO:tasks.workunit.client.1.smithi033.stdout:5/391: truncate d3/d12/f3f 1006820 0 2024-05-12T04:01:13.328 INFO:tasks.workunit.client.1.smithi033.stdout:4/466: creat dd/d7a/d87/f93 x:0 0 0 2024-05-12T04:01:13.329 INFO:tasks.workunit.client.0.smithi032.stdout:0/343: chown df/d10/d3d/d4a/f65 50762 1 2024-05-12T04:01:13.329 INFO:tasks.workunit.client.0.smithi032.stdout:1/290: truncate d1/d7/d30/d35/d46/f4b 353015 0 2024-05-12T04:01:13.331 INFO:tasks.workunit.client.1.smithi033.stdout:7/618: chown de/df/d60/c7e 3406480 1 2024-05-12T04:01:13.333 INFO:tasks.workunit.client.0.smithi032.stdout:2/296: unlink d1/d4d/d5e/f62 0 2024-05-12T04:01:13.334 INFO:tasks.workunit.client.0.smithi032.stdout:6/270: dwrite d7/de/d12/f4c [0,4194304] 0 2024-05-12T04:01:13.334 INFO:tasks.workunit.client.1.smithi033.stdout:3/437: link db/d17/d25/l6b db/d17/d3e/d65/l9b 0 2024-05-12T04:01:13.335 INFO:tasks.workunit.client.0.smithi032.stdout:4/214: dread d0/d3/d24/f2d [0,4194304] 0 2024-05-12T04:01:13.337 INFO:tasks.workunit.client.1.smithi033.stdout:1/415: dwrite d11/d17/d27/d36/f3c [0,4194304] 0 2024-05-12T04:01:13.337 INFO:tasks.workunit.client.0.smithi032.stdout:5/349: unlink d4/d7/d14/d26/d33/d4a/d38/f78 0 2024-05-12T04:01:13.338 INFO:tasks.workunit.client.0.smithi032.stdout:4/215: fdatasync d0/d3/f1c 0 2024-05-12T04:01:13.339 INFO:tasks.workunit.client.1.smithi033.stdout:1/416: chown d11/d17/d84 4 1 2024-05-12T04:01:13.340 INFO:tasks.workunit.client.0.smithi032.stdout:9/211: dwrite d0/d3/d6/d7/da/d17/f1e [4194304,4194304] 0 2024-05-12T04:01:13.340 INFO:tasks.workunit.client.1.smithi033.stdout:7/619: dwrite de/fb9 [0,4194304] 0 2024-05-12T04:01:13.343 INFO:tasks.workunit.client.0.smithi032.stdout:5/350: fdatasync d4/f22 0 2024-05-12T04:01:13.343 INFO:tasks.workunit.client.0.smithi032.stdout:7/276: getdents dd/d1b/d43 0 2024-05-12T04:01:13.344 INFO:tasks.workunit.client.1.smithi033.stdout:0/488: rename d6/d11/c49 to d6/d11/d57/ca3 0 2024-05-12T04:01:13.344 INFO:tasks.workunit.client.0.smithi032.stdout:0/344: dread df/d22/f39 [0,4194304] 0 2024-05-12T04:01:13.345 INFO:tasks.workunit.client.0.smithi032.stdout:7/277: readlink dd/l2e 0 2024-05-12T04:01:13.349 INFO:tasks.workunit.client.1.smithi033.stdout:9/692: mknod d2/d4/d2b/d6d/d3a/dca/cfb 0 2024-05-12T04:01:13.350 INFO:tasks.workunit.client.1.smithi033.stdout:7/620: dwrite de/fa1 [0,4194304] 0 2024-05-12T04:01:13.351 INFO:tasks.workunit.client.0.smithi032.stdout:9/212: write d0/d3/d6/d7/da/f1a [196219,73936] 0 2024-05-12T04:01:13.352 INFO:tasks.workunit.client.0.smithi032.stdout:7/278: dread - dd/d1b/d43/f58 zero size 2024-05-12T04:01:13.353 INFO:tasks.workunit.client.0.smithi032.stdout:1/291: dwrite d1/d7/d30/f2c [0,4194304] 0 2024-05-12T04:01:13.356 INFO:tasks.workunit.client.1.smithi033.stdout:6/471: unlink d4/d1a/d4e/d4f/f50 0 2024-05-12T04:01:13.358 INFO:tasks.workunit.client.0.smithi032.stdout:2/297: dwrite d1/d13/f5d [0,4194304] 0 2024-05-12T04:01:13.360 INFO:tasks.workunit.client.1.smithi033.stdout:5/392: creat d3/d12/d25/d26/d2e/d54/d57/d73/f7c x:0 0 0 2024-05-12T04:01:13.368 INFO:tasks.workunit.client.0.smithi032.stdout:3/331: mkdir d15/d20/d26/d6a/d6e 0 2024-05-12T04:01:13.369 INFO:tasks.workunit.client.1.smithi033.stdout:3/438: symlink db/d42/l9c 0 2024-05-12T04:01:13.370 INFO:tasks.workunit.client.1.smithi033.stdout:3/439: chown db/l11 15 1 2024-05-12T04:01:13.371 INFO:tasks.workunit.client.0.smithi032.stdout:3/332: truncate d15/d20/d22/f57 807303 0 2024-05-12T04:01:13.379 INFO:tasks.workunit.client.1.smithi033.stdout:9/693: dread - d2/dd/d39/d72/faf zero size 2024-05-12T04:01:13.380 INFO:tasks.workunit.client.1.smithi033.stdout:9/694: write d2/fa [341994,54392] 0 2024-05-12T04:01:13.381 INFO:tasks.workunit.client.1.smithi033.stdout:0/489: mkdir d6/dc/d66/da4 0 2024-05-12T04:01:13.385 INFO:tasks.workunit.client.0.smithi032.stdout:7/279: rename dd/d19 to dd/d1b/d43/d5e 0 2024-05-12T04:01:13.386 INFO:tasks.workunit.client.1.smithi033.stdout:7/621: readlink de/df/lc6 0 2024-05-12T04:01:13.392 INFO:tasks.workunit.client.0.smithi032.stdout:4/216: symlink d0/l34 0 2024-05-12T04:01:13.394 INFO:tasks.workunit.client.1.smithi033.stdout:7/622: dwrite de/d2b/d6c/f5a [0,4194304] 0 2024-05-12T04:01:13.394 INFO:tasks.workunit.client.0.smithi032.stdout:5/351: symlink d4/l7d 0 2024-05-12T04:01:13.395 INFO:tasks.workunit.client.1.smithi033.stdout:1/417: dwrite d11/d1c/f64 [0,4194304] 0 2024-05-12T04:01:13.397 INFO:tasks.workunit.client.0.smithi032.stdout:3/333: symlink d15/d20/d26/d6a/l6f 0 2024-05-12T04:01:13.397 INFO:tasks.workunit.client.0.smithi032.stdout:1/292: write d1/d7/d30/d18/f1a [3187643,23251] 0 2024-05-12T04:01:13.399 INFO:tasks.workunit.client.0.smithi032.stdout:1/293: dread - d1/f47 zero size 2024-05-12T04:01:13.400 INFO:tasks.workunit.client.0.smithi032.stdout:1/294: read - d1/d25/f4a zero size 2024-05-12T04:01:13.402 INFO:tasks.workunit.client.1.smithi033.stdout:3/440: mkdir db/d17/d3a/d8a/d9d 0 2024-05-12T04:01:13.402 INFO:tasks.workunit.client.1.smithi033.stdout:4/467: rename dd/d11/d5b/c8c to dd/d7a/c94 0 2024-05-12T04:01:13.403 INFO:tasks.workunit.client.0.smithi032.stdout:1/295: write d1/d7/d30/d35/f4e [281363,21346] 0 2024-05-12T04:01:13.404 INFO:tasks.workunit.client.1.smithi033.stdout:8/393: sync 2024-05-12T04:01:13.404 INFO:tasks.workunit.client.1.smithi033.stdout:7/623: truncate de/d20/d38/f9e 174997 0 2024-05-12T04:01:13.405 INFO:tasks.workunit.client.1.smithi033.stdout:0/490: dread d6/d11/f2f [0,4194304] 0 2024-05-12T04:01:13.408 INFO:tasks.workunit.client.1.smithi033.stdout:1/418: read d11/d17/d27/d36/f3d [1580413,77228] 0 2024-05-12T04:01:13.413 INFO:tasks.workunit.client.0.smithi032.stdout:3/334: creat d15/d1b/d65/f70 x:0 0 0 2024-05-12T04:01:13.419 INFO:tasks.workunit.client.0.smithi032.stdout:7/280: dread f2 [0,4194304] 0 2024-05-12T04:01:13.420 INFO:tasks.workunit.client.0.smithi032.stdout:9/213: truncate d0/f28 2788791 0 2024-05-12T04:01:13.422 INFO:tasks.workunit.client.0.smithi032.stdout:9/214: chown d0/d3/d6/d7/da/le 14137 1 2024-05-12T04:01:13.423 INFO:tasks.workunit.client.0.smithi032.stdout:9/215: write d0/d3/d6/d7/da/d17/d3f/d13/f3c [273255,11977] 0 2024-05-12T04:01:13.425 INFO:tasks.workunit.client.0.smithi032.stdout:4/217: creat d0/d3/d31/f35 x:0 0 0 2024-05-12T04:01:13.429 INFO:tasks.workunit.client.0.smithi032.stdout:0/345: getdents df/d30/d36 0 2024-05-12T04:01:13.433 INFO:tasks.workunit.client.0.smithi032.stdout:2/298: link d1/f31 d1/d34/d3e/d67/f69 0 2024-05-12T04:01:13.438 INFO:tasks.workunit.client.1.smithi033.stdout:9/695: rename d2/d70/c9d to d2/d4/d2b/d6d/db5/cfc 0 2024-05-12T04:01:13.439 INFO:tasks.workunit.client.0.smithi032.stdout:5/352: creat d4/d7/f7e x:0 0 0 2024-05-12T04:01:13.439 INFO:tasks.workunit.client.0.smithi032.stdout:2/299: dwrite d1/d13/d3d/d41/f2a [0,4194304] 0 2024-05-12T04:01:13.443 INFO:tasks.workunit.client.0.smithi032.stdout:2/300: chown d1/d13/c16 1 1 2024-05-12T04:01:13.443 INFO:tasks.workunit.client.1.smithi033.stdout:9/696: dwrite d2/d4/d29/d3c/fe9 [0,4194304] 0 2024-05-12T04:01:13.446 INFO:tasks.workunit.client.0.smithi032.stdout:0/346: creat df/d30/d4e/d55/f7f x:0 0 0 2024-05-12T04:01:13.447 INFO:tasks.workunit.client.0.smithi032.stdout:0/347: chown df/d30/d32/d60/l62 852415517 1 2024-05-12T04:01:13.448 INFO:tasks.workunit.client.1.smithi033.stdout:0/491: truncate d6/db/d2c/f3e 3682200 0 2024-05-12T04:01:13.448 INFO:tasks.workunit.client.0.smithi032.stdout:3/335: creat d15/d24/d2a/d5b/f71 x:0 0 0 2024-05-12T04:01:13.451 INFO:tasks.workunit.client.1.smithi033.stdout:6/472: creat d4/f9a x:0 0 0 2024-05-12T04:01:13.454 INFO:tasks.workunit.client.0.smithi032.stdout:1/296: creat d1/dd/d24/d57/f63 x:0 0 0 2024-05-12T04:01:13.455 INFO:tasks.workunit.client.1.smithi033.stdout:5/393: link d3/d15/d16/d45/f68 d3/f7d 0 2024-05-12T04:01:13.458 INFO:tasks.workunit.client.1.smithi033.stdout:1/419: truncate d11/d17/d27/d36/d3f/f50 458457 0 2024-05-12T04:01:13.459 INFO:tasks.workunit.client.0.smithi032.stdout:3/336: dwrite d15/d20/d26/f3c [4194304,4194304] 0 2024-05-12T04:01:13.464 INFO:tasks.workunit.client.0.smithi032.stdout:5/353: mkdir d4/d7/d14/d26/d33/d5f/d7f 0 2024-05-12T04:01:13.465 INFO:tasks.workunit.client.1.smithi033.stdout:2/466: sync 2024-05-12T04:01:13.465 INFO:tasks.workunit.client.1.smithi033.stdout:3/441: unlink db/d17/d25/l71 0 2024-05-12T04:01:13.470 INFO:tasks.workunit.client.0.smithi032.stdout:0/348: creat df/d10/f80 x:0 0 0 2024-05-12T04:01:13.471 INFO:tasks.workunit.client.1.smithi033.stdout:7/624: mkdir de/d2b/d6c/db5/dd0/dd6 0 2024-05-12T04:01:13.471 INFO:tasks.workunit.client.1.smithi033.stdout:8/394: dwrite d1/d16/d20/f65 [0,4194304] 0 2024-05-12T04:01:13.471 INFO:tasks.workunit.client.0.smithi032.stdout:9/216: dwrite d0/d3/d6/d7/da/d17/f20 [0,4194304] 0 2024-05-12T04:01:13.472 INFO:tasks.workunit.client.1.smithi033.stdout:8/395: fdatasync d1/d4/f39 0 2024-05-12T04:01:13.473 INFO:tasks.workunit.client.0.smithi032.stdout:9/217: write d0/d3/d6/d7/f2c [1036297,15840] 0 2024-05-12T04:01:13.473 INFO:tasks.workunit.client.0.smithi032.stdout:7/281: rmdir dd/d1b/d1e/d45/d55 39 2024-05-12T04:01:13.475 INFO:tasks.workunit.client.0.smithi032.stdout:7/282: write dd/d1b/d43/d5e/f33 [2293443,36789] 0 2024-05-12T04:01:13.475 INFO:tasks.workunit.client.0.smithi032.stdout:1/297: creat d1/d25/f64 x:0 0 0 2024-05-12T04:01:13.476 INFO:tasks.workunit.client.0.smithi032.stdout:4/218: link d0/d3/df/l15 d0/d3/d12/d1d/l36 0 2024-05-12T04:01:13.478 INFO:tasks.workunit.client.1.smithi033.stdout:9/697: creat d2/d4/d29/d3c/d52/ffd x:0 0 0 2024-05-12T04:01:13.492 INFO:tasks.workunit.client.1.smithi033.stdout:0/492: dread d6/db/f1a [0,4194304] 0 2024-05-12T04:01:13.495 INFO:tasks.workunit.client.0.smithi032.stdout:8/360: sync 2024-05-12T04:01:13.498 INFO:tasks.workunit.client.0.smithi032.stdout:3/337: symlink d15/d33/d3f/l72 0 2024-05-12T04:01:13.499 INFO:tasks.workunit.client.1.smithi033.stdout:5/394: chown d3/d12/f3f 1 1 2024-05-12T04:01:13.499 INFO:tasks.workunit.client.0.smithi032.stdout:3/338: dread d15/d24/f3e [0,4194304] 0 2024-05-12T04:01:13.500 INFO:tasks.workunit.client.1.smithi033.stdout:5/395: write d3/d15/d16/d45/d4b/f6c [376418,6724] 0 2024-05-12T04:01:13.501 INFO:tasks.workunit.client.0.smithi032.stdout:5/354: truncate f1 2241545 0 2024-05-12T04:01:13.503 INFO:tasks.workunit.client.1.smithi033.stdout:1/420: chown d11/d17/c29 16015301 1 2024-05-12T04:01:13.504 INFO:tasks.workunit.client.1.smithi033.stdout:1/421: fdatasync d11/d1c/d43/f5e 0 2024-05-12T04:01:13.505 INFO:tasks.workunit.client.0.smithi032.stdout:2/301: creat d1/d34/d3e/d67/f6a x:0 0 0 2024-05-12T04:01:13.507 INFO:tasks.workunit.client.1.smithi033.stdout:4/468: creat dd/d11/f95 x:0 0 0 2024-05-12T04:01:13.509 INFO:tasks.workunit.client.0.smithi032.stdout:8/361: dwrite d0/d9/d1b/d66/d75/f7c [0,4194304] 0 2024-05-12T04:01:13.514 INFO:tasks.workunit.client.0.smithi032.stdout:3/339: dread f12 [0,4194304] 0 2024-05-12T04:01:13.518 INFO:tasks.workunit.client.0.smithi032.stdout:3/340: stat d15/d33/d3f 0 2024-05-12T04:01:13.520 INFO:tasks.workunit.client.1.smithi033.stdout:5/396: dread d3/f3b [0,4194304] 0 2024-05-12T04:01:13.523 INFO:tasks.workunit.client.0.smithi032.stdout:2/302: dwrite d1/d13/d3d/d41/d2c/f56 [0,4194304] 0 2024-05-12T04:01:13.529 INFO:tasks.workunit.client.0.smithi032.stdout:8/362: dwrite d0/d9/d13/d21/d35/d37/f79 [0,4194304] 0 2024-05-12T04:01:13.530 INFO:tasks.workunit.client.0.smithi032.stdout:9/218: rename d0/d3/d6/d7/da/d17/d3f to d0/d3/d6/d7/d2a/d46 0 2024-05-12T04:01:13.533 INFO:tasks.workunit.client.0.smithi032.stdout:1/298: mknod d1/d7/d49/c65 0 2024-05-12T04:01:13.535 INFO:tasks.workunit.client.0.smithi032.stdout:1/299: write d1/d25/f4d [6210584,126545] 0 2024-05-12T04:01:13.536 INFO:tasks.workunit.client.1.smithi033.stdout:9/698: creat d2/d4/d2b/d6d/d4c/ffe x:0 0 0 2024-05-12T04:01:13.542 INFO:tasks.workunit.client.1.smithi033.stdout:6/473: symlink d4/d1a/d4e/d4f/l9b 0 2024-05-12T04:01:13.547 INFO:tasks.workunit.client.1.smithi033.stdout:3/442: write db/d17/d25/f58 [1983990,62762] 0 2024-05-12T04:01:13.548 INFO:tasks.workunit.client.1.smithi033.stdout:3/443: write db/d17/d3a/f63 [524538,22270] 0 2024-05-12T04:01:13.549 INFO:tasks.workunit.client.1.smithi033.stdout:1/422: fsync d11/d17/f1d 0 2024-05-12T04:01:13.550 INFO:tasks.workunit.client.1.smithi033.stdout:1/423: chown d11/d1c/d43/f5e 17 1 2024-05-12T04:01:13.552 INFO:tasks.workunit.client.1.smithi033.stdout:2/467: creat dc/d13/d1c/d98/f99 x:0 0 0 2024-05-12T04:01:13.553 INFO:tasks.workunit.client.1.smithi033.stdout:2/468: write dc/d13/d1f/f69 [382248,3439] 0 2024-05-12T04:01:13.557 INFO:tasks.workunit.client.0.smithi032.stdout:7/283: dwrite f6 [0,4194304] 0 2024-05-12T04:01:13.561 INFO:tasks.workunit.client.1.smithi033.stdout:3/444: dread db/f20 [0,4194304] 0 2024-05-12T04:01:13.562 INFO:tasks.workunit.client.1.smithi033.stdout:3/445: chown db/d17/d3e/d65/d91/d97 3 1 2024-05-12T04:01:13.562 INFO:tasks.workunit.client.1.smithi033.stdout:7/625: link de/df/d17/f88 de/d2b/d6c/fd7 0 2024-05-12T04:01:13.565 INFO:tasks.workunit.client.1.smithi033.stdout:8/396: mkdir d1/d16/d27/d7c 0 2024-05-12T04:01:13.566 INFO:tasks.workunit.client.1.smithi033.stdout:8/397: readlink d1/l9 0 2024-05-12T04:01:13.567 INFO:tasks.workunit.client.0.smithi032.stdout:8/363: dread d0/d9/d13/d26/d28/d3e/f57 [0,4194304] 0 2024-05-12T04:01:13.568 INFO:tasks.workunit.client.0.smithi032.stdout:8/364: dread - d0/d9/d1b/d33/f3b zero size 2024-05-12T04:01:13.569 INFO:tasks.workunit.client.1.smithi033.stdout:3/446: dwrite db/d59/f5b [0,4194304] 0 2024-05-12T04:01:13.572 INFO:tasks.workunit.client.1.smithi033.stdout:3/447: truncate db/d17/d3a/d8a/d96/f99 1816949 0 2024-05-12T04:01:13.572 INFO:tasks.workunit.client.1.smithi033.stdout:6/474: read d4/d5/f5c [315813,2616] 0 2024-05-12T04:01:13.577 INFO:tasks.workunit.client.0.smithi032.stdout:5/355: symlink d4/d7/d14/d26/d33/d4a/l80 0 2024-05-12T04:01:13.580 INFO:tasks.workunit.client.0.smithi032.stdout:2/303: truncate d1/f31 112583 0 2024-05-12T04:01:13.584 INFO:tasks.workunit.client.0.smithi032.stdout:7/284: readlink dd/l2c 0 2024-05-12T04:01:13.586 INFO:tasks.workunit.client.1.smithi033.stdout:2/469: creat dc/d17/d3a/f9a x:0 0 0 2024-05-12T04:01:13.591 INFO:tasks.workunit.client.1.smithi033.stdout:7/626: mkdir de/df/d17/d1e/dd8 0 2024-05-12T04:01:13.591 INFO:tasks.workunit.client.1.smithi033.stdout:7/627: stat de/df/d7a/fd3 0 2024-05-12T04:01:13.593 INFO:tasks.workunit.client.0.smithi032.stdout:4/219: rename d0/d3/le to d0/d3/d24/l37 0 2024-05-12T04:01:13.594 INFO:tasks.workunit.client.0.smithi032.stdout:4/220: read d0/fa [4079103,60317] 0 2024-05-12T04:01:13.595 INFO:tasks.workunit.client.0.smithi032.stdout:8/365: symlink d0/d9/d1b/d66/d75/l7d 0 2024-05-12T04:01:13.595 INFO:tasks.workunit.client.0.smithi032.stdout:4/221: chown d0/d3/d31/f35 9 1 2024-05-12T04:01:13.597 INFO:tasks.workunit.client.0.smithi032.stdout:5/356: dread d4/fa [0,4194304] 0 2024-05-12T04:01:13.598 INFO:tasks.workunit.client.0.smithi032.stdout:0/349: getdents df/d22/d58 0 2024-05-12T04:01:13.600 INFO:tasks.workunit.client.0.smithi032.stdout:2/304: mkdir d1/d13/d6b 0 2024-05-12T04:01:13.604 INFO:tasks.workunit.client.0.smithi032.stdout:7/285: creat dd/d1b/f5f x:0 0 0 2024-05-12T04:01:13.607 INFO:tasks.workunit.client.1.smithi033.stdout:6/475: creat d4/d1a/d6c/f9c x:0 0 0 2024-05-12T04:01:13.609 INFO:tasks.workunit.client.0.smithi032.stdout:9/219: rename d0/d3/d6/d7/da/d17/f1e to d0/d3/d1b/f47 0 2024-05-12T04:01:13.610 INFO:tasks.workunit.client.1.smithi033.stdout:3/448: mknod db/d17/d3a/d67/c9e 0 2024-05-12T04:01:13.610 INFO:tasks.workunit.client.0.smithi032.stdout:9/220: fsync d0/d3/d6/d7/da/f33 0 2024-05-12T04:01:13.610 INFO:tasks.workunit.client.1.smithi033.stdout:6/476: dread d4/f56 [0,4194304] 0 2024-05-12T04:01:13.611 INFO:tasks.workunit.client.1.smithi033.stdout:3/449: chown db/d17/d3a/d67/c9e 697391 1 2024-05-12T04:01:13.611 INFO:tasks.workunit.client.1.smithi033.stdout:3/450: dread - db/f90 zero size 2024-05-12T04:01:13.612 INFO:tasks.workunit.client.1.smithi033.stdout:4/469: rename dd/d11/d5b/c61 to dd/d11/d31/d66/d42/c96 0 2024-05-12T04:01:13.613 INFO:tasks.workunit.client.0.smithi032.stdout:8/366: rmdir d0/d9/d13/d21/d25 39 2024-05-12T04:01:13.615 INFO:tasks.workunit.client.0.smithi032.stdout:4/222: creat d0/d3/d31/f38 x:0 0 0 2024-05-12T04:01:13.616 INFO:tasks.workunit.client.1.smithi033.stdout:2/470: write dc/d17/d27/d2f/f41 [1853072,28660] 0 2024-05-12T04:01:13.617 INFO:tasks.workunit.client.1.smithi033.stdout:2/471: chown dc/d13/d1c/d98 13939570 1 2024-05-12T04:01:13.618 INFO:tasks.workunit.client.1.smithi033.stdout:9/699: rmdir d2/d4/d29/d3c/dfa 0 2024-05-12T04:01:13.620 INFO:tasks.workunit.client.0.smithi032.stdout:0/350: mkdir df/d30/d32/d81 0 2024-05-12T04:01:13.621 INFO:tasks.workunit.client.0.smithi032.stdout:4/223: dwrite d0/d3/d12/d1d/f2a [0,4194304] 0 2024-05-12T04:01:13.622 INFO:tasks.workunit.client.1.smithi033.stdout:8/398: symlink d1/d16/d27/d4d/d73/l7d 0 2024-05-12T04:01:13.622 INFO:tasks.workunit.client.0.smithi032.stdout:0/351: write df/f2b [2078705,9358] 0 2024-05-12T04:01:13.622 INFO:tasks.workunit.client.0.smithi032.stdout:4/224: stat d0/d3/df/c16 0 2024-05-12T04:01:13.624 INFO:tasks.workunit.client.1.smithi033.stdout:9/700: dwrite d2/d4/d29/d3c/fe9 [4194304,4194304] 0 2024-05-12T04:01:13.628 INFO:tasks.workunit.client.1.smithi033.stdout:0/493: read d6/db/d2c/f3e [374760,46425] 0 2024-05-12T04:01:13.629 INFO:tasks.workunit.client.1.smithi033.stdout:6/477: unlink d4/d43/d57/l6b 0 2024-05-12T04:01:13.629 INFO:tasks.workunit.client.1.smithi033.stdout:0/494: fsync d6/dc/d58/f88 0 2024-05-12T04:01:13.631 INFO:tasks.workunit.client.1.smithi033.stdout:3/451: symlink db/d42/l9f 0 2024-05-12T04:01:13.633 INFO:tasks.workunit.client.1.smithi033.stdout:5/397: rename d3/d12/d25/d26/d2e/d7b to d3/d12/d25/d26/d2e/d54/d57/d73/d74/d7e 0 2024-05-12T04:01:13.640 INFO:tasks.workunit.client.1.smithi033.stdout:2/472: creat dc/d13/d1f/f9b x:0 0 0 2024-05-12T04:01:13.641 INFO:tasks.workunit.client.1.smithi033.stdout:4/470: dread dd/d11/d31/d38/f68 [0,4194304] 0 2024-05-12T04:01:13.644 INFO:tasks.workunit.client.1.smithi033.stdout:6/478: mknod d4/d1a/d4e/c9d 0 2024-05-12T04:01:13.647 INFO:tasks.workunit.client.1.smithi033.stdout:7/628: rename de/df/d17/d1e/dc4/ccd to de/d2b/d6c/cd9 0 2024-05-12T04:01:13.648 INFO:tasks.workunit.client.1.smithi033.stdout:7/629: chown de/d20/f96 1688502500 1 2024-05-12T04:01:13.649 INFO:tasks.workunit.client.1.smithi033.stdout:5/398: unlink d3/d12/d25/d26/d2e/f5b 0 2024-05-12T04:01:13.651 INFO:tasks.workunit.client.1.smithi033.stdout:1/424: link d11/d17/d38/c70 d11/d17/d27/d36/d46/d4c/c8d 0 2024-05-12T04:01:13.651 INFO:tasks.workunit.client.1.smithi033.stdout:1/425: chown d11/d17/d27/c49 10262 1 2024-05-12T04:01:13.655 INFO:tasks.workunit.client.1.smithi033.stdout:0/495: mknod d6/dc/d58/ca5 0 2024-05-12T04:01:13.658 INFO:tasks.workunit.client.1.smithi033.stdout:9/701: rename d2/d4/d29/d32/c6b to d2/d70/d7b/d94/de3/cff 0 2024-05-12T04:01:13.659 INFO:tasks.workunit.client.1.smithi033.stdout:5/399: mkdir d3/d15/d16/d45/d62/d7f 0 2024-05-12T04:01:13.661 INFO:tasks.workunit.client.1.smithi033.stdout:5/400: read d3/d12/d25/d26/d2e/f49 [1648780,90098] 0 2024-05-12T04:01:13.662 INFO:tasks.workunit.client.1.smithi033.stdout:8/399: unlink d1/d4/d1d/f79 0 2024-05-12T04:01:13.670 INFO:tasks.workunit.client.0.smithi032.stdout:2/305: mknod d1/d4d/d5e/c6c 0 2024-05-12T04:01:13.670 INFO:tasks.workunit.client.0.smithi032.stdout:3/341: link d15/d24/d2a/c3a d15/c73 0 2024-05-12T04:01:13.671 INFO:tasks.workunit.client.1.smithi033.stdout:7/630: creat de/d2b/d6c/db5/dd0/dd6/fda x:0 0 0 2024-05-12T04:01:13.671 INFO:tasks.workunit.client.0.smithi032.stdout:4/225: write d0/f11 [4184112,7408] 0 2024-05-12T04:01:13.674 INFO:tasks.workunit.client.0.smithi032.stdout:5/357: mknod d4/d7/d14/d26/d33/d72/c81 0 2024-05-12T04:01:13.674 INFO:tasks.workunit.client.1.smithi033.stdout:9/702: dread d2/d4/d2b/d6d/f5c [0,4194304] 0 2024-05-12T04:01:13.674 INFO:tasks.workunit.client.1.smithi033.stdout:1/426: rename d11/d17/d27/c49 to d11/d17/d27/d36/d3f/d69/c8e 0 2024-05-12T04:01:13.675 INFO:tasks.workunit.client.1.smithi033.stdout:1/427: write d11/d1c/f34 [2253098,101009] 0 2024-05-12T04:01:13.675 INFO:tasks.workunit.client.1.smithi033.stdout:1/428: stat d11/d17/d27/f44 0 2024-05-12T04:01:13.676 INFO:tasks.workunit.client.1.smithi033.stdout:4/471: write dd/f13 [248607,71333] 0 2024-05-12T04:01:13.676 INFO:tasks.workunit.client.0.smithi032.stdout:5/358: fsync d4/d7/d14/d26/f5e 0 2024-05-12T04:01:13.678 INFO:tasks.workunit.client.0.smithi032.stdout:7/286: fsync dd/d1b/d43/d5e/d37/f46 0 2024-05-12T04:01:13.681 INFO:tasks.workunit.client.0.smithi032.stdout:3/342: dwrite d15/f19 [4194304,4194304] 0 2024-05-12T04:01:13.681 INFO:tasks.workunit.client.0.smithi032.stdout:5/359: write d4/f49 [739003,76926] 0 2024-05-12T04:01:13.689 INFO:tasks.workunit.client.0.smithi032.stdout:2/306: dread d1/f7 [0,4194304] 0 2024-05-12T04:01:13.690 INFO:tasks.workunit.client.0.smithi032.stdout:2/307: fsync d1/d34/d3e/f45 0 2024-05-12T04:01:13.692 INFO:tasks.workunit.client.0.smithi032.stdout:2/308: chown d1/f1e 938255092 1 2024-05-12T04:01:13.704 INFO:tasks.workunit.client.0.smithi032.stdout:2/309: dwrite d1/d13/d3d/d41/d2c/f59 [0,4194304] 0 2024-05-12T04:01:13.711 INFO:tasks.workunit.client.0.smithi032.stdout:5/360: dread d4/f59 [0,4194304] 0 2024-05-12T04:01:13.766 INFO:tasks.workunit.client.1.smithi033.stdout:5/401: sync 2024-05-12T04:01:13.767 INFO:tasks.workunit.client.0.smithi032.stdout:6/271: sync 2024-05-12T04:01:13.769 INFO:tasks.workunit.client.1.smithi033.stdout:2/473: truncate dc/d17/d27/d2f/d49/d59/f79 2939037 0 2024-05-12T04:01:13.771 INFO:tasks.workunit.client.1.smithi033.stdout:0/496: write d6/db/f19 [639455,113186] 0 2024-05-12T04:01:13.773 INFO:tasks.workunit.client.1.smithi033.stdout:2/474: truncate dc/d13/d1c/f21 4901489 0 2024-05-12T04:01:13.773 INFO:tasks.workunit.client.1.smithi033.stdout:5/402: dwrite d3/f77 [0,4194304] 0 2024-05-12T04:01:13.779 INFO:tasks.workunit.client.1.smithi033.stdout:6/479: dwrite d4/d1a/f26 [0,4194304] 0 2024-05-12T04:01:13.781 INFO:tasks.workunit.client.0.smithi032.stdout:0/352: truncate df/d30/d36/f41 350520 0 2024-05-12T04:01:13.785 INFO:tasks.workunit.client.1.smithi033.stdout:6/480: dwrite d4/f16 [4194304,4194304] 0 2024-05-12T04:01:13.786 INFO:tasks.workunit.client.1.smithi033.stdout:6/481: chown d4/l11 2 1 2024-05-12T04:01:13.791 INFO:tasks.workunit.client.1.smithi033.stdout:3/452: getdents db/d17/d3a/d8a 0 2024-05-12T04:01:13.797 INFO:tasks.workunit.client.0.smithi032.stdout:9/221: sync 2024-05-12T04:01:13.805 INFO:tasks.workunit.client.0.smithi032.stdout:3/343: creat d15/d33/f74 x:0 0 0 2024-05-12T04:01:13.806 INFO:tasks.workunit.client.0.smithi032.stdout:3/344: write d15/f45 [958876,114661] 0 2024-05-12T04:01:13.809 INFO:tasks.workunit.client.1.smithi033.stdout:1/429: rmdir d11/d1c 39 2024-05-12T04:01:13.814 INFO:tasks.workunit.client.0.smithi032.stdout:2/310: creat d1/d34/d4a/f6d x:0 0 0 2024-05-12T04:01:13.825 INFO:tasks.workunit.client.0.smithi032.stdout:6/272: mkdir d7/d9/d60 0 2024-05-12T04:01:13.825 INFO:tasks.workunit.client.0.smithi032.stdout:6/273: dread - d7/f59 zero size 2024-05-12T04:01:13.830 INFO:tasks.workunit.client.0.smithi032.stdout:0/353: write df/d30/d4e/d55/f67 [339623,120040] 0 2024-05-12T04:01:13.836 INFO:tasks.workunit.client.1.smithi033.stdout:5/403: rename d3/d12/d25/d26/d2e/d54/d57/f5f to d3/d12/f80 0 2024-05-12T04:01:13.838 INFO:tasks.workunit.client.0.smithi032.stdout:9/222: creat d0/d3/d6/d7/d27/f48 x:0 0 0 2024-05-12T04:01:13.840 INFO:tasks.workunit.client.0.smithi032.stdout:3/345: rmdir d15/d24/d2a 39 2024-05-12T04:01:13.842 INFO:tasks.workunit.client.0.smithi032.stdout:3/346: chown d15/d20/d26/f52 506 1 2024-05-12T04:01:13.843 INFO:tasks.workunit.client.0.smithi032.stdout:7/287: mkdir dd/d1b/d1e/d45/d55/d60 0 2024-05-12T04:01:13.845 INFO:tasks.workunit.client.1.smithi033.stdout:3/453: mkdir db/d17/d3a/d67/d76/da0 0 2024-05-12T04:01:13.847 INFO:tasks.workunit.client.1.smithi033.stdout:7/631: creat de/df/d65/dbc/fdb x:0 0 0 2024-05-12T04:01:13.847 INFO:tasks.workunit.client.0.smithi032.stdout:5/361: mknod d4/d7/d14/d39/d2b/d6e/c82 0 2024-05-12T04:01:13.848 INFO:tasks.workunit.client.1.smithi033.stdout:9/703: getdents d2/d4/d29/d32/db7 0 2024-05-12T04:01:13.849 INFO:tasks.workunit.client.1.smithi033.stdout:9/704: chown d2/d4/d29/d32/c80 1036788 1 2024-05-12T04:01:13.851 INFO:tasks.workunit.client.1.smithi033.stdout:4/472: creat dd/d69/f97 x:0 0 0 2024-05-12T04:01:13.852 INFO:tasks.workunit.client.1.smithi033.stdout:3/454: dread db/d17/f36 [0,4194304] 0 2024-05-12T04:01:13.854 INFO:tasks.workunit.client.0.smithi032.stdout:9/223: creat d0/d3/d6/d7/d2a/d46/f49 x:0 0 0 2024-05-12T04:01:13.857 INFO:tasks.workunit.client.1.smithi033.stdout:4/473: dwrite dd/d11/d75/f8d [0,4194304] 0 2024-05-12T04:01:13.857 INFO:tasks.workunit.client.0.smithi032.stdout:3/347: mkdir d15/d33/d3f/d75 0 2024-05-12T04:01:13.858 INFO:tasks.workunit.client.1.smithi033.stdout:4/474: chown dd/d11/f14 0 1 2024-05-12T04:01:13.859 INFO:tasks.workunit.client.1.smithi033.stdout:4/475: readlink dd/d11/d31/l50 0 2024-05-12T04:01:13.860 INFO:tasks.workunit.client.1.smithi033.stdout:8/400: truncate d1/d4/d1d/d36/d3c/d64/f7b 362992 0 2024-05-12T04:01:13.861 INFO:tasks.workunit.client.1.smithi033.stdout:8/401: chown d1/d16/d20/l61 47 1 2024-05-12T04:01:13.864 INFO:tasks.workunit.client.1.smithi033.stdout:8/402: dread d1/d16/d20/f65 [0,4194304] 0 2024-05-12T04:01:13.865 INFO:tasks.workunit.client.1.smithi033.stdout:1/430: rename d11/d17/d27/d36/d46/d4c/f73 to d11/d17/d38/f8f 0 2024-05-12T04:01:13.865 INFO:tasks.workunit.client.1.smithi033.stdout:8/403: readlink d1/d16/d27/l5f 0 2024-05-12T04:01:13.867 INFO:tasks.workunit.client.0.smithi032.stdout:5/362: rmdir d4/d7/d14/d39/d2b 39 2024-05-12T04:01:13.869 INFO:tasks.workunit.client.1.smithi033.stdout:5/404: truncate d3/d12/f1f 2075105 0 2024-05-12T04:01:13.873 INFO:tasks.workunit.client.0.smithi032.stdout:0/354: mknod df/c82 0 2024-05-12T04:01:13.874 INFO:tasks.workunit.client.0.smithi032.stdout:0/355: readlink df/d30/d4e/d55/l7c 0 2024-05-12T04:01:13.875 INFO:tasks.workunit.client.0.smithi032.stdout:0/356: chown df/d10/d3d/d4a/f2d 53 1 2024-05-12T04:01:13.882 INFO:tasks.workunit.client.1.smithi033.stdout:7/632: mknod de/d2b/d6c/cdc 0 2024-05-12T04:01:13.883 INFO:tasks.workunit.client.0.smithi032.stdout:9/224: dwrite d0/d3/d6/d7/d2a/f3b [0,4194304] 0 2024-05-12T04:01:13.884 INFO:tasks.workunit.client.0.smithi032.stdout:7/288: symlink dd/d1b/d1e/d45/l61 0 2024-05-12T04:01:13.884 INFO:tasks.workunit.client.0.smithi032.stdout:9/225: dread - d0/d3/f42 zero size 2024-05-12T04:01:13.893 INFO:tasks.workunit.client.1.smithi033.stdout:9/705: dread d2/d4/d2b/d6d/d7a/f81 [0,4194304] 0 2024-05-12T04:01:13.893 INFO:tasks.workunit.client.1.smithi033.stdout:4/476: mknod dd/d11/d5b/c98 0 2024-05-12T04:01:13.896 INFO:tasks.workunit.client.0.smithi032.stdout:3/348: mknod d15/d1b/c76 0 2024-05-12T04:01:13.905 INFO:tasks.workunit.client.0.smithi032.stdout:3/349: dwrite d15/d20/d22/f57 [0,4194304] 0 2024-05-12T04:01:13.907 INFO:tasks.workunit.client.1.smithi033.stdout:5/405: mkdir d3/d12/d25/d26/d2e/d54/d57/d73/d74/d7e/d81 0 2024-05-12T04:01:13.910 INFO:tasks.workunit.client.1.smithi033.stdout:1/431: dwrite d11/d1c/d59/f7f [0,4194304] 0 2024-05-12T04:01:13.912 INFO:tasks.workunit.client.0.smithi032.stdout:5/363: symlink d4/d7/d14/d39/d2b/d6e/l83 0 2024-05-12T04:01:13.915 INFO:tasks.workunit.client.1.smithi033.stdout:5/406: dwrite d3/f52 [0,4194304] 0 2024-05-12T04:01:13.918 INFO:tasks.workunit.client.1.smithi033.stdout:7/633: creat de/df/d17/d1e/fdd x:0 0 0 2024-05-12T04:01:13.918 INFO:tasks.workunit.client.0.smithi032.stdout:5/364: dread d4/fa [0,4194304] 0 2024-05-12T04:01:13.920 INFO:tasks.workunit.client.0.smithi032.stdout:7/289: creat dd/d1b/d1e/d45/d55/d60/f62 x:0 0 0 2024-05-12T04:01:13.923 INFO:tasks.workunit.client.1.smithi033.stdout:4/477: mknod dd/d7a/c99 0 2024-05-12T04:01:13.926 INFO:tasks.workunit.client.0.smithi032.stdout:1/300: rename d1/dd/c23 to d1/d25/c66 0 2024-05-12T04:01:13.935 INFO:tasks.workunit.client.0.smithi032.stdout:7/290: rmdir dd/d1b/d1e/d45/d55 39 2024-05-12T04:01:13.936 INFO:tasks.workunit.client.0.smithi032.stdout:7/291: write dd/d1b/d43/d5e/f1c [1258957,44024] 0 2024-05-12T04:01:13.938 INFO:tasks.workunit.client.0.smithi032.stdout:7/292: chown dd/d1b/d43/d5e/f4d 1546 1 2024-05-12T04:01:13.939 INFO:tasks.workunit.client.1.smithi033.stdout:1/432: rmdir d11/d17/d27 39 2024-05-12T04:01:13.943 INFO:tasks.workunit.client.1.smithi033.stdout:1/433: dread d11/d1c/f32 [0,4194304] 0 2024-05-12T04:01:13.944 INFO:tasks.workunit.client.0.smithi032.stdout:8/367: rename d0/d9/d1b/d27/d4c/d4e/c64 to d0/d9/d1b/d27/c7e 0 2024-05-12T04:01:13.948 INFO:tasks.workunit.client.1.smithi033.stdout:7/634: creat de/df/d7a/fde x:0 0 0 2024-05-12T04:01:13.953 INFO:tasks.workunit.client.0.smithi032.stdout:1/301: fsync d1/d7/d30/d18/f1b 0 2024-05-12T04:01:13.960 INFO:tasks.workunit.client.0.smithi032.stdout:5/365: creat d4/d7/d14/d26/d33/d5f/d7f/f84 x:0 0 0 2024-05-12T04:01:13.961 INFO:tasks.workunit.client.0.smithi032.stdout:5/366: fsync d4/d7/d14/d26/f5e 0 2024-05-12T04:01:13.965 INFO:tasks.workunit.client.0.smithi032.stdout:8/368: mkdir d0/d9/d1b/d27/d4c/d4e/d7f 0 2024-05-12T04:01:13.968 INFO:tasks.workunit.client.1.smithi033.stdout:7/635: rmdir de/d2b/d6c/da0 39 2024-05-12T04:01:13.971 INFO:tasks.workunit.client.0.smithi032.stdout:1/302: rename d1/dd/d24/d57 to d1/d7/d49/d67 0 2024-05-12T04:01:13.973 INFO:tasks.workunit.client.1.smithi033.stdout:7/636: dwrite de/df/d7a/fde [0,4194304] 0 2024-05-12T04:01:13.974 INFO:tasks.workunit.client.1.smithi033.stdout:8/404: sync 2024-05-12T04:01:13.975 INFO:tasks.workunit.client.1.smithi033.stdout:3/455: link db/l10 db/d17/d3a/la1 0 2024-05-12T04:01:13.987 INFO:tasks.workunit.client.1.smithi033.stdout:4/478: link dd/d11/d31/d38/l8e dd/d11/d5b/d84/l9a 0 2024-05-12T04:01:13.988 INFO:tasks.workunit.client.0.smithi032.stdout:2/311: getdents d1/d34/d4a 0 2024-05-12T04:01:13.991 INFO:tasks.workunit.client.0.smithi032.stdout:8/369: creat d0/d4b/f80 x:0 0 0 2024-05-12T04:01:13.994 INFO:tasks.workunit.client.0.smithi032.stdout:2/312: dwrite d1/d13/f2b [0,4194304] 0 2024-05-12T04:01:13.998 INFO:tasks.workunit.client.1.smithi033.stdout:0/497: write d6/db/f1a [4564469,39731] 0 2024-05-12T04:01:13.999 INFO:tasks.workunit.client.0.smithi032.stdout:1/303: creat d1/d7/d30/d61/f68 x:0 0 0 2024-05-12T04:01:14.003 INFO:tasks.workunit.client.1.smithi033.stdout:7/637: mknod de/d2b/d6c/da0/cdf 0 2024-05-12T04:01:14.008 INFO:tasks.workunit.client.1.smithi033.stdout:8/405: rmdir d1/d4/d1d/d36/d3c/d47 39 2024-05-12T04:01:14.011 INFO:tasks.workunit.client.0.smithi032.stdout:2/313: mknod d1/d13/d3d/d41/d52/c6e 0 2024-05-12T04:01:14.014 INFO:tasks.workunit.client.1.smithi033.stdout:3/456: rename db/l26 to db/d17/d3e/d65/d91/la2 0 2024-05-12T04:01:14.015 INFO:tasks.workunit.client.1.smithi033.stdout:3/457: write db/d17/d3a/d67/d76/f78 [893603,116770] 0 2024-05-12T04:01:14.016 INFO:tasks.workunit.client.1.smithi033.stdout:0/498: fdatasync d6/dc/d58/f85 0 2024-05-12T04:01:14.017 INFO:tasks.workunit.client.0.smithi032.stdout:2/314: dwrite d1/d13/f2b [0,4194304] 0 2024-05-12T04:01:14.020 INFO:tasks.workunit.client.1.smithi033.stdout:4/479: mknod dd/c9b 0 2024-05-12T04:01:14.022 INFO:tasks.workunit.client.1.smithi033.stdout:7/638: mkdir de/df/d17/d1b/de0 0 2024-05-12T04:01:14.025 INFO:tasks.workunit.client.0.smithi032.stdout:1/304: unlink d1/dd/c33 0 2024-05-12T04:01:14.026 INFO:tasks.workunit.client.1.smithi033.stdout:7/639: dread de/fa1 [0,4194304] 0 2024-05-12T04:01:14.028 INFO:tasks.workunit.client.1.smithi033.stdout:8/406: read d1/d4/d46/f4f [2550718,93340] 0 2024-05-12T04:01:14.030 INFO:tasks.workunit.client.0.smithi032.stdout:1/305: mknod d1/d7/d30/d61/c69 0 2024-05-12T04:01:14.031 INFO:tasks.workunit.client.0.smithi032.stdout:1/306: fsync d1/d7/d30/f28 0 2024-05-12T04:01:14.031 INFO:tasks.workunit.client.0.smithi032.stdout:6/274: write d7/de/f26 [174621,34315] 0 2024-05-12T04:01:14.032 INFO:tasks.workunit.client.0.smithi032.stdout:1/307: dread - d1/d7/d30/d35/f48 zero size 2024-05-12T04:01:14.033 INFO:tasks.workunit.client.0.smithi032.stdout:6/275: chown d7/de/d2e/f58 265 1 2024-05-12T04:01:14.033 INFO:tasks.workunit.client.0.smithi032.stdout:1/308: dread - d1/d25/f64 zero size 2024-05-12T04:01:14.034 INFO:tasks.workunit.client.0.smithi032.stdout:1/309: dread - d1/d7/d30/d61/f68 zero size 2024-05-12T04:01:14.036 INFO:tasks.workunit.client.0.smithi032.stdout:1/310: chown d1/d7/d30/d18 7321 1 2024-05-12T04:01:14.038 INFO:tasks.workunit.client.1.smithi033.stdout:2/475: mknod dc/c9c 0 2024-05-12T04:01:14.039 INFO:tasks.workunit.client.1.smithi033.stdout:2/476: dread - dc/d17/d27/d2f/f90 zero size 2024-05-12T04:01:14.039 INFO:tasks.workunit.client.0.smithi032.stdout:1/311: fdatasync d1/d7/d30/d35/f4e 0 2024-05-12T04:01:14.041 INFO:tasks.workunit.client.0.smithi032.stdout:6/276: dwrite d7/de/d12/f23 [0,4194304] 0 2024-05-12T04:01:14.041 INFO:tasks.workunit.client.0.smithi032.stdout:1/312: fdatasync d1/d7/d30/d35/f38 0 2024-05-12T04:01:14.049 INFO:tasks.workunit.client.0.smithi032.stdout:2/315: getdents d1/d13/d3d 0 2024-05-12T04:01:14.054 INFO:tasks.workunit.client.1.smithi033.stdout:6/482: write d4/d5/f27 [1038046,48050] 0 2024-05-12T04:01:14.057 INFO:tasks.workunit.client.0.smithi032.stdout:2/316: dwrite d1/d13/f5d [0,4194304] 0 2024-05-12T04:01:14.059 INFO:tasks.workunit.client.1.smithi033.stdout:1/434: rename d11/d17/f1d to d11/d17/d27/d36/d3f/d69/f90 0 2024-05-12T04:01:14.065 INFO:tasks.workunit.client.1.smithi033.stdout:2/477: chown dc/d17/d27/l70 35 1 2024-05-12T04:01:14.065 INFO:tasks.workunit.client.0.smithi032.stdout:1/313: mkdir d1/d6a 0 2024-05-12T04:01:14.067 INFO:tasks.workunit.client.1.smithi033.stdout:0/499: getdents d6/dc/d66/da4 0 2024-05-12T04:01:14.067 INFO:tasks.workunit.client.1.smithi033.stdout:1/435: dread d11/d17/d27/d36/d46/f51 [0,4194304] 0 2024-05-12T04:01:14.068 INFO:tasks.workunit.client.1.smithi033.stdout:0/500: dread - d6/f9e zero size 2024-05-12T04:01:14.068 INFO:tasks.workunit.client.1.smithi033.stdout:4/480: write dd/f2b [1587964,25616] 0 2024-05-12T04:01:14.069 INFO:tasks.workunit.client.0.smithi032.stdout:1/314: creat d1/d7/d49/f6b x:0 0 0 2024-05-12T04:01:14.069 INFO:tasks.workunit.client.1.smithi033.stdout:4/481: read - dd/d11/d31/d66/f7f zero size 2024-05-12T04:01:14.070 INFO:tasks.workunit.client.0.smithi032.stdout:1/315: write d1/f36 [2805509,121711] 0 2024-05-12T04:01:14.075 INFO:tasks.workunit.client.0.smithi032.stdout:7/293: rename l4 to dd/d1b/d1e/d45/d55/l63 0 2024-05-12T04:01:14.078 INFO:tasks.workunit.client.0.smithi032.stdout:1/316: rename d1/d7/c10 to d1/d7/d30/d35/c6c 0 2024-05-12T04:01:14.083 INFO:tasks.workunit.client.0.smithi032.stdout:1/317: symlink d1/d7/d49/d67/l6d 0 2024-05-12T04:01:14.084 INFO:tasks.workunit.client.1.smithi033.stdout:4/482: fdatasync dd/f1b 0 2024-05-12T04:01:14.085 INFO:tasks.workunit.client.0.smithi032.stdout:7/294: mkdir dd/d1b/d1e/d45/d64 0 2024-05-12T04:01:14.086 INFO:tasks.workunit.client.0.smithi032.stdout:7/295: write dd/d1b/d1e/f28 [127328,93285] 0 2024-05-12T04:01:14.090 INFO:tasks.workunit.client.0.smithi032.stdout:1/318: creat d1/d6a/f6e x:0 0 0 2024-05-12T04:01:14.091 INFO:tasks.workunit.client.0.smithi032.stdout:7/296: dread dd/d1b/d1e/d45/d55/f5b [0,4194304] 0 2024-05-12T04:01:14.099 INFO:tasks.workunit.client.1.smithi033.stdout:3/458: rename db/l73 to db/d17/d25/d32/d3c/la3 0 2024-05-12T04:01:14.100 INFO:tasks.workunit.client.0.smithi032.stdout:7/297: mknod dd/d1b/d1e/d45/c65 0 2024-05-12T04:01:14.100 INFO:tasks.workunit.client.1.smithi033.stdout:3/459: write db/d17/d3e/d65/d7d/f94 [745062,91401] 0 2024-05-12T04:01:14.102 INFO:tasks.workunit.client.1.smithi033.stdout:4/483: mkdir dd/d11/d31/d66/d42/d9c 0 2024-05-12T04:01:14.103 INFO:tasks.workunit.client.1.smithi033.stdout:4/484: write dd/d69/f97 [58197,21635] 0 2024-05-12T04:01:14.104 INFO:tasks.workunit.client.1.smithi033.stdout:2/478: rename dc/d17/c78 to dc/d13/d39/d8f/c9d 0 2024-05-12T04:01:14.106 INFO:tasks.workunit.client.1.smithi033.stdout:9/706: write d2/d4/d2b/d6d/db5/fcb [943632,98155] 0 2024-05-12T04:01:14.106 INFO:tasks.workunit.client.1.smithi033.stdout:3/460: fsync db/d17/d25/f40 0 2024-05-12T04:01:14.107 INFO:tasks.workunit.client.0.smithi032.stdout:9/226: truncate d0/d3/d6/d7/da/d17/f20 12970 0 2024-05-12T04:01:14.107 INFO:tasks.workunit.client.0.smithi032.stdout:1/319: dread d1/d7/d30/d18/f1b [0,4194304] 0 2024-05-12T04:01:14.108 INFO:tasks.workunit.client.1.smithi033.stdout:9/707: read d2/d4/d29/d3c/d52/f5e [2571550,92276] 0 2024-05-12T04:01:14.109 INFO:tasks.workunit.client.1.smithi033.stdout:4/485: rmdir dd/d11/d31/d66 39 2024-05-12T04:01:14.109 INFO:tasks.workunit.client.0.smithi032.stdout:1/320: read d1/d7/d30/d35/d46/f4b [152117,28910] 0 2024-05-12T04:01:14.110 INFO:tasks.workunit.client.0.smithi032.stdout:7/298: truncate dd/f1a 8616226 0 2024-05-12T04:01:14.112 INFO:tasks.workunit.client.1.smithi033.stdout:1/436: rename d11/d17/d27/d36/c79 to d11/d1c/d7d/c91 0 2024-05-12T04:01:14.113 INFO:tasks.workunit.client.0.smithi032.stdout:9/227: readlink d0/d3/d6/d7/da/d39/l37 0 2024-05-12T04:01:14.114 INFO:tasks.workunit.client.1.smithi033.stdout:2/479: mkdir dc/d13/d39/d8f/d9e 0 2024-05-12T04:01:14.115 INFO:tasks.workunit.client.1.smithi033.stdout:9/708: dwrite d2/d4/d2b/d49/ff0 [0,4194304] 0 2024-05-12T04:01:14.117 INFO:tasks.workunit.client.1.smithi033.stdout:9/709: write d2/d4/d2b/d6d/d7a/fed [747991,38134] 0 2024-05-12T04:01:14.119 INFO:tasks.workunit.client.0.smithi032.stdout:9/228: dwrite d0/d3/f25 [0,4194304] 0 2024-05-12T04:01:14.123 INFO:tasks.workunit.client.0.smithi032.stdout:9/229: truncate d0/d3/d6/d7/d27/f48 571445 0 2024-05-12T04:01:14.123 INFO:tasks.workunit.client.1.smithi033.stdout:3/461: unlink db/d17/d25/d32/d3c/d44/f4a 0 2024-05-12T04:01:14.124 INFO:tasks.workunit.client.0.smithi032.stdout:9/230: dread - d0/d3/d6/d7/d2a/d46/f49 zero size 2024-05-12T04:01:14.127 INFO:tasks.workunit.client.0.smithi032.stdout:7/299: rename dd/d1b/d43/d5e/c26 to dd/d1b/d1e/c66 0 2024-05-12T04:01:14.129 INFO:tasks.workunit.client.1.smithi033.stdout:5/407: write d3/d12/d25/d26/d2e/f3e [252109,82143] 0 2024-05-12T04:01:14.130 INFO:tasks.workunit.client.0.smithi032.stdout:3/350: truncate d15/d24/d40/f5a 116324 0 2024-05-12T04:01:14.132 INFO:tasks.workunit.client.1.smithi033.stdout:3/462: dwrite db/d17/d25/d32/d3c/d44/f55 [0,4194304] 0 2024-05-12T04:01:14.132 INFO:tasks.workunit.client.1.smithi033.stdout:0/501: mknod d6/ca6 0 2024-05-12T04:01:14.136 INFO:tasks.workunit.client.0.smithi032.stdout:1/321: symlink d1/d59/l6f 0 2024-05-12T04:01:14.137 INFO:tasks.workunit.client.1.smithi033.stdout:1/437: read d11/d17/f3e [737678,124105] 0 2024-05-12T04:01:14.139 INFO:tasks.workunit.client.0.smithi032.stdout:9/231: mknod d0/d3/d6/d7/da/d17/c4a 0 2024-05-12T04:01:14.141 INFO:tasks.workunit.client.0.smithi032.stdout:4/226: mkdir d0/d39 0 2024-05-12T04:01:14.142 INFO:tasks.workunit.client.0.smithi032.stdout:9/232: write d0/d3/d6/d7/da/f1c [34514,113119] 0 2024-05-12T04:01:14.145 INFO:tasks.workunit.client.1.smithi033.stdout:2/480: rename dc/d17/d27/d47 to dc/d17/d27/d68/d9f 0 2024-05-12T04:01:14.145 INFO:tasks.workunit.client.1.smithi033.stdout:3/463: dread db/d17/d3e/f48 [0,4194304] 0 2024-05-12T04:01:14.146 INFO:tasks.workunit.client.0.smithi032.stdout:0/357: sync 2024-05-12T04:01:14.147 INFO:tasks.workunit.client.1.smithi033.stdout:2/481: write dc/d17/d3a/f9a [966496,59891] 0 2024-05-12T04:01:14.147 INFO:tasks.workunit.client.0.smithi032.stdout:3/351: unlink d15/d33/f56 0 2024-05-12T04:01:14.152 INFO:tasks.workunit.client.0.smithi032.stdout:4/227: truncate d0/d3/df/f30 2180979 0 2024-05-12T04:01:14.153 INFO:tasks.workunit.client.0.smithi032.stdout:9/233: dwrite d0/d3/d6/d7/d2a/d46/f12 [0,4194304] 0 2024-05-12T04:01:14.155 INFO:tasks.workunit.client.1.smithi033.stdout:3/464: fdatasync db/d17/d25/d32/d3c/d44/f55 0 2024-05-12T04:01:14.156 INFO:tasks.workunit.client.0.smithi032.stdout:0/358: dwrite df/d30/d36/f6d [0,4194304] 0 2024-05-12T04:01:14.157 INFO:tasks.workunit.client.0.smithi032.stdout:0/359: dread - df/d10/f64 zero size 2024-05-12T04:01:14.169 INFO:tasks.workunit.client.1.smithi033.stdout:5/408: truncate d3/d27/f6a 2102791 0 2024-05-12T04:01:14.170 INFO:tasks.workunit.client.0.smithi032.stdout:5/367: truncate d4/f49 468034 0 2024-05-12T04:01:14.172 INFO:tasks.workunit.client.0.smithi032.stdout:5/368: truncate d4/d7/d14/d26/d33/d4a/f3e 1421549 0 2024-05-12T04:01:14.172 INFO:tasks.workunit.client.1.smithi033.stdout:0/502: creat d6/d11/d98/fa7 x:0 0 0 2024-05-12T04:01:14.173 INFO:tasks.workunit.client.0.smithi032.stdout:5/369: write d4/d7/d14/f1d [4134181,102972] 0 2024-05-12T04:01:14.173 INFO:tasks.workunit.client.1.smithi033.stdout:5/409: dwrite d3/d15/d16/d45/d4b/f56 [0,4194304] 0 2024-05-12T04:01:14.176 INFO:tasks.workunit.client.1.smithi033.stdout:1/438: creat d11/d17/d27/d36/d46/f92 x:0 0 0 2024-05-12T04:01:14.178 INFO:tasks.workunit.client.1.smithi033.stdout:1/439: write d11/d17/d27/d36/f3c [842308,21080] 0 2024-05-12T04:01:14.178 INFO:tasks.workunit.client.0.smithi032.stdout:4/228: symlink d0/l3a 0 2024-05-12T04:01:14.180 INFO:tasks.workunit.client.0.smithi032.stdout:9/234: mkdir d0/d3/d4b 0 2024-05-12T04:01:14.183 INFO:tasks.workunit.client.0.smithi032.stdout:0/360: mkdir df/d10/d3d/d83 0 2024-05-12T04:01:14.183 INFO:tasks.workunit.client.1.smithi033.stdout:2/482: truncate dc/d17/d27/f60 276688 0 2024-05-12T04:01:14.302 INFO:tasks.workunit.client.0.smithi032.stdout:5/370: unlink d4/d7/d14/d26/d33/d4a/f3e 0 2024-05-12T04:01:14.303 INFO:tasks.workunit.client.1.smithi033.stdout:3/465: mknod db/d17/d3a/d8a/ca4 0 2024-05-12T04:01:14.304 INFO:tasks.workunit.client.0.smithi032.stdout:4/229: truncate d0/d3/fd 1474035 0 2024-05-12T04:01:14.304 INFO:tasks.workunit.client.0.smithi032.stdout:4/230: readlink d0/l34 0 2024-05-12T04:01:14.305 INFO:tasks.workunit.client.1.smithi033.stdout:4/486: rmdir dd/d91 0 2024-05-12T04:01:14.306 INFO:tasks.workunit.client.0.smithi032.stdout:8/370: write d0/d9/d13/d26/f55 [4765949,112909] 0 2024-05-12T04:01:14.306 INFO:tasks.workunit.client.1.smithi033.stdout:4/487: chown dd/d11/f21 37 1 2024-05-12T04:01:14.307 INFO:tasks.workunit.client.0.smithi032.stdout:9/235: mknod d0/d3/d6/d7/c4c 0 2024-05-12T04:01:14.308 INFO:tasks.workunit.client.0.smithi032.stdout:8/371: dread - d0/d9/d13/d21/d35/d37/d56/f5b zero size 2024-05-12T04:01:14.312 INFO:tasks.workunit.client.0.smithi032.stdout:0/361: mkdir df/d10/d3d/d4a/d84 0 2024-05-12T04:01:14.322 INFO:tasks.workunit.client.1.smithi033.stdout:1/440: mkdir d11/d17/d27/d76/d93 0 2024-05-12T04:01:14.323 INFO:tasks.workunit.client.0.smithi032.stdout:5/371: creat d4/d7/d14/d39/d2b/d6e/f85 x:0 0 0 2024-05-12T04:01:14.325 INFO:tasks.workunit.client.0.smithi032.stdout:5/372: fdatasync d4/d7/d14/d26/d33/d4a/d38/d3b/d4c/f6a 0 2024-05-12T04:01:14.326 INFO:tasks.workunit.client.0.smithi032.stdout:4/231: mknod d0/d3/d31/c3b 0 2024-05-12T04:01:14.328 INFO:tasks.workunit.client.1.smithi033.stdout:8/407: write d1/f13 [675371,83855] 0 2024-05-12T04:01:14.329 INFO:tasks.workunit.client.1.smithi033.stdout:2/483: read dc/d13/d1f/f93 [420445,100581] 0 2024-05-12T04:01:14.329 INFO:tasks.workunit.client.0.smithi032.stdout:8/372: symlink d0/d4b/l81 0 2024-05-12T04:01:14.333 INFO:tasks.workunit.client.1.smithi033.stdout:4/488: readlink dd/d11/d31/d66/l8a 0 2024-05-12T04:01:14.335 INFO:tasks.workunit.client.1.smithi033.stdout:4/489: write dd/d11/d31/d66/d42/f82 [612544,68461] 0 2024-05-12T04:01:14.335 INFO:tasks.workunit.client.0.smithi032.stdout:6/277: dwrite d7/de/d12/f16 [0,4194304] 0 2024-05-12T04:01:14.335 INFO:tasks.workunit.client.1.smithi033.stdout:4/490: chown dd/d11 112 1 2024-05-12T04:01:14.336 INFO:tasks.workunit.client.1.smithi033.stdout:4/491: chown dd/d11/l3f 17 1 2024-05-12T04:01:14.337 INFO:tasks.workunit.client.1.smithi033.stdout:4/492: dread - dd/d11/d31/d66/f92 zero size 2024-05-12T04:01:14.339 INFO:tasks.workunit.client.1.smithi033.stdout:0/503: creat d6/db/d2c/d82/fa8 x:0 0 0 2024-05-12T04:01:14.343 INFO:tasks.workunit.client.1.smithi033.stdout:6/483: dwrite d4/f29 [4194304,4194304] 0 2024-05-12T04:01:14.343 INFO:tasks.workunit.client.1.smithi033.stdout:5/410: mknod d3/d12/d25/d26/d2e/d54/d57/d70/c82 0 2024-05-12T04:01:14.344 INFO:tasks.workunit.client.1.smithi033.stdout:6/484: chown d4/d5/l31 1 1 2024-05-12T04:01:14.344 INFO:tasks.workunit.client.1.smithi033.stdout:5/411: chown d3/d12/f3f 1 1 2024-05-12T04:01:14.346 INFO:tasks.workunit.client.0.smithi032.stdout:8/373: creat d0/d9/d13/d26/d28/f82 x:0 0 0 2024-05-12T04:01:14.349 INFO:tasks.workunit.client.1.smithi033.stdout:1/441: creat d11/d1c/f94 x:0 0 0 2024-05-12T04:01:14.351 INFO:tasks.workunit.client.1.smithi033.stdout:9/710: getdents d2/d4/d2b/d78/d76 0 2024-05-12T04:01:14.356 INFO:tasks.workunit.client.1.smithi033.stdout:8/408: rmdir d1/d16/d27/d4d 39 2024-05-12T04:01:14.358 INFO:tasks.workunit.client.1.smithi033.stdout:2/484: fdatasync dc/d17/f4d 0 2024-05-12T04:01:14.360 INFO:tasks.workunit.client.1.smithi033.stdout:3/466: symlink db/d17/d3a/d8a/d9d/la5 0 2024-05-12T04:01:14.363 INFO:tasks.workunit.client.0.smithi032.stdout:8/374: rmdir d0/d7 39 2024-05-12T04:01:14.367 INFO:tasks.workunit.client.0.smithi032.stdout:0/362: creat df/d10/d3d/f85 x:0 0 0 2024-05-12T04:01:14.368 INFO:tasks.workunit.client.1.smithi033.stdout:7/640: fdatasync de/df/d17/d1b/f89 0 2024-05-12T04:01:14.368 INFO:tasks.workunit.client.0.smithi032.stdout:0/363: write df/d10/d3d/d4a/f29 [560055,80308] 0 2024-05-12T04:01:14.371 INFO:tasks.workunit.client.0.smithi032.stdout:6/278: link d7/d9/d48/f5d d7/de/d12/d25/f61 0 2024-05-12T04:01:14.373 INFO:tasks.workunit.client.0.smithi032.stdout:6/279: chown d7/d9/l11 2240245 1 2024-05-12T04:01:14.373 INFO:tasks.workunit.client.1.smithi033.stdout:4/493: symlink dd/d7a/d87/d7e/l9d 0 2024-05-12T04:01:14.374 INFO:tasks.workunit.client.0.smithi032.stdout:0/364: dwrite f5 [0,4194304] 0 2024-05-12T04:01:14.382 INFO:tasks.workunit.client.1.smithi033.stdout:5/412: mkdir d3/d12/d83 0 2024-05-12T04:01:14.388 INFO:tasks.workunit.client.1.smithi033.stdout:1/442: chown d11/d17/f18 454 1 2024-05-12T04:01:14.388 INFO:tasks.workunit.client.1.smithi033.stdout:1/443: chown d11/d1c/d43/f6f 103306 1 2024-05-12T04:01:14.390 INFO:tasks.workunit.client.1.smithi033.stdout:9/711: symlink d2/d4/d2b/d6d/d7a/l100 0 2024-05-12T04:01:14.392 INFO:tasks.workunit.client.0.smithi032.stdout:0/365: rename df/l71 to df/d10/d3d/d6c/l86 0 2024-05-12T04:01:14.398 INFO:tasks.workunit.client.1.smithi033.stdout:2/485: creat dc/d17/d32/d84/fa0 x:0 0 0 2024-05-12T04:01:14.401 INFO:tasks.workunit.client.1.smithi033.stdout:3/467: mkdir db/d17/d3a/d67/da6 0 2024-05-12T04:01:14.405 INFO:tasks.workunit.client.1.smithi033.stdout:7/641: chown de/df/d17/d1b/d50 2084220 1 2024-05-12T04:01:14.406 INFO:tasks.workunit.client.0.smithi032.stdout:1/322: truncate d1/f13 429386 0 2024-05-12T04:01:14.407 INFO:tasks.workunit.client.0.smithi032.stdout:1/323: chown d1/d25/f4a 508 1 2024-05-12T04:01:14.409 INFO:tasks.workunit.client.0.smithi032.stdout:6/280: getdents d7/d9/d48 0 2024-05-12T04:01:14.410 INFO:tasks.workunit.client.0.smithi032.stdout:1/324: chown d1/d7/cc 2 1 2024-05-12T04:01:14.410 INFO:tasks.workunit.client.1.smithi033.stdout:0/504: mknod d6/dc/d66/da4/ca9 0 2024-05-12T04:01:14.412 INFO:tasks.workunit.client.0.smithi032.stdout:0/366: creat df/d10/d3d/f87 x:0 0 0 2024-05-12T04:01:14.414 INFO:tasks.workunit.client.1.smithi033.stdout:5/413: rmdir d3/d12/d25/d26/d2e/d54/d57/d73 39 2024-05-12T04:01:14.415 INFO:tasks.workunit.client.1.smithi033.stdout:5/414: dread - d3/d15/d16/d45/f68 zero size 2024-05-12T04:01:14.419 INFO:tasks.workunit.client.1.smithi033.stdout:1/444: rename d11/c28 to d11/d17/d27/d76/c95 0 2024-05-12T04:01:14.419 INFO:tasks.workunit.client.0.smithi032.stdout:6/281: dread d7/de/f1b [0,4194304] 0 2024-05-12T04:01:14.421 INFO:tasks.workunit.client.1.smithi033.stdout:1/445: read d11/d17/d27/d76/f6e [3794746,84804] 0 2024-05-12T04:01:14.421 INFO:tasks.workunit.client.1.smithi033.stdout:1/446: fsync d11/d17/d27/f57 0 2024-05-12T04:01:14.422 INFO:tasks.workunit.client.1.smithi033.stdout:9/712: unlink d2/d4/d2b/d33/l58 0 2024-05-12T04:01:14.423 INFO:tasks.workunit.client.1.smithi033.stdout:1/447: dread - d11/d17/d27/d36/d46/f92 zero size 2024-05-12T04:01:14.423 INFO:tasks.workunit.client.1.smithi033.stdout:9/713: stat d2/d4/d29/lb3 0 2024-05-12T04:01:14.425 INFO:tasks.workunit.client.1.smithi033.stdout:8/409: mknod d1/d4/d1d/d36/c7e 0 2024-05-12T04:01:14.425 INFO:tasks.workunit.client.1.smithi033.stdout:8/410: dread - d1/d4/d46/f7a zero size 2024-05-12T04:01:14.426 INFO:tasks.workunit.client.0.smithi032.stdout:1/325: truncate d1/d7/d30/d35/f4e 2826408 0 2024-05-12T04:01:14.428 INFO:tasks.workunit.client.1.smithi033.stdout:2/486: symlink dc/d17/d32/d84/la1 0 2024-05-12T04:01:14.433 INFO:tasks.workunit.client.0.smithi032.stdout:1/326: creat d1/d7/d30/d18/f70 x:0 0 0 2024-05-12T04:01:14.435 INFO:tasks.workunit.client.0.smithi032.stdout:0/367: link df/d30/d36/l68 df/d22/l88 0 2024-05-12T04:01:14.438 INFO:tasks.workunit.client.0.smithi032.stdout:6/282: getdents d7/d9/d48 0 2024-05-12T04:01:14.442 INFO:tasks.workunit.client.0.smithi032.stdout:1/327: unlink d1/d7/d30/d18/f1b 0 2024-05-12T04:01:14.445 INFO:tasks.workunit.client.0.smithi032.stdout:0/368: truncate df/d10/d3d/d4a/f2d 774485 0 2024-05-12T04:01:14.445 INFO:tasks.workunit.client.0.smithi032.stdout:1/328: dread d1/d7/d30/d35/f45 [0,4194304] 0 2024-05-12T04:01:14.446 INFO:tasks.workunit.client.0.smithi032.stdout:5/373: write d4/d7/d14/d39/d2b/d6e/f77 [3293071,35629] 0 2024-05-12T04:01:14.448 INFO:tasks.workunit.client.0.smithi032.stdout:6/283: write d7/de/d12/d1a/d20/f34 [448000,109525] 0 2024-05-12T04:01:14.448 INFO:tasks.workunit.client.0.smithi032.stdout:5/374: dread - d4/d7/d14/d26/f66 zero size 2024-05-12T04:01:14.449 INFO:tasks.workunit.client.0.smithi032.stdout:2/317: sync 2024-05-12T04:01:14.453 INFO:tasks.workunit.client.0.smithi032.stdout:9/236: truncate d0/d3/ff 11242597 0 2024-05-12T04:01:14.455 INFO:tasks.workunit.client.0.smithi032.stdout:6/284: chown d7/f38 1505 1 2024-05-12T04:01:14.457 INFO:tasks.workunit.client.0.smithi032.stdout:2/318: read d1/d13/f48 [2119063,2370] 0 2024-05-12T04:01:14.458 INFO:tasks.workunit.client.0.smithi032.stdout:6/285: fsync d7/f59 0 2024-05-12T04:01:14.464 INFO:tasks.workunit.client.0.smithi032.stdout:1/329: dread d1/d7/f5d [0,4194304] 0 2024-05-12T04:01:14.464 INFO:tasks.workunit.client.0.smithi032.stdout:5/375: dwrite d4/d7/d14/f1d [0,4194304] 0 2024-05-12T04:01:14.467 INFO:tasks.workunit.client.0.smithi032.stdout:9/237: creat d0/d3/d6/d7/da/d39/f4d x:0 0 0 2024-05-12T04:01:14.468 INFO:tasks.workunit.client.0.smithi032.stdout:2/319: dread d1/fa [0,4194304] 0 2024-05-12T04:01:14.481 INFO:tasks.workunit.client.0.smithi032.stdout:8/375: dwrite d0/d9/d13/f58 [0,4194304] 0 2024-05-12T04:01:14.483 INFO:tasks.workunit.client.0.smithi032.stdout:2/320: rename d1/d34/d3e/l63 to d1/d13/d6b/l6f 0 2024-05-12T04:01:14.488 INFO:tasks.workunit.client.0.smithi032.stdout:1/330: rename d1/d7/c3b to d1/d7/c71 0 2024-05-12T04:01:14.488 INFO:tasks.workunit.client.0.smithi032.stdout:4/232: write d0/d3/df/f30 [3122024,23489] 0 2024-05-12T04:01:14.490 INFO:tasks.workunit.client.0.smithi032.stdout:4/233: fdatasync d0/d3/df/f26 0 2024-05-12T04:01:14.497 INFO:tasks.workunit.client.1.smithi033.stdout:3/468: dwrite db/d17/d25/d32/f46 [0,4194304] 0 2024-05-12T04:01:14.509 INFO:tasks.workunit.client.0.smithi032.stdout:9/238: rename d0/d3/d1b/f24 to d0/d3/d6/d7/da/d17/f4e 0 2024-05-12T04:01:14.511 INFO:tasks.workunit.client.0.smithi032.stdout:2/321: mkdir d1/d13/d70 0 2024-05-12T04:01:14.513 INFO:tasks.workunit.client.0.smithi032.stdout:2/322: readlink d1/d13/d3d/d41/d39/l42 0 2024-05-12T04:01:14.514 INFO:tasks.workunit.client.1.smithi033.stdout:6/485: link d4/d5/f27 d4/d1a/d4e/d4f/f9e 0 2024-05-12T04:01:14.518 INFO:tasks.workunit.client.0.smithi032.stdout:1/331: mkdir d1/d6a/d72 0 2024-05-12T04:01:14.519 INFO:tasks.workunit.client.0.smithi032.stdout:1/332: write d1/d7/d30/f52 [713532,66069] 0 2024-05-12T04:01:14.524 INFO:tasks.workunit.client.0.smithi032.stdout:5/376: link d4/d7/d14/d26/d47/f65 d4/d7/d14/d26/d33/d4a/f86 0 2024-05-12T04:01:14.525 INFO:tasks.workunit.client.1.smithi033.stdout:1/448: mkdir d11/d17/d27/d36/d3f/d69/d87/d96 0 2024-05-12T04:01:14.528 INFO:tasks.workunit.client.0.smithi032.stdout:8/376: creat d0/d9/d13/d21/d25/f83 x:0 0 0 2024-05-12T04:01:14.532 INFO:tasks.workunit.client.1.smithi033.stdout:8/411: unlink d1/d16/f72 0 2024-05-12T04:01:14.533 INFO:tasks.workunit.client.0.smithi032.stdout:0/369: truncate fb 846772 0 2024-05-12T04:01:14.534 INFO:tasks.workunit.client.0.smithi032.stdout:9/239: mkdir d0/d3/d6/d7/d2a/d46/d4f 0 2024-05-12T04:01:14.535 INFO:tasks.workunit.client.0.smithi032.stdout:9/240: read d0/d3/d6/d7/d2a/f3b [1921427,15983] 0 2024-05-12T04:01:14.536 INFO:tasks.workunit.client.1.smithi033.stdout:2/487: fdatasync dc/d17/d27/d68/d9f/f58 0 2024-05-12T04:01:14.536 INFO:tasks.workunit.client.0.smithi032.stdout:7/300: sync 2024-05-12T04:01:14.537 INFO:tasks.workunit.client.0.smithi032.stdout:9/241: write d0/d3/d6/d7/d27/f31 [315176,99963] 0 2024-05-12T04:01:14.538 INFO:tasks.workunit.client.0.smithi032.stdout:9/242: dread - d0/d3/d6/d7/da/d39/f4d zero size 2024-05-12T04:01:14.540 INFO:tasks.workunit.client.0.smithi032.stdout:9/243: write d0/d3/d6/d7/d2a/d46/d13/f1d [5002576,20018] 0 2024-05-12T04:01:14.541 INFO:tasks.workunit.client.0.smithi032.stdout:2/323: rename d1/d34/f32 to d1/d4d/d5e/f71 0 2024-05-12T04:01:14.542 INFO:tasks.workunit.client.0.smithi032.stdout:6/286: getdents d7/d33 0 2024-05-12T04:01:14.544 INFO:tasks.workunit.client.0.smithi032.stdout:6/287: write d7/de/d12/d25/f3e [8451901,39269] 0 2024-05-12T04:01:14.547 INFO:tasks.workunit.client.0.smithi032.stdout:6/288: fsync d7/de/d12/d25/f3e 0 2024-05-12T04:01:14.547 INFO:tasks.workunit.client.0.smithi032.stdout:4/234: symlink d0/d3/d12/l3c 0 2024-05-12T04:01:14.549 INFO:tasks.workunit.client.0.smithi032.stdout:5/377: creat d4/d7/d14/d26/d33/d5f/f87 x:0 0 0 2024-05-12T04:01:14.550 INFO:tasks.workunit.client.1.smithi033.stdout:0/505: mknod d6/dc/d58/d6a/d94/caa 0 2024-05-12T04:01:14.550 INFO:tasks.workunit.client.0.smithi032.stdout:9/244: dwrite d0/d3/d6/d7/d2a/d46/d13/f1d [4194304,4194304] 0 2024-05-12T04:01:14.551 INFO:tasks.workunit.client.0.smithi032.stdout:5/378: stat d4/d7/d14/d26/c70 0 2024-05-12T04:01:14.554 INFO:tasks.workunit.client.0.smithi032.stdout:4/235: dread d0/d3/df/f29 [0,4194304] 0 2024-05-12T04:01:14.555 INFO:tasks.workunit.client.1.smithi033.stdout:5/415: mkdir d3/d15/d16/d45/d62/d7f/d84 0 2024-05-12T04:01:14.556 INFO:tasks.workunit.client.0.smithi032.stdout:8/377: mkdir d0/d9/d1b/d66/d75/d84 0 2024-05-12T04:01:14.556 INFO:tasks.workunit.client.0.smithi032.stdout:0/370: creat df/d28/f89 x:0 0 0 2024-05-12T04:01:14.558 INFO:tasks.workunit.client.1.smithi033.stdout:3/469: dread db/d17/d25/f52 [4194304,4194304] 0 2024-05-12T04:01:14.562 INFO:tasks.workunit.client.0.smithi032.stdout:7/301: stat dd/f12 0 2024-05-12T04:01:14.566 INFO:tasks.workunit.client.1.smithi033.stdout:8/412: truncate d1/d16/d27/d4d/d51/f4b 4613000 0 2024-05-12T04:01:14.567 INFO:tasks.workunit.client.1.smithi033.stdout:2/488: mkdir dc/d13/d1c/d98/da2 0 2024-05-12T04:01:14.568 INFO:tasks.workunit.client.0.smithi032.stdout:2/324: dread d1/f31 [0,4194304] 0 2024-05-12T04:01:14.569 INFO:tasks.workunit.client.1.smithi033.stdout:4/494: getdents dd/d11/d5b 0 2024-05-12T04:01:14.569 INFO:tasks.workunit.client.1.smithi033.stdout:7/642: write de/d20/d64/f93 [679742,8300] 0 2024-05-12T04:01:14.572 INFO:tasks.workunit.client.1.smithi033.stdout:7/643: write de/df/d17/d1e/dcc/fce [140235,82826] 0 2024-05-12T04:01:14.575 INFO:tasks.workunit.client.0.smithi032.stdout:1/333: creat d1/d7/d30/d35/d54/f73 x:0 0 0 2024-05-12T04:01:14.576 INFO:tasks.workunit.client.1.smithi033.stdout:0/506: dwrite d6/dc/d39/f40 [0,4194304] 0 2024-05-12T04:01:14.576 INFO:tasks.workunit.client.1.smithi033.stdout:0/507: chown d6/dc/f46 1015259 1 2024-05-12T04:01:14.578 INFO:tasks.workunit.client.1.smithi033.stdout:0/508: truncate d6/d11/d57/f93 45504 0 2024-05-12T04:01:14.585 INFO:tasks.workunit.client.0.smithi032.stdout:6/289: mkdir d7/d9/d5c/d62 0 2024-05-12T04:01:14.587 INFO:tasks.workunit.client.1.smithi033.stdout:3/470: creat db/d17/d3e/d65/d91/fa7 x:0 0 0 2024-05-12T04:01:14.588 INFO:tasks.workunit.client.0.smithi032.stdout:9/245: creat d0/d3/d6/d7/d2a/f50 x:0 0 0 2024-05-12T04:01:14.589 INFO:tasks.workunit.client.1.smithi033.stdout:9/714: link d2/dd/l91 d2/d4/d2b/d6d/d2f/l101 0 2024-05-12T04:01:14.591 INFO:tasks.workunit.client.0.smithi032.stdout:5/379: mkdir d4/d7/d14/d26/d33/d88 0 2024-05-12T04:01:14.594 INFO:tasks.workunit.client.1.smithi033.stdout:2/489: rmdir dc/d17/d32 39 2024-05-12T04:01:14.596 INFO:tasks.workunit.client.0.smithi032.stdout:0/371: mkdir df/d22/d58/d8a 0 2024-05-12T04:01:14.598 INFO:tasks.workunit.client.1.smithi033.stdout:4/495: rename dd/d11/f14 to dd/d11/d5b/d84/f9e 0 2024-05-12T04:01:14.603 INFO:tasks.workunit.client.1.smithi033.stdout:4/496: dwrite dd/f4b [4194304,4194304] 0 2024-05-12T04:01:14.603 INFO:tasks.workunit.client.0.smithi032.stdout:7/302: unlink dd/d1b/d43/d5e/f33 0 2024-05-12T04:01:14.606 INFO:tasks.workunit.client.1.smithi033.stdout:0/509: read d6/f26 [5881551,6865] 0 2024-05-12T04:01:14.613 INFO:tasks.workunit.client.0.smithi032.stdout:1/334: mknod d1/d7/d30/d61/c74 0 2024-05-12T04:01:14.618 INFO:tasks.workunit.client.0.smithi032.stdout:2/325: dwrite d1/d13/f1f [0,4194304] 0 2024-05-12T04:01:14.619 INFO:tasks.workunit.client.1.smithi033.stdout:9/715: creat d2/d4/d29/d3c/f102 x:0 0 0 2024-05-12T04:01:14.620 INFO:tasks.workunit.client.1.smithi033.stdout:4/497: dread dd/f13 [0,4194304] 0 2024-05-12T04:01:14.620 INFO:tasks.workunit.client.0.smithi032.stdout:6/290: mkdir d7/de/d12/d63 0 2024-05-12T04:01:14.621 INFO:tasks.workunit.client.1.smithi033.stdout:3/471: dwrite db/d17/d25/d32/f5d [4194304,4194304] 0 2024-05-12T04:01:14.622 INFO:tasks.workunit.client.1.smithi033.stdout:3/472: stat db/d17/d25/f40 0 2024-05-12T04:01:14.623 INFO:tasks.workunit.client.0.smithi032.stdout:6/291: write d7/de/f14 [552082,28841] 0 2024-05-12T04:01:14.624 INFO:tasks.workunit.client.1.smithi033.stdout:1/449: write d11/f2d [816443,48774] 0 2024-05-12T04:01:14.626 INFO:tasks.workunit.client.0.smithi032.stdout:9/246: mkdir d0/d3/d6/d7/da/d17/d51 0 2024-05-12T04:01:14.631 INFO:tasks.workunit.client.0.smithi032.stdout:8/378: symlink d0/l85 0 2024-05-12T04:01:14.632 INFO:tasks.workunit.client.1.smithi033.stdout:2/490: rename dc/d17/d27/d68/c72 to dc/d13/d1f/d6e/ca3 0 2024-05-12T04:01:14.634 INFO:tasks.workunit.client.0.smithi032.stdout:8/379: dread d0/d9/d1b/f4d [0,4194304] 0 2024-05-12T04:01:14.637 INFO:tasks.workunit.client.0.smithi032.stdout:8/380: write d0/d9/d13/d21/d35/d37/d56/f65 [328360,19969] 0 2024-05-12T04:01:14.640 INFO:tasks.workunit.client.1.smithi033.stdout:6/486: truncate d4/d5/f73 626164 0 2024-05-12T04:01:14.640 INFO:tasks.workunit.client.1.smithi033.stdout:0/510: fsync d6/dc/f16 0 2024-05-12T04:01:14.644 INFO:tasks.workunit.client.0.smithi032.stdout:1/335: chown d1/d7/d30/l5a 3603 1 2024-05-12T04:01:14.645 INFO:tasks.workunit.client.1.smithi033.stdout:4/498: chown dd/cf 161524 1 2024-05-12T04:01:14.646 INFO:tasks.workunit.client.0.smithi032.stdout:2/326: rmdir d1/d34 39 2024-05-12T04:01:14.650 INFO:tasks.workunit.client.1.smithi033.stdout:9/716: truncate d2/d4/d2b/d6d/fad 201001 0 2024-05-12T04:01:14.656 INFO:tasks.workunit.client.0.smithi032.stdout:5/380: mknod d4/d7/d14/d26/d33/d4a/c89 0 2024-05-12T04:01:14.657 INFO:tasks.workunit.client.1.smithi033.stdout:5/416: truncate d3/d12/f43 3443401 0 2024-05-12T04:01:14.660 INFO:tasks.workunit.client.1.smithi033.stdout:2/491: dread dc/d17/d27/d2f/f6a [0,4194304] 0 2024-05-12T04:01:14.663 INFO:tasks.workunit.client.0.smithi032.stdout:7/303: rename dd/f22 to dd/d1b/d43/d5e/f67 0 2024-05-12T04:01:14.664 INFO:tasks.workunit.client.1.smithi033.stdout:6/487: mknod d4/d1a/d4e/c9f 0 2024-05-12T04:01:14.666 INFO:tasks.workunit.client.0.smithi032.stdout:1/336: unlink l0 0 2024-05-12T04:01:14.668 INFO:tasks.workunit.client.0.smithi032.stdout:2/327: fsync d1/d13/f48 0 2024-05-12T04:01:14.671 INFO:tasks.workunit.client.0.smithi032.stdout:6/292: mkdir d7/de/d12/d63/d64 0 2024-05-12T04:01:14.674 INFO:tasks.workunit.client.1.smithi033.stdout:4/499: rename f3 to dd/d7a/f9f 0 2024-05-12T04:01:14.675 INFO:tasks.workunit.client.0.smithi032.stdout:0/372: dwrite df/d10/d48/d70/f76 [0,4194304] 0 2024-05-12T04:01:14.676 INFO:tasks.workunit.client.0.smithi032.stdout:2/328: dread d1/d13/d3d/d41/f2a [0,4194304] 0 2024-05-12T04:01:14.678 INFO:tasks.workunit.client.1.smithi033.stdout:8/413: link d1/d16/c18 d1/d16/d20/c7f 0 2024-05-12T04:01:14.678 INFO:tasks.workunit.client.0.smithi032.stdout:9/247: mknod d0/d3/d4b/c52 0 2024-05-12T04:01:14.679 INFO:tasks.workunit.client.0.smithi032.stdout:2/329: write d1/d13/d3d/d41/f3a [1937363,118243] 0 2024-05-12T04:01:14.680 INFO:tasks.workunit.client.0.smithi032.stdout:5/381: chown d4/d7/d14/d26/d3c/l3f 1168 1 2024-05-12T04:01:14.681 INFO:tasks.workunit.client.0.smithi032.stdout:5/382: chown d4/d7/d14/d26/d33/d4a/f51 38303 1 2024-05-12T04:01:14.681 INFO:tasks.workunit.client.0.smithi032.stdout:8/381: mknod d0/d9/d13/d26/c86 0 2024-05-12T04:01:14.682 INFO:tasks.workunit.client.0.smithi032.stdout:8/382: rename d0/d9/d1b/d33 to d0/d9/d1b/d33/d87 22 2024-05-12T04:01:14.683 INFO:tasks.workunit.client.1.smithi033.stdout:3/473: write db/d17/d25/f39 [4719846,64007] 0 2024-05-12T04:01:14.684 INFO:tasks.workunit.client.1.smithi033.stdout:5/417: read d3/d12/f1f [2071032,43799] 0 2024-05-12T04:01:14.711 INFO:tasks.workunit.client.1.smithi033.stdout:7/644: getdents de/df/d7a 0 2024-05-12T04:01:14.714 INFO:tasks.workunit.client.0.smithi032.stdout:7/304: creat dd/d1b/d1e/f68 x:0 0 0 2024-05-12T04:01:14.715 INFO:tasks.workunit.client.0.smithi032.stdout:7/305: stat dd/d1b/c3f 0 2024-05-12T04:01:14.716 INFO:tasks.workunit.client.0.smithi032.stdout:7/306: chown dd/d1b/c3f 699 1 2024-05-12T04:01:14.716 INFO:tasks.workunit.client.0.smithi032.stdout:1/337: rmdir d1/d7/d49/d67 39 2024-05-12T04:01:14.720 INFO:tasks.workunit.client.1.smithi033.stdout:6/488: truncate d4/d1a/d4e/f66 992997 0 2024-05-12T04:01:14.730 INFO:tasks.workunit.client.1.smithi033.stdout:9/717: rename d2/d4/d29/d3c/l63 to d2/dd/d39/d72/l103 0 2024-05-12T04:01:14.731 INFO:tasks.workunit.client.1.smithi033.stdout:9/718: chown d2/d4/d29/d3c/d52/fba 15 1 2024-05-12T04:01:14.733 INFO:tasks.workunit.client.1.smithi033.stdout:8/414: truncate d1/d4/d1d/d36/d3c/d47/f60 479868 0 2024-05-12T04:01:14.735 INFO:tasks.workunit.client.0.smithi032.stdout:2/330: truncate d1/d13/f48 3718350 0 2024-05-12T04:01:14.738 INFO:tasks.workunit.client.0.smithi032.stdout:5/383: read d4/f49 [387803,35509] 0 2024-05-12T04:01:14.739 INFO:tasks.workunit.client.0.smithi032.stdout:7/307: creat dd/d1b/d43/d5e/f69 x:0 0 0 2024-05-12T04:01:14.741 INFO:tasks.workunit.client.1.smithi033.stdout:5/418: fdatasync d3/d15/d16/f42 0 2024-05-12T04:01:14.745 INFO:tasks.workunit.client.0.smithi032.stdout:0/373: dwrite df/d10/d3d/d4a/f2e [0,4194304] 0 2024-05-12T04:01:14.750 INFO:tasks.workunit.client.1.smithi033.stdout:2/492: symlink dc/d17/d32/d84/la4 0 2024-05-12T04:01:14.752 INFO:tasks.workunit.client.0.smithi032.stdout:1/338: dwrite d1/d7/d49/d67/f63 [0,4194304] 0 2024-05-12T04:01:14.756 INFO:tasks.workunit.client.0.smithi032.stdout:6/293: creat d7/de/d12/d63/d64/f65 x:0 0 0 2024-05-12T04:01:14.758 INFO:tasks.workunit.client.1.smithi033.stdout:0/511: creat d6/dc/d58/fab x:0 0 0 2024-05-12T04:01:14.764 INFO:tasks.workunit.client.1.smithi033.stdout:4/500: fsync fc 0 2024-05-12T04:01:14.765 INFO:tasks.workunit.client.1.smithi033.stdout:4/501: dread - dd/d7a/d87/d7e/f8f zero size 2024-05-12T04:01:14.766 INFO:tasks.workunit.client.1.smithi033.stdout:4/502: fsync dd/d11/f21 0 2024-05-12T04:01:14.768 INFO:tasks.workunit.client.0.smithi032.stdout:2/331: mknod d1/d13/d3d/c72 0 2024-05-12T04:01:14.769 INFO:tasks.workunit.client.1.smithi033.stdout:9/719: mkdir d2/d4/d2b/d49/d104 0 2024-05-12T04:01:14.769 INFO:tasks.workunit.client.1.smithi033.stdout:2/493: dread dc/d17/d27/d68/d9f/f58 [0,4194304] 0 2024-05-12T04:01:14.770 INFO:tasks.workunit.client.1.smithi033.stdout:2/494: readlink dc/d17/d27/l70 0 2024-05-12T04:01:14.772 INFO:tasks.workunit.client.1.smithi033.stdout:8/415: mknod d1/c80 0 2024-05-12T04:01:14.775 INFO:tasks.workunit.client.1.smithi033.stdout:3/474: mknod db/d17/d3e/d65/d91/d97/ca8 0 2024-05-12T04:01:14.775 INFO:tasks.workunit.client.0.smithi032.stdout:7/308: dread dd/d1b/d1e/d45/d55/f5b [0,4194304] 0 2024-05-12T04:01:14.776 INFO:tasks.workunit.client.0.smithi032.stdout:7/309: stat c0 0 2024-05-12T04:01:14.782 INFO:tasks.workunit.client.0.smithi032.stdout:5/384: dwrite d4/fa [0,4194304] 0 2024-05-12T04:01:14.782 INFO:tasks.workunit.client.1.smithi033.stdout:7/645: mkdir de/d2b/d6c/de1 0 2024-05-12T04:01:14.783 INFO:tasks.workunit.client.0.smithi032.stdout:5/385: chown d4/d7/d14/d26/d33/d72/c81 0 1 2024-05-12T04:01:14.785 INFO:tasks.workunit.client.0.smithi032.stdout:7/310: dwrite dd/d1b/d43/d5e/f4c [0,4194304] 0 2024-05-12T04:01:14.789 INFO:tasks.workunit.client.0.smithi032.stdout:6/294: creat d7/f66 x:0 0 0 2024-05-12T04:01:14.792 INFO:tasks.workunit.client.1.smithi033.stdout:6/489: link d4/d1a/f91 d4/d1a/d4e/fa0 0 2024-05-12T04:01:14.794 INFO:tasks.workunit.client.0.smithi032.stdout:5/386: dwrite d4/d7/d14/d39/d1b/f76 [0,4194304] 0 2024-05-12T04:01:14.794 INFO:tasks.workunit.client.1.smithi033.stdout:0/512: chown d6/db/d2c/f31 11391482 1 2024-05-12T04:01:14.795 INFO:tasks.workunit.client.0.smithi032.stdout:5/387: chown d4/d7/d14/d39/c16 525923 1 2024-05-12T04:01:14.797 INFO:tasks.workunit.client.1.smithi033.stdout:5/419: dwrite d3/d12/d25/d26/d2e/f49 [0,4194304] 0 2024-05-12T04:01:14.798 INFO:tasks.workunit.client.0.smithi032.stdout:9/248: link d0/d3/d6/d7/da/f1c d0/d3/d6/d7/da/f53 0 2024-05-12T04:01:14.799 INFO:tasks.workunit.client.1.smithi033.stdout:5/420: read d3/d12/f3f [391337,59837] 0 2024-05-12T04:01:14.800 INFO:tasks.workunit.client.1.smithi033.stdout:4/503: chown dd/f32 3299 1 2024-05-12T04:01:14.804 INFO:tasks.workunit.client.1.smithi033.stdout:9/720: rmdir d2/d4/d2b/d6d 39 2024-05-12T04:01:14.804 INFO:tasks.workunit.client.1.smithi033.stdout:2/495: mknod dc/d13/d1c/d98/ca5 0 2024-05-12T04:01:14.805 INFO:tasks.workunit.client.1.smithi033.stdout:8/416: creat d1/f81 x:0 0 0 2024-05-12T04:01:14.806 INFO:tasks.workunit.client.0.smithi032.stdout:1/339: dread - d1/d7/d49/f4f zero size 2024-05-12T04:01:14.808 INFO:tasks.workunit.client.1.smithi033.stdout:1/450: getdents d11/d17/d38 0 2024-05-12T04:01:14.808 INFO:tasks.workunit.client.0.smithi032.stdout:1/340: readlink d1/d7/d49/l5c 0 2024-05-12T04:01:14.809 INFO:tasks.workunit.client.0.smithi032.stdout:1/341: write d1/d7/d30/f52 [832330,56722] 0 2024-05-12T04:01:14.818 INFO:tasks.workunit.client.1.smithi033.stdout:7/646: dread - de/d20/d38/fa2 zero size 2024-05-12T04:01:14.820 INFO:tasks.workunit.client.0.smithi032.stdout:0/374: creat df/d30/f8b x:0 0 0 2024-05-12T04:01:14.822 INFO:tasks.workunit.client.1.smithi033.stdout:6/490: rename d4/d1a/f83 to d4/d5/fa1 0 2024-05-12T04:01:14.823 INFO:tasks.workunit.client.0.smithi032.stdout:5/388: dwrite d4/d7/d14/d26/d47/f6d [0,4194304] 0 2024-05-12T04:01:14.823 INFO:tasks.workunit.client.1.smithi033.stdout:6/491: readlink d4/l11 0 2024-05-12T04:01:14.825 INFO:tasks.workunit.client.1.smithi033.stdout:0/513: rmdir d6/dc/d58 39 2024-05-12T04:01:14.827 INFO:tasks.workunit.client.1.smithi033.stdout:9/721: dread d2/d4/f2c [0,4194304] 0 2024-05-12T04:01:14.827 INFO:tasks.workunit.client.0.smithi032.stdout:2/332: dwrite d1/d13/d3d/d41/d52/f57 [0,4194304] 0 2024-05-12T04:01:14.828 INFO:tasks.workunit.client.1.smithi033.stdout:9/722: readlink d2/d70/l8f 0 2024-05-12T04:01:14.828 INFO:tasks.workunit.client.0.smithi032.stdout:2/333: stat d1/d13/f2b 0 2024-05-12T04:01:14.829 INFO:tasks.workunit.client.0.smithi032.stdout:7/311: rename dd/d1b/d1e/c57 to dd/d1b/d1e/d45/d55/c6a 0 2024-05-12T04:01:14.834 INFO:tasks.workunit.client.1.smithi033.stdout:4/504: fsync dd/d11/d31/d66/f49 0 2024-05-12T04:01:14.835 INFO:tasks.workunit.client.1.smithi033.stdout:4/505: chown dd/d11/f2f 70902 1 2024-05-12T04:01:14.843 INFO:tasks.workunit.client.0.smithi032.stdout:0/375: dread df/d10/f3c [0,4194304] 0 2024-05-12T04:01:14.844 INFO:tasks.workunit.client.1.smithi033.stdout:3/475: symlink db/d17/d25/d32/d3c/la9 0 2024-05-12T04:01:14.844 INFO:tasks.workunit.client.1.smithi033.stdout:3/476: dread - db/d17/d25/d32/d3c/d3d/f6f zero size 2024-05-12T04:01:14.845 INFO:tasks.workunit.client.1.smithi033.stdout:1/451: dread d11/d17/d27/d36/d3f/f50 [0,4194304] 0 2024-05-12T04:01:14.846 INFO:tasks.workunit.client.1.smithi033.stdout:1/452: write f8 [360105,91542] 0 2024-05-12T04:01:14.847 INFO:tasks.workunit.client.0.smithi032.stdout:6/295: rename d7/de/d12 to d7/d9/d48/d67 0 2024-05-12T04:01:14.848 INFO:tasks.workunit.client.1.smithi033.stdout:7/647: creat de/d20/d64/fe2 x:0 0 0 2024-05-12T04:01:14.849 INFO:tasks.workunit.client.0.smithi032.stdout:6/296: write d7/d9/d48/d67/d25/f29 [101555,127611] 0 2024-05-12T04:01:14.850 INFO:tasks.workunit.client.0.smithi032.stdout:1/342: symlink d1/d59/l75 0 2024-05-12T04:01:14.851 INFO:tasks.workunit.client.0.smithi032.stdout:7/312: creat dd/d1b/d1e/d45/d55/d60/f6b x:0 0 0 2024-05-12T04:01:14.851 INFO:tasks.workunit.client.0.smithi032.stdout:6/297: stat d7/de/d2e/c39 0 2024-05-12T04:01:14.853 INFO:tasks.workunit.client.0.smithi032.stdout:7/313: read - dd/d1b/d43/d5e/f5a zero size 2024-05-12T04:01:14.853 INFO:tasks.workunit.client.1.smithi033.stdout:6/492: symlink d4/d1a/d4e/d4f/d94/la2 0 2024-05-12T04:01:14.854 INFO:tasks.workunit.client.0.smithi032.stdout:5/389: creat d4/d7/d14/d26/d33/d4a/d38/d3b/d68/f8a x:0 0 0 2024-05-12T04:01:14.854 INFO:tasks.workunit.client.0.smithi032.stdout:3/352: sync 2024-05-12T04:01:14.855 INFO:tasks.workunit.client.0.smithi032.stdout:0/376: dread df/d10/f1b [0,4194304] 0 2024-05-12T04:01:14.855 INFO:tasks.workunit.client.0.smithi032.stdout:7/314: dread dd/d1b/d1e/d45/d55/f5b [0,4194304] 0 2024-05-12T04:01:14.857 INFO:tasks.workunit.client.0.smithi032.stdout:2/334: mkdir d1/d73 0 2024-05-12T04:01:14.858 INFO:tasks.workunit.client.1.smithi033.stdout:4/506: readlink dd/l6f 0 2024-05-12T04:01:14.861 INFO:tasks.workunit.client.1.smithi033.stdout:2/496: symlink dc/la6 0 2024-05-12T04:01:14.861 INFO:tasks.workunit.client.0.smithi032.stdout:6/298: dread d7/de/f14 [0,4194304] 0 2024-05-12T04:01:14.862 INFO:tasks.workunit.client.0.smithi032.stdout:0/377: dread df/d30/d4e/d55/f67 [0,4194304] 0 2024-05-12T04:01:14.869 INFO:tasks.workunit.client.0.smithi032.stdout:5/390: creat d4/d7/d14/d26/d33/f8b x:0 0 0 2024-05-12T04:01:14.870 INFO:tasks.workunit.client.1.smithi033.stdout:1/453: rmdir d11/d17/d27/d36/d3f/d4a 39 2024-05-12T04:01:14.873 INFO:tasks.workunit.client.1.smithi033.stdout:7/648: mkdir de/df/d65/de3 0 2024-05-12T04:01:14.878 INFO:tasks.workunit.client.0.smithi032.stdout:1/343: write d1/d7/d30/d35/d46/f4b [288123,45672] 0 2024-05-12T04:01:14.879 INFO:tasks.workunit.client.0.smithi032.stdout:2/335: rename d1/d13/d3d/d41/l4b to d1/d13/d6b/l74 0 2024-05-12T04:01:14.880 INFO:tasks.workunit.client.1.smithi033.stdout:8/417: truncate d1/d16/d27/d4d/d51/f4b 1591937 0 2024-05-12T04:01:14.881 INFO:tasks.workunit.client.1.smithi033.stdout:9/723: truncate d2/dd/f11 1091313 0 2024-05-12T04:01:14.883 INFO:tasks.workunit.client.1.smithi033.stdout:6/493: dread d4/f2f [0,4194304] 0 2024-05-12T04:01:14.885 INFO:tasks.workunit.client.0.smithi032.stdout:6/299: rmdir d7/d9/d48/d67/d25 39 2024-05-12T04:01:14.885 INFO:tasks.workunit.client.1.smithi033.stdout:0/514: write d6/dc/d58/f86 [721831,86576] 0 2024-05-12T04:01:14.888 INFO:tasks.workunit.client.1.smithi033.stdout:3/477: link db/d59/f5b db/d17/d4d/faa 0 2024-05-12T04:01:14.889 INFO:tasks.workunit.client.0.smithi032.stdout:7/315: mkdir dd/d6c 0 2024-05-12T04:01:14.889 INFO:tasks.workunit.client.1.smithi033.stdout:1/454: chown d11/d17/f35 3 1 2024-05-12T04:01:14.890 INFO:tasks.workunit.client.1.smithi033.stdout:7/649: symlink de/df/d17/d1e/dc4/le4 0 2024-05-12T04:01:14.891 INFO:tasks.workunit.client.0.smithi032.stdout:1/344: chown d1/dd/d24/c43 1514 1 2024-05-12T04:01:14.893 INFO:tasks.workunit.client.0.smithi032.stdout:1/345: truncate d1/d7/d49/f4f 742238 0 2024-05-12T04:01:14.895 INFO:tasks.workunit.client.0.smithi032.stdout:5/391: rename d4/d7/d14/d26/d33/d4a/d38/d3b/d68/f8a to d4/d7/d14/f8c 0 2024-05-12T04:01:14.895 INFO:tasks.workunit.client.0.smithi032.stdout:5/392: chown d4/l7d 370462524 1 2024-05-12T04:01:14.895 INFO:tasks.workunit.client.1.smithi033.stdout:7/650: dwrite de/df/d7a/f9f [0,4194304] 0 2024-05-12T04:01:14.899 INFO:tasks.workunit.client.0.smithi032.stdout:0/378: fdatasync df/d10/d3d/d4a/f2d 0 2024-05-12T04:01:14.901 INFO:tasks.workunit.client.0.smithi032.stdout:0/379: dread - df/d10/f80 zero size 2024-05-12T04:01:14.902 INFO:tasks.workunit.client.1.smithi033.stdout:6/494: rmdir d4 39 2024-05-12T04:01:14.906 INFO:tasks.workunit.client.0.smithi032.stdout:6/300: dread - d7/d9/d48/d67/d25/f61 zero size 2024-05-12T04:01:14.910 INFO:tasks.workunit.client.1.smithi033.stdout:2/497: mknod dc/d17/d27/ca7 0 2024-05-12T04:01:14.910 INFO:tasks.workunit.client.0.smithi032.stdout:7/316: fdatasync dd/d1b/d43/d5e/f34 0 2024-05-12T04:01:14.910 INFO:tasks.workunit.client.0.smithi032.stdout:0/380: dwrite df/d30/f8b [0,4194304] 0 2024-05-12T04:01:14.911 INFO:tasks.workunit.client.1.smithi033.stdout:2/498: truncate dc/d17/d3a/f9a 1709250 0 2024-05-12T04:01:14.912 INFO:tasks.workunit.client.0.smithi032.stdout:2/336: mknod d1/c75 0 2024-05-12T04:01:14.913 INFO:tasks.workunit.client.0.smithi032.stdout:6/301: write d7/de/d2e/f5a [999056,57252] 0 2024-05-12T04:01:14.913 INFO:tasks.workunit.client.1.smithi033.stdout:0/515: dread - d6/dc/d58/d6a/f7e zero size 2024-05-12T04:01:14.915 INFO:tasks.workunit.client.0.smithi032.stdout:6/302: chown d7/d9/d5c/c5e 0 1 2024-05-12T04:01:14.916 INFO:tasks.workunit.client.1.smithi033.stdout:3/478: fdatasync db/d17/d25/d32/d3c/f5a 0 2024-05-12T04:01:14.916 INFO:tasks.workunit.client.1.smithi033.stdout:8/418: write d1/d4/d46/f4f [1337784,126781] 0 2024-05-12T04:01:14.920 INFO:tasks.workunit.client.1.smithi033.stdout:1/455: rename d11/d17/d27/d36/d3f/d69/f90 to d11/d1c/d59/f97 0 2024-05-12T04:01:14.929 INFO:tasks.workunit.client.1.smithi033.stdout:0/516: dread d6/dc/f56 [0,4194304] 0 2024-05-12T04:01:14.932 INFO:tasks.workunit.client.1.smithi033.stdout:4/507: creat dd/d11/fa0 x:0 0 0 2024-05-12T04:01:14.933 INFO:tasks.workunit.client.1.smithi033.stdout:4/508: truncate dd/d11/f6d 859454 0 2024-05-12T04:01:14.934 INFO:tasks.workunit.client.1.smithi033.stdout:2/499: creat dc/d13/d65/fa8 x:0 0 0 2024-05-12T04:01:14.936 INFO:tasks.workunit.client.1.smithi033.stdout:0/517: dread d6/d11/d98/fa1 [0,4194304] 0 2024-05-12T04:01:14.936 INFO:tasks.workunit.client.1.smithi033.stdout:2/500: truncate dc/d17/d3a/f6d 8581829 0 2024-05-12T04:01:14.938 INFO:tasks.workunit.client.1.smithi033.stdout:3/479: creat db/d59/fab x:0 0 0 2024-05-12T04:01:14.939 INFO:tasks.workunit.client.0.smithi032.stdout:5/393: rename d4/d7/d14/d39/d1b/l6b to d4/d7/d14/d26/d33/d4a/d38/d79/l8d 0 2024-05-12T04:01:14.940 INFO:tasks.workunit.client.1.smithi033.stdout:7/651: write de/d2b/d6c/da0/fa8 [116208,30929] 0 2024-05-12T04:01:14.940 INFO:tasks.workunit.client.1.smithi033.stdout:8/419: mkdir d1/d16/d27/d4d/d82 0 2024-05-12T04:01:14.943 INFO:tasks.workunit.client.1.smithi033.stdout:9/724: dwrite d2/d4/d2b/d6d/f5c [4194304,4194304] 0 2024-05-12T04:01:14.943 INFO:tasks.workunit.client.0.smithi032.stdout:0/381: creat df/d10/d3d/d4a/d84/f8c x:0 0 0 2024-05-12T04:01:14.943 INFO:tasks.workunit.client.1.smithi033.stdout:1/456: chown d11/d17/d27/d36/d3f/d4a/c77 21345 1 2024-05-12T04:01:14.944 INFO:tasks.workunit.client.1.smithi033.stdout:7/652: truncate de/df/d17/d1e/d24/fc5 600524 0 2024-05-12T04:01:14.945 INFO:tasks.workunit.client.0.smithi032.stdout:7/317: unlink dd/d1b/d1e/f5d 0 2024-05-12T04:01:14.946 INFO:tasks.workunit.client.0.smithi032.stdout:6/303: creat d7/d9/d60/f68 x:0 0 0 2024-05-12T04:01:14.947 INFO:tasks.workunit.client.0.smithi032.stdout:5/394: rmdir d4/d7/d14/d39/d2b 39 2024-05-12T04:01:14.948 INFO:tasks.workunit.client.0.smithi032.stdout:5/395: chown d4/d7/d14/d39/l36 5 1 2024-05-12T04:01:14.952 INFO:tasks.workunit.client.0.smithi032.stdout:2/337: creat d1/d13/d3d/f76 x:0 0 0 2024-05-12T04:01:14.952 INFO:tasks.workunit.client.1.smithi033.stdout:4/509: dread dd/f13 [0,4194304] 0 2024-05-12T04:01:14.963 INFO:tasks.workunit.client.1.smithi033.stdout:9/725: fsync d2/d4/d29/f46 0 2024-05-12T04:01:14.967 INFO:tasks.workunit.client.1.smithi033.stdout:6/495: dwrite d4/d5/f5c [0,4194304] 0 2024-05-12T04:01:14.967 INFO:tasks.workunit.client.1.smithi033.stdout:4/510: write dd/d11/f46 [941572,87832] 0 2024-05-12T04:01:14.968 INFO:tasks.workunit.client.1.smithi033.stdout:6/496: stat d4/d1a/d67/d68/f8f 0 2024-05-12T04:01:14.969 INFO:tasks.workunit.client.1.smithi033.stdout:4/511: chown dd/d11/d31/d66/c41 25 1 2024-05-12T04:01:14.970 INFO:tasks.workunit.client.1.smithi033.stdout:3/480: mknod db/d17/d3a/d8a/d96/cac 0 2024-05-12T04:01:14.976 INFO:tasks.workunit.client.1.smithi033.stdout:7/653: dwrite de/df/d65/f86 [4194304,4194304] 0 2024-05-12T04:01:14.976 INFO:tasks.workunit.client.0.smithi032.stdout:5/396: unlink d4/d7/l24 0 2024-05-12T04:01:14.979 INFO:tasks.workunit.client.1.smithi033.stdout:1/457: creat d11/f98 x:0 0 0 2024-05-12T04:01:14.979 INFO:tasks.workunit.client.0.smithi032.stdout:0/382: mkdir df/d30/d8d 0 2024-05-12T04:01:14.980 INFO:tasks.workunit.client.1.smithi033.stdout:1/458: stat d11/d1c/c25 0 2024-05-12T04:01:14.982 INFO:tasks.workunit.client.1.smithi033.stdout:3/481: creat db/d17/d4d/fad x:0 0 0 2024-05-12T04:01:14.983 INFO:tasks.workunit.client.1.smithi033.stdout:3/482: write db/d17/d25/f40 [1798763,33598] 0 2024-05-12T04:01:14.984 INFO:tasks.workunit.client.1.smithi033.stdout:3/483: chown db/d17/d25/d32/f5d 24750 1 2024-05-12T04:01:14.985 INFO:tasks.workunit.client.1.smithi033.stdout:8/420: truncate d1/d4/d46/f4f 3135514 0 2024-05-12T04:01:14.986 INFO:tasks.workunit.client.1.smithi033.stdout:9/726: getdents d2/d4/d2b/d49/d104 0 2024-05-12T04:01:14.986 INFO:tasks.workunit.client.1.smithi033.stdout:8/421: chown d1/d16/d20/f2f 71929803 1 2024-05-12T04:01:14.987 INFO:tasks.workunit.client.1.smithi033.stdout:9/727: chown d2/d4/d29/d3c/dda 1027396 1 2024-05-12T04:01:14.987 INFO:tasks.workunit.client.1.smithi033.stdout:8/422: write d1/d4/d1d/f78 [390020,8233] 0 2024-05-12T04:01:14.989 INFO:tasks.workunit.client.0.smithi032.stdout:2/338: mknod d1/d13/d3d/d41/d39/c77 0 2024-05-12T04:01:14.990 INFO:tasks.workunit.client.1.smithi033.stdout:0/518: getdents d6/db/d2c/d73 0 2024-05-12T04:01:14.990 INFO:tasks.workunit.client.1.smithi033.stdout:6/497: mknod d4/d8d/ca3 0 2024-05-12T04:01:14.992 INFO:tasks.workunit.client.1.smithi033.stdout:6/498: readlink d4/d1a/l1b 0 2024-05-12T04:01:14.992 INFO:tasks.workunit.client.1.smithi033.stdout:9/728: truncate d2/d4/d2b/d6d/d2f/fd8 716136 0 2024-05-12T04:01:14.993 INFO:tasks.workunit.client.0.smithi032.stdout:6/304: dwrite d7/d9/d48/d67/d25/f29 [0,4194304] 0 2024-05-12T04:01:14.995 INFO:tasks.workunit.client.1.smithi033.stdout:4/512: dwrite dd/f2a [0,4194304] 0 2024-05-12T04:01:14.996 INFO:tasks.workunit.client.1.smithi033.stdout:4/513: chown dd/d11/l88 10020664 1 2024-05-12T04:01:14.997 INFO:tasks.workunit.client.0.smithi032.stdout:7/318: dwrite fa [0,4194304] 0 2024-05-12T04:01:15.004 INFO:tasks.workunit.client.0.smithi032.stdout:5/397: dwrite d4/d7/d14/d26/d33/d4a/d38/f74 [0,4194304] 0 2024-05-12T04:01:15.005 INFO:tasks.workunit.client.1.smithi033.stdout:9/729: dwrite d2/d4/d2b/d6d/d4c/fe0 [0,4194304] 0 2024-05-12T04:01:15.005 INFO:tasks.workunit.client.0.smithi032.stdout:7/319: write dd/d1b/d43/d5e/f4c [923221,31939] 0 2024-05-12T04:01:15.012 INFO:tasks.workunit.client.0.smithi032.stdout:7/320: write dd/d1b/d1e/f28 [1833483,82034] 0 2024-05-12T04:01:15.013 INFO:tasks.workunit.client.1.smithi033.stdout:1/459: rmdir d11/d17/d27/d36/d3f 39 2024-05-12T04:01:15.013 INFO:tasks.workunit.client.1.smithi033.stdout:3/484: symlink db/d17/d3e/d65/d7d/lae 0 2024-05-12T04:01:15.015 INFO:tasks.workunit.client.0.smithi032.stdout:6/305: symlink d7/d9/d48/d67/d63/l69 0 2024-05-12T04:01:15.023 INFO:tasks.workunit.client.0.smithi032.stdout:0/383: write df/d10/d48/f52 [1082324,121555] 0 2024-05-12T04:01:15.028 INFO:tasks.workunit.client.1.smithi033.stdout:4/514: write dd/f35 [4107636,131066] 0 2024-05-12T04:01:15.030 INFO:tasks.workunit.client.1.smithi033.stdout:4/515: read - dd/d11/d31/d66/f7f zero size 2024-05-12T04:01:15.030 INFO:tasks.workunit.client.0.smithi032.stdout:5/398: dwrite d4/f59 [0,4194304] 0 2024-05-12T04:01:15.030 INFO:tasks.workunit.client.1.smithi033.stdout:4/516: chown dd/d11 339 1 2024-05-12T04:01:15.036 INFO:tasks.workunit.client.0.smithi032.stdout:6/306: dwrite d7/d9/d60/f68 [0,4194304] 0 2024-05-12T04:01:15.039 INFO:tasks.workunit.client.1.smithi033.stdout:3/485: symlink db/d42/laf 0 2024-05-12T04:01:15.039 INFO:tasks.workunit.client.1.smithi033.stdout:3/486: dread - db/d17/d4d/fad zero size 2024-05-12T04:01:15.042 INFO:tasks.workunit.client.1.smithi033.stdout:1/460: mknod d11/d17/d27/d76/c99 0 2024-05-12T04:01:15.043 INFO:tasks.workunit.client.0.smithi032.stdout:2/339: dwrite d1/d13/d3d/f60 [0,4194304] 0 2024-05-12T04:01:15.046 INFO:tasks.workunit.client.0.smithi032.stdout:2/340: chown d1/f12 49628974 1 2024-05-12T04:01:15.049 INFO:tasks.workunit.client.0.smithi032.stdout:0/384: dwrite df/f51 [0,4194304] 0 2024-05-12T04:01:15.049 INFO:tasks.workunit.client.1.smithi033.stdout:6/499: write d4/d5/f76 [863039,119959] 0 2024-05-12T04:01:15.055 INFO:tasks.workunit.client.0.smithi032.stdout:6/307: dwrite d7/d9/d48/d67/f4d [0,4194304] 0 2024-05-12T04:01:15.060 INFO:tasks.workunit.client.0.smithi032.stdout:0/385: dwrite df/f51 [0,4194304] 0 2024-05-12T04:01:15.062 INFO:tasks.workunit.client.0.smithi032.stdout:7/321: write dd/f42 [340659,24331] 0 2024-05-12T04:01:15.064 INFO:tasks.workunit.client.0.smithi032.stdout:2/341: symlink d1/d13/d6b/l78 0 2024-05-12T04:01:15.067 INFO:tasks.workunit.client.0.smithi032.stdout:6/308: symlink d7/d9/d48/l6a 0 2024-05-12T04:01:15.068 INFO:tasks.workunit.client.1.smithi033.stdout:3/487: rename db/d17/d3a/d8a/d96 to db/d17/d3a/d67/db0 0 2024-05-12T04:01:15.070 INFO:tasks.workunit.client.0.smithi032.stdout:0/386: mkdir df/d10/d3d/d4a/d8e 0 2024-05-12T04:01:15.070 INFO:tasks.workunit.client.0.smithi032.stdout:5/399: read d4/d7/d14/d26/d33/f61 [4104172,25985] 0 2024-05-12T04:01:15.076 INFO:tasks.workunit.client.0.smithi032.stdout:6/309: dwrite d7/d9/f4a [0,4194304] 0 2024-05-12T04:01:15.077 INFO:tasks.workunit.client.0.smithi032.stdout:2/342: chown d1/d34/d3e/d67/f6a 514316890 1 2024-05-12T04:01:15.077 INFO:tasks.workunit.client.0.smithi032.stdout:0/387: read df/d10/d3d/d4a/f2d [28103,31866] 0 2024-05-12T04:01:15.078 INFO:tasks.workunit.client.1.smithi033.stdout:0/519: creat d6/fac x:0 0 0 2024-05-12T04:01:15.078 INFO:tasks.workunit.client.1.smithi033.stdout:8/423: dwrite d1/f2 [0,4194304] 0 2024-05-12T04:01:15.080 INFO:tasks.workunit.client.1.smithi033.stdout:0/520: truncate d6/d11/d57/f63 1126071 0 2024-05-12T04:01:15.080 INFO:tasks.workunit.client.1.smithi033.stdout:6/500: truncate d4/d43/f70 3385275 0 2024-05-12T04:01:15.086 INFO:tasks.workunit.client.1.smithi033.stdout:7/654: getdents de/d2b/d6c/db5 0 2024-05-12T04:01:15.087 INFO:tasks.workunit.client.1.smithi033.stdout:7/655: readlink de/df/d17/l33 0 2024-05-12T04:01:15.089 INFO:tasks.workunit.client.1.smithi033.stdout:9/730: creat d2/d4/d2b/f105 x:0 0 0 2024-05-12T04:01:15.093 INFO:tasks.workunit.client.0.smithi032.stdout:5/400: dwrite d4/d7/d14/f8c [0,4194304] 0 2024-05-12T04:01:15.097 INFO:tasks.workunit.client.0.smithi032.stdout:7/322: creat dd/d1b/d1e/d45/d64/f6d x:0 0 0 2024-05-12T04:01:15.101 INFO:tasks.workunit.client.1.smithi033.stdout:0/521: symlink d6/dc/d66/da4/lad 0 2024-05-12T04:01:15.103 INFO:tasks.workunit.client.1.smithi033.stdout:7/656: mknod de/d2f/d70/ce5 0 2024-05-12T04:01:15.106 INFO:tasks.workunit.client.1.smithi033.stdout:7/657: write de/df/d60/fc3 [259022,6306] 0 2024-05-12T04:01:15.106 INFO:tasks.workunit.client.1.smithi033.stdout:9/731: dread d2/d4/d2b/d6d/d2f/f38 [0,4194304] 0 2024-05-12T04:01:15.106 INFO:tasks.workunit.client.1.smithi033.stdout:7/658: fsync de/df/d17/d1e/d24/fc5 0 2024-05-12T04:01:15.106 INFO:tasks.workunit.client.1.smithi033.stdout:3/488: creat db/d17/d3a/d67/d76/da0/fb1 x:0 0 0 2024-05-12T04:01:15.107 INFO:tasks.workunit.client.0.smithi032.stdout:7/323: rename dd/l2c to dd/d1b/d43/l6e 0 2024-05-12T04:01:15.107 INFO:tasks.workunit.client.0.smithi032.stdout:5/401: dwrite d4/fa [0,4194304] 0 2024-05-12T04:01:15.111 INFO:tasks.workunit.client.1.smithi033.stdout:4/517: getdents dd/d7a/d87 0 2024-05-12T04:01:15.112 INFO:tasks.workunit.client.1.smithi033.stdout:1/461: write d11/d17/d27/d36/d3f/d4a/f7a [1428512,47157] 0 2024-05-12T04:01:15.113 INFO:tasks.workunit.client.0.smithi032.stdout:0/388: write df/d10/d3d/d4a/f2d [59743,118990] 0 2024-05-12T04:01:15.116 INFO:tasks.workunit.client.1.smithi033.stdout:0/522: mknod d6/dc/d66/da4/cae 0 2024-05-12T04:01:15.121 INFO:tasks.workunit.client.1.smithi033.stdout:6/501: dwrite d4/d5/f27 [0,4194304] 0 2024-05-12T04:01:15.121 INFO:tasks.workunit.client.0.smithi032.stdout:0/389: rmdir df/d30/d32/d60 39 2024-05-12T04:01:15.121 INFO:tasks.workunit.client.1.smithi033.stdout:5/421: sync 2024-05-12T04:01:15.122 INFO:tasks.workunit.client.0.smithi032.stdout:5/402: symlink d4/d7/d14/d39/d2b/l8e 0 2024-05-12T04:01:15.124 INFO:tasks.workunit.client.1.smithi033.stdout:7/659: unlink de/df/f48 0 2024-05-12T04:01:15.128 INFO:tasks.workunit.client.0.smithi032.stdout:7/324: dread dd/d1b/f2d [0,4194304] 0 2024-05-12T04:01:15.131 INFO:tasks.workunit.client.0.smithi032.stdout:5/403: chown d4/d7/d14/d39/l55 653 1 2024-05-12T04:01:15.132 INFO:tasks.workunit.client.1.smithi033.stdout:8/424: link d1/d16/c71 d1/d4/d1d/d36/d3c/d47/c83 0 2024-05-12T04:01:15.133 INFO:tasks.workunit.client.0.smithi032.stdout:7/325: dread - dd/d1b/d1e/d45/d55/d60/f62 zero size 2024-05-12T04:01:15.133 INFO:tasks.workunit.client.1.smithi033.stdout:6/502: creat d4/d1a/d6c/fa4 x:0 0 0 2024-05-12T04:01:15.134 INFO:tasks.workunit.client.0.smithi032.stdout:6/310: dread d7/fa [0,4194304] 0 2024-05-12T04:01:15.134 INFO:tasks.workunit.client.1.smithi033.stdout:5/422: mkdir d3/d12/d25/d26/d2e/d85 0 2024-05-12T04:01:15.134 INFO:tasks.workunit.client.1.smithi033.stdout:6/503: fdatasync d4/f9a 0 2024-05-12T04:01:15.134 INFO:tasks.workunit.client.1.smithi033.stdout:6/504: readlink d4/d1a/d4e/d4f/l84 0 2024-05-12T04:01:15.135 INFO:tasks.workunit.client.0.smithi032.stdout:7/326: write dd/d1b/d1e/d45/d55/d60/f62 [69195,42020] 0 2024-05-12T04:01:15.138 INFO:tasks.workunit.client.1.smithi033.stdout:3/489: link db/d17/f64 db/d17/d3a/d67/d76/fb2 0 2024-05-12T04:01:15.140 INFO:tasks.workunit.client.1.smithi033.stdout:5/423: dwrite d3/d12/d25/d26/d2e/f3e [0,4194304] 0 2024-05-12T04:01:15.141 INFO:tasks.workunit.client.1.smithi033.stdout:1/462: symlink d11/d17/d27/d36/d3f/d69/d87/d96/l9a 0 2024-05-12T04:01:15.143 INFO:tasks.workunit.client.0.smithi032.stdout:5/404: mknod d4/d7/d58/c8f 0 2024-05-12T04:01:15.146 INFO:tasks.workunit.client.0.smithi032.stdout:7/327: rename dd/d1b/f5f to dd/d1b/d43/d5e/f6f 0 2024-05-12T04:01:15.148 INFO:tasks.workunit.client.1.smithi033.stdout:9/732: dread d2/d70/d7b/d94/ff5 [0,4194304] 0 2024-05-12T04:01:15.150 INFO:tasks.workunit.client.1.smithi033.stdout:3/490: dwrite db/d17/d3a/d67/d76/da0/fb1 [0,4194304] 0 2024-05-12T04:01:15.152 INFO:tasks.workunit.client.0.smithi032.stdout:0/390: write df/d10/d3b/f66 [783920,123086] 0 2024-05-12T04:01:15.153 INFO:tasks.workunit.client.1.smithi033.stdout:5/424: write d3/f9 [4402397,73140] 0 2024-05-12T04:01:15.154 INFO:tasks.workunit.client.1.smithi033.stdout:8/425: dwrite d1/d16/d27/d28/f57 [0,4194304] 0 2024-05-12T04:01:15.155 INFO:tasks.workunit.client.0.smithi032.stdout:7/328: dwrite dd/d1b/d43/d5e/d37/f39 [0,4194304] 0 2024-05-12T04:01:15.156 INFO:tasks.workunit.client.1.smithi033.stdout:5/425: chown d3/d27/l1b 153 1 2024-05-12T04:01:15.156 INFO:tasks.workunit.client.1.smithi033.stdout:4/518: rename dd/c9b to dd/d11/d31/ca1 0 2024-05-12T04:01:15.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:14 smithi033 ceph-mon[49525]: pgmap v35: 65 pgs: 65 active+clean; 1.5 GiB data, 5.9 GiB used, 531 GiB / 536 GiB avail; 34 MiB/s rd, 188 MiB/s wr, 318 op/s 2024-05-12T04:01:15.162 INFO:tasks.workunit.client.0.smithi032.stdout:6/311: getdents d7/de 0 2024-05-12T04:01:15.163 INFO:tasks.workunit.client.1.smithi033.stdout:8/426: dwrite d1/d16/d20/f2f [4194304,4194304] 0 2024-05-12T04:01:15.166 INFO:tasks.workunit.client.1.smithi033.stdout:9/733: creat d2/d70/f106 x:0 0 0 2024-05-12T04:01:15.169 INFO:tasks.workunit.client.0.smithi032.stdout:7/329: dwrite dd/d1b/d43/d5e/f1c [0,4194304] 0 2024-05-12T04:01:15.171 INFO:tasks.workunit.client.1.smithi033.stdout:3/491: creat db/d17/d3e/d65/d91/fb3 x:0 0 0 2024-05-12T04:01:15.172 INFO:tasks.workunit.client.1.smithi033.stdout:8/427: dwrite d1/d4/d1d/d36/f50 [0,4194304] 0 2024-05-12T04:01:15.173 INFO:tasks.workunit.client.1.smithi033.stdout:8/428: write d1/d16/d27/d4d/d51/f31 [381326,27132] 0 2024-05-12T04:01:15.180 INFO:tasks.workunit.client.1.smithi033.stdout:9/734: dread d2/d4/d2b/d6d/d3a/f65 [0,4194304] 0 2024-05-12T04:01:15.181 INFO:tasks.workunit.client.0.smithi032.stdout:6/312: creat d7/d9/d5c/f6b x:0 0 0 2024-05-12T04:01:15.186 INFO:tasks.workunit.client.1.smithi033.stdout:5/426: write d3/d15/d16/f44 [1595052,2008] 0 2024-05-12T04:01:15.188 INFO:tasks.workunit.client.0.smithi032.stdout:4/236: sync 2024-05-12T04:01:15.188 INFO:tasks.workunit.client.0.smithi032.stdout:5/405: link d4/d7/d14/d26/d33/d4a/d38/d79/l46 d4/d7/d14/d39/l90 0 2024-05-12T04:01:15.190 INFO:tasks.workunit.client.0.smithi032.stdout:7/330: dwrite f6 [0,4194304] 0 2024-05-12T04:01:15.191 INFO:tasks.workunit.client.1.smithi033.stdout:2/501: sync 2024-05-12T04:01:15.192 INFO:tasks.workunit.client.0.smithi032.stdout:4/237: write d0/d3/d12/d1d/f2a [3765426,128976] 0 2024-05-12T04:01:15.193 INFO:tasks.workunit.client.0.smithi032.stdout:5/406: chown d4/d7/d14/d26/d33/d4a/d38/d79 452273057 1 2024-05-12T04:01:15.197 INFO:tasks.workunit.client.0.smithi032.stdout:5/407: write d4/d7/d14/d39/d1b/f23 [4553168,32277] 0 2024-05-12T04:01:15.198 INFO:tasks.workunit.client.0.smithi032.stdout:5/408: readlink d4/d7/d14/d26/d33/l42 0 2024-05-12T04:01:15.199 INFO:tasks.workunit.client.1.smithi033.stdout:0/523: rename d6/dc/d66 to d6/db/d2c/d73/daf 0 2024-05-12T04:01:15.199 INFO:tasks.workunit.client.0.smithi032.stdout:6/313: creat d7/d9/f6c x:0 0 0 2024-05-12T04:01:15.199 INFO:tasks.workunit.client.0.smithi032.stdout:4/238: write d0/d3/d12/d1d/f2a [3825298,6577] 0 2024-05-12T04:01:15.201 INFO:tasks.workunit.client.1.smithi033.stdout:6/505: creat d4/d1a/d4e/d4f/fa5 x:0 0 0 2024-05-12T04:01:15.203 INFO:tasks.workunit.client.1.smithi033.stdout:4/519: dread dd/f4f [0,4194304] 0 2024-05-12T04:01:15.204 INFO:tasks.workunit.client.0.smithi032.stdout:5/409: dread - d4/d7/d14/d26/d3c/f6f zero size 2024-05-12T04:01:15.204 INFO:tasks.workunit.client.1.smithi033.stdout:3/492: symlink db/d17/d3a/d8a/d9d/lb4 0 2024-05-12T04:01:15.204 INFO:tasks.workunit.client.0.smithi032.stdout:4/239: write d0/d3/d31/f38 [452178,92841] 0 2024-05-12T04:01:15.206 INFO:tasks.workunit.client.0.smithi032.stdout:4/240: chown d0/l17 2788 1 2024-05-12T04:01:15.206 INFO:tasks.workunit.client.0.smithi032.stdout:6/314: fdatasync d7/d9/d48/f5d 0 2024-05-12T04:01:15.209 INFO:tasks.workunit.client.1.smithi033.stdout:8/429: rmdir d1/d4/d1d/d36 39 2024-05-12T04:01:15.212 INFO:tasks.workunit.client.0.smithi032.stdout:0/391: write df/d10/d3d/d4a/f38 [1983839,127679] 0 2024-05-12T04:01:15.212 INFO:tasks.workunit.client.0.smithi032.stdout:7/331: rename dd/d1b/d1e/d45/d55/c6a to dd/d1b/d43/c70 0 2024-05-12T04:01:15.214 INFO:tasks.workunit.client.1.smithi033.stdout:1/463: creat d11/d17/f9b x:0 0 0 2024-05-12T04:01:15.215 INFO:tasks.workunit.client.0.smithi032.stdout:4/241: dread d0/d3/df/f26 [0,4194304] 0 2024-05-12T04:01:15.215 INFO:tasks.workunit.client.0.smithi032.stdout:5/410: symlink d4/d7/d14/d26/d33/d72/l91 0 2024-05-12T04:01:15.217 INFO:tasks.workunit.client.1.smithi033.stdout:9/735: mkdir d2/d4/d2b/d6d/d2f/d5f/d84/d107 0 2024-05-12T04:01:15.220 INFO:tasks.workunit.client.1.smithi033.stdout:5/427: fdatasync d3/d12/d29/d2b/f46 0 2024-05-12T04:01:15.223 INFO:tasks.workunit.client.1.smithi033.stdout:9/736: dwrite d2/d4/d2b/d6d/d2f/fd8 [0,4194304] 0 2024-05-12T04:01:15.224 INFO:tasks.workunit.client.1.smithi033.stdout:9/737: readlink d2/l51 0 2024-05-12T04:01:15.227 INFO:tasks.workunit.client.1.smithi033.stdout:0/524: truncate d6/db/d2c/f48 276513 0 2024-05-12T04:01:15.244 INFO:tasks.workunit.client.1.smithi033.stdout:5/428: read d3/f2c [257859,46533] 0 2024-05-12T04:01:15.245 INFO:tasks.workunit.client.1.smithi033.stdout:2/502: dwrite dc/d17/f23 [0,4194304] 0 2024-05-12T04:01:15.246 INFO:tasks.workunit.client.1.smithi033.stdout:2/503: chown dc/d17/d27/d2f/d49/d59/c7e 761705 1 2024-05-12T04:01:15.246 INFO:tasks.workunit.client.0.smithi032.stdout:6/315: unlink d7/de/l44 0 2024-05-12T04:01:15.247 INFO:tasks.workunit.client.1.smithi033.stdout:5/429: read d3/d15/d16/f20 [8461699,101593] 0 2024-05-12T04:01:15.252 INFO:tasks.workunit.client.1.smithi033.stdout:7/660: rename de/df/d60/d75/ca4 to de/df/d17/d1e/dc4/ce6 0 2024-05-12T04:01:15.252 INFO:tasks.workunit.client.1.smithi033.stdout:4/520: dwrite dd/f13 [0,4194304] 0 2024-05-12T04:01:15.254 INFO:tasks.workunit.client.1.smithi033.stdout:7/661: readlink de/df/l62 0 2024-05-12T04:01:15.255 INFO:tasks.workunit.client.1.smithi033.stdout:3/493: dwrite db/d59/f8d [4194304,4194304] 0 2024-05-12T04:01:15.256 INFO:tasks.workunit.client.0.smithi032.stdout:6/316: dwrite d7/d9/d5c/f6b [0,4194304] 0 2024-05-12T04:01:15.257 INFO:tasks.workunit.client.1.smithi033.stdout:7/662: write de/df/d65/f86 [7578691,51311] 0 2024-05-12T04:01:15.258 INFO:tasks.workunit.client.0.smithi032.stdout:5/411: mkdir d4/d7/d14/d26/d33/d72/d92 0 2024-05-12T04:01:15.262 INFO:tasks.workunit.client.1.smithi033.stdout:0/525: creat d6/db/fb0 x:0 0 0 2024-05-12T04:01:15.267 INFO:tasks.workunit.client.1.smithi033.stdout:3/494: dwrite db/d17/d4d/fad [0,4194304] 0 2024-05-12T04:01:15.268 INFO:tasks.workunit.client.0.smithi032.stdout:4/242: fsync d0/d3/fd 0 2024-05-12T04:01:15.269 INFO:tasks.workunit.client.1.smithi033.stdout:8/430: symlink d1/d16/d27/d4d/d82/l84 0 2024-05-12T04:01:15.270 INFO:tasks.workunit.client.0.smithi032.stdout:5/412: mknod d4/d7/d14/d26/d33/d5f/d7f/c93 0 2024-05-12T04:01:15.271 INFO:tasks.workunit.client.0.smithi032.stdout:5/413: readlink d4/d7/d14/d39/d2b/l75 0 2024-05-12T04:01:15.277 INFO:tasks.workunit.client.0.smithi032.stdout:4/243: unlink d0/l17 0 2024-05-12T04:01:15.277 INFO:tasks.workunit.client.0.smithi032.stdout:4/244: readlink d0/d3/df/l27 0 2024-05-12T04:01:15.281 INFO:tasks.workunit.client.0.smithi032.stdout:7/332: getdents dd/d1b/d1e/d45/d55/d60 0 2024-05-12T04:01:15.283 INFO:tasks.workunit.client.0.smithi032.stdout:5/414: symlink d4/d7/d14/d26/l94 0 2024-05-12T04:01:15.286 INFO:tasks.workunit.client.1.smithi033.stdout:9/738: dwrite d2/d4/d2b/d6d/d3a/f5a [8388608,4194304] 0 2024-05-12T04:01:15.287 INFO:tasks.workunit.client.1.smithi033.stdout:9/739: stat d2/d4/d29/d3c/d52/fba 0 2024-05-12T04:01:15.288 INFO:tasks.workunit.client.1.smithi033.stdout:1/464: write d11/d17/d27/d36/d3f/f61 [3040534,89299] 0 2024-05-12T04:01:15.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:14 smithi032 ceph-mon[40008]: pgmap v35: 65 pgs: 65 active+clean; 1.5 GiB data, 5.9 GiB used, 531 GiB / 536 GiB avail; 34 MiB/s rd, 188 MiB/s wr, 318 op/s 2024-05-12T04:01:15.290 INFO:tasks.workunit.client.0.smithi032.stdout:7/333: mknod dd/d1b/d43/c71 0 2024-05-12T04:01:15.293 INFO:tasks.workunit.client.0.smithi032.stdout:5/415: symlink d4/d7/d14/d26/l95 0 2024-05-12T04:01:15.295 INFO:tasks.workunit.client.0.smithi032.stdout:5/416: write d4/d7/d14/f1d [3016545,126421] 0 2024-05-12T04:01:15.300 INFO:tasks.workunit.client.1.smithi033.stdout:7/663: rename de/df/d17/d1b/d50/c7c to de/d2b/d6c/dbd/ce7 0 2024-05-12T04:01:15.301 INFO:tasks.workunit.client.0.smithi032.stdout:7/334: mkdir dd/d1b/d1e/d45/d55/d72 0 2024-05-12T04:01:15.304 INFO:tasks.workunit.client.0.smithi032.stdout:5/417: dwrite d4/d7/d14/d26/d33/d4a/f4e [0,4194304] 0 2024-05-12T04:01:15.304 INFO:tasks.workunit.client.0.smithi032.stdout:4/245: creat d0/f3d x:0 0 0 2024-05-12T04:01:15.306 INFO:tasks.workunit.client.1.smithi033.stdout:0/526: creat d6/dc/d96/fb1 x:0 0 0 2024-05-12T04:01:15.307 INFO:tasks.workunit.client.1.smithi033.stdout:0/527: chown d6/dc/d39/l69 5 1 2024-05-12T04:01:15.314 INFO:tasks.workunit.client.1.smithi033.stdout:3/495: creat db/d17/d3a/fb5 x:0 0 0 2024-05-12T04:01:15.316 INFO:tasks.workunit.client.0.smithi032.stdout:6/317: rename d7/d9/d5c/f6b to d7/de/f6d 0 2024-05-12T04:01:15.318 INFO:tasks.workunit.client.0.smithi032.stdout:7/335: unlink dd/f42 0 2024-05-12T04:01:15.326 INFO:tasks.workunit.client.0.smithi032.stdout:5/418: write d4/f20 [102458,27084] 0 2024-05-12T04:01:15.328 INFO:tasks.workunit.client.1.smithi033.stdout:5/430: symlink d3/d15/d16/d45/d62/d7f/d84/l86 0 2024-05-12T04:01:15.329 INFO:tasks.workunit.client.1.smithi033.stdout:5/431: stat d3/d12/l71 0 2024-05-12T04:01:15.330 INFO:tasks.workunit.client.1.smithi033.stdout:9/740: symlink d2/dd/d39/d72/l108 0 2024-05-12T04:01:15.333 INFO:tasks.workunit.client.1.smithi033.stdout:1/465: unlink d11/d17/d27/d36/f3c 0 2024-05-12T04:01:15.335 INFO:tasks.workunit.client.1.smithi033.stdout:1/466: write d11/d1c/f34 [324386,126722] 0 2024-05-12T04:01:15.336 INFO:tasks.workunit.client.1.smithi033.stdout:5/432: dwrite d3/d12/d29/f38 [0,4194304] 0 2024-05-12T04:01:15.336 INFO:tasks.workunit.client.0.smithi032.stdout:7/336: write dd/d1b/f4b [248059,82039] 0 2024-05-12T04:01:15.339 INFO:tasks.workunit.client.0.smithi032.stdout:5/419: creat d4/d7/d58/d5a/f96 x:0 0 0 2024-05-12T04:01:15.339 INFO:tasks.workunit.client.0.smithi032.stdout:5/420: fdatasync d4/d7/d14/d39/d1b/f23 0 2024-05-12T04:01:15.341 INFO:tasks.workunit.client.1.smithi033.stdout:6/506: getdents d4 0 2024-05-12T04:01:15.344 INFO:tasks.workunit.client.1.smithi033.stdout:8/431: fsync d1/d4/d1d/f26 0 2024-05-12T04:01:15.346 INFO:tasks.workunit.client.1.smithi033.stdout:2/504: rmdir dc/d13/d1c/d98/da2 0 2024-05-12T04:01:15.347 INFO:tasks.workunit.client.1.smithi033.stdout:6/507: dwrite d4/d5/f27 [0,4194304] 0 2024-05-12T04:01:15.349 INFO:tasks.workunit.client.0.smithi032.stdout:4/246: dread d0/d3/df/f30 [0,4194304] 0 2024-05-12T04:01:15.349 INFO:tasks.workunit.client.0.smithi032.stdout:6/318: creat d7/d9/d48/d67/f6e x:0 0 0 2024-05-12T04:01:15.350 INFO:tasks.workunit.client.0.smithi032.stdout:6/319: chown d7/l30 22720 1 2024-05-12T04:01:15.354 INFO:tasks.workunit.client.0.smithi032.stdout:5/421: rename d4/f1e to d4/d7/d14/d26/d33/d72/d92/f97 0 2024-05-12T04:01:15.356 INFO:tasks.workunit.client.0.smithi032.stdout:5/422: fdatasync d4/d7/d14/d26/d33/d4a/d38/d3b/d4c/f6a 0 2024-05-12T04:01:15.357 INFO:tasks.workunit.client.0.smithi032.stdout:5/423: stat d4/d7/d14/d26/d33/d4a/f51 0 2024-05-12T04:01:15.358 INFO:tasks.workunit.client.1.smithi033.stdout:4/521: rmdir dd/d7a/d87/d7e/d90 0 2024-05-12T04:01:15.359 INFO:tasks.workunit.client.0.smithi032.stdout:7/337: write dd/d1b/d43/d5e/d37/f41 [546958,37716] 0 2024-05-12T04:01:15.359 INFO:tasks.workunit.client.1.smithi033.stdout:4/522: truncate dd/d7a/f85 914724 0 2024-05-12T04:01:15.360 INFO:tasks.workunit.client.1.smithi033.stdout:4/523: chown dd/d11/d31/d66/l23 100442777 1 2024-05-12T04:01:15.363 INFO:tasks.workunit.client.0.smithi032.stdout:5/424: dread d4/fa [0,4194304] 0 2024-05-12T04:01:15.364 INFO:tasks.workunit.client.1.smithi033.stdout:5/433: truncate d3/d12/d29/d2b/f46 646044 0 2024-05-12T04:01:15.364 INFO:tasks.workunit.client.0.smithi032.stdout:7/338: dread fa [0,4194304] 0 2024-05-12T04:01:15.364 INFO:tasks.workunit.client.1.smithi033.stdout:5/434: chown d3/d27/f28 6 1 2024-05-12T04:01:15.366 INFO:tasks.workunit.client.1.smithi033.stdout:0/528: symlink d6/dc/lb2 0 2024-05-12T04:01:15.369 INFO:tasks.workunit.client.1.smithi033.stdout:3/496: creat db/d17/d25/d32/d53/fb6 x:0 0 0 2024-05-12T04:01:15.370 INFO:tasks.workunit.client.1.smithi033.stdout:3/497: write db/d17/d3a/d67/d76/f78 [309925,81426] 0 2024-05-12T04:01:15.372 INFO:tasks.workunit.client.0.smithi032.stdout:5/425: dwrite d4/d7/d14/d26/f57 [0,4194304] 0 2024-05-12T04:01:15.374 INFO:tasks.workunit.client.1.smithi033.stdout:3/498: dread db/d17/d3e/d65/d7d/f94 [0,4194304] 0 2024-05-12T04:01:15.375 INFO:tasks.workunit.client.0.smithi032.stdout:6/320: write d7/d9/d48/d67/d25/f61 [597295,117251] 0 2024-05-12T04:01:15.377 INFO:tasks.workunit.client.0.smithi032.stdout:6/321: chown d7/f38 179 1 2024-05-12T04:01:15.379 INFO:tasks.workunit.client.0.smithi032.stdout:6/322: read d7/de/f6d [981132,109082] 0 2024-05-12T04:01:15.382 INFO:tasks.workunit.client.1.smithi033.stdout:6/508: unlink d4/d1a/c28 0 2024-05-12T04:01:15.382 INFO:tasks.workunit.client.0.smithi032.stdout:5/426: dread d4/d7/d14/d26/d33/d4a/f51 [0,4194304] 0 2024-05-12T04:01:15.383 INFO:tasks.workunit.client.1.smithi033.stdout:8/432: dwrite d1/d4/d1d/d36/d3c/d64/f7b [0,4194304] 0 2024-05-12T04:01:15.384 INFO:tasks.workunit.client.0.smithi032.stdout:4/247: truncate d0/f11 2557965 0 2024-05-12T04:01:15.385 INFO:tasks.workunit.client.0.smithi032.stdout:7/339: chown dd/fe 24196306 1 2024-05-12T04:01:15.389 INFO:tasks.workunit.client.1.smithi033.stdout:7/664: link de/d2b/d6c/c40 de/df/d17/d1e/dd8/ce8 0 2024-05-12T04:01:15.390 INFO:tasks.workunit.client.0.smithi032.stdout:6/323: fsync d7/d9/d48/d67/d25/f2a 0 2024-05-12T04:01:15.390 INFO:tasks.workunit.client.1.smithi033.stdout:7/665: readlink de/df/d7a/l8c 0 2024-05-12T04:01:15.393 INFO:tasks.workunit.client.1.smithi033.stdout:5/435: creat d3/d12/d25/d26/d2e/d54/d57/d70/f87 x:0 0 0 2024-05-12T04:01:15.393 INFO:tasks.workunit.client.1.smithi033.stdout:4/524: dwrite dd/d11/f19 [0,4194304] 0 2024-05-12T04:01:15.402 INFO:tasks.workunit.client.1.smithi033.stdout:3/499: mknod db/d17/d25/cb7 0 2024-05-12T04:01:15.404 INFO:tasks.workunit.client.0.smithi032.stdout:7/340: dwrite fa [4194304,4194304] 0 2024-05-12T04:01:15.404 INFO:tasks.workunit.client.0.smithi032.stdout:6/324: creat d7/d9/d48/d67/d25/f6f x:0 0 0 2024-05-12T04:01:15.410 INFO:tasks.workunit.client.0.smithi032.stdout:4/248: symlink d0/d39/l3e 0 2024-05-12T04:01:15.412 INFO:tasks.workunit.client.1.smithi033.stdout:0/529: write d6/dc/f46 [2834231,37602] 0 2024-05-12T04:01:15.415 INFO:tasks.workunit.client.1.smithi033.stdout:6/509: creat d4/d1a/d4e/fa6 x:0 0 0 2024-05-12T04:01:15.417 INFO:tasks.workunit.client.0.smithi032.stdout:6/325: creat d7/d9/d48/f70 x:0 0 0 2024-05-12T04:01:15.423 INFO:tasks.workunit.client.1.smithi033.stdout:5/436: creat d3/d15/f88 x:0 0 0 2024-05-12T04:01:15.426 INFO:tasks.workunit.client.0.smithi032.stdout:5/427: getdents d4/d7/d14/d26/d33/d5f/d7f 0 2024-05-12T04:01:15.431 INFO:tasks.workunit.client.1.smithi033.stdout:3/500: symlink db/d17/d25/d32/d53/lb8 0 2024-05-12T04:01:15.432 INFO:tasks.workunit.client.0.smithi032.stdout:7/341: creat dd/d1b/d1e/d45/f73 x:0 0 0 2024-05-12T04:01:15.437 INFO:tasks.workunit.client.0.smithi032.stdout:4/249: rename d0/d3/l23 to d0/d3/df/l3f 0 2024-05-12T04:01:15.437 INFO:tasks.workunit.client.1.smithi033.stdout:4/525: dwrite fc [4194304,4194304] 0 2024-05-12T04:01:15.439 INFO:tasks.workunit.client.0.smithi032.stdout:5/428: mknod d4/d7/d14/d26/d33/d5f/c98 0 2024-05-12T04:01:15.442 INFO:tasks.workunit.client.0.smithi032.stdout:6/326: getdents d7/de/d57 0 2024-05-12T04:01:15.445 INFO:tasks.workunit.client.1.smithi033.stdout:3/501: dread db/d59/f8d [0,4194304] 0 2024-05-12T04:01:15.446 INFO:tasks.workunit.client.0.smithi032.stdout:5/429: dread - d4/d7/f7e zero size 2024-05-12T04:01:15.447 INFO:tasks.workunit.client.0.smithi032.stdout:5/430: write d4/d7/d14/f1d [5198447,50080] 0 2024-05-12T04:01:15.449 INFO:tasks.workunit.client.0.smithi032.stdout:5/431: readlink d4/d7/d14/d26/d33/l42 0 2024-05-12T04:01:15.449 INFO:tasks.workunit.client.0.smithi032.stdout:6/327: rename d7/d9/d48/f70 to d7/d9/d60/f71 0 2024-05-12T04:01:15.451 INFO:tasks.workunit.client.0.smithi032.stdout:5/432: chown d4/d7/d14/d39/d2b/d6e/l83 26979 1 2024-05-12T04:01:15.456 INFO:tasks.workunit.client.1.smithi033.stdout:4/526: dread dd/f2b [0,4194304] 0 2024-05-12T04:01:15.469 INFO:tasks.workunit.client.1.smithi033.stdout:9/741: rename d2/d4/d29/d32/db7/fd4 to d2/d4/d29/f109 0 2024-05-12T04:01:15.470 INFO:tasks.workunit.client.1.smithi033.stdout:1/467: getdents d11/d17/d27/d36/d46/d4c 0 2024-05-12T04:01:15.481 INFO:tasks.workunit.client.1.smithi033.stdout:2/505: rename dc/d17/d27/l8e to dc/d13/d1f/d6e/la9 0 2024-05-12T04:01:15.482 INFO:tasks.workunit.client.1.smithi033.stdout:2/506: fdatasync dc/d13/d1c/f21 0 2024-05-12T04:01:15.482 INFO:tasks.workunit.client.1.smithi033.stdout:8/433: creat d1/d16/d27/f85 x:0 0 0 2024-05-12T04:01:15.483 INFO:tasks.workunit.client.1.smithi033.stdout:2/507: chown dc/d13/d65/f83 429191 1 2024-05-12T04:01:15.486 INFO:tasks.workunit.client.1.smithi033.stdout:7/666: creat de/d2f/fe9 x:0 0 0 2024-05-12T04:01:15.487 INFO:tasks.workunit.client.1.smithi033.stdout:7/667: fdatasync de/df/d17/f51 0 2024-05-12T04:01:15.489 INFO:tasks.workunit.client.1.smithi033.stdout:2/508: dwrite dc/d13/d1c/f21 [0,4194304] 0 2024-05-12T04:01:15.489 INFO:tasks.workunit.client.1.smithi033.stdout:0/530: creat d6/dc/fb3 x:0 0 0 2024-05-12T04:01:15.498 INFO:tasks.workunit.client.1.smithi033.stdout:5/437: dwrite d3/d15/d16/f42 [0,4194304] 0 2024-05-12T04:01:15.500 INFO:tasks.workunit.client.0.smithi032.stdout:9/249: sync 2024-05-12T04:01:15.500 INFO:tasks.workunit.client.0.smithi032.stdout:4/250: link d0/l14 d0/d3/df/l40 0 2024-05-12T04:01:15.500 INFO:tasks.workunit.client.0.smithi032.stdout:8/383: sync 2024-05-12T04:01:15.502 INFO:tasks.workunit.client.0.smithi032.stdout:5/433: write f1 [1612375,121993] 0 2024-05-12T04:01:15.504 INFO:tasks.workunit.client.0.smithi032.stdout:5/434: chown d4/d7/d14/d26/d33/d5f/f87 62924906 1 2024-05-12T04:01:15.505 INFO:tasks.workunit.client.1.smithi033.stdout:0/531: truncate d6/db/d2c/f62 378588 0 2024-05-12T04:01:15.505 INFO:tasks.workunit.client.1.smithi033.stdout:3/502: dwrite db/d17/d4d/faa [0,4194304] 0 2024-05-12T04:01:15.506 INFO:tasks.workunit.client.1.smithi033.stdout:3/503: chown db/d17/d3a/l79 3397 1 2024-05-12T04:01:15.509 INFO:tasks.workunit.client.0.smithi032.stdout:4/251: fdatasync d0/d3/fd 0 2024-05-12T04:01:15.511 INFO:tasks.workunit.client.1.smithi033.stdout:4/527: getdents dd/d11/d31 0 2024-05-12T04:01:15.511 INFO:tasks.workunit.client.1.smithi033.stdout:1/468: dwrite d11/d17/f3e [0,4194304] 0 2024-05-12T04:01:15.513 INFO:tasks.workunit.client.1.smithi033.stdout:7/668: write de/d2b/d6c/da0/da6/dd1/faf [5136519,50050] 0 2024-05-12T04:01:15.513 INFO:tasks.workunit.client.0.smithi032.stdout:9/250: dwrite d0/d3/d6/f45 [4194304,4194304] 0 2024-05-12T04:01:15.517 INFO:tasks.workunit.client.0.smithi032.stdout:8/384: rename d0/d9/d13/d21/d35/d37 to d0/d9/d1b/d27/d88 0 2024-05-12T04:01:15.519 INFO:tasks.workunit.client.1.smithi033.stdout:9/742: getdents d2/d70/d7b/d94/de3 0 2024-05-12T04:01:15.523 INFO:tasks.workunit.client.1.smithi033.stdout:2/509: dwrite dc/d13/d1c/f4f [0,4194304] 0 2024-05-12T04:01:15.526 INFO:tasks.workunit.client.1.smithi033.stdout:0/532: creat d6/db/d2c/d82/fb4 x:0 0 0 2024-05-12T04:01:15.527 INFO:tasks.workunit.client.1.smithi033.stdout:8/434: rename d1/f81 to d1/d16/d27/f86 0 2024-05-12T04:01:15.528 INFO:tasks.workunit.client.0.smithi032.stdout:4/252: mknod d0/d3/d12/d1d/c41 0 2024-05-12T04:01:15.528 INFO:tasks.workunit.client.1.smithi033.stdout:4/528: dread - dd/f3b zero size 2024-05-12T04:01:15.529 INFO:tasks.workunit.client.0.smithi032.stdout:9/251: fsync d0/d3/d6/d7/f15 0 2024-05-12T04:01:15.531 INFO:tasks.workunit.client.1.smithi033.stdout:9/743: creat d2/d70/d7b/d94/de3/f10a x:0 0 0 2024-05-12T04:01:15.532 INFO:tasks.workunit.client.1.smithi033.stdout:6/510: sync 2024-05-12T04:01:15.537 INFO:tasks.workunit.client.0.smithi032.stdout:4/253: creat d0/d3/d24/f42 x:0 0 0 2024-05-12T04:01:15.538 INFO:tasks.workunit.client.1.smithi033.stdout:7/669: mkdir de/df/d17/d1e/d79/dd4/dea 0 2024-05-12T04:01:15.540 INFO:tasks.workunit.client.0.smithi032.stdout:8/385: getdents d0/d9/d13 0 2024-05-12T04:01:15.540 INFO:tasks.workunit.client.1.smithi033.stdout:0/533: dread d6/db/f19 [0,4194304] 0 2024-05-12T04:01:15.541 INFO:tasks.workunit.client.0.smithi032.stdout:9/252: dwrite d0/d3/d6/d7/da/d17/f4e [0,4194304] 0 2024-05-12T04:01:15.541 INFO:tasks.workunit.client.1.smithi033.stdout:0/534: readlink d6/db/l2d 0 2024-05-12T04:01:15.543 INFO:tasks.workunit.client.0.smithi032.stdout:8/386: dread - d0/d9/d1b/d27/d88/f62 zero size 2024-05-12T04:01:15.545 INFO:tasks.workunit.client.1.smithi033.stdout:6/511: creat d4/d1a/d67/d68/d88/fa7 x:0 0 0 2024-05-12T04:01:15.546 INFO:tasks.workunit.client.1.smithi033.stdout:9/744: dread d2/d4/d2b/d6d/d2f/f38 [0,4194304] 0 2024-05-12T04:01:15.547 INFO:tasks.workunit.client.0.smithi032.stdout:9/253: fsync d0/d3/d6/d7/da/f1c 0 2024-05-12T04:01:15.549 INFO:tasks.workunit.client.0.smithi032.stdout:8/387: symlink d0/d9/d1b/d27/d4c/l89 0 2024-05-12T04:01:15.549 INFO:tasks.workunit.client.0.smithi032.stdout:9/254: chown d0/d3/d6/d7/d2a 7 1 2024-05-12T04:01:15.551 INFO:tasks.workunit.client.1.smithi033.stdout:9/745: dread d2/d4/d2b/d78/d76/fc9 [0,4194304] 0 2024-05-12T04:01:15.551 INFO:tasks.workunit.client.0.smithi032.stdout:4/254: dwrite d0/d3/d31/f35 [0,4194304] 0 2024-05-12T04:01:15.554 INFO:tasks.workunit.client.1.smithi033.stdout:0/535: mknod d6/db/d2c/d82/cb5 0 2024-05-12T04:01:15.555 INFO:tasks.workunit.client.1.smithi033.stdout:6/512: fdatasync d4/d5/f20 0 2024-05-12T04:01:15.557 INFO:tasks.workunit.client.1.smithi033.stdout:5/438: rename d3/f3b to d3/d12/d25/d26/d2e/d54/d57/f89 0 2024-05-12T04:01:15.558 INFO:tasks.workunit.client.1.smithi033.stdout:7/670: mkdir de/d20/deb 0 2024-05-12T04:01:15.560 INFO:tasks.workunit.client.1.smithi033.stdout:2/510: getdents dc/d13/d1f 0 2024-05-12T04:01:15.562 INFO:tasks.workunit.client.1.smithi033.stdout:2/511: write dc/d13/d1c/d98/f99 [979018,9063] 0 2024-05-12T04:01:15.565 INFO:tasks.workunit.client.1.smithi033.stdout:7/671: mkdir de/d20/d38/dec 0 2024-05-12T04:01:15.567 INFO:tasks.workunit.client.1.smithi033.stdout:7/672: write de/d2f/d70/f74 [4195990,63861] 0 2024-05-12T04:01:15.569 INFO:tasks.workunit.client.1.smithi033.stdout:0/536: dwrite d6/db/f19 [0,4194304] 0 2024-05-12T04:01:15.569 INFO:tasks.workunit.client.1.smithi033.stdout:8/435: rename d1/d16/c18 to d1/d16/d27/d4d/c87 0 2024-05-12T04:01:15.570 INFO:tasks.workunit.client.1.smithi033.stdout:0/537: chown d6/dc/d39/f64 11 1 2024-05-12T04:01:15.571 INFO:tasks.workunit.client.0.smithi032.stdout:9/255: dread d0/d3/d1b/f47 [4194304,4194304] 0 2024-05-12T04:01:15.571 INFO:tasks.workunit.client.1.smithi033.stdout:7/673: readlink de/df/d17/d1b/l1d 0 2024-05-12T04:01:15.573 INFO:tasks.workunit.client.1.smithi033.stdout:0/538: truncate d6/dc/d39/f8d 747681 0 2024-05-12T04:01:15.575 INFO:tasks.workunit.client.1.smithi033.stdout:4/529: rename dd/d11/f46 to dd/d11/d75/fa2 0 2024-05-12T04:01:15.575 INFO:tasks.workunit.client.1.smithi033.stdout:8/436: mknod d1/d16/d27/d4d/d82/c88 0 2024-05-12T04:01:15.575 INFO:tasks.workunit.client.1.smithi033.stdout:4/530: fsync fc 0 2024-05-12T04:01:15.579 INFO:tasks.workunit.client.0.smithi032.stdout:9/256: rename d0/d3/d6/f30 to d0/d3/d6/d7/da/d17/d51/f54 0 2024-05-12T04:01:15.586 INFO:tasks.workunit.client.0.smithi032.stdout:8/388: dread d0/f2f [0,4194304] 0 2024-05-12T04:01:15.587 INFO:tasks.workunit.client.1.smithi033.stdout:2/512: unlink dc/d17/d27/f60 0 2024-05-12T04:01:15.591 INFO:tasks.workunit.client.1.smithi033.stdout:7/674: fdatasync de/df/d17/d1b/d36/f4a 0 2024-05-12T04:01:15.593 INFO:tasks.workunit.client.1.smithi033.stdout:0/539: truncate d6/dc/d58/f81 990123 0 2024-05-12T04:01:15.593 INFO:tasks.workunit.client.0.smithi032.stdout:8/389: dwrite d0/d9/d13/f58 [0,4194304] 0 2024-05-12T04:01:15.597 INFO:tasks.workunit.client.0.smithi032.stdout:5/435: dwrite d4/f49 [0,4194304] 0 2024-05-12T04:01:15.598 INFO:tasks.workunit.client.1.smithi033.stdout:7/675: dread de/df/d17/f34 [0,4194304] 0 2024-05-12T04:01:15.599 INFO:tasks.workunit.client.1.smithi033.stdout:1/469: write d11/d17/d27/f52 [551921,20328] 0 2024-05-12T04:01:15.604 INFO:tasks.workunit.client.1.smithi033.stdout:3/504: dwrite db/f5e [0,4194304] 0 2024-05-12T04:01:15.606 INFO:tasks.workunit.client.1.smithi033.stdout:3/505: write db/d17/d25/f40 [5252025,121926] 0 2024-05-12T04:01:15.609 INFO:tasks.workunit.client.0.smithi032.stdout:8/390: symlink d0/d9/d1b/d66/l8a 0 2024-05-12T04:01:15.610 INFO:tasks.workunit.client.0.smithi032.stdout:5/436: mkdir d4/d7/d58/d5a/d99 0 2024-05-12T04:01:15.613 INFO:tasks.workunit.client.1.smithi033.stdout:2/513: creat dc/d17/d74/faa x:0 0 0 2024-05-12T04:01:15.613 INFO:tasks.workunit.client.1.smithi033.stdout:2/514: stat c5 0 2024-05-12T04:01:15.616 INFO:tasks.workunit.client.1.smithi033.stdout:9/746: rename d2/d4/d2b/d6d/d2f/d5f/lea to d2/l10b 0 2024-05-12T04:01:15.618 INFO:tasks.workunit.client.0.smithi032.stdout:5/437: dwrite f1 [0,4194304] 0 2024-05-12T04:01:15.621 INFO:tasks.workunit.client.1.smithi033.stdout:7/676: creat de/df/d17/d1e/d79/fed x:0 0 0 2024-05-12T04:01:15.621 INFO:tasks.workunit.client.1.smithi033.stdout:9/747: dread d2/d4/d2b/d6d/d4c/fe0 [0,4194304] 0 2024-05-12T04:01:15.621 INFO:tasks.workunit.client.1.smithi033.stdout:7/677: chown de/d20 4 1 2024-05-12T04:01:15.624 INFO:tasks.workunit.client.0.smithi032.stdout:8/391: rename d0/d9/d1b/d27/d59/c5f to d0/d9/d1b/d66/d75/d84/c8b 0 2024-05-12T04:01:15.627 INFO:tasks.workunit.client.0.smithi032.stdout:4/255: truncate d0/d3/df/f30 433026 0 2024-05-12T04:01:15.630 INFO:tasks.workunit.client.1.smithi033.stdout:8/437: creat d1/d16/d27/d7c/f89 x:0 0 0 2024-05-12T04:01:15.631 INFO:tasks.workunit.client.1.smithi033.stdout:8/438: chown d1/d16/c41 764 1 2024-05-12T04:01:15.631 INFO:tasks.workunit.client.1.smithi033.stdout:5/439: truncate d3/f77 2982920 0 2024-05-12T04:01:15.632 INFO:tasks.workunit.client.0.smithi032.stdout:8/392: dwrite d0/d9/d13/f29 [0,4194304] 0 2024-05-12T04:01:15.632 INFO:tasks.workunit.client.1.smithi033.stdout:8/439: readlink l0 0 2024-05-12T04:01:15.632 INFO:tasks.workunit.client.1.smithi033.stdout:5/440: write d3/d15/f30 [808855,47334] 0 2024-05-12T04:01:15.637 INFO:tasks.workunit.client.1.smithi033.stdout:3/506: symlink db/d17/d3a/d8a/d9d/lb9 0 2024-05-12T04:01:15.638 INFO:tasks.workunit.client.1.smithi033.stdout:3/507: readlink db/d17/d4d/l9a 0 2024-05-12T04:01:15.639 INFO:tasks.workunit.client.1.smithi033.stdout:3/508: fdatasync db/d17/d25/d32/d3c/f5a 0 2024-05-12T04:01:15.640 INFO:tasks.workunit.client.1.smithi033.stdout:2/515: mkdir dc/d13/d39/dab 0 2024-05-12T04:01:15.641 INFO:tasks.workunit.client.0.smithi032.stdout:8/393: mknod d0/d9/d1b/d27/d88/c8c 0 2024-05-12T04:01:15.644 INFO:tasks.workunit.client.1.smithi033.stdout:0/540: rename d6/f7c to d6/d11/d57/fb6 0 2024-05-12T04:01:15.646 INFO:tasks.workunit.client.0.smithi032.stdout:5/438: creat d4/f9a x:0 0 0 2024-05-12T04:01:15.648 INFO:tasks.workunit.client.0.smithi032.stdout:9/257: truncate d0/d3/d6/d7/d27/f31 168493 0 2024-05-12T04:01:15.649 INFO:tasks.workunit.client.0.smithi032.stdout:4/256: symlink d0/d3/l43 0 2024-05-12T04:01:15.652 INFO:tasks.workunit.client.0.smithi032.stdout:8/394: symlink d0/d9/d1b/d66/l8d 0 2024-05-12T04:01:15.653 INFO:tasks.workunit.client.1.smithi033.stdout:7/678: symlink de/d2b/d6c/db5/dd0/lee 0 2024-05-12T04:01:15.655 INFO:tasks.workunit.client.1.smithi033.stdout:7/679: write de/d2b/d6c/da0/fa8 [350512,95164] 0 2024-05-12T04:01:15.656 INFO:tasks.workunit.client.1.smithi033.stdout:9/748: truncate d2/dd/d39/f67 5660953 0 2024-05-12T04:01:15.657 INFO:tasks.workunit.client.1.smithi033.stdout:9/749: readlink d2/d4/d2b/d6d/db5/le8 0 2024-05-12T04:01:15.659 INFO:tasks.workunit.client.0.smithi032.stdout:5/439: dwrite d4/d7/d14/d39/d2b/d6e/f77 [0,4194304] 0 2024-05-12T04:01:15.661 INFO:tasks.workunit.client.0.smithi032.stdout:4/257: dwrite d0/d3/d24/f42 [0,4194304] 0 2024-05-12T04:01:15.662 INFO:tasks.workunit.client.0.smithi032.stdout:8/395: rmdir d0/d9/d1b/d27/d4c 39 2024-05-12T04:01:15.663 INFO:tasks.workunit.client.1.smithi033.stdout:1/470: write d11/f6b [579453,25570] 0 2024-05-12T04:01:15.667 INFO:tasks.workunit.client.1.smithi033.stdout:8/440: mknod d1/d16/d27/d4d/d51/c8a 0 2024-05-12T04:01:15.669 INFO:tasks.workunit.client.1.smithi033.stdout:5/441: mknod d3/d12/d29/c8a 0 2024-05-12T04:01:15.671 INFO:tasks.workunit.client.0.smithi032.stdout:1/346: sync 2024-05-12T04:01:15.671 INFO:tasks.workunit.client.1.smithi033.stdout:5/442: write d3/d15/f88 [155988,63344] 0 2024-05-12T04:01:15.672 INFO:tasks.workunit.client.0.smithi032.stdout:3/353: sync 2024-05-12T04:01:15.672 INFO:tasks.workunit.client.0.smithi032.stdout:4/258: read d0/d3/df/f29 [5586,51304] 0 2024-05-12T04:01:15.673 INFO:tasks.workunit.client.0.smithi032.stdout:3/354: write d15/d33/f4c [54977,37292] 0 2024-05-12T04:01:15.674 INFO:tasks.workunit.client.1.smithi033.stdout:3/509: chown db/d17/d3e/c5f 1210732267 1 2024-05-12T04:01:15.674 INFO:tasks.workunit.client.0.smithi032.stdout:1/347: chown d1/d7/d49/d67/l6d 1 1 2024-05-12T04:01:15.677 INFO:tasks.workunit.client.1.smithi033.stdout:6/513: sync 2024-05-12T04:01:15.678 INFO:tasks.workunit.client.0.smithi032.stdout:3/355: write d15/d33/f4c [1019960,41267] 0 2024-05-12T04:01:15.679 INFO:tasks.workunit.client.0.smithi032.stdout:3/356: readlink d15/d20/l35 0 2024-05-12T04:01:15.680 INFO:tasks.workunit.client.0.smithi032.stdout:5/440: creat d4/d7/d14/d26/d47/f9b x:0 0 0 2024-05-12T04:01:15.687 INFO:tasks.workunit.client.0.smithi032.stdout:9/258: dread d0/d3/d6/f45 [0,4194304] 0 2024-05-12T04:01:15.688 INFO:tasks.workunit.client.0.smithi032.stdout:1/348: dwrite d1/d7/d30/f28 [0,4194304] 0 2024-05-12T04:01:15.692 INFO:tasks.workunit.client.0.smithi032.stdout:3/357: creat d15/d20/d26/d6a/f77 x:0 0 0 2024-05-12T04:01:15.692 INFO:tasks.workunit.client.0.smithi032.stdout:1/349: dread d1/dd/f53 [0,4194304] 0 2024-05-12T04:01:15.693 INFO:tasks.workunit.client.0.smithi032.stdout:5/441: dwrite d4/f12 [4194304,4194304] 0 2024-05-12T04:01:15.694 INFO:tasks.workunit.client.0.smithi032.stdout:9/259: read d0/d3/d6/d7/da/f53 [20599,53959] 0 2024-05-12T04:01:15.697 INFO:tasks.workunit.client.0.smithi032.stdout:8/396: creat d0/d9/f8e x:0 0 0 2024-05-12T04:01:15.700 INFO:tasks.workunit.client.0.smithi032.stdout:9/260: write d0/d3/d6/d7/d2a/d46/f12 [2820033,127167] 0 2024-05-12T04:01:15.705 INFO:tasks.workunit.client.1.smithi033.stdout:1/471: creat d11/d17/d27/d76/f9c x:0 0 0 2024-05-12T04:01:15.707 INFO:tasks.workunit.client.1.smithi033.stdout:1/472: truncate d11/d17/d27/f57 2228153 0 2024-05-12T04:01:15.707 INFO:tasks.workunit.client.1.smithi033.stdout:1/473: chown d11/l26 96862 1 2024-05-12T04:01:15.708 INFO:tasks.workunit.client.0.smithi032.stdout:3/358: creat d15/d1b/d65/f78 x:0 0 0 2024-05-12T04:01:15.710 INFO:tasks.workunit.client.1.smithi033.stdout:5/443: chown d3/d12/d25/d26/d2e/d54/d57/f89 20 1 2024-05-12T04:01:15.711 INFO:tasks.workunit.client.0.smithi032.stdout:5/442: dwrite d4/f9a [0,4194304] 0 2024-05-12T04:01:15.715 INFO:tasks.workunit.client.0.smithi032.stdout:1/350: unlink d1/d7/d49/d67/f63 0 2024-05-12T04:01:15.716 INFO:tasks.workunit.client.1.smithi033.stdout:3/510: creat db/d17/d3a/d67/d76/da0/fba x:0 0 0 2024-05-12T04:01:15.718 INFO:tasks.workunit.client.1.smithi033.stdout:6/514: creat d4/d1a/d3c/fa8 x:0 0 0 2024-05-12T04:01:15.719 INFO:tasks.workunit.client.1.smithi033.stdout:6/515: read d4/d5/f20 [141558,8318] 0 2024-05-12T04:01:15.725 INFO:tasks.workunit.client.1.smithi033.stdout:0/541: symlink d6/db/d2c/lb7 0 2024-05-12T04:01:15.727 INFO:tasks.workunit.client.1.smithi033.stdout:7/680: mkdir de/df/d17/def 0 2024-05-12T04:01:15.728 INFO:tasks.workunit.client.0.smithi032.stdout:3/359: symlink d15/d20/d26/d6a/l79 0 2024-05-12T04:01:15.729 INFO:tasks.workunit.client.1.smithi033.stdout:9/750: write d2/d70/d7b/f9f [1028280,100162] 0 2024-05-12T04:01:15.729 INFO:tasks.workunit.client.0.smithi032.stdout:3/360: fdatasync d15/d20/d26/f3c 0 2024-05-12T04:01:15.731 INFO:tasks.workunit.client.1.smithi033.stdout:0/542: dwrite d6/db/d2c/d73/f87 [4194304,4194304] 0 2024-05-12T04:01:15.732 INFO:tasks.workunit.client.0.smithi032.stdout:5/443: chown d4/d7/d14/d26/d33/d4a/c64 1 1 2024-05-12T04:01:15.733 INFO:tasks.workunit.client.1.smithi033.stdout:4/531: link dd/d11/d31/d66/c6b dd/ca3 0 2024-05-12T04:01:15.733 INFO:tasks.workunit.client.0.smithi032.stdout:5/444: write d4/d7/d14/d26/d33/f8b [646783,40606] 0 2024-05-12T04:01:15.740 INFO:tasks.workunit.client.0.smithi032.stdout:1/351: write d1/d25/f64 [259053,96930] 0 2024-05-12T04:01:15.741 INFO:tasks.workunit.client.1.smithi033.stdout:3/511: creat db/d17/d3e/d65/fbb x:0 0 0 2024-05-12T04:01:15.744 INFO:tasks.workunit.client.1.smithi033.stdout:2/516: truncate dc/d13/d1c/f21 280992 0 2024-05-12T04:01:15.745 INFO:tasks.workunit.client.0.smithi032.stdout:5/445: dwrite d4/d7/d14/d26/f66 [0,4194304] 0 2024-05-12T04:01:15.745 INFO:tasks.workunit.client.1.smithi033.stdout:6/516: mknod d4/d1a/d67/d68/ca9 0 2024-05-12T04:01:15.746 INFO:tasks.workunit.client.0.smithi032.stdout:9/261: write d0/d3/d6/d7/f15 [355545,75468] 0 2024-05-12T04:01:15.747 INFO:tasks.workunit.client.0.smithi032.stdout:9/262: fdatasync d0/d3/d1b/f38 0 2024-05-12T04:01:15.748 INFO:tasks.workunit.client.1.smithi033.stdout:3/512: dwrite db/d17/d3a/fb5 [0,4194304] 0 2024-05-12T04:01:15.749 INFO:tasks.workunit.client.1.smithi033.stdout:8/441: sync 2024-05-12T04:01:15.750 INFO:tasks.workunit.client.1.smithi033.stdout:3/513: chown db/d17/d3a/l4e 821628575 1 2024-05-12T04:01:15.750 INFO:tasks.workunit.client.1.smithi033.stdout:7/681: mkdir de/df/d17/d1e/dc4/df0 0 2024-05-12T04:01:15.752 INFO:tasks.workunit.client.0.smithi032.stdout:3/361: mkdir d15/d20/d22/d7a 0 2024-05-12T04:01:15.752 INFO:tasks.workunit.client.0.smithi032.stdout:5/446: dread f1 [0,4194304] 0 2024-05-12T04:01:15.754 INFO:tasks.workunit.client.0.smithi032.stdout:8/397: mkdir d0/d9/d1b/d27/d88/d56/d63/d8f 0 2024-05-12T04:01:15.755 INFO:tasks.workunit.client.1.smithi033.stdout:9/751: mknod d2/d4/d2b/d33/dd2/dde/c10c 0 2024-05-12T04:01:15.756 INFO:tasks.workunit.client.0.smithi032.stdout:1/352: creat d1/d7/d30/d18/f76 x:0 0 0 2024-05-12T04:01:15.758 INFO:tasks.workunit.client.1.smithi033.stdout:0/543: mkdir d6/dc/d58/d6a/d80/db8 0 2024-05-12T04:01:15.759 INFO:tasks.workunit.client.0.smithi032.stdout:1/353: read d1/d25/f64 [117497,54411] 0 2024-05-12T04:01:15.760 INFO:tasks.workunit.client.1.smithi033.stdout:4/532: mkdir dd/d69/da4 0 2024-05-12T04:01:15.760 INFO:tasks.workunit.client.1.smithi033.stdout:4/533: chown c6 181 1 2024-05-12T04:01:15.761 INFO:tasks.workunit.client.1.smithi033.stdout:4/534: dread - dd/d11/d31/d66/f92 zero size 2024-05-12T04:01:15.762 INFO:tasks.workunit.client.1.smithi033.stdout:5/444: mkdir d3/d15/d16/d78/d8b 0 2024-05-12T04:01:15.767 INFO:tasks.workunit.client.1.smithi033.stdout:3/514: dread db/d17/d25/d32/d3c/d3d/f72 [0,4194304] 0 2024-05-12T04:01:15.768 INFO:tasks.workunit.client.0.smithi032.stdout:3/362: fsync d15/d24/d2a/d5b/f71 0 2024-05-12T04:01:15.769 INFO:tasks.workunit.client.0.smithi032.stdout:5/447: symlink d4/d7/d58/l9c 0 2024-05-12T04:01:15.771 INFO:tasks.workunit.client.0.smithi032.stdout:1/354: dwrite d1/d7/d30/d35/d46/f4b [0,4194304] 0 2024-05-12T04:01:15.773 INFO:tasks.workunit.client.0.smithi032.stdout:1/355: truncate d1/d7/d49/f4f 1315739 0 2024-05-12T04:01:15.774 INFO:tasks.workunit.client.1.smithi033.stdout:6/517: rename d4/f51 to d4/d1a/d4e/d4f/d94/faa 0 2024-05-12T04:01:15.776 INFO:tasks.workunit.client.1.smithi033.stdout:8/442: chown d1/d4/d1d/d36/d3c/f55 2 1 2024-05-12T04:01:15.782 INFO:tasks.workunit.client.1.smithi033.stdout:4/535: dread dd/d11/d31/d38/f44 [0,4194304] 0 2024-05-12T04:01:15.785 INFO:tasks.workunit.client.1.smithi033.stdout:9/752: mkdir d2/d70/d10d 0 2024-05-12T04:01:15.792 INFO:tasks.workunit.client.0.smithi032.stdout:1/356: dread - d1/f47 zero size 2024-05-12T04:01:15.793 INFO:tasks.workunit.client.1.smithi033.stdout:5/445: mkdir d3/d15/d8c 0 2024-05-12T04:01:15.794 INFO:tasks.workunit.client.0.smithi032.stdout:8/398: mkdir d0/d9/d1b/d27/d4c/d4e/d90 0 2024-05-12T04:01:15.796 INFO:tasks.workunit.client.1.smithi033.stdout:2/517: symlink dc/lac 0 2024-05-12T04:01:15.798 INFO:tasks.workunit.client.1.smithi033.stdout:3/515: chown db/d17/d4d/l62 0 1 2024-05-12T04:01:15.799 INFO:tasks.workunit.client.1.smithi033.stdout:3/516: chown db/d17/d4d/faa 6 1 2024-05-12T04:01:15.799 INFO:tasks.workunit.client.0.smithi032.stdout:9/263: dwrite d0/d3/d6/d7/da/f1c [0,4194304] 0 2024-05-12T04:01:15.799 INFO:tasks.workunit.client.1.smithi033.stdout:3/517: chown db/d17/d25/d32/d53/lb8 435421432 1 2024-05-12T04:01:15.800 INFO:tasks.workunit.client.0.smithi032.stdout:5/448: mkdir d4/d7/d14/d26/d33/d4a/d9d 0 2024-05-12T04:01:15.805 INFO:tasks.workunit.client.1.smithi033.stdout:8/443: mknod d1/d16/d29/c8b 0 2024-05-12T04:01:15.806 INFO:tasks.workunit.client.1.smithi033.stdout:8/444: chown d1/d16/d27/d4d/l68 52 1 2024-05-12T04:01:15.808 INFO:tasks.workunit.client.1.smithi033.stdout:7/682: getdents de/df/d17/d1b/d36 0 2024-05-12T04:01:15.808 INFO:tasks.workunit.client.1.smithi033.stdout:7/683: chown de/df/d17/d1e/d79/dd4/dea 938663 1 2024-05-12T04:01:15.809 INFO:tasks.workunit.client.0.smithi032.stdout:1/357: dread d1/d25/f64 [0,4194304] 0 2024-05-12T04:01:15.810 INFO:tasks.workunit.client.1.smithi033.stdout:1/474: getdents d11/d17/d27/d36 0 2024-05-12T04:01:15.811 INFO:tasks.workunit.client.0.smithi032.stdout:1/358: chown d1/d7/d30/d35/d54 3 1 2024-05-12T04:01:15.811 INFO:tasks.workunit.client.1.smithi033.stdout:7/684: write de/df/d17/d1b/d36/f4a [1909475,67205] 0 2024-05-12T04:01:15.812 INFO:tasks.workunit.client.0.smithi032.stdout:1/359: dread d1/d25/f64 [0,4194304] 0 2024-05-12T04:01:15.813 INFO:tasks.workunit.client.0.smithi032.stdout:9/264: rename d0/d3/d1b/f47 to d0/d3/d6/d7/d2a/d46/d13/f55 0 2024-05-12T04:01:15.814 INFO:tasks.workunit.client.0.smithi032.stdout:9/265: chown d0/d3/d1b 32549 1 2024-05-12T04:01:15.814 INFO:tasks.workunit.client.0.smithi032.stdout:1/360: truncate d1/d7/d30/d18/f76 204750 0 2024-05-12T04:01:15.815 INFO:tasks.workunit.client.1.smithi033.stdout:0/544: symlink d6/dc/d96/d9d/lb9 0 2024-05-12T04:01:15.816 INFO:tasks.workunit.client.0.smithi032.stdout:1/361: read - d1/d7/d30/d18/f70 zero size 2024-05-12T04:01:15.816 INFO:tasks.workunit.client.0.smithi032.stdout:9/266: chown d0/d3/d6/d7/da/d39/c43 15633 1 2024-05-12T04:01:15.817 INFO:tasks.workunit.client.0.smithi032.stdout:5/449: truncate d4/d7/d14/d39/d2b/f32 2022195 0 2024-05-12T04:01:15.817 INFO:tasks.workunit.client.0.smithi032.stdout:1/362: fsync d1/d7/d49/f6b 0 2024-05-12T04:01:15.817 INFO:tasks.workunit.client.1.smithi033.stdout:7/685: dwrite de/d2f/d70/f74 [0,4194304] 0 2024-05-12T04:01:15.819 INFO:tasks.workunit.client.0.smithi032.stdout:5/450: dread - d4/d7/d14/d39/d2b/d6e/f85 zero size 2024-05-12T04:01:15.820 INFO:tasks.workunit.client.0.smithi032.stdout:8/399: mkdir d0/d9/d1b/d27/d4c/d4e/d7f/d91 0 2024-05-12T04:01:15.823 INFO:tasks.workunit.client.0.smithi032.stdout:3/363: truncate d15/d33/f4c 80777 0 2024-05-12T04:01:15.826 INFO:tasks.workunit.client.1.smithi033.stdout:3/518: mknod db/d42/cbc 0 2024-05-12T04:01:15.826 INFO:tasks.workunit.client.0.smithi032.stdout:9/267: dwrite d0/d3/d6/d7/da/d39/f4d [0,4194304] 0 2024-05-12T04:01:15.827 INFO:tasks.workunit.client.1.smithi033.stdout:3/519: write f6 [2088400,21464] 0 2024-05-12T04:01:15.829 INFO:tasks.workunit.client.0.smithi032.stdout:1/363: mkdir d1/d7/d30/d35/d77 0 2024-05-12T04:01:15.830 INFO:tasks.workunit.client.0.smithi032.stdout:5/451: creat d4/d7/d14/d26/d47/f9e x:0 0 0 2024-05-12T04:01:15.831 INFO:tasks.workunit.client.1.smithi033.stdout:1/475: creat d11/d1c/f9d x:0 0 0 2024-05-12T04:01:15.837 INFO:tasks.workunit.client.0.smithi032.stdout:3/364: dwrite d15/d1b/d65/f78 [0,4194304] 0 2024-05-12T04:01:15.838 INFO:tasks.workunit.client.0.smithi032.stdout:5/452: dread d4/f9a [0,4194304] 0 2024-05-12T04:01:15.840 INFO:tasks.workunit.client.1.smithi033.stdout:7/686: creat de/df/d17/d1e/dcc/ff1 x:0 0 0 2024-05-12T04:01:15.840 INFO:tasks.workunit.client.0.smithi032.stdout:5/453: fsync d4/d7/d14/d39/d2b/d6e/f85 0 2024-05-12T04:01:15.841 INFO:tasks.workunit.client.1.smithi033.stdout:7/687: chown de/d2f/d70/cb1 10 1 2024-05-12T04:01:15.847 INFO:tasks.workunit.client.0.smithi032.stdout:1/364: rename d1/f36 to d1/d7/f78 0 2024-05-12T04:01:15.847 INFO:tasks.workunit.client.0.smithi032.stdout:9/268: creat d0/d3/d6/d7/d2a/d46/d13/f56 x:0 0 0 2024-05-12T04:01:15.848 INFO:tasks.workunit.client.0.smithi032.stdout:1/365: write d1/d7/d30/d35/f42 [1647515,2020] 0 2024-05-12T04:01:15.857 INFO:tasks.workunit.client.1.smithi033.stdout:6/518: creat d4/d5/fab x:0 0 0 2024-05-12T04:01:15.859 INFO:tasks.workunit.client.1.smithi033.stdout:8/445: symlink d1/d4/l8c 0 2024-05-12T04:01:15.859 INFO:tasks.workunit.client.1.smithi033.stdout:8/446: stat d1/d16/c1f 0 2024-05-12T04:01:15.860 INFO:tasks.workunit.client.1.smithi033.stdout:9/753: link d2/d4/d29/lb3 d2/d70/d7b/d94/l10e 0 2024-05-12T04:01:15.862 INFO:tasks.workunit.client.1.smithi033.stdout:0/545: unlink d6/d11/d57/ca3 0 2024-05-12T04:01:15.864 INFO:tasks.workunit.client.1.smithi033.stdout:5/446: link d3/d12/l71 d3/d12/l8d 0 2024-05-12T04:01:15.866 INFO:tasks.workunit.client.0.smithi032.stdout:1/366: fdatasync d1/d7/d30/d35/f4e 0 2024-05-12T04:01:15.868 INFO:tasks.workunit.client.1.smithi033.stdout:8/447: stat d1/d4/d46/f4f 0 2024-05-12T04:01:15.870 INFO:tasks.workunit.client.1.smithi033.stdout:9/754: rename d2/d4/d29/d3c/d52/l71 to d2/d4/d29/d32/l10f 0 2024-05-12T04:01:15.871 INFO:tasks.workunit.client.1.smithi033.stdout:0/546: fsync d6/dc/f54 0 2024-05-12T04:01:15.874 INFO:tasks.workunit.client.1.smithi033.stdout:9/755: read d2/d4/d2b/d6d/db5/fcb [246033,3203] 0 2024-05-12T04:01:15.874 INFO:tasks.workunit.client.1.smithi033.stdout:5/447: rmdir d3/d12/d25 39 2024-05-12T04:01:15.875 INFO:tasks.workunit.client.1.smithi033.stdout:8/448: dwrite d1/f2 [0,4194304] 0 2024-05-12T04:01:15.882 INFO:tasks.workunit.client.1.smithi033.stdout:6/519: mkdir d4/d1a/d67/d6f/dac 0 2024-05-12T04:01:15.884 INFO:tasks.workunit.client.1.smithi033.stdout:2/518: truncate dc/d13/d1f/d6e/f81 3622910 0 2024-05-12T04:01:15.884 INFO:tasks.workunit.client.1.smithi033.stdout:8/449: rmdir d1/d16/d27/d4d 39 2024-05-12T04:01:15.889 INFO:tasks.workunit.client.0.smithi032.stdout:3/365: write d15/d33/f4c [778352,8600] 0 2024-05-12T04:01:15.891 INFO:tasks.workunit.client.1.smithi033.stdout:9/756: mkdir d2/d4/d29/d110 0 2024-05-12T04:01:15.892 INFO:tasks.workunit.client.1.smithi033.stdout:2/519: dwrite dc/d17/d27/d2f/f90 [0,4194304] 0 2024-05-12T04:01:15.895 INFO:tasks.workunit.client.1.smithi033.stdout:8/450: fsync d1/d16/d20/f43 0 2024-05-12T04:01:15.895 INFO:tasks.workunit.client.1.smithi033.stdout:3/520: dwrite db/f54 [0,4194304] 0 2024-05-12T04:01:15.895 INFO:tasks.workunit.client.1.smithi033.stdout:2/520: write dc/d17/d32/f5c [417351,97850] 0 2024-05-12T04:01:15.899 INFO:tasks.workunit.client.0.smithi032.stdout:9/269: dwrite d0/f28 [0,4194304] 0 2024-05-12T04:01:15.902 INFO:tasks.workunit.client.1.smithi033.stdout:7/688: truncate de/d80/f98 3309814 0 2024-05-12T04:01:15.906 INFO:tasks.workunit.client.1.smithi033.stdout:4/536: sync 2024-05-12T04:01:15.909 INFO:tasks.workunit.client.0.smithi032.stdout:9/270: creat d0/d3/d6/d7/d27/f57 x:0 0 0 2024-05-12T04:01:15.914 INFO:tasks.workunit.client.0.smithi032.stdout:9/271: write d0/d3/d6/d7/d27/f48 [698652,114009] 0 2024-05-12T04:01:15.916 INFO:tasks.workunit.client.0.smithi032.stdout:9/272: readlink d0/d3/d6/d7/da/d17/l44 0 2024-05-12T04:01:15.917 INFO:tasks.workunit.client.0.smithi032.stdout:9/273: readlink d0/d3/d6/d7/da/l11 0 2024-05-12T04:01:15.918 INFO:tasks.workunit.client.0.smithi032.stdout:9/274: write d0/d3/d6/d7/d2a/d46/d13/f3c [1147224,39131] 0 2024-05-12T04:01:15.921 INFO:tasks.workunit.client.1.smithi033.stdout:3/521: rmdir db/d59 39 2024-05-12T04:01:15.922 INFO:tasks.workunit.client.0.smithi032.stdout:3/366: truncate d15/d24/d2a/f63 2741195 0 2024-05-12T04:01:15.924 INFO:tasks.workunit.client.0.smithi032.stdout:3/367: write d15/d20/d22/d4f/f53 [2275606,79635] 0 2024-05-12T04:01:15.931 INFO:tasks.workunit.client.1.smithi033.stdout:7/689: creat de/d2b/d6c/db5/dd0/ff2 x:0 0 0 2024-05-12T04:01:15.931 INFO:tasks.workunit.client.1.smithi033.stdout:0/547: link d6/la2 d6/dc/d58/d6a/d94/lba 0 2024-05-12T04:01:15.931 INFO:tasks.workunit.client.1.smithi033.stdout:6/520: rename d4/d1a/d67/d68/d88 to d4/dad 0 2024-05-12T04:01:15.931 INFO:tasks.workunit.client.1.smithi033.stdout:7/690: dread de/df/d17/d1e/dcc/fce [0,4194304] 0 2024-05-12T04:01:15.932 INFO:tasks.workunit.client.0.smithi032.stdout:3/368: creat d15/d24/f7b x:0 0 0 2024-05-12T04:01:15.932 INFO:tasks.workunit.client.1.smithi033.stdout:6/521: dread - d4/d1a/f91 zero size 2024-05-12T04:01:15.932 INFO:tasks.workunit.client.1.smithi033.stdout:9/757: truncate d2/d4/d29/f109 1574122 0 2024-05-12T04:01:15.933 INFO:tasks.workunit.client.1.smithi033.stdout:3/522: rmdir db/d17/d3a/d67/d76 39 2024-05-12T04:01:15.934 INFO:tasks.workunit.client.1.smithi033.stdout:9/758: fdatasync d2/d4/d2b/d49/ff0 0 2024-05-12T04:01:15.935 INFO:tasks.workunit.client.0.smithi032.stdout:3/369: write d15/d20/d26/f52 [442942,56983] 0 2024-05-12T04:01:15.937 INFO:tasks.workunit.client.1.smithi033.stdout:4/537: mkdir dd/d11/da5 0 2024-05-12T04:01:15.941 INFO:tasks.workunit.client.1.smithi033.stdout:6/522: dwrite d4/dad/fa7 [0,4194304] 0 2024-05-12T04:01:15.942 INFO:tasks.workunit.client.1.smithi033.stdout:7/691: rename de/df/d7a/l8c to de/d20/d38/d7f/lf3 0 2024-05-12T04:01:15.945 INFO:tasks.workunit.client.1.smithi033.stdout:4/538: mknod dd/d11/d5b/ca6 0 2024-05-12T04:01:15.948 INFO:tasks.workunit.client.1.smithi033.stdout:9/759: rename d2/d4/d2b/d33/fe2 to d2/f111 0 2024-05-12T04:01:15.949 INFO:tasks.workunit.client.1.smithi033.stdout:6/523: creat d4/fae x:0 0 0 2024-05-12T04:01:15.950 INFO:tasks.workunit.client.1.smithi033.stdout:6/524: read - d4/fae zero size 2024-05-12T04:01:15.951 INFO:tasks.workunit.client.1.smithi033.stdout:4/539: rename dd/d11/d31 to dd/d7a/d87/d7e/da7 0 2024-05-12T04:01:15.953 INFO:tasks.workunit.client.1.smithi033.stdout:6/525: mkdir d4/d1a/d67/d6f/daf 0 2024-05-12T04:01:15.955 INFO:tasks.workunit.client.1.smithi033.stdout:6/526: read - d4/d1a/d67/d68/f72 zero size 2024-05-12T04:01:15.955 INFO:tasks.workunit.client.1.smithi033.stdout:9/760: dwrite d2/d4/d29/d3c/d52/fba [0,4194304] 0 2024-05-12T04:01:15.956 INFO:tasks.workunit.client.1.smithi033.stdout:6/527: stat d4/d1a/d3c/f65 0 2024-05-12T04:01:15.957 INFO:tasks.workunit.client.1.smithi033.stdout:9/761: chown d2/d4/d29/d3c/d52/c92 360325 1 2024-05-12T04:01:15.960 INFO:tasks.workunit.client.1.smithi033.stdout:6/528: rename d4/d5/c45 to d4/d1a/d67/d6f/cb0 0 2024-05-12T04:01:15.961 INFO:tasks.workunit.client.1.smithi033.stdout:9/762: symlink d2/d4/d2b/d78/d76/l112 0 2024-05-12T04:01:15.961 INFO:tasks.workunit.client.1.smithi033.stdout:6/529: truncate d4/d1a/d3c/f75 1120066 0 2024-05-12T04:01:15.964 INFO:tasks.workunit.client.1.smithi033.stdout:9/763: creat d2/d4/d29/d3c/f113 x:0 0 0 2024-05-12T04:01:15.965 INFO:tasks.workunit.client.1.smithi033.stdout:6/530: dread d4/d5/f5c [0,4194304] 0 2024-05-12T04:01:15.967 INFO:tasks.workunit.client.0.smithi032.stdout:0/392: sync 2024-05-12T04:01:15.967 INFO:tasks.workunit.client.0.smithi032.stdout:2/343: sync 2024-05-12T04:01:15.968 INFO:tasks.workunit.client.1.smithi033.stdout:1/476: sync 2024-05-12T04:01:15.968 INFO:tasks.workunit.client.1.smithi033.stdout:1/477: read - d11/d1c/f94 zero size 2024-05-12T04:01:15.974 INFO:tasks.workunit.client.1.smithi033.stdout:4/540: dread dd/f36 [0,4194304] 0 2024-05-12T04:01:15.975 INFO:tasks.workunit.client.1.smithi033.stdout:5/448: truncate d3/d15/d16/f44 3491659 0 2024-05-12T04:01:15.976 INFO:tasks.workunit.client.0.smithi032.stdout:2/344: unlink d1/fa 0 2024-05-12T04:01:15.976 INFO:tasks.workunit.client.1.smithi033.stdout:5/449: read - d3/d15/d16/d45/d62/f7a zero size 2024-05-12T04:01:15.981 INFO:tasks.workunit.client.1.smithi033.stdout:1/478: rmdir d11 39 2024-05-12T04:01:15.983 INFO:tasks.workunit.client.0.smithi032.stdout:2/345: dread d1/d13/d3d/d41/d2c/f37 [0,4194304] 0 2024-05-12T04:01:15.987 INFO:tasks.workunit.client.0.smithi032.stdout:0/393: mkdir df/d30/d8d/d8f 0 2024-05-12T04:01:15.988 INFO:tasks.workunit.client.0.smithi032.stdout:0/394: write df/d10/d3d/d4a/f29 [514926,73478] 0 2024-05-12T04:01:15.995 INFO:tasks.workunit.client.1.smithi033.stdout:5/450: mkdir d3/d12/d29/d2b/d8e 0 2024-05-12T04:01:16.000 INFO:tasks.workunit.client.1.smithi033.stdout:2/521: write dc/d17/d27/d2f/d49/d59/f80 [810917,37734] 0 2024-05-12T04:01:16.001 INFO:tasks.workunit.client.1.smithi033.stdout:2/522: chown dc/d17/d27 1017817 1 2024-05-12T04:01:16.001 INFO:tasks.workunit.client.1.smithi033.stdout:1/479: fdatasync d11/d1c/d59/f86 0 2024-05-12T04:01:16.002 INFO:tasks.workunit.client.0.smithi032.stdout:2/346: unlink d1/d4d/d5e/c68 0 2024-05-12T04:01:16.002 INFO:tasks.workunit.client.0.smithi032.stdout:2/347: stat d1/d34/d3e/f45 0 2024-05-12T04:01:16.006 INFO:tasks.workunit.client.1.smithi033.stdout:4/541: write dd/f62 [347382,92672] 0 2024-05-12T04:01:16.006 INFO:tasks.workunit.client.1.smithi033.stdout:8/451: truncate d1/d16/d20/f2f 4333963 0 2024-05-12T04:01:16.010 INFO:tasks.workunit.client.0.smithi032.stdout:0/395: creat df/d30/d32/d60/f90 x:0 0 0 2024-05-12T04:01:16.012 INFO:tasks.workunit.client.0.smithi032.stdout:2/348: mknod d1/c79 0 2024-05-12T04:01:16.013 INFO:tasks.workunit.client.1.smithi033.stdout:2/523: mknod dc/d17/d27/d2f/d49/d59/cad 0 2024-05-12T04:01:16.015 INFO:tasks.workunit.client.1.smithi033.stdout:8/452: dwrite d1/d16/f75 [0,4194304] 0 2024-05-12T04:01:16.016 INFO:tasks.workunit.client.0.smithi032.stdout:3/370: dwrite d15/d1b/f34 [0,4194304] 0 2024-05-12T04:01:16.019 INFO:tasks.workunit.client.1.smithi033.stdout:4/542: rename dd/f4f to dd/d69/da4/fa8 0 2024-05-12T04:01:16.019 INFO:tasks.workunit.client.1.smithi033.stdout:0/548: dwrite d6/dc/d58/d6a/f7e [0,4194304] 0 2024-05-12T04:01:16.020 INFO:tasks.workunit.client.1.smithi033.stdout:3/523: write db/d17/d3e/d65/d7d/f94 [34469,44961] 0 2024-05-12T04:01:16.022 INFO:tasks.workunit.client.1.smithi033.stdout:4/543: write dd/f32 [277762,116772] 0 2024-05-12T04:01:16.022 INFO:tasks.workunit.client.1.smithi033.stdout:3/524: dread - db/f90 zero size 2024-05-12T04:01:16.025 INFO:tasks.workunit.client.1.smithi033.stdout:8/453: dread d1/d16/d20/f43 [0,4194304] 0 2024-05-12T04:01:16.027 INFO:tasks.workunit.client.1.smithi033.stdout:1/480: mkdir d11/d17/d9e 0 2024-05-12T04:01:16.028 INFO:tasks.workunit.client.1.smithi033.stdout:1/481: chown d11/c2f 60104733 1 2024-05-12T04:01:16.037 INFO:tasks.workunit.client.0.smithi032.stdout:2/349: link d1/c79 d1/d34/d3e/d67/c7a 0 2024-05-12T04:01:16.039 INFO:tasks.workunit.client.0.smithi032.stdout:0/396: dread df/d10/d48/d70/f76 [0,4194304] 0 2024-05-12T04:01:16.047 INFO:tasks.workunit.client.0.smithi032.stdout:0/397: mknod df/d22/d58/c91 0 2024-05-12T04:01:16.048 INFO:tasks.workunit.client.1.smithi033.stdout:6/531: dwrite d4/d1a/f91 [0,4194304] 0 2024-05-12T04:01:16.050 INFO:tasks.workunit.client.1.smithi033.stdout:9/764: dwrite d2/d4/d2b/d6d/fad [0,4194304] 0 2024-05-12T04:01:16.051 INFO:tasks.workunit.client.0.smithi032.stdout:7/342: sync 2024-05-12T04:01:16.051 INFO:tasks.workunit.client.0.smithi032.stdout:4/259: sync 2024-05-12T04:01:16.051 INFO:tasks.workunit.client.0.smithi032.stdout:6/328: sync 2024-05-12T04:01:16.051 INFO:tasks.workunit.client.0.smithi032.stdout:3/371: link d15/d24/d2a/l31 d15/l7c 0 2024-05-12T04:01:16.051 INFO:tasks.workunit.client.1.smithi033.stdout:6/532: readlink d4/d1a/d4e/d4f/d94/la2 0 2024-05-12T04:01:16.053 INFO:tasks.workunit.client.0.smithi032.stdout:3/372: write d15/d1b/f34 [1193715,89096] 0 2024-05-12T04:01:16.053 INFO:tasks.workunit.client.0.smithi032.stdout:4/260: dread d0/d3/df/f26 [0,4194304] 0 2024-05-12T04:01:16.054 INFO:tasks.workunit.client.1.smithi033.stdout:5/451: symlink d3/d12/d25/d26/d2e/d85/l8f 0 2024-05-12T04:01:16.056 INFO:tasks.workunit.client.1.smithi033.stdout:0/549: symlink d6/dc/d39/lbb 0 2024-05-12T04:01:16.057 INFO:tasks.workunit.client.1.smithi033.stdout:0/550: fdatasync d6/dc/d58/f86 0 2024-05-12T04:01:16.057 INFO:tasks.workunit.client.0.smithi032.stdout:6/329: dread d7/d9/d60/f68 [0,4194304] 0 2024-05-12T04:01:16.058 INFO:tasks.workunit.client.1.smithi033.stdout:0/551: readlink d6/db/l10 0 2024-05-12T04:01:16.061 INFO:tasks.workunit.client.0.smithi032.stdout:3/373: dwrite fd [0,4194304] 0 2024-05-12T04:01:16.063 INFO:tasks.workunit.client.1.smithi033.stdout:4/544: unlink dd/d11/c67 0 2024-05-12T04:01:16.065 INFO:tasks.workunit.client.1.smithi033.stdout:4/545: truncate dd/d7a/d87/d7e/da7/f6c 1215188 0 2024-05-12T04:01:16.065 INFO:tasks.workunit.client.0.smithi032.stdout:7/343: mkdir dd/d1b/d43/d74 0 2024-05-12T04:01:16.066 INFO:tasks.workunit.client.1.smithi033.stdout:3/525: truncate db/d17/d3e/f48 965140 0 2024-05-12T04:01:16.070 INFO:tasks.workunit.client.0.smithi032.stdout:4/261: rename d0/d3 to d0/d3/d44 22 2024-05-12T04:01:16.072 INFO:tasks.workunit.client.0.smithi032.stdout:4/262: fsync d0/f3d 0 2024-05-12T04:01:16.073 INFO:tasks.workunit.client.0.smithi032.stdout:4/263: write d0/d3/d24/f42 [5156816,124432] 0 2024-05-12T04:01:16.086 INFO:tasks.workunit.client.1.smithi033.stdout:6/533: creat d4/d1a/d67/d6f/fb1 x:0 0 0 2024-05-12T04:01:16.087 INFO:tasks.workunit.client.0.smithi032.stdout:7/344: mkdir dd/d1b/d1e/d45/d55/d60/d75 0 2024-05-12T04:01:16.091 INFO:tasks.workunit.client.0.smithi032.stdout:0/398: creat df/f92 x:0 0 0 2024-05-12T04:01:16.092 INFO:tasks.workunit.client.0.smithi032.stdout:0/399: write df/d28/f89 [555093,49291] 0 2024-05-12T04:01:16.093 INFO:tasks.workunit.client.0.smithi032.stdout:2/350: write d1/d13/d3d/d41/f2a [4448346,54894] 0 2024-05-12T04:01:16.094 INFO:tasks.workunit.client.0.smithi032.stdout:0/400: chown df/d10/d48/d70 4493 1 2024-05-12T04:01:16.095 INFO:tasks.workunit.client.1.smithi033.stdout:5/452: dwrite d3/d15/d16/f51 [0,4194304] 0 2024-05-12T04:01:16.096 INFO:tasks.workunit.client.0.smithi032.stdout:0/401: truncate df/d10/d3d/d4a/f65 439146 0 2024-05-12T04:01:16.096 INFO:tasks.workunit.client.0.smithi032.stdout:4/264: symlink d0/d3/d31/l45 0 2024-05-12T04:01:16.097 INFO:tasks.workunit.client.1.smithi033.stdout:2/524: dwrite dc/d17/f63 [0,4194304] 0 2024-05-12T04:01:16.098 INFO:tasks.workunit.client.1.smithi033.stdout:5/453: chown d3/d12/d25/d26/d2e/d54/c5d 203 1 2024-05-12T04:01:16.099 INFO:tasks.workunit.client.1.smithi033.stdout:3/526: creat db/d17/d25/d32/d3c/d44/fbd x:0 0 0 2024-05-12T04:01:16.102 INFO:tasks.workunit.client.1.smithi033.stdout:7/692: link de/d20/d38/d7f/lf3 de/df/d17/d1e/dcc/lf4 0 2024-05-12T04:01:16.102 INFO:tasks.workunit.client.0.smithi032.stdout:0/402: dwrite f9 [8388608,4194304] 0 2024-05-12T04:01:16.105 INFO:tasks.workunit.client.0.smithi032.stdout:7/345: rmdir dd/d1b/d1e/d45/d55/d60 39 2024-05-12T04:01:16.115 INFO:tasks.workunit.client.0.smithi032.stdout:4/265: stat d0/d3/df/f30 0 2024-05-12T04:01:16.119 INFO:tasks.workunit.client.1.smithi033.stdout:4/546: symlink dd/d11/da5/la9 0 2024-05-12T04:01:16.127 INFO:tasks.workunit.client.0.smithi032.stdout:4/266: symlink d0/d39/l46 0 2024-05-12T04:01:16.129 INFO:tasks.workunit.client.0.smithi032.stdout:3/374: dwrite d15/d24/f51 [0,4194304] 0 2024-05-12T04:01:16.131 INFO:tasks.workunit.client.0.smithi032.stdout:6/330: getdents d7 0 2024-05-12T04:01:16.133 INFO:tasks.workunit.client.0.smithi032.stdout:0/403: mknod df/d30/d32/c93 0 2024-05-12T04:01:16.139 INFO:tasks.workunit.client.0.smithi032.stdout:7/346: truncate dd/f1d 1084234 0 2024-05-12T04:01:16.139 INFO:tasks.workunit.client.0.smithi032.stdout:7/347: chown dd/d1b/d1e/c53 792205301 1 2024-05-12T04:01:16.140 INFO:tasks.workunit.client.1.smithi033.stdout:3/527: symlink db/d17/d25/d32/d3c/d44/lbe 0 2024-05-12T04:01:16.140 INFO:tasks.workunit.client.0.smithi032.stdout:7/348: chown dd/d1b/f49 13556836 1 2024-05-12T04:01:16.142 INFO:tasks.workunit.client.1.smithi033.stdout:0/552: write d6/f23 [499690,5609] 0 2024-05-12T04:01:16.142 INFO:tasks.workunit.client.1.smithi033.stdout:4/547: symlink dd/d7a/d87/laa 0 2024-05-12T04:01:16.144 INFO:tasks.workunit.client.1.smithi033.stdout:9/765: dwrite d2/d4/d2b/d6d/d2f/d5f/f97 [0,4194304] 0 2024-05-12T04:01:16.144 INFO:tasks.workunit.client.1.smithi033.stdout:8/454: getdents d1/d16/d27/d7c 0 2024-05-12T04:01:16.145 INFO:tasks.workunit.client.1.smithi033.stdout:4/548: write dd/f13 [1389900,42314] 0 2024-05-12T04:01:16.146 INFO:tasks.workunit.client.1.smithi033.stdout:8/455: chown d1/d16/l1a 24 1 2024-05-12T04:01:16.148 INFO:tasks.workunit.client.1.smithi033.stdout:9/766: write d2/d4/d2b/d78/d76/ff8 [583914,20898] 0 2024-05-12T04:01:16.148 INFO:tasks.workunit.client.1.smithi033.stdout:8/456: readlink d1/d16/d27/l52 0 2024-05-12T04:01:16.148 INFO:tasks.workunit.client.0.smithi032.stdout:6/331: fdatasync d7/d9/d48/d67/d1a/d20/f42 0 2024-05-12T04:01:16.149 INFO:tasks.workunit.client.1.smithi033.stdout:6/534: creat d4/d1a/fb2 x:0 0 0 2024-05-12T04:01:16.150 INFO:tasks.workunit.client.1.smithi033.stdout:7/693: dread de/d2b/d6c/f39 [0,4194304] 0 2024-05-12T04:01:16.150 INFO:tasks.workunit.client.0.smithi032.stdout:7/349: dwrite dd/d1b/f4b [0,4194304] 0 2024-05-12T04:01:16.151 INFO:tasks.workunit.client.0.smithi032.stdout:6/332: readlink d7/d9/d48/d67/d63/l69 0 2024-05-12T04:01:16.155 INFO:tasks.workunit.client.0.smithi032.stdout:7/350: dread - dd/d1b/d43/d5e/f5a zero size 2024-05-12T04:01:16.157 INFO:tasks.workunit.client.1.smithi033.stdout:3/528: symlink db/d17/d3a/d8a/lbf 0 2024-05-12T04:01:16.158 INFO:tasks.workunit.client.0.smithi032.stdout:4/267: symlink d0/d3/l47 0 2024-05-12T04:01:16.161 INFO:tasks.workunit.client.0.smithi032.stdout:3/375: mkdir d15/d20/d26/d6a/d6e/d7d 0 2024-05-12T04:01:16.165 INFO:tasks.workunit.client.0.smithi032.stdout:0/404: dread df/d10/f1b [0,4194304] 0 2024-05-12T04:01:16.165 INFO:tasks.workunit.client.1.smithi033.stdout:4/549: symlink dd/d11/da5/lab 0 2024-05-12T04:01:16.166 INFO:tasks.workunit.client.1.smithi033.stdout:4/550: write dd/d11/d5b/f86 [955577,89903] 0 2024-05-12T04:01:16.167 INFO:tasks.workunit.client.1.smithi033.stdout:6/535: chown d4/f2f 28752 1 2024-05-12T04:01:16.168 INFO:tasks.workunit.client.1.smithi033.stdout:1/482: sync 2024-05-12T04:01:16.174 INFO:tasks.workunit.client.0.smithi032.stdout:6/333: symlink d7/d9/d48/d67/d63/l72 0 2024-05-12T04:01:16.174 INFO:tasks.workunit.client.1.smithi033.stdout:9/767: read d2/dd/d39/f59 [8028547,88701] 0 2024-05-12T04:01:16.174 INFO:tasks.workunit.client.1.smithi033.stdout:1/483: write d11/f2d [1259765,79537] 0 2024-05-12T04:01:16.174 INFO:tasks.workunit.client.1.smithi033.stdout:1/484: dread - d11/d1c/f94 zero size 2024-05-12T04:01:16.175 INFO:tasks.workunit.client.1.smithi033.stdout:8/457: creat d1/d4/d1d/d36/d3c/d64/f8d x:0 0 0 2024-05-12T04:01:16.175 INFO:tasks.workunit.client.1.smithi033.stdout:4/551: dread dd/d11/d5b/d84/f9e [4194304,4194304] 0 2024-05-12T04:01:16.176 INFO:tasks.workunit.client.1.smithi033.stdout:7/694: rename de/df/d9a/db7/cbf to de/d20/d64/cf5 0 2024-05-12T04:01:16.176 INFO:tasks.workunit.client.1.smithi033.stdout:8/458: dread d1/d16/d27/d28/f57 [0,4194304] 0 2024-05-12T04:01:16.178 INFO:tasks.workunit.client.0.smithi032.stdout:3/376: rename d15/d24/d2a/c3a to d15/d20/d22/d4f/c7e 0 2024-05-12T04:01:16.181 INFO:tasks.workunit.client.0.smithi032.stdout:7/351: dread dd/d1b/d1e/f38 [0,4194304] 0 2024-05-12T04:01:16.182 INFO:tasks.workunit.client.0.smithi032.stdout:2/351: write d1/d13/f2f [1477499,131042] 0 2024-05-12T04:01:16.182 INFO:tasks.workunit.client.1.smithi033.stdout:5/454: link d3/d12/d29/c8a d3/d12/d25/d26/d2e/d54/d57/d73/c90 0 2024-05-12T04:01:16.182 INFO:tasks.workunit.client.1.smithi033.stdout:5/455: stat d3/fc 0 2024-05-12T04:01:16.183 INFO:tasks.workunit.client.1.smithi033.stdout:5/456: chown d3/d12/d25/d26 70022 1 2024-05-12T04:01:16.186 INFO:tasks.workunit.client.0.smithi032.stdout:6/334: rename d7/f38 to d7/d9/d48/d67/d1a/f73 0 2024-05-12T04:01:16.189 INFO:tasks.workunit.client.0.smithi032.stdout:0/405: creat df/d30/d8d/d8f/f94 x:0 0 0 2024-05-12T04:01:16.190 INFO:tasks.workunit.client.0.smithi032.stdout:0/406: write df/d30/d36/f6d [1400868,126479] 0 2024-05-12T04:01:16.191 INFO:tasks.workunit.client.0.smithi032.stdout:7/352: write dd/f36 [605699,71786] 0 2024-05-12T04:01:16.193 INFO:tasks.workunit.client.0.smithi032.stdout:2/352: write d1/d13/d3d/f60 [1405369,72499] 0 2024-05-12T04:01:16.195 INFO:tasks.workunit.client.0.smithi032.stdout:4/268: creat d0/d3/d12/f48 x:0 0 0 2024-05-12T04:01:16.197 INFO:tasks.workunit.client.1.smithi033.stdout:8/459: chown d1/d4/d1d/f58 693 1 2024-05-12T04:01:16.200 INFO:tasks.workunit.client.1.smithi033.stdout:3/529: unlink db/l10 0 2024-05-12T04:01:16.201 INFO:tasks.workunit.client.0.smithi032.stdout:3/377: symlink d15/d1b/d43/l7f 0 2024-05-12T04:01:16.203 INFO:tasks.workunit.client.1.smithi033.stdout:0/553: symlink d6/d11/d57/lbc 0 2024-05-12T04:01:16.203 INFO:tasks.workunit.client.1.smithi033.stdout:9/768: write d2/d4/d2b/f4b [1796275,2919] 0 2024-05-12T04:01:16.205 INFO:tasks.workunit.client.1.smithi033.stdout:5/457: unlink d3/d12/d29/d2b/l4e 0 2024-05-12T04:01:16.206 INFO:tasks.workunit.client.1.smithi033.stdout:5/458: read - d3/d12/d25/d26/d2e/d54/d57/d70/f87 zero size 2024-05-12T04:01:16.207 INFO:tasks.workunit.client.1.smithi033.stdout:6/536: creat d4/d6e/fb3 x:0 0 0 2024-05-12T04:01:16.212 INFO:tasks.workunit.client.1.smithi033.stdout:1/485: creat d11/d17/d9e/f9f x:0 0 0 2024-05-12T04:01:16.216 INFO:tasks.workunit.client.0.smithi032.stdout:2/353: dread - d1/d34/f53 zero size 2024-05-12T04:01:16.219 INFO:tasks.workunit.client.1.smithi033.stdout:6/537: dread d4/d5/f2d [0,4194304] 0 2024-05-12T04:01:16.220 INFO:tasks.workunit.client.1.smithi033.stdout:6/538: write d4/d1a/d6c/f85 [362152,51774] 0 2024-05-12T04:01:16.221 INFO:tasks.workunit.client.1.smithi033.stdout:3/530: mkdir db/d17/d3a/dc0 0 2024-05-12T04:01:16.222 INFO:tasks.workunit.client.0.smithi032.stdout:6/335: write d7/d9/d48/d67/f18 [1129216,29128] 0 2024-05-12T04:01:16.224 INFO:tasks.workunit.client.1.smithi033.stdout:2/525: sync 2024-05-12T04:01:16.225 INFO:tasks.workunit.client.1.smithi033.stdout:9/769: creat d2/d4/d2b/d78/f114 x:0 0 0 2024-05-12T04:01:16.226 INFO:tasks.workunit.client.0.smithi032.stdout:0/407: write df/d30/d36/f41 [1377833,43150] 0 2024-05-12T04:01:16.229 INFO:tasks.workunit.client.1.smithi033.stdout:1/486: mkdir d11/d17/d27/d36/d3f/d4a/da0 0 2024-05-12T04:01:16.231 INFO:tasks.workunit.client.1.smithi033.stdout:4/552: creat dd/d11/fac x:0 0 0 2024-05-12T04:01:16.233 INFO:tasks.workunit.client.1.smithi033.stdout:6/539: write d4/d5/f5c [4570722,20130] 0 2024-05-12T04:01:16.236 INFO:tasks.workunit.client.1.smithi033.stdout:6/540: read d4/d1a/d4e/fa0 [1998728,21229] 0 2024-05-12T04:01:16.236 INFO:tasks.workunit.client.1.smithi033.stdout:1/487: dwrite d11/d17/f18 [0,4194304] 0 2024-05-12T04:01:16.238 INFO:tasks.workunit.client.1.smithi033.stdout:5/459: truncate d3/fb 409558 0 2024-05-12T04:01:16.239 INFO:tasks.workunit.client.1.smithi033.stdout:6/541: write d4/d1a/d67/d68/f8f [865085,27727] 0 2024-05-12T04:01:16.240 INFO:tasks.workunit.client.1.smithi033.stdout:3/531: dread db/d17/f36 [0,4194304] 0 2024-05-12T04:01:16.241 INFO:tasks.workunit.client.1.smithi033.stdout:1/488: readlink d11/d17/d27/l82 0 2024-05-12T04:01:16.245 INFO:tasks.workunit.client.0.smithi032.stdout:2/354: stat d1/f4 0 2024-05-12T04:01:16.247 INFO:tasks.workunit.client.1.smithi033.stdout:4/553: creat dd/d11/d5b/fad x:0 0 0 2024-05-12T04:01:16.247 INFO:tasks.workunit.client.1.smithi033.stdout:0/554: creat d6/dc/d58/fbd x:0 0 0 2024-05-12T04:01:16.249 INFO:tasks.workunit.client.1.smithi033.stdout:0/555: dread - d6/dc/f54 zero size 2024-05-12T04:01:16.250 INFO:tasks.workunit.client.1.smithi033.stdout:2/526: rename dc/d13/d39/c87 to dc/d17/d74/cae 0 2024-05-12T04:01:16.252 INFO:tasks.workunit.client.0.smithi032.stdout:3/378: symlink d15/d20/l80 0 2024-05-12T04:01:16.253 INFO:tasks.workunit.client.1.smithi033.stdout:9/770: dwrite d2/d70/d7b/d94/de3/ff7 [4194304,4194304] 0 2024-05-12T04:01:16.256 INFO:tasks.workunit.client.0.smithi032.stdout:6/336: rename d7/d9/d48/d67/d1a/d20/f34 to d7/d9/d48/d67/d25/f74 0 2024-05-12T04:01:16.258 INFO:tasks.workunit.client.1.smithi033.stdout:8/460: getdents d1 0 2024-05-12T04:01:16.259 INFO:tasks.workunit.client.1.smithi033.stdout:8/461: chown d1/d16/c41 288 1 2024-05-12T04:01:16.260 INFO:tasks.workunit.client.1.smithi033.stdout:3/532: symlink db/d17/lc1 0 2024-05-12T04:01:16.262 INFO:tasks.workunit.client.0.smithi032.stdout:7/353: creat dd/d1b/d1e/d45/d55/d60/d75/f76 x:0 0 0 2024-05-12T04:01:16.263 INFO:tasks.workunit.client.1.smithi033.stdout:1/489: creat d11/d1c/d43/fa1 x:0 0 0 2024-05-12T04:01:16.264 INFO:tasks.workunit.client.1.smithi033.stdout:1/490: chown d11/d17/d27/d36/d46/f66 917565337 1 2024-05-12T04:01:16.268 INFO:tasks.workunit.client.1.smithi033.stdout:0/556: mknod d6/dc/d96/d9d/cbe 0 2024-05-12T04:01:16.268 INFO:tasks.workunit.client.1.smithi033.stdout:0/557: fsync d6/dc/d58/d6a/f7e 0 2024-05-12T04:01:16.269 INFO:tasks.workunit.client.1.smithi033.stdout:0/558: stat d6/db/d2c/l43 0 2024-05-12T04:01:16.270 INFO:tasks.workunit.client.0.smithi032.stdout:4/269: link d0/f3d d0/d3/f49 0 2024-05-12T04:01:16.273 INFO:tasks.workunit.client.1.smithi033.stdout:2/527: symlink dc/d13/d1c/laf 0 2024-05-12T04:01:16.275 INFO:tasks.workunit.client.1.smithi033.stdout:2/528: truncate dc/d13/d1f/f9b 226844 0 2024-05-12T04:01:16.278 INFO:tasks.workunit.client.0.smithi032.stdout:0/408: write df/d28/f54 [766447,109099] 0 2024-05-12T04:01:16.279 INFO:tasks.workunit.client.1.smithi033.stdout:9/771: write d2/f111 [620890,124337] 0 2024-05-12T04:01:16.279 INFO:tasks.workunit.client.1.smithi033.stdout:9/772: fdatasync d2/d4/d29/d3c/fe9 0 2024-05-12T04:01:16.283 INFO:tasks.workunit.client.1.smithi033.stdout:7/695: sync 2024-05-12T04:01:16.287 INFO:tasks.workunit.client.0.smithi032.stdout:4/270: rmdir d0 39 2024-05-12T04:01:16.287 INFO:tasks.workunit.client.0.smithi032.stdout:7/354: dread dd/f13 [0,4194304] 0 2024-05-12T04:01:16.288 INFO:tasks.workunit.client.1.smithi033.stdout:1/491: rmdir d11/d1c/d43 39 2024-05-12T04:01:16.292 INFO:tasks.workunit.client.0.smithi032.stdout:0/409: mknod df/d10/d3d/d4a/c95 0 2024-05-12T04:01:16.292 INFO:tasks.workunit.client.1.smithi033.stdout:0/559: mknod d6/dc/d58/d91/cbf 0 2024-05-12T04:01:16.293 INFO:tasks.workunit.client.0.smithi032.stdout:2/355: write d1/f24 [2035115,125820] 0 2024-05-12T04:01:16.294 INFO:tasks.workunit.client.1.smithi033.stdout:0/560: fdatasync d6/d11/f33 0 2024-05-12T04:01:16.295 INFO:tasks.workunit.client.1.smithi033.stdout:5/460: rename d3/d27/f6a to d3/d15/d16/d45/d62/d7f/d84/f91 0 2024-05-12T04:01:16.296 INFO:tasks.workunit.client.1.smithi033.stdout:6/542: dwrite d4/d5/f73 [0,4194304] 0 2024-05-12T04:01:16.297 INFO:tasks.workunit.client.1.smithi033.stdout:0/561: write d6/db/d2c/d82/fb4 [863988,1569] 0 2024-05-12T04:01:16.300 INFO:tasks.workunit.client.0.smithi032.stdout:3/379: link d15/f54 d15/d24/f81 0 2024-05-12T04:01:16.300 INFO:tasks.workunit.client.0.smithi032.stdout:4/271: dread d0/d3/d31/f38 [0,4194304] 0 2024-05-12T04:01:16.302 INFO:tasks.workunit.client.1.smithi033.stdout:2/529: dread dc/f3f [0,4194304] 0 2024-05-12T04:01:16.303 INFO:tasks.workunit.client.0.smithi032.stdout:2/356: symlink d1/d34/d3e/l7b 0 2024-05-12T04:01:16.304 INFO:tasks.workunit.client.1.smithi033.stdout:9/773: mkdir d2/d4/d29/d32/db7/d115 0 2024-05-12T04:01:16.306 INFO:tasks.workunit.client.1.smithi033.stdout:8/462: mkdir d1/d16/d27/d4d/d51/d8e 0 2024-05-12T04:01:16.307 INFO:tasks.workunit.client.1.smithi033.stdout:8/463: fsync d1/d16/d27/f85 0 2024-05-12T04:01:16.307 INFO:tasks.workunit.client.0.smithi032.stdout:7/355: creat dd/d6c/f77 x:0 0 0 2024-05-12T04:01:16.308 INFO:tasks.workunit.client.1.smithi033.stdout:3/533: rmdir db/d59 39 2024-05-12T04:01:16.310 INFO:tasks.workunit.client.0.smithi032.stdout:7/356: dread - dd/d1b/d43/d5e/d37/f54 zero size 2024-05-12T04:01:16.310 INFO:tasks.workunit.client.0.smithi032.stdout:4/272: dwrite d0/f13 [0,4194304] 0 2024-05-12T04:01:16.316 INFO:tasks.workunit.client.0.smithi032.stdout:0/410: link df/d28/f89 df/d10/d48/f96 0 2024-05-12T04:01:16.316 INFO:tasks.workunit.client.0.smithi032.stdout:8/400: sync 2024-05-12T04:01:16.317 INFO:tasks.workunit.client.1.smithi033.stdout:3/534: dwrite db/d17/d3e/d65/d7d/f94 [0,4194304] 0 2024-05-12T04:01:16.319 INFO:tasks.workunit.client.0.smithi032.stdout:4/273: dwrite d0/d3/df/f26 [0,4194304] 0 2024-05-12T04:01:16.319 INFO:tasks.workunit.client.0.smithi032.stdout:0/411: chown df/d30/d8d/d8f/f94 15557 1 2024-05-12T04:01:16.321 INFO:tasks.workunit.client.1.smithi033.stdout:1/492: write d11/d1c/d43/f85 [2314685,116051] 0 2024-05-12T04:01:16.323 INFO:tasks.workunit.client.0.smithi032.stdout:2/357: symlink d1/d13/d6b/l7c 0 2024-05-12T04:01:16.325 INFO:tasks.workunit.client.0.smithi032.stdout:2/358: chown d1/d13/d3d/d41/d39/c43 7893 1 2024-05-12T04:01:16.333 INFO:tasks.workunit.client.1.smithi033.stdout:0/562: symlink d6/db/d2c/d73/daf/da4/lc0 0 2024-05-12T04:01:16.335 INFO:tasks.workunit.client.1.smithi033.stdout:0/563: readlink d6/dc/d58/l65 0 2024-05-12T04:01:16.335 INFO:tasks.workunit.client.1.smithi033.stdout:0/564: chown d6/dc/d58/d6a/d94 1796325730 1 2024-05-12T04:01:16.336 INFO:tasks.workunit.client.0.smithi032.stdout:8/401: unlink d0/d9/d1b/d66/l8d 0 2024-05-12T04:01:16.337 INFO:tasks.workunit.client.1.smithi033.stdout:6/543: mkdir d4/d1a/d4e/d4f/d94/db4 0 2024-05-12T04:01:16.342 INFO:tasks.workunit.client.0.smithi032.stdout:0/412: rmdir df/d10/d3b 39 2024-05-12T04:01:16.342 INFO:tasks.workunit.client.0.smithi032.stdout:0/413: chown df/d30/d8d/d8f 538458 1 2024-05-12T04:01:16.342 INFO:tasks.workunit.client.0.smithi032.stdout:0/414: dread - df/d30/d8d/d8f/f94 zero size 2024-05-12T04:01:16.342 INFO:tasks.workunit.client.0.smithi032.stdout:4/274: rename d0/d39/l3e to d0/d3/d24/l4a 0 2024-05-12T04:01:16.342 INFO:tasks.workunit.client.1.smithi033.stdout:3/535: dread db/d17/d25/d32/d3c/d44/f55 [4194304,4194304] 0 2024-05-12T04:01:16.343 INFO:tasks.workunit.client.0.smithi032.stdout:3/380: mknod d15/d20/d26/d6a/d6e/d7d/c82 0 2024-05-12T04:01:16.351 INFO:tasks.workunit.client.0.smithi032.stdout:8/402: mkdir d0/d9/d13/d92 0 2024-05-12T04:01:16.353 INFO:tasks.workunit.client.0.smithi032.stdout:2/359: dread d1/d13/f18 [0,4194304] 0 2024-05-12T04:01:16.353 INFO:tasks.workunit.client.0.smithi032.stdout:8/403: rename d0/d9/d1b/d27 to d0/d9/d1b/d27/d4c/d4e/d93 22 2024-05-12T04:01:16.355 INFO:tasks.workunit.client.0.smithi032.stdout:0/415: write df/d10/d48/d70/f76 [4820885,104353] 0 2024-05-12T04:01:16.358 INFO:tasks.workunit.client.1.smithi033.stdout:4/554: rename dd/d7a/d87/d7e/da7/d38/c56 to dd/d69/cae 0 2024-05-12T04:01:16.360 INFO:tasks.workunit.client.0.smithi032.stdout:4/275: creat d0/d39/f4b x:0 0 0 2024-05-12T04:01:16.360 INFO:tasks.workunit.client.1.smithi033.stdout:5/461: getdents d3/d15/d8c 0 2024-05-12T04:01:16.361 INFO:tasks.workunit.client.0.smithi032.stdout:2/360: dwrite d1/d13/d3d/d41/d2c/f56 [0,4194304] 0 2024-05-12T04:01:16.363 INFO:tasks.workunit.client.0.smithi032.stdout:3/381: unlink d15/d1b/f2f 0 2024-05-12T04:01:16.365 INFO:tasks.workunit.client.0.smithi032.stdout:3/382: chown d15/d24/d2a/l39 244 1 2024-05-12T04:01:16.366 INFO:tasks.workunit.client.1.smithi033.stdout:7/696: sync 2024-05-12T04:01:16.369 INFO:tasks.workunit.client.0.smithi032.stdout:7/357: dwrite dd/f1a [0,4194304] 0 2024-05-12T04:01:16.370 INFO:tasks.workunit.client.1.smithi033.stdout:6/544: dread d4/d1a/d4e/d4f/d94/faa [0,4194304] 0 2024-05-12T04:01:16.370 INFO:tasks.workunit.client.0.smithi032.stdout:0/416: creat df/d30/d8d/d8f/f97 x:0 0 0 2024-05-12T04:01:16.373 INFO:tasks.workunit.client.1.smithi033.stdout:9/774: rename d2/d4/d29/c85 to d2/d4/d2b/d33/c116 0 2024-05-12T04:01:16.378 INFO:tasks.workunit.client.1.smithi033.stdout:4/555: creat dd/d11/d5b/d84/faf x:0 0 0 2024-05-12T04:01:16.379 INFO:tasks.workunit.client.0.smithi032.stdout:4/276: dread d0/d3/d12/d1d/f2a [0,4194304] 0 2024-05-12T04:01:16.380 INFO:tasks.workunit.client.0.smithi032.stdout:0/417: dwrite df/d30/d4e/f73 [0,4194304] 0 2024-05-12T04:01:16.382 INFO:tasks.workunit.client.0.smithi032.stdout:3/383: creat d15/d33/d3f/f83 x:0 0 0 2024-05-12T04:01:16.382 INFO:tasks.workunit.client.0.smithi032.stdout:0/418: stat df/d10/d3d/d4a/l79 0 2024-05-12T04:01:16.385 INFO:tasks.workunit.client.1.smithi033.stdout:2/530: creat dc/d17/d32/fb0 x:0 0 0 2024-05-12T04:01:16.385 INFO:tasks.workunit.client.1.smithi033.stdout:4/556: dwrite dd/d11/f21 [0,4194304] 0 2024-05-12T04:01:16.387 INFO:tasks.workunit.client.0.smithi032.stdout:7/358: mknod dd/d1b/d1e/d45/d55/d60/c78 0 2024-05-12T04:01:16.388 INFO:tasks.workunit.client.1.smithi033.stdout:7/697: mknod de/df/d65/cf6 0 2024-05-12T04:01:16.390 INFO:tasks.workunit.client.0.smithi032.stdout:2/361: creat d1/d13/d70/f7d x:0 0 0 2024-05-12T04:01:16.391 INFO:tasks.workunit.client.0.smithi032.stdout:4/277: creat d0/d3/d12/d1d/f4c x:0 0 0 2024-05-12T04:01:16.393 INFO:tasks.workunit.client.1.smithi033.stdout:0/565: dwrite d6/dc/d58/d6a/d80/f84 [0,4194304] 0 2024-05-12T04:01:16.393 INFO:tasks.workunit.client.1.smithi033.stdout:2/531: write dc/d17/d32/d84/f89 [307868,78239] 0 2024-05-12T04:01:16.393 INFO:tasks.workunit.client.0.smithi032.stdout:3/384: dread - d15/f55 zero size 2024-05-12T04:01:16.396 INFO:tasks.workunit.client.1.smithi033.stdout:8/464: creat d1/d4/d1d/f8f x:0 0 0 2024-05-12T04:01:16.397 INFO:tasks.workunit.client.0.smithi032.stdout:0/419: rename df to df/d30/d32/d98 22 2024-05-12T04:01:16.398 INFO:tasks.workunit.client.1.smithi033.stdout:8/465: write d1/d16/f75 [106440,77433] 0 2024-05-12T04:01:16.398 INFO:tasks.workunit.client.1.smithi033.stdout:3/536: rename db/ce to db/d17/d3e/d65/d91/cc2 0 2024-05-12T04:01:16.398 INFO:tasks.workunit.client.1.smithi033.stdout:0/566: truncate d6/d11/d98/fa1 1429493 0 2024-05-12T04:01:16.401 INFO:tasks.workunit.client.1.smithi033.stdout:3/537: write f6 [2306327,7131] 0 2024-05-12T04:01:16.402 INFO:tasks.workunit.client.0.smithi032.stdout:3/385: mknod d15/d24/d2a/d5b/c84 0 2024-05-12T04:01:16.402 INFO:tasks.workunit.client.1.smithi033.stdout:7/698: dread de/d20/d64/f93 [0,4194304] 0 2024-05-12T04:01:16.402 INFO:tasks.workunit.client.1.smithi033.stdout:1/493: link d11/d17/d27/d36/d3f/d69/d87/l8a d11/d17/d27/d36/d46/d4c/la2 0 2024-05-12T04:01:16.403 INFO:tasks.workunit.client.1.smithi033.stdout:4/557: dwrite dd/f62 [0,4194304] 0 2024-05-12T04:01:16.403 INFO:tasks.workunit.client.0.smithi032.stdout:8/404: link d0/l85 d0/d9/d13/d21/l94 0 2024-05-12T04:01:16.404 INFO:tasks.workunit.client.1.smithi033.stdout:7/699: chown de/df/d17/f51 3392 1 2024-05-12T04:01:16.407 INFO:tasks.workunit.client.1.smithi033.stdout:2/532: mkdir dc/d13/d65/db1 0 2024-05-12T04:01:16.408 INFO:tasks.workunit.client.0.smithi032.stdout:5/454: sync 2024-05-12T04:01:16.408 INFO:tasks.workunit.client.0.smithi032.stdout:7/359: mknod dd/d1b/d1e/d45/c79 0 2024-05-12T04:01:16.413 INFO:tasks.workunit.client.0.smithi032.stdout:0/420: dwrite df/d30/d32/f5e [0,4194304] 0 2024-05-12T04:01:16.416 INFO:tasks.workunit.client.0.smithi032.stdout:3/386: rename d15/f55 to d15/d20/d22/d7a/f85 0 2024-05-12T04:01:16.418 INFO:tasks.workunit.client.1.smithi033.stdout:5/462: sync 2024-05-12T04:01:16.420 INFO:tasks.workunit.client.1.smithi033.stdout:0/567: read - d6/dc/d39/f64 zero size 2024-05-12T04:01:16.424 INFO:tasks.workunit.client.0.smithi032.stdout:5/455: mknod d4/d7/d58/c9f 0 2024-05-12T04:01:16.424 INFO:tasks.workunit.client.0.smithi032.stdout:3/387: dread d15/d24/f51 [0,4194304] 0 2024-05-12T04:01:16.429 INFO:tasks.workunit.client.0.smithi032.stdout:1/367: sync 2024-05-12T04:01:16.429 INFO:tasks.workunit.client.0.smithi032.stdout:9/275: sync 2024-05-12T04:01:16.429 INFO:tasks.workunit.client.0.smithi032.stdout:6/337: sync 2024-05-12T04:01:16.430 INFO:tasks.workunit.client.1.smithi033.stdout:2/533: dread dc/d17/d3a/f8b [0,4194304] 0 2024-05-12T04:01:16.431 INFO:tasks.workunit.client.0.smithi032.stdout:6/338: fsync d7/d9/f4a 0 2024-05-12T04:01:16.432 INFO:tasks.workunit.client.0.smithi032.stdout:1/368: chown d1/d7/d30/d35/d46/f4b 13638 1 2024-05-12T04:01:16.433 INFO:tasks.workunit.client.0.smithi032.stdout:7/360: rename dd/d1b/d43/f59 to dd/d1b/f7a 0 2024-05-12T04:01:16.434 INFO:tasks.workunit.client.1.smithi033.stdout:4/558: dread dd/d11/d75/fa2 [0,4194304] 0 2024-05-12T04:01:16.434 INFO:tasks.workunit.client.0.smithi032.stdout:1/369: chown d1/d7/d30/d35 0 1 2024-05-12T04:01:16.440 INFO:tasks.workunit.client.0.smithi032.stdout:8/405: readlink d0/d9/d1b/d27/d88/d56/d63/l71 0 2024-05-12T04:01:16.440 INFO:tasks.workunit.client.1.smithi033.stdout:1/494: dread d11/d17/f6d [0,4194304] 0 2024-05-12T04:01:16.442 INFO:tasks.workunit.client.0.smithi032.stdout:0/421: symlink df/d30/d32/l99 0 2024-05-12T04:01:16.442 INFO:tasks.workunit.client.0.smithi032.stdout:9/276: dwrite d0/d3/d6/d7/d2a/d46/d13/f3c [0,4194304] 0 2024-05-12T04:01:16.446 INFO:tasks.workunit.client.0.smithi032.stdout:0/422: truncate df/d10/f80 390123 0 2024-05-12T04:01:16.446 INFO:tasks.workunit.client.0.smithi032.stdout:7/361: dwrite dd/d1b/f4b [0,4194304] 0 2024-05-12T04:01:16.453 INFO:tasks.workunit.client.0.smithi032.stdout:3/388: rename d15/d1b/d65/c68 to d15/d1b/d43/c86 0 2024-05-12T04:01:16.459 INFO:tasks.workunit.client.0.smithi032.stdout:2/362: dwrite d1/f7 [0,4194304] 0 2024-05-12T04:01:16.459 INFO:tasks.workunit.client.0.smithi032.stdout:5/456: dread d4/d7/d14/d39/d1b/f23 [0,4194304] 0 2024-05-12T04:01:16.468 INFO:tasks.workunit.client.0.smithi032.stdout:0/423: mkdir df/d10/d3d/d4a/d9a 0 2024-05-12T04:01:16.470 INFO:tasks.workunit.client.0.smithi032.stdout:4/278: dwrite d0/f11 [0,4194304] 0 2024-05-12T04:01:16.472 INFO:tasks.workunit.client.0.smithi032.stdout:5/457: fdatasync d4/d7/d14/d26/d33/d5f/d7f/f84 0 2024-05-12T04:01:16.472 INFO:tasks.workunit.client.0.smithi032.stdout:4/279: read - d0/d3/f49 zero size 2024-05-12T04:01:16.474 INFO:tasks.workunit.client.0.smithi032.stdout:9/277: rename d0/d3/d6/d7/d2a/d46/d4f to d0/d3/d6/d58 0 2024-05-12T04:01:16.474 INFO:tasks.workunit.client.0.smithi032.stdout:6/339: write d7/f51 [862618,78787] 0 2024-05-12T04:01:16.477 INFO:tasks.workunit.client.0.smithi032.stdout:3/389: creat d15/d33/d3f/f87 x:0 0 0 2024-05-12T04:01:16.478 INFO:tasks.workunit.client.0.smithi032.stdout:3/390: dread - d15/d20/d22/f58 zero size 2024-05-12T04:01:16.478 INFO:tasks.workunit.client.0.smithi032.stdout:1/370: mkdir d1/d7/d49/d79 0 2024-05-12T04:01:16.485 INFO:tasks.workunit.client.0.smithi032.stdout:8/406: write d0/f68 [421449,8133] 0 2024-05-12T04:01:16.486 INFO:tasks.workunit.client.0.smithi032.stdout:8/407: stat d0/d9/d13/f29 0 2024-05-12T04:01:16.488 INFO:tasks.workunit.client.0.smithi032.stdout:8/408: truncate d0/d9/d1b/d27/f7b 884072 0 2024-05-12T04:01:16.490 INFO:tasks.workunit.client.0.smithi032.stdout:5/458: dread d4/d7/d14/d26/d33/d4a/f51 [0,4194304] 0 2024-05-12T04:01:16.491 INFO:tasks.workunit.client.0.smithi032.stdout:4/280: mknod d0/d3/df/c4d 0 2024-05-12T04:01:16.492 INFO:tasks.workunit.client.0.smithi032.stdout:4/281: dread - d0/d39/f4b zero size 2024-05-12T04:01:16.492 INFO:tasks.workunit.client.1.smithi033.stdout:9/775: link d2/d4/d2b/d49/d73/l75 d2/d4/d2b/d6d/d7a/l117 0 2024-05-12T04:01:16.494 INFO:tasks.workunit.client.1.smithi033.stdout:8/466: mknod d1/d4/c90 0 2024-05-12T04:01:16.496 INFO:tasks.workunit.client.0.smithi032.stdout:9/278: stat d0/d3/d6/d7/d2a/d46/d13/f55 0 2024-05-12T04:01:16.498 INFO:tasks.workunit.client.1.smithi033.stdout:6/545: rename d4/d1a/d67/d68/f7b to d4/d1a/d4e/fb5 0 2024-05-12T04:01:16.498 INFO:tasks.workunit.client.0.smithi032.stdout:1/371: rmdir d1/d59 39 2024-05-12T04:01:16.499 INFO:tasks.workunit.client.1.smithi033.stdout:6/546: write d4/d1a/d4e/d4f/f9e [4064312,53949] 0 2024-05-12T04:01:16.499 INFO:tasks.workunit.client.0.smithi032.stdout:1/372: dread - d1/d7/d49/f6b zero size 2024-05-12T04:01:16.500 INFO:tasks.workunit.client.1.smithi033.stdout:5/463: fsync d3/fc 0 2024-05-12T04:01:16.502 INFO:tasks.workunit.client.0.smithi032.stdout:3/391: creat d15/d20/d22/d7a/f88 x:0 0 0 2024-05-12T04:01:16.504 INFO:tasks.workunit.client.1.smithi033.stdout:0/568: creat d6/dc/d58/d6a/d94/fc1 x:0 0 0 2024-05-12T04:01:16.508 INFO:tasks.workunit.client.0.smithi032.stdout:0/424: rename df/d30/d4d/f7d to df/d10/d3b/d75/f9b 0 2024-05-12T04:01:16.509 INFO:tasks.workunit.client.1.smithi033.stdout:7/700: truncate de/d2f/d70/fa3 741990 0 2024-05-12T04:01:16.511 INFO:tasks.workunit.client.0.smithi032.stdout:8/409: creat d0/d9/d1b/d66/f95 x:0 0 0 2024-05-12T04:01:16.513 INFO:tasks.workunit.client.1.smithi033.stdout:6/547: dread d4/d1a/f26 [0,4194304] 0 2024-05-12T04:01:16.514 INFO:tasks.workunit.client.0.smithi032.stdout:4/282: chown d0/d3/l32 235841609 1 2024-05-12T04:01:16.516 INFO:tasks.workunit.client.0.smithi032.stdout:6/340: symlink d7/de/l75 0 2024-05-12T04:01:16.517 INFO:tasks.workunit.client.1.smithi033.stdout:1/495: dread - d11/d1c/d43/f4d zero size 2024-05-12T04:01:16.518 INFO:tasks.workunit.client.0.smithi032.stdout:9/279: rmdir d0/d3 39 2024-05-12T04:01:16.522 INFO:tasks.workunit.client.0.smithi032.stdout:6/341: dwrite d7/f2b [0,4194304] 0 2024-05-12T04:01:16.525 INFO:tasks.workunit.client.1.smithi033.stdout:3/538: dwrite db/d59/f8d [4194304,4194304] 0 2024-05-12T04:01:16.535 INFO:tasks.workunit.client.0.smithi032.stdout:0/425: creat df/d10/d3d/d4a/d84/f9c x:0 0 0 2024-05-12T04:01:16.536 INFO:tasks.workunit.client.0.smithi032.stdout:5/459: creat d4/d7/d58/d5a/d99/fa0 x:0 0 0 2024-05-12T04:01:16.538 INFO:tasks.workunit.client.1.smithi033.stdout:2/534: dwrite f9 [0,4194304] 0 2024-05-12T04:01:16.538 INFO:tasks.workunit.client.1.smithi033.stdout:9/776: dwrite d2/f27 [0,4194304] 0 2024-05-12T04:01:16.538 INFO:tasks.workunit.client.0.smithi032.stdout:5/460: write d4/d7/d14/d26/d3c/f6f [633281,59609] 0 2024-05-12T04:01:16.541 INFO:tasks.workunit.client.1.smithi033.stdout:2/535: truncate dc/d13/d1f/f9b 959465 0 2024-05-12T04:01:16.542 INFO:tasks.workunit.client.0.smithi032.stdout:1/373: creat d1/d6a/f7a x:0 0 0 2024-05-12T04:01:16.543 INFO:tasks.workunit.client.1.smithi033.stdout:7/701: chown de/d2b/d6c/da0/lb4 20915821 1 2024-05-12T04:01:16.545 INFO:tasks.workunit.client.0.smithi032.stdout:6/342: unlink d7/l53 0 2024-05-12T04:01:16.550 INFO:tasks.workunit.client.0.smithi032.stdout:9/280: truncate d0/d3/d6/d7/da/d17/f4e 4909638 0 2024-05-12T04:01:16.550 INFO:tasks.workunit.client.0.smithi032.stdout:4/283: dwrite d0/d3/d24/f2d [0,4194304] 0 2024-05-12T04:01:16.551 INFO:tasks.workunit.client.0.smithi032.stdout:4/284: chown d0/fa 3888599 1 2024-05-12T04:01:16.553 INFO:tasks.workunit.client.1.smithi033.stdout:4/559: mknod dd/d11/cb0 0 2024-05-12T04:01:16.555 INFO:tasks.workunit.client.0.smithi032.stdout:2/363: getdents d1/d13/d3d/d41 0 2024-05-12T04:01:16.556 INFO:tasks.workunit.client.1.smithi033.stdout:4/560: truncate dd/d7a/d87/d7e/da7/d66/f92 1042511 0 2024-05-12T04:01:16.558 INFO:tasks.workunit.client.0.smithi032.stdout:1/374: symlink d1/d7/d49/d67/l7b 0 2024-05-12T04:01:16.558 INFO:tasks.workunit.client.1.smithi033.stdout:3/539: unlink db/f8c 0 2024-05-12T04:01:16.559 INFO:tasks.workunit.client.0.smithi032.stdout:4/285: dread - d0/d3/d12/f48 zero size 2024-05-12T04:01:16.562 INFO:tasks.workunit.client.0.smithi032.stdout:9/281: rename d0/d3/d6/d7/d27/c3a to d0/d3/d6/c59 0 2024-05-12T04:01:16.563 INFO:tasks.workunit.client.1.smithi033.stdout:8/467: mkdir d1/d4/d91 0 2024-05-12T04:01:16.564 INFO:tasks.workunit.client.1.smithi033.stdout:5/464: dwrite d3/d12/d29/f3d [0,4194304] 0 2024-05-12T04:01:16.564 INFO:tasks.workunit.client.0.smithi032.stdout:3/392: truncate d15/d20/d26/f52 1576371 0 2024-05-12T04:01:16.564 INFO:tasks.workunit.client.1.smithi033.stdout:8/468: chown d1/d16/d27/f2d 111097 1 2024-05-12T04:01:16.565 INFO:tasks.workunit.client.0.smithi032.stdout:6/343: dwrite d7/d9/d48/d67/f6e [0,4194304] 0 2024-05-12T04:01:16.565 INFO:tasks.workunit.client.0.smithi032.stdout:0/426: mkdir df/d10/d3d/d4a/d9a/d9d 0 2024-05-12T04:01:16.569 INFO:tasks.workunit.client.0.smithi032.stdout:9/282: dread - d0/d3/d6/d7/d2a/d46/d13/f56 zero size 2024-05-12T04:01:16.570 INFO:tasks.workunit.client.0.smithi032.stdout:0/427: fsync df/d30/d32/d60/f90 0 2024-05-12T04:01:16.571 INFO:tasks.workunit.client.0.smithi032.stdout:0/428: chown df/l1d 2116 1 2024-05-12T04:01:16.574 INFO:tasks.workunit.client.0.smithi032.stdout:2/364: rename d1/d13/f5d to d1/d34/d3e/f7e 0 2024-05-12T04:01:16.574 INFO:tasks.workunit.client.1.smithi033.stdout:8/469: dwrite d1/d4/d1d/d36/d3c/d64/f8d [0,4194304] 0 2024-05-12T04:01:16.574 INFO:tasks.workunit.client.1.smithi033.stdout:6/548: dwrite d4/d1a/d3c/f65 [0,4194304] 0 2024-05-12T04:01:16.581 INFO:tasks.workunit.client.0.smithi032.stdout:6/344: fdatasync d7/d9/d48/d67/f28 0 2024-05-12T04:01:16.584 INFO:tasks.workunit.client.1.smithi033.stdout:7/702: symlink de/d2b/lf7 0 2024-05-12T04:01:16.584 INFO:tasks.workunit.client.0.smithi032.stdout:2/365: dwrite d1/d34/d3e/f45 [0,4194304] 0 2024-05-12T04:01:16.584 INFO:tasks.workunit.client.1.smithi033.stdout:7/703: stat de/d2b/d6c/dbd/ld2 0 2024-05-12T04:01:16.591 INFO:tasks.workunit.client.0.smithi032.stdout:9/283: creat d0/d3/d6/d58/f5a x:0 0 0 2024-05-12T04:01:16.591 INFO:tasks.workunit.client.0.smithi032.stdout:9/284: chown d0/d3/d6 74699 1 2024-05-12T04:01:16.593 INFO:tasks.workunit.client.1.smithi033.stdout:4/561: fsync dd/d7a/d87/d7e/da7/d66/d42/f4c 0 2024-05-12T04:01:16.596 INFO:tasks.workunit.client.0.smithi032.stdout:6/345: dwrite d7/d9/d48/d67/f6e [0,4194304] 0 2024-05-12T04:01:16.596 INFO:tasks.workunit.client.1.smithi033.stdout:1/496: mkdir d11/d17/d84/da3 0 2024-05-12T04:01:16.599 INFO:tasks.workunit.client.0.smithi032.stdout:0/429: rmdir df/d10/d48/d70 39 2024-05-12T04:01:16.602 INFO:tasks.workunit.client.0.smithi032.stdout:3/393: fsync d15/d33/f38 0 2024-05-12T04:01:16.603 INFO:tasks.workunit.client.0.smithi032.stdout:1/375: creat d1/dd/f7c x:0 0 0 2024-05-12T04:01:16.608 INFO:tasks.workunit.client.0.smithi032.stdout:4/286: dwrite d0/d3/d31/f38 [0,4194304] 0 2024-05-12T04:01:16.610 INFO:tasks.workunit.client.0.smithi032.stdout:9/285: symlink d0/d3/d6/l5b 0 2024-05-12T04:01:16.612 INFO:tasks.workunit.client.1.smithi033.stdout:9/777: dwrite d2/d4/d2b/d78/fc8 [0,4194304] 0 2024-05-12T04:01:16.614 INFO:tasks.workunit.client.1.smithi033.stdout:8/470: creat d1/d16/d27/d7c/f92 x:0 0 0 2024-05-12T04:01:16.617 INFO:tasks.workunit.client.1.smithi033.stdout:6/549: creat d4/d1a/d67/fb6 x:0 0 0 2024-05-12T04:01:16.618 INFO:tasks.workunit.client.0.smithi032.stdout:0/430: unlink df/d30/d32/c93 0 2024-05-12T04:01:16.621 INFO:tasks.workunit.client.0.smithi032.stdout:3/394: rmdir d15/d33/d3f 39 2024-05-12T04:01:16.623 INFO:tasks.workunit.client.0.smithi032.stdout:6/346: dwrite d7/de/f26 [0,4194304] 0 2024-05-12T04:01:16.627 INFO:tasks.workunit.client.0.smithi032.stdout:4/287: dwrite d0/d3/d12/f48 [0,4194304] 0 2024-05-12T04:01:16.628 INFO:tasks.workunit.client.0.smithi032.stdout:4/288: chown d0/d3/df/f29 0 1 2024-05-12T04:01:16.628 INFO:tasks.workunit.client.1.smithi033.stdout:3/540: dwrite db/d17/d3e/d65/d7d/f94 [4194304,4194304] 0 2024-05-12T04:01:16.631 INFO:tasks.workunit.client.1.smithi033.stdout:7/704: symlink de/df/d17/d1b/d50/lf8 0 2024-05-12T04:01:16.632 INFO:tasks.workunit.client.0.smithi032.stdout:4/289: dread d0/d3/fd [0,4194304] 0 2024-05-12T04:01:16.637 INFO:tasks.workunit.client.1.smithi033.stdout:1/497: rename d11/d17/d27/d36/d3f/d4a/da0 to d11/d17/d27/d36/d3f/d69/d87/da4 0 2024-05-12T04:01:16.639 INFO:tasks.workunit.client.1.smithi033.stdout:3/541: dwrite db/f54 [0,4194304] 0 2024-05-12T04:01:16.640 INFO:tasks.workunit.client.0.smithi032.stdout:0/431: rmdir df/d30/d8d/d8f 39 2024-05-12T04:01:16.640 INFO:tasks.workunit.client.1.smithi033.stdout:3/542: fsync db/d17/d3a/f63 0 2024-05-12T04:01:16.641 INFO:tasks.workunit.client.0.smithi032.stdout:1/376: mknod d1/d7/d30/d35/d77/c7d 0 2024-05-12T04:01:16.641 INFO:tasks.workunit.client.1.smithi033.stdout:3/543: fsync db/d17/f7b 0 2024-05-12T04:01:16.642 INFO:tasks.workunit.client.1.smithi033.stdout:9/778: fsync d2/dd/f13 0 2024-05-12T04:01:16.644 INFO:tasks.workunit.client.0.smithi032.stdout:1/377: write d1/d7/d30/d18/f41 [3891464,88468] 0 2024-05-12T04:01:16.648 INFO:tasks.workunit.client.1.smithi033.stdout:6/550: creat d4/dad/fb7 x:0 0 0 2024-05-12T04:01:16.653 INFO:tasks.workunit.client.1.smithi033.stdout:2/536: dwrite dc/d17/d27/d2f/d49/d59/f79 [0,4194304] 0 2024-05-12T04:01:16.654 INFO:tasks.workunit.client.1.smithi033.stdout:2/537: read - dc/d17/d32/fb0 zero size 2024-05-12T04:01:16.656 INFO:tasks.workunit.client.1.smithi033.stdout:2/538: write dc/d17/d32/f5c [954319,103125] 0 2024-05-12T04:01:16.656 INFO:tasks.workunit.client.0.smithi032.stdout:4/290: mkdir d0/d3/df/d4e 0 2024-05-12T04:01:16.658 INFO:tasks.workunit.client.0.smithi032.stdout:4/291: read d0/f11 [3675591,93344] 0 2024-05-12T04:01:16.662 INFO:tasks.workunit.client.0.smithi032.stdout:6/347: mknod d7/de/d2e/c76 0 2024-05-12T04:01:16.663 INFO:tasks.workunit.client.0.smithi032.stdout:1/378: dwrite d1/d7/d30/d18/f41 [4194304,4194304] 0 2024-05-12T04:01:16.664 INFO:tasks.workunit.client.0.smithi032.stdout:4/292: truncate d0/d39/f4b 130721 0 2024-05-12T04:01:16.665 INFO:tasks.workunit.client.0.smithi032.stdout:6/348: write d7/d9/d48/d67/f4c [2303982,93615] 0 2024-05-12T04:01:16.666 INFO:tasks.workunit.client.0.smithi032.stdout:4/293: fsync d0/d3/df/f26 0 2024-05-12T04:01:16.666 INFO:tasks.workunit.client.0.smithi032.stdout:6/349: chown d7/l30 475 1 2024-05-12T04:01:16.667 INFO:tasks.workunit.client.1.smithi033.stdout:7/705: mknod de/d2b/d6c/db5/dd0/cf9 0 2024-05-12T04:01:16.670 INFO:tasks.workunit.client.0.smithi032.stdout:2/366: getdents d1/d13 0 2024-05-12T04:01:16.670 INFO:tasks.workunit.client.1.smithi033.stdout:4/562: truncate dd/f29 1341697 0 2024-05-12T04:01:16.673 INFO:tasks.workunit.client.0.smithi032.stdout:4/294: dread d0/d3/d31/f38 [0,4194304] 0 2024-05-12T04:01:16.674 INFO:tasks.workunit.client.1.smithi033.stdout:5/465: creat d3/f92 x:0 0 0 2024-05-12T04:01:16.674 INFO:tasks.workunit.client.1.smithi033.stdout:1/498: creat d11/d17/d27/d36/d3f/d4a/fa5 x:0 0 0 2024-05-12T04:01:16.675 INFO:tasks.workunit.client.0.smithi032.stdout:2/367: dread d1/d34/d3e/f45 [0,4194304] 0 2024-05-12T04:01:16.676 INFO:tasks.workunit.client.0.smithi032.stdout:2/368: dread - d1/d34/f53 zero size 2024-05-12T04:01:16.679 INFO:tasks.workunit.client.0.smithi032.stdout:3/395: link d15/d1b/c76 d15/d20/d22/d4f/c89 0 2024-05-12T04:01:16.688 INFO:tasks.workunit.client.1.smithi033.stdout:6/551: unlink d4/d5/f76 0 2024-05-12T04:01:16.689 INFO:tasks.workunit.client.1.smithi033.stdout:6/552: truncate d4/dad/fb7 119961 0 2024-05-12T04:01:16.691 INFO:tasks.workunit.client.0.smithi032.stdout:6/350: dread d7/de/f6d [0,4194304] 0 2024-05-12T04:01:16.692 INFO:tasks.workunit.client.0.smithi032.stdout:4/295: symlink d0/d3/d12/d1d/l4f 0 2024-05-12T04:01:16.694 INFO:tasks.workunit.client.0.smithi032.stdout:1/379: getdents d1/d6a/d72 0 2024-05-12T04:01:16.694 INFO:tasks.workunit.client.1.smithi033.stdout:9/779: write d2/dd/f26 [38653,25767] 0 2024-05-12T04:01:16.699 INFO:tasks.workunit.client.0.smithi032.stdout:3/396: creat d15/d20/d26/d6a/f8a x:0 0 0 2024-05-12T04:01:16.699 INFO:tasks.workunit.client.1.smithi033.stdout:4/563: symlink dd/d7a/d87/d7e/lb1 0 2024-05-12T04:01:16.700 INFO:tasks.workunit.client.1.smithi033.stdout:4/564: chown dd/d11/c27 602447 1 2024-05-12T04:01:16.701 INFO:tasks.workunit.client.1.smithi033.stdout:1/499: rmdir d11/d17/d27/d36 39 2024-05-12T04:01:16.701 INFO:tasks.workunit.client.1.smithi033.stdout:4/565: dread - dd/d11/f95 zero size 2024-05-12T04:01:16.701 INFO:tasks.workunit.client.1.smithi033.stdout:2/539: dread dc/d17/d27/d2f/f75 [0,4194304] 0 2024-05-12T04:01:16.702 INFO:tasks.workunit.client.1.smithi033.stdout:2/540: stat dc/d17/d32/fb0 0 2024-05-12T04:01:16.704 INFO:tasks.workunit.client.0.smithi032.stdout:6/351: dread d7/d9/d48/d67/f16 [0,4194304] 0 2024-05-12T04:01:16.706 INFO:tasks.workunit.client.1.smithi033.stdout:5/466: truncate d3/d12/d25/d26/d2e/d54/d57/f89 841048 0 2024-05-12T04:01:16.707 INFO:tasks.workunit.client.0.smithi032.stdout:0/432: link df/c2a df/d30/d32/d60/c9e 0 2024-05-12T04:01:16.707 INFO:tasks.workunit.client.1.smithi033.stdout:5/467: stat d3/d15/d16/d45/d62/f67 0 2024-05-12T04:01:16.708 INFO:tasks.workunit.client.1.smithi033.stdout:3/544: creat db/d17/d3a/d67/da6/fc3 x:0 0 0 2024-05-12T04:01:16.712 INFO:tasks.workunit.client.0.smithi032.stdout:1/380: rmdir d1/d7/d30/d35 39 2024-05-12T04:01:16.714 INFO:tasks.workunit.client.1.smithi033.stdout:5/468: dwrite d3/d15/d16/f42 [0,4194304] 0 2024-05-12T04:01:16.715 INFO:tasks.workunit.client.1.smithi033.stdout:7/706: dwrite de/fc2 [0,4194304] 0 2024-05-12T04:01:16.721 INFO:tasks.workunit.client.0.smithi032.stdout:0/433: dread df/d10/d3d/f56 [0,4194304] 0 2024-05-12T04:01:16.721 INFO:tasks.workunit.client.1.smithi033.stdout:0/569: link d6/d11/d57/f55 d6/d11/d57/fc2 0 2024-05-12T04:01:16.723 INFO:tasks.workunit.client.0.smithi032.stdout:0/434: write df/d10/d3d/d4a/f38 [4922388,24168] 0 2024-05-12T04:01:16.724 INFO:tasks.workunit.client.1.smithi033.stdout:9/780: rmdir d2/dd/d79 39 2024-05-12T04:01:16.729 INFO:tasks.workunit.client.0.smithi032.stdout:3/397: mkdir d15/d20/d8b 0 2024-05-12T04:01:16.730 INFO:tasks.workunit.client.0.smithi032.stdout:6/352: mknod d7/d9/d5c/d62/c77 0 2024-05-12T04:01:16.731 INFO:tasks.workunit.client.0.smithi032.stdout:4/296: rename d0/d3/d12/l3c to d0/d3/d12/l50 0 2024-05-12T04:01:16.734 INFO:tasks.workunit.client.0.smithi032.stdout:0/435: fsync df/d10/d3b/d75/f78 0 2024-05-12T04:01:16.734 INFO:tasks.workunit.client.1.smithi033.stdout:2/541: dread dc/d13/d1c/f21 [0,4194304] 0 2024-05-12T04:01:16.735 INFO:tasks.workunit.client.0.smithi032.stdout:2/369: getdents d1/d13/d70 0 2024-05-12T04:01:16.737 INFO:tasks.workunit.client.0.smithi032.stdout:2/370: dread - d1/d34/d3e/d67/f6a zero size 2024-05-12T04:01:16.739 INFO:tasks.workunit.client.0.smithi032.stdout:1/381: mkdir d1/d7/d30/d35/d54/d7e 0 2024-05-12T04:01:16.741 INFO:tasks.workunit.client.1.smithi033.stdout:5/469: creat d3/d12/d25/d26/d2e/d54/d57/d73/d74/f93 x:0 0 0 2024-05-12T04:01:16.741 INFO:tasks.workunit.client.1.smithi033.stdout:8/471: link d1/d16/d27/d4d/d73/l7d d1/d4/d1d/d36/d3c/l93 0 2024-05-12T04:01:16.743 INFO:tasks.workunit.client.0.smithi032.stdout:6/353: creat d7/d9/d48/d67/d63/f78 x:0 0 0 2024-05-12T04:01:16.748 INFO:tasks.workunit.client.1.smithi033.stdout:6/553: rename d4/d5/f27 to d4/d43/d57/fb8 0 2024-05-12T04:01:16.749 INFO:tasks.workunit.client.1.smithi033.stdout:8/472: dread d1/f13 [0,4194304] 0 2024-05-12T04:01:16.754 INFO:tasks.workunit.client.0.smithi032.stdout:2/371: unlink d1/d13/f48 0 2024-05-12T04:01:16.754 INFO:tasks.workunit.client.0.smithi032.stdout:2/372: truncate d1/d34/d4a/f6d 822687 0 2024-05-12T04:01:16.754 INFO:tasks.workunit.client.1.smithi033.stdout:1/500: mkdir d11/d17/d27/da6 0 2024-05-12T04:01:16.754 INFO:tasks.workunit.client.0.smithi032.stdout:2/373: chown d1/d34/l50 3779239 1 2024-05-12T04:01:16.755 INFO:tasks.workunit.client.1.smithi033.stdout:6/554: dwrite d4/dad/fa7 [0,4194304] 0 2024-05-12T04:01:16.755 INFO:tasks.workunit.client.0.smithi032.stdout:1/382: creat d1/d7/f7f x:0 0 0 2024-05-12T04:01:16.756 INFO:tasks.workunit.client.1.smithi033.stdout:4/566: mknod dd/d7a/d87/d7e/da7/d66/cb2 0 2024-05-12T04:01:16.756 INFO:tasks.workunit.client.0.smithi032.stdout:3/398: dread f14 [0,4194304] 0 2024-05-12T04:01:16.757 INFO:tasks.workunit.client.0.smithi032.stdout:6/354: creat d7/d9/d48/d67/d1a/d20/f79 x:0 0 0 2024-05-12T04:01:16.759 INFO:tasks.workunit.client.1.smithi033.stdout:3/545: symlink db/d17/d3a/d67/d76/da0/lc4 0 2024-05-12T04:01:16.760 INFO:tasks.workunit.client.1.smithi033.stdout:2/542: mkdir dc/d13/d1c/d98/db2 0 2024-05-12T04:01:16.761 INFO:tasks.workunit.client.0.smithi032.stdout:0/436: mkdir df/d10/d3d/d83/d9f 0 2024-05-12T04:01:16.761 INFO:tasks.workunit.client.0.smithi032.stdout:0/437: chown l4 28613 1 2024-05-12T04:01:16.763 INFO:tasks.workunit.client.1.smithi033.stdout:4/567: dwrite dd/d11/f19 [0,4194304] 0 2024-05-12T04:01:16.765 INFO:tasks.workunit.client.0.smithi032.stdout:1/383: truncate d1/d25/f64 696895 0 2024-05-12T04:01:16.769 INFO:tasks.workunit.client.1.smithi033.stdout:3/546: dwrite db/d17/d3a/d67/d76/da0/fba [0,4194304] 0 2024-05-12T04:01:16.770 INFO:tasks.workunit.client.0.smithi032.stdout:2/374: dwrite d1/d13/f18 [0,4194304] 0 2024-05-12T04:01:16.771 INFO:tasks.workunit.client.1.smithi033.stdout:3/547: chown db/d17/d25/d32/d53/l68 2057 1 2024-05-12T04:01:16.772 INFO:tasks.workunit.client.0.smithi032.stdout:2/375: chown d1/d13/d3d 748 1 2024-05-12T04:01:16.772 INFO:tasks.workunit.client.0.smithi032.stdout:4/297: creat d0/d3/d12/f51 x:0 0 0 2024-05-12T04:01:16.775 INFO:tasks.workunit.client.0.smithi032.stdout:0/438: symlink df/d10/d3d/d4a/la0 0 2024-05-12T04:01:16.780 INFO:tasks.workunit.client.0.smithi032.stdout:4/298: dread d0/d3/df/f29 [0,4194304] 0 2024-05-12T04:01:16.782 INFO:tasks.workunit.client.1.smithi033.stdout:8/473: unlink d1/d4/d1d/d36/c7e 0 2024-05-12T04:01:16.785 INFO:tasks.workunit.client.0.smithi032.stdout:6/355: truncate d7/de/f14 1999651 0 2024-05-12T04:01:16.787 INFO:tasks.workunit.client.1.smithi033.stdout:1/501: fdatasync d11/d1c/f32 0 2024-05-12T04:01:16.789 INFO:tasks.workunit.client.0.smithi032.stdout:2/376: rename d1/d34/d4a/l64 to d1/d13/l7f 0 2024-05-12T04:01:16.796 INFO:tasks.workunit.client.1.smithi033.stdout:2/543: symlink dc/d17/d27/d68/d9f/d5e/lb3 0 2024-05-12T04:01:16.797 INFO:tasks.workunit.client.1.smithi033.stdout:4/568: mkdir dd/d69/da4/db3 0 2024-05-12T04:01:16.798 INFO:tasks.workunit.client.1.smithi033.stdout:4/569: truncate dd/d69/f97 605034 0 2024-05-12T04:01:16.802 INFO:tasks.workunit.client.0.smithi032.stdout:3/399: dwrite d15/d1b/f21 [4194304,4194304] 0 2024-05-12T04:01:16.804 INFO:tasks.workunit.client.0.smithi032.stdout:7/362: sync 2024-05-12T04:01:16.804 INFO:tasks.workunit.client.0.smithi032.stdout:6/356: fsync d7/d9/d48/d67/d1a/d20/f42 0 2024-05-12T04:01:16.806 INFO:tasks.workunit.client.0.smithi032.stdout:0/439: dread df/d30/f8b [0,4194304] 0 2024-05-12T04:01:16.806 INFO:tasks.workunit.client.1.smithi033.stdout:0/570: dwrite d6/f6b [0,4194304] 0 2024-05-12T04:01:16.807 INFO:tasks.workunit.client.0.smithi032.stdout:0/440: readlink df/d30/d36/l7a 0 2024-05-12T04:01:16.808 INFO:tasks.workunit.client.1.smithi033.stdout:0/571: chown d6/dc/f18 14060 1 2024-05-12T04:01:16.809 INFO:tasks.workunit.client.0.smithi032.stdout:2/377: rename d1/d13/l49 to d1/d34/d4a/l80 0 2024-05-12T04:01:16.810 INFO:tasks.workunit.client.0.smithi032.stdout:2/378: write d1/f24 [4605708,87997] 0 2024-05-12T04:01:16.814 INFO:tasks.workunit.client.1.smithi033.stdout:3/548: rename db/d17/f2d to db/d17/fc5 0 2024-05-12T04:01:16.818 INFO:tasks.workunit.client.1.smithi033.stdout:5/470: write d3/d15/d16/f44 [3559903,48134] 0 2024-05-12T04:01:16.822 INFO:tasks.workunit.client.0.smithi032.stdout:3/400: mkdir d15/d24/d2a/d8c 0 2024-05-12T04:01:16.825 INFO:tasks.workunit.client.1.smithi033.stdout:1/502: rename d11/d1c/d43/f85 to d11/d17/d27/d76/fa7 0 2024-05-12T04:01:16.826 INFO:tasks.workunit.client.0.smithi032.stdout:7/363: fsync dd/d1b/d43/d5e/f3e 0 2024-05-12T04:01:16.827 INFO:tasks.workunit.client.1.smithi033.stdout:2/544: unlink dc/d17/d27/d2f/d49/c7c 0 2024-05-12T04:01:16.828 INFO:tasks.workunit.client.0.smithi032.stdout:0/441: truncate df/d10/d3b/d75/f78 162612 0 2024-05-12T04:01:16.828 INFO:tasks.workunit.client.1.smithi033.stdout:2/545: chown dc/d17/d32/f43 63597515 1 2024-05-12T04:01:16.829 INFO:tasks.workunit.client.1.smithi033.stdout:4/570: chown dd/d69/da4/fa8 3033 1 2024-05-12T04:01:16.831 INFO:tasks.workunit.client.0.smithi032.stdout:6/357: rename d7/d9/d48/d67/d1a/d20/f50 to d7/d33/f7a 0 2024-05-12T04:01:16.833 INFO:tasks.workunit.client.1.smithi033.stdout:0/572: truncate d6/d11/d57/fb6 983874 0 2024-05-12T04:01:16.834 INFO:tasks.workunit.client.0.smithi032.stdout:2/379: mkdir d1/d34/d81 0 2024-05-12T04:01:16.835 INFO:tasks.workunit.client.1.smithi033.stdout:7/707: link de/df/d17/d1e/dc4/le4 de/d2f/lfa 0 2024-05-12T04:01:16.839 INFO:tasks.workunit.client.1.smithi033.stdout:3/549: chown db/d17/d25/d32/d3c/la3 701646 1 2024-05-12T04:01:16.839 INFO:tasks.workunit.client.0.smithi032.stdout:0/442: dread f5 [0,4194304] 0 2024-05-12T04:01:16.840 INFO:tasks.workunit.client.1.smithi033.stdout:7/708: dwrite de/df/d17/d1e/d24/f57 [0,4194304] 0 2024-05-12T04:01:16.841 INFO:tasks.workunit.client.0.smithi032.stdout:1/384: truncate d1/d25/f64 964749 0 2024-05-12T04:01:16.842 INFO:tasks.workunit.client.1.smithi033.stdout:8/474: mkdir d1/d16/d27/d4d/d51/d8e/d94 0 2024-05-12T04:01:16.844 INFO:tasks.workunit.client.0.smithi032.stdout:1/385: read - d1/d7/d30/d35/f48 zero size 2024-05-12T04:01:16.847 INFO:tasks.workunit.client.1.smithi033.stdout:9/781: link d2/dd/l91 d2/d4/d2b/d6d/d2f/d5f/d84/d107/l118 0 2024-05-12T04:01:16.848 INFO:tasks.workunit.client.1.smithi033.stdout:5/471: read d3/d12/f43 [1760346,38939] 0 2024-05-12T04:01:16.850 INFO:tasks.workunit.client.0.smithi032.stdout:0/443: dwrite df/d30/d32/f5e [0,4194304] 0 2024-05-12T04:01:16.850 INFO:tasks.workunit.client.0.smithi032.stdout:2/380: dread d1/d13/f2b [0,4194304] 0 2024-05-12T04:01:16.852 INFO:tasks.workunit.client.0.smithi032.stdout:2/381: readlink d1/d34/d4a/l5a 0 2024-05-12T04:01:16.858 INFO:tasks.workunit.client.0.smithi032.stdout:4/299: rename d0/d3/c19 to d0/d3/c52 0 2024-05-12T04:01:16.858 INFO:tasks.workunit.client.1.smithi033.stdout:4/571: rmdir dd/d7a/d87/d7e/da7/d38 39 2024-05-12T04:01:16.860 INFO:tasks.workunit.client.0.smithi032.stdout:3/401: dwrite d15/d20/d26/f5f [0,4194304] 0 2024-05-12T04:01:16.860 INFO:tasks.workunit.client.1.smithi033.stdout:0/573: truncate d6/dc/f54 192216 0 2024-05-12T04:01:16.865 INFO:tasks.workunit.client.0.smithi032.stdout:0/444: stat df/d10/d3d/d6c/l77 0 2024-05-12T04:01:16.866 INFO:tasks.workunit.client.0.smithi032.stdout:0/445: write df/d30/d32/d60/f90 [942213,78612] 0 2024-05-12T04:01:16.867 INFO:tasks.workunit.client.0.smithi032.stdout:0/446: stat df/d30/d32 0 2024-05-12T04:01:16.867 INFO:tasks.workunit.client.1.smithi033.stdout:3/550: unlink db/d42/l9f 0 2024-05-12T04:01:16.868 INFO:tasks.workunit.client.0.smithi032.stdout:0/447: chown f9 1211614 1 2024-05-12T04:01:16.870 INFO:tasks.workunit.client.1.smithi033.stdout:7/709: unlink de/d2b/d6c/da0/fa8 0 2024-05-12T04:01:16.873 INFO:tasks.workunit.client.1.smithi033.stdout:8/475: mknod d1/d16/c95 0 2024-05-12T04:01:16.874 INFO:tasks.workunit.client.0.smithi032.stdout:2/382: stat d1/d13/d6b/l6f 0 2024-05-12T04:01:16.876 INFO:tasks.workunit.client.1.smithi033.stdout:9/782: unlink d2/l23 0 2024-05-12T04:01:16.877 INFO:tasks.workunit.client.1.smithi033.stdout:9/783: chown d2/d4/d29/d3c/d52 21 1 2024-05-12T04:01:16.878 INFO:tasks.workunit.client.1.smithi033.stdout:9/784: stat d2/d4/d2b/d33/dd2 0 2024-05-12T04:01:16.879 INFO:tasks.workunit.client.0.smithi032.stdout:4/300: rmdir d0/d3/d24 39 2024-05-12T04:01:16.880 INFO:tasks.workunit.client.1.smithi033.stdout:9/785: chown d2/d4/d2b/d6d/d2f/d5f/f97 335 1 2024-05-12T04:01:16.881 INFO:tasks.workunit.client.1.smithi033.stdout:6/555: getdents d4/d1a/d4e/d4f/d94 0 2024-05-12T04:01:16.882 INFO:tasks.workunit.client.0.smithi032.stdout:6/358: dwrite d7/de/d2e/f36 [0,4194304] 0 2024-05-12T04:01:16.882 INFO:tasks.workunit.client.1.smithi033.stdout:6/556: fdatasync d4/d1a/fb2 0 2024-05-12T04:01:16.883 INFO:tasks.workunit.client.1.smithi033.stdout:6/557: chown d4 130562 1 2024-05-12T04:01:16.884 INFO:tasks.workunit.client.0.smithi032.stdout:6/359: chown d7/d9/d48/d67/d1a/d20/c46 587 1 2024-05-12T04:01:16.884 INFO:tasks.workunit.client.1.smithi033.stdout:2/546: dwrite dc/d17/d3a/f8b [0,4194304] 0 2024-05-12T04:01:16.885 INFO:tasks.workunit.client.0.smithi032.stdout:6/360: truncate d7/de/f26 4246779 0 2024-05-12T04:01:16.886 INFO:tasks.workunit.client.0.smithi032.stdout:1/386: rename d1/d59/l6f to d1/d7/d30/d35/d77/l80 0 2024-05-12T04:01:16.889 INFO:tasks.workunit.client.1.smithi033.stdout:0/574: unlink d6/db/f19 0 2024-05-12T04:01:16.895 INFO:tasks.workunit.client.0.smithi032.stdout:1/387: dwrite d1/d6a/f7a [0,4194304] 0 2024-05-12T04:01:16.896 INFO:tasks.workunit.client.1.smithi033.stdout:8/476: creat d1/d16/f96 x:0 0 0 2024-05-12T04:01:16.897 INFO:tasks.workunit.client.1.smithi033.stdout:7/710: dread de/d2b/d6c/f72 [0,4194304] 0 2024-05-12T04:01:16.897 INFO:tasks.workunit.client.0.smithi032.stdout:4/301: chown d0/d3/d12/l50 13779023 1 2024-05-12T04:01:16.898 INFO:tasks.workunit.client.0.smithi032.stdout:1/388: write d1/d7/d30/d18/f2b [1617756,120342] 0 2024-05-12T04:01:16.902 INFO:tasks.workunit.client.0.smithi032.stdout:1/389: read - d1/d7/d30/f60 zero size 2024-05-12T04:01:16.902 INFO:tasks.workunit.client.0.smithi032.stdout:3/402: symlink d15/d1b/l8d 0 2024-05-12T04:01:16.903 INFO:tasks.workunit.client.1.smithi033.stdout:9/786: mkdir d2/d4/d2b/d6d/db5/d119 0 2024-05-12T04:01:16.904 INFO:tasks.workunit.client.0.smithi032.stdout:6/361: creat d7/d9/d5c/f7b x:0 0 0 2024-05-12T04:01:16.905 INFO:tasks.workunit.client.0.smithi032.stdout:1/390: dread d1/d7/d30/d35/d46/f4b [0,4194304] 0 2024-05-12T04:01:16.908 INFO:tasks.workunit.client.1.smithi033.stdout:6/558: unlink d4/d5/c5f 0 2024-05-12T04:01:16.909 INFO:tasks.workunit.client.0.smithi032.stdout:2/383: mkdir d1/d82 0 2024-05-12T04:01:16.915 INFO:tasks.workunit.client.0.smithi032.stdout:4/302: creat d0/d3/d31/f53 x:0 0 0 2024-05-12T04:01:16.917 INFO:tasks.workunit.client.1.smithi033.stdout:0/575: mkdir d6/db/d2c/d73/daf/dc3 0 2024-05-12T04:01:16.920 INFO:tasks.workunit.client.0.smithi032.stdout:3/403: creat d15/d24/d2a/d5b/f8e x:0 0 0 2024-05-12T04:01:16.921 INFO:tasks.workunit.client.0.smithi032.stdout:3/404: dread - d15/d20/d22/f58 zero size 2024-05-12T04:01:16.924 INFO:tasks.workunit.client.0.smithi032.stdout:6/362: mknod d7/d9/d5c/d62/c7c 0 2024-05-12T04:01:16.925 INFO:tasks.workunit.client.1.smithi033.stdout:8/477: truncate d1/d4/f1c 2208816 0 2024-05-12T04:01:16.925 INFO:tasks.workunit.client.1.smithi033.stdout:8/478: chown d1/d16/d27/d7c 2513 1 2024-05-12T04:01:16.926 INFO:tasks.workunit.client.0.smithi032.stdout:0/448: link df/d10/d3d/d6c/l77 df/d28/la1 0 2024-05-12T04:01:16.926 INFO:tasks.workunit.client.0.smithi032.stdout:0/449: read - df/d30/d4e/f7e zero size 2024-05-12T04:01:16.930 INFO:tasks.workunit.client.1.smithi033.stdout:5/472: rename d3/d12/d25/d26/d2e to d3/d94 0 2024-05-12T04:01:16.930 INFO:tasks.workunit.client.1.smithi033.stdout:0/576: dread d6/db/d2c/d73/f87 [0,4194304] 0 2024-05-12T04:01:16.932 INFO:tasks.workunit.client.1.smithi033.stdout:5/473: read d3/d12/f1f [379361,114889] 0 2024-05-12T04:01:16.933 INFO:tasks.workunit.client.0.smithi032.stdout:2/384: creat d1/d34/d3e/d67/f83 x:0 0 0 2024-05-12T04:01:16.935 INFO:tasks.workunit.client.1.smithi033.stdout:4/572: fsync dd/d7a/d87/d7e/da7/d38/f44 0 2024-05-12T04:01:16.937 INFO:tasks.workunit.client.1.smithi033.stdout:7/711: write de/fbb [382124,45106] 0 2024-05-12T04:01:16.938 INFO:tasks.workunit.client.1.smithi033.stdout:2/547: creat dc/d13/d39/d8f/d9e/fb4 x:0 0 0 2024-05-12T04:01:16.940 INFO:tasks.workunit.client.1.smithi033.stdout:6/559: creat d4/d1a/d4e/d4f/d94/fb9 x:0 0 0 2024-05-12T04:01:16.941 INFO:tasks.workunit.client.1.smithi033.stdout:6/560: readlink d4/d1a/l61 0 2024-05-12T04:01:16.942 INFO:tasks.workunit.client.1.smithi033.stdout:5/474: dwrite d3/d15/d16/d45/d4b/f6c [0,4194304] 0 2024-05-12T04:01:16.942 INFO:tasks.workunit.client.0.smithi032.stdout:3/405: creat d15/d1b/d65/f8f x:0 0 0 2024-05-12T04:01:16.948 INFO:tasks.workunit.client.1.smithi033.stdout:8/479: creat d1/d16/d27/d4d/d73/f97 x:0 0 0 2024-05-12T04:01:16.952 INFO:tasks.workunit.client.0.smithi032.stdout:6/363: dwrite d7/d9/d60/f71 [0,4194304] 0 2024-05-12T04:01:16.952 INFO:tasks.workunit.client.1.smithi033.stdout:1/503: rename d11/d1c/f34 to d11/d17/d27/d36/d3f/d69/d87/da4/fa8 0 2024-05-12T04:01:16.954 INFO:tasks.workunit.client.1.smithi033.stdout:9/787: mkdir d2/d70/d10d/d11a 0 2024-05-12T04:01:16.956 INFO:tasks.workunit.client.0.smithi032.stdout:2/385: write d1/d34/d3e/d67/f69 [496155,78729] 0 2024-05-12T04:01:16.957 INFO:tasks.workunit.client.0.smithi032.stdout:1/391: creat d1/d7/d49/f81 x:0 0 0 2024-05-12T04:01:16.963 INFO:tasks.workunit.client.0.smithi032.stdout:6/364: write d7/fa [2541599,36090] 0 2024-05-12T04:01:16.963 INFO:tasks.workunit.client.1.smithi033.stdout:4/573: creat dd/d11/d75/fb4 x:0 0 0 2024-05-12T04:01:16.963 INFO:tasks.workunit.client.1.smithi033.stdout:7/712: rmdir de/df/d17/d1b/d36 39 2024-05-12T04:01:16.965 INFO:tasks.workunit.client.1.smithi033.stdout:2/548: creat dc/d13/d1c/d98/fb5 x:0 0 0 2024-05-12T04:01:16.965 INFO:tasks.workunit.client.0.smithi032.stdout:4/303: dread d0/d3/f1c [4194304,4194304] 0 2024-05-12T04:01:16.966 INFO:tasks.workunit.client.0.smithi032.stdout:0/450: creat df/d30/d8d/fa2 x:0 0 0 2024-05-12T04:01:16.966 INFO:tasks.workunit.client.1.smithi033.stdout:2/549: write dc/d17/d32/f5c [1874709,95946] 0 2024-05-12T04:01:16.967 INFO:tasks.workunit.client.0.smithi032.stdout:0/451: truncate df/d10/d3d/d4a/f65 1199990 0 2024-05-12T04:01:16.970 INFO:tasks.workunit.client.0.smithi032.stdout:6/365: creat d7/d9/d48/d67/d25/f7d x:0 0 0 2024-05-12T04:01:16.971 INFO:tasks.workunit.client.1.smithi033.stdout:3/551: getdents db/d17/d3a/d8a 0 2024-05-12T04:01:16.972 INFO:tasks.workunit.client.1.smithi033.stdout:3/552: fdatasync db/f5e 0 2024-05-12T04:01:16.972 INFO:tasks.workunit.client.0.smithi032.stdout:3/406: creat d15/d24/f90 x:0 0 0 2024-05-12T04:01:16.979 INFO:tasks.workunit.client.1.smithi033.stdout:0/577: rename d6/dc/d39/l8b to d6/dc/d58/d6a/lc4 0 2024-05-12T04:01:16.980 INFO:tasks.workunit.client.0.smithi032.stdout:8/410: sync 2024-05-12T04:01:16.981 INFO:tasks.workunit.client.1.smithi033.stdout:0/578: dread d6/dc/d39/f8d [0,4194304] 0 2024-05-12T04:01:16.983 INFO:tasks.workunit.client.1.smithi033.stdout:1/504: unlink d11/d1c/d59/f67 0 2024-05-12T04:01:16.984 INFO:tasks.workunit.client.1.smithi033.stdout:1/505: write d11/d17/f6d [2217148,111846] 0 2024-05-12T04:01:16.984 INFO:tasks.workunit.client.0.smithi032.stdout:1/392: mknod d1/d7/d49/c82 0 2024-05-12T04:01:16.988 INFO:tasks.workunit.client.1.smithi033.stdout:9/788: symlink d2/d4/d29/d32/l11b 0 2024-05-12T04:01:16.992 INFO:tasks.workunit.client.1.smithi033.stdout:1/506: dread f8 [4194304,4194304] 0 2024-05-12T04:01:16.993 INFO:tasks.workunit.client.0.smithi032.stdout:2/386: write d1/fc [2555939,114487] 0 2024-05-12T04:01:16.995 INFO:tasks.workunit.client.0.smithi032.stdout:4/304: symlink d0/d3/l54 0 2024-05-12T04:01:17.006 INFO:tasks.workunit.client.1.smithi033.stdout:2/550: rename f3 to dc/d13/d1f/d6e/fb6 0 2024-05-12T04:01:17.007 INFO:tasks.workunit.client.1.smithi033.stdout:3/553: creat db/d17/d3a/d93/fc6 x:0 0 0 2024-05-12T04:01:17.007 INFO:tasks.workunit.client.1.smithi033.stdout:0/579: rmdir d6/db/d2c/d73/daf 39 2024-05-12T04:01:17.009 INFO:tasks.workunit.client.1.smithi033.stdout:9/789: dread d2/d4/d2b/d78/f93 [0,4194304] 0 2024-05-12T04:01:17.010 INFO:tasks.workunit.client.1.smithi033.stdout:9/790: stat d2/d70/f106 0 2024-05-12T04:01:17.012 INFO:tasks.workunit.client.1.smithi033.stdout:7/713: chown de/df/d17/d1b/l1d 197371 1 2024-05-12T04:01:17.012 INFO:tasks.workunit.client.1.smithi033.stdout:7/714: write de/d2b/d6c/f72 [701750,43856] 0 2024-05-12T04:01:17.014 INFO:tasks.workunit.client.1.smithi033.stdout:7/715: truncate de/df/d17/d1e/d79/fed 770834 0 2024-05-12T04:01:17.017 INFO:tasks.workunit.client.1.smithi033.stdout:8/480: link d1/d16/d27/d28/c6f d1/d16/d27/d4d/d51/d8e/d94/c98 0 2024-05-12T04:01:17.020 INFO:tasks.workunit.client.1.smithi033.stdout:3/554: read - db/f81 zero size 2024-05-12T04:01:17.023 INFO:tasks.workunit.client.1.smithi033.stdout:1/507: rename d11/l26 to d11/d17/d27/d76/d93/la9 0 2024-05-12T04:01:17.026 INFO:tasks.workunit.client.1.smithi033.stdout:0/580: creat d6/dc/d58/d6a/d80/fc5 x:0 0 0 2024-05-12T04:01:17.028 INFO:tasks.workunit.client.1.smithi033.stdout:4/574: truncate dd/d7a/d87/d7e/da7/d66/f6a 71752 0 2024-05-12T04:01:17.029 INFO:tasks.workunit.client.1.smithi033.stdout:6/561: getdents d4/dad 0 2024-05-12T04:01:17.031 INFO:tasks.workunit.client.1.smithi033.stdout:7/716: truncate de/fa1 4547820 0 2024-05-12T04:01:17.036 INFO:tasks.workunit.client.1.smithi033.stdout:0/581: write d6/d11/d57/fb6 [153128,84144] 0 2024-05-12T04:01:17.036 INFO:tasks.workunit.client.1.smithi033.stdout:5/475: dwrite d3/f77 [0,4194304] 0 2024-05-12T04:01:17.038 INFO:tasks.workunit.client.1.smithi033.stdout:4/575: fsync dd/d69/da4/fa8 0 2024-05-12T04:01:17.040 INFO:tasks.workunit.client.1.smithi033.stdout:9/791: dread d2/d4/d29/d32/db7/fbf [0,4194304] 0 2024-05-12T04:01:17.041 INFO:tasks.workunit.client.0.smithi032.stdout:2/387: write d1/d13/f1f [3259477,110375] 0 2024-05-12T04:01:17.041 INFO:tasks.workunit.client.0.smithi032.stdout:0/452: dread df/d28/f59 [0,4194304] 0 2024-05-12T04:01:17.041 INFO:tasks.workunit.client.0.smithi032.stdout:1/393: dwrite d1/d7/d30/f52 [0,4194304] 0 2024-05-12T04:01:17.042 INFO:tasks.workunit.client.1.smithi033.stdout:6/562: rename d4/d43/l46 to d4/dad/lba 0 2024-05-12T04:01:17.046 INFO:tasks.workunit.client.0.smithi032.stdout:0/453: dread - df/d10/d3d/f85 zero size 2024-05-12T04:01:17.047 INFO:tasks.workunit.client.0.smithi032.stdout:0/454: dread - df/d10/d3d/f87 zero size 2024-05-12T04:01:17.050 INFO:tasks.workunit.client.1.smithi033.stdout:8/481: mknod d1/d16/d27/c99 0 2024-05-12T04:01:17.051 INFO:tasks.workunit.client.0.smithi032.stdout:2/388: dread d1/d13/d3d/d41/d2c/f56 [0,4194304] 0 2024-05-12T04:01:17.053 INFO:tasks.workunit.client.1.smithi033.stdout:2/551: creat dc/d17/d32/fb7 x:0 0 0 2024-05-12T04:01:17.053 INFO:tasks.workunit.client.1.smithi033.stdout:2/552: chown dc/d13/d65/db1 1734475172 1 2024-05-12T04:01:17.058 INFO:tasks.workunit.client.0.smithi032.stdout:3/407: mknod d15/d33/d3f/d75/c91 0 2024-05-12T04:01:17.060 INFO:tasks.workunit.client.0.smithi032.stdout:4/305: dread d0/d3/f1c [0,4194304] 0 2024-05-12T04:01:17.062 INFO:tasks.workunit.client.0.smithi032.stdout:4/306: dread - d0/d3/d12/f51 zero size 2024-05-12T04:01:17.062 INFO:tasks.workunit.client.0.smithi032.stdout:4/307: readlink d0/d39/l46 0 2024-05-12T04:01:17.064 INFO:tasks.workunit.client.0.smithi032.stdout:4/308: truncate d0/d3/d12/f51 153558 0 2024-05-12T04:01:17.065 INFO:tasks.workunit.client.1.smithi033.stdout:0/582: mknod d6/d11/d98/cc6 0 2024-05-12T04:01:17.067 INFO:tasks.workunit.client.1.smithi033.stdout:5/476: symlink d3/d94/d54/d57/d73/d74/d7e/l95 0 2024-05-12T04:01:17.067 INFO:tasks.workunit.client.1.smithi033.stdout:5/477: stat d3/d94/l66 0 2024-05-12T04:01:17.068 INFO:tasks.workunit.client.1.smithi033.stdout:3/555: truncate db/d17/d25/d32/f46 549067 0 2024-05-12T04:01:17.070 INFO:tasks.workunit.client.0.smithi032.stdout:2/389: stat d1/c5b 0 2024-05-12T04:01:17.071 INFO:tasks.workunit.client.0.smithi032.stdout:2/390: write d1/d13/d3d/f76 [160982,17351] 0 2024-05-12T04:01:17.073 INFO:tasks.workunit.client.0.smithi032.stdout:2/391: write d1/d13/d3d/d41/d2c/f59 [648710,117991] 0 2024-05-12T04:01:17.073 INFO:tasks.workunit.client.0.smithi032.stdout:2/392: stat d1/f24 0 2024-05-12T04:01:17.074 INFO:tasks.workunit.client.1.smithi033.stdout:7/717: rename de/df/d17/d1e/d24/f4f to de/d20/d38/dec/ffb 0 2024-05-12T04:01:17.076 INFO:tasks.workunit.client.0.smithi032.stdout:3/408: creat d15/d20/d22/d7a/f92 x:0 0 0 2024-05-12T04:01:17.077 INFO:tasks.workunit.client.1.smithi033.stdout:6/563: mkdir d4/d6e/dbb 0 2024-05-12T04:01:17.079 INFO:tasks.workunit.client.0.smithi032.stdout:1/394: unlink d1/c1c 0 2024-05-12T04:01:17.086 INFO:tasks.workunit.client.1.smithi033.stdout:1/508: link d11/d17/d27/f44 d11/d1c/d59/faa 0 2024-05-12T04:01:17.088 INFO:tasks.workunit.client.0.smithi032.stdout:1/395: dwrite d1/d7/d30/d18/f2b [0,4194304] 0 2024-05-12T04:01:17.088 INFO:tasks.workunit.client.1.smithi033.stdout:9/792: dwrite d2/d4/d2b/d6d/d2f/d5f/f61 [0,4194304] 0 2024-05-12T04:01:17.089 INFO:tasks.workunit.client.0.smithi032.stdout:1/396: dread - d1/d7/d30/d35/d54/f73 zero size 2024-05-12T04:01:17.091 INFO:tasks.workunit.client.0.smithi032.stdout:2/393: symlink d1/d13/d6b/l84 0 2024-05-12T04:01:17.095 INFO:tasks.workunit.client.1.smithi033.stdout:5/478: truncate d3/d12/f63 1027324 0 2024-05-12T04:01:17.096 INFO:tasks.workunit.client.0.smithi032.stdout:8/411: getdents d0/d9/d13/d26/d28/d3e 0 2024-05-12T04:01:17.099 INFO:tasks.workunit.client.1.smithi033.stdout:8/482: dwrite d1/d4/fd [0,4194304] 0 2024-05-12T04:01:17.100 INFO:tasks.workunit.client.1.smithi033.stdout:3/556: fdatasync db/d17/d3e/d65/d7d/f86 0 2024-05-12T04:01:17.102 INFO:tasks.workunit.client.1.smithi033.stdout:8/483: read d1/d4/d46/f4f [434134,23407] 0 2024-05-12T04:01:17.103 INFO:tasks.workunit.client.0.smithi032.stdout:0/455: creat df/fa3 x:0 0 0 2024-05-12T04:01:17.104 INFO:tasks.workunit.client.0.smithi032.stdout:1/397: mknod d1/d7/c83 0 2024-05-12T04:01:17.107 INFO:tasks.workunit.client.0.smithi032.stdout:2/394: rename d1/d34/d81 to d1/d13/d70/d85 0 2024-05-12T04:01:17.108 INFO:tasks.workunit.client.1.smithi033.stdout:7/718: chown de/d2b/d6c/dbd/ce7 1 1 2024-05-12T04:01:17.110 INFO:tasks.workunit.client.1.smithi033.stdout:6/564: symlink d4/d1a/d67/lbc 0 2024-05-12T04:01:17.113 INFO:tasks.workunit.client.0.smithi032.stdout:0/456: unlink df/d30/d4e/l50 0 2024-05-12T04:01:17.116 INFO:tasks.workunit.client.0.smithi032.stdout:1/398: unlink d1/d7/d49/f81 0 2024-05-12T04:01:17.119 INFO:tasks.workunit.client.0.smithi032.stdout:1/399: read d1/d7/d30/d18/f41 [7105987,49509] 0 2024-05-12T04:01:17.119 INFO:tasks.workunit.client.1.smithi033.stdout:9/793: symlink d2/d4/d2b/d49/d73/l11c 0 2024-05-12T04:01:17.121 INFO:tasks.workunit.client.0.smithi032.stdout:2/395: mkdir d1/d34/d4a/d86 0 2024-05-12T04:01:17.122 INFO:tasks.workunit.client.0.smithi032.stdout:0/457: dwrite df/f92 [0,4194304] 0 2024-05-12T04:01:17.123 INFO:tasks.workunit.client.0.smithi032.stdout:2/396: read - d1/d34/d3e/d67/f83 zero size 2024-05-12T04:01:17.124 INFO:tasks.workunit.client.0.smithi032.stdout:8/412: fsync d0/f2 0 2024-05-12T04:01:17.130 INFO:tasks.workunit.client.1.smithi033.stdout:1/509: write d11/d17/d27/f31 [1676354,87213] 0 2024-05-12T04:01:17.131 INFO:tasks.workunit.client.0.smithi032.stdout:4/309: getdents d0/d3/d12 0 2024-05-12T04:01:17.133 INFO:tasks.workunit.client.0.smithi032.stdout:0/458: dread df/d22/f39 [0,4194304] 0 2024-05-12T04:01:17.134 INFO:tasks.workunit.client.0.smithi032.stdout:8/413: dwrite d0/d9/d13/f58 [0,4194304] 0 2024-05-12T04:01:17.134 INFO:tasks.workunit.client.1.smithi033.stdout:3/557: rmdir db/d17/d3a/d8a 39 2024-05-12T04:01:17.137 INFO:tasks.workunit.client.1.smithi033.stdout:8/484: creat d1/d16/d29/f9a x:0 0 0 2024-05-12T04:01:17.140 INFO:tasks.workunit.client.0.smithi032.stdout:1/400: rename d1/d7/d30/d35/d46/f4b to d1/d6a/f84 0 2024-05-12T04:01:17.143 INFO:tasks.workunit.client.1.smithi033.stdout:4/576: rename dd/d7a/d87/d7e/da7/f33 to dd/d11/d5b/fb5 0 2024-05-12T04:01:17.144 INFO:tasks.workunit.client.0.smithi032.stdout:2/397: creat d1/d13/d3d/d41/d39/f87 x:0 0 0 2024-05-12T04:01:17.144 INFO:tasks.workunit.client.1.smithi033.stdout:5/479: dwrite d3/d15/d16/d45/d4b/f56 [0,4194304] 0 2024-05-12T04:01:17.146 INFO:tasks.workunit.client.1.smithi033.stdout:7/719: mknod de/d2b/d6c/da0/da6/dd1/cfc 0 2024-05-12T04:01:17.148 INFO:tasks.workunit.client.1.smithi033.stdout:2/553: creat dc/fb8 x:0 0 0 2024-05-12T04:01:17.148 INFO:tasks.workunit.client.0.smithi032.stdout:3/409: getdents d15/d20/d26/d6a/d6e 0 2024-05-12T04:01:17.152 INFO:tasks.workunit.client.1.smithi033.stdout:9/794: mkdir d2/d4/d2b/d49/d11d 0 2024-05-12T04:01:17.152 INFO:tasks.workunit.client.0.smithi032.stdout:4/310: unlink d0/d3/l47 0 2024-05-12T04:01:17.153 INFO:tasks.workunit.client.1.smithi033.stdout:9/795: stat d2/d4/f1f 0 2024-05-12T04:01:17.153 INFO:tasks.workunit.client.0.smithi032.stdout:4/311: stat d0/d3/d12 0 2024-05-12T04:01:17.155 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:16 smithi032 ceph-mon[40008]: pgmap v36: 65 pgs: 65 active+clean; 1.6 GiB data, 6.4 GiB used, 530 GiB / 536 GiB avail; 43 MiB/s rd, 214 MiB/s wr, 376 op/s 2024-05-12T04:01:17.156 INFO:tasks.workunit.client.1.smithi033.stdout:1/510: mkdir d11/d17/d27/d36/d46/d4c/dab 0 2024-05-12T04:01:17.156 INFO:tasks.workunit.client.0.smithi032.stdout:0/459: symlink df/d10/d3b/d75/la4 0 2024-05-12T04:01:17.157 INFO:tasks.workunit.client.0.smithi032.stdout:1/401: mkdir d1/d7/d30/d35/d54/d85 0 2024-05-12T04:01:17.157 INFO:tasks.workunit.client.0.smithi032.stdout:0/460: chown df/d28/l31 1038 1 2024-05-12T04:01:17.158 INFO:tasks.workunit.client.0.smithi032.stdout:1/402: truncate d1/dd/f7c 716426 0 2024-05-12T04:01:17.160 INFO:tasks.workunit.client.1.smithi033.stdout:3/558: rmdir db/d42 39 2024-05-12T04:01:17.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:16 smithi033 ceph-mon[49525]: pgmap v36: 65 pgs: 65 active+clean; 1.6 GiB data, 6.4 GiB used, 530 GiB / 536 GiB avail; 43 MiB/s rd, 214 MiB/s wr, 376 op/s 2024-05-12T04:01:17.168 INFO:tasks.workunit.client.0.smithi032.stdout:8/414: symlink d0/d7/l96 0 2024-05-12T04:01:17.171 INFO:tasks.workunit.client.0.smithi032.stdout:9/286: sync 2024-05-12T04:01:17.171 INFO:tasks.workunit.client.0.smithi032.stdout:5/461: sync 2024-05-12T04:01:17.171 INFO:tasks.workunit.client.0.smithi032.stdout:7/364: sync 2024-05-12T04:01:17.171 INFO:tasks.workunit.client.1.smithi033.stdout:6/565: mkdir d4/d1a/d67/d6f/dac/dbd 0 2024-05-12T04:01:17.174 INFO:tasks.workunit.client.0.smithi032.stdout:7/365: write dd/d1b/d1e/d45/d55/d60/f6b [291327,13300] 0 2024-05-12T04:01:17.174 INFO:tasks.workunit.client.0.smithi032.stdout:0/461: mkdir df/d30/d4e/da5 0 2024-05-12T04:01:17.176 INFO:tasks.workunit.client.1.smithi033.stdout:7/720: rename de/df/d60/c5e to de/df/d65/dbc/cfd 0 2024-05-12T04:01:17.178 INFO:tasks.workunit.client.0.smithi032.stdout:2/398: rename d1/d13/d3d/d41/d2c/f54 to d1/d13/d70/d85/f88 0 2024-05-12T04:01:17.180 INFO:tasks.workunit.client.0.smithi032.stdout:9/287: dwrite d0/d3/d6/d7/fc [0,4194304] 0 2024-05-12T04:01:17.180 INFO:tasks.workunit.client.1.smithi033.stdout:2/554: creat dc/d13/d1c/d98/fb9 x:0 0 0 2024-05-12T04:01:17.180 INFO:tasks.workunit.client.0.smithi032.stdout:9/288: fdatasync d0/d3/f25 0 2024-05-12T04:01:17.182 INFO:tasks.workunit.client.1.smithi033.stdout:0/583: getdents d6/dc/d6e 0 2024-05-12T04:01:17.182 INFO:tasks.workunit.client.1.smithi033.stdout:7/721: dwrite de/df/d17/d1e/d24/f57 [4194304,4194304] 0 2024-05-12T04:01:17.183 INFO:tasks.workunit.client.0.smithi032.stdout:9/289: write d0/d3/d6/d7/da/f53 [376276,118926] 0 2024-05-12T04:01:17.187 INFO:tasks.workunit.client.0.smithi032.stdout:8/415: write d0/d9/f1c [2019978,69124] 0 2024-05-12T04:01:17.188 INFO:tasks.workunit.client.0.smithi032.stdout:8/416: stat d0/d9/d1b/d66/l8a 0 2024-05-12T04:01:17.189 INFO:tasks.workunit.client.0.smithi032.stdout:8/417: fdatasync d0/d9/d1b/d66/f95 0 2024-05-12T04:01:17.190 INFO:tasks.workunit.client.1.smithi033.stdout:1/511: rmdir d11/d17/d27/d36/d46 39 2024-05-12T04:01:17.192 INFO:tasks.workunit.client.0.smithi032.stdout:6/366: sync 2024-05-12T04:01:17.193 INFO:tasks.workunit.client.1.smithi033.stdout:4/577: mknod dd/d7a/d87/d7e/da7/d66/d42/cb6 0 2024-05-12T04:01:17.194 INFO:tasks.workunit.client.1.smithi033.stdout:6/566: fsync d4/d1a/f26 0 2024-05-12T04:01:17.195 INFO:tasks.workunit.client.0.smithi032.stdout:0/462: symlink df/d10/d48/d70/la6 0 2024-05-12T04:01:17.195 INFO:tasks.workunit.client.0.smithi032.stdout:5/462: dread d4/d7/d14/f8c [0,4194304] 0 2024-05-12T04:01:17.197 INFO:tasks.workunit.client.1.smithi033.stdout:1/512: dwrite d11/d1c/d59/f86 [0,4194304] 0 2024-05-12T04:01:17.197 INFO:tasks.workunit.client.0.smithi032.stdout:5/463: chown d4/d7/d14/d26/d33/d4a/f51 20439700 1 2024-05-12T04:01:17.199 INFO:tasks.workunit.client.0.smithi032.stdout:5/464: fsync d4/f49 0 2024-05-12T04:01:17.204 INFO:tasks.workunit.client.1.smithi033.stdout:0/584: dwrite d6/d11/d98/fa1 [0,4194304] 0 2024-05-12T04:01:17.205 INFO:tasks.workunit.client.0.smithi032.stdout:0/463: dwrite df/d10/d3d/f85 [0,4194304] 0 2024-05-12T04:01:17.206 INFO:tasks.workunit.client.1.smithi033.stdout:7/722: mknod de/df/d17/d1e/dc4/cfe 0 2024-05-12T04:01:17.207 INFO:tasks.workunit.client.1.smithi033.stdout:7/723: fsync de/df/d7a/f9f 0 2024-05-12T04:01:17.209 INFO:tasks.workunit.client.0.smithi032.stdout:0/464: truncate df/d10/f3c 2095264 0 2024-05-12T04:01:17.211 INFO:tasks.workunit.client.1.smithi033.stdout:7/724: stat de/df/d17/d1e/dcc/fce 0 2024-05-12T04:01:17.214 INFO:tasks.workunit.client.1.smithi033.stdout:3/559: link db/d59/f8d db/d17/d25/d32/d53/fc7 0 2024-05-12T04:01:17.215 INFO:tasks.workunit.client.0.smithi032.stdout:8/418: truncate d0/d9/d1b/d27/f69 174137 0 2024-05-12T04:01:17.217 INFO:tasks.workunit.client.1.smithi033.stdout:9/796: dwrite d2/d4/d29/d32/db7/fbf [0,4194304] 0 2024-05-12T04:01:17.218 INFO:tasks.workunit.client.1.smithi033.stdout:9/797: read - d2/d4/d2b/d6d/d4c/ffe zero size 2024-05-12T04:01:17.221 INFO:tasks.workunit.client.0.smithi032.stdout:7/366: mkdir dd/d7b 0 2024-05-12T04:01:17.223 INFO:tasks.workunit.client.0.smithi032.stdout:6/367: creat d7/d9/d48/d67/d25/f7e x:0 0 0 2024-05-12T04:01:17.223 INFO:tasks.workunit.client.0.smithi032.stdout:2/399: creat d1/d13/d3d/d41/d58/f89 x:0 0 0 2024-05-12T04:01:17.223 INFO:tasks.workunit.client.0.smithi032.stdout:3/410: dwrite d15/d20/d22/d7a/f85 [0,4194304] 0 2024-05-12T04:01:17.225 INFO:tasks.workunit.client.0.smithi032.stdout:7/367: chown dd/d1b/d1e/d45/d55 2 1 2024-05-12T04:01:17.225 INFO:tasks.workunit.client.0.smithi032.stdout:5/465: creat d4/d7/d14/fa1 x:0 0 0 2024-05-12T04:01:17.225 INFO:tasks.workunit.client.0.smithi032.stdout:6/368: chown d7/d9/l11 98373279 1 2024-05-12T04:01:17.226 INFO:tasks.workunit.client.1.smithi033.stdout:0/585: truncate d6/d11/d57/fc2 894101 0 2024-05-12T04:01:17.227 INFO:tasks.workunit.client.0.smithi032.stdout:7/368: dread - dd/d1b/d1e/d45/f73 zero size 2024-05-12T04:01:17.228 INFO:tasks.workunit.client.0.smithi032.stdout:5/466: chown d4/d7/d58/c9f 77838255 1 2024-05-12T04:01:17.229 INFO:tasks.workunit.client.1.smithi033.stdout:7/725: symlink de/d2f/d70/lff 0 2024-05-12T04:01:17.230 INFO:tasks.workunit.client.1.smithi033.stdout:6/567: symlink d4/d1a/d67/d6f/dac/dbd/lbe 0 2024-05-12T04:01:17.231 INFO:tasks.workunit.client.0.smithi032.stdout:6/369: read f3 [1059459,51833] 0 2024-05-12T04:01:17.232 INFO:tasks.workunit.client.1.smithi033.stdout:7/726: read de/d20/d38/f61 [2993088,17092] 0 2024-05-12T04:01:17.232 INFO:tasks.workunit.client.0.smithi032.stdout:4/312: link d0/c25 d0/d3/c55 0 2024-05-12T04:01:17.234 INFO:tasks.workunit.client.1.smithi033.stdout:3/560: mkdir db/d17/d3a/d93/dc8 0 2024-05-12T04:01:17.234 INFO:tasks.workunit.client.1.smithi033.stdout:1/513: mknod d11/d17/d84/da3/cac 0 2024-05-12T04:01:17.238 INFO:tasks.workunit.client.1.smithi033.stdout:0/586: unlink d6/dc/f46 0 2024-05-12T04:01:17.240 INFO:tasks.workunit.client.1.smithi033.stdout:8/485: write d1/d16/d20/f2f [2139118,52253] 0 2024-05-12T04:01:17.242 INFO:tasks.workunit.client.1.smithi033.stdout:6/568: write d4/d1a/d4e/d4f/f63 [1267346,86994] 0 2024-05-12T04:01:17.243 INFO:tasks.workunit.client.1.smithi033.stdout:7/727: mkdir de/d20/d64/d100 0 2024-05-12T04:01:17.244 INFO:tasks.workunit.client.1.smithi033.stdout:7/728: chown de/df/d65/de3 28116925 1 2024-05-12T04:01:17.245 INFO:tasks.workunit.client.1.smithi033.stdout:7/729: dread - de/d2b/d6c/db5/dd0/ff2 zero size 2024-05-12T04:01:17.246 INFO:tasks.workunit.client.1.smithi033.stdout:7/730: dread - de/d2f/fe9 zero size 2024-05-12T04:01:17.246 INFO:tasks.workunit.client.1.smithi033.stdout:7/731: chown de/d2f/d70 219842265 1 2024-05-12T04:01:17.247 INFO:tasks.workunit.client.1.smithi033.stdout:9/798: symlink d2/dd/l11e 0 2024-05-12T04:01:17.247 INFO:tasks.workunit.client.1.smithi033.stdout:7/732: write de/d20/d38/f9e [479743,28877] 0 2024-05-12T04:01:17.248 INFO:tasks.workunit.client.1.smithi033.stdout:7/733: stat de/d2b/d6c 0 2024-05-12T04:01:17.251 INFO:tasks.workunit.client.1.smithi033.stdout:0/587: mkdir d6/d11/dc7 0 2024-05-12T04:01:17.255 INFO:tasks.workunit.client.1.smithi033.stdout:6/569: fdatasync d4/f9a 0 2024-05-12T04:01:17.256 INFO:tasks.workunit.client.1.smithi033.stdout:6/570: write d4/d1a/d3c/fa8 [280601,107959] 0 2024-05-12T04:01:17.257 INFO:tasks.workunit.client.0.smithi032.stdout:1/403: truncate d1/d25/f4d 1204613 0 2024-05-12T04:01:17.257 INFO:tasks.workunit.client.1.smithi033.stdout:6/571: read - d4/d1a/d6c/f7f zero size 2024-05-12T04:01:17.260 INFO:tasks.workunit.client.0.smithi032.stdout:9/290: creat d0/d3/f5c x:0 0 0 2024-05-12T04:01:17.261 INFO:tasks.workunit.client.1.smithi033.stdout:3/561: truncate db/d59/f8d 6983352 0 2024-05-12T04:01:17.263 INFO:tasks.workunit.client.1.smithi033.stdout:6/572: dwrite d4/d1a/fb2 [0,4194304] 0 2024-05-12T04:01:17.263 INFO:tasks.workunit.client.0.smithi032.stdout:7/369: unlink dd/d1b/d1e/d45/c65 0 2024-05-12T04:01:17.263 INFO:tasks.workunit.client.1.smithi033.stdout:3/562: write db/d17/d3e/d65/d91/fb3 [758103,28643] 0 2024-05-12T04:01:17.266 INFO:tasks.workunit.client.0.smithi032.stdout:5/467: mknod d4/d7/d58/d5a/ca2 0 2024-05-12T04:01:17.267 INFO:tasks.workunit.client.0.smithi032.stdout:5/468: stat d4/d7/d14/d26/d33/d4a/c89 0 2024-05-12T04:01:17.268 INFO:tasks.workunit.client.1.smithi033.stdout:9/799: mknod d2/d70/d7b/c11f 0 2024-05-12T04:01:17.272 INFO:tasks.workunit.client.1.smithi033.stdout:7/734: mkdir de/d2b/d6c/dbd/d101 0 2024-05-12T04:01:17.272 INFO:tasks.workunit.client.1.smithi033.stdout:6/573: dwrite d4/d5/f5c [0,4194304] 0 2024-05-12T04:01:17.273 INFO:tasks.workunit.client.0.smithi032.stdout:6/370: creat d7/d9/d48/d67/d25/f7f x:0 0 0 2024-05-12T04:01:17.274 INFO:tasks.workunit.client.1.smithi033.stdout:4/578: dwrite dd/d7a/d87/d7e/da7/d66/d42/f4c [0,4194304] 0 2024-05-12T04:01:17.274 INFO:tasks.workunit.client.0.smithi032.stdout:6/371: fdatasync d7/d9/d5c/f7b 0 2024-05-12T04:01:17.275 INFO:tasks.workunit.client.1.smithi033.stdout:6/574: chown d4/d1a/d4e/d4f/c5b 376077449 1 2024-05-12T04:01:17.276 INFO:tasks.workunit.client.1.smithi033.stdout:2/555: truncate dc/d17/d27/d2f/d49/d59/f80 257299 0 2024-05-12T04:01:17.277 INFO:tasks.workunit.client.1.smithi033.stdout:7/735: write de/df/d17/d1e/d79/fed [1543207,83542] 0 2024-05-12T04:01:17.277 INFO:tasks.workunit.client.1.smithi033.stdout:6/575: stat d4/d1a/d4e/f96 0 2024-05-12T04:01:17.281 INFO:tasks.workunit.client.0.smithi032.stdout:8/419: mkdir d0/d97 0 2024-05-12T04:01:17.284 INFO:tasks.workunit.client.1.smithi033.stdout:7/736: dwrite de/d2b/d6c/f5a [0,4194304] 0 2024-05-12T04:01:17.286 INFO:tasks.workunit.client.1.smithi033.stdout:1/514: rmdir d11/d17/d27/d36/d3f/d69/d87/da4 39 2024-05-12T04:01:17.286 INFO:tasks.workunit.client.1.smithi033.stdout:7/737: write de/d2b/d6c/f5a [3118290,15645] 0 2024-05-12T04:01:17.293 INFO:tasks.workunit.client.1.smithi033.stdout:9/800: dread - d2/d4/d2b/d49/d73/fdb zero size 2024-05-12T04:01:17.296 INFO:tasks.workunit.client.1.smithi033.stdout:0/588: mkdir d6/d11/dc7/dc8 0 2024-05-12T04:01:17.297 INFO:tasks.workunit.client.0.smithi032.stdout:7/370: mknod dd/d1b/d43/d5e/c7c 0 2024-05-12T04:01:17.298 INFO:tasks.workunit.client.0.smithi032.stdout:7/371: chown dd/d1b/d1e/c32 63566506 1 2024-05-12T04:01:17.301 INFO:tasks.workunit.client.0.smithi032.stdout:5/469: rmdir d4/d7/d14/d26/d3c 39 2024-05-12T04:01:17.302 INFO:tasks.workunit.client.0.smithi032.stdout:5/470: stat d4/d7/d14/d26/d33/d4a/c64 0 2024-05-12T04:01:17.302 INFO:tasks.workunit.client.0.smithi032.stdout:5/471: chown d4/d7/d14/d39/l1f 9738143 1 2024-05-12T04:01:17.306 INFO:tasks.workunit.client.1.smithi033.stdout:8/486: rename d1/d16/d27/d4d/l53 to d1/d4/d1d/d36/d3c/l9b 0 2024-05-12T04:01:17.307 INFO:tasks.workunit.client.0.smithi032.stdout:6/372: write d7/f51 [150638,68420] 0 2024-05-12T04:01:17.311 INFO:tasks.workunit.client.0.smithi032.stdout:4/313: chown d0/d3/d24/l37 16293 1 2024-05-12T04:01:17.311 INFO:tasks.workunit.client.0.smithi032.stdout:6/373: dread d7/f32 [0,4194304] 0 2024-05-12T04:01:17.312 INFO:tasks.workunit.client.0.smithi032.stdout:8/420: readlink d0/d9/d13/d21/l94 0 2024-05-12T04:01:17.313 INFO:tasks.workunit.client.0.smithi032.stdout:8/421: dread - d0/d7/f6c zero size 2024-05-12T04:01:17.314 INFO:tasks.workunit.client.1.smithi033.stdout:6/576: symlink d4/d1a/d6c/lbf 0 2024-05-12T04:01:17.314 INFO:tasks.workunit.client.0.smithi032.stdout:8/422: write d0/d9/d1b/d27/d88/f79 [2713388,29242] 0 2024-05-12T04:01:17.316 INFO:tasks.workunit.client.0.smithi032.stdout:8/423: write d0/d9/d13/d21/d25/f83 [854026,96234] 0 2024-05-12T04:01:17.317 INFO:tasks.workunit.client.0.smithi032.stdout:3/411: rename d15/d1b/d65 to d15/d24/d93 0 2024-05-12T04:01:17.329 INFO:tasks.workunit.client.0.smithi032.stdout:5/472: fdatasync d4/d7/d14/d26/d33/f61 0 2024-05-12T04:01:17.330 INFO:tasks.workunit.client.1.smithi033.stdout:9/801: symlink d2/d4/d2b/dc7/l120 0 2024-05-12T04:01:17.332 INFO:tasks.workunit.client.1.smithi033.stdout:0/589: creat d6/dc/d58/d6a/fc9 x:0 0 0 2024-05-12T04:01:17.333 INFO:tasks.workunit.client.1.smithi033.stdout:0/590: readlink d6/dc/d39/l8e 0 2024-05-12T04:01:17.333 INFO:tasks.workunit.client.1.smithi033.stdout:0/591: chown d6/dc/d39/l3d 3003548 1 2024-05-12T04:01:17.336 INFO:tasks.workunit.client.0.smithi032.stdout:0/465: dwrite df/d10/d3b/d75/f78 [0,4194304] 0 2024-05-12T04:01:17.337 INFO:tasks.workunit.client.1.smithi033.stdout:8/487: fsync d1/d4/d1d/d36/d3c/f55 0 2024-05-12T04:01:17.340 INFO:tasks.workunit.client.0.smithi032.stdout:0/466: write df/d10/d3d/d4a/f2d [1496376,80251] 0 2024-05-12T04:01:17.341 INFO:tasks.workunit.client.0.smithi032.stdout:6/374: unlink d7/d9/d48/d67/d1a/d20/f79 0 2024-05-12T04:01:17.343 INFO:tasks.workunit.client.0.smithi032.stdout:2/400: truncate d1/d13/f19 2637362 0 2024-05-12T04:01:17.350 INFO:tasks.workunit.client.1.smithi033.stdout:8/488: dread d1/f2 [0,4194304] 0 2024-05-12T04:01:17.352 INFO:tasks.workunit.client.0.smithi032.stdout:7/372: link dd/d1b/d1e/f68 dd/d7b/f7d 0 2024-05-12T04:01:17.353 INFO:tasks.workunit.client.1.smithi033.stdout:6/577: dread d4/d1a/d4e/fb5 [0,4194304] 0 2024-05-12T04:01:17.353 INFO:tasks.workunit.client.1.smithi033.stdout:0/592: dread d6/d11/f33 [0,4194304] 0 2024-05-12T04:01:17.354 INFO:tasks.workunit.client.0.smithi032.stdout:5/473: creat d4/d7/d14/fa3 x:0 0 0 2024-05-12T04:01:17.356 INFO:tasks.workunit.client.1.smithi033.stdout:0/593: chown d6/db/d2c/d82 4346 1 2024-05-12T04:01:17.356 INFO:tasks.workunit.client.1.smithi033.stdout:4/579: dwrite dd/d11/f2f [0,4194304] 0 2024-05-12T04:01:17.356 INFO:tasks.workunit.client.1.smithi033.stdout:9/802: symlink d2/d4/d2b/d6d/d2f/l121 0 2024-05-12T04:01:17.359 INFO:tasks.workunit.client.1.smithi033.stdout:9/803: read - d2/d4/d2b/d6d/db5/fa0 zero size 2024-05-12T04:01:17.359 INFO:tasks.workunit.client.1.smithi033.stdout:2/556: dwrite dc/d17/d27/d2f/f75 [0,4194304] 0 2024-05-12T04:01:17.361 INFO:tasks.workunit.client.1.smithi033.stdout:2/557: read - dc/d17/d32/fb7 zero size 2024-05-12T04:01:17.361 INFO:tasks.workunit.client.0.smithi032.stdout:4/314: mknod d0/d3/c56 0 2024-05-12T04:01:17.363 INFO:tasks.workunit.client.0.smithi032.stdout:9/291: truncate d0/d3/d6/d7/da/d17/f20 122504 0 2024-05-12T04:01:17.364 INFO:tasks.workunit.client.1.smithi033.stdout:9/804: dread d2/d4/d2b/d6d/d2f/d5f/f61 [0,4194304] 0 2024-05-12T04:01:17.365 INFO:tasks.workunit.client.1.smithi033.stdout:1/515: getdents d11/d17/d27/d36/d46/d4c/dab 0 2024-05-12T04:01:17.365 INFO:tasks.workunit.client.1.smithi033.stdout:2/558: dread dc/d17/d27/d2f/d49/d59/f79 [0,4194304] 0 2024-05-12T04:01:17.370 INFO:tasks.workunit.client.1.smithi033.stdout:3/563: write db/d17/d3a/f66 [681578,74793] 0 2024-05-12T04:01:17.370 INFO:tasks.workunit.client.1.smithi033.stdout:7/738: getdents de/df/d17/d1b/d36 0 2024-05-12T04:01:17.370 INFO:tasks.workunit.client.0.smithi032.stdout:4/315: chown d0/d3/d24/f2d 991987 1 2024-05-12T04:01:17.371 INFO:tasks.workunit.client.1.smithi033.stdout:8/489: rename d1/d16/d27/l52 to d1/l9c 0 2024-05-12T04:01:17.373 INFO:tasks.workunit.client.0.smithi032.stdout:9/292: write d0/d3/d6/d7/f2c [1077719,80517] 0 2024-05-12T04:01:17.374 INFO:tasks.workunit.client.0.smithi032.stdout:5/474: dread d4/d7/d14/d26/f57 [0,4194304] 0 2024-05-12T04:01:17.374 INFO:tasks.workunit.client.0.smithi032.stdout:1/404: dwrite d1/f13 [0,4194304] 0 2024-05-12T04:01:17.375 INFO:tasks.workunit.client.1.smithi033.stdout:6/578: rmdir d4/d43/d57 39 2024-05-12T04:01:17.375 INFO:tasks.workunit.client.0.smithi032.stdout:6/375: creat d7/d9/d60/f80 x:0 0 0 2024-05-12T04:01:17.378 INFO:tasks.workunit.client.0.smithi032.stdout:0/467: creat df/d30/d4e/d55/fa7 x:0 0 0 2024-05-12T04:01:17.379 INFO:tasks.workunit.client.1.smithi033.stdout:8/490: dwrite d1/d4/d1d/d36/f50 [0,4194304] 0 2024-05-12T04:01:17.380 INFO:tasks.workunit.client.1.smithi033.stdout:8/491: write d1/d4/fd [3312882,21078] 0 2024-05-12T04:01:17.381 INFO:tasks.workunit.client.0.smithi032.stdout:4/316: dread d0/d3/f1c [4194304,4194304] 0 2024-05-12T04:01:17.383 INFO:tasks.workunit.client.0.smithi032.stdout:0/468: truncate df/d30/d4e/d55/f7f 91403 0 2024-05-12T04:01:17.386 INFO:tasks.workunit.client.1.smithi033.stdout:4/580: rmdir dd/d11 39 2024-05-12T04:01:17.387 INFO:tasks.workunit.client.1.smithi033.stdout:4/581: read - dd/f3b zero size 2024-05-12T04:01:17.387 INFO:tasks.workunit.client.0.smithi032.stdout:3/412: write d15/f4d [298077,49529] 0 2024-05-12T04:01:17.388 INFO:tasks.workunit.client.1.smithi033.stdout:8/492: dwrite d1/d4/d1d/d36/f74 [0,4194304] 0 2024-05-12T04:01:17.398 INFO:tasks.workunit.client.0.smithi032.stdout:7/373: creat dd/d1b/d1e/f7e x:0 0 0 2024-05-12T04:01:17.406 INFO:tasks.workunit.client.1.smithi033.stdout:2/559: mknod dc/d13/d1f/d6e/cba 0 2024-05-12T04:01:17.410 INFO:tasks.workunit.client.1.smithi033.stdout:3/564: fdatasync db/f81 0 2024-05-12T04:01:17.412 INFO:tasks.workunit.client.0.smithi032.stdout:5/475: unlink d4/d7/d14/d26/l94 0 2024-05-12T04:01:17.413 INFO:tasks.workunit.client.1.smithi033.stdout:7/739: truncate de/df/d17/f34 38885 0 2024-05-12T04:01:17.414 INFO:tasks.workunit.client.0.smithi032.stdout:1/405: mknod d1/d7/d30/d18/c86 0 2024-05-12T04:01:17.415 INFO:tasks.workunit.client.0.smithi032.stdout:1/406: chown d1/d7/f7f 0 1 2024-05-12T04:01:17.419 INFO:tasks.workunit.client.0.smithi032.stdout:4/317: rmdir d0/d3/d12/d1d 39 2024-05-12T04:01:17.421 INFO:tasks.workunit.client.1.smithi033.stdout:6/579: truncate d4/d43/f60 870618 0 2024-05-12T04:01:17.421 INFO:tasks.workunit.client.1.smithi033.stdout:6/580: readlink d4/d1a/d3c/l4b 0 2024-05-12T04:01:17.422 INFO:tasks.workunit.client.0.smithi032.stdout:2/401: mkdir d1/d13/d3d/d8a 0 2024-05-12T04:01:17.423 INFO:tasks.workunit.client.1.smithi033.stdout:0/594: mknod d6/db/d2c/cca 0 2024-05-12T04:01:17.424 INFO:tasks.workunit.client.0.smithi032.stdout:3/413: mknod d15/d20/d26/d6a/c94 0 2024-05-12T04:01:17.426 INFO:tasks.workunit.client.1.smithi033.stdout:5/480: sync 2024-05-12T04:01:17.426 INFO:tasks.workunit.client.0.smithi032.stdout:8/424: creat d0/d9/d1b/f98 x:0 0 0 2024-05-12T04:01:17.428 INFO:tasks.workunit.client.0.smithi032.stdout:8/425: write d0/d4b/f80 [436644,96329] 0 2024-05-12T04:01:17.430 INFO:tasks.workunit.client.1.smithi033.stdout:0/595: dwrite d6/dc/d58/d6a/fc9 [0,4194304] 0 2024-05-12T04:01:17.431 INFO:tasks.workunit.client.1.smithi033.stdout:4/582: creat dd/d7a/d87/fb7 x:0 0 0 2024-05-12T04:01:17.432 INFO:tasks.workunit.client.1.smithi033.stdout:1/516: dwrite d11/d17/d27/f48 [0,4194304] 0 2024-05-12T04:01:17.435 INFO:tasks.workunit.client.1.smithi033.stdout:8/493: rmdir d1/d16 39 2024-05-12T04:01:17.437 INFO:tasks.workunit.client.0.smithi032.stdout:5/476: mkdir d4/d7/d14/d39/d1b/da4 0 2024-05-12T04:01:17.440 INFO:tasks.workunit.client.0.smithi032.stdout:1/407: mkdir d1/d7/d30/d18/d87 0 2024-05-12T04:01:17.446 INFO:tasks.workunit.client.0.smithi032.stdout:6/376: link d7/f66 d7/de/d2e/f81 0 2024-05-12T04:01:17.447 INFO:tasks.workunit.client.1.smithi033.stdout:9/805: getdents d2/d4/d29/d3c/dda 0 2024-05-12T04:01:17.447 INFO:tasks.workunit.client.0.smithi032.stdout:6/377: write d7/d9/d48/d67/d25/f3e [3783425,61227] 0 2024-05-12T04:01:17.449 INFO:tasks.workunit.client.0.smithi032.stdout:4/318: rmdir d0/d3/d31 39 2024-05-12T04:01:17.450 INFO:tasks.workunit.client.1.smithi033.stdout:2/560: stat dc/d17/d32/d84/l96 0 2024-05-12T04:01:17.451 INFO:tasks.workunit.client.1.smithi033.stdout:2/561: write f6 [1208900,42996] 0 2024-05-12T04:01:17.452 INFO:tasks.workunit.client.0.smithi032.stdout:2/402: mknod d1/d34/c8b 0 2024-05-12T04:01:17.452 INFO:tasks.workunit.client.0.smithi032.stdout:2/403: chown d1/d13/d3d/d41/d2c/c51 2 1 2024-05-12T04:01:17.455 INFO:tasks.workunit.client.1.smithi033.stdout:3/565: write db/f81 [80878,68439] 0 2024-05-12T04:01:17.456 INFO:tasks.workunit.client.0.smithi032.stdout:7/374: creat dd/d1b/d43/d74/f7f x:0 0 0 2024-05-12T04:01:17.457 INFO:tasks.workunit.client.0.smithi032.stdout:7/375: read - dd/d1b/d1e/d45/d64/f6d zero size 2024-05-12T04:01:17.459 INFO:tasks.workunit.client.0.smithi032.stdout:8/426: chown d0/d9/d1b/d27/d88/d56/d63/l6a 7828 1 2024-05-12T04:01:17.467 INFO:tasks.workunit.client.0.smithi032.stdout:3/414: write f14 [7650703,18296] 0 2024-05-12T04:01:17.469 INFO:tasks.workunit.client.0.smithi032.stdout:7/376: dread dd/fe [0,4194304] 0 2024-05-12T04:01:17.473 INFO:tasks.workunit.client.0.smithi032.stdout:7/377: dread dd/d1b/d1e/d45/d55/f5b [0,4194304] 0 2024-05-12T04:01:17.473 INFO:tasks.workunit.client.1.smithi033.stdout:7/740: dwrite de/d20/f21 [4194304,4194304] 0 2024-05-12T04:01:17.475 INFO:tasks.workunit.client.0.smithi032.stdout:8/427: symlink d0/d9/d13/d26/d28/l99 0 2024-05-12T04:01:17.476 INFO:tasks.workunit.client.0.smithi032.stdout:8/428: write d0/d9/d13/d26/f55 [403560,53354] 0 2024-05-12T04:01:17.478 INFO:tasks.workunit.client.0.smithi032.stdout:1/408: creat d1/d6a/d72/f88 x:0 0 0 2024-05-12T04:01:17.480 INFO:tasks.workunit.client.1.smithi033.stdout:7/741: dwrite de/d20/d38/f9e [0,4194304] 0 2024-05-12T04:01:17.481 INFO:tasks.workunit.client.1.smithi033.stdout:7/742: stat de/df/d60/fc3 0 2024-05-12T04:01:17.482 INFO:tasks.workunit.client.1.smithi033.stdout:7/743: truncate de/d20/d64/fe2 177936 0 2024-05-12T04:01:17.483 INFO:tasks.workunit.client.0.smithi032.stdout:6/378: link d7/d9/d48/f5d d7/d9/f82 0 2024-05-12T04:01:17.486 INFO:tasks.workunit.client.0.smithi032.stdout:2/404: truncate d1/d13/f18 3041616 0 2024-05-12T04:01:17.492 INFO:tasks.workunit.client.0.smithi032.stdout:5/477: creat d4/d7/fa5 x:0 0 0 2024-05-12T04:01:17.495 INFO:tasks.workunit.client.0.smithi032.stdout:3/415: mknod d15/d24/d2a/d8c/c95 0 2024-05-12T04:01:17.498 INFO:tasks.workunit.client.1.smithi033.stdout:6/581: creat d4/d1a/d67/d6f/dac/dbd/fc0 x:0 0 0 2024-05-12T04:01:17.498 INFO:tasks.workunit.client.0.smithi032.stdout:4/319: symlink d0/d3/d12/d1d/l57 0 2024-05-12T04:01:17.501 INFO:tasks.workunit.client.0.smithi032.stdout:2/405: rename d1/d13/d3d/f76 to d1/d34/f8c 0 2024-05-12T04:01:17.502 INFO:tasks.workunit.client.0.smithi032.stdout:6/379: read d7/d9/d48/d67/f45 [653379,53720] 0 2024-05-12T04:01:17.504 INFO:tasks.workunit.client.1.smithi033.stdout:4/583: chown dd/d11/c27 132290220 1 2024-05-12T04:01:17.505 INFO:tasks.workunit.client.1.smithi033.stdout:1/517: creat d11/d17/d9e/fad x:0 0 0 2024-05-12T04:01:17.506 INFO:tasks.workunit.client.0.smithi032.stdout:3/416: chown d15/d20/c60 1362758 1 2024-05-12T04:01:17.507 INFO:tasks.workunit.client.1.smithi033.stdout:1/518: fdatasync d11/d1c/d59/f7f 0 2024-05-12T04:01:17.507 INFO:tasks.workunit.client.1.smithi033.stdout:0/596: dread d6/dc/f27 [0,4194304] 0 2024-05-12T04:01:17.512 INFO:tasks.workunit.client.0.smithi032.stdout:4/320: dread d0/d3/d12/f48 [0,4194304] 0 2024-05-12T04:01:17.514 INFO:tasks.workunit.client.1.smithi033.stdout:2/562: rmdir dc/d13/d1f/d6e 39 2024-05-12T04:01:17.519 INFO:tasks.workunit.client.0.smithi032.stdout:3/417: creat d15/d20/d22/f96 x:0 0 0 2024-05-12T04:01:17.521 INFO:tasks.workunit.client.0.smithi032.stdout:1/409: rmdir d1/d7/d30/d35/d46 0 2024-05-12T04:01:17.522 INFO:tasks.workunit.client.0.smithi032.stdout:1/410: write d1/d7/d30/d35/f48 [226041,84640] 0 2024-05-12T04:01:17.525 INFO:tasks.workunit.client.1.smithi033.stdout:7/744: unlink de/d2f/d70/f74 0 2024-05-12T04:01:17.525 INFO:tasks.workunit.client.0.smithi032.stdout:7/378: write dd/d1b/d43/d5e/d37/f46 [1803355,126049] 0 2024-05-12T04:01:17.527 INFO:tasks.workunit.client.0.smithi032.stdout:7/379: chown dd/d1b/d1e/d45/c48 7645136 1 2024-05-12T04:01:17.527 INFO:tasks.workunit.client.1.smithi033.stdout:6/582: chown d4/c25 22099 1 2024-05-12T04:01:17.528 INFO:tasks.workunit.client.1.smithi033.stdout:6/583: truncate d4/d1a/d67/d68/f8b 664642 0 2024-05-12T04:01:17.531 INFO:tasks.workunit.client.0.smithi032.stdout:5/478: creat d4/d7/d14/d26/fa6 x:0 0 0 2024-05-12T04:01:17.540 INFO:tasks.workunit.client.0.smithi032.stdout:6/380: symlink d7/de/l83 0 2024-05-12T04:01:17.541 INFO:tasks.workunit.client.1.smithi033.stdout:5/481: write d3/d27/f22 [4285020,33698] 0 2024-05-12T04:01:17.542 INFO:tasks.workunit.client.0.smithi032.stdout:8/429: getdents d0/d9/d13/d26 0 2024-05-12T04:01:17.542 INFO:tasks.workunit.client.0.smithi032.stdout:3/418: unlink d15/d24/d2a/c59 0 2024-05-12T04:01:17.543 INFO:tasks.workunit.client.1.smithi033.stdout:4/584: dread dd/d7a/d87/d7e/da7/f72 [0,4194304] 0 2024-05-12T04:01:17.544 INFO:tasks.workunit.client.1.smithi033.stdout:0/597: unlink d6/dc/f27 0 2024-05-12T04:01:17.544 INFO:tasks.workunit.client.1.smithi033.stdout:0/598: chown d6/dc/d58/f86 0 1 2024-05-12T04:01:17.546 INFO:tasks.workunit.client.1.smithi033.stdout:9/806: creat d2/d4/d2b/d49/d11d/f122 x:0 0 0 2024-05-12T04:01:17.548 INFO:tasks.workunit.client.0.smithi032.stdout:8/430: dread d0/d9/d13/d26/f55 [4194304,4194304] 0 2024-05-12T04:01:17.550 INFO:tasks.workunit.client.0.smithi032.stdout:8/431: chown d0/d9/d1b/d66/d75/l7d 215 1 2024-05-12T04:01:17.554 INFO:tasks.workunit.client.0.smithi032.stdout:7/380: dread dd/d1b/d43/d5e/d37/f39 [0,4194304] 0 2024-05-12T04:01:17.554 INFO:tasks.workunit.client.1.smithi033.stdout:3/566: truncate db/d17/f7b 2598851 0 2024-05-12T04:01:17.556 INFO:tasks.workunit.client.0.smithi032.stdout:5/479: creat d4/d7/d14/d26/d33/d72/fa7 x:0 0 0 2024-05-12T04:01:17.558 INFO:tasks.workunit.client.0.smithi032.stdout:5/480: chown d4/d7/d14/d26/d33/f61 169003 1 2024-05-12T04:01:17.559 INFO:tasks.workunit.client.0.smithi032.stdout:6/381: mknod d7/d9/d48/d67/d25/c84 0 2024-05-12T04:01:17.560 INFO:tasks.workunit.client.0.smithi032.stdout:4/321: dwrite d0/d3/df/f29 [0,4194304] 0 2024-05-12T04:01:17.561 INFO:tasks.workunit.client.1.smithi033.stdout:7/745: rename de/df/d17/d1e/dc4/cfe to de/d2b/d6c/dbd/d101/c102 0 2024-05-12T04:01:17.566 INFO:tasks.workunit.client.1.smithi033.stdout:6/584: creat d4/d6e/fc1 x:0 0 0 2024-05-12T04:01:17.566 INFO:tasks.workunit.client.1.smithi033.stdout:8/494: write d1/d16/d27/d28/f57 [841674,90914] 0 2024-05-12T04:01:17.567 INFO:tasks.workunit.client.0.smithi032.stdout:3/419: mknod d15/d20/d26/d6a/c97 0 2024-05-12T04:01:17.568 INFO:tasks.workunit.client.1.smithi033.stdout:8/495: truncate d1/d16/d27/f86 387770 0 2024-05-12T04:01:17.574 INFO:tasks.workunit.client.1.smithi033.stdout:8/496: fdatasync d1/d16/d27/d28/f57 0 2024-05-12T04:01:17.574 INFO:tasks.workunit.client.1.smithi033.stdout:2/563: write dc/f3f [878522,25581] 0 2024-05-12T04:01:17.574 INFO:tasks.workunit.client.1.smithi033.stdout:2/564: read dc/d17/d32/f5c [1014904,2105] 0 2024-05-12T04:01:17.574 INFO:tasks.workunit.client.0.smithi032.stdout:1/411: creat d1/d7/d30/d18/d87/f89 x:0 0 0 2024-05-12T04:01:17.574 INFO:tasks.workunit.client.0.smithi032.stdout:6/382: dwrite d7/f2b [4194304,4194304] 0 2024-05-12T04:01:17.575 INFO:tasks.workunit.client.0.smithi032.stdout:1/412: chown d1/d7/d30/f28 424126 1 2024-05-12T04:01:17.576 INFO:tasks.workunit.client.0.smithi032.stdout:2/406: dwrite d1/d13/f18 [0,4194304] 0 2024-05-12T04:01:17.578 INFO:tasks.workunit.client.0.smithi032.stdout:1/413: dread - d1/d7/d30/d61/f68 zero size 2024-05-12T04:01:17.579 INFO:tasks.workunit.client.0.smithi032.stdout:8/432: rename d0/d9/d13/d26 to d0/d9/d1b/d27/d88/d56/d63/d8f/d9a 0 2024-05-12T04:01:17.580 INFO:tasks.workunit.client.0.smithi032.stdout:4/322: dwrite d0/d39/f4b [0,4194304] 0 2024-05-12T04:01:17.581 INFO:tasks.workunit.client.1.smithi033.stdout:4/585: chown dd/d7a/d87/d7e/da7/d66/d42/d59/l60 470144 1 2024-05-12T04:01:17.583 INFO:tasks.workunit.client.0.smithi032.stdout:7/381: symlink dd/d1b/d43/d74/l80 0 2024-05-12T04:01:17.585 INFO:tasks.workunit.client.1.smithi033.stdout:0/599: read - d6/d11/f99 zero size 2024-05-12T04:01:17.595 INFO:tasks.workunit.client.0.smithi032.stdout:8/433: read d0/d9/d1b/d66/d75/f7c [4151255,3975] 0 2024-05-12T04:01:17.595 INFO:tasks.workunit.client.0.smithi032.stdout:3/420: rmdir d15/d24/d2a/d8c 39 2024-05-12T04:01:17.596 INFO:tasks.workunit.client.0.smithi032.stdout:8/434: dread - d0/d7/f6c zero size 2024-05-12T04:01:17.602 INFO:tasks.workunit.client.1.smithi033.stdout:1/519: creat d11/d17/fae x:0 0 0 2024-05-12T04:01:17.606 INFO:tasks.workunit.client.0.smithi032.stdout:1/414: creat d1/d7/d49/d67/f8a x:0 0 0 2024-05-12T04:01:17.608 INFO:tasks.workunit.client.0.smithi032.stdout:1/415: dread - d1/d7/d30/d35/f38 zero size 2024-05-12T04:01:17.609 INFO:tasks.workunit.client.0.smithi032.stdout:4/323: dwrite d0/d3/d31/f53 [0,4194304] 0 2024-05-12T04:01:17.612 INFO:tasks.workunit.client.0.smithi032.stdout:7/382: unlink dd/d1b/d43/d5e/f3e 0 2024-05-12T04:01:17.612 INFO:tasks.workunit.client.1.smithi033.stdout:2/565: creat dc/d17/d27/d2f/d49/fbb x:0 0 0 2024-05-12T04:01:17.613 INFO:tasks.workunit.client.0.smithi032.stdout:5/481: mknod d4/ca8 0 2024-05-12T04:01:17.615 INFO:tasks.workunit.client.0.smithi032.stdout:7/383: dread - dd/d1b/d43/d5e/f69 zero size 2024-05-12T04:01:17.618 INFO:tasks.workunit.client.0.smithi032.stdout:7/384: chown dd/d1b/d1e/d45 273 1 2024-05-12T04:01:17.619 INFO:tasks.workunit.client.0.smithi032.stdout:1/416: dwrite d1/d7/d30/d18/f70 [0,4194304] 0 2024-05-12T04:01:17.620 INFO:tasks.workunit.client.0.smithi032.stdout:4/324: dread d0/d3/df/f29 [0,4194304] 0 2024-05-12T04:01:17.621 INFO:tasks.workunit.client.1.smithi033.stdout:4/586: unlink fa 0 2024-05-12T04:01:17.621 INFO:tasks.workunit.client.1.smithi033.stdout:6/585: write d4/d6e/f90 [668478,53736] 0 2024-05-12T04:01:17.622 INFO:tasks.workunit.client.1.smithi033.stdout:7/746: dwrite f3 [0,4194304] 0 2024-05-12T04:01:17.623 INFO:tasks.workunit.client.0.smithi032.stdout:8/435: chown d0/d9/d1b/d66/d75/d84/c8b 1 1 2024-05-12T04:01:17.625 INFO:tasks.workunit.client.1.smithi033.stdout:4/587: read dd/f13 [2146200,55956] 0 2024-05-12T04:01:17.625 INFO:tasks.workunit.client.1.smithi033.stdout:7/747: write de/df/d17/d1e/dcc/ff1 [410262,94430] 0 2024-05-12T04:01:17.628 INFO:tasks.workunit.client.1.smithi033.stdout:3/567: truncate db/d17/f1b 4370114 0 2024-05-12T04:01:17.633 INFO:tasks.workunit.client.0.smithi032.stdout:6/383: creat d7/d9/d48/d67/f85 x:0 0 0 2024-05-12T04:01:17.633 INFO:tasks.workunit.client.1.smithi033.stdout:8/497: dwrite d1/f2 [4194304,4194304] 0 2024-05-12T04:01:17.636 INFO:tasks.workunit.client.1.smithi033.stdout:8/498: stat d1/d4/d1d/d36/d3c/d47/f60 0 2024-05-12T04:01:17.637 INFO:tasks.workunit.client.0.smithi032.stdout:1/417: rmdir d1/dd 39 2024-05-12T04:01:17.639 INFO:tasks.workunit.client.0.smithi032.stdout:1/418: readlink d1/d7/d49/d67/l6d 0 2024-05-12T04:01:17.640 INFO:tasks.workunit.client.0.smithi032.stdout:6/384: dwrite d7/d9/d48/d67/f4d [0,4194304] 0 2024-05-12T04:01:17.641 INFO:tasks.workunit.client.0.smithi032.stdout:1/419: chown d1/d7/d30 66373702 1 2024-05-12T04:01:17.643 INFO:tasks.workunit.client.0.smithi032.stdout:7/385: mknod dd/d1b/d1e/d45/d55/d60/d75/c81 0 2024-05-12T04:01:17.647 INFO:tasks.workunit.client.1.smithi033.stdout:6/586: rmdir d4/d6e 39 2024-05-12T04:01:17.650 INFO:tasks.workunit.client.0.smithi032.stdout:8/436: creat d0/d9/d1b/d66/d75/f9b x:0 0 0 2024-05-12T04:01:17.650 INFO:tasks.workunit.client.0.smithi032.stdout:2/407: write d1/f12 [787043,90997] 0 2024-05-12T04:01:17.651 INFO:tasks.workunit.client.0.smithi032.stdout:7/386: dread dd/d1b/f7a [0,4194304] 0 2024-05-12T04:01:17.652 INFO:tasks.workunit.client.1.smithi033.stdout:9/807: rmdir d2/d4/d2b/d49/d73/df9 0 2024-05-12T04:01:17.658 INFO:tasks.workunit.client.1.smithi033.stdout:1/520: write d11/f30 [2090350,110750] 0 2024-05-12T04:01:17.659 INFO:tasks.workunit.client.0.smithi032.stdout:2/408: dwrite d1/f12 [0,4194304] 0 2024-05-12T04:01:17.660 INFO:tasks.workunit.client.0.smithi032.stdout:3/421: creat d15/f98 x:0 0 0 2024-05-12T04:01:17.661 INFO:tasks.workunit.client.0.smithi032.stdout:2/409: fsync d1/d13/d3d/d41/f3a 0 2024-05-12T04:01:17.664 INFO:tasks.workunit.client.0.smithi032.stdout:2/410: chown d1/d13/d3d/d41/f40 53421044 1 2024-05-12T04:01:17.668 INFO:tasks.workunit.client.1.smithi033.stdout:2/566: fsync dc/d17/d27/d2f/d49/f51 0 2024-05-12T04:01:17.669 INFO:tasks.workunit.client.1.smithi033.stdout:2/567: dread - dc/d13/d1c/d98/fb5 zero size 2024-05-12T04:01:17.670 INFO:tasks.workunit.client.0.smithi032.stdout:6/385: mkdir d7/d9/d5c/d86 0 2024-05-12T04:01:17.670 INFO:tasks.workunit.client.1.smithi033.stdout:8/499: rmdir d1/d16/d20 39 2024-05-12T04:01:17.672 INFO:tasks.workunit.client.1.smithi033.stdout:5/482: getdents d3/d15/d16/d78 0 2024-05-12T04:01:17.674 INFO:tasks.workunit.client.1.smithi033.stdout:7/748: dread de/d20/d38/dec/ffb [0,4194304] 0 2024-05-12T04:01:17.674 INFO:tasks.workunit.client.1.smithi033.stdout:6/587: mknod d4/d1a/d67/d6f/cc2 0 2024-05-12T04:01:17.675 INFO:tasks.workunit.client.0.smithi032.stdout:5/482: truncate d4/d7/d14/d26/f66 41480 0 2024-05-12T04:01:17.678 INFO:tasks.workunit.client.0.smithi032.stdout:8/437: write d0/d9/d1b/d33/f45 [2678682,14513] 0 2024-05-12T04:01:17.680 INFO:tasks.workunit.client.0.smithi032.stdout:7/387: rmdir dd/d1b 39 2024-05-12T04:01:17.681 INFO:tasks.workunit.client.1.smithi033.stdout:4/588: symlink dd/d7a/d87/d7e/da7/d66/lb8 0 2024-05-12T04:01:17.684 INFO:tasks.workunit.client.1.smithi033.stdout:1/521: creat d11/d1c/d43/faf x:0 0 0 2024-05-12T04:01:17.687 INFO:tasks.workunit.client.0.smithi032.stdout:3/422: truncate d15/d20/d22/f23 883994 0 2024-05-12T04:01:17.687 INFO:tasks.workunit.client.0.smithi032.stdout:3/423: fdatasync d15/d33/f38 0 2024-05-12T04:01:17.690 INFO:tasks.workunit.client.0.smithi032.stdout:2/411: mknod d1/d13/d3d/d41/d58/c8d 0 2024-05-12T04:01:17.694 INFO:tasks.workunit.client.0.smithi032.stdout:4/325: link d0/d3/d12/d1d/l36 d0/d3/df/d4e/l58 0 2024-05-12T04:01:17.695 INFO:tasks.workunit.client.1.smithi033.stdout:8/500: creat d1/d16/d27/d4d/f9d x:0 0 0 2024-05-12T04:01:17.697 INFO:tasks.workunit.client.1.smithi033.stdout:5/483: symlink d3/d94/d54/d57/d73/l96 0 2024-05-12T04:01:17.697 INFO:tasks.workunit.client.1.smithi033.stdout:4/589: dread dd/f32 [0,4194304] 0 2024-05-12T04:01:17.698 INFO:tasks.workunit.client.1.smithi033.stdout:5/484: chown d3/d12/d29/l50 317221 1 2024-05-12T04:01:17.700 INFO:tasks.workunit.client.1.smithi033.stdout:7/749: dread de/df/d17/d1e/f23 [0,4194304] 0 2024-05-12T04:01:17.701 INFO:tasks.workunit.client.1.smithi033.stdout:0/600: sync 2024-05-12T04:01:17.702 INFO:tasks.workunit.client.0.smithi032.stdout:5/483: rmdir d4/d7/d14/d26/d33 39 2024-05-12T04:01:17.703 INFO:tasks.workunit.client.1.smithi033.stdout:6/588: chown d4/d1a/d67/d6f/cb0 852331 1 2024-05-12T04:01:17.705 INFO:tasks.workunit.client.1.smithi033.stdout:9/808: symlink d2/dd/d79/l123 0 2024-05-12T04:01:17.705 INFO:tasks.workunit.client.1.smithi033.stdout:6/589: dread - d4/f9a zero size 2024-05-12T04:01:17.705 INFO:tasks.workunit.client.0.smithi032.stdout:1/420: truncate d1/d7/d30/d35/f42 3543327 0 2024-05-12T04:01:17.708 INFO:tasks.workunit.client.1.smithi033.stdout:1/522: fsync f8 0 2024-05-12T04:01:17.709 INFO:tasks.workunit.client.1.smithi033.stdout:0/601: dwrite d6/dc/d96/fb1 [0,4194304] 0 2024-05-12T04:01:17.711 INFO:tasks.workunit.client.0.smithi032.stdout:4/326: write d0/f11 [3278068,6790] 0 2024-05-12T04:01:17.713 INFO:tasks.workunit.client.0.smithi032.stdout:5/484: mkdir d4/d7/d14/da9 0 2024-05-12T04:01:17.715 INFO:tasks.workunit.client.1.smithi033.stdout:4/590: dread dd/f13 [0,4194304] 0 2024-05-12T04:01:17.717 INFO:tasks.workunit.client.1.smithi033.stdout:2/568: unlink dc/d13/d1f/d6e/l8a 0 2024-05-12T04:01:17.717 INFO:tasks.workunit.client.1.smithi033.stdout:9/809: dwrite d2/d4/d2b/d78/fc8 [0,4194304] 0 2024-05-12T04:01:17.718 INFO:tasks.workunit.client.0.smithi032.stdout:3/424: symlink d15/l99 0 2024-05-12T04:01:17.721 INFO:tasks.workunit.client.1.smithi033.stdout:8/501: mknod d1/d4/d1d/d36/d3c/d47/c9e 0 2024-05-12T04:01:17.722 INFO:tasks.workunit.client.1.smithi033.stdout:8/502: readlink d1/d4/d46/l6e 0 2024-05-12T04:01:17.723 INFO:tasks.workunit.client.0.smithi032.stdout:4/327: dwrite d0/d3/d12/d1d/f4c [0,4194304] 0 2024-05-12T04:01:17.725 INFO:tasks.workunit.client.1.smithi033.stdout:5/485: unlink d3/d12/f43 0 2024-05-12T04:01:17.726 INFO:tasks.workunit.client.0.smithi032.stdout:4/328: read d0/d3/d12/d1d/f2a [1776427,58860] 0 2024-05-12T04:01:17.727 INFO:tasks.workunit.client.0.smithi032.stdout:4/329: write d0/d3/d31/f35 [4447342,101038] 0 2024-05-12T04:01:17.729 INFO:tasks.workunit.client.0.smithi032.stdout:1/421: creat d1/d7/d30/d18/f8b x:0 0 0 2024-05-12T04:01:17.731 INFO:tasks.workunit.client.0.smithi032.stdout:1/422: write d1/d6a/f7a [2856799,7456] 0 2024-05-12T04:01:17.731 INFO:tasks.workunit.client.1.smithi033.stdout:7/750: mknod de/d2b/d6c/db5/c103 0 2024-05-12T04:01:17.732 INFO:tasks.workunit.client.1.smithi033.stdout:7/751: dread - de/df/d17/d1e/fdd zero size 2024-05-12T04:01:17.733 INFO:tasks.workunit.client.1.smithi033.stdout:7/752: chown de/d2b/d6c/db5/dd0/lee 13972 1 2024-05-12T04:01:17.733 INFO:tasks.workunit.client.1.smithi033.stdout:6/590: mkdir d4/d1a/d67/dc3 0 2024-05-12T04:01:17.736 INFO:tasks.workunit.client.0.smithi032.stdout:3/425: mkdir d15/d20/d26/d6a/d6e/d7d/d9a 0 2024-05-12T04:01:17.737 INFO:tasks.workunit.client.0.smithi032.stdout:2/412: rename d1/d13/f19 to d1/d13/d3d/f8e 0 2024-05-12T04:01:17.741 INFO:tasks.workunit.client.0.smithi032.stdout:4/330: symlink d0/d39/l59 0 2024-05-12T04:01:17.741 INFO:tasks.workunit.client.1.smithi033.stdout:0/602: mknod d6/dc/d6e/ccb 0 2024-05-12T04:01:17.742 INFO:tasks.workunit.client.0.smithi032.stdout:6/386: getdents d7/d9/d5c/d62 0 2024-05-12T04:01:17.743 INFO:tasks.workunit.client.1.smithi033.stdout:1/523: creat d11/d17/d27/d76/d93/fb0 x:0 0 0 2024-05-12T04:01:17.746 INFO:tasks.workunit.client.0.smithi032.stdout:5/485: creat d4/d7/d14/da9/faa x:0 0 0 2024-05-12T04:01:17.746 INFO:tasks.workunit.client.1.smithi033.stdout:3/568: write db/d17/f2c [873980,87087] 0 2024-05-12T04:01:17.751 INFO:tasks.workunit.client.1.smithi033.stdout:2/569: dread dc/d17/d3a/f6d [4194304,4194304] 0 2024-05-12T04:01:17.755 INFO:tasks.workunit.client.0.smithi032.stdout:2/413: creat d1/d13/f8f x:0 0 0 2024-05-12T04:01:17.759 INFO:tasks.workunit.client.0.smithi032.stdout:4/331: rmdir d0/d3/d31 39 2024-05-12T04:01:17.759 INFO:tasks.workunit.client.1.smithi033.stdout:5/486: dread d3/d12/f63 [0,4194304] 0 2024-05-12T04:01:17.759 INFO:tasks.workunit.client.0.smithi032.stdout:4/332: fsync d0/d39/f4b 0 2024-05-12T04:01:17.762 INFO:tasks.workunit.client.0.smithi032.stdout:6/387: creat d7/d33/f87 x:0 0 0 2024-05-12T04:01:17.766 INFO:tasks.workunit.client.0.smithi032.stdout:1/423: link d1/d7/f7f d1/d7/d30/d35/d54/f8c 0 2024-05-12T04:01:17.767 INFO:tasks.workunit.client.1.smithi033.stdout:0/603: truncate d6/f26 5697315 0 2024-05-12T04:01:17.767 INFO:tasks.workunit.client.0.smithi032.stdout:1/424: read - d1/d7/d30/d35/f38 zero size 2024-05-12T04:01:17.768 INFO:tasks.workunit.client.0.smithi032.stdout:3/426: mkdir d15/d20/d8b/d9b 0 2024-05-12T04:01:17.769 INFO:tasks.workunit.client.1.smithi033.stdout:1/524: symlink d11/d17/d84/lb1 0 2024-05-12T04:01:17.770 INFO:tasks.workunit.client.0.smithi032.stdout:2/414: creat d1/d13/d3d/d41/d2c/f90 x:0 0 0 2024-05-12T04:01:17.771 INFO:tasks.workunit.client.1.smithi033.stdout:6/591: write d4/d1a/f91 [1439333,48961] 0 2024-05-12T04:01:17.771 INFO:tasks.workunit.client.1.smithi033.stdout:8/503: dwrite d1/f6b [0,4194304] 0 2024-05-12T04:01:17.771 INFO:tasks.workunit.client.1.smithi033.stdout:1/525: chown d11/d1c/c8c 0 1 2024-05-12T04:01:17.774 INFO:tasks.workunit.client.1.smithi033.stdout:3/569: creat db/d17/fc9 x:0 0 0 2024-05-12T04:01:17.774 INFO:tasks.workunit.client.0.smithi032.stdout:6/388: mknod d7/d9/d48/d67/d1a/d20/c88 0 2024-05-12T04:01:17.776 INFO:tasks.workunit.client.0.smithi032.stdout:6/389: write d7/d9/d48/d67/d25/f29 [4106954,17858] 0 2024-05-12T04:01:17.776 INFO:tasks.workunit.client.0.smithi032.stdout:6/390: chown d7/d9/f82 931 1 2024-05-12T04:01:17.779 INFO:tasks.workunit.client.0.smithi032.stdout:5/486: rename d4/d7/l2d to d4/d7/d14/d26/d47/lab 0 2024-05-12T04:01:17.780 INFO:tasks.workunit.client.0.smithi032.stdout:5/487: fdatasync d4/d7/fa5 0 2024-05-12T04:01:17.786 INFO:tasks.workunit.client.1.smithi033.stdout:9/810: link d2/d4/f2c d2/d4/d29/d3c/dda/f124 0 2024-05-12T04:01:17.787 INFO:tasks.workunit.client.1.smithi033.stdout:9/811: chown d2/d4/d2b/d49/ff0 2 1 2024-05-12T04:01:17.789 INFO:tasks.workunit.client.0.smithi032.stdout:2/415: write d1/f31 [1288983,60372] 0 2024-05-12T04:01:17.798 INFO:tasks.workunit.client.0.smithi032.stdout:6/391: symlink d7/d9/d48/d67/d1a/l89 0 2024-05-12T04:01:17.799 INFO:tasks.workunit.client.1.smithi033.stdout:5/487: write d3/d94/f61 [123074,83128] 0 2024-05-12T04:01:17.800 INFO:tasks.workunit.client.1.smithi033.stdout:5/488: truncate d3/d15/f88 921308 0 2024-05-12T04:01:17.803 INFO:tasks.workunit.client.0.smithi032.stdout:3/427: creat d15/d20/d26/d6a/d6e/d7d/d9a/f9c x:0 0 0 2024-05-12T04:01:17.804 INFO:tasks.workunit.client.1.smithi033.stdout:4/591: creat dd/d11/fb9 x:0 0 0 2024-05-12T04:01:17.806 INFO:tasks.workunit.client.0.smithi032.stdout:2/416: readlink d1/d4d/l66 0 2024-05-12T04:01:17.807 INFO:tasks.workunit.client.0.smithi032.stdout:4/333: truncate d0/d39/f4b 3329284 0 2024-05-12T04:01:17.809 INFO:tasks.workunit.client.1.smithi033.stdout:8/504: unlink d1/d16/d27/d4d/d82/c88 0 2024-05-12T04:01:17.810 INFO:tasks.workunit.client.0.smithi032.stdout:5/488: mknod d4/d7/d14/d26/d33/d5f/d7f/cac 0 2024-05-12T04:01:17.813 INFO:tasks.workunit.client.1.smithi033.stdout:5/489: dread d3/d94/f3e [0,4194304] 0 2024-05-12T04:01:17.813 INFO:tasks.workunit.client.0.smithi032.stdout:2/417: creat d1/d13/d70/f91 x:0 0 0 2024-05-12T04:01:17.814 INFO:tasks.workunit.client.1.smithi033.stdout:8/505: dwrite d1/d4/d1d/d36/f50 [0,4194304] 0 2024-05-12T04:01:17.815 INFO:tasks.workunit.client.0.smithi032.stdout:4/334: mknod d0/d3/d12/d1d/c5a 0 2024-05-12T04:01:17.815 INFO:tasks.workunit.client.1.smithi033.stdout:5/490: dread - d3/d94/d54/d57/d73/d74/f93 zero size 2024-05-12T04:01:17.816 INFO:tasks.workunit.client.0.smithi032.stdout:6/392: mkdir d7/d9/d5c/d86/d8a 0 2024-05-12T04:01:17.816 INFO:tasks.workunit.client.1.smithi033.stdout:5/491: dread - d3/d15/d16/d45/d62/f7a zero size 2024-05-12T04:01:17.817 INFO:tasks.workunit.client.0.smithi032.stdout:6/393: chown d7/de/d2e/c76 180880 1 2024-05-12T04:01:17.817 INFO:tasks.workunit.client.0.smithi032.stdout:1/425: getdents d1/d7/d30/d35/d77 0 2024-05-12T04:01:17.819 INFO:tasks.workunit.client.1.smithi033.stdout:4/592: chown dd/ca3 2881376 1 2024-05-12T04:01:17.821 INFO:tasks.workunit.client.0.smithi032.stdout:3/428: fsync d15/d1b/f61 0 2024-05-12T04:01:17.822 INFO:tasks.workunit.client.1.smithi033.stdout:6/592: mkdir d4/d43/d57/dc4 0 2024-05-12T04:01:17.825 INFO:tasks.workunit.client.1.smithi033.stdout:1/526: creat d11/d17/d27/d36/d46/d4c/dab/fb2 x:0 0 0 2024-05-12T04:01:17.831 INFO:tasks.workunit.client.1.smithi033.stdout:1/527: dwrite d11/d17/d27/d76/f9c [0,4194304] 0 2024-05-12T04:01:17.834 INFO:tasks.workunit.client.0.smithi032.stdout:5/489: mknod d4/d7/d14/d26/d33/d4a/cad 0 2024-05-12T04:01:17.836 INFO:tasks.workunit.client.0.smithi032.stdout:3/429: dread d15/d33/f46 [0,4194304] 0 2024-05-12T04:01:17.838 INFO:tasks.workunit.client.1.smithi033.stdout:1/528: dwrite d11/d17/fae [0,4194304] 0 2024-05-12T04:01:17.842 INFO:tasks.workunit.client.0.smithi032.stdout:2/418: truncate d1/f1e 972383 0 2024-05-12T04:01:17.846 INFO:tasks.workunit.client.1.smithi033.stdout:7/753: getdents de/df/d7a 0 2024-05-12T04:01:17.846 INFO:tasks.workunit.client.0.smithi032.stdout:2/419: write d1/d13/d3d/d41/f5c [4535,85557] 0 2024-05-12T04:01:17.847 INFO:tasks.workunit.client.0.smithi032.stdout:3/430: dwrite d15/d24/d93/f8f [0,4194304] 0 2024-05-12T04:01:17.848 INFO:tasks.workunit.client.1.smithi033.stdout:0/604: rename d6/fac to d6/d11/d57/fcc 0 2024-05-12T04:01:17.849 INFO:tasks.workunit.client.0.smithi032.stdout:2/420: fdatasync d1/d34/d3e/d67/f69 0 2024-05-12T04:01:17.850 INFO:tasks.workunit.client.1.smithi033.stdout:9/812: write d2/dd/f11 [1109195,52172] 0 2024-05-12T04:01:17.852 INFO:tasks.workunit.client.1.smithi033.stdout:9/813: readlink d2/d4/d2b/d6d/d2f/d5f/d84/d107/l118 0 2024-05-12T04:01:17.856 INFO:tasks.workunit.client.1.smithi033.stdout:8/506: write d1/d16/d27/d4d/f5e [394299,83481] 0 2024-05-12T04:01:17.856 INFO:tasks.workunit.client.0.smithi032.stdout:2/421: dread d1/f31 [0,4194304] 0 2024-05-12T04:01:17.859 INFO:tasks.workunit.client.0.smithi032.stdout:6/394: dwrite d7/de/f1b [4194304,4194304] 0 2024-05-12T04:01:17.867 INFO:tasks.workunit.client.1.smithi033.stdout:2/570: link dc/l2c dc/d17/d27/d2f/lbc 0 2024-05-12T04:01:17.867 INFO:tasks.workunit.client.0.smithi032.stdout:4/335: creat d0/d3/d12/f5b x:0 0 0 2024-05-12T04:01:17.868 INFO:tasks.workunit.client.0.smithi032.stdout:3/431: mknod d15/d24/d40/c9d 0 2024-05-12T04:01:17.869 INFO:tasks.workunit.client.0.smithi032.stdout:4/336: chown d0/d3/d12/d1d 89832 1 2024-05-12T04:01:17.870 INFO:tasks.workunit.client.0.smithi032.stdout:5/490: mknod d4/d7/d14/d26/cae 0 2024-05-12T04:01:17.870 INFO:tasks.workunit.client.1.smithi033.stdout:7/754: chown cb 94 1 2024-05-12T04:01:17.872 INFO:tasks.workunit.client.0.smithi032.stdout:1/426: dwrite d1/d7/f78 [0,4194304] 0 2024-05-12T04:01:17.878 INFO:tasks.workunit.client.0.smithi032.stdout:3/432: dread d15/d24/f3e [0,4194304] 0 2024-05-12T04:01:17.878 INFO:tasks.workunit.client.1.smithi033.stdout:9/814: mknod d2/d70/c125 0 2024-05-12T04:01:17.881 INFO:tasks.workunit.client.0.smithi032.stdout:2/422: link d1/d13/d3d/f8e d1/d4d/d5e/f92 0 2024-05-12T04:01:17.886 INFO:tasks.workunit.client.0.smithi032.stdout:3/433: dwrite d15/d24/d93/f78 [0,4194304] 0 2024-05-12T04:01:17.889 INFO:tasks.workunit.client.1.smithi033.stdout:6/593: truncate d4/d43/f70 4115974 0 2024-05-12T04:01:17.890 INFO:tasks.workunit.client.0.smithi032.stdout:3/434: dread d15/f4d [0,4194304] 0 2024-05-12T04:01:17.891 INFO:tasks.workunit.client.1.smithi033.stdout:3/570: getdents db/d17/d3a/d67/da6 0 2024-05-12T04:01:17.892 INFO:tasks.workunit.client.0.smithi032.stdout:5/491: dread d4/d7/d14/d26/d33/f61 [0,4194304] 0 2024-05-12T04:01:17.893 INFO:tasks.workunit.client.0.smithi032.stdout:2/423: write d1/d34/f8c [1101663,84963] 0 2024-05-12T04:01:17.895 INFO:tasks.workunit.client.1.smithi033.stdout:1/529: unlink d11/d17/d27/d36/f41 0 2024-05-12T04:01:17.897 INFO:tasks.workunit.client.0.smithi032.stdout:4/337: rename d0/d3/d12/f51 to d0/d3/df/d4e/f5c 0 2024-05-12T04:01:17.897 INFO:tasks.workunit.client.1.smithi033.stdout:6/594: dwrite d4/d1a/d6c/f9c [0,4194304] 0 2024-05-12T04:01:17.898 INFO:tasks.workunit.client.0.smithi032.stdout:4/338: readlink d0/d3/l32 0 2024-05-12T04:01:17.898 INFO:tasks.workunit.client.0.smithi032.stdout:4/339: chown d0/l1a 0 1 2024-05-12T04:01:17.898 INFO:tasks.workunit.client.1.smithi033.stdout:7/755: mknod de/d80/c104 0 2024-05-12T04:01:17.899 INFO:tasks.workunit.client.1.smithi033.stdout:7/756: stat de/df/d17/d1b/d36/c3e 0 2024-05-12T04:01:17.900 INFO:tasks.workunit.client.1.smithi033.stdout:7/757: fsync de/df/d17/d1b/d36/f4a 0 2024-05-12T04:01:17.902 INFO:tasks.workunit.client.1.smithi033.stdout:7/758: chown de/d2b/d6c/de1 1658197 1 2024-05-12T04:01:17.915 INFO:tasks.workunit.client.0.smithi032.stdout:5/492: dread d4/d7/d14/f1d [0,4194304] 0 2024-05-12T04:01:17.939 INFO:tasks.workunit.client.0.smithi032.stdout:9/293: sync 2024-05-12T04:01:17.977 INFO:tasks.workunit.client.1.smithi033.stdout:2/571: sync 2024-05-12T04:01:17.978 INFO:tasks.workunit.client.1.smithi033.stdout:5/492: truncate d3/d15/d16/f42 3307068 0 2024-05-12T04:01:17.980 INFO:tasks.workunit.client.0.smithi032.stdout:1/427: creat d1/d7/d49/d79/f8d x:0 0 0 2024-05-12T04:01:17.981 INFO:tasks.workunit.client.0.smithi032.stdout:1/428: dread - d1/d7/d49/f6b zero size 2024-05-12T04:01:17.999 INFO:tasks.workunit.client.1.smithi033.stdout:8/507: mkdir d1/d4/d1d/d36/d3c/d9f 0 2024-05-12T04:01:18.000 INFO:tasks.workunit.client.1.smithi033.stdout:8/508: write d1/d16/f75 [2774297,119924] 0 2024-05-12T04:01:18.010 INFO:tasks.workunit.client.1.smithi033.stdout:9/815: write d2/d4/d2b/fd7 [168433,68700] 0 2024-05-12T04:01:18.011 INFO:tasks.workunit.client.0.smithi032.stdout:4/340: unlink d0/d3/df/c16 0 2024-05-12T04:01:18.012 INFO:tasks.workunit.client.1.smithi033.stdout:6/595: fsync d4/d5/fa1 0 2024-05-12T04:01:18.016 INFO:tasks.workunit.client.0.smithi032.stdout:9/294: symlink d0/d3/d6/d7/d2a/l5d 0 2024-05-12T04:01:18.022 INFO:tasks.workunit.client.1.smithi033.stdout:4/593: rename dd/d7a/d87/d7e/da7/d66/d42/f4e to dd/d7a/d87/d7e/da7/d66/fba 0 2024-05-12T04:01:18.023 INFO:tasks.workunit.client.0.smithi032.stdout:9/295: dwrite d0/d3/d6/d7/da/f1a [0,4194304] 0 2024-05-12T04:01:18.023 INFO:tasks.workunit.client.1.smithi033.stdout:4/594: chown dd/d11/d5b/l71 1 1 2024-05-12T04:01:18.023 INFO:tasks.workunit.client.0.smithi032.stdout:1/429: mknod d1/d6a/c8e 0 2024-05-12T04:01:18.026 INFO:tasks.workunit.client.0.smithi032.stdout:2/424: unlink d1/d4d/d5e/f92 0 2024-05-12T04:01:18.029 INFO:tasks.workunit.client.0.smithi032.stdout:4/341: truncate d0/d3/f49 772464 0 2024-05-12T04:01:18.031 INFO:tasks.workunit.client.0.smithi032.stdout:1/430: dwrite d1/d7/d30/d35/d54/f73 [0,4194304] 0 2024-05-12T04:01:18.032 INFO:tasks.workunit.client.0.smithi032.stdout:1/431: readlink d1/d7/l20 0 2024-05-12T04:01:18.033 INFO:tasks.workunit.client.0.smithi032.stdout:5/493: creat d4/d7/d14/d26/d33/d88/faf x:0 0 0 2024-05-12T04:01:18.033 INFO:tasks.workunit.client.0.smithi032.stdout:1/432: fdatasync d1/d7/d49/f6b 0 2024-05-12T04:01:18.035 INFO:tasks.workunit.client.0.smithi032.stdout:5/494: write d4/d7/d14/d39/d2b/d6e/f85 [701145,96181] 0 2024-05-12T04:01:18.035 INFO:tasks.workunit.client.0.smithi032.stdout:1/433: chown d1/d6a/f84 161764 1 2024-05-12T04:01:18.036 INFO:tasks.workunit.client.1.smithi033.stdout:3/571: mknod db/d42/cca 0 2024-05-12T04:01:18.037 INFO:tasks.workunit.client.0.smithi032.stdout:9/296: creat d0/d3/d6/d58/f5e x:0 0 0 2024-05-12T04:01:18.041 INFO:tasks.workunit.client.0.smithi032.stdout:2/425: chown d1/f21 26809 1 2024-05-12T04:01:18.043 INFO:tasks.workunit.client.0.smithi032.stdout:2/426: write d1/d13/d3d/d41/d39/f87 [683654,100480] 0 2024-05-12T04:01:18.043 INFO:tasks.workunit.client.1.smithi033.stdout:1/530: mknod d11/d17/d27/d36/cb3 0 2024-05-12T04:01:18.047 INFO:tasks.workunit.client.1.smithi033.stdout:9/816: rmdir d2/d4/d2b/dc7 39 2024-05-12T04:01:18.049 INFO:tasks.workunit.client.0.smithi032.stdout:3/435: rename d15/d20/d22/f58 to d15/d33/f9e 0 2024-05-12T04:01:18.050 INFO:tasks.workunit.client.0.smithi032.stdout:3/436: dread - d15/d24/d2a/d5b/f71 zero size 2024-05-12T04:01:18.051 INFO:tasks.workunit.client.0.smithi032.stdout:4/342: creat d0/d3/d12/d1d/f5d x:0 0 0 2024-05-12T04:01:18.052 INFO:tasks.workunit.client.1.smithi033.stdout:7/759: mknod de/d20/d64/d100/c105 0 2024-05-12T04:01:18.052 INFO:tasks.workunit.client.0.smithi032.stdout:4/343: write d0/d3/d12/d1d/f5d [397832,65877] 0 2024-05-12T04:01:18.056 INFO:tasks.workunit.client.0.smithi032.stdout:4/344: fdatasync d0/d3/d12/d1d/f4c 0 2024-05-12T04:01:18.056 INFO:tasks.workunit.client.1.smithi033.stdout:7/760: write de/df/d17/d1e/dcc/ff1 [136746,129551] 0 2024-05-12T04:01:18.056 INFO:tasks.workunit.client.1.smithi033.stdout:9/817: dwrite d2/d4/d2b/d6d/d4c/ffe [0,4194304] 0 2024-05-12T04:01:18.056 INFO:tasks.workunit.client.1.smithi033.stdout:8/509: dread d1/d16/d27/d4d/d51/f4b [0,4194304] 0 2024-05-12T04:01:18.056 INFO:tasks.workunit.client.1.smithi033.stdout:5/493: truncate d3/d12/d29/d2b/f46 1141339 0 2024-05-12T04:01:18.058 INFO:tasks.workunit.client.0.smithi032.stdout:9/297: symlink d0/d3/d4b/l5f 0 2024-05-12T04:01:18.059 INFO:tasks.workunit.client.0.smithi032.stdout:3/437: dwrite d15/d20/d26/f3c [0,4194304] 0 2024-05-12T04:01:18.061 INFO:tasks.workunit.client.1.smithi033.stdout:0/605: getdents d6/dc 0 2024-05-12T04:01:18.064 INFO:tasks.workunit.client.0.smithi032.stdout:2/427: rename d1/f12 to d1/d13/f93 0 2024-05-12T04:01:18.070 INFO:tasks.workunit.client.0.smithi032.stdout:5/495: mknod d4/d7/d14/d39/cb0 0 2024-05-12T04:01:18.070 INFO:tasks.workunit.client.1.smithi033.stdout:0/606: write d6/dc/d58/d6a/d94/fc1 [3048,38076] 0 2024-05-12T04:01:18.076 INFO:tasks.workunit.client.0.smithi032.stdout:4/345: readlink d0/d3/df/l3f 0 2024-05-12T04:01:18.081 INFO:tasks.workunit.client.0.smithi032.stdout:3/438: creat d15/d20/d22/d7a/f9f x:0 0 0 2024-05-12T04:01:18.082 INFO:tasks.workunit.client.0.smithi032.stdout:3/439: fsync d15/d24/d2a/d5b/f8e 0 2024-05-12T04:01:18.084 INFO:tasks.workunit.client.1.smithi033.stdout:9/818: creat d2/d4/d2b/d33/dd2/dde/f126 x:0 0 0 2024-05-12T04:01:18.084 INFO:tasks.workunit.client.1.smithi033.stdout:3/572: write db/d17/f36 [2035909,77491] 0 2024-05-12T04:01:18.086 INFO:tasks.workunit.client.0.smithi032.stdout:9/298: rename d0/d3/d6/d7/d2a/l5d to d0/d3/d4b/l60 0 2024-05-12T04:01:18.086 INFO:tasks.workunit.client.1.smithi033.stdout:3/573: write db/d17/d3a/f63 [3610087,4614] 0 2024-05-12T04:01:18.087 INFO:tasks.workunit.client.1.smithi033.stdout:5/494: symlink d3/d94/d85/l97 0 2024-05-12T04:01:18.088 INFO:tasks.workunit.client.0.smithi032.stdout:2/428: symlink d1/d13/d3d/d41/d52/l94 0 2024-05-12T04:01:18.090 INFO:tasks.workunit.client.1.smithi033.stdout:7/761: write de/df/d17/f88 [5019480,84042] 0 2024-05-12T04:01:18.092 INFO:tasks.workunit.client.1.smithi033.stdout:4/595: truncate dd/d7a/d87/d7e/da7/d66/d42/f63 2874649 0 2024-05-12T04:01:18.093 INFO:tasks.workunit.client.0.smithi032.stdout:5/496: read d4/d7/d14/d26/d33/d4a/d38/d3b/f48 [29080,105358] 0 2024-05-12T04:01:18.094 INFO:tasks.workunit.client.1.smithi033.stdout:4/596: chown dd/d11/f2f 3015 1 2024-05-12T04:01:18.094 INFO:tasks.workunit.client.0.smithi032.stdout:5/497: write d4/d7/d14/d39/d1b/f23 [1909562,447] 0 2024-05-12T04:01:18.095 INFO:tasks.workunit.client.0.smithi032.stdout:5/498: readlink d4/l7d 0 2024-05-12T04:01:18.095 INFO:tasks.workunit.client.1.smithi033.stdout:5/495: dwrite d3/d12/d29/f3d [0,4194304] 0 2024-05-12T04:01:18.096 INFO:tasks.workunit.client.0.smithi032.stdout:5/499: dread - d4/d7/d14/d26/d33/d5f/f87 zero size 2024-05-12T04:01:18.096 INFO:tasks.workunit.client.1.smithi033.stdout:5/496: stat d3/d94 0 2024-05-12T04:01:18.097 INFO:tasks.workunit.client.0.smithi032.stdout:4/346: readlink d0/d3/d12/l50 0 2024-05-12T04:01:18.098 INFO:tasks.workunit.client.1.smithi033.stdout:5/497: dread - d3/d12/f80 zero size 2024-05-12T04:01:18.099 INFO:tasks.workunit.client.1.smithi033.stdout:0/607: read - d6/d11/d57/f90 zero size 2024-05-12T04:01:18.101 INFO:tasks.workunit.client.0.smithi032.stdout:3/440: fdatasync d15/f54 0 2024-05-12T04:01:18.102 INFO:tasks.workunit.client.1.smithi033.stdout:4/597: dread dd/d7a/d87/d7e/da7/d66/d42/f82 [0,4194304] 0 2024-05-12T04:01:18.106 INFO:tasks.workunit.client.0.smithi032.stdout:8/438: sync 2024-05-12T04:01:18.106 INFO:tasks.workunit.client.0.smithi032.stdout:0/469: sync 2024-05-12T04:01:18.107 INFO:tasks.workunit.client.0.smithi032.stdout:5/500: dwrite d4/d7/d14/d26/d33/d5f/d7f/f84 [0,4194304] 0 2024-05-12T04:01:18.107 INFO:tasks.workunit.client.0.smithi032.stdout:9/299: rename d0/d3/d6/d7/da/c2b to d0/d3/d6/d7/d27/c61 0 2024-05-12T04:01:18.108 INFO:tasks.workunit.client.0.smithi032.stdout:9/300: readlink d0/d3/d6/d7/d27/l2e 0 2024-05-12T04:01:18.113 INFO:tasks.workunit.client.0.smithi032.stdout:1/434: getdents d1/d7/d49/d79 0 2024-05-12T04:01:18.115 INFO:tasks.workunit.client.0.smithi032.stdout:8/439: dwrite d0/d9/f12 [0,4194304] 0 2024-05-12T04:01:18.116 INFO:tasks.workunit.client.1.smithi033.stdout:9/819: symlink d2/dd/d39/l127 0 2024-05-12T04:01:18.117 INFO:tasks.workunit.client.0.smithi032.stdout:1/435: write d1/d7/d30/f2c [1123013,120950] 0 2024-05-12T04:01:18.119 INFO:tasks.workunit.client.0.smithi032.stdout:1/436: write d1/d6a/d72/f88 [892125,86729] 0 2024-05-12T04:01:18.121 INFO:tasks.workunit.client.0.smithi032.stdout:1/437: fsync d1/f47 0 2024-05-12T04:01:18.123 INFO:tasks.workunit.client.1.smithi033.stdout:8/510: dwrite d1/d16/d20/f43 [0,4194304] 0 2024-05-12T04:01:18.123 INFO:tasks.workunit.client.1.smithi033.stdout:8/511: chown l0 329395721 1 2024-05-12T04:01:18.125 INFO:tasks.workunit.client.1.smithi033.stdout:6/596: link d4/d1a/d4e/d4f/d94/la2 d4/d43/lc5 0 2024-05-12T04:01:18.126 INFO:tasks.workunit.client.1.smithi033.stdout:6/597: chown d4/d1a/d4e/c9f 221688 1 2024-05-12T04:01:18.126 INFO:tasks.workunit.client.0.smithi032.stdout:0/470: symlink df/d10/d3d/d4a/la8 0 2024-05-12T04:01:18.128 INFO:tasks.workunit.client.1.smithi033.stdout:5/498: rmdir d3/d15/d16/d45 39 2024-05-12T04:01:18.131 INFO:tasks.workunit.client.1.smithi033.stdout:0/608: rmdir d6/d11/dc7 39 2024-05-12T04:01:18.132 INFO:tasks.workunit.client.1.smithi033.stdout:0/609: dread - d6/dc/d6e/f51 zero size 2024-05-12T04:01:18.133 INFO:tasks.workunit.client.0.smithi032.stdout:2/429: dwrite d1/d13/d3d/d41/f28 [0,4194304] 0 2024-05-12T04:01:18.134 INFO:tasks.workunit.client.1.smithi033.stdout:9/820: rmdir d2/d4/d2b/d6d 39 2024-05-12T04:01:18.135 INFO:tasks.workunit.client.1.smithi033.stdout:9/821: chown d2/fa 1216633337 1 2024-05-12T04:01:18.137 INFO:tasks.workunit.client.0.smithi032.stdout:9/301: dread d0/d3/d6/d7/f15 [0,4194304] 0 2024-05-12T04:01:18.138 INFO:tasks.workunit.client.0.smithi032.stdout:8/440: creat d0/d9/d1b/d27/d4c/d4e/d7f/f9c x:0 0 0 2024-05-12T04:01:18.149 INFO:tasks.workunit.client.1.smithi033.stdout:7/762: dwrite de/df/d7a/f8b [4194304,4194304] 0 2024-05-12T04:01:18.153 INFO:tasks.workunit.client.0.smithi032.stdout:0/471: unlink df/d10/d3b/l6f 0 2024-05-12T04:01:18.154 INFO:tasks.workunit.client.1.smithi033.stdout:4/598: dread dd/f2b [0,4194304] 0 2024-05-12T04:01:18.157 INFO:tasks.workunit.client.1.smithi033.stdout:8/512: dwrite d1/d16/d20/f43 [4194304,4194304] 0 2024-05-12T04:01:18.157 INFO:tasks.workunit.client.1.smithi033.stdout:5/499: mkdir d3/d94/d54/d98 0 2024-05-12T04:01:18.159 INFO:tasks.workunit.client.0.smithi032.stdout:1/438: readlink d1/dd/l2e 0 2024-05-12T04:01:18.159 INFO:tasks.workunit.client.1.smithi033.stdout:8/513: readlink d1/d4/d1d/d36/d3c/l93 0 2024-05-12T04:01:18.161 INFO:tasks.workunit.client.0.smithi032.stdout:1/439: truncate d1/d25/f4d 4710774 0 2024-05-12T04:01:18.161 INFO:tasks.workunit.client.1.smithi033.stdout:7/763: rmdir de/d20/d64/d100 39 2024-05-12T04:01:18.162 INFO:tasks.workunit.client.1.smithi033.stdout:7/764: stat de/d2b/d6c/da0/da6 0 2024-05-12T04:01:18.163 INFO:tasks.workunit.client.1.smithi033.stdout:7/765: readlink de/df/lc6 0 2024-05-12T04:01:18.164 INFO:tasks.workunit.client.0.smithi032.stdout:4/347: creat d0/d3/f5e x:0 0 0 2024-05-12T04:01:18.164 INFO:tasks.workunit.client.0.smithi032.stdout:2/430: dread d1/d34/d3e/f7e [0,4194304] 0 2024-05-12T04:01:18.168 INFO:tasks.workunit.client.1.smithi033.stdout:4/599: rename dd/f62 to dd/d69/da4/db3/fbb 0 2024-05-12T04:01:18.170 INFO:tasks.workunit.client.1.smithi033.stdout:2/572: sync 2024-05-12T04:01:18.171 INFO:tasks.workunit.client.1.smithi033.stdout:2/573: dread - dc/d13/d39/d8f/d9e/fb4 zero size 2024-05-12T04:01:18.172 INFO:tasks.workunit.client.1.smithi033.stdout:2/574: chown dc/d17/d27/c55 238012386 1 2024-05-12T04:01:18.173 INFO:tasks.workunit.client.1.smithi033.stdout:2/575: truncate dc/d17/d32/fb7 750325 0 2024-05-12T04:01:18.174 INFO:tasks.workunit.client.1.smithi033.stdout:5/500: mkdir d3/d94/d85/d99 0 2024-05-12T04:01:18.175 INFO:tasks.workunit.client.1.smithi033.stdout:5/501: chown d3/d94 3200071 1 2024-05-12T04:01:18.175 INFO:tasks.workunit.client.1.smithi033.stdout:2/576: rename dc to dc/d17/d27/d68/dbd 22 2024-05-12T04:01:18.175 INFO:tasks.workunit.client.1.smithi033.stdout:3/574: getdents db/d17/d3e/d65/d91/d97 0 2024-05-12T04:01:18.177 INFO:tasks.workunit.client.1.smithi033.stdout:3/575: read db/d17/d25/d32/f5d [695554,82298] 0 2024-05-12T04:01:18.178 INFO:tasks.workunit.client.1.smithi033.stdout:8/514: creat d1/d4/d1d/d36/d3c/d64/fa0 x:0 0 0 2024-05-12T04:01:18.179 INFO:tasks.workunit.client.1.smithi033.stdout:8/515: write d1/d4/d1d/d36/f50 [3916542,96107] 0 2024-05-12T04:01:18.180 INFO:tasks.workunit.client.0.smithi032.stdout:9/302: truncate d0/d3/d6/d7/da/d39/f4d 3496910 0 2024-05-12T04:01:18.180 INFO:tasks.workunit.client.1.smithi033.stdout:7/766: dread de/d2b/d6c/f3d [0,4194304] 0 2024-05-12T04:01:18.181 INFO:tasks.workunit.client.0.smithi032.stdout:4/348: dread d0/f3d [0,4194304] 0 2024-05-12T04:01:18.181 INFO:tasks.workunit.client.1.smithi033.stdout:8/516: write d1/d4/f39 [4386310,61088] 0 2024-05-12T04:01:18.183 INFO:tasks.workunit.client.0.smithi032.stdout:3/441: link d15/d1b/l8d d15/d1b/la0 0 2024-05-12T04:01:18.184 INFO:tasks.workunit.client.0.smithi032.stdout:3/442: fsync d15/d20/d26/f52 0 2024-05-12T04:01:18.191 INFO:tasks.workunit.client.0.smithi032.stdout:5/501: link d4/d7/d14/d26/d3c/l3f d4/d7/d14/d39/d2b/lb1 0 2024-05-12T04:01:18.193 INFO:tasks.workunit.client.0.smithi032.stdout:3/443: dwrite d15/d20/d26/f5f [0,4194304] 0 2024-05-12T04:01:18.195 INFO:tasks.workunit.client.1.smithi033.stdout:0/610: symlink d6/d11/dc7/dc8/lcd 0 2024-05-12T04:01:18.197 INFO:tasks.workunit.client.0.smithi032.stdout:1/440: symlink d1/d7/d30/l8f 0 2024-05-12T04:01:18.200 INFO:tasks.workunit.client.0.smithi032.stdout:9/303: rename d0/d3/d6/d7/d2a/d46/d13/f1d to d0/d3/d6/f62 0 2024-05-12T04:01:18.202 INFO:tasks.workunit.client.0.smithi032.stdout:2/431: dread d1/d13/d3d/d41/f40 [0,4194304] 0 2024-05-12T04:01:18.205 INFO:tasks.workunit.client.0.smithi032.stdout:5/502: readlink d4/d7/d14/d39/l90 0 2024-05-12T04:01:18.206 INFO:tasks.workunit.client.1.smithi033.stdout:7/767: symlink de/df/d9a/db7/l106 0 2024-05-12T04:01:18.207 INFO:tasks.workunit.client.0.smithi032.stdout:8/441: link d0/d9/d13/d21/f2a d0/d9/d13/d21/f9d 0 2024-05-12T04:01:18.208 INFO:tasks.workunit.client.1.smithi033.stdout:6/598: link d4/d1a/d3c/l4b d4/d6e/lc6 0 2024-05-12T04:01:18.209 INFO:tasks.workunit.client.1.smithi033.stdout:6/599: truncate d4/d5/fa1 494809 0 2024-05-12T04:01:18.210 INFO:tasks.workunit.client.0.smithi032.stdout:3/444: creat d15/d20/d26/d6a/d6e/d7d/fa1 x:0 0 0 2024-05-12T04:01:18.211 INFO:tasks.workunit.client.1.smithi033.stdout:9/822: getdents d2/d4/d2b 0 2024-05-12T04:01:18.212 INFO:tasks.workunit.client.1.smithi033.stdout:0/611: chown d6/db/d2c/d73/daf/l74 13203 1 2024-05-12T04:01:18.214 INFO:tasks.workunit.client.0.smithi032.stdout:8/442: dwrite d0/f2f [4194304,4194304] 0 2024-05-12T04:01:18.216 INFO:tasks.workunit.client.0.smithi032.stdout:8/443: write d0/d9/d13/d21/d25/f83 [1494271,81050] 0 2024-05-12T04:01:18.219 INFO:tasks.workunit.client.0.smithi032.stdout:0/472: getdents df/d10/d3b/d75 0 2024-05-12T04:01:18.227 INFO:tasks.workunit.client.0.smithi032.stdout:9/304: unlink d0/d3/d4b/c52 0 2024-05-12T04:01:18.228 INFO:tasks.workunit.client.1.smithi033.stdout:9/823: dread d2/d4/d2b/d78/f88 [4194304,4194304] 0 2024-05-12T04:01:18.230 INFO:tasks.workunit.client.1.smithi033.stdout:6/600: creat d4/d1a/d6c/fc7 x:0 0 0 2024-05-12T04:01:18.231 INFO:tasks.workunit.client.0.smithi032.stdout:8/444: dwrite d0/d4b/f80 [0,4194304] 0 2024-05-12T04:01:18.233 INFO:tasks.workunit.client.0.smithi032.stdout:8/445: chown d0/l23 6 1 2024-05-12T04:01:18.234 INFO:tasks.workunit.client.1.smithi033.stdout:0/612: creat d6/d11/dc7/fce x:0 0 0 2024-05-12T04:01:18.236 INFO:tasks.workunit.client.1.smithi033.stdout:6/601: dwrite d4/dad/fb7 [0,4194304] 0 2024-05-12T04:01:18.239 INFO:tasks.workunit.client.1.smithi033.stdout:5/502: fsync d3/d15/d16/d45/d62/d7f/d84/f91 0 2024-05-12T04:01:18.241 INFO:tasks.workunit.client.0.smithi032.stdout:6/395: sync 2024-05-12T04:01:18.241 INFO:tasks.workunit.client.0.smithi032.stdout:7/388: sync 2024-05-12T04:01:18.243 INFO:tasks.workunit.client.0.smithi032.stdout:5/503: write d4/d7/d14/f1d [1295688,74838] 0 2024-05-12T04:01:18.244 INFO:tasks.workunit.client.1.smithi033.stdout:8/517: creat d1/d16/fa1 x:0 0 0 2024-05-12T04:01:18.246 INFO:tasks.workunit.client.0.smithi032.stdout:8/446: dread d0/d9/f12 [0,4194304] 0 2024-05-12T04:01:18.246 INFO:tasks.workunit.client.1.smithi033.stdout:7/768: dwrite de/d2b/d6c/f39 [0,4194304] 0 2024-05-12T04:01:18.246 INFO:tasks.workunit.client.0.smithi032.stdout:5/504: write d4/d7/d14/d26/d47/f9b [230433,71366] 0 2024-05-12T04:01:18.249 INFO:tasks.workunit.client.0.smithi032.stdout:3/445: write d15/d24/f3d [975917,44371] 0 2024-05-12T04:01:18.252 INFO:tasks.workunit.client.0.smithi032.stdout:5/505: chown d4/d7/d14/d26/d33/d72/d92 3 1 2024-05-12T04:01:18.252 INFO:tasks.workunit.client.0.smithi032.stdout:5/506: chown d4 482 1 2024-05-12T04:01:18.255 INFO:tasks.workunit.client.0.smithi032.stdout:7/389: dwrite dd/f1a [0,4194304] 0 2024-05-12T04:01:18.256 INFO:tasks.workunit.client.1.smithi033.stdout:8/518: dread d1/d4/d1d/d36/f74 [0,4194304] 0 2024-05-12T04:01:18.256 INFO:tasks.workunit.client.1.smithi033.stdout:6/602: rmdir d4/d1a/d4e/d4f 39 2024-05-12T04:01:18.256 INFO:tasks.workunit.client.1.smithi033.stdout:7/769: dwrite de/d20/d64/fe2 [0,4194304] 0 2024-05-12T04:01:18.258 INFO:tasks.workunit.client.1.smithi033.stdout:3/576: getdents db/d17/d3a/d67/d76/da0 0 2024-05-12T04:01:18.260 INFO:tasks.workunit.client.0.smithi032.stdout:2/432: dread d1/f3 [0,4194304] 0 2024-05-12T04:01:18.261 INFO:tasks.workunit.client.1.smithi033.stdout:7/770: read de/d2b/d6c/da0/da6/dd1/faf [3422134,38181] 0 2024-05-12T04:01:18.264 INFO:tasks.workunit.client.0.smithi032.stdout:2/433: read d1/d13/d3d/d41/f40 [215768,48328] 0 2024-05-12T04:01:18.265 INFO:tasks.workunit.client.1.smithi033.stdout:9/824: mkdir d2/d4/d2b/d6d/d2f/d5f/d84/d107/d128 0 2024-05-12T04:01:18.270 INFO:tasks.workunit.client.1.smithi033.stdout:0/613: creat d6/db/d2c/d73/daf/dc3/fcf x:0 0 0 2024-05-12T04:01:18.270 INFO:tasks.workunit.client.1.smithi033.stdout:0/614: dread - d6/dc/fb3 zero size 2024-05-12T04:01:18.276 INFO:tasks.workunit.client.1.smithi033.stdout:5/503: write d3/d15/d16/d45/d62/f64 [6561,53091] 0 2024-05-12T04:01:18.279 INFO:tasks.workunit.client.0.smithi032.stdout:9/305: creat d0/d3/d6/d7/d2a/f63 x:0 0 0 2024-05-12T04:01:18.281 INFO:tasks.workunit.client.1.smithi033.stdout:7/771: truncate de/d2f/fba 1024266 0 2024-05-12T04:01:18.283 INFO:tasks.workunit.client.1.smithi033.stdout:5/504: dwrite d3/d94/f49 [0,4194304] 0 2024-05-12T04:01:18.283 INFO:tasks.workunit.client.0.smithi032.stdout:4/349: link d0/l1a d0/l5f 0 2024-05-12T04:01:18.284 INFO:tasks.workunit.client.0.smithi032.stdout:4/350: truncate d0/d3/d12/f5b 720762 0 2024-05-12T04:01:18.286 INFO:tasks.workunit.client.1.smithi033.stdout:0/615: creat d6/d11/dc7/dc8/fd0 x:0 0 0 2024-05-12T04:01:18.287 INFO:tasks.workunit.client.0.smithi032.stdout:1/441: creat d1/d7/d49/f90 x:0 0 0 2024-05-12T04:01:18.288 INFO:tasks.workunit.client.1.smithi033.stdout:6/603: dread d4/f16 [0,4194304] 0 2024-05-12T04:01:18.288 INFO:tasks.workunit.client.0.smithi032.stdout:1/442: write d1/d7/d30/d18/f1a [303080,79181] 0 2024-05-12T04:01:18.289 INFO:tasks.workunit.client.1.smithi033.stdout:8/519: rename d1/d4/d1d/d36/c4e to d1/d16/d29/ca2 0 2024-05-12T04:01:18.289 INFO:tasks.workunit.client.0.smithi032.stdout:1/443: chown d1/d59 1915036177 1 2024-05-12T04:01:18.289 INFO:tasks.workunit.client.1.smithi033.stdout:8/520: readlink d1/d16/l4a 0 2024-05-12T04:01:18.292 INFO:tasks.workunit.client.0.smithi032.stdout:0/473: mknod df/d10/d3d/d4a/d8e/ca9 0 2024-05-12T04:01:18.295 INFO:tasks.workunit.client.0.smithi032.stdout:8/447: rename d0/d9/d1b/d27/l72 to d0/d9/d13/d21/d35/l9e 0 2024-05-12T04:01:18.296 INFO:tasks.workunit.client.1.smithi033.stdout:7/772: creat de/df/d7a/f107 x:0 0 0 2024-05-12T04:01:18.297 INFO:tasks.workunit.client.1.smithi033.stdout:7/773: write de/d20/f96 [4363847,92923] 0 2024-05-12T04:01:18.298 INFO:tasks.workunit.client.1.smithi033.stdout:5/505: symlink d3/d12/d29/l9a 0 2024-05-12T04:01:18.300 INFO:tasks.workunit.client.0.smithi032.stdout:2/434: mkdir d1/d34/d95 0 2024-05-12T04:01:18.302 INFO:tasks.workunit.client.0.smithi032.stdout:9/306: mkdir d0/d3/d1b/d64 0 2024-05-12T04:01:18.303 INFO:tasks.workunit.client.1.smithi033.stdout:8/521: rmdir d1/d16/d27/d7c 39 2024-05-12T04:01:18.304 INFO:tasks.workunit.client.0.smithi032.stdout:8/448: dwrite d0/d9/d1b/d27/d88/d56/d63/d8f/d9a/d28/f82 [0,4194304] 0 2024-05-12T04:01:18.305 INFO:tasks.workunit.client.0.smithi032.stdout:4/351: write d0/d3/df/d4e/f5c [1076305,29228] 0 2024-05-12T04:01:18.306 INFO:tasks.workunit.client.0.smithi032.stdout:8/449: chown d0/d9/d13/c47 7737443 1 2024-05-12T04:01:18.308 INFO:tasks.workunit.client.0.smithi032.stdout:8/450: stat d0/d9/d1b/d27/d88/d56/d63/d8f/d9a/d28/d3e/f57 0 2024-05-12T04:01:18.308 INFO:tasks.workunit.client.1.smithi033.stdout:3/577: rename db/d17/d25/d32/d3c/l45 to db/d17/d3a/d67/da6/lcb 0 2024-05-12T04:01:18.309 INFO:tasks.workunit.client.0.smithi032.stdout:8/451: chown d0/d9/d1b/d27/d88/d56 0 1 2024-05-12T04:01:18.311 INFO:tasks.workunit.client.0.smithi032.stdout:0/474: fdatasync df/d10/d48/f96 0 2024-05-12T04:01:18.312 INFO:tasks.workunit.client.0.smithi032.stdout:0/475: dread - df/fa3 zero size 2024-05-12T04:01:18.313 INFO:tasks.workunit.client.1.smithi033.stdout:9/825: creat d2/d4/d29/f129 x:0 0 0 2024-05-12T04:01:18.314 INFO:tasks.workunit.client.0.smithi032.stdout:0/476: readlink df/d28/l31 0 2024-05-12T04:01:18.314 INFO:tasks.workunit.client.1.smithi033.stdout:6/604: dread d4/d43/f70 [0,4194304] 0 2024-05-12T04:01:18.319 INFO:tasks.workunit.client.0.smithi032.stdout:9/307: chown d0/d3/d6/d7/da/d39/f4d 59255 1 2024-05-12T04:01:18.320 INFO:tasks.workunit.client.0.smithi032.stdout:9/308: truncate d0/d3/f42 209011 0 2024-05-12T04:01:18.321 INFO:tasks.workunit.client.1.smithi033.stdout:1/531: sync 2024-05-12T04:01:18.321 INFO:tasks.workunit.client.1.smithi033.stdout:8/522: mkdir d1/d16/d29/d63/da3 0 2024-05-12T04:01:18.322 INFO:tasks.workunit.client.1.smithi033.stdout:1/532: fsync d11/d17/d27/d76/f9c 0 2024-05-12T04:01:18.323 INFO:tasks.workunit.client.1.smithi033.stdout:8/523: truncate d1/d16/d27/f86 477169 0 2024-05-12T04:01:18.326 INFO:tasks.workunit.client.0.smithi032.stdout:7/390: dread dd/d1b/d1e/f38 [0,4194304] 0 2024-05-12T04:01:18.328 INFO:tasks.workunit.client.0.smithi032.stdout:9/309: dwrite d0/d3/d6/d7/f2c [0,4194304] 0 2024-05-12T04:01:18.332 INFO:tasks.workunit.client.1.smithi033.stdout:9/826: creat d2/d70/d7b/d94/f12a x:0 0 0 2024-05-12T04:01:18.339 INFO:tasks.workunit.client.1.smithi033.stdout:9/827: dread d2/f111 [0,4194304] 0 2024-05-12T04:01:18.342 INFO:tasks.workunit.client.0.smithi032.stdout:8/452: dread d0/d9/fe [0,4194304] 0 2024-05-12T04:01:18.345 INFO:tasks.workunit.client.0.smithi032.stdout:6/396: getdents d7/de/d2e 0 2024-05-12T04:01:18.347 INFO:tasks.workunit.client.0.smithi032.stdout:6/397: dread - d7/d9/d48/d67/d25/f2a zero size 2024-05-12T04:01:18.349 INFO:tasks.workunit.client.0.smithi032.stdout:8/453: dwrite d0/d9/d13/f29 [0,4194304] 0 2024-05-12T04:01:18.350 INFO:tasks.workunit.client.1.smithi033.stdout:0/616: truncate d6/f6b 2255526 0 2024-05-12T04:01:18.352 INFO:tasks.workunit.client.0.smithi032.stdout:3/446: rename d15/d33/d3f/l72 to d15/d20/la2 0 2024-05-12T04:01:18.354 INFO:tasks.workunit.client.0.smithi032.stdout:0/477: creat df/d10/d3d/d83/d9f/faa x:0 0 0 2024-05-12T04:01:18.355 INFO:tasks.workunit.client.0.smithi032.stdout:4/352: write d0/d3/df/f29 [1938703,44687] 0 2024-05-12T04:01:18.357 INFO:tasks.workunit.client.1.smithi033.stdout:7/774: write de/f26 [1376653,125510] 0 2024-05-12T04:01:18.360 INFO:tasks.workunit.client.1.smithi033.stdout:5/506: truncate d3/d12/d29/f3d 1200169 0 2024-05-12T04:01:18.361 INFO:tasks.workunit.client.1.smithi033.stdout:5/507: fsync d3/d94/d54/d57/d73/d74/f93 0 2024-05-12T04:01:18.363 INFO:tasks.workunit.client.0.smithi032.stdout:9/310: symlink d0/d3/d6/d7/da/l65 0 2024-05-12T04:01:18.364 INFO:tasks.workunit.client.1.smithi033.stdout:0/617: write d6/d11/f47 [448423,41320] 0 2024-05-12T04:01:18.367 INFO:tasks.workunit.client.1.smithi033.stdout:7/775: readlink de/d2f/d70/l85 0 2024-05-12T04:01:18.369 INFO:tasks.workunit.client.1.smithi033.stdout:7/776: chown de/df/d65/cf6 5 1 2024-05-12T04:01:18.371 INFO:tasks.workunit.client.1.smithi033.stdout:6/605: rename d4/d43/f70 to d4/fc8 0 2024-05-12T04:01:18.372 INFO:tasks.workunit.client.0.smithi032.stdout:0/478: fdatasync df/d30/d4e/d55/f67 0 2024-05-12T04:01:18.373 INFO:tasks.workunit.client.1.smithi033.stdout:3/578: dwrite f6 [0,4194304] 0 2024-05-12T04:01:18.375 INFO:tasks.workunit.client.1.smithi033.stdout:8/524: creat d1/fa4 x:0 0 0 2024-05-12T04:01:18.375 INFO:tasks.workunit.client.0.smithi032.stdout:4/353: symlink d0/d3/df/d4e/l60 0 2024-05-12T04:01:18.377 INFO:tasks.workunit.client.1.smithi033.stdout:5/508: symlink d3/d15/d16/d45/d62/d7f/d84/l9b 0 2024-05-12T04:01:18.378 INFO:tasks.workunit.client.0.smithi032.stdout:2/435: getdents d1/d13/d3d/d41/d39 0 2024-05-12T04:01:18.381 INFO:tasks.workunit.client.0.smithi032.stdout:2/436: write d1/d13/f2f [1990224,12095] 0 2024-05-12T04:01:18.383 INFO:tasks.workunit.client.0.smithi032.stdout:6/398: getdents d7/de/d57 0 2024-05-12T04:01:18.383 INFO:tasks.workunit.client.1.smithi033.stdout:8/525: dwrite d1/d4/f39 [0,4194304] 0 2024-05-12T04:01:18.384 INFO:tasks.workunit.client.1.smithi033.stdout:0/618: mkdir d6/dc/dd1 0 2024-05-12T04:01:18.387 INFO:tasks.workunit.client.0.smithi032.stdout:9/311: rmdir d0/d3/d6/d7/d2a/d46 39 2024-05-12T04:01:18.387 INFO:tasks.workunit.client.1.smithi033.stdout:5/509: dwrite d3/f92 [0,4194304] 0 2024-05-12T04:01:18.388 INFO:tasks.workunit.client.0.smithi032.stdout:4/354: dwrite d0/d3/df/d4e/f5c [0,4194304] 0 2024-05-12T04:01:18.394 INFO:tasks.workunit.client.0.smithi032.stdout:6/399: dread d7/d33/f7a [0,4194304] 0 2024-05-12T04:01:18.394 INFO:tasks.workunit.client.1.smithi033.stdout:8/526: dread d1/f6b [0,4194304] 0 2024-05-12T04:01:18.395 INFO:tasks.workunit.client.0.smithi032.stdout:8/454: dread d0/d9/d13/d21/d25/f42 [0,4194304] 0 2024-05-12T04:01:18.397 INFO:tasks.workunit.client.0.smithi032.stdout:6/400: truncate d7/d9/d48/d67/d25/f74 1006018 0 2024-05-12T04:01:18.399 INFO:tasks.workunit.client.1.smithi033.stdout:1/533: rename d11/d1c/d59/c81 to d11/d17/d27/d36/d3f/d69/d87/da4/cb4 0 2024-05-12T04:01:18.399 INFO:tasks.workunit.client.1.smithi033.stdout:8/527: truncate d1/d16/f96 336217 0 2024-05-12T04:01:18.399 INFO:tasks.workunit.client.1.smithi033.stdout:6/606: mknod d4/d5/cc9 0 2024-05-12T04:01:18.402 INFO:tasks.workunit.client.1.smithi033.stdout:3/579: mkdir db/d59/dcc 0 2024-05-12T04:01:18.402 INFO:tasks.workunit.client.0.smithi032.stdout:0/479: write df/d30/d8d/d8f/f97 [15721,124973] 0 2024-05-12T04:01:18.414 INFO:tasks.workunit.client.0.smithi032.stdout:9/312: unlink d0/d3/d6/d7/da/l11 0 2024-05-12T04:01:18.415 INFO:tasks.workunit.client.1.smithi033.stdout:2/577: sync 2024-05-12T04:01:18.415 INFO:tasks.workunit.client.1.smithi033.stdout:4/600: sync 2024-05-12T04:01:18.416 INFO:tasks.workunit.client.1.smithi033.stdout:7/777: dwrite de/df/d17/d1e/f23 [0,4194304] 0 2024-05-12T04:01:18.417 INFO:tasks.workunit.client.0.smithi032.stdout:4/355: dwrite d0/d3/d31/f35 [0,4194304] 0 2024-05-12T04:01:18.417 INFO:tasks.workunit.client.1.smithi033.stdout:4/601: chown dd/d69/f7d 132257152 1 2024-05-12T04:01:18.418 INFO:tasks.workunit.client.0.smithi032.stdout:7/391: rename dd/d1b/d43/f58 to dd/f82 0 2024-05-12T04:01:18.423 INFO:tasks.workunit.client.0.smithi032.stdout:3/447: write d15/d24/d93/f6b [9784662,84414] 0 2024-05-12T04:01:18.424 INFO:tasks.workunit.client.0.smithi032.stdout:6/401: unlink d7/d9/d5c/f7b 0 2024-05-12T04:01:18.426 INFO:tasks.workunit.client.1.smithi033.stdout:1/534: unlink d11/d1c/c25 0 2024-05-12T04:01:18.429 INFO:tasks.workunit.client.1.smithi033.stdout:8/528: rmdir d1/d4 39 2024-05-12T04:01:18.429 INFO:tasks.workunit.client.1.smithi033.stdout:6/607: mknod d4/d1a/d67/cca 0 2024-05-12T04:01:18.430 INFO:tasks.workunit.client.1.smithi033.stdout:6/608: dread - d4/fae zero size 2024-05-12T04:01:18.433 INFO:tasks.workunit.client.1.smithi033.stdout:3/580: creat db/d17/d4d/fcd x:0 0 0 2024-05-12T04:01:18.434 INFO:tasks.workunit.client.1.smithi033.stdout:5/510: write d3/d12/d29/f59 [770560,2162] 0 2024-05-12T04:01:18.435 INFO:tasks.workunit.client.0.smithi032.stdout:9/313: creat d0/d3/d6/d58/f66 x:0 0 0 2024-05-12T04:01:18.436 INFO:tasks.workunit.client.1.smithi033.stdout:0/619: write d6/dc/f5e [4034246,36896] 0 2024-05-12T04:01:18.437 INFO:tasks.workunit.client.1.smithi033.stdout:0/620: write d6/db/f1a [1477414,4361] 0 2024-05-12T04:01:18.438 INFO:tasks.workunit.client.1.smithi033.stdout:9/828: sync 2024-05-12T04:01:18.439 INFO:tasks.workunit.client.1.smithi033.stdout:9/829: rename d2/d4/d29 to d2/d4/d29/d110/d12b 22 2024-05-12T04:01:18.439 INFO:tasks.workunit.client.1.smithi033.stdout:2/578: creat dc/d17/d27/d2f/d49/fbe x:0 0 0 2024-05-12T04:01:18.440 INFO:tasks.workunit.client.1.smithi033.stdout:7/778: symlink de/df/d17/d1e/l108 0 2024-05-12T04:01:18.441 INFO:tasks.workunit.client.0.smithi032.stdout:3/448: read d15/f4d [157089,102769] 0 2024-05-12T04:01:18.441 INFO:tasks.workunit.client.1.smithi033.stdout:7/779: chown de/d20/f83 162539 1 2024-05-12T04:01:18.443 INFO:tasks.workunit.client.0.smithi032.stdout:6/402: mkdir d7/d9/d5c/d8b 0 2024-05-12T04:01:18.447 INFO:tasks.workunit.client.0.smithi032.stdout:7/392: link dd/d1b/d43/d5e/f6f dd/d7b/f83 0 2024-05-12T04:01:18.449 INFO:tasks.workunit.client.0.smithi032.stdout:9/314: mknod d0/d3/d4b/c67 0 2024-05-12T04:01:18.450 INFO:tasks.workunit.client.0.smithi032.stdout:9/315: readlink d0/d3/d6/d7/da/l1f 0 2024-05-12T04:01:18.452 INFO:tasks.workunit.client.0.smithi032.stdout:7/393: dread dd/d1b/d43/d5e/f67 [0,4194304] 0 2024-05-12T04:01:18.453 INFO:tasks.workunit.client.0.smithi032.stdout:7/394: stat dd/d1b/d1e/d45/d55/d60/d75/c81 0 2024-05-12T04:01:18.456 INFO:tasks.workunit.client.0.smithi032.stdout:3/449: creat d15/d20/d26/d6a/d6e/d7d/d9a/fa3 x:0 0 0 2024-05-12T04:01:18.457 INFO:tasks.workunit.client.1.smithi033.stdout:4/602: write dd/d11/f21 [940437,80868] 0 2024-05-12T04:01:18.460 INFO:tasks.workunit.client.0.smithi032.stdout:0/480: link df/d10/d3d/d6c/l77 df/d28/lab 0 2024-05-12T04:01:18.465 INFO:tasks.workunit.client.0.smithi032.stdout:4/356: truncate d0/d3/d31/f53 3482434 0 2024-05-12T04:01:18.467 INFO:tasks.workunit.client.0.smithi032.stdout:3/450: creat d15/d20/d22/fa4 x:0 0 0 2024-05-12T04:01:18.470 INFO:tasks.workunit.client.0.smithi032.stdout:6/403: write f3 [633921,40405] 0 2024-05-12T04:01:18.556 INFO:tasks.workunit.client.0.smithi032.stdout:9/316: creat d0/d3/d6/d7/d2a/f68 x:0 0 0 2024-05-12T04:01:18.559 INFO:tasks.workunit.client.0.smithi032.stdout:6/404: creat d7/de/d2e/f8c x:0 0 0 2024-05-12T04:01:18.565 INFO:tasks.workunit.client.0.smithi032.stdout:3/451: rename d15/d24/d2a/f63 to d15/d20/d26/fa5 0 2024-05-12T04:01:18.566 INFO:tasks.workunit.client.0.smithi032.stdout:6/405: dwrite d7/d9/d48/d67/d25/f29 [0,4194304] 0 2024-05-12T04:01:18.568 INFO:tasks.workunit.client.0.smithi032.stdout:0/481: truncate df/d30/f8b 316374 0 2024-05-12T04:01:18.568 INFO:tasks.workunit.client.0.smithi032.stdout:9/317: creat d0/d3/d6/f69 x:0 0 0 2024-05-12T04:01:18.569 INFO:tasks.workunit.client.0.smithi032.stdout:0/482: chown df/d10/l15 91 1 2024-05-12T04:01:18.571 INFO:tasks.workunit.client.0.smithi032.stdout:0/483: truncate df/d30/d32/d60/f90 1297290 0 2024-05-12T04:01:18.574 INFO:tasks.workunit.client.0.smithi032.stdout:7/395: getdents dd/d1b/d1e/d45/d55 0 2024-05-12T04:01:18.577 INFO:tasks.workunit.client.0.smithi032.stdout:3/452: creat d15/d20/d22/d7a/fa6 x:0 0 0 2024-05-12T04:01:18.579 INFO:tasks.workunit.client.0.smithi032.stdout:0/484: dwrite df/d10/d3b/d75/f78 [0,4194304] 0 2024-05-12T04:01:18.581 INFO:tasks.workunit.client.1.smithi033.stdout:8/529: truncate d1/d4/d1d/d36/d3c/d47/f60 1450706 0 2024-05-12T04:01:18.581 INFO:tasks.workunit.client.1.smithi033.stdout:8/530: dread - d1/d4/d1d/f8f zero size 2024-05-12T04:01:18.583 INFO:tasks.workunit.client.1.smithi033.stdout:8/531: write d1/d16/d27/d4d/d73/f97 [444715,15495] 0 2024-05-12T04:01:18.585 INFO:tasks.workunit.client.1.smithi033.stdout:6/609: dread d4/f87 [0,4194304] 0 2024-05-12T04:01:18.587 INFO:tasks.workunit.client.0.smithi032.stdout:4/357: getdents d0/d3/d24 0 2024-05-12T04:01:18.590 INFO:tasks.workunit.client.0.smithi032.stdout:4/358: read d0/d3/df/f26 [1280074,89534] 0 2024-05-12T04:01:18.590 INFO:tasks.workunit.client.0.smithi032.stdout:7/396: unlink dd/d7b/f83 0 2024-05-12T04:01:18.598 INFO:tasks.workunit.client.1.smithi033.stdout:0/621: dwrite d6/dc/d58/f86 [0,4194304] 0 2024-05-12T04:01:18.598 INFO:tasks.workunit.client.1.smithi033.stdout:2/579: unlink dc/d17/d32/f5c 0 2024-05-12T04:01:18.599 INFO:tasks.workunit.client.0.smithi032.stdout:3/453: dread - d15/f4e zero size 2024-05-12T04:01:18.600 INFO:tasks.workunit.client.1.smithi033.stdout:7/780: mkdir de/df/d60/d109 0 2024-05-12T04:01:18.601 INFO:tasks.workunit.client.0.smithi032.stdout:5/507: sync 2024-05-12T04:01:18.603 INFO:tasks.workunit.client.1.smithi033.stdout:0/622: dread d6/d11/f33 [0,4194304] 0 2024-05-12T04:01:18.605 INFO:tasks.workunit.client.1.smithi033.stdout:0/623: truncate d6/db/fb0 540270 0 2024-05-12T04:01:18.607 INFO:tasks.workunit.client.1.smithi033.stdout:7/781: dwrite de/df/d17/d1b/d36/f4a [4194304,4194304] 0 2024-05-12T04:01:18.608 INFO:tasks.workunit.client.1.smithi033.stdout:1/535: unlink c10 0 2024-05-12T04:01:18.609 INFO:tasks.workunit.client.0.smithi032.stdout:0/485: dwrite df/d10/f3c [0,4194304] 0 2024-05-12T04:01:18.610 INFO:tasks.workunit.client.0.smithi032.stdout:4/359: fdatasync d0/fa 0 2024-05-12T04:01:18.613 INFO:tasks.workunit.client.0.smithi032.stdout:6/406: rmdir d7/d9/d5c/d86/d8a 0 2024-05-12T04:01:18.614 INFO:tasks.workunit.client.0.smithi032.stdout:9/318: write d0/d3/ff [6994783,126348] 0 2024-05-12T04:01:18.615 INFO:tasks.workunit.client.1.smithi033.stdout:8/532: chown d1/d16/d27/d7c 12838 1 2024-05-12T04:01:18.617 INFO:tasks.workunit.client.0.smithi032.stdout:3/454: creat d15/d20/d26/d6a/d6e/d7d/d9a/fa7 x:0 0 0 2024-05-12T04:01:18.619 INFO:tasks.workunit.client.1.smithi033.stdout:3/581: creat db/d59/dcc/fce x:0 0 0 2024-05-12T04:01:18.622 INFO:tasks.workunit.client.1.smithi033.stdout:9/830: write d2/d70/f87 [1867503,78896] 0 2024-05-12T04:01:18.625 INFO:tasks.workunit.client.1.smithi033.stdout:6/610: symlink d4/d6e/lcb 0 2024-05-12T04:01:18.626 INFO:tasks.workunit.client.1.smithi033.stdout:4/603: write dd/d69/f7d [286928,117206] 0 2024-05-12T04:01:18.627 INFO:tasks.workunit.client.1.smithi033.stdout:4/604: dread - dd/d11/f95 zero size 2024-05-12T04:01:18.632 INFO:tasks.workunit.client.0.smithi032.stdout:9/319: unlink d0/d3/f5c 0 2024-05-12T04:01:18.633 INFO:tasks.workunit.client.0.smithi032.stdout:3/455: unlink d15/d33/d3f/f83 0 2024-05-12T04:01:18.635 INFO:tasks.workunit.client.1.smithi033.stdout:4/605: dwrite dd/d11/d5b/f86 [4194304,4194304] 0 2024-05-12T04:01:18.637 INFO:tasks.workunit.client.1.smithi033.stdout:7/782: rmdir de/d20/d38/d7f 39 2024-05-12T04:01:18.638 INFO:tasks.workunit.client.1.smithi033.stdout:7/783: chown de/df/d65/cf6 1917108313 1 2024-05-12T04:01:18.642 INFO:tasks.workunit.client.1.smithi033.stdout:3/582: unlink db/d17/d25/d32/d3c/f56 0 2024-05-12T04:01:18.647 INFO:tasks.workunit.client.0.smithi032.stdout:6/407: dwrite f6 [0,4194304] 0 2024-05-12T04:01:18.648 INFO:tasks.workunit.client.1.smithi033.stdout:2/580: dwrite dc/d17/d32/f43 [0,4194304] 0 2024-05-12T04:01:18.648 INFO:tasks.workunit.client.0.smithi032.stdout:5/508: link d4/f20 d4/d7/d14/d26/d33/d5f/fb2 0 2024-05-12T04:01:18.649 INFO:tasks.workunit.client.1.smithi033.stdout:2/581: truncate dc/d17/d27/d2f/d49/fbe 26560 0 2024-05-12T04:01:18.650 INFO:tasks.workunit.client.1.smithi033.stdout:0/624: symlink d6/dc/d39/ld2 0 2024-05-12T04:01:18.654 INFO:tasks.workunit.client.1.smithi033.stdout:7/784: write de/d20/f21 [57487,20108] 0 2024-05-12T04:01:18.657 INFO:tasks.workunit.client.1.smithi033.stdout:8/533: getdents d1/d4/d91 0 2024-05-12T04:01:18.659 INFO:tasks.workunit.client.0.smithi032.stdout:3/456: creat d15/d20/d8b/d9b/fa8 x:0 0 0 2024-05-12T04:01:18.661 INFO:tasks.workunit.client.0.smithi032.stdout:5/509: dwrite d4/f12 [8388608,4194304] 0 2024-05-12T04:01:18.663 INFO:tasks.workunit.client.0.smithi032.stdout:3/457: fsync d15/d24/d2a/d5b/f8e 0 2024-05-12T04:01:18.667 INFO:tasks.workunit.client.1.smithi033.stdout:5/511: getdents d3/d94/d54/d57 0 2024-05-12T04:01:18.670 INFO:tasks.workunit.client.1.smithi033.stdout:9/831: rename d2/d4/d29/d3c/d52/cee to d2/d4/d2b/d49/d104/c12c 0 2024-05-12T04:01:18.670 INFO:tasks.workunit.client.0.smithi032.stdout:4/360: link d0/d3/d12/d1d/l4f d0/d3/d12/l61 0 2024-05-12T04:01:18.674 INFO:tasks.workunit.client.1.smithi033.stdout:2/582: mkdir dc/d17/d27/d68/d9f/d5e/dbf 0 2024-05-12T04:01:18.677 INFO:tasks.workunit.client.0.smithi032.stdout:6/408: mkdir d7/d9/d48/d67/d8d 0 2024-05-12T04:01:18.679 INFO:tasks.workunit.client.1.smithi033.stdout:0/625: fdatasync d6/dc/d58/d6a/f89 0 2024-05-12T04:01:18.711 INFO:tasks.workunit.client.1.smithi033.stdout:6/611: sync 2024-05-12T04:01:18.714 INFO:tasks.workunit.client.0.smithi032.stdout:5/510: mknod d4/d7/d14/d39/d1b/cb3 0 2024-05-12T04:01:18.716 INFO:tasks.workunit.client.0.smithi032.stdout:5/511: read - d4/d7/d14/d26/d33/d72/fa7 zero size 2024-05-12T04:01:18.717 INFO:tasks.workunit.client.0.smithi032.stdout:5/512: chown d4/d7/d14/d26/d33 273856 1 2024-05-12T04:01:18.717 INFO:tasks.workunit.client.0.smithi032.stdout:5/513: stat d4/l7d 0 2024-05-12T04:01:18.737 INFO:tasks.workunit.client.0.smithi032.stdout:1/444: sync 2024-05-12T04:01:18.737 INFO:tasks.workunit.client.0.smithi032.stdout:2/437: sync 2024-05-12T04:01:18.737 INFO:tasks.workunit.client.0.smithi032.stdout:2/438: fdatasync d1/f7 0 2024-05-12T04:01:18.738 INFO:tasks.workunit.client.0.smithi032.stdout:1/445: chown d1/d7/d30/d35 1840958 1 2024-05-12T04:01:18.762 INFO:tasks.workunit.client.1.smithi033.stdout:7/785: chown de/df/d17/d1e/dcc/lf4 308321170 1 2024-05-12T04:01:18.763 INFO:tasks.workunit.client.0.smithi032.stdout:3/458: symlink d15/d20/d22/d7a/la9 0 2024-05-12T04:01:18.768 INFO:tasks.workunit.client.0.smithi032.stdout:9/320: rename d0/d3/d6/d7/d27/l40 to d0/l6a 0 2024-05-12T04:01:18.770 INFO:tasks.workunit.client.0.smithi032.stdout:9/321: write d0/d3/d6/d7/d2a/f63 [268425,43399] 0 2024-05-12T04:01:18.770 INFO:tasks.workunit.client.1.smithi033.stdout:8/534: fdatasync d1/d4/d1d/f58 0 2024-05-12T04:01:18.775 INFO:tasks.workunit.client.1.smithi033.stdout:3/583: mknod db/d17/d3a/d93/dc8/ccf 0 2024-05-12T04:01:18.777 INFO:tasks.workunit.client.0.smithi032.stdout:0/486: dwrite df/d30/f8b [0,4194304] 0 2024-05-12T04:01:18.789 INFO:tasks.workunit.client.0.smithi032.stdout:9/322: unlink d0/d3/d1b/f38 0 2024-05-12T04:01:18.790 INFO:tasks.workunit.client.0.smithi032.stdout:9/323: chown d0/d3/d6/d7/d27/l2e 65163306 1 2024-05-12T04:01:18.792 INFO:tasks.workunit.client.1.smithi033.stdout:2/583: creat dc/d3b/d88/fc0 x:0 0 0 2024-05-12T04:01:18.793 INFO:tasks.workunit.client.0.smithi032.stdout:6/409: unlink d7/de/f4b 0 2024-05-12T04:01:18.794 INFO:tasks.workunit.client.0.smithi032.stdout:6/410: write d7/de/f1b [239626,58629] 0 2024-05-12T04:01:18.795 INFO:tasks.workunit.client.0.smithi032.stdout:6/411: chown d7/de/d2e/f81 790478293 1 2024-05-12T04:01:18.816 INFO:tasks.workunit.client.0.smithi032.stdout:6/412: creat d7/de/d2e/f8e x:0 0 0 2024-05-12T04:01:18.819 INFO:tasks.workunit.client.1.smithi033.stdout:9/832: creat d2/d4/d2b/d33/dd2/ddc/f12d x:0 0 0 2024-05-12T04:01:18.821 INFO:tasks.workunit.client.0.smithi032.stdout:5/514: rename d4/d7/d14/c7c to d4/d7/d14/d26/cb4 0 2024-05-12T04:01:18.823 INFO:tasks.workunit.client.0.smithi032.stdout:5/515: fsync d4/d7/d14/d26/d33/d4a/f4e 0 2024-05-12T04:01:18.826 INFO:tasks.workunit.client.0.smithi032.stdout:8/455: sync 2024-05-12T04:01:18.828 INFO:tasks.workunit.client.0.smithi032.stdout:8/456: chown d0/d9/d1b/d27/d4c/d4e 19784557 1 2024-05-12T04:01:18.835 INFO:tasks.workunit.client.1.smithi033.stdout:4/606: rename dd/d11/d5b/fad to dd/d7a/d87/d7e/da7/d38/fbc 0 2024-05-12T04:01:18.837 INFO:tasks.workunit.client.1.smithi033.stdout:9/833: creat d2/d4/d2b/d6d/d2f/d5f/f12e x:0 0 0 2024-05-12T04:01:18.838 INFO:tasks.workunit.client.0.smithi032.stdout:5/516: mkdir d4/d7/d14/db5 0 2024-05-12T04:01:18.839 INFO:tasks.workunit.client.0.smithi032.stdout:3/459: mknod d15/d20/d26/caa 0 2024-05-12T04:01:18.843 INFO:tasks.workunit.client.0.smithi032.stdout:8/457: symlink d0/d9/d1b/d33/l9f 0 2024-05-12T04:01:18.848 INFO:tasks.workunit.client.1.smithi033.stdout:8/535: link d1/d16/d27/d4d/d82/l84 d1/d4/d91/la5 0 2024-05-12T04:01:18.853 INFO:tasks.workunit.client.0.smithi032.stdout:3/460: fsync d15/d24/f81 0 2024-05-12T04:01:18.859 INFO:tasks.workunit.client.1.smithi033.stdout:4/607: symlink dd/d7a/d87/d7e/da7/d66/d42/d59/lbd 0 2024-05-12T04:01:18.863 INFO:tasks.workunit.client.1.smithi033.stdout:9/834: fdatasync d2/dd/d39/d72/d82/fcd 0 2024-05-12T04:01:18.864 INFO:tasks.workunit.client.1.smithi033.stdout:6/612: link d4/l19 d4/d43/d57/lcc 0 2024-05-12T04:01:18.866 INFO:tasks.workunit.client.0.smithi032.stdout:3/461: mknod d15/d20/d26/d6a/d6e/cab 0 2024-05-12T04:01:18.867 INFO:tasks.workunit.client.1.smithi033.stdout:8/536: truncate d1/d4/d1d/d36/d3c/f55 581074 0 2024-05-12T04:01:18.869 INFO:tasks.workunit.client.0.smithi032.stdout:5/517: creat d4/d7/d14/d26/d33/d4a/fb6 x:0 0 0 2024-05-12T04:01:18.870 INFO:tasks.workunit.client.1.smithi033.stdout:9/835: dwrite d2/d70/f106 [0,4194304] 0 2024-05-12T04:01:18.872 INFO:tasks.workunit.client.0.smithi032.stdout:3/462: creat d15/d20/d8b/fac x:0 0 0 2024-05-12T04:01:18.874 INFO:tasks.workunit.client.1.smithi033.stdout:4/608: rmdir dd/d11/d5b/d84 39 2024-05-12T04:01:18.875 INFO:tasks.workunit.client.1.smithi033.stdout:7/786: symlink de/df/d17/d1e/l10a 0 2024-05-12T04:01:18.880 INFO:tasks.workunit.client.1.smithi033.stdout:6/613: mkdir d4/d5/dcd 0 2024-05-12T04:01:18.880 INFO:tasks.workunit.client.1.smithi033.stdout:4/609: dwrite dd/f1b [4194304,4194304] 0 2024-05-12T04:01:18.885 INFO:tasks.workunit.client.1.smithi033.stdout:6/614: dread d4/d1a/d4e/fb5 [0,4194304] 0 2024-05-12T04:01:18.889 INFO:tasks.workunit.client.1.smithi033.stdout:7/787: creat de/d2b/d6c/dbd/f10b x:0 0 0 2024-05-12T04:01:18.891 INFO:tasks.workunit.client.1.smithi033.stdout:7/788: stat de/df/d65/dbc 0 2024-05-12T04:01:18.892 INFO:tasks.workunit.client.1.smithi033.stdout:4/610: mknod dd/d69/da4/db3/cbe 0 2024-05-12T04:01:18.894 INFO:tasks.workunit.client.1.smithi033.stdout:4/611: dread - dd/d11/f95 zero size 2024-05-12T04:01:18.894 INFO:tasks.workunit.client.1.smithi033.stdout:9/836: dwrite d2/d4/d2b/d6d/db5/fa0 [0,4194304] 0 2024-05-12T04:01:18.902 INFO:tasks.workunit.client.1.smithi033.stdout:4/612: creat dd/d7a/d87/d7e/da7/d66/fbf x:0 0 0 2024-05-12T04:01:18.905 INFO:tasks.workunit.client.1.smithi033.stdout:6/615: dwrite d4/d1a/d4e/d4f/d94/fb9 [0,4194304] 0 2024-05-12T04:01:18.907 INFO:tasks.workunit.client.1.smithi033.stdout:4/613: truncate dd/d69/da4/db3/fbb 5090271 0 2024-05-12T04:01:18.908 INFO:tasks.workunit.client.1.smithi033.stdout:6/616: creat d4/dad/fce x:0 0 0 2024-05-12T04:01:18.910 INFO:tasks.workunit.client.1.smithi033.stdout:4/614: truncate dd/f32 3054403 0 2024-05-12T04:01:18.919 INFO:tasks.workunit.client.1.smithi033.stdout:9/837: dread d2/d70/d7b/d94/ff5 [4194304,4194304] 0 2024-05-12T04:01:18.924 INFO:tasks.workunit.client.1.smithi033.stdout:9/838: dread d2/f111 [0,4194304] 0 2024-05-12T04:01:18.929 INFO:tasks.workunit.client.0.smithi032.stdout:9/324: getdents d0/d3/d1b 0 2024-05-12T04:01:18.930 INFO:tasks.workunit.client.0.smithi032.stdout:4/361: sync 2024-05-12T04:01:18.931 INFO:tasks.workunit.client.0.smithi032.stdout:7/397: sync 2024-05-12T04:01:18.935 INFO:tasks.workunit.client.0.smithi032.stdout:7/398: creat dd/d1b/d43/d74/f84 x:0 0 0 2024-05-12T04:01:18.937 INFO:tasks.workunit.client.1.smithi033.stdout:0/626: write d6/d11/f99 [592213,113607] 0 2024-05-12T04:01:18.938 INFO:tasks.workunit.client.0.smithi032.stdout:7/399: creat dd/d6c/f85 x:0 0 0 2024-05-12T04:01:18.940 INFO:tasks.workunit.client.1.smithi033.stdout:1/536: creat d11/fb5 x:0 0 0 2024-05-12T04:01:18.941 INFO:tasks.workunit.client.0.smithi032.stdout:9/325: dread d0/d3/d6/d7/da/f53 [0,4194304] 0 2024-05-12T04:01:18.942 INFO:tasks.workunit.client.0.smithi032.stdout:7/400: creat dd/d1b/d1e/d45/d55/d60/d75/f86 x:0 0 0 2024-05-12T04:01:18.944 INFO:tasks.workunit.client.1.smithi033.stdout:0/627: symlink d6/dc/ld3 0 2024-05-12T04:01:18.946 INFO:tasks.workunit.client.1.smithi033.stdout:1/537: fdatasync d11/f19 0 2024-05-12T04:01:18.947 INFO:tasks.workunit.client.0.smithi032.stdout:4/362: dread d0/d3/d24/f42 [0,4194304] 0 2024-05-12T04:01:18.947 INFO:tasks.workunit.client.0.smithi032.stdout:7/401: symlink dd/d1b/d1e/d45/d55/d60/d75/l87 0 2024-05-12T04:01:18.951 INFO:tasks.workunit.client.0.smithi032.stdout:7/402: creat dd/d1b/d1e/d45/d55/d60/f88 x:0 0 0 2024-05-12T04:01:18.952 INFO:tasks.workunit.client.1.smithi033.stdout:0/628: dwrite d6/db/d2c/d73/f87 [0,4194304] 0 2024-05-12T04:01:18.953 INFO:tasks.workunit.client.0.smithi032.stdout:9/326: creat d0/f6b x:0 0 0 2024-05-12T04:01:18.958 INFO:tasks.workunit.client.0.smithi032.stdout:9/327: creat d0/d3/d6/d7/da/d17/f6c x:0 0 0 2024-05-12T04:01:18.958 INFO:tasks.workunit.client.0.smithi032.stdout:7/403: dwrite dd/d1b/d1e/d45/f73 [0,4194304] 0 2024-05-12T04:01:18.960 INFO:tasks.workunit.client.1.smithi033.stdout:0/629: fdatasync d6/dc/d58/f81 0 2024-05-12T04:01:18.962 INFO:tasks.workunit.client.0.smithi032.stdout:9/328: creat d0/d3/d4b/f6d x:0 0 0 2024-05-12T04:01:18.965 INFO:tasks.workunit.client.0.smithi032.stdout:2/439: rename d1/d13/d3d to d1/d34/d96 0 2024-05-12T04:01:18.971 INFO:tasks.workunit.client.0.smithi032.stdout:6/413: dwrite d7/d9/d48/d67/d1a/d20/f3d [0,4194304] 0 2024-05-12T04:01:18.976 INFO:tasks.workunit.client.0.smithi032.stdout:2/440: readlink d1/l35 0 2024-05-12T04:01:18.978 INFO:tasks.workunit.client.1.smithi033.stdout:0/630: creat d6/dc/d96/d9d/fd4 x:0 0 0 2024-05-12T04:01:18.979 INFO:tasks.workunit.client.1.smithi033.stdout:2/584: truncate dc/f3f 289804 0 2024-05-12T04:01:18.981 INFO:tasks.workunit.client.1.smithi033.stdout:2/585: write dc/d17/d27/d68/d9f/f58 [3571115,39427] 0 2024-05-12T04:01:18.983 INFO:tasks.workunit.client.1.smithi033.stdout:0/631: dread d6/db/d2c/d73/f87 [0,4194304] 0 2024-05-12T04:01:18.985 INFO:tasks.workunit.client.1.smithi033.stdout:4/615: fsync dd/d7a/d87/d7e/da7/d38/fbc 0 2024-05-12T04:01:18.985 INFO:tasks.workunit.client.1.smithi033.stdout:4/616: dread - dd/f3b zero size 2024-05-12T04:01:18.986 INFO:tasks.workunit.client.1.smithi033.stdout:4/617: chown dd/f35 1478 1 2024-05-12T04:01:18.988 INFO:tasks.workunit.client.0.smithi032.stdout:6/414: unlink d7/d9/d48/d67/d1a/d20/f42 0 2024-05-12T04:01:18.989 INFO:tasks.workunit.client.0.smithi032.stdout:6/415: fsync d7/de/d2e/f36 0 2024-05-12T04:01:18.992 INFO:tasks.workunit.client.0.smithi032.stdout:8/458: write d0/d9/d13/f24 [2569308,112293] 0 2024-05-12T04:01:18.992 INFO:tasks.workunit.client.1.smithi033.stdout:0/632: symlink d6/db/d2c/d73/daf/da4/ld5 0 2024-05-12T04:01:18.993 INFO:tasks.workunit.client.1.smithi033.stdout:0/633: readlink d6/dc/d58/l65 0 2024-05-12T04:01:18.994 INFO:tasks.workunit.client.0.smithi032.stdout:1/446: rename d1/d7/d30/d61/c74 to d1/dd/c91 0 2024-05-12T04:01:18.995 INFO:tasks.workunit.client.1.smithi033.stdout:4/618: symlink dd/d69/da4/lc0 0 2024-05-12T04:01:18.996 INFO:tasks.workunit.client.0.smithi032.stdout:6/416: creat d7/d9/d48/d67/d25/f8f x:0 0 0 2024-05-12T04:01:18.997 INFO:tasks.workunit.client.0.smithi032.stdout:6/417: read d7/d9/d48/d67/d25/f29 [2310436,47311] 0 2024-05-12T04:01:19.000 INFO:tasks.workunit.client.0.smithi032.stdout:2/441: rename d1/d34/d96/c72 to d1/d34/d96/d41/d52/c97 0 2024-05-12T04:01:19.001 INFO:tasks.workunit.client.0.smithi032.stdout:2/442: read - d1/d34/d3e/d67/f83 zero size 2024-05-12T04:01:19.007 INFO:tasks.workunit.client.1.smithi033.stdout:6/617: creat d4/fcf x:0 0 0 2024-05-12T04:01:19.007 INFO:tasks.workunit.client.1.smithi033.stdout:9/839: dwrite d2/dd/d39/d72/d82/fcd [0,4194304] 0 2024-05-12T04:01:19.007 INFO:tasks.workunit.client.1.smithi033.stdout:9/840: truncate d2/d4/d29/d3c/f102 49968 0 2024-05-12T04:01:19.010 INFO:tasks.workunit.client.1.smithi033.stdout:2/586: rmdir dc/d17/d27/d68/d9f/d5e/dbf 0 2024-05-12T04:01:19.011 INFO:tasks.workunit.client.0.smithi032.stdout:8/459: fdatasync d0/d9/f2d 0 2024-05-12T04:01:19.016 INFO:tasks.workunit.client.0.smithi032.stdout:6/418: mknod d7/d9/d48/d67/d63/c90 0 2024-05-12T04:01:19.016 INFO:tasks.workunit.client.0.smithi032.stdout:6/419: stat d7/d9/d48/d67/f85 0 2024-05-12T04:01:19.017 INFO:tasks.workunit.client.0.smithi032.stdout:6/420: chown d7/d9/d48/d67/d25/f8f 2093692953 1 2024-05-12T04:01:19.018 INFO:tasks.workunit.client.1.smithi033.stdout:0/634: fdatasync d6/d11/d57/f21 0 2024-05-12T04:01:19.022 INFO:tasks.workunit.client.0.smithi032.stdout:6/421: dread d7/d9/d48/d67/f28 [0,4194304] 0 2024-05-12T04:01:19.023 INFO:tasks.workunit.client.0.smithi032.stdout:6/422: stat d7/d9/d48/d67/d25/f7d 0 2024-05-12T04:01:19.023 INFO:tasks.workunit.client.1.smithi033.stdout:4/619: unlink dd/d7a/d87/d7e/da7/d66/d42/f63 0 2024-05-12T04:01:19.024 INFO:tasks.workunit.client.1.smithi033.stdout:4/620: chown dd/d7a/d87/d7e/da7/d66/d42/d59/l60 0 1 2024-05-12T04:01:19.025 INFO:tasks.workunit.client.1.smithi033.stdout:4/621: chown dd/f36 2298334 1 2024-05-12T04:01:19.025 INFO:tasks.workunit.client.0.smithi032.stdout:1/447: symlink d1/d7/d49/l92 0 2024-05-12T04:01:19.026 INFO:tasks.workunit.client.1.smithi033.stdout:4/622: write dd/d11/d75/f8d [3003758,4896] 0 2024-05-12T04:01:19.027 INFO:tasks.workunit.client.1.smithi033.stdout:6/618: mknod d4/d1a/d67/d6f/cd0 0 2024-05-12T04:01:19.029 INFO:tasks.workunit.client.1.smithi033.stdout:9/841: rmdir d2/d4/d29/d32 39 2024-05-12T04:01:19.030 INFO:tasks.workunit.client.1.smithi033.stdout:9/842: chown d2/d4/d2b/d6d/db5/fa0 3 1 2024-05-12T04:01:19.032 INFO:tasks.workunit.client.0.smithi032.stdout:3/463: dwrite d15/d1b/f25 [8388608,4194304] 0 2024-05-12T04:01:19.033 INFO:tasks.workunit.client.0.smithi032.stdout:3/464: chown d15/d24/l5d 56838 1 2024-05-12T04:01:19.034 INFO:tasks.workunit.client.0.smithi032.stdout:8/460: rename d0/d9/d13/f24 to d0/d9/d13/fa0 0 2024-05-12T04:01:19.036 INFO:tasks.workunit.client.1.smithi033.stdout:8/537: dwrite d1/d16/d27/d4d/d51/f4b [0,4194304] 0 2024-05-12T04:01:19.041 INFO:tasks.workunit.client.1.smithi033.stdout:4/623: dread dd/f2a [0,4194304] 0 2024-05-12T04:01:19.045 INFO:tasks.workunit.client.1.smithi033.stdout:0/635: chown d6/dc/d39/l68 90442221 1 2024-05-12T04:01:19.047 INFO:tasks.workunit.client.0.smithi032.stdout:6/423: mknod d7/d9/d48/d67/d1a/c91 0 2024-05-12T04:01:19.048 INFO:tasks.workunit.client.0.smithi032.stdout:6/424: dread - d7/d9/f6c zero size 2024-05-12T04:01:19.049 INFO:tasks.workunit.client.0.smithi032.stdout:6/425: write d7/d9/f6c [643032,120767] 0 2024-05-12T04:01:19.061 INFO:tasks.workunit.client.0.smithi032.stdout:0/487: mknod df/cac 0 2024-05-12T04:01:19.062 INFO:tasks.workunit.client.0.smithi032.stdout:6/426: symlink d7/d9/d48/d67/d1a/d20/l92 0 2024-05-12T04:01:19.062 INFO:tasks.workunit.client.0.smithi032.stdout:6/427: readlink d7/l1c 0 2024-05-12T04:01:19.063 INFO:tasks.workunit.client.0.smithi032.stdout:6/428: dread - d7/de/d2e/f81 zero size 2024-05-12T04:01:19.063 INFO:tasks.workunit.client.0.smithi032.stdout:0/488: dread df/d10/d3d/d4a/f65 [0,4194304] 0 2024-05-12T04:01:19.065 INFO:tasks.workunit.client.1.smithi033.stdout:5/512: rename d3/f1e to d3/d12/d25/d26/f9c 0 2024-05-12T04:01:19.067 INFO:tasks.workunit.client.1.smithi033.stdout:5/513: write d3/d15/f88 [107341,128098] 0 2024-05-12T04:01:19.067 INFO:tasks.workunit.client.1.smithi033.stdout:6/619: symlink d4/d5/dcd/ld1 0 2024-05-12T04:01:19.069 INFO:tasks.workunit.client.1.smithi033.stdout:6/620: truncate d4/d6e/f90 1406356 0 2024-05-12T04:01:19.072 INFO:tasks.workunit.client.1.smithi033.stdout:9/843: dread d2/d4/f28 [0,4194304] 0 2024-05-12T04:01:19.073 INFO:tasks.workunit.client.1.smithi033.stdout:3/584: rename db/d17/d3a/d67/db0 to db/d17/d3a/d8a/d9d/dd0 0 2024-05-12T04:01:19.074 INFO:tasks.workunit.client.1.smithi033.stdout:3/585: fsync db/f90 0 2024-05-12T04:01:19.075 INFO:tasks.workunit.client.1.smithi033.stdout:3/586: fsync db/d17/d3a/f63 0 2024-05-12T04:01:19.076 INFO:tasks.workunit.client.0.smithi032.stdout:3/465: creat d15/d24/d2a/fad x:0 0 0 2024-05-12T04:01:19.078 INFO:tasks.workunit.client.1.smithi033.stdout:8/538: creat d1/d16/fa6 x:0 0 0 2024-05-12T04:01:19.081 INFO:tasks.workunit.client.0.smithi032.stdout:6/429: mkdir d7/d9/d48/d67/d1a/d93 0 2024-05-12T04:01:19.083 INFO:tasks.workunit.client.0.smithi032.stdout:6/430: write d7/d9/d48/d67/d25/f6f [517034,74898] 0 2024-05-12T04:01:19.089 INFO:tasks.workunit.client.0.smithi032.stdout:6/431: mkdir d7/d9/d48/d67/d63/d64/d94 0 2024-05-12T04:01:19.090 INFO:tasks.workunit.client.1.smithi033.stdout:7/789: rename de/df/d60/d75/ca5 to de/d2f/d42/c10c 0 2024-05-12T04:01:19.091 INFO:tasks.workunit.client.0.smithi032.stdout:9/329: write d0/d3/d6/d7/d2a/f3b [1661054,5530] 0 2024-05-12T04:01:19.092 INFO:tasks.workunit.client.0.smithi032.stdout:7/404: truncate dd/d1b/d43/d5e/d37/f39 628596 0 2024-05-12T04:01:19.093 INFO:tasks.workunit.client.1.smithi033.stdout:3/587: symlink db/d17/d3e/ld1 0 2024-05-12T04:01:19.094 INFO:tasks.workunit.client.1.smithi033.stdout:3/588: dread - db/d17/d25/d32/d3c/d3d/f6f zero size 2024-05-12T04:01:19.095 INFO:tasks.workunit.client.0.smithi032.stdout:7/405: stat dd/d1b/d43/d5e/d37/f46 0 2024-05-12T04:01:19.097 INFO:tasks.workunit.client.0.smithi032.stdout:6/432: dwrite d7/de/d2e/f8e [0,4194304] 0 2024-05-12T04:01:19.100 INFO:tasks.workunit.client.0.smithi032.stdout:9/330: dread d0/d3/d6/d7/da/f1a [0,4194304] 0 2024-05-12T04:01:19.100 INFO:tasks.workunit.client.0.smithi032.stdout:7/406: truncate dd/d1b/d1e/d45/d55/d60/d75/f86 534111 0 2024-05-12T04:01:19.110 INFO:tasks.workunit.client.0.smithi032.stdout:9/331: stat d0/d3/d6/d7/d2a/d46/d13/f3c 0 2024-05-12T04:01:19.112 INFO:tasks.workunit.client.0.smithi032.stdout:6/433: mknod d7/d9/d48/d67/d1a/d93/c95 0 2024-05-12T04:01:19.113 INFO:tasks.workunit.client.0.smithi032.stdout:6/434: fsync d7/d9/d48/d67/d25/f7e 0 2024-05-12T04:01:19.121 INFO:tasks.workunit.client.0.smithi032.stdout:9/332: dwrite d0/d3/d6/d7/da/f1a [4194304,4194304] 0 2024-05-12T04:01:19.122 INFO:tasks.workunit.client.1.smithi033.stdout:1/538: rename d11/f98 to d11/d17/d27/d36/fb6 0 2024-05-12T04:01:19.123 INFO:tasks.workunit.client.0.smithi032.stdout:7/407: dread dd/f13 [0,4194304] 0 2024-05-12T04:01:19.125 INFO:tasks.workunit.client.0.smithi032.stdout:7/408: chown dd/d6c/f77 475701 1 2024-05-12T04:01:19.126 INFO:tasks.workunit.client.0.smithi032.stdout:2/443: write d1/d34/d96/d41/f40 [2434684,116712] 0 2024-05-12T04:01:19.130 INFO:tasks.workunit.client.0.smithi032.stdout:9/333: dwrite d0/d3/d6/d7/da/f1a [4194304,4194304] 0 2024-05-12T04:01:19.130 INFO:tasks.workunit.client.1.smithi033.stdout:3/589: getdents db/d17/d25 0 2024-05-12T04:01:19.132 INFO:tasks.workunit.client.0.smithi032.stdout:7/409: symlink dd/d1b/d1e/d45/d64/l89 0 2024-05-12T04:01:19.132 INFO:tasks.workunit.client.0.smithi032.stdout:2/444: rmdir d1/d13/d6b 39 2024-05-12T04:01:19.134 INFO:tasks.workunit.client.0.smithi032.stdout:2/445: write d1/d34/d3e/d67/f83 [281204,104093] 0 2024-05-12T04:01:19.134 INFO:tasks.workunit.client.1.smithi033.stdout:1/539: creat d11/fb7 x:0 0 0 2024-05-12T04:01:19.135 INFO:tasks.workunit.client.0.smithi032.stdout:9/334: chown d0/d3/d4b/l60 799748432 1 2024-05-12T04:01:19.135 INFO:tasks.workunit.client.0.smithi032.stdout:7/410: symlink dd/d1b/d1e/l8a 0 2024-05-12T04:01:19.151 INFO:tasks.workunit.client.0.smithi032.stdout:2/446: creat d1/d4d/f98 x:0 0 0 2024-05-12T04:01:19.151 INFO:tasks.workunit.client.0.smithi032.stdout:2/447: chown d1/d34/d96 24 1 2024-05-12T04:01:19.153 INFO:tasks.workunit.client.0.smithi032.stdout:2/448: fdatasync d1/d34/d96/d41/f3a 0 2024-05-12T04:01:19.153 INFO:tasks.workunit.client.0.smithi032.stdout:9/335: rename d0/d3/d6/d7/c4c to d0/d3/d6/d7/da/d39/c6e 0 2024-05-12T04:01:19.153 INFO:tasks.workunit.client.0.smithi032.stdout:2/449: stat d1/d4d/f98 0 2024-05-12T04:01:19.155 INFO:tasks.workunit.client.0.smithi032.stdout:2/450: chown d1/d34/d96/d41/d39/l42 254 1 2024-05-12T04:01:19.156 INFO:tasks.workunit.client.1.smithi033.stdout:2/587: write dc/f8d [351741,106352] 0 2024-05-12T04:01:19.157 INFO:tasks.workunit.client.1.smithi033.stdout:2/588: write dc/d17/d32/d84/fa0 [497789,52523] 0 2024-05-12T04:01:19.157 INFO:tasks.workunit.client.0.smithi032.stdout:9/336: mkdir d0/d3/d4b/d6f 0 2024-05-12T04:01:19.158 INFO:tasks.workunit.client.0.smithi032.stdout:9/337: readlink d0/d3/l2d 0 2024-05-12T04:01:19.159 INFO:tasks.workunit.client.1.smithi033.stdout:1/540: dread d11/d17/d27/f48 [0,4194304] 0 2024-05-12T04:01:19.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:18 smithi033 ceph-mon[49525]: pgmap v37: 65 pgs: 65 active+clean; 2.0 GiB data, 7.3 GiB used, 529 GiB / 536 GiB avail; 59 MiB/s rd, 253 MiB/s wr, 469 op/s 2024-05-12T04:01:19.162 INFO:tasks.workunit.client.0.smithi032.stdout:7/411: link dd/f13 dd/d1b/d1e/d45/d64/f8b 0 2024-05-12T04:01:19.166 INFO:tasks.workunit.client.1.smithi033.stdout:1/541: dwrite d11/f2d [0,4194304] 0 2024-05-12T04:01:19.168 INFO:tasks.workunit.client.0.smithi032.stdout:2/451: unlink d1/f29 0 2024-05-12T04:01:19.170 INFO:tasks.workunit.client.0.smithi032.stdout:7/412: symlink dd/d1b/d43/d5e/d37/l8c 0 2024-05-12T04:01:19.171 INFO:tasks.workunit.client.0.smithi032.stdout:7/413: read - dd/d7b/f7d zero size 2024-05-12T04:01:19.172 INFO:tasks.workunit.client.1.smithi033.stdout:1/542: rmdir d11/d1c 39 2024-05-12T04:01:19.172 INFO:tasks.workunit.client.1.smithi033.stdout:2/589: dread dc/d17/d27/d2f/f41 [0,4194304] 0 2024-05-12T04:01:19.173 INFO:tasks.workunit.client.0.smithi032.stdout:7/414: read dd/d1b/d43/d5e/d37/f46 [82032,86053] 0 2024-05-12T04:01:19.173 INFO:tasks.workunit.client.1.smithi033.stdout:1/543: chown d11/d17/d27/d36/d3f/d69 7818 1 2024-05-12T04:01:19.174 INFO:tasks.workunit.client.1.smithi033.stdout:2/590: read - dc/d13/d1c/d98/fb9 zero size 2024-05-12T04:01:19.174 INFO:tasks.workunit.client.0.smithi032.stdout:7/415: write dd/d1b/d1e/d45/d64/f6d [466544,18431] 0 2024-05-12T04:01:19.176 INFO:tasks.workunit.client.1.smithi033.stdout:2/591: write dc/d17/d32/d84/fa0 [485710,100451] 0 2024-05-12T04:01:19.176 INFO:tasks.workunit.client.0.smithi032.stdout:2/452: creat d1/d34/d96/d41/d2c/f99 x:0 0 0 2024-05-12T04:01:19.177 INFO:tasks.workunit.client.0.smithi032.stdout:8/461: write d0/d7/f11 [238694,3727] 0 2024-05-12T04:01:19.177 INFO:tasks.workunit.client.0.smithi032.stdout:2/453: fsync d1/d34/d3e/d67/f83 0 2024-05-12T04:01:19.180 INFO:tasks.workunit.client.1.smithi033.stdout:2/592: truncate dc/d17/d27/d2f/d49/d59/f79 4250318 0 2024-05-12T04:01:19.181 INFO:tasks.workunit.client.0.smithi032.stdout:1/448: write d1/f3 [8811655,77142] 0 2024-05-12T04:01:19.183 INFO:tasks.workunit.client.1.smithi033.stdout:1/544: fsync d11/d1c/f33 0 2024-05-12T04:01:19.184 INFO:tasks.workunit.client.1.smithi033.stdout:1/545: chown d11/d1c/d43/f4d 1210676 1 2024-05-12T04:01:19.185 INFO:tasks.workunit.client.0.smithi032.stdout:2/454: dwrite d1/f31 [0,4194304] 0 2024-05-12T04:01:19.190 INFO:tasks.workunit.client.1.smithi033.stdout:4/624: sync 2024-05-12T04:01:19.191 INFO:tasks.workunit.client.0.smithi032.stdout:7/416: rename f2 to dd/d1b/d1e/f8d 0 2024-05-12T04:01:19.191 INFO:tasks.workunit.client.1.smithi033.stdout:0/636: write d6/dc/f8c [573779,44216] 0 2024-05-12T04:01:19.192 INFO:tasks.workunit.client.0.smithi032.stdout:6/435: write d7/de/d2e/f81 [569259,48770] 0 2024-05-12T04:01:19.193 INFO:tasks.workunit.client.0.smithi032.stdout:8/462: creat d0/d4b/fa1 x:0 0 0 2024-05-12T04:01:19.197 INFO:tasks.workunit.client.1.smithi033.stdout:0/637: unlink f4 0 2024-05-12T04:01:19.198 INFO:tasks.workunit.client.0.smithi032.stdout:1/449: fsync d1/d7/d30/d35/f45 0 2024-05-12T04:01:19.199 INFO:tasks.workunit.client.0.smithi032.stdout:8/463: dwrite d0/d9/d1b/d33/f45 [4194304,4194304] 0 2024-05-12T04:01:19.200 INFO:tasks.workunit.client.1.smithi033.stdout:2/593: dread dc/d13/d1f/d6e/fb6 [0,4194304] 0 2024-05-12T04:01:19.203 INFO:tasks.workunit.client.1.smithi033.stdout:6/621: write d4/d1a/d4e/fb5 [1751099,60420] 0 2024-05-12T04:01:19.203 INFO:tasks.workunit.client.0.smithi032.stdout:9/338: getdents d0/d3/d6/d7/d2a/d46/d13 0 2024-05-12T04:01:19.206 INFO:tasks.workunit.client.1.smithi033.stdout:1/546: rename d11/d1c/d7d to d11/d1c/db8 0 2024-05-12T04:01:19.207 INFO:tasks.workunit.client.0.smithi032.stdout:0/489: dwrite df/d10/d3d/f56 [0,4194304] 0 2024-05-12T04:01:19.208 INFO:tasks.workunit.client.0.smithi032.stdout:2/455: mkdir d1/d4d/d5e/d9a 0 2024-05-12T04:01:19.211 INFO:tasks.workunit.client.1.smithi033.stdout:9/844: dwrite d2/d4/f16 [4194304,4194304] 0 2024-05-12T04:01:19.215 INFO:tasks.workunit.client.0.smithi032.stdout:7/417: symlink dd/d1b/d43/d74/l8e 0 2024-05-12T04:01:19.215 INFO:tasks.workunit.client.0.smithi032.stdout:6/436: symlink d7/d9/d48/d67/d1a/l96 0 2024-05-12T04:01:19.222 INFO:tasks.workunit.client.0.smithi032.stdout:1/450: mkdir d1/d7/d30/d18/d93 0 2024-05-12T04:01:19.224 INFO:tasks.workunit.client.1.smithi033.stdout:6/622: dwrite d4/f16 [4194304,4194304] 0 2024-05-12T04:01:19.224 INFO:tasks.workunit.client.1.smithi033.stdout:8/539: dwrite d1/d16/d27/f2d [0,4194304] 0 2024-05-12T04:01:19.225 INFO:tasks.workunit.client.0.smithi032.stdout:1/451: fsync d1/d7/d30/d18/f76 0 2024-05-12T04:01:19.226 INFO:tasks.workunit.client.0.smithi032.stdout:7/418: dwrite dd/d1b/d43/d5e/f69 [0,4194304] 0 2024-05-12T04:01:19.227 INFO:tasks.workunit.client.1.smithi033.stdout:6/623: chown d4/d5/l31 7684401 1 2024-05-12T04:01:19.228 INFO:tasks.workunit.client.1.smithi033.stdout:8/540: stat d1/d16/fa1 0 2024-05-12T04:01:19.233 INFO:tasks.workunit.client.0.smithi032.stdout:9/339: creat d0/d3/d1b/f70 x:0 0 0 2024-05-12T04:01:19.234 INFO:tasks.workunit.client.0.smithi032.stdout:7/419: read f5 [4321989,90679] 0 2024-05-12T04:01:19.235 INFO:tasks.workunit.client.0.smithi032.stdout:9/340: chown d0/d3/d4b/c67 4 1 2024-05-12T04:01:19.237 INFO:tasks.workunit.client.0.smithi032.stdout:9/341: truncate d0/d3/d4b/f6d 26529 0 2024-05-12T04:01:19.238 INFO:tasks.workunit.client.0.smithi032.stdout:2/456: readlink d1/ld 0 2024-05-12T04:01:19.238 INFO:tasks.workunit.client.1.smithi033.stdout:1/547: symlink d11/d1c/lb9 0 2024-05-12T04:01:19.240 INFO:tasks.workunit.client.1.smithi033.stdout:7/790: dwrite de/d20/d38/fa2 [0,4194304] 0 2024-05-12T04:01:19.240 INFO:tasks.workunit.client.1.smithi033.stdout:5/514: dwrite d3/d15/d16/d45/d4b/f56 [0,4194304] 0 2024-05-12T04:01:19.241 INFO:tasks.workunit.client.1.smithi033.stdout:7/791: stat de 0 2024-05-12T04:01:19.244 INFO:tasks.workunit.client.0.smithi032.stdout:6/437: unlink d7/d9/d48/d67/d1a/l55 0 2024-05-12T04:01:19.245 INFO:tasks.workunit.client.1.smithi033.stdout:7/792: chown de/d2b/d6c/da0/da6/lc9 219240489 1 2024-05-12T04:01:19.245 INFO:tasks.workunit.client.1.smithi033.stdout:3/590: write db/d17/d25/f52 [5259688,42054] 0 2024-05-12T04:01:19.247 INFO:tasks.workunit.client.1.smithi033.stdout:6/624: dwrite d4/d5/f5c [0,4194304] 0 2024-05-12T04:01:19.251 INFO:tasks.workunit.client.1.smithi033.stdout:3/591: truncate db/d17/d3a/d67/d76/f78 1625626 0 2024-05-12T04:01:19.252 INFO:tasks.workunit.client.0.smithi032.stdout:8/464: mknod d0/d9/d13/d21/ca2 0 2024-05-12T04:01:19.257 INFO:tasks.workunit.client.0.smithi032.stdout:9/342: dwrite d0/d3/d6/d58/f5a [0,4194304] 0 2024-05-12T04:01:19.257 INFO:tasks.workunit.client.1.smithi033.stdout:1/548: creat d11/d17/d27/d36/d3f/d69/d87/da4/fba x:0 0 0 2024-05-12T04:01:19.258 INFO:tasks.workunit.client.1.smithi033.stdout:1/549: chown d11/f30 256834 1 2024-05-12T04:01:19.263 INFO:tasks.workunit.client.0.smithi032.stdout:8/465: dwrite d0/d9/d13/f29 [0,4194304] 0 2024-05-12T04:01:19.266 INFO:tasks.workunit.client.0.smithi032.stdout:8/466: write d0/d9/d1b/d27/d4c/d4e/d7f/f9c [263452,7617] 0 2024-05-12T04:01:19.269 INFO:tasks.workunit.client.1.smithi033.stdout:7/793: creat de/d2b/d6c/de1/f10d x:0 0 0 2024-05-12T04:01:19.275 INFO:tasks.workunit.client.1.smithi033.stdout:8/541: rename d1/d16/d20 to d1/d4/d1d/d36/d3c/da7 0 2024-05-12T04:01:19.279 INFO:tasks.workunit.client.1.smithi033.stdout:7/794: unlink de/df/d65/cf6 0 2024-05-12T04:01:19.283 INFO:tasks.workunit.client.1.smithi033.stdout:5/515: rename d3/d94/d54/d57/d70/f87 to d3/d12/d29/d2b/d4f/f9d 0 2024-05-12T04:01:19.286 INFO:tasks.workunit.client.1.smithi033.stdout:8/542: symlink d1/d16/d27/d4d/d51/d8e/la8 0 2024-05-12T04:01:19.288 INFO:tasks.workunit.client.1.smithi033.stdout:8/543: write d1/d4/d1d/f78 [531825,65823] 0 2024-05-12T04:01:19.289 INFO:tasks.workunit.client.1.smithi033.stdout:0/638: dwrite d6/db/d2c/f3e [4194304,4194304] 0 2024-05-12T04:01:19.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:18 smithi032 ceph-mon[40008]: pgmap v37: 65 pgs: 65 active+clean; 2.0 GiB data, 7.3 GiB used, 529 GiB / 536 GiB avail; 59 MiB/s rd, 253 MiB/s wr, 469 op/s 2024-05-12T04:01:19.291 INFO:tasks.workunit.client.1.smithi033.stdout:5/516: creat d3/d94/f9e x:0 0 0 2024-05-12T04:01:19.293 INFO:tasks.workunit.client.1.smithi033.stdout:2/594: truncate dc/d17/d32/d84/f89 436986 0 2024-05-12T04:01:19.299 INFO:tasks.workunit.client.0.smithi032.stdout:0/490: symlink df/d30/d32/d81/lad 0 2024-05-12T04:01:19.300 INFO:tasks.workunit.client.1.smithi033.stdout:5/517: dwrite d3/d94/d54/d57/d73/d74/f93 [0,4194304] 0 2024-05-12T04:01:19.305 INFO:tasks.workunit.client.0.smithi032.stdout:6/438: mkdir d7/d97 0 2024-05-12T04:01:19.307 INFO:tasks.workunit.client.1.smithi033.stdout:2/595: dread dc/f3f [0,4194304] 0 2024-05-12T04:01:19.308 INFO:tasks.workunit.client.1.smithi033.stdout:2/596: write dc/f31 [36330,23845] 0 2024-05-12T04:01:19.311 INFO:tasks.workunit.client.1.smithi033.stdout:7/795: rename de/df/l62 to de/d20/d38/l10e 0 2024-05-12T04:01:19.312 INFO:tasks.workunit.client.0.smithi032.stdout:1/452: dwrite d1/dd/f53 [0,4194304] 0 2024-05-12T04:01:19.312 INFO:tasks.workunit.client.0.smithi032.stdout:4/363: sync 2024-05-12T04:01:19.312 INFO:tasks.workunit.client.0.smithi032.stdout:5/518: sync 2024-05-12T04:01:19.313 INFO:tasks.workunit.client.1.smithi033.stdout:7/796: fsync de/d2b/d6c/f5a 0 2024-05-12T04:01:19.314 INFO:tasks.workunit.client.0.smithi032.stdout:5/519: fsync d4/d7/d14/d26/d33/d4a/f4e 0 2024-05-12T04:01:19.315 INFO:tasks.workunit.client.1.smithi033.stdout:6/625: write d4/d1a/d67/f8c [3281372,82723] 0 2024-05-12T04:01:19.316 INFO:tasks.workunit.client.1.smithi033.stdout:2/597: dwrite dc/d17/d32/f43 [0,4194304] 0 2024-05-12T04:01:19.325 INFO:tasks.workunit.client.0.smithi032.stdout:0/491: truncate df/d10/d48/f96 1125431 0 2024-05-12T04:01:19.328 INFO:tasks.workunit.client.0.smithi032.stdout:8/467: dwrite d0/d9/d1b/f4d [0,4194304] 0 2024-05-12T04:01:19.331 INFO:tasks.workunit.client.0.smithi032.stdout:0/492: write df/d10/d3d/d83/d9f/faa [243152,78463] 0 2024-05-12T04:01:19.333 INFO:tasks.workunit.client.1.smithi033.stdout:2/598: chown dc/c35 35896234 1 2024-05-12T04:01:19.337 INFO:tasks.workunit.client.1.smithi033.stdout:0/639: getdents d6/dc/d58/d6a 0 2024-05-12T04:01:19.338 INFO:tasks.workunit.client.1.smithi033.stdout:0/640: write d6/dc/d58/f88 [338068,58298] 0 2024-05-12T04:01:19.338 INFO:tasks.workunit.client.1.smithi033.stdout:7/797: truncate de/df/d17/d1b/f89 4539281 0 2024-05-12T04:01:19.340 INFO:tasks.workunit.client.1.smithi033.stdout:7/798: write de/df/d17/d1e/fdd [80011,6331] 0 2024-05-12T04:01:19.341 INFO:tasks.workunit.client.1.smithi033.stdout:6/626: creat d4/d1a/d67/dc3/fd2 x:0 0 0 2024-05-12T04:01:19.342 INFO:tasks.workunit.client.0.smithi032.stdout:1/453: creat d1/d7/d30/d35/d54/f94 x:0 0 0 2024-05-12T04:01:19.342 INFO:tasks.workunit.client.1.smithi033.stdout:7/799: fsync de/df/d7a/f9f 0 2024-05-12T04:01:19.342 INFO:tasks.workunit.client.0.smithi032.stdout:9/343: creat d0/d3/d4b/d6f/f71 x:0 0 0 2024-05-12T04:01:19.345 INFO:tasks.workunit.client.0.smithi032.stdout:5/520: stat d4/d7/d14/d26/l71 0 2024-05-12T04:01:19.345 INFO:tasks.workunit.client.0.smithi032.stdout:9/344: dread d0/d3/d6/d7/da/d17/f20 [0,4194304] 0 2024-05-12T04:01:19.346 INFO:tasks.workunit.client.0.smithi032.stdout:9/345: read - d0/d3/d6/d7/d27/f57 zero size 2024-05-12T04:01:19.346 INFO:tasks.workunit.client.0.smithi032.stdout:5/521: dread - d4/d7/d14/d26/d47/f9e zero size 2024-05-12T04:01:19.347 INFO:tasks.workunit.client.1.smithi033.stdout:2/599: read dc/d13/d1c/f25 [357669,88148] 0 2024-05-12T04:01:19.348 INFO:tasks.workunit.client.0.smithi032.stdout:5/522: fsync d4/d7/d14/d26/d33/d5f/f87 0 2024-05-12T04:01:19.350 INFO:tasks.workunit.client.1.smithi033.stdout:8/544: dwrite d1/d16/d27/d28/f77 [0,4194304] 0 2024-05-12T04:01:19.353 INFO:tasks.workunit.client.0.smithi032.stdout:8/468: mkdir d0/d9/d1b/d27/d4c/d4e/d7f/da3 0 2024-05-12T04:01:19.354 INFO:tasks.workunit.client.1.smithi033.stdout:7/800: dwrite de/d2b/d6c/f39 [0,4194304] 0 2024-05-12T04:01:19.354 INFO:tasks.workunit.client.0.smithi032.stdout:8/469: chown d0/d9/d1b/d27/d88/d56/d63/d8f/d9a/d28/f82 408753292 1 2024-05-12T04:01:19.358 INFO:tasks.workunit.client.1.smithi033.stdout:2/600: dread dc/d17/d27/d2f/f41 [0,4194304] 0 2024-05-12T04:01:19.358 INFO:tasks.workunit.client.0.smithi032.stdout:0/493: mknod df/d30/d4e/cae 0 2024-05-12T04:01:19.358 INFO:tasks.workunit.client.1.smithi033.stdout:6/627: mknod d4/d43/d57/dc4/cd3 0 2024-05-12T04:01:19.360 INFO:tasks.workunit.client.0.smithi032.stdout:0/494: chown df/d10/d3d/d4a/f2e 2732920 1 2024-05-12T04:01:19.362 INFO:tasks.workunit.client.0.smithi032.stdout:7/420: dwrite dd/d1b/d1e/d45/d64/f8b [0,4194304] 0 2024-05-12T04:01:19.362 INFO:tasks.workunit.client.0.smithi032.stdout:0/495: chown df/d30/d32/d81 10474469 1 2024-05-12T04:01:19.363 INFO:tasks.workunit.client.0.smithi032.stdout:7/421: readlink dd/d1b/d1e/l52 0 2024-05-12T04:01:19.365 INFO:tasks.workunit.client.1.smithi033.stdout:0/641: write d6/d11/d57/f21 [7126576,59528] 0 2024-05-12T04:01:19.371 INFO:tasks.workunit.client.1.smithi033.stdout:0/642: mkdir d6/db/dd6 0 2024-05-12T04:01:19.371 INFO:tasks.workunit.client.0.smithi032.stdout:5/523: mknod d4/d7/d14/d26/d47/cb7 0 2024-05-12T04:01:19.373 INFO:tasks.workunit.client.1.smithi033.stdout:0/643: write d6/db/d2c/f3e [2196255,130308] 0 2024-05-12T04:01:19.374 INFO:tasks.workunit.client.1.smithi033.stdout:0/644: chown d6/d11/dc7/dc8 4931684 1 2024-05-12T04:01:19.376 INFO:tasks.workunit.client.0.smithi032.stdout:9/346: dwrite d0/d3/d6/d7/da/d17/d51/f54 [0,4194304] 0 2024-05-12T04:01:19.377 INFO:tasks.workunit.client.0.smithi032.stdout:2/457: getdents d1/d34/d3e 0 2024-05-12T04:01:19.378 INFO:tasks.workunit.client.0.smithi032.stdout:3/466: sync 2024-05-12T04:01:19.379 INFO:tasks.workunit.client.1.smithi033.stdout:6/628: creat d4/d1a/d4e/d4f/fd4 x:0 0 0 2024-05-12T04:01:19.380 INFO:tasks.workunit.client.1.smithi033.stdout:8/545: dread d1/d16/d27/d28/f57 [0,4194304] 0 2024-05-12T04:01:19.381 INFO:tasks.workunit.client.1.smithi033.stdout:7/801: dread de/f12 [0,4194304] 0 2024-05-12T04:01:19.382 INFO:tasks.workunit.client.0.smithi032.stdout:8/470: creat d0/d9/d1b/d27/d59/fa4 x:0 0 0 2024-05-12T04:01:19.384 INFO:tasks.workunit.client.0.smithi032.stdout:7/422: rmdir dd/d1b/d43/d74 39 2024-05-12T04:01:19.385 INFO:tasks.workunit.client.0.smithi032.stdout:0/496: stat df/d10/d3d/c49 0 2024-05-12T04:01:19.388 INFO:tasks.workunit.client.0.smithi032.stdout:4/364: creat d0/f62 x:0 0 0 2024-05-12T04:01:19.389 INFO:tasks.workunit.client.0.smithi032.stdout:5/524: creat d4/d7/d14/d26/d33/d4a/d38/fb8 x:0 0 0 2024-05-12T04:01:19.391 INFO:tasks.workunit.client.1.smithi033.stdout:8/546: symlink d1/d4/d91/la9 0 2024-05-12T04:01:19.393 INFO:tasks.workunit.client.0.smithi032.stdout:9/347: rmdir d0/d3/d6/d7/da/d17/d51 39 2024-05-12T04:01:19.394 INFO:tasks.workunit.client.1.smithi033.stdout:7/802: mknod de/df/d17/d1b/de0/c10f 0 2024-05-12T04:01:19.395 INFO:tasks.workunit.client.1.smithi033.stdout:6/629: creat d4/fd5 x:0 0 0 2024-05-12T04:01:19.395 INFO:tasks.workunit.client.0.smithi032.stdout:7/423: mkdir dd/d1b/d1e/d45/d55/d60/d75/d8f 0 2024-05-12T04:01:19.395 INFO:tasks.workunit.client.1.smithi033.stdout:7/803: read de/d2b/d6c/fd7 [1644221,40802] 0 2024-05-12T04:01:19.396 INFO:tasks.workunit.client.1.smithi033.stdout:7/804: chown de/df/d17/f51 0 1 2024-05-12T04:01:19.396 INFO:tasks.workunit.client.0.smithi032.stdout:0/497: rmdir df/d30/d8d/d8f 39 2024-05-12T04:01:19.397 INFO:tasks.workunit.client.0.smithi032.stdout:0/498: fdatasync df/d28/f54 0 2024-05-12T04:01:19.399 INFO:tasks.workunit.client.1.smithi033.stdout:0/645: getdents d6 0 2024-05-12T04:01:19.399 INFO:tasks.workunit.client.0.smithi032.stdout:4/365: mkdir d0/d3/d12/d1d/d63 0 2024-05-12T04:01:19.401 INFO:tasks.workunit.client.0.smithi032.stdout:5/525: chown d4/d7/d14/d26/d33/d72/d92/f97 34798734 1 2024-05-12T04:01:19.402 INFO:tasks.workunit.client.1.smithi033.stdout:6/630: creat d4/d5/fd6 x:0 0 0 2024-05-12T04:01:19.402 INFO:tasks.workunit.client.0.smithi032.stdout:5/526: write d4/f22 [4993565,36287] 0 2024-05-12T04:01:19.406 INFO:tasks.workunit.client.0.smithi032.stdout:0/499: dwrite df/d10/d3d/f56 [0,4194304] 0 2024-05-12T04:01:19.406 INFO:tasks.workunit.client.0.smithi032.stdout:8/471: mknod d0/d9/ca5 0 2024-05-12T04:01:19.409 INFO:tasks.workunit.client.0.smithi032.stdout:7/424: unlink dd/d1b/d1e/d45/c79 0 2024-05-12T04:01:19.410 INFO:tasks.workunit.client.1.smithi033.stdout:8/547: creat d1/d4/d1d/d36/faa x:0 0 0 2024-05-12T04:01:19.411 INFO:tasks.workunit.client.1.smithi033.stdout:7/805: dread de/df/d7a/f9f [0,4194304] 0 2024-05-12T04:01:19.411 INFO:tasks.workunit.client.1.smithi033.stdout:6/631: symlink d4/d1a/d4e/d4f/d94/ld7 0 2024-05-12T04:01:19.411 INFO:tasks.workunit.client.1.smithi033.stdout:6/632: fdatasync d4/d5/f73 0 2024-05-12T04:01:19.414 INFO:tasks.workunit.client.1.smithi033.stdout:8/548: creat d1/d16/d29/fab x:0 0 0 2024-05-12T04:01:19.420 INFO:tasks.workunit.client.1.smithi033.stdout:8/549: unlink d1/d16/d27/d7c/f92 0 2024-05-12T04:01:19.421 INFO:tasks.workunit.client.0.smithi032.stdout:9/348: creat d0/d3/d6/d7/da/f72 x:0 0 0 2024-05-12T04:01:19.422 INFO:tasks.workunit.client.1.smithi033.stdout:7/806: mkdir de/df/d17/d1b/d110 0 2024-05-12T04:01:19.424 INFO:tasks.workunit.client.1.smithi033.stdout:7/807: fdatasync de/df/d65/dbc/fdb 0 2024-05-12T04:01:19.424 INFO:tasks.workunit.client.0.smithi032.stdout:6/439: truncate d7/de/f26 36596 0 2024-05-12T04:01:19.425 INFO:tasks.workunit.client.0.smithi032.stdout:0/500: mkdir df/d30/d4e/da5/daf 0 2024-05-12T04:01:19.426 INFO:tasks.workunit.client.1.smithi033.stdout:7/808: read - de/d2b/d6c/dbd/f10b zero size 2024-05-12T04:01:19.427 INFO:tasks.workunit.client.0.smithi032.stdout:7/425: dwrite f5 [4194304,4194304] 0 2024-05-12T04:01:19.429 INFO:tasks.workunit.client.0.smithi032.stdout:4/366: dread d0/d3/df/f30 [0,4194304] 0 2024-05-12T04:01:19.432 INFO:tasks.workunit.client.1.smithi033.stdout:6/633: unlink d4/l11 0 2024-05-12T04:01:19.433 INFO:tasks.workunit.client.1.smithi033.stdout:6/634: stat d4/d6e 0 2024-05-12T04:01:19.433 INFO:tasks.workunit.client.0.smithi032.stdout:8/472: creat d0/d97/fa6 x:0 0 0 2024-05-12T04:01:19.433 INFO:tasks.workunit.client.1.smithi033.stdout:7/809: dread - de/df/d65/dbc/fdb zero size 2024-05-12T04:01:19.434 INFO:tasks.workunit.client.1.smithi033.stdout:8/550: mknod d1/d16/d27/d4d/cac 0 2024-05-12T04:01:19.436 INFO:tasks.workunit.client.0.smithi032.stdout:9/349: unlink d0/d3/d6/d7/d2a/f63 0 2024-05-12T04:01:19.436 INFO:tasks.workunit.client.0.smithi032.stdout:7/426: dwrite fa [0,4194304] 0 2024-05-12T04:01:19.437 INFO:tasks.workunit.client.1.smithi033.stdout:2/601: fsync dc/d17/d27/d2f/d49/d59/f79 0 2024-05-12T04:01:19.439 INFO:tasks.workunit.client.0.smithi032.stdout:7/427: chown dd/d1b 208 1 2024-05-12T04:01:19.439 INFO:tasks.workunit.client.0.smithi032.stdout:6/440: creat d7/d9/d5c/f98 x:0 0 0 2024-05-12T04:01:19.442 INFO:tasks.workunit.client.1.smithi033.stdout:7/810: dwrite de/df/d17/d1e/d24/f57 [0,4194304] 0 2024-05-12T04:01:19.442 INFO:tasks.workunit.client.0.smithi032.stdout:1/454: dwrite d1/d7/f7f [0,4194304] 0 2024-05-12T04:01:19.443 INFO:tasks.workunit.client.0.smithi032.stdout:6/441: read d7/f66 [461258,97243] 0 2024-05-12T04:01:19.446 INFO:tasks.workunit.client.0.smithi032.stdout:9/350: read d0/d3/f25 [13598,65690] 0 2024-05-12T04:01:19.449 INFO:tasks.workunit.client.0.smithi032.stdout:9/351: truncate d0/d3/d6/d58/f5a 4964652 0 2024-05-12T04:01:19.455 INFO:tasks.workunit.client.0.smithi032.stdout:9/352: dwrite d0/d3/ff [0,4194304] 0 2024-05-12T04:01:19.461 INFO:tasks.workunit.client.0.smithi032.stdout:2/458: dread d1/f21 [0,4194304] 0 2024-05-12T04:01:19.462 INFO:tasks.workunit.client.0.smithi032.stdout:2/459: truncate d1/f24 5601991 0 2024-05-12T04:01:19.468 INFO:tasks.workunit.client.0.smithi032.stdout:2/460: dwrite d1/d13/d70/f91 [0,4194304] 0 2024-05-12T04:01:19.502 INFO:tasks.workunit.client.0.smithi032.stdout:4/367: creat d0/d39/f64 x:0 0 0 2024-05-12T04:01:19.502 INFO:tasks.workunit.client.0.smithi032.stdout:3/467: write d15/d24/d93/f70 [886653,106808] 0 2024-05-12T04:01:19.503 INFO:tasks.workunit.client.0.smithi032.stdout:4/368: dread - d0/f62 zero size 2024-05-12T04:01:19.504 INFO:tasks.workunit.client.0.smithi032.stdout:4/369: write d0/d3/df/f26 [2540565,6221] 0 2024-05-12T04:01:19.511 INFO:tasks.workunit.client.0.smithi032.stdout:7/428: rmdir dd/d1b/d1e/d45/d55/d60/d75 39 2024-05-12T04:01:19.514 INFO:tasks.workunit.client.0.smithi032.stdout:1/455: mknod d1/d7/c95 0 2024-05-12T04:01:19.517 INFO:tasks.workunit.client.0.smithi032.stdout:9/353: mkdir d0/d3/d4b/d73 0 2024-05-12T04:01:19.518 INFO:tasks.workunit.client.1.smithi033.stdout:0/646: write d6/dc/f16 [3723839,85660] 0 2024-05-12T04:01:19.522 INFO:tasks.workunit.client.0.smithi032.stdout:2/461: fdatasync d1/f4 0 2024-05-12T04:01:19.527 INFO:tasks.workunit.client.0.smithi032.stdout:9/354: dwrite d0/f6b [0,4194304] 0 2024-05-12T04:01:19.529 INFO:tasks.workunit.client.0.smithi032.stdout:5/527: dwrite d4/d7/d14/d26/d47/f65 [4194304,4194304] 0 2024-05-12T04:01:19.535 INFO:tasks.workunit.client.0.smithi032.stdout:9/355: dread - d0/d3/d6/d58/f66 zero size 2024-05-12T04:01:19.536 INFO:tasks.workunit.client.1.smithi033.stdout:8/551: mknod d1/d16/d27/d28/cad 0 2024-05-12T04:01:19.536 INFO:tasks.workunit.client.1.smithi033.stdout:9/845: sync 2024-05-12T04:01:19.536 INFO:tasks.workunit.client.1.smithi033.stdout:4/625: sync 2024-05-12T04:01:19.537 INFO:tasks.workunit.client.1.smithi033.stdout:4/626: fsync dd/d11/f19 0 2024-05-12T04:01:19.540 INFO:tasks.workunit.client.1.smithi033.stdout:2/602: mkdir dc/d13/d1c/d98/dc1 0 2024-05-12T04:01:19.542 INFO:tasks.workunit.client.1.smithi033.stdout:2/603: write dc/d17/d3a/f8b [2404078,39548] 0 2024-05-12T04:01:19.543 INFO:tasks.workunit.client.1.smithi033.stdout:7/811: mkdir de/d2b/d6c/da0/d111 0 2024-05-12T04:01:19.546 INFO:tasks.workunit.client.1.smithi033.stdout:0/647: creat d6/dc/d39/fd7 x:0 0 0 2024-05-12T04:01:19.548 INFO:tasks.workunit.client.1.smithi033.stdout:8/552: chown d1/d4/d1d/d36/d3c/l9b 0 1 2024-05-12T04:01:19.549 INFO:tasks.workunit.client.1.smithi033.stdout:9/846: mknod d2/d4/c12f 0 2024-05-12T04:01:19.556 INFO:tasks.workunit.client.1.smithi033.stdout:0/648: dwrite d6/db/d2c/d82/fb4 [0,4194304] 0 2024-05-12T04:01:19.556 INFO:tasks.workunit.client.0.smithi032.stdout:5/528: unlink d4/d7/d14/fa1 0 2024-05-12T04:01:19.558 INFO:tasks.workunit.client.0.smithi032.stdout:5/529: fsync d4/d7/d14/d39/d2b/d6e/f85 0 2024-05-12T04:01:19.558 INFO:tasks.workunit.client.1.smithi033.stdout:7/812: creat de/df/d60/d75/f112 x:0 0 0 2024-05-12T04:01:19.560 INFO:tasks.workunit.client.0.smithi032.stdout:9/356: write d0/d3/d6/d7/da/f1c [766016,3371] 0 2024-05-12T04:01:19.562 INFO:tasks.workunit.client.1.smithi033.stdout:6/635: dread d4/d1a/f42 [0,4194304] 0 2024-05-12T04:01:19.564 INFO:tasks.workunit.client.0.smithi032.stdout:0/501: getdents df/d10/d48/d70 0 2024-05-12T04:01:19.565 INFO:tasks.workunit.client.1.smithi033.stdout:4/627: rmdir dd/d11/d5b/d84 39 2024-05-12T04:01:19.565 INFO:tasks.workunit.client.0.smithi032.stdout:3/468: link d15/d24/l49 d15/d20/d26/lae 0 2024-05-12T04:01:19.566 INFO:tasks.workunit.client.0.smithi032.stdout:3/469: dread - d15/d33/d3f/f87 zero size 2024-05-12T04:01:19.567 INFO:tasks.workunit.client.1.smithi033.stdout:0/649: read d6/db/d2c/f62 [114957,90114] 0 2024-05-12T04:01:19.568 INFO:tasks.workunit.client.1.smithi033.stdout:0/650: write d6/d11/f47 [4365281,114345] 0 2024-05-12T04:01:19.569 INFO:tasks.workunit.client.0.smithi032.stdout:4/370: rename d0/d3/df/d4e/f5c to d0/f65 0 2024-05-12T04:01:19.570 INFO:tasks.workunit.client.1.smithi033.stdout:7/813: mkdir de/d2b/d113 0 2024-05-12T04:01:19.571 INFO:tasks.workunit.client.1.smithi033.stdout:7/814: dread - de/d2b/d6c/dbd/f10b zero size 2024-05-12T04:01:19.571 INFO:tasks.workunit.client.1.smithi033.stdout:7/815: chown de/d20/d38 423 1 2024-05-12T04:01:19.573 INFO:tasks.workunit.client.0.smithi032.stdout:1/456: write d1/d25/f34 [327509,120674] 0 2024-05-12T04:01:19.575 INFO:tasks.workunit.client.1.smithi033.stdout:9/847: mknod d2/d4/d29/d110/c130 0 2024-05-12T04:01:19.576 INFO:tasks.workunit.client.0.smithi032.stdout:5/530: truncate d4/f9a 4155978 0 2024-05-12T04:01:19.578 INFO:tasks.workunit.client.1.smithi033.stdout:6/636: symlink d4/d5/dcd/ld8 0 2024-05-12T04:01:19.579 INFO:tasks.workunit.client.1.smithi033.stdout:4/628: readlink dd/d7a/d87/d7e/da7/d38/l80 0 2024-05-12T04:01:19.584 INFO:tasks.workunit.client.0.smithi032.stdout:2/462: dwrite d1/f3 [0,4194304] 0 2024-05-12T04:01:19.590 INFO:tasks.workunit.client.1.smithi033.stdout:8/553: link d1/d16/d27/f2d d1/d16/d27/fae 0 2024-05-12T04:01:19.590 INFO:tasks.workunit.client.1.smithi033.stdout:8/554: stat d1/d16/d27 0 2024-05-12T04:01:19.591 INFO:tasks.workunit.client.1.smithi033.stdout:8/555: fdatasync d1/d4/d1d/d36/f50 0 2024-05-12T04:01:19.592 INFO:tasks.workunit.client.1.smithi033.stdout:9/848: unlink d2/d4/d2b/d6d/d2f/d5f/d84/d107/l118 0 2024-05-12T04:01:19.593 INFO:tasks.workunit.client.0.smithi032.stdout:1/457: symlink d1/d7/d49/d67/l96 0 2024-05-12T04:01:19.593 INFO:tasks.workunit.client.1.smithi033.stdout:8/556: chown d1/d16/f96 3124 1 2024-05-12T04:01:19.593 INFO:tasks.workunit.client.1.smithi033.stdout:6/637: mkdir d4/d1a/d3c/dd9 0 2024-05-12T04:01:19.596 INFO:tasks.workunit.client.0.smithi032.stdout:5/531: creat d4/d7/d14/d26/d33/d4a/d38/d79/fb9 x:0 0 0 2024-05-12T04:01:19.597 INFO:tasks.workunit.client.1.smithi033.stdout:4/629: creat dd/d69/da4/db3/fc1 x:0 0 0 2024-05-12T04:01:19.598 INFO:tasks.workunit.client.0.smithi032.stdout:0/502: mkdir df/d10/d3d/db0 0 2024-05-12T04:01:19.598 INFO:tasks.workunit.client.0.smithi032.stdout:0/503: dread - df/d30/d4e/d55/fa7 zero size 2024-05-12T04:01:19.609 INFO:tasks.workunit.client.1.smithi033.stdout:9/849: rename d2/d4/d29/d110 to d2/d4/d2b/d6d/db5/d131 0 2024-05-12T04:01:19.610 INFO:tasks.workunit.client.0.smithi032.stdout:3/470: write d15/d24/f81 [333599,84460] 0 2024-05-12T04:01:19.613 INFO:tasks.workunit.client.0.smithi032.stdout:0/504: mknod df/d10/d3d/d83/d9f/cb1 0 2024-05-12T04:01:19.615 INFO:tasks.workunit.client.1.smithi033.stdout:0/651: creat d6/db/d2c/d73/fd8 x:0 0 0 2024-05-12T04:01:19.623 INFO:tasks.workunit.client.0.smithi032.stdout:0/505: dwrite df/d10/d3d/f87 [0,4194304] 0 2024-05-12T04:01:19.623 INFO:tasks.workunit.client.0.smithi032.stdout:0/506: write df/d10/d3d/d83/d9f/faa [209927,56710] 0 2024-05-12T04:01:19.623 INFO:tasks.workunit.client.0.smithi032.stdout:9/357: rename d0/d3/d6/d7/da/d17 to d0/d3/d74 0 2024-05-12T04:01:19.624 INFO:tasks.workunit.client.0.smithi032.stdout:4/371: creat d0/d3/d12/f66 x:0 0 0 2024-05-12T04:01:19.625 INFO:tasks.workunit.client.0.smithi032.stdout:4/372: write d0/f11 [649153,66010] 0 2024-05-12T04:01:19.626 INFO:tasks.workunit.client.0.smithi032.stdout:4/373: chown d0/d3/d24/l4a 226107 1 2024-05-12T04:01:19.626 INFO:tasks.workunit.client.1.smithi033.stdout:9/850: mkdir d2/d4/d2b/d33/dd2/d132 0 2024-05-12T04:01:19.627 INFO:tasks.workunit.client.1.smithi033.stdout:7/816: write de/df/d17/d1e/d24/f6f [358987,56238] 0 2024-05-12T04:01:19.628 INFO:tasks.workunit.client.1.smithi033.stdout:7/817: readlink de/df/lc6 0 2024-05-12T04:01:19.629 INFO:tasks.workunit.client.1.smithi033.stdout:6/638: symlink d4/d1a/lda 0 2024-05-12T04:01:19.629 INFO:tasks.workunit.client.1.smithi033.stdout:9/851: chown d2/dd/d39/lb8 1564947 1 2024-05-12T04:01:19.631 INFO:tasks.workunit.client.1.smithi033.stdout:4/630: mknod dd/cc2 0 2024-05-12T04:01:19.633 INFO:tasks.workunit.client.0.smithi032.stdout:2/463: dwrite d1/d34/d96/d41/d2c/f56 [0,4194304] 0 2024-05-12T04:01:19.637 INFO:tasks.workunit.client.1.smithi033.stdout:5/518: sync 2024-05-12T04:01:19.638 INFO:tasks.workunit.client.1.smithi033.stdout:1/550: sync 2024-05-12T04:01:19.638 INFO:tasks.workunit.client.1.smithi033.stdout:3/592: sync 2024-05-12T04:01:19.638 INFO:tasks.workunit.client.1.smithi033.stdout:8/557: creat d1/d4/d1d/d36/faf x:0 0 0 2024-05-12T04:01:19.639 INFO:tasks.workunit.client.0.smithi032.stdout:9/358: write d0/d3/d6/f62 [4973470,1832] 0 2024-05-12T04:01:19.642 INFO:tasks.workunit.client.1.smithi033.stdout:8/558: chown d1/d16/d27/d4d/d51/d8e 676 1 2024-05-12T04:01:19.643 INFO:tasks.workunit.client.1.smithi033.stdout:6/639: chown d4/c80 223 1 2024-05-12T04:01:19.649 INFO:tasks.workunit.client.1.smithi033.stdout:7/818: mkdir de/df/d17/d1e/dd8/d114 0 2024-05-12T04:01:19.653 INFO:tasks.workunit.client.1.smithi033.stdout:0/652: dwrite d6/dc/d39/f8d [0,4194304] 0 2024-05-12T04:01:19.653 INFO:tasks.workunit.client.1.smithi033.stdout:3/593: unlink db/d17/d3a/d93/dc8/ccf 0 2024-05-12T04:01:19.654 INFO:tasks.workunit.client.1.smithi033.stdout:9/852: write d2/d4/d29/d3c/d52/ffd [594901,80893] 0 2024-05-12T04:01:19.655 INFO:tasks.workunit.client.1.smithi033.stdout:3/594: chown db/d17/d3a/dc0 1 1 2024-05-12T04:01:19.657 INFO:tasks.workunit.client.1.smithi033.stdout:6/640: stat d4/f82 0 2024-05-12T04:01:19.659 INFO:tasks.workunit.client.0.smithi032.stdout:5/532: chown d4/f9a 923319 1 2024-05-12T04:01:19.663 INFO:tasks.workunit.client.0.smithi032.stdout:4/374: dwrite d0/d3/d12/f48 [4194304,4194304] 0 2024-05-12T04:01:19.663 INFO:tasks.workunit.client.1.smithi033.stdout:4/631: rename dd/d7a/d87/d7e/da7/d38/f68 to dd/d7a/d87/d7e/da7/d66/d42/d9c/fc3 0 2024-05-12T04:01:19.664 INFO:tasks.workunit.client.0.smithi032.stdout:1/458: getdents d1 0 2024-05-12T04:01:19.666 INFO:tasks.workunit.client.0.smithi032.stdout:3/471: link d15/d1b/c76 d15/d20/d26/caf 0 2024-05-12T04:01:19.666 INFO:tasks.workunit.client.0.smithi032.stdout:5/533: write d4/d7/d14/d26/d33/d5f/d7f/f84 [2898218,123738] 0 2024-05-12T04:01:19.670 INFO:tasks.workunit.client.0.smithi032.stdout:2/464: link d1/d34/d96/d41/f28 d1/d34/d96/d8a/f9b 0 2024-05-12T04:01:19.672 INFO:tasks.workunit.client.1.smithi033.stdout:5/519: creat d3/d94/d54/d98/f9f x:0 0 0 2024-05-12T04:01:19.672 INFO:tasks.workunit.client.0.smithi032.stdout:7/429: sync 2024-05-12T04:01:19.672 INFO:tasks.workunit.client.0.smithi032.stdout:6/442: sync 2024-05-12T04:01:19.672 INFO:tasks.workunit.client.0.smithi032.stdout:8/473: sync 2024-05-12T04:01:19.673 INFO:tasks.workunit.client.1.smithi033.stdout:5/520: fdatasync d3/d27/f22 0 2024-05-12T04:01:19.673 INFO:tasks.workunit.client.1.smithi033.stdout:8/559: write d1/f13 [312583,121080] 0 2024-05-12T04:01:19.676 INFO:tasks.workunit.client.0.smithi032.stdout:4/375: dwrite d0/d3/df/f29 [0,4194304] 0 2024-05-12T04:01:19.677 INFO:tasks.workunit.client.1.smithi033.stdout:5/521: truncate d3/d12/d29/d2b/d4f/f9d 760 0 2024-05-12T04:01:19.677 INFO:tasks.workunit.client.0.smithi032.stdout:0/507: dread fa [0,4194304] 0 2024-05-12T04:01:19.681 INFO:tasks.workunit.client.1.smithi033.stdout:3/595: unlink db/d17/d3e/c57 0 2024-05-12T04:01:19.683 INFO:tasks.workunit.client.1.smithi033.stdout:6/641: symlink d4/d1a/d67/ldb 0 2024-05-12T04:01:19.684 INFO:tasks.workunit.client.1.smithi033.stdout:7/819: dwrite de/df/d7a/fde [0,4194304] 0 2024-05-12T04:01:19.686 INFO:tasks.workunit.client.1.smithi033.stdout:3/596: stat db/d17/d3a/d93/l95 0 2024-05-12T04:01:19.686 INFO:tasks.workunit.client.0.smithi032.stdout:5/534: unlink d4/f49 0 2024-05-12T04:01:19.688 INFO:tasks.workunit.client.0.smithi032.stdout:3/472: mknod d15/d20/d22/d7a/cb0 0 2024-05-12T04:01:19.691 INFO:tasks.workunit.client.0.smithi032.stdout:1/459: symlink d1/d7/d30/d35/l97 0 2024-05-12T04:01:19.692 INFO:tasks.workunit.client.0.smithi032.stdout:1/460: chown d1/d7/d30/d35/d54/f73 34935 1 2024-05-12T04:01:19.693 INFO:tasks.workunit.client.1.smithi033.stdout:6/642: dread d4/d1a/d4e/fb5 [0,4194304] 0 2024-05-12T04:01:19.693 INFO:tasks.workunit.client.0.smithi032.stdout:1/461: truncate d1/d6a/f7a 4505338 0 2024-05-12T04:01:19.695 INFO:tasks.workunit.client.0.smithi032.stdout:1/462: write d1/d7/d49/d67/f8a [505466,81223] 0 2024-05-12T04:01:19.695 INFO:tasks.workunit.client.1.smithi033.stdout:0/653: dwrite d6/dc/d58/d6a/f89 [0,4194304] 0 2024-05-12T04:01:19.696 INFO:tasks.workunit.client.1.smithi033.stdout:4/632: rename dd/d69/da4/db3 to dd/d69/dc4 0 2024-05-12T04:01:19.696 INFO:tasks.workunit.client.0.smithi032.stdout:1/463: fsync d1/d25/f4a 0 2024-05-12T04:01:19.696 INFO:tasks.workunit.client.1.smithi033.stdout:6/643: truncate d4/fd5 274115 0 2024-05-12T04:01:19.697 INFO:tasks.workunit.client.0.smithi032.stdout:1/464: dread - d1/d7/d30/d35/f38 zero size 2024-05-12T04:01:19.699 INFO:tasks.workunit.client.1.smithi033.stdout:0/654: write d6/db/f1a [3288140,122923] 0 2024-05-12T04:01:19.701 INFO:tasks.workunit.client.1.smithi033.stdout:9/853: dwrite d2/d70/d7b/fbe [0,4194304] 0 2024-05-12T04:01:19.701 INFO:tasks.workunit.client.0.smithi032.stdout:9/359: creat d0/f75 x:0 0 0 2024-05-12T04:01:19.704 INFO:tasks.workunit.client.0.smithi032.stdout:9/360: write d0/d3/d6/d7/d2a/d46/d13/f56 [375122,70032] 0 2024-05-12T04:01:19.705 INFO:tasks.workunit.client.0.smithi032.stdout:6/443: dread f6 [0,4194304] 0 2024-05-12T04:01:19.708 INFO:tasks.workunit.client.1.smithi033.stdout:8/560: unlink d1/d4/d1d/d36/d3c/l93 0 2024-05-12T04:01:19.711 INFO:tasks.workunit.client.1.smithi033.stdout:5/522: symlink d3/d15/d16/d45/d62/la0 0 2024-05-12T04:01:19.713 INFO:tasks.workunit.client.0.smithi032.stdout:0/508: readlink df/d10/d48/l61 0 2024-05-12T04:01:19.717 INFO:tasks.workunit.client.1.smithi033.stdout:3/597: creat db/d17/d3a/d8a/d9d/dd0/fd2 x:0 0 0 2024-05-12T04:01:19.717 INFO:tasks.workunit.client.0.smithi032.stdout:3/473: fsync d15/d33/f9e 0 2024-05-12T04:01:19.718 INFO:tasks.workunit.client.1.smithi033.stdout:3/598: chown db/d17/d3a/d8a/c8f 10 1 2024-05-12T04:01:19.724 INFO:tasks.workunit.client.0.smithi032.stdout:2/465: rename d1/d34/d96/d41/f28 to d1/d13/d70/d85/f9c 0 2024-05-12T04:01:19.725 INFO:tasks.workunit.client.0.smithi032.stdout:2/466: chown d1/d4d/d5e/c6c 6755 1 2024-05-12T04:01:19.729 INFO:tasks.workunit.client.1.smithi033.stdout:0/655: mkdir d6/d11/dc7/dd9 0 2024-05-12T04:01:19.745 INFO:tasks.workunit.client.1.smithi033.stdout:2/604: sync 2024-05-12T04:01:19.774 INFO:tasks.workunit.client.1.smithi033.stdout:9/854: creat d2/d4/d2b/d6d/f133 x:0 0 0 2024-05-12T04:01:19.774 INFO:tasks.workunit.client.0.smithi032.stdout:8/474: mknod d0/d9/d13/d21/ca7 0 2024-05-12T04:01:19.776 INFO:tasks.workunit.client.1.smithi033.stdout:8/561: readlink d1/l9c 0 2024-05-12T04:01:19.776 INFO:tasks.workunit.client.1.smithi033.stdout:6/644: write d4/f87 [512283,68094] 0 2024-05-12T04:01:19.777 INFO:tasks.workunit.client.0.smithi032.stdout:7/430: write dd/d1b/d43/d5e/d37/f54 [534115,97049] 0 2024-05-12T04:01:19.778 INFO:tasks.workunit.client.0.smithi032.stdout:6/444: write d7/d9/d60/f68 [496327,28059] 0 2024-05-12T04:01:19.778 INFO:tasks.workunit.client.1.smithi033.stdout:6/645: write d4/d6e/fb3 [22329,73798] 0 2024-05-12T04:01:19.795 INFO:tasks.workunit.client.1.smithi033.stdout:1/551: sync 2024-05-12T04:01:19.870 INFO:tasks.workunit.client.0.smithi032.stdout:0/509: mknod df/d10/d3d/d4a/d8e/cb2 0 2024-05-12T04:01:19.871 INFO:tasks.workunit.client.0.smithi032.stdout:4/376: truncate d0/d3/d12/f5b 572416 0 2024-05-12T04:01:19.871 INFO:tasks.workunit.client.0.smithi032.stdout:5/535: getdents d4/d7/d14/db5 0 2024-05-12T04:01:19.873 INFO:tasks.workunit.client.0.smithi032.stdout:5/536: chown d4/d7/d14/d26/d47/cb7 76476307 1 2024-05-12T04:01:19.874 INFO:tasks.workunit.client.1.smithi033.stdout:4/633: mknod dd/d11/cc5 0 2024-05-12T04:01:19.878 INFO:tasks.workunit.client.1.smithi033.stdout:2/605: creat dc/d13/d1f/d6e/fc2 x:0 0 0 2024-05-12T04:01:19.879 INFO:tasks.workunit.client.1.smithi033.stdout:9/855: dread - d2/d4/feb zero size 2024-05-12T04:01:19.880 INFO:tasks.workunit.client.0.smithi032.stdout:5/537: dwrite d4/d7/d14/d26/d33/f61 [0,4194304] 0 2024-05-12T04:01:19.882 INFO:tasks.workunit.client.1.smithi033.stdout:5/523: mknod d3/d12/ca1 0 2024-05-12T04:01:19.883 INFO:tasks.workunit.client.0.smithi032.stdout:5/538: chown d4/f6 2352729 1 2024-05-12T04:01:19.884 INFO:tasks.workunit.client.0.smithi032.stdout:1/465: truncate d1/d7/d30/d18/f2b 2588622 0 2024-05-12T04:01:19.889 INFO:tasks.workunit.client.1.smithi033.stdout:1/552: rmdir d11/d17/d27/d36/d46/d4c 39 2024-05-12T04:01:19.890 INFO:tasks.workunit.client.1.smithi033.stdout:0/656: dwrite d6/f61 [0,4194304] 0 2024-05-12T04:01:19.893 INFO:tasks.workunit.client.1.smithi033.stdout:3/599: truncate db/f20 1068538 0 2024-05-12T04:01:19.896 INFO:tasks.workunit.client.0.smithi032.stdout:4/377: rename d0/d3/d12/d1d/l36 to d0/d39/l67 0 2024-05-12T04:01:19.899 INFO:tasks.workunit.client.0.smithi032.stdout:7/431: write dd/d1b/d1e/f68 [895996,76107] 0 2024-05-12T04:01:19.899 INFO:tasks.workunit.client.0.smithi032.stdout:4/378: chown d0/d3/d24/f42 17 1 2024-05-12T04:01:19.901 INFO:tasks.workunit.client.0.smithi032.stdout:7/432: fsync dd/d1b/d1e/f7e 0 2024-05-12T04:01:19.901 INFO:tasks.workunit.client.0.smithi032.stdout:9/361: creat d0/d3/f76 x:0 0 0 2024-05-12T04:01:19.902 INFO:tasks.workunit.client.1.smithi033.stdout:2/606: creat dc/d3b/d88/fc3 x:0 0 0 2024-05-12T04:01:19.906 INFO:tasks.workunit.client.0.smithi032.stdout:5/539: creat d4/d7/d14/d26/d33/d4a/d38/d3b/fba x:0 0 0 2024-05-12T04:01:19.947 INFO:tasks.workunit.client.1.smithi033.stdout:6/646: symlink d4/d43/ldc 0 2024-05-12T04:01:19.948 INFO:tasks.workunit.client.1.smithi033.stdout:1/553: unlink d11/d17/d27/d76/c7e 0 2024-05-12T04:01:19.948 INFO:tasks.workunit.client.1.smithi033.stdout:8/562: truncate d1/d4/d1d/d36/d3c/da7/f43 4873175 0 2024-05-12T04:01:19.948 INFO:tasks.workunit.client.1.smithi033.stdout:0/657: creat d6/d11/d98/fda x:0 0 0 2024-05-12T04:01:19.948 INFO:tasks.workunit.client.1.smithi033.stdout:4/634: symlink dd/d7a/d87/d7e/da7/d66/d42/lc6 0 2024-05-12T04:01:19.948 INFO:tasks.workunit.client.1.smithi033.stdout:5/524: mkdir d3/d94/d85/d99/da2 0 2024-05-12T04:01:19.948 INFO:tasks.workunit.client.1.smithi033.stdout:5/525: dwrite d3/d94/f49 [0,4194304] 0 2024-05-12T04:01:19.948 INFO:tasks.workunit.client.1.smithi033.stdout:6/647: symlink d4/dad/ldd 0 2024-05-12T04:01:19.948 INFO:tasks.workunit.client.1.smithi033.stdout:7/820: link de/df/d17/d1b/cac de/d2b/d6c/c115 0 2024-05-12T04:01:19.949 INFO:tasks.workunit.client.0.smithi032.stdout:5/540: truncate d4/d7/d58/d5a/f96 1039201 0 2024-05-12T04:01:19.949 INFO:tasks.workunit.client.0.smithi032.stdout:0/510: creat df/d30/d8d/d8f/fb3 x:0 0 0 2024-05-12T04:01:19.949 INFO:tasks.workunit.client.0.smithi032.stdout:6/445: truncate d7/fa 3010466 0 2024-05-12T04:01:19.949 INFO:tasks.workunit.client.0.smithi032.stdout:6/446: write f3 [2542889,421] 0 2024-05-12T04:01:19.949 INFO:tasks.workunit.client.0.smithi032.stdout:7/433: dread - dd/d1b/d43/d74/f7f zero size 2024-05-12T04:01:19.949 INFO:tasks.workunit.client.0.smithi032.stdout:9/362: unlink d0/d3/d1b/f70 0 2024-05-12T04:01:19.949 INFO:tasks.workunit.client.0.smithi032.stdout:9/363: dread d0/d3/d6/d7/d2a/d46/f12 [0,4194304] 0 2024-05-12T04:01:19.949 INFO:tasks.workunit.client.0.smithi032.stdout:9/364: chown d0/d3/d4b/l5f 31583567 1 2024-05-12T04:01:19.949 INFO:tasks.workunit.client.0.smithi032.stdout:3/474: getdents d15/d20/d26/d6a/d6e 0 2024-05-12T04:01:19.949 INFO:tasks.workunit.client.0.smithi032.stdout:3/475: read d15/d20/d26/f52 [1021512,106499] 0 2024-05-12T04:01:19.950 INFO:tasks.workunit.client.0.smithi032.stdout:4/379: symlink d0/d3/l68 0 2024-05-12T04:01:19.950 INFO:tasks.workunit.client.1.smithi033.stdout:1/554: creat d11/d17/d27/d76/fbb x:0 0 0 2024-05-12T04:01:19.952 INFO:tasks.workunit.client.1.smithi033.stdout:5/526: dwrite d3/d94/d54/d57/d73/d74/f93 [0,4194304] 0 2024-05-12T04:01:19.953 INFO:tasks.workunit.client.1.smithi033.stdout:1/555: write d11/d17/d9e/fad [190041,40887] 0 2024-05-12T04:01:19.955 INFO:tasks.workunit.client.1.smithi033.stdout:7/821: dwrite de/d20/d64/fe2 [0,4194304] 0 2024-05-12T04:01:19.956 INFO:tasks.workunit.client.0.smithi032.stdout:2/467: sync 2024-05-12T04:01:19.960 INFO:tasks.workunit.client.1.smithi033.stdout:3/600: creat db/d17/d25/d32/d3c/d3d/d47/fd3 x:0 0 0 2024-05-12T04:01:19.961 INFO:tasks.workunit.client.1.smithi033.stdout:2/607: symlink dc/d13/d1c/d98/dc1/lc4 0 2024-05-12T04:01:19.963 INFO:tasks.workunit.client.0.smithi032.stdout:8/475: getdents d0/d9 0 2024-05-12T04:01:19.964 INFO:tasks.workunit.client.1.smithi033.stdout:2/608: dread - dc/d13/d39/d8f/d9e/fb4 zero size 2024-05-12T04:01:19.965 INFO:tasks.workunit.client.1.smithi033.stdout:9/856: mkdir d2/d4/d29/d32/db7/d115/d134 0 2024-05-12T04:01:19.965 INFO:tasks.workunit.client.1.smithi033.stdout:7/822: dwrite de/d2f/d70/fa3 [0,4194304] 0 2024-05-12T04:01:19.967 INFO:tasks.workunit.client.1.smithi033.stdout:9/857: chown d2/d4/d29/d3c/d52/c92 25 1 2024-05-12T04:01:19.970 INFO:tasks.workunit.client.0.smithi032.stdout:0/511: symlink df/d10/lb4 0 2024-05-12T04:01:19.970 INFO:tasks.workunit.client.1.smithi033.stdout:9/858: write d2/d4/d2b/d49/ff0 [1223831,86750] 0 2024-05-12T04:01:19.971 INFO:tasks.workunit.client.1.smithi033.stdout:6/648: mknod d4/d1a/d4e/d4f/d94/cde 0 2024-05-12T04:01:19.972 INFO:tasks.workunit.client.0.smithi032.stdout:1/466: dwrite d1/d7/d30/d18/f2b [0,4194304] 0 2024-05-12T04:01:19.974 INFO:tasks.workunit.client.1.smithi033.stdout:8/563: dwrite d1/d4/d1d/f26 [0,4194304] 0 2024-05-12T04:01:19.977 INFO:tasks.workunit.client.0.smithi032.stdout:4/380: creat d0/d3/df/d4e/f69 x:0 0 0 2024-05-12T04:01:19.979 INFO:tasks.workunit.client.0.smithi032.stdout:6/447: creat d7/d97/f99 x:0 0 0 2024-05-12T04:01:19.983 INFO:tasks.workunit.client.1.smithi033.stdout:5/527: mkdir d3/d94/d54/d57/d70/da3 0 2024-05-12T04:01:19.983 INFO:tasks.workunit.client.0.smithi032.stdout:2/468: rename d1/c5b to d1/d34/d96/d41/d58/c9d 0 2024-05-12T04:01:19.984 INFO:tasks.workunit.client.1.smithi033.stdout:5/528: stat d3/d94/d85/d99/da2 0 2024-05-12T04:01:19.985 INFO:tasks.workunit.client.0.smithi032.stdout:2/469: truncate d1/d34/d3e/d67/f6a 636804 0 2024-05-12T04:01:19.987 INFO:tasks.workunit.client.0.smithi032.stdout:7/434: creat dd/d1b/d1e/d45/d55/d60/d75/f90 x:0 0 0 2024-05-12T04:01:19.988 INFO:tasks.workunit.client.0.smithi032.stdout:7/435: fsync dd/d1b/d1e/d45/d64/f8b 0 2024-05-12T04:01:19.995 INFO:tasks.workunit.client.0.smithi032.stdout:1/467: mknod d1/dd/d24/c98 0 2024-05-12T04:01:19.997 INFO:tasks.workunit.client.1.smithi033.stdout:6/649: unlink d4/dad/fb7 0 2024-05-12T04:01:19.997 INFO:tasks.workunit.client.1.smithi033.stdout:6/650: readlink d4/dad/l93 0 2024-05-12T04:01:19.998 INFO:tasks.workunit.client.0.smithi032.stdout:3/476: mkdir d15/d24/d2a/d8c/db1 0 2024-05-12T04:01:20.005 INFO:tasks.workunit.client.0.smithi032.stdout:2/470: rmdir d1/d4d 39 2024-05-12T04:01:20.006 INFO:tasks.workunit.client.1.smithi033.stdout:5/529: mknod d3/d94/d54/d57/ca4 0 2024-05-12T04:01:20.009 INFO:tasks.workunit.client.0.smithi032.stdout:5/541: getdents d4/d7/d14/d26/d33/d4a/d38/d3b 0 2024-05-12T04:01:20.011 INFO:tasks.workunit.client.1.smithi033.stdout:7/823: write de/df/d60/d75/fa7 [1025852,81097] 0 2024-05-12T04:01:20.011 INFO:tasks.workunit.client.0.smithi032.stdout:5/542: fsync d4/d7/d14/d26/d33/d4a/d38/fb8 0 2024-05-12T04:01:20.012 INFO:tasks.workunit.client.0.smithi032.stdout:5/543: chown d4/d7/d14/d26/d33/d72/l91 1137 1 2024-05-12T04:01:20.013 INFO:tasks.workunit.client.1.smithi033.stdout:7/824: chown de/df/d17/d1e/d24/f57 1439839 1 2024-05-12T04:01:20.015 INFO:tasks.workunit.client.0.smithi032.stdout:3/477: creat d15/d20/d22/d7a/fb2 x:0 0 0 2024-05-12T04:01:20.016 INFO:tasks.workunit.client.1.smithi033.stdout:8/564: write d1/d4/d1d/d36/d3c/da7/f33 [5146705,47016] 0 2024-05-12T04:01:20.018 INFO:tasks.workunit.client.0.smithi032.stdout:4/381: fsync d0/d39/f4b 0 2024-05-12T04:01:20.018 INFO:tasks.workunit.client.0.smithi032.stdout:4/382: chown d0/l34 1382358 1 2024-05-12T04:01:20.021 INFO:tasks.workunit.client.1.smithi033.stdout:6/651: dwrite d4/d5/f2d [0,4194304] 0 2024-05-12T04:01:20.021 INFO:tasks.workunit.client.0.smithi032.stdout:0/512: link df/d10/d3d/d4a/d84/f9c df/d10/d3d/d4a/d9a/d9d/fb5 0 2024-05-12T04:01:20.023 INFO:tasks.workunit.client.1.smithi033.stdout:0/658: link d6/f6b d6/db/d2c/fdb 0 2024-05-12T04:01:20.027 INFO:tasks.workunit.client.1.smithi033.stdout:1/556: creat d11/d17/d27/fbc x:0 0 0 2024-05-12T04:01:20.029 INFO:tasks.workunit.client.0.smithi032.stdout:9/365: getdents d0/d3/d6/d7 0 2024-05-12T04:01:20.030 INFO:tasks.workunit.client.1.smithi033.stdout:5/530: rmdir d3/d15/d16 39 2024-05-12T04:01:20.035 INFO:tasks.workunit.client.0.smithi032.stdout:9/366: dread - d0/d3/d6/d7/da/f72 zero size 2024-05-12T04:01:20.035 INFO:tasks.workunit.client.0.smithi032.stdout:4/383: dwrite d0/d3/d24/f2d [0,4194304] 0 2024-05-12T04:01:20.035 INFO:tasks.workunit.client.0.smithi032.stdout:0/513: dwrite df/d10/d48/f52 [0,4194304] 0 2024-05-12T04:01:20.035 INFO:tasks.workunit.client.0.smithi032.stdout:4/384: write d0/d3/d24/f2d [2743818,55334] 0 2024-05-12T04:01:20.036 INFO:tasks.workunit.client.1.smithi033.stdout:4/635: getdents dd/d11/d75 0 2024-05-12T04:01:20.036 INFO:tasks.workunit.client.1.smithi033.stdout:2/609: link dc/d3b/c44 dc/d13/d1c/d98/cc5 0 2024-05-12T04:01:20.036 INFO:tasks.workunit.client.1.smithi033.stdout:4/636: truncate dd/d69/dc4/fc1 1012107 0 2024-05-12T04:01:20.036 INFO:tasks.workunit.client.1.smithi033.stdout:9/859: dread d2/d4/d2b/d78/f93 [0,4194304] 0 2024-05-12T04:01:20.037 INFO:tasks.workunit.client.1.smithi033.stdout:9/860: stat d2/d70/d7b/d94 0 2024-05-12T04:01:20.037 INFO:tasks.workunit.client.1.smithi033.stdout:9/861: dread - d2/d4/d2b/d78/f114 zero size 2024-05-12T04:01:20.037 INFO:tasks.workunit.client.0.smithi032.stdout:9/367: chown d0/d3/d6/d7/da/l26 208136 1 2024-05-12T04:01:20.037 INFO:tasks.workunit.client.0.smithi032.stdout:5/544: mkdir d4/d7/d14/da9/dbb 0 2024-05-12T04:01:20.038 INFO:tasks.workunit.client.1.smithi033.stdout:7/825: truncate de/df/d17/d1e/d24/f95 1223183 0 2024-05-12T04:01:20.039 INFO:tasks.workunit.client.0.smithi032.stdout:5/545: readlink d4/d7/d14/d26/d33/d72/l91 0 2024-05-12T04:01:20.044 INFO:tasks.workunit.client.0.smithi032.stdout:3/478: rename d15/d1b/f61 to d15/d33/d3f/fb3 0 2024-05-12T04:01:20.048 INFO:tasks.workunit.client.1.smithi033.stdout:6/652: rename d4/d1a/d67/d6f/dac/dbd to d4/d43/d57/dc4/ddf 0 2024-05-12T04:01:20.049 INFO:tasks.workunit.client.1.smithi033.stdout:6/653: chown d4/d1a/d4e/d4f/f9e 2313 1 2024-05-12T04:01:20.055 INFO:tasks.workunit.client.1.smithi033.stdout:2/610: unlink dc/d3b/l71 0 2024-05-12T04:01:20.056 INFO:tasks.workunit.client.0.smithi032.stdout:4/385: chown d0/d3/df/d4e/l58 6071010 1 2024-05-12T04:01:20.057 INFO:tasks.workunit.client.0.smithi032.stdout:1/468: truncate d1/dd/f53 1631300 0 2024-05-12T04:01:20.061 INFO:tasks.workunit.client.1.smithi033.stdout:7/826: symlink de/df/d17/d1e/d79/dd4/l116 0 2024-05-12T04:01:20.062 INFO:tasks.workunit.client.1.smithi033.stdout:7/827: write de/df/d17/d1e/f23 [2021355,94373] 0 2024-05-12T04:01:20.063 INFO:tasks.workunit.client.0.smithi032.stdout:5/546: rename d4/d7/d14/d39/l90 to d4/d7/d14/lbc 0 2024-05-12T04:01:20.064 INFO:tasks.workunit.client.0.smithi032.stdout:5/547: stat d4/d7/d14/d26/l71 0 2024-05-12T04:01:20.065 INFO:tasks.workunit.client.0.smithi032.stdout:3/479: creat d15/d20/d26/d6a/fb4 x:0 0 0 2024-05-12T04:01:20.067 INFO:tasks.workunit.client.0.smithi032.stdout:3/480: write d15/d20/d22/d4f/f53 [424331,26020] 0 2024-05-12T04:01:20.068 INFO:tasks.workunit.client.1.smithi033.stdout:0/659: creat d6/dc/d6e/fdc x:0 0 0 2024-05-12T04:01:20.068 INFO:tasks.workunit.client.0.smithi032.stdout:3/481: fsync d15/d20/d26/f5f 0 2024-05-12T04:01:20.069 INFO:tasks.workunit.client.0.smithi032.stdout:3/482: write d15/d20/d22/d7a/fa6 [454163,23357] 0 2024-05-12T04:01:20.077 INFO:tasks.workunit.client.0.smithi032.stdout:5/548: creat d4/d7/d14/d39/d1b/fbd x:0 0 0 2024-05-12T04:01:20.082 INFO:tasks.workunit.client.1.smithi033.stdout:1/557: dwrite d11/d17/d27/d76/f72 [0,4194304] 0 2024-05-12T04:01:20.084 INFO:tasks.workunit.client.1.smithi033.stdout:9/862: write d2/d4/d2b/d78/f8c [1533513,119879] 0 2024-05-12T04:01:20.084 INFO:tasks.workunit.client.1.smithi033.stdout:7/828: unlink de/d2b/d6c/db5/dd0/lee 0 2024-05-12T04:01:20.085 INFO:tasks.workunit.client.0.smithi032.stdout:5/549: dwrite d4/d7/d14/d26/d33/d5f/d7f/f84 [4194304,4194304] 0 2024-05-12T04:01:20.085 INFO:tasks.workunit.client.1.smithi033.stdout:7/829: read - de/df/d65/dbc/fdb zero size 2024-05-12T04:01:20.093 INFO:tasks.workunit.client.1.smithi033.stdout:8/565: truncate d1/d4/f39 294986 0 2024-05-12T04:01:20.093 INFO:tasks.workunit.client.1.smithi033.stdout:8/566: stat d1/d4/d1d/f8f 0 2024-05-12T04:01:20.093 INFO:tasks.workunit.client.0.smithi032.stdout:3/483: rmdir d15/d1b/d43 39 2024-05-12T04:01:20.098 INFO:tasks.workunit.client.0.smithi032.stdout:9/368: rmdir d0/d3/d1b/d64 0 2024-05-12T04:01:20.099 INFO:tasks.workunit.client.1.smithi033.stdout:2/611: write dc/d13/d65/f83 [61853,86486] 0 2024-05-12T04:01:20.100 INFO:tasks.workunit.client.0.smithi032.stdout:9/369: write d0/d3/d6/d7/da/f72 [728969,113747] 0 2024-05-12T04:01:20.101 INFO:tasks.workunit.client.0.smithi032.stdout:1/469: creat d1/d7/d30/d35/d54/d7e/f99 x:0 0 0 2024-05-12T04:01:20.102 INFO:tasks.workunit.client.0.smithi032.stdout:1/470: chown d1/d7/d49/l5c 136665 1 2024-05-12T04:01:20.103 INFO:tasks.workunit.client.0.smithi032.stdout:2/471: getdents d1/d4d/d5e 0 2024-05-12T04:01:20.104 INFO:tasks.workunit.client.0.smithi032.stdout:9/370: read d0/d3/d6/d7/d2a/d46/d13/f3c [3930164,90700] 0 2024-05-12T04:01:20.105 INFO:tasks.workunit.client.1.smithi033.stdout:4/637: creat dd/d7a/d87/d7e/da7/fc7 x:0 0 0 2024-05-12T04:01:20.105 INFO:tasks.workunit.client.0.smithi032.stdout:2/472: write d1/d34/d96/d41/f40 [2394385,11433] 0 2024-05-12T04:01:20.109 INFO:tasks.workunit.client.0.smithi032.stdout:5/550: creat d4/d7/d58/fbe x:0 0 0 2024-05-12T04:01:20.109 INFO:tasks.workunit.client.0.smithi032.stdout:4/386: write d0/d3/d12/f5b [643151,63005] 0 2024-05-12T04:01:20.110 INFO:tasks.workunit.client.0.smithi032.stdout:4/387: fdatasync d0/d3/d12/f48 0 2024-05-12T04:01:20.111 INFO:tasks.workunit.client.1.smithi033.stdout:3/601: sync 2024-05-12T04:01:20.113 INFO:tasks.workunit.client.1.smithi033.stdout:5/531: truncate d3/d15/d16/d45/d4b/f56 1844126 0 2024-05-12T04:01:20.115 INFO:tasks.workunit.client.1.smithi033.stdout:5/532: chown d3/d15/d16/d45/f68 234 1 2024-05-12T04:01:20.116 INFO:tasks.workunit.client.1.smithi033.stdout:0/660: creat d6/db/dd6/fdd x:0 0 0 2024-05-12T04:01:20.117 INFO:tasks.workunit.client.0.smithi032.stdout:5/551: dread d4/d7/d14/d26/d47/f9b [0,4194304] 0 2024-05-12T04:01:20.118 INFO:tasks.workunit.client.1.smithi033.stdout:8/567: rmdir d1/d16/d27/d4d/d82 39 2024-05-12T04:01:20.120 INFO:tasks.workunit.client.1.smithi033.stdout:3/602: dwrite db/d59/fab [0,4194304] 0 2024-05-12T04:01:20.124 INFO:tasks.workunit.client.1.smithi033.stdout:8/568: stat d1/d4/d1d/d36/d3c/da7/f2f 0 2024-05-12T04:01:20.124 INFO:tasks.workunit.client.1.smithi033.stdout:4/638: dread dd/d11/d5b/fb5 [0,4194304] 0 2024-05-12T04:01:20.124 INFO:tasks.workunit.client.1.smithi033.stdout:1/558: rename d11/d1c/db8/c91 to d11/d17/d27/d76/cbd 0 2024-05-12T04:01:20.127 INFO:tasks.workunit.client.1.smithi033.stdout:4/639: chown dd/d69/dc4/fc1 4530725 1 2024-05-12T04:01:20.129 INFO:tasks.workunit.client.1.smithi033.stdout:0/661: dwrite d6/d11/d57/fb6 [0,4194304] 0 2024-05-12T04:01:20.130 INFO:tasks.workunit.client.1.smithi033.stdout:0/662: fsync d6/db/fb0 0 2024-05-12T04:01:20.131 INFO:tasks.workunit.client.1.smithi033.stdout:9/863: link d2/d70/d7b/d94/ff5 d2/d4/d2b/d78/f135 0 2024-05-12T04:01:20.137 INFO:tasks.workunit.client.1.smithi033.stdout:2/612: write dc/d17/d27/d68/d9f/d5e/f77 [2067797,81234] 0 2024-05-12T04:01:20.139 INFO:tasks.workunit.client.1.smithi033.stdout:2/613: write dc/d13/d1c/d98/fb5 [958704,59592] 0 2024-05-12T04:01:20.143 INFO:tasks.workunit.client.0.smithi032.stdout:5/552: dread d4/d7/d14/d39/d1b/f23 [0,4194304] 0 2024-05-12T04:01:20.145 INFO:tasks.workunit.client.0.smithi032.stdout:3/484: mknod d15/cb5 0 2024-05-12T04:01:20.148 INFO:tasks.workunit.client.1.smithi033.stdout:8/569: unlink d1/d16/d27/d28/f77 0 2024-05-12T04:01:20.148 INFO:tasks.workunit.client.0.smithi032.stdout:3/485: dread d15/d24/f81 [0,4194304] 0 2024-05-12T04:01:20.151 INFO:tasks.workunit.client.1.smithi033.stdout:7/830: rename de/df/d17/d1e/dcc to de/d20/d38/dec/d117 0 2024-05-12T04:01:20.151 INFO:tasks.workunit.client.1.smithi033.stdout:7/831: readlink de/l14 0 2024-05-12T04:01:20.152 INFO:tasks.workunit.client.0.smithi032.stdout:1/471: symlink d1/d7/d30/d18/d93/l9a 0 2024-05-12T04:01:20.152 INFO:tasks.workunit.client.0.smithi032.stdout:1/472: dread - d1/d7/d49/f6b zero size 2024-05-12T04:01:20.155 INFO:tasks.workunit.client.1.smithi033.stdout:0/663: symlink d6/d11/dc7/lde 0 2024-05-12T04:01:20.158 INFO:tasks.workunit.client.0.smithi032.stdout:2/473: link d1/d34/d3e/f7e d1/d34/d3e/f9e 0 2024-05-12T04:01:20.158 INFO:tasks.workunit.client.0.smithi032.stdout:2/474: write d1/d34/d96/d41/f40 [1359242,79001] 0 2024-05-12T04:01:20.159 INFO:tasks.workunit.client.0.smithi032.stdout:2/475: write d1/d34/d3e/d67/f69 [3139054,33907] 0 2024-05-12T04:01:20.161 INFO:tasks.workunit.client.0.smithi032.stdout:4/388: getdents d0/d3/d24 0 2024-05-12T04:01:20.162 INFO:tasks.workunit.client.0.smithi032.stdout:5/553: mknod d4/d7/d14/d26/d33/d72/cbf 0 2024-05-12T04:01:20.170 INFO:tasks.workunit.client.1.smithi033.stdout:8/570: mknod d1/d16/d27/d7c/cb0 0 2024-05-12T04:01:20.170 INFO:tasks.workunit.client.1.smithi033.stdout:8/571: readlink d1/d4/d1d/d36/d3c/da7/l61 0 2024-05-12T04:01:20.171 INFO:tasks.workunit.client.0.smithi032.stdout:5/554: dwrite d4/d7/d14/d39/d1b/fbd [0,4194304] 0 2024-05-12T04:01:20.171 INFO:tasks.workunit.client.1.smithi033.stdout:8/572: write d1/f2 [8318393,1095] 0 2024-05-12T04:01:20.172 INFO:tasks.workunit.client.1.smithi033.stdout:1/559: mknod d11/d17/d27/da6/cbe 0 2024-05-12T04:01:20.173 INFO:tasks.workunit.client.1.smithi033.stdout:5/533: truncate d3/d15/f30 3169579 0 2024-05-12T04:01:20.173 INFO:tasks.workunit.client.1.smithi033.stdout:5/534: chown d3/d94/d85 1850399 1 2024-05-12T04:01:20.174 INFO:tasks.workunit.client.0.smithi032.stdout:1/473: mkdir d1/dd/d24/d9b 0 2024-05-12T04:01:20.178 INFO:tasks.workunit.client.1.smithi033.stdout:3/603: rename db/d42/laf to db/d17/d3a/d93/ld4 0 2024-05-12T04:01:20.182 INFO:tasks.workunit.client.0.smithi032.stdout:2/476: symlink d1/d34/d96/d41/d39/l9f 0 2024-05-12T04:01:20.191 INFO:tasks.workunit.client.0.smithi032.stdout:9/371: getdents d0/d3/d6/d7/d2a 0 2024-05-12T04:01:20.192 INFO:tasks.workunit.client.0.smithi032.stdout:9/372: dread d0/d3/d74/f20 [0,4194304] 0 2024-05-12T04:01:20.193 INFO:tasks.workunit.client.1.smithi033.stdout:4/640: rename dd/d7a/d87/d7e/da7/d66/d42/d59/l5e to dd/d7a/d87/d7e/da7/d66/d42/d9c/lc8 0 2024-05-12T04:01:20.194 INFO:tasks.workunit.client.0.smithi032.stdout:9/373: truncate d0/d3/d4b/f6d 1026655 0 2024-05-12T04:01:20.196 INFO:tasks.workunit.client.0.smithi032.stdout:1/474: creat d1/dd/d24/f9c x:0 0 0 2024-05-12T04:01:20.199 INFO:tasks.workunit.client.0.smithi032.stdout:2/477: unlink d1/f7 0 2024-05-12T04:01:20.199 INFO:tasks.workunit.client.1.smithi033.stdout:3/604: dwrite db/d17/d25/d32/f5d [0,4194304] 0 2024-05-12T04:01:20.202 INFO:tasks.workunit.client.1.smithi033.stdout:1/560: write d11/d17/d27/f44 [4987831,129070] 0 2024-05-12T04:01:20.202 INFO:tasks.workunit.client.1.smithi033.stdout:2/614: rmdir dc/d13/d1c/d98/db2 0 2024-05-12T04:01:20.206 INFO:tasks.workunit.client.0.smithi032.stdout:1/475: dwrite d1/d25/f34 [0,4194304] 0 2024-05-12T04:01:20.213 INFO:tasks.workunit.client.1.smithi033.stdout:0/664: rename d6/d11/d57/fb6 to d6/d11/dc7/fdf 0 2024-05-12T04:01:20.215 INFO:tasks.workunit.client.1.smithi033.stdout:9/864: getdents d2/d4/d2b/d49/d11d 0 2024-05-12T04:01:20.219 INFO:tasks.workunit.client.0.smithi032.stdout:4/389: truncate d0/d3/df/f26 966247 0 2024-05-12T04:01:20.220 INFO:tasks.workunit.client.0.smithi032.stdout:4/390: chown d0/d3/df/d4e/l58 5693 1 2024-05-12T04:01:20.221 INFO:tasks.workunit.client.0.smithi032.stdout:4/391: write d0/f13 [4095863,18792] 0 2024-05-12T04:01:20.222 INFO:tasks.workunit.client.1.smithi033.stdout:1/561: mkdir d11/d17/d27/d76/d93/dbf 0 2024-05-12T04:01:20.225 INFO:tasks.workunit.client.1.smithi033.stdout:6/654: sync 2024-05-12T04:01:20.225 INFO:tasks.workunit.client.1.smithi033.stdout:1/562: write d11/d17/d27/f31 [3357698,120349] 0 2024-05-12T04:01:20.226 INFO:tasks.workunit.client.0.smithi032.stdout:1/476: rename d1/d7/d30/d18/c1d to d1/d7/d49/d79/c9d 0 2024-05-12T04:01:20.226 INFO:tasks.workunit.client.1.smithi033.stdout:5/535: rmdir d3/d94/d54/d57/d70 39 2024-05-12T04:01:20.227 INFO:tasks.workunit.client.0.smithi032.stdout:1/477: truncate d1/dd/f7c 1188301 0 2024-05-12T04:01:20.228 INFO:tasks.workunit.client.0.smithi032.stdout:3/486: getdents d15/d20/d26/d6a/d6e/d7d 0 2024-05-12T04:01:20.229 INFO:tasks.workunit.client.1.smithi033.stdout:8/573: dwrite d1/d4/f39 [0,4194304] 0 2024-05-12T04:01:20.229 INFO:tasks.workunit.client.1.smithi033.stdout:6/655: write d4/d1a/d67/f8c [5015243,100264] 0 2024-05-12T04:01:20.242 INFO:tasks.workunit.client.0.smithi032.stdout:5/555: rmdir d4/d7/d14/d26/d33/d4a/d9d 0 2024-05-12T04:01:20.242 INFO:tasks.workunit.client.0.smithi032.stdout:9/374: creat d0/d3/d74/d51/f77 x:0 0 0 2024-05-12T04:01:20.243 INFO:tasks.workunit.client.0.smithi032.stdout:9/375: chown d0/d3/d74/c4a 117070495 1 2024-05-12T04:01:20.243 INFO:tasks.workunit.client.1.smithi033.stdout:0/665: creat d6/db/d2c/d73/daf/da4/fe0 x:0 0 0 2024-05-12T04:01:20.243 INFO:tasks.workunit.client.1.smithi033.stdout:0/666: fdatasync d6/dc/d58/d6a/f7e 0 2024-05-12T04:01:20.244 INFO:tasks.workunit.client.0.smithi032.stdout:2/478: mkdir d1/d13/da0 0 2024-05-12T04:01:20.245 INFO:tasks.workunit.client.1.smithi033.stdout:2/615: dwrite dc/d17/d27/d2f/f38 [0,4194304] 0 2024-05-12T04:01:20.245 INFO:tasks.workunit.client.1.smithi033.stdout:0/667: chown d6/dc/d39/l3d 1806822 1 2024-05-12T04:01:20.245 INFO:tasks.workunit.client.1.smithi033.stdout:9/865: fdatasync d2/d4/d2b/d49/fb9 0 2024-05-12T04:01:20.246 INFO:tasks.workunit.client.0.smithi032.stdout:4/392: write d0/d3/d24/f42 [1786318,42237] 0 2024-05-12T04:01:20.248 INFO:tasks.workunit.client.0.smithi032.stdout:1/478: unlink d1/d7/d30/d18/f2b 0 2024-05-12T04:01:20.252 INFO:tasks.workunit.client.0.smithi032.stdout:3/487: creat d15/d20/d22/fb6 x:0 0 0 2024-05-12T04:01:20.254 INFO:tasks.workunit.client.0.smithi032.stdout:4/393: dwrite d0/d3/d12/f66 [0,4194304] 0 2024-05-12T04:01:20.255 INFO:tasks.workunit.client.1.smithi033.stdout:7/832: sync 2024-05-12T04:01:20.256 INFO:tasks.workunit.client.1.smithi033.stdout:4/641: dwrite dd/d7a/d87/d7e/da7/d66/fba [0,4194304] 0 2024-05-12T04:01:20.257 INFO:tasks.workunit.client.0.smithi032.stdout:4/394: write d0/d3/d24/f2d [3458021,77089] 0 2024-05-12T04:01:20.257 INFO:tasks.workunit.client.1.smithi033.stdout:8/574: mkdir d1/d16/d29/db1 0 2024-05-12T04:01:20.260 INFO:tasks.workunit.client.1.smithi033.stdout:6/656: mknod d4/d1a/d67/dc3/ce0 0 2024-05-12T04:01:20.261 INFO:tasks.workunit.client.1.smithi033.stdout:8/575: write d1/d4/d1d/d36/d3c/d64/f7b [3663534,109778] 0 2024-05-12T04:01:20.262 INFO:tasks.workunit.client.1.smithi033.stdout:8/576: dread - d1/d16/fa6 zero size 2024-05-12T04:01:20.263 INFO:tasks.workunit.client.1.smithi033.stdout:8/577: readlink d1/d16/d27/l5f 0 2024-05-12T04:01:20.263 INFO:tasks.workunit.client.1.smithi033.stdout:6/657: read d4/d1a/f91 [510100,53585] 0 2024-05-12T04:01:20.265 INFO:tasks.workunit.client.1.smithi033.stdout:6/658: fdatasync d4/dad/fa7 0 2024-05-12T04:01:20.266 INFO:tasks.workunit.client.1.smithi033.stdout:3/605: creat db/d17/d25/d32/fd5 x:0 0 0 2024-05-12T04:01:20.268 INFO:tasks.workunit.client.1.smithi033.stdout:3/606: fsync db/d17/d25/f40 0 2024-05-12T04:01:20.272 INFO:tasks.workunit.client.1.smithi033.stdout:0/668: readlink d6/dc/l25 0 2024-05-12T04:01:20.273 INFO:tasks.workunit.client.1.smithi033.stdout:2/616: symlink dc/d17/d27/d68/lc6 0 2024-05-12T04:01:20.275 INFO:tasks.workunit.client.1.smithi033.stdout:6/659: dwrite d4/d5/f73 [0,4194304] 0 2024-05-12T04:01:20.278 INFO:tasks.workunit.client.1.smithi033.stdout:6/660: readlink d4/d1a/d4e/d4f/l9b 0 2024-05-12T04:01:20.283 INFO:tasks.workunit.client.1.smithi033.stdout:9/866: dread d2/d4/d29/d3c/d52/f86 [0,4194304] 0 2024-05-12T04:01:20.284 INFO:tasks.workunit.client.1.smithi033.stdout:3/607: dwrite db/d59/dcc/fce [0,4194304] 0 2024-05-12T04:01:20.285 INFO:tasks.workunit.client.1.smithi033.stdout:9/867: chown d2/d4/d2b/d6d/d2f/d5f 792 1 2024-05-12T04:01:20.293 INFO:tasks.workunit.client.1.smithi033.stdout:5/536: creat d3/fa5 x:0 0 0 2024-05-12T04:01:20.294 INFO:tasks.workunit.client.0.smithi032.stdout:1/479: creat d1/d6a/d72/f9e x:0 0 0 2024-05-12T04:01:20.296 INFO:tasks.workunit.client.1.smithi033.stdout:7/833: creat de/df/d17/d1b/d110/f118 x:0 0 0 2024-05-12T04:01:20.296 INFO:tasks.workunit.client.1.smithi033.stdout:0/669: dread d6/d11/d57/f55 [0,4194304] 0 2024-05-12T04:01:20.303 INFO:tasks.workunit.client.1.smithi033.stdout:3/608: dread db/d17/d25/d32/f46 [0,4194304] 0 2024-05-12T04:01:20.304 INFO:tasks.workunit.client.0.smithi032.stdout:5/556: symlink d4/d7/lc0 0 2024-05-12T04:01:20.307 INFO:tasks.workunit.client.1.smithi033.stdout:8/578: mkdir d1/d4/db2 0 2024-05-12T04:01:20.309 INFO:tasks.workunit.client.1.smithi033.stdout:4/642: dwrite dd/d7a/d87/d7e/da7/d66/d42/f82 [0,4194304] 0 2024-05-12T04:01:20.310 INFO:tasks.workunit.client.0.smithi032.stdout:4/395: link d0/d3/d24/f2d d0/d39/f6a 0 2024-05-12T04:01:20.310 INFO:tasks.workunit.client.0.smithi032.stdout:4/396: chown d0/d3/d12/l50 1470 1 2024-05-12T04:01:20.314 INFO:tasks.workunit.client.1.smithi033.stdout:9/868: chown d2/dd/l91 17286251 1 2024-05-12T04:01:20.317 INFO:tasks.workunit.client.0.smithi032.stdout:9/376: rename d0/d3/d74/d51/f54 to d0/d3/d74/f78 0 2024-05-12T04:01:20.319 INFO:tasks.workunit.client.1.smithi033.stdout:1/563: rename d11/d17/d27/d76/f9c to d11/d17/d27/d36/fc0 0 2024-05-12T04:01:20.321 INFO:tasks.workunit.client.0.smithi032.stdout:4/397: rmdir d0/d3/df 39 2024-05-12T04:01:20.323 INFO:tasks.workunit.client.0.smithi032.stdout:4/398: chown d0/d39/l59 13 1 2024-05-12T04:01:20.325 INFO:tasks.workunit.client.0.smithi032.stdout:9/377: dwrite d0/d3/d6/d7/d2a/d46/d13/f56 [0,4194304] 0 2024-05-12T04:01:20.330 INFO:tasks.workunit.client.0.smithi032.stdout:4/399: mkdir d0/d3/d12/d1d/d6b 0 2024-05-12T04:01:20.332 INFO:tasks.workunit.client.1.smithi033.stdout:0/670: mknod d6/d11/d98/ce1 0 2024-05-12T04:01:20.334 INFO:tasks.workunit.client.1.smithi033.stdout:3/609: symlink db/d17/d25/d32/d3c/d3d/ld6 0 2024-05-12T04:01:20.335 INFO:tasks.workunit.client.1.smithi033.stdout:3/610: dread - db/d17/fc9 zero size 2024-05-12T04:01:20.337 INFO:tasks.workunit.client.1.smithi033.stdout:8/579: fsync d1/d4/d1d/f58 0 2024-05-12T04:01:20.338 INFO:tasks.workunit.client.1.smithi033.stdout:4/643: symlink dd/d69/dc4/lc9 0 2024-05-12T04:01:20.339 INFO:tasks.workunit.client.1.smithi033.stdout:8/580: chown d1/d16/d27/l5f 5854 1 2024-05-12T04:01:20.340 INFO:tasks.workunit.client.1.smithi033.stdout:4/644: read dd/d11/d5b/fb5 [790291,117223] 0 2024-05-12T04:01:20.341 INFO:tasks.workunit.client.1.smithi033.stdout:6/661: dwrite d4/f78 [4194304,4194304] 0 2024-05-12T04:01:20.342 INFO:tasks.workunit.client.1.smithi033.stdout:7/834: rename de/df/c78 to de/d2f/d70/c119 0 2024-05-12T04:01:20.343 INFO:tasks.workunit.client.1.smithi033.stdout:2/617: truncate dc/d13/d1c/f4f 3322560 0 2024-05-12T04:01:20.345 INFO:tasks.workunit.client.1.smithi033.stdout:5/537: truncate d3/d15/f30 3588437 0 2024-05-12T04:01:20.346 INFO:tasks.workunit.client.1.smithi033.stdout:7/835: truncate de/df/d60/d75/f112 826926 0 2024-05-12T04:01:20.347 INFO:tasks.workunit.client.1.smithi033.stdout:0/671: symlink d6/dc/d58/d91/le2 0 2024-05-12T04:01:20.348 INFO:tasks.workunit.client.1.smithi033.stdout:9/869: dread d2/d4/f1f [0,4194304] 0 2024-05-12T04:01:20.353 INFO:tasks.workunit.client.1.smithi033.stdout:1/564: link d11/d17/f6d d11/d17/d27/d76/d93/fc1 0 2024-05-12T04:01:20.354 INFO:tasks.workunit.client.1.smithi033.stdout:1/565: chown d11/d17/d27/d76/d93 2991247 1 2024-05-12T04:01:20.357 INFO:tasks.workunit.client.1.smithi033.stdout:7/836: unlink de/df/d17/d1e/c43 0 2024-05-12T04:01:20.358 INFO:tasks.workunit.client.1.smithi033.stdout:0/672: symlink d6/dc/d6e/le3 0 2024-05-12T04:01:20.358 INFO:tasks.workunit.client.1.smithi033.stdout:7/837: write de/d2b/d6c/de1/f10d [1020676,5297] 0 2024-05-12T04:01:20.360 INFO:tasks.workunit.client.1.smithi033.stdout:4/645: link dd/d11/f2f dd/d11/d75/fca 0 2024-05-12T04:01:20.361 INFO:tasks.workunit.client.1.smithi033.stdout:1/566: creat d11/d17/d27/da6/fc2 x:0 0 0 2024-05-12T04:01:20.363 INFO:tasks.workunit.client.1.smithi033.stdout:9/870: symlink d2/dd/l136 0 2024-05-12T04:01:20.364 INFO:tasks.workunit.client.1.smithi033.stdout:3/611: getdents db/d17/d3a/d8a 0 2024-05-12T04:01:20.370 INFO:tasks.workunit.client.1.smithi033.stdout:4/646: mknod dd/d7a/d87/d7e/da7/d66/d42/d9c/ccb 0 2024-05-12T04:01:20.371 INFO:tasks.workunit.client.1.smithi033.stdout:6/662: link d4/d1a/d4e/d4f/d94/la2 d4/d1a/d67/le1 0 2024-05-12T04:01:20.375 INFO:tasks.workunit.client.1.smithi033.stdout:8/581: getdents d1/d4/d1d/d36 0 2024-05-12T04:01:20.376 INFO:tasks.workunit.client.1.smithi033.stdout:6/663: dread d4/f16 [4194304,4194304] 0 2024-05-12T04:01:20.377 INFO:tasks.workunit.client.1.smithi033.stdout:3/612: rename db/d17/d3a/d8a/d9d to db/d17/d3e/dd7 0 2024-05-12T04:01:20.378 INFO:tasks.workunit.client.1.smithi033.stdout:5/538: write d3/d12/f63 [865534,96235] 0 2024-05-12T04:01:20.379 INFO:tasks.workunit.client.1.smithi033.stdout:7/838: unlink de/df/d17/d1b/d46/lcb 0 2024-05-12T04:01:20.381 INFO:tasks.workunit.client.1.smithi033.stdout:4/647: rmdir dd 39 2024-05-12T04:01:20.381 INFO:tasks.workunit.client.1.smithi033.stdout:0/673: dread d6/dc/d58/d6a/d80/f84 [0,4194304] 0 2024-05-12T04:01:20.382 INFO:tasks.workunit.client.1.smithi033.stdout:6/664: write d4/d1a/d4e/d4f/d94/fb9 [3143618,4648] 0 2024-05-12T04:01:20.384 INFO:tasks.workunit.client.1.smithi033.stdout:1/567: readlink d11/d17/d27/d36/d46/d4c/la2 0 2024-05-12T04:01:20.385 INFO:tasks.workunit.client.1.smithi033.stdout:2/618: dwrite dc/d17/d27/f94 [0,4194304] 0 2024-05-12T04:01:20.386 INFO:tasks.workunit.client.0.smithi032.stdout:5/557: write f1 [4001220,79142] 0 2024-05-12T04:01:20.387 INFO:tasks.workunit.client.1.smithi033.stdout:6/665: chown d4/d1a/d4e/c9d 0 1 2024-05-12T04:01:20.389 INFO:tasks.workunit.client.1.smithi033.stdout:1/568: read - d11/d17/d27/d76/d93/fb0 zero size 2024-05-12T04:01:20.391 INFO:tasks.workunit.client.0.smithi032.stdout:9/378: dwrite d0/d3/d6/d58/f66 [0,4194304] 0 2024-05-12T04:01:20.391 INFO:tasks.workunit.client.0.smithi032.stdout:3/488: dwrite d15/f4e [0,4194304] 0 2024-05-12T04:01:20.392 INFO:tasks.workunit.client.1.smithi033.stdout:3/613: dwrite db/d17/d3a/fb5 [4194304,4194304] 0 2024-05-12T04:01:20.396 INFO:tasks.workunit.client.0.smithi032.stdout:4/400: symlink d0/d3/d12/d1d/d63/l6c 0 2024-05-12T04:01:20.404 INFO:tasks.workunit.client.0.smithi032.stdout:9/379: getdents d0/d3/d6/d7/d2a/d46/d13 0 2024-05-12T04:01:20.406 INFO:tasks.workunit.client.1.smithi033.stdout:9/871: dwrite d2/d4/d29/d32/db7/fc6 [0,4194304] 0 2024-05-12T04:01:20.408 INFO:tasks.workunit.client.1.smithi033.stdout:0/674: rename d6/d11/d98/fda to d6/dc/d58/d6a/d80/db8/fe4 0 2024-05-12T04:01:20.410 INFO:tasks.workunit.client.0.smithi032.stdout:5/558: creat d4/d7/fc1 x:0 0 0 2024-05-12T04:01:20.411 INFO:tasks.workunit.client.0.smithi032.stdout:5/559: stat d4/d7/d14/f1d 0 2024-05-12T04:01:20.411 INFO:tasks.workunit.client.1.smithi033.stdout:5/539: creat d3/d94/d54/d57/d73/fa6 x:0 0 0 2024-05-12T04:01:20.412 INFO:tasks.workunit.client.0.smithi032.stdout:9/380: mkdir d0/d3/d6/d79 0 2024-05-12T04:01:20.414 INFO:tasks.workunit.client.0.smithi032.stdout:4/401: creat d0/d3/d12/d1d/d6b/f6d x:0 0 0 2024-05-12T04:01:20.416 INFO:tasks.workunit.client.1.smithi033.stdout:2/619: symlink dc/d13/d39/d8f/lc7 0 2024-05-12T04:01:20.417 INFO:tasks.workunit.client.0.smithi032.stdout:4/402: write d0/d3/d12/f5b [1614050,59640] 0 2024-05-12T04:01:20.419 INFO:tasks.workunit.client.1.smithi033.stdout:8/582: link d1/c70 d1/d4/d91/cb3 0 2024-05-12T04:01:20.420 INFO:tasks.workunit.client.1.smithi033.stdout:3/614: dwrite db/d17/d25/d32/f46 [0,4194304] 0 2024-05-12T04:01:20.422 INFO:tasks.workunit.client.0.smithi032.stdout:4/403: dread d0/d3/d24/f42 [0,4194304] 0 2024-05-12T04:01:20.425 INFO:tasks.workunit.client.1.smithi033.stdout:9/872: mkdir d2/d4/d2b/d6d/d2f/d5f/d84/d137 0 2024-05-12T04:01:20.426 INFO:tasks.workunit.client.0.smithi032.stdout:5/560: link d4/d7/d14/d26/d33/d4a/d38/d79/l8d d4/d7/d14/da9/lc2 0 2024-05-12T04:01:20.426 INFO:tasks.workunit.client.1.smithi033.stdout:5/540: dread d3/d12/f1f [0,4194304] 0 2024-05-12T04:01:20.427 INFO:tasks.workunit.client.1.smithi033.stdout:2/620: dwrite dc/d17/d32/d84/fa0 [0,4194304] 0 2024-05-12T04:01:20.428 INFO:tasks.workunit.client.0.smithi032.stdout:4/404: truncate d0/d39/f4b 858194 0 2024-05-12T04:01:20.433 INFO:tasks.workunit.client.0.smithi032.stdout:0/514: sync 2024-05-12T04:01:20.433 INFO:tasks.workunit.client.0.smithi032.stdout:6/448: sync 2024-05-12T04:01:20.433 INFO:tasks.workunit.client.0.smithi032.stdout:8/476: sync 2024-05-12T04:01:20.433 INFO:tasks.workunit.client.0.smithi032.stdout:7/436: sync 2024-05-12T04:01:20.435 INFO:tasks.workunit.client.0.smithi032.stdout:4/405: readlink d0/l14 0 2024-05-12T04:01:20.437 INFO:tasks.workunit.client.0.smithi032.stdout:7/437: chown dd/d1b/d43/d5e/f4d 0 1 2024-05-12T04:01:20.437 INFO:tasks.workunit.client.0.smithi032.stdout:4/406: chown d0/f65 0 1 2024-05-12T04:01:20.437 INFO:tasks.workunit.client.0.smithi032.stdout:0/515: read - df/d22/d58/f6a zero size 2024-05-12T04:01:20.438 INFO:tasks.workunit.client.1.smithi033.stdout:3/615: symlink db/d17/d3a/d93/ld8 0 2024-05-12T04:01:20.441 INFO:tasks.workunit.client.0.smithi032.stdout:5/561: dread d4/d7/d14/d26/d33/f61 [0,4194304] 0 2024-05-12T04:01:20.441 INFO:tasks.workunit.client.0.smithi032.stdout:6/449: rmdir d7/d9/d5c 39 2024-05-12T04:01:20.442 INFO:tasks.workunit.client.1.smithi033.stdout:9/873: dread d2/d4/f9e [0,4194304] 0 2024-05-12T04:01:20.447 INFO:tasks.workunit.client.0.smithi032.stdout:6/450: write d7/d9/d48/d67/f85 [205589,72527] 0 2024-05-12T04:01:20.447 INFO:tasks.workunit.client.1.smithi033.stdout:1/569: write d11/d1c/d43/f4d [523252,65180] 0 2024-05-12T04:01:20.448 INFO:tasks.workunit.client.1.smithi033.stdout:2/621: mknod dc/d13/d1c/cc8 0 2024-05-12T04:01:20.448 INFO:tasks.workunit.client.1.smithi033.stdout:7/839: dwrite de/fb9 [4194304,4194304] 0 2024-05-12T04:01:20.448 INFO:tasks.workunit.client.0.smithi032.stdout:6/451: read d7/d9/d48/d67/d1a/d20/f3d [1097703,115228] 0 2024-05-12T04:01:20.448 INFO:tasks.workunit.client.0.smithi032.stdout:6/452: write d7/de/d2e/f8e [4735058,12659] 0 2024-05-12T04:01:20.449 INFO:tasks.workunit.client.1.smithi033.stdout:2/622: write dc/d13/d39/d8f/d9e/fb4 [1002154,8916] 0 2024-05-12T04:01:20.449 INFO:tasks.workunit.client.0.smithi032.stdout:0/516: readlink df/d10/d3d/d6c/l86 0 2024-05-12T04:01:20.449 INFO:tasks.workunit.client.0.smithi032.stdout:0/517: stat f5 0 2024-05-12T04:01:20.450 INFO:tasks.workunit.client.1.smithi033.stdout:7/840: fdatasync de/d2b/d6c/f5a 0 2024-05-12T04:01:20.451 INFO:tasks.workunit.client.1.smithi033.stdout:7/841: dread - de/d2f/fe9 zero size 2024-05-12T04:01:20.451 INFO:tasks.workunit.client.0.smithi032.stdout:9/381: dread d0/d3/d6/d7/d2a/f3b [0,4194304] 0 2024-05-12T04:01:20.452 INFO:tasks.workunit.client.0.smithi032.stdout:9/382: fdatasync d0/f28 0 2024-05-12T04:01:20.452 INFO:tasks.workunit.client.1.smithi033.stdout:7/842: write de/d20/f21 [756431,74977] 0 2024-05-12T04:01:20.453 INFO:tasks.workunit.client.1.smithi033.stdout:2/623: fsync dc/f31 0 2024-05-12T04:01:20.453 INFO:tasks.workunit.client.0.smithi032.stdout:9/383: write d0/f6b [163881,60899] 0 2024-05-12T04:01:20.454 INFO:tasks.workunit.client.0.smithi032.stdout:7/438: rename dd/d1b/f49 to dd/d1b/d1e/d45/d64/f91 0 2024-05-12T04:01:20.456 INFO:tasks.workunit.client.1.smithi033.stdout:4/648: dread dd/d11/d5b/d84/f9e [4194304,4194304] 0 2024-05-12T04:01:20.457 INFO:tasks.workunit.client.0.smithi032.stdout:9/384: stat d0/d3/d74/f6c 0 2024-05-12T04:01:20.457 INFO:tasks.workunit.client.0.smithi032.stdout:4/407: truncate d0/d3/f49 112286 0 2024-05-12T04:01:20.457 INFO:tasks.workunit.client.0.smithi032.stdout:9/385: fsync d0/d3/d6/d7/da/f1c 0 2024-05-12T04:01:20.464 INFO:tasks.workunit.client.0.smithi032.stdout:8/477: mknod d0/d9/d1b/d33/d73/ca8 0 2024-05-12T04:01:20.464 INFO:tasks.workunit.client.1.smithi033.stdout:8/583: write d1/d4/d1d/d36/d3c/da7/f65 [4731505,27806] 0 2024-05-12T04:01:20.481 INFO:tasks.workunit.client.0.smithi032.stdout:4/408: mkdir d0/d3/d31/d6e 0 2024-05-12T04:01:20.482 INFO:tasks.workunit.client.0.smithi032.stdout:4/409: stat d0/d3/d12 0 2024-05-12T04:01:20.484 INFO:tasks.workunit.client.0.smithi032.stdout:9/386: creat d0/d3/d6/d7/d27/f7a x:0 0 0 2024-05-12T04:01:20.488 INFO:tasks.workunit.client.0.smithi032.stdout:8/478: truncate d0/d9/d13/d21/d25/f52 1255983 0 2024-05-12T04:01:20.492 INFO:tasks.workunit.client.0.smithi032.stdout:6/453: dwrite d7/d9/f82 [0,4194304] 0 2024-05-12T04:01:20.493 INFO:tasks.workunit.client.0.smithi032.stdout:0/518: dwrite df/d10/d3d/d4a/d84/f8c [0,4194304] 0 2024-05-12T04:01:20.494 INFO:tasks.workunit.client.0.smithi032.stdout:6/454: chown d7/d9/d48/d67/d1a/l96 769726669 1 2024-05-12T04:01:20.501 INFO:tasks.workunit.client.0.smithi032.stdout:9/387: mknod d0/d3/d6/d7/c7b 0 2024-05-12T04:01:20.505 INFO:tasks.workunit.client.0.smithi032.stdout:7/439: dwrite dd/d1b/f2d [0,4194304] 0 2024-05-12T04:01:20.506 INFO:tasks.workunit.client.0.smithi032.stdout:0/519: dwrite df/d10/d3d/f56 [0,4194304] 0 2024-05-12T04:01:20.508 INFO:tasks.workunit.client.0.smithi032.stdout:0/520: readlink df/d28/la1 0 2024-05-12T04:01:20.513 INFO:tasks.workunit.client.1.smithi033.stdout:6/666: sync 2024-05-12T04:01:20.513 INFO:tasks.workunit.client.1.smithi033.stdout:6/667: readlink d4/d1a/l1b 0 2024-05-12T04:01:20.515 INFO:tasks.workunit.client.0.smithi032.stdout:6/455: creat d7/d9/d60/f9a x:0 0 0 2024-05-12T04:01:20.516 INFO:tasks.workunit.client.0.smithi032.stdout:5/562: rename d4/d7/d14/d26/d33/d72/c81 to d4/d7/d14/d26/d33/d4a/cc3 0 2024-05-12T04:01:20.518 INFO:tasks.workunit.client.0.smithi032.stdout:7/440: dwrite dd/d1b/d1e/d45/d64/f6d [0,4194304] 0 2024-05-12T04:01:20.520 INFO:tasks.workunit.client.0.smithi032.stdout:7/441: write dd/f13 [5023465,95458] 0 2024-05-12T04:01:20.522 INFO:tasks.workunit.client.0.smithi032.stdout:9/388: creat d0/d3/d6/d7/d2a/d46/d13/f7c x:0 0 0 2024-05-12T04:01:20.522 INFO:tasks.workunit.client.0.smithi032.stdout:7/442: write dd/d1b/d43/d74/f84 [473636,119196] 0 2024-05-12T04:01:20.523 INFO:tasks.workunit.client.0.smithi032.stdout:9/389: dread - d0/d3/f76 zero size 2024-05-12T04:01:20.524 INFO:tasks.workunit.client.0.smithi032.stdout:0/521: rmdir df/d10/d48/d70 39 2024-05-12T04:01:20.525 INFO:tasks.workunit.client.0.smithi032.stdout:0/522: chown df/d10/d3d/d4a/d9a/d9d 11 1 2024-05-12T04:01:20.531 INFO:tasks.workunit.client.0.smithi032.stdout:4/410: creat d0/f6f x:0 0 0 2024-05-12T04:01:20.536 INFO:tasks.workunit.client.0.smithi032.stdout:7/443: write dd/d1b/d1e/f38 [4708763,98743] 0 2024-05-12T04:01:20.538 INFO:tasks.workunit.client.0.smithi032.stdout:9/390: creat d0/d3/d6/d7/f7d x:0 0 0 2024-05-12T04:01:20.540 INFO:tasks.workunit.client.1.smithi033.stdout:2/624: readlink dc/l11 0 2024-05-12T04:01:20.542 INFO:tasks.workunit.client.0.smithi032.stdout:8/479: read d0/d9/d13/d21/d25/f52 [572644,39365] 0 2024-05-12T04:01:20.543 INFO:tasks.workunit.client.0.smithi032.stdout:8/480: fdatasync d0/f68 0 2024-05-12T04:01:20.543 INFO:tasks.workunit.client.0.smithi032.stdout:9/391: dread d0/f6b [0,4194304] 0 2024-05-12T04:01:20.544 INFO:tasks.workunit.client.1.smithi033.stdout:8/584: mknod d1/d16/d29/cb4 0 2024-05-12T04:01:20.545 INFO:tasks.workunit.client.1.smithi033.stdout:8/585: write d1/d16/d27/d4d/d51/f4b [977011,19658] 0 2024-05-12T04:01:20.547 INFO:tasks.workunit.client.0.smithi032.stdout:4/411: creat d0/d3/d12/d1d/d63/f70 x:0 0 0 2024-05-12T04:01:20.548 INFO:tasks.workunit.client.1.smithi033.stdout:0/675: link d6/dc/d58/d6a/d80/c97 d6/db/d2c/d73/daf/da4/ce5 0 2024-05-12T04:01:20.550 INFO:tasks.workunit.client.0.smithi032.stdout:5/563: creat d4/d7/d14/d39/d1b/da4/fc4 x:0 0 0 2024-05-12T04:01:20.552 INFO:tasks.workunit.client.1.smithi033.stdout:8/586: dwrite d1/d4/d1d/f78 [0,4194304] 0 2024-05-12T04:01:20.558 INFO:tasks.workunit.client.1.smithi033.stdout:7/843: creat de/df/d7a/dc0/f11a x:0 0 0 2024-05-12T04:01:20.558 INFO:tasks.workunit.client.0.smithi032.stdout:6/456: write d7/f59 [826766,33330] 0 2024-05-12T04:01:20.558 INFO:tasks.workunit.client.0.smithi032.stdout:7/444: dwrite dd/d1b/d43/d5e/d37/f4f [4194304,4194304] 0 2024-05-12T04:01:20.560 INFO:tasks.workunit.client.0.smithi032.stdout:7/445: stat dd/d1b/d43/d5e/d37/f41 0 2024-05-12T04:01:20.560 INFO:tasks.workunit.client.1.smithi033.stdout:2/625: symlink dc/d17/d27/d68/lc9 0 2024-05-12T04:01:20.565 INFO:tasks.workunit.client.1.smithi033.stdout:9/874: dwrite d2/d4/f30 [0,4194304] 0 2024-05-12T04:01:20.565 INFO:tasks.workunit.client.0.smithi032.stdout:9/392: creat d0/d3/d6/d7/f7e x:0 0 0 2024-05-12T04:01:20.565 INFO:tasks.workunit.client.0.smithi032.stdout:2/479: sync 2024-05-12T04:01:20.566 INFO:tasks.workunit.client.0.smithi032.stdout:2/480: readlink d1/ld 0 2024-05-12T04:01:20.569 INFO:tasks.workunit.client.0.smithi032.stdout:8/481: write d0/d9/d1b/d66/d75/f7c [830625,116610] 0 2024-05-12T04:01:20.570 INFO:tasks.workunit.client.0.smithi032.stdout:2/481: chown d1/d34/d96/d41/d39/l42 1197 1 2024-05-12T04:01:20.572 INFO:tasks.workunit.client.0.smithi032.stdout:6/457: dwrite d7/d9/f4a [0,4194304] 0 2024-05-12T04:01:20.574 INFO:tasks.workunit.client.0.smithi032.stdout:9/393: dread d0/d3/d6/d7/d2a/d46/d13/f56 [0,4194304] 0 2024-05-12T04:01:20.579 INFO:tasks.workunit.client.0.smithi032.stdout:0/523: write fb [1227240,119090] 0 2024-05-12T04:01:20.584 INFO:tasks.workunit.client.0.smithi032.stdout:7/446: creat dd/d1b/d43/d5e/f92 x:0 0 0 2024-05-12T04:01:20.588 INFO:tasks.workunit.client.1.smithi033.stdout:5/541: link d3/d12/d25/c53 d3/d12/d29/d2b/d8e/ca7 0 2024-05-12T04:01:20.589 INFO:tasks.workunit.client.0.smithi032.stdout:8/482: unlink d0/f2f 0 2024-05-12T04:01:20.590 INFO:tasks.workunit.client.1.smithi033.stdout:2/626: dwrite dc/f31 [0,4194304] 0 2024-05-12T04:01:20.591 INFO:tasks.workunit.client.1.smithi033.stdout:3/616: rename db/d17/f36 to db/fd9 0 2024-05-12T04:01:20.595 INFO:tasks.workunit.client.0.smithi032.stdout:6/458: truncate d7/d33/f7a 1625583 0 2024-05-12T04:01:20.596 INFO:tasks.workunit.client.0.smithi032.stdout:9/394: creat d0/d3/d74/d51/f7f x:0 0 0 2024-05-12T04:01:20.597 INFO:tasks.workunit.client.1.smithi033.stdout:1/570: dwrite d11/d17/d27/d36/d46/f74 [0,4194304] 0 2024-05-12T04:01:20.600 INFO:tasks.workunit.client.0.smithi032.stdout:7/447: symlink dd/d1b/d1e/d45/d64/l93 0 2024-05-12T04:01:20.602 INFO:tasks.workunit.client.1.smithi033.stdout:9/875: dwrite d2/d4/d2b/d6d/f69 [4194304,4194304] 0 2024-05-12T04:01:20.602 INFO:tasks.workunit.client.1.smithi033.stdout:8/587: creat d1/d16/d29/db1/fb5 x:0 0 0 2024-05-12T04:01:20.605 INFO:tasks.workunit.client.0.smithi032.stdout:4/412: rmdir d0/d3/d31/d6e 0 2024-05-12T04:01:20.606 INFO:tasks.workunit.client.0.smithi032.stdout:4/413: write d0/d3/d12/d1d/f4c [4753276,12593] 0 2024-05-12T04:01:20.607 INFO:tasks.workunit.client.1.smithi033.stdout:4/649: rename c6 to dd/d7a/d87/d7e/da7/d66/d42/d9c/ccc 0 2024-05-12T04:01:20.608 INFO:tasks.workunit.client.1.smithi033.stdout:8/588: read d1/d4/d1d/d36/f50 [1185429,115580] 0 2024-05-12T04:01:20.610 INFO:tasks.workunit.client.1.smithi033.stdout:2/627: chown dc/d13/d1f/l2d 55 1 2024-05-12T04:01:20.610 INFO:tasks.workunit.client.0.smithi032.stdout:5/564: link d4/d7/d14/d39/d2b/l75 d4/d7/d14/d26/d33/d4a/d38/lc5 0 2024-05-12T04:01:20.613 INFO:tasks.workunit.client.1.smithi033.stdout:3/617: unlink db/d17/d25/d32/d53/l85 0 2024-05-12T04:01:20.614 INFO:tasks.workunit.client.0.smithi032.stdout:2/482: creat d1/d34/d95/fa1 x:0 0 0 2024-05-12T04:01:20.616 INFO:tasks.workunit.client.1.smithi033.stdout:4/650: dwrite dd/d69/f97 [0,4194304] 0 2024-05-12T04:01:20.618 INFO:tasks.workunit.client.1.smithi033.stdout:1/571: rmdir d11 39 2024-05-12T04:01:20.619 INFO:tasks.workunit.client.0.smithi032.stdout:6/459: creat d7/d97/f9b x:0 0 0 2024-05-12T04:01:20.620 INFO:tasks.workunit.client.0.smithi032.stdout:9/395: fsync d0/d3/d6/d7/d2a/d46/d13/f56 0 2024-05-12T04:01:20.620 INFO:tasks.workunit.client.1.smithi033.stdout:8/589: dwrite d1/d16/f96 [0,4194304] 0 2024-05-12T04:01:20.622 INFO:tasks.workunit.client.1.smithi033.stdout:8/590: fdatasync d1/d16/d29/f9a 0 2024-05-12T04:01:20.622 INFO:tasks.workunit.client.1.smithi033.stdout:9/876: rmdir d2/d4/d29/d3c/dda 39 2024-05-12T04:01:20.624 INFO:tasks.workunit.client.0.smithi032.stdout:4/414: readlink d0/d3/df/l3f 0 2024-05-12T04:01:20.626 INFO:tasks.workunit.client.1.smithi033.stdout:0/676: truncate d6/db/f1a 3104286 0 2024-05-12T04:01:20.627 INFO:tasks.workunit.client.1.smithi033.stdout:0/677: dread - d6/dc/d58/fbd zero size 2024-05-12T04:01:20.632 INFO:tasks.workunit.client.1.smithi033.stdout:2/628: symlink dc/d17/d27/d2f/d49/d59/lca 0 2024-05-12T04:01:20.632 INFO:tasks.workunit.client.1.smithi033.stdout:0/678: dread d6/dc/d58/d6a/f89 [0,4194304] 0 2024-05-12T04:01:20.637 INFO:tasks.workunit.client.1.smithi033.stdout:1/572: chown d11/d1c/d59/faa 30 1 2024-05-12T04:01:20.640 INFO:tasks.workunit.client.1.smithi033.stdout:5/542: dwrite d3/d12/f1f [0,4194304] 0 2024-05-12T04:01:20.640 INFO:tasks.workunit.client.0.smithi032.stdout:0/524: dwrite df/d28/f59 [0,4194304] 0 2024-05-12T04:01:20.641 INFO:tasks.workunit.client.1.smithi033.stdout:3/618: rename db/d17/d25/d32/d53/fc7 to db/d17/d3a/d93/dc8/fda 0 2024-05-12T04:01:20.642 INFO:tasks.workunit.client.1.smithi033.stdout:3/619: fdatasync db/d17/d3a/fb5 0 2024-05-12T04:01:20.643 INFO:tasks.workunit.client.0.smithi032.stdout:9/396: mknod d0/d3/d6/d58/c80 0 2024-05-12T04:01:20.644 INFO:tasks.workunit.client.1.smithi033.stdout:6/668: sync 2024-05-12T04:01:20.645 INFO:tasks.workunit.client.1.smithi033.stdout:6/669: dread - d4/d43/d57/dc4/ddf/fc0 zero size 2024-05-12T04:01:20.650 INFO:tasks.workunit.client.0.smithi032.stdout:4/415: rmdir d0/d3/d31 39 2024-05-12T04:01:20.653 INFO:tasks.workunit.client.0.smithi032.stdout:2/483: dread d1/d34/d3e/f7e [0,4194304] 0 2024-05-12T04:01:20.655 INFO:tasks.workunit.client.1.smithi033.stdout:0/679: dread d6/db/d2c/d73/f87 [4194304,4194304] 0 2024-05-12T04:01:20.656 INFO:tasks.workunit.client.0.smithi032.stdout:6/460: dread d7/d9/d48/d67/f6e [0,4194304] 0 2024-05-12T04:01:20.656 INFO:tasks.workunit.client.1.smithi033.stdout:0/680: chown d6/db/d2c/d73/daf/da4/lad 51859 1 2024-05-12T04:01:20.658 INFO:tasks.workunit.client.1.smithi033.stdout:9/877: creat d2/dd/d39/d72/f138 x:0 0 0 2024-05-12T04:01:20.661 INFO:tasks.workunit.client.1.smithi033.stdout:9/878: chown d2/dd/d39/f59 496 1 2024-05-12T04:01:20.662 INFO:tasks.workunit.client.0.smithi032.stdout:8/483: link d0/l85 d0/d9/d1b/d27/d59/la9 0 2024-05-12T04:01:20.663 INFO:tasks.workunit.client.0.smithi032.stdout:8/484: chown d0/d9/f1c 500394 1 2024-05-12T04:01:20.668 INFO:tasks.workunit.client.1.smithi033.stdout:1/573: creat d11/d17/d9e/fc3 x:0 0 0 2024-05-12T04:01:20.669 INFO:tasks.workunit.client.0.smithi032.stdout:0/525: symlink df/d30/d36/lb6 0 2024-05-12T04:01:20.670 INFO:tasks.workunit.client.1.smithi033.stdout:3/620: dread db/d17/d25/d32/d3c/d44/f55 [0,4194304] 0 2024-05-12T04:01:20.671 INFO:tasks.workunit.client.0.smithi032.stdout:0/526: chown df/d10/d3d/d4a/d84/f8c 0 1 2024-05-12T04:01:20.674 INFO:tasks.workunit.client.0.smithi032.stdout:9/397: creat d0/d3/d4b/f81 x:0 0 0 2024-05-12T04:01:20.676 INFO:tasks.workunit.client.1.smithi033.stdout:3/621: dwrite db/d17/d3a/fb5 [0,4194304] 0 2024-05-12T04:01:20.678 INFO:tasks.workunit.client.0.smithi032.stdout:0/527: dwrite df/d10/d48/f52 [0,4194304] 0 2024-05-12T04:01:20.679 INFO:tasks.workunit.client.0.smithi032.stdout:4/416: unlink d0/d3/df/l3f 0 2024-05-12T04:01:20.679 INFO:tasks.workunit.client.0.smithi032.stdout:2/484: dread d1/d13/f1f [0,4194304] 0 2024-05-12T04:01:20.688 INFO:tasks.workunit.client.1.smithi033.stdout:8/591: truncate d1/d4/d1d/d36/d3c/f55 712830 0 2024-05-12T04:01:20.689 INFO:tasks.workunit.client.1.smithi033.stdout:0/681: readlink d6/dc/lb2 0 2024-05-12T04:01:20.692 INFO:tasks.workunit.client.0.smithi032.stdout:5/565: rename d4/d7/d14/da9/dbb to d4/d7/d14/d26/dc6 0 2024-05-12T04:01:20.696 INFO:tasks.workunit.client.1.smithi033.stdout:2/629: symlink dc/d13/lcb 0 2024-05-12T04:01:20.696 INFO:tasks.workunit.client.1.smithi033.stdout:2/630: readlink dc/d3b/l5b 0 2024-05-12T04:01:20.697 INFO:tasks.workunit.client.1.smithi033.stdout:1/574: write d11/d17/d27/d36/fc0 [3658764,127156] 0 2024-05-12T04:01:20.697 INFO:tasks.workunit.client.1.smithi033.stdout:7/844: sync 2024-05-12T04:01:20.698 INFO:tasks.workunit.client.1.smithi033.stdout:1/575: fdatasync d11/d17/f18 0 2024-05-12T04:01:20.699 INFO:tasks.workunit.client.1.smithi033.stdout:1/576: write d11/d17/f3e [612580,54277] 0 2024-05-12T04:01:20.703 INFO:tasks.workunit.client.1.smithi033.stdout:9/879: write d2/d4/d2b/d6d/d7a/f90 [1953684,3895] 0 2024-05-12T04:01:20.704 INFO:tasks.workunit.client.1.smithi033.stdout:9/880: read - d2/d4/d2b/d78/f114 zero size 2024-05-12T04:01:20.705 INFO:tasks.workunit.client.1.smithi033.stdout:5/543: unlink d3/d12/l8d 0 2024-05-12T04:01:20.706 INFO:tasks.workunit.client.1.smithi033.stdout:5/544: chown d3/d15/d16/d45/d62/d7f/d84/l86 7916 1 2024-05-12T04:01:20.707 INFO:tasks.workunit.client.0.smithi032.stdout:8/485: mknod d0/d9/d1b/d27/d4c/d4e/d90/caa 0 2024-05-12T04:01:20.707 INFO:tasks.workunit.client.1.smithi033.stdout:6/670: rmdir d4/d8d 39 2024-05-12T04:01:20.708 INFO:tasks.workunit.client.1.smithi033.stdout:3/622: write db/d17/d3a/d67/d76/da0/fb1 [4961180,130158] 0 2024-05-12T04:01:20.709 INFO:tasks.workunit.client.0.smithi032.stdout:6/461: truncate d7/de/f1b 6843090 0 2024-05-12T04:01:20.709 INFO:tasks.workunit.client.1.smithi033.stdout:3/623: readlink db/d17/lc1 0 2024-05-12T04:01:20.710 INFO:tasks.workunit.client.1.smithi033.stdout:6/671: fsync d4/d1a/d4e/d4f/d94/fb9 0 2024-05-12T04:01:20.710 INFO:tasks.workunit.client.1.smithi033.stdout:8/592: mkdir d1/d16/d27/d4d/d51/d8e/db6 0 2024-05-12T04:01:20.710 INFO:tasks.workunit.client.1.smithi033.stdout:5/545: dread d3/d12/f1f [0,4194304] 0 2024-05-12T04:01:20.711 INFO:tasks.workunit.client.1.smithi033.stdout:0/682: mknod d6/dc/d58/d6a/d80/ce6 0 2024-05-12T04:01:20.712 INFO:tasks.workunit.client.1.smithi033.stdout:8/593: dread - d1/d16/d29/fab zero size 2024-05-12T04:01:20.712 INFO:tasks.workunit.client.0.smithi032.stdout:0/528: fsync df/d28/f89 0 2024-05-12T04:01:20.713 INFO:tasks.workunit.client.1.smithi033.stdout:5/546: write d3/d12/f63 [515378,96099] 0 2024-05-12T04:01:20.713 INFO:tasks.workunit.client.0.smithi032.stdout:4/417: truncate d0/d3/d24/f2d 744393 0 2024-05-12T04:01:20.714 INFO:tasks.workunit.client.1.smithi033.stdout:8/594: chown d1/d4/d1d/d36/d3c/d9f 209741 1 2024-05-12T04:01:20.725 INFO:tasks.workunit.client.1.smithi033.stdout:2/631: write dc/d13/d1f/f93 [4116905,95763] 0 2024-05-12T04:01:20.725 INFO:tasks.workunit.client.1.smithi033.stdout:2/632: stat dc/lac 0 2024-05-12T04:01:20.726 INFO:tasks.workunit.client.0.smithi032.stdout:2/485: truncate d1/d13/d70/f91 1654685 0 2024-05-12T04:01:20.728 INFO:tasks.workunit.client.1.smithi033.stdout:1/577: unlink d11/d17/d27/d36/d3f/d4a/l53 0 2024-05-12T04:01:20.730 INFO:tasks.workunit.client.0.smithi032.stdout:4/418: write d0/d3/d24/f42 [3280321,32107] 0 2024-05-12T04:01:20.731 INFO:tasks.workunit.client.1.smithi033.stdout:9/881: mkdir d2/d4/d2b/d33/d139 0 2024-05-12T04:01:20.732 INFO:tasks.workunit.client.0.smithi032.stdout:8/486: mknod d0/cab 0 2024-05-12T04:01:20.733 INFO:tasks.workunit.client.1.smithi033.stdout:7/845: dread de/d2b/f2c [0,4194304] 0 2024-05-12T04:01:20.734 INFO:tasks.workunit.client.1.smithi033.stdout:7/846: chown f3 3607728 1 2024-05-12T04:01:20.735 INFO:tasks.workunit.client.0.smithi032.stdout:6/462: symlink d7/d9/d48/d67/l9c 0 2024-05-12T04:01:20.739 INFO:tasks.workunit.client.1.smithi033.stdout:3/624: stat db/d17/d3a/la1 0 2024-05-12T04:01:20.741 INFO:tasks.workunit.client.0.smithi032.stdout:4/419: dread d0/fa [0,4194304] 0 2024-05-12T04:01:20.742 INFO:tasks.workunit.client.0.smithi032.stdout:2/486: mkdir d1/d34/d96/d41/da2 0 2024-05-12T04:01:20.745 INFO:tasks.workunit.client.1.smithi033.stdout:0/683: mknod d6/dc/d58/d6a/d80/db8/ce7 0 2024-05-12T04:01:20.746 INFO:tasks.workunit.client.1.smithi033.stdout:0/684: chown d6/db/l30 0 1 2024-05-12T04:01:20.746 INFO:tasks.workunit.client.0.smithi032.stdout:0/529: truncate df/d22/f39 556107 0 2024-05-12T04:01:20.749 INFO:tasks.workunit.client.1.smithi033.stdout:8/595: mkdir d1/d16/d27/d7c/db7 0 2024-05-12T04:01:20.750 INFO:tasks.workunit.client.1.smithi033.stdout:8/596: fdatasync d1/d16/d27/d7c/f89 0 2024-05-12T04:01:20.751 INFO:tasks.workunit.client.1.smithi033.stdout:8/597: readlink d1/d16/d29/l5c 0 2024-05-12T04:01:20.752 INFO:tasks.workunit.client.1.smithi033.stdout:0/685: dwrite d6/d11/d57/f21 [0,4194304] 0 2024-05-12T04:01:20.753 INFO:tasks.workunit.client.1.smithi033.stdout:4/651: rename dd/d7a/d87/d7e/da7/d66/d42/d9c/fc3 to dd/d11/d5b/fcd 0 2024-05-12T04:01:20.753 INFO:tasks.workunit.client.0.smithi032.stdout:6/463: mkdir d7/de/d2e/d9d 0 2024-05-12T04:01:20.755 INFO:tasks.workunit.client.1.smithi033.stdout:2/633: fsync dc/d17/d27/d2f/d49/f85 0 2024-05-12T04:01:20.755 INFO:tasks.workunit.client.0.smithi032.stdout:1/480: sync 2024-05-12T04:01:20.755 INFO:tasks.workunit.client.0.smithi032.stdout:3/489: sync 2024-05-12T04:01:20.755 INFO:tasks.workunit.client.0.smithi032.stdout:5/566: getdents d4/d7/d14/d26/d3c 0 2024-05-12T04:01:20.759 INFO:tasks.workunit.client.0.smithi032.stdout:4/420: creat d0/d3/d24/f71 x:0 0 0 2024-05-12T04:01:20.760 INFO:tasks.workunit.client.0.smithi032.stdout:4/421: dread - d0/d3/d24/f71 zero size 2024-05-12T04:01:20.761 INFO:tasks.workunit.client.1.smithi033.stdout:9/882: stat d2/d4/d29/lb3 0 2024-05-12T04:01:20.763 INFO:tasks.workunit.client.1.smithi033.stdout:9/883: write d2/d4/d2b/fd7 [388299,97030] 0 2024-05-12T04:01:20.765 INFO:tasks.workunit.client.1.smithi033.stdout:5/547: dwrite d3/d12/f3f [0,4194304] 0 2024-05-12T04:01:20.766 INFO:tasks.workunit.client.1.smithi033.stdout:9/884: dread - d2/d4/d29/d3c/f113 zero size 2024-05-12T04:01:20.767 INFO:tasks.workunit.client.1.smithi033.stdout:3/625: unlink f6 0 2024-05-12T04:01:20.768 INFO:tasks.workunit.client.1.smithi033.stdout:9/885: chown d2/d4/d2b/d33/dd2/dde/c10c 3666538 1 2024-05-12T04:01:20.768 INFO:tasks.workunit.client.1.smithi033.stdout:6/672: mkdir d4/d1a/d3c/dd9/de2 0 2024-05-12T04:01:20.770 INFO:tasks.workunit.client.0.smithi032.stdout:6/464: creat d7/d97/f9e x:0 0 0 2024-05-12T04:01:20.776 INFO:tasks.workunit.client.0.smithi032.stdout:6/465: dread d7/d9/f4a [0,4194304] 0 2024-05-12T04:01:20.778 INFO:tasks.workunit.client.0.smithi032.stdout:3/490: rename d15/f1c to d15/d33/d3f/fb7 0 2024-05-12T04:01:20.778 INFO:tasks.workunit.client.1.smithi033.stdout:8/598: dwrite d1/f6b [0,4194304] 0 2024-05-12T04:01:20.780 INFO:tasks.workunit.client.0.smithi032.stdout:3/491: truncate d15/d20/d8b/d9b/fa8 338915 0 2024-05-12T04:01:20.781 INFO:tasks.workunit.client.0.smithi032.stdout:4/422: unlink d0/d3/d12/d1d/c41 0 2024-05-12T04:01:20.782 INFO:tasks.workunit.client.0.smithi032.stdout:2/487: symlink d1/d13/d6b/la3 0 2024-05-12T04:01:20.787 INFO:tasks.workunit.client.0.smithi032.stdout:1/481: dread d1/d7/d30/f2c [8388608,4194304] 0 2024-05-12T04:01:20.790 INFO:tasks.workunit.client.1.smithi033.stdout:0/686: write d6/dc/d58/f81 [594116,44709] 0 2024-05-12T04:01:20.793 INFO:tasks.workunit.client.1.smithi033.stdout:7/847: symlink de/df/d17/d1e/dc4/df0/l11b 0 2024-05-12T04:01:20.793 INFO:tasks.workunit.client.1.smithi033.stdout:5/548: creat d3/d27/fa8 x:0 0 0 2024-05-12T04:01:20.794 INFO:tasks.workunit.client.1.smithi033.stdout:9/886: fdatasync d2/dd/d39/f54 0 2024-05-12T04:01:20.796 INFO:tasks.workunit.client.0.smithi032.stdout:5/567: link d4/d7/d14/d26/d33/d4a/d38/fb8 d4/d7/d14/d26/d33/d4a/d38/d79/fc7 0 2024-05-12T04:01:20.797 INFO:tasks.workunit.client.0.smithi032.stdout:4/423: truncate d0/d3/df/f30 1423031 0 2024-05-12T04:01:20.801 INFO:tasks.workunit.client.0.smithi032.stdout:2/488: dread d1/d13/f1f [0,4194304] 0 2024-05-12T04:01:20.802 INFO:tasks.workunit.client.1.smithi033.stdout:2/634: mkdir dc/d17/dcc 0 2024-05-12T04:01:20.803 INFO:tasks.workunit.client.0.smithi032.stdout:2/489: write d1/d34/d96/d41/d2c/f90 [484388,94853] 0 2024-05-12T04:01:20.804 INFO:tasks.workunit.client.0.smithi032.stdout:1/482: unlink d1/d59/l75 0 2024-05-12T04:01:20.805 INFO:tasks.workunit.client.0.smithi032.stdout:1/483: write d1/d7/d49/f6b [693631,102594] 0 2024-05-12T04:01:20.806 INFO:tasks.workunit.client.1.smithi033.stdout:1/578: creat d11/d17/d27/d36/d46/fc4 x:0 0 0 2024-05-12T04:01:20.807 INFO:tasks.workunit.client.0.smithi032.stdout:8/487: write d0/d9/d1b/d33/f3b [397280,43757] 0 2024-05-12T04:01:20.810 INFO:tasks.workunit.client.0.smithi032.stdout:1/484: dread d1/d25/f4a [0,4194304] 0 2024-05-12T04:01:20.814 INFO:tasks.workunit.client.0.smithi032.stdout:6/466: mknod d7/d9/d48/d67/d8d/c9f 0 2024-05-12T04:01:20.815 INFO:tasks.workunit.client.1.smithi033.stdout:7/848: symlink de/df/d9a/l11c 0 2024-05-12T04:01:20.818 INFO:tasks.workunit.client.0.smithi032.stdout:4/424: rmdir d0/d3/df/d4e 39 2024-05-12T04:01:20.820 INFO:tasks.workunit.client.0.smithi032.stdout:4/425: write d0/d3/d12/d1d/d6b/f6d [17028,64118] 0 2024-05-12T04:01:20.821 INFO:tasks.workunit.client.1.smithi033.stdout:7/849: dwrite f3 [0,4194304] 0 2024-05-12T04:01:20.821 INFO:tasks.workunit.client.0.smithi032.stdout:0/530: getdents df/d30 0 2024-05-12T04:01:20.822 INFO:tasks.workunit.client.1.smithi033.stdout:7/850: write de/fbb [1423104,129627] 0 2024-05-12T04:01:20.822 INFO:tasks.workunit.client.0.smithi032.stdout:0/531: dread df/d10/f80 [0,4194304] 0 2024-05-12T04:01:20.826 INFO:tasks.workunit.client.1.smithi033.stdout:6/673: mknod d4/d1a/d67/d6f/daf/ce3 0 2024-05-12T04:01:20.828 INFO:tasks.workunit.client.0.smithi032.stdout:1/485: dread d1/d7/d30/d35/f4e [0,4194304] 0 2024-05-12T04:01:20.830 INFO:tasks.workunit.client.1.smithi033.stdout:4/652: creat dd/d7a/d87/d7e/da7/fce x:0 0 0 2024-05-12T04:01:20.831 INFO:tasks.workunit.client.0.smithi032.stdout:3/492: write d15/d33/d3f/fb7 [2146863,39420] 0 2024-05-12T04:01:20.832 INFO:tasks.workunit.client.0.smithi032.stdout:8/488: rmdir d0/d9/d1b/d27 39 2024-05-12T04:01:20.834 INFO:tasks.workunit.client.0.smithi032.stdout:6/467: mkdir d7/d9/d48/d67/d1a/da0 0 2024-05-12T04:01:20.837 INFO:tasks.workunit.client.1.smithi033.stdout:6/674: dwrite d4/d5/f5c [4194304,4194304] 0 2024-05-12T04:01:20.840 INFO:tasks.workunit.client.0.smithi032.stdout:6/468: dwrite d7/d9/d48/d67/f4d [0,4194304] 0 2024-05-12T04:01:20.840 INFO:tasks.workunit.client.1.smithi033.stdout:9/887: dwrite d2/d4/f9e [0,4194304] 0 2024-05-12T04:01:20.844 INFO:tasks.workunit.client.1.smithi033.stdout:4/653: dwrite dd/d7a/d87/d7e/da7/d66/d42/f82 [0,4194304] 0 2024-05-12T04:01:20.892 INFO:tasks.workunit.client.1.smithi033.stdout:1/579: write d11/d1c/d43/f45 [4415317,61791] 0 2024-05-12T04:01:20.894 INFO:tasks.workunit.client.0.smithi032.stdout:5/568: write d4/d7/d14/d26/d33/d4a/f51 [472851,57826] 0 2024-05-12T04:01:20.898 INFO:tasks.workunit.client.1.smithi033.stdout:1/580: dwrite d11/d17/d27/f44 [0,4194304] 0 2024-05-12T04:01:20.914 INFO:tasks.workunit.client.1.smithi033.stdout:3/626: sync 2024-05-12T04:01:20.914 INFO:tasks.workunit.client.1.smithi033.stdout:8/599: sync 2024-05-12T04:01:20.951 INFO:tasks.workunit.client.0.smithi032.stdout:4/426: rename d0/f13 to d0/d3/d24/f72 0 2024-05-12T04:01:20.953 INFO:tasks.workunit.client.0.smithi032.stdout:0/532: fsync df/f2b 0 2024-05-12T04:01:20.953 INFO:tasks.workunit.client.1.smithi033.stdout:7/851: mknod de/df/d17/d1e/dc4/c11d 0 2024-05-12T04:01:20.954 INFO:tasks.workunit.client.1.smithi033.stdout:7/852: chown de/df/d17/d1e/f23 769676914 1 2024-05-12T04:01:20.955 INFO:tasks.workunit.client.0.smithi032.stdout:2/490: mkdir d1/d34/d96/da4 0 2024-05-12T04:01:20.956 INFO:tasks.workunit.client.1.smithi033.stdout:2/635: getdents dc/d17/dcc 0 2024-05-12T04:01:20.958 INFO:tasks.workunit.client.0.smithi032.stdout:3/493: mknod d15/d20/d8b/cb8 0 2024-05-12T04:01:20.959 INFO:tasks.workunit.client.0.smithi032.stdout:3/494: write d15/d33/f9e [1024677,49014] 0 2024-05-12T04:01:20.960 INFO:tasks.workunit.client.0.smithi032.stdout:3/495: truncate f14 9621618 0 2024-05-12T04:01:20.962 INFO:tasks.workunit.client.0.smithi032.stdout:3/496: write d15/d33/d3f/f87 [1024981,41440] 0 2024-05-12T04:01:20.962 INFO:tasks.workunit.client.1.smithi033.stdout:0/687: creat d6/dc/d58/fe8 x:0 0 0 2024-05-12T04:01:20.963 INFO:tasks.workunit.client.0.smithi032.stdout:3/497: write d15/f19 [963307,54477] 0 2024-05-12T04:01:20.967 INFO:tasks.workunit.client.0.smithi032.stdout:8/489: write d0/d9/d13/d21/d25/f52 [1058275,26567] 0 2024-05-12T04:01:20.968 INFO:tasks.workunit.client.0.smithi032.stdout:8/490: fdatasync d0/d9/d13/f29 0 2024-05-12T04:01:20.969 INFO:tasks.workunit.client.1.smithi033.stdout:6/675: chown d4/d8d 32 1 2024-05-12T04:01:20.970 INFO:tasks.workunit.client.1.smithi033.stdout:6/676: truncate d4/d5/fd6 397340 0 2024-05-12T04:01:20.972 INFO:tasks.workunit.client.1.smithi033.stdout:4/654: creat dd/d7a/d87/d7e/fcf x:0 0 0 2024-05-12T04:01:20.974 INFO:tasks.workunit.client.0.smithi032.stdout:6/469: dwrite d7/d9/d48/d67/f2c [0,4194304] 0 2024-05-12T04:01:20.975 INFO:tasks.workunit.client.0.smithi032.stdout:6/470: dread - d7/de/d2e/f4e zero size 2024-05-12T04:01:20.982 INFO:tasks.workunit.client.0.smithi032.stdout:2/491: mkdir d1/d34/da5 0 2024-05-12T04:01:20.985 INFO:tasks.workunit.client.1.smithi033.stdout:7/853: mkdir de/df/d17/d1b/d110/d11e 0 2024-05-12T04:01:20.986 INFO:tasks.workunit.client.1.smithi033.stdout:3/627: write db/d17/fc5 [2170690,110833] 0 2024-05-12T04:01:20.986 INFO:tasks.workunit.client.1.smithi033.stdout:7/854: chown de/df/d17/d1e/dc4 867199 1 2024-05-12T04:01:20.987 INFO:tasks.workunit.client.1.smithi033.stdout:2/636: unlink dc/d3b/d88/c6b 0 2024-05-12T04:01:20.988 INFO:tasks.workunit.client.0.smithi032.stdout:4/427: dwrite d0/fa [4194304,4194304] 0 2024-05-12T04:01:20.991 INFO:tasks.workunit.client.1.smithi033.stdout:0/688: creat d6/fe9 x:0 0 0 2024-05-12T04:01:20.993 INFO:tasks.workunit.client.0.smithi032.stdout:3/498: symlink d15/d20/d22/d7a/lb9 0 2024-05-12T04:01:20.993 INFO:tasks.workunit.client.0.smithi032.stdout:3/499: stat d15/d1b/l6d 0 2024-05-12T04:01:20.994 INFO:tasks.workunit.client.1.smithi033.stdout:9/888: creat d2/d4/d2b/d6d/d2f/df6/f13a x:0 0 0 2024-05-12T04:01:20.994 INFO:tasks.workunit.client.0.smithi032.stdout:3/500: chown d15/d24/d2a/d5b/f8e 1307 1 2024-05-12T04:01:20.995 INFO:tasks.workunit.client.1.smithi033.stdout:9/889: write d2/d4/f16 [6588349,130165] 0 2024-05-12T04:01:20.996 INFO:tasks.workunit.client.1.smithi033.stdout:4/655: mkdir dd/d7a/d87/d7e/da7/d66/d42/d59/dd0 0 2024-05-12T04:01:20.997 INFO:tasks.workunit.client.1.smithi033.stdout:5/549: link d3/fc d3/d94/d54/d57/d70/fa9 0 2024-05-12T04:01:20.999 INFO:tasks.workunit.client.1.smithi033.stdout:1/581: symlink d11/lc5 0 2024-05-12T04:01:21.000 INFO:tasks.workunit.client.1.smithi033.stdout:8/600: dread d1/d16/d27/d4d/d51/f31 [0,4194304] 0 2024-05-12T04:01:21.001 INFO:tasks.workunit.client.1.smithi033.stdout:1/582: write d11/d1c/d43/f45 [3477587,89140] 0 2024-05-12T04:01:21.001 INFO:tasks.workunit.client.1.smithi033.stdout:3/628: creat db/d17/d4d/fdb x:0 0 0 2024-05-12T04:01:21.004 INFO:tasks.workunit.client.1.smithi033.stdout:7/855: mkdir de/d2b/d6c/da0/da6/dd1/d11f 0 2024-05-12T04:01:21.005 INFO:tasks.workunit.client.1.smithi033.stdout:2/637: dread dc/d17/d27/d2f/f41 [0,4194304] 0 2024-05-12T04:01:21.006 INFO:tasks.workunit.client.1.smithi033.stdout:2/638: chown dc/d13/d1c/c3c 4957 1 2024-05-12T04:01:21.009 INFO:tasks.workunit.client.0.smithi032.stdout:0/533: creat df/d10/d3d/d4a/d9a/d9d/fb7 x:0 0 0 2024-05-12T04:01:21.011 INFO:tasks.workunit.client.1.smithi033.stdout:6/677: mknod d4/d1a/d3c/dd9/de2/ce4 0 2024-05-12T04:01:21.012 INFO:tasks.workunit.client.0.smithi032.stdout:2/492: symlink d1/d34/d96/d41/d39/la6 0 2024-05-12T04:01:21.013 INFO:tasks.workunit.client.0.smithi032.stdout:2/493: chown d1/d34/d4a 0 1 2024-05-12T04:01:21.015 INFO:tasks.workunit.client.0.smithi032.stdout:7/448: sync 2024-05-12T04:01:21.015 INFO:tasks.workunit.client.1.smithi033.stdout:9/890: creat d2/d4/d2b/f13b x:0 0 0 2024-05-12T04:01:21.016 INFO:tasks.workunit.client.1.smithi033.stdout:6/678: dwrite d4/dad/fa7 [0,4194304] 0 2024-05-12T04:01:21.016 INFO:tasks.workunit.client.0.smithi032.stdout:4/428: mknod d0/d39/c73 0 2024-05-12T04:01:21.018 INFO:tasks.workunit.client.0.smithi032.stdout:4/429: dread - d0/d3/f5e zero size 2024-05-12T04:01:21.020 INFO:tasks.workunit.client.0.smithi032.stdout:3/501: mknod d15/d20/d22/cba 0 2024-05-12T04:01:21.021 INFO:tasks.workunit.client.0.smithi032.stdout:3/502: chown f14 303 1 2024-05-12T04:01:21.023 INFO:tasks.workunit.client.0.smithi032.stdout:7/449: dwrite dd/d1b/d43/d74/f7f [0,4194304] 0 2024-05-12T04:01:21.024 INFO:tasks.workunit.client.1.smithi033.stdout:6/679: dwrite d4/fc8 [4194304,4194304] 0 2024-05-12T04:01:21.029 INFO:tasks.workunit.client.0.smithi032.stdout:0/534: dread - df/d10/f64 zero size 2024-05-12T04:01:21.030 INFO:tasks.workunit.client.0.smithi032.stdout:0/535: chown df/d30/d8d/d8f 16454296 1 2024-05-12T04:01:21.031 INFO:tasks.workunit.client.1.smithi033.stdout:1/583: mkdir d11/d17/d27/d36/d3f/d4a/dc6 0 2024-05-12T04:01:21.032 INFO:tasks.workunit.client.0.smithi032.stdout:2/494: mknod d1/d34/ca7 0 2024-05-12T04:01:21.033 INFO:tasks.workunit.client.1.smithi033.stdout:1/584: write d11/d1c/d59/f86 [589371,46002] 0 2024-05-12T04:01:21.038 INFO:tasks.workunit.client.0.smithi032.stdout:8/491: mknod d0/d9/d1b/d27/d4c/d4e/d7f/d91/cac 0 2024-05-12T04:01:21.041 INFO:tasks.workunit.client.1.smithi033.stdout:6/680: dwrite d4/d1a/d67/d68/f8b [0,4194304] 0 2024-05-12T04:01:21.043 INFO:tasks.workunit.client.0.smithi032.stdout:5/569: getdents d4/d7/d14/d39 0 2024-05-12T04:01:21.045 INFO:tasks.workunit.client.1.smithi033.stdout:4/656: dread - dd/d7a/d87/d7e/da7/d38/fbc zero size 2024-05-12T04:01:21.047 INFO:tasks.workunit.client.1.smithi033.stdout:5/550: mkdir d3/d15/daa 0 2024-05-12T04:01:21.049 INFO:tasks.workunit.client.0.smithi032.stdout:6/471: mkdir d7/d9/d5c/d8b/da1 0 2024-05-12T04:01:21.050 INFO:tasks.workunit.client.1.smithi033.stdout:8/601: mkdir d1/d4/db2/db8 0 2024-05-12T04:01:21.051 INFO:tasks.workunit.client.0.smithi032.stdout:2/495: rmdir d1/d34/d95 39 2024-05-12T04:01:21.051 INFO:tasks.workunit.client.1.smithi033.stdout:8/602: stat d1/d16/d27/d4d/d51/d8e/la8 0 2024-05-12T04:01:21.059 INFO:tasks.workunit.client.1.smithi033.stdout:8/603: dread d1/f13 [0,4194304] 0 2024-05-12T04:01:21.059 INFO:tasks.workunit.client.1.smithi033.stdout:7/856: write de/df/d60/f76 [823270,63906] 0 2024-05-12T04:01:21.059 INFO:tasks.workunit.client.0.smithi032.stdout:4/430: truncate d0/d3/f49 146441 0 2024-05-12T04:01:21.059 INFO:tasks.workunit.client.0.smithi032.stdout:8/492: mkdir d0/d4b/dad 0 2024-05-12T04:01:21.059 INFO:tasks.workunit.client.0.smithi032.stdout:8/493: chown d0/d9/d1b/d27/d88/f79 147219839 1 2024-05-12T04:01:21.060 INFO:tasks.workunit.client.0.smithi032.stdout:5/570: creat d4/d7/d14/d26/dc6/fc8 x:0 0 0 2024-05-12T04:01:21.060 INFO:tasks.workunit.client.1.smithi033.stdout:7/857: readlink de/d2b/d6c/lc8 0 2024-05-12T04:01:21.061 INFO:tasks.workunit.client.0.smithi032.stdout:8/494: read d0/d9/d13/f29 [989381,117107] 0 2024-05-12T04:01:21.062 INFO:tasks.workunit.client.0.smithi032.stdout:7/450: mknod dd/d1b/d1e/d45/d55/d72/c94 0 2024-05-12T04:01:21.063 INFO:tasks.workunit.client.0.smithi032.stdout:7/451: chown dd/f82 1 1 2024-05-12T04:01:21.064 INFO:tasks.workunit.client.1.smithi033.stdout:8/604: dwrite d1/d16/d29/f9a [0,4194304] 0 2024-05-12T04:01:21.066 INFO:tasks.workunit.client.0.smithi032.stdout:2/496: creat d1/d13/d70/fa8 x:0 0 0 2024-05-12T04:01:21.069 INFO:tasks.workunit.client.1.smithi033.stdout:1/585: dwrite d11/d17/d27/d36/d3f/f61 [0,4194304] 0 2024-05-12T04:01:21.073 INFO:tasks.workunit.client.0.smithi032.stdout:2/497: dwrite d1/d34/f53 [0,4194304] 0 2024-05-12T04:01:21.075 INFO:tasks.workunit.client.0.smithi032.stdout:9/398: sync 2024-05-12T04:01:21.076 INFO:tasks.workunit.client.0.smithi032.stdout:1/486: sync 2024-05-12T04:01:21.076 INFO:tasks.workunit.client.0.smithi032.stdout:9/399: dread - d0/d3/d6/d58/f5e zero size 2024-05-12T04:01:21.078 INFO:tasks.workunit.client.0.smithi032.stdout:9/400: chown d0/d3/d74 1680 1 2024-05-12T04:01:21.079 INFO:tasks.workunit.client.0.smithi032.stdout:5/571: mkdir d4/d7/d14/d26/d33/d72/dc9 0 2024-05-12T04:01:21.080 INFO:tasks.workunit.client.0.smithi032.stdout:9/401: write d0/d3/d6/d7/d27/f57 [127948,114934] 0 2024-05-12T04:01:21.083 INFO:tasks.workunit.client.1.smithi033.stdout:5/551: fsync d3/fc 0 2024-05-12T04:01:21.084 INFO:tasks.workunit.client.0.smithi032.stdout:8/495: creat d0/d9/d1b/d27/d88/d56/fae x:0 0 0 2024-05-12T04:01:21.084 INFO:tasks.workunit.client.1.smithi033.stdout:3/629: creat db/d17/d25/d32/d3c/d3d/d47/fdc x:0 0 0 2024-05-12T04:01:21.085 INFO:tasks.workunit.client.0.smithi032.stdout:7/452: dread - dd/d1b/d1e/d45/d64/f91 zero size 2024-05-12T04:01:21.087 INFO:tasks.workunit.client.0.smithi032.stdout:6/472: creat d7/d9/d5c/d8b/da1/fa2 x:0 0 0 2024-05-12T04:01:21.089 INFO:tasks.workunit.client.1.smithi033.stdout:8/605: creat d1/d16/d27/d28/fb9 x:0 0 0 2024-05-12T04:01:21.091 INFO:tasks.workunit.client.1.smithi033.stdout:6/681: truncate d4/d1a/d4e/f66 1986391 0 2024-05-12T04:01:21.093 INFO:tasks.workunit.client.1.smithi033.stdout:6/682: dread - d4/d1a/d6c/fc7 zero size 2024-05-12T04:01:21.093 INFO:tasks.workunit.client.0.smithi032.stdout:3/503: getdents d15/d24 0 2024-05-12T04:01:21.095 INFO:tasks.workunit.client.0.smithi032.stdout:8/496: dwrite d0/d9/d13/d21/d25/f83 [0,4194304] 0 2024-05-12T04:01:21.096 INFO:tasks.workunit.client.0.smithi032.stdout:9/402: unlink d0/d3/d6/d7/f2c 0 2024-05-12T04:01:21.096 INFO:tasks.workunit.client.1.smithi033.stdout:6/683: dread d4/d5/fd6 [0,4194304] 0 2024-05-12T04:01:21.096 INFO:tasks.workunit.client.1.smithi033.stdout:8/606: dwrite d1/d16/d29/db1/fb5 [0,4194304] 0 2024-05-12T04:01:21.097 INFO:tasks.workunit.client.1.smithi033.stdout:6/684: write d4/d5/f5c [6727546,18200] 0 2024-05-12T04:01:21.098 INFO:tasks.workunit.client.1.smithi033.stdout:9/891: dread - d2/d4/d2b/d49/d73/fd6 zero size 2024-05-12T04:01:21.098 INFO:tasks.workunit.client.1.smithi033.stdout:8/607: fsync d1/d4/d1d/d36/d3c/da7/f33 0 2024-05-12T04:01:21.099 INFO:tasks.workunit.client.0.smithi032.stdout:0/536: truncate df/d30/d8d/d8f/f97 125226 0 2024-05-12T04:01:21.100 INFO:tasks.workunit.client.1.smithi033.stdout:9/892: chown d2/d4/d29/d32/db7/d115 0 1 2024-05-12T04:01:21.100 INFO:tasks.workunit.client.1.smithi033.stdout:8/608: fsync d1/d4/d1d/f8f 0 2024-05-12T04:01:21.101 INFO:tasks.workunit.client.0.smithi032.stdout:8/497: write d0/d9/d1b/d27/d88/d56/d63/d8f/d9a/d28/f82 [3442153,126060] 0 2024-05-12T04:01:21.103 INFO:tasks.workunit.client.0.smithi032.stdout:5/572: dread d4/d7/d14/d26/f57 [0,4194304] 0 2024-05-12T04:01:21.107 INFO:tasks.workunit.client.1.smithi033.stdout:2/639: getdents dc/d17/d27/d2f/d49 0 2024-05-12T04:01:21.108 INFO:tasks.workunit.client.1.smithi033.stdout:2/640: chown dc/d17/d27/d2f/f90 13 1 2024-05-12T04:01:21.109 INFO:tasks.workunit.client.1.smithi033.stdout:2/641: dread - dc/d3b/d88/fc3 zero size 2024-05-12T04:01:21.109 INFO:tasks.workunit.client.1.smithi033.stdout:5/552: write d3/d12/f1f [5226757,115863] 0 2024-05-12T04:01:21.110 INFO:tasks.workunit.client.1.smithi033.stdout:3/630: chown db/d17/d3a/c43 7470903 1 2024-05-12T04:01:21.113 INFO:tasks.workunit.client.1.smithi033.stdout:7/858: symlink de/df/d17/d1b/d110/d11e/l120 0 2024-05-12T04:01:21.115 INFO:tasks.workunit.client.0.smithi032.stdout:4/431: dwrite d0/d39/f4b [0,4194304] 0 2024-05-12T04:01:21.118 INFO:tasks.workunit.client.1.smithi033.stdout:0/689: creat d6/db/d2c/fea x:0 0 0 2024-05-12T04:01:21.119 INFO:tasks.workunit.client.0.smithi032.stdout:4/432: write d0/f11 [4190176,52230] 0 2024-05-12T04:01:21.120 INFO:tasks.workunit.client.0.smithi032.stdout:7/453: write dd/d1b/d43/d5e/f4d [1551014,128109] 0 2024-05-12T04:01:21.123 INFO:tasks.workunit.client.0.smithi032.stdout:6/473: unlink d7/de/f1b 0 2024-05-12T04:01:21.123 INFO:tasks.workunit.client.0.smithi032.stdout:7/454: write dd/d1b/d43/d5e/d37/f41 [538289,118631] 0 2024-05-12T04:01:21.128 INFO:tasks.workunit.client.0.smithi032.stdout:8/498: creat d0/d9/d1b/d27/d4c/d4e/d90/faf x:0 0 0 2024-05-12T04:01:21.137 INFO:tasks.workunit.client.0.smithi032.stdout:9/403: mkdir d0/d3/d4b/d73/d82 0 2024-05-12T04:01:21.137 INFO:tasks.workunit.client.1.smithi033.stdout:6/685: write d4/d1a/d4e/f96 [791343,106511] 0 2024-05-12T04:01:21.137 INFO:tasks.workunit.client.0.smithi032.stdout:3/504: truncate d15/d24/d93/f70 846669 0 2024-05-12T04:01:21.137 INFO:tasks.workunit.client.0.smithi032.stdout:9/404: readlink d0/d3/d4b/l60 0 2024-05-12T04:01:21.143 INFO:tasks.workunit.client.1.smithi033.stdout:6/686: dwrite d4/d5/f73 [0,4194304] 0 2024-05-12T04:01:21.144 INFO:tasks.workunit.client.0.smithi032.stdout:4/433: chown d0/d3/d31/l45 0 1 2024-05-12T04:01:21.148 INFO:tasks.workunit.client.0.smithi032.stdout:6/474: symlink d7/d9/d48/d67/d1a/d93/la3 0 2024-05-12T04:01:21.151 INFO:tasks.workunit.client.1.smithi033.stdout:3/631: unlink db/d17/d25/f40 0 2024-05-12T04:01:21.152 INFO:tasks.workunit.client.0.smithi032.stdout:7/455: creat dd/d1b/d1e/d45/d64/f95 x:0 0 0 2024-05-12T04:01:21.153 INFO:tasks.workunit.client.0.smithi032.stdout:4/434: dwrite d0/d3/d24/f71 [0,4194304] 0 2024-05-12T04:01:21.154 INFO:tasks.workunit.client.1.smithi033.stdout:0/690: chown d6/dc/f8c 4 1 2024-05-12T04:01:21.155 INFO:tasks.workunit.client.0.smithi032.stdout:7/456: stat dd/d1b/d1e/d45/d55 0 2024-05-12T04:01:21.156 INFO:tasks.workunit.client.0.smithi032.stdout:8/499: mkdir d0/d9/d1b/d27/d88/d56/d63/d8f/db0 0 2024-05-12T04:01:21.157 INFO:tasks.workunit.client.1.smithi033.stdout:1/586: rename d11/lc5 to d11/d17/lc7 0 2024-05-12T04:01:21.158 INFO:tasks.workunit.client.1.smithi033.stdout:3/632: dwrite db/f5e [4194304,4194304] 0 2024-05-12T04:01:21.158 INFO:tasks.workunit.client.0.smithi032.stdout:8/500: write d0/d9/d1b/d27/f7b [1286220,114624] 0 2024-05-12T04:01:21.158 INFO:tasks.workunit.client.0.smithi032.stdout:5/573: creat d4/d7/d14/d26/d33/d72/dc9/fca x:0 0 0 2024-05-12T04:01:21.160 INFO:tasks.workunit.client.0.smithi032.stdout:3/505: mkdir d15/d20/d8b/d9b/dbb 0 2024-05-12T04:01:21.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:20 smithi033 ceph-mon[49525]: pgmap v38: 65 pgs: 65 active+clean; 2.0 GiB data, 7.3 GiB used, 529 GiB / 536 GiB avail; 51 MiB/s rd, 191 MiB/s wr, 350 op/s 2024-05-12T04:01:21.165 INFO:tasks.workunit.client.0.smithi032.stdout:6/475: creat d7/d97/fa4 x:0 0 0 2024-05-12T04:01:21.166 INFO:tasks.workunit.client.0.smithi032.stdout:4/435: dread d0/d3/f1c [0,4194304] 0 2024-05-12T04:01:21.166 INFO:tasks.workunit.client.1.smithi033.stdout:4/657: getdents dd/d7a/d87 0 2024-05-12T04:01:21.167 INFO:tasks.workunit.client.0.smithi032.stdout:4/436: fdatasync d0/fa 0 2024-05-12T04:01:21.167 INFO:tasks.workunit.client.0.smithi032.stdout:6/476: write d7/d9/d48/d67/f4d [4342246,25223] 0 2024-05-12T04:01:21.168 INFO:tasks.workunit.client.1.smithi033.stdout:8/609: unlink d1/d4/d1d/d36/d3c/da7/c7f 0 2024-05-12T04:01:21.171 INFO:tasks.workunit.client.1.smithi033.stdout:6/687: mkdir d4/d1a/d67/de5 0 2024-05-12T04:01:21.173 INFO:tasks.workunit.client.0.smithi032.stdout:9/405: mknod d0/d3/d74/c83 0 2024-05-12T04:01:21.174 INFO:tasks.workunit.client.0.smithi032.stdout:9/406: chown d0/d3/d6/f62 415 1 2024-05-12T04:01:21.176 INFO:tasks.workunit.client.1.smithi033.stdout:7/859: mkdir de/df/d121 0 2024-05-12T04:01:21.178 INFO:tasks.workunit.client.0.smithi032.stdout:0/537: getdents df/d10/d3d/d4a/d84 0 2024-05-12T04:01:21.182 INFO:tasks.workunit.client.0.smithi032.stdout:9/407: dwrite d0/d3/d6/d7/d2a/f68 [0,4194304] 0 2024-05-12T04:01:21.183 INFO:tasks.workunit.client.1.smithi033.stdout:3/633: mknod db/d17/d3e/d65/d91/cdd 0 2024-05-12T04:01:21.184 INFO:tasks.workunit.client.1.smithi033.stdout:3/634: truncate db/d17/d4d/fdb 592970 0 2024-05-12T04:01:21.185 INFO:tasks.workunit.client.1.smithi033.stdout:3/635: chown db/d17/d25/cb7 18 1 2024-05-12T04:01:21.189 INFO:tasks.workunit.client.0.smithi032.stdout:7/457: mknod dd/d1b/d1e/d45/d55/d60/c96 0 2024-05-12T04:01:21.191 INFO:tasks.workunit.client.0.smithi032.stdout:7/458: fdatasync dd/d1b/d43/d74/f84 0 2024-05-12T04:01:21.195 INFO:tasks.workunit.client.0.smithi032.stdout:5/574: creat d4/d7/d14/db5/fcb x:0 0 0 2024-05-12T04:01:21.196 INFO:tasks.workunit.client.1.smithi033.stdout:0/691: dread d6/f61 [0,4194304] 0 2024-05-12T04:01:21.196 INFO:tasks.workunit.client.0.smithi032.stdout:5/575: chown d4/d7/d14/d26/d33/d72/fa7 96200341 1 2024-05-12T04:01:21.197 INFO:tasks.workunit.client.1.smithi033.stdout:1/587: readlink d11/d17/lc7 0 2024-05-12T04:01:21.198 INFO:tasks.workunit.client.1.smithi033.stdout:9/893: dread d2/d4/d2b/dc7/fe6 [0,4194304] 0 2024-05-12T04:01:21.199 INFO:tasks.workunit.client.1.smithi033.stdout:5/553: creat d3/d15/fab x:0 0 0 2024-05-12T04:01:21.201 INFO:tasks.workunit.client.1.smithi033.stdout:7/860: readlink de/df/d17/d1b/d36/l6e 0 2024-05-12T04:01:21.203 INFO:tasks.workunit.client.0.smithi032.stdout:0/538: rmdir df/d10/d3b 39 2024-05-12T04:01:21.203 INFO:tasks.workunit.client.1.smithi033.stdout:9/894: dread - d2/d4/d2b/d49/d73/fdb zero size 2024-05-12T04:01:21.203 INFO:tasks.workunit.client.1.smithi033.stdout:6/688: dwrite d4/d1a/d4e/f89 [0,4194304] 0 2024-05-12T04:01:21.205 INFO:tasks.workunit.client.1.smithi033.stdout:9/895: chown d2/dd/d39/d72/d82/fcd 76104 1 2024-05-12T04:01:21.210 INFO:tasks.workunit.client.1.smithi033.stdout:8/610: mkdir d1/d16/d29/d63/da3/dba 0 2024-05-12T04:01:21.214 INFO:tasks.workunit.client.1.smithi033.stdout:2/642: link dc/d17/d32/d84/f89 dc/d13/d65/db1/fcd 0 2024-05-12T04:01:21.215 INFO:tasks.workunit.client.0.smithi032.stdout:7/459: dread dd/d1b/f4b [0,4194304] 0 2024-05-12T04:01:21.216 INFO:tasks.workunit.client.0.smithi032.stdout:6/477: getdents d7/de/d57 0 2024-05-12T04:01:21.217 INFO:tasks.workunit.client.0.smithi032.stdout:7/460: write dd/f1a [9515207,77483] 0 2024-05-12T04:01:21.218 INFO:tasks.workunit.client.0.smithi032.stdout:7/461: chown dd/d1b/f7a 1215670 1 2024-05-12T04:01:21.219 INFO:tasks.workunit.client.0.smithi032.stdout:7/462: readlink dd/d1b/d43/d74/l8e 0 2024-05-12T04:01:21.222 INFO:tasks.workunit.client.0.smithi032.stdout:8/501: write d0/d9/d1b/d33/f45 [5345688,104251] 0 2024-05-12T04:01:21.226 INFO:tasks.workunit.client.0.smithi032.stdout:3/506: truncate d15/d20/d26/f3c 7986525 0 2024-05-12T04:01:21.227 INFO:tasks.workunit.client.0.smithi032.stdout:0/539: mknod df/d30/d32/d81/cb8 0 2024-05-12T04:01:21.229 INFO:tasks.workunit.client.0.smithi032.stdout:9/408: mkdir d0/d84 0 2024-05-12T04:01:21.230 INFO:tasks.workunit.client.1.smithi033.stdout:6/689: truncate d4/d5/f41 2295438 0 2024-05-12T04:01:21.233 INFO:tasks.workunit.client.1.smithi033.stdout:4/658: dwrite dd/d7a/f9f [0,4194304] 0 2024-05-12T04:01:21.235 INFO:tasks.workunit.client.1.smithi033.stdout:9/896: read d2/d4/d2b/d6d/d7a/f90 [3065360,6127] 0 2024-05-12T04:01:21.236 INFO:tasks.workunit.client.1.smithi033.stdout:3/636: dwrite db/d17/d3e/d65/d7d/f86 [0,4194304] 0 2024-05-12T04:01:21.237 INFO:tasks.workunit.client.0.smithi032.stdout:7/463: symlink dd/d1b/d1e/l97 0 2024-05-12T04:01:21.237 INFO:tasks.workunit.client.0.smithi032.stdout:4/437: dwrite d0/d3/df/f30 [0,4194304] 0 2024-05-12T04:01:21.238 INFO:tasks.workunit.client.1.smithi033.stdout:2/643: symlink dc/d13/d39/d8f/d9e/lce 0 2024-05-12T04:01:21.239 INFO:tasks.workunit.client.0.smithi032.stdout:7/464: fdatasync f5 0 2024-05-12T04:01:21.240 INFO:tasks.workunit.client.0.smithi032.stdout:8/502: creat d0/d9/d1b/d27/d59/fb1 x:0 0 0 2024-05-12T04:01:21.241 INFO:tasks.workunit.client.1.smithi033.stdout:2/644: write dc/d13/d65/f83 [618307,7848] 0 2024-05-12T04:01:21.242 INFO:tasks.workunit.client.1.smithi033.stdout:5/554: mkdir d3/d12/dac 0 2024-05-12T04:01:21.245 INFO:tasks.workunit.client.1.smithi033.stdout:7/861: fdatasync de/fa1 0 2024-05-12T04:01:21.247 INFO:tasks.workunit.client.1.smithi033.stdout:1/588: link d11/d1c/d43/f4d d11/d17/d27/d36/d46/d4c/fc8 0 2024-05-12T04:01:21.249 INFO:tasks.workunit.client.1.smithi033.stdout:0/692: dwrite d6/d11/d57/fcc [0,4194304] 0 2024-05-12T04:01:21.249 INFO:tasks.workunit.client.1.smithi033.stdout:1/589: dread - d11/d1c/f9d zero size 2024-05-12T04:01:21.251 INFO:tasks.workunit.client.1.smithi033.stdout:2/645: dread dc/d17/d32/d84/fa0 [0,4194304] 0 2024-05-12T04:01:21.252 INFO:tasks.workunit.client.1.smithi033.stdout:1/590: chown d11/d17/d9e/fc3 13882816 1 2024-05-12T04:01:21.253 INFO:tasks.workunit.client.1.smithi033.stdout:0/693: read d6/d11/f99 [76823,52012] 0 2024-05-12T04:01:21.253 INFO:tasks.workunit.client.0.smithi032.stdout:5/576: dwrite d4/d7/d14/d26/d33/d72/d92/f97 [0,4194304] 0 2024-05-12T04:01:21.254 INFO:tasks.workunit.client.1.smithi033.stdout:7/862: dread de/d2b/f2c [0,4194304] 0 2024-05-12T04:01:21.255 INFO:tasks.workunit.client.1.smithi033.stdout:1/591: read d11/d17/d27/d76/fa7 [3704261,91466] 0 2024-05-12T04:01:21.258 INFO:tasks.workunit.client.0.smithi032.stdout:3/507: rmdir d15/d20/d26/d6a/d6e 39 2024-05-12T04:01:21.258 INFO:tasks.workunit.client.0.smithi032.stdout:9/409: mkdir d0/d3/d4b/d85 0 2024-05-12T04:01:21.258 INFO:tasks.workunit.client.0.smithi032.stdout:7/465: fsync dd/f82 0 2024-05-12T04:01:21.258 INFO:tasks.workunit.client.0.smithi032.stdout:6/478: creat d7/de/d2e/d9d/fa5 x:0 0 0 2024-05-12T04:01:21.259 INFO:tasks.workunit.client.0.smithi032.stdout:8/503: dread - d0/d4b/fa1 zero size 2024-05-12T04:01:21.262 INFO:tasks.workunit.client.0.smithi032.stdout:7/466: write dd/f13 [2698186,103475] 0 2024-05-12T04:01:21.262 INFO:tasks.workunit.client.0.smithi032.stdout:8/504: chown d0/d9/f2d 31059 1 2024-05-12T04:01:21.263 INFO:tasks.workunit.client.0.smithi032.stdout:8/505: read - d0/d97/fa6 zero size 2024-05-12T04:01:21.263 INFO:tasks.workunit.client.0.smithi032.stdout:7/467: chown dd/d1b/c35 34009 1 2024-05-12T04:01:21.264 INFO:tasks.workunit.client.0.smithi032.stdout:8/506: chown d0/d9/d1b/d66 6550 1 2024-05-12T04:01:21.265 INFO:tasks.workunit.client.1.smithi033.stdout:9/897: rename d2/d4/lac to d2/d4/d2b/d6d/d2f/d5f/d84/d107/l13c 0 2024-05-12T04:01:21.265 INFO:tasks.workunit.client.0.smithi032.stdout:8/507: stat d0/d9/d1b/d27/d88/d56/d63/d8f 0 2024-05-12T04:01:21.273 INFO:tasks.workunit.client.1.smithi033.stdout:0/694: chown d6/dc/l92 2890 1 2024-05-12T04:01:21.276 INFO:tasks.workunit.client.1.smithi033.stdout:1/592: read d11/d17/d27/d36/d46/f51 [782786,14994] 0 2024-05-12T04:01:21.278 INFO:tasks.workunit.client.1.smithi033.stdout:4/659: mkdir dd/d11/d5b/dd1 0 2024-05-12T04:01:21.281 INFO:tasks.workunit.client.1.smithi033.stdout:1/593: dwrite d11/d17/f3e [0,4194304] 0 2024-05-12T04:01:21.282 INFO:tasks.workunit.client.1.smithi033.stdout:7/863: rename de/d2f/d42/l7d to de/df/d17/d1b/de0/l122 0 2024-05-12T04:01:21.284 INFO:tasks.workunit.client.1.smithi033.stdout:1/594: chown d11/d17/d27/d36/d46/d4c/c5b 1879474892 1 2024-05-12T04:01:21.284 INFO:tasks.workunit.client.1.smithi033.stdout:7/864: write de/df/d60/d75/fa7 [1023151,87328] 0 2024-05-12T04:01:21.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:20 smithi032 ceph-mon[40008]: pgmap v38: 65 pgs: 65 active+clean; 2.0 GiB data, 7.3 GiB used, 529 GiB / 536 GiB avail; 51 MiB/s rd, 191 MiB/s wr, 350 op/s 2024-05-12T04:01:21.290 INFO:tasks.workunit.client.0.smithi032.stdout:4/438: creat d0/d3/d12/d1d/f74 x:0 0 0 2024-05-12T04:01:21.292 INFO:tasks.workunit.client.1.smithi033.stdout:8/611: getdents d1/d16/d27/d4d 0 2024-05-12T04:01:21.293 INFO:tasks.workunit.client.1.smithi033.stdout:4/660: readlink dd/d7a/d87/d7e/da7/l3a 0 2024-05-12T04:01:21.293 INFO:tasks.workunit.client.0.smithi032.stdout:3/508: mknod d15/d33/d3f/cbc 0 2024-05-12T04:01:21.294 INFO:tasks.workunit.client.0.smithi032.stdout:5/577: rmdir d4/d7/d14/d26/d33/d72/d92 39 2024-05-12T04:01:21.296 INFO:tasks.workunit.client.1.smithi033.stdout:9/898: mkdir d2/d4/d2b/d78/d76/d13d 0 2024-05-12T04:01:21.299 INFO:tasks.workunit.client.0.smithi032.stdout:3/509: dwrite d15/d20/d22/d4f/f53 [0,4194304] 0 2024-05-12T04:01:21.299 INFO:tasks.workunit.client.1.smithi033.stdout:4/661: dwrite dd/d7a/d87/d7e/da7/fc7 [0,4194304] 0 2024-05-12T04:01:21.300 INFO:tasks.workunit.client.0.smithi032.stdout:3/510: chown d15/d24/d2a/d8c 201 1 2024-05-12T04:01:21.301 INFO:tasks.workunit.client.0.smithi032.stdout:3/511: fdatasync d15/f4e 0 2024-05-12T04:01:21.301 INFO:tasks.workunit.client.0.smithi032.stdout:3/512: readlink d15/l16 0 2024-05-12T04:01:21.302 INFO:tasks.workunit.client.0.smithi032.stdout:4/439: chown d0/d3/l20 1020274257 1 2024-05-12T04:01:21.304 INFO:tasks.workunit.client.0.smithi032.stdout:4/440: write d0/d3/d12/f5b [921063,88049] 0 2024-05-12T04:01:21.305 INFO:tasks.workunit.client.0.smithi032.stdout:0/540: link df/d30/d32/d81/lad df/d30/d4e/d55/lb9 0 2024-05-12T04:01:21.306 INFO:tasks.workunit.client.0.smithi032.stdout:0/541: chown df/d30/d32/d81/cb8 4681906 1 2024-05-12T04:01:21.311 INFO:tasks.workunit.client.0.smithi032.stdout:5/578: symlink d4/d7/d14/d26/d33/d5f/d7f/lcc 0 2024-05-12T04:01:21.312 INFO:tasks.workunit.client.1.smithi033.stdout:0/695: link d6/dc/d58/f88 d6/dc/d58/d6a/feb 0 2024-05-12T04:01:21.315 INFO:tasks.workunit.client.1.smithi033.stdout:9/899: dwrite d2/d4/d29/d3c/f102 [0,4194304] 0 2024-05-12T04:01:21.315 INFO:tasks.workunit.client.0.smithi032.stdout:7/468: dread dd/d1b/d1e/f68 [0,4194304] 0 2024-05-12T04:01:21.316 INFO:tasks.workunit.client.1.smithi033.stdout:9/900: write d2/d4/d2b/d49/ff0 [2571060,64893] 0 2024-05-12T04:01:21.319 INFO:tasks.workunit.client.1.smithi033.stdout:8/612: creat d1/d4/db2/fbb x:0 0 0 2024-05-12T04:01:21.320 INFO:tasks.workunit.client.0.smithi032.stdout:8/508: dread d0/d9/d1b/d27/f7b [0,4194304] 0 2024-05-12T04:01:21.323 INFO:tasks.workunit.client.1.smithi033.stdout:0/696: read d6/db/d2c/d73/f87 [167478,103812] 0 2024-05-12T04:01:21.325 INFO:tasks.workunit.client.0.smithi032.stdout:3/513: symlink d15/d24/lbd 0 2024-05-12T04:01:21.329 INFO:tasks.workunit.client.1.smithi033.stdout:5/555: getdents d3/d94 0 2024-05-12T04:01:21.332 INFO:tasks.workunit.client.1.smithi033.stdout:5/556: read d3/f92 [1179814,4151] 0 2024-05-12T04:01:21.334 INFO:tasks.workunit.client.1.smithi033.stdout:5/557: write d3/d15/d16/f51 [928274,45054] 0 2024-05-12T04:01:21.334 INFO:tasks.workunit.client.0.smithi032.stdout:0/542: write df/d10/d3b/f66 [876023,12066] 0 2024-05-12T04:01:21.336 INFO:tasks.workunit.client.1.smithi033.stdout:9/901: mkdir d2/d4/d2b/d6d/d2f/df6/d13e 0 2024-05-12T04:01:21.338 INFO:tasks.workunit.client.0.smithi032.stdout:5/579: read d4/f59 [3379809,37992] 0 2024-05-12T04:01:21.339 INFO:tasks.workunit.client.1.smithi033.stdout:6/690: dwrite d4/f82 [0,4194304] 0 2024-05-12T04:01:21.340 INFO:tasks.workunit.client.1.smithi033.stdout:4/662: symlink dd/d7a/d87/d7e/da7/ld2 0 2024-05-12T04:01:21.340 INFO:tasks.workunit.client.1.smithi033.stdout:3/637: write db/d17/d4d/fad [1145373,130512] 0 2024-05-12T04:01:21.342 INFO:tasks.workunit.client.0.smithi032.stdout:3/514: mkdir d15/d20/d22/d7a/dbe 0 2024-05-12T04:01:21.343 INFO:tasks.workunit.client.1.smithi033.stdout:0/697: mkdir d6/dc/d58/d6a/dec 0 2024-05-12T04:01:21.344 INFO:tasks.workunit.client.0.smithi032.stdout:7/469: dwrite dd/f36 [0,4194304] 0 2024-05-12T04:01:21.346 INFO:tasks.workunit.client.0.smithi032.stdout:4/441: mkdir d0/d75 0 2024-05-12T04:01:21.346 INFO:tasks.workunit.client.0.smithi032.stdout:7/470: write dd/d1b/d1e/d45/d64/f95 [633320,92400] 0 2024-05-12T04:01:21.352 INFO:tasks.workunit.client.0.smithi032.stdout:0/543: mkdir df/d10/d3b/dba 0 2024-05-12T04:01:21.353 INFO:tasks.workunit.client.0.smithi032.stdout:3/515: dwrite d15/d20/d22/d7a/f85 [0,4194304] 0 2024-05-12T04:01:21.355 INFO:tasks.workunit.client.1.smithi033.stdout:8/613: rename d1/d4/f1c to d1/d4/db2/fbc 0 2024-05-12T04:01:21.361 INFO:tasks.workunit.client.1.smithi033.stdout:5/558: creat d3/d15/d16/d45/d62/fad x:0 0 0 2024-05-12T04:01:21.364 INFO:tasks.workunit.client.1.smithi033.stdout:9/902: creat d2/dd/d39/d72/d82/f13f x:0 0 0 2024-05-12T04:01:21.366 INFO:tasks.workunit.client.1.smithi033.stdout:6/691: symlink d4/d8d/le6 0 2024-05-12T04:01:21.367 INFO:tasks.workunit.client.1.smithi033.stdout:6/692: chown d4/d1a/d67/d68/f72 209461817 1 2024-05-12T04:01:21.369 INFO:tasks.workunit.client.1.smithi033.stdout:4/663: write dd/f13 [2204106,44761] 0 2024-05-12T04:01:21.376 INFO:tasks.workunit.client.0.smithi032.stdout:8/509: creat d0/fb2 x:0 0 0 2024-05-12T04:01:21.377 INFO:tasks.workunit.client.1.smithi033.stdout:3/638: rename db/d17/fc9 to db/d17/d4d/fde 0 2024-05-12T04:01:21.380 INFO:tasks.workunit.client.1.smithi033.stdout:6/693: symlink d4/d1a/d3c/dd9/le7 0 2024-05-12T04:01:21.382 INFO:tasks.workunit.client.1.smithi033.stdout:9/903: read d2/d4/d2b/d6d/d2f/fd8 [774920,70488] 0 2024-05-12T04:01:21.382 INFO:tasks.workunit.client.1.smithi033.stdout:4/664: creat dd/d11/da5/fd3 x:0 0 0 2024-05-12T04:01:21.384 INFO:tasks.workunit.client.0.smithi032.stdout:8/510: creat d0/d9/d1b/d27/d88/d56/d63/d8f/d9a/d28/d3e/fb3 x:0 0 0 2024-05-12T04:01:21.384 INFO:tasks.workunit.client.1.smithi033.stdout:9/904: chown d2/d4/d2b/d78 844 1 2024-05-12T04:01:21.384 INFO:tasks.workunit.client.1.smithi033.stdout:4/665: fsync dd/d7a/d87/fb7 0 2024-05-12T04:01:21.385 INFO:tasks.workunit.client.0.smithi032.stdout:0/544: getdents df/d10/d48/d70 0 2024-05-12T04:01:21.387 INFO:tasks.workunit.client.0.smithi032.stdout:4/442: dread d0/f11 [0,4194304] 0 2024-05-12T04:01:21.390 INFO:tasks.workunit.client.0.smithi032.stdout:0/545: dread df/d30/d32/f34 [4194304,4194304] 0 2024-05-12T04:01:21.390 INFO:tasks.workunit.client.1.smithi033.stdout:8/614: rename d1/d16/d27/d4d/d51/f31 to d1/d4/d91/fbd 0 2024-05-12T04:01:21.392 INFO:tasks.workunit.client.1.smithi033.stdout:5/559: symlink d3/d94/d85/d99/da2/lae 0 2024-05-12T04:01:21.395 INFO:tasks.workunit.client.1.smithi033.stdout:4/666: mkdir dd/d11/d75/dd4 0 2024-05-12T04:01:21.396 INFO:tasks.workunit.client.0.smithi032.stdout:4/443: truncate d0/d3/fd 517129 0 2024-05-12T04:01:21.396 INFO:tasks.workunit.client.1.smithi033.stdout:4/667: readlink dd/d69/da4/lc0 0 2024-05-12T04:01:21.397 INFO:tasks.workunit.client.1.smithi033.stdout:4/668: chown dd/d69/da4/lc0 21 1 2024-05-12T04:01:21.397 INFO:tasks.workunit.client.0.smithi032.stdout:0/546: unlink df/d28/f89 0 2024-05-12T04:01:21.398 INFO:tasks.workunit.client.0.smithi032.stdout:0/547: fdatasync df/d30/d4e/d55/f7f 0 2024-05-12T04:01:21.400 INFO:tasks.workunit.client.1.smithi033.stdout:8/615: fsync d1/d4/d1d/f58 0 2024-05-12T04:01:21.400 INFO:tasks.workunit.client.1.smithi033.stdout:7/865: write de/d20/d38/dec/ffb [3822160,83641] 0 2024-05-12T04:01:21.401 INFO:tasks.workunit.client.0.smithi032.stdout:4/444: rmdir d0/d3/d12/d1d/d6b 39 2024-05-12T04:01:21.402 INFO:tasks.workunit.client.1.smithi033.stdout:8/616: write d1/d16/d27/d28/fb9 [834372,67941] 0 2024-05-12T04:01:21.403 INFO:tasks.workunit.client.1.smithi033.stdout:7/866: write de/d20/d38/dec/d117/ff1 [235584,17938] 0 2024-05-12T04:01:21.405 INFO:tasks.workunit.client.1.smithi033.stdout:7/867: fdatasync de/df/d17/d1e/d24/f6f 0 2024-05-12T04:01:21.406 INFO:tasks.workunit.client.1.smithi033.stdout:6/694: symlink d4/d1a/d67/de5/le8 0 2024-05-12T04:01:21.407 INFO:tasks.workunit.client.0.smithi032.stdout:0/548: mkdir df/d10/d48/d70/dbb 0 2024-05-12T04:01:21.410 INFO:tasks.workunit.client.0.smithi032.stdout:4/445: unlink d0/fa 0 2024-05-12T04:01:21.417 INFO:tasks.workunit.client.0.smithi032.stdout:4/446: creat d0/d3/d24/f76 x:0 0 0 2024-05-12T04:01:21.418 INFO:tasks.workunit.client.0.smithi032.stdout:0/549: getdents df/d10/d3b/d75 0 2024-05-12T04:01:21.418 INFO:tasks.workunit.client.1.smithi033.stdout:1/595: dwrite d11/d17/f2e [4194304,4194304] 0 2024-05-12T04:01:21.422 INFO:tasks.workunit.client.1.smithi033.stdout:1/596: dread d11/f6b [0,4194304] 0 2024-05-12T04:01:21.423 INFO:tasks.workunit.client.0.smithi032.stdout:4/447: mkdir d0/d3/d12/d1d/d63/d77 0 2024-05-12T04:01:21.425 INFO:tasks.workunit.client.1.smithi033.stdout:6/695: creat d4/d5/dcd/fe9 x:0 0 0 2024-05-12T04:01:21.427 INFO:tasks.workunit.client.0.smithi032.stdout:4/448: dread d0/d3/d12/f66 [0,4194304] 0 2024-05-12T04:01:21.430 INFO:tasks.workunit.client.0.smithi032.stdout:3/516: rename ca to d15/d24/d40/cbf 0 2024-05-12T04:01:21.431 INFO:tasks.workunit.client.1.smithi033.stdout:4/669: mknod dd/d7a/d87/d7e/da7/d66/cd5 0 2024-05-12T04:01:21.434 INFO:tasks.workunit.client.0.smithi032.stdout:0/550: symlink df/d10/d3d/db0/lbc 0 2024-05-12T04:01:21.440 INFO:tasks.workunit.client.0.smithi032.stdout:3/517: unlink d15/d24/d40/c9d 0 2024-05-12T04:01:21.441 INFO:tasks.workunit.client.0.smithi032.stdout:5/580: dwrite d4/d7/d14/d26/d33/d4a/d38/d3b/f48 [0,4194304] 0 2024-05-12T04:01:21.442 INFO:tasks.workunit.client.1.smithi033.stdout:4/670: dwrite dd/d11/d5b/f86 [4194304,4194304] 0 2024-05-12T04:01:21.442 INFO:tasks.workunit.client.0.smithi032.stdout:5/581: chown d4/d7/d14/d39/l55 10304065 1 2024-05-12T04:01:21.445 INFO:tasks.workunit.client.1.smithi033.stdout:7/868: mkdir de/d20/d123 0 2024-05-12T04:01:21.448 INFO:tasks.workunit.client.1.smithi033.stdout:2/646: sync 2024-05-12T04:01:21.451 INFO:tasks.workunit.client.0.smithi032.stdout:4/449: creat d0/d75/f78 x:0 0 0 2024-05-12T04:01:21.451 INFO:tasks.workunit.client.0.smithi032.stdout:7/471: dwrite f6 [0,4194304] 0 2024-05-12T04:01:21.452 INFO:tasks.workunit.client.0.smithi032.stdout:4/450: readlink d0/d3/l68 0 2024-05-12T04:01:21.454 INFO:tasks.workunit.client.0.smithi032.stdout:7/472: dread - dd/d1b/d1e/f7e zero size 2024-05-12T04:01:21.455 INFO:tasks.workunit.client.0.smithi032.stdout:7/473: chown dd/d1b/d1e/f8d 1132 1 2024-05-12T04:01:21.457 INFO:tasks.workunit.client.1.smithi033.stdout:9/905: getdents d2/d4/d2b/d6d/d7a 0 2024-05-12T04:01:21.459 INFO:tasks.workunit.client.0.smithi032.stdout:8/511: write d0/d9/fe [465283,43986] 0 2024-05-12T04:01:21.460 INFO:tasks.workunit.client.0.smithi032.stdout:3/518: truncate d15/d20/d26/d6a/d6e/d7d/fa1 418016 0 2024-05-12T04:01:21.461 INFO:tasks.workunit.client.0.smithi032.stdout:5/582: mknod d4/d7/d14/d26/dc6/ccd 0 2024-05-12T04:01:21.466 INFO:tasks.workunit.client.1.smithi033.stdout:0/698: dwrite d6/f26 [0,4194304] 0 2024-05-12T04:01:21.466 INFO:tasks.workunit.client.1.smithi033.stdout:8/617: link d1/d4/d1d/d36/f74 d1/d16/fbe 0 2024-05-12T04:01:21.470 INFO:tasks.workunit.client.1.smithi033.stdout:0/699: dwrite d6/db/d2c/f3e [0,4194304] 0 2024-05-12T04:01:21.476 INFO:tasks.workunit.client.1.smithi033.stdout:5/560: write d3/f52 [4808455,11667] 0 2024-05-12T04:01:21.482 INFO:tasks.workunit.client.0.smithi032.stdout:4/451: dread d0/d3/d12/d1d/f4c [0,4194304] 0 2024-05-12T04:01:21.483 INFO:tasks.workunit.client.0.smithi032.stdout:5/583: mknod d4/d7/d14/d26/d33/d4a/d38/cce 0 2024-05-12T04:01:21.491 INFO:tasks.workunit.client.0.smithi032.stdout:7/474: creat dd/d1b/d1e/d45/d55/d60/d75/d8f/f98 x:0 0 0 2024-05-12T04:01:21.495 INFO:tasks.workunit.client.0.smithi032.stdout:8/512: truncate d0/d9/d1b/d33/f78 524821 0 2024-05-12T04:01:21.501 INFO:tasks.workunit.client.0.smithi032.stdout:0/551: truncate df/d10/d48/f96 1264802 0 2024-05-12T04:01:21.504 INFO:tasks.workunit.client.0.smithi032.stdout:2/498: sync 2024-05-12T04:01:21.504 INFO:tasks.workunit.client.0.smithi032.stdout:1/487: sync 2024-05-12T04:01:21.504 INFO:tasks.workunit.client.0.smithi032.stdout:2/499: stat d1/d34 0 2024-05-12T04:01:21.507 INFO:tasks.workunit.client.0.smithi032.stdout:7/475: creat dd/d1b/d1e/d45/d55/d60/d75/d8f/f99 x:0 0 0 2024-05-12T04:01:21.508 INFO:tasks.workunit.client.0.smithi032.stdout:1/488: dread d1/d6a/d72/f88 [0,4194304] 0 2024-05-12T04:01:21.509 INFO:tasks.workunit.client.0.smithi032.stdout:8/513: creat d0/d9/d1b/d27/d88/d56/d63/d8f/d9a/fb4 x:0 0 0 2024-05-12T04:01:21.511 INFO:tasks.workunit.client.0.smithi032.stdout:4/452: mknod d0/d3/d12/d1d/d63/d77/c79 0 2024-05-12T04:01:21.512 INFO:tasks.workunit.client.0.smithi032.stdout:4/453: readlink d0/d3/d12/l50 0 2024-05-12T04:01:21.512 INFO:tasks.workunit.client.0.smithi032.stdout:8/514: truncate d0/d9/d1b/d27/d4c/d4e/d7f/f9c 853705 0 2024-05-12T04:01:21.514 INFO:tasks.workunit.client.0.smithi032.stdout:5/584: link d4/d7/fc1 d4/d7/d14/d39/fcf 0 2024-05-12T04:01:21.519 INFO:tasks.workunit.client.0.smithi032.stdout:3/519: dwrite d15/d33/f5c [0,4194304] 0 2024-05-12T04:01:21.522 INFO:tasks.workunit.client.0.smithi032.stdout:2/500: chown d1/d73 829609114 1 2024-05-12T04:01:21.523 INFO:tasks.workunit.client.0.smithi032.stdout:2/501: readlink d1/d34/d4a/l80 0 2024-05-12T04:01:21.524 INFO:tasks.workunit.client.1.smithi033.stdout:3/639: sync 2024-05-12T04:01:21.524 INFO:tasks.workunit.client.0.smithi032.stdout:0/552: dwrite df/d22/d58/f6a [0,4194304] 0 2024-05-12T04:01:21.526 INFO:tasks.workunit.client.0.smithi032.stdout:2/502: dread d1/d34/d4a/f6d [0,4194304] 0 2024-05-12T04:01:21.530 INFO:tasks.workunit.client.0.smithi032.stdout:1/489: mknod d1/d6a/c9f 0 2024-05-12T04:01:21.533 INFO:tasks.workunit.client.0.smithi032.stdout:4/454: chown d0/d3/c2c 14302326 1 2024-05-12T04:01:21.541 INFO:tasks.workunit.client.0.smithi032.stdout:8/515: dread d0/d9/f12 [0,4194304] 0 2024-05-12T04:01:21.549 INFO:tasks.workunit.client.0.smithi032.stdout:0/553: creat df/d10/d3d/db0/fbd x:0 0 0 2024-05-12T04:01:21.553 INFO:tasks.workunit.client.0.smithi032.stdout:4/455: dread d0/d3/d31/f35 [0,4194304] 0 2024-05-12T04:01:21.554 INFO:tasks.workunit.client.0.smithi032.stdout:1/490: dread - d1/d6a/f6e zero size 2024-05-12T04:01:21.555 INFO:tasks.workunit.client.0.smithi032.stdout:1/491: stat d1/f47 0 2024-05-12T04:01:21.556 INFO:tasks.workunit.client.0.smithi032.stdout:1/492: write d1/d6a/f7a [541243,33591] 0 2024-05-12T04:01:21.561 INFO:tasks.workunit.client.1.smithi033.stdout:4/671: chown dd/d7a/d87/d7e/da7/d66/d42/c96 182264 1 2024-05-12T04:01:21.562 INFO:tasks.workunit.client.0.smithi032.stdout:2/503: dread d1/d13/d70/d85/f9c [0,4194304] 0 2024-05-12T04:01:21.563 INFO:tasks.workunit.client.0.smithi032.stdout:2/504: chown d1/fc 8 1 2024-05-12T04:01:21.563 INFO:tasks.workunit.client.1.smithi033.stdout:7/869: creat de/d2b/d6c/db5/dd0/dd6/f124 x:0 0 0 2024-05-12T04:01:21.567 INFO:tasks.workunit.client.1.smithi033.stdout:2/647: rmdir dc/d17/d27/d68 39 2024-05-12T04:01:21.569 INFO:tasks.workunit.client.0.smithi032.stdout:2/505: dwrite d1/d34/d3e/d67/f69 [0,4194304] 0 2024-05-12T04:01:21.570 INFO:tasks.workunit.client.0.smithi032.stdout:2/506: read - d1/d13/d70/f7d zero size 2024-05-12T04:01:21.570 INFO:tasks.workunit.client.1.smithi033.stdout:6/696: truncate d4/d43/f60 798519 0 2024-05-12T04:01:21.571 INFO:tasks.workunit.client.1.smithi033.stdout:6/697: fdatasync d4/d5/f73 0 2024-05-12T04:01:21.575 INFO:tasks.workunit.client.1.smithi033.stdout:9/906: rmdir d2/d4/d2b/d6d/d2f/d5f/d84 39 2024-05-12T04:01:21.575 INFO:tasks.workunit.client.0.smithi032.stdout:2/507: dread d1/d34/d3e/d67/f69 [0,4194304] 0 2024-05-12T04:01:21.576 INFO:tasks.workunit.client.0.smithi032.stdout:4/456: write d0/d3/d12/d1d/f4c [80989,78469] 0 2024-05-12T04:01:21.579 INFO:tasks.workunit.client.1.smithi033.stdout:8/618: creat d1/d16/d29/d63/fbf x:0 0 0 2024-05-12T04:01:21.586 INFO:tasks.workunit.client.0.smithi032.stdout:1/493: rename d1/d7/d49/l5c to d1/d7/d30/d35/d54/d7e/la0 0 2024-05-12T04:01:21.587 INFO:tasks.workunit.client.0.smithi032.stdout:1/494: dread - d1/d7/d30/d18/f8b zero size 2024-05-12T04:01:21.587 INFO:tasks.workunit.client.0.smithi032.stdout:2/508: dread d1/d34/f8c [0,4194304] 0 2024-05-12T04:01:21.588 INFO:tasks.workunit.client.1.smithi033.stdout:5/561: creat d3/d12/d29/d2b/d4f/faf x:0 0 0 2024-05-12T04:01:21.588 INFO:tasks.workunit.client.0.smithi032.stdout:1/495: readlink d1/d7/d30/d18/l37 0 2024-05-12T04:01:21.591 INFO:tasks.workunit.client.0.smithi032.stdout:3/520: symlink d15/d24/d2a/d8c/db1/lc0 0 2024-05-12T04:01:21.594 INFO:tasks.workunit.client.0.smithi032.stdout:5/585: mkdir d4/d7/d14/d39/d1b/da4/dd0 0 2024-05-12T04:01:21.596 INFO:tasks.workunit.client.1.smithi033.stdout:1/597: rename d11/d17/f9b to d11/d17/d27/fc9 0 2024-05-12T04:01:21.598 INFO:tasks.workunit.client.1.smithi033.stdout:2/648: creat dc/d17/d27/d2f/d49/d59/fcf x:0 0 0 2024-05-12T04:01:21.603 INFO:tasks.workunit.client.1.smithi033.stdout:4/672: write dd/f36 [1480176,78115] 0 2024-05-12T04:01:21.605 INFO:tasks.workunit.client.1.smithi033.stdout:4/673: write dd/d11/d75/fb4 [14498,67063] 0 2024-05-12T04:01:21.613 INFO:tasks.workunit.client.1.smithi033.stdout:0/700: creat d6/dc/d58/d6a/dec/fed x:0 0 0 2024-05-12T04:01:21.614 INFO:tasks.workunit.client.1.smithi033.stdout:7/870: dwrite de/d2b/d6c/db5/dd0/dd6/fda [0,4194304] 0 2024-05-12T04:01:21.615 INFO:tasks.workunit.client.1.smithi033.stdout:0/701: read d6/dc/d58/f59 [3407799,41367] 0 2024-05-12T04:01:21.616 INFO:tasks.workunit.client.1.smithi033.stdout:5/562: mkdir d3/d12/d25/db0 0 2024-05-12T04:01:21.617 INFO:tasks.workunit.client.1.smithi033.stdout:0/702: chown d6/dc/d58/f81 125275360 1 2024-05-12T04:01:21.620 INFO:tasks.workunit.client.1.smithi033.stdout:8/619: rename d1/d4/d1d/f26 to d1/d4/d1d/d36/d3c/d64/fc0 0 2024-05-12T04:01:21.622 INFO:tasks.workunit.client.1.smithi033.stdout:1/598: symlink d11/d17/d27/d36/d3f/d69/lca 0 2024-05-12T04:01:21.622 INFO:tasks.workunit.client.1.smithi033.stdout:6/698: dwrite d4/f16 [0,4194304] 0 2024-05-12T04:01:21.627 INFO:tasks.workunit.client.0.smithi032.stdout:0/554: rename df/d10/d48/l61 to df/d30/d8d/d8f/lbe 0 2024-05-12T04:01:21.629 INFO:tasks.workunit.client.0.smithi032.stdout:2/509: fsync d1/d34/d96/f8e 0 2024-05-12T04:01:21.632 INFO:tasks.workunit.client.0.smithi032.stdout:0/555: truncate df/d10/d3b/d75/f78 4849676 0 2024-05-12T04:01:21.633 INFO:tasks.workunit.client.0.smithi032.stdout:2/510: chown d1/d34/d96/d41/d2c/f90 184096 1 2024-05-12T04:01:21.633 INFO:tasks.workunit.client.0.smithi032.stdout:5/586: dwrite d4/d7/d14/d26/d47/f9b [0,4194304] 0 2024-05-12T04:01:21.633 INFO:tasks.workunit.client.0.smithi032.stdout:3/521: unlink d15/d24/d93/f6b 0 2024-05-12T04:01:21.635 INFO:tasks.workunit.client.1.smithi033.stdout:4/674: creat dd/d7a/fd6 x:0 0 0 2024-05-12T04:01:21.635 INFO:tasks.workunit.client.0.smithi032.stdout:4/457: write d0/d39/f6a [203262,22336] 0 2024-05-12T04:01:21.636 INFO:tasks.workunit.client.1.smithi033.stdout:4/675: readlink dd/d7a/d87/d7e/lb1 0 2024-05-12T04:01:21.636 INFO:tasks.workunit.client.1.smithi033.stdout:9/907: mkdir d2/d4/d2b/d78/d76/d13d/d140 0 2024-05-12T04:01:21.637 INFO:tasks.workunit.client.1.smithi033.stdout:4/676: fdatasync fc 0 2024-05-12T04:01:21.639 INFO:tasks.workunit.client.1.smithi033.stdout:3/640: sync 2024-05-12T04:01:21.641 INFO:tasks.workunit.client.0.smithi032.stdout:1/496: rename d1/dd/d24/c98 to d1/d7/d30/d18/ca1 0 2024-05-12T04:01:21.642 INFO:tasks.workunit.client.1.smithi033.stdout:5/563: fdatasync d3/d15/d16/f20 0 2024-05-12T04:01:21.645 INFO:tasks.workunit.client.1.smithi033.stdout:4/677: dread fc [4194304,4194304] 0 2024-05-12T04:01:21.646 INFO:tasks.workunit.client.1.smithi033.stdout:9/908: dwrite d2/d4/d2b/d78/f8c [0,4194304] 0 2024-05-12T04:01:21.647 INFO:tasks.workunit.client.1.smithi033.stdout:9/909: chown d2/d4/d2b/d6d/d3a 1 1 2024-05-12T04:01:21.649 INFO:tasks.workunit.client.1.smithi033.stdout:7/871: rename de/d2b/d6c/cdc to de/d80/c125 0 2024-05-12T04:01:21.651 INFO:tasks.workunit.client.1.smithi033.stdout:8/620: chown d1/d4/d1d/d36/d3c/l56 529471048 1 2024-05-12T04:01:21.652 INFO:tasks.workunit.client.1.smithi033.stdout:0/703: symlink d6/dc/d58/d91/lee 0 2024-05-12T04:01:21.653 INFO:tasks.workunit.client.1.smithi033.stdout:0/704: fdatasync d6/d11/f47 0 2024-05-12T04:01:21.654 INFO:tasks.workunit.client.1.smithi033.stdout:1/599: fsync d11/d1c/d59/f97 0 2024-05-12T04:01:21.655 INFO:tasks.workunit.client.1.smithi033.stdout:6/699: truncate d4/f2f 2491097 0 2024-05-12T04:01:21.658 INFO:tasks.workunit.client.0.smithi032.stdout:2/511: mknod d1/ca9 0 2024-05-12T04:01:21.659 INFO:tasks.workunit.client.0.smithi032.stdout:3/522: dread d15/f54 [0,4194304] 0 2024-05-12T04:01:21.661 INFO:tasks.workunit.client.0.smithi032.stdout:5/587: truncate d4/d7/d14/d26/d33/d4a/f4e 1567086 0 2024-05-12T04:01:21.662 INFO:tasks.workunit.client.1.smithi033.stdout:3/641: write db/d17/d3a/d93/dc8/fda [4799544,37338] 0 2024-05-12T04:01:21.663 INFO:tasks.workunit.client.0.smithi032.stdout:1/497: creat d1/dd/d24/d9b/fa2 x:0 0 0 2024-05-12T04:01:21.670 INFO:tasks.workunit.client.1.smithi033.stdout:4/678: mkdir dd/d11/d5b/d84/dd7 0 2024-05-12T04:01:21.670 INFO:tasks.workunit.client.1.smithi033.stdout:2/649: truncate dc/d17/d32/f43 856053 0 2024-05-12T04:01:21.671 INFO:tasks.workunit.client.1.smithi033.stdout:2/650: chown dc/f31 1 1 2024-05-12T04:01:21.671 INFO:tasks.workunit.client.0.smithi032.stdout:6/479: sync 2024-05-12T04:01:21.671 INFO:tasks.workunit.client.0.smithi032.stdout:9/410: sync 2024-05-12T04:01:21.672 INFO:tasks.workunit.client.1.smithi033.stdout:7/872: dread de/fa1 [0,4194304] 0 2024-05-12T04:01:21.672 INFO:tasks.workunit.client.0.smithi032.stdout:3/523: rmdir d15/d20/d22/d7a 39 2024-05-12T04:01:21.673 INFO:tasks.workunit.client.0.smithi032.stdout:6/480: chown d7/d9/d48/d67/f16 718940216 1 2024-05-12T04:01:21.674 INFO:tasks.workunit.client.1.smithi033.stdout:2/651: dread dc/d17/d27/d2f/f41 [0,4194304] 0 2024-05-12T04:01:21.674 INFO:tasks.workunit.client.0.smithi032.stdout:5/588: symlink d4/d7/d14/d26/d33/d4a/d38/d79/ld1 0 2024-05-12T04:01:21.677 INFO:tasks.workunit.client.1.smithi033.stdout:5/564: write d3/d12/d29/d2b/f6e [844491,9694] 0 2024-05-12T04:01:21.678 INFO:tasks.workunit.client.1.smithi033.stdout:1/600: truncate d11/d1c/d43/f5e 468956 0 2024-05-12T04:01:21.679 INFO:tasks.workunit.client.0.smithi032.stdout:9/411: symlink d0/d3/d4b/d73/l86 0 2024-05-12T04:01:21.681 INFO:tasks.workunit.client.0.smithi032.stdout:9/412: fdatasync d0/d3/d6/d7/d2a/d46/d13/f7c 0 2024-05-12T04:01:21.682 INFO:tasks.workunit.client.1.smithi033.stdout:2/652: dread dc/d17/d27/d2f/d49/f51 [0,4194304] 0 2024-05-12T04:01:21.685 INFO:tasks.workunit.client.0.smithi032.stdout:1/498: dread d1/d7/d30/d18/f41 [0,4194304] 0 2024-05-12T04:01:21.686 INFO:tasks.workunit.client.0.smithi032.stdout:5/589: rmdir d4/d7/d14/d26/d47 39 2024-05-12T04:01:21.689 INFO:tasks.workunit.client.1.smithi033.stdout:3/642: unlink db/d17/d3a/c43 0 2024-05-12T04:01:21.694 INFO:tasks.workunit.client.0.smithi032.stdout:3/524: write d15/d20/d22/f57 [786078,55775] 0 2024-05-12T04:01:21.695 INFO:tasks.workunit.client.1.smithi033.stdout:9/910: link d2/d70/d7b/d94/ff5 d2/d4/d2b/d6d/d2f/df6/d13e/f141 0 2024-05-12T04:01:21.697 INFO:tasks.workunit.client.1.smithi033.stdout:4/679: truncate dd/d7a/d87/d7e/da7/d66/f2d 58668 0 2024-05-12T04:01:21.698 INFO:tasks.workunit.client.1.smithi033.stdout:8/621: symlink d1/d16/d27/d4d/d51/d8e/db6/lc1 0 2024-05-12T04:01:21.699 INFO:tasks.workunit.client.1.smithi033.stdout:7/873: mknod de/d2b/d6c/dbd/d101/c126 0 2024-05-12T04:01:21.700 INFO:tasks.workunit.client.0.smithi032.stdout:3/525: dwrite d15/d24/d2a/d5b/f71 [0,4194304] 0 2024-05-12T04:01:21.700 INFO:tasks.workunit.client.0.smithi032.stdout:1/499: rename d1/dd/d24/d9b/fa2 to d1/d7/d30/d35/fa3 0 2024-05-12T04:01:21.717 INFO:tasks.workunit.client.0.smithi032.stdout:1/500: dread d1/d7/d30/d18/f1a [0,4194304] 0 2024-05-12T04:01:21.718 INFO:tasks.workunit.client.0.smithi032.stdout:1/501: readlink d1/d7/d49/d67/l6d 0 2024-05-12T04:01:21.719 INFO:tasks.workunit.client.1.smithi033.stdout:0/705: fdatasync d6/db/d2c/f48 0 2024-05-12T04:01:21.719 INFO:tasks.workunit.client.0.smithi032.stdout:1/502: read - d1/d7/d49/d79/f8d zero size 2024-05-12T04:01:21.720 INFO:tasks.workunit.client.1.smithi033.stdout:5/565: mkdir d3/d94/d54/d57/d73/d74/d7e/db1 0 2024-05-12T04:01:21.721 INFO:tasks.workunit.client.0.smithi032.stdout:2/512: getdents d1/d34/d96/d41/d39 0 2024-05-12T04:01:21.728 INFO:tasks.workunit.client.0.smithi032.stdout:9/413: mkdir d0/d3/d4b/d85/d87 0 2024-05-12T04:01:21.729 INFO:tasks.workunit.client.1.smithi033.stdout:3/643: unlink db/d17/d4d/c98 0 2024-05-12T04:01:21.740 INFO:tasks.workunit.client.0.smithi032.stdout:1/503: mkdir d1/d6a/d72/da4 0 2024-05-12T04:01:21.740 INFO:tasks.workunit.client.1.smithi033.stdout:7/874: mknod de/df/d7a/c127 0 2024-05-12T04:01:21.742 INFO:tasks.workunit.client.0.smithi032.stdout:6/481: link d7/d9/d48/d67/l47 d7/d9/d48/d67/d1a/d20/la6 0 2024-05-12T04:01:21.743 INFO:tasks.workunit.client.0.smithi032.stdout:6/482: fsync d7/d9/d60/f80 0 2024-05-12T04:01:21.744 INFO:tasks.workunit.client.1.smithi033.stdout:0/706: unlink d6/dc/d6e/fdc 0 2024-05-12T04:01:21.745 INFO:tasks.workunit.client.0.smithi032.stdout:9/414: mkdir d0/d3/d4b/d88 0 2024-05-12T04:01:21.746 INFO:tasks.workunit.client.1.smithi033.stdout:5/566: creat d3/d94/d54/d57/d73/d74/d7e/fb2 x:0 0 0 2024-05-12T04:01:21.747 INFO:tasks.workunit.client.1.smithi033.stdout:6/700: link d4/d1a/d67/lbc d4/d1a/d67/d68/lea 0 2024-05-12T04:01:21.749 INFO:tasks.workunit.client.1.smithi033.stdout:6/701: truncate d4/d6e/fb3 760082 0 2024-05-12T04:01:21.749 INFO:tasks.workunit.client.1.smithi033.stdout:6/702: chown d4/d1a/d67/d68/c77 3238 1 2024-05-12T04:01:21.751 INFO:tasks.workunit.client.0.smithi032.stdout:2/513: write d1/d34/d96/d8a/f9b [298205,47372] 0 2024-05-12T04:01:21.752 INFO:tasks.workunit.client.0.smithi032.stdout:9/415: dwrite d0/d3/d6/d7/da/f1a [8388608,4194304] 0 2024-05-12T04:01:21.754 INFO:tasks.workunit.client.0.smithi032.stdout:1/504: mkdir d1/d25/da5 0 2024-05-12T04:01:21.754 INFO:tasks.workunit.client.0.smithi032.stdout:9/416: read - d0/d3/d6/d7/d27/f7a zero size 2024-05-12T04:01:21.759 INFO:tasks.workunit.client.1.smithi033.stdout:9/911: rename d2/l1e to d2/d4/d2b/d33/l142 0 2024-05-12T04:01:21.761 INFO:tasks.workunit.client.0.smithi032.stdout:6/483: mknod d7/d9/d48/d67/d8d/ca7 0 2024-05-12T04:01:21.765 INFO:tasks.workunit.client.0.smithi032.stdout:5/590: getdents d4/d7/d14/da9 0 2024-05-12T04:01:21.767 INFO:tasks.workunit.client.0.smithi032.stdout:3/526: rename d15/d20/d26/d6a/f8a to d15/d24/d2a/fc1 0 2024-05-12T04:01:21.771 INFO:tasks.workunit.client.0.smithi032.stdout:1/505: unlink d1/d25/f4a 0 2024-05-12T04:01:21.772 INFO:tasks.workunit.client.1.smithi033.stdout:8/622: fsync d1/d16/d27/f86 0 2024-05-12T04:01:21.774 INFO:tasks.workunit.client.1.smithi033.stdout:8/623: write d1/d4/d1d/d36/d3c/da7/f65 [287117,66935] 0 2024-05-12T04:01:21.774 INFO:tasks.workunit.client.0.smithi032.stdout:6/484: write d7/de/f6d [824671,88634] 0 2024-05-12T04:01:21.775 INFO:tasks.workunit.client.1.smithi033.stdout:8/624: chown d1/d16/f75 536371071 1 2024-05-12T04:01:21.775 INFO:tasks.workunit.client.1.smithi033.stdout:5/567: fsync d3/d27/f28 0 2024-05-12T04:01:21.777 INFO:tasks.workunit.client.1.smithi033.stdout:6/703: fdatasync d4/d1a/f91 0 2024-05-12T04:01:21.780 INFO:tasks.workunit.client.1.smithi033.stdout:0/707: dread d6/d11/d57/f63 [0,4194304] 0 2024-05-12T04:01:21.780 INFO:tasks.workunit.client.0.smithi032.stdout:5/591: rename d4/d7/d14/d26/d33/d4a/d38/d3b/d4c to d4/d7/d14/d26/d33/d4a/d38/dd2 0 2024-05-12T04:01:21.780 INFO:tasks.workunit.client.1.smithi033.stdout:6/704: write d4/d1a/d4e/d4f/d94/fb9 [1083080,35056] 0 2024-05-12T04:01:21.782 INFO:tasks.workunit.client.1.smithi033.stdout:1/601: sync 2024-05-12T04:01:21.782 INFO:tasks.workunit.client.1.smithi033.stdout:0/708: write d6/d11/d98/fa1 [2504683,70331] 0 2024-05-12T04:01:21.783 INFO:tasks.workunit.client.1.smithi033.stdout:9/912: rename d2/d4/d29/d3c/le5 to d2/d4/l143 0 2024-05-12T04:01:21.785 INFO:tasks.workunit.client.0.smithi032.stdout:9/417: mkdir d0/d3/d6/d7/da/d89 0 2024-05-12T04:01:21.786 INFO:tasks.workunit.client.1.smithi033.stdout:8/625: dwrite d1/d16/d29/db1/fb5 [0,4194304] 0 2024-05-12T04:01:21.786 INFO:tasks.workunit.client.0.smithi032.stdout:2/514: dread d1/d34/d96/f60 [0,4194304] 0 2024-05-12T04:01:21.788 INFO:tasks.workunit.client.0.smithi032.stdout:6/485: mknod d7/d9/d48/d67/d63/ca8 0 2024-05-12T04:01:21.789 INFO:tasks.workunit.client.1.smithi033.stdout:8/626: write d1/d16/d29/db1/fb5 [1340945,58058] 0 2024-05-12T04:01:21.789 INFO:tasks.workunit.client.0.smithi032.stdout:3/527: mkdir d15/dc2 0 2024-05-12T04:01:21.790 INFO:tasks.workunit.client.1.smithi033.stdout:4/680: creat dd/d11/d5b/fd8 x:0 0 0 2024-05-12T04:01:21.791 INFO:tasks.workunit.client.1.smithi033.stdout:7/875: unlink de/df/d17/d1b/c30 0 2024-05-12T04:01:21.793 INFO:tasks.workunit.client.0.smithi032.stdout:1/506: mknod d1/dd/ca6 0 2024-05-12T04:01:21.793 INFO:tasks.workunit.client.1.smithi033.stdout:6/705: dread d4/f82 [0,4194304] 0 2024-05-12T04:01:21.795 INFO:tasks.workunit.client.1.smithi033.stdout:9/913: dwrite d2/d4/d2b/d78/f8c [0,4194304] 0 2024-05-12T04:01:21.795 INFO:tasks.workunit.client.0.smithi032.stdout:2/515: mknod d1/d34/d96/d41/d52/caa 0 2024-05-12T04:01:21.796 INFO:tasks.workunit.client.0.smithi032.stdout:6/486: fsync d7/d9/d48/d67/f23 0 2024-05-12T04:01:21.799 INFO:tasks.workunit.client.0.smithi032.stdout:6/487: write d7/d9/d48/d67/d25/f3e [4400583,69479] 0 2024-05-12T04:01:21.799 INFO:tasks.workunit.client.0.smithi032.stdout:3/528: chown d15/d20/d22/d7a/fa6 20586 1 2024-05-12T04:01:21.800 INFO:tasks.workunit.client.1.smithi033.stdout:2/653: getdents dc/d17/d74 0 2024-05-12T04:01:21.802 INFO:tasks.workunit.client.0.smithi032.stdout:1/507: unlink d1/d6a/c8e 0 2024-05-12T04:01:21.803 INFO:tasks.workunit.client.0.smithi032.stdout:9/418: mkdir d0/d84/d8a 0 2024-05-12T04:01:21.803 INFO:tasks.workunit.client.1.smithi033.stdout:6/706: dwrite d4/d6e/fc1 [0,4194304] 0 2024-05-12T04:01:21.804 INFO:tasks.workunit.client.0.smithi032.stdout:9/419: write d0/d3/d6/f69 [496215,18570] 0 2024-05-12T04:01:21.806 INFO:tasks.workunit.client.1.smithi033.stdout:1/602: rmdir d11/d1c 39 2024-05-12T04:01:21.810 INFO:tasks.workunit.client.0.smithi032.stdout:6/488: creat d7/d9/d5c/d8b/fa9 x:0 0 0 2024-05-12T04:01:21.811 INFO:tasks.workunit.client.0.smithi032.stdout:1/508: dwrite d1/d7/d30/d35/d54/f94 [0,4194304] 0 2024-05-12T04:01:21.811 INFO:tasks.workunit.client.0.smithi032.stdout:2/516: mknod d1/d73/cab 0 2024-05-12T04:01:21.817 INFO:tasks.workunit.client.0.smithi032.stdout:9/420: creat d0/d3/d4b/d6f/f8b x:0 0 0 2024-05-12T04:01:21.818 INFO:tasks.workunit.client.0.smithi032.stdout:9/421: chown d0/d3/f42 2745 1 2024-05-12T04:01:21.819 INFO:tasks.workunit.client.1.smithi033.stdout:4/681: mkdir dd/d7a/d87/dd9 0 2024-05-12T04:01:21.819 INFO:tasks.workunit.client.1.smithi033.stdout:4/682: dread - dd/d11/da5/fd3 zero size 2024-05-12T04:01:21.820 INFO:tasks.workunit.client.0.smithi032.stdout:9/422: dread d0/d3/d74/f20 [0,4194304] 0 2024-05-12T04:01:21.821 INFO:tasks.workunit.client.1.smithi033.stdout:8/627: creat d1/d16/d27/d4d/d82/fc2 x:0 0 0 2024-05-12T04:01:21.822 INFO:tasks.workunit.client.0.smithi032.stdout:5/592: write d4/d7/d14/d39/d1b/f76 [3863732,111818] 0 2024-05-12T04:01:21.824 INFO:tasks.workunit.client.0.smithi032.stdout:1/509: dread d1/d6a/d72/f88 [0,4194304] 0 2024-05-12T04:01:21.824 INFO:tasks.workunit.client.1.smithi033.stdout:7/876: chown de/d2b/d6c/dbd/d101/c102 352284 1 2024-05-12T04:01:21.825 INFO:tasks.workunit.client.0.smithi032.stdout:5/593: write d4/d7/d14/d26/d33/d4a/d38/d3b/f48 [4676986,12665] 0 2024-05-12T04:01:21.826 INFO:tasks.workunit.client.1.smithi033.stdout:7/877: write de/d2b/d6c/db5/dd0/ff2 [1018285,52160] 0 2024-05-12T04:01:21.827 INFO:tasks.workunit.client.1.smithi033.stdout:5/568: mkdir d3/d15/daa/db3 0 2024-05-12T04:01:21.828 INFO:tasks.workunit.client.1.smithi033.stdout:5/569: chown d3/d15/d16/d45/d62/fad 1 1 2024-05-12T04:01:21.828 INFO:tasks.workunit.client.1.smithi033.stdout:5/570: chown d3/d15/daa 1 1 2024-05-12T04:01:21.834 INFO:tasks.workunit.client.1.smithi033.stdout:2/654: rmdir dc/d13/d1c 39 2024-05-12T04:01:21.834 INFO:tasks.workunit.client.0.smithi032.stdout:3/529: write d15/d33/f74 [586316,37111] 0 2024-05-12T04:01:21.838 INFO:tasks.workunit.client.0.smithi032.stdout:2/517: mknod d1/d34/d4a/d86/cac 0 2024-05-12T04:01:21.839 INFO:tasks.workunit.client.0.smithi032.stdout:5/594: dread d4/d7/d14/d26/d33/d4a/f86 [0,4194304] 0 2024-05-12T04:01:21.841 INFO:tasks.workunit.client.0.smithi032.stdout:1/510: unlink d1/d7/d30/d18/c5f 0 2024-05-12T04:01:21.843 INFO:tasks.workunit.client.1.smithi033.stdout:1/603: creat d11/d17/d27/d36/d3f/d69/d87/d96/fcb x:0 0 0 2024-05-12T04:01:21.844 INFO:tasks.workunit.client.1.smithi033.stdout:1/604: chown d11/d17/f6d 232387603 1 2024-05-12T04:01:21.845 INFO:tasks.workunit.client.0.smithi032.stdout:1/511: read d1/d7/d30/d35/f48 [160347,108126] 0 2024-05-12T04:01:21.846 INFO:tasks.workunit.client.1.smithi033.stdout:0/709: symlink d6/d11/lef 0 2024-05-12T04:01:21.849 INFO:tasks.workunit.client.1.smithi033.stdout:3/644: creat db/d17/d3a/dc0/fdf x:0 0 0 2024-05-12T04:01:21.852 INFO:tasks.workunit.client.1.smithi033.stdout:5/571: dread d3/d94/f3e [0,4194304] 0 2024-05-12T04:01:21.852 INFO:tasks.workunit.client.0.smithi032.stdout:2/518: creat d1/d4d/fad x:0 0 0 2024-05-12T04:01:21.853 INFO:tasks.workunit.client.1.smithi033.stdout:5/572: dread - d3/d27/fa8 zero size 2024-05-12T04:01:21.854 INFO:tasks.workunit.client.1.smithi033.stdout:8/628: fdatasync d1/d4/d1d/d36/f74 0 2024-05-12T04:01:21.855 INFO:tasks.workunit.client.1.smithi033.stdout:6/707: dread d4/d1a/d67/d68/f6a [0,4194304] 0 2024-05-12T04:01:21.855 INFO:tasks.workunit.client.1.smithi033.stdout:8/629: readlink d1/d4/d46/l6e 0 2024-05-12T04:01:21.856 INFO:tasks.workunit.client.1.smithi033.stdout:8/630: chown d1/d16/c95 28207121 1 2024-05-12T04:01:21.857 INFO:tasks.workunit.client.1.smithi033.stdout:7/878: creat de/df/d60/f128 x:0 0 0 2024-05-12T04:01:21.864 INFO:tasks.workunit.client.0.smithi032.stdout:3/530: write d15/f4d [1049123,48009] 0 2024-05-12T04:01:21.866 INFO:tasks.workunit.client.0.smithi032.stdout:3/531: read d15/d20/d22/d7a/fa6 [24987,18249] 0 2024-05-12T04:01:21.869 INFO:tasks.workunit.client.1.smithi033.stdout:9/914: truncate d2/d4/d2b/d78/f8c 3926182 0 2024-05-12T04:01:21.870 INFO:tasks.workunit.client.0.smithi032.stdout:5/595: dwrite d4/d7/d14/d39/fcf [0,4194304] 0 2024-05-12T04:01:21.874 INFO:tasks.workunit.client.0.smithi032.stdout:9/423: getdents d0 0 2024-05-12T04:01:21.875 INFO:tasks.workunit.client.1.smithi033.stdout:5/573: creat d3/d12/d29/d2b/d4f/fb4 x:0 0 0 2024-05-12T04:01:21.877 INFO:tasks.workunit.client.0.smithi032.stdout:1/512: link d1/d7/d30/d35/fa3 d1/d7/d30/d35/d54/d7e/fa7 0 2024-05-12T04:01:21.878 INFO:tasks.workunit.client.0.smithi032.stdout:1/513: readlink d1/d7/d49/d67/l7b 0 2024-05-12T04:01:21.881 INFO:tasks.workunit.client.1.smithi033.stdout:8/631: mkdir d1/d4/d1d/d36/d3c/d47/dc3 0 2024-05-12T04:01:21.881 INFO:tasks.workunit.client.1.smithi033.stdout:4/683: dwrite dd/d7a/d87/d7e/da7/d38/f44 [8388608,4194304] 0 2024-05-12T04:01:21.882 INFO:tasks.workunit.client.0.smithi032.stdout:2/519: mkdir d1/d13/da0/dae 0 2024-05-12T04:01:21.883 INFO:tasks.workunit.client.0.smithi032.stdout:2/520: fdatasync d1/d4d/f98 0 2024-05-12T04:01:21.884 INFO:tasks.workunit.client.0.smithi032.stdout:2/521: chown d1/d34/d96/d41/f2a 9 1 2024-05-12T04:01:21.885 INFO:tasks.workunit.client.1.smithi033.stdout:2/655: mkdir dc/d13/d65/db1/dd0 0 2024-05-12T04:01:21.887 INFO:tasks.workunit.client.0.smithi032.stdout:5/596: rename d4/d7/d14/d26/d33/d5f/fb2 to d4/d7/d14/db5/fd3 0 2024-05-12T04:01:21.889 INFO:tasks.workunit.client.1.smithi033.stdout:1/605: symlink d11/d17/d27/d36/lcc 0 2024-05-12T04:01:21.890 INFO:tasks.workunit.client.0.smithi032.stdout:9/424: symlink d0/d3/d4b/l8c 0 2024-05-12T04:01:21.898 INFO:tasks.workunit.client.0.smithi032.stdout:3/532: creat d15/dc2/fc3 x:0 0 0 2024-05-12T04:01:21.900 INFO:tasks.workunit.client.0.smithi032.stdout:3/533: write d15/d20/d22/fb6 [495681,127702] 0 2024-05-12T04:01:21.900 INFO:tasks.workunit.client.1.smithi033.stdout:9/915: dread d2/d4/d2b/d6d/d3a/f65 [0,4194304] 0 2024-05-12T04:01:21.904 INFO:tasks.workunit.client.0.smithi032.stdout:5/597: dread d4/d7/d14/f1d [0,4194304] 0 2024-05-12T04:01:21.904 INFO:tasks.workunit.client.0.smithi032.stdout:2/522: fdatasync d1/d34/d4a/f6d 0 2024-05-12T04:01:21.906 INFO:tasks.workunit.client.1.smithi033.stdout:9/916: dwrite d2/d4/d2b/d78/f114 [0,4194304] 0 2024-05-12T04:01:21.911 INFO:tasks.workunit.client.0.smithi032.stdout:9/425: symlink d0/d3/d6/d7/d2a/d46/d13/l8d 0 2024-05-12T04:01:21.911 INFO:tasks.workunit.client.1.smithi033.stdout:0/710: dwrite d6/dc/f18 [0,4194304] 0 2024-05-12T04:01:21.918 INFO:tasks.workunit.client.0.smithi032.stdout:3/534: rmdir d15/d24/d2a/d5b 39 2024-05-12T04:01:21.919 INFO:tasks.workunit.client.0.smithi032.stdout:3/535: write d15/d33/f9e [1780261,129645] 0 2024-05-12T04:01:21.921 INFO:tasks.workunit.client.0.smithi032.stdout:2/523: unlink d1/d13/d70/fa8 0 2024-05-12T04:01:21.923 INFO:tasks.workunit.client.0.smithi032.stdout:5/598: rename d4/d7/d14/d26/d33/d4a/l62 to d4/d7/d14/d26/d33/d4a/d38/d3b/d68/ld4 0 2024-05-12T04:01:21.928 INFO:tasks.workunit.client.1.smithi033.stdout:5/574: dread d3/d15/d16/f20 [4194304,4194304] 0 2024-05-12T04:01:21.937 INFO:tasks.workunit.client.0.smithi032.stdout:9/426: dread d0/d3/d6/d58/f66 [0,4194304] 0 2024-05-12T04:01:21.938 INFO:tasks.workunit.client.0.smithi032.stdout:7/476: sync 2024-05-12T04:01:21.944 INFO:tasks.workunit.client.1.smithi033.stdout:4/684: write dd/d7a/d87/d7e/da7/d66/f7f [64922,88757] 0 2024-05-12T04:01:21.950 INFO:tasks.workunit.client.1.smithi033.stdout:6/708: rename d4/d1a/d4e/c9f to d4/ceb 0 2024-05-12T04:01:21.951 INFO:tasks.workunit.client.1.smithi033.stdout:1/606: dwrite d11/d17/f35 [4194304,4194304] 0 2024-05-12T04:01:21.951 INFO:tasks.workunit.client.1.smithi033.stdout:0/711: mknod d6/dc/d96/d9d/cf0 0 2024-05-12T04:01:21.952 INFO:tasks.workunit.client.1.smithi033.stdout:8/632: mknod d1/d16/d29/d63/da3/dba/cc4 0 2024-05-12T04:01:21.954 INFO:tasks.workunit.client.1.smithi033.stdout:7/879: link de/df/d17/d1e/l10a de/d2b/d6c/da0/da6/l129 0 2024-05-12T04:01:21.956 INFO:tasks.workunit.client.0.smithi032.stdout:9/427: symlink d0/d84/l8e 0 2024-05-12T04:01:21.958 INFO:tasks.workunit.client.0.smithi032.stdout:7/477: mkdir dd/d7b/d9a 0 2024-05-12T04:01:21.960 INFO:tasks.workunit.client.0.smithi032.stdout:7/478: chown dd/d1b/d1e/l8a 23647 1 2024-05-12T04:01:21.960 INFO:tasks.workunit.client.1.smithi033.stdout:2/656: mkdir dc/d17/d27/d68/dd1 0 2024-05-12T04:01:21.961 INFO:tasks.workunit.client.0.smithi032.stdout:3/536: dread d15/d20/d26/fa5 [0,4194304] 0 2024-05-12T04:01:21.962 INFO:tasks.workunit.client.0.smithi032.stdout:1/514: getdents d1/d6a 0 2024-05-12T04:01:21.962 INFO:tasks.workunit.client.0.smithi032.stdout:7/479: read dd/fe [1562793,104652] 0 2024-05-12T04:01:21.963 INFO:tasks.workunit.client.0.smithi032.stdout:3/537: readlink d15/d20/d22/d7a/la9 0 2024-05-12T04:01:21.964 INFO:tasks.workunit.client.0.smithi032.stdout:2/524: mknod d1/d34/da5/caf 0 2024-05-12T04:01:21.964 INFO:tasks.workunit.client.1.smithi033.stdout:4/685: read dd/d7a/f85 [452576,56405] 0 2024-05-12T04:01:21.965 INFO:tasks.workunit.client.1.smithi033.stdout:4/686: readlink dd/d7a/l7b 0 2024-05-12T04:01:21.966 INFO:tasks.workunit.client.1.smithi033.stdout:9/917: creat d2/d4/d2b/d33/dd2/d132/f144 x:0 0 0 2024-05-12T04:01:21.968 INFO:tasks.workunit.client.1.smithi033.stdout:7/880: dread de/d20/d38/f9e [0,4194304] 0 2024-05-12T04:01:21.969 INFO:tasks.workunit.client.0.smithi032.stdout:3/538: dread d15/d20/d22/fb6 [0,4194304] 0 2024-05-12T04:01:21.970 INFO:tasks.workunit.client.0.smithi032.stdout:5/599: dwrite d4/d7/d14/d26/d33/d4a/d38/d79/fc7 [0,4194304] 0 2024-05-12T04:01:21.970 INFO:tasks.workunit.client.0.smithi032.stdout:2/525: dread d1/d34/d3e/d67/f83 [0,4194304] 0 2024-05-12T04:01:21.972 INFO:tasks.workunit.client.0.smithi032.stdout:7/480: fsync dd/d1b/d43/d5e/f34 0 2024-05-12T04:01:21.976 INFO:tasks.workunit.client.0.smithi032.stdout:9/428: mknod d0/d3/c8f 0 2024-05-12T04:01:21.978 INFO:tasks.workunit.client.0.smithi032.stdout:1/515: dread d1/dd/f53 [0,4194304] 0 2024-05-12T04:01:21.981 INFO:tasks.workunit.client.1.smithi033.stdout:5/575: creat d3/d94/d54/d57/d73/d74/d7e/db1/fb5 x:0 0 0 2024-05-12T04:01:21.981 INFO:tasks.workunit.client.0.smithi032.stdout:9/429: dread d0/d3/d6/d7/d2a/f68 [0,4194304] 0 2024-05-12T04:01:21.984 INFO:tasks.workunit.client.0.smithi032.stdout:3/539: read fd [257708,44270] 0 2024-05-12T04:01:21.986 INFO:tasks.workunit.client.1.smithi033.stdout:2/657: dread dc/d13/d1c/d98/fb5 [0,4194304] 0 2024-05-12T04:01:21.986 INFO:tasks.workunit.client.0.smithi032.stdout:7/481: fdatasync dd/d1b/d1e/f28 0 2024-05-12T04:01:21.989 INFO:tasks.workunit.client.0.smithi032.stdout:9/430: dwrite d0/d3/f76 [0,4194304] 0 2024-05-12T04:01:21.990 INFO:tasks.workunit.client.0.smithi032.stdout:5/600: rename d4/d7/d58/l9c to d4/d7/d14/d26/d33/d5f/ld5 0 2024-05-12T04:01:21.990 INFO:tasks.workunit.client.1.smithi033.stdout:8/633: dwrite d1/f13 [0,4194304] 0 2024-05-12T04:01:21.991 INFO:tasks.workunit.client.1.smithi033.stdout:7/881: rmdir de/d2b/d6c/db5 39 2024-05-12T04:01:21.991 INFO:tasks.workunit.client.0.smithi032.stdout:9/431: write d0/d3/d6/d7/da/f72 [652430,123229] 0 2024-05-12T04:01:21.995 INFO:tasks.workunit.client.1.smithi033.stdout:0/712: mkdir d6/dc/df1 0 2024-05-12T04:01:21.996 INFO:tasks.workunit.client.0.smithi032.stdout:2/526: truncate d1/d34/d96/f8e 1973583 0 2024-05-12T04:01:21.999 INFO:tasks.workunit.client.0.smithi032.stdout:1/516: creat d1/d7/d30/d18/d93/fa8 x:0 0 0 2024-05-12T04:01:22.000 INFO:tasks.workunit.client.0.smithi032.stdout:2/527: write d1/d13/d70/d85/f9c [1777809,43916] 0 2024-05-12T04:01:22.005 INFO:tasks.workunit.client.1.smithi033.stdout:3/645: rename db/d17/d3e/d65/d91/cdd to db/d17/d25/d32/ce0 0 2024-05-12T04:01:22.006 INFO:tasks.workunit.client.0.smithi032.stdout:7/482: creat dd/d1b/d1e/d45/d55/d60/d75/f9b x:0 0 0 2024-05-12T04:01:22.009 INFO:tasks.workunit.client.1.smithi033.stdout:4/687: symlink dd/d7a/d87/d7e/da7/d66/d42/d59/dd0/lda 0 2024-05-12T04:01:22.010 INFO:tasks.workunit.client.0.smithi032.stdout:2/528: dwrite d1/d34/d3e/d67/f83 [0,4194304] 0 2024-05-12T04:01:22.011 INFO:tasks.workunit.client.1.smithi033.stdout:4/688: write dd/d11/d75/fb4 [818651,30917] 0 2024-05-12T04:01:22.019 INFO:tasks.workunit.client.1.smithi033.stdout:2/658: creat dc/d3b/d88/fd2 x:0 0 0 2024-05-12T04:01:22.020 INFO:tasks.workunit.client.0.smithi032.stdout:9/432: symlink d0/d3/d4b/d73/l90 0 2024-05-12T04:01:22.020 INFO:tasks.workunit.client.1.smithi033.stdout:5/576: dread d3/d15/d16/d45/d4b/f6c [0,4194304] 0 2024-05-12T04:01:22.026 INFO:tasks.workunit.client.1.smithi033.stdout:7/882: dread de/df/d17/d1e/d24/fc5 [0,4194304] 0 2024-05-12T04:01:22.027 INFO:tasks.workunit.client.1.smithi033.stdout:7/883: chown de/df/d60/lb8 520383 1 2024-05-12T04:01:22.028 INFO:tasks.workunit.client.0.smithi032.stdout:3/540: mknod d15/d24/cc4 0 2024-05-12T04:01:22.030 INFO:tasks.workunit.client.1.smithi033.stdout:0/713: creat d6/db/d2c/d73/daf/da4/ff2 x:0 0 0 2024-05-12T04:01:22.032 INFO:tasks.workunit.client.1.smithi033.stdout:1/607: write d11/d17/d38/f71 [49410,6020] 0 2024-05-12T04:01:22.037 INFO:tasks.workunit.client.0.smithi032.stdout:9/433: dread d0/d3/ff [0,4194304] 0 2024-05-12T04:01:22.037 INFO:tasks.workunit.client.1.smithi033.stdout:3/646: chown db/d17/l38 105455453 1 2024-05-12T04:01:22.038 INFO:tasks.workunit.client.0.smithi032.stdout:9/434: dread - d0/d3/d6/d58/f5e zero size 2024-05-12T04:01:22.039 INFO:tasks.workunit.client.0.smithi032.stdout:7/483: write dd/d1b/f7a [247195,73333] 0 2024-05-12T04:01:22.041 INFO:tasks.workunit.client.0.smithi032.stdout:2/529: mknod d1/d13/da0/cb0 0 2024-05-12T04:01:22.045 INFO:tasks.workunit.client.0.smithi032.stdout:2/530: write d1/d4d/f98 [21892,112689] 0 2024-05-12T04:01:22.050 INFO:tasks.workunit.client.1.smithi033.stdout:8/634: dwrite d1/d16/d27/d28/f76 [0,4194304] 0 2024-05-12T04:01:22.055 INFO:tasks.workunit.client.0.smithi032.stdout:8/516: sync 2024-05-12T04:01:22.060 INFO:tasks.workunit.client.0.smithi032.stdout:9/435: mkdir d0/d3/d6/d7/d2a/d46/d91 0 2024-05-12T04:01:22.060 INFO:tasks.workunit.client.0.smithi032.stdout:7/484: rmdir dd/d1b 39 2024-05-12T04:01:22.062 INFO:tasks.workunit.client.1.smithi033.stdout:1/608: dread - d11/d17/d27/d36/d46/f92 zero size 2024-05-12T04:01:22.062 INFO:tasks.workunit.client.0.smithi032.stdout:9/436: stat d0/d3/d6/d7/da/l26 0 2024-05-12T04:01:22.064 INFO:tasks.workunit.client.0.smithi032.stdout:9/437: chown d0/d3/d6/d7/da/f1c 7583 1 2024-05-12T04:01:22.069 INFO:tasks.workunit.client.1.smithi033.stdout:2/659: symlink dc/d17/d27/ld3 0 2024-05-12T04:01:22.075 INFO:tasks.workunit.client.0.smithi032.stdout:5/601: getdents d4/d7/d14/db5 0 2024-05-12T04:01:22.078 INFO:tasks.workunit.client.0.smithi032.stdout:9/438: dread d0/d3/d6/f62 [0,4194304] 0 2024-05-12T04:01:22.078 INFO:tasks.workunit.client.1.smithi033.stdout:4/689: dwrite dd/d11/d75/fa2 [0,4194304] 0 2024-05-12T04:01:22.079 INFO:tasks.workunit.client.1.smithi033.stdout:4/690: write dd/d11/d5b/fd8 [90011,115056] 0 2024-05-12T04:01:22.082 INFO:tasks.workunit.client.1.smithi033.stdout:0/714: unlink d6/f23 0 2024-05-12T04:01:22.083 INFO:tasks.workunit.client.0.smithi032.stdout:1/517: dwrite d1/d25/f64 [0,4194304] 0 2024-05-12T04:01:22.086 INFO:tasks.workunit.client.1.smithi033.stdout:1/609: rename d11/d17/d27/d36/d46/f92 to d11/d17/d27/d36/d3f/d69/d87/d96/fcd 0 2024-05-12T04:01:22.087 INFO:tasks.workunit.client.0.smithi032.stdout:7/485: mkdir dd/d7b/d9a/d9c 0 2024-05-12T04:01:22.088 INFO:tasks.workunit.client.0.smithi032.stdout:8/517: write d0/d9/f12 [4832578,9864] 0 2024-05-12T04:01:22.092 INFO:tasks.workunit.client.0.smithi032.stdout:5/602: unlink d4/d7/d14/d26/d3c/l50 0 2024-05-12T04:01:22.093 INFO:tasks.workunit.client.1.smithi033.stdout:2/660: creat dc/d3b/d88/fd4 x:0 0 0 2024-05-12T04:01:22.094 INFO:tasks.workunit.client.0.smithi032.stdout:9/439: mknod d0/d3/d1b/c92 0 2024-05-12T04:01:22.094 INFO:tasks.workunit.client.1.smithi033.stdout:5/577: link d3/c79 d3/d94/d85/cb6 0 2024-05-12T04:01:22.095 INFO:tasks.workunit.client.1.smithi033.stdout:7/884: link de/d20/d64/c73 de/df/d17/d1b/d36/c12a 0 2024-05-12T04:01:22.097 INFO:tasks.workunit.client.0.smithi032.stdout:3/541: getdents d15/d1b 0 2024-05-12T04:01:22.099 INFO:tasks.workunit.client.1.smithi033.stdout:8/635: link d1/d4/d1d/d36/d3c/d47/f60 d1/d16/d27/d7c/fc5 0 2024-05-12T04:01:22.100 INFO:tasks.workunit.client.1.smithi033.stdout:4/691: truncate dd/f2a 1886306 0 2024-05-12T04:01:22.101 INFO:tasks.workunit.client.0.smithi032.stdout:9/440: dwrite d0/d3/d6/d7/da/f72 [0,4194304] 0 2024-05-12T04:01:22.101 INFO:tasks.workunit.client.0.smithi032.stdout:1/518: unlink d1/d7/d49/d67/l6d 0 2024-05-12T04:01:22.103 INFO:tasks.workunit.client.0.smithi032.stdout:1/519: write d1/f3 [1824875,22827] 0 2024-05-12T04:01:22.105 INFO:tasks.workunit.client.1.smithi033.stdout:4/692: dread dd/d7a/d87/d7e/da7/d66/d42/f82 [0,4194304] 0 2024-05-12T04:01:22.106 INFO:tasks.workunit.client.0.smithi032.stdout:7/486: readlink dd/d1b/d1e/l29 0 2024-05-12T04:01:22.110 INFO:tasks.workunit.client.1.smithi033.stdout:0/715: dwrite d6/d11/d57/f21 [4194304,4194304] 0 2024-05-12T04:01:22.110 INFO:tasks.workunit.client.0.smithi032.stdout:9/441: dwrite d0/d3/d6/d7/d27/f7a [0,4194304] 0 2024-05-12T04:01:22.119 INFO:tasks.workunit.client.1.smithi033.stdout:2/661: mkdir dc/d13/d39/d8f/d9e/dd5 0 2024-05-12T04:01:22.122 INFO:tasks.workunit.client.1.smithi033.stdout:2/662: dread dc/d13/d1f/f9b [0,4194304] 0 2024-05-12T04:01:22.123 INFO:tasks.workunit.client.1.smithi033.stdout:5/578: rename d3/d12/ca1 to d3/d15/d8c/cb7 0 2024-05-12T04:01:22.124 INFO:tasks.workunit.client.1.smithi033.stdout:5/579: fdatasync d3/d94/f49 0 2024-05-12T04:01:22.125 INFO:tasks.workunit.client.1.smithi033.stdout:5/580: readlink d3/d94/d85/d99/da2/lae 0 2024-05-12T04:01:22.127 INFO:tasks.workunit.client.1.smithi033.stdout:6/709: sync 2024-05-12T04:01:22.127 INFO:tasks.workunit.client.1.smithi033.stdout:9/918: sync 2024-05-12T04:01:22.128 INFO:tasks.workunit.client.0.smithi032.stdout:1/520: rename d1/d7/d49/d67 to d1/d6a/d72/da9 0 2024-05-12T04:01:22.128 INFO:tasks.workunit.client.1.smithi033.stdout:3/647: truncate db/d17/d4d/fad 2490660 0 2024-05-12T04:01:22.129 INFO:tasks.workunit.client.1.smithi033.stdout:7/885: creat de/d2b/f12b x:0 0 0 2024-05-12T04:01:22.130 INFO:tasks.workunit.client.1.smithi033.stdout:7/886: readlink de/df/d17/l33 0 2024-05-12T04:01:22.131 INFO:tasks.workunit.client.1.smithi033.stdout:7/887: fdatasync de/d2b/d6c/de1/f10d 0 2024-05-12T04:01:22.132 INFO:tasks.workunit.client.0.smithi032.stdout:7/487: creat dd/d1b/d1e/d45/d55/f9d x:0 0 0 2024-05-12T04:01:22.133 INFO:tasks.workunit.client.0.smithi032.stdout:5/603: mkdir d4/d7/d14/d39/d2b/dd6 0 2024-05-12T04:01:22.136 INFO:tasks.workunit.client.1.smithi033.stdout:4/693: symlink dd/d69/ldb 0 2024-05-12T04:01:22.138 INFO:tasks.workunit.client.0.smithi032.stdout:3/542: dread - d15/d24/d2a/d5b/f8e zero size 2024-05-12T04:01:22.144 INFO:tasks.workunit.client.0.smithi032.stdout:7/488: symlink dd/d1b/d1e/l9e 0 2024-05-12T04:01:22.151 INFO:tasks.workunit.client.0.smithi032.stdout:5/604: creat d4/d7/d14/d26/d33/d4a/d38/fd7 x:0 0 0 2024-05-12T04:01:22.153 INFO:tasks.workunit.client.1.smithi033.stdout:2/663: symlink dc/d13/d39/d8f/d9e/ld6 0 2024-05-12T04:01:22.159 INFO:tasks.workunit.client.1.smithi033.stdout:5/581: mkdir d3/d94/db8 0 2024-05-12T04:01:22.160 INFO:tasks.workunit.client.0.smithi032.stdout:9/442: link d0/d3/d6/d7/c7b d0/d3/d4b/d88/c93 0 2024-05-12T04:01:22.167 INFO:tasks.workunit.client.1.smithi033.stdout:9/919: mkdir d2/d4/d2b/d49/d104/d145 0 2024-05-12T04:01:22.167 INFO:tasks.workunit.client.0.smithi032.stdout:1/521: dwrite d1/d7/d30/d35/fa3 [0,4194304] 0 2024-05-12T04:01:22.168 INFO:tasks.workunit.client.1.smithi033.stdout:2/664: dread dc/d13/d1c/f25 [0,4194304] 0 2024-05-12T04:01:22.168 INFO:tasks.workunit.client.0.smithi032.stdout:1/522: readlink d1/d7/d30/l8f 0 2024-05-12T04:01:22.169 INFO:tasks.workunit.client.0.smithi032.stdout:3/543: mknod d15/d20/d8b/d9b/dbb/cc5 0 2024-05-12T04:01:22.171 INFO:tasks.workunit.client.0.smithi032.stdout:3/544: readlink d15/d20/d22/d7a/lb9 0 2024-05-12T04:01:22.175 INFO:tasks.workunit.client.0.smithi032.stdout:3/545: truncate d15/d33/f9e 2467718 0 2024-05-12T04:01:22.177 INFO:tasks.workunit.client.0.smithi032.stdout:5/605: dwrite d4/d7/d14/d39/fcf [4194304,4194304] 0 2024-05-12T04:01:22.185 INFO:tasks.workunit.client.1.smithi033.stdout:1/610: creat d11/d17/d27/d36/d3f/d69/d87/fce x:0 0 0 2024-05-12T04:01:22.186 INFO:tasks.workunit.client.0.smithi032.stdout:9/443: dread d0/d3/d74/f4e [0,4194304] 0 2024-05-12T04:01:22.186 INFO:tasks.workunit.client.0.smithi032.stdout:1/523: rename d1/d7/cc to d1/d7/caa 0 2024-05-12T04:01:22.187 INFO:tasks.workunit.client.0.smithi032.stdout:1/524: fdatasync d1/f47 0 2024-05-12T04:01:22.187 INFO:tasks.workunit.client.1.smithi033.stdout:8/636: dwrite d1/d4/d1d/d36/d3c/da7/f43 [0,4194304] 0 2024-05-12T04:01:22.188 INFO:tasks.workunit.client.1.smithi033.stdout:4/694: mknod dd/d69/cdc 0 2024-05-12T04:01:22.189 INFO:tasks.workunit.client.0.smithi032.stdout:1/525: write d1/d7/d30/d35/d54/f94 [4603691,101784] 0 2024-05-12T04:01:22.189 INFO:tasks.workunit.client.1.smithi033.stdout:8/637: fsync d1/d4/d1d/d36/d3c/da7/f65 0 2024-05-12T04:01:22.190 INFO:tasks.workunit.client.1.smithi033.stdout:8/638: stat d1/d16/c1f 0 2024-05-12T04:01:22.190 INFO:tasks.workunit.client.0.smithi032.stdout:1/526: readlink d1/d7/d30/d35/d77/l80 0 2024-05-12T04:01:22.191 INFO:tasks.workunit.client.0.smithi032.stdout:0/556: sync 2024-05-12T04:01:22.191 INFO:tasks.workunit.client.0.smithi032.stdout:4/458: sync 2024-05-12T04:01:22.191 INFO:tasks.workunit.client.1.smithi033.stdout:8/639: truncate d1/d4/db2/fbb 491699 0 2024-05-12T04:01:22.194 INFO:tasks.workunit.client.0.smithi032.stdout:3/546: dread - d15/d24/f7b zero size 2024-05-12T04:01:22.197 INFO:tasks.workunit.client.0.smithi032.stdout:9/444: dread d0/d3/d6/d7/d27/f31 [0,4194304] 0 2024-05-12T04:01:22.201 INFO:tasks.workunit.client.0.smithi032.stdout:7/489: mkdir dd/d1b/d1e/d45/d55/d60/d9f 0 2024-05-12T04:01:22.202 INFO:tasks.workunit.client.0.smithi032.stdout:1/527: dwrite d1/d7/d30/d18/f76 [0,4194304] 0 2024-05-12T04:01:22.203 INFO:tasks.workunit.client.1.smithi033.stdout:2/665: mkdir dc/d13/d39/d8f/d9e/dd7 0 2024-05-12T04:01:22.207 INFO:tasks.workunit.client.1.smithi033.stdout:7/888: symlink de/d20/deb/l12c 0 2024-05-12T04:01:22.208 INFO:tasks.workunit.client.0.smithi032.stdout:0/557: symlink df/d10/d3d/d83/d9f/lbf 0 2024-05-12T04:01:22.210 INFO:tasks.workunit.client.1.smithi033.stdout:4/695: rmdir dd/d7a/d87/d7e 39 2024-05-12T04:01:22.211 INFO:tasks.workunit.client.0.smithi032.stdout:3/547: chown d15/d24/d40 0 1 2024-05-12T04:01:22.214 INFO:tasks.workunit.client.0.smithi032.stdout:5/606: rename d4/d7/d14/d26/d33/d4a/d38/f74 to d4/d7/d14/d39/d1b/fd8 0 2024-05-12T04:01:22.215 INFO:tasks.workunit.client.0.smithi032.stdout:9/445: dwrite d0/d3/d4b/d6f/f71 [0,4194304] 0 2024-05-12T04:01:22.216 INFO:tasks.workunit.client.1.smithi033.stdout:5/582: dread d3/d15/f30 [0,4194304] 0 2024-05-12T04:01:22.219 INFO:tasks.workunit.client.0.smithi032.stdout:4/459: dwrite d0/f65 [0,4194304] 0 2024-05-12T04:01:22.220 INFO:tasks.workunit.client.0.smithi032.stdout:4/460: fdatasync d0/d3/d24/f76 0 2024-05-12T04:01:22.221 INFO:tasks.workunit.client.0.smithi032.stdout:9/446: write d0/d3/d6/d58/f5a [1888514,89182] 0 2024-05-12T04:01:22.225 INFO:tasks.workunit.client.1.smithi033.stdout:8/640: creat d1/d16/d27/d7c/fc6 x:0 0 0 2024-05-12T04:01:22.226 INFO:tasks.workunit.client.1.smithi033.stdout:8/641: chown d1/d16/d27/d4d/d82/fc2 32847 1 2024-05-12T04:01:22.227 INFO:tasks.workunit.client.0.smithi032.stdout:7/490: chown dd/d1b/d43/d5e/d37/f39 0 1 2024-05-12T04:01:22.229 INFO:tasks.workunit.client.0.smithi032.stdout:1/528: dread d1/d6a/f84 [0,4194304] 0 2024-05-12T04:01:22.230 INFO:tasks.workunit.client.0.smithi032.stdout:1/529: readlink d1/d7/d30/d18/l37 0 2024-05-12T04:01:22.232 INFO:tasks.workunit.client.0.smithi032.stdout:0/558: mkdir df/d10/d3d/d4a/d84/dc0 0 2024-05-12T04:01:22.232 INFO:tasks.workunit.client.1.smithi033.stdout:9/920: dwrite d2/d4/d2b/d6d/d7a/f90 [0,4194304] 0 2024-05-12T04:01:22.237 INFO:tasks.workunit.client.0.smithi032.stdout:5/607: creat d4/d7/d14/d26/d33/d4a/d38/dd2/fd9 x:0 0 0 2024-05-12T04:01:22.239 INFO:tasks.workunit.client.1.smithi033.stdout:4/696: rmdir dd/d11/da5 39 2024-05-12T04:01:22.248 INFO:tasks.workunit.client.1.smithi033.stdout:5/583: creat d3/d94/d54/d57/d73/d74/d7e/db1/fb9 x:0 0 0 2024-05-12T04:01:22.250 INFO:tasks.workunit.client.1.smithi033.stdout:0/716: fdatasync d6/dc/d58/d6a/f89 0 2024-05-12T04:01:22.252 INFO:tasks.workunit.client.1.smithi033.stdout:6/710: getdents d4/d5/dcd 0 2024-05-12T04:01:22.254 INFO:tasks.workunit.client.0.smithi032.stdout:4/461: dread d0/d3/fd [0,4194304] 0 2024-05-12T04:01:22.255 INFO:tasks.workunit.client.1.smithi033.stdout:8/642: rename d1/d4/c6 to d1/d16/d29/db1/cc7 0 2024-05-12T04:01:22.255 INFO:tasks.workunit.client.0.smithi032.stdout:1/530: fsync d1/d7/d30/d18/f41 0 2024-05-12T04:01:22.257 INFO:tasks.workunit.client.1.smithi033.stdout:2/666: creat dc/d17/d27/d68/dd1/fd8 x:0 0 0 2024-05-12T04:01:22.257 INFO:tasks.workunit.client.0.smithi032.stdout:1/531: dread d1/d6a/d72/f88 [0,4194304] 0 2024-05-12T04:01:22.258 INFO:tasks.workunit.client.1.smithi033.stdout:7/889: dread de/df/f29 [0,4194304] 0 2024-05-12T04:01:22.259 INFO:tasks.workunit.client.0.smithi032.stdout:7/491: write dd/d1b/d43/d5e/f5a [61263,33370] 0 2024-05-12T04:01:22.261 INFO:tasks.workunit.client.1.smithi033.stdout:7/890: fdatasync de/df/d60/f76 0 2024-05-12T04:01:22.261 INFO:tasks.workunit.client.1.smithi033.stdout:9/921: unlink d2/dd/d79/l123 0 2024-05-12T04:01:22.263 INFO:tasks.workunit.client.1.smithi033.stdout:8/643: dwrite d1/d16/d29/d63/fbf [0,4194304] 0 2024-05-12T04:01:22.263 INFO:tasks.workunit.client.0.smithi032.stdout:3/548: mknod d15/d24/d2a/cc6 0 2024-05-12T04:01:22.265 INFO:tasks.workunit.client.0.smithi032.stdout:1/532: dwrite d1/f3 [0,4194304] 0 2024-05-12T04:01:22.267 INFO:tasks.workunit.client.1.smithi033.stdout:0/717: mkdir d6/dc/d96/d9d/df3 0 2024-05-12T04:01:22.268 INFO:tasks.workunit.client.1.smithi033.stdout:6/711: fsync d4/d1a/d3c/f75 0 2024-05-12T04:01:22.271 INFO:tasks.workunit.client.0.smithi032.stdout:0/559: dwrite df/d10/f13 [0,4194304] 0 2024-05-12T04:01:22.282 INFO:tasks.workunit.client.1.smithi033.stdout:9/922: mknod d2/d70/d7b/c146 0 2024-05-12T04:01:22.282 INFO:tasks.workunit.client.1.smithi033.stdout:4/697: symlink dd/d11/d5b/ldd 0 2024-05-12T04:01:22.289 INFO:tasks.workunit.client.1.smithi033.stdout:6/712: mknod d4/d1a/d67/dc3/cec 0 2024-05-12T04:01:22.291 INFO:tasks.workunit.client.1.smithi033.stdout:3/648: sync 2024-05-12T04:01:22.294 INFO:tasks.workunit.client.1.smithi033.stdout:5/584: rename d3/d12/l71 to d3/d15/daa/lba 0 2024-05-12T04:01:22.295 INFO:tasks.workunit.client.1.smithi033.stdout:3/649: truncate db/d17/d3e/dd7/dd0/fd2 207926 0 2024-05-12T04:01:22.298 INFO:tasks.workunit.client.1.smithi033.stdout:2/667: write dc/d17/d27/d2f/d49/f51 [732719,20193] 0 2024-05-12T04:01:22.302 INFO:tasks.workunit.client.0.smithi032.stdout:0/560: dread fa [4194304,4194304] 0 2024-05-12T04:01:22.304 INFO:tasks.workunit.client.1.smithi033.stdout:7/891: dwrite de/d20/f83 [0,4194304] 0 2024-05-12T04:01:22.310 INFO:tasks.workunit.client.1.smithi033.stdout:3/650: dread - db/d17/d3e/d65/fbb zero size 2024-05-12T04:01:22.312 INFO:tasks.workunit.client.1.smithi033.stdout:2/668: rename dc/d3b/l4b to dc/d17/d27/d68/dd1/ld9 0 2024-05-12T04:01:22.313 INFO:tasks.workunit.client.1.smithi033.stdout:0/718: dwrite d6/d11/f1c [0,4194304] 0 2024-05-12T04:01:22.314 INFO:tasks.workunit.client.1.smithi033.stdout:2/669: chown dc/d13/d39/l3e 494169732 1 2024-05-12T04:01:22.315 INFO:tasks.workunit.client.1.smithi033.stdout:0/719: dread - d6/dc/d39/f64 zero size 2024-05-12T04:01:22.315 INFO:tasks.workunit.client.1.smithi033.stdout:9/923: creat d2/d4/d2b/d6d/d2f/d5f/d84/f147 x:0 0 0 2024-05-12T04:01:22.317 INFO:tasks.workunit.client.1.smithi033.stdout:4/698: chown dd/d7a/d87/d7e/da7/l39 6 1 2024-05-12T04:01:22.320 INFO:tasks.workunit.client.1.smithi033.stdout:7/892: creat de/d2f/d42/f12d x:0 0 0 2024-05-12T04:01:22.325 INFO:tasks.workunit.client.1.smithi033.stdout:5/585: truncate d3/d94/d54/d57/f89 1353207 0 2024-05-12T04:01:22.328 INFO:tasks.workunit.client.1.smithi033.stdout:2/670: fsync dc/d17/d27/d2f/f41 0 2024-05-12T04:01:22.330 INFO:tasks.workunit.client.0.smithi032.stdout:9/447: creat d0/d84/d8a/f94 x:0 0 0 2024-05-12T04:01:22.331 INFO:tasks.workunit.client.1.smithi033.stdout:6/713: dwrite d4/d1a/d4e/fb5 [0,4194304] 0 2024-05-12T04:01:22.331 INFO:tasks.workunit.client.1.smithi033.stdout:2/671: chown dc/d13/d1f/d6e/c86 535717617 1 2024-05-12T04:01:22.333 INFO:tasks.workunit.client.1.smithi033.stdout:5/586: symlink d3/d94/d54/d57/d73/d74/lbb 0 2024-05-12T04:01:22.336 INFO:tasks.workunit.client.0.smithi032.stdout:4/462: write d0/d3/d12/d1d/d6b/f6d [305295,109264] 0 2024-05-12T04:01:22.337 INFO:tasks.workunit.client.1.smithi033.stdout:0/720: symlink d6/dc/d58/lf4 0 2024-05-12T04:01:22.340 INFO:tasks.workunit.client.0.smithi032.stdout:2/531: sync 2024-05-12T04:01:22.340 INFO:tasks.workunit.client.0.smithi032.stdout:6/489: sync 2024-05-12T04:01:22.341 INFO:tasks.workunit.client.0.smithi032.stdout:2/532: readlink d1/ld 0 2024-05-12T04:01:22.346 INFO:tasks.workunit.client.0.smithi032.stdout:3/549: rmdir d15/d24/d2a/d5b 39 2024-05-12T04:01:22.349 INFO:tasks.workunit.client.1.smithi033.stdout:5/587: creat d3/d15/daa/fbc x:0 0 0 2024-05-12T04:01:22.349 INFO:tasks.workunit.client.1.smithi033.stdout:4/699: write dd/d69/da4/fa8 [995648,95523] 0 2024-05-12T04:01:22.349 INFO:tasks.workunit.client.0.smithi032.stdout:1/533: truncate d1/d7/d30/d35/f4e 3186975 0 2024-05-12T04:01:22.350 INFO:tasks.workunit.client.1.smithi033.stdout:5/588: readlink d3/d94/d54/d57/d73/d74/d7e/l95 0 2024-05-12T04:01:22.350 INFO:tasks.workunit.client.1.smithi033.stdout:3/651: link db/d17/d3a/d67/d76/da0/lc4 db/d17/d3e/d65/d7d/le1 0 2024-05-12T04:01:22.351 INFO:tasks.workunit.client.1.smithi033.stdout:7/893: write de/df/d17/d1b/d46/f53 [2531269,34169] 0 2024-05-12T04:01:22.351 INFO:tasks.workunit.client.0.smithi032.stdout:0/561: truncate df/d30/d4e/f7e 31161 0 2024-05-12T04:01:22.353 INFO:tasks.workunit.client.0.smithi032.stdout:0/562: truncate df/d30/d4e/d55/fa7 921064 0 2024-05-12T04:01:22.355 INFO:tasks.workunit.client.1.smithi033.stdout:3/652: dread db/d17/d3e/d65/d7d/f86 [0,4194304] 0 2024-05-12T04:01:22.357 INFO:tasks.workunit.client.1.smithi033.stdout:1/611: sync 2024-05-12T04:01:22.358 INFO:tasks.workunit.client.1.smithi033.stdout:9/924: rename d2/d4/d29/d3c/d52 to d2/d4/d29/d148 0 2024-05-12T04:01:22.360 INFO:tasks.workunit.client.0.smithi032.stdout:7/492: write dd/d1b/d1e/d45/d55/d60/d75/f76 [1015968,83329] 0 2024-05-12T04:01:22.368 INFO:tasks.workunit.client.0.smithi032.stdout:4/463: write d0/d3/d12/f66 [212328,30834] 0 2024-05-12T04:01:22.368 INFO:tasks.workunit.client.0.smithi032.stdout:3/550: fsync d15/d24/d93/f78 0 2024-05-12T04:01:22.369 INFO:tasks.workunit.client.1.smithi033.stdout:5/589: write d3/d12/d29/d2b/d4f/f9d [320989,31427] 0 2024-05-12T04:01:22.370 INFO:tasks.workunit.client.1.smithi033.stdout:5/590: readlink d3/d94/d54/d57/d73/d74/lbb 0 2024-05-12T04:01:22.372 INFO:tasks.workunit.client.0.smithi032.stdout:5/608: rename d4/d7/d14/db5/fcb to d4/d7/d14/d26/d33/d4a/fda 0 2024-05-12T04:01:22.372 INFO:tasks.workunit.client.0.smithi032.stdout:9/448: dwrite d0/d3/ff [8388608,4194304] 0 2024-05-12T04:01:22.376 INFO:tasks.workunit.client.0.smithi032.stdout:9/449: dread - d0/f75 zero size 2024-05-12T04:01:22.376 INFO:tasks.workunit.client.0.smithi032.stdout:1/534: creat d1/d7/d30/d18/fab x:0 0 0 2024-05-12T04:01:22.377 INFO:tasks.workunit.client.0.smithi032.stdout:3/551: truncate d15/d20/d22/f57 4261685 0 2024-05-12T04:01:22.379 INFO:tasks.workunit.client.1.smithi033.stdout:3/653: creat db/d42/fe2 x:0 0 0 2024-05-12T04:01:22.382 INFO:tasks.workunit.client.1.smithi033.stdout:7/894: rename de/d20/d38/fa2 to de/df/d17/d1b/d36/f12e 0 2024-05-12T04:01:22.383 INFO:tasks.workunit.client.0.smithi032.stdout:2/533: dwrite d1/d34/d96/d41/d2c/f37 [0,4194304] 0 2024-05-12T04:01:22.384 INFO:tasks.workunit.client.1.smithi033.stdout:9/925: creat d2/d4/d2b/d6d/d2f/d5f/d84/d107/f149 x:0 0 0 2024-05-12T04:01:22.384 INFO:tasks.workunit.client.1.smithi033.stdout:9/926: chown d2/d4/d2b/d6d/d2f/df6 449393 1 2024-05-12T04:01:22.385 INFO:tasks.workunit.client.1.smithi033.stdout:6/714: creat d4/d1a/fed x:0 0 0 2024-05-12T04:01:22.386 INFO:tasks.workunit.client.1.smithi033.stdout:2/672: creat dc/d17/d27/fda x:0 0 0 2024-05-12T04:01:22.387 INFO:tasks.workunit.client.1.smithi033.stdout:9/927: truncate d2/d4/d2b/d6d/d2f/df6/f13a 406223 0 2024-05-12T04:01:22.387 INFO:tasks.workunit.client.1.smithi033.stdout:2/673: chown dc/d13/d1f/d6e/fb6 0 1 2024-05-12T04:01:22.388 INFO:tasks.workunit.client.1.smithi033.stdout:9/928: chown d2/d4/d29/d3c/dda/f124 247946 1 2024-05-12T04:01:22.389 INFO:tasks.workunit.client.1.smithi033.stdout:9/929: write d2/dd/f26 [291723,31536] 0 2024-05-12T04:01:22.391 INFO:tasks.workunit.client.0.smithi032.stdout:0/563: dwrite df/d10/d3b/d75/f9b [0,4194304] 0 2024-05-12T04:01:22.392 INFO:tasks.workunit.client.0.smithi032.stdout:5/609: dwrite d4/d7/d14/d26/d33/d4a/d38/d3b/fba [0,4194304] 0 2024-05-12T04:01:22.394 INFO:tasks.workunit.client.1.smithi033.stdout:3/654: creat db/d17/d3a/d67/da6/fe3 x:0 0 0 2024-05-12T04:01:22.394 INFO:tasks.workunit.client.0.smithi032.stdout:6/490: rename d7/d9/d48/d67/f23 to d7/d9/d48/faa 0 2024-05-12T04:01:22.395 INFO:tasks.workunit.client.0.smithi032.stdout:3/552: rmdir d15/d24/d93 39 2024-05-12T04:01:22.396 INFO:tasks.workunit.client.0.smithi032.stdout:3/553: fsync d15/d33/f5c 0 2024-05-12T04:01:22.397 INFO:tasks.workunit.client.1.smithi033.stdout:2/674: dwrite dc/d3b/d88/fc3 [0,4194304] 0 2024-05-12T04:01:22.400 INFO:tasks.workunit.client.1.smithi033.stdout:7/895: symlink de/df/d17/d1e/dc4/df0/l12f 0 2024-05-12T04:01:22.400 INFO:tasks.workunit.client.0.smithi032.stdout:3/554: write d15/d20/d22/d7a/fb2 [382563,58088] 0 2024-05-12T04:01:22.401 INFO:tasks.workunit.client.0.smithi032.stdout:7/493: mkdir dd/d1b/d1e/d45/d55/d72/da0 0 2024-05-12T04:01:22.403 INFO:tasks.workunit.client.0.smithi032.stdout:3/555: dread - d15/d24/d2a/fc1 zero size 2024-05-12T04:01:22.404 INFO:tasks.workunit.client.1.smithi033.stdout:4/700: creat dd/d7a/d87/d7e/da7/d66/d42/fde x:0 0 0 2024-05-12T04:01:22.409 INFO:tasks.workunit.client.0.smithi032.stdout:5/610: dwrite d4/f22 [8388608,4194304] 0 2024-05-12T04:01:22.410 INFO:tasks.workunit.client.1.smithi033.stdout:7/896: dwrite de/df/d17/d1e/d79/fed [0,4194304] 0 2024-05-12T04:01:22.410 INFO:tasks.workunit.client.1.smithi033.stdout:7/897: chown de/d2b/d6c/de1 3272432 1 2024-05-12T04:01:22.417 INFO:tasks.workunit.client.1.smithi033.stdout:0/721: creat d6/db/d2c/d73/ff5 x:0 0 0 2024-05-12T04:01:22.420 INFO:tasks.workunit.client.0.smithi032.stdout:4/464: rename d0/d3/d24/l2e to d0/d3/d31/l7a 0 2024-05-12T04:01:22.423 INFO:tasks.workunit.client.1.smithi033.stdout:3/655: mknod db/d17/d3e/d65/d91/d97/ce4 0 2024-05-12T04:01:22.426 INFO:tasks.workunit.client.0.smithi032.stdout:6/491: fdatasync d7/f66 0 2024-05-12T04:01:22.429 INFO:tasks.workunit.client.1.smithi033.stdout:1/612: dwrite d11/d1c/f64 [4194304,4194304] 0 2024-05-12T04:01:22.430 INFO:tasks.workunit.client.0.smithi032.stdout:7/494: symlink dd/d1b/d1e/d45/d64/la1 0 2024-05-12T04:01:22.430 INFO:tasks.workunit.client.1.smithi033.stdout:1/613: stat d11/d1c/d43/c5c 0 2024-05-12T04:01:22.431 INFO:tasks.workunit.client.1.smithi033.stdout:2/675: symlink dc/d17/d27/d68/d9f/d5e/ldb 0 2024-05-12T04:01:22.433 INFO:tasks.workunit.client.1.smithi033.stdout:5/591: link d3/d12/d29/d2b/c2d d3/d15/d8c/cbd 0 2024-05-12T04:01:22.434 INFO:tasks.workunit.client.1.smithi033.stdout:5/592: fdatasync d3/d15/d16/d45/d62/fad 0 2024-05-12T04:01:22.435 INFO:tasks.workunit.client.0.smithi032.stdout:0/564: write f5 [2391570,50855] 0 2024-05-12T04:01:22.437 INFO:tasks.workunit.client.0.smithi032.stdout:5/611: chown d4/d7/d14/d39/c16 1 1 2024-05-12T04:01:22.439 INFO:tasks.workunit.client.1.smithi033.stdout:8/644: sync 2024-05-12T04:01:22.440 INFO:tasks.workunit.client.1.smithi033.stdout:8/645: stat d1/l9c 0 2024-05-12T04:01:22.440 INFO:tasks.workunit.client.0.smithi032.stdout:0/565: write df/d22/d58/f6a [5036884,22262] 0 2024-05-12T04:01:22.441 INFO:tasks.workunit.client.1.smithi033.stdout:9/930: dwrite d2/dd/d79/fae [0,4194304] 0 2024-05-12T04:01:22.442 INFO:tasks.workunit.client.0.smithi032.stdout:0/566: write f5 [3772183,78124] 0 2024-05-12T04:01:22.443 INFO:tasks.workunit.client.0.smithi032.stdout:2/534: dwrite d1/d34/f8c [0,4194304] 0 2024-05-12T04:01:22.449 INFO:tasks.workunit.client.0.smithi032.stdout:6/492: creat d7/d9/d48/d67/d1a/d20/fab x:0 0 0 2024-05-12T04:01:22.450 INFO:tasks.workunit.client.1.smithi033.stdout:3/656: mknod db/d17/d3a/d67/d76/ce5 0 2024-05-12T04:01:22.451 INFO:tasks.workunit.client.1.smithi033.stdout:1/614: mknod d11/d1c/d43/ccf 0 2024-05-12T04:01:22.451 INFO:tasks.workunit.client.1.smithi033.stdout:2/676: mknod dc/d17/d27/d2f/cdc 0 2024-05-12T04:01:22.452 INFO:tasks.workunit.client.0.smithi032.stdout:3/556: mkdir d15/d24/d2a/dc7 0 2024-05-12T04:01:22.452 INFO:tasks.workunit.client.1.smithi033.stdout:2/677: stat c5 0 2024-05-12T04:01:22.453 INFO:tasks.workunit.client.1.smithi033.stdout:2/678: chown dc/l11 3551 1 2024-05-12T04:01:22.454 INFO:tasks.workunit.client.1.smithi033.stdout:4/701: symlink dd/d7a/d87/d7e/da7/ldf 0 2024-05-12T04:01:22.455 INFO:tasks.workunit.client.1.smithi033.stdout:5/593: truncate d3/d27/f28 2066401 0 2024-05-12T04:01:22.456 INFO:tasks.workunit.client.1.smithi033.stdout:2/679: write dc/d13/d65/f83 [808291,88244] 0 2024-05-12T04:01:22.456 INFO:tasks.workunit.client.1.smithi033.stdout:5/594: chown d3/d27/l1b 52448 1 2024-05-12T04:01:22.457 INFO:tasks.workunit.client.1.smithi033.stdout:2/680: stat dc/d17/c52 0 2024-05-12T04:01:22.458 INFO:tasks.workunit.client.1.smithi033.stdout:4/702: chown l4 122 1 2024-05-12T04:01:22.460 INFO:tasks.workunit.client.0.smithi032.stdout:0/567: rmdir df 39 2024-05-12T04:01:22.461 INFO:tasks.workunit.client.0.smithi032.stdout:9/450: getdents d0/d84 0 2024-05-12T04:01:22.462 INFO:tasks.workunit.client.0.smithi032.stdout:2/535: rmdir d1/d34/d96/d8a 39 2024-05-12T04:01:22.464 INFO:tasks.workunit.client.1.smithi033.stdout:2/681: dwrite dc/d17/d27/d2f/f38 [0,4194304] 0 2024-05-12T04:01:22.465 INFO:tasks.workunit.client.0.smithi032.stdout:7/495: mknod dd/d1b/d1e/d45/d55/ca2 0 2024-05-12T04:01:22.465 INFO:tasks.workunit.client.0.smithi032.stdout:7/496: stat dd/d1b/l47 0 2024-05-12T04:01:22.465 INFO:tasks.workunit.client.1.smithi033.stdout:2/682: stat dc/fb8 0 2024-05-12T04:01:22.466 INFO:tasks.workunit.client.0.smithi032.stdout:6/493: mknod d7/d9/d48/d67/d25/cac 0 2024-05-12T04:01:22.468 INFO:tasks.workunit.client.0.smithi032.stdout:3/557: mkdir d15/d24/d2a/d8c/db1/dc8 0 2024-05-12T04:01:22.471 INFO:tasks.workunit.client.0.smithi032.stdout:4/465: creat d0/f7b x:0 0 0 2024-05-12T04:01:22.474 INFO:tasks.workunit.client.0.smithi032.stdout:7/497: dwrite dd/d1b/d1e/f8d [0,4194304] 0 2024-05-12T04:01:22.476 INFO:tasks.workunit.client.0.smithi032.stdout:5/612: creat d4/d7/d14/d26/d33/d72/d92/fdb x:0 0 0 2024-05-12T04:01:22.479 INFO:tasks.workunit.client.0.smithi032.stdout:0/568: chown df/d30/d4e/d55/c6e 140287 1 2024-05-12T04:01:22.481 INFO:tasks.workunit.client.0.smithi032.stdout:9/451: symlink d0/d3/d1b/l95 0 2024-05-12T04:01:22.486 INFO:tasks.workunit.client.0.smithi032.stdout:3/558: dread d15/f4e [0,4194304] 0 2024-05-12T04:01:22.486 INFO:tasks.workunit.client.0.smithi032.stdout:9/452: dread d0/d3/d6/f69 [0,4194304] 0 2024-05-12T04:01:22.494 INFO:tasks.workunit.client.0.smithi032.stdout:7/498: dread - dd/d1b/d1e/d45/d55/d60/f88 zero size 2024-05-12T04:01:22.494 INFO:tasks.workunit.client.0.smithi032.stdout:7/499: chown dd/d1b/d43/d74/f7f 3014668 1 2024-05-12T04:01:22.495 INFO:tasks.workunit.client.0.smithi032.stdout:5/613: mknod d4/d7/d14/d26/d33/cdc 0 2024-05-12T04:01:22.497 INFO:tasks.workunit.client.0.smithi032.stdout:5/614: fdatasync d4/d7/d14/d26/d33/d4a/d38/fb8 0 2024-05-12T04:01:22.503 INFO:tasks.workunit.client.0.smithi032.stdout:3/559: dread d15/d24/d40/f5a [0,4194304] 0 2024-05-12T04:01:22.505 INFO:tasks.workunit.client.0.smithi032.stdout:2/536: truncate d1/d13/d70/f91 146865 0 2024-05-12T04:01:22.507 INFO:tasks.workunit.client.1.smithi033.stdout:6/715: sync 2024-05-12T04:01:22.510 INFO:tasks.workunit.client.0.smithi032.stdout:4/466: link d0/d3/df/f29 d0/d39/f7c 0 2024-05-12T04:01:22.512 INFO:tasks.workunit.client.0.smithi032.stdout:7/500: creat dd/d1b/d1e/d45/d55/d60/d75/fa3 x:0 0 0 2024-05-12T04:01:22.512 INFO:tasks.workunit.client.0.smithi032.stdout:0/569: dread df/f51 [0,4194304] 0 2024-05-12T04:01:22.513 INFO:tasks.workunit.client.0.smithi032.stdout:6/494: write d7/de/d2e/f4e [547603,119026] 0 2024-05-12T04:01:22.515 INFO:tasks.workunit.client.0.smithi032.stdout:7/501: chown dd/d1b/d1e/d45/d55/d60/f6b 2656704 1 2024-05-12T04:01:22.522 INFO:tasks.workunit.client.0.smithi032.stdout:9/453: creat d0/d3/d4b/d88/f96 x:0 0 0 2024-05-12T04:01:22.524 INFO:tasks.workunit.client.0.smithi032.stdout:7/502: dwrite dd/d1b/d43/d5e/d37/f54 [0,4194304] 0 2024-05-12T04:01:22.526 INFO:tasks.workunit.client.0.smithi032.stdout:0/570: dread df/d10/d3b/f66 [0,4194304] 0 2024-05-12T04:01:22.533 INFO:tasks.workunit.client.1.smithi033.stdout:9/931: mkdir d2/d4/d2b/d6d/d2f/d5f/d84/d14a 0 2024-05-12T04:01:22.539 INFO:tasks.workunit.client.1.smithi033.stdout:9/932: dread d2/d4/d2b/d6d/d2f/f38 [0,4194304] 0 2024-05-12T04:01:22.540 INFO:tasks.workunit.client.1.smithi033.stdout:7/898: link de/d80/l81 de/df/d17/def/l130 0 2024-05-12T04:01:22.547 INFO:tasks.workunit.client.1.smithi033.stdout:8/646: rename d1/d4/d1d/d36/d3c/f55 to d1/d4/d46/fc8 0 2024-05-12T04:01:22.549 INFO:tasks.workunit.client.1.smithi033.stdout:4/703: unlink dd/d7a/d87/laa 0 2024-05-12T04:01:22.550 INFO:tasks.workunit.client.1.smithi033.stdout:2/683: mkdir dc/d17/d27/d2f/d49/ddd 0 2024-05-12T04:01:22.552 INFO:tasks.workunit.client.1.smithi033.stdout:6/716: unlink d4/d43/d57/dc4/ddf/fc0 0 2024-05-12T04:01:22.553 INFO:tasks.workunit.client.1.smithi033.stdout:6/717: truncate d4/d1a/d67/dc3/fd2 253564 0 2024-05-12T04:01:22.553 INFO:tasks.workunit.client.0.smithi032.stdout:8/518: sync 2024-05-12T04:01:22.554 INFO:tasks.workunit.client.1.smithi033.stdout:3/657: unlink db/d17/f64 0 2024-05-12T04:01:22.559 INFO:tasks.workunit.client.1.smithi033.stdout:9/933: dwrite d2/d4/d2b/d6d/d2f/d5f/f61 [0,4194304] 0 2024-05-12T04:01:22.611 INFO:tasks.workunit.client.1.smithi033.stdout:0/722: rename d6/dc/d58/d91/cbf to d6/dc/d96/cf6 0 2024-05-12T04:01:22.612 INFO:tasks.workunit.client.1.smithi033.stdout:0/723: chown d6/db/d2c/d73/daf 69903382 1 2024-05-12T04:01:22.612 INFO:tasks.workunit.client.1.smithi033.stdout:8/647: creat d1/d4/db2/fc9 x:0 0 0 2024-05-12T04:01:22.613 INFO:tasks.workunit.client.0.smithi032.stdout:6/495: creat d7/d9/d5c/d62/fad x:0 0 0 2024-05-12T04:01:22.615 INFO:tasks.workunit.client.1.smithi033.stdout:4/704: chown c8 211 1 2024-05-12T04:01:22.615 INFO:tasks.workunit.client.0.smithi032.stdout:7/503: creat dd/d1b/d1e/fa4 x:0 0 0 2024-05-12T04:01:22.622 INFO:tasks.workunit.client.0.smithi032.stdout:3/560: creat d15/d24/d2a/d5b/fc9 x:0 0 0 2024-05-12T04:01:22.623 INFO:tasks.workunit.client.1.smithi033.stdout:3/658: fdatasync db/d17/fc5 0 2024-05-12T04:01:22.623 INFO:tasks.workunit.client.1.smithi033.stdout:3/659: chown db/d17/d3a 14 1 2024-05-12T04:01:22.624 INFO:tasks.workunit.client.0.smithi032.stdout:9/454: dread d0/f6b [0,4194304] 0 2024-05-12T04:01:22.625 INFO:tasks.workunit.client.1.smithi033.stdout:3/660: read db/d17/d3e/d65/d7d/f86 [1699629,126550] 0 2024-05-12T04:01:22.625 INFO:tasks.workunit.client.0.smithi032.stdout:5/615: dwrite d4/d7/d14/f8c [4194304,4194304] 0 2024-05-12T04:01:22.625 INFO:tasks.workunit.client.1.smithi033.stdout:3/661: readlink db/d17/d3a/d93/l95 0 2024-05-12T04:01:22.628 INFO:tasks.workunit.client.0.smithi032.stdout:9/455: read d0/d3/d6/d7/da/f1c [3808922,1110] 0 2024-05-12T04:01:22.631 INFO:tasks.workunit.client.0.smithi032.stdout:9/456: read d0/d3/ff [9902041,1535] 0 2024-05-12T04:01:22.632 INFO:tasks.workunit.client.0.smithi032.stdout:6/496: dread d7/d9/d48/d67/f28 [0,4194304] 0 2024-05-12T04:01:22.633 INFO:tasks.workunit.client.0.smithi032.stdout:8/519: mkdir d0/d9/d13/d21/d25/db5 0 2024-05-12T04:01:22.637 INFO:tasks.workunit.client.1.smithi033.stdout:6/718: rename d4/d1a/d6c/f7f to d4/d1a/d67/dc3/fee 0 2024-05-12T04:01:22.638 INFO:tasks.workunit.client.1.smithi033.stdout:6/719: chown d4/f87 183618 1 2024-05-12T04:01:22.639 INFO:tasks.workunit.client.0.smithi032.stdout:2/537: rmdir d1/d82 0 2024-05-12T04:01:22.644 INFO:tasks.workunit.client.1.smithi033.stdout:8/648: unlink d1/d4/d1d/d36/d3c/d64/fc0 0 2024-05-12T04:01:22.645 INFO:tasks.workunit.client.0.smithi032.stdout:7/504: dread dd/d1b/d1e/d45/d64/f6d [0,4194304] 0 2024-05-12T04:01:22.647 INFO:tasks.workunit.client.1.smithi033.stdout:2/684: getdents dc/d13/d65/db1/dd0 0 2024-05-12T04:01:22.656 INFO:tasks.workunit.client.1.smithi033.stdout:5/595: truncate d3/d27/f28 2913612 0 2024-05-12T04:01:22.657 INFO:tasks.workunit.client.1.smithi033.stdout:0/724: write d6/d11/d98/fa7 [587727,26348] 0 2024-05-12T04:01:22.658 INFO:tasks.workunit.client.1.smithi033.stdout:1/615: link d11/d17/d27/d36/d3f/d4a/fa5 d11/d17/d27/d36/d3f/fd0 0 2024-05-12T04:01:22.661 INFO:tasks.workunit.client.1.smithi033.stdout:7/899: creat de/df/f131 x:0 0 0 2024-05-12T04:01:22.664 INFO:tasks.workunit.client.0.smithi032.stdout:9/457: rename d0/d3/d6/d7/d27/f57 to d0/d3/d6/d7/d27/f97 0 2024-05-12T04:01:22.667 INFO:tasks.workunit.client.0.smithi032.stdout:3/561: dwrite d15/d33/f4c [0,4194304] 0 2024-05-12T04:01:22.674 INFO:tasks.workunit.client.1.smithi033.stdout:6/720: mkdir d4/d1a/d3c/dd9/de2/def 0 2024-05-12T04:01:22.675 INFO:tasks.workunit.client.1.smithi033.stdout:6/721: write d4/d6e/f90 [648977,31746] 0 2024-05-12T04:01:22.678 INFO:tasks.workunit.client.1.smithi033.stdout:8/649: mkdir d1/d4/d46/dca 0 2024-05-12T04:01:22.679 INFO:tasks.workunit.client.1.smithi033.stdout:8/650: dread - d1/d4/d1d/f8f zero size 2024-05-12T04:01:22.682 INFO:tasks.workunit.client.1.smithi033.stdout:2/685: rename dc/d13/d39/d8f to dc/d17/d32/d84/dde 0 2024-05-12T04:01:22.684 INFO:tasks.workunit.client.0.smithi032.stdout:0/571: truncate df/d10/d48/f52 732252 0 2024-05-12T04:01:22.685 INFO:tasks.workunit.client.1.smithi033.stdout:3/662: truncate db/d17/d25/d32/f46 1769089 0 2024-05-12T04:01:22.686 INFO:tasks.workunit.client.0.smithi032.stdout:5/616: mknod d4/d7/d14/d39/d2b/cdd 0 2024-05-12T04:01:22.690 INFO:tasks.workunit.client.0.smithi032.stdout:4/467: link d0/d3/l32 d0/l7d 0 2024-05-12T04:01:22.693 INFO:tasks.workunit.client.1.smithi033.stdout:0/725: dwrite d6/dc/d58/d6a/d80/f84 [0,4194304] 0 2024-05-12T04:01:22.695 INFO:tasks.workunit.client.1.smithi033.stdout:7/900: dread de/df/d60/fc3 [0,4194304] 0 2024-05-12T04:01:22.696 INFO:tasks.workunit.client.1.smithi033.stdout:5/596: write d3/d15/d16/d45/d62/f67 [52273,98930] 0 2024-05-12T04:01:22.698 INFO:tasks.workunit.client.1.smithi033.stdout:5/597: fsync d3/d94/f9e 0 2024-05-12T04:01:22.698 INFO:tasks.workunit.client.1.smithi033.stdout:9/934: link d2/d4/d29/d32/c80 d2/dd/d39/c14b 0 2024-05-12T04:01:22.700 INFO:tasks.workunit.client.1.smithi033.stdout:9/935: stat d2/d4/d2b/d6d/d2f/f38 0 2024-05-12T04:01:22.700 INFO:tasks.workunit.client.1.smithi033.stdout:9/936: chown d2/d4/d2b/d6d/d4c 57230 1 2024-05-12T04:01:22.701 INFO:tasks.workunit.client.1.smithi033.stdout:9/937: chown d2/d4/d29/d148 1479638050 1 2024-05-12T04:01:22.702 INFO:tasks.workunit.client.1.smithi033.stdout:9/938: chown d2/d70/d7b/d94/cc4 1 1 2024-05-12T04:01:22.703 INFO:tasks.workunit.client.1.smithi033.stdout:6/722: fdatasync d4/d43/d57/fb8 0 2024-05-12T04:01:22.706 INFO:tasks.workunit.client.0.smithi032.stdout:6/497: dwrite d7/de/f14 [0,4194304] 0 2024-05-12T04:01:22.707 INFO:tasks.workunit.client.1.smithi033.stdout:4/705: creat dd/d7a/d87/d7e/da7/d66/fe0 x:0 0 0 2024-05-12T04:01:22.709 INFO:tasks.workunit.client.0.smithi032.stdout:6/498: dread - d7/d9/d48/d67/d25/f8f zero size 2024-05-12T04:01:22.711 INFO:tasks.workunit.client.0.smithi032.stdout:6/499: write d7/d9/f6c [1568544,50111] 0 2024-05-12T04:01:22.712 INFO:tasks.workunit.client.1.smithi033.stdout:2/686: symlink dc/d17/d27/d2f/d49/d59/ldf 0 2024-05-12T04:01:22.714 INFO:tasks.workunit.client.1.smithi033.stdout:2/687: fdatasync dc/d17/d27/d68/d9f/d5e/f77 0 2024-05-12T04:01:22.715 INFO:tasks.workunit.client.1.smithi033.stdout:1/616: mknod d11/d17/d27/cd1 0 2024-05-12T04:01:22.717 INFO:tasks.workunit.client.1.smithi033.stdout:1/617: write d11/d17/f2e [8677485,94084] 0 2024-05-12T04:01:22.721 INFO:tasks.workunit.client.0.smithi032.stdout:5/617: dwrite d4/d7/d14/d26/d33/f61 [0,4194304] 0 2024-05-12T04:01:22.722 INFO:tasks.workunit.client.0.smithi032.stdout:5/618: chown d4/f20 840211 1 2024-05-12T04:01:22.724 INFO:tasks.workunit.client.1.smithi033.stdout:5/598: rename d3/d15 to d3/d94/d54/d57/d70/da3/dbe 0 2024-05-12T04:01:22.725 INFO:tasks.workunit.client.1.smithi033.stdout:5/599: chown d3/d94/d54/d57/d73/d74/d7e/db1 2 1 2024-05-12T04:01:22.726 INFO:tasks.workunit.client.1.smithi033.stdout:5/600: chown d3/d12/f3f 7 1 2024-05-12T04:01:22.728 INFO:tasks.workunit.client.0.smithi032.stdout:8/520: link d0/d9/d1b/d27/f7b d0/d9/d1b/d27/d88/d56/d63/d8f/d9a/d28/d3e/fb6 0 2024-05-12T04:01:22.731 INFO:tasks.workunit.client.1.smithi033.stdout:4/706: dread dd/d7a/f9f [0,4194304] 0 2024-05-12T04:01:22.731 INFO:tasks.workunit.client.0.smithi032.stdout:3/562: dread d15/d24/d93/f8f [0,4194304] 0 2024-05-12T04:01:22.732 INFO:tasks.workunit.client.1.smithi033.stdout:9/939: dread - d2/d4/d2b/d49/d73/fb4 zero size 2024-05-12T04:01:22.733 INFO:tasks.workunit.client.1.smithi033.stdout:5/601: dwrite d3/f52 [0,4194304] 0 2024-05-12T04:01:22.734 INFO:tasks.workunit.client.0.smithi032.stdout:2/538: link d1/d34/d96/d41/d2c/f59 d1/d34/d4a/d86/fb1 0 2024-05-12T04:01:22.734 INFO:tasks.workunit.client.1.smithi033.stdout:4/707: dread dd/d69/dc4/fc1 [0,4194304] 0 2024-05-12T04:01:22.735 INFO:tasks.workunit.client.1.smithi033.stdout:5/602: chown d3/d94/f49 553746 1 2024-05-12T04:01:22.737 INFO:tasks.workunit.client.1.smithi033.stdout:0/726: write d6/d11/f2f [1988285,73882] 0 2024-05-12T04:01:22.737 INFO:tasks.workunit.client.1.smithi033.stdout:0/727: chown d6/d11/d98/fa1 297 1 2024-05-12T04:01:22.744 INFO:tasks.workunit.client.1.smithi033.stdout:3/663: dread - db/d17/d25/d32/d53/f87 zero size 2024-05-12T04:01:22.747 INFO:tasks.workunit.client.0.smithi032.stdout:4/468: creat d0/d3/df/d4e/f7e x:0 0 0 2024-05-12T04:01:22.748 INFO:tasks.workunit.client.0.smithi032.stdout:4/469: write d0/d3/d24/f2d [1630687,123995] 0 2024-05-12T04:01:22.751 INFO:tasks.workunit.client.0.smithi032.stdout:6/500: unlink d7/d9/d48/d67/d8d/c9f 0 2024-05-12T04:01:22.752 INFO:tasks.workunit.client.0.smithi032.stdout:6/501: read - d7/d9/d48/d67/d25/f7f zero size 2024-05-12T04:01:22.754 INFO:tasks.workunit.client.0.smithi032.stdout:0/572: truncate f5 2624824 0 2024-05-12T04:01:22.757 INFO:tasks.workunit.client.1.smithi033.stdout:7/901: truncate de/df/d17/d1b/f89 5183315 0 2024-05-12T04:01:22.758 INFO:tasks.workunit.client.0.smithi032.stdout:5/619: rename d4/d7/d14/d39/d2b/l8e to d4/d7/d14/d26/d33/d5f/d7f/lde 0 2024-05-12T04:01:22.759 INFO:tasks.workunit.client.0.smithi032.stdout:8/521: truncate d0/d9/d1b/d66/f95 323201 0 2024-05-12T04:01:22.761 INFO:tasks.workunit.client.1.smithi033.stdout:2/688: dwrite dc/d13/d1c/d98/f99 [0,4194304] 0 2024-05-12T04:01:22.764 INFO:tasks.workunit.client.0.smithi032.stdout:8/522: dread d0/d9/d1b/d27/d59/f6d [0,4194304] 0 2024-05-12T04:01:22.764 INFO:tasks.workunit.client.0.smithi032.stdout:2/539: fsync d1/d34/d95/fa1 0 2024-05-12T04:01:22.765 INFO:tasks.workunit.client.0.smithi032.stdout:7/505: link dd/d1b/d1e/d45/d64/f91 dd/d1b/d1e/d45/d55/fa5 0 2024-05-12T04:01:22.766 INFO:tasks.workunit.client.0.smithi032.stdout:2/540: stat d1/d34/d96/d41/d39 0 2024-05-12T04:01:22.767 INFO:tasks.workunit.client.0.smithi032.stdout:5/620: dwrite d4/d7/d14/d26/d33/d4a/d38/d79/fc7 [0,4194304] 0 2024-05-12T04:01:22.768 INFO:tasks.workunit.client.1.smithi033.stdout:9/940: unlink d2/d70/l8f 0 2024-05-12T04:01:22.770 INFO:tasks.workunit.client.0.smithi032.stdout:2/541: truncate d1/d34/d3e/d67/f6a 1203543 0 2024-05-12T04:01:22.770 INFO:tasks.workunit.client.0.smithi032.stdout:4/470: mknod d0/d3/d12/d1d/d6b/c7f 0 2024-05-12T04:01:22.775 INFO:tasks.workunit.client.0.smithi032.stdout:9/458: getdents d0/d3/d6/d7/d2a/d46 0 2024-05-12T04:01:22.777 INFO:tasks.workunit.client.0.smithi032.stdout:8/523: dwrite d0/f68 [0,4194304] 0 2024-05-12T04:01:22.778 INFO:tasks.workunit.client.1.smithi033.stdout:8/651: creat d1/fcb x:0 0 0 2024-05-12T04:01:22.779 INFO:tasks.workunit.client.1.smithi033.stdout:8/652: chown d1/d16/c1f 99417544 1 2024-05-12T04:01:22.780 INFO:tasks.workunit.client.1.smithi033.stdout:3/664: truncate db/f77 569129 0 2024-05-12T04:01:22.781 INFO:tasks.workunit.client.1.smithi033.stdout:3/665: chown db/d17/d3e/dd7/la5 13558633 1 2024-05-12T04:01:22.783 INFO:tasks.workunit.client.0.smithi032.stdout:4/471: mkdir d0/d3/d31/d80 0 2024-05-12T04:01:22.787 INFO:tasks.workunit.client.1.smithi033.stdout:2/689: mknod dc/d13/d1f/d6e/ce0 0 2024-05-12T04:01:22.794 INFO:tasks.workunit.client.1.smithi033.stdout:0/728: dread d6/dc/d58/f85 [0,4194304] 0 2024-05-12T04:01:22.797 INFO:tasks.workunit.client.0.smithi032.stdout:0/573: rename df/d30/d36/f6d to df/fc1 0 2024-05-12T04:01:22.798 INFO:tasks.workunit.client.1.smithi033.stdout:5/603: mknod d3/d12/d25/db0/cbf 0 2024-05-12T04:01:22.800 INFO:tasks.workunit.client.1.smithi033.stdout:0/729: dwrite d6/d11/d57/fcc [0,4194304] 0 2024-05-12T04:01:22.800 INFO:tasks.workunit.client.0.smithi032.stdout:4/472: unlink d0/d3/d12/d1d/d6b/f6d 0 2024-05-12T04:01:22.801 INFO:tasks.workunit.client.0.smithi032.stdout:4/473: dread - d0/d39/f64 zero size 2024-05-12T04:01:22.803 INFO:tasks.workunit.client.0.smithi032.stdout:9/459: creat d0/d3/d4b/d85/d87/f98 x:0 0 0 2024-05-12T04:01:22.804 INFO:tasks.workunit.client.0.smithi032.stdout:1/535: sync 2024-05-12T04:01:22.805 INFO:tasks.workunit.client.1.smithi033.stdout:3/666: rmdir db 39 2024-05-12T04:01:22.808 INFO:tasks.workunit.client.1.smithi033.stdout:5/604: dwrite d3/d94/d54/d57/d70/da3/dbe/d16/d45/d62/f64 [0,4194304] 0 2024-05-12T04:01:22.809 INFO:tasks.workunit.client.1.smithi033.stdout:7/902: dread de/fc2 [0,4194304] 0 2024-05-12T04:01:22.811 INFO:tasks.workunit.client.0.smithi032.stdout:3/563: write d15/f19 [4845039,59717] 0 2024-05-12T04:01:22.812 INFO:tasks.workunit.client.0.smithi032.stdout:7/506: write dd/d1b/d1e/d45/d55/fa5 [33729,42210] 0 2024-05-12T04:01:22.812 INFO:tasks.workunit.client.1.smithi033.stdout:2/690: creat dc/d3b/d88/fe1 x:0 0 0 2024-05-12T04:01:22.812 INFO:tasks.workunit.client.0.smithi032.stdout:6/502: dwrite d7/d9/d48/d67/f41 [0,4194304] 0 2024-05-12T04:01:22.813 INFO:tasks.workunit.client.0.smithi032.stdout:8/524: mkdir d0/d9/d1b/d33/d73/db7 0 2024-05-12T04:01:22.816 INFO:tasks.workunit.client.1.smithi033.stdout:9/941: mkdir d2/d4/d2b/d78/d76/d13d/d140/d14c 0 2024-05-12T04:01:22.817 INFO:tasks.workunit.client.1.smithi033.stdout:6/723: getdents d4/d43/d57 0 2024-05-12T04:01:22.819 INFO:tasks.workunit.client.1.smithi033.stdout:6/724: write d4/d5/f73 [4233719,98503] 0 2024-05-12T04:01:22.820 INFO:tasks.workunit.client.0.smithi032.stdout:5/621: rename d4/l7d to d4/d7/d14/ldf 0 2024-05-12T04:01:22.822 INFO:tasks.workunit.client.0.smithi032.stdout:5/622: write d4/d7/d14/d39/fcf [3766080,12380] 0 2024-05-12T04:01:22.822 INFO:tasks.workunit.client.0.smithi032.stdout:0/574: mkdir df/d30/dc2 0 2024-05-12T04:01:22.831 INFO:tasks.workunit.client.0.smithi032.stdout:9/460: mkdir d0/d3/d4b/d88/d99 0 2024-05-12T04:01:22.832 INFO:tasks.workunit.client.0.smithi032.stdout:2/542: truncate d1/d34/d3e/d67/f83 4101920 0 2024-05-12T04:01:22.833 INFO:tasks.workunit.client.1.smithi033.stdout:0/730: rmdir d6/db/d2c/d73/daf/dc3 39 2024-05-12T04:01:22.834 INFO:tasks.workunit.client.1.smithi033.stdout:8/653: dwrite d1/d4/d1d/f58 [0,4194304] 0 2024-05-12T04:01:22.835 INFO:tasks.workunit.client.1.smithi033.stdout:0/731: chown d6/db/d2c 107856 1 2024-05-12T04:01:22.836 INFO:tasks.workunit.client.1.smithi033.stdout:0/732: fdatasync d6/dc/f18 0 2024-05-12T04:01:22.839 INFO:tasks.workunit.client.1.smithi033.stdout:1/618: rename lc to d11/d1c/ld2 0 2024-05-12T04:01:22.842 INFO:tasks.workunit.client.0.smithi032.stdout:8/525: fsync d0/d9/d1b/d27/d88/f40 0 2024-05-12T04:01:22.847 INFO:tasks.workunit.client.0.smithi032.stdout:6/503: creat d7/d9/d5c/d8b/da1/fae x:0 0 0 2024-05-12T04:01:22.856 INFO:tasks.workunit.client.0.smithi032.stdout:5/623: rename d4/d7/fc1 to d4/d7/d14/d26/d3c/fe0 0 2024-05-12T04:01:22.857 INFO:tasks.workunit.client.1.smithi033.stdout:4/708: link dd/d7a/d87/d7e/da7/d38/l52 dd/d7a/d87/d7e/da7/d66/d42/d59/dd0/le1 0 2024-05-12T04:01:22.858 INFO:tasks.workunit.client.0.smithi032.stdout:4/474: mknod d0/d3/df/c81 0 2024-05-12T04:01:22.862 INFO:tasks.workunit.client.0.smithi032.stdout:2/543: symlink d1/d34/da5/lb2 0 2024-05-12T04:01:22.864 INFO:tasks.workunit.client.1.smithi033.stdout:3/667: unlink db/d17/d25/l4c 0 2024-05-12T04:01:22.865 INFO:tasks.workunit.client.0.smithi032.stdout:3/564: mknod d15/d20/d26/d6a/cca 0 2024-05-12T04:01:22.865 INFO:tasks.workunit.client.1.smithi033.stdout:3/668: write db/d17/d3e/dd7/dd0/fd2 [468674,14587] 0 2024-05-12T04:01:22.868 INFO:tasks.workunit.client.1.smithi033.stdout:1/619: read d11/d17/d27/f52 [218770,114278] 0 2024-05-12T04:01:22.871 INFO:tasks.workunit.client.1.smithi033.stdout:5/605: link d3/d94/d54/d57/d73/d74/d7e/fb2 d3/d94/d54/d57/d70/da3/dbe/d16/d45/d62/fc0 0 2024-05-12T04:01:22.874 INFO:tasks.workunit.client.1.smithi033.stdout:4/709: creat dd/d69/dc4/fe2 x:0 0 0 2024-05-12T04:01:22.876 INFO:tasks.workunit.client.1.smithi033.stdout:0/733: unlink d6/f61 0 2024-05-12T04:01:22.877 INFO:tasks.workunit.client.1.smithi033.stdout:9/942: dread d2/d4/d2b/d6d/d3a/f5a [4194304,4194304] 0 2024-05-12T04:01:22.878 INFO:tasks.workunit.client.1.smithi033.stdout:9/943: dread - d2/d4/d2b/d6d/d2f/d5f/d84/d107/f149 zero size 2024-05-12T04:01:22.883 INFO:tasks.workunit.client.0.smithi032.stdout:1/536: creat d1/d7/d30/fac x:0 0 0 2024-05-12T04:01:22.883 INFO:tasks.workunit.client.1.smithi033.stdout:1/620: dread d11/d17/d27/d36/d3f/d69/d87/da4/fa8 [0,4194304] 0 2024-05-12T04:01:22.887 INFO:tasks.workunit.client.1.smithi033.stdout:7/903: creat de/d20/d38/f132 x:0 0 0 2024-05-12T04:01:22.888 INFO:tasks.workunit.client.0.smithi032.stdout:3/565: unlink d15/d20/d26/f3c 0 2024-05-12T04:01:22.890 INFO:tasks.workunit.client.1.smithi033.stdout:2/691: creat dc/d13/d1f/fe2 x:0 0 0 2024-05-12T04:01:22.894 INFO:tasks.workunit.client.1.smithi033.stdout:6/725: creat d4/d1a/ff0 x:0 0 0 2024-05-12T04:01:22.894 INFO:tasks.workunit.client.1.smithi033.stdout:6/726: chown d4/fd5 28051292 1 2024-05-12T04:01:22.895 INFO:tasks.workunit.client.0.smithi032.stdout:4/475: dread d0/d3/d31/f35 [0,4194304] 0 2024-05-12T04:01:22.895 INFO:tasks.workunit.client.0.smithi032.stdout:8/526: mkdir d0/d9/d1b/db8 0 2024-05-12T04:01:22.895 INFO:tasks.workunit.client.0.smithi032.stdout:8/527: fdatasync d0/d9/f12 0 2024-05-12T04:01:22.896 INFO:tasks.workunit.client.1.smithi033.stdout:5/606: unlink d3/d94/d54/d57/d70/da3/dbe/d16/f51 0 2024-05-12T04:01:22.897 INFO:tasks.workunit.client.1.smithi033.stdout:5/607: write d3/f52 [2552424,99145] 0 2024-05-12T04:01:22.899 INFO:tasks.workunit.client.1.smithi033.stdout:8/654: truncate d1/d16/d27/d28/f76 4101956 0 2024-05-12T04:01:22.900 INFO:tasks.workunit.client.0.smithi032.stdout:6/504: mknod d7/d9/d48/d67/caf 0 2024-05-12T04:01:22.902 INFO:tasks.workunit.client.0.smithi032.stdout:0/575: creat df/fc3 x:0 0 0 2024-05-12T04:01:22.904 INFO:tasks.workunit.client.0.smithi032.stdout:6/505: write d7/d9/d5c/d8b/fa9 [476104,68132] 0 2024-05-12T04:01:22.904 INFO:tasks.workunit.client.0.smithi032.stdout:8/528: dwrite d0/d7/f11 [0,4194304] 0 2024-05-12T04:01:22.907 INFO:tasks.workunit.client.0.smithi032.stdout:7/507: rename dd/l15 to dd/d1b/d1e/d45/d55/d72/la6 0 2024-05-12T04:01:22.908 INFO:tasks.workunit.client.1.smithi033.stdout:8/655: dwrite d1/d16/d27/f86 [0,4194304] 0 2024-05-12T04:01:22.908 INFO:tasks.workunit.client.0.smithi032.stdout:5/624: mknod d4/d7/d14/d26/d47/ce1 0 2024-05-12T04:01:22.912 INFO:tasks.workunit.client.1.smithi033.stdout:7/904: creat de/df/d9a/f133 x:0 0 0 2024-05-12T04:01:22.913 INFO:tasks.workunit.client.1.smithi033.stdout:2/692: dread - dc/d13/d65/fa8 zero size 2024-05-12T04:01:22.914 INFO:tasks.workunit.client.0.smithi032.stdout:0/576: dwrite df/fc3 [0,4194304] 0 2024-05-12T04:01:22.916 INFO:tasks.workunit.client.1.smithi033.stdout:0/734: dwrite d6/db/d2c/d73/f87 [4194304,4194304] 0 2024-05-12T04:01:22.918 INFO:tasks.workunit.client.1.smithi033.stdout:7/905: write de/d20/d38/dec/d117/ff1 [681482,82718] 0 2024-05-12T04:01:22.920 INFO:tasks.workunit.client.1.smithi033.stdout:3/669: dwrite db/d17/d25/d32/d3c/d44/fbd [0,4194304] 0 2024-05-12T04:01:22.921 INFO:tasks.workunit.client.0.smithi032.stdout:3/566: chown d15/d20/d22/f23 41 1 2024-05-12T04:01:22.921 INFO:tasks.workunit.client.0.smithi032.stdout:6/506: dwrite d7/d9/d60/f80 [0,4194304] 0 2024-05-12T04:01:22.930 INFO:tasks.workunit.client.1.smithi033.stdout:9/944: symlink d2/d4/d2b/d78/d76/d13d/d140/d14c/l14d 0 2024-05-12T04:01:22.930 INFO:tasks.workunit.client.0.smithi032.stdout:6/507: write d7/de/f14 [234469,115393] 0 2024-05-12T04:01:22.936 INFO:tasks.workunit.client.0.smithi032.stdout:9/461: getdents d0/d3/d4b/d73 0 2024-05-12T04:01:22.939 INFO:tasks.workunit.client.0.smithi032.stdout:8/529: creat d0/d9/d1b/d27/d4c/d4e/d90/fb9 x:0 0 0 2024-05-12T04:01:22.939 INFO:tasks.workunit.client.0.smithi032.stdout:5/625: readlink d4/d7/d14/d39/l1f 0 2024-05-12T04:01:22.944 INFO:tasks.workunit.client.0.smithi032.stdout:8/530: write d0/d9/d13/f29 [3782128,31593] 0 2024-05-12T04:01:22.945 INFO:tasks.workunit.client.0.smithi032.stdout:6/508: dwrite d7/d9/f82 [0,4194304] 0 2024-05-12T04:01:22.946 INFO:tasks.workunit.client.0.smithi032.stdout:5/626: chown d4/d7/d14/d26/d33/d4a/cad 34 1 2024-05-12T04:01:22.947 INFO:tasks.workunit.client.0.smithi032.stdout:4/476: link d0/d3/d24/f2d d0/d3/d31/d80/f82 0 2024-05-12T04:01:22.961 INFO:tasks.workunit.client.1.smithi033.stdout:5/608: creat d3/d94/d54/d57/d70/da3/dbe/fc1 x:0 0 0 2024-05-12T04:01:22.962 INFO:tasks.workunit.client.0.smithi032.stdout:2/544: truncate d1/d34/d96/d8a/f9b 1141960 0 2024-05-12T04:01:22.963 INFO:tasks.workunit.client.0.smithi032.stdout:2/545: chown d1/d34/d96/d8a 15164 1 2024-05-12T04:01:22.967 INFO:tasks.workunit.client.1.smithi033.stdout:4/710: getdents dd/d11 0 2024-05-12T04:01:22.968 INFO:tasks.workunit.client.0.smithi032.stdout:3/567: fsync d15/d33/d3f/fb3 0 2024-05-12T04:01:22.970 INFO:tasks.workunit.client.1.smithi033.stdout:6/727: dwrite d4/d1a/d67/d68/f72 [0,4194304] 0 2024-05-12T04:01:22.972 INFO:tasks.workunit.client.1.smithi033.stdout:9/945: link d2/d4/d2b/d33/dd2/dde/f126 d2/d4/d2b/d6d/d2f/d5f/d84/d107/f14e 0 2024-05-12T04:01:22.974 INFO:tasks.workunit.client.0.smithi032.stdout:4/477: rename d0/d3/d31/f38 to d0/d75/f83 0 2024-05-12T04:01:22.977 INFO:tasks.workunit.client.0.smithi032.stdout:8/531: fsync d0/d9/d1b/d27/d4c/d4e/d90/fb9 0 2024-05-12T04:01:22.978 INFO:tasks.workunit.client.0.smithi032.stdout:8/532: dread - d0/d4b/fa1 zero size 2024-05-12T04:01:22.979 INFO:tasks.workunit.client.1.smithi033.stdout:5/609: symlink d3/d94/d54/d57/d70/da3/dbe/d16/d45/d62/d7f/d84/lc2 0 2024-05-12T04:01:22.981 INFO:tasks.workunit.client.1.smithi033.stdout:3/670: write db/d17/d25/d32/d53/f87 [410924,116569] 0 2024-05-12T04:01:22.983 INFO:tasks.workunit.client.1.smithi033.stdout:3/671: write db/d17/d3a/f63 [4602786,115220] 0 2024-05-12T04:01:22.985 INFO:tasks.workunit.client.0.smithi032.stdout:7/508: creat dd/d1b/d1e/d45/d55/fa7 x:0 0 0 2024-05-12T04:01:22.986 INFO:tasks.workunit.client.0.smithi032.stdout:1/537: link d1/d25/c66 d1/d7/d30/d18/d93/cad 0 2024-05-12T04:01:22.986 INFO:tasks.workunit.client.0.smithi032.stdout:7/509: readlink dd/d1b/d1e/l52 0 2024-05-12T04:01:22.986 INFO:tasks.workunit.client.1.smithi033.stdout:4/711: unlink dd/d7a/d87/d7e/da7/f72 0 2024-05-12T04:01:22.987 INFO:tasks.workunit.client.0.smithi032.stdout:2/546: dread - d1/d34/d96/d41/d58/f89 zero size 2024-05-12T04:01:22.988 INFO:tasks.workunit.client.1.smithi033.stdout:8/656: getdents d1/d16/d29/d63/da3/dba 0 2024-05-12T04:01:22.990 INFO:tasks.workunit.client.1.smithi033.stdout:6/728: mknod d4/d1a/d3c/dd9/cf1 0 2024-05-12T04:01:22.991 INFO:tasks.workunit.client.1.smithi033.stdout:8/657: read d1/d4/d1d/d36/d3c/d64/f7b [4051787,90150] 0 2024-05-12T04:01:22.992 INFO:tasks.workunit.client.0.smithi032.stdout:0/577: creat df/d30/d4e/d55/fc4 x:0 0 0 2024-05-12T04:01:22.995 INFO:tasks.workunit.client.1.smithi033.stdout:9/946: dread d2/dd/d39/f43 [0,4194304] 0 2024-05-12T04:01:22.995 INFO:tasks.workunit.client.1.smithi033.stdout:7/906: dwrite de/df/d17/f34 [0,4194304] 0 2024-05-12T04:01:22.996 INFO:tasks.workunit.client.1.smithi033.stdout:7/907: read - de/d2f/fe9 zero size 2024-05-12T04:01:22.998 INFO:tasks.workunit.client.1.smithi033.stdout:0/735: rename d6/dc/d58/f78 to d6/ff7 0 2024-05-12T04:01:22.998 INFO:tasks.workunit.client.0.smithi032.stdout:3/568: read - d15/d20/d22/d7a/f88 zero size 2024-05-12T04:01:23.001 INFO:tasks.workunit.client.1.smithi033.stdout:5/610: creat d3/d94/d54/d57/d70/da3/dbe/daa/fc3 x:0 0 0 2024-05-12T04:01:23.002 INFO:tasks.workunit.client.1.smithi033.stdout:5/611: chown d3/d94/d54/d57/d70/da3/dbe/daa/fc3 6884 1 2024-05-12T04:01:23.005 INFO:tasks.workunit.client.0.smithi032.stdout:9/462: dwrite d0/d3/d6/d7/d2a/d46/f12 [0,4194304] 0 2024-05-12T04:01:23.014 INFO:tasks.workunit.client.0.smithi032.stdout:6/509: dwrite d7/f66 [0,4194304] 0 2024-05-12T04:01:23.024 INFO:tasks.workunit.client.0.smithi032.stdout:1/538: truncate d1/d7/d30/f60 905130 0 2024-05-12T04:01:23.025 INFO:tasks.workunit.client.0.smithi032.stdout:1/539: write d1/f47 [1041920,119601] 0 2024-05-12T04:01:23.027 INFO:tasks.workunit.client.0.smithi032.stdout:1/540: chown d1/d7/d49/l92 1415 1 2024-05-12T04:01:23.028 INFO:tasks.workunit.client.0.smithi032.stdout:2/547: mkdir d1/d34/da5/db3 0 2024-05-12T04:01:23.030 INFO:tasks.workunit.client.0.smithi032.stdout:5/627: truncate d4/d7/d14/f8c 6014096 0 2024-05-12T04:01:23.032 INFO:tasks.workunit.client.1.smithi033.stdout:9/947: mkdir d2/d4/d2b/d78/d76/d14f 0 2024-05-12T04:01:23.033 INFO:tasks.workunit.client.1.smithi033.stdout:9/948: stat d2/d4/d2b/d33/dd2/dde/f126 0 2024-05-12T04:01:23.033 INFO:tasks.workunit.client.0.smithi032.stdout:3/569: creat d15/d33/d3f/d75/fcb x:0 0 0 2024-05-12T04:01:23.038 INFO:tasks.workunit.client.0.smithi032.stdout:2/548: dwrite d1/d34/d96/d41/d39/f87 [0,4194304] 0 2024-05-12T04:01:23.041 INFO:tasks.workunit.client.0.smithi032.stdout:4/478: truncate d0/f3d 1074343 0 2024-05-12T04:01:23.041 INFO:tasks.workunit.client.0.smithi032.stdout:9/463: creat d0/d3/d6/d58/f9a x:0 0 0 2024-05-12T04:01:23.042 INFO:tasks.workunit.client.1.smithi033.stdout:8/658: dread d1/d4/d91/fbd [0,4194304] 0 2024-05-12T04:01:23.046 INFO:tasks.workunit.client.1.smithi033.stdout:3/672: symlink db/d17/le6 0 2024-05-12T04:01:23.046 INFO:tasks.workunit.client.1.smithi033.stdout:0/736: creat d6/db/d2c/d82/ff8 x:0 0 0 2024-05-12T04:01:23.046 INFO:tasks.workunit.client.1.smithi033.stdout:5/612: mkdir d3/d94/d54/dc4 0 2024-05-12T04:01:23.046 INFO:tasks.workunit.client.1.smithi033.stdout:8/659: read d1/d16/d27/d4d/f5e [194223,72971] 0 2024-05-12T04:01:23.047 INFO:tasks.workunit.client.1.smithi033.stdout:8/660: readlink l0 0 2024-05-12T04:01:23.047 INFO:tasks.workunit.client.1.smithi033.stdout:0/737: write d6/db/d2c/f3e [2520729,4158] 0 2024-05-12T04:01:23.049 INFO:tasks.workunit.client.1.smithi033.stdout:8/661: read - d1/d4/d1d/d36/faa zero size 2024-05-12T04:01:23.049 INFO:tasks.workunit.client.1.smithi033.stdout:0/738: dread - d6/dc/d58/d6a/f7f zero size 2024-05-12T04:01:23.050 INFO:tasks.workunit.client.1.smithi033.stdout:6/729: mkdir d4/df2 0 2024-05-12T04:01:23.051 INFO:tasks.workunit.client.0.smithi032.stdout:8/533: symlink d0/d9/d1b/d27/d88/d56/d63/lba 0 2024-05-12T04:01:23.052 INFO:tasks.workunit.client.0.smithi032.stdout:8/534: fdatasync d0/fb2 0 2024-05-12T04:01:23.052 INFO:tasks.workunit.client.0.smithi032.stdout:9/464: chown d0/d84/d8a/f94 7 1 2024-05-12T04:01:23.053 INFO:tasks.workunit.client.1.smithi033.stdout:8/662: read d1/d4/d1d/d36/d3c/da7/f43 [4116074,97593] 0 2024-05-12T04:01:23.054 INFO:tasks.workunit.client.1.smithi033.stdout:8/663: chown d1/d16/d29/d63/da3/dba 152135 1 2024-05-12T04:01:23.055 INFO:tasks.workunit.client.0.smithi032.stdout:8/535: write d0/d9/d1b/d27/d88/f79 [1737178,95365] 0 2024-05-12T04:01:23.058 INFO:tasks.workunit.client.0.smithi032.stdout:8/536: dread - d0/d9/d1b/d27/d59/fa4 zero size 2024-05-12T04:01:23.059 INFO:tasks.workunit.client.0.smithi032.stdout:4/479: dwrite d0/d39/f64 [0,4194304] 0 2024-05-12T04:01:23.059 INFO:tasks.workunit.client.0.smithi032.stdout:1/541: unlink d1/f3 0 2024-05-12T04:01:23.061 INFO:tasks.workunit.client.0.smithi032.stdout:1/542: chown d1/d7/d30/d35/f45 2952 1 2024-05-12T04:01:23.069 INFO:tasks.workunit.client.0.smithi032.stdout:5/628: fsync d4/d7/d14/d39/d2b/d6e/f77 0 2024-05-12T04:01:23.070 INFO:tasks.workunit.client.0.smithi032.stdout:5/629: dread - d4/d7/d14/da9/faa zero size 2024-05-12T04:01:23.074 INFO:tasks.workunit.client.1.smithi033.stdout:6/730: fdatasync d4/d1a/d4e/fa6 0 2024-05-12T04:01:23.074 INFO:tasks.workunit.client.0.smithi032.stdout:5/630: write d4/d7/d14/fa3 [642141,94644] 0 2024-05-12T04:01:23.077 INFO:tasks.workunit.client.0.smithi032.stdout:7/510: creat dd/d1b/d43/fa8 x:0 0 0 2024-05-12T04:01:23.079 INFO:tasks.workunit.client.1.smithi033.stdout:0/739: creat d6/dc/d58/d6a/d80/db8/ff9 x:0 0 0 2024-05-12T04:01:23.080 INFO:tasks.workunit.client.1.smithi033.stdout:0/740: write d6/d11/d57/f21 [8043209,49876] 0 2024-05-12T04:01:23.081 INFO:tasks.workunit.client.1.smithi033.stdout:8/664: chown d1/d4/d46/fc8 1872 1 2024-05-12T04:01:23.082 INFO:tasks.workunit.client.1.smithi033.stdout:8/665: chown d1/d4/f39 64470 1 2024-05-12T04:01:23.084 INFO:tasks.workunit.client.0.smithi032.stdout:5/631: dwrite d4/d7/d14/d39/d2b/d6e/f85 [0,4194304] 0 2024-05-12T04:01:23.086 INFO:tasks.workunit.client.1.smithi033.stdout:4/712: rename dd/d11/c2c to dd/ce3 0 2024-05-12T04:01:23.089 INFO:tasks.workunit.client.0.smithi032.stdout:4/480: dread d0/d3/d12/f48 [0,4194304] 0 2024-05-12T04:01:23.092 INFO:tasks.workunit.client.1.smithi033.stdout:9/949: write d2/d70/d7b/d94/ff5 [1094409,126222] 0 2024-05-12T04:01:23.094 INFO:tasks.workunit.client.0.smithi032.stdout:8/537: dread - d0/d9/d1b/d27/d88/f62 zero size 2024-05-12T04:01:23.096 INFO:tasks.workunit.client.1.smithi033.stdout:1/621: sync 2024-05-12T04:01:23.096 INFO:tasks.workunit.client.1.smithi033.stdout:2/693: sync 2024-05-12T04:01:23.097 INFO:tasks.workunit.client.0.smithi032.stdout:7/511: dread dd/f1a [4194304,4194304] 0 2024-05-12T04:01:23.098 INFO:tasks.workunit.client.1.smithi033.stdout:2/694: write dc/d17/d27/d2f/d49/fbe [429109,74527] 0 2024-05-12T04:01:23.098 INFO:tasks.workunit.client.0.smithi032.stdout:1/543: chown d1/d7/c71 502803100 1 2024-05-12T04:01:23.102 INFO:tasks.workunit.client.0.smithi032.stdout:1/544: dread - d1/d6a/f6e zero size 2024-05-12T04:01:23.102 INFO:tasks.workunit.client.0.smithi032.stdout:3/570: write d15/d1b/f21 [9129736,95227] 0 2024-05-12T04:01:23.103 INFO:tasks.workunit.client.1.smithi033.stdout:6/731: symlink d4/d1a/d67/dc3/lf3 0 2024-05-12T04:01:23.103 INFO:tasks.workunit.client.1.smithi033.stdout:5/613: write d3/d12/f14 [1180834,89837] 0 2024-05-12T04:01:23.105 INFO:tasks.workunit.client.0.smithi032.stdout:7/512: dread dd/d1b/d1e/d45/d64/f6d [0,4194304] 0 2024-05-12T04:01:23.109 INFO:tasks.workunit.client.0.smithi032.stdout:6/510: creat d7/d9/d48/d67/fb0 x:0 0 0 2024-05-12T04:01:23.109 INFO:tasks.workunit.client.0.smithi032.stdout:1/545: dwrite d1/d7/d30/d18/f41 [0,4194304] 0 2024-05-12T04:01:23.111 INFO:tasks.workunit.client.0.smithi032.stdout:1/546: readlink d1/d6a/d72/da9/l96 0 2024-05-12T04:01:23.112 INFO:tasks.workunit.client.0.smithi032.stdout:1/547: write d1/d7/f78 [4926115,57775] 0 2024-05-12T04:01:23.119 INFO:tasks.workunit.client.1.smithi033.stdout:3/673: mkdir db/d17/d3a/d67/d76/da0/de7 0 2024-05-12T04:01:23.120 INFO:tasks.workunit.client.1.smithi033.stdout:1/622: dread - d11/d1c/f94 zero size 2024-05-12T04:01:23.124 INFO:tasks.workunit.client.1.smithi033.stdout:2/695: truncate dc/d17/d27/d2f/f90 239567 0 2024-05-12T04:01:23.125 INFO:tasks.workunit.client.0.smithi032.stdout:2/549: link d1/d13/d6b/l84 d1/d4d/d5e/lb4 0 2024-05-12T04:01:23.126 INFO:tasks.workunit.client.0.smithi032.stdout:2/550: chown d1/ca9 589223 1 2024-05-12T04:01:23.127 INFO:tasks.workunit.client.1.smithi033.stdout:5/614: symlink d3/d94/d54/d57/d73/lc5 0 2024-05-12T04:01:23.130 INFO:tasks.workunit.client.0.smithi032.stdout:5/632: write d4/d7/d14/d39/d1b/f23 [1691850,111570] 0 2024-05-12T04:01:23.133 INFO:tasks.workunit.client.1.smithi033.stdout:8/666: creat d1/d4/d1d/d36/d3c/d9f/fcc x:0 0 0 2024-05-12T04:01:23.138 INFO:tasks.workunit.client.0.smithi032.stdout:0/578: rename df/d30/c3e to df/d30/d32/cc5 0 2024-05-12T04:01:23.139 INFO:tasks.workunit.client.1.smithi033.stdout:9/950: mkdir d2/d4/d2b/d6d/db5/d119/d150 0 2024-05-12T04:01:23.142 INFO:tasks.workunit.client.1.smithi033.stdout:4/713: stat dd/d11/da5/la9 0 2024-05-12T04:01:23.144 INFO:tasks.workunit.client.0.smithi032.stdout:5/633: dread d4/d7/d14/d26/f57 [0,4194304] 0 2024-05-12T04:01:23.146 INFO:tasks.workunit.client.1.smithi033.stdout:3/674: mkdir db/d17/d3e/d65/d91/d97/de8 0 2024-05-12T04:01:23.146 INFO:tasks.workunit.client.0.smithi032.stdout:5/634: stat d4/d7/d14/d39/d1b/da4/fc4 0 2024-05-12T04:01:23.146 INFO:tasks.workunit.client.1.smithi033.stdout:0/741: dwrite d6/db/f1a [0,4194304] 0 2024-05-12T04:01:23.149 INFO:tasks.workunit.client.1.smithi033.stdout:1/623: mknod d11/d17/d27/d36/d3f/d69/d87/da4/cd3 0 2024-05-12T04:01:23.149 INFO:tasks.workunit.client.1.smithi033.stdout:0/742: read d6/dc/d58/d6a/f7e [3724854,14100] 0 2024-05-12T04:01:23.151 INFO:tasks.workunit.client.1.smithi033.stdout:1/624: chown d11/d17/d27/d36/d46/f66 1023 1 2024-05-12T04:01:23.152 INFO:tasks.workunit.client.0.smithi032.stdout:3/571: link d15/d20/c27 d15/d1b/ccc 0 2024-05-12T04:01:23.155 INFO:tasks.workunit.client.0.smithi032.stdout:9/465: rename d0/d3/d6/d7/d27/l2e to d0/d3/d4b/d73/l9b 0 2024-05-12T04:01:23.157 INFO:tasks.workunit.client.0.smithi032.stdout:2/551: truncate d1/d34/d96/f8e 179158 0 2024-05-12T04:01:23.159 INFO:tasks.workunit.client.0.smithi032.stdout:5/635: dwrite d4/d7/d14/d26/d33/d88/faf [0,4194304] 0 2024-05-12T04:01:23.159 INFO:tasks.workunit.client.0.smithi032.stdout:9/466: fdatasync d0/d3/d6/d7/da/f53 0 2024-05-12T04:01:23.160 INFO:tasks.workunit.client.0.smithi032.stdout:4/481: rename d0/d39 to d0/d3/d31/d80/d84 0 2024-05-12T04:01:23.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:22 smithi033 ceph-mon[49525]: pgmap v39: 65 pgs: 65 active+clean; 2.5 GiB data, 8.8 GiB used, 528 GiB / 536 GiB avail; 84 MiB/s rd, 283 MiB/s wr, 534 op/s 2024-05-12T04:01:23.165 INFO:tasks.workunit.client.1.smithi033.stdout:7/908: rename de/df/d7a/fd3 to de/d20/f134 0 2024-05-12T04:01:23.166 INFO:tasks.workunit.client.1.smithi033.stdout:7/909: stat de/d20/d38/dec/d117/fce 0 2024-05-12T04:01:23.167 INFO:tasks.workunit.client.0.smithi032.stdout:2/552: chown d1/d34/d3e/d67/c7a 60021473 1 2024-05-12T04:01:23.169 INFO:tasks.workunit.client.0.smithi032.stdout:5/636: mkdir d4/d7/d14/d26/d33/de2 0 2024-05-12T04:01:23.170 INFO:tasks.workunit.client.0.smithi032.stdout:6/511: truncate d7/d9/d48/d67/f6e 3301531 0 2024-05-12T04:01:23.173 INFO:tasks.workunit.client.1.smithi033.stdout:5/615: write d3/d94/d54/d57/d73/d74/d7e/fb2 [912643,124419] 0 2024-05-12T04:01:23.174 INFO:tasks.workunit.client.1.smithi033.stdout:3/675: mkdir db/d59/dcc/de9 0 2024-05-12T04:01:23.177 INFO:tasks.workunit.client.1.smithi033.stdout:6/732: truncate d4/d1a/d4e/f89 3992808 0 2024-05-12T04:01:23.183 INFO:tasks.workunit.client.0.smithi032.stdout:6/512: creat d7/d9/d48/d67/d1a/d20/fb1 x:0 0 0 2024-05-12T04:01:23.183 INFO:tasks.workunit.client.1.smithi033.stdout:8/667: write d1/d16/d27/f2d [3922405,89320] 0 2024-05-12T04:01:23.184 INFO:tasks.workunit.client.0.smithi032.stdout:3/572: dread fd [0,4194304] 0 2024-05-12T04:01:23.188 INFO:tasks.workunit.client.0.smithi032.stdout:9/467: dread d0/d3/d74/f4e [0,4194304] 0 2024-05-12T04:01:23.189 INFO:tasks.workunit.client.0.smithi032.stdout:9/468: dread - d0/d3/d4b/f81 zero size 2024-05-12T04:01:23.193 INFO:tasks.workunit.client.0.smithi032.stdout:8/538: rename d0/d9/fe to d0/d9/d1b/d27/d88/d56/d63/fbb 0 2024-05-12T04:01:23.195 INFO:tasks.workunit.client.0.smithi032.stdout:9/469: dread d0/d3/d6/d7/d2a/d46/f12 [0,4194304] 0 2024-05-12T04:01:23.196 INFO:tasks.workunit.client.1.smithi033.stdout:9/951: mknod d2/d4/d2b/c151 0 2024-05-12T04:01:23.200 INFO:tasks.workunit.client.1.smithi033.stdout:4/714: readlink dd/d11/d5b/d84/l9a 0 2024-05-12T04:01:23.201 INFO:tasks.workunit.client.1.smithi033.stdout:3/676: fdatasync db/fd9 0 2024-05-12T04:01:23.205 INFO:tasks.workunit.client.1.smithi033.stdout:5/616: rename d3/d94/d54/d57/d70/da3/dbe/fab to d3/d94/db8/fc6 0 2024-05-12T04:01:23.206 INFO:tasks.workunit.client.1.smithi033.stdout:6/733: mknod d4/d1a/d3c/dd9/de2/cf4 0 2024-05-12T04:01:23.209 INFO:tasks.workunit.client.1.smithi033.stdout:5/617: dread d3/d94/d54/d57/d73/d74/d7e/fb2 [0,4194304] 0 2024-05-12T04:01:23.209 INFO:tasks.workunit.client.0.smithi032.stdout:7/513: rename dd/d1b/d1e/fa4 to dd/d1b/fa9 0 2024-05-12T04:01:23.213 INFO:tasks.workunit.client.0.smithi032.stdout:7/514: write dd/d1b/d1e/d45/d55/d60/d75/f9b [874910,88361] 0 2024-05-12T04:01:23.213 INFO:tasks.workunit.client.0.smithi032.stdout:4/482: dwrite d0/f3d [0,4194304] 0 2024-05-12T04:01:23.214 INFO:tasks.workunit.client.1.smithi033.stdout:8/668: creat d1/d16/d27/d4d/d73/fcd x:0 0 0 2024-05-12T04:01:23.215 INFO:tasks.workunit.client.1.smithi033.stdout:6/734: dwrite d4/d1a/d4e/d4f/f9e [0,4194304] 0 2024-05-12T04:01:23.215 INFO:tasks.workunit.client.0.smithi032.stdout:4/483: write d0/f7b [198357,45677] 0 2024-05-12T04:01:23.216 INFO:tasks.workunit.client.1.smithi033.stdout:7/910: dwrite de/df/d17/d1b/d36/f8a [0,4194304] 0 2024-05-12T04:01:23.216 INFO:tasks.workunit.client.0.smithi032.stdout:4/484: stat d0/d3/d31/d80/d84/f6a 0 2024-05-12T04:01:23.217 INFO:tasks.workunit.client.0.smithi032.stdout:9/470: mknod d0/d3/d4b/d6f/c9c 0 2024-05-12T04:01:23.217 INFO:tasks.workunit.client.1.smithi033.stdout:6/735: stat d4/d1a/d3c/fa8 0 2024-05-12T04:01:23.218 INFO:tasks.workunit.client.0.smithi032.stdout:9/471: stat d0 0 2024-05-12T04:01:23.219 INFO:tasks.workunit.client.1.smithi033.stdout:6/736: write d4/d1a/ff0 [160761,91387] 0 2024-05-12T04:01:23.233 INFO:tasks.workunit.client.0.smithi032.stdout:1/548: rename d1/d6a/c9f to d1/d6a/d72/da9/cae 0 2024-05-12T04:01:23.239 INFO:tasks.workunit.client.1.smithi033.stdout:5/618: mknod d3/d27/cc7 0 2024-05-12T04:01:23.240 INFO:tasks.workunit.client.1.smithi033.stdout:1/625: dwrite d11/d17/d27/fc9 [0,4194304] 0 2024-05-12T04:01:23.240 INFO:tasks.workunit.client.0.smithi032.stdout:7/515: creat dd/d1b/d1e/faa x:0 0 0 2024-05-12T04:01:23.242 INFO:tasks.workunit.client.0.smithi032.stdout:7/516: readlink dd/d1b/d1e/l2f 0 2024-05-12T04:01:23.243 INFO:tasks.workunit.client.1.smithi033.stdout:5/619: chown d3/d94/d54/d57/d70/da3/dbe/d16/d45/d62/f7a 7885 1 2024-05-12T04:01:23.244 INFO:tasks.workunit.client.1.smithi033.stdout:7/911: chown de/d2f/d42/c10c 1466333 1 2024-05-12T04:01:23.245 INFO:tasks.workunit.client.1.smithi033.stdout:2/696: sync 2024-05-12T04:01:23.246 INFO:tasks.workunit.client.1.smithi033.stdout:1/626: dread d11/d17/d27/d36/d3f/d69/d87/da4/fa8 [0,4194304] 0 2024-05-12T04:01:23.246 INFO:tasks.workunit.client.0.smithi032.stdout:2/553: rename d1/d34/d3e/d67/f69 to d1/d34/da5/fb5 0 2024-05-12T04:01:23.247 INFO:tasks.workunit.client.1.smithi033.stdout:2/697: write dc/f8d [49261,7635] 0 2024-05-12T04:01:23.249 INFO:tasks.workunit.client.0.smithi032.stdout:8/539: creat d0/d9/d1b/d27/d4c/d4e/d90/fbc x:0 0 0 2024-05-12T04:01:23.251 INFO:tasks.workunit.client.0.smithi032.stdout:8/540: read d0/d9/d1b/d27/d59/f6d [2908543,104509] 0 2024-05-12T04:01:23.251 INFO:tasks.workunit.client.1.smithi033.stdout:8/669: rename d1/d16/d27/d28/fb9 to d1/d4/d1d/d36/fce 0 2024-05-12T04:01:23.252 INFO:tasks.workunit.client.1.smithi033.stdout:3/677: write db/d17/d4d/faa [888004,3197] 0 2024-05-12T04:01:23.253 INFO:tasks.workunit.client.1.smithi033.stdout:8/670: write d1/d16/d29/db1/fb5 [2593217,38485] 0 2024-05-12T04:01:23.253 INFO:tasks.workunit.client.1.smithi033.stdout:3/678: stat db/d17/d3a/d67/da6/fe3 0 2024-05-12T04:01:23.254 INFO:tasks.workunit.client.1.smithi033.stdout:8/671: fdatasync d1/fa4 0 2024-05-12T04:01:23.255 INFO:tasks.workunit.client.1.smithi033.stdout:4/715: mknod dd/d7a/d87/d7e/da7/d38/ce4 0 2024-05-12T04:01:23.258 INFO:tasks.workunit.client.0.smithi032.stdout:1/549: fsync d1/d25/f34 0 2024-05-12T04:01:23.259 INFO:tasks.workunit.client.0.smithi032.stdout:1/550: write d1/d7/f7f [2398624,52351] 0 2024-05-12T04:01:23.259 INFO:tasks.workunit.client.0.smithi032.stdout:5/637: rename d4/d7/d14/d26/d33/d4a/d38/dd2/f6a to d4/d7/d14/d39/d1b/fe3 0 2024-05-12T04:01:23.266 INFO:tasks.workunit.client.0.smithi032.stdout:4/485: dwrite d0/d3/df/f29 [0,4194304] 0 2024-05-12T04:01:23.275 INFO:tasks.workunit.client.0.smithi032.stdout:7/517: dwrite dd/d1b/d1e/d45/d55/d60/f62 [0,4194304] 0 2024-05-12T04:01:23.277 INFO:tasks.workunit.client.0.smithi032.stdout:8/541: write d0/d9/d1b/d27/f7b [890518,110556] 0 2024-05-12T04:01:23.278 INFO:tasks.workunit.client.0.smithi032.stdout:7/518: write dd/d1b/d1e/d45/d55/d60/d75/f76 [1410663,5156] 0 2024-05-12T04:01:23.284 INFO:tasks.workunit.client.1.smithi033.stdout:3/679: mknod db/d17/d3e/dd7/dd0/cea 0 2024-05-12T04:01:23.286 INFO:tasks.workunit.client.0.smithi032.stdout:7/519: dwrite dd/d1b/f7a [0,4194304] 0 2024-05-12T04:01:23.289 INFO:tasks.workunit.client.1.smithi033.stdout:4/716: dwrite dd/d69/dc4/fbb [0,4194304] 0 2024-05-12T04:01:23.291 INFO:tasks.workunit.client.1.smithi033.stdout:9/952: creat d2/d4/d2b/d6d/d2f/d5f/f152 x:0 0 0 2024-05-12T04:01:23.292 INFO:tasks.workunit.client.1.smithi033.stdout:0/743: sync 2024-05-12T04:01:23.292 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:22 smithi032 ceph-mon[40008]: pgmap v39: 65 pgs: 65 active+clean; 2.5 GiB data, 8.8 GiB used, 528 GiB / 536 GiB avail; 84 MiB/s rd, 283 MiB/s wr, 534 op/s 2024-05-12T04:01:23.298 INFO:tasks.workunit.client.0.smithi032.stdout:7/520: dwrite dd/d1b/d43/d74/f7f [4194304,4194304] 0 2024-05-12T04:01:23.301 INFO:tasks.workunit.client.0.smithi032.stdout:7/521: write dd/d1b/d1e/d45/d55/d60/d75/f76 [1231103,110190] 0 2024-05-12T04:01:23.306 INFO:tasks.workunit.client.0.smithi032.stdout:7/522: truncate dd/d1b/d1e/faa 884237 0 2024-05-12T04:01:23.306 INFO:tasks.workunit.client.0.smithi032.stdout:7/523: write dd/d1b/d1e/f7e [845894,67892] 0 2024-05-12T04:01:23.307 INFO:tasks.workunit.client.1.smithi033.stdout:6/737: rename d4/d5/f5c to d4/d43/ff5 0 2024-05-12T04:01:23.312 INFO:tasks.workunit.client.1.smithi033.stdout:8/672: rmdir d1/d4/d1d/d36/d3c 39 2024-05-12T04:01:23.314 INFO:tasks.workunit.client.1.smithi033.stdout:2/698: link dc/d17/d32/fb0 dc/fe3 0 2024-05-12T04:01:23.315 INFO:tasks.workunit.client.1.smithi033.stdout:5/620: dwrite d3/d12/d25/d26/f9c [4194304,4194304] 0 2024-05-12T04:01:23.316 INFO:tasks.workunit.client.1.smithi033.stdout:2/699: write dc/d13/d1f/d6e/fc2 [913431,5164] 0 2024-05-12T04:01:23.319 INFO:tasks.workunit.client.1.smithi033.stdout:1/627: rename d11/d17/d84/da3/cac to d11/d17/d27/da6/cd4 0 2024-05-12T04:01:23.320 INFO:tasks.workunit.client.0.smithi032.stdout:1/551: creat d1/d7/d30/d35/d77/faf x:0 0 0 2024-05-12T04:01:23.324 INFO:tasks.workunit.client.0.smithi032.stdout:6/513: rename d7/l30 to d7/d9/d48/d67/d63/d64/d94/lb2 0 2024-05-12T04:01:23.326 INFO:tasks.workunit.client.1.smithi033.stdout:9/953: unlink d2/dd/d39/d72/l103 0 2024-05-12T04:01:23.327 INFO:tasks.workunit.client.0.smithi032.stdout:5/638: symlink d4/d7/d58/d5a/le4 0 2024-05-12T04:01:23.332 INFO:tasks.workunit.client.1.smithi033.stdout:6/738: write d4/f56 [4795150,130250] 0 2024-05-12T04:01:23.334 INFO:tasks.workunit.client.0.smithi032.stdout:4/486: creat d0/d3/d31/f85 x:0 0 0 2024-05-12T04:01:23.334 INFO:tasks.workunit.client.1.smithi033.stdout:6/739: chown d4/d43/d57/fb8 4916 1 2024-05-12T04:01:23.336 INFO:tasks.workunit.client.1.smithi033.stdout:3/680: dwrite db/d17/d25/d32/d3c/d3d/f72 [0,4194304] 0 2024-05-12T04:01:23.338 INFO:tasks.workunit.client.1.smithi033.stdout:7/912: rename de/d2f/d70/lff to de/d20/d38/d7f/l135 0 2024-05-12T04:01:23.340 INFO:tasks.workunit.client.0.smithi032.stdout:5/639: dread d4/d7/d14/d39/d1b/fbd [0,4194304] 0 2024-05-12T04:01:23.341 INFO:tasks.workunit.client.1.smithi033.stdout:1/628: symlink d11/d17/d84/ld5 0 2024-05-12T04:01:23.342 INFO:tasks.workunit.client.0.smithi032.stdout:4/487: dwrite d0/d3/d31/f85 [0,4194304] 0 2024-05-12T04:01:23.343 INFO:tasks.workunit.client.0.smithi032.stdout:5/640: fdatasync d4/d7/d58/fbe 0 2024-05-12T04:01:23.346 INFO:tasks.workunit.client.1.smithi033.stdout:5/621: truncate d3/f7 1510254 0 2024-05-12T04:01:23.349 INFO:tasks.workunit.client.1.smithi033.stdout:4/717: dwrite dd/f3b [0,4194304] 0 2024-05-12T04:01:23.352 INFO:tasks.workunit.client.1.smithi033.stdout:3/681: mkdir db/d17/d3a/d93/deb 0 2024-05-12T04:01:23.354 INFO:tasks.workunit.client.0.smithi032.stdout:7/524: unlink f6 0 2024-05-12T04:01:23.357 INFO:tasks.workunit.client.1.smithi033.stdout:0/744: rename d6/d11/d57/f90 to d6/d11/dc7/ffa 0 2024-05-12T04:01:23.359 INFO:tasks.workunit.client.0.smithi032.stdout:7/525: dread dd/d1b/d1e/d45/d64/f95 [0,4194304] 0 2024-05-12T04:01:23.361 INFO:tasks.workunit.client.1.smithi033.stdout:7/913: mkdir de/df/d17/d1b/d110/d136 0 2024-05-12T04:01:23.361 INFO:tasks.workunit.client.1.smithi033.stdout:7/914: chown de/d20/d64/c66 1 1 2024-05-12T04:01:23.362 INFO:tasks.workunit.client.0.smithi032.stdout:5/641: read d4/f20 [59643,110283] 0 2024-05-12T04:01:23.363 INFO:tasks.workunit.client.0.smithi032.stdout:1/552: symlink d1/d7/d30/d18/d93/lb0 0 2024-05-12T04:01:23.368 INFO:tasks.workunit.client.0.smithi032.stdout:6/514: rmdir d7/d9/d48/d67/d8d 39 2024-05-12T04:01:23.378 INFO:tasks.workunit.client.0.smithi032.stdout:4/488: fsync d0/d3/d31/d80/f82 0 2024-05-12T04:01:23.380 INFO:tasks.workunit.client.0.smithi032.stdout:4/489: chown d0/d3/df/c81 109546403 1 2024-05-12T04:01:23.383 INFO:tasks.workunit.client.1.smithi033.stdout:3/682: write db/d17/d25/d32/f5d [867312,53434] 0 2024-05-12T04:01:23.385 INFO:tasks.workunit.client.1.smithi033.stdout:9/954: truncate d2/d4/d2b/d6d/d2f/df6/d13e/f141 2782497 0 2024-05-12T04:01:23.385 INFO:tasks.workunit.client.0.smithi032.stdout:6/515: dread d7/de/f6d [0,4194304] 0 2024-05-12T04:01:23.387 INFO:tasks.workunit.client.1.smithi033.stdout:6/740: rename d4/d1a/d3c/f75 to d4/d1a/d67/d6f/dac/ff6 0 2024-05-12T04:01:23.388 INFO:tasks.workunit.client.0.smithi032.stdout:7/526: symlink dd/d1b/lab 0 2024-05-12T04:01:23.389 INFO:tasks.workunit.client.0.smithi032.stdout:7/527: chown dd/d1b/d43/d5e/f4d 205 1 2024-05-12T04:01:23.389 INFO:tasks.workunit.client.1.smithi033.stdout:0/745: truncate d6/dc/d39/f40 5010764 0 2024-05-12T04:01:23.390 INFO:tasks.workunit.client.0.smithi032.stdout:5/642: truncate d4/d7/d14/d26/d47/f6d 8144324 0 2024-05-12T04:01:23.391 INFO:tasks.workunit.client.0.smithi032.stdout:5/643: dread - d4/d7/d14/d26/dc6/fc8 zero size 2024-05-12T04:01:23.394 INFO:tasks.workunit.client.0.smithi032.stdout:3/573: rename d15/d1b/f21 to d15/d20/d8b/d9b/fcd 0 2024-05-12T04:01:23.394 INFO:tasks.workunit.client.1.smithi033.stdout:7/915: dread de/d2f/f5c [0,4194304] 0 2024-05-12T04:01:23.396 INFO:tasks.workunit.client.1.smithi033.stdout:2/700: getdents dc/d17/d3a 0 2024-05-12T04:01:23.398 INFO:tasks.workunit.client.0.smithi032.stdout:2/554: getdents d1/d13 0 2024-05-12T04:01:23.403 INFO:tasks.workunit.client.1.smithi033.stdout:0/746: dread d6/d11/dc7/fdf [0,4194304] 0 2024-05-12T04:01:23.405 INFO:tasks.workunit.client.0.smithi032.stdout:6/516: unlink d7/d9/d48/d67/d1a/f56 0 2024-05-12T04:01:23.409 INFO:tasks.workunit.client.1.smithi033.stdout:6/741: creat d4/d1a/d67/d6f/daf/ff7 x:0 0 0 2024-05-12T04:01:23.414 INFO:tasks.workunit.client.1.smithi033.stdout:4/718: write dd/d7a/d87/d7e/da7/d66/f6a [812344,62367] 0 2024-05-12T04:01:23.415 INFO:tasks.workunit.client.1.smithi033.stdout:7/916: dread de/df/d60/d75/f8f [0,4194304] 0 2024-05-12T04:01:23.416 INFO:tasks.workunit.client.0.smithi032.stdout:3/574: chown d15/d1b/d43/l4a 2381 1 2024-05-12T04:01:23.417 INFO:tasks.workunit.client.1.smithi033.stdout:1/629: rmdir d11/d17/d27/d76/d93/dbf 0 2024-05-12T04:01:23.421 INFO:tasks.workunit.client.0.smithi032.stdout:2/555: fdatasync d1/d34/d4a/d86/fb1 0 2024-05-12T04:01:23.423 INFO:tasks.workunit.client.1.smithi033.stdout:3/683: dwrite db/d17/d25/d32/d53/fb6 [0,4194304] 0 2024-05-12T04:01:23.425 INFO:tasks.workunit.client.0.smithi032.stdout:5/644: write d4/d7/f7e [720971,35424] 0 2024-05-12T04:01:23.427 INFO:tasks.workunit.client.1.smithi033.stdout:0/747: rename d6/dc/d58/d91 to d6/dc/dd1/dfb 0 2024-05-12T04:01:23.428 INFO:tasks.workunit.client.1.smithi033.stdout:0/748: read d6/db/f1a [305388,109122] 0 2024-05-12T04:01:23.430 INFO:tasks.workunit.client.1.smithi033.stdout:9/955: creat d2/d4/d2b/dc7/f153 x:0 0 0 2024-05-12T04:01:23.430 INFO:tasks.workunit.client.1.smithi033.stdout:0/749: chown d6/dc/d58/d6a/dec 410305 1 2024-05-12T04:01:23.433 INFO:tasks.workunit.client.1.smithi033.stdout:6/742: unlink d4/d6e/l7d 0 2024-05-12T04:01:23.435 INFO:tasks.workunit.client.0.smithi032.stdout:7/528: truncate fa 429793 0 2024-05-12T04:01:23.436 INFO:tasks.workunit.client.0.smithi032.stdout:7/529: chown dd/d1b/d1e/faa 1890385 1 2024-05-12T04:01:23.436 INFO:tasks.workunit.client.1.smithi033.stdout:4/719: chown dd/d11/d75/fa2 190 1 2024-05-12T04:01:23.441 INFO:tasks.workunit.client.1.smithi033.stdout:8/673: sync 2024-05-12T04:01:23.442 INFO:tasks.workunit.client.0.smithi032.stdout:4/490: creat d0/d3/df/f86 x:0 0 0 2024-05-12T04:01:23.446 INFO:tasks.workunit.client.0.smithi032.stdout:6/517: dread d7/d9/f4a [0,4194304] 0 2024-05-12T04:01:23.447 INFO:tasks.workunit.client.1.smithi033.stdout:1/630: write d11/f6b [1352043,70357] 0 2024-05-12T04:01:23.448 INFO:tasks.workunit.client.0.smithi032.stdout:5/645: symlink d4/d7/d14/d26/d33/d4a/d38/d3b/le5 0 2024-05-12T04:01:23.449 INFO:tasks.workunit.client.0.smithi032.stdout:5/646: stat d4/d7/d14/d39/d1b/f23 0 2024-05-12T04:01:23.453 INFO:tasks.workunit.client.0.smithi032.stdout:3/575: unlink d15/d1b/la0 0 2024-05-12T04:01:23.454 INFO:tasks.workunit.client.1.smithi033.stdout:5/622: getdents d3/d94/d54/d57/d70/da3/dbe/d16/d45/d62/d7f 0 2024-05-12T04:01:23.456 INFO:tasks.workunit.client.1.smithi033.stdout:7/917: dwrite de/d20/d38/f61 [0,4194304] 0 2024-05-12T04:01:23.456 INFO:tasks.workunit.client.1.smithi033.stdout:7/918: readlink de/df/l5d 0 2024-05-12T04:01:23.457 INFO:tasks.workunit.client.1.smithi033.stdout:3/684: write db/d17/d3a/f66 [1715961,91263] 0 2024-05-12T04:01:23.457 INFO:tasks.workunit.client.1.smithi033.stdout:7/919: stat de/d2f/d42 0 2024-05-12T04:01:23.458 INFO:tasks.workunit.client.1.smithi033.stdout:3/685: stat db/d59/f5b 0 2024-05-12T04:01:23.463 INFO:tasks.workunit.client.0.smithi032.stdout:4/491: mkdir d0/d3/d12/d1d/d63/d87 0 2024-05-12T04:01:23.464 INFO:tasks.workunit.client.1.smithi033.stdout:6/743: mknod d4/d1a/d67/d68/cf8 0 2024-05-12T04:01:23.465 INFO:tasks.workunit.client.1.smithi033.stdout:6/744: write d4/dad/fce [756006,37075] 0 2024-05-12T04:01:23.466 INFO:tasks.workunit.client.0.smithi032.stdout:6/518: mkdir d7/d97/db3 0 2024-05-12T04:01:23.469 INFO:tasks.workunit.client.0.smithi032.stdout:8/542: rename d0/d9/ca5 to d0/d9/d1b/d27/d88/d56/d63/cbd 0 2024-05-12T04:01:23.471 INFO:tasks.workunit.client.1.smithi033.stdout:2/701: creat dc/d17/d32/fe4 x:0 0 0 2024-05-12T04:01:23.472 INFO:tasks.workunit.client.0.smithi032.stdout:2/556: dwrite d1/d13/d70/f91 [0,4194304] 0 2024-05-12T04:01:23.472 INFO:tasks.workunit.client.1.smithi033.stdout:8/674: creat d1/d16/d27/d7c/fcf x:0 0 0 2024-05-12T04:01:23.472 INFO:tasks.workunit.client.1.smithi033.stdout:2/702: chown dc/d13/d1c/f21 54462866 1 2024-05-12T04:01:23.477 INFO:tasks.workunit.client.0.smithi032.stdout:6/519: chown d7/de/c5f 407082 1 2024-05-12T04:01:23.480 INFO:tasks.workunit.client.0.smithi032.stdout:5/647: symlink d4/d7/d14/d26/d33/d72/le6 0 2024-05-12T04:01:23.481 INFO:tasks.workunit.client.0.smithi032.stdout:5/648: chown d4/d7/d58/d5a/le4 15 1 2024-05-12T04:01:23.488 INFO:tasks.workunit.client.0.smithi032.stdout:2/557: symlink d1/d34/d96/d41/d2c/lb6 0 2024-05-12T04:01:23.491 INFO:tasks.workunit.client.0.smithi032.stdout:2/558: read d1/d34/d96/d41/d2c/f37 [427967,30436] 0 2024-05-12T04:01:23.494 INFO:tasks.workunit.client.0.smithi032.stdout:5/649: dread d4/d7/d14/d26/d33/d4a/d38/d3b/f48 [0,4194304] 0 2024-05-12T04:01:23.495 INFO:tasks.workunit.client.0.smithi032.stdout:1/553: rename d1/dd/l3a to d1/d7/d30/d35/lb1 0 2024-05-12T04:01:23.497 INFO:tasks.workunit.client.0.smithi032.stdout:7/530: getdents dd/d1b 0 2024-05-12T04:01:23.507 INFO:tasks.workunit.client.0.smithi032.stdout:3/576: dwrite d15/d24/f81 [0,4194304] 0 2024-05-12T04:01:23.508 INFO:tasks.workunit.client.0.smithi032.stdout:3/577: dread - d15/d20/d8b/fac zero size 2024-05-12T04:01:23.509 INFO:tasks.workunit.client.0.smithi032.stdout:4/492: creat d0/d3/f88 x:0 0 0 2024-05-12T04:01:23.512 INFO:tasks.workunit.client.0.smithi032.stdout:2/559: fsync d1/d34/d4a/f6d 0 2024-05-12T04:01:23.516 INFO:tasks.workunit.client.0.smithi032.stdout:3/578: dwrite d15/d20/d26/f52 [0,4194304] 0 2024-05-12T04:01:23.518 INFO:tasks.workunit.client.0.smithi032.stdout:5/650: readlink d4/d7/d14/d26/d33/d4a/d38/d3b/d68/ld4 0 2024-05-12T04:01:23.521 INFO:tasks.workunit.client.0.smithi032.stdout:5/651: dread - d4/d7/d14/d26/d33/d72/fa7 zero size 2024-05-12T04:01:23.522 INFO:tasks.workunit.client.0.smithi032.stdout:8/543: write d0/d9/d13/d21/d25/f50 [361004,54358] 0 2024-05-12T04:01:23.522 INFO:tasks.workunit.client.0.smithi032.stdout:1/554: rename d1/dd/ca6 to d1/d6a/cb2 0 2024-05-12T04:01:23.524 INFO:tasks.workunit.client.0.smithi032.stdout:1/555: dread - d1/d7/d30/fac zero size 2024-05-12T04:01:23.524 INFO:tasks.workunit.client.0.smithi032.stdout:8/544: dread - d0/d9/d1b/d27/d88/d56/d63/d8f/d9a/fb4 zero size 2024-05-12T04:01:23.525 INFO:tasks.workunit.client.0.smithi032.stdout:7/531: mknod dd/d1b/d1e/d45/d55/d72/cac 0 2024-05-12T04:01:23.579 INFO:tasks.workunit.client.0.smithi032.stdout:7/532: dwrite dd/d1b/d43/d5e/f92 [0,4194304] 0 2024-05-12T04:01:23.597 INFO:tasks.workunit.client.0.smithi032.stdout:7/533: dread dd/f13 [0,4194304] 0 2024-05-12T04:01:23.601 INFO:tasks.workunit.client.1.smithi033.stdout:3/686: sync 2024-05-12T04:01:23.643 INFO:tasks.workunit.client.0.smithi032.stdout:4/493: truncate d0/d3/d12/d1d/f5d 479455 0 2024-05-12T04:01:23.645 INFO:tasks.workunit.client.0.smithi032.stdout:2/560: write d1/d34/d3e/f9e [5107361,65375] 0 2024-05-12T04:01:23.648 INFO:tasks.workunit.client.0.smithi032.stdout:6/520: creat d7/de/fb4 x:0 0 0 2024-05-12T04:01:23.651 INFO:tasks.workunit.client.0.smithi032.stdout:3/579: mknod d15/d1b/d43/cce 0 2024-05-12T04:01:23.654 INFO:tasks.workunit.client.0.smithi032.stdout:5/652: unlink d4/d7/d14/d26/d33/d4a/fb6 0 2024-05-12T04:01:23.665 INFO:tasks.workunit.client.0.smithi032.stdout:7/534: creat dd/d1b/d1e/d45/d55/d60/d75/fad x:0 0 0 2024-05-12T04:01:23.667 INFO:tasks.workunit.client.0.smithi032.stdout:4/494: creat d0/d75/f89 x:0 0 0 2024-05-12T04:01:23.667 INFO:tasks.workunit.client.0.smithi032.stdout:8/545: write d0/d9/d1b/d27/d88/d56/d63/d8f/d9a/d28/f2b [4929260,82603] 0 2024-05-12T04:01:23.670 INFO:tasks.workunit.client.0.smithi032.stdout:8/546: write d0/d9/d13/d21/d25/f83 [1933977,79146] 0 2024-05-12T04:01:23.672 INFO:tasks.workunit.client.0.smithi032.stdout:8/547: stat d0/d9/d1b/d27/d88/d56/d63/d8f/db0 0 2024-05-12T04:01:23.673 INFO:tasks.workunit.client.1.smithi033.stdout:7/920: unlink de/d2b/d6c/da0/cdf 0 2024-05-12T04:01:23.674 INFO:tasks.workunit.client.1.smithi033.stdout:0/750: dwrite d6/dc/d58/f59 [0,4194304] 0 2024-05-12T04:01:23.675 INFO:tasks.workunit.client.0.smithi032.stdout:4/495: dread d0/d3/d12/f5b [0,4194304] 0 2024-05-12T04:01:23.675 INFO:tasks.workunit.client.0.smithi032.stdout:5/653: creat d4/d7/d14/d26/d33/fe7 x:0 0 0 2024-05-12T04:01:23.676 INFO:tasks.workunit.client.1.smithi033.stdout:6/745: symlink d4/d43/d57/lf9 0 2024-05-12T04:01:23.677 INFO:tasks.workunit.client.1.smithi033.stdout:6/746: write d4/d6e/fb3 [1387539,43970] 0 2024-05-12T04:01:23.679 INFO:tasks.workunit.client.1.smithi033.stdout:4/720: rename dd/d7a/d87/d7e/da7/d38/f78 to dd/d7a/d87/d7e/fe5 0 2024-05-12T04:01:23.679 INFO:tasks.workunit.client.0.smithi032.stdout:5/654: truncate d4/d7/d14/d26/d33/d4a/f51 1188624 0 2024-05-12T04:01:23.680 INFO:tasks.workunit.client.0.smithi032.stdout:4/496: dread d0/f3d [0,4194304] 0 2024-05-12T04:01:23.681 INFO:tasks.workunit.client.0.smithi032.stdout:5/655: dread - d4/d7/d14/d26/d33/d4a/d38/fd7 zero size 2024-05-12T04:01:23.681 INFO:tasks.workunit.client.0.smithi032.stdout:4/497: dread - d0/d3/d24/f76 zero size 2024-05-12T04:01:23.686 INFO:tasks.workunit.client.1.smithi033.stdout:4/721: dwrite dd/d69/f97 [0,4194304] 0 2024-05-12T04:01:23.687 INFO:tasks.workunit.client.1.smithi033.stdout:4/722: stat dd/d11/d5b/fb5 0 2024-05-12T04:01:23.697 INFO:tasks.workunit.client.1.smithi033.stdout:4/723: dwrite dd/d7a/d87/d7e/da7/fce [0,4194304] 0 2024-05-12T04:01:23.699 INFO:tasks.workunit.client.1.smithi033.stdout:6/747: dread d4/d1a/d67/d68/f8b [0,4194304] 0 2024-05-12T04:01:23.699 INFO:tasks.workunit.client.0.smithi032.stdout:0/579: sync 2024-05-12T04:01:23.801 INFO:tasks.workunit.client.1.smithi033.stdout:3/687: chown db/d17/c4f 64998 1 2024-05-12T04:01:23.801 INFO:tasks.workunit.client.1.smithi033.stdout:3/688: fdatasync db/d42/fe2 0 2024-05-12T04:01:23.802 INFO:tasks.workunit.client.0.smithi032.stdout:7/535: dread dd/d1b/d1e/d45/d64/f8b [0,4194304] 0 2024-05-12T04:01:23.803 INFO:tasks.workunit.client.0.smithi032.stdout:7/536: fsync dd/d1b/d43/d5e/f5a 0 2024-05-12T04:01:23.805 INFO:tasks.workunit.client.0.smithi032.stdout:7/537: chown dd/d1b/d43/d5e/d37/l8c 184405351 1 2024-05-12T04:01:23.808 INFO:tasks.workunit.client.0.smithi032.stdout:2/561: mkdir d1/d34/da5/db3/db7 0 2024-05-12T04:01:23.810 INFO:tasks.workunit.client.1.smithi033.stdout:7/921: truncate de/d20/f96 36407 0 2024-05-12T04:01:23.811 INFO:tasks.workunit.client.0.smithi032.stdout:7/538: dwrite f5 [4194304,4194304] 0 2024-05-12T04:01:23.811 INFO:tasks.workunit.client.1.smithi033.stdout:7/922: chown de/df/d17/l33 643540 1 2024-05-12T04:01:23.817 INFO:tasks.workunit.client.1.smithi033.stdout:0/751: creat d6/d11/d98/ffc x:0 0 0 2024-05-12T04:01:23.818 INFO:tasks.workunit.client.1.smithi033.stdout:0/752: chown d6/dc/f16 1 1 2024-05-12T04:01:23.821 INFO:tasks.workunit.client.0.smithi032.stdout:8/548: dread - d0/d7/f4a zero size 2024-05-12T04:01:23.823 INFO:tasks.workunit.client.1.smithi033.stdout:8/675: dwrite d1/d16/fbe [0,4194304] 0 2024-05-12T04:01:23.828 INFO:tasks.workunit.client.1.smithi033.stdout:5/623: rename d3/d27 to d3/d94/dc8 0 2024-05-12T04:01:23.838 INFO:tasks.workunit.client.0.smithi032.stdout:0/580: creat df/d28/fc6 x:0 0 0 2024-05-12T04:01:23.840 INFO:tasks.workunit.client.1.smithi033.stdout:8/676: dread d1/f2 [0,4194304] 0 2024-05-12T04:01:23.847 INFO:tasks.workunit.client.1.smithi033.stdout:1/631: link d11/d1c/f94 d11/d17/d84/da3/fd6 0 2024-05-12T04:01:23.853 INFO:tasks.workunit.client.0.smithi032.stdout:2/562: symlink d1/d34/d3e/d67/lb8 0 2024-05-12T04:01:23.858 INFO:tasks.workunit.client.1.smithi033.stdout:9/956: link d2/f111 d2/d4/d2b/d49/f154 0 2024-05-12T04:01:23.860 INFO:tasks.workunit.client.0.smithi032.stdout:2/563: dwrite d1/d34/d95/fa1 [0,4194304] 0 2024-05-12T04:01:23.862 INFO:tasks.workunit.client.0.smithi032.stdout:2/564: stat d1/d34/d4a/l5a 0 2024-05-12T04:01:23.863 INFO:tasks.workunit.client.1.smithi033.stdout:7/923: unlink de/d20/f21 0 2024-05-12T04:01:23.874 INFO:tasks.workunit.client.1.smithi033.stdout:0/753: rename d6/dc/d58/d6a/l7b to d6/dc/d39/lfd 0 2024-05-12T04:01:23.876 INFO:tasks.workunit.client.0.smithi032.stdout:9/472: sync 2024-05-12T04:01:23.881 INFO:tasks.workunit.client.1.smithi033.stdout:4/724: symlink dd/d7a/d87/d7e/da7/d66/d42/le6 0 2024-05-12T04:01:23.881 INFO:tasks.workunit.client.1.smithi033.stdout:4/725: chown dd/d11/d75/dd4 69 1 2024-05-12T04:01:23.886 INFO:tasks.workunit.client.1.smithi033.stdout:1/632: chown d11/d17/d27/d36/d3f/d69/c8e 1640 1 2024-05-12T04:01:23.889 INFO:tasks.workunit.client.0.smithi032.stdout:2/565: creat d1/d4d/d5e/fb9 x:0 0 0 2024-05-12T04:01:23.892 INFO:tasks.workunit.client.1.smithi033.stdout:3/689: symlink db/d17/d3e/d65/d91/d97/de8/lec 0 2024-05-12T04:01:23.894 INFO:tasks.workunit.client.0.smithi032.stdout:9/473: symlink d0/d84/l9d 0 2024-05-12T04:01:23.895 INFO:tasks.workunit.client.1.smithi033.stdout:9/957: fdatasync d2/d4/d2b/d6d/d7a/fed 0 2024-05-12T04:01:23.899 INFO:tasks.workunit.client.1.smithi033.stdout:7/924: rmdir de/df/d7a/dc0 39 2024-05-12T04:01:23.905 INFO:tasks.workunit.client.1.smithi033.stdout:5/624: rename d3/d94/d54/d57/d73/d74/d7e to d3/d94/d54/d57/d70/da3/dc9 0 2024-05-12T04:01:23.910 INFO:tasks.workunit.client.0.smithi032.stdout:8/549: getdents d0/d9/d13 0 2024-05-12T04:01:23.911 INFO:tasks.workunit.client.1.smithi033.stdout:5/625: dwrite d3/d94/d54/d98/f9f [0,4194304] 0 2024-05-12T04:01:23.919 INFO:tasks.workunit.client.1.smithi033.stdout:3/690: fsync db/d17/d3a/f3b 0 2024-05-12T04:01:23.919 INFO:tasks.workunit.client.0.smithi032.stdout:8/550: dwrite d0/d9/d13/d21/d25/f50 [0,4194304] 0 2024-05-12T04:01:23.919 INFO:tasks.workunit.client.0.smithi032.stdout:8/551: chown d0/d9 1083801 1 2024-05-12T04:01:23.920 INFO:tasks.workunit.client.0.smithi032.stdout:2/566: getdents d1/d13/d70 0 2024-05-12T04:01:23.926 INFO:tasks.workunit.client.1.smithi033.stdout:0/754: truncate d6/db/d2c/d73/f7a 2830913 0 2024-05-12T04:01:23.929 INFO:tasks.workunit.client.0.smithi032.stdout:5/656: sync 2024-05-12T04:01:23.934 INFO:tasks.workunit.client.0.smithi032.stdout:2/567: dread d1/d34/d96/f60 [0,4194304] 0 2024-05-12T04:01:23.939 INFO:tasks.workunit.client.1.smithi033.stdout:5/626: dread d3/d12/d29/f3d [0,4194304] 0 2024-05-12T04:01:23.944 INFO:tasks.workunit.client.1.smithi033.stdout:9/958: unlink d2/d4/d2b/c89 0 2024-05-12T04:01:23.945 INFO:tasks.workunit.client.1.smithi033.stdout:9/959: read - d2/dd/d39/d72/f138 zero size 2024-05-12T04:01:23.946 INFO:tasks.workunit.client.0.smithi032.stdout:2/568: creat d1/d34/d3e/d67/fba x:0 0 0 2024-05-12T04:01:23.947 INFO:tasks.workunit.client.0.smithi032.stdout:9/474: mkdir d0/d3/d6/d7/da/d9e 0 2024-05-12T04:01:23.949 INFO:tasks.workunit.client.1.smithi033.stdout:0/755: fdatasync d6/dc/d58/d6a/fc9 0 2024-05-12T04:01:23.952 INFO:tasks.workunit.client.0.smithi032.stdout:2/569: creat d1/d34/d96/d8a/fbb x:0 0 0 2024-05-12T04:01:23.955 INFO:tasks.workunit.client.1.smithi033.stdout:1/633: creat d11/d17/fd7 x:0 0 0 2024-05-12T04:01:23.957 INFO:tasks.workunit.client.1.smithi033.stdout:5/627: symlink d3/d94/d54/d57/d70/da3/dc9/lca 0 2024-05-12T04:01:23.959 INFO:tasks.workunit.client.0.smithi032.stdout:2/570: symlink d1/d34/d96/d41/lbc 0 2024-05-12T04:01:23.959 INFO:tasks.workunit.client.0.smithi032.stdout:2/571: dread - d1/d4d/fad zero size 2024-05-12T04:01:23.961 INFO:tasks.workunit.client.0.smithi032.stdout:2/572: fsync d1/d34/d96/d41/d2c/f90 0 2024-05-12T04:01:23.961 INFO:tasks.workunit.client.1.smithi033.stdout:9/960: mkdir d2/d4/d2b/d6d/db5/d119/d155 0 2024-05-12T04:01:23.961 INFO:tasks.workunit.client.0.smithi032.stdout:2/573: read d1/d4d/f98 [83179,32560] 0 2024-05-12T04:01:23.963 INFO:tasks.workunit.client.1.smithi033.stdout:5/628: dwrite d3/f52 [0,4194304] 0 2024-05-12T04:01:23.964 INFO:tasks.workunit.client.0.smithi032.stdout:9/475: dread d0/d3/d6/d7/da/f1a [4194304,4194304] 0 2024-05-12T04:01:23.965 INFO:tasks.workunit.client.1.smithi033.stdout:5/629: write d3/d94/d54/d98/f9f [2977999,18827] 0 2024-05-12T04:01:23.965 INFO:tasks.workunit.client.0.smithi032.stdout:9/476: readlink d0/d3/d6/l5b 0 2024-05-12T04:01:23.966 INFO:tasks.workunit.client.1.smithi033.stdout:5/630: chown d3/d12/d25/db0/cbf 166215654 1 2024-05-12T04:01:23.973 INFO:tasks.workunit.client.1.smithi033.stdout:1/634: mkdir d11/d17/d27/d36/d3f/d69/d87/d96/dd8 0 2024-05-12T04:01:23.986 INFO:tasks.workunit.client.1.smithi033.stdout:5/631: mkdir d3/d94/dc8/dcb 0 2024-05-12T04:01:23.987 INFO:tasks.workunit.client.1.smithi033.stdout:0/756: creat d6/d11/dc7/dd9/ffe x:0 0 0 2024-05-12T04:01:23.989 INFO:tasks.workunit.client.1.smithi033.stdout:5/632: rmdir d3/d94/d54/d57/d70 39 2024-05-12T04:01:23.990 INFO:tasks.workunit.client.1.smithi033.stdout:5/633: write d3/d12/f1f [1413585,53582] 0 2024-05-12T04:01:23.994 INFO:tasks.workunit.client.1.smithi033.stdout:1/635: rename d11/d17/d27/d36/d3f/d69/d87/d96/fcd to d11/d17/d27/d36/d3f/d69/d87/da4/fd9 0 2024-05-12T04:01:23.998 INFO:tasks.workunit.client.1.smithi033.stdout:5/634: mkdir d3/d94/d54/d98/dcc 0 2024-05-12T04:01:23.999 INFO:tasks.workunit.client.1.smithi033.stdout:0/757: truncate d6/f6b 1006097 0 2024-05-12T04:01:24.005 INFO:tasks.workunit.client.1.smithi033.stdout:0/758: dwrite d6/d11/d57/f21 [0,4194304] 0 2024-05-12T04:01:24.007 INFO:tasks.workunit.client.1.smithi033.stdout:5/635: fdatasync d3/d94/d54/d57/d70/da3/dbe/d16/f44 0 2024-05-12T04:01:24.020 INFO:tasks.workunit.client.1.smithi033.stdout:5/636: rename d3/fa5 to d3/d12/d29/fcd 0 2024-05-12T04:01:24.024 INFO:tasks.workunit.client.1.smithi033.stdout:5/637: creat d3/d12/dac/fce x:0 0 0 2024-05-12T04:01:24.037 INFO:tasks.workunit.client.1.smithi033.stdout:5/638: dread d3/d94/d54/d57/d70/da3/dbe/f30 [0,4194304] 0 2024-05-12T04:01:24.042 INFO:tasks.workunit.client.1.smithi033.stdout:5/639: mkdir d3/d94/d54/d57/d70/da3/dbe/d16/d45/d62/d7f/dcf 0 2024-05-12T04:01:24.046 INFO:tasks.workunit.client.0.smithi032.stdout:6/521: dwrite d7/f51 [0,4194304] 0 2024-05-12T04:01:24.056 INFO:tasks.workunit.client.1.smithi033.stdout:6/748: write d4/d1a/d4e/f71 [2428395,35479] 0 2024-05-12T04:01:24.057 INFO:tasks.workunit.client.1.smithi033.stdout:6/749: stat d4/d1a/d67/ldb 0 2024-05-12T04:01:24.057 INFO:tasks.workunit.client.1.smithi033.stdout:6/750: readlink d4/d6e/lc6 0 2024-05-12T04:01:24.058 INFO:tasks.workunit.client.1.smithi033.stdout:2/703: write dc/d13/d1f/d6e/f81 [2889407,43113] 0 2024-05-12T04:01:24.061 INFO:tasks.workunit.client.1.smithi033.stdout:2/704: getdents dc/d17/d27/d2f/d49/ddd 0 2024-05-12T04:01:24.064 INFO:tasks.workunit.client.0.smithi032.stdout:6/522: dread d7/f2b [0,4194304] 0 2024-05-12T04:01:24.064 INFO:tasks.workunit.client.1.smithi033.stdout:2/705: unlink dc/d17/d32/d84/fa0 0 2024-05-12T04:01:24.065 INFO:tasks.workunit.client.1.smithi033.stdout:2/706: chown dc/d3b/d88/c7f 863190644 1 2024-05-12T04:01:24.066 INFO:tasks.workunit.client.1.smithi033.stdout:2/707: write dc/f8d [878922,55878] 0 2024-05-12T04:01:24.072 INFO:tasks.workunit.client.0.smithi032.stdout:6/523: dwrite d7/f2b [0,4194304] 0 2024-05-12T04:01:24.076 INFO:tasks.workunit.client.0.smithi032.stdout:4/498: truncate d0/d3/d31/d80/d84/f64 1653336 0 2024-05-12T04:01:24.079 INFO:tasks.workunit.client.0.smithi032.stdout:4/499: truncate d0/f65 4480659 0 2024-05-12T04:01:24.080 INFO:tasks.workunit.client.1.smithi033.stdout:8/677: write d1/d16/d27/d4d/f5e [1451558,34133] 0 2024-05-12T04:01:24.081 INFO:tasks.workunit.client.0.smithi032.stdout:4/500: readlink d0/d3/l1e 0 2024-05-12T04:01:24.081 INFO:tasks.workunit.client.0.smithi032.stdout:4/501: chown d0/d3/l1e 1946940 1 2024-05-12T04:01:24.083 INFO:tasks.workunit.client.1.smithi033.stdout:8/678: mkdir d1/d16/d27/d4d/d73/dd0 0 2024-05-12T04:01:24.085 INFO:tasks.workunit.client.0.smithi032.stdout:4/502: creat d0/d75/f8a x:0 0 0 2024-05-12T04:01:24.085 INFO:tasks.workunit.client.0.smithi032.stdout:4/503: readlink d0/d3/l1e 0 2024-05-12T04:01:24.086 INFO:tasks.workunit.client.0.smithi032.stdout:4/504: dread - d0/d3/d24/f76 zero size 2024-05-12T04:01:24.086 INFO:tasks.workunit.client.1.smithi033.stdout:0/759: mkdir d6/dc/d58/dff 0 2024-05-12T04:01:24.087 INFO:tasks.workunit.client.0.smithi032.stdout:4/505: dread - d0/d3/d12/d1d/d63/f70 zero size 2024-05-12T04:01:24.088 INFO:tasks.workunit.client.1.smithi033.stdout:0/760: truncate d6/db/d2c/d73/fd8 494773 0 2024-05-12T04:01:24.088 INFO:tasks.workunit.client.0.smithi032.stdout:4/506: write d0/d3/df/f29 [2999245,2905] 0 2024-05-12T04:01:24.090 INFO:tasks.workunit.client.1.smithi033.stdout:3/691: sync 2024-05-12T04:01:24.090 INFO:tasks.workunit.client.1.smithi033.stdout:7/925: sync 2024-05-12T04:01:24.091 INFO:tasks.workunit.client.1.smithi033.stdout:0/761: creat d6/db/d2c/d73/daf/da4/f100 x:0 0 0 2024-05-12T04:01:24.093 INFO:tasks.workunit.client.1.smithi033.stdout:7/926: symlink de/df/d17/d1e/dc4/l137 0 2024-05-12T04:01:24.096 INFO:tasks.workunit.client.1.smithi033.stdout:3/692: mknod db/d17/d3a/d93/dc8/ced 0 2024-05-12T04:01:24.097 INFO:tasks.workunit.client.1.smithi033.stdout:0/762: mkdir d6/dc/d6e/d101 0 2024-05-12T04:01:24.097 INFO:tasks.workunit.client.0.smithi032.stdout:0/581: write df/d22/f39 [26734,25070] 0 2024-05-12T04:01:24.099 INFO:tasks.workunit.client.0.smithi032.stdout:0/582: dread - df/d10/d3d/d4a/d84/f9c zero size 2024-05-12T04:01:24.100 INFO:tasks.workunit.client.0.smithi032.stdout:4/507: getdents d0/d3/d24 0 2024-05-12T04:01:24.102 INFO:tasks.workunit.client.0.smithi032.stdout:0/583: read df/d10/d3d/d83/d9f/faa [197024,31089] 0 2024-05-12T04:01:24.106 INFO:tasks.workunit.client.0.smithi032.stdout:4/508: readlink d0/d3/d12/d1d/l4f 0 2024-05-12T04:01:24.111 INFO:tasks.workunit.client.1.smithi033.stdout:7/927: creat de/d20/d38/f138 x:0 0 0 2024-05-12T04:01:24.113 INFO:tasks.workunit.client.0.smithi032.stdout:0/584: unlink df/cac 0 2024-05-12T04:01:24.113 INFO:tasks.workunit.client.0.smithi032.stdout:4/509: dwrite d0/d3/f88 [0,4194304] 0 2024-05-12T04:01:24.115 INFO:tasks.workunit.client.1.smithi033.stdout:0/763: dread d6/dc/f1d [4194304,4194304] 0 2024-05-12T04:01:24.117 INFO:tasks.workunit.client.1.smithi033.stdout:0/764: chown d6/db/d2c/d73/daf/da4/f100 21012683 1 2024-05-12T04:01:24.117 INFO:tasks.workunit.client.1.smithi033.stdout:4/726: write dd/d11/f1d [3864330,3718] 0 2024-05-12T04:01:24.118 INFO:tasks.workunit.client.0.smithi032.stdout:0/585: mkdir df/d30/d32/d60/dc7 0 2024-05-12T04:01:24.126 INFO:tasks.workunit.client.0.smithi032.stdout:8/552: truncate d0/d9/d1b/d27/d88/d56/d63/d8f/d9a/d28/f2b 6691981 0 2024-05-12T04:01:24.126 INFO:tasks.workunit.client.1.smithi033.stdout:0/765: write d6/dc/f56 [4276422,58443] 0 2024-05-12T04:01:24.128 INFO:tasks.workunit.client.1.smithi033.stdout:0/766: creat d6/d11/d98/f102 x:0 0 0 2024-05-12T04:01:24.130 INFO:tasks.workunit.client.0.smithi032.stdout:8/553: fdatasync d0/d9/d1b/d27/d88/d56/d63/d8f/d9a/d28/d3e/f57 0 2024-05-12T04:01:24.132 INFO:tasks.workunit.client.1.smithi033.stdout:4/727: truncate dd/d11/f2f 2697325 0 2024-05-12T04:01:24.134 INFO:tasks.workunit.client.1.smithi033.stdout:4/728: truncate dd/d7a/d87/d7e/da7/d66/f6a 1691620 0 2024-05-12T04:01:24.136 INFO:tasks.workunit.client.1.smithi033.stdout:0/767: truncate d6/dc/fb3 930310 0 2024-05-12T04:01:24.137 INFO:tasks.workunit.client.0.smithi032.stdout:8/554: dwrite d0/d97/fa6 [0,4194304] 0 2024-05-12T04:01:24.137 INFO:tasks.workunit.client.1.smithi033.stdout:0/768: chown d6/db/d2c/l3a 1355752584 1 2024-05-12T04:01:24.137 INFO:tasks.workunit.client.0.smithi032.stdout:4/510: dread d0/d3/d24/f71 [0,4194304] 0 2024-05-12T04:01:24.138 INFO:tasks.workunit.client.1.smithi033.stdout:0/769: chown d6/dc/f18 10964 1 2024-05-12T04:01:24.139 INFO:tasks.workunit.client.0.smithi032.stdout:4/511: dread - d0/d3/d12/d1d/f74 zero size 2024-05-12T04:01:24.140 INFO:tasks.workunit.client.0.smithi032.stdout:1/556: rename d1/d7/d49/l51 to d1/d7/d30/lb3 0 2024-05-12T04:01:24.142 INFO:tasks.workunit.client.1.smithi033.stdout:4/729: link dd/d7a/d87/fb7 dd/d11/d75/fe7 0 2024-05-12T04:01:24.144 INFO:tasks.workunit.client.1.smithi033.stdout:0/770: rmdir d6/d11/dc7 39 2024-05-12T04:01:24.146 INFO:tasks.workunit.client.0.smithi032.stdout:3/580: rename d15/c1f to d15/d24/d2a/d5b/ccf 0 2024-05-12T04:01:24.147 INFO:tasks.workunit.client.0.smithi032.stdout:3/581: readlink d15/d1b/d43/l66 0 2024-05-12T04:01:24.147 INFO:tasks.workunit.client.1.smithi033.stdout:4/730: dread - dd/d11/fac zero size 2024-05-12T04:01:24.148 INFO:tasks.workunit.client.0.smithi032.stdout:4/512: creat d0/d3/d31/f8b x:0 0 0 2024-05-12T04:01:24.149 INFO:tasks.workunit.client.0.smithi032.stdout:4/513: truncate d0/f6f 1002809 0 2024-05-12T04:01:24.151 INFO:tasks.workunit.client.0.smithi032.stdout:4/514: dread - d0/d3/df/d4e/f69 zero size 2024-05-12T04:01:24.152 INFO:tasks.workunit.client.0.smithi032.stdout:2/574: rename d1/d34/d3e/d67/f6a to d1/d34/da5/fbd 0 2024-05-12T04:01:24.153 INFO:tasks.workunit.client.1.smithi033.stdout:4/731: read dd/d7a/d87/d7e/da7/d66/f92 [867787,102344] 0 2024-05-12T04:01:24.154 INFO:tasks.workunit.client.0.smithi032.stdout:3/582: fsync d15/d20/d22/d7a/f9f 0 2024-05-12T04:01:24.155 INFO:tasks.workunit.client.0.smithi032.stdout:3/583: readlink d15/d20/d22/d7a/lb9 0 2024-05-12T04:01:24.156 INFO:tasks.workunit.client.0.smithi032.stdout:3/584: dread - d15/d20/d22/d7a/f88 zero size 2024-05-12T04:01:24.158 INFO:tasks.workunit.client.1.smithi033.stdout:4/732: mknod dd/d7a/d87/d7e/da7/d66/d42/d9c/ce8 0 2024-05-12T04:01:24.161 INFO:tasks.workunit.client.1.smithi033.stdout:4/733: fdatasync dd/d7a/d87/d7e/da7/d66/d42/f82 0 2024-05-12T04:01:24.163 INFO:tasks.workunit.client.0.smithi032.stdout:5/657: mkdir d4/d7/d14/de8 0 2024-05-12T04:01:24.165 INFO:tasks.workunit.client.0.smithi032.stdout:8/555: rename d0/d9/d1b/d27/d59/c61 to d0/d9/d1b/d27/d88/d56/d63/d8f/db0/cbe 0 2024-05-12T04:01:24.167 INFO:tasks.workunit.client.0.smithi032.stdout:2/575: mknod d1/d13/da0/dae/cbe 0 2024-05-12T04:01:24.169 INFO:tasks.workunit.client.0.smithi032.stdout:9/477: write d0/d3/d6/d7/d27/f31 [635412,105419] 0 2024-05-12T04:01:24.172 INFO:tasks.workunit.client.0.smithi032.stdout:3/585: truncate d15/d33/f5c 3547852 0 2024-05-12T04:01:24.174 INFO:tasks.workunit.client.0.smithi032.stdout:5/658: symlink d4/d7/d14/d26/d33/d88/le9 0 2024-05-12T04:01:24.176 INFO:tasks.workunit.client.0.smithi032.stdout:2/576: symlink d1/d13/da0/lbf 0 2024-05-12T04:01:24.177 INFO:tasks.workunit.client.0.smithi032.stdout:8/556: dread d0/d9/f2d [0,4194304] 0 2024-05-12T04:01:24.178 INFO:tasks.workunit.client.0.smithi032.stdout:2/577: write d1/d34/d96/d41/f40 [335561,65189] 0 2024-05-12T04:01:24.178 INFO:tasks.workunit.client.1.smithi033.stdout:9/961: dwrite d2/d4/d2b/d6d/d2f/d5f/f97 [0,4194304] 0 2024-05-12T04:01:24.188 INFO:tasks.workunit.client.0.smithi032.stdout:8/557: mknod d0/d9/d1b/d27/d4c/d4e/d90/cbf 0 2024-05-12T04:01:24.193 INFO:tasks.workunit.client.1.smithi033.stdout:6/751: unlink d4/d1a/d67/cca 0 2024-05-12T04:01:24.193 INFO:tasks.workunit.client.0.smithi032.stdout:3/586: dread d15/d20/d8b/d9b/fcd [4194304,4194304] 0 2024-05-12T04:01:24.193 INFO:tasks.workunit.client.1.smithi033.stdout:6/752: stat d4/d6e/fb3 0 2024-05-12T04:01:24.194 INFO:tasks.workunit.client.0.smithi032.stdout:3/587: write d15/f4d [48649,23381] 0 2024-05-12T04:01:24.198 INFO:tasks.workunit.client.1.smithi033.stdout:1/636: write d11/d1c/d43/f4d [702349,57928] 0 2024-05-12T04:01:24.199 INFO:tasks.workunit.client.1.smithi033.stdout:6/753: readlink d4/l69 0 2024-05-12T04:01:24.200 INFO:tasks.workunit.client.1.smithi033.stdout:6/754: readlink d4/d1a/d6c/lbf 0 2024-05-12T04:01:24.201 INFO:tasks.workunit.client.0.smithi032.stdout:3/588: mkdir d15/d1b/d43/dd0 0 2024-05-12T04:01:24.203 INFO:tasks.workunit.client.1.smithi033.stdout:1/637: creat d11/d17/d27/d36/d3f/d69/d87/da4/fda x:0 0 0 2024-05-12T04:01:24.210 INFO:tasks.workunit.client.0.smithi032.stdout:7/539: mkdir dd/dae 0 2024-05-12T04:01:24.216 INFO:tasks.workunit.client.0.smithi032.stdout:7/540: dwrite dd/d1b/d1e/d45/d55/d60/d75/f9b [0,4194304] 0 2024-05-12T04:01:24.217 INFO:tasks.workunit.client.0.smithi032.stdout:3/589: dread d15/d20/d22/d7a/f85 [0,4194304] 0 2024-05-12T04:01:24.217 INFO:tasks.workunit.client.1.smithi033.stdout:0/771: sync 2024-05-12T04:01:24.318 INFO:tasks.workunit.client.1.smithi033.stdout:7/928: symlink de/d2b/l139 0 2024-05-12T04:01:24.322 INFO:tasks.workunit.client.1.smithi033.stdout:4/734: rename dd/d7a/d87/d7e/fcf to dd/d69/fe9 0 2024-05-12T04:01:24.322 INFO:tasks.workunit.client.1.smithi033.stdout:4/735: fdatasync dd/d7a/d87/d7e/da7/fc7 0 2024-05-12T04:01:24.323 INFO:tasks.workunit.client.1.smithi033.stdout:8/679: creat d1/d4/d1d/fd1 x:0 0 0 2024-05-12T04:01:24.332 INFO:tasks.workunit.client.1.smithi033.stdout:5/640: dwrite d3/d94/d54/d57/d70/da3/dbe/d16/d45/f68 [0,4194304] 0 2024-05-12T04:01:24.333 INFO:tasks.workunit.client.0.smithi032.stdout:5/659: symlink d4/d7/d14/d26/lea 0 2024-05-12T04:01:24.339 INFO:tasks.workunit.client.1.smithi033.stdout:4/736: creat dd/d11/d75/dd4/fea x:0 0 0 2024-05-12T04:01:24.344 INFO:tasks.workunit.client.1.smithi033.stdout:2/708: dwrite f7 [0,4194304] 0 2024-05-12T04:01:24.345 INFO:tasks.workunit.client.1.smithi033.stdout:8/680: mkdir d1/d4/d1d/d36/dd2 0 2024-05-12T04:01:24.347 INFO:tasks.workunit.client.0.smithi032.stdout:6/524: dwrite d7/d9/d48/faa [0,4194304] 0 2024-05-12T04:01:24.347 INFO:tasks.workunit.client.0.smithi032.stdout:6/525: dread - d7/d9/d60/f9a zero size 2024-05-12T04:01:24.357 INFO:tasks.workunit.client.1.smithi033.stdout:5/641: mknod d3/d12/d29/d2b/d8e/cd0 0 2024-05-12T04:01:24.361 INFO:tasks.workunit.client.1.smithi033.stdout:2/709: mknod dc/d13/d65/db1/ce5 0 2024-05-12T04:01:24.362 INFO:tasks.workunit.client.1.smithi033.stdout:2/710: chown dc/d3b/c44 7431156 1 2024-05-12T04:01:24.364 INFO:tasks.workunit.client.1.smithi033.stdout:2/711: dread - dc/d3b/d88/fd2 zero size 2024-05-12T04:01:24.364 INFO:tasks.workunit.client.1.smithi033.stdout:4/737: mkdir dd/d7a/d87/d7e/da7/d38/deb 0 2024-05-12T04:01:24.366 INFO:tasks.workunit.client.1.smithi033.stdout:8/681: mknod d1/d4/d1d/d36/d3c/d9f/cd3 0 2024-05-12T04:01:24.369 INFO:tasks.workunit.client.1.smithi033.stdout:3/693: dwrite db/d17/d3a/d67/d76/fb2 [0,4194304] 0 2024-05-12T04:01:24.378 INFO:tasks.workunit.client.1.smithi033.stdout:8/682: creat d1/d16/d27/d4d/fd4 x:0 0 0 2024-05-12T04:01:24.379 INFO:tasks.workunit.client.0.smithi032.stdout:0/586: write df/d10/d3d/d4a/f29 [679892,111814] 0 2024-05-12T04:01:24.379 INFO:tasks.workunit.client.0.smithi032.stdout:8/558: write d0/d9/d1b/d27/d88/d56/d63/d8f/d9a/d28/d3e/f57 [1856303,103982] 0 2024-05-12T04:01:24.381 INFO:tasks.workunit.client.0.smithi032.stdout:8/559: readlink d0/d9/d1b/d66/d75/l7d 0 2024-05-12T04:01:24.381 INFO:tasks.workunit.client.0.smithi032.stdout:0/587: chown df/d28/c44 33482 1 2024-05-12T04:01:24.382 INFO:tasks.workunit.client.0.smithi032.stdout:0/588: dread - df/d10/f64 zero size 2024-05-12T04:01:24.389 INFO:tasks.workunit.client.0.smithi032.stdout:3/590: symlink d15/d20/ld1 0 2024-05-12T04:01:24.392 INFO:tasks.workunit.client.0.smithi032.stdout:7/541: mkdir dd/d1b/d1e/d45/d55/d60/d9f/daf 0 2024-05-12T04:01:24.396 INFO:tasks.workunit.client.0.smithi032.stdout:5/660: creat d4/d7/d14/d26/d3c/feb x:0 0 0 2024-05-12T04:01:24.398 INFO:tasks.workunit.client.1.smithi033.stdout:3/694: dread db/d17/d25/d32/d3c/d44/f55 [4194304,4194304] 0 2024-05-12T04:01:24.399 INFO:tasks.workunit.client.0.smithi032.stdout:5/661: chown d4/d7/d58/f7b 19851 1 2024-05-12T04:01:24.399 INFO:tasks.workunit.client.0.smithi032.stdout:1/557: dwrite d1/d6a/d72/f88 [0,4194304] 0 2024-05-12T04:01:24.399 INFO:tasks.workunit.client.0.smithi032.stdout:8/560: chown d0/d9/f8e 475 1 2024-05-12T04:01:24.401 INFO:tasks.workunit.client.1.smithi033.stdout:5/642: getdents d3/d94/d54/d57/d70/da3/dbe/d16 0 2024-05-12T04:01:24.406 INFO:tasks.workunit.client.1.smithi033.stdout:4/738: getdents dd 0 2024-05-12T04:01:24.407 INFO:tasks.workunit.client.1.smithi033.stdout:5/643: dwrite d3/f7d [0,4194304] 0 2024-05-12T04:01:24.407 INFO:tasks.workunit.client.1.smithi033.stdout:3/695: creat db/d17/d3a/d67/d76/da0/fee x:0 0 0 2024-05-12T04:01:24.408 INFO:tasks.workunit.client.1.smithi033.stdout:4/739: stat dd/d7a/d87/d7e/da7/d66/f92 0 2024-05-12T04:01:24.410 INFO:tasks.workunit.client.1.smithi033.stdout:4/740: write dd/d69/f7d [746560,104989] 0 2024-05-12T04:01:24.411 INFO:tasks.workunit.client.1.smithi033.stdout:4/741: write dd/f3b [4046884,71208] 0 2024-05-12T04:01:24.413 INFO:tasks.workunit.client.0.smithi032.stdout:7/542: rename dd/d1b/l47 to dd/d1b/d1e/d45/d55/d60/d9f/lb0 0 2024-05-12T04:01:24.415 INFO:tasks.workunit.client.0.smithi032.stdout:6/526: creat d7/d9/d48/d67/d8d/fb5 x:0 0 0 2024-05-12T04:01:24.416 INFO:tasks.workunit.client.0.smithi032.stdout:6/527: write d7/d9/d48/faa [597951,32846] 0 2024-05-12T04:01:24.417 INFO:tasks.workunit.client.0.smithi032.stdout:9/478: write d0/d3/d74/f20 [79026,24165] 0 2024-05-12T04:01:24.419 INFO:tasks.workunit.client.1.smithi033.stdout:9/962: write d2/d4/d2b/d49/d73/fb4 [303109,125257] 0 2024-05-12T04:01:24.420 INFO:tasks.workunit.client.0.smithi032.stdout:8/561: rmdir d0/d9/d1b/d27/d4c/d4e/d7f 39 2024-05-12T04:01:24.425 INFO:tasks.workunit.client.1.smithi033.stdout:4/742: unlink dd/d7a/d87/d7e/da7/d66/c18 0 2024-05-12T04:01:24.427 INFO:tasks.workunit.client.0.smithi032.stdout:3/591: mknod d15/d20/d22/d7a/dbe/cd2 0 2024-05-12T04:01:24.427 INFO:tasks.workunit.client.0.smithi032.stdout:2/578: dwrite d1/d34/d96/d41/f2a [0,4194304] 0 2024-05-12T04:01:24.428 INFO:tasks.workunit.client.0.smithi032.stdout:2/579: chown d1/d13/f8f 24152 1 2024-05-12T04:01:24.430 INFO:tasks.workunit.client.1.smithi033.stdout:5/644: link d3/f7d d3/d12/d29/d2b/fd1 0 2024-05-12T04:01:24.432 INFO:tasks.workunit.client.1.smithi033.stdout:4/743: dwrite dd/d7a/d87/d7e/da7/d38/fbc [0,4194304] 0 2024-05-12T04:01:24.434 INFO:tasks.workunit.client.1.smithi033.stdout:9/963: rmdir d2/d4/d2b/d6d/db5 39 2024-05-12T04:01:24.435 INFO:tasks.workunit.client.0.smithi032.stdout:7/543: creat dd/d7b/fb1 x:0 0 0 2024-05-12T04:01:24.443 INFO:tasks.workunit.client.0.smithi032.stdout:1/558: symlink d1/d7/d49/lb4 0 2024-05-12T04:01:24.443 INFO:tasks.workunit.client.1.smithi033.stdout:6/755: dwrite d4/d5/fab [0,4194304] 0 2024-05-12T04:01:24.445 INFO:tasks.workunit.client.1.smithi033.stdout:5/645: mknod d3/d94/d54/d98/cd2 0 2024-05-12T04:01:24.446 INFO:tasks.workunit.client.1.smithi033.stdout:1/638: dwrite d11/d17/d27/d36/d3f/d69/d87/da4/fd9 [0,4194304] 0 2024-05-12T04:01:24.446 INFO:tasks.workunit.client.0.smithi032.stdout:9/479: mkdir d0/d3/d6/d7/d2a/d9f 0 2024-05-12T04:01:24.450 INFO:tasks.workunit.client.1.smithi033.stdout:9/964: symlink d2/d4/d2b/d6d/d4c/l156 0 2024-05-12T04:01:24.456 INFO:tasks.workunit.client.0.smithi032.stdout:3/592: symlink d15/d20/d8b/d9b/ld3 0 2024-05-12T04:01:24.456 INFO:tasks.workunit.client.0.smithi032.stdout:3/593: chown d15/f54 933005 1 2024-05-12T04:01:24.459 INFO:tasks.workunit.client.1.smithi033.stdout:0/772: write d6/d11/d57/f63 [1224849,113664] 0 2024-05-12T04:01:24.461 INFO:tasks.workunit.client.1.smithi033.stdout:6/756: mknod d4/d1a/d67/d6f/dac/cfa 0 2024-05-12T04:01:24.465 INFO:tasks.workunit.client.0.smithi032.stdout:7/544: truncate dd/d1b/f4b 1113501 0 2024-05-12T04:01:24.465 INFO:tasks.workunit.client.0.smithi032.stdout:2/580: dwrite d1/d13/f2f [0,4194304] 0 2024-05-12T04:01:24.466 INFO:tasks.workunit.client.0.smithi032.stdout:7/545: fsync dd/d1b/d43/d5e/d37/f54 0 2024-05-12T04:01:24.471 INFO:tasks.workunit.client.0.smithi032.stdout:1/559: unlink d1/d7/d30/d35/d54/d7e/fa7 0 2024-05-12T04:01:24.472 INFO:tasks.workunit.client.1.smithi033.stdout:7/929: dwrite de/df/d17/d1e/d24/fad [0,4194304] 0 2024-05-12T04:01:24.474 INFO:tasks.workunit.client.0.smithi032.stdout:9/480: creat d0/d3/d6/fa0 x:0 0 0 2024-05-12T04:01:24.475 INFO:tasks.workunit.client.1.smithi033.stdout:1/639: dread d11/d1c/d59/faa [0,4194304] 0 2024-05-12T04:01:24.477 INFO:tasks.workunit.client.0.smithi032.stdout:8/562: unlink d0/d9/d1b/d27/d88/d56/d63/l6b 0 2024-05-12T04:01:24.480 INFO:tasks.workunit.client.0.smithi032.stdout:0/589: getdents df/d22 0 2024-05-12T04:01:24.488 INFO:tasks.workunit.client.1.smithi033.stdout:3/696: sync 2024-05-12T04:01:24.497 INFO:tasks.workunit.client.0.smithi032.stdout:9/481: fdatasync d0/d3/d6/d7/d2a/d46/f49 0 2024-05-12T04:01:24.502 INFO:tasks.workunit.client.0.smithi032.stdout:8/563: unlink d0/d9/d1b/d27/d4c/d4e/d90/cbf 0 2024-05-12T04:01:24.506 INFO:tasks.workunit.client.0.smithi032.stdout:0/590: mknod df/d10/d3d/db0/cc8 0 2024-05-12T04:01:24.506 INFO:tasks.workunit.client.0.smithi032.stdout:9/482: dwrite d0/d3/d6/d58/f5a [4194304,4194304] 0 2024-05-12T04:01:24.508 INFO:tasks.workunit.client.0.smithi032.stdout:1/560: dread d1/d25/f64 [0,4194304] 0 2024-05-12T04:01:24.509 INFO:tasks.workunit.client.0.smithi032.stdout:2/581: dread d1/d34/d96/d41/d2c/f56 [0,4194304] 0 2024-05-12T04:01:24.510 INFO:tasks.workunit.client.0.smithi032.stdout:1/561: fdatasync d1/d7/d49/d79/f8d 0 2024-05-12T04:01:24.511 INFO:tasks.workunit.client.0.smithi032.stdout:2/582: chown d1/d13/c55 129264 1 2024-05-12T04:01:24.512 INFO:tasks.workunit.client.0.smithi032.stdout:3/594: dread d15/d24/f51 [0,4194304] 0 2024-05-12T04:01:24.516 INFO:tasks.workunit.client.0.smithi032.stdout:2/583: dread d1/d34/d95/fa1 [0,4194304] 0 2024-05-12T04:01:24.519 INFO:tasks.workunit.client.1.smithi033.stdout:2/712: dwrite dc/d17/d32/fb0 [0,4194304] 0 2024-05-12T04:01:24.523 INFO:tasks.workunit.client.1.smithi033.stdout:2/713: dread f7 [0,4194304] 0 2024-05-12T04:01:24.527 INFO:tasks.workunit.client.0.smithi032.stdout:9/483: rename d0/d3/f76 to d0/d3/d6/d7/d2a/d9f/fa1 0 2024-05-12T04:01:24.528 INFO:tasks.workunit.client.0.smithi032.stdout:9/484: readlink d0/d3/d6/d7/da/l26 0 2024-05-12T04:01:24.534 INFO:tasks.workunit.client.0.smithi032.stdout:9/485: dwrite d0/d3/d6/d7/da/f72 [0,4194304] 0 2024-05-12T04:01:24.540 INFO:tasks.workunit.client.0.smithi032.stdout:3/595: creat d15/d24/d2a/d8c/fd4 x:0 0 0 2024-05-12T04:01:24.552 INFO:tasks.workunit.client.0.smithi032.stdout:6/528: dwrite d7/de/d2e/f58 [0,4194304] 0 2024-05-12T04:01:24.553 INFO:tasks.workunit.client.0.smithi032.stdout:8/564: rename d0/d9/d1b/d27/d88/d56/d63/d8f/d9a/d28/d3e/f57 to d0/d9/d1b/d27/d88/d56/d63/d8f/db0/fc0 0 2024-05-12T04:01:24.554 INFO:tasks.workunit.client.0.smithi032.stdout:8/565: chown d0/d9/d1b/d27/d88/d56/d63/l6a 6494199 1 2024-05-12T04:01:24.557 INFO:tasks.workunit.client.0.smithi032.stdout:1/562: mkdir d1/d25/da5/db5 0 2024-05-12T04:01:24.559 INFO:tasks.workunit.client.0.smithi032.stdout:1/563: write d1/d7/d30/d18/d93/fa8 [611175,109519] 0 2024-05-12T04:01:24.566 INFO:tasks.workunit.client.0.smithi032.stdout:3/596: creat d15/d20/d26/d6a/d6e/d7d/fd5 x:0 0 0 2024-05-12T04:01:24.568 INFO:tasks.workunit.client.0.smithi032.stdout:3/597: fsync d15/d33/f4c 0 2024-05-12T04:01:24.569 INFO:tasks.workunit.client.0.smithi032.stdout:9/486: dwrite d0/d3/d74/f78 [0,4194304] 0 2024-05-12T04:01:24.569 INFO:tasks.workunit.client.1.smithi033.stdout:4/744: write dd/d7a/d87/d7e/da7/f45 [563418,1718] 0 2024-05-12T04:01:24.570 INFO:tasks.workunit.client.1.smithi033.stdout:4/745: dread - dd/d11/d75/dd4/fea zero size 2024-05-12T04:01:24.571 INFO:tasks.workunit.client.0.smithi032.stdout:7/546: getdents dd/d1b 0 2024-05-12T04:01:24.574 INFO:tasks.workunit.client.0.smithi032.stdout:9/487: truncate d0/d3/d6/d7/d27/f97 561685 0 2024-05-12T04:01:24.575 INFO:tasks.workunit.client.1.smithi033.stdout:5/646: dwrite d3/d94/d54/d57/d70/da3/dbe/d16/f20 [8388608,4194304] 0 2024-05-12T04:01:24.575 INFO:tasks.workunit.client.0.smithi032.stdout:9/488: dread - d0/d3/d4b/d85/d87/f98 zero size 2024-05-12T04:01:24.575 INFO:tasks.workunit.client.0.smithi032.stdout:2/584: creat d1/d34/d96/da4/fc0 x:0 0 0 2024-05-12T04:01:24.576 INFO:tasks.workunit.client.1.smithi033.stdout:5/647: readlink d3/d12/d29/l9a 0 2024-05-12T04:01:24.581 INFO:tasks.workunit.client.1.smithi033.stdout:0/773: mkdir d6/dc/d96/d9d/d103 0 2024-05-12T04:01:24.587 INFO:tasks.workunit.client.0.smithi032.stdout:8/566: truncate d0/d9/d13/f5d 838443 0 2024-05-12T04:01:24.594 INFO:tasks.workunit.client.0.smithi032.stdout:7/547: mkdir dd/d1b/d1e/d45/d55/d60/d75/d8f/db2 0 2024-05-12T04:01:24.596 INFO:tasks.workunit.client.1.smithi033.stdout:9/965: mknod d2/d4/d2b/d78/c157 0 2024-05-12T04:01:24.598 INFO:tasks.workunit.client.1.smithi033.stdout:1/640: readlink d11/d17/lc7 0 2024-05-12T04:01:24.600 INFO:tasks.workunit.client.0.smithi032.stdout:9/489: symlink d0/la2 0 2024-05-12T04:01:24.602 INFO:tasks.workunit.client.1.smithi033.stdout:3/697: mkdir db/d17/d3e/d65/d91/d97/def 0 2024-05-12T04:01:24.604 INFO:tasks.workunit.client.0.smithi032.stdout:8/567: creat d0/d9/d1b/d66/d75/fc1 x:0 0 0 2024-05-12T04:01:24.606 INFO:tasks.workunit.client.0.smithi032.stdout:2/585: dread d1/f31 [0,4194304] 0 2024-05-12T04:01:24.610 INFO:tasks.workunit.client.0.smithi032.stdout:0/591: write df/d30/d32/f34 [276426,87685] 0 2024-05-12T04:01:24.613 INFO:tasks.workunit.client.1.smithi033.stdout:2/714: unlink dc/d17/d27/d68/d9f/d5e/ldb 0 2024-05-12T04:01:24.617 INFO:tasks.workunit.client.1.smithi033.stdout:4/746: chown dd/d7a/d87/d7e/da7/d38/c4d 14600 1 2024-05-12T04:01:24.617 INFO:tasks.workunit.client.0.smithi032.stdout:8/568: creat d0/d9/d1b/d27/d88/d56/d63/d8f/d9a/d28/d3e/fc2 x:0 0 0 2024-05-12T04:01:24.618 INFO:tasks.workunit.client.1.smithi033.stdout:5/648: mknod d3/d94/d54/cd3 0 2024-05-12T04:01:24.618 INFO:tasks.workunit.client.1.smithi033.stdout:2/715: dwrite dc/d17/d3a/f8b [0,4194304] 0 2024-05-12T04:01:24.620 INFO:tasks.workunit.client.0.smithi032.stdout:3/598: creat d15/d20/fd6 x:0 0 0 2024-05-12T04:01:24.621 INFO:tasks.workunit.client.0.smithi032.stdout:3/599: fsync d15/d20/d22/d7a/fb2 0 2024-05-12T04:01:24.622 INFO:tasks.workunit.client.1.smithi033.stdout:0/774: fsync d6/dc/d58/fe8 0 2024-05-12T04:01:24.624 INFO:tasks.workunit.client.0.smithi032.stdout:4/515: sync 2024-05-12T04:01:24.627 INFO:tasks.workunit.client.0.smithi032.stdout:3/600: dwrite d15/f4e [0,4194304] 0 2024-05-12T04:01:24.633 INFO:tasks.workunit.client.0.smithi032.stdout:8/569: rename d0/d9/d1b/d66/d75/l7d to d0/d9/d1b/d27/d4c/d4e/d90/lc3 0 2024-05-12T04:01:24.638 INFO:tasks.workunit.client.1.smithi033.stdout:8/683: mknod d1/d16/d27/d4d/d73/cd5 0 2024-05-12T04:01:24.642 INFO:tasks.workunit.client.0.smithi032.stdout:7/548: link dd/d1b/l40 dd/d1b/d1e/d45/d55/d60/d75/d8f/lb3 0 2024-05-12T04:01:24.642 INFO:tasks.workunit.client.0.smithi032.stdout:7/549: stat dd/d1b/lab 0 2024-05-12T04:01:24.644 INFO:tasks.workunit.client.0.smithi032.stdout:4/516: creat d0/d3/d31/d80/f8c x:0 0 0 2024-05-12T04:01:24.645 INFO:tasks.workunit.client.1.smithi033.stdout:2/716: dread - dc/d13/d1c/d98/fb9 zero size 2024-05-12T04:01:24.646 INFO:tasks.workunit.client.1.smithi033.stdout:9/966: dread d2/d4/d29/d32/db7/fbf [0,4194304] 0 2024-05-12T04:01:24.649 INFO:tasks.workunit.client.0.smithi032.stdout:7/550: dwrite dd/d1b/d1e/d45/d55/d60/d75/f76 [0,4194304] 0 2024-05-12T04:01:24.650 INFO:tasks.workunit.client.0.smithi032.stdout:7/551: chown dd/d1b/d43/d74/f84 6 1 2024-05-12T04:01:24.650 INFO:tasks.workunit.client.0.smithi032.stdout:3/601: mknod d15/d24/d40/cd7 0 2024-05-12T04:01:24.652 INFO:tasks.workunit.client.0.smithi032.stdout:7/552: write dd/d1b/d1e/d45/d64/f91 [133335,52255] 0 2024-05-12T04:01:24.653 INFO:tasks.workunit.client.0.smithi032.stdout:7/553: chown dd/d1b/d1e/c53 928 1 2024-05-12T04:01:24.654 INFO:tasks.workunit.client.1.smithi033.stdout:8/684: dread d1/d4/d1d/f78 [0,4194304] 0 2024-05-12T04:01:24.655 INFO:tasks.workunit.client.1.smithi033.stdout:8/685: fdatasync d1/d16/d29/d63/fbf 0 2024-05-12T04:01:24.656 INFO:tasks.workunit.client.1.smithi033.stdout:8/686: chown d1/d16/d27/d7c/fcf 809820182 1 2024-05-12T04:01:24.657 INFO:tasks.workunit.client.1.smithi033.stdout:8/687: read - d1/fa4 zero size 2024-05-12T04:01:24.658 INFO:tasks.workunit.client.0.smithi032.stdout:1/564: write d1/d7/d30/d18/f70 [2122576,119473] 0 2024-05-12T04:01:24.660 INFO:tasks.workunit.client.0.smithi032.stdout:0/592: creat df/d10/d48/d70/dbb/fc9 x:0 0 0 2024-05-12T04:01:24.661 INFO:tasks.workunit.client.0.smithi032.stdout:1/565: chown d1/d7/ce 58725 1 2024-05-12T04:01:24.663 INFO:tasks.workunit.client.0.smithi032.stdout:6/529: dwrite d7/d9/d48/d67/d63/d64/f65 [0,4194304] 0 2024-05-12T04:01:24.665 INFO:tasks.workunit.client.0.smithi032.stdout:0/593: dread - df/d10/d3d/d4a/d9a/d9d/fb7 zero size 2024-05-12T04:01:24.666 INFO:tasks.workunit.client.1.smithi033.stdout:5/649: symlink d3/d12/ld4 0 2024-05-12T04:01:24.669 INFO:tasks.workunit.client.1.smithi033.stdout:9/967: read d2/f8e [63746,25796] 0 2024-05-12T04:01:24.670 INFO:tasks.workunit.client.1.smithi033.stdout:9/968: readlink d2/d4/d2b/d6d/d3a/dca/lcc 0 2024-05-12T04:01:24.671 INFO:tasks.workunit.client.1.smithi033.stdout:2/717: creat dc/d13/d39/fe6 x:0 0 0 2024-05-12T04:01:24.675 INFO:tasks.workunit.client.0.smithi032.stdout:7/554: unlink dd/d1b/d1e/d45/d55/d60/c96 0 2024-05-12T04:01:24.676 INFO:tasks.workunit.client.1.smithi033.stdout:2/718: dread dc/d17/d3a/f8b [0,4194304] 0 2024-05-12T04:01:24.678 INFO:tasks.workunit.client.0.smithi032.stdout:9/490: write d0/f6b [1059226,52211] 0 2024-05-12T04:01:24.681 INFO:tasks.workunit.client.0.smithi032.stdout:7/555: dwrite dd/f36 [0,4194304] 0 2024-05-12T04:01:24.683 INFO:tasks.workunit.client.0.smithi032.stdout:5/662: sync 2024-05-12T04:01:24.685 INFO:tasks.workunit.client.0.smithi032.stdout:7/556: write dd/d1b/d1e/d45/d55/d60/d75/d8f/f98 [470477,63702] 0 2024-05-12T04:01:24.686 INFO:tasks.workunit.client.1.smithi033.stdout:8/688: write d1/d16/d27/d4d/d82/fc2 [1040832,73331] 0 2024-05-12T04:01:24.690 INFO:tasks.workunit.client.0.smithi032.stdout:0/594: symlink df/d10/d3d/d4a/d84/lca 0 2024-05-12T04:01:24.690 INFO:tasks.workunit.client.1.smithi033.stdout:5/650: symlink d3/d94/d54/d57/d70/da3/dbe/d16/d45/d62/ld5 0 2024-05-12T04:01:24.691 INFO:tasks.workunit.client.1.smithi033.stdout:6/757: dwrite d4/f2f [0,4194304] 0 2024-05-12T04:01:24.691 INFO:tasks.workunit.client.0.smithi032.stdout:0/595: fdatasync fb 0 2024-05-12T04:01:24.691 INFO:tasks.workunit.client.0.smithi032.stdout:2/586: truncate d1/d13/d70/f91 2034681 0 2024-05-12T04:01:24.692 INFO:tasks.workunit.client.0.smithi032.stdout:2/587: chown d1/d34/d96/d41/d52/c6e 2280642 1 2024-05-12T04:01:24.694 INFO:tasks.workunit.client.0.smithi032.stdout:8/570: rename d0/d9/d1b/d27/d4c/c74 to d0/d9/cc4 0 2024-05-12T04:01:24.696 INFO:tasks.workunit.client.1.smithi033.stdout:9/969: write d2/d4/d2b/d6d/ff3 [8752987,121790] 0 2024-05-12T04:01:24.697 INFO:tasks.workunit.client.1.smithi033.stdout:3/698: dwrite db/d17/d3e/d65/d7d/f94 [4194304,4194304] 0 2024-05-12T04:01:24.697 INFO:tasks.workunit.client.0.smithi032.stdout:8/571: fdatasync d0/d9/d1b/d27/d88/d56/d63/d8f/d9a/d28/d3e/fc2 0 2024-05-12T04:01:24.700 INFO:tasks.workunit.client.1.smithi033.stdout:3/699: chown db/d59/f5b 1807 1 2024-05-12T04:01:24.702 INFO:tasks.workunit.client.0.smithi032.stdout:8/572: read d0/d9/d13/d21/d25/f52 [393885,14294] 0 2024-05-12T04:01:24.702 INFO:tasks.workunit.client.1.smithi033.stdout:6/758: dread d4/d1a/d67/d68/f72 [0,4194304] 0 2024-05-12T04:01:24.706 INFO:tasks.workunit.client.0.smithi032.stdout:9/491: mkdir d0/d84/d8a/da3 0 2024-05-12T04:01:24.706 INFO:tasks.workunit.client.0.smithi032.stdout:4/517: write d0/d3/d12/d1d/f2a [1203654,45119] 0 2024-05-12T04:01:24.707 INFO:tasks.workunit.client.1.smithi033.stdout:2/719: symlink dc/d17/d32/d84/dde/d9e/le7 0 2024-05-12T04:01:24.707 INFO:tasks.workunit.client.1.smithi033.stdout:5/651: dwrite d3/d94/d54/d57/d70/da3/dbe/d16/d45/d62/f64 [0,4194304] 0 2024-05-12T04:01:24.707 INFO:tasks.workunit.client.1.smithi033.stdout:7/930: link de/d20/d64/cf5 de/d20/c13a 0 2024-05-12T04:01:24.710 INFO:tasks.workunit.client.1.smithi033.stdout:1/641: link d11/d1c/d59/f7f d11/d17/d27/d36/d3f/d69/d87/fdb 0 2024-05-12T04:01:24.719 INFO:tasks.workunit.client.0.smithi032.stdout:2/588: creat d1/d13/da0/dae/fc1 x:0 0 0 2024-05-12T04:01:24.723 INFO:tasks.workunit.client.0.smithi032.stdout:5/663: rename d4/d7/d14/d26/d33/d4a/d38/d3b to d4/d7/d14/d26/d33/d72/dc9/dec 0 2024-05-12T04:01:24.726 INFO:tasks.workunit.client.1.smithi033.stdout:0/775: getdents d6/dc/d58/d6a/d80 0 2024-05-12T04:01:24.727 INFO:tasks.workunit.client.1.smithi033.stdout:0/776: write d6/db/d2c/f3e [1863544,6834] 0 2024-05-12T04:01:24.728 INFO:tasks.workunit.client.0.smithi032.stdout:3/602: creat d15/d1b/fd8 x:0 0 0 2024-05-12T04:01:24.730 INFO:tasks.workunit.client.0.smithi032.stdout:3/603: dread d15/d24/d40/f5a [0,4194304] 0 2024-05-12T04:01:24.731 INFO:tasks.workunit.client.1.smithi033.stdout:6/759: creat d4/d43/d57/dc4/ffb x:0 0 0 2024-05-12T04:01:24.732 INFO:tasks.workunit.client.0.smithi032.stdout:8/573: mknod d0/d9/d1b/d27/d59/cc5 0 2024-05-12T04:01:24.735 INFO:tasks.workunit.client.1.smithi033.stdout:2/720: rmdir dc/d17/d27/d68/d9f/d5e 39 2024-05-12T04:01:24.738 INFO:tasks.workunit.client.1.smithi033.stdout:2/721: write dc/d17/d27/d2f/f38 [4997415,5235] 0 2024-05-12T04:01:24.738 INFO:tasks.workunit.client.1.smithi033.stdout:8/689: dwrite d1/d4/d91/fbd [0,4194304] 0 2024-05-12T04:01:24.741 INFO:tasks.workunit.client.1.smithi033.stdout:7/931: rename de/d20/d123 to de/df/d65/dbc/d13b 0 2024-05-12T04:01:24.744 INFO:tasks.workunit.client.1.smithi033.stdout:1/642: chown d11/d17/d27/d36/f3d 11653487 1 2024-05-12T04:01:24.749 INFO:tasks.workunit.client.1.smithi033.stdout:5/652: write d3/d94/d54/d57/d70/da3/dbe/d16/d45/d4b/f6c [3197017,92359] 0 2024-05-12T04:01:24.750 INFO:tasks.workunit.client.0.smithi032.stdout:5/664: dread - d4/d7/fa5 zero size 2024-05-12T04:01:24.752 INFO:tasks.workunit.client.1.smithi033.stdout:0/777: chown d6/dc/d39/f40 57298103 1 2024-05-12T04:01:24.754 INFO:tasks.workunit.client.1.smithi033.stdout:3/700: mkdir db/d17/df0 0 2024-05-12T04:01:24.756 INFO:tasks.workunit.client.1.smithi033.stdout:9/970: creat d2/d4/d2b/d6d/d2f/d5f/d84/d137/f158 x:0 0 0 2024-05-12T04:01:24.757 INFO:tasks.workunit.client.0.smithi032.stdout:0/596: dwrite df/d10/d48/f96 [0,4194304] 0 2024-05-12T04:01:24.760 INFO:tasks.workunit.client.0.smithi032.stdout:9/492: creat d0/d84/d8a/da3/fa4 x:0 0 0 2024-05-12T04:01:24.761 INFO:tasks.workunit.client.0.smithi032.stdout:6/530: getdents d7/d9/d5c 0 2024-05-12T04:01:24.762 INFO:tasks.workunit.client.1.smithi033.stdout:7/932: fsync de/df/d7a/f107 0 2024-05-12T04:01:24.764 INFO:tasks.workunit.client.0.smithi032.stdout:4/518: mkdir d0/d3/d12/d8d 0 2024-05-12T04:01:24.765 INFO:tasks.workunit.client.1.smithi033.stdout:8/690: truncate d1/d4/d1d/d36/d3c/d64/fa0 130419 0 2024-05-12T04:01:24.766 INFO:tasks.workunit.client.0.smithi032.stdout:4/519: chown d0/d3/d12/d1d/d63/d77/c79 7545167 1 2024-05-12T04:01:24.768 INFO:tasks.workunit.client.1.smithi033.stdout:6/760: rename d4/d1a/d67/d6f/cc2 to d4/d8d/cfc 0 2024-05-12T04:01:24.770 INFO:tasks.workunit.client.1.smithi033.stdout:1/643: symlink d11/d1c/d43/ldc 0 2024-05-12T04:01:24.770 INFO:tasks.workunit.client.1.smithi033.stdout:1/644: chown d11/d17/d27/d36/cb3 514742800 1 2024-05-12T04:01:24.771 INFO:tasks.workunit.client.0.smithi032.stdout:1/566: getdents d1/d6a/d72/da9 0 2024-05-12T04:01:24.772 INFO:tasks.workunit.client.0.smithi032.stdout:1/567: fdatasync d1/d6a/d72/f88 0 2024-05-12T04:01:24.772 INFO:tasks.workunit.client.0.smithi032.stdout:5/665: dread d4/d7/d14/d26/d47/f9b [0,4194304] 0 2024-05-12T04:01:24.773 INFO:tasks.workunit.client.1.smithi033.stdout:5/653: mknod d3/d94/d54/d57/d70/da3/dbe/d16/d45/d62/d7f/d84/cd6 0 2024-05-12T04:01:24.774 INFO:tasks.workunit.client.0.smithi032.stdout:1/568: write d1/d7/d30/d18/f70 [3578999,14359] 0 2024-05-12T04:01:24.774 INFO:tasks.workunit.client.1.smithi033.stdout:3/701: truncate db/d17/d4d/fcd 433995 0 2024-05-12T04:01:24.776 INFO:tasks.workunit.client.0.smithi032.stdout:2/589: rename d1/c23 to d1/d34/da5/db3/db7/cc2 0 2024-05-12T04:01:24.777 INFO:tasks.workunit.client.1.smithi033.stdout:2/722: write dc/fb8 [12543,70547] 0 2024-05-12T04:01:24.784 INFO:tasks.workunit.client.1.smithi033.stdout:4/747: sync 2024-05-12T04:01:24.786 INFO:tasks.workunit.client.1.smithi033.stdout:8/691: unlink d1/d16/d29/cb4 0 2024-05-12T04:01:24.787 INFO:tasks.workunit.client.1.smithi033.stdout:8/692: dread - d1/d16/d27/d7c/fcf zero size 2024-05-12T04:01:24.788 INFO:tasks.workunit.client.1.smithi033.stdout:8/693: truncate d1/d4/db2/fbb 912318 0 2024-05-12T04:01:24.790 INFO:tasks.workunit.client.1.smithi033.stdout:9/971: dread d2/d70/d7b/f8a [0,4194304] 0 2024-05-12T04:01:24.791 INFO:tasks.workunit.client.0.smithi032.stdout:9/493: read d0/d3/d6/d7/d2a/d46/f12 [3330175,63689] 0 2024-05-12T04:01:24.792 INFO:tasks.workunit.client.1.smithi033.stdout:1/645: read d11/d17/d27/f48 [3187819,70717] 0 2024-05-12T04:01:24.792 INFO:tasks.workunit.client.0.smithi032.stdout:6/531: mknod d7/d9/d48/d67/d1a/d20/cb6 0 2024-05-12T04:01:24.793 INFO:tasks.workunit.client.0.smithi032.stdout:6/532: stat d7/d9/d48/d67/d1a/l2d 0 2024-05-12T04:01:24.795 INFO:tasks.workunit.client.1.smithi033.stdout:5/654: rename d3/d94/d54/d57/d70/da3/dc9/db1 to d3/d94/d54/d57/d70/da3/dbe/d16/dd7 0 2024-05-12T04:01:24.796 INFO:tasks.workunit.client.0.smithi032.stdout:7/557: getdents dd/d6c 0 2024-05-12T04:01:24.796 INFO:tasks.workunit.client.1.smithi033.stdout:5/655: fsync d3/d12/f14 0 2024-05-12T04:01:24.797 INFO:tasks.workunit.client.1.smithi033.stdout:7/933: dwrite de/df/d17/d1e/d24/fc5 [0,4194304] 0 2024-05-12T04:01:24.798 INFO:tasks.workunit.client.1.smithi033.stdout:3/702: mknod db/d17/d3a/d67/da6/cf1 0 2024-05-12T04:01:24.799 INFO:tasks.workunit.client.0.smithi032.stdout:4/520: creat d0/d3/d31/d80/f8e x:0 0 0 2024-05-12T04:01:24.799 INFO:tasks.workunit.client.1.smithi033.stdout:2/723: fdatasync dc/d17/d27/d2f/d49/f85 0 2024-05-12T04:01:24.804 INFO:tasks.workunit.client.1.smithi033.stdout:4/748: rmdir dd/d7a/d87/d7e 39 2024-05-12T04:01:24.805 INFO:tasks.workunit.client.0.smithi032.stdout:5/666: creat d4/d7/d14/d26/d33/d5f/fed x:0 0 0 2024-05-12T04:01:24.806 INFO:tasks.workunit.client.0.smithi032.stdout:1/569: readlink d1/d7/d30/d35/d54/d7e/la0 0 2024-05-12T04:01:24.806 INFO:tasks.workunit.client.0.smithi032.stdout:0/597: dwrite df/d10/f3c [4194304,4194304] 0 2024-05-12T04:01:24.809 INFO:tasks.workunit.client.1.smithi033.stdout:6/761: mknod d4/d6e/dbb/cfd 0 2024-05-12T04:01:24.815 INFO:tasks.workunit.client.1.smithi033.stdout:9/972: creat d2/d4/d2b/d78/d76/f159 x:0 0 0 2024-05-12T04:01:24.816 INFO:tasks.workunit.client.1.smithi033.stdout:6/762: dread d4/d6e/f90 [0,4194304] 0 2024-05-12T04:01:24.818 INFO:tasks.workunit.client.1.smithi033.stdout:6/763: read d4/d1a/ff0 [181566,98839] 0 2024-05-12T04:01:24.819 INFO:tasks.workunit.client.1.smithi033.stdout:8/694: dread d1/d16/d27/d4d/f5e [0,4194304] 0 2024-05-12T04:01:24.819 INFO:tasks.workunit.client.0.smithi032.stdout:9/494: rename d0/d3/d6/d7/l22 to d0/d3/d6/d7/la5 0 2024-05-12T04:01:24.820 INFO:tasks.workunit.client.0.smithi032.stdout:9/495: chown d0/d3/l2d 15538913 1 2024-05-12T04:01:24.821 INFO:tasks.workunit.client.0.smithi032.stdout:6/533: creat d7/d9/d5c/d86/fb7 x:0 0 0 2024-05-12T04:01:24.822 INFO:tasks.workunit.client.0.smithi032.stdout:9/496: read d0/d3/d6/d58/f66 [2814047,61818] 0 2024-05-12T04:01:24.825 INFO:tasks.workunit.client.1.smithi033.stdout:7/934: chown de/d2f/d70/c119 45038 1 2024-05-12T04:01:24.826 INFO:tasks.workunit.client.0.smithi032.stdout:4/521: creat d0/d3/d12/d1d/d63/d77/f8f x:0 0 0 2024-05-12T04:01:24.827 INFO:tasks.workunit.client.1.smithi033.stdout:3/703: creat db/d17/d25/d32/d3c/d3d/d47/ff2 x:0 0 0 2024-05-12T04:01:24.829 INFO:tasks.workunit.client.0.smithi032.stdout:5/667: read d4/d7/d14/f8c [4214890,90275] 0 2024-05-12T04:01:24.832 INFO:tasks.workunit.client.1.smithi033.stdout:2/724: rename dc/d17/d74 to dc/d13/d1f/d6e/de8 0 2024-05-12T04:01:24.834 INFO:tasks.workunit.client.1.smithi033.stdout:5/656: write d3/d94/d54/d57/d70/fa9 [1466138,53157] 0 2024-05-12T04:01:24.834 INFO:tasks.workunit.client.1.smithi033.stdout:5/657: chown d3/d94/db8 2978 1 2024-05-12T04:01:24.839 INFO:tasks.workunit.client.1.smithi033.stdout:1/646: mkdir d11/d17/d27/d36/d46/ddd 0 2024-05-12T04:01:24.841 INFO:tasks.workunit.client.0.smithi032.stdout:3/604: getdents d15/d20/d8b/d9b 0 2024-05-12T04:01:24.842 INFO:tasks.workunit.client.0.smithi032.stdout:2/590: truncate d1/d34/d96/d41/d2c/f56 173855 0 2024-05-12T04:01:24.842 INFO:tasks.workunit.client.1.smithi033.stdout:9/973: rmdir d2/d4/d29/d3c/dda 39 2024-05-12T04:01:24.843 INFO:tasks.workunit.client.0.smithi032.stdout:2/591: chown d1/d34/d96/d41/d2c/f59 1724 1 2024-05-12T04:01:24.843 INFO:tasks.workunit.client.0.smithi032.stdout:8/574: link d0/d9/d1b/d27/d88/d56/d63/d8f/d9a/f55 d0/d97/fc6 0 2024-05-12T04:01:24.845 INFO:tasks.workunit.client.0.smithi032.stdout:6/534: rmdir d7/d9/d48/d67 39 2024-05-12T04:01:24.846 INFO:tasks.workunit.client.1.smithi033.stdout:6/764: fdatasync d4/fcf 0 2024-05-12T04:01:24.849 INFO:tasks.workunit.client.1.smithi033.stdout:8/695: mkdir d1/d16/d29/d63/da3/dd6 0 2024-05-12T04:01:24.850 INFO:tasks.workunit.client.0.smithi032.stdout:1/570: dwrite d1/f13 [0,4194304] 0 2024-05-12T04:01:24.850 INFO:tasks.workunit.client.0.smithi032.stdout:7/558: mkdir dd/d7b/d9a/d9c/db4 0 2024-05-12T04:01:24.852 INFO:tasks.workunit.client.0.smithi032.stdout:7/559: chown dd/d1b/d1e/f28 16028544 1 2024-05-12T04:01:24.855 INFO:tasks.workunit.client.0.smithi032.stdout:4/522: symlink d0/d3/d24/l90 0 2024-05-12T04:01:24.858 INFO:tasks.workunit.client.0.smithi032.stdout:9/497: write d0/d3/d6/d7/d2a/d46/f12 [5175251,57166] 0 2024-05-12T04:01:24.860 INFO:tasks.workunit.client.0.smithi032.stdout:0/598: creat df/d30/d8d/d8f/fcb x:0 0 0 2024-05-12T04:01:24.861 INFO:tasks.workunit.client.0.smithi032.stdout:0/599: chown df/d10/l15 1 1 2024-05-12T04:01:24.864 INFO:tasks.workunit.client.0.smithi032.stdout:3/605: creat d15/d20/d26/d6a/d6e/d7d/fd9 x:0 0 0 2024-05-12T04:01:24.866 INFO:tasks.workunit.client.1.smithi033.stdout:2/725: creat dc/d17/d27/d2f/fe9 x:0 0 0 2024-05-12T04:01:24.867 INFO:tasks.workunit.client.0.smithi032.stdout:2/592: creat d1/d34/d96/d41/d52/fc3 x:0 0 0 2024-05-12T04:01:24.868 INFO:tasks.workunit.client.0.smithi032.stdout:8/575: symlink d0/d7/lc7 0 2024-05-12T04:01:24.868 INFO:tasks.workunit.client.0.smithi032.stdout:8/576: readlink d0/l32 0 2024-05-12T04:01:24.872 INFO:tasks.workunit.client.1.smithi033.stdout:1/647: mkdir d11/d17/d27/d36/d46/d4c/dde 0 2024-05-12T04:01:24.873 INFO:tasks.workunit.client.1.smithi033.stdout:1/648: fsync d11/d17/d27/d36/d46/d4c/fc8 0 2024-05-12T04:01:24.873 INFO:tasks.workunit.client.0.smithi032.stdout:1/571: symlink d1/d7/lb6 0 2024-05-12T04:01:24.874 INFO:tasks.workunit.client.1.smithi033.stdout:1/649: read - d11/d17/d27/d36/fb6 zero size 2024-05-12T04:01:24.874 INFO:tasks.workunit.client.1.smithi033.stdout:1/650: dread - d11/d17/d27/fbc zero size 2024-05-12T04:01:24.877 INFO:tasks.workunit.client.0.smithi032.stdout:1/572: dread d1/f13 [0,4194304] 0 2024-05-12T04:01:24.878 INFO:tasks.workunit.client.1.smithi033.stdout:6/765: creat d4/d1a/d67/de5/ffe x:0 0 0 2024-05-12T04:01:24.878 INFO:tasks.workunit.client.0.smithi032.stdout:1/573: dread - d1/d7/d30/d18/f8b zero size 2024-05-12T04:01:24.879 INFO:tasks.workunit.client.0.smithi032.stdout:1/574: readlink d1/d7/lb6 0 2024-05-12T04:01:24.881 INFO:tasks.workunit.client.0.smithi032.stdout:0/600: creat df/d30/d8d/d8f/fcc x:0 0 0 2024-05-12T04:01:24.883 INFO:tasks.workunit.client.1.smithi033.stdout:7/935: write de/d2b/d6c/db5/dd0/ff2 [1969143,94813] 0 2024-05-12T04:01:24.885 INFO:tasks.workunit.client.1.smithi033.stdout:2/726: truncate dc/d13/d1c/f21 1045620 0 2024-05-12T04:01:24.885 INFO:tasks.workunit.client.1.smithi033.stdout:8/696: dwrite d1/d4/d1d/d36/faf [0,4194304] 0 2024-05-12T04:01:24.886 INFO:tasks.workunit.client.0.smithi032.stdout:9/498: dwrite d0/d3/d6/d7/d27/f97 [0,4194304] 0 2024-05-12T04:01:24.888 INFO:tasks.workunit.client.0.smithi032.stdout:6/535: dwrite d7/de/f6d [0,4194304] 0 2024-05-12T04:01:24.890 INFO:tasks.workunit.client.1.smithi033.stdout:1/651: dread - d11/fb7 zero size 2024-05-12T04:01:24.896 INFO:tasks.workunit.client.1.smithi033.stdout:3/704: write db/d17/d3a/d67/d76/f78 [625492,101002] 0 2024-05-12T04:01:24.897 INFO:tasks.workunit.client.1.smithi033.stdout:6/766: symlink d4/d1a/d67/d6f/lff 0 2024-05-12T04:01:24.898 INFO:tasks.workunit.client.1.smithi033.stdout:6/767: chown d4/d1a/l61 10031 1 2024-05-12T04:01:24.899 INFO:tasks.workunit.client.0.smithi032.stdout:1/575: write d1/d25/f64 [1930851,70193] 0 2024-05-12T04:01:24.901 INFO:tasks.workunit.client.0.smithi032.stdout:1/576: read - d1/d7/d49/d79/f8d zero size 2024-05-12T04:01:24.904 INFO:tasks.workunit.client.1.smithi033.stdout:3/705: dread db/d17/d25/d32/d3c/d3d/f72 [0,4194304] 0 2024-05-12T04:01:24.905 INFO:tasks.workunit.client.1.smithi033.stdout:3/706: write db/fd9 [3232325,53884] 0 2024-05-12T04:01:24.916 INFO:tasks.workunit.client.1.smithi033.stdout:0/778: sync 2024-05-12T04:01:25.003 INFO:tasks.workunit.client.0.smithi032.stdout:0/601: dread df/d10/f80 [0,4194304] 0 2024-05-12T04:01:25.003 INFO:tasks.workunit.client.0.smithi032.stdout:0/602: write df/d10/d48/f96 [3323265,15269] 0 2024-05-12T04:01:25.007 INFO:tasks.workunit.client.0.smithi032.stdout:2/593: rename d1/d34/d96/c4f to d1/d34/d96/d41/cc4 0 2024-05-12T04:01:25.009 INFO:tasks.workunit.client.0.smithi032.stdout:2/594: read d1/f3 [1527153,77432] 0 2024-05-12T04:01:25.010 INFO:tasks.workunit.client.1.smithi033.stdout:5/658: rename d3/d12/f63 to d3/d94/d54/d57/d70/da3/dbe/d16/d78/d8b/fd8 0 2024-05-12T04:01:25.011 INFO:tasks.workunit.client.0.smithi032.stdout:8/577: symlink d0/d9/d1b/d27/lc8 0 2024-05-12T04:01:25.012 INFO:tasks.workunit.client.1.smithi033.stdout:8/697: write d1/d4/d1d/d36/f50 [1950938,20712] 0 2024-05-12T04:01:25.012 INFO:tasks.workunit.client.1.smithi033.stdout:8/698: stat d1 0 2024-05-12T04:01:25.017 INFO:tasks.workunit.client.1.smithi033.stdout:6/768: truncate d4/d1a/d4e/d4f/d94/faa 1123627 0 2024-05-12T04:01:25.019 INFO:tasks.workunit.client.1.smithi033.stdout:3/707: dread db/d17/d4d/fcd [0,4194304] 0 2024-05-12T04:01:25.020 INFO:tasks.workunit.client.0.smithi032.stdout:9/499: write d0/d3/d6/d7/d2a/f68 [1583063,125442] 0 2024-05-12T04:01:25.020 INFO:tasks.workunit.client.1.smithi033.stdout:0/779: symlink d6/dc/d58/d6a/dec/l104 0 2024-05-12T04:01:25.021 INFO:tasks.workunit.client.0.smithi032.stdout:7/560: dwrite dd/f1d [0,4194304] 0 2024-05-12T04:01:25.021 INFO:tasks.workunit.client.1.smithi033.stdout:4/749: sync 2024-05-12T04:01:25.024 INFO:tasks.workunit.client.1.smithi033.stdout:7/936: mknod de/d2b/d6c/dbd/c13c 0 2024-05-12T04:01:25.025 INFO:tasks.workunit.client.1.smithi033.stdout:7/937: write f3 [56358,34348] 0 2024-05-12T04:01:25.028 INFO:tasks.workunit.client.0.smithi032.stdout:3/606: dwrite d15/d24/f3e [0,4194304] 0 2024-05-12T04:01:25.029 INFO:tasks.workunit.client.1.smithi033.stdout:2/727: truncate dc/d13/d1c/f4f 3573793 0 2024-05-12T04:01:25.033 INFO:tasks.workunit.client.0.smithi032.stdout:0/603: truncate df/d10/d3d/d4a/d84/f9c 618050 0 2024-05-12T04:01:25.041 INFO:tasks.workunit.client.1.smithi033.stdout:5/659: mkdir d3/d12/dac/dd9 0 2024-05-12T04:01:25.041 INFO:tasks.workunit.client.0.smithi032.stdout:2/595: read d1/d34/d96/d41/d2c/f56 [167699,69800] 0 2024-05-12T04:01:25.042 INFO:tasks.workunit.client.1.smithi033.stdout:8/699: creat d1/d16/d27/d4d/d73/fd7 x:0 0 0 2024-05-12T04:01:25.047 INFO:tasks.workunit.client.0.smithi032.stdout:1/577: write d1/d7/d30/d35/f48 [843328,124499] 0 2024-05-12T04:01:25.049 INFO:tasks.workunit.client.0.smithi032.stdout:4/523: getdents d0/d3/df 0 2024-05-12T04:01:25.051 INFO:tasks.workunit.client.1.smithi033.stdout:0/780: fsync d6/dc/d58/f81 0 2024-05-12T04:01:25.051 INFO:tasks.workunit.client.1.smithi033.stdout:7/938: dread de/d20/f83 [0,4194304] 0 2024-05-12T04:01:25.051 INFO:tasks.workunit.client.0.smithi032.stdout:7/561: creat dd/d7b/fb5 x:0 0 0 2024-05-12T04:01:25.052 INFO:tasks.workunit.client.1.smithi033.stdout:4/750: chown dd/d7a/d87/d7e/da7/d66/f7f 273 1 2024-05-12T04:01:25.053 INFO:tasks.workunit.client.0.smithi032.stdout:9/500: mkdir d0/d84/da6 0 2024-05-12T04:01:25.054 INFO:tasks.workunit.client.0.smithi032.stdout:9/501: chown d0/d3/d6/d7/d2a/d46/d91 959125 1 2024-05-12T04:01:25.061 INFO:tasks.workunit.client.0.smithi032.stdout:8/578: rename d0/d9/d1b/d27/d88/l53 to d0/d9/d1b/d33/d73/db7/lc9 0 2024-05-12T04:01:25.062 INFO:tasks.workunit.client.0.smithi032.stdout:3/607: rmdir d15/d24 39 2024-05-12T04:01:25.063 INFO:tasks.workunit.client.0.smithi032.stdout:3/608: chown d15/d20/d8b/d9b/dbb/cc5 413118 1 2024-05-12T04:01:25.064 INFO:tasks.workunit.client.1.smithi033.stdout:7/939: dread de/d2f/d70/f94 [0,4194304] 0 2024-05-12T04:01:25.064 INFO:tasks.workunit.client.1.smithi033.stdout:8/700: mkdir d1/d16/d29/db1/dd8 0 2024-05-12T04:01:25.065 INFO:tasks.workunit.client.1.smithi033.stdout:7/940: write de/d20/d38/f132 [308626,52010] 0 2024-05-12T04:01:25.067 INFO:tasks.workunit.client.0.smithi032.stdout:3/609: dread d15/d20/d22/d7a/fb2 [0,4194304] 0 2024-05-12T04:01:25.067 INFO:tasks.workunit.client.1.smithi033.stdout:5/660: rename d3/d94/d54/d57/ca4 to d3/d94/d54/d57/d70/cda 0 2024-05-12T04:01:25.067 INFO:tasks.workunit.client.1.smithi033.stdout:9/974: sync 2024-05-12T04:01:25.068 INFO:tasks.workunit.client.1.smithi033.stdout:1/652: sync 2024-05-12T04:01:25.069 INFO:tasks.workunit.client.0.smithi032.stdout:1/578: mkdir d1/d7/d30/d18/d87/db7 0 2024-05-12T04:01:25.070 INFO:tasks.workunit.client.0.smithi032.stdout:6/536: truncate d7/d9/d48/faa 3999129 0 2024-05-12T04:01:25.071 INFO:tasks.workunit.client.0.smithi032.stdout:1/579: truncate d1/d7/d30/d18/fab 587544 0 2024-05-12T04:01:25.072 INFO:tasks.workunit.client.1.smithi033.stdout:3/708: creat db/d59/dcc/de9/ff3 x:0 0 0 2024-05-12T04:01:25.077 INFO:tasks.workunit.client.1.smithi033.stdout:6/769: dwrite d4/d1a/d4e/d4f/fd4 [0,4194304] 0 2024-05-12T04:01:25.077 INFO:tasks.workunit.client.0.smithi032.stdout:0/604: rename df/d10/d3b/d75/la4 to df/d28/lcd 0 2024-05-12T04:01:25.077 INFO:tasks.workunit.client.0.smithi032.stdout:4/524: rename d0/d3/df to d0/d3/df/d91 22 2024-05-12T04:01:25.079 INFO:tasks.workunit.client.1.smithi033.stdout:6/770: read - d4/d1a/fed zero size 2024-05-12T04:01:25.081 INFO:tasks.workunit.client.0.smithi032.stdout:2/596: truncate d1/d13/d70/d85/f88 884982 0 2024-05-12T04:01:25.081 INFO:tasks.workunit.client.1.smithi033.stdout:8/701: fdatasync d1/d4/d46/f4f 0 2024-05-12T04:01:25.084 INFO:tasks.workunit.client.1.smithi033.stdout:7/941: read de/df/d17/d1e/d24/f57 [7811403,4080] 0 2024-05-12T04:01:25.084 INFO:tasks.workunit.client.1.smithi033.stdout:2/728: dwrite f7 [0,4194304] 0 2024-05-12T04:01:25.085 INFO:tasks.workunit.client.1.smithi033.stdout:0/781: dwrite d6/dc/d58/fbd [0,4194304] 0 2024-05-12T04:01:25.085 INFO:tasks.workunit.client.1.smithi033.stdout:4/751: rename dd/d7a/d87/d7e/da7/d66/d42/d59/lbd to dd/d11/d5b/d84/dd7/lec 0 2024-05-12T04:01:25.087 INFO:tasks.workunit.client.0.smithi032.stdout:1/580: creat d1/d25/fb8 x:0 0 0 2024-05-12T04:01:25.090 INFO:tasks.workunit.client.0.smithi032.stdout:7/562: mknod dd/d1b/d43/cb6 0 2024-05-12T04:01:25.097 INFO:tasks.workunit.client.1.smithi033.stdout:3/709: symlink db/d59/dcc/lf4 0 2024-05-12T04:01:25.097 INFO:tasks.workunit.client.1.smithi033.stdout:3/710: fdatasync db/d17/fc5 0 2024-05-12T04:01:25.099 INFO:tasks.workunit.client.0.smithi032.stdout:7/563: read dd/d1b/d1e/d45/d64/f91 [114985,35114] 0 2024-05-12T04:01:25.100 INFO:tasks.workunit.client.0.smithi032.stdout:9/502: rename d0/d3/d6/d7/da/f1c to d0/d3/d6/d58/fa7 0 2024-05-12T04:01:25.103 INFO:tasks.workunit.client.0.smithi032.stdout:8/579: symlink d0/d9/d1b/lca 0 2024-05-12T04:01:25.106 INFO:tasks.workunit.client.0.smithi032.stdout:8/580: readlink d0/d9/d1b/d27/d88/d56/d63/l6a 0 2024-05-12T04:01:25.106 INFO:tasks.workunit.client.0.smithi032.stdout:3/610: symlink d15/d1b/lda 0 2024-05-12T04:01:25.108 INFO:tasks.workunit.client.0.smithi032.stdout:3/611: write d15/f54 [4133853,92285] 0 2024-05-12T04:01:25.108 INFO:tasks.workunit.client.0.smithi032.stdout:4/525: creat d0/d3/df/d4e/f92 x:0 0 0 2024-05-12T04:01:25.114 INFO:tasks.workunit.client.1.smithi033.stdout:8/702: chown d1/d16/d29/ca2 29183 1 2024-05-12T04:01:25.114 INFO:tasks.workunit.client.0.smithi032.stdout:4/526: write d0/d3/d31/d80/d84/f7c [3933130,115947] 0 2024-05-12T04:01:25.114 INFO:tasks.workunit.client.0.smithi032.stdout:1/581: dread d1/d6a/f7a [0,4194304] 0 2024-05-12T04:01:25.115 INFO:tasks.workunit.client.1.smithi033.stdout:8/703: write d1/d4/db2/fbb [351951,34870] 0 2024-05-12T04:01:25.116 INFO:tasks.workunit.client.0.smithi032.stdout:3/612: dread d15/d20/d22/f23 [0,4194304] 0 2024-05-12T04:01:25.116 INFO:tasks.workunit.client.0.smithi032.stdout:2/597: creat d1/d34/da5/db3/db7/fc5 x:0 0 0 2024-05-12T04:01:25.118 INFO:tasks.workunit.client.1.smithi033.stdout:9/975: write d2/d4/d29/d3c/f9a [5031551,87384] 0 2024-05-12T04:01:25.118 INFO:tasks.workunit.client.1.smithi033.stdout:7/942: read de/d2f/fba [180164,3238] 0 2024-05-12T04:01:25.120 INFO:tasks.workunit.client.0.smithi032.stdout:8/581: dwrite d0/d9/d1b/d27/d4c/d4e/d90/fbc [0,4194304] 0 2024-05-12T04:01:25.121 INFO:tasks.workunit.client.0.smithi032.stdout:8/582: readlink d0/d9/d1b/d27/lc8 0 2024-05-12T04:01:25.125 INFO:tasks.workunit.client.0.smithi032.stdout:0/605: fdatasync df/d10/d48/f52 0 2024-05-12T04:01:25.128 INFO:tasks.workunit.client.1.smithi033.stdout:0/782: creat d6/dc/d58/d6a/dec/f105 x:0 0 0 2024-05-12T04:01:25.131 INFO:tasks.workunit.client.1.smithi033.stdout:5/661: truncate d3/d94/d54/d57/d70/da3/dbe/d16/f42 2378401 0 2024-05-12T04:01:25.132 INFO:tasks.workunit.client.1.smithi033.stdout:5/662: fsync d3/d94/d54/d57/d73/d74/f93 0 2024-05-12T04:01:25.132 INFO:tasks.workunit.client.0.smithi032.stdout:6/537: truncate d7/de/d2e/f58 1466726 0 2024-05-12T04:01:25.133 INFO:tasks.workunit.client.1.smithi033.stdout:5/663: truncate d3/d12/d29/d2b/d4f/faf 125032 0 2024-05-12T04:01:25.141 INFO:tasks.workunit.client.1.smithi033.stdout:2/729: dwrite dc/d17/d27/d2f/f6a [0,4194304] 0 2024-05-12T04:01:25.147 INFO:tasks.workunit.client.1.smithi033.stdout:4/752: dwrite dd/d69/fe9 [0,4194304] 0 2024-05-12T04:01:25.150 INFO:tasks.workunit.client.0.smithi032.stdout:9/503: dwrite d0/d3/d6/d7/d2a/f50 [0,4194304] 0 2024-05-12T04:01:25.153 INFO:tasks.workunit.client.1.smithi033.stdout:6/771: write d4/f29 [8930906,23864] 0 2024-05-12T04:01:25.154 INFO:tasks.workunit.client.0.smithi032.stdout:3/613: truncate d15/d24/d2a/d5b/f8e 107267 0 2024-05-12T04:01:25.154 INFO:tasks.workunit.client.0.smithi032.stdout:3/614: chown d15/d33/f46 1755 1 2024-05-12T04:01:25.156 INFO:tasks.workunit.client.0.smithi032.stdout:3/615: write d15/d20/d8b/d9b/fa8 [585762,20760] 0 2024-05-12T04:01:25.160 INFO:tasks.workunit.client.0.smithi032.stdout:8/583: creat d0/d4b/fcb x:0 0 0 2024-05-12T04:01:25.161 INFO:tasks.workunit.client.0.smithi032.stdout:8/584: stat d0/d9/d13/f58 0 2024-05-12T04:01:25.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:24 smithi033 ceph-mon[49525]: pgmap v40: 65 pgs: 65 active+clean; 2.5 GiB data, 8.8 GiB used, 528 GiB / 536 GiB avail; 62 MiB/s rd, 187 MiB/s wr, 365 op/s 2024-05-12T04:01:25.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:24 smithi033 ceph-mon[49525]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:01:25.164 INFO:tasks.workunit.client.0.smithi032.stdout:0/606: rmdir df/d10/d3d/d4a/d8e 39 2024-05-12T04:01:25.168 INFO:tasks.workunit.client.0.smithi032.stdout:6/538: mknod d7/d9/d5c/d86/cb8 0 2024-05-12T04:01:25.170 INFO:tasks.workunit.client.0.smithi032.stdout:2/598: write d1/d13/f4c [388644,98280] 0 2024-05-12T04:01:25.171 INFO:tasks.workunit.client.0.smithi032.stdout:0/607: dwrite df/d10/f3c [0,4194304] 0 2024-05-12T04:01:25.175 INFO:tasks.workunit.client.1.smithi033.stdout:5/664: dwrite d3/d94/d54/d57/d70/da3/dbe/d16/d45/d62/f64 [0,4194304] 0 2024-05-12T04:01:25.176 INFO:tasks.workunit.client.0.smithi032.stdout:2/599: write d1/d34/d96/d41/f2a [7433,31568] 0 2024-05-12T04:01:25.177 INFO:tasks.workunit.client.1.smithi033.stdout:0/783: write d6/db/d2c/f31 [2197956,85678] 0 2024-05-12T04:01:25.179 INFO:tasks.workunit.client.1.smithi033.stdout:0/784: chown d6/dc/d58/d6a/d80/db8/ff9 3 1 2024-05-12T04:01:25.179 INFO:tasks.workunit.client.0.smithi032.stdout:0/608: dread - df/d30/d8d/d8f/fcb zero size 2024-05-12T04:01:25.182 INFO:tasks.workunit.client.1.smithi033.stdout:1/653: rename d11/d17/d27/d36/d46 to d11/d1c/ddf 0 2024-05-12T04:01:25.183 INFO:tasks.workunit.client.0.smithi032.stdout:3/616: creat d15/d24/d2a/d8c/db1/fdb x:0 0 0 2024-05-12T04:01:25.185 INFO:tasks.workunit.client.0.smithi032.stdout:3/617: dread d15/d24/d40/f5a [0,4194304] 0 2024-05-12T04:01:25.188 INFO:tasks.workunit.client.1.smithi033.stdout:3/711: rmdir db/d17/df0 0 2024-05-12T04:01:25.189 INFO:tasks.workunit.client.1.smithi033.stdout:3/712: stat db/d59/f5b 0 2024-05-12T04:01:25.189 INFO:tasks.workunit.client.1.smithi033.stdout:2/730: creat dc/d17/dcc/fea x:0 0 0 2024-05-12T04:01:25.190 INFO:tasks.workunit.client.1.smithi033.stdout:2/731: read - dc/d13/d65/fa8 zero size 2024-05-12T04:01:25.191 INFO:tasks.workunit.client.0.smithi032.stdout:7/564: rename dd/d7b/d9a/d9c/db4 to dd/d1b/d1e/d45/d55/db7 0 2024-05-12T04:01:25.192 INFO:tasks.workunit.client.1.smithi033.stdout:7/943: rmdir de/df/d17/d1e/dd8/d114 0 2024-05-12T04:01:25.193 INFO:tasks.workunit.client.1.smithi033.stdout:7/944: read - de/d2f/fe9 zero size 2024-05-12T04:01:25.194 INFO:tasks.workunit.client.0.smithi032.stdout:6/539: mkdir d7/de/d2e/db9 0 2024-05-12T04:01:25.194 INFO:tasks.workunit.client.1.smithi033.stdout:7/945: fdatasync de/df/d7a/fde 0 2024-05-12T04:01:25.196 INFO:tasks.workunit.client.0.smithi032.stdout:4/527: creat d0/f93 x:0 0 0 2024-05-12T04:01:25.198 INFO:tasks.workunit.client.1.smithi033.stdout:6/772: rename d4/f29 to d4/d1a/d67/d6f/dac/f100 0 2024-05-12T04:01:25.199 INFO:tasks.workunit.client.1.smithi033.stdout:1/654: mknod d11/d17/d27/d36/d3f/d4a/ce0 0 2024-05-12T04:01:25.200 INFO:tasks.workunit.client.0.smithi032.stdout:1/582: write d1/d7/d30/d35/fa3 [4559527,28895] 0 2024-05-12T04:01:25.202 INFO:tasks.workunit.client.1.smithi033.stdout:1/655: read d11/f4f [1114931,74257] 0 2024-05-12T04:01:25.202 INFO:tasks.workunit.client.1.smithi033.stdout:9/976: truncate d2/d4/f30 4109639 0 2024-05-12T04:01:25.202 INFO:tasks.workunit.client.0.smithi032.stdout:9/504: symlink d0/d3/d6/d7/da/d89/la8 0 2024-05-12T04:01:25.203 INFO:tasks.workunit.client.1.smithi033.stdout:9/977: dread - d2/d4/d2b/d6d/d2f/d5f/d84/d107/f149 zero size 2024-05-12T04:01:25.205 INFO:tasks.workunit.client.1.smithi033.stdout:3/713: dread db/f81 [0,4194304] 0 2024-05-12T04:01:25.205 INFO:tasks.workunit.client.0.smithi032.stdout:0/609: creat df/d10/d3b/d75/fce x:0 0 0 2024-05-12T04:01:25.206 INFO:tasks.workunit.client.1.smithi033.stdout:5/665: dread d3/d94/f3e [0,4194304] 0 2024-05-12T04:01:25.207 INFO:tasks.workunit.client.1.smithi033.stdout:5/666: stat d3/l4d 0 2024-05-12T04:01:25.209 INFO:tasks.workunit.client.0.smithi032.stdout:2/600: dread d1/d34/f8c [0,4194304] 0 2024-05-12T04:01:25.210 INFO:tasks.workunit.client.1.smithi033.stdout:7/946: unlink de/df/d17/d1b/d36/f12e 0 2024-05-12T04:01:25.211 INFO:tasks.workunit.client.1.smithi033.stdout:7/947: dread - de/df/d65/dbc/fdb zero size 2024-05-12T04:01:25.211 INFO:tasks.workunit.client.1.smithi033.stdout:8/704: sync 2024-05-12T04:01:25.212 INFO:tasks.workunit.client.0.smithi032.stdout:8/585: creat d0/d4b/dad/fcc x:0 0 0 2024-05-12T04:01:25.214 INFO:tasks.workunit.client.1.smithi033.stdout:6/773: mknod d4/d1a/d3c/c101 0 2024-05-12T04:01:25.215 INFO:tasks.workunit.client.0.smithi032.stdout:3/618: chown d15/d1b/c1e 8987 1 2024-05-12T04:01:25.219 INFO:tasks.workunit.client.1.smithi033.stdout:8/705: dwrite d1/d4/f39 [0,4194304] 0 2024-05-12T04:01:25.220 INFO:tasks.workunit.client.0.smithi032.stdout:9/505: dread d0/d3/d6/d7/d2a/d46/d13/f56 [0,4194304] 0 2024-05-12T04:01:25.222 INFO:tasks.workunit.client.1.smithi033.stdout:9/978: dread d2/d4/f28 [0,4194304] 0 2024-05-12T04:01:25.223 INFO:tasks.workunit.client.1.smithi033.stdout:6/774: dwrite d4/d1a/d67/f8c [4194304,4194304] 0 2024-05-12T04:01:25.226 INFO:tasks.workunit.client.1.smithi033.stdout:8/706: dread d1/d16/fbe [0,4194304] 0 2024-05-12T04:01:25.227 INFO:tasks.workunit.client.0.smithi032.stdout:5/668: sync 2024-05-12T04:01:25.228 INFO:tasks.workunit.client.1.smithi033.stdout:3/714: truncate db/d17/d3a/d93/fc6 525524 0 2024-05-12T04:01:25.235 INFO:tasks.workunit.client.0.smithi032.stdout:2/601: dread d1/f21 [0,4194304] 0 2024-05-12T04:01:25.242 INFO:tasks.workunit.client.0.smithi032.stdout:3/619: mkdir d15/d20/d26/d6a/d6e/d7d/ddc 0 2024-05-12T04:01:25.244 INFO:tasks.workunit.client.0.smithi032.stdout:2/602: dwrite d1/d13/f2f [0,4194304] 0 2024-05-12T04:01:25.244 INFO:tasks.workunit.client.0.smithi032.stdout:5/669: dread d4/f22 [8388608,4194304] 0 2024-05-12T04:01:25.245 INFO:tasks.workunit.client.0.smithi032.stdout:5/670: readlink d4/l27 0 2024-05-12T04:01:25.246 INFO:tasks.workunit.client.0.smithi032.stdout:7/565: link dd/d1b/d1e/d45/d55/d60/f88 dd/d7b/d9a/fb8 0 2024-05-12T04:01:25.247 INFO:tasks.workunit.client.0.smithi032.stdout:7/566: chown dd/d1b/d1e/d45/d55/fa7 2853 1 2024-05-12T04:01:25.248 INFO:tasks.workunit.client.0.smithi032.stdout:9/506: write d0/d3/d6/d7/da/f1a [4093687,111300] 0 2024-05-12T04:01:25.252 INFO:tasks.workunit.client.0.smithi032.stdout:6/540: link d7/d9/d5c/d8b/da1/fa2 d7/de/d57/fba 0 2024-05-12T04:01:25.255 INFO:tasks.workunit.client.0.smithi032.stdout:4/528: symlink d0/d3/l94 0 2024-05-12T04:01:25.256 INFO:tasks.workunit.client.0.smithi032.stdout:0/610: write df/d10/d3d/d4a/f2e [2299787,7646] 0 2024-05-12T04:01:25.261 INFO:tasks.workunit.client.0.smithi032.stdout:1/583: dwrite d1/d6a/f6e [0,4194304] 0 2024-05-12T04:01:25.274 INFO:tasks.workunit.client.0.smithi032.stdout:5/671: write d4/f20 [1933358,64653] 0 2024-05-12T04:01:25.275 INFO:tasks.workunit.client.1.smithi033.stdout:2/732: link dc/l2c dc/d17/d32/d84/dde/d9e/dd7/leb 0 2024-05-12T04:01:25.276 INFO:tasks.workunit.client.1.smithi033.stdout:5/667: mknod d3/d12/d25/d26/cdb 0 2024-05-12T04:01:25.276 INFO:tasks.workunit.client.0.smithi032.stdout:2/603: read d1/d34/d96/d41/d2c/f59 [1628702,17532] 0 2024-05-12T04:01:25.277 INFO:tasks.workunit.client.1.smithi033.stdout:7/948: fdatasync de/d2b/d6c/fd7 0 2024-05-12T04:01:25.278 INFO:tasks.workunit.client.1.smithi033.stdout:1/656: write d11/d1c/f94 [967014,73909] 0 2024-05-12T04:01:25.278 INFO:tasks.workunit.client.1.smithi033.stdout:0/785: dwrite d6/db/d2c/d73/f7a [0,4194304] 0 2024-05-12T04:01:25.278 INFO:tasks.workunit.client.1.smithi033.stdout:2/733: fsync dc/d17/d27/d2f/f38 0 2024-05-12T04:01:25.281 INFO:tasks.workunit.client.0.smithi032.stdout:5/672: stat d4/d7/d58/d5a/d99 0 2024-05-12T04:01:25.283 INFO:tasks.workunit.client.1.smithi033.stdout:1/657: truncate d11/d1c/ddf/d4c/fc8 1721733 0 2024-05-12T04:01:25.283 INFO:tasks.workunit.client.0.smithi032.stdout:1/584: dwrite d1/dd/d24/f9c [0,4194304] 0 2024-05-12T04:01:25.284 INFO:tasks.workunit.client.1.smithi033.stdout:1/658: stat d11/d17/d27/d36/d3f/d69/d87/d96 0 2024-05-12T04:01:25.288 INFO:tasks.workunit.client.0.smithi032.stdout:9/507: dwrite d0/d3/ff [0,4194304] 0 2024-05-12T04:01:25.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:24 smithi032 ceph-mon[40008]: pgmap v40: 65 pgs: 65 active+clean; 2.5 GiB data, 8.8 GiB used, 528 GiB / 536 GiB avail; 62 MiB/s rd, 187 MiB/s wr, 365 op/s 2024-05-12T04:01:25.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:24 smithi032 ceph-mon[40008]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:01:25.291 INFO:tasks.workunit.client.0.smithi032.stdout:3/620: dread d15/f19 [4194304,4194304] 0 2024-05-12T04:01:25.293 INFO:tasks.workunit.client.1.smithi033.stdout:6/775: rename d4/d1a/d4e/d4f/f63 to d4/d1a/f102 0 2024-05-12T04:01:25.295 INFO:tasks.workunit.client.1.smithi033.stdout:6/776: dread d4/d5/f20 [0,4194304] 0 2024-05-12T04:01:25.301 INFO:tasks.workunit.client.1.smithi033.stdout:2/734: creat dc/d17/d32/d84/dde/d9e/dd7/fec x:0 0 0 2024-05-12T04:01:25.303 INFO:tasks.workunit.client.1.smithi033.stdout:3/715: rename db/d17/d25/d32/d53/c7a to db/d17/d25/d32/d3c/d3d/d47/cf5 0 2024-05-12T04:01:25.305 INFO:tasks.workunit.client.1.smithi033.stdout:1/659: mkdir d11/d17/d27/d36/d3f/d4a/dc6/de1 0 2024-05-12T04:01:25.309 INFO:tasks.workunit.client.1.smithi033.stdout:7/949: rmdir de/df/d65/dbc/d13b 0 2024-05-12T04:01:25.310 INFO:tasks.workunit.client.1.smithi033.stdout:4/753: sync 2024-05-12T04:01:25.310 INFO:tasks.workunit.client.1.smithi033.stdout:5/668: link d3/d94/d54/d57/d70/da3/dbe/f30 d3/d12/d29/fdc 0 2024-05-12T04:01:25.310 INFO:tasks.workunit.client.1.smithi033.stdout:3/716: dwrite db/d17/d3a/d67/d76/fb2 [4194304,4194304] 0 2024-05-12T04:01:25.312 INFO:tasks.workunit.client.1.smithi033.stdout:5/669: dread - d3/d94/d54/d57/d70/da3/dbe/fc1 zero size 2024-05-12T04:01:25.313 INFO:tasks.workunit.client.1.smithi033.stdout:2/735: creat dc/d13/d1c/d98/fed x:0 0 0 2024-05-12T04:01:25.315 INFO:tasks.workunit.client.1.smithi033.stdout:0/786: rename d6/dc/f1d to d6/dc/dd1/f106 0 2024-05-12T04:01:25.321 INFO:tasks.workunit.client.1.smithi033.stdout:8/707: getdents d1/d16/d27/d4d/d51/d8e 0 2024-05-12T04:01:25.322 INFO:tasks.workunit.client.1.smithi033.stdout:8/708: chown d1/d16/d27/d28/l5b 49650431 1 2024-05-12T04:01:25.324 INFO:tasks.workunit.client.0.smithi032.stdout:6/541: symlink d7/de/d2e/db9/lbb 0 2024-05-12T04:01:25.327 INFO:tasks.workunit.client.1.smithi033.stdout:4/754: rmdir dd/d7a/d87/d7e/da7 39 2024-05-12T04:01:25.328 INFO:tasks.workunit.client.0.smithi032.stdout:4/529: mkdir d0/d3/d12/d1d/d63/d87/d95 0 2024-05-12T04:01:25.328 INFO:tasks.workunit.client.0.smithi032.stdout:4/530: readlink d0/d3/l1e 0 2024-05-12T04:01:25.328 INFO:tasks.workunit.client.1.smithi033.stdout:8/709: dwrite d1/d4/d1d/d36/f50 [4194304,4194304] 0 2024-05-12T04:01:25.329 INFO:tasks.workunit.client.1.smithi033.stdout:4/755: write dd/d7a/fd6 [497109,48763] 0 2024-05-12T04:01:25.329 INFO:tasks.workunit.client.1.smithi033.stdout:3/717: rmdir db/d17/d25/d32/d3c 39 2024-05-12T04:01:25.330 INFO:tasks.workunit.client.1.smithi033.stdout:8/710: chown d1/d16/d27/d7c/fc6 1114377342 1 2024-05-12T04:01:25.333 INFO:tasks.workunit.client.1.smithi033.stdout:2/736: unlink dc/c9c 0 2024-05-12T04:01:25.335 INFO:tasks.workunit.client.0.smithi032.stdout:2/604: symlink d1/d34/d4a/lc6 0 2024-05-12T04:01:25.336 INFO:tasks.workunit.client.0.smithi032.stdout:2/605: fdatasync d1/d13/da0/dae/fc1 0 2024-05-12T04:01:25.338 INFO:tasks.workunit.client.0.smithi032.stdout:1/585: read d1/d7/d49/f4f [210325,38068] 0 2024-05-12T04:01:25.340 INFO:tasks.workunit.client.1.smithi033.stdout:1/660: truncate d11/d1c/d59/f7f 5227177 0 2024-05-12T04:01:25.341 INFO:tasks.workunit.client.0.smithi032.stdout:9/508: readlink d0/lb 0 2024-05-12T04:01:25.342 INFO:tasks.workunit.client.1.smithi033.stdout:1/661: write d11/d17/d27/da6/fc2 [300501,67018] 0 2024-05-12T04:01:25.343 INFO:tasks.workunit.client.1.smithi033.stdout:1/662: write d11/d1c/ddf/f74 [1548758,29001] 0 2024-05-12T04:01:25.343 INFO:tasks.workunit.client.1.smithi033.stdout:9/979: sync 2024-05-12T04:01:25.343 INFO:tasks.workunit.client.0.smithi032.stdout:3/621: rmdir d15/d20/d8b 39 2024-05-12T04:01:25.344 INFO:tasks.workunit.client.1.smithi033.stdout:6/777: write d4/d1a/d4e/d4f/f5e [876699,120780] 0 2024-05-12T04:01:25.347 INFO:tasks.workunit.client.0.smithi032.stdout:8/586: rename d0/d9/d1b/d27/d4c/d4e/d7f/d91/cac to d0/d9/d1b/d27/d4c/d4e/ccd 0 2024-05-12T04:01:25.349 INFO:tasks.workunit.client.0.smithi032.stdout:8/587: write d0/d9/d1b/d27/d59/fa4 [336961,31684] 0 2024-05-12T04:01:25.353 INFO:tasks.workunit.client.0.smithi032.stdout:4/531: symlink d0/d3/d12/d1d/d63/l96 0 2024-05-12T04:01:25.357 INFO:tasks.workunit.client.0.smithi032.stdout:0/611: creat df/d30/d32/fcf x:0 0 0 2024-05-12T04:01:25.358 INFO:tasks.workunit.client.0.smithi032.stdout:6/542: dwrite d7/d9/d48/d67/d25/f2a [0,4194304] 0 2024-05-12T04:01:25.359 INFO:tasks.workunit.client.1.smithi033.stdout:2/737: rmdir dc/d13/d1c/d98 39 2024-05-12T04:01:25.363 INFO:tasks.workunit.client.0.smithi032.stdout:0/612: dread df/d22/f39 [0,4194304] 0 2024-05-12T04:01:25.364 INFO:tasks.workunit.client.0.smithi032.stdout:1/586: creat d1/d6a/fb9 x:0 0 0 2024-05-12T04:01:25.367 INFO:tasks.workunit.client.0.smithi032.stdout:5/673: mknod d4/d7/d14/de8/cee 0 2024-05-12T04:01:25.372 INFO:tasks.workunit.client.1.smithi033.stdout:5/670: write d3/d94/d54/d57/d70/da3/dbe/d16/d45/d62/fc0 [1355099,88479] 0 2024-05-12T04:01:25.375 INFO:tasks.workunit.client.0.smithi032.stdout:7/567: rename dd/d1b/d1e/d45/d55/d60 to dd/d7b/d9a/d9c/db9 0 2024-05-12T04:01:25.376 INFO:tasks.workunit.client.1.smithi033.stdout:6/778: read - d4/f9a zero size 2024-05-12T04:01:25.376 INFO:tasks.workunit.client.1.smithi033.stdout:6/779: readlink d4/d8d/le6 0 2024-05-12T04:01:25.379 INFO:tasks.workunit.client.0.smithi032.stdout:8/588: mknod d0/d9/d1b/d27/d59/cce 0 2024-05-12T04:01:25.383 INFO:tasks.workunit.client.1.smithi033.stdout:4/756: dwrite dd/f2a [0,4194304] 0 2024-05-12T04:01:25.385 INFO:tasks.workunit.client.1.smithi033.stdout:8/711: dwrite d1/d4/d1d/d36/d3c/d64/f7b [0,4194304] 0 2024-05-12T04:01:25.386 INFO:tasks.workunit.client.1.smithi033.stdout:2/738: rename dc/d17/d27/d2f/d49/c95 to dc/d17/d32/d84/dde/cee 0 2024-05-12T04:01:25.387 INFO:tasks.workunit.client.1.smithi033.stdout:2/739: fsync dc/fe3 0 2024-05-12T04:01:25.392 INFO:tasks.workunit.client.0.smithi032.stdout:1/587: creat d1/d7/fba x:0 0 0 2024-05-12T04:01:25.393 INFO:tasks.workunit.client.1.smithi033.stdout:3/718: write db/d17/d25/d32/d3c/d3d/f8b [276205,12256] 0 2024-05-12T04:01:25.394 INFO:tasks.workunit.client.1.smithi033.stdout:7/950: getdents de/d20 0 2024-05-12T04:01:25.395 INFO:tasks.workunit.client.0.smithi032.stdout:5/674: rmdir d4/d7/d14/d39/d1b 39 2024-05-12T04:01:25.395 INFO:tasks.workunit.client.1.smithi033.stdout:7/951: chown de/d2b/lf7 3 1 2024-05-12T04:01:25.397 INFO:tasks.workunit.client.0.smithi032.stdout:5/675: chown d4/d7/d14/d26/d33/d4a/d38/fb8 569434 1 2024-05-12T04:01:25.397 INFO:tasks.workunit.client.1.smithi033.stdout:7/952: chown de/df/d17/d1b/d36/f8a 25 1 2024-05-12T04:01:25.398 INFO:tasks.workunit.client.1.smithi033.stdout:5/671: creat d3/d94/d85/d99/da2/fdd x:0 0 0 2024-05-12T04:01:25.398 INFO:tasks.workunit.client.1.smithi033.stdout:5/672: truncate d3/d12/f3f 4307071 0 2024-05-12T04:01:25.399 INFO:tasks.workunit.client.0.smithi032.stdout:0/613: dread df/d30/d4e/f73 [0,4194304] 0 2024-05-12T04:01:25.446 INFO:tasks.workunit.client.0.smithi032.stdout:6/543: rename d7/d9/d60 to d7/d9/d48/d67/d8d/dbc 0 2024-05-12T04:01:25.446 INFO:tasks.workunit.client.0.smithi032.stdout:6/544: stat d7/d33/f7a 0 2024-05-12T04:01:25.448 INFO:tasks.workunit.client.0.smithi032.stdout:7/568: rmdir dd/d7b/d9a/d9c/db9/d75 39 2024-05-12T04:01:25.451 INFO:tasks.workunit.client.1.smithi033.stdout:1/663: sync 2024-05-12T04:01:25.453 INFO:tasks.workunit.client.0.smithi032.stdout:3/622: truncate d15/d33/f9e 2093417 0 2024-05-12T04:01:25.454 INFO:tasks.workunit.client.0.smithi032.stdout:3/623: chown d15/dc2/fc3 14117 1 2024-05-12T04:01:25.457 INFO:tasks.workunit.client.0.smithi032.stdout:4/532: truncate d0/d3/d12/d1d/f2a 2209314 0 2024-05-12T04:01:25.458 INFO:tasks.workunit.client.0.smithi032.stdout:4/533: read d0/f11 [2100608,89542] 0 2024-05-12T04:01:25.465 INFO:tasks.workunit.client.1.smithi033.stdout:0/787: link d6/dc/d39/f8a d6/dc/f107 0 2024-05-12T04:01:25.466 INFO:tasks.workunit.client.1.smithi033.stdout:3/719: mkdir db/d17/d25/d32/d53/df6 0 2024-05-12T04:01:25.466 INFO:tasks.workunit.client.0.smithi032.stdout:4/534: dwrite d0/f7b [0,4194304] 0 2024-05-12T04:01:25.467 INFO:tasks.workunit.client.1.smithi033.stdout:0/788: stat d6/db/d2c/c4a 0 2024-05-12T04:01:25.468 INFO:tasks.workunit.client.0.smithi032.stdout:5/676: mknod d4/d7/d14/d39/d2b/cef 0 2024-05-12T04:01:25.473 INFO:tasks.workunit.client.0.smithi032.stdout:0/614: rename df/d10/d3d/d4a/d84/lca to df/d10/d3d/d83/d9f/ld0 0 2024-05-12T04:01:25.476 INFO:tasks.workunit.client.0.smithi032.stdout:6/545: creat d7/d9/d5c/fbd x:0 0 0 2024-05-12T04:01:25.478 INFO:tasks.workunit.client.0.smithi032.stdout:6/546: chown d7/d9/d48/d67/d25/f7e 3 1 2024-05-12T04:01:25.479 INFO:tasks.workunit.client.1.smithi033.stdout:4/757: creat dd/d7a/d87/d7e/da7/d66/d42/d59/fed x:0 0 0 2024-05-12T04:01:25.486 INFO:tasks.workunit.client.0.smithi032.stdout:2/606: link d1/c3c d1/d34/d95/cc7 0 2024-05-12T04:01:25.489 INFO:tasks.workunit.client.0.smithi032.stdout:1/588: link d1/d7/d30/d18/f8b d1/d7/d30/d18/d87/fbb 0 2024-05-12T04:01:25.490 INFO:tasks.workunit.client.0.smithi032.stdout:3/624: write d15/d24/d2a/fad [821071,60159] 0 2024-05-12T04:01:25.491 INFO:tasks.workunit.client.0.smithi032.stdout:4/535: creat d0/d3/d12/d1d/d63/f97 x:0 0 0 2024-05-12T04:01:25.493 INFO:tasks.workunit.client.0.smithi032.stdout:7/569: dwrite dd/d1b/d1e/d45/d64/f91 [0,4194304] 0 2024-05-12T04:01:25.493 INFO:tasks.workunit.client.0.smithi032.stdout:4/536: readlink d0/d3/l54 0 2024-05-12T04:01:25.495 INFO:tasks.workunit.client.0.smithi032.stdout:7/570: chown dd/d1b/d43/cb6 0 1 2024-05-12T04:01:25.500 INFO:tasks.workunit.client.0.smithi032.stdout:3/625: dread d15/d20/d22/d7a/f85 [0,4194304] 0 2024-05-12T04:01:25.504 INFO:tasks.workunit.client.0.smithi032.stdout:7/571: dwrite dd/d7b/fb1 [0,4194304] 0 2024-05-12T04:01:25.507 INFO:tasks.workunit.client.0.smithi032.stdout:6/547: rmdir d7/d9/d5c/d8b/da1 39 2024-05-12T04:01:25.507 INFO:tasks.workunit.client.0.smithi032.stdout:5/677: write d4/d7/d14/d26/d47/f9e [363497,99462] 0 2024-05-12T04:01:25.511 INFO:tasks.workunit.client.0.smithi032.stdout:8/589: creat d0/d9/d1b/d27/d88/d56/d63/d8f/d9a/fcf x:0 0 0 2024-05-12T04:01:25.513 INFO:tasks.workunit.client.0.smithi032.stdout:2/607: mknod d1/d34/d4a/cc8 0 2024-05-12T04:01:25.518 INFO:tasks.workunit.client.0.smithi032.stdout:1/589: symlink d1/d59/lbc 0 2024-05-12T04:01:25.519 INFO:tasks.workunit.client.0.smithi032.stdout:0/615: dwrite df/d10/d3d/f5c [0,4194304] 0 2024-05-12T04:01:25.520 INFO:tasks.workunit.client.0.smithi032.stdout:0/616: readlink df/d10/d3d/d4a/la8 0 2024-05-12T04:01:25.527 INFO:tasks.workunit.client.0.smithi032.stdout:4/537: rename d0/d3/d31/d80/d84/f4b to d0/d3/d12/d8d/f98 0 2024-05-12T04:01:25.527 INFO:tasks.workunit.client.0.smithi032.stdout:0/617: dwrite fb [0,4194304] 0 2024-05-12T04:01:25.533 INFO:tasks.workunit.client.0.smithi032.stdout:7/572: mknod dd/d1b/d1e/d45/d64/cba 0 2024-05-12T04:01:25.533 INFO:tasks.workunit.client.0.smithi032.stdout:0/618: dread df/d10/f3c [4194304,4194304] 0 2024-05-12T04:01:25.544 INFO:tasks.workunit.client.1.smithi033.stdout:2/740: sync 2024-05-12T04:01:25.544 INFO:tasks.workunit.client.0.smithi032.stdout:4/538: fsync d0/d3/d12/f5b 0 2024-05-12T04:01:25.549 INFO:tasks.workunit.client.0.smithi032.stdout:7/573: mknod dd/d1b/cbb 0 2024-05-12T04:01:25.549 INFO:tasks.workunit.client.0.smithi032.stdout:7/574: read dd/d1b/d1e/faa [139954,22178] 0 2024-05-12T04:01:25.550 INFO:tasks.workunit.client.0.smithi032.stdout:0/619: rmdir df/d10/d3b 39 2024-05-12T04:01:25.550 INFO:tasks.workunit.client.0.smithi032.stdout:7/575: fsync dd/d1b/d43/fa8 0 2024-05-12T04:01:25.551 INFO:tasks.workunit.client.0.smithi032.stdout:0/620: dread - df/d10/d3d/db0/fbd zero size 2024-05-12T04:01:25.552 INFO:tasks.workunit.client.0.smithi032.stdout:7/576: write dd/d1b/d43/d5e/d37/f4f [5152496,68276] 0 2024-05-12T04:01:25.553 INFO:tasks.workunit.client.0.smithi032.stdout:8/590: creat d0/d9/d1b/d27/d4c/d4e/d7f/da3/fd0 x:0 0 0 2024-05-12T04:01:25.554 INFO:tasks.workunit.client.0.smithi032.stdout:2/608: symlink d1/d34/lc9 0 2024-05-12T04:01:25.558 INFO:tasks.workunit.client.0.smithi032.stdout:4/539: creat d0/d3/d12/d1d/d63/f99 x:0 0 0 2024-05-12T04:01:25.558 INFO:tasks.workunit.client.0.smithi032.stdout:7/577: dread dd/d1b/d1e/f7e [0,4194304] 0 2024-05-12T04:01:25.562 INFO:tasks.workunit.client.0.smithi032.stdout:4/540: stat d0/d3/d31/d80/f82 0 2024-05-12T04:01:25.563 INFO:tasks.workunit.client.1.smithi033.stdout:3/720: creat db/d17/d3e/dd7/ff7 x:0 0 0 2024-05-12T04:01:25.564 INFO:tasks.workunit.client.1.smithi033.stdout:3/721: chown db/d17/l38 0 1 2024-05-12T04:01:25.564 INFO:tasks.workunit.client.0.smithi032.stdout:0/621: dwrite df/d10/d48/d70/f76 [0,4194304] 0 2024-05-12T04:01:25.567 INFO:tasks.workunit.client.1.smithi033.stdout:7/953: fdatasync de/d2b/d6c/da0/da6/dd1/faf 0 2024-05-12T04:01:25.568 INFO:tasks.workunit.client.0.smithi032.stdout:2/609: dwrite d1/d34/d96/d41/d39/f87 [0,4194304] 0 2024-05-12T04:01:25.569 INFO:tasks.workunit.client.1.smithi033.stdout:9/980: link d2/d4/d2b/d49/d73/l11c d2/d4/d2b/d33/d139/l15a 0 2024-05-12T04:01:25.576 INFO:tasks.workunit.client.0.smithi032.stdout:8/591: symlink d0/d9/d1b/d27/d4c/ld1 0 2024-05-12T04:01:25.577 INFO:tasks.workunit.client.0.smithi032.stdout:8/592: write d0/d9/d1b/d27/d4c/d4e/d7f/f9c [457732,23194] 0 2024-05-12T04:01:25.578 INFO:tasks.workunit.client.1.smithi033.stdout:1/664: symlink d11/d17/d38/le2 0 2024-05-12T04:01:25.579 INFO:tasks.workunit.client.0.smithi032.stdout:8/593: chown d0/d9/d1b/d27/d59/cc5 31998 1 2024-05-12T04:01:25.585 INFO:tasks.workunit.client.1.smithi033.stdout:6/780: rename d4/d1a/lda to d4/l103 0 2024-05-12T04:01:25.588 INFO:tasks.workunit.client.0.smithi032.stdout:5/678: dwrite d4/d7/d14/d26/d33/f8b [0,4194304] 0 2024-05-12T04:01:25.589 INFO:tasks.workunit.client.0.smithi032.stdout:4/541: creat d0/d3/d12/d1d/d6b/f9a x:0 0 0 2024-05-12T04:01:25.591 INFO:tasks.workunit.client.0.smithi032.stdout:5/679: chown d4/d7/d14/d26/d33/d4a/d38/dd2 237614 1 2024-05-12T04:01:25.592 INFO:tasks.workunit.client.1.smithi033.stdout:0/789: unlink d6/c1b 0 2024-05-12T04:01:25.593 INFO:tasks.workunit.client.0.smithi032.stdout:7/578: dread dd/d1b/d1e/d45/d64/f95 [0,4194304] 0 2024-05-12T04:01:25.596 INFO:tasks.workunit.client.1.smithi033.stdout:0/790: dread d6/dc/d58/d6a/f89 [0,4194304] 0 2024-05-12T04:01:25.597 INFO:tasks.workunit.client.0.smithi032.stdout:7/579: write dd/d1b/d43/d74/f7f [1911651,101544] 0 2024-05-12T04:01:25.598 INFO:tasks.workunit.client.1.smithi033.stdout:7/954: mknod de/d2b/d6c/dbd/d101/c13d 0 2024-05-12T04:01:25.600 INFO:tasks.workunit.client.1.smithi033.stdout:9/981: dread - d2/d4/d2b/f105 zero size 2024-05-12T04:01:25.603 INFO:tasks.workunit.client.0.smithi032.stdout:3/626: link d15/d1b/l1d d15/d24/d2a/ldd 0 2024-05-12T04:01:25.604 INFO:tasks.workunit.client.0.smithi032.stdout:3/627: stat l4 0 2024-05-12T04:01:25.604 INFO:tasks.workunit.client.0.smithi032.stdout:4/542: dwrite d0/d3/d12/d1d/d63/f70 [0,4194304] 0 2024-05-12T04:01:25.608 INFO:tasks.workunit.client.0.smithi032.stdout:5/680: dwrite d4/d7/d14/d26/d33/d4a/d38/dd2/fd9 [0,4194304] 0 2024-05-12T04:01:25.609 INFO:tasks.workunit.client.0.smithi032.stdout:4/543: readlink d0/d3/df/l40 0 2024-05-12T04:01:25.610 INFO:tasks.workunit.client.0.smithi032.stdout:5/681: read - d4/d7/d14/d26/d33/d72/dc9/fca zero size 2024-05-12T04:01:25.611 INFO:tasks.workunit.client.1.smithi033.stdout:8/712: rename d1/d4/d1d to d1/d16/d27/d4d/d82/dd9 0 2024-05-12T04:01:25.613 INFO:tasks.workunit.client.1.smithi033.stdout:6/781: mknod d4/d1a/d67/d6f/daf/c104 0 2024-05-12T04:01:25.614 INFO:tasks.workunit.client.1.smithi033.stdout:6/782: stat d4/d1a/d3c/c54 0 2024-05-12T04:01:25.615 INFO:tasks.workunit.client.1.smithi033.stdout:2/741: truncate dc/d17/d27/d2f/f90 717467 0 2024-05-12T04:01:25.616 INFO:tasks.workunit.client.1.smithi033.stdout:2/742: write dc/fb8 [588646,125018] 0 2024-05-12T04:01:25.617 INFO:tasks.workunit.client.0.smithi032.stdout:0/622: symlink df/d10/ld1 0 2024-05-12T04:01:25.622 INFO:tasks.workunit.client.1.smithi033.stdout:9/982: mkdir d2/d70/d7b/d15b 0 2024-05-12T04:01:25.626 INFO:tasks.workunit.client.0.smithi032.stdout:8/594: symlink d0/d9/d13/d21/d25/db5/ld2 0 2024-05-12T04:01:25.627 INFO:tasks.workunit.client.0.smithi032.stdout:8/595: readlink d0/l51 0 2024-05-12T04:01:25.628 INFO:tasks.workunit.client.1.smithi033.stdout:3/722: rename db/d17/d4d/l9a to db/d42/lf8 0 2024-05-12T04:01:25.632 INFO:tasks.workunit.client.1.smithi033.stdout:5/673: truncate d3/d94/d54/d98/f9f 1338510 0 2024-05-12T04:01:25.633 INFO:tasks.workunit.client.1.smithi033.stdout:5/674: dread - d3/d94/d54/d57/d70/da3/dbe/daa/fbc zero size 2024-05-12T04:01:25.634 INFO:tasks.workunit.client.1.smithi033.stdout:6/783: mknod d4/d5/dcd/c105 0 2024-05-12T04:01:25.635 INFO:tasks.workunit.client.0.smithi032.stdout:5/682: creat d4/d7/d14/d26/d33/d72/dc9/ff0 x:0 0 0 2024-05-12T04:01:25.640 INFO:tasks.workunit.client.0.smithi032.stdout:4/544: dread d0/d3/d12/d1d/f2a [0,4194304] 0 2024-05-12T04:01:25.640 INFO:tasks.workunit.client.1.smithi033.stdout:2/743: unlink dc/d17/d3a/f8b 0 2024-05-12T04:01:25.642 INFO:tasks.workunit.client.0.smithi032.stdout:7/580: mknod dd/d7b/d9a/d9c/db9/d9f/daf/cbc 0 2024-05-12T04:01:25.643 INFO:tasks.workunit.client.0.smithi032.stdout:7/581: readlink dd/d1b/l31 0 2024-05-12T04:01:25.648 INFO:tasks.workunit.client.0.smithi032.stdout:3/628: symlink d15/d24/d2a/d8c/db1/dc8/lde 0 2024-05-12T04:01:25.648 INFO:tasks.workunit.client.0.smithi032.stdout:4/545: dwrite d0/d3/f88 [0,4194304] 0 2024-05-12T04:01:25.649 INFO:tasks.workunit.client.0.smithi032.stdout:8/596: dread d0/d9/d13/d21/d25/f6e [0,4194304] 0 2024-05-12T04:01:25.652 INFO:tasks.workunit.client.0.smithi032.stdout:3/629: readlink d15/d1b/lda 0 2024-05-12T04:01:25.652 INFO:tasks.workunit.client.1.smithi033.stdout:4/758: write dd/d11/fa0 [392210,114415] 0 2024-05-12T04:01:25.653 INFO:tasks.workunit.client.1.smithi033.stdout:4/759: chown dd/l5c 45355 1 2024-05-12T04:01:25.656 INFO:tasks.workunit.client.0.smithi032.stdout:0/623: dread f9 [4194304,4194304] 0 2024-05-12T04:01:25.656 INFO:tasks.workunit.client.1.smithi033.stdout:1/665: rmdir d11/d1c/ddf/ddd 0 2024-05-12T04:01:25.658 INFO:tasks.workunit.client.0.smithi032.stdout:2/610: dwrite d1/fc [0,4194304] 0 2024-05-12T04:01:25.660 INFO:tasks.workunit.client.1.smithi033.stdout:0/791: rename d6/dc/d58/d6a/d80/fc5 to d6/dc/dd1/f108 0 2024-05-12T04:01:25.662 INFO:tasks.workunit.client.0.smithi032.stdout:3/630: dwrite d15/d24/f81 [0,4194304] 0 2024-05-12T04:01:25.667 INFO:tasks.workunit.client.1.smithi033.stdout:3/723: unlink db/d42/c4b 0 2024-05-12T04:01:25.671 INFO:tasks.workunit.client.1.smithi033.stdout:5/675: fdatasync d3/d94/d54/d57/d73/fa6 0 2024-05-12T04:01:25.674 INFO:tasks.workunit.client.1.smithi033.stdout:6/784: mkdir d4/d1a/d67/d6f/d106 0 2024-05-12T04:01:25.675 INFO:tasks.workunit.client.1.smithi033.stdout:6/785: stat d4/d43/d57/l64 0 2024-05-12T04:01:25.676 INFO:tasks.workunit.client.1.smithi033.stdout:6/786: chown d4/d1a/d6c/c99 6348767 1 2024-05-12T04:01:25.677 INFO:tasks.workunit.client.1.smithi033.stdout:2/744: fsync dc/d17/d27/d2f/d49/fbb 0 2024-05-12T04:01:25.679 INFO:tasks.workunit.client.1.smithi033.stdout:7/955: rmdir de/df/d60/d109 0 2024-05-12T04:01:25.679 INFO:tasks.workunit.client.0.smithi032.stdout:4/546: fdatasync d0/d3/d31/d80/d84/f6a 0 2024-05-12T04:01:25.681 INFO:tasks.workunit.client.0.smithi032.stdout:4/547: chown d0/d3/df/f26 50506 1 2024-05-12T04:01:25.681 INFO:tasks.workunit.client.1.smithi033.stdout:8/713: dwrite d1/d16/d27/d4d/d82/dd9/f78 [0,4194304] 0 2024-05-12T04:01:25.682 INFO:tasks.workunit.client.0.smithi032.stdout:4/548: dread d0/f6f [0,4194304] 0 2024-05-12T04:01:25.685 INFO:tasks.workunit.client.1.smithi033.stdout:8/714: write d1/d16/d27/d4d/fd4 [158414,2198] 0 2024-05-12T04:01:25.685 INFO:tasks.workunit.client.1.smithi033.stdout:9/983: link d2/dd/d79/fae d2/d4/d2b/d33/dd2/d132/f15c 0 2024-05-12T04:01:25.688 INFO:tasks.workunit.client.1.smithi033.stdout:4/760: rmdir dd/d7a/d87/d7e/da7/d66/d42 39 2024-05-12T04:01:25.691 INFO:tasks.workunit.client.1.smithi033.stdout:1/666: symlink d11/d17/d27/d36/d3f/d4a/le3 0 2024-05-12T04:01:25.693 INFO:tasks.workunit.client.1.smithi033.stdout:9/984: dwrite d2/d4/d2b/d6d/d2f/d5f/d84/d137/f158 [0,4194304] 0 2024-05-12T04:01:25.694 INFO:tasks.workunit.client.0.smithi032.stdout:3/631: rmdir d15/d20/d26/d6a/d6e/d7d 39 2024-05-12T04:01:25.695 INFO:tasks.workunit.client.0.smithi032.stdout:3/632: write d15/d20/d22/f57 [4674153,118124] 0 2024-05-12T04:01:25.697 INFO:tasks.workunit.client.0.smithi032.stdout:5/683: creat d4/d7/d14/d26/d33/d4a/ff1 x:0 0 0 2024-05-12T04:01:25.699 INFO:tasks.workunit.client.1.smithi033.stdout:0/792: symlink d6/dc/d96/d9d/l109 0 2024-05-12T04:01:25.703 INFO:tasks.workunit.client.0.smithi032.stdout:3/633: dwrite d15/d20/d22/d4f/f53 [0,4194304] 0 2024-05-12T04:01:25.712 INFO:tasks.workunit.client.0.smithi032.stdout:4/549: dread d0/f65 [0,4194304] 0 2024-05-12T04:01:25.718 INFO:tasks.workunit.client.1.smithi033.stdout:2/745: fdatasync dc/d13/d1f/f69 0 2024-05-12T04:01:25.718 INFO:tasks.workunit.client.1.smithi033.stdout:7/956: chown de/d2b/d6c/c115 168923947 1 2024-05-12T04:01:25.720 INFO:tasks.workunit.client.1.smithi033.stdout:8/715: dread - d1/d16/d27/d4d/d82/dd9/d36/faa zero size 2024-05-12T04:01:25.724 INFO:tasks.workunit.client.0.smithi032.stdout:0/624: dwrite df/f51 [0,4194304] 0 2024-05-12T04:01:25.725 INFO:tasks.workunit.client.0.smithi032.stdout:8/597: creat d0/d9/d13/d21/fd3 x:0 0 0 2024-05-12T04:01:25.727 INFO:tasks.workunit.client.0.smithi032.stdout:8/598: chown d0/l23 3 1 2024-05-12T04:01:25.729 INFO:tasks.workunit.client.0.smithi032.stdout:5/684: dwrite d4/d7/d14/d26/d47/f9b [4194304,4194304] 0 2024-05-12T04:01:25.729 INFO:tasks.workunit.client.0.smithi032.stdout:8/599: chown d0/l32 1019763450 1 2024-05-12T04:01:25.732 INFO:tasks.workunit.client.0.smithi032.stdout:8/600: chown d0/d9/d1b/d66/d75/d84 162 1 2024-05-12T04:01:25.732 INFO:tasks.workunit.client.1.smithi033.stdout:9/985: mknod d2/d4/d2b/d33/dd2/c15d 0 2024-05-12T04:01:25.734 INFO:tasks.workunit.client.1.smithi033.stdout:0/793: symlink d6/dc/dd1/dfb/l10a 0 2024-05-12T04:01:25.736 INFO:tasks.workunit.client.0.smithi032.stdout:4/550: creat d0/d3/d12/d1d/d63/f9b x:0 0 0 2024-05-12T04:01:25.740 INFO:tasks.workunit.client.0.smithi032.stdout:2/611: rename d1/d34/d96/d41/d39/c77 to d1/cca 0 2024-05-12T04:01:25.742 INFO:tasks.workunit.client.0.smithi032.stdout:8/601: dwrite d0/d9/d1b/d66/d75/fc1 [0,4194304] 0 2024-05-12T04:01:25.742 INFO:tasks.workunit.client.1.smithi033.stdout:6/787: dwrite d4/d5/f10 [0,4194304] 0 2024-05-12T04:01:25.743 INFO:tasks.workunit.client.0.smithi032.stdout:2/612: dread - d1/d34/da5/db3/db7/fc5 zero size 2024-05-12T04:01:25.744 INFO:tasks.workunit.client.0.smithi032.stdout:3/634: mkdir d15/d20/d8b/d9b/dbb/ddf 0 2024-05-12T04:01:25.748 INFO:tasks.workunit.client.0.smithi032.stdout:4/551: dread d0/d3/d31/d80/d84/f6a [0,4194304] 0 2024-05-12T04:01:25.753 INFO:tasks.workunit.client.1.smithi033.stdout:1/667: write d11/d17/f20 [643546,19258] 0 2024-05-12T04:01:25.755 INFO:tasks.workunit.client.1.smithi033.stdout:1/668: chown d11/d17/d27/d36/d3f/d69/d87/da4/fd9 1941553 1 2024-05-12T04:01:25.755 INFO:tasks.workunit.client.0.smithi032.stdout:7/582: rename dd/d7b/d9a/d9c/db9/d75/d8f to dd/d1b/d1e/d45/d55/d72/da0/dbd 0 2024-05-12T04:01:25.755 INFO:tasks.workunit.client.1.smithi033.stdout:1/669: stat d11/d17/d27/d36/d3f/fd0 0 2024-05-12T04:01:25.756 INFO:tasks.workunit.client.1.smithi033.stdout:9/986: mkdir d2/d4/d2b/d49/d73/d15e 0 2024-05-12T04:01:25.759 INFO:tasks.workunit.client.0.smithi032.stdout:3/635: mkdir d15/d20/d22/d4f/de0 0 2024-05-12T04:01:25.760 INFO:tasks.workunit.client.0.smithi032.stdout:2/613: rmdir d1/d34/d96/d41/d58 39 2024-05-12T04:01:25.762 INFO:tasks.workunit.client.1.smithi033.stdout:3/724: creat db/d17/d25/d32/d3c/ff9 x:0 0 0 2024-05-12T04:01:25.763 INFO:tasks.workunit.client.0.smithi032.stdout:2/614: write d1/d34/d3e/d67/fba [419665,17015] 0 2024-05-12T04:01:25.763 INFO:tasks.workunit.client.0.smithi032.stdout:5/685: mknod d4/d7/d14/d26/d33/de2/cf2 0 2024-05-12T04:01:25.763 INFO:tasks.workunit.client.1.smithi033.stdout:5/676: link d3/d94/d54/d57/d70/c82 d3/d12/d83/cde 0 2024-05-12T04:01:25.766 INFO:tasks.workunit.client.0.smithi032.stdout:0/625: rename df/d10/d3d/d6c/l86 to df/d30/d4d/ld2 0 2024-05-12T04:01:25.767 INFO:tasks.workunit.client.0.smithi032.stdout:8/602: getdents d0/d9/d13/d92 0 2024-05-12T04:01:25.769 INFO:tasks.workunit.client.1.smithi033.stdout:4/761: creat dd/d11/d5b/fee x:0 0 0 2024-05-12T04:01:25.769 INFO:tasks.workunit.client.1.smithi033.stdout:6/788: dwrite d4/f82 [0,4194304] 0 2024-05-12T04:01:25.770 INFO:tasks.workunit.client.0.smithi032.stdout:2/615: dwrite d1/d4d/fad [0,4194304] 0 2024-05-12T04:01:25.774 INFO:tasks.workunit.client.0.smithi032.stdout:4/552: mknod d0/d3/c9c 0 2024-05-12T04:01:25.778 INFO:tasks.workunit.client.1.smithi033.stdout:0/794: rename d6/dc/dd1 to d6/d11/dc7/dd9/d10b 0 2024-05-12T04:01:25.788 INFO:tasks.workunit.client.0.smithi032.stdout:3/636: creat d15/d20/d26/d6a/d6e/d7d/fe1 x:0 0 0 2024-05-12T04:01:25.793 INFO:tasks.workunit.client.0.smithi032.stdout:2/616: fdatasync d1/d4d/d5e/f71 0 2024-05-12T04:01:25.793 INFO:tasks.workunit.client.1.smithi033.stdout:4/762: read dd/d11/f21 [3022061,6214] 0 2024-05-12T04:01:25.794 INFO:tasks.workunit.client.1.smithi033.stdout:1/670: truncate d11/d17/f18 187258 0 2024-05-12T04:01:25.796 INFO:tasks.workunit.client.0.smithi032.stdout:7/583: dread dd/d7b/d9a/d9c/db9/f62 [0,4194304] 0 2024-05-12T04:01:25.800 INFO:tasks.workunit.client.0.smithi032.stdout:7/584: dread dd/f36 [0,4194304] 0 2024-05-12T04:01:25.801 INFO:tasks.workunit.client.0.smithi032.stdout:3/637: mkdir d15/d20/d26/de2 0 2024-05-12T04:01:25.801 INFO:tasks.workunit.client.1.smithi033.stdout:9/987: write d2/d4/d29/d148/f86 [1770294,46573] 0 2024-05-12T04:01:25.803 INFO:tasks.workunit.client.1.smithi033.stdout:9/988: write d2/d4/d2b/d6d/d2f/d5f/f97 [1469536,15910] 0 2024-05-12T04:01:25.805 INFO:tasks.workunit.client.0.smithi032.stdout:4/553: dread d0/d3/d31/d80/d84/f7c [0,4194304] 0 2024-05-12T04:01:25.807 INFO:tasks.workunit.client.0.smithi032.stdout:2/617: symlink d1/d13/d70/lcb 0 2024-05-12T04:01:25.808 INFO:tasks.workunit.client.1.smithi033.stdout:8/716: getdents d1/d16/d29/db1 0 2024-05-12T04:01:25.808 INFO:tasks.workunit.client.0.smithi032.stdout:2/618: rename d1/d34/d96 to d1/d34/d96/da4/dcc 22 2024-05-12T04:01:25.812 INFO:tasks.workunit.client.1.smithi033.stdout:5/677: dread d3/d94/d54/d57/d70/da3/dbe/d16/f20 [4194304,4194304] 0 2024-05-12T04:01:25.813 INFO:tasks.workunit.client.0.smithi032.stdout:0/626: dwrite df/d10/d3d/f87 [0,4194304] 0 2024-05-12T04:01:25.814 INFO:tasks.workunit.client.0.smithi032.stdout:8/603: dwrite d0/d9/d13/d21/f2a [0,4194304] 0 2024-05-12T04:01:25.815 INFO:tasks.workunit.client.0.smithi032.stdout:3/638: rmdir d15/d24/d2a/d8c 39 2024-05-12T04:01:25.820 INFO:tasks.workunit.client.0.smithi032.stdout:7/585: dread fa [0,4194304] 0 2024-05-12T04:01:25.826 INFO:tasks.workunit.client.0.smithi032.stdout:4/554: dwrite d0/d3/d12/d8d/f98 [0,4194304] 0 2024-05-12T04:01:25.827 INFO:tasks.workunit.client.1.smithi033.stdout:2/746: sync 2024-05-12T04:01:25.827 INFO:tasks.workunit.client.1.smithi033.stdout:7/957: sync 2024-05-12T04:01:25.829 INFO:tasks.workunit.client.0.smithi032.stdout:7/586: dwrite dd/d7b/fb5 [0,4194304] 0 2024-05-12T04:01:25.830 INFO:tasks.workunit.client.0.smithi032.stdout:7/587: write dd/d1b/d1e/d45/d64/f91 [2633771,38493] 0 2024-05-12T04:01:25.834 INFO:tasks.workunit.client.1.smithi033.stdout:7/958: dwrite de/d2b/f12b [0,4194304] 0 2024-05-12T04:01:25.835 INFO:tasks.workunit.client.0.smithi032.stdout:7/588: dread dd/d1b/d43/d5e/d37/f41 [0,4194304] 0 2024-05-12T04:01:25.887 INFO:tasks.workunit.client.1.smithi033.stdout:0/795: link d6/d11/d98/ffc d6/dc/d96/d9d/d103/f10c 0 2024-05-12T04:01:25.892 INFO:tasks.workunit.client.0.smithi032.stdout:0/627: mkdir df/d10/d3d/d83/d9f/dd3 0 2024-05-12T04:01:25.896 INFO:tasks.workunit.client.1.smithi033.stdout:1/671: write d11/d17/f18 [790461,88103] 0 2024-05-12T04:01:25.898 INFO:tasks.workunit.client.0.smithi032.stdout:3/639: creat d15/d20/d8b/fe3 x:0 0 0 2024-05-12T04:01:25.905 INFO:tasks.workunit.client.1.smithi033.stdout:5/678: creat d3/d94/d54/d98/fdf x:0 0 0 2024-05-12T04:01:25.911 INFO:tasks.workunit.client.0.smithi032.stdout:8/604: dwrite d0/d4b/f80 [0,4194304] 0 2024-05-12T04:01:25.912 INFO:tasks.workunit.client.1.smithi033.stdout:2/747: unlink dc/d13/d1c/f25 0 2024-05-12T04:01:25.913 INFO:tasks.workunit.client.0.smithi032.stdout:7/589: write dd/d1b/d43/d5e/f34 [680821,38518] 0 2024-05-12T04:01:25.914 INFO:tasks.workunit.client.1.smithi033.stdout:8/717: dread d1/d16/d27/d28/f57 [0,4194304] 0 2024-05-12T04:01:25.916 INFO:tasks.workunit.client.0.smithi032.stdout:3/640: truncate d15/f45 561082 0 2024-05-12T04:01:25.916 INFO:tasks.workunit.client.1.smithi033.stdout:7/959: dread de/df/d60/d75/f112 [0,4194304] 0 2024-05-12T04:01:25.920 INFO:tasks.workunit.client.0.smithi032.stdout:3/641: dread d15/f19 [4194304,4194304] 0 2024-05-12T04:01:25.920 INFO:tasks.workunit.client.1.smithi033.stdout:0/796: rename d6/db/d2c/l3a to d6/d11/dc7/dd9/l10d 0 2024-05-12T04:01:25.920 INFO:tasks.workunit.client.1.smithi033.stdout:3/725: link db/d17/d25/f52 db/d17/d3a/d67/ffa 0 2024-05-12T04:01:25.922 INFO:tasks.workunit.client.1.smithi033.stdout:9/989: creat d2/d4/d2b/d6d/db5/d119/f15f x:0 0 0 2024-05-12T04:01:25.923 INFO:tasks.workunit.client.1.smithi033.stdout:7/960: dread de/df/d60/f76 [0,4194304] 0 2024-05-12T04:01:25.927 INFO:tasks.workunit.client.0.smithi032.stdout:8/605: symlink d0/d4b/dad/ld4 0 2024-05-12T04:01:25.930 INFO:tasks.workunit.client.1.smithi033.stdout:6/789: getdents d4/d1a/d3c/dd9 0 2024-05-12T04:01:25.930 INFO:tasks.workunit.client.0.smithi032.stdout:4/555: truncate d0/d3/d12/d1d/d63/f70 4166435 0 2024-05-12T04:01:25.930 INFO:tasks.workunit.client.1.smithi033.stdout:3/726: dwrite db/d17/d3a/fb5 [0,4194304] 0 2024-05-12T04:01:25.932 INFO:tasks.workunit.client.0.smithi032.stdout:7/590: mknod dd/d1b/d1e/d45/d55/d72/cbe 0 2024-05-12T04:01:25.932 INFO:tasks.workunit.client.1.smithi033.stdout:6/790: chown d4/d5/dcd/fe9 1859 1 2024-05-12T04:01:25.935 INFO:tasks.workunit.client.1.smithi033.stdout:8/718: dread d1/d16/d27/d4d/d82/dd9/d36/d3c/da7/f33 [0,4194304] 0 2024-05-12T04:01:25.936 INFO:tasks.workunit.client.0.smithi032.stdout:0/628: write df/d30/d36/f41 [1550480,90410] 0 2024-05-12T04:01:25.938 INFO:tasks.workunit.client.1.smithi033.stdout:5/679: write d3/d94/d54/d57/d73/f7c [265737,92577] 0 2024-05-12T04:01:25.938 INFO:tasks.workunit.client.0.smithi032.stdout:0/629: chown df/d30/d32/f34 62 1 2024-05-12T04:01:25.939 INFO:tasks.workunit.client.1.smithi033.stdout:8/719: write d1/d4/f39 [2086857,58715] 0 2024-05-12T04:01:25.939 INFO:tasks.workunit.client.1.smithi033.stdout:4/763: getdents dd/d7a/d87/d7e 0 2024-05-12T04:01:25.940 INFO:tasks.workunit.client.0.smithi032.stdout:0/630: chown df/d10/d3d/db0/cc8 60458049 1 2024-05-12T04:01:25.941 INFO:tasks.workunit.client.1.smithi033.stdout:3/727: dwrite db/d17/d3e/d65/d7d/f94 [4194304,4194304] 0 2024-05-12T04:01:25.942 INFO:tasks.workunit.client.1.smithi033.stdout:3/728: chown db/d17/d3a/d93/dc8 96161 1 2024-05-12T04:01:25.944 INFO:tasks.workunit.client.1.smithi033.stdout:3/729: dread - db/d42/fe2 zero size 2024-05-12T04:01:25.944 INFO:tasks.workunit.client.0.smithi032.stdout:7/591: creat dd/d7b/fbf x:0 0 0 2024-05-12T04:01:25.945 INFO:tasks.workunit.client.0.smithi032.stdout:3/642: mkdir d15/d24/d2a/d8c/db1/dc8/de4 0 2024-05-12T04:01:25.954 INFO:tasks.workunit.client.1.smithi033.stdout:2/748: mknod dc/cef 0 2024-05-12T04:01:25.955 INFO:tasks.workunit.client.0.smithi032.stdout:4/556: mknod d0/d3/d12/d1d/d63/d87/d95/c9d 0 2024-05-12T04:01:25.955 INFO:tasks.workunit.client.1.smithi033.stdout:9/990: write d2/d4/d2b/d33/dd2/d132/f15c [197189,96751] 0 2024-05-12T04:01:25.955 INFO:tasks.workunit.client.1.smithi033.stdout:6/791: fdatasync d4/d1a/d6c/f85 0 2024-05-12T04:01:25.958 INFO:tasks.workunit.client.0.smithi032.stdout:0/631: mkdir df/d30/d4e/d55/dd4 0 2024-05-12T04:01:25.960 INFO:tasks.workunit.client.0.smithi032.stdout:8/606: write d0/d9/d1b/d27/d88/d56/d63/d8f/d9a/d28/f2b [3884640,6606] 0 2024-05-12T04:01:25.960 INFO:tasks.workunit.client.0.smithi032.stdout:7/592: read dd/d1b/d43/d5e/d37/f46 [1584538,5797] 0 2024-05-12T04:01:25.962 INFO:tasks.workunit.client.0.smithi032.stdout:8/607: dread - d0/d9/d1b/d27/d4c/d4e/d7f/da3/fd0 zero size 2024-05-12T04:01:25.963 INFO:tasks.workunit.client.1.smithi033.stdout:1/672: rename d11/d17/d38/c70 to d11/d1c/ddf/d4c/dab/ce4 0 2024-05-12T04:01:25.963 INFO:tasks.workunit.client.1.smithi033.stdout:6/792: dwrite d4/d1a/d67/de5/ffe [0,4194304] 0 2024-05-12T04:01:25.964 INFO:tasks.workunit.client.0.smithi032.stdout:3/643: symlink d15/d20/d26/le5 0 2024-05-12T04:01:25.964 INFO:tasks.workunit.client.0.smithi032.stdout:3/644: chown d15/d33/f46 5 1 2024-05-12T04:01:25.967 INFO:tasks.workunit.client.0.smithi032.stdout:0/632: mkdir df/d10/d3d/db0/dd5 0 2024-05-12T04:01:25.970 INFO:tasks.workunit.client.0.smithi032.stdout:0/633: dread - df/d10/d3d/d4a/d9a/d9d/fb7 zero size 2024-05-12T04:01:25.971 INFO:tasks.workunit.client.1.smithi033.stdout:8/720: mkdir d1/d16/d27/d4d/d82/dd9/d36/d3c/d64/dda 0 2024-05-12T04:01:25.971 INFO:tasks.workunit.client.1.smithi033.stdout:8/721: readlink d1/d4/l8c 0 2024-05-12T04:01:25.972 INFO:tasks.workunit.client.1.smithi033.stdout:3/730: mkdir db/d17/d3e/d65/dfb 0 2024-05-12T04:01:25.973 INFO:tasks.workunit.client.0.smithi032.stdout:8/608: dread - d0/d7/f6c zero size 2024-05-12T04:01:25.980 INFO:tasks.workunit.client.1.smithi033.stdout:9/991: rmdir d2/d4/d2b/d6d/d2f/d5f/d84 39 2024-05-12T04:01:25.981 INFO:tasks.workunit.client.0.smithi032.stdout:8/609: symlink d0/d9/d1b/d27/d88/d56/d63/d8f/d9a/d28/ld5 0 2024-05-12T04:01:25.983 INFO:tasks.workunit.client.0.smithi032.stdout:7/593: dread dd/d1b/d1e/f8d [4194304,4194304] 0 2024-05-12T04:01:25.985 INFO:tasks.workunit.client.0.smithi032.stdout:7/594: read dd/d1b/d43/d5e/d37/f54 [996991,29631] 0 2024-05-12T04:01:25.986 INFO:tasks.workunit.client.1.smithi033.stdout:0/797: rename d6/dc/d96/d9d/df3 to d6/d11/dc7/d10e 0 2024-05-12T04:01:25.990 INFO:tasks.workunit.client.0.smithi032.stdout:8/610: stat d0/d9/d13/d21/d35/l9e 0 2024-05-12T04:01:25.990 INFO:tasks.workunit.client.0.smithi032.stdout:8/611: readlink d0/d7/l96 0 2024-05-12T04:01:25.991 INFO:tasks.workunit.client.0.smithi032.stdout:8/612: chown d0/d9/d13/f29 406 1 2024-05-12T04:01:25.991 INFO:tasks.workunit.client.1.smithi033.stdout:5/680: mknod d3/ce0 0 2024-05-12T04:01:25.992 INFO:tasks.workunit.client.0.smithi032.stdout:8/613: chown d0/d9/d1b/c46 2568129 1 2024-05-12T04:01:25.995 INFO:tasks.workunit.client.1.smithi033.stdout:1/673: creat d11/d17/d27/d36/d3f/d4a/fe5 x:0 0 0 2024-05-12T04:01:25.995 INFO:tasks.workunit.client.1.smithi033.stdout:1/674: chown d11/d17/fd7 28917 1 2024-05-12T04:01:25.996 INFO:tasks.workunit.client.1.smithi033.stdout:1/675: stat d11/d17/d27/d36/d3f/d69 0 2024-05-12T04:01:25.998 INFO:tasks.workunit.client.0.smithi032.stdout:3/645: write d15/d24/d40/f5a [90765,49133] 0 2024-05-12T04:01:25.998 INFO:tasks.workunit.client.0.smithi032.stdout:0/634: write df/d10/d3d/d4a/f65 [1584183,76339] 0 2024-05-12T04:01:26.003 INFO:tasks.workunit.client.1.smithi033.stdout:3/731: rmdir db/d17/d25/d32/d3c/d3d 39 2024-05-12T04:01:26.006 INFO:tasks.workunit.client.1.smithi033.stdout:6/793: write d4/f78 [6677730,38540] 0 2024-05-12T04:01:26.006 INFO:tasks.workunit.client.1.smithi033.stdout:7/961: link de/df/d17/d1e/dc4/df0/l11b de/df/d9a/l13e 0 2024-05-12T04:01:26.008 INFO:tasks.workunit.client.0.smithi032.stdout:3/646: rename d15/d20/d26/d6a/l79 to d15/d20/d8b/le6 0 2024-05-12T04:01:26.009 INFO:tasks.workunit.client.1.smithi033.stdout:2/749: mkdir dc/d13/d1c/d98/df0 0 2024-05-12T04:01:26.012 INFO:tasks.workunit.client.1.smithi033.stdout:2/750: dread - dc/d13/d65/fa8 zero size 2024-05-12T04:01:26.012 INFO:tasks.workunit.client.1.smithi033.stdout:2/751: truncate dc/d17/d27/d2f/f6a 5071001 0 2024-05-12T04:01:26.015 INFO:tasks.workunit.client.0.smithi032.stdout:7/595: dwrite dd/d7b/d9a/d9c/db9/f6b [0,4194304] 0 2024-05-12T04:01:26.015 INFO:tasks.workunit.client.0.smithi032.stdout:0/635: mkdir df/d10/d3b/dba/dd6 0 2024-05-12T04:01:26.016 INFO:tasks.workunit.client.1.smithi033.stdout:0/798: creat d6/dc/d58/d6a/d94/f10f x:0 0 0 2024-05-12T04:01:26.018 INFO:tasks.workunit.client.0.smithi032.stdout:8/614: dwrite d0/d9/d1b/d33/f3a [4194304,4194304] 0 2024-05-12T04:01:26.021 INFO:tasks.workunit.client.1.smithi033.stdout:9/992: write d2/d4/d2b/d6d/d2f/d5f/fd5 [968570,23991] 0 2024-05-12T04:01:26.023 INFO:tasks.workunit.client.1.smithi033.stdout:5/681: dread - d3/d94/d54/d57/d70/da3/dbe/d16/dd7/fb5 zero size 2024-05-12T04:01:26.023 INFO:tasks.workunit.client.0.smithi032.stdout:3/647: write d15/f19 [1570513,111522] 0 2024-05-12T04:01:26.027 INFO:tasks.workunit.client.0.smithi032.stdout:7/596: readlink dd/d7b/d9a/d9c/db9/d9f/lb0 0 2024-05-12T04:01:26.027 INFO:tasks.workunit.client.0.smithi032.stdout:8/615: unlink d0/d9/d1b/d66/d75/fc1 0 2024-05-12T04:01:26.032 INFO:tasks.workunit.client.0.smithi032.stdout:3/648: dwrite d15/d24/f3e [0,4194304] 0 2024-05-12T04:01:26.036 INFO:tasks.workunit.client.0.smithi032.stdout:7/597: unlink dd/f36 0 2024-05-12T04:01:26.038 INFO:tasks.workunit.client.0.smithi032.stdout:8/616: creat d0/d9/d1b/d27/d4c/d4e/d7f/da3/fd6 x:0 0 0 2024-05-12T04:01:26.039 INFO:tasks.workunit.client.1.smithi033.stdout:7/962: fsync de/df/d17/d1b/d110/f118 0 2024-05-12T04:01:26.041 INFO:tasks.workunit.client.0.smithi032.stdout:3/649: rmdir d15/d20/d26 39 2024-05-12T04:01:26.042 INFO:tasks.workunit.client.1.smithi033.stdout:2/752: chown dc/c40 34412 1 2024-05-12T04:01:26.045 INFO:tasks.workunit.client.1.smithi033.stdout:3/732: write db/f54 [2796563,40239] 0 2024-05-12T04:01:26.046 INFO:tasks.workunit.client.1.smithi033.stdout:3/733: dread - db/d17/d3a/dc0/fdf zero size 2024-05-12T04:01:26.049 INFO:tasks.workunit.client.1.smithi033.stdout:9/993: symlink d2/d4/d2b/d6d/d7a/l160 0 2024-05-12T04:01:26.051 INFO:tasks.workunit.client.1.smithi033.stdout:9/994: chown d2/d4/d2b/d49/fb9 192119 1 2024-05-12T04:01:26.051 INFO:tasks.workunit.client.0.smithi032.stdout:3/650: creat d15/d20/d22/d4f/de0/fe7 x:0 0 0 2024-05-12T04:01:26.052 INFO:tasks.workunit.client.1.smithi033.stdout:9/995: write d2/d4/d2b/d6d/d2f/d5f/f97 [2640391,116351] 0 2024-05-12T04:01:26.053 INFO:tasks.workunit.client.1.smithi033.stdout:4/764: getdents dd/d69 0 2024-05-12T04:01:26.055 INFO:tasks.workunit.client.1.smithi033.stdout:6/794: truncate d4/d43/f60 1364038 0 2024-05-12T04:01:26.055 INFO:tasks.workunit.client.1.smithi033.stdout:6/795: chown d4/d6e/dbb 20050 1 2024-05-12T04:01:26.058 INFO:tasks.workunit.client.1.smithi033.stdout:7/963: rename de/df/d7a to de/df/d65/de3/d13f 0 2024-05-12T04:01:26.059 INFO:tasks.workunit.client.1.smithi033.stdout:2/753: truncate dc/d13/f15 4514819 0 2024-05-12T04:01:26.061 INFO:tasks.workunit.client.1.smithi033.stdout:0/799: mkdir d6/db/d2c/d73/daf/dc3/d110 0 2024-05-12T04:01:26.062 INFO:tasks.workunit.client.0.smithi032.stdout:8/617: write d0/d9/d1b/f98 [982310,1673] 0 2024-05-12T04:01:26.063 INFO:tasks.workunit.client.1.smithi033.stdout:5/682: write d3/d12/f80 [504240,14631] 0 2024-05-12T04:01:26.065 INFO:tasks.workunit.client.1.smithi033.stdout:8/722: getdents d1/d16/d27/d4d/d82/dd9/d36/d3c/d47 0 2024-05-12T04:01:26.067 INFO:tasks.workunit.client.0.smithi032.stdout:7/598: dwrite dd/d1b/d1e/f28 [0,4194304] 0 2024-05-12T04:01:26.068 INFO:tasks.workunit.client.1.smithi033.stdout:7/964: dwrite de/d2f/fe9 [0,4194304] 0 2024-05-12T04:01:26.070 INFO:tasks.workunit.client.0.smithi032.stdout:7/599: chown dd/d1b/d43/d74/f84 24 1 2024-05-12T04:01:26.070 INFO:tasks.workunit.client.1.smithi033.stdout:4/765: dread dd/f13 [0,4194304] 0 2024-05-12T04:01:26.071 INFO:tasks.workunit.client.1.smithi033.stdout:1/676: getdents d11/d1c/db8 0 2024-05-12T04:01:26.074 INFO:tasks.workunit.client.1.smithi033.stdout:9/996: creat d2/d4/d2b/d49/d104/d145/f161 x:0 0 0 2024-05-12T04:01:26.075 INFO:tasks.workunit.client.0.smithi032.stdout:8/618: dwrite d0/d9/d1b/d27/d88/d56/d63/d8f/d9a/d28/d3e/fb3 [0,4194304] 0 2024-05-12T04:01:26.076 INFO:tasks.workunit.client.1.smithi033.stdout:2/754: dwrite dc/d17/d27/d68/dd1/fd8 [0,4194304] 0 2024-05-12T04:01:26.078 INFO:tasks.workunit.client.0.smithi032.stdout:6/548: sync 2024-05-12T04:01:26.078 INFO:tasks.workunit.client.0.smithi032.stdout:1/590: sync 2024-05-12T04:01:26.079 INFO:tasks.workunit.client.1.smithi033.stdout:9/997: chown d2/d4/d2b/d6d/d7a/fed 0 1 2024-05-12T04:01:26.079 INFO:tasks.workunit.client.0.smithi032.stdout:5/686: sync 2024-05-12T04:01:26.079 INFO:tasks.workunit.client.0.smithi032.stdout:9/509: sync 2024-05-12T04:01:26.084 INFO:tasks.workunit.client.0.smithi032.stdout:8/619: mkdir d0/d9/d1b/d27/d88/dd7 0 2024-05-12T04:01:26.088 INFO:tasks.workunit.client.0.smithi032.stdout:9/510: symlink d0/d3/d6/d58/la9 0 2024-05-12T04:01:26.088 INFO:tasks.workunit.client.1.smithi033.stdout:6/796: write d4/d1a/d3c/f65 [3556150,106153] 0 2024-05-12T04:01:26.089 INFO:tasks.workunit.client.0.smithi032.stdout:5/687: write d4/d7/d14/d26/f57 [3785881,17800] 0 2024-05-12T04:01:26.091 INFO:tasks.workunit.client.1.smithi033.stdout:0/800: mknod d6/dc/df1/c111 0 2024-05-12T04:01:26.095 INFO:tasks.workunit.client.0.smithi032.stdout:1/591: creat d1/d7/d30/d35/d54/d7e/fbd x:0 0 0 2024-05-12T04:01:26.095 INFO:tasks.workunit.client.1.smithi033.stdout:1/677: read - d11/fb5 zero size 2024-05-12T04:01:26.096 INFO:tasks.workunit.client.0.smithi032.stdout:8/620: creat d0/d97/fd8 x:0 0 0 2024-05-12T04:01:26.098 INFO:tasks.workunit.client.1.smithi033.stdout:4/766: unlink dd/d7a/d87/d7e/l9d 0 2024-05-12T04:01:26.099 INFO:tasks.workunit.client.0.smithi032.stdout:9/511: mkdir d0/d3/d6/d7/d27/daa 0 2024-05-12T04:01:26.100 INFO:tasks.workunit.client.0.smithi032.stdout:9/512: write d0/d3/d74/f20 [138016,109595] 0 2024-05-12T04:01:26.101 INFO:tasks.workunit.client.1.smithi033.stdout:2/755: mkdir dc/d17/dcc/df1 0 2024-05-12T04:01:26.101 INFO:tasks.workunit.client.1.smithi033.stdout:2/756: chown dc/d17/d27/d2f/d49/fbb 97 1 2024-05-12T04:01:26.102 INFO:tasks.workunit.client.0.smithi032.stdout:5/688: chown d4/d7/c10 10661485 1 2024-05-12T04:01:26.103 INFO:tasks.workunit.client.1.smithi033.stdout:9/998: stat d2/d4/d2b/d6d/d2f/d5f/d84/d137/f158 0 2024-05-12T04:01:26.103 INFO:tasks.workunit.client.1.smithi033.stdout:2/757: readlink dc/d17/d32/d84/dde/d9e/lce 0 2024-05-12T04:01:26.103 INFO:tasks.workunit.client.1.smithi033.stdout:2/758: chown dc 0 1 2024-05-12T04:01:26.106 INFO:tasks.workunit.client.1.smithi033.stdout:5/683: truncate d3/d94/d54/d57/d70/da3/dbe/d16/d45/d4b/f56 121048 0 2024-05-12T04:01:26.107 INFO:tasks.workunit.client.0.smithi032.stdout:8/621: creat d0/d9/d1b/d27/d88/d56/d63/d8f/fd9 x:0 0 0 2024-05-12T04:01:26.108 INFO:tasks.workunit.client.1.smithi033.stdout:0/801: creat d6/dc/d58/d6a/f112 x:0 0 0 2024-05-12T04:01:26.110 INFO:tasks.workunit.client.1.smithi033.stdout:3/734: link db/d17/d25/d32/c61 db/d59/dcc/de9/cfc 0 2024-05-12T04:01:26.112 INFO:tasks.workunit.client.1.smithi033.stdout:3/735: write db/f54 [4933944,89433] 0 2024-05-12T04:01:26.112 INFO:tasks.workunit.client.0.smithi032.stdout:6/549: rename d7/d9/d48/d67/f16 to d7/d9/d48/fbe 0 2024-05-12T04:01:26.114 INFO:tasks.workunit.client.0.smithi032.stdout:5/689: mkdir d4/d7/d14/df3 0 2024-05-12T04:01:26.115 INFO:tasks.workunit.client.0.smithi032.stdout:9/513: dwrite d0/d3/d6/d7/da/f53 [0,4194304] 0 2024-05-12T04:01:26.116 INFO:tasks.workunit.client.1.smithi033.stdout:0/802: dwrite d6/fe9 [0,4194304] 0 2024-05-12T04:01:26.116 INFO:tasks.workunit.client.1.smithi033.stdout:4/767: creat dd/d11/d75/dd4/fef x:0 0 0 2024-05-12T04:01:26.116 INFO:tasks.workunit.client.1.smithi033.stdout:8/723: write d1/d16/d27/d4d/d73/f97 [76080,48908] 0 2024-05-12T04:01:26.117 INFO:tasks.workunit.client.0.smithi032.stdout:7/600: link dd/d1b/d1e/d45/d64/cba dd/d1b/d43/cc0 0 2024-05-12T04:01:26.118 INFO:tasks.workunit.client.0.smithi032.stdout:9/514: chown d0/d3/d74/f6c 38 1 2024-05-12T04:01:26.119 INFO:tasks.workunit.client.0.smithi032.stdout:9/515: chown d0/d3/d4b/d73/l90 165402380 1 2024-05-12T04:01:26.119 INFO:tasks.workunit.client.1.smithi033.stdout:9/999: mknod d2/d4/c162 0 2024-05-12T04:01:26.122 INFO:tasks.workunit.client.1.smithi033.stdout:5/684: readlink d3/d94/d54/d57/d70/da3/dbe/d16/l4c 0 2024-05-12T04:01:26.124 INFO:tasks.workunit.client.1.smithi033.stdout:0/803: dread d6/fe9 [0,4194304] 0 2024-05-12T04:01:26.125 INFO:tasks.workunit.client.1.smithi033.stdout:8/724: dread d1/d16/d27/d4d/d82/fc2 [0,4194304] 0 2024-05-12T04:01:26.126 INFO:tasks.workunit.client.1.smithi033.stdout:8/725: chown d1/d16/d29/f9a 300 1 2024-05-12T04:01:26.127 INFO:tasks.workunit.client.0.smithi032.stdout:1/592: dread d1/d25/f64 [0,4194304] 0 2024-05-12T04:01:26.127 INFO:tasks.workunit.client.0.smithi032.stdout:8/622: symlink d0/d9/d1b/d27/d88/d56/d63/lda 0 2024-05-12T04:01:26.129 INFO:tasks.workunit.client.0.smithi032.stdout:8/623: fsync d0/d9/d1b/d33/f45 0 2024-05-12T04:01:26.129 INFO:tasks.workunit.client.1.smithi033.stdout:3/736: unlink db/c1f 0 2024-05-12T04:01:26.130 INFO:tasks.workunit.client.1.smithi033.stdout:3/737: write db/d17/d3a/f63 [660181,23223] 0 2024-05-12T04:01:26.135 INFO:tasks.workunit.client.0.smithi032.stdout:1/593: read d1/d7/d30/d18/f1a [3003838,22078] 0 2024-05-12T04:01:26.136 INFO:tasks.workunit.client.1.smithi033.stdout:7/965: getdents de/d80 0 2024-05-12T04:01:26.137 INFO:tasks.workunit.client.0.smithi032.stdout:6/550: creat d7/d9/d48/fbf x:0 0 0 2024-05-12T04:01:26.137 INFO:tasks.workunit.client.0.smithi032.stdout:6/551: stat d7/de/l75 0 2024-05-12T04:01:26.138 INFO:tasks.workunit.client.1.smithi033.stdout:5/685: creat d3/d94/fe1 x:0 0 0 2024-05-12T04:01:26.142 INFO:tasks.workunit.client.0.smithi032.stdout:7/601: rmdir dd/dae 0 2024-05-12T04:01:26.145 INFO:tasks.workunit.client.0.smithi032.stdout:1/594: mkdir d1/d7/d30/d35/dbe 0 2024-05-12T04:01:26.150 INFO:tasks.workunit.client.1.smithi033.stdout:8/726: dread d1/d4/db2/fbc [0,4194304] 0 2024-05-12T04:01:26.151 INFO:tasks.workunit.client.1.smithi033.stdout:1/678: rename d11/d17/d84/lb1 to d11/d1c/ddf/le6 0 2024-05-12T04:01:26.156 INFO:tasks.workunit.client.0.smithi032.stdout:7/602: dread dd/d1b/d1e/f7e [0,4194304] 0 2024-05-12T04:01:26.156 INFO:tasks.workunit.client.1.smithi033.stdout:3/738: chown db/d17/d25/d32/d3c/d3d/d47/ff2 934217 1 2024-05-12T04:01:26.163 INFO:tasks.workunit.client.1.smithi033.stdout:7/966: rename de/df/d9a/l11c to de/df/d9a/l140 0 2024-05-12T04:01:26.164 INFO:tasks.workunit.client.0.smithi032.stdout:8/624: creat d0/d9/d1b/fdb x:0 0 0 2024-05-12T04:01:26.166 INFO:tasks.workunit.client.1.smithi033.stdout:5/686: mkdir d3/d94/d54/d57/d70/da3/dbe/d16/d45/d62/d7f/d84/de2 0 2024-05-12T04:01:26.166 INFO:tasks.workunit.client.1.smithi033.stdout:5/687: stat d3/f52 0 2024-05-12T04:01:26.167 INFO:tasks.workunit.client.0.smithi032.stdout:8/625: dread - d0/d9/d1b/d27/d88/f62 zero size 2024-05-12T04:01:26.167 INFO:tasks.workunit.client.1.smithi033.stdout:5/688: fdatasync d3/d12/f80 0 2024-05-12T04:01:26.168 INFO:tasks.workunit.client.1.smithi033.stdout:0/804: mkdir d6/db/d2c/d113 0 2024-05-12T04:01:26.169 INFO:tasks.workunit.client.0.smithi032.stdout:5/690: dwrite d4/d7/d14/d26/d3c/f6f [0,4194304] 0 2024-05-12T04:01:26.170 INFO:tasks.workunit.client.0.smithi032.stdout:7/603: mknod dd/d7b/d9a/cc1 0 2024-05-12T04:01:26.172 INFO:tasks.workunit.client.0.smithi032.stdout:7/604: readlink dd/d7b/d9a/d9c/db9/d75/l87 0 2024-05-12T04:01:26.173 INFO:tasks.workunit.client.1.smithi033.stdout:5/689: dwrite d3/d94/dc8/fa8 [0,4194304] 0 2024-05-12T04:01:26.174 INFO:tasks.workunit.client.0.smithi032.stdout:1/595: creat d1/d25/da5/db5/fbf x:0 0 0 2024-05-12T04:01:26.180 INFO:tasks.workunit.client.1.smithi033.stdout:8/727: rename d1/d4/d46/fc8 to d1/d16/d27/d4d/d51/fdb 0 2024-05-12T04:01:26.181 INFO:tasks.workunit.client.0.smithi032.stdout:7/605: dwrite dd/d1b/d43/d5e/f5a [0,4194304] 0 2024-05-12T04:01:26.183 INFO:tasks.workunit.client.1.smithi033.stdout:2/759: dwrite dc/d13/d65/db1/fcd [0,4194304] 0 2024-05-12T04:01:26.183 INFO:tasks.workunit.client.0.smithi032.stdout:8/626: creat d0/d9/d13/d21/d25/db5/fdc x:0 0 0 2024-05-12T04:01:26.184 INFO:tasks.workunit.client.1.smithi033.stdout:3/739: readlink db/d17/l33 0 2024-05-12T04:01:26.184 INFO:tasks.workunit.client.1.smithi033.stdout:2/760: fsync dc/d17/dcc/fea 0 2024-05-12T04:01:26.185 INFO:tasks.workunit.client.0.smithi032.stdout:5/691: fsync d4/d7/d14/f8c 0 2024-05-12T04:01:26.186 INFO:tasks.workunit.client.1.smithi033.stdout:2/761: stat dc/d17/d27/d68/lc9 0 2024-05-12T04:01:26.189 INFO:tasks.workunit.client.0.smithi032.stdout:1/596: creat d1/d25/da5/fc0 x:0 0 0 2024-05-12T04:01:26.192 INFO:tasks.workunit.client.1.smithi033.stdout:0/805: creat d6/dc/d58/d6a/d80/db8/f114 x:0 0 0 2024-05-12T04:01:26.193 INFO:tasks.workunit.client.0.smithi032.stdout:6/552: truncate d7/d9/d48/d67/d25/f3e 4500131 0 2024-05-12T04:01:26.196 INFO:tasks.workunit.client.0.smithi032.stdout:7/606: rename dd/d7b/d9a/d9c/db9/d9f to dd/d7b/d9a/d9c/dc2 0 2024-05-12T04:01:26.197 INFO:tasks.workunit.client.0.smithi032.stdout:7/607: stat dd/d1b/d1e 0 2024-05-12T04:01:26.202 INFO:tasks.workunit.client.1.smithi033.stdout:5/690: rmdir d3/d94/d54/d57/d70/da3/dbe/d16/d45/d62/d7f 39 2024-05-12T04:01:26.203 INFO:tasks.workunit.client.0.smithi032.stdout:1/597: mkdir d1/d7/d30/d18/d93/dc1 0 2024-05-12T04:01:26.205 INFO:tasks.workunit.client.0.smithi032.stdout:8/627: symlink d0/d9/d13/d92/ldd 0 2024-05-12T04:01:26.205 INFO:tasks.workunit.client.1.smithi033.stdout:1/679: dwrite d11/d17/d27/f31 [0,4194304] 0 2024-05-12T04:01:26.206 INFO:tasks.workunit.client.0.smithi032.stdout:8/628: readlink d0/d9/d1b/d27/lc8 0 2024-05-12T04:01:26.206 INFO:tasks.workunit.client.1.smithi033.stdout:3/740: dread - db/d17/d25/d32/fd5 zero size 2024-05-12T04:01:26.207 INFO:tasks.workunit.client.1.smithi033.stdout:3/741: dread - db/d17/d3e/dd7/ff7 zero size 2024-05-12T04:01:26.208 INFO:tasks.workunit.client.0.smithi032.stdout:5/692: link d4/d7/d14/d26/d3c/f6f d4/d7/d14/d26/d33/ff4 0 2024-05-12T04:01:26.210 INFO:tasks.workunit.client.0.smithi032.stdout:8/629: dread d0/d9/f2d [0,4194304] 0 2024-05-12T04:01:26.211 INFO:tasks.workunit.client.0.smithi032.stdout:5/693: chown d4/d7/d14/d26/d3c/fe0 230583280 1 2024-05-12T04:01:26.212 INFO:tasks.workunit.client.1.smithi033.stdout:4/768: fsync dd/d7a/d87/d7e/da7/d66/d42/f82 0 2024-05-12T04:01:26.213 INFO:tasks.workunit.client.1.smithi033.stdout:4/769: dread - dd/d69/dc4/fe2 zero size 2024-05-12T04:01:26.214 INFO:tasks.workunit.client.0.smithi032.stdout:1/598: dwrite d1/d7/d30/d35/d54/f8c [4194304,4194304] 0 2024-05-12T04:01:26.214 INFO:tasks.workunit.client.1.smithi033.stdout:4/770: readlink dd/d7a/d87/d7e/da7/ld2 0 2024-05-12T04:01:26.214 INFO:tasks.workunit.client.1.smithi033.stdout:4/771: chown l7 60902993 1 2024-05-12T04:01:26.216 INFO:tasks.workunit.client.0.smithi032.stdout:1/599: write d1/f47 [597482,45931] 0 2024-05-12T04:01:26.221 INFO:tasks.workunit.client.0.smithi032.stdout:8/630: creat d0/d9/d1b/d27/d4c/d4e/d7f/d91/fde x:0 0 0 2024-05-12T04:01:26.223 INFO:tasks.workunit.client.1.smithi033.stdout:5/691: creat d3/d94/d54/d57/d70/da3/dc9/d81/fe3 x:0 0 0 2024-05-12T04:01:26.225 INFO:tasks.workunit.client.1.smithi033.stdout:1/680: getdents d11/d1c/db8 0 2024-05-12T04:01:26.232 INFO:tasks.workunit.client.1.smithi033.stdout:4/772: dread dd/f2b [0,4194304] 0 2024-05-12T04:01:26.234 INFO:tasks.workunit.client.0.smithi032.stdout:5/694: creat d4/d7/d14/d39/d1b/ff5 x:0 0 0 2024-05-12T04:01:26.235 INFO:tasks.workunit.client.1.smithi033.stdout:8/728: write d1/d4/d46/f7a [94464,119632] 0 2024-05-12T04:01:26.237 INFO:tasks.workunit.client.1.smithi033.stdout:8/729: stat d1/d16/d27/d4d/d82/dd9/d36/dd2 0 2024-05-12T04:01:26.239 INFO:tasks.workunit.client.1.smithi033.stdout:4/773: dwrite dd/d69/dc4/fe2 [0,4194304] 0 2024-05-12T04:01:26.241 INFO:tasks.workunit.client.0.smithi032.stdout:8/631: dread d0/d9/f12 [4194304,4194304] 0 2024-05-12T04:01:26.244 INFO:tasks.workunit.client.0.smithi032.stdout:8/632: fsync d0/f2 0 2024-05-12T04:01:26.248 INFO:tasks.workunit.client.0.smithi032.stdout:5/695: link d4/d7/d14/d26/d47/f9b d4/d7/d14/d39/d2b/ff6 0 2024-05-12T04:01:26.249 INFO:tasks.workunit.client.1.smithi033.stdout:4/774: symlink dd/d69/da4/lf0 0 2024-05-12T04:01:26.250 INFO:tasks.workunit.client.1.smithi033.stdout:2/762: dwrite dc/d13/f1b [0,4194304] 0 2024-05-12T04:01:26.252 INFO:tasks.workunit.client.1.smithi033.stdout:2/763: truncate dc/d3b/d88/fd4 1243 0 2024-05-12T04:01:26.255 INFO:tasks.workunit.client.1.smithi033.stdout:3/742: truncate db/d17/d3a/f63 3848821 0 2024-05-12T04:01:26.259 INFO:tasks.workunit.client.1.smithi033.stdout:1/681: write d11/d17/d27/d36/d3f/d4a/f7a [1737594,109133] 0 2024-05-12T04:01:26.260 INFO:tasks.workunit.client.1.smithi033.stdout:2/764: dwrite dc/d17/d27/d68/dd1/fd8 [0,4194304] 0 2024-05-12T04:01:26.261 INFO:tasks.workunit.client.0.smithi032.stdout:1/600: dwrite d1/d7/d30/d61/f68 [0,4194304] 0 2024-05-12T04:01:26.262 INFO:tasks.workunit.client.1.smithi033.stdout:1/682: write d11/d1c/d43/f4d [1499703,59030] 0 2024-05-12T04:01:26.262 INFO:tasks.workunit.client.0.smithi032.stdout:1/601: stat d1/d6a/d72/da9 0 2024-05-12T04:01:26.263 INFO:tasks.workunit.client.1.smithi033.stdout:4/775: fsync dd/d7a/d87/d7e/da7/d66/d42/f82 0 2024-05-12T04:01:26.263 INFO:tasks.workunit.client.1.smithi033.stdout:2/765: chown dc/d13/d1f/d6e/de8/cae 3 1 2024-05-12T04:01:26.265 INFO:tasks.workunit.client.1.smithi033.stdout:1/683: write d11/d17/d38/f71 [977895,43445] 0 2024-05-12T04:01:26.266 INFO:tasks.workunit.client.0.smithi032.stdout:5/696: mknod d4/d7/d14/d39/cf7 0 2024-05-12T04:01:26.266 INFO:tasks.workunit.client.1.smithi033.stdout:5/692: getdents d3/d94/db8 0 2024-05-12T04:01:26.269 INFO:tasks.workunit.client.0.smithi032.stdout:5/697: truncate d4/d7/d14/d26/d33/d72/d92/fdb 250728 0 2024-05-12T04:01:26.270 INFO:tasks.workunit.client.0.smithi032.stdout:5/698: fsync d4/f20 0 2024-05-12T04:01:26.270 INFO:tasks.workunit.client.1.smithi033.stdout:2/766: read dc/d17/d27/d2f/d49/f51 [208638,73157] 0 2024-05-12T04:01:26.278 INFO:tasks.workunit.client.0.smithi032.stdout:8/633: truncate d0/d9/d1b/d66/d75/f7c 575975 0 2024-05-12T04:01:26.279 INFO:tasks.workunit.client.0.smithi032.stdout:8/634: chown d0/d9/d1b/d27/d4c/f70 4655768 1 2024-05-12T04:01:26.281 INFO:tasks.workunit.client.1.smithi033.stdout:6/797: sync 2024-05-12T04:01:26.282 INFO:tasks.workunit.client.0.smithi032.stdout:2/619: sync 2024-05-12T04:01:26.282 INFO:tasks.workunit.client.1.smithi033.stdout:4/776: dread - dd/d11/d5b/d84/faf zero size 2024-05-12T04:01:26.284 INFO:tasks.workunit.client.1.smithi033.stdout:4/777: truncate dd/d69/da4/fa8 1604012 0 2024-05-12T04:01:26.286 INFO:tasks.workunit.client.1.smithi033.stdout:1/684: mkdir d11/d17/d84/de7 0 2024-05-12T04:01:26.287 INFO:tasks.workunit.client.1.smithi033.stdout:5/693: fdatasync d3/d94/d54/d57/d70/da3/dbe/d16/f20 0 2024-05-12T04:01:26.288 INFO:tasks.workunit.client.0.smithi032.stdout:2/620: dwrite d1/d34/da5/db3/db7/fc5 [0,4194304] 0 2024-05-12T04:01:26.288 INFO:tasks.workunit.client.0.smithi032.stdout:2/621: stat d1/f24 0 2024-05-12T04:01:26.294 INFO:tasks.workunit.client.1.smithi033.stdout:2/767: mkdir dc/d17/d27/d2f/d49/df2 0 2024-05-12T04:01:26.295 INFO:tasks.workunit.client.1.smithi033.stdout:8/730: link d1/d4/db2/fbc d1/d16/d27/d4d/d82/dd9/d36/d3c/fdc 0 2024-05-12T04:01:26.297 INFO:tasks.workunit.client.1.smithi033.stdout:6/798: truncate d4/fcf 453968 0 2024-05-12T04:01:26.298 INFO:tasks.workunit.client.1.smithi033.stdout:6/799: readlink d4/d1a/d4e/d4f/l84 0 2024-05-12T04:01:26.299 INFO:tasks.workunit.client.1.smithi033.stdout:4/778: creat dd/d11/d5b/d84/dd7/ff1 x:0 0 0 2024-05-12T04:01:26.305 INFO:tasks.workunit.client.1.smithi033.stdout:2/768: creat dc/d17/d32/d84/dde/d9e/dd7/ff3 x:0 0 0 2024-05-12T04:01:26.307 INFO:tasks.workunit.client.1.smithi033.stdout:8/731: mknod d1/d16/d27/d4d/d82/dd9/d36/d3c/d9f/cdd 0 2024-05-12T04:01:26.311 INFO:tasks.workunit.client.1.smithi033.stdout:6/800: mknod d4/d1a/d67/d6f/c107 0 2024-05-12T04:01:26.313 INFO:tasks.workunit.client.1.smithi033.stdout:8/732: dwrite d1/d16/d27/f86 [0,4194304] 0 2024-05-12T04:01:26.315 INFO:tasks.workunit.client.1.smithi033.stdout:3/743: link db/d17/d4d/fad db/d17/d3a/d8a/ffd 0 2024-05-12T04:01:26.315 INFO:tasks.workunit.client.1.smithi033.stdout:5/694: dread d3/d12/d25/d26/f31 [0,4194304] 0 2024-05-12T04:01:26.315 INFO:tasks.workunit.client.1.smithi033.stdout:3/744: readlink db/d17/d3e/dd7/lb9 0 2024-05-12T04:01:26.329 INFO:tasks.workunit.client.1.smithi033.stdout:5/695: mknod d3/d94/dc8/dcb/ce4 0 2024-05-12T04:01:26.331 INFO:tasks.workunit.client.1.smithi033.stdout:2/769: creat dc/ff4 x:0 0 0 2024-05-12T04:01:26.331 INFO:tasks.workunit.client.1.smithi033.stdout:1/685: write d11/d17/d27/d36/d3f/f50 [776105,15484] 0 2024-05-12T04:01:26.332 INFO:tasks.workunit.client.1.smithi033.stdout:1/686: dread - d11/d1c/f9d zero size 2024-05-12T04:01:26.333 INFO:tasks.workunit.client.1.smithi033.stdout:3/745: read db/d17/d4d/fad [815030,75499] 0 2024-05-12T04:01:26.337 INFO:tasks.workunit.client.1.smithi033.stdout:5/696: symlink d3/d12/d29/le5 0 2024-05-12T04:01:26.342 INFO:tasks.workunit.client.1.smithi033.stdout:3/746: creat db/d17/d25/d32/d3c/d3d/d47/ffe x:0 0 0 2024-05-12T04:01:26.343 INFO:tasks.workunit.client.1.smithi033.stdout:3/747: fsync db/d17/d3a/d67/da6/fe3 0 2024-05-12T04:01:26.344 INFO:tasks.workunit.client.1.smithi033.stdout:5/697: mknod d3/d94/d85/ce6 0 2024-05-12T04:01:26.346 INFO:tasks.workunit.client.1.smithi033.stdout:2/770: link dc/d3b/d88/fd4 dc/d17/d27/d2f/d49/ff5 0 2024-05-12T04:01:26.348 INFO:tasks.workunit.client.1.smithi033.stdout:1/687: getdents d11/d17/d84/de7 0 2024-05-12T04:01:26.349 INFO:tasks.workunit.client.0.smithi032.stdout:1/602: mknod d1/d7/d30/cc2 0 2024-05-12T04:01:26.351 INFO:tasks.workunit.client.0.smithi032.stdout:5/699: mknod d4/d7/d14/d26/d33/cf8 0 2024-05-12T04:01:26.352 INFO:tasks.workunit.client.0.smithi032.stdout:5/700: chown d4/d7/d14/f8c 3 1 2024-05-12T04:01:26.352 INFO:tasks.workunit.client.1.smithi033.stdout:6/801: write d4/d1a/d4e/f66 [1692955,113296] 0 2024-05-12T04:01:26.354 INFO:tasks.workunit.client.1.smithi033.stdout:2/771: symlink dc/d13/d65/lf6 0 2024-05-12T04:01:26.357 INFO:tasks.workunit.client.1.smithi033.stdout:1/688: creat d11/d17/d27/d36/d3f/d69/fe8 x:0 0 0 2024-05-12T04:01:26.358 INFO:tasks.workunit.client.1.smithi033.stdout:6/802: dread d4/d6e/fb3 [0,4194304] 0 2024-05-12T04:01:26.359 INFO:tasks.workunit.client.0.smithi032.stdout:2/622: creat d1/d34/d3e/fcd x:0 0 0 2024-05-12T04:01:26.359 INFO:tasks.workunit.client.0.smithi032.stdout:2/623: chown d1/d13/c55 217441 1 2024-05-12T04:01:26.362 INFO:tasks.workunit.client.1.smithi033.stdout:3/748: dread db/d59/fab [0,4194304] 0 2024-05-12T04:01:26.365 INFO:tasks.workunit.client.1.smithi033.stdout:2/772: unlink dc/d17/d3a/l4e 0 2024-05-12T04:01:26.366 INFO:tasks.workunit.client.0.smithi032.stdout:8/635: mkdir d0/d9/d1b/d27/d88/dd7/ddf 0 2024-05-12T04:01:26.367 INFO:tasks.workunit.client.1.smithi033.stdout:1/689: readlink d11/d1c/ddf/le6 0 2024-05-12T04:01:26.367 INFO:tasks.workunit.client.0.smithi032.stdout:8/636: read d0/f2 [3787076,80787] 0 2024-05-12T04:01:26.367 INFO:tasks.workunit.client.1.smithi033.stdout:1/690: chown d11/d1c/f32 12744 1 2024-05-12T04:01:26.368 INFO:tasks.workunit.client.0.smithi032.stdout:5/701: mknod d4/d7/d14/d26/d33/d4a/d38/dd2/cf9 0 2024-05-12T04:01:26.370 INFO:tasks.workunit.client.0.smithi032.stdout:5/702: chown d4/d7/d14/f8c 101 1 2024-05-12T04:01:26.371 INFO:tasks.workunit.client.1.smithi033.stdout:5/698: link d3/d12/d29/f3d d3/d94/d54/d57/d70/da3/dbe/d16/d78/fe7 0 2024-05-12T04:01:26.372 INFO:tasks.workunit.client.0.smithi032.stdout:1/603: mkdir d1/d7/d30/d18/d93/dc1/dc3 0 2024-05-12T04:01:26.374 INFO:tasks.workunit.client.0.smithi032.stdout:1/604: fsync d1/d7/f7f 0 2024-05-12T04:01:26.375 INFO:tasks.workunit.client.0.smithi032.stdout:8/637: dwrite d0/d9/f12 [4194304,4194304] 0 2024-05-12T04:01:26.375 INFO:tasks.workunit.client.1.smithi033.stdout:2/773: truncate dc/d13/d1c/d98/fb9 470501 0 2024-05-12T04:01:26.377 INFO:tasks.workunit.client.0.smithi032.stdout:5/703: fdatasync d4/d7/d14/d26/d33/d4a/d38/f45 0 2024-05-12T04:01:26.377 INFO:tasks.workunit.client.1.smithi033.stdout:2/774: dread - dc/d17/d3a/f56 zero size 2024-05-12T04:01:26.378 INFO:tasks.workunit.client.0.smithi032.stdout:5/704: write d4/d7/d14/d39/d1b/f23 [1154935,76936] 0 2024-05-12T04:01:26.379 INFO:tasks.workunit.client.1.smithi033.stdout:1/691: mknod d11/d17/d27/d36/d3f/d69/d87/da4/ce9 0 2024-05-12T04:01:26.379 INFO:tasks.workunit.client.0.smithi032.stdout:5/705: readlink d4/d7/d14/d26/d33/d72/dc9/dec/le5 0 2024-05-12T04:01:26.386 INFO:tasks.workunit.client.1.smithi033.stdout:1/692: dwrite d11/d1c/f64 [0,4194304] 0 2024-05-12T04:01:26.387 INFO:tasks.workunit.client.1.smithi033.stdout:1/693: chown d11/d17/d9e/f9f 1 1 2024-05-12T04:01:26.395 INFO:tasks.workunit.client.1.smithi033.stdout:1/694: dwrite d11/d17/d27/d36/d3f/f50 [0,4194304] 0 2024-05-12T04:01:26.396 INFO:tasks.workunit.client.1.smithi033.stdout:7/967: sync 2024-05-12T04:01:26.398 INFO:tasks.workunit.client.0.smithi032.stdout:5/706: read d4/d7/d14/d26/f5e [50721,45108] 0 2024-05-12T04:01:26.399 INFO:tasks.workunit.client.0.smithi032.stdout:2/624: getdents d1/d13/d70/d85 0 2024-05-12T04:01:26.402 INFO:tasks.workunit.client.1.smithi033.stdout:1/695: dread d11/d17/d27/da6/fc2 [0,4194304] 0 2024-05-12T04:01:26.404 INFO:tasks.workunit.client.1.smithi033.stdout:2/775: mkdir dc/d17/d27/d68/d9f/df7 0 2024-05-12T04:01:26.406 INFO:tasks.workunit.client.1.smithi033.stdout:6/803: truncate d4/d1a/d4e/d4f/d94/fb9 4119629 0 2024-05-12T04:01:26.407 INFO:tasks.workunit.client.1.smithi033.stdout:6/804: chown d4/d1a/d3c/dd9/cf1 18 1 2024-05-12T04:01:26.409 INFO:tasks.workunit.client.1.smithi033.stdout:7/968: creat de/df/d17/d1b/d110/f141 x:0 0 0 2024-05-12T04:01:26.411 INFO:tasks.workunit.client.1.smithi033.stdout:5/699: creat d3/d94/d54/d57/d70/da3/dbe/d16/d45/d62/d7f/d84/de2/fe8 x:0 0 0 2024-05-12T04:01:26.411 INFO:tasks.workunit.client.0.smithi032.stdout:1/605: dread d1/d7/d30/d35/f4e [0,4194304] 0 2024-05-12T04:01:26.413 INFO:tasks.workunit.client.1.smithi033.stdout:1/696: dread - d11/d17/d27/d76/d93/fb0 zero size 2024-05-12T04:01:26.414 INFO:tasks.workunit.client.0.smithi032.stdout:8/638: dwrite d0/d7/f6c [0,4194304] 0 2024-05-12T04:01:26.415 INFO:tasks.workunit.client.1.smithi033.stdout:5/700: dread d3/d94/f3e [0,4194304] 0 2024-05-12T04:01:26.415 INFO:tasks.workunit.client.1.smithi033.stdout:1/697: read d11/d17/f35 [7448420,89242] 0 2024-05-12T04:01:26.420 INFO:tasks.workunit.client.0.smithi032.stdout:5/707: write d4/f9a [818356,6566] 0 2024-05-12T04:01:26.424 INFO:tasks.workunit.client.0.smithi032.stdout:8/639: dread - d0/d9/d1b/d27/d59/fb1 zero size 2024-05-12T04:01:26.424 INFO:tasks.workunit.client.1.smithi033.stdout:6/805: creat d4/d43/d57/f108 x:0 0 0 2024-05-12T04:01:26.426 INFO:tasks.workunit.client.0.smithi032.stdout:2/625: getdents d1/d34/d3e 0 2024-05-12T04:01:26.428 INFO:tasks.workunit.client.0.smithi032.stdout:5/708: mkdir d4/d7/d14/dfa 0 2024-05-12T04:01:26.429 INFO:tasks.workunit.client.1.smithi033.stdout:5/701: rmdir d3/d94/d54/d57/d70/da3 39 2024-05-12T04:01:26.430 INFO:tasks.workunit.client.0.smithi032.stdout:5/709: fsync d4/d7/d14/d26/dc6/fc8 0 2024-05-12T04:01:26.432 INFO:tasks.workunit.client.0.smithi032.stdout:8/640: creat d0/d9/d1b/d33/d73/fe0 x:0 0 0 2024-05-12T04:01:26.434 INFO:tasks.workunit.client.0.smithi032.stdout:5/710: rmdir d4/d7/d14/d26/d33/d72/dc9/dec/d68 39 2024-05-12T04:01:26.435 INFO:tasks.workunit.client.1.smithi033.stdout:2/776: dwrite dc/d3b/d88/fd4 [0,4194304] 0 2024-05-12T04:01:26.437 INFO:tasks.workunit.client.1.smithi033.stdout:7/969: truncate de/df/d17/d1b/d50/f67 365857 0 2024-05-12T04:01:26.442 INFO:tasks.workunit.client.0.smithi032.stdout:1/606: dread d1/d7/d30/d35/fa3 [0,4194304] 0 2024-05-12T04:01:26.443 INFO:tasks.workunit.client.0.smithi032.stdout:5/711: mkdir d4/d7/d14/d26/d33/d88/dfb 0 2024-05-12T04:01:26.444 INFO:tasks.workunit.client.0.smithi032.stdout:5/712: stat d4/d7/d14/d26/d33/d5f/c98 0 2024-05-12T04:01:26.445 INFO:tasks.workunit.client.1.smithi033.stdout:0/806: sync 2024-05-12T04:01:26.446 INFO:tasks.workunit.client.1.smithi033.stdout:1/698: rename d11/d17/d38/c47 to d11/d17/d9e/cea 0 2024-05-12T04:01:26.448 INFO:tasks.workunit.client.0.smithi032.stdout:2/626: creat d1/d34/fce x:0 0 0 2024-05-12T04:01:26.449 INFO:tasks.workunit.client.0.smithi032.stdout:2/627: chown d1/fc 41 1 2024-05-12T04:01:26.450 INFO:tasks.workunit.client.0.smithi032.stdout:2/628: dread - d1/d34/d96/d8a/fbb zero size 2024-05-12T04:01:26.451 INFO:tasks.workunit.client.1.smithi033.stdout:6/806: dwrite d4/d1a/d6c/fa4 [0,4194304] 0 2024-05-12T04:01:26.453 INFO:tasks.workunit.client.0.smithi032.stdout:8/641: creat d0/d9/fe1 x:0 0 0 2024-05-12T04:01:26.454 INFO:tasks.workunit.client.0.smithi032.stdout:2/629: creat d1/d34/da5/db3/db7/fcf x:0 0 0 2024-05-12T04:01:26.456 INFO:tasks.workunit.client.1.smithi033.stdout:7/970: creat de/d20/d38/d7f/f142 x:0 0 0 2024-05-12T04:01:26.457 INFO:tasks.workunit.client.1.smithi033.stdout:5/702: fsync d3/d94/d54/d57/d70/da3/dbe/d16/d78/d8b/fd8 0 2024-05-12T04:01:26.459 INFO:tasks.workunit.client.0.smithi032.stdout:8/642: mkdir d0/d9/d1b/d66/d75/de2 0 2024-05-12T04:01:26.461 INFO:tasks.workunit.client.1.smithi033.stdout:0/807: creat d6/d11/dc7/dd9/d10b/f115 x:0 0 0 2024-05-12T04:01:26.461 INFO:tasks.workunit.client.1.smithi033.stdout:0/808: chown d6/db/d2c/d73 241 1 2024-05-12T04:01:26.462 INFO:tasks.workunit.client.1.smithi033.stdout:0/809: chown d6/dc/d96/d9d 378557 1 2024-05-12T04:01:26.462 INFO:tasks.workunit.client.1.smithi033.stdout:0/810: readlink d6/dc/d39/l68 0 2024-05-12T04:01:26.463 INFO:tasks.workunit.client.1.smithi033.stdout:0/811: chown d6/dc/d6e/c4c 5567 1 2024-05-12T04:01:26.463 INFO:tasks.workunit.client.1.smithi033.stdout:0/812: chown d6/d11/dc7/lde 189902 1 2024-05-12T04:01:26.466 INFO:tasks.workunit.client.0.smithi032.stdout:2/630: creat d1/d34/d96/d41/d52/fd0 x:0 0 0 2024-05-12T04:01:26.468 INFO:tasks.workunit.client.0.smithi032.stdout:2/631: fdatasync d1/d13/f4c 0 2024-05-12T04:01:26.468 INFO:tasks.workunit.client.0.smithi032.stdout:8/643: symlink d0/d9/d1b/d27/d4c/d4e/le3 0 2024-05-12T04:01:26.470 INFO:tasks.workunit.client.0.smithi032.stdout:8/644: chown d0/l44 125379 1 2024-05-12T04:01:26.472 INFO:tasks.workunit.client.1.smithi033.stdout:2/777: symlink dc/d13/d1c/lf8 0 2024-05-12T04:01:26.472 INFO:tasks.workunit.client.1.smithi033.stdout:6/807: truncate d4/d1a/fb2 2316132 0 2024-05-12T04:01:26.472 INFO:tasks.workunit.client.0.smithi032.stdout:5/713: dwrite d4/d7/d14/d26/d33/d72/dc9/dec/f48 [0,4194304] 0 2024-05-12T04:01:26.472 INFO:tasks.workunit.client.1.smithi033.stdout:7/971: mknod de/d2b/d6c/db5/c143 0 2024-05-12T04:01:26.473 INFO:tasks.workunit.client.0.smithi032.stdout:2/632: symlink d1/d34/d4a/d86/ld1 0 2024-05-12T04:01:26.475 INFO:tasks.workunit.client.1.smithi033.stdout:0/813: symlink d6/d11/dc7/d10e/l116 0 2024-05-12T04:01:26.475 INFO:tasks.workunit.client.1.smithi033.stdout:0/814: stat d6/dc/d58/d6a/dec 0 2024-05-12T04:01:26.476 INFO:tasks.workunit.client.0.smithi032.stdout:8/645: rmdir d0/d9/d1b/d27/d88 39 2024-05-12T04:01:26.477 INFO:tasks.workunit.client.1.smithi033.stdout:1/699: write f8 [1818854,1130] 0 2024-05-12T04:01:26.480 INFO:tasks.workunit.client.0.smithi032.stdout:2/633: creat d1/d34/d96/d41/d39/fd2 x:0 0 0 2024-05-12T04:01:26.483 INFO:tasks.workunit.client.0.smithi032.stdout:5/714: symlink d4/d7/d14/d26/lfc 0 2024-05-12T04:01:26.485 INFO:tasks.workunit.client.0.smithi032.stdout:2/634: creat d1/d34/d96/d41/d52/fd3 x:0 0 0 2024-05-12T04:01:26.486 INFO:tasks.workunit.client.1.smithi033.stdout:8/733: sync 2024-05-12T04:01:26.486 INFO:tasks.workunit.client.1.smithi033.stdout:4/779: sync 2024-05-12T04:01:26.486 INFO:tasks.workunit.client.1.smithi033.stdout:3/749: sync 2024-05-12T04:01:26.486 INFO:tasks.workunit.client.1.smithi033.stdout:8/734: read - d1/fa4 zero size 2024-05-12T04:01:26.487 INFO:tasks.workunit.client.1.smithi033.stdout:3/750: stat db/d17/d3e/d65/dfb 0 2024-05-12T04:01:26.489 INFO:tasks.workunit.client.0.smithi032.stdout:5/715: symlink d4/d7/d14/d39/d1b/da4/lfd 0 2024-05-12T04:01:26.492 INFO:tasks.workunit.client.0.smithi032.stdout:2/635: mkdir d1/d34/d96/d41/d2c/dd4 0 2024-05-12T04:01:26.494 INFO:tasks.workunit.client.1.smithi033.stdout:4/780: dwrite dd/d7a/d87/d7e/da7/d66/f6a [0,4194304] 0 2024-05-12T04:01:26.512 INFO:tasks.workunit.client.0.smithi032.stdout:0/636: sync 2024-05-12T04:01:26.513 INFO:tasks.workunit.client.0.smithi032.stdout:4/557: sync 2024-05-12T04:01:26.513 INFO:tasks.workunit.client.0.smithi032.stdout:3/651: sync 2024-05-12T04:01:26.513 INFO:tasks.workunit.client.0.smithi032.stdout:5/716: creat d4/d7/d14/d26/d33/d88/ffe x:0 0 0 2024-05-12T04:01:26.515 INFO:tasks.workunit.client.0.smithi032.stdout:8/646: write d0/d9/d1b/d27/d88/d56/fae [550457,92614] 0 2024-05-12T04:01:26.516 INFO:tasks.workunit.client.0.smithi032.stdout:2/636: mkdir d1/d13/d70/d85/dd5 0 2024-05-12T04:01:26.518 INFO:tasks.workunit.client.0.smithi032.stdout:8/647: chown d0/d9/d1b/d33/f3a 0 1 2024-05-12T04:01:26.519 INFO:tasks.workunit.client.0.smithi032.stdout:0/637: fdatasync df/d30/d32/d60/f90 0 2024-05-12T04:01:26.520 INFO:tasks.workunit.client.0.smithi032.stdout:8/648: dread - d0/d9/d1b/d27/d4c/d4e/d7f/d91/fde zero size 2024-05-12T04:01:26.521 INFO:tasks.workunit.client.0.smithi032.stdout:5/717: creat d4/d7/d14/d26/d3c/fff x:0 0 0 2024-05-12T04:01:26.522 INFO:tasks.workunit.client.0.smithi032.stdout:0/638: dread df/d30/d4e/d55/fa7 [0,4194304] 0 2024-05-12T04:01:26.523 INFO:tasks.workunit.client.0.smithi032.stdout:0/639: write df/f51 [4402853,125259] 0 2024-05-12T04:01:26.526 INFO:tasks.workunit.client.0.smithi032.stdout:0/640: dread - df/d10/f64 zero size 2024-05-12T04:01:26.526 INFO:tasks.workunit.client.0.smithi032.stdout:2/637: fsync d1/d34/d96/d41/d2c/f99 0 2024-05-12T04:01:26.527 INFO:tasks.workunit.client.1.smithi033.stdout:6/808: readlink d4/d1a/d4e/d4f/d94/l97 0 2024-05-12T04:01:26.527 INFO:tasks.workunit.client.0.smithi032.stdout:0/641: read - df/d10/d3b/d75/fce zero size 2024-05-12T04:01:26.531 INFO:tasks.workunit.client.0.smithi032.stdout:3/652: dread fd [0,4194304] 0 2024-05-12T04:01:26.535 INFO:tasks.workunit.client.0.smithi032.stdout:4/558: creat d0/d3/d12/d1d/f9e x:0 0 0 2024-05-12T04:01:26.538 INFO:tasks.workunit.client.0.smithi032.stdout:5/718: rename d4/d7/d14/d26/d33/d4a/d38/cce to d4/d7/d14/d39/d1b/da4/c100 0 2024-05-12T04:01:26.539 INFO:tasks.workunit.client.1.smithi033.stdout:3/751: mknod db/d17/d3e/d65/d91/cff 0 2024-05-12T04:01:26.539 INFO:tasks.workunit.client.0.smithi032.stdout:5/719: read d4/d7/d14/d26/f57 [2306064,23467] 0 2024-05-12T04:01:26.540 INFO:tasks.workunit.client.0.smithi032.stdout:5/720: readlink d4/d7/d14/d39/l1f 0 2024-05-12T04:01:26.541 INFO:tasks.workunit.client.1.smithi033.stdout:8/735: creat d1/d16/d27/d4d/d82/dd9/d36/d3c/d47/fde x:0 0 0 2024-05-12T04:01:26.545 INFO:tasks.workunit.client.0.smithi032.stdout:8/649: mknod d0/d9/ce4 0 2024-05-12T04:01:26.547 INFO:tasks.workunit.client.0.smithi032.stdout:8/650: write d0/d9/d1b/d27/d4c/d4e/d7f/da3/fd6 [478784,22082] 0 2024-05-12T04:01:26.548 INFO:tasks.workunit.client.1.smithi033.stdout:8/736: dwrite d1/d4/d46/f7a [0,4194304] 0 2024-05-12T04:01:26.549 INFO:tasks.workunit.client.1.smithi033.stdout:8/737: fdatasync d1/d16/d27/d4d/d73/f97 0 2024-05-12T04:01:26.683 INFO:tasks.workunit.client.0.smithi032.stdout:4/559: creat d0/d3/df/d4e/f9f x:0 0 0 2024-05-12T04:01:26.683 INFO:tasks.workunit.client.0.smithi032.stdout:4/560: dread - d0/d3/d12/d1d/d6b/f9a zero size 2024-05-12T04:01:26.684 INFO:tasks.workunit.client.1.smithi033.stdout:2/778: rename dc/f31 to dc/d17/d27/d2f/d49/ff9 0 2024-05-12T04:01:26.726 INFO:tasks.workunit.client.0.smithi032.stdout:9/516: sync 2024-05-12T04:01:26.726 INFO:tasks.workunit.client.0.smithi032.stdout:6/553: sync 2024-05-12T04:01:26.726 INFO:tasks.workunit.client.0.smithi032.stdout:7/608: sync 2024-05-12T04:01:26.728 INFO:tasks.workunit.client.1.smithi033.stdout:7/972: truncate de/df/d17/d1b/d50/f67 31744 0 2024-05-12T04:01:26.729 INFO:tasks.workunit.client.0.smithi032.stdout:0/642: creat df/d30/d32/d60/dc7/fd7 x:0 0 0 2024-05-12T04:01:26.732 INFO:tasks.workunit.client.1.smithi033.stdout:1/700: creat d11/d17/d27/d36/d3f/d4a/dc6/de1/feb x:0 0 0 2024-05-12T04:01:26.734 INFO:tasks.workunit.client.0.smithi032.stdout:6/554: dwrite d7/d9/d48/d67/fb0 [0,4194304] 0 2024-05-12T04:01:26.735 INFO:tasks.workunit.client.0.smithi032.stdout:6/555: readlink d7/d9/d48/l6a 0 2024-05-12T04:01:26.736 INFO:tasks.workunit.client.0.smithi032.stdout:8/651: rename d0/d9/d13/d21/f2a to d0/d9/d1b/d27/d88/d56/fe5 0 2024-05-12T04:01:26.737 INFO:tasks.workunit.client.0.smithi032.stdout:8/652: fsync d0/d7/f11 0 2024-05-12T04:01:26.738 INFO:tasks.workunit.client.1.smithi033.stdout:3/752: mkdir db/d17/d3a/d93/dc8/d100 0 2024-05-12T04:01:26.739 INFO:tasks.workunit.client.0.smithi032.stdout:4/561: mknod d0/d3/d24/ca0 0 2024-05-12T04:01:26.740 INFO:tasks.workunit.client.0.smithi032.stdout:8/653: truncate d0/d9/d1b/d27/d4c/d4e/d7f/d91/fde 35702 0 2024-05-12T04:01:26.743 INFO:tasks.workunit.client.0.smithi032.stdout:9/517: mknod d0/d3/d4b/d85/cab 0 2024-05-12T04:01:26.745 INFO:tasks.workunit.client.1.smithi033.stdout:5/703: getdents d3/d12/dac 0 2024-05-12T04:01:26.746 INFO:tasks.workunit.client.0.smithi032.stdout:7/609: readlink dd/d1b/d1e/d45/d55/d72/la6 0 2024-05-12T04:01:26.747 INFO:tasks.workunit.client.1.smithi033.stdout:3/753: dwrite db/d17/d25/d32/d53/f87 [0,4194304] 0 2024-05-12T04:01:26.749 INFO:tasks.workunit.client.0.smithi032.stdout:3/653: creat d15/d33/fe8 x:0 0 0 2024-05-12T04:01:26.753 INFO:tasks.workunit.client.1.smithi033.stdout:0/815: dwrite d6/db/fb0 [0,4194304] 0 2024-05-12T04:01:26.754 INFO:tasks.workunit.client.0.smithi032.stdout:7/610: dwrite dd/d7b/d9a/d9c/db9/f6b [0,4194304] 0 2024-05-12T04:01:26.754 INFO:tasks.workunit.client.1.smithi033.stdout:2/779: mknod dc/d17/d27/d2f/cfa 0 2024-05-12T04:01:26.757 INFO:tasks.workunit.client.1.smithi033.stdout:6/809: write d4/d6e/fb3 [39967,75376] 0 2024-05-12T04:01:26.758 INFO:tasks.workunit.client.0.smithi032.stdout:6/556: mknod d7/d9/d48/d67/d1a/d20/cc0 0 2024-05-12T04:01:26.761 INFO:tasks.workunit.client.1.smithi033.stdout:4/781: dwrite dd/d11/f21 [0,4194304] 0 2024-05-12T04:01:26.767 INFO:tasks.workunit.client.0.smithi032.stdout:8/654: write d0/d9/d1b/d27/d88/d56/fe5 [103426,11846] 0 2024-05-12T04:01:26.768 INFO:tasks.workunit.client.0.smithi032.stdout:1/607: sync 2024-05-12T04:01:26.770 INFO:tasks.workunit.client.1.smithi033.stdout:8/738: mkdir d1/d16/d27/ddf 0 2024-05-12T04:01:26.772 INFO:tasks.workunit.client.1.smithi033.stdout:4/782: write dd/d7a/d87/d7e/da7/d66/d42/fde [1038916,70825] 0 2024-05-12T04:01:26.772 INFO:tasks.workunit.client.0.smithi032.stdout:9/518: mknod d0/d3/d6/d7/da/d89/cac 0 2024-05-12T04:01:26.774 INFO:tasks.workunit.client.1.smithi033.stdout:3/754: mknod db/d17/d3a/d8a/c101 0 2024-05-12T04:01:26.778 INFO:tasks.workunit.client.1.smithi033.stdout:0/816: creat d6/dc/df1/f117 x:0 0 0 2024-05-12T04:01:26.781 INFO:tasks.workunit.client.1.smithi033.stdout:4/783: dwrite dd/d69/f97 [0,4194304] 0 2024-05-12T04:01:26.782 INFO:tasks.workunit.client.1.smithi033.stdout:4/784: chown dd/f2a 11847 1 2024-05-12T04:01:26.788 INFO:tasks.workunit.client.1.smithi033.stdout:8/739: creat d1/d16/d29/d63/da3/dba/fe0 x:0 0 0 2024-05-12T04:01:26.790 INFO:tasks.workunit.client.0.smithi032.stdout:6/557: chown d7/d9/d5c/d8b/da1 42 1 2024-05-12T04:01:26.792 INFO:tasks.workunit.client.1.smithi033.stdout:5/704: truncate d3/d94/d54/d57/d70/da3/dbe/f30 1420322 0 2024-05-12T04:01:26.796 INFO:tasks.workunit.client.1.smithi033.stdout:0/817: rmdir d6/dc/d6e 39 2024-05-12T04:01:26.799 INFO:tasks.workunit.client.0.smithi032.stdout:1/608: fsync d1/dd/f53 0 2024-05-12T04:01:26.800 INFO:tasks.workunit.client.0.smithi032.stdout:1/609: stat d1/d25/f4d 0 2024-05-12T04:01:26.800 INFO:tasks.workunit.client.1.smithi033.stdout:4/785: unlink l7 0 2024-05-12T04:01:26.803 INFO:tasks.workunit.client.1.smithi033.stdout:7/973: dread de/d2f/d70/fa3 [0,4194304] 0 2024-05-12T04:01:26.805 INFO:tasks.workunit.client.1.smithi033.stdout:7/974: dread - de/df/d9a/f133 zero size 2024-05-12T04:01:26.805 INFO:tasks.workunit.client.1.smithi033.stdout:8/740: mkdir d1/d4/d91/de1 0 2024-05-12T04:01:26.807 INFO:tasks.workunit.client.1.smithi033.stdout:3/755: mknod db/d17/d3e/d65/d88/c102 0 2024-05-12T04:01:26.808 INFO:tasks.workunit.client.0.smithi032.stdout:4/562: dwrite d0/d3/f49 [0,4194304] 0 2024-05-12T04:01:26.812 INFO:tasks.workunit.client.0.smithi032.stdout:0/643: truncate df/d10/d48/f96 3341051 0 2024-05-12T04:01:26.815 INFO:tasks.workunit.client.0.smithi032.stdout:9/519: dread d0/f28 [0,4194304] 0 2024-05-12T04:01:26.816 INFO:tasks.workunit.client.0.smithi032.stdout:7/611: dwrite dd/d1b/d43/d5e/f1c [0,4194304] 0 2024-05-12T04:01:26.817 INFO:tasks.workunit.client.0.smithi032.stdout:9/520: stat d0/d3/c5 0 2024-05-12T04:01:26.818 INFO:tasks.workunit.client.1.smithi033.stdout:2/780: dread dc/d13/d1c/f92 [0,4194304] 0 2024-05-12T04:01:26.820 INFO:tasks.workunit.client.1.smithi033.stdout:6/810: dwrite d4/d5/fa1 [0,4194304] 0 2024-05-12T04:01:26.822 INFO:tasks.workunit.client.0.smithi032.stdout:3/654: write d15/d1b/f25 [1559666,125669] 0 2024-05-12T04:01:26.826 INFO:tasks.workunit.client.0.smithi032.stdout:3/655: dread - d15/d24/d2a/d8c/db1/fdb zero size 2024-05-12T04:01:26.828 INFO:tasks.workunit.client.0.smithi032.stdout:3/656: truncate d15/d24/d2a/d5b/fc9 962214 0 2024-05-12T04:01:26.830 INFO:tasks.workunit.client.0.smithi032.stdout:3/657: dread d15/d24/d2a/d5b/fc9 [0,4194304] 0 2024-05-12T04:01:26.831 INFO:tasks.workunit.client.0.smithi032.stdout:4/563: mkdir d0/d3/d12/d8d/da1 0 2024-05-12T04:01:26.833 INFO:tasks.workunit.client.0.smithi032.stdout:6/558: dread d7/d9/d48/d67/f45 [0,4194304] 0 2024-05-12T04:01:26.833 INFO:tasks.workunit.client.0.smithi032.stdout:4/564: chown d0/f6f 53289 1 2024-05-12T04:01:26.835 INFO:tasks.workunit.client.0.smithi032.stdout:2/638: sync 2024-05-12T04:01:26.835 INFO:tasks.workunit.client.0.smithi032.stdout:5/721: sync 2024-05-12T04:01:26.835 INFO:tasks.workunit.client.0.smithi032.stdout:4/565: dread - d0/d75/f8a zero size 2024-05-12T04:01:26.835 INFO:tasks.workunit.client.1.smithi033.stdout:0/818: mkdir d6/dc/d6e/d118 0 2024-05-12T04:01:26.836 INFO:tasks.workunit.client.0.smithi032.stdout:0/644: read df/fc1 [1207817,12911] 0 2024-05-12T04:01:26.836 INFO:tasks.workunit.client.1.smithi033.stdout:1/701: sync 2024-05-12T04:01:26.837 INFO:tasks.workunit.client.0.smithi032.stdout:9/521: read d0/d3/f25 [839282,55094] 0 2024-05-12T04:01:26.837 INFO:tasks.workunit.client.0.smithi032.stdout:6/559: truncate d7/d9/d48/d67/d8d/fb5 647335 0 2024-05-12T04:01:26.838 INFO:tasks.workunit.client.0.smithi032.stdout:8/655: truncate d0/d9/d1b/d27/d88/d56/d63/d8f/d9a/d28/d3e/fb6 738412 0 2024-05-12T04:01:26.842 INFO:tasks.workunit.client.0.smithi032.stdout:9/522: write d0/d3/d6/d7/d2a/d46/d13/f7c [351742,76303] 0 2024-05-12T04:01:26.842 INFO:tasks.workunit.client.1.smithi033.stdout:4/786: mkdir dd/d7a/d87/d7e/da7/d66/df2 0 2024-05-12T04:01:26.843 INFO:tasks.workunit.client.0.smithi032.stdout:6/560: dread - d7/d9/d48/d67/d25/f8f zero size 2024-05-12T04:01:26.849 INFO:tasks.workunit.client.0.smithi032.stdout:5/722: dwrite d4/d7/d14/d26/d3c/feb [0,4194304] 0 2024-05-12T04:01:26.850 INFO:tasks.workunit.client.0.smithi032.stdout:2/639: mknod d1/d34/d96/d41/d39/cd6 0 2024-05-12T04:01:26.850 INFO:tasks.workunit.client.1.smithi033.stdout:5/705: write d3/d12/d29/d2b/f46 [400746,52956] 0 2024-05-12T04:01:26.851 INFO:tasks.workunit.client.1.smithi033.stdout:5/706: readlink d3/d94/dc8/l1b 0 2024-05-12T04:01:26.853 INFO:tasks.workunit.client.1.smithi033.stdout:5/707: fsync d3/d94/d54/d57/d70/da3/dbe/d16/d45/d62/f67 0 2024-05-12T04:01:26.854 INFO:tasks.workunit.client.0.smithi032.stdout:3/658: creat d15/d1b/d43/dd0/fe9 x:0 0 0 2024-05-12T04:01:26.854 INFO:tasks.workunit.client.0.smithi032.stdout:2/640: stat d1/d34/d3e 0 2024-05-12T04:01:26.854 INFO:tasks.workunit.client.0.smithi032.stdout:1/610: dwrite d1/d6a/d72/da9/f8a [0,4194304] 0 2024-05-12T04:01:26.854 INFO:tasks.workunit.client.0.smithi032.stdout:5/723: stat d4/d7/f7e 0 2024-05-12T04:01:26.859 INFO:tasks.workunit.client.1.smithi033.stdout:3/756: symlink db/d17/d25/d32/d53/df6/l103 0 2024-05-12T04:01:26.865 INFO:tasks.workunit.client.0.smithi032.stdout:8/656: mknod d0/d9/d1b/d27/d4c/ce6 0 2024-05-12T04:01:26.868 INFO:tasks.workunit.client.0.smithi032.stdout:7/612: truncate dd/d7b/d9a/d9c/db9/f62 2665965 0 2024-05-12T04:01:26.868 INFO:tasks.workunit.client.0.smithi032.stdout:2/641: dwrite d1/d34/d3e/fcd [0,4194304] 0 2024-05-12T04:01:26.868 INFO:tasks.workunit.client.0.smithi032.stdout:4/566: dread d0/d3/d24/f72 [0,4194304] 0 2024-05-12T04:01:26.871 INFO:tasks.workunit.client.1.smithi033.stdout:0/819: unlink d6/d11/d57/f21 0 2024-05-12T04:01:26.872 INFO:tasks.workunit.client.0.smithi032.stdout:9/523: fsync d0/d3/d6/f62 0 2024-05-12T04:01:26.873 INFO:tasks.workunit.client.0.smithi032.stdout:9/524: chown d0/d3/d6/d7/da/d9e 228420 1 2024-05-12T04:01:26.874 INFO:tasks.workunit.client.0.smithi032.stdout:9/525: chown d0/d3/d6 21914 1 2024-05-12T04:01:26.878 INFO:tasks.workunit.client.0.smithi032.stdout:5/724: rmdir d4/d7/d14 39 2024-05-12T04:01:26.880 INFO:tasks.workunit.client.0.smithi032.stdout:5/725: read d4/f20 [1322682,54846] 0 2024-05-12T04:01:26.881 INFO:tasks.workunit.client.0.smithi032.stdout:3/659: symlink d15/d24/d40/lea 0 2024-05-12T04:01:26.884 INFO:tasks.workunit.client.0.smithi032.stdout:0/645: getdents df/d10/d3d/db0/dd5 0 2024-05-12T04:01:26.885 INFO:tasks.workunit.client.1.smithi033.stdout:8/741: dwrite d1/d16/d27/d4d/d82/dd9/d36/f74 [0,4194304] 0 2024-05-12T04:01:26.893 INFO:tasks.workunit.client.1.smithi033.stdout:5/708: creat d3/d94/d85/fe9 x:0 0 0 2024-05-12T04:01:26.895 INFO:tasks.workunit.client.1.smithi033.stdout:3/757: truncate db/d17/d25/d32/d3c/d3d/f6f 310009 0 2024-05-12T04:01:26.896 INFO:tasks.workunit.client.1.smithi033.stdout:3/758: dread - db/d17/d25/d32/d3c/d3d/d47/ffe zero size 2024-05-12T04:01:26.896 INFO:tasks.workunit.client.0.smithi032.stdout:7/613: mkdir dd/d6c/dc3 0 2024-05-12T04:01:26.899 INFO:tasks.workunit.client.1.smithi033.stdout:0/820: truncate d6/dc/d39/f76 1049227 0 2024-05-12T04:01:26.899 INFO:tasks.workunit.client.0.smithi032.stdout:9/526: mknod d0/d3/d6/d58/cad 0 2024-05-12T04:01:26.900 INFO:tasks.workunit.client.1.smithi033.stdout:1/702: truncate d11/d1c/d59/f7b 2996030 0 2024-05-12T04:01:26.900 INFO:tasks.workunit.client.1.smithi033.stdout:1/703: stat d11/d17/d38/le2 0 2024-05-12T04:01:26.902 INFO:tasks.workunit.client.1.smithi033.stdout:1/704: read d11/d17/d27/d36/d3f/f50 [735505,102064] 0 2024-05-12T04:01:26.902 INFO:tasks.workunit.client.0.smithi032.stdout:9/527: truncate d0/d3/d6/d7/d27/f97 4865510 0 2024-05-12T04:01:26.905 INFO:tasks.workunit.client.1.smithi033.stdout:2/781: rename dc/d17/d32/d84/dde/d9e to dc/d13/dfb 0 2024-05-12T04:01:26.906 INFO:tasks.workunit.client.0.smithi032.stdout:0/646: chown df/d10/d48/f96 8 1 2024-05-12T04:01:26.907 INFO:tasks.workunit.client.0.smithi032.stdout:0/647: read df/d10/f80 [48123,42661] 0 2024-05-12T04:01:26.908 INFO:tasks.workunit.client.0.smithi032.stdout:1/611: getdents d1/d6a/d72/da9 0 2024-05-12T04:01:26.909 INFO:tasks.workunit.client.0.smithi032.stdout:1/612: read d1/d7/d30/d18/d93/fa8 [223462,7458] 0 2024-05-12T04:01:26.910 INFO:tasks.workunit.client.0.smithi032.stdout:1/613: read - d1/d7/d30/d18/d87/fbb zero size 2024-05-12T04:01:26.911 INFO:tasks.workunit.client.1.smithi033.stdout:6/811: rmdir d4/df2 0 2024-05-12T04:01:26.913 INFO:tasks.workunit.client.1.smithi033.stdout:7/975: getdents de/d2b/d6c/db5/dd0 0 2024-05-12T04:01:26.917 INFO:tasks.workunit.client.1.smithi033.stdout:2/782: dread dc/d13/d1f/d6e/f81 [0,4194304] 0 2024-05-12T04:01:26.919 INFO:tasks.workunit.client.1.smithi033.stdout:3/759: mkdir db/d17/d3a/d93/d104 0 2024-05-12T04:01:26.919 INFO:tasks.workunit.client.0.smithi032.stdout:3/660: write d15/d20/d22/d7a/fa6 [1218788,107092] 0 2024-05-12T04:01:26.925 INFO:tasks.workunit.client.1.smithi033.stdout:8/742: write d1/d16/f75 [1755349,117448] 0 2024-05-12T04:01:26.926 INFO:tasks.workunit.client.0.smithi032.stdout:5/726: symlink d4/d7/d14/d26/d33/d5f/d7f/l101 0 2024-05-12T04:01:26.927 INFO:tasks.workunit.client.0.smithi032.stdout:5/727: truncate d4/d7/f7e 1726027 0 2024-05-12T04:01:26.928 INFO:tasks.workunit.client.1.smithi033.stdout:4/787: creat dd/d11/d5b/d84/dd7/ff3 x:0 0 0 2024-05-12T04:01:26.929 INFO:tasks.workunit.client.0.smithi032.stdout:9/528: symlink d0/d3/d6/d7/d2a/lae 0 2024-05-12T04:01:26.932 INFO:tasks.workunit.client.0.smithi032.stdout:7/614: write dd/d1b/d43/d5e/f67 [994966,12816] 0 2024-05-12T04:01:26.938 INFO:tasks.workunit.client.1.smithi033.stdout:5/709: creat d3/d94/d54/dc4/fea x:0 0 0 2024-05-12T04:01:26.940 INFO:tasks.workunit.client.0.smithi032.stdout:6/561: getdents d7/de 0 2024-05-12T04:01:26.940 INFO:tasks.workunit.client.1.smithi033.stdout:2/783: symlink dc/d17/d32/d84/dde/lfc 0 2024-05-12T04:01:26.943 INFO:tasks.workunit.client.1.smithi033.stdout:3/760: rename db/d17/d25/c50 to db/d17/d3e/d65/d91/d97/c105 0 2024-05-12T04:01:26.944 INFO:tasks.workunit.client.0.smithi032.stdout:4/567: creat d0/d3/d12/fa2 x:0 0 0 2024-05-12T04:01:26.945 INFO:tasks.workunit.client.0.smithi032.stdout:4/568: stat d0/d3/d12/d1d/l57 0 2024-05-12T04:01:26.947 INFO:tasks.workunit.client.1.smithi033.stdout:8/743: creat d1/d16/d27/d4d/d51/fe2 x:0 0 0 2024-05-12T04:01:26.949 INFO:tasks.workunit.client.1.smithi033.stdout:4/788: read - dd/d7a/d87/d7e/da7/d66/fbf zero size 2024-05-12T04:01:26.953 INFO:tasks.workunit.client.0.smithi032.stdout:0/648: mkdir df/d10/d3d/dd8 0 2024-05-12T04:01:26.957 INFO:tasks.workunit.client.0.smithi032.stdout:7/615: creat dd/d7b/d9a/d9c/db9/d75/fc4 x:0 0 0 2024-05-12T04:01:26.959 INFO:tasks.workunit.client.0.smithi032.stdout:1/614: creat d1/d7/d30/d18/d93/dc1/dc3/fc4 x:0 0 0 2024-05-12T04:01:26.959 INFO:tasks.workunit.client.1.smithi033.stdout:0/821: dwrite d6/dc/fb3 [0,4194304] 0 2024-05-12T04:01:26.960 INFO:tasks.workunit.client.0.smithi032.stdout:2/642: link d1/d13/d6b/l74 d1/d34/d4a/d86/ld7 0 2024-05-12T04:01:26.960 INFO:tasks.workunit.client.0.smithi032.stdout:1/615: stat d1/d6a/fb9 0 2024-05-12T04:01:26.962 INFO:tasks.workunit.client.1.smithi033.stdout:7/976: truncate de/d2b/d6c/f5a 1667636 0 2024-05-12T04:01:26.962 INFO:tasks.workunit.client.0.smithi032.stdout:5/728: dwrite d4/d7/d14/d26/d33/d4a/f86 [0,4194304] 0 2024-05-12T04:01:26.963 INFO:tasks.workunit.client.0.smithi032.stdout:1/616: readlink d1/d7/d30/d35/l97 0 2024-05-12T04:01:26.963 INFO:tasks.workunit.client.1.smithi033.stdout:5/710: chown d3/d94/d54/d57/d70/da3/dbe/d16/d45/d4b/f56 1624756663 1 2024-05-12T04:01:26.965 INFO:tasks.workunit.client.0.smithi032.stdout:3/661: symlink d15/d20/d26/d6a/d6e/d7d/leb 0 2024-05-12T04:01:26.968 INFO:tasks.workunit.client.0.smithi032.stdout:1/617: write d1/f47 [1435360,62662] 0 2024-05-12T04:01:26.973 INFO:tasks.workunit.client.1.smithi033.stdout:4/789: symlink dd/d11/d75/lf4 0 2024-05-12T04:01:26.973 INFO:tasks.workunit.client.0.smithi032.stdout:9/529: link d0/d3/d6/d7/da/f33 d0/d84/faf 0 2024-05-12T04:01:26.974 INFO:tasks.workunit.client.0.smithi032.stdout:5/729: write d4/d7/d14/d26/d33/d5f/fed [231247,113140] 0 2024-05-12T04:01:26.976 INFO:tasks.workunit.client.0.smithi032.stdout:3/662: write d15/d20/d22/d7a/fa6 [269495,108039] 0 2024-05-12T04:01:26.981 INFO:tasks.workunit.client.0.smithi032.stdout:1/618: rmdir d1/d7/d30/d35/d77 39 2024-05-12T04:01:26.981 INFO:tasks.workunit.client.0.smithi032.stdout:3/663: chown d15/d24/f3e 138950541 1 2024-05-12T04:01:26.981 INFO:tasks.workunit.client.0.smithi032.stdout:4/569: dwrite d0/d3/d24/f71 [0,4194304] 0 2024-05-12T04:01:26.982 INFO:tasks.workunit.client.0.smithi032.stdout:7/616: dwrite dd/d1b/d1e/d45/d64/f91 [0,4194304] 0 2024-05-12T04:01:26.983 INFO:tasks.workunit.client.1.smithi033.stdout:6/812: rmdir d4/d1a/d4e/d4f/d94/db4 0 2024-05-12T04:01:26.984 INFO:tasks.workunit.client.1.smithi033.stdout:6/813: fsync d4/f56 0 2024-05-12T04:01:26.984 INFO:tasks.workunit.client.0.smithi032.stdout:3/664: write d15/d1b/fd8 [408633,107546] 0 2024-05-12T04:01:26.986 INFO:tasks.workunit.client.0.smithi032.stdout:5/730: read d4/d7/d14/d26/d33/d4a/d38/f45 [1025787,20587] 0 2024-05-12T04:01:26.990 INFO:tasks.workunit.client.0.smithi032.stdout:6/562: creat d7/d9/fc1 x:0 0 0 2024-05-12T04:01:26.994 INFO:tasks.workunit.client.0.smithi032.stdout:1/619: dread d1/d7/d30/d35/f48 [0,4194304] 0 2024-05-12T04:01:26.995 INFO:tasks.workunit.client.0.smithi032.stdout:5/731: write d4/f9a [353684,116348] 0 2024-05-12T04:01:26.996 INFO:tasks.workunit.client.0.smithi032.stdout:0/649: write df/d10/d3d/f56 [4641678,107600] 0 2024-05-12T04:01:26.999 INFO:tasks.workunit.client.1.smithi033.stdout:8/744: write d1/f6b [4520347,94172] 0 2024-05-12T04:01:27.000 INFO:tasks.workunit.client.1.smithi033.stdout:2/784: dwrite dc/d17/f63 [0,4194304] 0 2024-05-12T04:01:27.002 INFO:tasks.workunit.client.1.smithi033.stdout:1/705: truncate d11/d17/d27/d36/f80 3165171 0 2024-05-12T04:01:27.012 INFO:tasks.workunit.client.0.smithi032.stdout:2/643: dwrite d1/d4d/f98 [0,4194304] 0 2024-05-12T04:01:27.013 INFO:tasks.workunit.client.0.smithi032.stdout:4/570: write d0/f6f [1548728,4959] 0 2024-05-12T04:01:27.015 INFO:tasks.workunit.client.1.smithi033.stdout:5/711: fdatasync d3/d94/dc8/f28 0 2024-05-12T04:01:27.016 INFO:tasks.workunit.client.1.smithi033.stdout:5/712: write d3/d12/f1f [461868,32529] 0 2024-05-12T04:01:27.020 INFO:tasks.workunit.client.0.smithi032.stdout:1/620: rename d1/d6a/f84 to d1/d7/d30/d18/d87/db7/fc5 0 2024-05-12T04:01:27.022 INFO:tasks.workunit.client.1.smithi033.stdout:8/745: creat d1/d16/d27/d4d/d73/fe3 x:0 0 0 2024-05-12T04:01:27.022 INFO:tasks.workunit.client.1.smithi033.stdout:8/746: chown d1/d16/d27/d4d/d82 220846555 1 2024-05-12T04:01:27.023 INFO:tasks.workunit.client.0.smithi032.stdout:4/571: write d0/f6f [2000073,56324] 0 2024-05-12T04:01:27.023 INFO:tasks.workunit.client.1.smithi033.stdout:4/790: mknod dd/d11/d5b/cf5 0 2024-05-12T04:01:27.023 INFO:tasks.workunit.client.1.smithi033.stdout:8/747: chown d1/d4/db2/fbb 23694884 1 2024-05-12T04:01:27.024 INFO:tasks.workunit.client.0.smithi032.stdout:0/650: truncate df/d10/d3d/f85 2831108 0 2024-05-12T04:01:27.025 INFO:tasks.workunit.client.1.smithi033.stdout:8/748: write d1/d16/d27/d4d/d73/f97 [386623,60775] 0 2024-05-12T04:01:27.027 INFO:tasks.workunit.client.1.smithi033.stdout:8/749: truncate d1/d16/d27/d7c/fcf 181866 0 2024-05-12T04:01:27.028 INFO:tasks.workunit.client.1.smithi033.stdout:8/750: chown d1/d16/d27/d4d/d82/dd9/d36/d3c/d47 654193 1 2024-05-12T04:01:27.028 INFO:tasks.workunit.client.0.smithi032.stdout:2/644: dwrite d1/d34/d96/d41/d52/fc3 [0,4194304] 0 2024-05-12T04:01:27.029 INFO:tasks.workunit.client.1.smithi033.stdout:2/785: truncate dc/d17/d3a/f56 628064 0 2024-05-12T04:01:27.031 INFO:tasks.workunit.client.0.smithi032.stdout:5/732: dread d4/d7/d14/d39/d1b/f23 [0,4194304] 0 2024-05-12T04:01:27.034 INFO:tasks.workunit.client.0.smithi032.stdout:3/665: mkdir d15/dec 0 2024-05-12T04:01:27.039 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:26 smithi032 ceph-mon[40008]: pgmap v41: 65 pgs: 65 active+clean; 2.7 GiB data, 9.2 GiB used, 527 GiB / 536 GiB avail; 72 MiB/s rd, 213 MiB/s wr, 403 op/s 2024-05-12T04:01:27.041 INFO:tasks.workunit.client.1.smithi033.stdout:7/977: creat de/df/d17/d1b/f144 x:0 0 0 2024-05-12T04:01:27.044 INFO:tasks.workunit.client.1.smithi033.stdout:4/791: dread fc [4194304,4194304] 0 2024-05-12T04:01:27.048 INFO:tasks.workunit.client.1.smithi033.stdout:3/761: getdents db/d17/d3a 0 2024-05-12T04:01:27.052 INFO:tasks.workunit.client.0.smithi032.stdout:1/621: mkdir d1/d7/d30/d18/d87/dc6 0 2024-05-12T04:01:27.054 INFO:tasks.workunit.client.0.smithi032.stdout:4/572: symlink d0/d3/d12/d1d/d63/d87/la3 0 2024-05-12T04:01:27.055 INFO:tasks.workunit.client.1.smithi033.stdout:1/706: dwrite d11/d1c/ddf/f51 [0,4194304] 0 2024-05-12T04:01:27.057 INFO:tasks.workunit.client.1.smithi033.stdout:3/762: dwrite db/d17/d3a/d93/dc8/fda [0,4194304] 0 2024-05-12T04:01:27.062 INFO:tasks.workunit.client.0.smithi032.stdout:1/622: dwrite d1/d25/da5/fc0 [0,4194304] 0 2024-05-12T04:01:27.063 INFO:tasks.workunit.client.1.smithi033.stdout:0/822: link d6/dc/c41 d6/dc/d58/d6a/c119 0 2024-05-12T04:01:27.068 INFO:tasks.workunit.client.0.smithi032.stdout:3/666: unlink d15/d20/d22/cba 0 2024-05-12T04:01:27.075 INFO:tasks.workunit.client.1.smithi033.stdout:4/792: rmdir dd/d69 39 2024-05-12T04:01:27.076 INFO:tasks.workunit.client.0.smithi032.stdout:7/617: getdents dd/d7b 0 2024-05-12T04:01:27.078 INFO:tasks.workunit.client.1.smithi033.stdout:5/713: mkdir d3/d12/d83/deb 0 2024-05-12T04:01:27.078 INFO:tasks.workunit.client.0.smithi032.stdout:1/623: dwrite d1/d6a/d72/da9/f8a [0,4194304] 0 2024-05-12T04:01:27.081 INFO:tasks.workunit.client.1.smithi033.stdout:8/751: mknod d1/d16/d27/d4d/d73/dd0/ce4 0 2024-05-12T04:01:27.082 INFO:tasks.workunit.client.0.smithi032.stdout:6/563: getdents d7/d97 0 2024-05-12T04:01:27.086 INFO:tasks.workunit.client.1.smithi033.stdout:2/786: mknod dc/cfd 0 2024-05-12T04:01:27.090 INFO:tasks.workunit.client.0.smithi032.stdout:0/651: rmdir df/d10/d3b/dba/dd6 0 2024-05-12T04:01:27.091 INFO:tasks.workunit.client.0.smithi032.stdout:5/733: dwrite d4/d7/d14/d26/d33/d72/dc9/dec/f2a [0,4194304] 0 2024-05-12T04:01:27.091 INFO:tasks.workunit.client.0.smithi032.stdout:0/652: stat df 0 2024-05-12T04:01:27.093 INFO:tasks.workunit.client.0.smithi032.stdout:1/624: dwrite d1/d25/da5/db5/fbf [0,4194304] 0 2024-05-12T04:01:27.094 INFO:tasks.workunit.client.1.smithi033.stdout:6/814: getdents d4/d1a/d4e/d4f 0 2024-05-12T04:01:27.097 INFO:tasks.workunit.client.1.smithi033.stdout:0/823: creat d6/d11/dc7/dc8/f11a x:0 0 0 2024-05-12T04:01:27.099 INFO:tasks.workunit.client.1.smithi033.stdout:7/978: symlink de/df/d121/l145 0 2024-05-12T04:01:27.100 INFO:tasks.workunit.client.0.smithi032.stdout:6/564: creat d7/d9/d48/d67/d1a/d20/fc2 x:0 0 0 2024-05-12T04:01:27.112 INFO:tasks.workunit.client.0.smithi032.stdout:7/618: write dd/d1b/d1e/faa [1290259,25443] 0 2024-05-12T04:01:27.115 INFO:tasks.workunit.client.1.smithi033.stdout:2/787: symlink dc/d13/d1c/d98/dc1/lfe 0 2024-05-12T04:01:27.117 INFO:tasks.workunit.client.0.smithi032.stdout:1/625: creat d1/d7/d30/d18/d87/fc7 x:0 0 0 2024-05-12T04:01:27.119 INFO:tasks.workunit.client.0.smithi032.stdout:3/667: dwrite d15/d33/f5c [0,4194304] 0 2024-05-12T04:01:27.120 INFO:tasks.workunit.client.1.smithi033.stdout:2/788: dread dc/d17/d3a/f9a [0,4194304] 0 2024-05-12T04:01:27.123 INFO:tasks.workunit.client.0.smithi032.stdout:6/565: creat d7/d9/d48/d67/d63/d64/d94/fc3 x:0 0 0 2024-05-12T04:01:27.124 INFO:tasks.workunit.client.0.smithi032.stdout:6/566: fsync d7/de/d2e/f4e 0 2024-05-12T04:01:27.126 INFO:tasks.workunit.client.1.smithi033.stdout:3/763: write db/d17/d25/d32/f46 [2714092,100302] 0 2024-05-12T04:01:27.129 INFO:tasks.workunit.client.0.smithi032.stdout:0/653: dwrite df/d30/d4e/d55/f67 [0,4194304] 0 2024-05-12T04:01:27.129 INFO:tasks.workunit.client.1.smithi033.stdout:5/714: dread d3/d94/d54/d57/d70/da3/dbe/d16/f44 [0,4194304] 0 2024-05-12T04:01:27.129 INFO:tasks.workunit.client.1.smithi033.stdout:7/979: mkdir de/d2f/d42/d146 0 2024-05-12T04:01:27.136 INFO:tasks.workunit.client.0.smithi032.stdout:1/626: mkdir d1/d7/d30/d61/dc8 0 2024-05-12T04:01:27.136 INFO:tasks.workunit.client.1.smithi033.stdout:6/815: dwrite d4/d1a/f42 [0,4194304] 0 2024-05-12T04:01:27.137 INFO:tasks.workunit.client.0.smithi032.stdout:5/734: write d4/d7/d14/d26/f66 [157004,129939] 0 2024-05-12T04:01:27.139 INFO:tasks.workunit.client.0.smithi032.stdout:3/668: rmdir d15/d33 39 2024-05-12T04:01:27.140 INFO:tasks.workunit.client.0.smithi032.stdout:6/567: dwrite d7/d9/d48/d67/d8d/dbc/f9a [0,4194304] 0 2024-05-12T04:01:27.140 INFO:tasks.workunit.client.1.smithi033.stdout:1/707: creat d11/d17/d27/d36/fec x:0 0 0 2024-05-12T04:01:27.143 INFO:tasks.workunit.client.0.smithi032.stdout:7/619: mknod dd/d1b/d43/cc5 0 2024-05-12T04:01:27.144 INFO:tasks.workunit.client.1.smithi033.stdout:3/764: creat db/d59/dcc/f106 x:0 0 0 2024-05-12T04:01:27.147 INFO:tasks.workunit.client.1.smithi033.stdout:4/793: write dd/d11/d5b/f86 [4169179,105469] 0 2024-05-12T04:01:27.148 INFO:tasks.workunit.client.1.smithi033.stdout:7/980: truncate de/d20/d64/fe2 4917165 0 2024-05-12T04:01:27.153 INFO:tasks.workunit.client.1.smithi033.stdout:8/752: creat d1/d16/d27/d4d/d82/dd9/d36/fe5 x:0 0 0 2024-05-12T04:01:27.153 INFO:tasks.workunit.client.1.smithi033.stdout:2/789: dread dc/d17/d27/d2f/f38 [0,4194304] 0 2024-05-12T04:01:27.154 INFO:tasks.workunit.client.1.smithi033.stdout:1/708: dwrite d11/d17/f2e [8388608,4194304] 0 2024-05-12T04:01:27.156 INFO:tasks.workunit.client.1.smithi033.stdout:6/816: creat d4/d43/d57/dc4/ddf/f109 x:0 0 0 2024-05-12T04:01:27.157 INFO:tasks.workunit.client.1.smithi033.stdout:2/790: chown dc/d13/d1c/d98 139 1 2024-05-12T04:01:27.158 INFO:tasks.workunit.client.0.smithi032.stdout:3/669: mknod d15/d1b/d43/dd0/ced 0 2024-05-12T04:01:27.160 INFO:tasks.workunit.client.1.smithi033.stdout:3/765: creat db/d17/d3a/d8a/f107 x:0 0 0 2024-05-12T04:01:27.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:26 smithi033 ceph-mon[49525]: pgmap v41: 65 pgs: 65 active+clean; 2.7 GiB data, 9.2 GiB used, 527 GiB / 536 GiB avail; 72 MiB/s rd, 213 MiB/s wr, 403 op/s 2024-05-12T04:01:27.162 INFO:tasks.workunit.client.0.smithi032.stdout:5/735: truncate d4/d7/d14/d26/d33/d4a/d38/f45 687175 0 2024-05-12T04:01:27.165 INFO:tasks.workunit.client.1.smithi033.stdout:7/981: mknod de/df/d17/d1e/d24/c147 0 2024-05-12T04:01:27.165 INFO:tasks.workunit.client.0.smithi032.stdout:0/654: link df/d28/c4c df/d30/d32/d60/cd9 0 2024-05-12T04:01:27.167 INFO:tasks.workunit.client.0.smithi032.stdout:7/620: creat dd/d6c/dc3/fc6 x:0 0 0 2024-05-12T04:01:27.168 INFO:tasks.workunit.client.0.smithi032.stdout:7/621: chown dd/d1b/d43/d5e/d37/f46 10530 1 2024-05-12T04:01:27.170 INFO:tasks.workunit.client.1.smithi033.stdout:6/817: creat d4/d1a/d3c/dd9/de2/f10a x:0 0 0 2024-05-12T04:01:27.170 INFO:tasks.workunit.client.1.smithi033.stdout:0/824: dread d6/dc/d58/d6a/f89 [0,4194304] 0 2024-05-12T04:01:27.170 INFO:tasks.workunit.client.1.smithi033.stdout:0/825: readlink d6/db/d2c/d73/daf/l74 0 2024-05-12T04:01:27.175 INFO:tasks.workunit.client.0.smithi032.stdout:6/568: dwrite d7/de/d2e/f5a [0,4194304] 0 2024-05-12T04:01:27.177 INFO:tasks.workunit.client.1.smithi033.stdout:3/766: rename db/d59/dcc to db/d17/d3a/d108 0 2024-05-12T04:01:27.180 INFO:tasks.workunit.client.0.smithi032.stdout:5/736: creat d4/d7/d58/f102 x:0 0 0 2024-05-12T04:01:27.181 INFO:tasks.workunit.client.0.smithi032.stdout:0/655: truncate df/d30/d32/d60/f90 1690695 0 2024-05-12T04:01:27.184 INFO:tasks.workunit.client.1.smithi033.stdout:7/982: unlink de/df/d65/de3/d13f/f8b 0 2024-05-12T04:01:27.186 INFO:tasks.workunit.client.1.smithi033.stdout:4/794: write dd/f1b [4121369,94344] 0 2024-05-12T04:01:27.189 INFO:tasks.workunit.client.0.smithi032.stdout:7/622: rename dd/d1b/d1e/d45/f73 to dd/d1b/d1e/fc7 0 2024-05-12T04:01:27.190 INFO:tasks.workunit.client.0.smithi032.stdout:0/656: dwrite df/d10/d48/d70/f76 [0,4194304] 0 2024-05-12T04:01:27.190 INFO:tasks.workunit.client.1.smithi033.stdout:4/795: read dd/d7a/d87/d7e/da7/fce [2310875,3373] 0 2024-05-12T04:01:27.192 INFO:tasks.workunit.client.0.smithi032.stdout:1/627: link d1/d7/d30/d18/l37 d1/d7/d30/lc9 0 2024-05-12T04:01:27.195 INFO:tasks.workunit.client.0.smithi032.stdout:1/628: read d1/dd/f53 [295202,69713] 0 2024-05-12T04:01:27.199 INFO:tasks.workunit.client.0.smithi032.stdout:7/623: dwrite dd/d1b/d1e/d45/d55/f9d [0,4194304] 0 2024-05-12T04:01:27.201 INFO:tasks.workunit.client.1.smithi033.stdout:1/709: dwrite d11/d17/d27/d36/d3f/d4a/fa5 [0,4194304] 0 2024-05-12T04:01:27.202 INFO:tasks.workunit.client.1.smithi033.stdout:1/710: stat d11/d17/d38/le2 0 2024-05-12T04:01:27.208 INFO:tasks.workunit.client.1.smithi033.stdout:3/767: rename db/d17/d3e/dd7/ff7 to db/d17/d25/d32/d53/df6/f109 0 2024-05-12T04:01:27.209 INFO:tasks.workunit.client.1.smithi033.stdout:7/983: dread de/d2b/d6c/db5/dd0/dd6/fda [0,4194304] 0 2024-05-12T04:01:27.209 INFO:tasks.workunit.client.1.smithi033.stdout:3/768: stat db/d17/d3e/d65/d7d 0 2024-05-12T04:01:27.215 INFO:tasks.workunit.client.0.smithi032.stdout:1/629: unlink d1/d7/d30/d18/d93/dc1/dc3/fc4 0 2024-05-12T04:01:27.217 INFO:tasks.workunit.client.1.smithi033.stdout:2/791: rmdir dc/d13/d65/db1/dd0 0 2024-05-12T04:01:27.219 INFO:tasks.workunit.client.1.smithi033.stdout:4/796: dread dd/d11/f1d [0,4194304] 0 2024-05-12T04:01:27.220 INFO:tasks.workunit.client.1.smithi033.stdout:4/797: chown dd/d7a/d87/d7e/da7/d66/f6a 14 1 2024-05-12T04:01:27.221 INFO:tasks.workunit.client.1.smithi033.stdout:0/826: mknod d6/dc/d6e/d101/c11b 0 2024-05-12T04:01:27.222 INFO:tasks.workunit.client.0.smithi032.stdout:3/670: getdents d15/d24/d40 0 2024-05-12T04:01:27.223 INFO:tasks.workunit.client.0.smithi032.stdout:6/569: write d7/de/d2e/f8e [153195,62765] 0 2024-05-12T04:01:27.229 INFO:tasks.workunit.client.1.smithi033.stdout:7/984: truncate de/d2b/d6c/de1/f10d 1777367 0 2024-05-12T04:01:27.230 INFO:tasks.workunit.client.0.smithi032.stdout:3/671: dwrite d15/d24/f3e [0,4194304] 0 2024-05-12T04:01:27.230 INFO:tasks.workunit.client.0.smithi032.stdout:5/737: creat d4/d7/d14/d39/f103 x:0 0 0 2024-05-12T04:01:27.232 INFO:tasks.workunit.client.1.smithi033.stdout:3/769: truncate db/d59/fab 3276705 0 2024-05-12T04:01:27.233 INFO:tasks.workunit.client.0.smithi032.stdout:1/630: creat d1/d7/d30/d35/d54/d7e/fca x:0 0 0 2024-05-12T04:01:27.236 INFO:tasks.workunit.client.0.smithi032.stdout:7/624: write dd/d7b/d9a/d9c/db9/f88 [148500,57460] 0 2024-05-12T04:01:27.236 INFO:tasks.workunit.client.1.smithi033.stdout:7/985: dread de/d2f/fba [0,4194304] 0 2024-05-12T04:01:27.239 INFO:tasks.workunit.client.1.smithi033.stdout:4/798: dread dd/d11/d5b/f86 [4194304,4194304] 0 2024-05-12T04:01:27.239 INFO:tasks.workunit.client.1.smithi033.stdout:8/753: link d1/d16/d27/d4d/d82/dd9/d36/fce d1/d16/d27/d4d/d82/dd9/d36/d3c/fe6 0 2024-05-12T04:01:27.240 INFO:tasks.workunit.client.1.smithi033.stdout:4/799: fsync dd/d11/d75/dd4/fef 0 2024-05-12T04:01:27.242 INFO:tasks.workunit.client.1.smithi033.stdout:6/818: creat d4/d1a/f10b x:0 0 0 2024-05-12T04:01:27.244 INFO:tasks.workunit.client.1.smithi033.stdout:2/792: mknod dc/d17/d3a/cff 0 2024-05-12T04:01:27.245 INFO:tasks.workunit.client.1.smithi033.stdout:1/711: dwrite d11/d17/d27/f57 [0,4194304] 0 2024-05-12T04:01:27.245 INFO:tasks.workunit.client.1.smithi033.stdout:2/793: chown dc/d13/d1c/f92 1981064 1 2024-05-12T04:01:27.246 INFO:tasks.workunit.client.0.smithi032.stdout:0/657: dwrite df/d30/d32/d60/f90 [0,4194304] 0 2024-05-12T04:01:27.247 INFO:tasks.workunit.client.1.smithi033.stdout:0/827: symlink d6/dc/d6e/l11c 0 2024-05-12T04:01:27.247 INFO:tasks.workunit.client.0.smithi032.stdout:6/570: truncate d7/fa 1706011 0 2024-05-12T04:01:27.248 INFO:tasks.workunit.client.0.smithi032.stdout:5/738: mknod d4/d7/d14/d26/d33/d88/dfb/c104 0 2024-05-12T04:01:27.250 INFO:tasks.workunit.client.0.smithi032.stdout:1/631: unlink d1/dd/cf 0 2024-05-12T04:01:27.252 INFO:tasks.workunit.client.1.smithi033.stdout:0/828: dwrite d6/d11/dc7/dd9/d10b/f115 [0,4194304] 0 2024-05-12T04:01:27.253 INFO:tasks.workunit.client.0.smithi032.stdout:7/625: mknod dd/cc8 0 2024-05-12T04:01:27.254 INFO:tasks.workunit.client.1.smithi033.stdout:0/829: fdatasync d6/dc/d58/d6a/dec/f105 0 2024-05-12T04:01:27.257 INFO:tasks.workunit.client.0.smithi032.stdout:6/571: write d7/d9/d5c/d86/fb7 [961462,114090] 0 2024-05-12T04:01:27.257 INFO:tasks.workunit.client.0.smithi032.stdout:5/739: creat d4/d7/d14/d39/d2b/d6e/f105 x:0 0 0 2024-05-12T04:01:27.258 INFO:tasks.workunit.client.1.smithi033.stdout:8/754: unlink d1/c80 0 2024-05-12T04:01:27.259 INFO:tasks.workunit.client.1.smithi033.stdout:8/755: fdatasync d1/d16/d29/d63/fbf 0 2024-05-12T04:01:27.260 INFO:tasks.workunit.client.1.smithi033.stdout:4/800: creat dd/d11/d5b/d84/dd7/ff6 x:0 0 0 2024-05-12T04:01:27.260 INFO:tasks.workunit.client.1.smithi033.stdout:8/756: read d1/d4/d46/f7a [3339795,59274] 0 2024-05-12T04:01:27.264 INFO:tasks.workunit.client.1.smithi033.stdout:1/712: unlink d11/d1c/d43/ldc 0 2024-05-12T04:01:27.265 INFO:tasks.workunit.client.0.smithi032.stdout:7/626: mknod dd/cc9 0 2024-05-12T04:01:27.267 INFO:tasks.workunit.client.1.smithi033.stdout:2/794: rmdir dc/d17/d27/d2f 39 2024-05-12T04:01:27.273 INFO:tasks.workunit.client.1.smithi033.stdout:3/770: write db/d17/d3a/d67/d76/fb2 [3252857,99005] 0 2024-05-12T04:01:27.277 INFO:tasks.workunit.client.0.smithi032.stdout:0/658: rename df/d10/d3d/d4a/d84/f9c to df/d30/d4e/da5/daf/fda 0 2024-05-12T04:01:27.284 INFO:tasks.workunit.client.0.smithi032.stdout:7/627: mknod dd/d1b/d1e/d45/d64/cca 0 2024-05-12T04:01:27.284 INFO:tasks.workunit.client.0.smithi032.stdout:0/659: chown df/d30/d32/d81/cb8 364975832 1 2024-05-12T04:01:27.286 INFO:tasks.workunit.client.1.smithi033.stdout:8/757: symlink d1/d16/d27/d4d/d82/dd9/d36/d3c/d64/le7 0 2024-05-12T04:01:27.293 INFO:tasks.workunit.client.1.smithi033.stdout:3/771: dread db/d17/d25/d32/d53/fb6 [0,4194304] 0 2024-05-12T04:01:27.294 INFO:tasks.workunit.client.1.smithi033.stdout:8/758: dwrite d1/d4/db2/fbb [0,4194304] 0 2024-05-12T04:01:27.297 INFO:tasks.workunit.client.0.smithi032.stdout:1/632: truncate d1/f47 1353902 0 2024-05-12T04:01:27.297 INFO:tasks.workunit.client.1.smithi033.stdout:7/986: dwrite de/df/d65/de3/d13f/f9f [0,4194304] 0 2024-05-12T04:01:27.300 INFO:tasks.workunit.client.1.smithi033.stdout:4/801: symlink dd/d7a/d87/dd9/lf7 0 2024-05-12T04:01:27.300 INFO:tasks.workunit.client.0.smithi032.stdout:6/572: write d7/d9/d48/d67/d25/f29 [1562736,44677] 0 2024-05-12T04:01:27.301 INFO:tasks.workunit.client.1.smithi033.stdout:4/802: fsync dd/d11/da5/fd3 0 2024-05-12T04:01:27.301 INFO:tasks.workunit.client.1.smithi033.stdout:4/803: chown dd/ce3 130 1 2024-05-12T04:01:27.307 INFO:tasks.workunit.client.1.smithi033.stdout:1/713: link d11/d17/d27/f31 d11/d17/d27/d36/d3f/d4a/dc6/de1/fed 0 2024-05-12T04:01:27.307 INFO:tasks.workunit.client.0.smithi032.stdout:8/657: sync 2024-05-12T04:01:27.311 INFO:tasks.workunit.client.0.smithi032.stdout:6/573: dwrite d7/d9/d5c/d62/fad [0,4194304] 0 2024-05-12T04:01:27.316 INFO:tasks.workunit.client.1.smithi033.stdout:3/772: unlink db/d17/d25/d32/d53/df6/l103 0 2024-05-12T04:01:27.321 INFO:tasks.workunit.client.1.smithi033.stdout:4/804: stat dd/d69/cae 0 2024-05-12T04:01:27.323 INFO:tasks.workunit.client.1.smithi033.stdout:2/795: rename dc/d13/d1f/d6e/cba to dc/d13/d1c/c100 0 2024-05-12T04:01:27.325 INFO:tasks.workunit.client.1.smithi033.stdout:7/987: dread de/d2b/f12b [0,4194304] 0 2024-05-12T04:01:27.325 INFO:tasks.workunit.client.1.smithi033.stdout:2/796: chown dc/d17/d32/d84/la4 34307 1 2024-05-12T04:01:27.326 INFO:tasks.workunit.client.1.smithi033.stdout:7/988: read de/d2f/d70/fa3 [3270786,20900] 0 2024-05-12T04:01:27.327 INFO:tasks.workunit.client.1.smithi033.stdout:0/830: getdents d6/db/d2c/d73/daf 0 2024-05-12T04:01:27.329 INFO:tasks.workunit.client.1.smithi033.stdout:3/773: mkdir db/d17/d25/d32/d3c/d3d/d47/d10a 0 2024-05-12T04:01:27.332 INFO:tasks.workunit.client.0.smithi032.stdout:5/740: rename d4/d7/d58/d5a/ca2 to d4/d7/d58/c106 0 2024-05-12T04:01:27.334 INFO:tasks.workunit.client.0.smithi032.stdout:5/741: write d4/d7/d14/d26/d33/d72/dc9/dec/fba [2694571,5062] 0 2024-05-12T04:01:27.335 INFO:tasks.workunit.client.0.smithi032.stdout:0/660: fdatasync df/fa3 0 2024-05-12T04:01:27.337 INFO:tasks.workunit.client.0.smithi032.stdout:5/742: chown d4/d7/d14/d26/d33/d5f/d7f/f84 410401421 1 2024-05-12T04:01:27.337 INFO:tasks.workunit.client.1.smithi033.stdout:8/759: rename d1/d16/d27/ddf to d1/d4/d46/dca/de8 0 2024-05-12T04:01:27.338 INFO:tasks.workunit.client.0.smithi032.stdout:7/628: symlink dd/d1b/d1e/d45/d55/db7/lcb 0 2024-05-12T04:01:27.340 INFO:tasks.workunit.client.0.smithi032.stdout:7/629: chown dd/d1b/fa9 31699354 1 2024-05-12T04:01:27.340 INFO:tasks.workunit.client.1.smithi033.stdout:2/797: fsync f6 0 2024-05-12T04:01:27.340 INFO:tasks.workunit.client.1.smithi033.stdout:2/798: chown dc/d13/f1b 206086986 1 2024-05-12T04:01:27.345 INFO:tasks.workunit.client.0.smithi032.stdout:5/743: dwrite d4/d7/d14/d26/d33/d88/ffe [0,4194304] 0 2024-05-12T04:01:27.351 INFO:tasks.workunit.client.0.smithi032.stdout:8/658: rename d0/d9/d1b/d27/d4c/d4e/d7f/da3/fd6 to d0/d9/d1b/d33/d73/fe7 0 2024-05-12T04:01:27.352 INFO:tasks.workunit.client.0.smithi032.stdout:8/659: stat d0/d9/d1b/d27/d88/d56/f5b 0 2024-05-12T04:01:27.357 INFO:tasks.workunit.client.0.smithi032.stdout:0/661: symlink df/d30/d4e/da5/ldb 0 2024-05-12T04:01:27.361 INFO:tasks.workunit.client.1.smithi033.stdout:8/760: rename d1/d16/d27/d4d/d82/dd9/d36/d3c/l56 to d1/d16/d27/d4d/d51/d8e/le9 0 2024-05-12T04:01:27.362 INFO:tasks.workunit.client.0.smithi032.stdout:1/633: symlink d1/d7/d30/d35/d77/lcb 0 2024-05-12T04:01:27.365 INFO:tasks.workunit.client.1.smithi033.stdout:2/799: symlink dc/d13/d39/l101 0 2024-05-12T04:01:27.366 INFO:tasks.workunit.client.0.smithi032.stdout:5/744: readlink d4/d7/d14/d39/l55 0 2024-05-12T04:01:27.368 INFO:tasks.workunit.client.1.smithi033.stdout:0/831: mknod d6/d11/c11d 0 2024-05-12T04:01:27.371 INFO:tasks.workunit.client.1.smithi033.stdout:5/715: sync 2024-05-12T04:01:27.371 INFO:tasks.workunit.client.0.smithi032.stdout:7/630: write dd/d6c/f85 [169952,56293] 0 2024-05-12T04:01:27.372 INFO:tasks.workunit.client.1.smithi033.stdout:5/716: fdatasync d3/d12/d29/d2b/d4f/f9d 0 2024-05-12T04:01:27.373 INFO:tasks.workunit.client.1.smithi033.stdout:5/717: chown d3/d12/ld4 941483134 1 2024-05-12T04:01:27.373 INFO:tasks.workunit.client.0.smithi032.stdout:8/660: creat d0/d9/d1b/d66/fe8 x:0 0 0 2024-05-12T04:01:27.374 INFO:tasks.workunit.client.1.smithi033.stdout:5/718: readlink d3/d12/d29/le5 0 2024-05-12T04:01:27.374 INFO:tasks.workunit.client.1.smithi033.stdout:5/719: chown d3/d94/d54 2411571 1 2024-05-12T04:01:27.375 INFO:tasks.workunit.client.1.smithi033.stdout:5/720: readlink d3/d94/d54/d57/d70/da3/dbe/d16/d45/d62/la0 0 2024-05-12T04:01:27.375 INFO:tasks.workunit.client.1.smithi033.stdout:4/805: link dd/d11/d75/fa2 dd/d11/d5b/dd1/ff8 0 2024-05-12T04:01:27.376 INFO:tasks.workunit.client.1.smithi033.stdout:4/806: chown dd/d7a/d87/d7e/da7/d38 11374968 1 2024-05-12T04:01:27.377 INFO:tasks.workunit.client.1.smithi033.stdout:3/774: rename db/d17/d3a/d108/lf4 to db/d17/d25/d32/d3c/d3d/d47/l10b 0 2024-05-12T04:01:27.379 INFO:tasks.workunit.client.0.smithi032.stdout:0/662: rename df/d22/c57 to df/d10/d3b/dba/cdc 0 2024-05-12T04:01:27.380 INFO:tasks.workunit.client.1.smithi033.stdout:7/989: truncate f3 3928581 0 2024-05-12T04:01:27.383 INFO:tasks.workunit.client.1.smithi033.stdout:2/800: write dc/d17/d27/d2f/f41 [3781917,34279] 0 2024-05-12T04:01:27.383 INFO:tasks.workunit.client.1.smithi033.stdout:2/801: chown dc/c35 2328799 1 2024-05-12T04:01:27.385 INFO:tasks.workunit.client.0.smithi032.stdout:6/574: link d7/c54 d7/d33/cc4 0 2024-05-12T04:01:27.385 INFO:tasks.workunit.client.1.smithi033.stdout:2/802: write dc/d17/d27/d2f/f38 [6025461,98807] 0 2024-05-12T04:01:27.387 INFO:tasks.workunit.client.0.smithi032.stdout:7/631: read dd/d1b/f2d [263837,54778] 0 2024-05-12T04:01:27.387 INFO:tasks.workunit.client.1.smithi033.stdout:5/721: rmdir d3/d12/d25/db0 39 2024-05-12T04:01:27.388 INFO:tasks.workunit.client.0.smithi032.stdout:8/661: mkdir d0/d9/d13/d21/d35/de9 0 2024-05-12T04:01:27.390 INFO:tasks.workunit.client.0.smithi032.stdout:8/662: write d0/d9/d1b/d33/f45 [434079,47768] 0 2024-05-12T04:01:27.391 INFO:tasks.workunit.client.0.smithi032.stdout:0/663: chown df/d30/d32/cc5 21715 1 2024-05-12T04:01:27.395 INFO:tasks.workunit.client.0.smithi032.stdout:1/634: unlink d1/dd/l50 0 2024-05-12T04:01:27.395 INFO:tasks.workunit.client.1.smithi033.stdout:4/807: dwrite dd/d7a/d87/d7e/da7/fce [4194304,4194304] 0 2024-05-12T04:01:27.398 INFO:tasks.workunit.client.1.smithi033.stdout:8/761: mkdir d1/d16/d27/d4d/d82/dd9/d36/dea 0 2024-05-12T04:01:27.398 INFO:tasks.workunit.client.0.smithi032.stdout:8/663: dwrite d0/d9/d1b/d27/d4c/d4e/d7f/f9c [0,4194304] 0 2024-05-12T04:01:27.398 INFO:tasks.workunit.client.0.smithi032.stdout:5/745: write d4/d7/d14/d26/d3c/f6f [3773317,73936] 0 2024-05-12T04:01:27.400 INFO:tasks.workunit.client.1.smithi033.stdout:7/990: rmdir de/df/d17/d1e/d79 39 2024-05-12T04:01:27.400 INFO:tasks.workunit.client.0.smithi032.stdout:8/664: truncate d0/d4b/fcb 802569 0 2024-05-12T04:01:27.401 INFO:tasks.workunit.client.0.smithi032.stdout:8/665: fdatasync d0/d9/fe1 0 2024-05-12T04:01:27.405 INFO:tasks.workunit.client.1.smithi033.stdout:8/762: dwrite d1/d16/d27/d4d/d82/dd9/f58 [0,4194304] 0 2024-05-12T04:01:27.406 INFO:tasks.workunit.client.0.smithi032.stdout:8/666: write d0/d9/d1b/d27/d88/d56/d63/d8f/d9a/fb4 [910588,55037] 0 2024-05-12T04:01:27.406 INFO:tasks.workunit.client.1.smithi033.stdout:8/763: fdatasync d1/fa4 0 2024-05-12T04:01:27.406 INFO:tasks.workunit.client.0.smithi032.stdout:0/664: mkdir df/d30/d36/ddd 0 2024-05-12T04:01:27.407 INFO:tasks.workunit.client.1.smithi033.stdout:2/803: dread dc/d17/d3a/f56 [0,4194304] 0 2024-05-12T04:01:27.409 INFO:tasks.workunit.client.1.smithi033.stdout:5/722: symlink d3/d94/dc8/dcb/lec 0 2024-05-12T04:01:27.416 INFO:tasks.workunit.client.0.smithi032.stdout:6/575: fsync d7/d9/d48/faa 0 2024-05-12T04:01:27.418 INFO:tasks.workunit.client.0.smithi032.stdout:8/667: write d0/d9/d13/fa0 [84991,113854] 0 2024-05-12T04:01:27.419 INFO:tasks.workunit.client.0.smithi032.stdout:8/668: chown d0/d9/d13/d21/d35 747 1 2024-05-12T04:01:27.421 INFO:tasks.workunit.client.0.smithi032.stdout:8/669: read - d0/d9/d1b/d33/d73/fe0 zero size 2024-05-12T04:01:27.423 INFO:tasks.workunit.client.1.smithi033.stdout:0/832: write d6/dc/f5e [1716179,128852] 0 2024-05-12T04:01:27.424 INFO:tasks.workunit.client.0.smithi032.stdout:6/576: creat d7/d97/fc5 x:0 0 0 2024-05-12T04:01:27.426 INFO:tasks.workunit.client.1.smithi033.stdout:2/804: write dc/d17/d27/d68/d9f/d5e/f77 [4046881,83055] 0 2024-05-12T04:01:27.428 INFO:tasks.workunit.client.1.smithi033.stdout:3/775: truncate db/d17/d25/d32/d53/f87 2533501 0 2024-05-12T04:01:27.429 INFO:tasks.workunit.client.1.smithi033.stdout:8/764: creat d1/d16/d27/d4d/d82/dd9/d36/d3c/da7/feb x:0 0 0 2024-05-12T04:01:27.430 INFO:tasks.workunit.client.1.smithi033.stdout:8/765: chown d1/d16/d29/d63/da3/dba/fe0 52049574 1 2024-05-12T04:01:27.431 INFO:tasks.workunit.client.1.smithi033.stdout:8/766: chown d1/f6b 3700991 1 2024-05-12T04:01:27.431 INFO:tasks.workunit.client.0.smithi032.stdout:8/670: mknod d0/d9/d1b/d27/d4c/d4e/d7f/da3/cea 0 2024-05-12T04:01:27.431 INFO:tasks.workunit.client.1.smithi033.stdout:8/767: chown d1/d16/d29/d63/da3/dba 364 1 2024-05-12T04:01:27.433 INFO:tasks.workunit.client.1.smithi033.stdout:0/833: fsync d6/dc/f5e 0 2024-05-12T04:01:27.435 INFO:tasks.workunit.client.1.smithi033.stdout:4/808: rename dd/d11/d5b/fcd to dd/d11/d5b/dd1/ff9 0 2024-05-12T04:01:27.438 INFO:tasks.workunit.client.0.smithi032.stdout:5/746: getdents d4/d7/d14 0 2024-05-12T04:01:27.439 INFO:tasks.workunit.client.0.smithi032.stdout:6/577: mkdir d7/d9/d48/d67/d1a/da0/dc6 0 2024-05-12T04:01:27.443 INFO:tasks.workunit.client.1.smithi033.stdout:8/768: mknod d1/d4/d46/cec 0 2024-05-12T04:01:27.444 INFO:tasks.workunit.client.0.smithi032.stdout:0/665: dwrite df/d10/d3d/f85 [0,4194304] 0 2024-05-12T04:01:27.445 INFO:tasks.workunit.client.0.smithi032.stdout:6/578: creat d7/fc7 x:0 0 0 2024-05-12T04:01:27.450 INFO:tasks.workunit.client.1.smithi033.stdout:7/991: rename de/d2b to de/df/d9a/d148 0 2024-05-12T04:01:27.453 INFO:tasks.workunit.client.1.smithi033.stdout:4/809: unlink dd/d69/fe9 0 2024-05-12T04:01:27.453 INFO:tasks.workunit.client.1.smithi033.stdout:2/805: mknod dc/d17/d27/d2f/d49/ddd/c102 0 2024-05-12T04:01:27.454 INFO:tasks.workunit.client.0.smithi032.stdout:6/579: mknod d7/de/d2e/db9/cc8 0 2024-05-12T04:01:27.455 INFO:tasks.workunit.client.1.smithi033.stdout:6/819: sync 2024-05-12T04:01:27.455 INFO:tasks.workunit.client.1.smithi033.stdout:5/723: link d3/d12/d29/d2b/d4f/c6f d3/d94/d54/d57/d70/da3/dbe/d16/dd7/ced 0 2024-05-12T04:01:27.455 INFO:tasks.workunit.client.1.smithi033.stdout:2/806: truncate dc/fb8 1653004 0 2024-05-12T04:01:27.456 INFO:tasks.workunit.client.0.smithi032.stdout:5/747: creat d4/d7/d14/d26/f107 x:0 0 0 2024-05-12T04:01:27.459 INFO:tasks.workunit.client.0.smithi032.stdout:8/671: rename d0/d9/d1b/d27/d88/d56/d63/c67 to d0/d9/d1b/d33/ceb 0 2024-05-12T04:01:27.460 INFO:tasks.workunit.client.1.smithi033.stdout:0/834: mknod d6/c11e 0 2024-05-12T04:01:27.463 INFO:tasks.workunit.client.1.smithi033.stdout:2/807: dwrite dc/d17/d27/d2f/fe9 [0,4194304] 0 2024-05-12T04:01:27.465 INFO:tasks.workunit.client.1.smithi033.stdout:2/808: chown dc/d17/d27/d2f/f38 141 1 2024-05-12T04:01:27.465 INFO:tasks.workunit.client.1.smithi033.stdout:3/776: rename db/d17/d3e/d65/d91/d97/ce4 to db/d17/d25/d32/d3c/d3d/d47/d10a/c10c 0 2024-05-12T04:01:27.466 INFO:tasks.workunit.client.0.smithi032.stdout:8/672: dwrite d0/d9/d1b/d27/d88/d56/d63/d8f/d9a/fcf [0,4194304] 0 2024-05-12T04:01:27.467 INFO:tasks.workunit.client.0.smithi032.stdout:6/580: link d7/de/d2e/f8e d7/d9/d48/d67/d1a/da0/fc9 0 2024-05-12T04:01:27.468 INFO:tasks.workunit.client.0.smithi032.stdout:8/673: chown d0/d7/c3d 11629 1 2024-05-12T04:01:27.470 INFO:tasks.workunit.client.1.smithi033.stdout:8/769: dread d1/f13 [0,4194304] 0 2024-05-12T04:01:27.474 INFO:tasks.workunit.client.0.smithi032.stdout:0/666: rename df/d30/d4d/c4f to df/d10/d3d/d4a/d84/dc0/cde 0 2024-05-12T04:01:27.479 INFO:tasks.workunit.client.1.smithi033.stdout:4/810: mkdir dd/d69/dfa 0 2024-05-12T04:01:27.479 INFO:tasks.workunit.client.1.smithi033.stdout:6/820: creat d4/d1a/d3c/f10c x:0 0 0 2024-05-12T04:01:27.481 INFO:tasks.workunit.client.0.smithi032.stdout:6/581: fdatasync d7/d9/d48/d67/d25/f7d 0 2024-05-12T04:01:27.487 INFO:tasks.workunit.client.0.smithi032.stdout:5/748: rename d4/d7/d14/d26/d3c to d4/d7/d14/d108 0 2024-05-12T04:01:27.489 INFO:tasks.workunit.client.0.smithi032.stdout:0/667: dread df/d30/d4e/da5/daf/fda [0,4194304] 0 2024-05-12T04:01:27.491 INFO:tasks.workunit.client.1.smithi033.stdout:5/724: rename d3/d94/d54/cd3 to d3/d94/d54/d57/d70/da3/cee 0 2024-05-12T04:01:27.492 INFO:tasks.workunit.client.1.smithi033.stdout:6/821: dread d4/d1a/d4e/f66 [0,4194304] 0 2024-05-12T04:01:27.493 INFO:tasks.workunit.client.1.smithi033.stdout:6/822: read d4/d5/f10 [3717335,56909] 0 2024-05-12T04:01:27.494 INFO:tasks.workunit.client.1.smithi033.stdout:2/809: dread dc/d17/f4d [0,4194304] 0 2024-05-12T04:01:27.494 INFO:tasks.workunit.client.0.smithi032.stdout:6/582: stat d7/d9/d48/d67/d1a/d20/l43 0 2024-05-12T04:01:27.495 INFO:tasks.workunit.client.1.smithi033.stdout:6/823: write d4/d1a/d67/d6f/daf/ff7 [706927,107860] 0 2024-05-12T04:01:27.496 INFO:tasks.workunit.client.1.smithi033.stdout:7/992: symlink de/df/d9a/d148/d6c/da0/da6/dd1/d11f/l149 0 2024-05-12T04:01:27.498 INFO:tasks.workunit.client.0.smithi032.stdout:0/668: symlink df/d30/d32/d60/dc7/ldf 0 2024-05-12T04:01:27.500 INFO:tasks.workunit.client.0.smithi032.stdout:6/583: rename d7/d9/d48/d67/d25/f7d to d7/d9/d48/d67/d63/d64/fca 0 2024-05-12T04:01:27.503 INFO:tasks.workunit.client.0.smithi032.stdout:0/669: mkdir df/d30/d4e/de0 0 2024-05-12T04:01:27.510 INFO:tasks.workunit.client.0.smithi032.stdout:0/670: creat df/d10/fe1 x:0 0 0 2024-05-12T04:01:27.514 INFO:tasks.workunit.client.0.smithi032.stdout:0/671: unlink df/d10/d3d/d4a/l79 0 2024-05-12T04:01:27.515 INFO:tasks.workunit.client.0.smithi032.stdout:0/672: write df/d10/d3d/f85 [3223962,25358] 0 2024-05-12T04:01:27.519 INFO:tasks.workunit.client.0.smithi032.stdout:0/673: readlink df/d30/d4e/d55/lb9 0 2024-05-12T04:01:27.520 INFO:tasks.workunit.client.0.smithi032.stdout:0/674: rmdir df/d30 39 2024-05-12T04:01:27.521 INFO:tasks.workunit.client.0.smithi032.stdout:0/675: dread - df/d10/d3b/d75/fce zero size 2024-05-12T04:01:27.534 INFO:tasks.workunit.client.1.smithi033.stdout:4/811: truncate dd/d11/d75/f8d 1985048 0 2024-05-12T04:01:27.537 INFO:tasks.workunit.client.0.smithi032.stdout:2/645: sync 2024-05-12T04:01:27.537 INFO:tasks.workunit.client.0.smithi032.stdout:9/530: sync 2024-05-12T04:01:27.540 INFO:tasks.workunit.client.1.smithi033.stdout:3/777: mknod db/d17/d3a/d93/dc8/c10d 0 2024-05-12T04:01:27.541 INFO:tasks.workunit.client.1.smithi033.stdout:3/778: write db/d17/d25/d32/d3c/d3d/d47/ffe [923888,72670] 0 2024-05-12T04:01:27.542 INFO:tasks.workunit.client.1.smithi033.stdout:2/810: symlink dc/d13/d1f/d6e/de8/l103 0 2024-05-12T04:01:27.543 INFO:tasks.workunit.client.0.smithi032.stdout:8/674: dread d0/d9/d1b/d33/f3b [0,4194304] 0 2024-05-12T04:01:27.545 INFO:tasks.workunit.client.1.smithi033.stdout:1/714: sync 2024-05-12T04:01:27.551 INFO:tasks.workunit.client.0.smithi032.stdout:8/675: fdatasync d0/f2 0 2024-05-12T04:01:27.552 INFO:tasks.workunit.client.0.smithi032.stdout:8/676: readlink d0/d9/d1b/d27/d4c/d4e/le3 0 2024-05-12T04:01:27.558 INFO:tasks.workunit.client.0.smithi032.stdout:8/677: dwrite d0/d97/fa6 [0,4194304] 0 2024-05-12T04:01:27.562 INFO:tasks.workunit.client.0.smithi032.stdout:2/646: dread d1/d34/d96/d41/d2c/f59 [0,4194304] 0 2024-05-12T04:01:27.562 INFO:tasks.workunit.client.0.smithi032.stdout:9/531: link d0/d3/d6/d7/f7d d0/d3/d4b/d85/d87/fb0 0 2024-05-12T04:01:27.563 INFO:tasks.workunit.client.0.smithi032.stdout:2/647: fdatasync d1/d13/f2f 0 2024-05-12T04:01:27.564 INFO:tasks.workunit.client.1.smithi033.stdout:5/725: rmdir d3 39 2024-05-12T04:01:27.566 INFO:tasks.workunit.client.1.smithi033.stdout:2/811: dread dc/d13/d1c/d98/fb5 [0,4194304] 0 2024-05-12T04:01:27.567 INFO:tasks.workunit.client.1.smithi033.stdout:3/779: rename db/d17/d3e/dd7/lb4 to db/d17/d25/d32/d53/df6/l10e 0 2024-05-12T04:01:27.567 INFO:tasks.workunit.client.1.smithi033.stdout:0/835: dwrite d6/dc/d58/d6a/feb [0,4194304] 0 2024-05-12T04:01:27.567 INFO:tasks.workunit.client.0.smithi032.stdout:9/532: dread d0/d3/d6/d7/d2a/d46/d13/f7c [0,4194304] 0 2024-05-12T04:01:27.569 INFO:tasks.workunit.client.0.smithi032.stdout:9/533: chown d0/d3/d6/d7/da/c18 497 1 2024-05-12T04:01:27.570 INFO:tasks.workunit.client.1.smithi033.stdout:1/715: mkdir d11/d1c/ddf/d4c/dab/dee 0 2024-05-12T04:01:27.573 INFO:tasks.workunit.client.0.smithi032.stdout:2/648: rmdir d1/d34/d96/d41/d39 39 2024-05-12T04:01:27.574 INFO:tasks.workunit.client.0.smithi032.stdout:2/649: readlink d1/d34/da5/lb2 0 2024-05-12T04:01:27.584 INFO:tasks.workunit.client.0.smithi032.stdout:5/749: dwrite d4/d7/d14/d39/d1b/fd8 [4194304,4194304] 0 2024-05-12T04:01:27.586 INFO:tasks.workunit.client.0.smithi032.stdout:6/584: truncate d7/d9/d48/fbe 568029 0 2024-05-12T04:01:27.589 INFO:tasks.workunit.client.0.smithi032.stdout:8/678: rename d0/d4b/dad/fcc to d0/d9/d1b/fec 0 2024-05-12T04:01:27.590 INFO:tasks.workunit.client.1.smithi033.stdout:3/780: symlink db/d17/d25/l10f 0 2024-05-12T04:01:27.591 INFO:tasks.workunit.client.1.smithi033.stdout:3/781: chown db/d17/d25/d32/d3c/la9 31 1 2024-05-12T04:01:27.592 INFO:tasks.workunit.client.1.smithi033.stdout:2/812: creat dc/d17/d27/d2f/d49/d59/f104 x:0 0 0 2024-05-12T04:01:27.593 INFO:tasks.workunit.client.0.smithi032.stdout:2/650: symlink d1/d34/ld8 0 2024-05-12T04:01:27.595 INFO:tasks.workunit.client.1.smithi033.stdout:0/836: rename d6/d11/dc7/dc8/fd0 to d6/db/d2c/d73/daf/dc3/d110/f11f 0 2024-05-12T04:01:27.596 INFO:tasks.workunit.client.1.smithi033.stdout:1/716: rename d11/d17/d27/d36/d3f to d11/d17/d27/d36/d3f/d69/d87/da4/def 22 2024-05-12T04:01:27.597 INFO:tasks.workunit.client.0.smithi032.stdout:2/651: dread d1/d34/d3e/fcd [0,4194304] 0 2024-05-12T04:01:27.600 INFO:tasks.workunit.client.1.smithi033.stdout:6/824: getdents d4/d43/d57/dc4 0 2024-05-12T04:01:27.604 INFO:tasks.workunit.client.1.smithi033.stdout:8/770: sync 2024-05-12T04:01:27.605 INFO:tasks.workunit.client.1.smithi033.stdout:8/771: stat d1/d16/d27/d4d/d51/c67 0 2024-05-12T04:01:27.609 INFO:tasks.workunit.client.1.smithi033.stdout:8/772: dread d1/d4/db2/fbb [0,4194304] 0 2024-05-12T04:01:27.613 INFO:tasks.workunit.client.1.smithi033.stdout:0/837: creat d6/db/d2c/d73/daf/dc3/f120 x:0 0 0 2024-05-12T04:01:27.613 INFO:tasks.workunit.client.1.smithi033.stdout:0/838: truncate d6/d11/d57/f63 1741724 0 2024-05-12T04:01:27.613 INFO:tasks.workunit.client.1.smithi033.stdout:1/717: truncate d11/d17/d27/da6/fc2 1165515 0 2024-05-12T04:01:27.614 INFO:tasks.workunit.client.1.smithi033.stdout:4/812: getdents dd/d7a/d87/d7e/da7/d66/d42/d59/dd0 0 2024-05-12T04:01:27.618 INFO:tasks.workunit.client.1.smithi033.stdout:6/825: mknod d4/d43/d57/c10d 0 2024-05-12T04:01:27.620 INFO:tasks.workunit.client.0.smithi032.stdout:0/676: dwrite df/d30/d8d/d8f/f97 [0,4194304] 0 2024-05-12T04:01:27.621 INFO:tasks.workunit.client.1.smithi033.stdout:5/726: unlink d3/d12/d25/db0/cbf 0 2024-05-12T04:01:27.624 INFO:tasks.workunit.client.0.smithi032.stdout:0/677: read df/d10/d3d/d4a/f29 [467267,113862] 0 2024-05-12T04:01:27.626 INFO:tasks.workunit.client.1.smithi033.stdout:3/782: mknod db/d17/d3a/d8a/c110 0 2024-05-12T04:01:27.627 INFO:tasks.workunit.client.0.smithi032.stdout:2/652: mknod d1/d34/d96/cd9 0 2024-05-12T04:01:27.633 INFO:tasks.workunit.client.1.smithi033.stdout:8/773: dread d1/d16/d27/d4d/d51/fdb [0,4194304] 0 2024-05-12T04:01:27.637 INFO:tasks.workunit.client.0.smithi032.stdout:0/678: creat df/d30/d4d/fe2 x:0 0 0 2024-05-12T04:01:27.638 INFO:tasks.workunit.client.1.smithi033.stdout:1/718: rmdir d11/d17/d27/da6 39 2024-05-12T04:01:27.639 INFO:tasks.workunit.client.1.smithi033.stdout:1/719: write d11/d1c/f94 [299908,120455] 0 2024-05-12T04:01:27.640 INFO:tasks.workunit.client.0.smithi032.stdout:2/653: mkdir d1/d34/d3e/d67/dda 0 2024-05-12T04:01:27.643 INFO:tasks.workunit.client.1.smithi033.stdout:7/993: dwrite de/df/d60/d75/f112 [0,4194304] 0 2024-05-12T04:01:27.646 INFO:tasks.workunit.client.0.smithi032.stdout:0/679: read df/d10/d3d/d83/d9f/faa [81162,113864] 0 2024-05-12T04:01:27.648 INFO:tasks.workunit.client.0.smithi032.stdout:2/654: mkdir d1/d13/d70/ddb 0 2024-05-12T04:01:27.650 INFO:tasks.workunit.client.1.smithi033.stdout:3/783: mkdir db/d17/d25/d32/d3c/d3d/d111 0 2024-05-12T04:01:27.656 INFO:tasks.workunit.client.0.smithi032.stdout:9/534: write d0/d3/d74/f4e [312634,94953] 0 2024-05-12T04:01:27.658 INFO:tasks.workunit.client.1.smithi033.stdout:8/774: unlink d1/d4/d46/f7a 0 2024-05-12T04:01:27.660 INFO:tasks.workunit.client.0.smithi032.stdout:0/680: mknod df/d10/d3d/db0/dd5/ce3 0 2024-05-12T04:01:27.660 INFO:tasks.workunit.client.1.smithi033.stdout:8/775: chown d1/d16/d27/d4d/d51/d8e/db6/lc1 672535 1 2024-05-12T04:01:27.660 INFO:tasks.workunit.client.1.smithi033.stdout:5/727: dread d3/d94/d54/d57/d70/da3/dbe/d16/d45/f68 [0,4194304] 0 2024-05-12T04:01:27.661 INFO:tasks.workunit.client.0.smithi032.stdout:0/681: fdatasync df/d30/d8d/d8f/fcb 0 2024-05-12T04:01:27.661 INFO:tasks.workunit.client.1.smithi033.stdout:0/839: fsync d6/d11/d57/f60 0 2024-05-12T04:01:27.662 INFO:tasks.workunit.client.0.smithi032.stdout:4/573: sync 2024-05-12T04:01:27.662 INFO:tasks.workunit.client.0.smithi032.stdout:3/672: sync 2024-05-12T04:01:27.665 INFO:tasks.workunit.client.0.smithi032.stdout:9/535: rename d0/d3/d6/f45 to d0/d3/d4b/d88/d99/fb1 0 2024-05-12T04:01:27.667 INFO:tasks.workunit.client.0.smithi032.stdout:9/536: write d0/d3/d4b/d88/f96 [573336,123132] 0 2024-05-12T04:01:27.669 INFO:tasks.workunit.client.0.smithi032.stdout:4/574: dread d0/f6f [0,4194304] 0 2024-05-12T04:01:27.673 INFO:tasks.workunit.client.0.smithi032.stdout:8/679: write d0/f5 [7855588,23155] 0 2024-05-12T04:01:27.675 INFO:tasks.workunit.client.0.smithi032.stdout:5/750: truncate d4/d7/d14/d26/d33/f8b 3351801 0 2024-05-12T04:01:27.677 INFO:tasks.workunit.client.1.smithi033.stdout:7/994: truncate de/df/d9a/d148/d6c/da0/da6/dd1/faf 3161394 0 2024-05-12T04:01:27.677 INFO:tasks.workunit.client.0.smithi032.stdout:6/585: write d7/de/d2e/f58 [2482314,102591] 0 2024-05-12T04:01:27.678 INFO:tasks.workunit.client.1.smithi033.stdout:7/995: read - de/d20/d38/f138 zero size 2024-05-12T04:01:27.679 INFO:tasks.workunit.client.0.smithi032.stdout:6/586: write d7/de/d2e/f5a [296385,119608] 0 2024-05-12T04:01:27.683 INFO:tasks.workunit.client.1.smithi033.stdout:2/813: write dc/d13/d1c/f4f [636980,67314] 0 2024-05-12T04:01:27.684 INFO:tasks.workunit.client.1.smithi033.stdout:2/814: fsync f7 0 2024-05-12T04:01:27.685 INFO:tasks.workunit.client.1.smithi033.stdout:2/815: write dc/d13/d1c/d98/f99 [3317448,32747] 0 2024-05-12T04:01:27.687 INFO:tasks.workunit.client.0.smithi032.stdout:5/751: dread d4/f20 [0,4194304] 0 2024-05-12T04:01:27.689 INFO:tasks.workunit.client.1.smithi033.stdout:0/840: symlink d6/dc/d96/l121 0 2024-05-12T04:01:27.689 INFO:tasks.workunit.client.1.smithi033.stdout:0/841: chown d6/dc/d39/l68 25714706 1 2024-05-12T04:01:27.690 INFO:tasks.workunit.client.0.smithi032.stdout:3/673: truncate d15/d33/f46 1567635 0 2024-05-12T04:01:27.691 INFO:tasks.workunit.client.1.smithi033.stdout:8/776: truncate d1/d16/d27/d7c/fc6 791788 0 2024-05-12T04:01:27.691 INFO:tasks.workunit.client.1.smithi033.stdout:8/777: chown d1/d4/db2/fbc 37461181 1 2024-05-12T04:01:27.693 INFO:tasks.workunit.client.1.smithi033.stdout:4/813: write dd/d11/fac [953666,25986] 0 2024-05-12T04:01:27.695 INFO:tasks.workunit.client.1.smithi033.stdout:6/826: write d4/dad/fa7 [3294469,23908] 0 2024-05-12T04:01:27.702 INFO:tasks.workunit.client.0.smithi032.stdout:0/682: getdents df/d10/d3d/d6c 0 2024-05-12T04:01:27.704 INFO:tasks.workunit.client.0.smithi032.stdout:2/655: dwrite d1/d34/d96/d41/d2c/f99 [0,4194304] 0 2024-05-12T04:01:27.704 INFO:tasks.workunit.client.0.smithi032.stdout:0/683: dread - df/d30/d4d/fe2 zero size 2024-05-12T04:01:27.704 INFO:tasks.workunit.client.1.smithi033.stdout:5/728: unlink d3/d12/d29/fdc 0 2024-05-12T04:01:27.705 INFO:tasks.workunit.client.0.smithi032.stdout:2/656: chown d1/d34/d4a/cc8 12773 1 2024-05-12T04:01:27.708 INFO:tasks.workunit.client.0.smithi032.stdout:2/657: dread d1/d34/da5/fbd [0,4194304] 0 2024-05-12T04:01:27.709 INFO:tasks.workunit.client.1.smithi033.stdout:1/720: write d11/d17/d27/d36/fc0 [2857495,36870] 0 2024-05-12T04:01:27.710 INFO:tasks.workunit.client.0.smithi032.stdout:4/575: creat d0/d3/d12/d1d/fa4 x:0 0 0 2024-05-12T04:01:27.712 INFO:tasks.workunit.client.1.smithi033.stdout:1/721: write d11/d17/d27/d36/d3f/d4a/fa5 [2141189,15129] 0 2024-05-12T04:01:27.714 INFO:tasks.workunit.client.0.smithi032.stdout:5/752: rename d4/d7/d14/d26/d33/d4a/f4e to d4/d7/d14/d26/d33/d72/d92/f109 0 2024-05-12T04:01:27.714 INFO:tasks.workunit.client.1.smithi033.stdout:1/722: write d11/d17/d27/d36/d3f/fd0 [879299,49701] 0 2024-05-12T04:01:27.714 INFO:tasks.workunit.client.1.smithi033.stdout:1/723: dread - d11/d17/d27/d76/d93/fb0 zero size 2024-05-12T04:01:27.718 INFO:tasks.workunit.client.0.smithi032.stdout:5/753: dread d4/d7/d14/db5/fd3 [0,4194304] 0 2024-05-12T04:01:27.719 INFO:tasks.workunit.client.0.smithi032.stdout:5/754: chown d4/d7/d14/d26/c70 17 1 2024-05-12T04:01:27.721 INFO:tasks.workunit.client.0.smithi032.stdout:5/755: chown d4/d7/d14/d39/d2b 22511794 1 2024-05-12T04:01:27.721 INFO:tasks.workunit.client.0.smithi032.stdout:1/635: sync 2024-05-12T04:01:27.721 INFO:tasks.workunit.client.0.smithi032.stdout:7/632: sync 2024-05-12T04:01:27.724 INFO:tasks.workunit.client.1.smithi033.stdout:3/784: creat db/f112 x:0 0 0 2024-05-12T04:01:27.724 INFO:tasks.workunit.client.0.smithi032.stdout:1/636: write d1/dd/d24/f9c [3557574,125783] 0 2024-05-12T04:01:27.728 INFO:tasks.workunit.client.1.smithi033.stdout:3/785: dread db/d17/d3e/dd7/dd0/fd2 [0,4194304] 0 2024-05-12T04:01:27.728 INFO:tasks.workunit.client.0.smithi032.stdout:7/633: write dd/d7b/d9a/d9c/db9/d75/f76 [1145419,127520] 0 2024-05-12T04:01:27.728 INFO:tasks.workunit.client.0.smithi032.stdout:2/658: creat d1/d4d/d5e/fdc x:0 0 0 2024-05-12T04:01:27.729 INFO:tasks.workunit.client.1.smithi033.stdout:7/996: symlink de/d2f/l14a 0 2024-05-12T04:01:27.732 INFO:tasks.workunit.client.0.smithi032.stdout:4/576: fsync d0/d3/df/f26 0 2024-05-12T04:01:27.732 INFO:tasks.workunit.client.0.smithi032.stdout:8/680: dwrite d0/d9/d1b/d66/d75/f9b [0,4194304] 0 2024-05-12T04:01:27.734 INFO:tasks.workunit.client.1.smithi033.stdout:5/729: unlink d3/d12/d29/d2b/d4f/fb4 0 2024-05-12T04:01:27.739 INFO:tasks.workunit.client.0.smithi032.stdout:4/577: read - d0/d3/d12/d1d/f74 zero size 2024-05-12T04:01:27.739 INFO:tasks.workunit.client.0.smithi032.stdout:9/537: dread d0/d3/d4b/d88/d99/fb1 [0,4194304] 0 2024-05-12T04:01:27.740 INFO:tasks.workunit.client.0.smithi032.stdout:3/674: rename d15/d24/f81 to d15/d20/d26/fee 0 2024-05-12T04:01:27.740 INFO:tasks.workunit.client.0.smithi032.stdout:4/578: stat d0/d3/d12/f66 0 2024-05-12T04:01:27.746 INFO:tasks.workunit.client.0.smithi032.stdout:3/675: fdatasync d15/d20/d26/d6a/d6e/d7d/fd5 0 2024-05-12T04:01:27.746 INFO:tasks.workunit.client.0.smithi032.stdout:5/756: creat d4/d7/d14/d26/d33/f10a x:0 0 0 2024-05-12T04:01:27.747 INFO:tasks.workunit.client.0.smithi032.stdout:6/587: dwrite d7/d9/d48/d67/f2c [0,4194304] 0 2024-05-12T04:01:27.748 INFO:tasks.workunit.client.0.smithi032.stdout:3/676: readlink d15/l99 0 2024-05-12T04:01:27.748 INFO:tasks.workunit.client.1.smithi033.stdout:2/816: dwrite dc/d13/d1f/d6e/f81 [4194304,4194304] 0 2024-05-12T04:01:27.752 INFO:tasks.workunit.client.0.smithi032.stdout:0/684: mkdir df/d10/d3d/de4 0 2024-05-12T04:01:27.752 INFO:tasks.workunit.client.1.smithi033.stdout:0/842: write d6/dc/d96/fb1 [1636120,102720] 0 2024-05-12T04:01:27.753 INFO:tasks.workunit.client.0.smithi032.stdout:3/677: dread - d15/d24/d2a/d8c/db1/fdb zero size 2024-05-12T04:01:27.754 INFO:tasks.workunit.client.0.smithi032.stdout:9/538: dwrite d0/d3/d6/d58/f9a [0,4194304] 0 2024-05-12T04:01:27.754 INFO:tasks.workunit.client.0.smithi032.stdout:1/637: symlink d1/d6a/lcc 0 2024-05-12T04:01:27.763 INFO:tasks.workunit.client.1.smithi033.stdout:8/778: dwrite d1/d16/d27/d4d/f5e [0,4194304] 0 2024-05-12T04:01:27.763 INFO:tasks.workunit.client.0.smithi032.stdout:9/539: dread d0/d3/d4b/d88/f96 [0,4194304] 0 2024-05-12T04:01:27.764 INFO:tasks.workunit.client.1.smithi033.stdout:8/779: fdatasync d1/d16/d27/d4d/d82/dd9/f78 0 2024-05-12T04:01:27.766 INFO:tasks.workunit.client.1.smithi033.stdout:8/780: write d1/f6b [420729,128538] 0 2024-05-12T04:01:27.770 INFO:tasks.workunit.client.0.smithi032.stdout:9/540: read d0/d3/d6/d7/da/d39/f4d [2505611,36727] 0 2024-05-12T04:01:27.771 INFO:tasks.workunit.client.1.smithi033.stdout:7/997: creat de/d20/d38/dec/d117/f14b x:0 0 0 2024-05-12T04:01:27.778 INFO:tasks.workunit.client.0.smithi032.stdout:5/757: creat d4/d7/d58/d5a/d99/f10b x:0 0 0 2024-05-12T04:01:27.779 INFO:tasks.workunit.client.1.smithi033.stdout:3/786: dwrite db/d17/d4d/fde [0,4194304] 0 2024-05-12T04:01:27.779 INFO:tasks.workunit.client.1.smithi033.stdout:1/724: dread d11/d17/d27/d76/f72 [0,4194304] 0 2024-05-12T04:01:27.779 INFO:tasks.workunit.client.1.smithi033.stdout:7/998: write de/d20/d38/dec/d117/f14b [806627,54091] 0 2024-05-12T04:01:27.779 INFO:tasks.workunit.client.1.smithi033.stdout:0/843: dread - d6/db/d2c/d73/daf/dc3/d110/f11f zero size 2024-05-12T04:01:27.786 INFO:tasks.workunit.client.0.smithi032.stdout:1/638: mknod d1/d6a/d72/ccd 0 2024-05-12T04:01:27.790 INFO:tasks.workunit.client.0.smithi032.stdout:1/639: dread d1/d25/f4d [4194304,4194304] 0 2024-05-12T04:01:27.798 INFO:tasks.workunit.client.0.smithi032.stdout:9/541: read d0/d3/f42 [148736,31246] 0 2024-05-12T04:01:27.798 INFO:tasks.workunit.client.1.smithi033.stdout:2/817: dwrite dc/d17/d27/d2f/d49/fbb [0,4194304] 0 2024-05-12T04:01:27.799 INFO:tasks.workunit.client.0.smithi032.stdout:6/588: write d7/de/d2e/f8c [289199,12704] 0 2024-05-12T04:01:27.800 INFO:tasks.workunit.client.0.smithi032.stdout:9/542: chown d0/d3/d6/d7/d2a/d46/d13/f7c 13927454 1 2024-05-12T04:01:27.800 INFO:tasks.workunit.client.1.smithi033.stdout:2/818: write dc/d17/d27/d2f/f38 [383848,75901] 0 2024-05-12T04:01:27.802 INFO:tasks.workunit.client.0.smithi032.stdout:9/543: chown d0/d3/d6/d7/d2a/d46/d13/f3c 60 1 2024-05-12T04:01:27.803 INFO:tasks.workunit.client.1.smithi033.stdout:2/819: dread - dc/d13/d65/fa8 zero size 2024-05-12T04:01:27.805 INFO:tasks.workunit.client.1.smithi033.stdout:0/844: fdatasync d6/dc/d58/d6a/fc9 0 2024-05-12T04:01:27.805 INFO:tasks.workunit.client.1.smithi033.stdout:7/999: rmdir de/df/d9a/d148/d6c/db5/dd0 39 2024-05-12T04:01:27.806 INFO:tasks.workunit.client.1.smithi033.stdout:0/845: chown d6/dc/d58/d6a/d94/f10f 9 1 2024-05-12T04:01:27.806 INFO:tasks.workunit.client.1.smithi033.stdout:0/846: chown d6/d11/f33 94934 1 2024-05-12T04:01:27.807 INFO:tasks.workunit.client.0.smithi032.stdout:0/685: dwrite df/d10/d3d/d4a/f29 [0,4194304] 0 2024-05-12T04:01:27.808 INFO:tasks.workunit.client.1.smithi033.stdout:6/827: rename d4/d5/c14 to d4/d1a/d4e/c10e 0 2024-05-12T04:01:27.809 INFO:tasks.workunit.client.1.smithi033.stdout:6/828: dread - d4/d1a/d3c/dd9/de2/f10a zero size 2024-05-12T04:01:27.809 INFO:tasks.workunit.client.0.smithi032.stdout:3/678: write d15/d20/d26/d6a/d6e/d7d/d9a/fa7 [489317,55966] 0 2024-05-12T04:01:27.811 INFO:tasks.workunit.client.0.smithi032.stdout:2/659: write d1/d13/f18 [1113341,51653] 0 2024-05-12T04:01:27.813 INFO:tasks.workunit.client.0.smithi032.stdout:2/660: dread - d1/d34/d96/d41/d52/fd3 zero size 2024-05-12T04:01:27.818 INFO:tasks.workunit.client.0.smithi032.stdout:7/634: creat dd/d7b/d9a/d9c/db9/fcc x:0 0 0 2024-05-12T04:01:27.818 INFO:tasks.workunit.client.0.smithi032.stdout:3/679: fdatasync d15/d20/d26/d6a/d6e/d7d/d9a/fa7 0 2024-05-12T04:01:27.819 INFO:tasks.workunit.client.1.smithi033.stdout:1/725: dwrite d11/d1c/d43/f45 [0,4194304] 0 2024-05-12T04:01:27.819 INFO:tasks.workunit.client.1.smithi033.stdout:2/820: creat dc/d13/d39/f105 x:0 0 0 2024-05-12T04:01:27.821 INFO:tasks.workunit.client.0.smithi032.stdout:6/589: creat d7/de/fcb x:0 0 0 2024-05-12T04:01:27.822 INFO:tasks.workunit.client.0.smithi032.stdout:8/681: link d0/l32 d0/d9/d1b/d27/d4c/d4e/d7f/led 0 2024-05-12T04:01:27.825 INFO:tasks.workunit.client.1.smithi033.stdout:3/787: dread db/d17/d25/d32/d3c/d3d/f72 [0,4194304] 0 2024-05-12T04:01:27.829 INFO:tasks.workunit.client.0.smithi032.stdout:7/635: dwrite dd/d7b/fbf [0,4194304] 0 2024-05-12T04:01:27.832 INFO:tasks.workunit.client.1.smithi033.stdout:3/788: dwrite db/d17/d25/d32/f46 [0,4194304] 0 2024-05-12T04:01:27.839 INFO:tasks.workunit.client.1.smithi033.stdout:6/829: dread d4/d1a/d6c/f86 [0,4194304] 0 2024-05-12T04:01:27.849 INFO:tasks.workunit.client.1.smithi033.stdout:4/814: sync 2024-05-12T04:01:27.872 INFO:tasks.workunit.client.1.smithi033.stdout:5/730: sync 2024-05-12T04:01:27.873 INFO:tasks.workunit.client.1.smithi033.stdout:5/731: readlink d3/d94/d54/d57/d70/da3/dbe/d16/d45/d62/d7f/d84/l9b 0 2024-05-12T04:01:27.915 INFO:tasks.workunit.client.1.smithi033.stdout:8/781: rmdir d1/d4/db2/db8 0 2024-05-12T04:01:27.921 INFO:tasks.workunit.client.1.smithi033.stdout:0/847: creat d6/dc/d6e/d118/f122 x:0 0 0 2024-05-12T04:01:27.927 INFO:tasks.workunit.client.1.smithi033.stdout:4/815: chown dd/d11/d5b/l71 740924 1 2024-05-12T04:01:27.928 INFO:tasks.workunit.client.0.smithi032.stdout:9/544: mknod d0/d3/d4b/d73/d82/cb2 0 2024-05-12T04:01:27.928 INFO:tasks.workunit.client.0.smithi032.stdout:9/545: chown d0/d3/d4b/d73 686 1 2024-05-12T04:01:27.929 INFO:tasks.workunit.client.0.smithi032.stdout:9/546: dread - d0/d3/d6/fa0 zero size 2024-05-12T04:01:27.931 INFO:tasks.workunit.client.0.smithi032.stdout:8/682: read d0/d9/d13/f58 [2816096,112156] 0 2024-05-12T04:01:27.932 INFO:tasks.workunit.client.0.smithi032.stdout:7/636: mkdir dd/d6c/dc3/dcd 0 2024-05-12T04:01:27.935 INFO:tasks.workunit.client.1.smithi033.stdout:2/821: symlink dc/d17/d32/l106 0 2024-05-12T04:01:27.936 INFO:tasks.workunit.client.0.smithi032.stdout:1/640: creat d1/dd/fce x:0 0 0 2024-05-12T04:01:27.937 INFO:tasks.workunit.client.1.smithi033.stdout:5/732: dwrite d3/d12/d29/d2b/f6e [0,4194304] 0 2024-05-12T04:01:27.938 INFO:tasks.workunit.client.1.smithi033.stdout:1/726: creat d11/d17/d84/de7/ff0 x:0 0 0 2024-05-12T04:01:27.939 INFO:tasks.workunit.client.0.smithi032.stdout:2/661: creat d1/d34/d96/d41/d58/fdd x:0 0 0 2024-05-12T04:01:27.940 INFO:tasks.workunit.client.1.smithi033.stdout:0/848: creat d6/d11/dc7/dd9/d10b/f123 x:0 0 0 2024-05-12T04:01:27.941 INFO:tasks.workunit.client.0.smithi032.stdout:3/680: rename d15/d1b/d43/c86 to d15/d20/d26/d6a/d6e/d7d/ddc/cef 0 2024-05-12T04:01:27.942 INFO:tasks.workunit.client.1.smithi033.stdout:3/789: truncate db/d17/d3a/d93/fc6 247197 0 2024-05-12T04:01:27.943 INFO:tasks.workunit.client.1.smithi033.stdout:8/782: dwrite d1/d16/d27/d4d/d82/dd9/d36/d3c/da7/f65 [0,4194304] 0 2024-05-12T04:01:27.944 INFO:tasks.workunit.client.0.smithi032.stdout:6/590: symlink d7/d9/d48/d67/lcc 0 2024-05-12T04:01:27.946 INFO:tasks.workunit.client.1.smithi033.stdout:8/783: truncate d1/d16/d27/d4d/d82/dd9/d36/fce 1135587 0 2024-05-12T04:01:27.949 INFO:tasks.workunit.client.0.smithi032.stdout:7/637: unlink dd/d1b/d1e/d45/d55/d72/c94 0 2024-05-12T04:01:27.950 INFO:tasks.workunit.client.0.smithi032.stdout:0/686: link df/d10/d3d/d4a/f38 df/d30/d4e/d55/dd4/fe5 0 2024-05-12T04:01:27.951 INFO:tasks.workunit.client.0.smithi032.stdout:0/687: fdatasync df/d28/fc6 0 2024-05-12T04:01:27.955 INFO:tasks.workunit.client.1.smithi033.stdout:2/822: dread dc/d13/d1f/f9b [0,4194304] 0 2024-05-12T04:01:27.956 INFO:tasks.workunit.client.0.smithi032.stdout:2/662: unlink d1/d34/d96/d41/f2a 0 2024-05-12T04:01:27.958 INFO:tasks.workunit.client.0.smithi032.stdout:3/681: stat d15/d20/c32 0 2024-05-12T04:01:27.959 INFO:tasks.workunit.client.0.smithi032.stdout:3/682: fsync d15/d33/d3f/d75/fcb 0 2024-05-12T04:01:27.959 INFO:tasks.workunit.client.1.smithi033.stdout:0/849: rmdir d6/dc/d58 39 2024-05-12T04:01:27.961 INFO:tasks.workunit.client.0.smithi032.stdout:6/591: creat d7/d9/d48/d67/d63/d64/fcd x:0 0 0 2024-05-12T04:01:27.962 INFO:tasks.workunit.client.1.smithi033.stdout:3/790: truncate db/d17/d25/f39 174499 0 2024-05-12T04:01:27.965 INFO:tasks.workunit.client.1.smithi033.stdout:6/830: link d4/fae d4/d1a/d67/f10f 0 2024-05-12T04:01:27.969 INFO:tasks.workunit.client.1.smithi033.stdout:8/784: mknod d1/d16/d27/d28/ced 0 2024-05-12T04:01:27.972 INFO:tasks.workunit.client.1.smithi033.stdout:4/816: mknod dd/d7a/d87/d7e/da7/d66/cfb 0 2024-05-12T04:01:27.972 INFO:tasks.workunit.client.1.smithi033.stdout:4/817: fsync dd/d11/fac 0 2024-05-12T04:01:27.973 INFO:tasks.workunit.client.1.smithi033.stdout:5/733: mknod d3/d94/d54/d57/d70/cef 0 2024-05-12T04:01:27.974 INFO:tasks.workunit.client.0.smithi032.stdout:7/638: dread dd/d1b/d1e/f68 [0,4194304] 0 2024-05-12T04:01:27.978 INFO:tasks.workunit.client.0.smithi032.stdout:2/663: dread d1/fc [0,4194304] 0 2024-05-12T04:01:27.978 INFO:tasks.workunit.client.0.smithi032.stdout:0/688: dread df/d28/f54 [0,4194304] 0 2024-05-12T04:01:27.979 INFO:tasks.workunit.client.0.smithi032.stdout:7/639: chown dd/d1b/d1e/d45/d64/l93 444734 1 2024-05-12T04:01:27.980 INFO:tasks.workunit.client.0.smithi032.stdout:9/547: write d0/d3/d6/d7/f7d [104747,104265] 0 2024-05-12T04:01:27.980 INFO:tasks.workunit.client.0.smithi032.stdout:1/641: dwrite d1/d7/d30/d35/f45 [0,4194304] 0 2024-05-12T04:01:27.984 INFO:tasks.workunit.client.0.smithi032.stdout:8/683: truncate d0/d9/d1b/d27/d59/f6d 4155524 0 2024-05-12T04:01:27.984 INFO:tasks.workunit.client.0.smithi032.stdout:8/684: readlink d0/l44 0 2024-05-12T04:01:27.985 INFO:tasks.workunit.client.0.smithi032.stdout:8/685: stat d0/d9/d1b/d27/d4c 0 2024-05-12T04:01:27.986 INFO:tasks.workunit.client.0.smithi032.stdout:3/683: creat d15/d20/d8b/d9b/dbb/ff0 x:0 0 0 2024-05-12T04:01:27.988 INFO:tasks.workunit.client.0.smithi032.stdout:8/686: readlink d0/d9/d1b/d27/d88/d56/d63/lba 0 2024-05-12T04:01:27.989 INFO:tasks.workunit.client.0.smithi032.stdout:3/684: write d15/d24/d2a/d5b/f8e [966403,130969] 0 2024-05-12T04:01:27.992 INFO:tasks.workunit.client.1.smithi033.stdout:8/785: dread - d1/d16/d27/f85 zero size 2024-05-12T04:01:27.994 INFO:tasks.workunit.client.1.smithi033.stdout:8/786: dread - d1/d16/d27/d4d/d82/dd9/fd1 zero size 2024-05-12T04:01:27.995 INFO:tasks.workunit.client.1.smithi033.stdout:2/823: write dc/f3f [1160729,59171] 0 2024-05-12T04:01:27.997 INFO:tasks.workunit.client.0.smithi032.stdout:9/548: dread d0/d84/faf [0,4194304] 0 2024-05-12T04:01:28.001 INFO:tasks.workunit.client.1.smithi033.stdout:1/727: creat d11/d17/d27/d36/d3f/ff1 x:0 0 0 2024-05-12T04:01:28.003 INFO:tasks.workunit.client.0.smithi032.stdout:7/640: creat dd/d7b/d9a/d9c/dc2/fce x:0 0 0 2024-05-12T04:01:28.003 INFO:tasks.workunit.client.0.smithi032.stdout:7/641: chown dd/f1d 506 1 2024-05-12T04:01:28.005 INFO:tasks.workunit.client.1.smithi033.stdout:5/734: dread d3/d94/d54/d57/f89 [0,4194304] 0 2024-05-12T04:01:28.005 INFO:tasks.workunit.client.0.smithi032.stdout:2/664: creat d1/d34/d96/da4/fde x:0 0 0 2024-05-12T04:01:28.006 INFO:tasks.workunit.client.0.smithi032.stdout:2/665: chown d1/d34/d3e/d67/lb8 0 1 2024-05-12T04:01:28.010 INFO:tasks.workunit.client.1.smithi033.stdout:5/735: readlink d3/d12/d29/l50 0 2024-05-12T04:01:28.010 INFO:tasks.workunit.client.1.smithi033.stdout:8/787: dwrite d1/d16/d27/d4d/d82/dd9/f8f [0,4194304] 0 2024-05-12T04:01:28.012 INFO:tasks.workunit.client.1.smithi033.stdout:8/788: chown d1/d16/l4a 1 1 2024-05-12T04:01:28.013 INFO:tasks.workunit.client.0.smithi032.stdout:2/666: dread d1/d34/d3e/d67/fba [0,4194304] 0 2024-05-12T04:01:28.015 INFO:tasks.workunit.client.1.smithi033.stdout:0/850: dwrite d6/dc/d39/fd7 [0,4194304] 0 2024-05-12T04:01:28.015 INFO:tasks.workunit.client.0.smithi032.stdout:6/592: dread d7/d9/d48/d67/f6e [0,4194304] 0 2024-05-12T04:01:28.016 INFO:tasks.workunit.client.1.smithi033.stdout:2/824: dwrite dc/d13/d1c/d98/f99 [0,4194304] 0 2024-05-12T04:01:28.019 INFO:tasks.workunit.client.1.smithi033.stdout:3/791: dwrite db/d17/d25/d32/d3c/d3d/d47/fdc [0,4194304] 0 2024-05-12T04:01:28.021 INFO:tasks.workunit.client.1.smithi033.stdout:5/736: creat d3/d12/d83/ff0 x:0 0 0 2024-05-12T04:01:28.026 INFO:tasks.workunit.client.0.smithi032.stdout:7/642: dread dd/d1b/d43/d5e/d37/f39 [0,4194304] 0 2024-05-12T04:01:28.027 INFO:tasks.workunit.client.0.smithi032.stdout:2/667: unlink d1/d13/d6b/l78 0 2024-05-12T04:01:28.028 INFO:tasks.workunit.client.0.smithi032.stdout:0/689: dread df/d10/d3b/d75/f78 [0,4194304] 0 2024-05-12T04:01:28.034 INFO:tasks.workunit.client.1.smithi033.stdout:2/825: creat dc/d13/d65/f107 x:0 0 0 2024-05-12T04:01:28.035 INFO:tasks.workunit.client.1.smithi033.stdout:2/826: readlink dc/d13/dfb/dd7/leb 0 2024-05-12T04:01:28.038 INFO:tasks.workunit.client.0.smithi032.stdout:2/668: dwrite d1/d4d/f98 [0,4194304] 0 2024-05-12T04:01:28.044 INFO:tasks.workunit.client.1.smithi033.stdout:0/851: mkdir d6/dc/d58/d6a/d80/db8/d124 0 2024-05-12T04:01:28.046 INFO:tasks.workunit.client.1.smithi033.stdout:1/728: creat d11/d17/ff2 x:0 0 0 2024-05-12T04:01:28.047 INFO:tasks.workunit.client.0.smithi032.stdout:1/642: dwrite d1/d7/d30/d18/d93/fa8 [0,4194304] 0 2024-05-12T04:01:28.049 INFO:tasks.workunit.client.0.smithi032.stdout:7/643: creat dd/d7b/d9a/d9c/dc2/fcf x:0 0 0 2024-05-12T04:01:28.055 INFO:tasks.workunit.client.1.smithi033.stdout:3/792: creat db/d17/d25/d32/d3c/d3d/d111/f113 x:0 0 0 2024-05-12T04:01:28.057 INFO:tasks.workunit.client.1.smithi033.stdout:3/793: fdatasync db/d17/d3a/d67/d76/da0/fee 0 2024-05-12T04:01:28.061 INFO:tasks.workunit.client.1.smithi033.stdout:8/789: dwrite d1/d16/d27/d4d/d82/dd9/d36/d3c/da7/f2f [0,4194304] 0 2024-05-12T04:01:28.063 INFO:tasks.workunit.client.1.smithi033.stdout:8/790: chown d1/d16/d27/d4d/d82/dd9/d36/d3c/da7/f43 0 1 2024-05-12T04:01:28.068 INFO:tasks.workunit.client.0.smithi032.stdout:0/690: write fa [5758425,18647] 0 2024-05-12T04:01:28.071 INFO:tasks.workunit.client.0.smithi032.stdout:8/687: getdents d0 0 2024-05-12T04:01:28.071 INFO:tasks.workunit.client.1.smithi033.stdout:1/729: creat d11/d17/d27/da6/ff3 x:0 0 0 2024-05-12T04:01:28.073 INFO:tasks.workunit.client.0.smithi032.stdout:3/685: getdents d15/d33/d3f 0 2024-05-12T04:01:28.074 INFO:tasks.workunit.client.0.smithi032.stdout:1/643: dread d1/d7/f78 [0,4194304] 0 2024-05-12T04:01:28.075 INFO:tasks.workunit.client.0.smithi032.stdout:1/644: chown d1/f13 0 1 2024-05-12T04:01:28.078 INFO:tasks.workunit.client.1.smithi033.stdout:0/852: write d6/d11/d98/fa1 [4221620,10639] 0 2024-05-12T04:01:28.079 INFO:tasks.workunit.client.1.smithi033.stdout:5/737: link d3/d94/d54/d57/d70/da3/dbe/daa/lba d3/d94/d54/d57/d70/da3/dbe/d16/d45/lf1 0 2024-05-12T04:01:28.079 INFO:tasks.workunit.client.1.smithi033.stdout:2/827: dwrite dc/d13/d1c/d98/fb5 [0,4194304] 0 2024-05-12T04:01:28.083 INFO:tasks.workunit.client.0.smithi032.stdout:8/688: fdatasync d0/d9/d1b/d27/d4c/f70 0 2024-05-12T04:01:28.084 INFO:tasks.workunit.client.1.smithi033.stdout:3/794: creat db/d17/f114 x:0 0 0 2024-05-12T04:01:28.085 INFO:tasks.workunit.client.1.smithi033.stdout:2/828: dread dc/d17/d27/d2f/fe9 [0,4194304] 0 2024-05-12T04:01:28.086 INFO:tasks.workunit.client.0.smithi032.stdout:6/593: dwrite d7/de/d2e/f8e [4194304,4194304] 0 2024-05-12T04:01:28.088 INFO:tasks.workunit.client.0.smithi032.stdout:3/686: mknod d15/d24/d40/cf1 0 2024-05-12T04:01:28.090 INFO:tasks.workunit.client.1.smithi033.stdout:8/791: rename d1/d4/d91/cb3 to d1/d16/d27/d4d/d73/cee 0 2024-05-12T04:01:28.090 INFO:tasks.workunit.client.0.smithi032.stdout:7/644: dwrite dd/d1b/d1e/f20 [0,4194304] 0 2024-05-12T04:01:28.094 INFO:tasks.workunit.client.0.smithi032.stdout:1/645: creat d1/d6a/fcf x:0 0 0 2024-05-12T04:01:28.095 INFO:tasks.workunit.client.0.smithi032.stdout:3/687: readlink d15/d20/d8b/d9b/ld3 0 2024-05-12T04:01:28.095 INFO:tasks.workunit.client.0.smithi032.stdout:7/645: fsync dd/d7b/fb1 0 2024-05-12T04:01:28.095 INFO:tasks.workunit.client.1.smithi033.stdout:5/738: unlink d3/d12/f1f 0 2024-05-12T04:01:28.097 INFO:tasks.workunit.client.1.smithi033.stdout:0/853: mkdir d6/dc/d58/d6a/d94/d125 0 2024-05-12T04:01:28.097 INFO:tasks.workunit.client.0.smithi032.stdout:7/646: stat dd/d7b/d9a/d9c/db9/c78 0 2024-05-12T04:01:28.098 INFO:tasks.workunit.client.0.smithi032.stdout:0/691: mknod df/d10/d48/ce6 0 2024-05-12T04:01:28.101 INFO:tasks.workunit.client.0.smithi032.stdout:0/692: fsync df/d30/d8d/d8f/fcb 0 2024-05-12T04:01:28.102 INFO:tasks.workunit.client.0.smithi032.stdout:8/689: chown d0/d4b/c6f 2 1 2024-05-12T04:01:28.102 INFO:tasks.workunit.client.1.smithi033.stdout:2/829: creat dc/d13/d1f/d6e/de8/f108 x:0 0 0 2024-05-12T04:01:28.103 INFO:tasks.workunit.client.0.smithi032.stdout:2/669: dwrite d1/d13/d70/f91 [0,4194304] 0 2024-05-12T04:01:28.106 INFO:tasks.workunit.client.0.smithi032.stdout:2/670: dread - d1/d34/d96/d41/d58/fdd zero size 2024-05-12T04:01:28.111 INFO:tasks.workunit.client.1.smithi033.stdout:5/739: fdatasync d3/f2c 0 2024-05-12T04:01:28.112 INFO:tasks.workunit.client.1.smithi033.stdout:5/740: chown d3/d12/d25 266401290 1 2024-05-12T04:01:28.113 INFO:tasks.workunit.client.1.smithi033.stdout:0/854: unlink d6/dc/d39/f64 0 2024-05-12T04:01:28.135 INFO:tasks.workunit.client.1.smithi033.stdout:3/795: mknod db/d17/d25/d32/c115 0 2024-05-12T04:01:28.135 INFO:tasks.workunit.client.1.smithi033.stdout:3/796: stat db/d17/d25/d32/d3c/d44/fbd 0 2024-05-12T04:01:28.137 INFO:tasks.workunit.client.1.smithi033.stdout:2/830: creat dc/d3b/f109 x:0 0 0 2024-05-12T04:01:28.139 INFO:tasks.workunit.client.1.smithi033.stdout:8/792: mkdir d1/d16/d27/d7c/db7/def 0 2024-05-12T04:01:28.141 INFO:tasks.workunit.client.0.smithi032.stdout:8/690: mkdir d0/d9/d1b/d66/d75/d84/dee 0 2024-05-12T04:01:28.144 INFO:tasks.workunit.client.0.smithi032.stdout:6/594: rename d7/f2b to d7/d9/d48/d67/d63/d64/d94/fce 0 2024-05-12T04:01:28.145 INFO:tasks.workunit.client.1.smithi033.stdout:5/741: rmdir d3/d94/d54/d57/d70/da3/dbe/d16 39 2024-05-12T04:01:28.145 INFO:tasks.workunit.client.0.smithi032.stdout:8/691: chown d0/d9/d1b/d27/d88/d56/d63/d8f/d9a/d28/f49 62 1 2024-05-12T04:01:28.146 INFO:tasks.workunit.client.1.smithi033.stdout:0/855: dread d6/db/d2c/f48 [0,4194304] 0 2024-05-12T04:01:28.147 INFO:tasks.workunit.client.0.smithi032.stdout:7/647: dwrite dd/d1b/d1e/f7e [0,4194304] 0 2024-05-12T04:01:28.149 INFO:tasks.workunit.client.0.smithi032.stdout:1/646: creat d1/d7/d30/d18/d87/dc6/fd0 x:0 0 0 2024-05-12T04:01:28.154 INFO:tasks.workunit.client.0.smithi032.stdout:1/647: write d1/d7/d30/d35/d77/faf [636984,87123] 0 2024-05-12T04:01:28.154 INFO:tasks.workunit.client.1.smithi033.stdout:5/742: mknod d3/d94/d85/d99/cf2 0 2024-05-12T04:01:28.157 INFO:tasks.workunit.client.0.smithi032.stdout:3/688: creat d15/d20/ff2 x:0 0 0 2024-05-12T04:01:28.157 INFO:tasks.workunit.client.0.smithi032.stdout:6/595: truncate d7/d97/f9e 372974 0 2024-05-12T04:01:28.158 INFO:tasks.workunit.client.1.smithi033.stdout:8/793: dwrite d1/f13 [4194304,4194304] 0 2024-05-12T04:01:28.158 INFO:tasks.workunit.client.1.smithi033.stdout:0/856: creat d6/db/d2c/d113/f126 x:0 0 0 2024-05-12T04:01:28.160 INFO:tasks.workunit.client.0.smithi032.stdout:8/692: dwrite d0/d9/d1b/d27/d88/d56/d63/d8f/d9a/f55 [4194304,4194304] 0 2024-05-12T04:01:28.160 INFO:tasks.workunit.client.1.smithi033.stdout:8/794: write d1/d16/d27/fae [935875,60669] 0 2024-05-12T04:01:28.162 INFO:tasks.workunit.client.1.smithi033.stdout:8/795: write d1/d4/d91/fbd [195719,89001] 0 2024-05-12T04:01:28.163 INFO:tasks.workunit.client.1.smithi033.stdout:5/743: unlink d3/d94/d85/d99/da2/fdd 0 2024-05-12T04:01:28.172 INFO:tasks.workunit.client.1.smithi033.stdout:3/797: dread db/d17/d3a/d67/d76/da0/fba [0,4194304] 0 2024-05-12T04:01:28.173 INFO:tasks.workunit.client.0.smithi032.stdout:8/693: dwrite d0/d9/d1b/d27/d4c/d4e/d90/fb9 [0,4194304] 0 2024-05-12T04:01:28.173 INFO:tasks.workunit.client.1.smithi033.stdout:6/831: sync 2024-05-12T04:01:28.174 INFO:tasks.workunit.client.1.smithi033.stdout:4/818: sync 2024-05-12T04:01:28.175 INFO:tasks.workunit.client.1.smithi033.stdout:4/819: stat dd/d7a/d87/d7e/da7/d66/lb8 0 2024-05-12T04:01:28.176 INFO:tasks.workunit.client.0.smithi032.stdout:3/689: dread - d15/d20/d26/d6a/fb4 zero size 2024-05-12T04:01:28.177 INFO:tasks.workunit.client.0.smithi032.stdout:3/690: truncate d15/d33/fe8 162282 0 2024-05-12T04:01:28.180 INFO:tasks.workunit.client.0.smithi032.stdout:3/691: fsync d15/d20/d8b/d9b/dbb/ff0 0 2024-05-12T04:01:28.180 INFO:tasks.workunit.client.0.smithi032.stdout:2/671: getdents d1/d4d/d5e 0 2024-05-12T04:01:28.181 INFO:tasks.workunit.client.1.smithi033.stdout:2/831: dread f9 [0,4194304] 0 2024-05-12T04:01:28.182 INFO:tasks.workunit.client.0.smithi032.stdout:3/692: truncate d15/d20/d22/d7a/fb2 716071 0 2024-05-12T04:01:28.183 INFO:tasks.workunit.client.1.smithi033.stdout:8/796: mknod d1/d16/d27/d7c/db7/cf0 0 2024-05-12T04:01:28.185 INFO:tasks.workunit.client.1.smithi033.stdout:5/744: chown d3/d94/d54/d57/d70/da3/dbe/d16/l4c 6 1 2024-05-12T04:01:28.190 INFO:tasks.workunit.client.1.smithi033.stdout:6/832: unlink d4/f9a 0 2024-05-12T04:01:28.191 INFO:tasks.workunit.client.0.smithi032.stdout:8/694: creat d0/d9/d1b/d33/d73/db7/fef x:0 0 0 2024-05-12T04:01:28.192 INFO:tasks.workunit.client.0.smithi032.stdout:7/648: dwrite dd/d1b/d43/d74/f84 [0,4194304] 0 2024-05-12T04:01:28.194 INFO:tasks.workunit.client.0.smithi032.stdout:2/672: mknod d1/d34/d96/da4/cdf 0 2024-05-12T04:01:28.194 INFO:tasks.workunit.client.1.smithi033.stdout:0/857: mkdir d6/dc/d58/d6a/d94/d125/d127 0 2024-05-12T04:01:28.197 INFO:tasks.workunit.client.0.smithi032.stdout:8/695: dread d0/d9/d1b/d33/d73/fe7 [0,4194304] 0 2024-05-12T04:01:28.203 INFO:tasks.workunit.client.1.smithi033.stdout:3/798: symlink db/d17/d3a/d93/deb/l116 0 2024-05-12T04:01:28.206 INFO:tasks.workunit.client.1.smithi033.stdout:6/833: mkdir d4/dad/d110 0 2024-05-12T04:01:28.207 INFO:tasks.workunit.client.0.smithi032.stdout:7/649: rmdir dd/d7b/d9a/d9c 39 2024-05-12T04:01:28.211 INFO:tasks.workunit.client.1.smithi033.stdout:6/834: chown d4/d1a/d67/d68/c77 388867 1 2024-05-12T04:01:28.211 INFO:tasks.workunit.client.1.smithi033.stdout:4/820: unlink dd/d11/f95 0 2024-05-12T04:01:28.211 INFO:tasks.workunit.client.1.smithi033.stdout:2/832: write dc/d17/d27/f94 [1998690,69100] 0 2024-05-12T04:01:28.211 INFO:tasks.workunit.client.0.smithi032.stdout:7/650: write dd/d1b/d1e/d45/d55/d72/da0/dbd/f98 [1139801,111468] 0 2024-05-12T04:01:28.212 INFO:tasks.workunit.client.1.smithi033.stdout:3/799: dwrite db/d17/d25/d32/d3c/d3d/d111/f113 [0,4194304] 0 2024-05-12T04:01:28.212 INFO:tasks.workunit.client.1.smithi033.stdout:4/821: chown dd/d7a/d87/d7e/da7/f45 48307847 1 2024-05-12T04:01:28.213 INFO:tasks.workunit.client.0.smithi032.stdout:1/648: getdents d1/dd/d24/d9b 0 2024-05-12T04:01:28.213 INFO:tasks.workunit.client.1.smithi033.stdout:2/833: chown dc/d17/d27/d2f/d49/d59/c7e 264828606 1 2024-05-12T04:01:28.213 INFO:tasks.workunit.client.1.smithi033.stdout:0/858: truncate d6/db/d2c/d73/daf/dc3/fcf 17364 0 2024-05-12T04:01:28.214 INFO:tasks.workunit.client.1.smithi033.stdout:4/822: write dd/d69/f7d [1838883,52920] 0 2024-05-12T04:01:28.215 INFO:tasks.workunit.client.1.smithi033.stdout:8/797: creat d1/d16/d27/d4d/d82/dd9/d36/dea/ff1 x:0 0 0 2024-05-12T04:01:28.215 INFO:tasks.workunit.client.1.smithi033.stdout:2/834: chown dc/d13/dfb/dd5 0 1 2024-05-12T04:01:28.215 INFO:tasks.workunit.client.0.smithi032.stdout:3/693: mknod d15/d20/cf3 0 2024-05-12T04:01:28.217 INFO:tasks.workunit.client.1.smithi033.stdout:5/745: truncate d3/d94/d54/d98/f9f 1898157 0 2024-05-12T04:01:28.217 INFO:tasks.workunit.client.1.smithi033.stdout:1/730: sync 2024-05-12T04:01:28.219 INFO:tasks.workunit.client.0.smithi032.stdout:6/596: getdents d7/d9 0 2024-05-12T04:01:28.219 INFO:tasks.workunit.client.0.smithi032.stdout:2/673: write d1/fc [2988265,98269] 0 2024-05-12T04:01:28.220 INFO:tasks.workunit.client.0.smithi032.stdout:6/597: write d7/d33/f7a [989023,94659] 0 2024-05-12T04:01:28.221 INFO:tasks.workunit.client.1.smithi033.stdout:6/835: mknod d4/d43/d57/c111 0 2024-05-12T04:01:28.223 INFO:tasks.workunit.client.1.smithi033.stdout:6/836: write d4/d43/d57/dc4/ddf/f109 [507004,112073] 0 2024-05-12T04:01:28.225 INFO:tasks.workunit.client.1.smithi033.stdout:0/859: mknod d6/db/d2c/d73/daf/c128 0 2024-05-12T04:01:28.226 INFO:tasks.workunit.client.0.smithi032.stdout:3/694: symlink d15/d24/d40/lf4 0 2024-05-12T04:01:28.226 INFO:tasks.workunit.client.1.smithi033.stdout:0/860: dread - d6/dc/d58/d6a/d80/db8/f114 zero size 2024-05-12T04:01:28.226 INFO:tasks.workunit.client.0.smithi032.stdout:3/695: fsync d15/d33/f4c 0 2024-05-12T04:01:28.227 INFO:tasks.workunit.client.1.smithi033.stdout:2/835: unlink dc/d3b/f109 0 2024-05-12T04:01:28.233 INFO:tasks.workunit.client.0.smithi032.stdout:6/598: fsync d7/d9/d5c/d8b/da1/fa2 0 2024-05-12T04:01:28.236 INFO:tasks.workunit.client.0.smithi032.stdout:1/649: mkdir d1/d7/d49/dd1 0 2024-05-12T04:01:28.240 INFO:tasks.workunit.client.0.smithi032.stdout:8/696: creat d0/d9/d1b/d27/d4c/d4e/ff0 x:0 0 0 2024-05-12T04:01:28.241 INFO:tasks.workunit.client.0.smithi032.stdout:6/599: dwrite d7/d9/d5c/d8b/da1/fae [0,4194304] 0 2024-05-12T04:01:28.242 INFO:tasks.workunit.client.0.smithi032.stdout:3/696: creat d15/d33/d3f/ff5 x:0 0 0 2024-05-12T04:01:28.243 INFO:tasks.workunit.client.0.smithi032.stdout:3/697: fsync d15/f4e 0 2024-05-12T04:01:28.244 INFO:tasks.workunit.client.1.smithi033.stdout:6/837: mknod d4/d6e/dbb/c112 0 2024-05-12T04:01:28.244 INFO:tasks.workunit.client.0.smithi032.stdout:8/697: write d0/d9/d1b/d27/d88/d56/d63/d8f/d9a/d28/d3e/fb3 [3193816,130410] 0 2024-05-12T04:01:28.247 INFO:tasks.workunit.client.1.smithi033.stdout:3/800: write db/d17/d3a/d93/fc6 [134418,69848] 0 2024-05-12T04:01:28.248 INFO:tasks.workunit.client.1.smithi033.stdout:3/801: stat db/d59/f5b 0 2024-05-12T04:01:28.252 INFO:tasks.workunit.client.0.smithi032.stdout:6/600: creat d7/d9/d48/d67/d1a/da0/fcf x:0 0 0 2024-05-12T04:01:28.253 INFO:tasks.workunit.client.1.smithi033.stdout:4/823: mknod dd/d7a/d87/d7e/da7/d38/cfc 0 2024-05-12T04:01:28.254 INFO:tasks.workunit.client.0.smithi032.stdout:6/601: dread - d7/d9/d48/d67/d1a/d20/fc2 zero size 2024-05-12T04:01:28.255 INFO:tasks.workunit.client.0.smithi032.stdout:6/602: write d7/d9/d5c/fbd [304134,49327] 0 2024-05-12T04:01:28.256 INFO:tasks.workunit.client.0.smithi032.stdout:6/603: readlink d7/l1c 0 2024-05-12T04:01:28.257 INFO:tasks.workunit.client.0.smithi032.stdout:6/604: readlink d7/d9/d48/l6a 0 2024-05-12T04:01:28.258 INFO:tasks.workunit.client.1.smithi033.stdout:1/731: mknod d11/cf4 0 2024-05-12T04:01:28.259 INFO:tasks.workunit.client.0.smithi032.stdout:6/605: dread - d7/de/fcb zero size 2024-05-12T04:01:28.263 INFO:tasks.workunit.client.0.smithi032.stdout:2/674: dwrite d1/d34/d96/d8a/f9b [0,4194304] 0 2024-05-12T04:01:28.264 INFO:tasks.workunit.client.1.smithi033.stdout:8/798: mkdir d1/d16/d27/d4d/d51/d8e/d94/df2 0 2024-05-12T04:01:28.265 INFO:tasks.workunit.client.1.smithi033.stdout:1/732: dwrite d11/d17/d27/d36/d3f/d69/fe8 [0,4194304] 0 2024-05-12T04:01:28.267 INFO:tasks.workunit.client.1.smithi033.stdout:5/746: write d3/d94/d54/d57/d70/da3/dbe/f30 [955531,28873] 0 2024-05-12T04:01:28.267 INFO:tasks.workunit.client.1.smithi033.stdout:1/733: chown d11/d17/d27/d36/d3f/d69/d87/l8a 2210112 1 2024-05-12T04:01:28.268 INFO:tasks.workunit.client.0.smithi032.stdout:1/650: fsync d1/f47 0 2024-05-12T04:01:28.270 INFO:tasks.workunit.client.1.smithi033.stdout:4/824: dread dd/f1b [4194304,4194304] 0 2024-05-12T04:01:28.271 INFO:tasks.workunit.client.1.smithi033.stdout:3/802: mkdir db/d17/d25/d32/d3c/d44/d117 0 2024-05-12T04:01:28.273 INFO:tasks.workunit.client.1.smithi033.stdout:0/861: symlink d6/dc/d58/d6a/d94/d125/d127/l129 0 2024-05-12T04:01:28.275 INFO:tasks.workunit.client.1.smithi033.stdout:1/734: dwrite d11/d1c/f64 [0,4194304] 0 2024-05-12T04:01:28.283 INFO:tasks.workunit.client.0.smithi032.stdout:3/698: creat d15/ff6 x:0 0 0 2024-05-12T04:01:28.283 INFO:tasks.workunit.client.1.smithi033.stdout:6/838: write d4/d1a/d6c/f9c [4316471,81772] 0 2024-05-12T04:01:28.291 INFO:tasks.workunit.client.0.smithi032.stdout:6/606: fsync d7/d9/d48/d67/d63/d64/d94/fce 0 2024-05-12T04:01:28.291 INFO:tasks.workunit.client.1.smithi033.stdout:3/803: creat db/d42/f118 x:0 0 0 2024-05-12T04:01:28.292 INFO:tasks.workunit.client.0.smithi032.stdout:8/698: write d0/d9/d1b/d27/d59/f6d [418442,102946] 0 2024-05-12T04:01:28.293 INFO:tasks.workunit.client.0.smithi032.stdout:3/699: write d15/d24/d2a/d8c/db1/fdb [770326,63826] 0 2024-05-12T04:01:28.294 INFO:tasks.workunit.client.1.smithi033.stdout:2/836: creat dc/d17/d27/d68/d9f/f10a x:0 0 0 2024-05-12T04:01:28.300 INFO:tasks.workunit.client.1.smithi033.stdout:0/862: dread d6/dc/d58/f59 [0,4194304] 0 2024-05-12T04:01:28.302 INFO:tasks.workunit.client.0.smithi032.stdout:2/675: dwrite d1/d34/d3e/d67/fba [0,4194304] 0 2024-05-12T04:01:28.302 INFO:tasks.workunit.client.0.smithi032.stdout:6/607: dwrite d7/de/fcb [0,4194304] 0 2024-05-12T04:01:28.302 INFO:tasks.workunit.client.1.smithi033.stdout:8/799: symlink d1/d16/d27/d4d/d82/dd9/d36/lf3 0 2024-05-12T04:01:28.339 INFO:tasks.workunit.client.0.smithi032.stdout:4/579: sync 2024-05-12T04:01:28.341 INFO:tasks.workunit.client.1.smithi033.stdout:5/747: symlink d3/d94/d54/d57/d70/da3/dbe/d16/lf3 0 2024-05-12T04:01:28.344 INFO:tasks.workunit.client.1.smithi033.stdout:6/839: rename d4/d43/d57/c111 to d4/d1a/d67/dc3/c113 0 2024-05-12T04:01:28.344 INFO:tasks.workunit.client.1.smithi033.stdout:1/735: write d11/d17/d27/d76/fa7 [2894726,19799] 0 2024-05-12T04:01:28.345 INFO:tasks.workunit.client.1.smithi033.stdout:6/840: readlink d4/l19 0 2024-05-12T04:01:28.346 INFO:tasks.workunit.client.1.smithi033.stdout:1/736: write d11/d17/d27/d76/f6e [879575,82260] 0 2024-05-12T04:01:28.351 INFO:tasks.workunit.client.1.smithi033.stdout:5/748: mknod d3/d94/d54/d57/d70/da3/dbe/d16/d45/d62/d7f/cf4 0 2024-05-12T04:01:28.352 INFO:tasks.workunit.client.1.smithi033.stdout:5/749: chown d3/d12/d83/ff0 444929 1 2024-05-12T04:01:28.354 INFO:tasks.workunit.client.1.smithi033.stdout:5/750: truncate d3/d94/fe1 714600 0 2024-05-12T04:01:28.355 INFO:tasks.workunit.client.1.smithi033.stdout:3/804: symlink db/d17/d25/d32/d3c/l119 0 2024-05-12T04:01:28.355 INFO:tasks.workunit.client.1.smithi033.stdout:5/751: stat d3/d94/d54/d57/d70/da3/dbe/d16/d45/d4b/f56 0 2024-05-12T04:01:28.357 INFO:tasks.workunit.client.1.smithi033.stdout:4/825: dread dd/d11/d5b/f86 [0,4194304] 0 2024-05-12T04:01:28.361 INFO:tasks.workunit.client.1.smithi033.stdout:8/800: mkdir d1/d16/d27/d4d/d82/dd9/df4 0 2024-05-12T04:01:28.362 INFO:tasks.workunit.client.1.smithi033.stdout:8/801: write d1/d4/f39 [3021755,103798] 0 2024-05-12T04:01:28.367 INFO:tasks.workunit.client.1.smithi033.stdout:5/752: mkdir d3/d12/d29/d2b/df5 0 2024-05-12T04:01:28.367 INFO:tasks.workunit.client.1.smithi033.stdout:1/737: dread d11/d17/d27/d76/d93/fc1 [0,4194304] 0 2024-05-12T04:01:28.368 INFO:tasks.workunit.client.1.smithi033.stdout:1/738: chown d11/d1c/ddf 12909 1 2024-05-12T04:01:28.369 INFO:tasks.workunit.client.1.smithi033.stdout:1/739: chown d11/d1c/ddf/c6a 53308925 1 2024-05-12T04:01:28.369 INFO:tasks.workunit.client.1.smithi033.stdout:4/826: dread - dd/d11/d75/dd4/fef zero size 2024-05-12T04:01:28.376 INFO:tasks.workunit.client.1.smithi033.stdout:5/753: symlink d3/d94/d54/d57/d70/da3/dbe/d8c/lf6 0 2024-05-12T04:01:28.376 INFO:tasks.workunit.client.1.smithi033.stdout:6/841: dwrite d4/d5/f2d [0,4194304] 0 2024-05-12T04:01:28.377 INFO:tasks.workunit.client.1.smithi033.stdout:2/837: write dc/d13/f15 [3447602,75127] 0 2024-05-12T04:01:28.379 INFO:tasks.workunit.client.1.smithi033.stdout:2/838: write dc/f3f [1952058,49158] 0 2024-05-12T04:01:28.379 INFO:tasks.workunit.client.1.smithi033.stdout:1/740: truncate d11/d17/d9e/f9f 576690 0 2024-05-12T04:01:28.380 INFO:tasks.workunit.client.1.smithi033.stdout:0/863: sync 2024-05-12T04:01:28.383 INFO:tasks.workunit.client.1.smithi033.stdout:8/802: link d1/d16/d27/d4d/d51/fe2 d1/d16/d27/d4d/d51/ff5 0 2024-05-12T04:01:28.386 INFO:tasks.workunit.client.1.smithi033.stdout:6/842: mknod d4/d43/d57/dc4/c114 0 2024-05-12T04:01:28.389 INFO:tasks.workunit.client.1.smithi033.stdout:2/839: dwrite dc/d13/d1c/d98/fb5 [0,4194304] 0 2024-05-12T04:01:28.390 INFO:tasks.workunit.client.1.smithi033.stdout:3/805: dwrite db/f90 [0,4194304] 0 2024-05-12T04:01:28.398 INFO:tasks.workunit.client.1.smithi033.stdout:0/864: dread - d6/dc/d96/d9d/fd4 zero size 2024-05-12T04:01:28.399 INFO:tasks.workunit.client.1.smithi033.stdout:1/741: unlink d11/d17/d27/d36/d3f/d69/d87/da4/fa8 0 2024-05-12T04:01:28.405 INFO:tasks.workunit.client.1.smithi033.stdout:6/843: mkdir d4/d6e/dbb/d115 0 2024-05-12T04:01:28.406 INFO:tasks.workunit.client.0.smithi032.stdout:3/700: creat d15/d1b/d43/dd0/ff7 x:0 0 0 2024-05-12T04:01:28.413 INFO:tasks.workunit.client.0.smithi032.stdout:2/676: stat d1/d34/d96/d41/d52/c97 0 2024-05-12T04:01:28.415 INFO:tasks.workunit.client.1.smithi033.stdout:4/827: dwrite dd/d11/f2f [0,4194304] 0 2024-05-12T04:01:28.416 INFO:tasks.workunit.client.0.smithi032.stdout:4/580: creat d0/d3/df/fa5 x:0 0 0 2024-05-12T04:01:28.419 INFO:tasks.workunit.client.1.smithi033.stdout:1/742: dread - d11/d17/d27/d36/d3f/d69/d87/da4/fba zero size 2024-05-12T04:01:28.427 INFO:tasks.workunit.client.1.smithi033.stdout:2/840: symlink dc/d13/d1c/l10b 0 2024-05-12T04:01:28.429 INFO:tasks.workunit.client.0.smithi032.stdout:2/677: mknod d1/d34/d96/d41/d2c/ce0 0 2024-05-12T04:01:28.429 INFO:tasks.workunit.client.1.smithi033.stdout:6/844: dread d4/d1a/d4e/f96 [0,4194304] 0 2024-05-12T04:01:28.431 INFO:tasks.workunit.client.0.smithi032.stdout:1/651: getdents d1/d7/d49/d79 0 2024-05-12T04:01:28.433 INFO:tasks.workunit.client.0.smithi032.stdout:1/652: chown d1/c2a 532594391 1 2024-05-12T04:01:28.435 INFO:tasks.workunit.client.0.smithi032.stdout:6/608: dwrite d7/f32 [0,4194304] 0 2024-05-12T04:01:28.436 INFO:tasks.workunit.client.0.smithi032.stdout:8/699: write d0/d9/d1b/d66/d75/f7c [923998,123856] 0 2024-05-12T04:01:28.437 INFO:tasks.workunit.client.1.smithi033.stdout:2/841: dwrite dc/d17/d27/d68/d9f/f10a [0,4194304] 0 2024-05-12T04:01:28.438 INFO:tasks.workunit.client.1.smithi033.stdout:5/754: dwrite d3/d94/d54/d57/d70/da3/dbe/d16/d78/fe7 [0,4194304] 0 2024-05-12T04:01:28.438 INFO:tasks.workunit.client.1.smithi033.stdout:1/743: unlink d11/d17/d27/d36/f3d 0 2024-05-12T04:01:28.440 INFO:tasks.workunit.client.0.smithi032.stdout:2/678: mkdir d1/d34/d96/d41/d58/de1 0 2024-05-12T04:01:28.441 INFO:tasks.workunit.client.1.smithi033.stdout:2/842: fsync dc/d17/d32/d84/f89 0 2024-05-12T04:01:28.443 INFO:tasks.workunit.client.0.smithi032.stdout:2/679: chown d1/d13/d6b/l7c 79765278 1 2024-05-12T04:01:28.445 INFO:tasks.workunit.client.0.smithi032.stdout:4/581: rename d0/d3/d12/l2f to d0/d3/d12/d1d/d63/d87/la6 0 2024-05-12T04:01:28.445 INFO:tasks.workunit.client.0.smithi032.stdout:8/700: dwrite d0/f68 [0,4194304] 0 2024-05-12T04:01:28.446 INFO:tasks.workunit.client.1.smithi033.stdout:4/828: write dd/d11/d5b/d84/faf [851996,36239] 0 2024-05-12T04:01:28.449 INFO:tasks.workunit.client.0.smithi032.stdout:8/701: fdatasync d0/d9/d1b/d33/f45 0 2024-05-12T04:01:28.450 INFO:tasks.workunit.client.1.smithi033.stdout:6/845: mkdir d4/d1a/d67/d6f/daf/d116 0 2024-05-12T04:01:28.452 INFO:tasks.workunit.client.1.smithi033.stdout:3/806: dwrite db/d17/d3a/f3b [0,4194304] 0 2024-05-12T04:01:28.452 INFO:tasks.workunit.client.0.smithi032.stdout:1/653: rmdir d1/d25/da5/db5 39 2024-05-12T04:01:28.455 INFO:tasks.workunit.client.1.smithi033.stdout:0/865: mknod d6/db/d2c/d73/daf/dc3/d110/c12a 0 2024-05-12T04:01:28.455 INFO:tasks.workunit.client.0.smithi032.stdout:3/701: dwrite d15/d24/d93/f70 [0,4194304] 0 2024-05-12T04:01:28.456 INFO:tasks.workunit.client.1.smithi033.stdout:2/843: dwrite dc/d17/d27/d68/d9f/d5e/f77 [0,4194304] 0 2024-05-12T04:01:28.458 INFO:tasks.workunit.client.0.smithi032.stdout:4/582: mkdir d0/d3/d31/d80/d84/da7 0 2024-05-12T04:01:28.463 INFO:tasks.workunit.client.0.smithi032.stdout:8/702: mkdir d0/d9/d1b/d33/d73/db7/df1 0 2024-05-12T04:01:28.465 INFO:tasks.workunit.client.1.smithi033.stdout:1/744: dread - d11/d1c/d43/fa1 zero size 2024-05-12T04:01:28.465 INFO:tasks.workunit.client.1.smithi033.stdout:5/755: creat d3/d12/d25/db0/ff7 x:0 0 0 2024-05-12T04:01:28.466 INFO:tasks.workunit.client.1.smithi033.stdout:8/803: getdents d1/d16/d29/d63/da3 0 2024-05-12T04:01:28.467 INFO:tasks.workunit.client.0.smithi032.stdout:1/654: rename d1/d7/c71 to d1/d6a/d72/cd2 0 2024-05-12T04:01:28.470 INFO:tasks.workunit.client.0.smithi032.stdout:2/680: unlink d1/cca 0 2024-05-12T04:01:28.473 INFO:tasks.workunit.client.1.smithi033.stdout:4/829: creat dd/d11/d5b/dd1/ffd x:0 0 0 2024-05-12T04:01:28.474 INFO:tasks.workunit.client.1.smithi033.stdout:5/756: chown d3/d12/d29/f38 21 1 2024-05-12T04:01:28.474 INFO:tasks.workunit.client.1.smithi033.stdout:0/866: readlink d6/l38 0 2024-05-12T04:01:28.475 INFO:tasks.workunit.client.0.smithi032.stdout:8/703: symlink d0/d97/lf2 0 2024-05-12T04:01:28.478 INFO:tasks.workunit.client.1.smithi033.stdout:8/804: dread - d1/d16/d27/d7c/f89 zero size 2024-05-12T04:01:28.483 INFO:tasks.workunit.client.0.smithi032.stdout:8/704: dwrite d0/d9/d13/d21/d25/db5/fdc [0,4194304] 0 2024-05-12T04:01:28.485 INFO:tasks.workunit.client.0.smithi032.stdout:4/583: getdents d0/d3/d31/d80/d84/da7 0 2024-05-12T04:01:28.487 INFO:tasks.workunit.client.0.smithi032.stdout:1/655: creat d1/d7/d30/d61/dc8/fd3 x:0 0 0 2024-05-12T04:01:28.488 INFO:tasks.workunit.client.0.smithi032.stdout:2/681: mknod d1/d34/d96/d41/d58/de1/ce2 0 2024-05-12T04:01:28.488 INFO:tasks.workunit.client.1.smithi033.stdout:5/757: mknod d3/d94/d54/d57/d70/da3/cf8 0 2024-05-12T04:01:28.490 INFO:tasks.workunit.client.1.smithi033.stdout:3/807: mkdir db/d17/d3e/d65/dfb/d11a 0 2024-05-12T04:01:28.491 INFO:tasks.workunit.client.0.smithi032.stdout:3/702: link d15/d20/d22/fb6 d15/d33/d3f/d75/ff8 0 2024-05-12T04:01:28.493 INFO:tasks.workunit.client.0.smithi032.stdout:4/584: creat d0/d3/d12/d1d/d63/d77/fa8 x:0 0 0 2024-05-12T04:01:28.495 INFO:tasks.workunit.client.0.smithi032.stdout:1/656: creat d1/d7/d30/d61/dc8/fd4 x:0 0 0 2024-05-12T04:01:28.495 INFO:tasks.workunit.client.0.smithi032.stdout:8/705: read d0/d9/d13/f29 [3662327,34183] 0 2024-05-12T04:01:28.498 INFO:tasks.workunit.client.0.smithi032.stdout:8/706: readlink d0/d9/d1b/d27/d88/d56/d63/lba 0 2024-05-12T04:01:28.499 INFO:tasks.workunit.client.0.smithi032.stdout:3/703: dwrite d15/d20/d22/d4f/f53 [0,4194304] 0 2024-05-12T04:01:28.500 INFO:tasks.workunit.client.0.smithi032.stdout:8/707: dread - d0/d9/d13/d21/fd3 zero size 2024-05-12T04:01:28.501 INFO:tasks.workunit.client.0.smithi032.stdout:3/704: write d15/d20/d8b/d9b/dbb/ff0 [772401,69684] 0 2024-05-12T04:01:28.505 INFO:tasks.workunit.client.0.smithi032.stdout:4/585: rename d0/d3/f49 to d0/d3/d31/d80/d84/fa9 0 2024-05-12T04:01:28.506 INFO:tasks.workunit.client.0.smithi032.stdout:5/758: sync 2024-05-12T04:01:28.507 INFO:tasks.workunit.client.0.smithi032.stdout:1/657: creat d1/d7/d30/d35/d77/fd5 x:0 0 0 2024-05-12T04:01:28.509 INFO:tasks.workunit.client.0.smithi032.stdout:2/682: link d1/d34/d96/d41/d2c/f59 d1/d34/d96/d41/d52/fe3 0 2024-05-12T04:01:28.510 INFO:tasks.workunit.client.0.smithi032.stdout:2/683: fdatasync d1/d13/f4c 0 2024-05-12T04:01:28.510 INFO:tasks.workunit.client.0.smithi032.stdout:2/684: chown d1/d34/d95 6654 1 2024-05-12T04:01:28.514 INFO:tasks.workunit.client.0.smithi032.stdout:3/705: mknod d15/dc2/cf9 0 2024-05-12T04:01:28.517 INFO:tasks.workunit.client.0.smithi032.stdout:5/759: creat d4/d7/d14/f10c x:0 0 0 2024-05-12T04:01:28.519 INFO:tasks.workunit.client.0.smithi032.stdout:1/658: truncate d1/d7/f5d 1308864 0 2024-05-12T04:01:28.520 INFO:tasks.workunit.client.0.smithi032.stdout:1/659: chown d1/d7/d30/f28 1782066713 1 2024-05-12T04:01:28.521 INFO:tasks.workunit.client.0.smithi032.stdout:1/660: chown d1/d6a/fb9 53 1 2024-05-12T04:01:28.524 INFO:tasks.workunit.client.0.smithi032.stdout:8/708: mkdir d0/d9/d1b/d27/d88/dd7/ddf/df3 0 2024-05-12T04:01:28.530 INFO:tasks.workunit.client.0.smithi032.stdout:5/760: dread d4/d7/d14/d39/fcf [4194304,4194304] 0 2024-05-12T04:01:28.533 INFO:tasks.workunit.client.0.smithi032.stdout:5/761: chown d4/d7/d14/d26/fa6 155280 1 2024-05-12T04:01:28.533 INFO:tasks.workunit.client.0.smithi032.stdout:1/661: truncate d1/d7/f78 1752307 0 2024-05-12T04:01:28.534 INFO:tasks.workunit.client.0.smithi032.stdout:6/609: dwrite d7/fa [0,4194304] 0 2024-05-12T04:01:28.535 INFO:tasks.workunit.client.1.smithi033.stdout:6/846: write d4/d1a/d6c/f85 [1462156,4736] 0 2024-05-12T04:01:28.536 INFO:tasks.workunit.client.0.smithi032.stdout:4/586: creat d0/d3/d12/faa x:0 0 0 2024-05-12T04:01:28.537 INFO:tasks.workunit.client.0.smithi032.stdout:5/762: chown d4/d7/d14/d26/d33/d5f/d7f/l101 33320033 1 2024-05-12T04:01:28.540 INFO:tasks.workunit.client.0.smithi032.stdout:8/709: rmdir d0/d9/d1b/d27/d88/d56/d63/d8f/d9a/d28 39 2024-05-12T04:01:28.542 INFO:tasks.workunit.client.0.smithi032.stdout:8/710: write d0/f68 [3704976,111137] 0 2024-05-12T04:01:28.543 INFO:tasks.workunit.client.0.smithi032.stdout:1/662: dwrite d1/d25/fb8 [0,4194304] 0 2024-05-12T04:01:28.543 INFO:tasks.workunit.client.0.smithi032.stdout:8/711: chown d0/d97 186818916 1 2024-05-12T04:01:28.548 INFO:tasks.workunit.client.0.smithi032.stdout:3/706: creat d15/ffa x:0 0 0 2024-05-12T04:01:28.550 INFO:tasks.workunit.client.0.smithi032.stdout:3/707: chown d15/ff6 11373 1 2024-05-12T04:01:28.555 INFO:tasks.workunit.client.0.smithi032.stdout:8/712: dread d0/d9/d1b/d27/d88/d56/fae [0,4194304] 0 2024-05-12T04:01:28.561 INFO:tasks.workunit.client.0.smithi032.stdout:2/685: rename d1/d34/d96/d41/d39/fd2 to d1/fe4 0 2024-05-12T04:01:28.562 INFO:tasks.workunit.client.0.smithi032.stdout:2/686: chown d1/d13/d70/f91 16537340 1 2024-05-12T04:01:28.562 INFO:tasks.workunit.client.0.smithi032.stdout:2/687: dread - d1/d4d/d5e/fdc zero size 2024-05-12T04:01:28.566 INFO:tasks.workunit.client.0.smithi032.stdout:1/663: symlink d1/d7/d30/d18/d93/dc1/dc3/ld6 0 2024-05-12T04:01:28.567 INFO:tasks.workunit.client.0.smithi032.stdout:3/708: symlink d15/d20/d8b/lfb 0 2024-05-12T04:01:28.568 INFO:tasks.workunit.client.0.smithi032.stdout:3/709: readlink d15/d24/d2a/l39 0 2024-05-12T04:01:28.574 INFO:tasks.workunit.client.0.smithi032.stdout:4/587: read d0/d3/df/f29 [3606646,115766] 0 2024-05-12T04:01:28.574 INFO:tasks.workunit.client.0.smithi032.stdout:5/763: mknod d4/d7/d14/d26/d33/d72/dc9/dec/c10d 0 2024-05-12T04:01:28.576 INFO:tasks.workunit.client.0.smithi032.stdout:8/713: unlink d0/d9/d1b/d27/f69 0 2024-05-12T04:01:28.584 INFO:tasks.workunit.client.0.smithi032.stdout:2/688: dread d1/d34/d4a/f6d [0,4194304] 0 2024-05-12T04:01:28.586 INFO:tasks.workunit.client.0.smithi032.stdout:5/764: dread d4/d7/d14/d26/d33/ff4 [0,4194304] 0 2024-05-12T04:01:28.587 INFO:tasks.workunit.client.1.smithi033.stdout:2/844: creat dc/d17/d27/d2f/d49/df2/f10c x:0 0 0 2024-05-12T04:01:28.588 INFO:tasks.workunit.client.0.smithi032.stdout:5/765: write d4/d7/d14/d26/d33/d4a/d38/dd2/fd9 [4749603,43400] 0 2024-05-12T04:01:28.589 INFO:tasks.workunit.client.0.smithi032.stdout:3/710: unlink d15/d20/d22/d7a/lb9 0 2024-05-12T04:01:28.590 INFO:tasks.workunit.client.0.smithi032.stdout:5/766: read - d4/d7/d14/d26/f107 zero size 2024-05-12T04:01:28.596 INFO:tasks.workunit.client.0.smithi032.stdout:4/588: mknod d0/d3/d12/d1d/d63/d87/d95/cab 0 2024-05-12T04:01:28.597 INFO:tasks.workunit.client.1.smithi033.stdout:0/867: truncate d6/f9e 515246 0 2024-05-12T04:01:28.600 INFO:tasks.workunit.client.0.smithi032.stdout:1/664: dread d1/d6a/f7a [0,4194304] 0 2024-05-12T04:01:28.600 INFO:tasks.workunit.client.0.smithi032.stdout:5/767: dwrite d4/d7/d14/d26/d33/d72/dc9/ff0 [0,4194304] 0 2024-05-12T04:01:28.603 INFO:tasks.workunit.client.0.smithi032.stdout:5/768: fsync d4/d7/d14/d26/d33/fe7 0 2024-05-12T04:01:28.603 INFO:tasks.workunit.client.1.smithi033.stdout:4/830: link dd/f3b dd/d7a/d87/dd9/ffe 0 2024-05-12T04:01:28.604 INFO:tasks.workunit.client.0.smithi032.stdout:2/689: symlink d1/d34/d96/d41/d2c/le5 0 2024-05-12T04:01:28.611 INFO:tasks.workunit.client.1.smithi033.stdout:2/845: mknod dc/d17/d27/d68/dd1/c10d 0 2024-05-12T04:01:28.613 INFO:tasks.workunit.client.0.smithi032.stdout:6/610: write d7/d9/d48/d67/d8d/dbc/f68 [4594550,16386] 0 2024-05-12T04:01:28.614 INFO:tasks.workunit.client.0.smithi032.stdout:3/711: dread d15/f19 [0,4194304] 0 2024-05-12T04:01:28.615 INFO:tasks.workunit.client.0.smithi032.stdout:3/712: fsync d15/d20/d26/f52 0 2024-05-12T04:01:28.617 INFO:tasks.workunit.client.1.smithi033.stdout:0/868: dread - d6/dc/d58/fab zero size 2024-05-12T04:01:28.617 INFO:tasks.workunit.client.1.smithi033.stdout:0/869: fdatasync d6/d11/dc7/dd9/d10b/f115 0 2024-05-12T04:01:28.618 INFO:tasks.workunit.client.0.smithi032.stdout:1/665: symlink d1/d7/ld7 0 2024-05-12T04:01:28.621 INFO:tasks.workunit.client.0.smithi032.stdout:5/769: mkdir d4/d7/d14/d39/d2b/d6e/d10e 0 2024-05-12T04:01:28.622 INFO:tasks.workunit.client.1.smithi033.stdout:5/758: creat d3/d12/d83/deb/ff9 x:0 0 0 2024-05-12T04:01:28.625 INFO:tasks.workunit.client.1.smithi033.stdout:3/808: creat db/d17/d3a/d67/d76/da0/de7/f11b x:0 0 0 2024-05-12T04:01:28.628 INFO:tasks.workunit.client.0.smithi032.stdout:4/589: mkdir d0/d3/df/dac 0 2024-05-12T04:01:28.628 INFO:tasks.workunit.client.0.smithi032.stdout:4/590: chown d0/d3/d12/d8d/da1 2311269 1 2024-05-12T04:01:28.629 INFO:tasks.workunit.client.1.smithi033.stdout:2/846: creat dc/d13/d65/f10e x:0 0 0 2024-05-12T04:01:28.630 INFO:tasks.workunit.client.1.smithi033.stdout:2/847: write dc/fe3 [4484727,82952] 0 2024-05-12T04:01:28.634 INFO:tasks.workunit.client.1.smithi033.stdout:0/870: creat d6/db/f12b x:0 0 0 2024-05-12T04:01:28.634 INFO:tasks.workunit.client.0.smithi032.stdout:6/611: creat d7/d9/fd0 x:0 0 0 2024-05-12T04:01:28.634 INFO:tasks.workunit.client.0.smithi032.stdout:8/714: link d0/d9/d1b/d27/d4c/d4e/d90/faf d0/d9/d1b/d27/d88/dd7/ff4 0 2024-05-12T04:01:28.635 INFO:tasks.workunit.client.0.smithi032.stdout:8/715: stat d0/d9/d1b/d33/d73/ca8 0 2024-05-12T04:01:28.637 INFO:tasks.workunit.client.0.smithi032.stdout:3/713: creat d15/d20/d22/d7a/dbe/ffc x:0 0 0 2024-05-12T04:01:28.640 INFO:tasks.workunit.client.0.smithi032.stdout:1/666: fdatasync d1/d6a/d72/f9e 0 2024-05-12T04:01:28.642 INFO:tasks.workunit.client.1.smithi033.stdout:6/847: link d4/d5/c59 d4/d1a/d67/dc3/c117 0 2024-05-12T04:01:28.646 INFO:tasks.workunit.client.1.smithi033.stdout:2/848: creat dc/d13/d39/f10f x:0 0 0 2024-05-12T04:01:28.649 INFO:tasks.workunit.client.0.smithi032.stdout:9/549: sync 2024-05-12T04:01:28.649 INFO:tasks.workunit.client.0.smithi032.stdout:9/550: chown d0/d3/d74/f4e 1316587 1 2024-05-12T04:01:28.650 INFO:tasks.workunit.client.0.smithi032.stdout:4/591: truncate d0/d3/df/d4e/f69 78823 0 2024-05-12T04:01:28.650 INFO:tasks.workunit.client.1.smithi033.stdout:0/871: fdatasync d6/db/d2c/d73/daf/dc3/d110/f11f 0 2024-05-12T04:01:28.653 INFO:tasks.workunit.client.1.smithi033.stdout:1/745: dwrite d11/d17/d27/d36/f80 [0,4194304] 0 2024-05-12T04:01:28.654 INFO:tasks.workunit.client.0.smithi032.stdout:8/716: mknod d0/d9/d1b/d66/d75/cf5 0 2024-05-12T04:01:28.655 INFO:tasks.workunit.client.0.smithi032.stdout:3/714: dread - d15/d20/d26/d6a/d6e/d7d/d9a/fa3 zero size 2024-05-12T04:01:28.656 INFO:tasks.workunit.client.0.smithi032.stdout:3/715: dread - d15/d20/ff2 zero size 2024-05-12T04:01:28.657 INFO:tasks.workunit.client.1.smithi033.stdout:1/746: truncate d11/d17/d27/d36/fec 33945 0 2024-05-12T04:01:28.657 INFO:tasks.workunit.client.0.smithi032.stdout:4/592: dread d0/d3/d31/d80/f82 [0,4194304] 0 2024-05-12T04:01:28.659 INFO:tasks.workunit.client.1.smithi033.stdout:8/805: dwrite d1/d4/db2/fbb [0,4194304] 0 2024-05-12T04:01:28.664 INFO:tasks.workunit.client.0.smithi032.stdout:5/770: mknod d4/d7/d14/d39/d1b/da4/dd0/c10f 0 2024-05-12T04:01:28.668 INFO:tasks.workunit.client.1.smithi033.stdout:0/872: mkdir d6/d11/dc7/d12c 0 2024-05-12T04:01:28.671 INFO:tasks.workunit.client.0.smithi032.stdout:9/551: creat d0/d3/d6/d7/d2a/fb3 x:0 0 0 2024-05-12T04:01:28.672 INFO:tasks.workunit.client.1.smithi033.stdout:3/809: creat db/f11c x:0 0 0 2024-05-12T04:01:28.675 INFO:tasks.workunit.client.0.smithi032.stdout:3/716: symlink d15/d20/d8b/lfd 0 2024-05-12T04:01:28.678 INFO:tasks.workunit.client.1.smithi033.stdout:8/806: symlink d1/d4/d46/dca/de8/lf6 0 2024-05-12T04:01:28.679 INFO:tasks.workunit.client.1.smithi033.stdout:8/807: chown d1/d16/d27/d4d/d82/dd9/d36/d3c/da7 0 1 2024-05-12T04:01:28.691 INFO:tasks.workunit.client.0.smithi032.stdout:8/717: dread d0/d9/d1b/d33/f45 [4194304,4194304] 0 2024-05-12T04:01:28.698 INFO:tasks.workunit.client.0.smithi032.stdout:0/693: sync 2024-05-12T04:01:28.698 INFO:tasks.workunit.client.0.smithi032.stdout:7/651: sync 2024-05-12T04:01:28.700 INFO:tasks.workunit.client.0.smithi032.stdout:7/652: write dd/f1d [1284214,29279] 0 2024-05-12T04:01:28.702 INFO:tasks.workunit.client.0.smithi032.stdout:7/653: fdatasync dd/d1b/d1e/d45/d55/fa5 0 2024-05-12T04:01:28.718 INFO:tasks.workunit.client.0.smithi032.stdout:0/694: dread df/f51 [0,4194304] 0 2024-05-12T04:01:28.731 INFO:tasks.workunit.client.0.smithi032.stdout:5/771: truncate d4/d7/d14/d39/d1b/da4/fc4 502979 0 2024-05-12T04:01:28.790 INFO:tasks.workunit.client.1.smithi033.stdout:4/831: write dd/d7a/d87/d7e/fe5 [617654,106206] 0 2024-05-12T04:01:28.791 INFO:tasks.workunit.client.1.smithi033.stdout:0/873: chown d6/db/d2c/d73/daf/da4/ca9 13742 1 2024-05-12T04:01:28.793 INFO:tasks.workunit.client.1.smithi033.stdout:5/759: write d3/f7d [3650936,47700] 0 2024-05-12T04:01:28.794 INFO:tasks.workunit.client.1.smithi033.stdout:3/810: creat db/d17/d3e/dd7/dd0/f11d x:0 0 0 2024-05-12T04:01:28.795 INFO:tasks.workunit.client.0.smithi032.stdout:4/593: creat d0/d3/d31/d80/d84/da7/fad x:0 0 0 2024-05-12T04:01:28.796 INFO:tasks.workunit.client.0.smithi032.stdout:4/594: readlink d0/d3/d12/d1d/d63/l96 0 2024-05-12T04:01:28.798 INFO:tasks.workunit.client.0.smithi032.stdout:2/690: write d1/d34/d3e/d67/f83 [2939611,67862] 0 2024-05-12T04:01:28.806 INFO:tasks.workunit.client.0.smithi032.stdout:8/718: creat d0/d9/d1b/d33/d73/db7/ff6 x:0 0 0 2024-05-12T04:01:28.807 INFO:tasks.workunit.client.1.smithi033.stdout:4/832: mkdir dd/d69/dff 0 2024-05-12T04:01:28.809 INFO:tasks.workunit.client.0.smithi032.stdout:1/667: write d1/d7/d30/d35/f38 [954243,11457] 0 2024-05-12T04:01:28.812 INFO:tasks.workunit.client.1.smithi033.stdout:0/874: write d6/dc/d58/f88 [2060155,59158] 0 2024-05-12T04:01:28.814 INFO:tasks.workunit.client.1.smithi033.stdout:6/848: dwrite d4/dad/fce [0,4194304] 0 2024-05-12T04:01:28.816 INFO:tasks.workunit.client.1.smithi033.stdout:4/833: dread dd/d7a/d87/d7e/da7/fce [4194304,4194304] 0 2024-05-12T04:01:28.818 INFO:tasks.workunit.client.1.smithi033.stdout:1/747: creat d11/d17/d38/ff5 x:0 0 0 2024-05-12T04:01:28.819 INFO:tasks.workunit.client.1.smithi033.stdout:1/748: chown d11/f4f 8875 1 2024-05-12T04:01:28.824 INFO:tasks.workunit.client.0.smithi032.stdout:0/695: truncate df/d10/f3c 1800255 0 2024-05-12T04:01:28.825 INFO:tasks.workunit.client.0.smithi032.stdout:7/654: dwrite dd/d7b/d9a/d9c/db9/d75/fad [0,4194304] 0 2024-05-12T04:01:28.826 INFO:tasks.workunit.client.1.smithi033.stdout:8/808: rename d1/d16/d27/d4d/d82/dd9/d36/d3c/d64/fa0 to d1/d16/d27/d4d/d82/dd9/d36/d3c/d47/ff7 0 2024-05-12T04:01:28.827 INFO:tasks.workunit.client.1.smithi033.stdout:8/809: write d1/d16/d27/d4d/d82/dd9/d36/f50 [3490697,85751] 0 2024-05-12T04:01:28.836 INFO:tasks.workunit.client.0.smithi032.stdout:9/552: mknod d0/d3/d6/d7/da/cb4 0 2024-05-12T04:01:28.836 INFO:tasks.workunit.client.1.smithi033.stdout:2/849: write dc/d17/d27/d2f/d49/d59/f80 [887972,118594] 0 2024-05-12T04:01:28.843 INFO:tasks.workunit.client.1.smithi033.stdout:2/850: dwrite f7 [0,4194304] 0 2024-05-12T04:01:28.846 INFO:tasks.workunit.client.1.smithi033.stdout:0/875: creat d6/db/dd6/f12d x:0 0 0 2024-05-12T04:01:28.849 INFO:tasks.workunit.client.0.smithi032.stdout:6/612: rmdir d7/d9/d48/d67/d8d/dbc 39 2024-05-12T04:01:28.851 INFO:tasks.workunit.client.0.smithi032.stdout:6/613: write d7/d9/d48/d67/d25/f8f [633777,130130] 0 2024-05-12T04:01:28.868 INFO:tasks.workunit.client.0.smithi032.stdout:0/696: symlink df/d30/d32/d81/le7 0 2024-05-12T04:01:28.868 INFO:tasks.workunit.client.0.smithi032.stdout:0/697: readlink l4 0 2024-05-12T04:01:28.874 INFO:tasks.workunit.client.0.smithi032.stdout:9/553: dread d0/d3/d6/d7/d2a/d46/d13/f7c [0,4194304] 0 2024-05-12T04:01:28.882 INFO:tasks.workunit.client.1.smithi033.stdout:2/851: link dc/d3b/d88/fd4 dc/d17/d27/d68/f110 0 2024-05-12T04:01:28.885 INFO:tasks.workunit.client.0.smithi032.stdout:9/554: dwrite d0/d3/d74/f4e [0,4194304] 0 2024-05-12T04:01:28.886 INFO:tasks.workunit.client.0.smithi032.stdout:9/555: write d0/d3/d6/d7/f15 [3533,95295] 0 2024-05-12T04:01:28.887 INFO:tasks.workunit.client.1.smithi033.stdout:3/811: getdents db/d17/d25/d32/d53/df6 0 2024-05-12T04:01:28.894 INFO:tasks.workunit.client.1.smithi033.stdout:3/812: chown db/d17/d3a/d93/ld4 128 1 2024-05-12T04:01:28.900 INFO:tasks.workunit.client.1.smithi033.stdout:3/813: write db/d17/d3a/fb5 [22443,32673] 0 2024-05-12T04:01:28.903 INFO:tasks.workunit.client.0.smithi032.stdout:2/691: getdents d1/d4d 0 2024-05-12T04:01:28.907 INFO:tasks.workunit.client.0.smithi032.stdout:9/556: mknod d0/d3/d6/d79/cb5 0 2024-05-12T04:01:28.913 INFO:tasks.workunit.client.1.smithi033.stdout:3/814: link db/d17/d25/d32/d3c/d44/lbe db/d17/d25/d32/d3c/d3d/d47/d10a/l11e 0 2024-05-12T04:01:28.914 INFO:tasks.workunit.client.1.smithi033.stdout:3/815: write db/fd9 [1110325,108026] 0 2024-05-12T04:01:28.915 INFO:tasks.workunit.client.1.smithi033.stdout:3/816: stat db/d17/d3a/d108/de9/ff3 0 2024-05-12T04:01:28.919 INFO:tasks.workunit.client.0.smithi032.stdout:6/614: link d7/c1d d7/d9/d5c/d86/cd1 0 2024-05-12T04:01:28.919 INFO:tasks.workunit.client.0.smithi032.stdout:0/698: getdents df/d10/d3b/dba 0 2024-05-12T04:01:28.920 INFO:tasks.workunit.client.0.smithi032.stdout:2/692: mknod d1/d34/d3e/d67/dda/ce6 0 2024-05-12T04:01:28.921 INFO:tasks.workunit.client.1.smithi033.stdout:3/817: fdatasync db/d17/d3e/dd7/dd0/fd2 0 2024-05-12T04:01:28.924 INFO:tasks.workunit.client.0.smithi032.stdout:0/699: mkdir df/d30/d32/d81/de8 0 2024-05-12T04:01:28.929 INFO:tasks.workunit.client.1.smithi033.stdout:3/818: rename db/d17/d3a/d8a/c101 to db/d17/d3e/d65/d7d/c11f 0 2024-05-12T04:01:28.931 INFO:tasks.workunit.client.0.smithi032.stdout:6/615: dread d7/d9/d48/faa [0,4194304] 0 2024-05-12T04:01:28.936 INFO:tasks.workunit.client.1.smithi033.stdout:3/819: mknod db/d17/d25/d32/d3c/d3d/c120 0 2024-05-12T04:01:28.946 INFO:tasks.workunit.client.1.smithi033.stdout:3/820: rmdir db/d17/d3a/d93/dc8/d100 0 2024-05-12T04:01:28.948 INFO:tasks.workunit.client.1.smithi033.stdout:3/821: write db/d17/d25/d32/d3c/d3d/d47/ff2 [338688,97233] 0 2024-05-12T04:01:28.968 INFO:tasks.workunit.client.1.smithi033.stdout:6/849: link d4/d43/ldc d4/d1a/d67/d6f/l118 0 2024-05-12T04:01:28.970 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:28 smithi032 ceph-mon[40008]: pgmap v42: 65 pgs: 65 active+clean; 3.0 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 92 MiB/s rd, 242 MiB/s wr, 495 op/s 2024-05-12T04:01:28.971 INFO:tasks.workunit.client.1.smithi033.stdout:6/850: fdatasync d4/d1a/d67/d6f/dac/ff6 0 2024-05-12T04:01:28.975 INFO:tasks.workunit.client.1.smithi033.stdout:6/851: rename d4/d43/d57/dc4/c114 to d4/d1a/d3c/dd9/de2/def/c119 0 2024-05-12T04:01:28.975 INFO:tasks.workunit.client.1.smithi033.stdout:6/852: dread - d4/d1a/f10b zero size 2024-05-12T04:01:28.991 INFO:tasks.workunit.client.1.smithi033.stdout:6/853: dread d4/d1a/d67/de5/ffe [0,4194304] 0 2024-05-12T04:01:28.991 INFO:tasks.workunit.client.1.smithi033.stdout:6/854: dread - d4/d1a/fed zero size 2024-05-12T04:01:28.995 INFO:tasks.workunit.client.0.smithi032.stdout:3/717: dwrite d15/d20/d26/d6a/d6e/d7d/fa1 [0,4194304] 0 2024-05-12T04:01:28.998 INFO:tasks.workunit.client.1.smithi033.stdout:5/760: dwrite d3/f77 [0,4194304] 0 2024-05-12T04:01:29.001 INFO:tasks.workunit.client.0.smithi032.stdout:3/718: creat d15/dec/ffe x:0 0 0 2024-05-12T04:01:29.005 INFO:tasks.workunit.client.1.smithi033.stdout:5/761: mknod d3/d12/d29/d2b/cfa 0 2024-05-12T04:01:29.013 INFO:tasks.workunit.client.1.smithi033.stdout:5/762: dread d3/d94/d54/d57/d70/da3/dbe/f88 [0,4194304] 0 2024-05-12T04:01:29.014 INFO:tasks.workunit.client.0.smithi032.stdout:4/595: write d0/d3/d12/d1d/f4c [3509310,128055] 0 2024-05-12T04:01:29.016 INFO:tasks.workunit.client.0.smithi032.stdout:5/772: dwrite d4/f22 [4194304,4194304] 0 2024-05-12T04:01:29.027 INFO:tasks.workunit.client.0.smithi032.stdout:5/773: mknod d4/d7/d14/d26/d33/d88/c110 0 2024-05-12T04:01:29.031 INFO:tasks.workunit.client.0.smithi032.stdout:4/596: dread d0/d3/d12/d1d/f5d [0,4194304] 0 2024-05-12T04:01:29.032 INFO:tasks.workunit.client.0.smithi032.stdout:5/774: mkdir d4/d7/d14/d26/d33/d72/d92/d111 0 2024-05-12T04:01:29.036 INFO:tasks.workunit.client.0.smithi032.stdout:4/597: write d0/d3/d31/d80/d84/f7c [4858065,8518] 0 2024-05-12T04:01:29.037 INFO:tasks.workunit.client.1.smithi033.stdout:4/834: unlink dd/f1b 0 2024-05-12T04:01:29.038 INFO:tasks.workunit.client.0.smithi032.stdout:5/775: symlink d4/d7/d14/d26/dc6/l112 0 2024-05-12T04:01:29.043 INFO:tasks.workunit.client.1.smithi033.stdout:4/835: dread dd/d69/f7d [0,4194304] 0 2024-05-12T04:01:29.045 INFO:tasks.workunit.client.1.smithi033.stdout:1/749: write d11/d1c/f54 [1049024,114511] 0 2024-05-12T04:01:29.049 INFO:tasks.workunit.client.1.smithi033.stdout:4/836: symlink dd/d69/l100 0 2024-05-12T04:01:29.050 INFO:tasks.workunit.client.1.smithi033.stdout:8/810: write d1/d16/fa1 [309648,125731] 0 2024-05-12T04:01:29.051 INFO:tasks.workunit.client.1.smithi033.stdout:8/811: fdatasync d1/d16/d27/d4d/d82/dd9/d36/dea/ff1 0 2024-05-12T04:01:29.052 INFO:tasks.workunit.client.1.smithi033.stdout:1/750: rename d11/d17/d38/ff5 to d11/d17/d27/d36/d3f/d69/d87/da4/ff6 0 2024-05-12T04:01:29.057 INFO:tasks.workunit.client.1.smithi033.stdout:0/876: write d6/d11/dc7/dd9/d10b/f108 [802671,3434] 0 2024-05-12T04:01:29.058 INFO:tasks.workunit.client.1.smithi033.stdout:1/751: mkdir d11/d17/d84/da3/df7 0 2024-05-12T04:01:29.059 INFO:tasks.workunit.client.1.smithi033.stdout:1/752: fdatasync d11/d17/d27/d36/d3f/d4a/f7a 0 2024-05-12T04:01:29.063 INFO:tasks.workunit.client.0.smithi032.stdout:4/598: dread d0/d3/d31/f35 [0,4194304] 0 2024-05-12T04:01:29.063 INFO:tasks.workunit.client.1.smithi033.stdout:1/753: creat d11/d17/d27/d36/d3f/d69/d87/da4/ff8 x:0 0 0 2024-05-12T04:01:29.064 INFO:tasks.workunit.client.1.smithi033.stdout:0/877: dwrite d6/d11/d57/f63 [0,4194304] 0 2024-05-12T04:01:29.066 INFO:tasks.workunit.client.1.smithi033.stdout:0/878: read - d6/d11/dc7/dd9/d10b/f123 zero size 2024-05-12T04:01:29.066 INFO:tasks.workunit.client.1.smithi033.stdout:1/754: chown d11/d17/d27/d36/d3f/d69/d87/da4/fda 10503 1 2024-05-12T04:01:29.069 INFO:tasks.workunit.client.1.smithi033.stdout:1/755: fsync d11/d17/f2e 0 2024-05-12T04:01:29.071 INFO:tasks.workunit.client.1.smithi033.stdout:0/879: creat d6/dc/d6e/f12e x:0 0 0 2024-05-12T04:01:29.072 INFO:tasks.workunit.client.0.smithi032.stdout:4/599: rmdir d0/d3/d12 39 2024-05-12T04:01:29.073 INFO:tasks.workunit.client.1.smithi033.stdout:4/837: dread dd/f4b [0,4194304] 0 2024-05-12T04:01:29.076 INFO:tasks.workunit.client.0.smithi032.stdout:9/557: write d0/d3/d6/d7/f7e [704964,43558] 0 2024-05-12T04:01:29.077 INFO:tasks.workunit.client.1.smithi033.stdout:4/838: symlink dd/d11/d5b/d84/l101 0 2024-05-12T04:01:29.078 INFO:tasks.workunit.client.1.smithi033.stdout:4/839: chown dd/d7a/f85 6225153 1 2024-05-12T04:01:29.079 INFO:tasks.workunit.client.1.smithi033.stdout:0/880: dwrite d6/db/dd6/f12d [0,4194304] 0 2024-05-12T04:01:29.082 INFO:tasks.workunit.client.0.smithi032.stdout:9/558: dread - d0/d3/d74/d51/f77 zero size 2024-05-12T04:01:29.084 INFO:tasks.workunit.client.1.smithi033.stdout:0/881: creat d6/dc/d96/f12f x:0 0 0 2024-05-12T04:01:29.089 INFO:tasks.workunit.client.1.smithi033.stdout:4/840: dwrite dd/d11/d5b/fb5 [0,4194304] 0 2024-05-12T04:01:29.090 INFO:tasks.workunit.client.1.smithi033.stdout:0/882: fsync d6/dc/f54 0 2024-05-12T04:01:29.090 INFO:tasks.workunit.client.1.smithi033.stdout:4/841: chown l4 50111823 1 2024-05-12T04:01:29.092 INFO:tasks.workunit.client.0.smithi032.stdout:8/719: mkdir d0/d9/d1b/df7 0 2024-05-12T04:01:29.093 INFO:tasks.workunit.client.0.smithi032.stdout:9/559: dwrite d0/d3/d74/f20 [0,4194304] 0 2024-05-12T04:01:29.093 INFO:tasks.workunit.client.1.smithi033.stdout:0/883: fsync d6/ff7 0 2024-05-12T04:01:29.098 INFO:tasks.workunit.client.1.smithi033.stdout:4/842: rename dd/l6f to dd/d69/dfa/l102 0 2024-05-12T04:01:29.098 INFO:tasks.workunit.client.1.smithi033.stdout:4/843: write dd/f2a [4052728,47870] 0 2024-05-12T04:01:29.101 INFO:tasks.workunit.client.0.smithi032.stdout:0/700: dwrite df/d10/d3d/d83/d9f/faa [0,4194304] 0 2024-05-12T04:01:29.101 INFO:tasks.workunit.client.0.smithi032.stdout:7/655: creat dd/d1b/d43/fd0 x:0 0 0 2024-05-12T04:01:29.102 INFO:tasks.workunit.client.1.smithi033.stdout:0/884: creat d6/d11/dc7/dd9/d10b/dfb/f130 x:0 0 0 2024-05-12T04:01:29.103 INFO:tasks.workunit.client.0.smithi032.stdout:0/701: write df/d10/d3d/d4a/d9a/d9d/fb7 [540056,21526] 0 2024-05-12T04:01:29.103 INFO:tasks.workunit.client.0.smithi032.stdout:6/616: write d7/d9/f4a [397012,96064] 0 2024-05-12T04:01:29.111 INFO:tasks.workunit.client.1.smithi033.stdout:4/844: creat dd/d69/dc4/f103 x:0 0 0 2024-05-12T04:01:29.111 INFO:tasks.workunit.client.1.smithi033.stdout:4/845: dread - dd/d7a/d87/fb7 zero size 2024-05-12T04:01:29.112 INFO:tasks.workunit.client.0.smithi032.stdout:9/560: creat d0/d3/d6/d7/d2a/d46/d13/fb6 x:0 0 0 2024-05-12T04:01:29.114 INFO:tasks.workunit.client.0.smithi032.stdout:7/656: creat dd/d1b/d1e/fd1 x:0 0 0 2024-05-12T04:01:29.116 INFO:tasks.workunit.client.0.smithi032.stdout:6/617: dread - d7/d9/d48/d67/d63/f78 zero size 2024-05-12T04:01:29.120 INFO:tasks.workunit.client.1.smithi033.stdout:4/846: fsync dd/f29 0 2024-05-12T04:01:29.120 INFO:tasks.workunit.client.1.smithi033.stdout:3/822: dwrite db/d17/d3e/d65/fbb [0,4194304] 0 2024-05-12T04:01:29.123 INFO:tasks.workunit.client.0.smithi032.stdout:0/702: dread df/d10/d3d/d4a/d9a/d9d/fb7 [0,4194304] 0 2024-05-12T04:01:29.123 INFO:tasks.workunit.client.0.smithi032.stdout:8/720: unlink d0/d9/d1b/d33/f78 0 2024-05-12T04:01:29.126 INFO:tasks.workunit.client.1.smithi033.stdout:0/885: mkdir d6/d11/d131 0 2024-05-12T04:01:29.126 INFO:tasks.workunit.client.0.smithi032.stdout:9/561: unlink d0/d3/d6/d7/d2a/f3b 0 2024-05-12T04:01:29.127 INFO:tasks.workunit.client.0.smithi032.stdout:9/562: chown d0/d3/d74/f4e 20351408 1 2024-05-12T04:01:29.131 INFO:tasks.workunit.client.1.smithi033.stdout:2/852: symlink dc/d3b/l111 0 2024-05-12T04:01:29.132 INFO:tasks.workunit.client.1.smithi033.stdout:4/847: chown dd/d7a/d87/d7e/da7/c51 80990 1 2024-05-12T04:01:29.132 INFO:tasks.workunit.client.1.smithi033.stdout:4/848: write dd/f2a [4559957,46754] 0 2024-05-12T04:01:29.134 INFO:tasks.workunit.client.1.smithi033.stdout:3/823: truncate db/d17/d3a/d67/da6/fc3 1020933 0 2024-05-12T04:01:29.136 INFO:tasks.workunit.client.0.smithi032.stdout:8/721: unlink d0/d9/d1b/d27/d4c/l89 0 2024-05-12T04:01:29.137 INFO:tasks.workunit.client.1.smithi033.stdout:2/853: dread - dc/d3b/d88/fc0 zero size 2024-05-12T04:01:29.139 INFO:tasks.workunit.client.0.smithi032.stdout:9/563: mkdir d0/d84/d8a/da3/db7 0 2024-05-12T04:01:29.142 INFO:tasks.workunit.client.1.smithi033.stdout:2/854: dwrite dc/d13/d1f/d6e/f81 [0,4194304] 0 2024-05-12T04:01:29.149 INFO:tasks.workunit.client.0.smithi032.stdout:8/722: write d0/d9/d1b/fec [942176,130400] 0 2024-05-12T04:01:29.154 INFO:tasks.workunit.client.0.smithi032.stdout:7/657: link dd/d1b/d1e/d45/d55/d72/cac dd/d1b/d1e/d45/d55/d72/cd2 0 2024-05-12T04:01:29.155 INFO:tasks.workunit.client.0.smithi032.stdout:8/723: mknod d0/d9/d13/d21/d25/db5/cf8 0 2024-05-12T04:01:29.156 INFO:tasks.workunit.client.0.smithi032.stdout:6/618: link d7/d97/f9e d7/d9/d5c/d62/fd2 0 2024-05-12T04:01:29.157 INFO:tasks.workunit.client.0.smithi032.stdout:6/619: read - d7/d9/d48/d67/d63/f78 zero size 2024-05-12T04:01:29.159 INFO:tasks.workunit.client.0.smithi032.stdout:7/658: rmdir dd/d1b/d1e/d45/d64 39 2024-05-12T04:01:29.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:28 smithi033 ceph-mon[49525]: pgmap v42: 65 pgs: 65 active+clean; 3.0 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 92 MiB/s rd, 242 MiB/s wr, 495 op/s 2024-05-12T04:01:29.161 INFO:tasks.workunit.client.0.smithi032.stdout:8/724: mkdir d0/d9/d13/d21/d25/df9 0 2024-05-12T04:01:29.164 INFO:tasks.workunit.client.0.smithi032.stdout:6/620: symlink d7/d9/d5c/d62/ld3 0 2024-05-12T04:01:29.165 INFO:tasks.workunit.client.0.smithi032.stdout:1/668: rename d1/d7/l20 to d1/dd/d24/d9b/ld8 0 2024-05-12T04:01:29.165 INFO:tasks.workunit.client.0.smithi032.stdout:9/564: link d0/d3/d6/d7/da/d89/cac d0/d3/cb8 0 2024-05-12T04:01:29.167 INFO:tasks.workunit.client.0.smithi032.stdout:1/669: write d1/d25/da5/fc0 [3729017,83270] 0 2024-05-12T04:01:29.172 INFO:tasks.workunit.client.0.smithi032.stdout:3/719: rename d15/d1b/d43/dd0/ced to d15/d20/d26/d6a/d6e/d7d/cff 0 2024-05-12T04:01:29.175 INFO:tasks.workunit.client.0.smithi032.stdout:9/565: dread - d0/f75 zero size 2024-05-12T04:01:29.180 INFO:tasks.workunit.client.0.smithi032.stdout:0/703: rename df/d10/d3d/d4a/l37 to df/d10/d3d/db0/le9 0 2024-05-12T04:01:29.182 INFO:tasks.workunit.client.0.smithi032.stdout:9/566: dwrite d0/d3/d6/d7/da/f72 [0,4194304] 0 2024-05-12T04:01:29.182 INFO:tasks.workunit.client.0.smithi032.stdout:3/720: mknod d15/d20/d22/d7a/dbe/c100 0 2024-05-12T04:01:29.188 INFO:tasks.workunit.client.0.smithi032.stdout:9/567: fsync d0/d3/d4b/f81 0 2024-05-12T04:01:29.190 INFO:tasks.workunit.client.0.smithi032.stdout:0/704: mkdir df/d10/d3d/de4/dea 0 2024-05-12T04:01:29.190 INFO:tasks.workunit.client.0.smithi032.stdout:9/568: stat d0/d3/d4b/d88/d99/fb1 0 2024-05-12T04:01:29.195 INFO:tasks.workunit.client.0.smithi032.stdout:3/721: dwrite d15/d20/d26/f64 [0,4194304] 0 2024-05-12T04:01:29.195 INFO:tasks.workunit.client.1.smithi033.stdout:4/849: mkdir dd/d7a/d87/d7e/da7/d38/d104 0 2024-05-12T04:01:29.203 INFO:tasks.workunit.client.0.smithi032.stdout:3/722: creat d15/d20/d22/d7a/dbe/f101 x:0 0 0 2024-05-12T04:01:29.204 INFO:tasks.workunit.client.1.smithi033.stdout:6/855: dwrite d4/d1a/d6c/fc7 [0,4194304] 0 2024-05-12T04:01:29.205 INFO:tasks.workunit.client.0.smithi032.stdout:0/705: symlink df/d10/d3d/dd8/leb 0 2024-05-12T04:01:29.205 INFO:tasks.workunit.client.1.smithi033.stdout:5/763: dwrite d3/d12/d25/d26/f9c [4194304,4194304] 0 2024-05-12T04:01:29.207 INFO:tasks.workunit.client.1.smithi033.stdout:8/812: dwrite d1/d16/d27/d4d/d82/fc2 [0,4194304] 0 2024-05-12T04:01:29.208 INFO:tasks.workunit.client.1.smithi033.stdout:4/850: mkdir dd/d11/d5b/d84/d105 0 2024-05-12T04:01:29.209 INFO:tasks.workunit.client.1.smithi033.stdout:6/856: stat d4/d1a/d67/d6f/lff 0 2024-05-12T04:01:29.210 INFO:tasks.workunit.client.0.smithi032.stdout:3/723: chown d15/d24/l49 3603 1 2024-05-12T04:01:29.212 INFO:tasks.workunit.client.0.smithi032.stdout:3/724: truncate d15/d20/fd6 139276 0 2024-05-12T04:01:29.214 INFO:tasks.workunit.client.0.smithi032.stdout:0/706: mknod df/d10/d3d/cec 0 2024-05-12T04:01:29.214 INFO:tasks.workunit.client.1.smithi033.stdout:8/813: dread - d1/d16/d27/d4d/d82/dd9/fd1 zero size 2024-05-12T04:01:29.216 INFO:tasks.workunit.client.1.smithi033.stdout:2/855: truncate dc/d17/d27/d68/f110 2530341 0 2024-05-12T04:01:29.218 INFO:tasks.workunit.client.1.smithi033.stdout:1/756: dwrite d11/d1c/d43/f6f [0,4194304] 0 2024-05-12T04:01:29.219 INFO:tasks.workunit.client.0.smithi032.stdout:3/725: unlink d15/d20/d22/d7a/f92 0 2024-05-12T04:01:29.220 INFO:tasks.workunit.client.1.smithi033.stdout:6/857: stat d4/d5/c59 0 2024-05-12T04:01:29.220 INFO:tasks.workunit.client.1.smithi033.stdout:6/858: chown d4/d1a/d3c 15 1 2024-05-12T04:01:29.222 INFO:tasks.workunit.client.1.smithi033.stdout:5/764: mkdir d3/d94/d54/d57/d70/dfb 0 2024-05-12T04:01:29.227 INFO:tasks.workunit.client.1.smithi033.stdout:1/757: dwrite f8 [0,4194304] 0 2024-05-12T04:01:29.228 INFO:tasks.workunit.client.1.smithi033.stdout:2/856: mkdir dc/d17/d27/d2f/d49/ddd/d112 0 2024-05-12T04:01:29.228 INFO:tasks.workunit.client.0.smithi032.stdout:0/707: getdents df/d22/d58/d8a 0 2024-05-12T04:01:29.231 INFO:tasks.workunit.client.1.smithi033.stdout:6/859: mknod d4/d43/d57/c11a 0 2024-05-12T04:01:29.231 INFO:tasks.workunit.client.0.smithi032.stdout:3/726: dwrite d15/f54 [0,4194304] 0 2024-05-12T04:01:29.237 INFO:tasks.workunit.client.0.smithi032.stdout:3/727: read - d15/d24/d2a/fc1 zero size 2024-05-12T04:01:29.237 INFO:tasks.workunit.client.0.smithi032.stdout:3/728: stat d15/d1b/d43 0 2024-05-12T04:01:29.238 INFO:tasks.workunit.client.0.smithi032.stdout:3/729: dread - d15/d1b/d43/dd0/ff7 zero size 2024-05-12T04:01:29.240 INFO:tasks.workunit.client.0.smithi032.stdout:3/730: read d15/d20/d26/fee [954697,129692] 0 2024-05-12T04:01:29.240 INFO:tasks.workunit.client.1.smithi033.stdout:1/758: rename d11/d17/f3e to d11/d17/d27/d36/d3f/d4a/dc6/de1/ff9 0 2024-05-12T04:01:29.245 INFO:tasks.workunit.client.1.smithi033.stdout:6/860: symlink d4/d43/d57/dc4/l11b 0 2024-05-12T04:01:29.245 INFO:tasks.workunit.client.1.smithi033.stdout:6/861: chown d4/d5/l3a 273813 1 2024-05-12T04:01:29.247 INFO:tasks.workunit.client.1.smithi033.stdout:2/857: getdents dc/d17/d27/d68/d9f/df7 0 2024-05-12T04:01:29.248 INFO:tasks.workunit.client.1.smithi033.stdout:2/858: fdatasync dc/d13/d1c/d98/fed 0 2024-05-12T04:01:29.249 INFO:tasks.workunit.client.1.smithi033.stdout:2/859: chown dc/d17/d27/d68 4112969 1 2024-05-12T04:01:29.253 INFO:tasks.workunit.client.1.smithi033.stdout:8/814: link d1/d16/d29/d63/c6c d1/d16/d27/cf8 0 2024-05-12T04:01:29.253 INFO:tasks.workunit.client.1.smithi033.stdout:4/851: dread dd/d7a/d87/dd9/ffe [0,4194304] 0 2024-05-12T04:01:29.255 INFO:tasks.workunit.client.1.smithi033.stdout:3/824: dwrite db/d17/d25/d32/d3c/f5a [0,4194304] 0 2024-05-12T04:01:29.256 INFO:tasks.workunit.client.1.smithi033.stdout:8/815: symlink d1/d16/d27/d4d/d73/lf9 0 2024-05-12T04:01:29.257 INFO:tasks.workunit.client.1.smithi033.stdout:8/816: write d1/d16/d27/d4d/d73/fe3 [764632,63945] 0 2024-05-12T04:01:29.257 INFO:tasks.workunit.client.1.smithi033.stdout:2/860: rename dc/d17/d32/d84/l96 to dc/d17/d27/d2f/d49/d59/l113 0 2024-05-12T04:01:29.260 INFO:tasks.workunit.client.1.smithi033.stdout:4/852: fsync dd/d7a/d87/dd9/ffe 0 2024-05-12T04:01:29.262 INFO:tasks.workunit.client.1.smithi033.stdout:6/862: creat d4/f11c x:0 0 0 2024-05-12T04:01:29.265 INFO:tasks.workunit.client.0.smithi032.stdout:2/693: sync 2024-05-12T04:01:29.268 INFO:tasks.workunit.client.0.smithi032.stdout:7/659: dwrite dd/d7b/d9a/d9c/db9/d75/fa3 [0,4194304] 0 2024-05-12T04:01:29.270 INFO:tasks.workunit.client.1.smithi033.stdout:4/853: mknod dd/d11/d5b/d84/dd7/c106 0 2024-05-12T04:01:29.271 INFO:tasks.workunit.client.1.smithi033.stdout:3/825: mkdir db/d17/d3a/d121 0 2024-05-12T04:01:29.272 INFO:tasks.workunit.client.0.smithi032.stdout:8/725: dwrite d0/d9/d13/d21/d25/f2c [0,4194304] 0 2024-05-12T04:01:29.276 INFO:tasks.workunit.client.0.smithi032.stdout:2/694: unlink d1/d34/d96/f60 0 2024-05-12T04:01:29.279 INFO:tasks.workunit.client.1.smithi033.stdout:4/854: creat dd/d69/da4/f107 x:0 0 0 2024-05-12T04:01:29.279 INFO:tasks.workunit.client.0.smithi032.stdout:6/621: dwrite d7/d9/f6c [0,4194304] 0 2024-05-12T04:01:29.286 INFO:tasks.workunit.client.0.smithi032.stdout:1/670: dwrite d1/d7/d30/d35/f42 [0,4194304] 0 2024-05-12T04:01:29.287 INFO:tasks.workunit.client.0.smithi032.stdout:8/726: symlink d0/d9/d1b/d27/d88/d56/d63/d8f/lfa 0 2024-05-12T04:01:29.287 INFO:tasks.workunit.client.1.smithi033.stdout:8/817: dread d1/d16/d27/d4d/d82/dd9/d36/d3c/da7/f43 [0,4194304] 0 2024-05-12T04:01:29.287 INFO:tasks.workunit.client.0.smithi032.stdout:9/569: write d0/d3/d6/d7/d27/f48 [622834,13057] 0 2024-05-12T04:01:29.289 INFO:tasks.workunit.client.1.smithi033.stdout:2/861: dread dc/d17/d32/d84/f89 [0,4194304] 0 2024-05-12T04:01:29.291 INFO:tasks.workunit.client.1.smithi033.stdout:2/862: fdatasync dc/d13/d65/f10e 0 2024-05-12T04:01:29.297 INFO:tasks.workunit.client.1.smithi033.stdout:4/855: dwrite dd/d69/f7d [0,4194304] 0 2024-05-12T04:01:29.299 INFO:tasks.workunit.client.1.smithi033.stdout:4/856: fdatasync dd/d7a/d87/d7e/da7/d38/fbc 0 2024-05-12T04:01:29.300 INFO:tasks.workunit.client.1.smithi033.stdout:4/857: fsync dd/d11/d5b/dd1/ffd 0 2024-05-12T04:01:29.301 INFO:tasks.workunit.client.0.smithi032.stdout:2/695: rename d1/d34/d96/d41/d39/c43 to d1/d13/d70/d85/dd5/ce7 0 2024-05-12T04:01:29.301 INFO:tasks.workunit.client.0.smithi032.stdout:8/727: rmdir d0 39 2024-05-12T04:01:29.301 INFO:tasks.workunit.client.0.smithi032.stdout:2/696: dread - d1/fe4 zero size 2024-05-12T04:01:29.301 INFO:tasks.workunit.client.0.smithi032.stdout:2/697: chown d1/d13/d6b 38599 1 2024-05-12T04:01:29.310 INFO:tasks.workunit.client.0.smithi032.stdout:1/671: dwrite d1/dd/d24/f9c [0,4194304] 0 2024-05-12T04:01:29.316 INFO:tasks.workunit.client.1.smithi033.stdout:3/826: getdents db/d17/d25/d32/d3c/d44 0 2024-05-12T04:01:29.320 INFO:tasks.workunit.client.0.smithi032.stdout:0/708: dwrite df/d30/d4e/f73 [4194304,4194304] 0 2024-05-12T04:01:29.322 INFO:tasks.workunit.client.1.smithi033.stdout:1/759: write d11/fb5 [140554,54758] 0 2024-05-12T04:01:29.322 INFO:tasks.workunit.client.0.smithi032.stdout:3/731: write d15/d33/d3f/fb7 [3115860,49885] 0 2024-05-12T04:01:29.326 INFO:tasks.workunit.client.1.smithi033.stdout:3/827: rmdir db/d17/d3e/dd7/dd0 39 2024-05-12T04:01:29.328 INFO:tasks.workunit.client.1.smithi033.stdout:3/828: write db/d17/d25/d32/d3c/f5a [4871431,126930] 0 2024-05-12T04:01:29.330 INFO:tasks.workunit.client.1.smithi033.stdout:3/829: write db/d17/d3a/d93/dc8/fda [5176145,119025] 0 2024-05-12T04:01:29.331 INFO:tasks.workunit.client.1.smithi033.stdout:8/818: rename d1/fa4 to d1/d16/d27/d4d/d82/dd9/ffa 0 2024-05-12T04:01:29.332 INFO:tasks.workunit.client.1.smithi033.stdout:8/819: rename d1/d16/d27/d4d/d82 to d1/d16/d27/d4d/d82/dfb 22 2024-05-12T04:01:29.333 INFO:tasks.workunit.client.1.smithi033.stdout:8/820: chown d1/d16/d27/d4d/d51 1 1 2024-05-12T04:01:29.336 INFO:tasks.workunit.client.1.smithi033.stdout:6/863: write d4/d1a/ff0 [71671,39533] 0 2024-05-12T04:01:29.338 INFO:tasks.workunit.client.0.smithi032.stdout:7/660: write dd/fe [2205137,116063] 0 2024-05-12T04:01:29.344 INFO:tasks.workunit.client.1.smithi033.stdout:8/821: fsync d1/d16/f96 0 2024-05-12T04:01:29.345 INFO:tasks.workunit.client.0.smithi032.stdout:7/661: dwrite dd/d7b/d9a/d9c/db9/d75/fa3 [0,4194304] 0 2024-05-12T04:01:29.350 INFO:tasks.workunit.client.1.smithi033.stdout:2/863: dwrite dc/d13/d1f/f69 [0,4194304] 0 2024-05-12T04:01:29.351 INFO:tasks.workunit.client.1.smithi033.stdout:6/864: mkdir d4/d1a/d6c/d11d 0 2024-05-12T04:01:29.362 INFO:tasks.workunit.client.1.smithi033.stdout:1/760: write d11/d1c/ddf/fc4 [219277,123946] 0 2024-05-12T04:01:29.364 INFO:tasks.workunit.client.1.smithi033.stdout:1/761: stat d11/d17/d27/d36/d3f/d4a/fe5 0 2024-05-12T04:01:29.364 INFO:tasks.workunit.client.1.smithi033.stdout:4/858: write dd/d7a/d87/d7e/da7/d66/fbf [1092,91069] 0 2024-05-12T04:01:29.366 INFO:tasks.workunit.client.1.smithi033.stdout:2/864: creat dc/d13/dfb/dd7/f114 x:0 0 0 2024-05-12T04:01:29.366 INFO:tasks.workunit.client.1.smithi033.stdout:1/762: chown d11/d17/d27/d36/d3f/d4a/fa5 24 1 2024-05-12T04:01:29.368 INFO:tasks.workunit.client.0.smithi032.stdout:6/622: rename d7/d9/d5c/d86 to d7/de/d2e/dd4 0 2024-05-12T04:01:29.373 INFO:tasks.workunit.client.0.smithi032.stdout:9/570: mknod d0/d3/d6/d7/da/d89/cb9 0 2024-05-12T04:01:29.373 INFO:tasks.workunit.client.0.smithi032.stdout:9/571: readlink d0/d3/d6/d7/da/le 0 2024-05-12T04:01:29.374 INFO:tasks.workunit.client.0.smithi032.stdout:9/572: fsync d0/d3/d6/d7/da/f72 0 2024-05-12T04:01:29.375 INFO:tasks.workunit.client.1.smithi033.stdout:6/865: mknod d4/d1a/d4e/c11e 0 2024-05-12T04:01:29.376 INFO:tasks.workunit.client.0.smithi032.stdout:9/573: chown d0/d3/d6/d7/d2a/d46/d13/f7c 16908105 1 2024-05-12T04:01:29.377 INFO:tasks.workunit.client.1.smithi033.stdout:4/859: creat dd/d11/d5b/dd1/f108 x:0 0 0 2024-05-12T04:01:29.378 INFO:tasks.workunit.client.0.smithi032.stdout:1/672: symlink d1/d7/d30/d35/d77/ld9 0 2024-05-12T04:01:29.379 INFO:tasks.workunit.client.0.smithi032.stdout:1/673: dread - d1/d7/d30/d35/d54/d7e/fbd zero size 2024-05-12T04:01:29.380 INFO:tasks.workunit.client.1.smithi033.stdout:2/865: unlink dc/d13/d1c/l30 0 2024-05-12T04:01:29.380 INFO:tasks.workunit.client.1.smithi033.stdout:3/830: write db/d59/fab [377439,100409] 0 2024-05-12T04:01:29.380 INFO:tasks.workunit.client.1.smithi033.stdout:6/866: dwrite d4/d43/d57/f108 [0,4194304] 0 2024-05-12T04:01:29.382 INFO:tasks.workunit.client.1.smithi033.stdout:2/866: fdatasync dc/d13/d39/f10f 0 2024-05-12T04:01:29.382 INFO:tasks.workunit.client.1.smithi033.stdout:6/867: chown d4/d1a/d67/dc3 107139613 1 2024-05-12T04:01:29.383 INFO:tasks.workunit.client.1.smithi033.stdout:3/831: read - db/d17/d25/d32/fd5 zero size 2024-05-12T04:01:29.384 INFO:tasks.workunit.client.1.smithi033.stdout:2/867: write f7 [1044091,58945] 0 2024-05-12T04:01:29.389 INFO:tasks.workunit.client.0.smithi032.stdout:7/662: dread dd/d7b/d9a/d9c/db9/d75/f86 [0,4194304] 0 2024-05-12T04:01:29.391 INFO:tasks.workunit.client.1.smithi033.stdout:8/822: dwrite d1/d16/d27/d4d/d82/dd9/d36/d3c/fe6 [0,4194304] 0 2024-05-12T04:01:29.391 INFO:tasks.workunit.client.0.smithi032.stdout:6/623: creat d7/d9/d5c/fd5 x:0 0 0 2024-05-12T04:01:29.393 INFO:tasks.workunit.client.0.smithi032.stdout:8/728: symlink d0/d9/d1b/d27/d4c/d4e/d7f/lfb 0 2024-05-12T04:01:29.395 INFO:tasks.workunit.client.1.smithi033.stdout:6/868: mknod d4/d5/c11f 0 2024-05-12T04:01:29.399 INFO:tasks.workunit.client.1.smithi033.stdout:4/860: mkdir dd/d7a/d87/d7e/da7/d38/d104/d109 0 2024-05-12T04:01:29.400 INFO:tasks.workunit.client.0.smithi032.stdout:9/574: chown d0/d3/d4b/d88/c93 108613 1 2024-05-12T04:01:29.402 INFO:tasks.workunit.client.1.smithi033.stdout:2/868: mkdir dc/d17/d27/d2f/d49/df2/d115 0 2024-05-12T04:01:29.402 INFO:tasks.workunit.client.1.smithi033.stdout:0/886: sync 2024-05-12T04:01:29.407 INFO:tasks.workunit.client.1.smithi033.stdout:1/763: truncate d11/d17/d84/da3/fd6 785267 0 2024-05-12T04:01:29.407 INFO:tasks.workunit.client.0.smithi032.stdout:9/575: dwrite d0/d3/d6/d7/d2a/f50 [0,4194304] 0 2024-05-12T04:01:29.408 INFO:tasks.workunit.client.1.smithi033.stdout:2/869: dwrite dc/d17/d27/d68/d9f/f10a [0,4194304] 0 2024-05-12T04:01:29.409 INFO:tasks.workunit.client.0.smithi032.stdout:3/732: getdents d15/d20/d26/de2 0 2024-05-12T04:01:29.409 INFO:tasks.workunit.client.1.smithi033.stdout:6/869: creat d4/d43/d57/dc4/f120 x:0 0 0 2024-05-12T04:01:29.417 INFO:tasks.workunit.client.0.smithi032.stdout:8/729: creat d0/d9/d1b/d27/d59/ffc x:0 0 0 2024-05-12T04:01:29.418 INFO:tasks.workunit.client.1.smithi033.stdout:1/764: dwrite d11/d17/d27/d76/fa7 [0,4194304] 0 2024-05-12T04:01:29.418 INFO:tasks.workunit.client.0.smithi032.stdout:2/698: getdents d1/d4d/d5e 0 2024-05-12T04:01:29.424 INFO:tasks.workunit.client.0.smithi032.stdout:1/674: rename d1/d7/d30/d18/d93/fa8 to d1/d25/da5/db5/fda 0 2024-05-12T04:01:29.426 INFO:tasks.workunit.client.1.smithi033.stdout:2/870: rmdir dc/d17/d27/d68/d9f 39 2024-05-12T04:01:29.430 INFO:tasks.workunit.client.1.smithi033.stdout:6/870: symlink d4/d6e/dbb/l121 0 2024-05-12T04:01:29.432 INFO:tasks.workunit.client.1.smithi033.stdout:3/832: creat db/d17/d3a/f122 x:0 0 0 2024-05-12T04:01:29.432 INFO:tasks.workunit.client.0.smithi032.stdout:5/776: sync 2024-05-12T04:01:29.432 INFO:tasks.workunit.client.0.smithi032.stdout:4/600: sync 2024-05-12T04:01:29.436 INFO:tasks.workunit.client.0.smithi032.stdout:7/663: write dd/d6c/f77 [375654,61015] 0 2024-05-12T04:01:29.436 INFO:tasks.workunit.client.0.smithi032.stdout:6/624: write d7/de/d2e/f36 [2465523,112004] 0 2024-05-12T04:01:29.437 INFO:tasks.workunit.client.1.smithi033.stdout:4/861: write dd/d7a/d87/d7e/da7/d66/d42/f82 [2981005,73933] 0 2024-05-12T04:01:29.439 INFO:tasks.workunit.client.1.smithi033.stdout:1/765: unlink d11/d17/d27/d36/d3f/d4a/le3 0 2024-05-12T04:01:29.442 INFO:tasks.workunit.client.1.smithi033.stdout:8/823: write d1/d16/d27/d7c/fc6 [414279,55527] 0 2024-05-12T04:01:29.444 INFO:tasks.workunit.client.1.smithi033.stdout:8/824: chown d1/d16/d27/d4d/d82/dd9/d36/d3c/d47/fde 164798 1 2024-05-12T04:01:29.444 INFO:tasks.workunit.client.1.smithi033.stdout:6/871: mknod d4/d1a/d3c/dd9/de2/c122 0 2024-05-12T04:01:29.447 INFO:tasks.workunit.client.0.smithi032.stdout:0/709: link df/d10/d3d/d4a/d84/f8c df/d10/fed 0 2024-05-12T04:01:29.453 INFO:tasks.workunit.client.0.smithi032.stdout:3/733: write d15/d33/d3f/fb3 [264144,56004] 0 2024-05-12T04:01:29.454 INFO:tasks.workunit.client.0.smithi032.stdout:4/601: mkdir d0/d3/d24/dae 0 2024-05-12T04:01:29.457 INFO:tasks.workunit.client.1.smithi033.stdout:8/825: mknod d1/d4/d46/dca/cfc 0 2024-05-12T04:01:29.458 INFO:tasks.workunit.client.1.smithi033.stdout:6/872: read - d4/d1a/d67/d6f/fb1 zero size 2024-05-12T04:01:29.459 INFO:tasks.workunit.client.1.smithi033.stdout:8/826: stat d1/d16/d27/d4d/d51/d8e/la8 0 2024-05-12T04:01:29.459 INFO:tasks.workunit.client.0.smithi032.stdout:7/664: stat dd/c1f 0 2024-05-12T04:01:29.459 INFO:tasks.workunit.client.0.smithi032.stdout:0/710: dread df/d10/d3d/f56 [0,4194304] 0 2024-05-12T04:01:29.461 INFO:tasks.workunit.client.0.smithi032.stdout:5/777: write d4/d7/d14/d26/d33/d4a/d38/d79/fc7 [816020,19363] 0 2024-05-12T04:01:29.461 INFO:tasks.workunit.client.1.smithi033.stdout:2/871: dwrite dc/d17/d27/d2f/d49/f85 [0,4194304] 0 2024-05-12T04:01:29.461 INFO:tasks.workunit.client.0.smithi032.stdout:0/711: chown df/d30/d4e/da5/daf 37895 1 2024-05-12T04:01:29.463 INFO:tasks.workunit.client.0.smithi032.stdout:0/712: write df/d30/d8d/d8f/fcb [505052,119680] 0 2024-05-12T04:01:29.463 INFO:tasks.workunit.client.0.smithi032.stdout:6/625: unlink d7/de/d2e/f4e 0 2024-05-12T04:01:29.467 INFO:tasks.workunit.client.1.smithi033.stdout:4/862: truncate dd/d7a/d87/d7e/da7/d38/fbc 3016787 0 2024-05-12T04:01:29.470 INFO:tasks.workunit.client.1.smithi033.stdout:1/766: truncate d11/d1c/d59/f97 3722766 0 2024-05-12T04:01:29.470 INFO:tasks.workunit.client.0.smithi032.stdout:1/675: mknod d1/d7/d49/dd1/cdb 0 2024-05-12T04:01:29.471 INFO:tasks.workunit.client.1.smithi033.stdout:2/872: dread dc/f3f [0,4194304] 0 2024-05-12T04:01:29.473 INFO:tasks.workunit.client.0.smithi032.stdout:3/734: dread - d15/dc2/fc3 zero size 2024-05-12T04:01:29.474 INFO:tasks.workunit.client.1.smithi033.stdout:3/833: dwrite db/d17/d3a/f66 [0,4194304] 0 2024-05-12T04:01:29.475 INFO:tasks.workunit.client.1.smithi033.stdout:6/873: read d4/d1a/d6c/f86 [2843193,11844] 0 2024-05-12T04:01:29.477 INFO:tasks.workunit.client.1.smithi033.stdout:8/827: dread d1/d4/db2/fbc [0,4194304] 0 2024-05-12T04:01:29.482 INFO:tasks.workunit.client.1.smithi033.stdout:3/834: dwrite db/d59/fab [0,4194304] 0 2024-05-12T04:01:29.488 INFO:tasks.workunit.client.1.smithi033.stdout:4/863: dread dd/d11/d5b/dd1/ff9 [0,4194304] 0 2024-05-12T04:01:29.496 INFO:tasks.workunit.client.0.smithi032.stdout:9/576: getdents d0/d3/d4b/d73 0 2024-05-12T04:01:29.497 INFO:tasks.workunit.client.0.smithi032.stdout:0/713: write df/d30/d32/f5e [3958021,99292] 0 2024-05-12T04:01:29.499 INFO:tasks.workunit.client.1.smithi033.stdout:6/874: rename d4/d1a/d3c/l4b to d4/d1a/d67/l123 0 2024-05-12T04:01:29.501 INFO:tasks.workunit.client.0.smithi032.stdout:5/778: dwrite d4/d7/d14/d26/d33/d72/fa7 [0,4194304] 0 2024-05-12T04:01:29.503 INFO:tasks.workunit.client.1.smithi033.stdout:8/828: dread - d1/d4/db2/fc9 zero size 2024-05-12T04:01:29.506 INFO:tasks.workunit.client.1.smithi033.stdout:5/765: sync 2024-05-12T04:01:29.508 INFO:tasks.workunit.client.1.smithi033.stdout:5/766: write d3/d94/d54/d57/d70/da3/dbe/d16/d45/d62/f64 [357375,38762] 0 2024-05-12T04:01:29.508 INFO:tasks.workunit.client.0.smithi032.stdout:0/714: dwrite df/d30/d32/f34 [4194304,4194304] 0 2024-05-12T04:01:29.514 INFO:tasks.workunit.client.0.smithi032.stdout:8/730: getdents d0/d9/d1b/d27/d88/d56/d63/d8f 0 2024-05-12T04:01:29.515 INFO:tasks.workunit.client.0.smithi032.stdout:9/577: unlink d0/d3/d6/d7/d2a/d46/d13/f55 0 2024-05-12T04:01:29.524 INFO:tasks.workunit.client.0.smithi032.stdout:6/626: write d7/d9/d48/d67/d8d/dbc/f71 [4119906,119371] 0 2024-05-12T04:01:29.525 INFO:tasks.workunit.client.1.smithi033.stdout:3/835: dread db/d17/f1b [0,4194304] 0 2024-05-12T04:01:29.526 INFO:tasks.workunit.client.1.smithi033.stdout:3/836: chown db/d17/d25/d32/d3c/d3d/d47 414669 1 2024-05-12T04:01:29.527 INFO:tasks.workunit.client.1.smithi033.stdout:3/837: stat db/d17/d25/d32/d53/l69 0 2024-05-12T04:01:29.528 INFO:tasks.workunit.client.0.smithi032.stdout:7/665: creat dd/d7b/d9a/fd3 x:0 0 0 2024-05-12T04:01:29.530 INFO:tasks.workunit.client.0.smithi032.stdout:0/715: symlink df/d30/d32/d60/dc7/lee 0 2024-05-12T04:01:29.533 INFO:tasks.workunit.client.0.smithi032.stdout:6/627: dwrite d7/de/d2e/f8c [0,4194304] 0 2024-05-12T04:01:29.538 INFO:tasks.workunit.client.0.smithi032.stdout:1/676: link d1/d6a/d72/da9/cae d1/d7/d49/dd1/cdc 0 2024-05-12T04:01:29.539 INFO:tasks.workunit.client.0.smithi032.stdout:3/735: write d15/d33/d3f/d75/ff8 [159623,75555] 0 2024-05-12T04:01:29.542 INFO:tasks.workunit.client.0.smithi032.stdout:3/736: stat d15/d20/d26/d6a/d6e/d7d/fd5 0 2024-05-12T04:01:29.542 INFO:tasks.workunit.client.0.smithi032.stdout:4/602: creat d0/d3/d12/faf x:0 0 0 2024-05-12T04:01:29.543 INFO:tasks.workunit.client.0.smithi032.stdout:6/628: dwrite d7/fc7 [0,4194304] 0 2024-05-12T04:01:29.546 INFO:tasks.workunit.client.1.smithi033.stdout:4/864: mkdir dd/d11/d5b/dd1/d10a 0 2024-05-12T04:01:29.548 INFO:tasks.workunit.client.1.smithi033.stdout:4/865: stat dd/d7a/d87/d7e/da7/d66/d42/l79 0 2024-05-12T04:01:29.548 INFO:tasks.workunit.client.1.smithi033.stdout:0/887: sync 2024-05-12T04:01:29.551 INFO:tasks.workunit.client.1.smithi033.stdout:6/875: chown d4/c81 814227021 1 2024-05-12T04:01:29.551 INFO:tasks.workunit.client.1.smithi033.stdout:6/876: write d4/d43/d57/dc4/ddf/f109 [33225,18926] 0 2024-05-12T04:01:29.553 INFO:tasks.workunit.client.0.smithi032.stdout:1/677: creat d1/d7/fdd x:0 0 0 2024-05-12T04:01:29.554 INFO:tasks.workunit.client.0.smithi032.stdout:1/678: write d1/d7/d30/d35/d54/d7e/fca [1047594,3763] 0 2024-05-12T04:01:29.556 INFO:tasks.workunit.client.0.smithi032.stdout:3/737: creat d15/d24/d93/f102 x:0 0 0 2024-05-12T04:01:29.557 INFO:tasks.workunit.client.0.smithi032.stdout:3/738: stat d15/d20/d22/d7a/la9 0 2024-05-12T04:01:29.562 INFO:tasks.workunit.client.1.smithi033.stdout:5/767: rename d3/d94/dc8/c55 to d3/d12/dac/cfc 0 2024-05-12T04:01:29.565 INFO:tasks.workunit.client.1.smithi033.stdout:3/838: chown db/d42/lf8 10931523 1 2024-05-12T04:01:29.565 INFO:tasks.workunit.client.0.smithi032.stdout:5/779: getdents d4/d7/d14/d39 0 2024-05-12T04:01:29.566 INFO:tasks.workunit.client.1.smithi033.stdout:2/873: link dc/d13/d1c/f92 dc/d17/d32/d84/f116 0 2024-05-12T04:01:29.568 INFO:tasks.workunit.client.1.smithi033.stdout:2/874: chown dc/d17/d32/d84/la4 268 1 2024-05-12T04:01:29.569 INFO:tasks.workunit.client.1.smithi033.stdout:8/829: dread d1/d16/d27/d4d/d82/dd9/d36/f59 [0,4194304] 0 2024-05-12T04:01:29.570 INFO:tasks.workunit.client.0.smithi032.stdout:7/666: dread dd/d7b/d9a/d9c/db9/d75/f9b [0,4194304] 0 2024-05-12T04:01:29.570 INFO:tasks.workunit.client.1.smithi033.stdout:4/866: creat dd/d69/da4/f10b x:0 0 0 2024-05-12T04:01:29.572 INFO:tasks.workunit.client.1.smithi033.stdout:4/867: chown dd/d7a/d87/d7e/da7/d38/d104/d109 5462 1 2024-05-12T04:01:29.573 INFO:tasks.workunit.client.1.smithi033.stdout:0/888: dread d6/d11/d57/f93 [0,4194304] 0 2024-05-12T04:01:29.573 INFO:tasks.workunit.client.0.smithi032.stdout:4/603: truncate d0/d3/df/f26 949153 0 2024-05-12T04:01:29.575 INFO:tasks.workunit.client.1.smithi033.stdout:8/830: dwrite d1/d16/d29/d63/fbf [0,4194304] 0 2024-05-12T04:01:29.576 INFO:tasks.workunit.client.0.smithi032.stdout:8/731: link d0/d9/d1b/d27/d4c/d4e/d90/lc3 d0/d9/d1b/d27/d4c/d4e/d90/lfd 0 2024-05-12T04:01:29.578 INFO:tasks.workunit.client.0.smithi032.stdout:5/780: readlink d4/d7/d14/d26/d33/d5f/ld5 0 2024-05-12T04:01:29.580 INFO:tasks.workunit.client.1.smithi033.stdout:1/767: getdents d11/d1c/ddf/d4c/dab 0 2024-05-12T04:01:29.582 INFO:tasks.workunit.client.1.smithi033.stdout:5/768: write d3/d94/f3e [2581527,35830] 0 2024-05-12T04:01:29.587 INFO:tasks.workunit.client.1.smithi033.stdout:4/868: creat dd/d11/d5b/dd1/f10c x:0 0 0 2024-05-12T04:01:29.587 INFO:tasks.workunit.client.0.smithi032.stdout:6/629: write d7/d9/d48/d67/f52 [453330,129185] 0 2024-05-12T04:01:29.590 INFO:tasks.workunit.client.0.smithi032.stdout:1/679: write d1/d7/d30/f28 [3416488,17040] 0 2024-05-12T04:01:29.592 INFO:tasks.workunit.client.1.smithi033.stdout:4/869: dwrite dd/f35 [0,4194304] 0 2024-05-12T04:01:29.593 INFO:tasks.workunit.client.0.smithi032.stdout:1/680: write d1/d7/d49/d79/f8d [244831,53175] 0 2024-05-12T04:01:29.596 INFO:tasks.workunit.client.1.smithi033.stdout:6/877: write d4/d5/fab [1533578,32957] 0 2024-05-12T04:01:29.601 INFO:tasks.workunit.client.0.smithi032.stdout:6/630: creat d7/d9/d5c/d62/fd6 x:0 0 0 2024-05-12T04:01:29.605 INFO:tasks.workunit.client.0.smithi032.stdout:5/781: mkdir d4/d113 0 2024-05-12T04:01:29.606 INFO:tasks.workunit.client.1.smithi033.stdout:8/831: dwrite d1/d16/d27/d4d/d82/dd9/d36/d3c/d47/f60 [0,4194304] 0 2024-05-12T04:01:29.607 INFO:tasks.workunit.client.1.smithi033.stdout:0/889: mknod d6/dc/d58/dff/c132 0 2024-05-12T04:01:29.609 INFO:tasks.workunit.client.0.smithi032.stdout:3/739: getdents d15/dc2 0 2024-05-12T04:01:29.611 INFO:tasks.workunit.client.1.smithi033.stdout:1/768: mknod d11/d17/d27/d36/d3f/d69/d87/cfa 0 2024-05-12T04:01:29.612 INFO:tasks.workunit.client.1.smithi033.stdout:6/878: dread d4/d1a/d6c/f9c [0,4194304] 0 2024-05-12T04:01:29.613 INFO:tasks.workunit.client.1.smithi033.stdout:1/769: stat d11/d17/d27/d36/d3f/d69/d87/cfa 0 2024-05-12T04:01:29.616 INFO:tasks.workunit.client.0.smithi032.stdout:6/631: read - d7/d9/d48/d67/d1a/d20/fab zero size 2024-05-12T04:01:29.618 INFO:tasks.workunit.client.1.smithi033.stdout:5/769: mkdir d3/d94/d54/d57/d70/da3/dbe/d16/d45/d62/d7f/dcf/dfd 0 2024-05-12T04:01:29.619 INFO:tasks.workunit.client.0.smithi032.stdout:1/681: write d1/dd/f7c [189861,23388] 0 2024-05-12T04:01:29.620 INFO:tasks.workunit.client.1.smithi033.stdout:2/875: creat dc/f117 x:0 0 0 2024-05-12T04:01:29.620 INFO:tasks.workunit.client.0.smithi032.stdout:8/732: dwrite d0/d9/f2d [0,4194304] 0 2024-05-12T04:01:29.622 INFO:tasks.workunit.client.0.smithi032.stdout:8/733: dread - d0/d9/f8e zero size 2024-05-12T04:01:29.623 INFO:tasks.workunit.client.1.smithi033.stdout:4/870: creat dd/d7a/d87/d7e/da7/d66/df2/f10d x:0 0 0 2024-05-12T04:01:29.624 INFO:tasks.workunit.client.0.smithi032.stdout:3/740: unlink fd 0 2024-05-12T04:01:29.626 INFO:tasks.workunit.client.1.smithi033.stdout:8/832: creat d1/d16/d27/d4d/d51/d8e/ffd x:0 0 0 2024-05-12T04:01:29.626 INFO:tasks.workunit.client.0.smithi032.stdout:7/667: getdents dd/d1b/d1e/d45/d55/db7 0 2024-05-12T04:01:29.629 INFO:tasks.workunit.client.0.smithi032.stdout:4/604: getdents d0/d3 0 2024-05-12T04:01:29.630 INFO:tasks.workunit.client.0.smithi032.stdout:4/605: read - d0/f93 zero size 2024-05-12T04:01:29.633 INFO:tasks.workunit.client.1.smithi033.stdout:6/879: mkdir d4/d1a/d67/d124 0 2024-05-12T04:01:29.636 INFO:tasks.workunit.client.1.smithi033.stdout:1/770: symlink d11/d17/d84/de7/lfb 0 2024-05-12T04:01:29.638 INFO:tasks.workunit.client.0.smithi032.stdout:7/668: dread dd/d1b/d43/d5e/d37/f46 [0,4194304] 0 2024-05-12T04:01:29.640 INFO:tasks.workunit.client.1.smithi033.stdout:3/839: rename db/d17/d25/d32/d53/c75 to db/d17/c123 0 2024-05-12T04:01:29.645 INFO:tasks.workunit.client.0.smithi032.stdout:3/741: rmdir d15 39 2024-05-12T04:01:29.647 INFO:tasks.workunit.client.1.smithi033.stdout:5/770: rmdir d3/d94/d85/d99/da2 39 2024-05-12T04:01:29.649 INFO:tasks.workunit.client.0.smithi032.stdout:5/782: dread d4/d7/d14/d26/d33/d5f/d7f/f84 [0,4194304] 0 2024-05-12T04:01:29.653 INFO:tasks.workunit.client.0.smithi032.stdout:4/606: mknod d0/d3/d12/d1d/d63/d77/cb0 0 2024-05-12T04:01:29.658 INFO:tasks.workunit.client.0.smithi032.stdout:8/734: link d0/d9/d1b/f98 d0/d7/ffe 0 2024-05-12T04:01:29.660 INFO:tasks.workunit.client.0.smithi032.stdout:6/632: dwrite d7/d97/f99 [0,4194304] 0 2024-05-12T04:01:29.661 INFO:tasks.workunit.client.1.smithi033.stdout:2/876: write dc/d17/d27/fda [727997,103821] 0 2024-05-12T04:01:29.662 INFO:tasks.workunit.client.1.smithi033.stdout:6/880: fsync d4/d1a/d4e/fa6 0 2024-05-12T04:01:29.665 INFO:tasks.workunit.client.1.smithi033.stdout:3/840: creat db/d17/d25/d32/d3c/d3d/d47/d10a/f124 x:0 0 0 2024-05-12T04:01:29.667 INFO:tasks.workunit.client.0.smithi032.stdout:5/783: write d4/d7/d14/d39/d1b/fbd [2258489,109528] 0 2024-05-12T04:01:29.669 INFO:tasks.workunit.client.0.smithi032.stdout:4/607: fsync d0/d3/d31/d80/d84/fa9 0 2024-05-12T04:01:29.670 INFO:tasks.workunit.client.1.smithi033.stdout:4/871: symlink dd/d7a/d87/d7e/da7/d38/deb/l10e 0 2024-05-12T04:01:29.673 INFO:tasks.workunit.client.1.smithi033.stdout:0/890: link d6/d11/f1c d6/dc/d58/d6a/f133 0 2024-05-12T04:01:29.673 INFO:tasks.workunit.client.0.smithi032.stdout:5/784: truncate d4/d7/d14/d26/d33/d72/dc9/fca 4592 0 2024-05-12T04:01:29.674 INFO:tasks.workunit.client.0.smithi032.stdout:6/633: mknod d7/de/d2e/db9/cd7 0 2024-05-12T04:01:29.677 INFO:tasks.workunit.client.0.smithi032.stdout:7/669: dread dd/d1b/d43/d5e/f4c [0,4194304] 0 2024-05-12T04:01:29.677 INFO:tasks.workunit.client.0.smithi032.stdout:3/742: unlink d15/d20/d22/d7a/cb0 0 2024-05-12T04:01:29.681 INFO:tasks.workunit.client.0.smithi032.stdout:4/608: rename d0/f7b to d0/d3/d12/d8d/da1/fb1 0 2024-05-12T04:01:29.684 INFO:tasks.workunit.client.0.smithi032.stdout:6/634: chown d7/d97/f9e 838216 1 2024-05-12T04:01:29.687 INFO:tasks.workunit.client.0.smithi032.stdout:7/670: mknod dd/d1b/d43/d5e/d37/cd4 0 2024-05-12T04:01:29.690 INFO:tasks.workunit.client.0.smithi032.stdout:8/735: dwrite d0/d9/d13/d21/d25/f42 [0,4194304] 0 2024-05-12T04:01:29.696 INFO:tasks.workunit.client.1.smithi033.stdout:8/833: link d1/d4/cc d1/d16/d27/d7c/db7/cfe 0 2024-05-12T04:01:29.698 INFO:tasks.workunit.client.1.smithi033.stdout:8/834: stat d1/d16/d27/d4d/d82/dd9/d36/d3c/d47 0 2024-05-12T04:01:29.700 INFO:tasks.workunit.client.0.smithi032.stdout:6/635: creat d7/d33/fd8 x:0 0 0 2024-05-12T04:01:29.702 INFO:tasks.workunit.client.0.smithi032.stdout:6/636: dread - d7/d9/d48/d67/d63/d64/d94/fc3 zero size 2024-05-12T04:01:29.706 INFO:tasks.workunit.client.1.smithi033.stdout:5/771: dwrite d3/d94/d54/d98/f9f [0,4194304] 0 2024-05-12T04:01:29.707 INFO:tasks.workunit.client.1.smithi033.stdout:3/841: link db/d17/d3e/f48 db/d17/d25/d32/d53/df6/f125 0 2024-05-12T04:01:29.708 INFO:tasks.workunit.client.0.smithi032.stdout:4/609: dwrite d0/d3/f1c [4194304,4194304] 0 2024-05-12T04:01:29.709 INFO:tasks.workunit.client.1.smithi033.stdout:4/872: dwrite dd/f4b [0,4194304] 0 2024-05-12T04:01:29.709 INFO:tasks.workunit.client.0.smithi032.stdout:4/610: stat d0/d3/d12/d1d/d63/d87/la3 0 2024-05-12T04:01:29.710 INFO:tasks.workunit.client.0.smithi032.stdout:8/736: unlink d0/d9/d1b/d27/d4c/d4e/d7f/d91/fde 0 2024-05-12T04:01:29.712 INFO:tasks.workunit.client.1.smithi033.stdout:4/873: dread - dd/d69/dc4/f103 zero size 2024-05-12T04:01:29.713 INFO:tasks.workunit.client.0.smithi032.stdout:5/785: write d4/d7/d14/d26/d33/d4a/d38/f45 [747731,129476] 0 2024-05-12T04:01:29.713 INFO:tasks.workunit.client.1.smithi033.stdout:0/891: dwrite d6/dc/d58/d6a/dec/fed [0,4194304] 0 2024-05-12T04:01:29.715 INFO:tasks.workunit.client.1.smithi033.stdout:0/892: readlink d6/dc/d96/d9d/lb9 0 2024-05-12T04:01:29.716 INFO:tasks.workunit.client.1.smithi033.stdout:2/877: creat dc/d17/d27/d68/f118 x:0 0 0 2024-05-12T04:01:29.717 INFO:tasks.workunit.client.0.smithi032.stdout:3/743: rmdir d15/d20/d8b/d9b/dbb/ddf 0 2024-05-12T04:01:29.722 INFO:tasks.workunit.client.1.smithi033.stdout:0/893: readlink d6/dc/d6e/l11c 0 2024-05-12T04:01:29.722 INFO:tasks.workunit.client.0.smithi032.stdout:4/611: symlink d0/d3/d12/d1d/d63/lb2 0 2024-05-12T04:01:29.722 INFO:tasks.workunit.client.1.smithi033.stdout:6/881: getdents d4/d1a/d3c/dd9 0 2024-05-12T04:01:29.723 INFO:tasks.workunit.client.1.smithi033.stdout:3/842: dread db/d59/fab [0,4194304] 0 2024-05-12T04:01:29.726 INFO:tasks.workunit.client.1.smithi033.stdout:1/771: getdents d11/d17/d27 0 2024-05-12T04:01:29.727 INFO:tasks.workunit.client.1.smithi033.stdout:3/843: write db/d17/d25/d32/d3c/d3d/d47/fdc [4855848,80268] 0 2024-05-12T04:01:29.729 INFO:tasks.workunit.client.0.smithi032.stdout:5/786: unlink d4/d7/d14/d39/l1f 0 2024-05-12T04:01:29.729 INFO:tasks.workunit.client.1.smithi033.stdout:3/844: truncate db/d17/d3a/d67/da6/fe3 704461 0 2024-05-12T04:01:29.739 INFO:tasks.workunit.client.1.smithi033.stdout:8/835: mkdir d1/d4/d91/de1/dff 0 2024-05-12T04:01:29.739 INFO:tasks.workunit.client.0.smithi032.stdout:6/637: dwrite d7/d9/d5c/d8b/fa9 [0,4194304] 0 2024-05-12T04:01:29.739 INFO:tasks.workunit.client.1.smithi033.stdout:8/836: chown d1/d4/db2 18085 1 2024-05-12T04:01:29.742 INFO:tasks.workunit.client.1.smithi033.stdout:0/894: creat d6/d11/dc7/dd9/f134 x:0 0 0 2024-05-12T04:01:29.742 INFO:tasks.workunit.client.0.smithi032.stdout:4/612: mknod d0/d3/df/dac/cb3 0 2024-05-12T04:01:29.743 INFO:tasks.workunit.client.0.smithi032.stdout:4/613: chown d0/d75/f83 1 1 2024-05-12T04:01:29.743 INFO:tasks.workunit.client.1.smithi033.stdout:0/895: dread - d6/dc/d96/d9d/fd4 zero size 2024-05-12T04:01:29.745 INFO:tasks.workunit.client.0.smithi032.stdout:8/737: write d0/d9/d1b/f4d [2696999,41978] 0 2024-05-12T04:01:29.748 INFO:tasks.workunit.client.0.smithi032.stdout:6/638: symlink d7/d9/d48/d67/d63/ld9 0 2024-05-12T04:01:29.751 INFO:tasks.workunit.client.0.smithi032.stdout:4/614: dwrite d0/d3/df/d4e/f92 [0,4194304] 0 2024-05-12T04:01:29.751 INFO:tasks.workunit.client.0.smithi032.stdout:8/738: truncate d0/d9/d13/d21/d25/db5/fdc 4864096 0 2024-05-12T04:01:29.753 INFO:tasks.workunit.client.1.smithi033.stdout:4/874: write dd/d7a/d87/d7e/da7/fc7 [801674,52684] 0 2024-05-12T04:01:29.756 INFO:tasks.workunit.client.1.smithi033.stdout:3/845: mkdir db/d17/d3a/d108/de9/d126 0 2024-05-12T04:01:29.760 INFO:tasks.workunit.client.1.smithi033.stdout:5/772: creat d3/d94/d54/d57/d70/da3/dbe/d16/d45/d62/d7f/dcf/dfd/ffe x:0 0 0 2024-05-12T04:01:29.765 INFO:tasks.workunit.client.0.smithi032.stdout:3/744: dwrite d15/d33/f46 [0,4194304] 0 2024-05-12T04:01:29.765 INFO:tasks.workunit.client.0.smithi032.stdout:3/745: chown d15/d24/d93/f102 6190 1 2024-05-12T04:01:29.765 INFO:tasks.workunit.client.0.smithi032.stdout:3/746: write d15/d20/d26/d6a/d6e/d7d/fe1 [435955,52544] 0 2024-05-12T04:01:29.766 INFO:tasks.workunit.client.0.smithi032.stdout:8/739: creat d0/d9/d1b/d27/d88/dd7/fff x:0 0 0 2024-05-12T04:01:29.766 INFO:tasks.workunit.client.0.smithi032.stdout:6/639: mknod d7/d9/d48/cda 0 2024-05-12T04:01:29.769 INFO:tasks.workunit.client.0.smithi032.stdout:5/787: symlink d4/d7/d14/d39/d2b/l114 0 2024-05-12T04:01:29.770 INFO:tasks.workunit.client.0.smithi032.stdout:8/740: stat d0/d9/d1b/d27/d88/d56/d63/d8f/db0 0 2024-05-12T04:01:29.771 INFO:tasks.workunit.client.0.smithi032.stdout:3/747: write d15/d24/d2a/d8c/fd4 [629294,49082] 0 2024-05-12T04:01:29.771 INFO:tasks.workunit.client.1.smithi033.stdout:0/896: symlink d6/db/dd6/l135 0 2024-05-12T04:01:29.772 INFO:tasks.workunit.client.1.smithi033.stdout:0/897: readlink d6/db/dd6/l135 0 2024-05-12T04:01:29.772 INFO:tasks.workunit.client.0.smithi032.stdout:5/788: stat d4/d7/d14/d26/d33/d72/dc9/fca 0 2024-05-12T04:01:29.778 INFO:tasks.workunit.client.0.smithi032.stdout:6/640: mknod d7/d9/d5c/d62/cdb 0 2024-05-12T04:01:29.780 INFO:tasks.workunit.client.1.smithi033.stdout:3/846: creat db/d17/d25/d32/d3c/d3d/f127 x:0 0 0 2024-05-12T04:01:29.781 INFO:tasks.workunit.client.0.smithi032.stdout:3/748: fsync d15/d33/f74 0 2024-05-12T04:01:29.783 INFO:tasks.workunit.client.0.smithi032.stdout:4/615: dwrite d0/d3/d31/d80/f82 [0,4194304] 0 2024-05-12T04:01:29.783 INFO:tasks.workunit.client.1.smithi033.stdout:6/882: dwrite d4/d1a/d67/f10f [0,4194304] 0 2024-05-12T04:01:29.785 INFO:tasks.workunit.client.1.smithi033.stdout:5/773: rename d3/d12/d29/d2b/f6e to d3/d12/d83/deb/fff 0 2024-05-12T04:01:29.788 INFO:tasks.workunit.client.0.smithi032.stdout:8/741: rmdir d0/d9/d13/d21/d25/db5 39 2024-05-12T04:01:29.789 INFO:tasks.workunit.client.1.smithi033.stdout:1/772: write d11/d17/d27/d36/d3f/d4a/dc6/de1/ff9 [4216878,115148] 0 2024-05-12T04:01:29.791 INFO:tasks.workunit.client.1.smithi033.stdout:2/878: creat dc/f119 x:0 0 0 2024-05-12T04:01:29.798 INFO:tasks.workunit.client.0.smithi032.stdout:3/749: creat d15/d20/d22/d4f/de0/f103 x:0 0 0 2024-05-12T04:01:29.800 INFO:tasks.workunit.client.0.smithi032.stdout:3/750: stat d15/d20/d26/d6a/d6e/d7d/c82 0 2024-05-12T04:01:29.800 INFO:tasks.workunit.client.1.smithi033.stdout:0/898: dread d6/dc/f8c [0,4194304] 0 2024-05-12T04:01:29.800 INFO:tasks.workunit.client.0.smithi032.stdout:4/616: mknod d0/d3/d31/d80/d84/cb4 0 2024-05-12T04:01:29.802 INFO:tasks.workunit.client.1.smithi033.stdout:4/875: creat dd/d7a/d87/d7e/da7/d38/d104/d109/f10f x:0 0 0 2024-05-12T04:01:29.803 INFO:tasks.workunit.client.1.smithi033.stdout:8/837: truncate d1/d16/d27/d4d/d82/dd9/d36/faf 3564462 0 2024-05-12T04:01:29.806 INFO:tasks.workunit.client.1.smithi033.stdout:3/847: stat db/d17/d25/d32/ce0 0 2024-05-12T04:01:29.807 INFO:tasks.workunit.client.0.smithi032.stdout:6/641: creat d7/d97/db3/fdc x:0 0 0 2024-05-12T04:01:29.807 INFO:tasks.workunit.client.0.smithi032.stdout:5/789: dwrite d4/d7/d14/d26/fa6 [0,4194304] 0 2024-05-12T04:01:29.815 INFO:tasks.workunit.client.1.smithi033.stdout:1/773: mkdir d11/d17/d27/d76/dfc 0 2024-05-12T04:01:29.817 INFO:tasks.workunit.client.0.smithi032.stdout:3/751: unlink d15/d20/d22/fb6 0 2024-05-12T04:01:29.819 INFO:tasks.workunit.client.1.smithi033.stdout:8/838: dread d1/d16/f75 [0,4194304] 0 2024-05-12T04:01:29.823 INFO:tasks.workunit.client.1.smithi033.stdout:2/879: read dc/d17/d27/d2f/d49/ff9 [274787,55249] 0 2024-05-12T04:01:29.824 INFO:tasks.workunit.client.1.smithi033.stdout:4/876: creat dd/d11/d5b/dd1/f110 x:0 0 0 2024-05-12T04:01:29.825 INFO:tasks.workunit.client.0.smithi032.stdout:6/642: read d7/d9/d48/d67/d25/f6f [407129,114125] 0 2024-05-12T04:01:29.826 INFO:tasks.workunit.client.1.smithi033.stdout:4/877: write dd/d11/d5b/dd1/f10c [259901,86260] 0 2024-05-12T04:01:29.826 INFO:tasks.workunit.client.0.smithi032.stdout:3/752: dwrite d15/d20/d26/d6a/d6e/d7d/fa1 [0,4194304] 0 2024-05-12T04:01:29.829 INFO:tasks.workunit.client.0.smithi032.stdout:4/617: mkdir d0/db5 0 2024-05-12T04:01:29.830 INFO:tasks.workunit.client.0.smithi032.stdout:4/618: stat d0/d3/d31/d80/d84/da7/fad 0 2024-05-12T04:01:29.832 INFO:tasks.workunit.client.1.smithi033.stdout:3/848: rename db/c14 to db/d17/d3e/d65/d91/d97/c128 0 2024-05-12T04:01:29.833 INFO:tasks.workunit.client.1.smithi033.stdout:0/899: write d6/d11/f47 [5381409,127228] 0 2024-05-12T04:01:29.834 INFO:tasks.workunit.client.0.smithi032.stdout:3/753: dwrite d15/f4e [0,4194304] 0 2024-05-12T04:01:29.843 INFO:tasks.workunit.client.0.smithi032.stdout:8/742: creat d0/d9/d13/d21/f100 x:0 0 0 2024-05-12T04:01:29.843 INFO:tasks.workunit.client.0.smithi032.stdout:5/790: mknod d4/d7/d14/df3/c115 0 2024-05-12T04:01:29.844 INFO:tasks.workunit.client.0.smithi032.stdout:6/643: creat d7/de/d2e/dd4/fdd x:0 0 0 2024-05-12T04:01:29.846 INFO:tasks.workunit.client.0.smithi032.stdout:4/619: mknod d0/d3/df/dac/cb6 0 2024-05-12T04:01:29.848 INFO:tasks.workunit.client.0.smithi032.stdout:3/754: dwrite d15/d20/d8b/fe3 [0,4194304] 0 2024-05-12T04:01:29.848 INFO:tasks.workunit.client.0.smithi032.stdout:6/644: fsync d7/d9/d48/d67/d1a/d20/fc2 0 2024-05-12T04:01:29.855 INFO:tasks.workunit.client.1.smithi033.stdout:3/849: chown db/d17/d3e/dd7/dd0/cac 31372169 1 2024-05-12T04:01:29.858 INFO:tasks.workunit.client.1.smithi033.stdout:0/900: fdatasync d6/dc/d39/f8a 0 2024-05-12T04:01:29.860 INFO:tasks.workunit.client.1.smithi033.stdout:5/774: creat d3/d12/d25/f100 x:0 0 0 2024-05-12T04:01:29.861 INFO:tasks.workunit.client.0.smithi032.stdout:4/620: fsync d0/d3/d12/d1d/f2a 0 2024-05-12T04:01:29.863 INFO:tasks.workunit.client.1.smithi033.stdout:1/774: dwrite d11/d17/d27/d76/fbb [0,4194304] 0 2024-05-12T04:01:29.865 INFO:tasks.workunit.client.1.smithi033.stdout:8/839: symlink d1/d16/d27/d4d/d82/dd9/d36/dd2/l100 0 2024-05-12T04:01:29.866 INFO:tasks.workunit.client.1.smithi033.stdout:8/840: readlink d1/d16/d27/d4d/d82/dd9/d36/dd2/l100 0 2024-05-12T04:01:29.868 INFO:tasks.workunit.client.1.smithi033.stdout:2/880: link dc/d13/d1f/f69 dc/d13/d39/f11a 0 2024-05-12T04:01:29.871 INFO:tasks.workunit.client.1.smithi033.stdout:4/878: symlink dd/d7a/d87/l111 0 2024-05-12T04:01:29.873 INFO:tasks.workunit.client.0.smithi032.stdout:2/699: sync 2024-05-12T04:01:29.873 INFO:tasks.workunit.client.0.smithi032.stdout:8/743: dread d0/f68 [0,4194304] 0 2024-05-12T04:01:29.874 INFO:tasks.workunit.client.0.smithi032.stdout:2/700: readlink d1/d34/da5/lb2 0 2024-05-12T04:01:29.878 INFO:tasks.workunit.client.1.smithi033.stdout:4/879: dwrite dd/d69/dc4/f103 [0,4194304] 0 2024-05-12T04:01:29.879 INFO:tasks.workunit.client.0.smithi032.stdout:5/791: creat d4/d7/d14/d26/d33/d72/dc9/dec/f116 x:0 0 0 2024-05-12T04:01:29.883 INFO:tasks.workunit.client.1.smithi033.stdout:8/841: fsync d1/d16/d27/d4d/d82/dd9/d36/d3c/d9f/fcc 0 2024-05-12T04:01:29.886 INFO:tasks.workunit.client.1.smithi033.stdout:2/881: rmdir dc/d13/d39 39 2024-05-12T04:01:29.887 INFO:tasks.workunit.client.0.smithi032.stdout:3/755: link d15/d20/d22/d7a/dbe/cd2 d15/d1b/d43/c104 0 2024-05-12T04:01:29.889 INFO:tasks.workunit.client.0.smithi032.stdout:4/621: write d0/d3/d12/d1d/f5d [1357274,125780] 0 2024-05-12T04:01:29.891 INFO:tasks.workunit.client.1.smithi033.stdout:8/842: dwrite d1/d16/d27/d4d/d51/d8e/ffd [0,4194304] 0 2024-05-12T04:01:29.894 INFO:tasks.workunit.client.0.smithi032.stdout:4/622: truncate d0/d3/d12/d1d/d63/f9b 836645 0 2024-05-12T04:01:29.896 INFO:tasks.workunit.client.0.smithi032.stdout:3/756: dwrite d15/d24/d2a/d5b/f8e [0,4194304] 0 2024-05-12T04:01:29.899 INFO:tasks.workunit.client.1.smithi033.stdout:4/880: creat dd/d7a/d87/d7e/da7/d66/d42/d59/dd0/f112 x:0 0 0 2024-05-12T04:01:29.899 INFO:tasks.workunit.client.0.smithi032.stdout:2/701: write d1/d34/d96/d41/d58/f89 [239381,104312] 0 2024-05-12T04:01:29.900 INFO:tasks.workunit.client.1.smithi033.stdout:1/775: getdents d11/d17/d27/d76/dfc 0 2024-05-12T04:01:29.901 INFO:tasks.workunit.client.0.smithi032.stdout:4/623: mkdir d0/d3/d12/d1d/d6b/db7 0 2024-05-12T04:01:29.902 INFO:tasks.workunit.client.1.smithi033.stdout:5/775: dwrite d3/d94/d54/d57/d70/da3/dbe/d16/f44 [0,4194304] 0 2024-05-12T04:01:29.904 INFO:tasks.workunit.client.0.smithi032.stdout:4/624: stat d0/d3/d12/f48 0 2024-05-12T04:01:29.905 INFO:tasks.workunit.client.0.smithi032.stdout:8/744: dread d0/d9/d13/f58 [0,4194304] 0 2024-05-12T04:01:29.905 INFO:tasks.workunit.client.0.smithi032.stdout:2/702: rmdir d1/d73 39 2024-05-12T04:01:29.909 INFO:tasks.workunit.client.0.smithi032.stdout:8/745: write d0/d97/fd8 [409544,107926] 0 2024-05-12T04:01:29.909 INFO:tasks.workunit.client.1.smithi033.stdout:3/850: creat db/d17/d3a/f129 x:0 0 0 2024-05-12T04:01:29.913 INFO:tasks.workunit.client.1.smithi033.stdout:4/881: dwrite dd/d11/d5b/fb5 [0,4194304] 0 2024-05-12T04:01:29.913 INFO:tasks.workunit.client.0.smithi032.stdout:5/792: dwrite d4/d7/d14/d39/d2b/d6e/f85 [4194304,4194304] 0 2024-05-12T04:01:29.914 INFO:tasks.workunit.client.0.smithi032.stdout:4/625: rename d0/d3/d31/l45 to d0/db5/lb8 0 2024-05-12T04:01:29.915 INFO:tasks.workunit.client.1.smithi033.stdout:4/882: write dd/d69/da4/f10b [723135,67103] 0 2024-05-12T04:01:29.918 INFO:tasks.workunit.client.1.smithi033.stdout:2/882: mkdir dc/d17/dcc/df1/d11b 0 2024-05-12T04:01:29.921 INFO:tasks.workunit.client.0.smithi032.stdout:3/757: write d15/d20/d26/f5f [3611171,55904] 0 2024-05-12T04:01:29.928 INFO:tasks.workunit.client.0.smithi032.stdout:2/703: creat d1/d34/d96/fe8 x:0 0 0 2024-05-12T04:01:29.928 INFO:tasks.workunit.client.1.smithi033.stdout:2/883: stat dc/d17/d27/d68/d9f/d5e/lb3 0 2024-05-12T04:01:29.928 INFO:tasks.workunit.client.0.smithi032.stdout:2/704: chown d1/d13/da0 23324698 1 2024-05-12T04:01:29.929 INFO:tasks.workunit.client.0.smithi032.stdout:5/793: mknod d4/c117 0 2024-05-12T04:01:29.931 INFO:tasks.workunit.client.1.smithi033.stdout:1/776: getdents d11/d17/d27/d36/d3f/d4a 0 2024-05-12T04:01:29.935 INFO:tasks.workunit.client.1.smithi033.stdout:2/884: rename dc/d17/d27/d2f/d49/f85 to dc/d13/dfb/f11c 0 2024-05-12T04:01:29.937 INFO:tasks.workunit.client.1.smithi033.stdout:4/883: creat dd/d11/f113 x:0 0 0 2024-05-12T04:01:29.939 INFO:tasks.workunit.client.0.smithi032.stdout:8/746: creat d0/d9/d1b/f101 x:0 0 0 2024-05-12T04:01:29.939 INFO:tasks.workunit.client.1.smithi033.stdout:3/851: fsync db/d17/d3a/f129 0 2024-05-12T04:01:29.944 INFO:tasks.workunit.client.1.smithi033.stdout:1/777: creat d11/d17/d27/d76/d93/ffd x:0 0 0 2024-05-12T04:01:29.947 INFO:tasks.workunit.client.1.smithi033.stdout:5/776: dread d3/d94/dc8/f22 [0,4194304] 0 2024-05-12T04:01:29.948 INFO:tasks.workunit.client.0.smithi032.stdout:2/705: symlink d1/d73/le9 0 2024-05-12T04:01:29.950 INFO:tasks.workunit.client.1.smithi033.stdout:8/843: truncate d1/d16/d27/d4d/d82/fc2 3453789 0 2024-05-12T04:01:29.950 INFO:tasks.workunit.client.0.smithi032.stdout:8/747: truncate d0/d9/d1b/d27/d88/d56/f5b 63017 0 2024-05-12T04:01:29.951 INFO:tasks.workunit.client.0.smithi032.stdout:4/626: write d0/d3/d12/f66 [1524908,39883] 0 2024-05-12T04:01:29.953 INFO:tasks.workunit.client.0.smithi032.stdout:5/794: rmdir d4/d7/d14/d26/d33/d72/dc9/dec/d68 39 2024-05-12T04:01:29.954 INFO:tasks.workunit.client.1.smithi033.stdout:4/884: dwrite dd/d7a/f9f [4194304,4194304] 0 2024-05-12T04:01:29.955 INFO:tasks.workunit.client.1.smithi033.stdout:4/885: chown dd/d7a/d87/fb7 7834265 1 2024-05-12T04:01:29.955 INFO:tasks.workunit.client.1.smithi033.stdout:1/778: mkdir d11/d17/d27/da6/dfe 0 2024-05-12T04:01:29.956 INFO:tasks.workunit.client.0.smithi032.stdout:3/758: getdents d15/d20/d22/d7a/dbe 0 2024-05-12T04:01:29.958 INFO:tasks.workunit.client.1.smithi033.stdout:2/885: link dc/d17/d27/d2f/d49/d59/f104 dc/d13/dfb/f11d 0 2024-05-12T04:01:29.967 INFO:tasks.workunit.client.0.smithi032.stdout:8/748: rename d0/d7/f4a to d0/d9/d13/f102 0 2024-05-12T04:01:29.969 INFO:tasks.workunit.client.1.smithi033.stdout:3/852: write db/d59/f74 [750013,93860] 0 2024-05-12T04:01:29.973 INFO:tasks.workunit.client.1.smithi033.stdout:1/779: creat d11/d1c/ddf/d4c/dab/fff x:0 0 0 2024-05-12T04:01:29.974 INFO:tasks.workunit.client.1.smithi033.stdout:2/886: dread dc/d17/d27/d2f/d49/ff9 [0,4194304] 0 2024-05-12T04:01:29.979 INFO:tasks.workunit.client.0.smithi032.stdout:8/749: unlink d0/d9/d13/d21/f9d 0 2024-05-12T04:01:29.979 INFO:tasks.workunit.client.1.smithi033.stdout:3/853: rmdir db/d17/d3e/dd7/dd0 39 2024-05-12T04:01:29.980 INFO:tasks.workunit.client.0.smithi032.stdout:8/750: write d0/d97/fa6 [4770746,8622] 0 2024-05-12T04:01:29.981 INFO:tasks.workunit.client.1.smithi033.stdout:8/844: dwrite d1/d16/d29/db1/fb5 [0,4194304] 0 2024-05-12T04:01:29.983 INFO:tasks.workunit.client.1.smithi033.stdout:6/883: sync 2024-05-12T04:01:29.983 INFO:tasks.workunit.client.1.smithi033.stdout:3/854: write db/d17/d25/d32/d3c/d3d/f127 [705338,91346] 0 2024-05-12T04:01:29.983 INFO:tasks.workunit.client.1.smithi033.stdout:8/845: chown d1/d16/d29/db1/cc7 2 1 2024-05-12T04:01:29.984 INFO:tasks.workunit.client.1.smithi033.stdout:5/777: truncate d3/d94/d54/d57/d70/da3/dc9/fb2 1439763 0 2024-05-12T04:01:29.984 INFO:tasks.workunit.client.1.smithi033.stdout:4/886: fdatasync dd/d11/d75/f8d 0 2024-05-12T04:01:29.985 INFO:tasks.workunit.client.0.smithi032.stdout:5/795: dwrite d4/d7/d14/d26/f57 [0,4194304] 0 2024-05-12T04:01:29.987 INFO:tasks.workunit.client.0.smithi032.stdout:5/796: dread - d4/d7/d14/d26/d33/d4a/ff1 zero size 2024-05-12T04:01:29.989 INFO:tasks.workunit.client.1.smithi033.stdout:1/780: rmdir d11/d17/d27/d36/d3f/d4a/dc6/de1 39 2024-05-12T04:01:29.990 INFO:tasks.workunit.client.1.smithi033.stdout:2/887: symlink dc/d13/d1f/d6e/l11e 0 2024-05-12T04:01:29.996 INFO:tasks.workunit.client.0.smithi032.stdout:3/759: creat d15/d24/f105 x:0 0 0 2024-05-12T04:01:29.996 INFO:tasks.workunit.client.1.smithi033.stdout:8/846: dread d1/d16/d27/d7c/fc5 [0,4194304] 0 2024-05-12T04:01:29.998 INFO:tasks.workunit.client.1.smithi033.stdout:4/887: rename fc to dd/d7a/d87/d7e/da7/d66/d42/d9c/f114 0 2024-05-12T04:01:29.999 INFO:tasks.workunit.client.0.smithi032.stdout:2/706: getdents d1/d13/d70/d85/dd5 0 2024-05-12T04:01:30.001 INFO:tasks.workunit.client.0.smithi032.stdout:4/627: getdents d0/d75 0 2024-05-12T04:01:30.002 INFO:tasks.workunit.client.1.smithi033.stdout:5/778: read - d3/d94/d54/d57/d70/da3/dbe/d16/dd7/fb9 zero size 2024-05-12T04:01:30.004 INFO:tasks.workunit.client.0.smithi032.stdout:8/751: mkdir d0/d9/d1b/d66/d75/de2/d103 0 2024-05-12T04:01:30.005 INFO:tasks.workunit.client.1.smithi033.stdout:2/888: read - dc/d17/d27/d2f/d49/d59/fcf zero size 2024-05-12T04:01:30.007 INFO:tasks.workunit.client.1.smithi033.stdout:3/855: symlink db/d17/d3a/dc0/l12a 0 2024-05-12T04:01:30.009 INFO:tasks.workunit.client.1.smithi033.stdout:1/781: creat d11/d17/d27/d36/d3f/d69/d87/d96/f100 x:0 0 0 2024-05-12T04:01:30.012 INFO:tasks.workunit.client.0.smithi032.stdout:5/797: creat d4/d7/d14/d39/d2b/dd6/f118 x:0 0 0 2024-05-12T04:01:30.013 INFO:tasks.workunit.client.0.smithi032.stdout:2/707: dwrite d1/d34/da5/fbd [0,4194304] 0 2024-05-12T04:01:30.013 INFO:tasks.workunit.client.0.smithi032.stdout:3/760: link d15/d24/d93/f70 d15/d20/d22/d7a/f106 0 2024-05-12T04:01:30.014 INFO:tasks.workunit.client.0.smithi032.stdout:5/798: chown d4/d7/d14/d26/d33/d4a/d38/fd7 377320 1 2024-05-12T04:01:30.015 INFO:tasks.workunit.client.1.smithi033.stdout:8/847: unlink d1/d16/d27/d4d/d73/lf9 0 2024-05-12T04:01:30.016 INFO:tasks.workunit.client.1.smithi033.stdout:1/782: dwrite d11/d17/d27/d76/fa7 [4194304,4194304] 0 2024-05-12T04:01:30.018 INFO:tasks.workunit.client.1.smithi033.stdout:4/888: fsync dd/f29 0 2024-05-12T04:01:30.020 INFO:tasks.workunit.client.0.smithi032.stdout:3/761: dread d15/d20/d26/fee [0,4194304] 0 2024-05-12T04:01:30.020 INFO:tasks.workunit.client.1.smithi033.stdout:5/779: stat d3/c11 0 2024-05-12T04:01:30.022 INFO:tasks.workunit.client.0.smithi032.stdout:5/799: mkdir d4/d7/d14/d26/d33/d5f/d7f/d119 0 2024-05-12T04:01:30.023 INFO:tasks.workunit.client.1.smithi033.stdout:5/780: truncate d3/d94/dc8/fa8 4885096 0 2024-05-12T04:01:30.025 INFO:tasks.workunit.client.0.smithi032.stdout:8/752: rename d0/d9/d13/d92/ldd to d0/d9/d13/d21/d25/db5/l104 0 2024-05-12T04:01:30.027 INFO:tasks.workunit.client.0.smithi032.stdout:8/753: write d0/d7/f6c [1903405,119520] 0 2024-05-12T04:01:30.027 INFO:tasks.workunit.client.1.smithi033.stdout:5/781: chown d3/d94/d54/d57/d70/da3/dbe/d16/d45/d62/f64 0 1 2024-05-12T04:01:30.029 INFO:tasks.workunit.client.0.smithi032.stdout:4/628: creat d0/fb9 x:0 0 0 2024-05-12T04:01:30.031 INFO:tasks.workunit.client.0.smithi032.stdout:4/629: read d0/d3/fd [170073,72946] 0 2024-05-12T04:01:30.032 INFO:tasks.workunit.client.0.smithi032.stdout:4/630: read - d0/d75/f78 zero size 2024-05-12T04:01:30.032 INFO:tasks.workunit.client.1.smithi033.stdout:8/848: mknod d1/d16/d27/d28/c101 0 2024-05-12T04:01:30.033 INFO:tasks.workunit.client.0.smithi032.stdout:4/631: readlink d0/d3/d12/d1d/l2b 0 2024-05-12T04:01:30.037 INFO:tasks.workunit.client.1.smithi033.stdout:1/783: creat d11/d17/d84/de7/f101 x:0 0 0 2024-05-12T04:01:30.040 INFO:tasks.workunit.client.1.smithi033.stdout:4/889: creat dd/d7a/d87/d7e/da7/d66/d42/d59/f115 x:0 0 0 2024-05-12T04:01:30.045 INFO:tasks.workunit.client.1.smithi033.stdout:6/884: truncate d4/d5/fab 2571267 0 2024-05-12T04:01:30.046 INFO:tasks.workunit.client.1.smithi033.stdout:6/885: stat d4/d5/f10 0 2024-05-12T04:01:30.047 INFO:tasks.workunit.client.0.smithi032.stdout:2/708: creat d1/d34/d96/d41/fea x:0 0 0 2024-05-12T04:01:30.052 INFO:tasks.workunit.client.1.smithi033.stdout:1/784: creat d11/d17/d27/d36/d3f/d69/f102 x:0 0 0 2024-05-12T04:01:30.052 INFO:tasks.workunit.client.1.smithi033.stdout:1/785: chown d11/d1c/f33 7 1 2024-05-12T04:01:30.054 INFO:tasks.workunit.client.1.smithi033.stdout:8/849: rmdir d1/d16/d27/d28 39 2024-05-12T04:01:30.080 INFO:tasks.workunit.client.1.smithi033.stdout:0/901: sync 2024-05-12T04:01:30.106 INFO:tasks.workunit.client.0.smithi032.stdout:9/578: sync 2024-05-12T04:01:30.156 INFO:tasks.workunit.client.1.smithi033.stdout:3/856: creat db/f12b x:0 0 0 2024-05-12T04:01:30.160 INFO:tasks.workunit.client.1.smithi033.stdout:6/886: creat d4/d1a/d6c/f125 x:0 0 0 2024-05-12T04:01:30.168 INFO:tasks.workunit.client.1.smithi033.stdout:4/890: rename dd/d11/c70 to dd/d7a/c116 0 2024-05-12T04:01:30.168 INFO:tasks.workunit.client.0.smithi032.stdout:8/754: write d0/d9/d1b/d27/d59/fb1 [174020,9731] 0 2024-05-12T04:01:30.169 INFO:tasks.workunit.client.1.smithi033.stdout:4/891: readlink dd/d7a/d87/d7e/da7/ldf 0 2024-05-12T04:01:30.169 INFO:tasks.workunit.client.1.smithi033.stdout:8/850: creat d1/d16/d27/d4d/d73/f102 x:0 0 0 2024-05-12T04:01:30.170 INFO:tasks.workunit.client.0.smithi032.stdout:9/579: mknod d0/d3/d4b/d88/cba 0 2024-05-12T04:01:30.176 INFO:tasks.workunit.client.1.smithi033.stdout:4/892: dwrite dd/d7a/d87/d7e/fe5 [0,4194304] 0 2024-05-12T04:01:30.177 INFO:tasks.workunit.client.1.smithi033.stdout:8/851: dread d1/d16/d27/d4d/d82/dd9/d36/d3c/d47/f60 [0,4194304] 0 2024-05-12T04:01:30.182 INFO:tasks.workunit.client.1.smithi033.stdout:2/889: sync 2024-05-12T04:01:30.182 INFO:tasks.workunit.client.0.smithi032.stdout:8/755: rmdir d0/d9/d1b 39 2024-05-12T04:01:30.183 INFO:tasks.workunit.client.1.smithi033.stdout:1/786: rmdir d11/d17/d27/d36/d3f/d4a/dc6/de1 39 2024-05-12T04:01:30.184 INFO:tasks.workunit.client.1.smithi033.stdout:1/787: dread - d11/d17/d27/d36/d3f/ff1 zero size 2024-05-12T04:01:30.185 INFO:tasks.workunit.client.0.smithi032.stdout:5/800: getdents d4/d7/d58/d5a 0 2024-05-12T04:01:30.187 INFO:tasks.workunit.client.0.smithi032.stdout:9/580: symlink d0/d3/d4b/d88/d99/lbb 0 2024-05-12T04:01:30.193 INFO:tasks.workunit.client.1.smithi033.stdout:5/782: link d3/d12/d25/d26/l76 d3/d94/d85/d99/l101 0 2024-05-12T04:01:30.196 INFO:tasks.workunit.client.1.smithi033.stdout:4/893: readlink dd/d11/d5b/d84/dd7/lec 0 2024-05-12T04:01:30.196 INFO:tasks.workunit.client.0.smithi032.stdout:4/632: dwrite d0/d3/f5e [0,4194304] 0 2024-05-12T04:01:30.198 INFO:tasks.workunit.client.0.smithi032.stdout:8/756: mkdir d0/d9/d1b/d66/d75/de2/d105 0 2024-05-12T04:01:30.199 INFO:tasks.workunit.client.1.smithi033.stdout:5/783: dwrite d3/fc [0,4194304] 0 2024-05-12T04:01:30.203 INFO:tasks.workunit.client.0.smithi032.stdout:4/633: truncate d0/d3/df/fa5 376921 0 2024-05-12T04:01:30.203 INFO:tasks.workunit.client.0.smithi032.stdout:5/801: link d4/d7/d14/d26/d33/d72/dc9/dec/fba d4/d7/d58/f11a 0 2024-05-12T04:01:30.204 INFO:tasks.workunit.client.1.smithi033.stdout:8/852: symlink d1/d16/d27/d4d/d51/d8e/db6/l103 0 2024-05-12T04:01:30.206 INFO:tasks.workunit.client.0.smithi032.stdout:3/762: dwrite d15/d33/d3f/d75/ff8 [0,4194304] 0 2024-05-12T04:01:30.207 INFO:tasks.workunit.client.1.smithi033.stdout:3/857: write db/d17/d3e/d65/d91/fb3 [1825278,56233] 0 2024-05-12T04:01:30.209 INFO:tasks.workunit.client.0.smithi032.stdout:4/634: chown d0/d3/d12/d1d/l57 1 1 2024-05-12T04:01:30.210 INFO:tasks.workunit.client.0.smithi032.stdout:2/709: dwrite d1/f24 [0,4194304] 0 2024-05-12T04:01:30.210 INFO:tasks.workunit.client.0.smithi032.stdout:9/581: mknod d0/d84/d8a/da3/db7/cbc 0 2024-05-12T04:01:30.211 INFO:tasks.workunit.client.0.smithi032.stdout:9/582: dread - d0/d3/d6/d7/d2a/d46/d13/fb6 zero size 2024-05-12T04:01:30.214 INFO:tasks.workunit.client.1.smithi033.stdout:0/902: link d6/dc/d39/lbb d6/dc/d58/d6a/d80/db8/d124/l136 0 2024-05-12T04:01:30.215 INFO:tasks.workunit.client.1.smithi033.stdout:0/903: chown d6/d11 1 1 2024-05-12T04:01:30.218 INFO:tasks.workunit.client.0.smithi032.stdout:0/716: sync 2024-05-12T04:01:30.219 INFO:tasks.workunit.client.1.smithi033.stdout:6/887: dwrite d4/d6e/f90 [0,4194304] 0 2024-05-12T04:01:30.220 INFO:tasks.workunit.client.0.smithi032.stdout:0/717: write df/d10/d3d/f85 [4320229,77405] 0 2024-05-12T04:01:30.225 INFO:tasks.workunit.client.1.smithi033.stdout:4/894: unlink dd/d7a/d87/d7e/da7/d38/c4d 0 2024-05-12T04:01:30.231 INFO:tasks.workunit.client.0.smithi032.stdout:2/710: creat d1/d34/d4a/feb x:0 0 0 2024-05-12T04:01:30.233 INFO:tasks.workunit.client.0.smithi032.stdout:2/711: truncate d1/d4d/d5e/fdc 80063 0 2024-05-12T04:01:30.234 INFO:tasks.workunit.client.1.smithi033.stdout:2/890: mkdir dc/d17/dcc/df1/d11b/d11f 0 2024-05-12T04:01:30.235 INFO:tasks.workunit.client.0.smithi032.stdout:0/718: dread df/d10/f1b [0,4194304] 0 2024-05-12T04:01:30.236 INFO:tasks.workunit.client.0.smithi032.stdout:8/757: creat d0/d9/d1b/d27/d88/d56/d63/d8f/d9a/d28/f106 x:0 0 0 2024-05-12T04:01:30.238 INFO:tasks.workunit.client.0.smithi032.stdout:8/758: write d0/d9/d1b/d27/d88/d56/d63/d8f/d9a/d28/d3e/fb3 [2595464,112016] 0 2024-05-12T04:01:30.238 INFO:tasks.workunit.client.1.smithi033.stdout:3/858: write db/d17/d25/d32/d53/fb6 [3059221,91204] 0 2024-05-12T04:01:30.240 INFO:tasks.workunit.client.1.smithi033.stdout:1/788: creat d11/d1c/ddf/d4c/dab/dee/f103 x:0 0 0 2024-05-12T04:01:30.240 INFO:tasks.workunit.client.1.smithi033.stdout:1/789: stat d11/d17/fd7 0 2024-05-12T04:01:30.241 INFO:tasks.workunit.client.0.smithi032.stdout:3/763: symlink d15/d20/d26/d6a/d6e/l107 0 2024-05-12T04:01:30.241 INFO:tasks.workunit.client.1.smithi033.stdout:1/790: dread - d11/d17/d27/fbc zero size 2024-05-12T04:01:30.245 INFO:tasks.workunit.client.1.smithi033.stdout:0/904: unlink d6/dc/d39/l69 0 2024-05-12T04:01:30.246 INFO:tasks.workunit.client.1.smithi033.stdout:0/905: chown d6/db/dd6 0 1 2024-05-12T04:01:30.246 INFO:tasks.workunit.client.1.smithi033.stdout:0/906: stat d6/d11/dc7/lde 0 2024-05-12T04:01:30.247 INFO:tasks.workunit.client.1.smithi033.stdout:6/888: read d4/f16 [7029361,1174] 0 2024-05-12T04:01:30.248 INFO:tasks.workunit.client.0.smithi032.stdout:4/635: dread d0/f3d [0,4194304] 0 2024-05-12T04:01:30.250 INFO:tasks.workunit.client.1.smithi033.stdout:4/895: mkdir dd/d69/dc4/d117 0 2024-05-12T04:01:30.252 INFO:tasks.workunit.client.0.smithi032.stdout:0/719: symlink df/d30/d32/d60/dc7/lef 0 2024-05-12T04:01:30.253 INFO:tasks.workunit.client.0.smithi032.stdout:5/802: dwrite d4/d7/d14/f8c [4194304,4194304] 0 2024-05-12T04:01:30.257 INFO:tasks.workunit.client.1.smithi033.stdout:8/853: fsync d1/d16/d27/d4d/d82/dd9/d36/d3c/d47/ff7 0 2024-05-12T04:01:30.259 INFO:tasks.workunit.client.1.smithi033.stdout:2/891: truncate dc/d17/f4d 966033 0 2024-05-12T04:01:30.264 INFO:tasks.workunit.client.0.smithi032.stdout:9/583: truncate d0/d3/d6/d58/fa7 3449945 0 2024-05-12T04:01:30.266 INFO:tasks.workunit.client.0.smithi032.stdout:2/712: write d1/d34/d96/d41/d2c/f90 [1558960,22523] 0 2024-05-12T04:01:30.269 INFO:tasks.workunit.client.0.smithi032.stdout:1/682: sync 2024-05-12T04:01:30.270 INFO:tasks.workunit.client.0.smithi032.stdout:6/645: sync 2024-05-12T04:01:30.270 INFO:tasks.workunit.client.0.smithi032.stdout:7/671: sync 2024-05-12T04:01:30.270 INFO:tasks.workunit.client.0.smithi032.stdout:1/683: write d1/d7/d30/d18/d87/dc6/fd0 [921593,23896] 0 2024-05-12T04:01:30.271 INFO:tasks.workunit.client.1.smithi033.stdout:1/791: write d11/d17/d27/d36/d3f/d69/d87/da4/ff6 [3338,110930] 0 2024-05-12T04:01:30.271 INFO:tasks.workunit.client.0.smithi032.stdout:5/803: fdatasync d4/d7/d14/d39/d2b/ff6 0 2024-05-12T04:01:30.273 INFO:tasks.workunit.client.0.smithi032.stdout:1/684: read d1/d7/d30/d35/d54/d7e/fca [545004,105198] 0 2024-05-12T04:01:30.276 INFO:tasks.workunit.client.1.smithi033.stdout:5/784: rename d3/d94/d54/d57/d70/da3/dbe/d16/d45/d62/d7f/d84/cd6 to d3/d12/d25/c102 0 2024-05-12T04:01:30.280 INFO:tasks.workunit.client.0.smithi032.stdout:8/759: dwrite d0/d9/d1b/d27/d88/f39 [0,4194304] 0 2024-05-12T04:01:30.284 INFO:tasks.workunit.client.1.smithi033.stdout:6/889: dwrite d4/d1a/d4e/fb5 [0,4194304] 0 2024-05-12T04:01:30.284 INFO:tasks.workunit.client.0.smithi032.stdout:9/584: creat d0/d3/d4b/d88/d99/fbd x:0 0 0 2024-05-12T04:01:30.284 INFO:tasks.workunit.client.0.smithi032.stdout:3/764: dwrite d15/d20/d26/d6a/f77 [0,4194304] 0 2024-05-12T04:01:30.289 INFO:tasks.workunit.client.0.smithi032.stdout:3/765: write d15/d20/d26/f5f [1579988,116658] 0 2024-05-12T04:01:30.294 INFO:tasks.workunit.client.1.smithi033.stdout:0/907: unlink d6/c42 0 2024-05-12T04:01:30.295 INFO:tasks.workunit.client.1.smithi033.stdout:0/908: chown d6/d11/d98/fa1 230 1 2024-05-12T04:01:30.300 INFO:tasks.workunit.client.1.smithi033.stdout:1/792: creat d11/d17/d27/d36/d3f/d69/d87/da4/f104 x:0 0 0 2024-05-12T04:01:30.304 INFO:tasks.workunit.client.0.smithi032.stdout:5/804: symlink d4/d7/d14/l11b 0 2024-05-12T04:01:30.304 INFO:tasks.workunit.client.0.smithi032.stdout:3/766: chown d15/d24/d2a/d8c/db1/dc8/de4 91 1 2024-05-12T04:01:30.305 INFO:tasks.workunit.client.0.smithi032.stdout:1/685: creat d1/d7/d30/d18/d87/dc6/fde x:0 0 0 2024-05-12T04:01:30.306 INFO:tasks.workunit.client.0.smithi032.stdout:4/636: creat d0/d3/d12/d1d/fba x:0 0 0 2024-05-12T04:01:30.309 INFO:tasks.workunit.client.0.smithi032.stdout:3/767: chown d15/d20/d26/d6a/d6e/d7d/fd9 4 1 2024-05-12T04:01:30.311 INFO:tasks.workunit.client.0.smithi032.stdout:2/713: dwrite d1/d4d/d5e/f71 [0,4194304] 0 2024-05-12T04:01:30.312 INFO:tasks.workunit.client.1.smithi033.stdout:2/892: link dc/d17/d27/fda dc/d17/dcc/f120 0 2024-05-12T04:01:30.313 INFO:tasks.workunit.client.1.smithi033.stdout:2/893: write dc/d13/d1c/f4f [3311537,6075] 0 2024-05-12T04:01:30.314 INFO:tasks.workunit.client.0.smithi032.stdout:3/768: fdatasync d15/d20/d22/d4f/de0/fe7 0 2024-05-12T04:01:30.316 INFO:tasks.workunit.client.1.smithi033.stdout:5/785: write d3/d94/f49 [3574562,7768] 0 2024-05-12T04:01:30.318 INFO:tasks.workunit.client.0.smithi032.stdout:1/686: dwrite d1/d7/d30/d35/d77/faf [0,4194304] 0 2024-05-12T04:01:30.320 INFO:tasks.workunit.client.0.smithi032.stdout:1/687: fsync d1/d6a/d72/da9/f8a 0 2024-05-12T04:01:30.322 INFO:tasks.workunit.client.1.smithi033.stdout:8/854: truncate d1/d16/d27/f86 106684 0 2024-05-12T04:01:30.322 INFO:tasks.workunit.client.1.smithi033.stdout:0/909: dread d6/d11/d98/fa1 [0,4194304] 0 2024-05-12T04:01:30.324 INFO:tasks.workunit.client.0.smithi032.stdout:9/585: rename d0/d3/d4b/d73 to d0/d3/d4b/d85/dbe 0 2024-05-12T04:01:30.326 INFO:tasks.workunit.client.1.smithi033.stdout:1/793: symlink d11/d17/d27/d76/d93/l105 0 2024-05-12T04:01:30.327 INFO:tasks.workunit.client.1.smithi033.stdout:1/794: stat d11/d1c/ddf/f66 0 2024-05-12T04:01:30.328 INFO:tasks.workunit.client.1.smithi033.stdout:1/795: fsync d11/d17/d27/d36/d3f/fd0 0 2024-05-12T04:01:30.329 INFO:tasks.workunit.client.0.smithi032.stdout:7/672: write dd/d1b/f4b [619251,77991] 0 2024-05-12T04:01:30.334 INFO:tasks.workunit.client.1.smithi033.stdout:2/894: unlink dc/d17/d27/d2f/d49/d59/f79 0 2024-05-12T04:01:30.336 INFO:tasks.workunit.client.1.smithi033.stdout:3/859: getdents db/d17/d25/d32/d3c 0 2024-05-12T04:01:30.341 INFO:tasks.workunit.client.1.smithi033.stdout:2/895: dwrite dc/d13/d1f/d6e/de8/f108 [0,4194304] 0 2024-05-12T04:01:30.345 INFO:tasks.workunit.client.1.smithi033.stdout:4/896: getdents dd/d7a/d87/d7e/da7/d38/deb 0 2024-05-12T04:01:30.346 INFO:tasks.workunit.client.1.smithi033.stdout:8/855: mknod d1/d16/d27/d7c/db7/c104 0 2024-05-12T04:01:30.347 INFO:tasks.workunit.client.1.smithi033.stdout:6/890: truncate d4/d1a/d67/d68/f72 1081827 0 2024-05-12T04:01:30.348 INFO:tasks.workunit.client.1.smithi033.stdout:8/856: read d1/d16/d27/d4d/d82/dd9/d36/fce [2467119,82114] 0 2024-05-12T04:01:30.350 INFO:tasks.workunit.client.1.smithi033.stdout:0/910: mkdir d6/dc/df1/d137 0 2024-05-12T04:01:30.355 INFO:tasks.workunit.client.1.smithi033.stdout:3/860: fsync db/d59/f5b 0 2024-05-12T04:01:30.363 INFO:tasks.workunit.client.1.smithi033.stdout:2/896: truncate dc/d17/d32/fe4 344905 0 2024-05-12T04:01:30.369 INFO:tasks.workunit.client.1.smithi033.stdout:2/897: dwrite dc/d17/d27/d2f/f6a [0,4194304] 0 2024-05-12T04:01:30.375 INFO:tasks.workunit.client.1.smithi033.stdout:1/796: dwrite d11/d1c/d43/fa1 [0,4194304] 0 2024-05-12T04:01:30.382 INFO:tasks.workunit.client.1.smithi033.stdout:0/911: dwrite d6/dc/d58/f86 [0,4194304] 0 2024-05-12T04:01:30.383 INFO:tasks.workunit.client.1.smithi033.stdout:0/912: stat d6/d11/d57/lbc 0 2024-05-12T04:01:30.388 INFO:tasks.workunit.client.1.smithi033.stdout:5/786: rmdir d3/d94/d54/d98/dcc 0 2024-05-12T04:01:30.391 INFO:tasks.workunit.client.1.smithi033.stdout:4/897: dwrite dd/d7a/d87/d7e/da7/d66/fe0 [0,4194304] 0 2024-05-12T04:01:30.392 INFO:tasks.workunit.client.1.smithi033.stdout:8/857: write d1/d16/d27/d4d/d82/dd9/d36/d3c/da7/f43 [3206890,39943] 0 2024-05-12T04:01:30.394 INFO:tasks.workunit.client.1.smithi033.stdout:8/858: dread - d1/d16/d27/d4d/d82/dd9/d36/faa zero size 2024-05-12T04:01:30.395 INFO:tasks.workunit.client.1.smithi033.stdout:4/898: write dd/d11/d5b/dd1/f108 [1013372,17043] 0 2024-05-12T04:01:30.395 INFO:tasks.workunit.client.0.smithi032.stdout:8/760: getdents d0/d9/d1b/d66/d75/d84/dee 0 2024-05-12T04:01:30.396 INFO:tasks.workunit.client.0.smithi032.stdout:8/761: readlink d0/d4b/dad/ld4 0 2024-05-12T04:01:30.401 INFO:tasks.workunit.client.0.smithi032.stdout:5/805: mknod d4/d7/d14/d26/d33/d4a/d38/c11c 0 2024-05-12T04:01:30.406 INFO:tasks.workunit.client.0.smithi032.stdout:0/720: link df/d10/d3d/d4a/f38 df/d30/dc2/ff0 0 2024-05-12T04:01:30.412 INFO:tasks.workunit.client.1.smithi033.stdout:0/913: read d6/db/d2c/d73/daf/dc3/fcf [15175,85215] 0 2024-05-12T04:01:30.413 INFO:tasks.workunit.client.1.smithi033.stdout:0/914: write d6/dc/d58/f86 [5092530,21016] 0 2024-05-12T04:01:30.415 INFO:tasks.workunit.client.1.smithi033.stdout:5/787: creat d3/d12/d29/d2b/d4f/f103 x:0 0 0 2024-05-12T04:01:30.417 INFO:tasks.workunit.client.0.smithi032.stdout:1/688: mknod d1/d6a/d72/da4/cdf 0 2024-05-12T04:01:30.423 INFO:tasks.workunit.client.1.smithi033.stdout:8/859: rename d1/d16/d29/db1/fb5 to d1/d4/d46/f105 0 2024-05-12T04:01:30.424 INFO:tasks.workunit.client.0.smithi032.stdout:4/637: dwrite d0/d3/fd [0,4194304] 0 2024-05-12T04:01:30.424 INFO:tasks.workunit.client.1.smithi033.stdout:8/860: write d1/d16/fbe [4848505,27812] 0 2024-05-12T04:01:30.425 INFO:tasks.workunit.client.0.smithi032.stdout:4/638: fsync d0/d3/df/d4e/f69 0 2024-05-12T04:01:30.426 INFO:tasks.workunit.client.0.smithi032.stdout:0/721: creat df/d22/d58/d8a/ff1 x:0 0 0 2024-05-12T04:01:30.429 INFO:tasks.workunit.client.0.smithi032.stdout:3/769: write d15/d24/f90 [949086,78236] 0 2024-05-12T04:01:30.432 INFO:tasks.workunit.client.1.smithi033.stdout:3/861: dwrite db/d17/d3a/f63 [0,4194304] 0 2024-05-12T04:01:30.433 INFO:tasks.workunit.client.0.smithi032.stdout:2/714: rename d1/d34/d4a/l4e to d1/d34/lec 0 2024-05-12T04:01:30.435 INFO:tasks.workunit.client.0.smithi032.stdout:1/689: dread d1/d7/d30/d61/f68 [0,4194304] 0 2024-05-12T04:01:30.435 INFO:tasks.workunit.client.0.smithi032.stdout:2/715: chown d1/d34/d3e/f45 26763333 1 2024-05-12T04:01:30.437 INFO:tasks.workunit.client.0.smithi032.stdout:0/722: dwrite df/d30/d32/d60/f90 [0,4194304] 0 2024-05-12T04:01:30.439 INFO:tasks.workunit.client.1.smithi033.stdout:2/898: truncate dc/fe3 1982410 0 2024-05-12T04:01:30.442 INFO:tasks.workunit.client.1.smithi033.stdout:5/788: mkdir d3/d94/d54/d57/d70/da3/dbe/d16/d78/d104 0 2024-05-12T04:01:30.442 INFO:tasks.workunit.client.0.smithi032.stdout:1/690: dread d1/d7/d30/d61/f68 [0,4194304] 0 2024-05-12T04:01:30.443 INFO:tasks.workunit.client.1.smithi033.stdout:5/789: chown d3/d94/d54/d57/d73/d74 630645 1 2024-05-12T04:01:30.444 INFO:tasks.workunit.client.1.smithi033.stdout:0/915: dread d6/d11/d98/fa7 [0,4194304] 0 2024-05-12T04:01:30.445 INFO:tasks.workunit.client.1.smithi033.stdout:6/891: link d4/d6e/l7c d4/dad/l126 0 2024-05-12T04:01:30.452 INFO:tasks.workunit.client.0.smithi032.stdout:0/723: fsync df/d28/f59 0 2024-05-12T04:01:30.453 INFO:tasks.workunit.client.0.smithi032.stdout:2/716: dread d1/d34/d96/d41/d2c/f37 [0,4194304] 0 2024-05-12T04:01:30.454 INFO:tasks.workunit.client.0.smithi032.stdout:4/639: unlink d0/d3/df/l15 0 2024-05-12T04:01:30.455 INFO:tasks.workunit.client.0.smithi032.stdout:2/717: write d1/d34/d96/d41/d2c/f90 [1665604,94649] 0 2024-05-12T04:01:30.455 INFO:tasks.workunit.client.0.smithi032.stdout:0/724: truncate df/d30/d8d/d8f/fcb 870055 0 2024-05-12T04:01:30.456 INFO:tasks.workunit.client.1.smithi033.stdout:1/797: link d11/d17/d27/d36/d3f/d69/d87/fce d11/d17/d27/da6/f106 0 2024-05-12T04:01:30.456 INFO:tasks.workunit.client.0.smithi032.stdout:2/718: chown d1/d34/d4a/d86/cac 450201 1 2024-05-12T04:01:30.457 INFO:tasks.workunit.client.0.smithi032.stdout:2/719: chown d1/d4d/d5e/d9a 114774913 1 2024-05-12T04:01:30.460 INFO:tasks.workunit.client.1.smithi033.stdout:5/790: rmdir d3/d12/d29/d2b/d4f 39 2024-05-12T04:01:30.462 INFO:tasks.workunit.client.0.smithi032.stdout:1/691: creat d1/d7/d30/d35/dbe/fe0 x:0 0 0 2024-05-12T04:01:30.467 INFO:tasks.workunit.client.0.smithi032.stdout:9/586: rename d0/d3/d6/d7/d27/daa to d0/d3/d74/dbf 0 2024-05-12T04:01:30.469 INFO:tasks.workunit.client.0.smithi032.stdout:1/692: symlink d1/d7/d30/d18/d93/dc1/le1 0 2024-05-12T04:01:30.470 INFO:tasks.workunit.client.1.smithi033.stdout:3/862: mknod db/c12c 0 2024-05-12T04:01:30.474 INFO:tasks.workunit.client.0.smithi032.stdout:8/762: rename d0/d9/d1b/d66/l8a to d0/d9/d13/d21/d25/db5/l107 0 2024-05-12T04:01:30.475 INFO:tasks.workunit.client.1.smithi033.stdout:6/892: dread d4/f82 [0,4194304] 0 2024-05-12T04:01:30.475 INFO:tasks.workunit.client.0.smithi032.stdout:9/587: truncate d0/f75 845567 0 2024-05-12T04:01:30.476 INFO:tasks.workunit.client.0.smithi032.stdout:9/588: chown d0/d3/d6/d7/da/l65 18023 1 2024-05-12T04:01:30.478 INFO:tasks.workunit.client.0.smithi032.stdout:3/770: write d15/d20/d22/d7a/f106 [2802655,104420] 0 2024-05-12T04:01:30.480 INFO:tasks.workunit.client.1.smithi033.stdout:8/861: creat d1/d16/f106 x:0 0 0 2024-05-12T04:01:30.482 INFO:tasks.workunit.client.0.smithi032.stdout:8/763: rmdir d0/d4b 39 2024-05-12T04:01:30.484 INFO:tasks.workunit.client.1.smithi033.stdout:4/899: truncate dd/d69/f7d 1059336 0 2024-05-12T04:01:30.489 INFO:tasks.workunit.client.0.smithi032.stdout:9/589: unlink d0/d3/d6/d7/da/d39/l37 0 2024-05-12T04:01:30.493 INFO:tasks.workunit.client.0.smithi032.stdout:3/771: symlink d15/d24/d2a/d8c/db1/l108 0 2024-05-12T04:01:30.493 INFO:tasks.workunit.client.0.smithi032.stdout:4/640: write d0/d75/f78 [306791,8510] 0 2024-05-12T04:01:30.493 INFO:tasks.workunit.client.1.smithi033.stdout:3/863: read db/d17/d3a/d8a/ffd [2202251,103333] 0 2024-05-12T04:01:30.493 INFO:tasks.workunit.client.1.smithi033.stdout:5/791: mknod d3/d94/d54/d57/d70/da3/dbe/d16/d78/d104/c105 0 2024-05-12T04:01:30.494 INFO:tasks.workunit.client.1.smithi033.stdout:0/916: write d6/d11/dc7/fdf [4043415,17954] 0 2024-05-12T04:01:30.494 INFO:tasks.workunit.client.1.smithi033.stdout:8/862: dread - d1/d16/fa6 zero size 2024-05-12T04:01:30.497 INFO:tasks.workunit.client.1.smithi033.stdout:1/798: creat d11/d17/d27/d36/d3f/f107 x:0 0 0 2024-05-12T04:01:30.498 INFO:tasks.workunit.client.1.smithi033.stdout:2/899: dwrite dc/d13/d1f/d6e/de8/faa [0,4194304] 0 2024-05-12T04:01:30.500 INFO:tasks.workunit.client.1.smithi033.stdout:1/799: read - d11/d17/d84/de7/f101 zero size 2024-05-12T04:01:30.500 INFO:tasks.workunit.client.1.smithi033.stdout:4/900: truncate dd/d11/f21 4641825 0 2024-05-12T04:01:30.502 INFO:tasks.workunit.client.0.smithi032.stdout:5/806: rename d4/d7/d14/df3 to d4/d7/d14/d26/d33/d4a/d11d 0 2024-05-12T04:01:30.505 INFO:tasks.workunit.client.1.smithi033.stdout:5/792: dwrite d3/d94/d54/d57/d70/da3/dbe/d16/d45/d62/f64 [0,4194304] 0 2024-05-12T04:01:30.507 INFO:tasks.workunit.client.0.smithi032.stdout:4/641: chown d0/d3/d12/c33 99292 1 2024-05-12T04:01:30.511 INFO:tasks.workunit.client.0.smithi032.stdout:4/642: write d0/d3/d12/fa2 [537416,11752] 0 2024-05-12T04:01:30.514 INFO:tasks.workunit.client.0.smithi032.stdout:4/643: fsync d0/d75/f78 0 2024-05-12T04:01:30.515 INFO:tasks.workunit.client.0.smithi032.stdout:1/693: dwrite d1/d7/d30/f60 [0,4194304] 0 2024-05-12T04:01:30.518 INFO:tasks.workunit.client.0.smithi032.stdout:1/694: read - d1/d7/d30/d35/dbe/fe0 zero size 2024-05-12T04:01:30.524 INFO:tasks.workunit.client.0.smithi032.stdout:8/764: dwrite d0/d9/d1b/d66/fe8 [0,4194304] 0 2024-05-12T04:01:30.524 INFO:tasks.workunit.client.0.smithi032.stdout:1/695: read - d1/d7/d30/d35/d54/d7e/fbd zero size 2024-05-12T04:01:30.526 INFO:tasks.workunit.client.1.smithi033.stdout:3/864: dread db/d17/d25/f52 [0,4194304] 0 2024-05-12T04:01:30.532 INFO:tasks.workunit.client.0.smithi032.stdout:9/590: creat d0/d3/d74/fc0 x:0 0 0 2024-05-12T04:01:30.541 INFO:tasks.workunit.client.0.smithi032.stdout:5/807: creat d4/d7/d14/d39/d1b/da4/f11e x:0 0 0 2024-05-12T04:01:30.541 INFO:tasks.workunit.client.0.smithi032.stdout:5/808: read - d4/d7/d58/f102 zero size 2024-05-12T04:01:30.543 INFO:tasks.workunit.client.0.smithi032.stdout:3/772: creat d15/d33/d3f/d75/f109 x:0 0 0 2024-05-12T04:01:30.548 INFO:tasks.workunit.client.0.smithi032.stdout:9/591: rmdir d0/d3/d6/d7/da 39 2024-05-12T04:01:30.552 INFO:tasks.workunit.client.0.smithi032.stdout:0/725: rename df/d10/d3d/db0 to df/df2 0 2024-05-12T04:01:30.554 INFO:tasks.workunit.client.0.smithi032.stdout:5/809: mknod d4/d7/d14/d26/d33/c11f 0 2024-05-12T04:01:30.557 INFO:tasks.workunit.client.0.smithi032.stdout:5/810: dread d4/d7/d14/d26/f66 [0,4194304] 0 2024-05-12T04:01:30.558 INFO:tasks.workunit.client.0.smithi032.stdout:3/773: dread d15/d33/f74 [0,4194304] 0 2024-05-12T04:01:30.560 INFO:tasks.workunit.client.0.smithi032.stdout:8/765: mkdir d0/d9/d1b/d66/d75/de2/d103/d108 0 2024-05-12T04:01:30.562 INFO:tasks.workunit.client.0.smithi032.stdout:1/696: write d1/d7/d30/d35/f4e [1066637,17780] 0 2024-05-12T04:01:30.565 INFO:tasks.workunit.client.0.smithi032.stdout:0/726: readlink df/d30/l3f 0 2024-05-12T04:01:30.574 INFO:tasks.workunit.client.0.smithi032.stdout:1/697: symlink d1/d6a/d72/da9/le2 0 2024-05-12T04:01:30.576 INFO:tasks.workunit.client.0.smithi032.stdout:9/592: write d0/d3/d4b/d6f/f71 [2350823,53135] 0 2024-05-12T04:01:30.580 INFO:tasks.workunit.client.0.smithi032.stdout:5/811: write d4/d7/d58/f7b [995412,58485] 0 2024-05-12T04:01:30.581 INFO:tasks.workunit.client.0.smithi032.stdout:5/812: truncate d4/f20 2337806 0 2024-05-12T04:01:30.594 INFO:tasks.workunit.client.1.smithi033.stdout:6/893: mknod d4/d6e/dbb/d115/c127 0 2024-05-12T04:01:30.598 INFO:tasks.workunit.client.1.smithi033.stdout:4/901: rmdir dd/d7a 39 2024-05-12T04:01:30.600 INFO:tasks.workunit.client.1.smithi033.stdout:2/900: creat dc/d17/d27/d68/d9f/d5e/f121 x:0 0 0 2024-05-12T04:01:30.601 INFO:tasks.workunit.client.1.smithi033.stdout:2/901: chown dc/d3b/l45 3452 1 2024-05-12T04:01:30.601 INFO:tasks.workunit.client.1.smithi033.stdout:5/793: fsync d3/d94/d54/d57/d70/da3/dbe/fc1 0 2024-05-12T04:01:30.602 INFO:tasks.workunit.client.1.smithi033.stdout:2/902: truncate dc/d17/d27/fda 883379 0 2024-05-12T04:01:30.603 INFO:tasks.workunit.client.1.smithi033.stdout:5/794: chown d3/d12/dac/fce 61 1 2024-05-12T04:01:30.606 INFO:tasks.workunit.client.1.smithi033.stdout:6/894: fsync d4/d1a/d67/d6f/dac/ff6 0 2024-05-12T04:01:30.607 INFO:tasks.workunit.client.1.smithi033.stdout:4/902: creat dd/d69/dc4/f118 x:0 0 0 2024-05-12T04:01:30.608 INFO:tasks.workunit.client.1.smithi033.stdout:1/800: mkdir d11/d17/d27/d36/d3f/d69/d87/d108 0 2024-05-12T04:01:30.609 INFO:tasks.workunit.client.0.smithi032.stdout:1/698: creat d1/d7/d30/d18/fe3 x:0 0 0 2024-05-12T04:01:30.610 INFO:tasks.workunit.client.0.smithi032.stdout:0/727: unlink df/c20 0 2024-05-12T04:01:30.610 INFO:tasks.workunit.client.0.smithi032.stdout:9/593: fdatasync d0/f28 0 2024-05-12T04:01:30.612 INFO:tasks.workunit.client.0.smithi032.stdout:1/699: chown d1/d25/f5b 0 1 2024-05-12T04:01:30.613 INFO:tasks.workunit.client.1.smithi033.stdout:2/903: dread dc/d13/d1f/d6e/fc2 [0,4194304] 0 2024-05-12T04:01:30.614 INFO:tasks.workunit.client.1.smithi033.stdout:3/865: write db/f81 [243825,22726] 0 2024-05-12T04:01:30.615 INFO:tasks.workunit.client.0.smithi032.stdout:3/774: dwrite d15/d20/d22/fa4 [0,4194304] 0 2024-05-12T04:01:30.617 INFO:tasks.workunit.client.0.smithi032.stdout:5/813: rename d4/d7/d14/de8 to d4/d7/d14/d26/d47/d120 0 2024-05-12T04:01:30.619 INFO:tasks.workunit.client.0.smithi032.stdout:5/814: fsync d4/d7/d14/d39/d2b/d6e/f85 0 2024-05-12T04:01:30.619 INFO:tasks.workunit.client.1.smithi033.stdout:6/895: fdatasync d4/d1a/d67/d68/f8b 0 2024-05-12T04:01:30.624 INFO:tasks.workunit.client.1.smithi033.stdout:1/801: rmdir d11/d17/d9e 39 2024-05-12T04:01:30.627 INFO:tasks.workunit.client.0.smithi032.stdout:1/700: creat d1/d7/fe4 x:0 0 0 2024-05-12T04:01:30.628 INFO:tasks.workunit.client.0.smithi032.stdout:9/594: creat d0/d3/d6/d7/d2a/fc1 x:0 0 0 2024-05-12T04:01:30.630 INFO:tasks.workunit.client.0.smithi032.stdout:0/728: truncate df/d30/d8d/d8f/fcc 881835 0 2024-05-12T04:01:30.633 INFO:tasks.workunit.client.0.smithi032.stdout:3/775: creat d15/d24/d2a/d5b/f10a x:0 0 0 2024-05-12T04:01:30.634 INFO:tasks.workunit.client.1.smithi033.stdout:0/917: getdents d6 0 2024-05-12T04:01:30.635 INFO:tasks.workunit.client.0.smithi032.stdout:0/729: dread df/d30/d4e/f73 [4194304,4194304] 0 2024-05-12T04:01:30.640 INFO:tasks.workunit.client.1.smithi033.stdout:6/896: creat d4/d1a/d3c/dd9/de2/def/f128 x:0 0 0 2024-05-12T04:01:30.641 INFO:tasks.workunit.client.0.smithi032.stdout:3/776: dwrite d15/d24/d93/f102 [0,4194304] 0 2024-05-12T04:01:30.642 INFO:tasks.workunit.client.0.smithi032.stdout:1/701: rmdir d1/d7/d30/d18/d93/dc1/dc3 39 2024-05-12T04:01:30.645 INFO:tasks.workunit.client.1.smithi033.stdout:2/904: dwrite dc/d17/d27/d2f/f75 [0,4194304] 0 2024-05-12T04:01:30.649 INFO:tasks.workunit.client.0.smithi032.stdout:0/730: unlink df/l1d 0 2024-05-12T04:01:30.649 INFO:tasks.workunit.client.0.smithi032.stdout:3/777: read d15/d24/f3e [386556,9667] 0 2024-05-12T04:01:30.649 INFO:tasks.workunit.client.0.smithi032.stdout:5/815: dwrite d4/d7/d14/d26/d47/f9b [0,4194304] 0 2024-05-12T04:01:30.651 INFO:tasks.workunit.client.1.smithi033.stdout:6/897: dwrite d4/d1a/d6c/fa4 [0,4194304] 0 2024-05-12T04:01:30.653 INFO:tasks.workunit.client.1.smithi033.stdout:5/795: rename d3/l4d to d3/d94/d54/d57/d70/da3/dbe/d16/d45/d62/l106 0 2024-05-12T04:01:30.653 INFO:tasks.workunit.client.1.smithi033.stdout:4/903: truncate dd/d11/d5b/fb5 3336511 0 2024-05-12T04:01:30.658 INFO:tasks.workunit.client.1.smithi033.stdout:2/905: dwrite dc/d17/d27/d68/dd1/fd8 [0,4194304] 0 2024-05-12T04:01:30.665 INFO:tasks.workunit.client.1.smithi033.stdout:6/898: dread - d4/d43/d57/dc4/ffb zero size 2024-05-12T04:01:30.668 INFO:tasks.workunit.client.1.smithi033.stdout:2/906: stat l2 0 2024-05-12T04:01:30.672 INFO:tasks.workunit.client.1.smithi033.stdout:1/802: getdents d11/d1c/d43 0 2024-05-12T04:01:30.673 INFO:tasks.workunit.client.1.smithi033.stdout:2/907: rmdir dc/d17/d27/d68/d9f/d5e 39 2024-05-12T04:01:30.676 INFO:tasks.workunit.client.1.smithi033.stdout:5/796: rename d3/d12/d83/cde to d3/d12/d29/d2b/c107 0 2024-05-12T04:01:30.677 INFO:tasks.workunit.client.1.smithi033.stdout:6/899: mkdir d4/d1a/d67/d6f/d106/d129 0 2024-05-12T04:01:30.677 INFO:tasks.workunit.client.1.smithi033.stdout:5/797: chown d3/d94/d54/d57/d73/l96 90755716 1 2024-05-12T04:01:30.678 INFO:tasks.workunit.client.1.smithi033.stdout:2/908: truncate f6 5286342 0 2024-05-12T04:01:30.678 INFO:tasks.workunit.client.1.smithi033.stdout:2/909: readlink dc/d3b/d88/l91 0 2024-05-12T04:01:30.679 INFO:tasks.workunit.client.1.smithi033.stdout:4/904: creat dd/d7a/d87/d7e/f119 x:0 0 0 2024-05-12T04:01:30.680 INFO:tasks.workunit.client.1.smithi033.stdout:1/803: unlink d11/d17/c29 0 2024-05-12T04:01:30.682 INFO:tasks.workunit.client.1.smithi033.stdout:1/804: fdatasync d11/d17/d84/de7/f101 0 2024-05-12T04:01:30.682 INFO:tasks.workunit.client.1.smithi033.stdout:6/900: creat d4/d43/d57/dc4/f12a x:0 0 0 2024-05-12T04:01:30.685 INFO:tasks.workunit.client.1.smithi033.stdout:5/798: fsync d3/f92 0 2024-05-12T04:01:30.686 INFO:tasks.workunit.client.1.smithi033.stdout:0/918: dread d6/dc/d58/d6a/f133 [0,4194304] 0 2024-05-12T04:01:30.686 INFO:tasks.workunit.client.1.smithi033.stdout:2/910: mkdir dc/d17/dcc/df1/d11b/d122 0 2024-05-12T04:01:30.688 INFO:tasks.workunit.client.1.smithi033.stdout:4/905: chown dd/d7a/d87/d7e/f8f 2963363 1 2024-05-12T04:01:30.689 INFO:tasks.workunit.client.1.smithi033.stdout:8/863: sync 2024-05-12T04:01:30.690 INFO:tasks.workunit.client.1.smithi033.stdout:1/805: mkdir d11/d1c/ddf/d4c/dab/d109 0 2024-05-12T04:01:30.691 INFO:tasks.workunit.client.1.smithi033.stdout:0/919: symlink d6/dc/d58/d6a/d80/l138 0 2024-05-12T04:01:30.693 INFO:tasks.workunit.client.1.smithi033.stdout:2/911: mkdir dc/d17/d32/d84/d123 0 2024-05-12T04:01:30.695 INFO:tasks.workunit.client.1.smithi033.stdout:6/901: unlink d4/d1a/d67/l123 0 2024-05-12T04:01:30.697 INFO:tasks.workunit.client.1.smithi033.stdout:1/806: dread - d11/d17/d27/d36/d3f/d69/d87/da4/fda zero size 2024-05-12T04:01:30.698 INFO:tasks.workunit.client.1.smithi033.stdout:8/864: chown d1/d16/d27/d28/c62 331072627 1 2024-05-12T04:01:30.700 INFO:tasks.workunit.client.1.smithi033.stdout:0/920: dwrite d6/db/d2c/d113/f126 [0,4194304] 0 2024-05-12T04:01:30.700 INFO:tasks.workunit.client.1.smithi033.stdout:4/906: dread dd/d11/fac [0,4194304] 0 2024-05-12T04:01:30.700 INFO:tasks.workunit.client.1.smithi033.stdout:2/912: read dc/d13/d1c/f21 [399451,82684] 0 2024-05-12T04:01:30.705 INFO:tasks.workunit.client.1.smithi033.stdout:4/907: dread - dd/d7a/d87/f93 zero size 2024-05-12T04:01:30.707 INFO:tasks.workunit.client.1.smithi033.stdout:8/865: truncate d1/d16/d29/f9a 837742 0 2024-05-12T04:01:30.708 INFO:tasks.workunit.client.1.smithi033.stdout:6/902: dwrite d4/d1a/ff0 [0,4194304] 0 2024-05-12T04:01:30.714 INFO:tasks.workunit.client.1.smithi033.stdout:2/913: mkdir dc/d17/d32/d84/d123/d124 0 2024-05-12T04:01:30.719 INFO:tasks.workunit.client.1.smithi033.stdout:1/807: dwrite d11/d1c/ddf/f66 [0,4194304] 0 2024-05-12T04:01:30.719 INFO:tasks.workunit.client.1.smithi033.stdout:4/908: mknod dd/d7a/d87/d7e/da7/d38/c11a 0 2024-05-12T04:01:30.721 INFO:tasks.workunit.client.1.smithi033.stdout:2/914: mkdir dc/d17/d27/d2f/d49/ddd/d112/d125 0 2024-05-12T04:01:30.724 INFO:tasks.workunit.client.1.smithi033.stdout:4/909: mkdir dd/d11/d5b/dd1/d11b 0 2024-05-12T04:01:30.725 INFO:tasks.workunit.client.1.smithi033.stdout:0/921: write d6/db/d2c/d73/daf/dc3/f120 [1906,108628] 0 2024-05-12T04:01:30.726 INFO:tasks.workunit.client.1.smithi033.stdout:2/915: unlink dc/d17/d27/d2f/d49/d59/f104 0 2024-05-12T04:01:30.735 INFO:tasks.workunit.client.1.smithi033.stdout:6/903: dread d4/d43/d57/f108 [0,4194304] 0 2024-05-12T04:01:30.736 INFO:tasks.workunit.client.1.smithi033.stdout:6/904: stat d4/d1a/d4e/f66 0 2024-05-12T04:01:30.736 INFO:tasks.workunit.client.1.smithi033.stdout:6/905: readlink l3 0 2024-05-12T04:01:30.737 INFO:tasks.workunit.client.1.smithi033.stdout:8/866: write d1/d16/d27/d4d/d82/dd9/d36/faf [2872792,97169] 0 2024-05-12T04:01:30.741 INFO:tasks.workunit.client.1.smithi033.stdout:0/922: getdents d6/dc/d6e 0 2024-05-12T04:01:30.744 INFO:tasks.workunit.client.1.smithi033.stdout:0/923: dread d6/d11/d98/fa7 [0,4194304] 0 2024-05-12T04:01:30.745 INFO:tasks.workunit.client.1.smithi033.stdout:1/808: dwrite d11/d17/f6d [0,4194304] 0 2024-05-12T04:01:30.748 INFO:tasks.workunit.client.0.smithi032.stdout:0/731: dwrite df/d30/d32/f5e [0,4194304] 0 2024-05-12T04:01:30.749 INFO:tasks.workunit.client.0.smithi032.stdout:5/816: dwrite d4/d7/d14/d108/fff [0,4194304] 0 2024-05-12T04:01:30.749 INFO:tasks.workunit.client.0.smithi032.stdout:3/778: symlink d15/d33/d3f/d75/l10b 0 2024-05-12T04:01:30.751 INFO:tasks.workunit.client.1.smithi033.stdout:4/910: dwrite dd/d7a/d87/d7e/da7/d66/fba [0,4194304] 0 2024-05-12T04:01:30.753 INFO:tasks.workunit.client.0.smithi032.stdout:9/595: rename d0/d3/d6/d7/d27/c61 to d0/d3/d6/d7/da/d89/cc2 0 2024-05-12T04:01:30.754 INFO:tasks.workunit.client.1.smithi033.stdout:2/916: dwrite dc/d13/dfb/f11d [0,4194304] 0 2024-05-12T04:01:30.759 INFO:tasks.workunit.client.1.smithi033.stdout:6/906: creat d4/dad/d110/f12b x:0 0 0 2024-05-12T04:01:30.764 INFO:tasks.workunit.client.0.smithi032.stdout:6/646: sync 2024-05-12T04:01:30.764 INFO:tasks.workunit.client.0.smithi032.stdout:7/673: sync 2024-05-12T04:01:30.765 INFO:tasks.workunit.client.0.smithi032.stdout:0/732: unlink df/d10/d3d/d83/d9f/lbf 0 2024-05-12T04:01:30.765 INFO:tasks.workunit.client.1.smithi033.stdout:8/867: dread d1/d16/d27/d4d/d82/fc2 [0,4194304] 0 2024-05-12T04:01:30.768 INFO:tasks.workunit.client.0.smithi032.stdout:7/674: fsync dd/d7b/d9a/fd3 0 2024-05-12T04:01:30.769 INFO:tasks.workunit.client.0.smithi032.stdout:9/596: mknod d0/d3/d1b/cc3 0 2024-05-12T04:01:30.771 INFO:tasks.workunit.client.1.smithi033.stdout:0/924: rmdir d6/db/d2c 39 2024-05-12T04:01:30.772 INFO:tasks.workunit.client.0.smithi032.stdout:5/817: write d4/d7/d14/f1d [4621325,102161] 0 2024-05-12T04:01:30.773 INFO:tasks.workunit.client.0.smithi032.stdout:1/702: dread d1/d7/f7f [4194304,4194304] 0 2024-05-12T04:01:30.776 INFO:tasks.workunit.client.0.smithi032.stdout:3/779: mkdir d15/d20/d26/de2/d10c 0 2024-05-12T04:01:30.776 INFO:tasks.workunit.client.1.smithi033.stdout:4/911: symlink dd/d7a/l11c 0 2024-05-12T04:01:30.777 INFO:tasks.workunit.client.1.smithi033.stdout:4/912: write dd/d11/d5b/d84/dd7/ff1 [635861,85673] 0 2024-05-12T04:01:30.778 INFO:tasks.workunit.client.0.smithi032.stdout:3/780: truncate d15/d20/d26/f64 4454270 0 2024-05-12T04:01:30.778 INFO:tasks.workunit.client.0.smithi032.stdout:0/733: creat df/d10/d48/d70/ff3 x:0 0 0 2024-05-12T04:01:30.781 INFO:tasks.workunit.client.0.smithi032.stdout:6/647: mkdir d7/d9/d48/d67/d1a/d93/dde 0 2024-05-12T04:01:30.782 INFO:tasks.workunit.client.1.smithi033.stdout:6/907: dread - d4/d1a/d67/dc3/fee zero size 2024-05-12T04:01:30.785 INFO:tasks.workunit.client.0.smithi032.stdout:9/597: rename d0/d3/d6/d79 to d0/d3/d4b/d88/dc4 0 2024-05-12T04:01:30.786 INFO:tasks.workunit.client.1.smithi033.stdout:8/868: readlink d1/d16/d27/d4d/d82/l84 0 2024-05-12T04:01:30.793 INFO:tasks.workunit.client.1.smithi033.stdout:6/908: creat d4/d1a/d67/de5/f12c x:0 0 0 2024-05-12T04:01:30.796 INFO:tasks.workunit.client.0.smithi032.stdout:6/648: rename d7/f32 to d7/d9/d48/d67/d63/d64/d94/fdf 0 2024-05-12T04:01:30.798 INFO:tasks.workunit.client.1.smithi033.stdout:1/809: dread fd [0,4194304] 0 2024-05-12T04:01:30.801 INFO:tasks.workunit.client.0.smithi032.stdout:3/781: symlink d15/d1b/l10d 0 2024-05-12T04:01:30.803 INFO:tasks.workunit.client.1.smithi033.stdout:4/913: creat dd/d11/f11d x:0 0 0 2024-05-12T04:01:30.804 INFO:tasks.workunit.client.1.smithi033.stdout:3/866: sync 2024-05-12T04:01:30.805 INFO:tasks.workunit.client.1.smithi033.stdout:1/810: dwrite d11/d17/d27/d76/f6e [0,4194304] 0 2024-05-12T04:01:30.815 INFO:tasks.workunit.client.0.smithi032.stdout:6/649: unlink d7/de/d2e/dd4/cb8 0 2024-05-12T04:01:30.816 INFO:tasks.workunit.client.1.smithi033.stdout:4/914: chown dd/d11/d5b/fd8 3874313 1 2024-05-12T04:01:30.817 INFO:tasks.workunit.client.1.smithi033.stdout:2/917: write dc/d17/d27/fda [1124148,66673] 0 2024-05-12T04:01:30.817 INFO:tasks.workunit.client.0.smithi032.stdout:9/598: mkdir d0/d3/d6/d7/d2a/d46/d91/dc5 0 2024-05-12T04:01:30.819 INFO:tasks.workunit.client.0.smithi032.stdout:9/599: chown d0/d3/d6/d7/d2a/d46/d91 4816391 1 2024-05-12T04:01:30.822 INFO:tasks.workunit.client.0.smithi032.stdout:5/818: creat d4/f121 x:0 0 0 2024-05-12T04:01:30.824 INFO:tasks.workunit.client.1.smithi033.stdout:0/925: write d6/db/d2c/d73/ff5 [654363,26710] 0 2024-05-12T04:01:30.824 INFO:tasks.workunit.client.0.smithi032.stdout:1/703: link d1/d7/d30/d35/c39 d1/d7/d30/d18/d87/dc6/ce5 0 2024-05-12T04:01:30.824 INFO:tasks.workunit.client.0.smithi032.stdout:9/600: dread d0/d3/d4b/d85/d87/fb0 [0,4194304] 0 2024-05-12T04:01:30.826 INFO:tasks.workunit.client.1.smithi033.stdout:6/909: symlink d4/d6e/l12d 0 2024-05-12T04:01:30.828 INFO:tasks.workunit.client.0.smithi032.stdout:3/782: mknod d15/d24/d40/c10e 0 2024-05-12T04:01:30.828 INFO:tasks.workunit.client.0.smithi032.stdout:0/734: write df/d10/d3d/d4a/d84/f8c [4827558,78067] 0 2024-05-12T04:01:30.830 INFO:tasks.workunit.client.0.smithi032.stdout:9/601: read d0/d3/f25 [1132497,122623] 0 2024-05-12T04:01:30.830 INFO:tasks.workunit.client.0.smithi032.stdout:3/783: fsync d15/d33/d3f/fb7 0 2024-05-12T04:01:30.831 INFO:tasks.workunit.client.1.smithi033.stdout:1/811: mkdir d11/d10a 0 2024-05-12T04:01:30.832 INFO:tasks.workunit.client.0.smithi032.stdout:0/735: truncate df/d30/d32/f34 8721959 0 2024-05-12T04:01:30.833 INFO:tasks.workunit.client.1.smithi033.stdout:3/867: link db/d17/f114 db/d17/d25/d32/d53/f12d 0 2024-05-12T04:01:30.835 INFO:tasks.workunit.client.1.smithi033.stdout:0/926: unlink d6/db/d2c/f48 0 2024-05-12T04:01:30.836 INFO:tasks.workunit.client.0.smithi032.stdout:5/819: fdatasync d4/d7/d58/fbe 0 2024-05-12T04:01:30.839 INFO:tasks.workunit.client.0.smithi032.stdout:1/704: mkdir d1/d6a/de6 0 2024-05-12T04:01:30.843 INFO:tasks.workunit.client.0.smithi032.stdout:3/784: creat d15/d1b/d43/f10f x:0 0 0 2024-05-12T04:01:30.846 INFO:tasks.workunit.client.0.smithi032.stdout:6/650: rename d7/d9/l11 to d7/de/d2e/db9/le0 0 2024-05-12T04:01:30.847 INFO:tasks.workunit.client.1.smithi033.stdout:0/927: rmdir d6/db/d2c/d73 39 2024-05-12T04:01:30.849 INFO:tasks.workunit.client.0.smithi032.stdout:9/602: mknod d0/d3/d6/d7/d2a/d46/d13/cc6 0 2024-05-12T04:01:30.851 INFO:tasks.workunit.client.0.smithi032.stdout:0/736: mkdir df/d10/d3d/d6c/df4 0 2024-05-12T04:01:30.855 INFO:tasks.workunit.client.1.smithi033.stdout:6/910: write d4/d1a/d4e/fa6 [754527,75672] 0 2024-05-12T04:01:30.858 INFO:tasks.workunit.client.0.smithi032.stdout:5/820: mkdir d4/d7/d14/d108/d122 0 2024-05-12T04:01:30.859 INFO:tasks.workunit.client.0.smithi032.stdout:1/705: mkdir d1/d7/d30/d35/d77/de7 0 2024-05-12T04:01:30.863 INFO:tasks.workunit.client.1.smithi033.stdout:6/911: fsync d4/d1a/f91 0 2024-05-12T04:01:30.864 INFO:tasks.workunit.client.0.smithi032.stdout:0/737: fdatasync df/fc1 0 2024-05-12T04:01:30.866 INFO:tasks.workunit.client.1.smithi033.stdout:0/928: truncate d6/db/d2c/d73/daf/da4/fe0 190298 0 2024-05-12T04:01:30.866 INFO:tasks.workunit.client.1.smithi033.stdout:3/868: write db/f20 [141426,18527] 0 2024-05-12T04:01:30.869 INFO:tasks.workunit.client.0.smithi032.stdout:5/821: read - d4/d7/d58/d5a/d99/fa0 zero size 2024-05-12T04:01:30.870 INFO:tasks.workunit.client.1.smithi033.stdout:2/918: dwrite dc/d13/dfb/dd7/fec [0,4194304] 0 2024-05-12T04:01:30.871 INFO:tasks.workunit.client.1.smithi033.stdout:1/812: dread d11/d1c/d59/f7f [0,4194304] 0 2024-05-12T04:01:30.873 INFO:tasks.workunit.client.0.smithi032.stdout:3/785: dread d15/d20/d26/f52 [0,4194304] 0 2024-05-12T04:01:30.874 INFO:tasks.workunit.client.1.smithi033.stdout:0/929: read d6/dc/fb3 [2818182,84877] 0 2024-05-12T04:01:30.879 INFO:tasks.workunit.client.1.smithi033.stdout:3/869: symlink db/d17/d3e/d65/d7d/l12e 0 2024-05-12T04:01:30.879 INFO:tasks.workunit.client.0.smithi032.stdout:0/738: creat df/d30/d4d/ff5 x:0 0 0 2024-05-12T04:01:30.880 INFO:tasks.workunit.client.1.smithi033.stdout:1/813: rmdir d11/d1c 39 2024-05-12T04:01:30.880 INFO:tasks.workunit.client.1.smithi033.stdout:6/912: dwrite d4/d1a/f91 [4194304,4194304] 0 2024-05-12T04:01:30.882 INFO:tasks.workunit.client.1.smithi033.stdout:1/814: stat d11/d17/d27/d36/d3f/f107 0 2024-05-12T04:01:30.882 INFO:tasks.workunit.client.1.smithi033.stdout:6/913: chown d4/d43/d57/c5d 7815 1 2024-05-12T04:01:30.883 INFO:tasks.workunit.client.0.smithi032.stdout:5/822: dread d4/d7/d14/d39/d2b/ff6 [4194304,4194304] 0 2024-05-12T04:01:30.883 INFO:tasks.workunit.client.1.smithi033.stdout:1/815: dread - d11/d17/d27/d36/d3f/d69/d87/da4/fda zero size 2024-05-12T04:01:30.884 INFO:tasks.workunit.client.0.smithi032.stdout:3/786: creat d15/d20/d26/f110 x:0 0 0 2024-05-12T04:01:30.884 INFO:tasks.workunit.client.0.smithi032.stdout:5/823: stat d4/d7/d14/d26/d47/f9e 0 2024-05-12T04:01:30.886 INFO:tasks.workunit.client.1.smithi033.stdout:0/930: creat d6/dc/d58/d6a/d80/db8/f139 x:0 0 0 2024-05-12T04:01:30.886 INFO:tasks.workunit.client.0.smithi032.stdout:6/651: mknod d7/d9/d48/d67/d63/d64/ce1 0 2024-05-12T04:01:30.886 INFO:tasks.workunit.client.0.smithi032.stdout:5/824: chown d4/d7/d14/d108/l3f 182953 1 2024-05-12T04:01:30.892 INFO:tasks.workunit.client.1.smithi033.stdout:2/919: creat dc/d17/d27/d2f/d49/df2/d115/f126 x:0 0 0 2024-05-12T04:01:30.893 INFO:tasks.workunit.client.1.smithi033.stdout:0/931: dwrite d6/dc/d58/d6a/dec/fed [0,4194304] 0 2024-05-12T04:01:30.894 INFO:tasks.workunit.client.0.smithi032.stdout:5/825: fsync d4/d7/d58/d5a/d99/fa0 0 2024-05-12T04:01:30.895 INFO:tasks.workunit.client.0.smithi032.stdout:5/826: stat d4/d7/d14/d26/d33/d88/faf 0 2024-05-12T04:01:30.897 INFO:tasks.workunit.client.1.smithi033.stdout:0/932: truncate d6/dc/d58/d6a/d80/db8/f139 91574 0 2024-05-12T04:01:30.898 INFO:tasks.workunit.client.0.smithi032.stdout:6/652: rename d7/de/d2e to d7/d9/d48/d67/d8d/dbc/de2 0 2024-05-12T04:01:30.902 INFO:tasks.workunit.client.1.smithi033.stdout:2/920: creat dc/d17/d27/d2f/d49/d59/f127 x:0 0 0 2024-05-12T04:01:30.902 INFO:tasks.workunit.client.1.smithi033.stdout:2/921: chown dc/cfd 12705 1 2024-05-12T04:01:30.906 INFO:tasks.workunit.client.1.smithi033.stdout:0/933: symlink d6/dc/d58/d6a/d94/d125/l13a 0 2024-05-12T04:01:30.909 INFO:tasks.workunit.client.0.smithi032.stdout:0/739: link c6 df/d10/d3d/d83/cf6 0 2024-05-12T04:01:30.910 INFO:tasks.workunit.client.1.smithi033.stdout:2/922: dwrite dc/d17/d27/d2f/d49/df2/d115/f126 [0,4194304] 0 2024-05-12T04:01:30.910 INFO:tasks.workunit.client.0.smithi032.stdout:5/827: dread d4/d7/d14/d26/d33/d72/dc9/dec/fba [0,4194304] 0 2024-05-12T04:01:30.911 INFO:tasks.workunit.client.1.smithi033.stdout:2/923: chown dc/d13/d1f/d6e/de8/l97 604795 1 2024-05-12T04:01:30.913 INFO:tasks.workunit.client.1.smithi033.stdout:2/924: fsync dc/d17/d27/d68/dd1/fd8 0 2024-05-12T04:01:30.915 INFO:tasks.workunit.client.1.smithi033.stdout:2/925: chown dc/d13/d1c/d98/ca5 369561 1 2024-05-12T04:01:30.917 INFO:tasks.workunit.client.1.smithi033.stdout:3/870: dread db/d17/d4d/fde [0,4194304] 0 2024-05-12T04:01:30.918 INFO:tasks.workunit.client.1.smithi033.stdout:3/871: readlink db/d42/l9c 0 2024-05-12T04:01:30.920 INFO:tasks.workunit.client.1.smithi033.stdout:3/872: readlink db/d17/l24 0 2024-05-12T04:01:30.920 INFO:tasks.workunit.client.1.smithi033.stdout:3/873: stat db/d17/d25/c28 0 2024-05-12T04:01:30.923 INFO:tasks.workunit.client.1.smithi033.stdout:2/926: link dc/l12 dc/d17/dcc/df1/d11b/d122/l128 0 2024-05-12T04:01:30.923 INFO:tasks.workunit.client.1.smithi033.stdout:0/934: dread d6/db/dd6/f12d [0,4194304] 0 2024-05-12T04:01:30.925 INFO:tasks.workunit.client.1.smithi033.stdout:0/935: rename d6 to d6/d11/d57/d13b 22 2024-05-12T04:01:30.925 INFO:tasks.workunit.client.1.smithi033.stdout:2/927: truncate dc/d17/d27/d2f/d49/d59/f127 859398 0 2024-05-12T04:01:30.928 INFO:tasks.workunit.client.1.smithi033.stdout:2/928: unlink dc/cfd 0 2024-05-12T04:01:30.929 INFO:tasks.workunit.client.1.smithi033.stdout:0/936: dread - d6/dc/d58/d6a/d80/db8/ff9 zero size 2024-05-12T04:01:30.931 INFO:tasks.workunit.client.1.smithi033.stdout:3/874: dread db/d17/d3a/d67/d76/f78 [0,4194304] 0 2024-05-12T04:01:30.931 INFO:tasks.workunit.client.1.smithi033.stdout:5/799: sync 2024-05-12T04:01:30.933 INFO:tasks.workunit.client.1.smithi033.stdout:6/914: dwrite d4/d1a/d4e/d4f/fd4 [4194304,4194304] 0 2024-05-12T04:01:30.933 INFO:tasks.workunit.client.1.smithi033.stdout:2/929: mkdir dc/d13/d1c/d98/df0/d129 0 2024-05-12T04:01:30.934 INFO:tasks.workunit.client.1.smithi033.stdout:3/875: chown db/d17/d3a/f3b 1989 1 2024-05-12T04:01:30.935 INFO:tasks.workunit.client.1.smithi033.stdout:6/915: chown d4/d1a/d67/d68/l8e 147742 1 2024-05-12T04:01:30.936 INFO:tasks.workunit.client.1.smithi033.stdout:4/915: sync 2024-05-12T04:01:30.937 INFO:tasks.workunit.client.1.smithi033.stdout:8/869: sync 2024-05-12T04:01:30.939 INFO:tasks.workunit.client.1.smithi033.stdout:2/930: creat dc/d17/d32/f12a x:0 0 0 2024-05-12T04:01:30.942 INFO:tasks.workunit.client.1.smithi033.stdout:5/800: dread d3/d94/d54/d57/d70/da3/dbe/f88 [0,4194304] 0 2024-05-12T04:01:30.944 INFO:tasks.workunit.client.1.smithi033.stdout:5/801: chown d3/d94/d54/c5d 4058 1 2024-05-12T04:01:30.944 INFO:tasks.workunit.client.1.smithi033.stdout:3/876: mkdir db/d17/d3e/d65/d91/d12f 0 2024-05-12T04:01:30.946 INFO:tasks.workunit.client.1.smithi033.stdout:2/931: creat dc/d13/d1f/d6e/f12b x:0 0 0 2024-05-12T04:01:30.948 INFO:tasks.workunit.client.1.smithi033.stdout:4/916: truncate dd/d69/f7d 1005889 0 2024-05-12T04:01:30.950 INFO:tasks.workunit.client.1.smithi033.stdout:1/816: dwrite d11/d1c/ddf/f74 [0,4194304] 0 2024-05-12T04:01:30.952 INFO:tasks.workunit.client.1.smithi033.stdout:5/802: mkdir d3/d94/d54/d57/d70/da3/dbe/d8c/d108 0 2024-05-12T04:01:30.952 INFO:tasks.workunit.client.1.smithi033.stdout:3/877: write db/d17/d4d/fde [2620506,46174] 0 2024-05-12T04:01:30.953 INFO:tasks.workunit.client.1.smithi033.stdout:0/937: dread d6/d11/d57/fcc [0,4194304] 0 2024-05-12T04:01:30.954 INFO:tasks.workunit.client.1.smithi033.stdout:2/932: stat dc/d13/d1f/d6e/fb6 0 2024-05-12T04:01:30.956 INFO:tasks.workunit.client.1.smithi033.stdout:8/870: creat d1/d16/d27/d4d/d82/dd9/d36/d3c/f107 x:0 0 0 2024-05-12T04:01:30.956 INFO:tasks.workunit.client.1.smithi033.stdout:6/916: link d4/d6e/dbb/c112 d4/d1a/d6c/c12e 0 2024-05-12T04:01:30.958 INFO:tasks.workunit.client.0.smithi032.stdout:2/720: sync 2024-05-12T04:01:30.958 INFO:tasks.workunit.client.0.smithi032.stdout:4/644: sync 2024-05-12T04:01:30.959 INFO:tasks.workunit.client.1.smithi033.stdout:0/938: creat d6/db/d2c/d82/f13c x:0 0 0 2024-05-12T04:01:30.960 INFO:tasks.workunit.client.0.smithi032.stdout:3/787: getdents d15/d24/d2a/d8c/db1 0 2024-05-12T04:01:30.962 INFO:tasks.workunit.client.0.smithi032.stdout:1/706: write d1/d25/f4d [2375754,13454] 0 2024-05-12T04:01:30.962 INFO:tasks.workunit.client.0.smithi032.stdout:5/828: write d4/d7/d14/d39/d2b/d6e/f105 [214252,100408] 0 2024-05-12T04:01:30.965 INFO:tasks.workunit.client.1.smithi033.stdout:0/939: truncate d6/db/f12b 881204 0 2024-05-12T04:01:30.965 INFO:tasks.workunit.client.0.smithi032.stdout:5/829: dread d4/d7/d14/d26/d33/d72/d92/fdb [0,4194304] 0 2024-05-12T04:01:30.968 INFO:tasks.workunit.client.1.smithi033.stdout:8/871: link d1/d16/d29/d63/c6d d1/d16/d29/d63/da3/c108 0 2024-05-12T04:01:30.970 INFO:tasks.workunit.client.1.smithi033.stdout:5/803: dread d3/d94/d54/d57/d70/da3/dbe/d16/d78/d8b/fd8 [0,4194304] 0 2024-05-12T04:01:30.970 INFO:tasks.workunit.client.0.smithi032.stdout:4/645: mkdir d0/d3/d24/dbb 0 2024-05-12T04:01:30.971 INFO:tasks.workunit.client.1.smithi033.stdout:2/933: dwrite dc/d13/d65/f107 [0,4194304] 0 2024-05-12T04:01:30.971 INFO:tasks.workunit.client.0.smithi032.stdout:2/721: mkdir d1/d13/da0/ded 0 2024-05-12T04:01:30.972 INFO:tasks.workunit.client.0.smithi032.stdout:2/722: readlink d1/ld 0 2024-05-12T04:01:30.972 INFO:tasks.workunit.client.1.smithi033.stdout:1/817: dwrite d11/d17/d27/d36/d3f/d69/d87/da4/ff8 [0,4194304] 0 2024-05-12T04:01:30.973 INFO:tasks.workunit.client.0.smithi032.stdout:1/707: mknod d1/d25/ce8 0 2024-05-12T04:01:30.975 INFO:tasks.workunit.client.0.smithi032.stdout:4/646: dread d0/d3/fd [0,4194304] 0 2024-05-12T04:01:30.977 INFO:tasks.workunit.client.1.smithi033.stdout:8/872: fsync d1/d4/db2/fbc 0 2024-05-12T04:01:30.979 INFO:tasks.workunit.client.1.smithi033.stdout:0/940: unlink d6/dc/d58/lf4 0 2024-05-12T04:01:30.980 INFO:tasks.workunit.client.0.smithi032.stdout:6/653: rename d7/d9/d48/d67/d8d/dbc/de2/db9/cd7 to d7/d9/d5c/d8b/ce3 0 2024-05-12T04:01:30.981 INFO:tasks.workunit.client.1.smithi033.stdout:1/818: dread d11/d17/d27/d36/d3f/d69/d87/da4/ff6 [0,4194304] 0 2024-05-12T04:01:30.982 INFO:tasks.workunit.client.1.smithi033.stdout:1/819: readlink d11/d17/d84/ld5 0 2024-05-12T04:01:30.983 INFO:tasks.workunit.client.1.smithi033.stdout:3/878: dread db/d59/f8d [0,4194304] 0 2024-05-12T04:01:30.984 INFO:tasks.workunit.client.1.smithi033.stdout:6/917: dread d4/d1a/d6c/f86 [0,4194304] 0 2024-05-12T04:01:30.984 INFO:tasks.workunit.client.0.smithi032.stdout:6/654: dread d7/d97/f99 [0,4194304] 0 2024-05-12T04:01:30.988 INFO:tasks.workunit.client.1.smithi033.stdout:5/804: creat d3/d12/dac/dd9/f109 x:0 0 0 2024-05-12T04:01:30.994 INFO:tasks.workunit.client.1.smithi033.stdout:8/873: rmdir d1/d16/d27/d4d/d82/dd9/d36/dea 39 2024-05-12T04:01:30.996 INFO:tasks.workunit.client.1.smithi033.stdout:0/941: truncate d6/f9e 285742 0 2024-05-12T04:01:30.999 INFO:tasks.workunit.client.0.smithi032.stdout:0/740: dread df/d30/d8d/d8f/f97 [0,4194304] 0 2024-05-12T04:01:30.999 INFO:tasks.workunit.client.0.smithi032.stdout:5/830: mknod d4/d7/d14/d108/d122/c123 0 2024-05-12T04:01:31.000 INFO:tasks.workunit.client.1.smithi033.stdout:6/918: dread d4/d1a/d67/de5/ffe [0,4194304] 0 2024-05-12T04:01:31.000 INFO:tasks.workunit.client.1.smithi033.stdout:3/879: creat db/d42/f130 x:0 0 0 2024-05-12T04:01:31.003 INFO:tasks.workunit.client.0.smithi032.stdout:3/788: creat d15/d1b/f111 x:0 0 0 2024-05-12T04:01:31.007 INFO:tasks.workunit.client.1.smithi033.stdout:8/874: write d1/d16/d27/d4d/d82/dd9/d36/d3c/fe6 [2606500,62831] 0 2024-05-12T04:01:31.008 INFO:tasks.workunit.client.0.smithi032.stdout:2/723: truncate d1/d13/d70/d85/f88 1715175 0 2024-05-12T04:01:31.009 INFO:tasks.workunit.client.0.smithi032.stdout:3/789: dwrite d15/d1b/f25 [4194304,4194304] 0 2024-05-12T04:01:31.010 INFO:tasks.workunit.client.1.smithi033.stdout:1/820: creat d11/d17/d27/d36/d3f/d4a/dc6/de1/f10b x:0 0 0 2024-05-12T04:01:31.013 INFO:tasks.workunit.client.1.smithi033.stdout:6/919: creat d4/d1a/d3c/dd9/f12f x:0 0 0 2024-05-12T04:01:31.015 INFO:tasks.workunit.client.1.smithi033.stdout:3/880: creat db/d17/d3a/d108/f131 x:0 0 0 2024-05-12T04:01:31.015 INFO:tasks.workunit.client.0.smithi032.stdout:0/741: readlink df/d28/lcd 0 2024-05-12T04:01:31.017 INFO:tasks.workunit.client.1.smithi033.stdout:5/805: symlink d3/d94/d54/d57/l10a 0 2024-05-12T04:01:31.020 INFO:tasks.workunit.client.1.smithi033.stdout:2/934: rename dc/d13/d1c/l33 to dc/d17/d32/l12c 0 2024-05-12T04:01:31.021 INFO:tasks.workunit.client.1.smithi033.stdout:2/935: chown dc/d17/d27/d68/lc9 1077 1 2024-05-12T04:01:31.022 INFO:tasks.workunit.client.1.smithi033.stdout:8/875: rmdir d1/d16/d27/d4d/d51/d8e/d94 39 2024-05-12T04:01:31.030 INFO:tasks.workunit.client.1.smithi033.stdout:3/881: mkdir db/d17/d25/d32/d3c/d3d/d111/d132 0 2024-05-12T04:01:31.034 INFO:tasks.workunit.client.0.smithi032.stdout:4/647: dwrite d0/f65 [0,4194304] 0 2024-05-12T04:01:31.036 INFO:tasks.workunit.client.0.smithi032.stdout:4/648: truncate d0/d3/d12/faa 451287 0 2024-05-12T04:01:31.039 INFO:tasks.workunit.client.1.smithi033.stdout:0/942: rename d6/dc/l25 to d6/dc/d58/d6a/dec/l13d 0 2024-05-12T04:01:31.040 INFO:tasks.workunit.client.0.smithi032.stdout:0/742: fsync df/d10/f80 0 2024-05-12T04:01:31.041 INFO:tasks.workunit.client.0.smithi032.stdout:6/655: write d7/d9/d48/d67/f6e [521335,9928] 0 2024-05-12T04:01:31.042 INFO:tasks.workunit.client.1.smithi033.stdout:2/936: dread dc/d17/d32/fb7 [0,4194304] 0 2024-05-12T04:01:31.043 INFO:tasks.workunit.client.0.smithi032.stdout:0/743: read - df/d30/d4d/ff5 zero size 2024-05-12T04:01:31.050 INFO:tasks.workunit.client.0.smithi032.stdout:1/708: link d1/d7/d30/d18/d93/cad d1/d7/d30/d61/ce9 0 2024-05-12T04:01:31.052 INFO:tasks.workunit.client.0.smithi032.stdout:1/709: chown d1/d7/d30/c22 1974845913 1 2024-05-12T04:01:31.052 INFO:tasks.workunit.client.0.smithi032.stdout:5/831: write d4/d7/d14/d39/d2b/d6e/f77 [3140876,129072] 0 2024-05-12T04:01:31.056 INFO:tasks.workunit.client.1.smithi033.stdout:3/882: symlink db/d17/d3a/dc0/l133 0 2024-05-12T04:01:31.058 INFO:tasks.workunit.client.1.smithi033.stdout:5/806: mkdir d3/d12/d25/d10b 0 2024-05-12T04:01:31.059 INFO:tasks.workunit.client.1.smithi033.stdout:5/807: readlink d3/d94/d85/l8f 0 2024-05-12T04:01:31.062 INFO:tasks.workunit.client.0.smithi032.stdout:2/724: write d1/d34/d3e/fcd [584834,19507] 0 2024-05-12T04:01:31.065 INFO:tasks.workunit.client.1.smithi033.stdout:8/876: truncate d1/d16/d27/f86 590268 0 2024-05-12T04:01:31.069 INFO:tasks.workunit.client.1.smithi033.stdout:1/821: dwrite d11/d1c/d43/f5e [0,4194304] 0 2024-05-12T04:01:31.070 INFO:tasks.workunit.client.0.smithi032.stdout:3/790: dwrite d15/d20/d22/f96 [0,4194304] 0 2024-05-12T04:01:31.070 INFO:tasks.workunit.client.1.smithi033.stdout:6/920: dwrite d4/d5/dcd/fe9 [0,4194304] 0 2024-05-12T04:01:31.070 INFO:tasks.workunit.client.0.smithi032.stdout:7/675: sync 2024-05-12T04:01:31.071 INFO:tasks.workunit.client.0.smithi032.stdout:8/766: sync 2024-05-12T04:01:31.072 INFO:tasks.workunit.client.0.smithi032.stdout:4/649: dread d0/d3/d24/f42 [0,4194304] 0 2024-05-12T04:01:31.073 INFO:tasks.workunit.client.1.smithi033.stdout:6/921: chown d4/d1a/d3c 92380362 1 2024-05-12T04:01:31.073 INFO:tasks.workunit.client.1.smithi033.stdout:1/822: chown d11/d17/d27/d36/d3f/d4a/dc6/de1/ff9 1038 1 2024-05-12T04:01:31.073 INFO:tasks.workunit.client.0.smithi032.stdout:6/656: rename d7/de to d7/d9/d48/d67/d63/d64/d94/de4 0 2024-05-12T04:01:31.081 INFO:tasks.workunit.client.1.smithi033.stdout:1/823: dwrite d11/d17/d27/d36/d3f/d69/d87/d96/f100 [0,4194304] 0 2024-05-12T04:01:31.085 INFO:tasks.workunit.client.1.smithi033.stdout:1/824: fsync d11/d1c/d43/fa1 0 2024-05-12T04:01:31.087 INFO:tasks.workunit.client.0.smithi032.stdout:2/725: mkdir d1/d34/d3e/dee 0 2024-05-12T04:01:31.087 INFO:tasks.workunit.client.0.smithi032.stdout:1/710: symlink d1/d25/da5/lea 0 2024-05-12T04:01:31.089 INFO:tasks.workunit.client.0.smithi032.stdout:5/832: symlink d4/d7/l124 0 2024-05-12T04:01:31.092 INFO:tasks.workunit.client.0.smithi032.stdout:8/767: dread d0/d9/d1b/d66/d75/f7c [0,4194304] 0 2024-05-12T04:01:31.094 INFO:tasks.workunit.client.1.smithi033.stdout:8/877: fsync d1/d16/d27/d4d/d82/dd9/d36/d3c/da7/f33 0 2024-05-12T04:01:31.097 INFO:tasks.workunit.client.0.smithi032.stdout:3/791: creat d15/d24/d2a/d8c/db1/f112 x:0 0 0 2024-05-12T04:01:31.097 INFO:tasks.workunit.client.0.smithi032.stdout:1/711: dread d1/d7/d30/d35/f38 [0,4194304] 0 2024-05-12T04:01:31.098 INFO:tasks.workunit.client.1.smithi033.stdout:0/943: rename d6/dc/c41 to d6/dc/d58/c13e 0 2024-05-12T04:01:31.099 INFO:tasks.workunit.client.0.smithi032.stdout:4/650: creat d0/db5/fbc x:0 0 0 2024-05-12T04:01:31.102 INFO:tasks.workunit.client.0.smithi032.stdout:0/744: dwrite df/f2b [0,4194304] 0 2024-05-12T04:01:31.106 INFO:tasks.workunit.client.0.smithi032.stdout:3/792: dwrite d15/d20/d26/f64 [0,4194304] 0 2024-05-12T04:01:31.109 INFO:tasks.workunit.client.0.smithi032.stdout:2/726: rmdir d1/d4d/d5e 39 2024-05-12T04:01:31.111 INFO:tasks.workunit.client.1.smithi033.stdout:1/825: creat d11/d17/d27/d36/d3f/d69/d87/d96/f10c x:0 0 0 2024-05-12T04:01:31.112 INFO:tasks.workunit.client.0.smithi032.stdout:9/603: sync 2024-05-12T04:01:31.115 INFO:tasks.workunit.client.0.smithi032.stdout:9/604: fsync d0/d3/d6/d7/d2a/fc1 0 2024-05-12T04:01:31.116 INFO:tasks.workunit.client.1.smithi033.stdout:3/883: creat db/d17/d3e/dd7/f134 x:0 0 0 2024-05-12T04:01:31.118 INFO:tasks.workunit.client.0.smithi032.stdout:3/793: dwrite d15/d20/d8b/fe3 [4194304,4194304] 0 2024-05-12T04:01:31.129 INFO:tasks.workunit.client.1.smithi033.stdout:2/937: link dc/d13/d1c/d98/dc1/lc4 dc/d13/l12d 0 2024-05-12T04:01:31.130 INFO:tasks.workunit.client.1.smithi033.stdout:2/938: fdatasync dc/d17/d27/fda 0 2024-05-12T04:01:31.131 INFO:tasks.workunit.client.0.smithi032.stdout:6/657: creat d7/d9/d48/d67/d1a/da0/dc6/fe5 x:0 0 0 2024-05-12T04:01:31.132 INFO:tasks.workunit.client.0.smithi032.stdout:4/651: mkdir d0/d3/d24/dbd 0 2024-05-12T04:01:31.132 INFO:tasks.workunit.client.1.smithi033.stdout:2/939: truncate dc/d13/dfb/dd7/ff3 193170 0 2024-05-12T04:01:31.133 INFO:tasks.workunit.client.0.smithi032.stdout:3/794: dwrite d15/d20/d22/f96 [4194304,4194304] 0 2024-05-12T04:01:31.133 INFO:tasks.workunit.client.1.smithi033.stdout:1/826: dread - d11/d17/d38/f8f zero size 2024-05-12T04:01:31.136 INFO:tasks.workunit.client.1.smithi033.stdout:5/808: creat d3/d94/d54/d57/d70/da3/dbe/d16/d45/d62/f10c x:0 0 0 2024-05-12T04:01:31.136 INFO:tasks.workunit.client.1.smithi033.stdout:0/944: dread d6/db/d2c/fdb [0,4194304] 0 2024-05-12T04:01:31.137 INFO:tasks.workunit.client.0.smithi032.stdout:9/605: dwrite d0/d3/d6/d7/f15 [0,4194304] 0 2024-05-12T04:01:31.140 INFO:tasks.workunit.client.1.smithi033.stdout:3/884: unlink db/d17/d3a/f66 0 2024-05-12T04:01:31.140 INFO:tasks.workunit.client.0.smithi032.stdout:8/768: dread d0/d9/f12 [0,4194304] 0 2024-05-12T04:01:31.145 INFO:tasks.workunit.client.0.smithi032.stdout:4/652: dwrite d0/d3/f1c [4194304,4194304] 0 2024-05-12T04:01:31.152 INFO:tasks.workunit.client.0.smithi032.stdout:0/745: write df/d10/d48/f96 [602290,12944] 0 2024-05-12T04:01:31.152 INFO:tasks.workunit.client.1.smithi033.stdout:1/827: chown d11/d1c/f94 14 1 2024-05-12T04:01:31.156 INFO:tasks.workunit.client.1.smithi033.stdout:3/885: mknod db/d17/d3e/d65/c135 0 2024-05-12T04:01:31.157 INFO:tasks.workunit.client.0.smithi032.stdout:7/676: truncate dd/d7b/d9a/d9c/db9/d75/fad 1476224 0 2024-05-12T04:01:31.159 INFO:tasks.workunit.client.1.smithi033.stdout:8/878: rename d1/d16/d27/d4d/d82/dd9/d36/d3c/d47/f60 to d1/d16/d27/d4d/d82/dd9/d36/f109 0 2024-05-12T04:01:31.160 INFO:tasks.workunit.client.1.smithi033.stdout:8/879: chown d1/d16/d27/d4d/d82/dd9/d36/d3c/d9f 391060868 1 2024-05-12T04:01:31.161 INFO:tasks.workunit.client.1.smithi033.stdout:8/880: readlink d1/d16/d27/d28/l5b 0 2024-05-12T04:01:31.161 INFO:tasks.workunit.client.1.smithi033.stdout:8/881: stat d1/d16/d27/d4d/d82/fc2 0 2024-05-12T04:01:31.167 INFO:tasks.workunit.client.0.smithi032.stdout:3/795: creat d15/dc2/f113 x:0 0 0 2024-05-12T04:01:31.167 INFO:tasks.workunit.client.1.smithi033.stdout:0/945: creat d6/dc/df1/d137/f13f x:0 0 0 2024-05-12T04:01:31.172 INFO:tasks.workunit.client.1.smithi033.stdout:3/886: creat db/d17/d3a/d67/d76/da0/de7/f136 x:0 0 0 2024-05-12T04:01:31.173 INFO:tasks.workunit.client.1.smithi033.stdout:2/940: creat dc/d13/d39/dab/f12e x:0 0 0 2024-05-12T04:01:31.173 INFO:tasks.workunit.client.0.smithi032.stdout:8/769: truncate d0/d9/d1b/d27/d88/d56/f65 528680 0 2024-05-12T04:01:31.174 INFO:tasks.workunit.client.0.smithi032.stdout:9/606: fdatasync d0/d3/d4b/f6d 0 2024-05-12T04:01:31.174 INFO:tasks.workunit.client.1.smithi033.stdout:3/887: write db/d17/d25/d32/d3c/d3d/d47/d10a/f124 [362889,10211] 0 2024-05-12T04:01:31.175 INFO:tasks.workunit.client.1.smithi033.stdout:2/941: read - dc/d17/d27/d2f/d49/d59/fcf zero size 2024-05-12T04:01:31.176 INFO:tasks.workunit.client.0.smithi032.stdout:2/727: symlink d1/d13/d70/ddb/lef 0 2024-05-12T04:01:31.177 INFO:tasks.workunit.client.1.smithi033.stdout:8/882: chown d1/d16/d27/d4d/d82/dd9/d36/d3c/d47/c83 1 1 2024-05-12T04:01:31.182 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:30 smithi032 ceph-mon[40008]: pgmap v43: 65 pgs: 65 active+clean; 3.0 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 72 MiB/s rd, 174 MiB/s wr, 373 op/s 2024-05-12T04:01:31.184 INFO:tasks.workunit.client.1.smithi033.stdout:0/946: creat d6/dc/d58/d6a/d94/d125/d127/f140 x:0 0 0 2024-05-12T04:01:31.185 INFO:tasks.workunit.client.1.smithi033.stdout:1/828: dwrite d11/f30 [0,4194304] 0 2024-05-12T04:01:31.186 INFO:tasks.workunit.client.0.smithi032.stdout:5/833: creat d4/d7/d14/d26/d33/d4a/f125 x:0 0 0 2024-05-12T04:01:31.186 INFO:tasks.workunit.client.1.smithi033.stdout:1/829: readlink d11/d17/d38/le2 0 2024-05-12T04:01:31.190 INFO:tasks.workunit.client.1.smithi033.stdout:5/809: truncate d3/d12/d29/d2b/f46 194355 0 2024-05-12T04:01:31.195 INFO:tasks.workunit.client.0.smithi032.stdout:1/712: rename d1/dd/d24/c58 to d1/dd/ceb 0 2024-05-12T04:01:31.195 INFO:tasks.workunit.client.0.smithi032.stdout:6/658: dwrite d7/d9/d48/d67/d8d/dbc/de2/d9d/fa5 [0,4194304] 0 2024-05-12T04:01:31.197 INFO:tasks.workunit.client.0.smithi032.stdout:5/834: dwrite d4/d7/d14/d26/d33/d72/dc9/dec/f116 [0,4194304] 0 2024-05-12T04:01:31.198 INFO:tasks.workunit.client.1.smithi033.stdout:8/883: unlink d1/d16/d27/d28/f57 0 2024-05-12T04:01:31.199 INFO:tasks.workunit.client.0.smithi032.stdout:5/835: write d4/f20 [1089013,119462] 0 2024-05-12T04:01:31.205 INFO:tasks.workunit.client.0.smithi032.stdout:1/713: dwrite d1/d25/da5/fc0 [0,4194304] 0 2024-05-12T04:01:31.206 INFO:tasks.workunit.client.1.smithi033.stdout:0/947: mkdir d6/d11/dc7/dd9/d141 0 2024-05-12T04:01:31.207 INFO:tasks.workunit.client.1.smithi033.stdout:4/917: sync 2024-05-12T04:01:31.211 INFO:tasks.workunit.client.0.smithi032.stdout:8/770: readlink d0/l2e 0 2024-05-12T04:01:31.211 INFO:tasks.workunit.client.0.smithi032.stdout:4/653: write d0/d3/d31/f35 [1149318,63929] 0 2024-05-12T04:01:31.212 INFO:tasks.workunit.client.1.smithi033.stdout:5/810: unlink d3/d94/dc8/c39 0 2024-05-12T04:01:31.214 INFO:tasks.workunit.client.0.smithi032.stdout:9/607: creat d0/fc7 x:0 0 0 2024-05-12T04:01:31.214 INFO:tasks.workunit.client.1.smithi033.stdout:5/811: write d3/d12/d29/f3d [2982193,60115] 0 2024-05-12T04:01:31.215 INFO:tasks.workunit.client.1.smithi033.stdout:3/888: write db/d17/d25/d32/d3c/d3d/d47/fd3 [934632,109368] 0 2024-05-12T04:01:31.216 INFO:tasks.workunit.client.1.smithi033.stdout:5/812: chown d3/d12/d25/d10b 5 1 2024-05-12T04:01:31.224 INFO:tasks.workunit.client.0.smithi032.stdout:0/746: creat df/d10/d3d/de4/dea/ff7 x:0 0 0 2024-05-12T04:01:31.224 INFO:tasks.workunit.client.0.smithi032.stdout:3/796: write d15/d20/d26/d6a/d6e/d7d/fd9 [354750,130175] 0 2024-05-12T04:01:31.225 INFO:tasks.workunit.client.1.smithi033.stdout:2/942: dwrite dc/d13/d1f/f69 [0,4194304] 0 2024-05-12T04:01:31.227 INFO:tasks.workunit.client.1.smithi033.stdout:4/918: chown dd/d7a/d87/d7e/da7/d66/d42/d9c/lc8 32 1 2024-05-12T04:01:31.234 INFO:tasks.workunit.client.1.smithi033.stdout:1/830: dwrite d11/d17/d27/d76/f72 [0,4194304] 0 2024-05-12T04:01:31.239 INFO:tasks.workunit.client.1.smithi033.stdout:8/884: dwrite d1/d16/d27/d4d/d82/dd9/d36/d3c/d47/ff7 [0,4194304] 0 2024-05-12T04:01:31.253 INFO:tasks.workunit.client.0.smithi032.stdout:9/608: dread d0/d3/d4b/d88/d99/fb1 [0,4194304] 0 2024-05-12T04:01:31.283 INFO:tasks.workunit.client.0.smithi032.stdout:6/659: unlink d7/l1c 0 2024-05-12T04:01:31.287 INFO:tasks.workunit.client.1.smithi033.stdout:0/948: dwrite d6/d11/f2f [0,4194304] 0 2024-05-12T04:01:31.291 INFO:tasks.workunit.client.0.smithi032.stdout:5/836: mkdir d4/d7/d14/d108/d126 0 2024-05-12T04:01:31.294 INFO:tasks.workunit.client.1.smithi033.stdout:5/813: fdatasync d3/d94/d54/d57/d70/da3/dbe/d16/d45/d62/fc0 0 2024-05-12T04:01:31.294 INFO:tasks.workunit.client.1.smithi033.stdout:5/814: dread - d3/d12/d29/fcd zero size 2024-05-12T04:01:31.295 INFO:tasks.workunit.client.0.smithi032.stdout:1/714: mknod d1/d7/d30/d18/d87/dc6/cec 0 2024-05-12T04:01:31.295 INFO:tasks.workunit.client.0.smithi032.stdout:1/715: fsync d1/d25/da5/fc0 0 2024-05-12T04:01:31.298 INFO:tasks.workunit.client.1.smithi033.stdout:8/885: unlink d1/d16/d27/c99 0 2024-05-12T04:01:31.299 INFO:tasks.workunit.client.1.smithi033.stdout:2/943: mknod dc/d17/d32/d84/d123/d124/c12f 0 2024-05-12T04:01:31.300 INFO:tasks.workunit.client.0.smithi032.stdout:4/654: mknod d0/d3/d12/d1d/d6b/cbe 0 2024-05-12T04:01:31.301 INFO:tasks.workunit.client.1.smithi033.stdout:0/949: creat d6/dc/d58/d6a/f142 x:0 0 0 2024-05-12T04:01:31.310 INFO:tasks.workunit.client.0.smithi032.stdout:7/677: link dd/d7b/d9a/d9c/db9/d75/f90 dd/d1b/d1e/d45/d55/db7/fd5 0 2024-05-12T04:01:31.311 INFO:tasks.workunit.client.0.smithi032.stdout:6/660: symlink d7/d9/d48/d67/d8d/dbc/le6 0 2024-05-12T04:01:31.312 INFO:tasks.workunit.client.0.smithi032.stdout:6/661: fdatasync d7/d9/d48/d67/d63/d64/fcd 0 2024-05-12T04:01:31.313 INFO:tasks.workunit.client.0.smithi032.stdout:6/662: write d7/d33/f7a [2914502,26774] 0 2024-05-12T04:01:31.314 INFO:tasks.workunit.client.0.smithi032.stdout:0/747: write df/d10/d3d/d4a/f2e [636015,90580] 0 2024-05-12T04:01:31.318 INFO:tasks.workunit.client.1.smithi033.stdout:8/886: fsync d1/d4/d46/f4f 0 2024-05-12T04:01:31.320 INFO:tasks.workunit.client.0.smithi032.stdout:8/771: mkdir d0/d109 0 2024-05-12T04:01:31.322 INFO:tasks.workunit.client.1.smithi033.stdout:2/944: rmdir dc/d13/d1f/d6e/de8 39 2024-05-12T04:01:31.324 INFO:tasks.workunit.client.0.smithi032.stdout:9/609: dwrite d0/d3/d74/f78 [4194304,4194304] 0 2024-05-12T04:01:31.324 INFO:tasks.workunit.client.1.smithi033.stdout:1/831: truncate d11/d1c/f64 5039325 0 2024-05-12T04:01:31.325 INFO:tasks.workunit.client.1.smithi033.stdout:5/815: dwrite d3/d12/f14 [0,4194304] 0 2024-05-12T04:01:31.327 INFO:tasks.workunit.client.0.smithi032.stdout:9/610: chown d0/d3/d74/f20 2648 1 2024-05-12T04:01:31.330 INFO:tasks.workunit.client.0.smithi032.stdout:2/728: creat d1/d4d/d5e/d9a/ff0 x:0 0 0 2024-05-12T04:01:31.334 INFO:tasks.workunit.client.1.smithi033.stdout:0/950: write d6/dc/d58/f81 [1514782,36787] 0 2024-05-12T04:01:31.336 INFO:tasks.workunit.client.0.smithi032.stdout:0/748: dread df/d30/dc2/ff0 [0,4194304] 0 2024-05-12T04:01:31.336 INFO:tasks.workunit.client.0.smithi032.stdout:8/772: dwrite d0/d9/fe1 [0,4194304] 0 2024-05-12T04:01:31.338 INFO:tasks.workunit.client.1.smithi033.stdout:4/919: rename dd/d7a/d87/d7e/da7/d66/c6b to dd/d7a/d87/d7e/da7/d66/c11e 0 2024-05-12T04:01:31.339 INFO:tasks.workunit.client.0.smithi032.stdout:8/773: chown d0/d9/d1b/d27/d88/d56/d63/d8f/d9a/f55 833776450 1 2024-05-12T04:01:31.344 INFO:tasks.workunit.client.1.smithi033.stdout:2/945: dread dc/d17/d32/d84/f89 [0,4194304] 0 2024-05-12T04:01:31.344 INFO:tasks.workunit.client.1.smithi033.stdout:1/832: unlink d11/d17/d27/d36/fc0 0 2024-05-12T04:01:31.345 INFO:tasks.workunit.client.1.smithi033.stdout:2/946: chown dc/la6 14 1 2024-05-12T04:01:31.346 INFO:tasks.workunit.client.1.smithi033.stdout:2/947: chown dc/d13/d1c/d98/df0 47142936 1 2024-05-12T04:01:31.347 INFO:tasks.workunit.client.1.smithi033.stdout:2/948: readlink dc/d17/d27/d2f/lbc 0 2024-05-12T04:01:31.347 INFO:tasks.workunit.client.1.smithi033.stdout:5/816: rmdir d3/d94/d54/d57/d70/da3/dc9 39 2024-05-12T04:01:31.348 INFO:tasks.workunit.client.1.smithi033.stdout:1/833: dread d11/d1c/f33 [0,4194304] 0 2024-05-12T04:01:31.348 INFO:tasks.workunit.client.1.smithi033.stdout:2/949: chown dc/d13/d1f/d6e/c86 51 1 2024-05-12T04:01:31.349 INFO:tasks.workunit.client.1.smithi033.stdout:5/817: chown d3/d94/d85/l97 6201303 1 2024-05-12T04:01:31.349 INFO:tasks.workunit.client.1.smithi033.stdout:2/950: dread - dc/ff4 zero size 2024-05-12T04:01:31.352 INFO:tasks.workunit.client.1.smithi033.stdout:0/951: rename d6/db/d2c/d73/daf to d6/d11/d98/d143 0 2024-05-12T04:01:31.355 INFO:tasks.workunit.client.1.smithi033.stdout:4/920: unlink dd/d7a/d87/d7e/da7/d66/d42/d9c/lc8 0 2024-05-12T04:01:31.358 INFO:tasks.workunit.client.1.smithi033.stdout:8/887: symlink d1/d16/d27/d4d/l10a 0 2024-05-12T04:01:31.365 INFO:tasks.workunit.client.1.smithi033.stdout:1/834: mkdir d11/d17/d27/d10d 0 2024-05-12T04:01:31.368 INFO:tasks.workunit.client.1.smithi033.stdout:0/952: write d6/dc/d58/f59 [2505639,55158] 0 2024-05-12T04:01:31.369 INFO:tasks.workunit.client.1.smithi033.stdout:4/921: creat dd/d7a/d87/d7e/da7/d38/d104/f11f x:0 0 0 2024-05-12T04:01:31.370 INFO:tasks.workunit.client.1.smithi033.stdout:4/922: chown dd/l5c 28 1 2024-05-12T04:01:31.371 INFO:tasks.workunit.client.1.smithi033.stdout:8/888: dread - d1/d16/d27/d4d/d82/dd9/d36/d3c/da7/feb zero size 2024-05-12T04:01:31.374 INFO:tasks.workunit.client.1.smithi033.stdout:6/922: sync 2024-05-12T04:01:31.374 INFO:tasks.workunit.client.1.smithi033.stdout:5/818: symlink d3/d94/d54/d57/d70/da3/dc9/l10d 0 2024-05-12T04:01:31.377 INFO:tasks.workunit.client.1.smithi033.stdout:0/953: creat d6/d11/dc7/dc8/f144 x:0 0 0 2024-05-12T04:01:31.379 INFO:tasks.workunit.client.1.smithi033.stdout:4/923: mknod dd/d11/d5b/d84/dd7/c120 0 2024-05-12T04:01:31.385 INFO:tasks.workunit.client.1.smithi033.stdout:5/819: rmdir d3/d94/d54/d57/d73/d74 39 2024-05-12T04:01:31.386 INFO:tasks.workunit.client.1.smithi033.stdout:5/820: fdatasync d3/d94/d54/d57/d70/da3/dbe/daa/fbc 0 2024-05-12T04:01:31.388 INFO:tasks.workunit.client.1.smithi033.stdout:0/954: rename d6/db/d2c/f31 to d6/d11/dc7/dc8/f145 0 2024-05-12T04:01:31.389 INFO:tasks.workunit.client.1.smithi033.stdout:0/955: truncate d6/d11/d98/d143/dc3/f120 894373 0 2024-05-12T04:01:31.391 INFO:tasks.workunit.client.1.smithi033.stdout:8/889: mkdir d1/d16/d27/d4d/d73/dd0/d10b 0 2024-05-12T04:01:31.393 INFO:tasks.workunit.client.1.smithi033.stdout:2/951: getdents dc/d17 0 2024-05-12T04:01:31.396 INFO:tasks.workunit.client.1.smithi033.stdout:5/821: creat d3/d12/d83/f10e x:0 0 0 2024-05-12T04:01:31.398 INFO:tasks.workunit.client.1.smithi033.stdout:4/924: mkdir dd/d69/dc4/d117/d121 0 2024-05-12T04:01:31.401 INFO:tasks.workunit.client.1.smithi033.stdout:3/889: sync 2024-05-12T04:01:31.402 INFO:tasks.workunit.client.1.smithi033.stdout:3/890: dread - db/d17/d3a/dc0/fdf zero size 2024-05-12T04:01:31.403 INFO:tasks.workunit.client.1.smithi033.stdout:8/890: dread d1/f6b [0,4194304] 0 2024-05-12T04:01:31.403 INFO:tasks.workunit.client.1.smithi033.stdout:2/952: rename dc/d17/d27/d2f/lbc to dc/d17/l130 0 2024-05-12T04:01:31.405 INFO:tasks.workunit.client.1.smithi033.stdout:1/835: mknod d11/d17/d27/da6/c10e 0 2024-05-12T04:01:31.405 INFO:tasks.workunit.client.1.smithi033.stdout:8/891: write d1/d16/d27/d4d/d82/dd9/d36/f50 [2302851,74928] 0 2024-05-12T04:01:31.406 INFO:tasks.workunit.client.1.smithi033.stdout:5/822: truncate d3/d12/d29/d2b/d4f/f103 1005747 0 2024-05-12T04:01:31.406 INFO:tasks.workunit.client.1.smithi033.stdout:0/956: mknod d6/dc/d58/d6a/d80/db8/d124/c146 0 2024-05-12T04:01:31.408 INFO:tasks.workunit.client.1.smithi033.stdout:5/823: chown d3/d94/d54/d57/d73/lc5 5590 1 2024-05-12T04:01:31.408 INFO:tasks.workunit.client.1.smithi033.stdout:0/957: write d6/d11/dc7/dd9/d10b/dfb/f130 [628060,42008] 0 2024-05-12T04:01:31.409 INFO:tasks.workunit.client.1.smithi033.stdout:5/824: write d3/d12/d25/db0/ff7 [1046378,51481] 0 2024-05-12T04:01:31.410 INFO:tasks.workunit.client.0.smithi032.stdout:5/837: mknod d4/d7/d14/d108/d126/c127 0 2024-05-12T04:01:31.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:30 smithi033 ceph-mon[49525]: pgmap v43: 65 pgs: 65 active+clean; 3.0 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 72 MiB/s rd, 174 MiB/s wr, 373 op/s 2024-05-12T04:01:31.412 INFO:tasks.workunit.client.1.smithi033.stdout:6/923: link d4/dad/ldd d4/d1a/l130 0 2024-05-12T04:01:31.414 INFO:tasks.workunit.client.1.smithi033.stdout:3/891: read - db/d17/d3e/dd7/dd0/f11d zero size 2024-05-12T04:01:31.416 INFO:tasks.workunit.client.0.smithi032.stdout:4/655: unlink d0/d3/d12/d1d/d63/d87/la6 0 2024-05-12T04:01:31.425 INFO:tasks.workunit.client.0.smithi032.stdout:2/729: rename d1/d34/d4a/l80 to d1/d34/d3e/dee/lf1 0 2024-05-12T04:01:31.426 INFO:tasks.workunit.client.0.smithi032.stdout:5/838: dread d4/d7/d14/d39/d1b/f23 [0,4194304] 0 2024-05-12T04:01:31.427 INFO:tasks.workunit.client.1.smithi033.stdout:1/836: creat d11/d1c/ddf/d4c/f10f x:0 0 0 2024-05-12T04:01:31.435 INFO:tasks.workunit.client.1.smithi033.stdout:8/892: dread d1/f2 [4194304,4194304] 0 2024-05-12T04:01:31.436 INFO:tasks.workunit.client.1.smithi033.stdout:6/924: mkdir d4/d1a/d3c/dd9/de2/d131 0 2024-05-12T04:01:31.438 INFO:tasks.workunit.client.1.smithi033.stdout:3/892: mkdir db/d17/d3e/d65/d7d/d137 0 2024-05-12T04:01:31.438 INFO:tasks.workunit.client.1.smithi033.stdout:3/893: readlink db/d17/d25/d32/d3c/l6c 0 2024-05-12T04:01:31.439 INFO:tasks.workunit.client.0.smithi032.stdout:4/656: truncate d0/f11 4260137 0 2024-05-12T04:01:31.439 INFO:tasks.workunit.client.1.smithi033.stdout:3/894: dread - db/d17/d25/d32/fd5 zero size 2024-05-12T04:01:31.446 INFO:tasks.workunit.client.0.smithi032.stdout:2/730: rename d1/d13/d70/d85/dd5/ce7 to d1/d34/da5/db3/db7/cf2 0 2024-05-12T04:01:31.447 INFO:tasks.workunit.client.1.smithi033.stdout:8/893: fdatasync d1/d16/d27/d4d/d73/fd7 0 2024-05-12T04:01:31.448 INFO:tasks.workunit.client.0.smithi032.stdout:2/731: write d1/d34/d96/d8a/f9b [4156428,110523] 0 2024-05-12T04:01:31.449 INFO:tasks.workunit.client.1.smithi033.stdout:6/925: dread d4/f16 [4194304,4194304] 0 2024-05-12T04:01:31.450 INFO:tasks.workunit.client.0.smithi032.stdout:5/839: mkdir d4/d7/d14/d108/d126/d128 0 2024-05-12T04:01:31.456 INFO:tasks.workunit.client.1.smithi033.stdout:5/825: creat d3/d94/d54/d57/d73/f10f x:0 0 0 2024-05-12T04:01:31.457 INFO:tasks.workunit.client.1.smithi033.stdout:5/826: dread - d3/d12/d83/deb/ff9 zero size 2024-05-12T04:01:31.458 INFO:tasks.workunit.client.0.smithi032.stdout:6/663: write f3 [2476356,111808] 0 2024-05-12T04:01:31.461 INFO:tasks.workunit.client.1.smithi033.stdout:8/894: creat d1/d16/d29/f10c x:0 0 0 2024-05-12T04:01:31.461 INFO:tasks.workunit.client.0.smithi032.stdout:1/716: getdents d1/d7/d30/d61/dc8 0 2024-05-12T04:01:31.461 INFO:tasks.workunit.client.1.smithi033.stdout:4/925: dwrite dd/f3b [4194304,4194304] 0 2024-05-12T04:01:31.462 INFO:tasks.workunit.client.0.smithi032.stdout:1/717: fsync d1/d7/d30/d35/dbe/fe0 0 2024-05-12T04:01:31.463 INFO:tasks.workunit.client.0.smithi032.stdout:9/611: write d0/d3/d6/d7/d2a/f68 [2773121,120804] 0 2024-05-12T04:01:31.464 INFO:tasks.workunit.client.1.smithi033.stdout:4/926: read dd/d7a/d87/d7e/da7/d66/d42/fde [272568,66893] 0 2024-05-12T04:01:31.465 INFO:tasks.workunit.client.0.smithi032.stdout:1/718: truncate d1/d7/d30/d61/dc8/fd3 775919 0 2024-05-12T04:01:31.467 INFO:tasks.workunit.client.0.smithi032.stdout:0/749: write df/f92 [2810154,95078] 0 2024-05-12T04:01:31.469 INFO:tasks.workunit.client.0.smithi032.stdout:2/732: read - d1/d4d/d5e/fb9 zero size 2024-05-12T04:01:31.472 INFO:tasks.workunit.client.0.smithi032.stdout:8/774: rmdir d0/d9/d1b/d66/d75/d84/dee 0 2024-05-12T04:01:31.473 INFO:tasks.workunit.client.1.smithi033.stdout:0/958: dwrite d6/dc/f16 [0,4194304] 0 2024-05-12T04:01:31.474 INFO:tasks.workunit.client.0.smithi032.stdout:0/750: dread df/d10/d3d/d4a/d9a/d9d/fb7 [0,4194304] 0 2024-05-12T04:01:31.474 INFO:tasks.workunit.client.1.smithi033.stdout:0/959: dread - d6/d11/dc7/dc8/f144 zero size 2024-05-12T04:01:31.475 INFO:tasks.workunit.client.1.smithi033.stdout:1/837: truncate d11/d17/d27/d36/d3f/fd0 2433154 0 2024-05-12T04:01:31.477 INFO:tasks.workunit.client.1.smithi033.stdout:4/927: dread dd/d69/da4/fa8 [0,4194304] 0 2024-05-12T04:01:31.477 INFO:tasks.workunit.client.1.smithi033.stdout:5/827: dread - d3/d94/d54/d57/d70/da3/dbe/d16/d45/d62/fad zero size 2024-05-12T04:01:31.477 INFO:tasks.workunit.client.1.smithi033.stdout:0/960: chown d6/db/d2c/d73/l79 0 1 2024-05-12T04:01:31.478 INFO:tasks.workunit.client.0.smithi032.stdout:6/664: creat d7/d9/d48/d67/d25/fe7 x:0 0 0 2024-05-12T04:01:31.481 INFO:tasks.workunit.client.0.smithi032.stdout:8/775: dwrite d0/d9/d1b/d66/fe8 [0,4194304] 0 2024-05-12T04:01:31.482 INFO:tasks.workunit.client.0.smithi032.stdout:6/665: write d7/d9/d48/d67/d8d/fb5 [686034,13055] 0 2024-05-12T04:01:31.483 INFO:tasks.workunit.client.0.smithi032.stdout:6/666: readlink d7/d9/d48/d67/l9c 0 2024-05-12T04:01:31.484 INFO:tasks.workunit.client.1.smithi033.stdout:3/895: dwrite db/d17/fc5 [0,4194304] 0 2024-05-12T04:01:31.486 INFO:tasks.workunit.client.0.smithi032.stdout:6/667: truncate d7/d9/d48/d67/d63/d64/d94/fdf 5179381 0 2024-05-12T04:01:31.493 INFO:tasks.workunit.client.0.smithi032.stdout:2/733: mkdir d1/d13/d70/d85/df3 0 2024-05-12T04:01:31.498 INFO:tasks.workunit.client.0.smithi032.stdout:1/719: write d1/d7/d30/d18/d87/db7/fc5 [2795853,96813] 0 2024-05-12T04:01:31.500 INFO:tasks.workunit.client.0.smithi032.stdout:7/678: link dd/d1b/d1e/d45/d64/l89 dd/ld6 0 2024-05-12T04:01:31.500 INFO:tasks.workunit.client.0.smithi032.stdout:6/668: dread d7/d33/f7a [0,4194304] 0 2024-05-12T04:01:31.503 INFO:tasks.workunit.client.0.smithi032.stdout:4/657: rename d0/d3/d12/l61 to d0/d3/df/lbf 0 2024-05-12T04:01:31.505 INFO:tasks.workunit.client.1.smithi033.stdout:5/828: unlink d3/d94/d54/d57/d70/da3/dc9/lca 0 2024-05-12T04:01:31.507 INFO:tasks.workunit.client.1.smithi033.stdout:6/926: link d4/d1a/d67/d68/ca9 d4/d1a/d6c/c132 0 2024-05-12T04:01:31.508 INFO:tasks.workunit.client.1.smithi033.stdout:3/896: read - db/d17/d3e/d65/d91/fa7 zero size 2024-05-12T04:01:31.510 INFO:tasks.workunit.client.1.smithi033.stdout:3/897: write db/d17/d3a/d108/f106 [897903,109914] 0 2024-05-12T04:01:31.511 INFO:tasks.workunit.client.0.smithi032.stdout:2/734: dread - d1/d34/d96/da4/fc0 zero size 2024-05-12T04:01:31.513 INFO:tasks.workunit.client.0.smithi032.stdout:2/735: dread d1/d4d/d5e/fdc [0,4194304] 0 2024-05-12T04:01:31.513 INFO:tasks.workunit.client.0.smithi032.stdout:0/751: creat df/d30/dc2/ff8 x:0 0 0 2024-05-12T04:01:31.514 INFO:tasks.workunit.client.0.smithi032.stdout:1/720: mkdir d1/d6a/d72/da9/ded 0 2024-05-12T04:01:31.517 INFO:tasks.workunit.client.0.smithi032.stdout:6/669: mknod d7/d33/ce8 0 2024-05-12T04:01:31.519 INFO:tasks.workunit.client.0.smithi032.stdout:5/840: rename d4/d7/d14/d39/d2b/cef to d4/d7/d14/d108/c129 0 2024-05-12T04:01:31.521 INFO:tasks.workunit.client.0.smithi032.stdout:4/658: unlink d0/d3/l32 0 2024-05-12T04:01:31.524 INFO:tasks.workunit.client.0.smithi032.stdout:8/776: creat d0/d9/d13/d21/d35/de9/f10a x:0 0 0 2024-05-12T04:01:31.525 INFO:tasks.workunit.client.0.smithi032.stdout:5/841: dwrite d4/d7/d14/f1d [0,4194304] 0 2024-05-12T04:01:31.526 INFO:tasks.workunit.client.0.smithi032.stdout:9/612: creat d0/d3/d74/fc8 x:0 0 0 2024-05-12T04:01:31.527 INFO:tasks.workunit.client.0.smithi032.stdout:8/777: chown d0/d9/d1b/d27/d4c/d4e/d90/caa 49322048 1 2024-05-12T04:01:31.530 INFO:tasks.workunit.client.0.smithi032.stdout:5/842: dread d4/d7/d14/d39/d1b/f23 [0,4194304] 0 2024-05-12T04:01:31.531 INFO:tasks.workunit.client.0.smithi032.stdout:2/736: symlink d1/d34/d3e/dee/lf4 0 2024-05-12T04:01:31.535 INFO:tasks.workunit.client.0.smithi032.stdout:1/721: creat d1/d7/d49/dd1/fee x:0 0 0 2024-05-12T04:01:31.536 INFO:tasks.workunit.client.0.smithi032.stdout:1/722: chown d1/d6a/d72/da4 67191 1 2024-05-12T04:01:31.536 INFO:tasks.workunit.client.0.smithi032.stdout:7/679: symlink dd/d1b/d1e/d45/ld7 0 2024-05-12T04:01:31.542 INFO:tasks.workunit.client.0.smithi032.stdout:9/613: mkdir d0/d3/d4b/d85/dc9 0 2024-05-12T04:01:31.543 INFO:tasks.workunit.client.0.smithi032.stdout:9/614: chown d0/d3/d6/f69 6506 1 2024-05-12T04:01:31.544 INFO:tasks.workunit.client.0.smithi032.stdout:0/752: dread df/d10/d48/f52 [0,4194304] 0 2024-05-12T04:01:31.555 INFO:tasks.workunit.client.0.smithi032.stdout:5/843: dwrite d4/d7/d14/d108/fe0 [8388608,4194304] 0 2024-05-12T04:01:31.555 INFO:tasks.workunit.client.0.smithi032.stdout:7/680: creat dd/d1b/d43/d5e/d37/fd8 x:0 0 0 2024-05-12T04:01:31.564 INFO:tasks.workunit.client.0.smithi032.stdout:0/753: symlink df/d30/d4d/lf9 0 2024-05-12T04:01:31.568 INFO:tasks.workunit.client.1.smithi033.stdout:0/961: sync 2024-05-12T04:01:31.569 INFO:tasks.workunit.client.0.smithi032.stdout:2/737: rename d1/d34/d96/d41/d52/fe3 to d1/d73/ff5 0 2024-05-12T04:01:31.576 INFO:tasks.workunit.client.0.smithi032.stdout:2/738: dwrite d1/d34/fce [0,4194304] 0 2024-05-12T04:01:31.576 INFO:tasks.workunit.client.0.smithi032.stdout:6/670: creat d7/d9/d48/d67/fe9 x:0 0 0 2024-05-12T04:01:31.578 INFO:tasks.workunit.client.0.smithi032.stdout:6/671: readlink d7/d9/d48/d67/d1a/d20/l5b 0 2024-05-12T04:01:31.594 INFO:tasks.workunit.client.0.smithi032.stdout:7/681: dread dd/d1b/d1e/f8d [4194304,4194304] 0 2024-05-12T04:01:31.604 INFO:tasks.workunit.client.0.smithi032.stdout:7/682: dread dd/fe [0,4194304] 0 2024-05-12T04:01:31.685 INFO:tasks.workunit.client.0.smithi032.stdout:0/754: mknod df/df2/cfa 0 2024-05-12T04:01:31.690 INFO:tasks.workunit.client.0.smithi032.stdout:3/797: sync 2024-05-12T04:01:31.691 INFO:tasks.workunit.client.1.smithi033.stdout:1/838: mknod d11/d1c/ddf/d4c/dde/c110 0 2024-05-12T04:01:31.695 INFO:tasks.workunit.client.1.smithi033.stdout:4/928: truncate dd/d11/f6d 1552858 0 2024-05-12T04:01:31.751 INFO:tasks.workunit.client.1.smithi033.stdout:4/929: sync 2024-05-12T04:01:31.752 INFO:tasks.workunit.client.1.smithi033.stdout:4/930: chown dd/d7a/d87/d7e/da7/d66/d42/d59/f115 859 1 2024-05-12T04:01:31.819 INFO:tasks.workunit.client.0.smithi032.stdout:4/659: symlink d0/d3/d12/d1d/d63/lc0 0 2024-05-12T04:01:31.822 INFO:tasks.workunit.client.1.smithi033.stdout:6/927: fsync d4/d1a/d67/dc3/fee 0 2024-05-12T04:01:31.823 INFO:tasks.workunit.client.0.smithi032.stdout:7/683: truncate dd/d1b/d43/d5e/f34 1279058 0 2024-05-12T04:01:31.824 INFO:tasks.workunit.client.0.smithi032.stdout:7/684: stat dd/d7b/fbf 0 2024-05-12T04:01:31.824 INFO:tasks.workunit.client.0.smithi032.stdout:8/778: dwrite d0/d9/d1b/d27/d4c/d4e/d90/faf [0,4194304] 0 2024-05-12T04:01:31.827 INFO:tasks.workunit.client.1.smithi033.stdout:3/898: mknod db/d59/c138 0 2024-05-12T04:01:31.831 INFO:tasks.workunit.client.0.smithi032.stdout:1/723: truncate d1/d7/d30/d18/d87/db7/fc5 134453 0 2024-05-12T04:01:31.831 INFO:tasks.workunit.client.1.smithi033.stdout:0/962: mknod d6/dc/d58/d6a/c147 0 2024-05-12T04:01:31.834 INFO:tasks.workunit.client.1.smithi033.stdout:1/839: mknod d11/d17/d27/d36/d3f/d69/d87/d96/c111 0 2024-05-12T04:01:31.843 INFO:tasks.workunit.client.0.smithi032.stdout:5/844: write d4/d7/d14/d26/d47/f6d [1688048,39666] 0 2024-05-12T04:01:31.846 INFO:tasks.workunit.client.0.smithi032.stdout:5/845: truncate d4/d7/d14/d108/fe0 13074100 0 2024-05-12T04:01:31.848 INFO:tasks.workunit.client.0.smithi032.stdout:9/615: rename d0/d3/d6/c35 to d0/d3/d6/d7/da/cca 0 2024-05-12T04:01:31.851 INFO:tasks.workunit.client.1.smithi033.stdout:8/895: link d1/d16/d27/f85 d1/d16/d27/d4d/f10d 0 2024-05-12T04:01:31.854 INFO:tasks.workunit.client.1.smithi033.stdout:3/899: fdatasync db/d17/d25/d32/d3c/ff9 0 2024-05-12T04:01:31.855 INFO:tasks.workunit.client.1.smithi033.stdout:3/900: chown db/d17/d3e/d65 24118990 1 2024-05-12T04:01:31.855 INFO:tasks.workunit.client.0.smithi032.stdout:1/724: chown d1/d7/d30/d18/d93/dc1/dc3 75400445 1 2024-05-12T04:01:31.859 INFO:tasks.workunit.client.0.smithi032.stdout:6/672: link d7/d9/d48/d67/d25/f7e d7/d9/d48/d67/d25/fea 0 2024-05-12T04:01:31.863 INFO:tasks.workunit.client.0.smithi032.stdout:4/660: write d0/d3/d12/fa2 [61191,44620] 0 2024-05-12T04:01:31.865 INFO:tasks.workunit.client.1.smithi033.stdout:2/953: link dc/d17/d27/d2f/d49/df2/f10c dc/d17/d32/f131 0 2024-05-12T04:01:31.866 INFO:tasks.workunit.client.0.smithi032.stdout:6/673: dread f3 [0,4194304] 0 2024-05-12T04:01:31.868 INFO:tasks.workunit.client.1.smithi033.stdout:4/931: truncate dd/d7a/d87/d7e/da7/d38/fbc 226723 0 2024-05-12T04:01:31.871 INFO:tasks.workunit.client.1.smithi033.stdout:5/829: link d3/d94/d54/d57/d70/da3/dbe/fc1 d3/d12/d29/d2b/d8e/f110 0 2024-05-12T04:01:31.873 INFO:tasks.workunit.client.0.smithi032.stdout:5/846: rmdir d4/d7/d58/d5a/d99 39 2024-05-12T04:01:31.876 INFO:tasks.workunit.client.0.smithi032.stdout:7/685: dwrite dd/d1b/d1e/d45/d64/f8b [4194304,4194304] 0 2024-05-12T04:01:31.878 INFO:tasks.workunit.client.0.smithi032.stdout:0/755: rename df/d10/d48/f96 to df/d10/d48/d70/ffb 0 2024-05-12T04:01:31.880 INFO:tasks.workunit.client.1.smithi033.stdout:6/928: mknod d4/d43/c133 0 2024-05-12T04:01:31.881 INFO:tasks.workunit.client.1.smithi033.stdout:0/963: write d6/dc/d39/f40 [1168188,12529] 0 2024-05-12T04:01:31.884 INFO:tasks.workunit.client.1.smithi033.stdout:6/929: write d4/d1a/d67/d6f/daf/ff7 [421522,44519] 0 2024-05-12T04:01:31.884 INFO:tasks.workunit.client.1.smithi033.stdout:3/901: rename db/d17/d3e/d65/d7d/l12e to db/d17/d3e/d65/d88/l139 0 2024-05-12T04:01:31.885 INFO:tasks.workunit.client.0.smithi032.stdout:8/779: dwrite d0/d9/d1b/d27/d4c/d4e/d90/fbc [0,4194304] 0 2024-05-12T04:01:31.887 INFO:tasks.workunit.client.0.smithi032.stdout:3/798: link d15/d20/d22/d7a/fa6 d15/d20/f114 0 2024-05-12T04:01:31.891 INFO:tasks.workunit.client.0.smithi032.stdout:1/725: mkdir d1/d6a/def 0 2024-05-12T04:01:31.892 INFO:tasks.workunit.client.0.smithi032.stdout:5/847: dwrite d4/f121 [0,4194304] 0 2024-05-12T04:01:31.895 INFO:tasks.workunit.client.1.smithi033.stdout:1/840: read d11/d17/d9e/f9f [116140,21395] 0 2024-05-12T04:01:31.897 INFO:tasks.workunit.client.1.smithi033.stdout:3/902: dread db/d17/d3e/d65/d91/fb3 [0,4194304] 0 2024-05-12T04:01:31.900 INFO:tasks.workunit.client.1.smithi033.stdout:8/896: write d1/fcb [386021,126112] 0 2024-05-12T04:01:31.902 INFO:tasks.workunit.client.1.smithi033.stdout:5/830: rmdir d3/d12 39 2024-05-12T04:01:31.906 INFO:tasks.workunit.client.0.smithi032.stdout:7/686: chown dd/d1b/d1e/d45/d55/f5b 605469 1 2024-05-12T04:01:31.906 INFO:tasks.workunit.client.0.smithi032.stdout:7/687: chown dd/d1b/d1e/fd1 8300 1 2024-05-12T04:01:31.911 INFO:tasks.workunit.client.0.smithi032.stdout:6/674: rename d7/d9/d48/d67/d8d/dbc/de2/f8e to d7/d9/d48/d67/d25/feb 0 2024-05-12T04:01:31.911 INFO:tasks.workunit.client.1.smithi033.stdout:2/954: write dc/d17/d27/d2f/d49/f51 [239015,110151] 0 2024-05-12T04:01:31.915 INFO:tasks.workunit.client.0.smithi032.stdout:9/616: mknod d0/d3/d6/d7/da/ccb 0 2024-05-12T04:01:31.916 INFO:tasks.workunit.client.1.smithi033.stdout:6/930: rmdir d4/dad/d110 39 2024-05-12T04:01:31.917 INFO:tasks.workunit.client.0.smithi032.stdout:0/756: dread - df/d30/d4e/d55/fc4 zero size 2024-05-12T04:01:31.919 INFO:tasks.workunit.client.1.smithi033.stdout:1/841: truncate d11/d17/d9e/fc3 49001 0 2024-05-12T04:01:31.921 INFO:tasks.workunit.client.1.smithi033.stdout:3/903: chown db/d17/d3e/d65/d91/d97/c128 2075875 1 2024-05-12T04:01:31.923 INFO:tasks.workunit.client.0.smithi032.stdout:3/799: creat d15/d1b/d43/f115 x:0 0 0 2024-05-12T04:01:31.923 INFO:tasks.workunit.client.0.smithi032.stdout:3/800: chown d15/d33/d3f/f87 758498 1 2024-05-12T04:01:31.924 INFO:tasks.workunit.client.0.smithi032.stdout:3/801: fsync d15/d33/d3f/fb7 0 2024-05-12T04:01:31.926 INFO:tasks.workunit.client.0.smithi032.stdout:2/739: sync 2024-05-12T04:01:31.928 INFO:tasks.workunit.client.0.smithi032.stdout:1/726: symlink d1/d7/d30/d18/d93/lf0 0 2024-05-12T04:01:31.930 INFO:tasks.workunit.client.0.smithi032.stdout:5/848: dread d4/d7/d14/d39/d2b/f32 [0,4194304] 0 2024-05-12T04:01:31.931 INFO:tasks.workunit.client.1.smithi033.stdout:5/831: write d3/d12/d29/d2b/d4f/f103 [468394,22326] 0 2024-05-12T04:01:31.931 INFO:tasks.workunit.client.0.smithi032.stdout:4/661: fsync d0/d3/d31/d80/d84/da7/fad 0 2024-05-12T04:01:31.933 INFO:tasks.workunit.client.1.smithi033.stdout:8/897: dread d1/d16/d27/d4d/d82/dd9/d36/f50 [4194304,4194304] 0 2024-05-12T04:01:31.935 INFO:tasks.workunit.client.1.smithi033.stdout:2/955: dread dc/d13/d1c/d98/fb9 [0,4194304] 0 2024-05-12T04:01:31.937 INFO:tasks.workunit.client.0.smithi032.stdout:6/675: creat d7/d97/fec x:0 0 0 2024-05-12T04:01:31.940 INFO:tasks.workunit.client.0.smithi032.stdout:5/849: dwrite d4/d7/d14/f1d [0,4194304] 0 2024-05-12T04:01:31.943 INFO:tasks.workunit.client.1.smithi033.stdout:3/904: unlink db/c2f 0 2024-05-12T04:01:31.945 INFO:tasks.workunit.client.1.smithi033.stdout:4/932: link dd/d11/d5b/d84/dd7/ff3 dd/d7a/d87/f122 0 2024-05-12T04:01:31.947 INFO:tasks.workunit.client.0.smithi032.stdout:2/740: symlink d1/d34/da5/db3/db7/lf6 0 2024-05-12T04:01:31.950 INFO:tasks.workunit.client.0.smithi032.stdout:3/802: rename d15/d24/d93/f70 to d15/d24/d2a/d8c/f116 0 2024-05-12T04:01:31.951 INFO:tasks.workunit.client.0.smithi032.stdout:6/676: dwrite d7/d9/d5c/d8b/da1/fae [0,4194304] 0 2024-05-12T04:01:31.955 INFO:tasks.workunit.client.1.smithi033.stdout:0/964: creat d6/db/d2c/f148 x:0 0 0 2024-05-12T04:01:31.961 INFO:tasks.workunit.client.1.smithi033.stdout:5/832: truncate d3/d94/d54/d57/d70/da3/dc9/fb2 2359706 0 2024-05-12T04:01:31.962 INFO:tasks.workunit.client.1.smithi033.stdout:6/931: write d4/f78 [6544821,95628] 0 2024-05-12T04:01:31.962 INFO:tasks.workunit.client.0.smithi032.stdout:4/662: unlink d0/d3/d12/d8d/da1/fb1 0 2024-05-12T04:01:31.963 INFO:tasks.workunit.client.1.smithi033.stdout:5/833: write d3/d12/f14 [1046521,18399] 0 2024-05-12T04:01:31.963 INFO:tasks.workunit.client.1.smithi033.stdout:3/905: dread db/d17/d25/d32/d3c/d3d/d111/f113 [0,4194304] 0 2024-05-12T04:01:31.964 INFO:tasks.workunit.client.0.smithi032.stdout:5/850: dread d4/d7/d14/d26/f57 [0,4194304] 0 2024-05-12T04:01:31.972 INFO:tasks.workunit.client.0.smithi032.stdout:8/780: dwrite d0/d9/d1b/d27/d88/d56/d63/d8f/d9a/d28/d3e/fb6 [0,4194304] 0 2024-05-12T04:01:31.973 INFO:tasks.workunit.client.0.smithi032.stdout:0/757: dwrite df/d10/d48/f52 [0,4194304] 0 2024-05-12T04:01:31.977 INFO:tasks.workunit.client.1.smithi033.stdout:6/932: mknod d4/d1a/d67/d6f/dac/c134 0 2024-05-12T04:01:31.978 INFO:tasks.workunit.client.0.smithi032.stdout:2/741: dread - d1/d13/da0/dae/fc1 zero size 2024-05-12T04:01:31.981 INFO:tasks.workunit.client.1.smithi033.stdout:5/834: rmdir d3/d12 39 2024-05-12T04:01:31.984 INFO:tasks.workunit.client.1.smithi033.stdout:5/835: read d3/d94/dc8/fa8 [1409179,76535] 0 2024-05-12T04:01:31.986 INFO:tasks.workunit.client.0.smithi032.stdout:1/727: unlink d1/dd/d24/c29 0 2024-05-12T04:01:31.988 INFO:tasks.workunit.client.0.smithi032.stdout:2/742: dwrite d1/f24 [0,4194304] 0 2024-05-12T04:01:31.993 INFO:tasks.workunit.client.0.smithi032.stdout:7/688: creat dd/d1b/d1e/d45/fd9 x:0 0 0 2024-05-12T04:01:31.993 INFO:tasks.workunit.client.1.smithi033.stdout:1/842: write d11/d17/d27/f31 [2927426,71891] 0 2024-05-12T04:01:31.995 INFO:tasks.workunit.client.1.smithi033.stdout:4/933: link dd/d11/d5b/d84/dd7/ff6 dd/d7a/d87/d7e/da7/d38/d104/f123 0 2024-05-12T04:01:31.996 INFO:tasks.workunit.client.0.smithi032.stdout:9/617: truncate d0/d3/d6/d7/d2a/f50 3413097 0 2024-05-12T04:01:32.003 INFO:tasks.workunit.client.0.smithi032.stdout:5/851: creat d4/d7/d14/d26/d47/d120/f12a x:0 0 0 2024-05-12T04:01:32.004 INFO:tasks.workunit.client.1.smithi033.stdout:2/956: sync 2024-05-12T04:01:32.004 INFO:tasks.workunit.client.1.smithi033.stdout:6/933: mknod d4/d8d/c135 0 2024-05-12T04:01:32.005 INFO:tasks.workunit.client.0.smithi032.stdout:0/758: creat df/df2/ffc x:0 0 0 2024-05-12T04:01:32.005 INFO:tasks.workunit.client.1.smithi033.stdout:0/965: dwrite d6/dc/d58/fbd [0,4194304] 0 2024-05-12T04:01:32.007 INFO:tasks.workunit.client.0.smithi032.stdout:8/781: mknod d0/d9/d1b/d27/d4c/d4e/d7f/da3/c10b 0 2024-05-12T04:01:32.009 INFO:tasks.workunit.client.1.smithi033.stdout:5/836: creat d3/d94/d54/f111 x:0 0 0 2024-05-12T04:01:32.013 INFO:tasks.workunit.client.0.smithi032.stdout:4/663: rmdir d0/d3/d12/d1d/d63/d77 39 2024-05-12T04:01:32.019 INFO:tasks.workunit.client.0.smithi032.stdout:8/782: dwrite d0/d9/d13/d21/d25/db5/fdc [0,4194304] 0 2024-05-12T04:01:32.022 INFO:tasks.workunit.client.1.smithi033.stdout:3/906: dwrite db/d17/d3e/d65/d91/fa7 [0,4194304] 0 2024-05-12T04:01:32.024 INFO:tasks.workunit.client.0.smithi032.stdout:2/743: mkdir d1/d34/d96/d41/d52/df7 0 2024-05-12T04:01:32.024 INFO:tasks.workunit.client.1.smithi033.stdout:8/898: link d1/d16/d29/f9a d1/d16/d27/d4d/d82/dd9/d36/dd2/f10e 0 2024-05-12T04:01:32.026 INFO:tasks.workunit.client.0.smithi032.stdout:3/803: mkdir d15/d20/d22/d117 0 2024-05-12T04:01:32.028 INFO:tasks.workunit.client.0.smithi032.stdout:1/728: creat d1/d7/d30/d35/d54/ff1 x:0 0 0 2024-05-12T04:01:32.030 INFO:tasks.workunit.client.0.smithi032.stdout:7/689: mkdir dd/d6c/dc3/dda 0 2024-05-12T04:01:32.033 INFO:tasks.workunit.client.1.smithi033.stdout:8/899: dwrite d1/d16/fbe [4194304,4194304] 0 2024-05-12T04:01:32.035 INFO:tasks.workunit.client.0.smithi032.stdout:9/618: truncate d0/f28 4629147 0 2024-05-12T04:01:32.037 INFO:tasks.workunit.client.0.smithi032.stdout:9/619: truncate d0/d84/faf 9338537 0 2024-05-12T04:01:32.037 INFO:tasks.workunit.client.0.smithi032.stdout:9/620: chown d0/d84/l9d 245999615 1 2024-05-12T04:01:32.040 INFO:tasks.workunit.client.1.smithi033.stdout:1/843: write d11/d17/d27/d36/d3f/d69/d87/fdb [5650500,25584] 0 2024-05-12T04:01:32.047 INFO:tasks.workunit.client.1.smithi033.stdout:4/934: dwrite dd/f2b [0,4194304] 0 2024-05-12T04:01:32.047 INFO:tasks.workunit.client.1.smithi033.stdout:4/935: chown dd/d69 771298 1 2024-05-12T04:01:32.050 INFO:tasks.workunit.client.0.smithi032.stdout:6/677: creat d7/d9/d48/fed x:0 0 0 2024-05-12T04:01:32.055 INFO:tasks.workunit.client.1.smithi033.stdout:6/934: mknod d4/d43/c136 0 2024-05-12T04:01:32.060 INFO:tasks.workunit.client.0.smithi032.stdout:3/804: mknod d15/d20/d22/d7a/dbe/c118 0 2024-05-12T04:01:32.065 INFO:tasks.workunit.client.0.smithi032.stdout:2/744: rmdir d1/d34/d96/d41 39 2024-05-12T04:01:32.067 INFO:tasks.workunit.client.0.smithi032.stdout:4/664: creat d0/d3/d12/d1d/d6b/db7/fc1 x:0 0 0 2024-05-12T04:01:32.069 INFO:tasks.workunit.client.0.smithi032.stdout:2/745: readlink d1/d13/da0/lbf 0 2024-05-12T04:01:32.072 INFO:tasks.workunit.client.0.smithi032.stdout:6/678: dwrite d7/d9/d48/d67/d25/fe7 [0,4194304] 0 2024-05-12T04:01:32.072 INFO:tasks.workunit.client.0.smithi032.stdout:5/852: rename d4/d7/d14/d39/d1b/fd8 to d4/d7/d14/d26/d33/d72/dc9/dec/f12b 0 2024-05-12T04:01:32.078 INFO:tasks.workunit.client.1.smithi033.stdout:2/957: link dc/d13/d65/fa8 dc/d13/d1c/d98/df0/f132 0 2024-05-12T04:01:32.080 INFO:tasks.workunit.client.1.smithi033.stdout:8/900: symlink d1/d4/l10f 0 2024-05-12T04:01:32.088 INFO:tasks.workunit.client.0.smithi032.stdout:3/805: dread d15/d20/d26/d6a/d6e/d7d/fa1 [0,4194304] 0 2024-05-12T04:01:32.090 INFO:tasks.workunit.client.0.smithi032.stdout:4/665: mkdir d0/d3/d12/d1d/d6b/dc2 0 2024-05-12T04:01:32.091 INFO:tasks.workunit.client.1.smithi033.stdout:3/907: dwrite db/d42/fe2 [0,4194304] 0 2024-05-12T04:01:32.093 INFO:tasks.workunit.client.0.smithi032.stdout:0/759: rename df/d30/d32/d60/dc7/lee to df/d22/d58/lfd 0 2024-05-12T04:01:32.095 INFO:tasks.workunit.client.1.smithi033.stdout:2/958: truncate dc/d17/d27/d2f/d49/d59/fcf 926739 0 2024-05-12T04:01:32.097 INFO:tasks.workunit.client.1.smithi033.stdout:0/966: getdents d6/d11/d98/d143/dc3 0 2024-05-12T04:01:32.097 INFO:tasks.workunit.client.1.smithi033.stdout:0/967: readlink d6/l8 0 2024-05-12T04:01:32.098 INFO:tasks.workunit.client.1.smithi033.stdout:0/968: fsync d6/db/d2c/d82/f13c 0 2024-05-12T04:01:32.099 INFO:tasks.workunit.client.0.smithi032.stdout:5/853: mkdir d4/d7/d14/d26/d33/d4a/d11d/d12c 0 2024-05-12T04:01:32.101 INFO:tasks.workunit.client.1.smithi033.stdout:1/844: creat d11/f112 x:0 0 0 2024-05-12T04:01:32.108 INFO:tasks.workunit.client.1.smithi033.stdout:3/908: creat db/d17/d3a/d108/f13a x:0 0 0 2024-05-12T04:01:32.114 INFO:tasks.workunit.client.0.smithi032.stdout:0/760: dread df/d28/f59 [0,4194304] 0 2024-05-12T04:01:32.117 INFO:tasks.workunit.client.1.smithi033.stdout:5/837: sync 2024-05-12T04:01:32.118 INFO:tasks.workunit.client.0.smithi032.stdout:9/621: dwrite d0/d3/d4b/d88/f96 [0,4194304] 0 2024-05-12T04:01:32.119 INFO:tasks.workunit.client.0.smithi032.stdout:4/666: fsync d0/d3/df/d4e/f9f 0 2024-05-12T04:01:32.121 INFO:tasks.workunit.client.1.smithi033.stdout:6/935: dwrite d4/fcf [0,4194304] 0 2024-05-12T04:01:32.123 INFO:tasks.workunit.client.1.smithi033.stdout:4/936: link dd/d69/da4/lc0 dd/d11/l124 0 2024-05-12T04:01:32.125 INFO:tasks.workunit.client.1.smithi033.stdout:4/937: fdatasync dd/d11/d75/fca 0 2024-05-12T04:01:32.127 INFO:tasks.workunit.client.0.smithi032.stdout:1/729: dwrite d1/d7/d30/d61/dc8/fd4 [0,4194304] 0 2024-05-12T04:01:32.128 INFO:tasks.workunit.client.0.smithi032.stdout:1/730: fdatasync d1/dd/fce 0 2024-05-12T04:01:32.131 INFO:tasks.workunit.client.1.smithi033.stdout:6/936: dwrite d4/d1a/d4e/fb5 [0,4194304] 0 2024-05-12T04:01:32.132 INFO:tasks.workunit.client.1.smithi033.stdout:3/909: rename db/d17/d3a/d93/dc8 to db/d17/d3e/d65/dfb/d11a/d13b 0 2024-05-12T04:01:32.138 INFO:tasks.workunit.client.1.smithi033.stdout:2/959: mknod dc/d13/d1f/c133 0 2024-05-12T04:01:32.139 INFO:tasks.workunit.client.1.smithi033.stdout:2/960: chown dc/lac 1886 1 2024-05-12T04:01:32.141 INFO:tasks.workunit.client.1.smithi033.stdout:0/969: mkdir d6/d11/d131/d149 0 2024-05-12T04:01:32.142 INFO:tasks.workunit.client.0.smithi032.stdout:2/746: dread d1/d34/d96/d41/d39/f87 [0,4194304] 0 2024-05-12T04:01:32.146 INFO:tasks.workunit.client.0.smithi032.stdout:2/747: fsync d1/d34/d96/d8a/fbb 0 2024-05-12T04:01:32.147 INFO:tasks.workunit.client.1.smithi033.stdout:0/970: dwrite d6/ff7 [0,4194304] 0 2024-05-12T04:01:32.150 INFO:tasks.workunit.client.1.smithi033.stdout:0/971: write d6/dc/d58/d6a/d80/db8/f139 [98099,125887] 0 2024-05-12T04:01:32.150 INFO:tasks.workunit.client.1.smithi033.stdout:0/972: stat d6/fe9 0 2024-05-12T04:01:32.151 INFO:tasks.workunit.client.1.smithi033.stdout:4/938: creat dd/d7a/d87/d7e/da7/d38/d104/f125 x:0 0 0 2024-05-12T04:01:32.152 INFO:tasks.workunit.client.1.smithi033.stdout:0/973: stat d6/d11/d98/d143/da4/f100 0 2024-05-12T04:01:32.154 INFO:tasks.workunit.client.1.smithi033.stdout:0/974: write d6/d11/dc7/dc8/f11a [1023270,38782] 0 2024-05-12T04:01:32.157 INFO:tasks.workunit.client.0.smithi032.stdout:3/806: rename d15/d33/f74 to d15/d20/f119 0 2024-05-12T04:01:32.159 INFO:tasks.workunit.client.0.smithi032.stdout:3/807: chown d15/d20/d26/f5f 6267865 1 2024-05-12T04:01:32.160 INFO:tasks.workunit.client.0.smithi032.stdout:6/679: getdents d7/d9/d48/d67/d25 0 2024-05-12T04:01:32.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:31 smithi033 ceph-mon[49525]: pgmap v44: 65 pgs: 65 active+clean; 3.5 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 106 MiB/s rd, 263 MiB/s wr, 562 op/s 2024-05-12T04:01:32.164 INFO:tasks.workunit.client.0.smithi032.stdout:2/748: truncate d1/d4d/d5e/fdc 401706 0 2024-05-12T04:01:32.165 INFO:tasks.workunit.client.1.smithi033.stdout:2/961: rename dc/d17/d27/d2f/d49/df2/d115 to dc/d17/d27/d2f/d49/d134 0 2024-05-12T04:01:32.166 INFO:tasks.workunit.client.0.smithi032.stdout:2/749: readlink d1/d13/d6b/l6f 0 2024-05-12T04:01:32.167 INFO:tasks.workunit.client.0.smithi032.stdout:4/667: truncate d0/d3/d31/f53 1193713 0 2024-05-12T04:01:32.171 INFO:tasks.workunit.client.1.smithi033.stdout:3/910: write db/d17/d25/f58 [1757049,38293] 0 2024-05-12T04:01:32.172 INFO:tasks.workunit.client.0.smithi032.stdout:5/854: rename d4/d7/d14/d26/d33/f10a to d4/d7/d14/d26/d33/d72/d92/d111/f12d 0 2024-05-12T04:01:32.172 INFO:tasks.workunit.client.0.smithi032.stdout:6/680: dwrite d7/d9/d48/d67/d63/d64/d94/de4/fcb [4194304,4194304] 0 2024-05-12T04:01:32.173 INFO:tasks.workunit.client.1.smithi033.stdout:3/911: chown db/d17/d3a/d93/l95 6832759 1 2024-05-12T04:01:32.173 INFO:tasks.workunit.client.1.smithi033.stdout:0/975: dread d6/f26 [0,4194304] 0 2024-05-12T04:01:32.175 INFO:tasks.workunit.client.1.smithi033.stdout:6/937: symlink d4/d1a/d67/d6f/daf/d116/l137 0 2024-05-12T04:01:32.177 INFO:tasks.workunit.client.0.smithi032.stdout:5/855: write d4/d7/d14/d39/d2b/d6e/f77 [4664873,48815] 0 2024-05-12T04:01:32.177 INFO:tasks.workunit.client.1.smithi033.stdout:8/901: sync 2024-05-12T04:01:32.178 INFO:tasks.workunit.client.1.smithi033.stdout:8/902: chown d1/d16/d29/d63/da3/dd6 249 1 2024-05-12T04:01:32.178 INFO:tasks.workunit.client.0.smithi032.stdout:4/668: truncate d0/d3/d31/d80/d84/f7c 5043742 0 2024-05-12T04:01:32.179 INFO:tasks.workunit.client.1.smithi033.stdout:8/903: chown d1/d16/d27/d7c/f89 10952 1 2024-05-12T04:01:32.179 INFO:tasks.workunit.client.0.smithi032.stdout:6/681: write d7/d9/d48/d67/d8d/dbc/f71 [4059175,18403] 0 2024-05-12T04:01:32.182 INFO:tasks.workunit.client.0.smithi032.stdout:0/761: dread df/d30/d4e/f73 [4194304,4194304] 0 2024-05-12T04:01:32.183 INFO:tasks.workunit.client.0.smithi032.stdout:5/856: stat d4/d7/d14/d26/d33/d4a/d38/f45 0 2024-05-12T04:01:32.183 INFO:tasks.workunit.client.1.smithi033.stdout:4/939: creat dd/d7a/d87/d7e/da7/d38/d104/d109/f126 x:0 0 0 2024-05-12T04:01:32.185 INFO:tasks.workunit.client.1.smithi033.stdout:1/845: getdents d11 0 2024-05-12T04:01:32.185 INFO:tasks.workunit.client.1.smithi033.stdout:4/940: dread - dd/d69/da4/f107 zero size 2024-05-12T04:01:32.187 INFO:tasks.workunit.client.0.smithi032.stdout:2/750: rmdir d1/d34/d3e 39 2024-05-12T04:01:32.189 INFO:tasks.workunit.client.1.smithi033.stdout:2/962: truncate dc/d3b/d88/fc0 972850 0 2024-05-12T04:01:32.192 INFO:tasks.workunit.client.0.smithi032.stdout:9/622: rename d0/d3/d6/d7/d27/f7a to d0/d3/d4b/fcc 0 2024-05-12T04:01:32.193 INFO:tasks.workunit.client.0.smithi032.stdout:5/857: dwrite d4/d7/d58/f7b [0,4194304] 0 2024-05-12T04:01:32.193 INFO:tasks.workunit.client.0.smithi032.stdout:0/762: dread df/d10/d3b/f66 [4194304,4194304] 0 2024-05-12T04:01:32.199 INFO:tasks.workunit.client.0.smithi032.stdout:3/808: truncate d15/d24/f3d 595381 0 2024-05-12T04:01:32.200 INFO:tasks.workunit.client.1.smithi033.stdout:0/976: rmdir d6/dc/d58/d6a/d80/db8 39 2024-05-12T04:01:32.202 INFO:tasks.workunit.client.0.smithi032.stdout:4/669: symlink d0/d3/df/d4e/lc3 0 2024-05-12T04:01:32.205 INFO:tasks.workunit.client.1.smithi033.stdout:8/904: creat d1/d4/d46/dca/f110 x:0 0 0 2024-05-12T04:01:32.205 INFO:tasks.workunit.client.1.smithi033.stdout:1/846: symlink d11/d17/d27/d36/d3f/d4a/l113 0 2024-05-12T04:01:32.207 INFO:tasks.workunit.client.1.smithi033.stdout:2/963: rmdir dc/d13/d1c/d98/df0 39 2024-05-12T04:01:32.208 INFO:tasks.workunit.client.0.smithi032.stdout:3/809: dwrite d15/d1b/d43/f10f [0,4194304] 0 2024-05-12T04:01:32.209 INFO:tasks.workunit.client.1.smithi033.stdout:8/905: dread d1/d16/fbe [4194304,4194304] 0 2024-05-12T04:01:32.213 INFO:tasks.workunit.client.1.smithi033.stdout:8/906: dread d1/d16/d27/d4d/d82/dd9/d36/f74 [4194304,4194304] 0 2024-05-12T04:01:32.214 INFO:tasks.workunit.client.0.smithi032.stdout:1/731: rename d1/d6a/d72/ccd to d1/d7/d30/d35/d54/cf2 0 2024-05-12T04:01:32.215 INFO:tasks.workunit.client.1.smithi033.stdout:3/912: creat db/d17/d3e/d65/d7d/d137/f13c x:0 0 0 2024-05-12T04:01:32.217 INFO:tasks.workunit.client.0.smithi032.stdout:9/623: creat d0/d3/d4b/d85/fcd x:0 0 0 2024-05-12T04:01:32.221 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:31 smithi032 ceph-mon[40008]: pgmap v44: 65 pgs: 65 active+clean; 3.5 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 106 MiB/s rd, 263 MiB/s wr, 562 op/s 2024-05-12T04:01:32.221 INFO:tasks.workunit.client.1.smithi033.stdout:0/977: truncate d6/f26 2491023 0 2024-05-12T04:01:32.222 INFO:tasks.workunit.client.1.smithi033.stdout:0/978: stat d6/db/d2c/lb7 0 2024-05-12T04:01:32.223 INFO:tasks.workunit.client.1.smithi033.stdout:3/913: dwrite db/d17/d25/d32/d3c/d3d/d47/fdc [4194304,4194304] 0 2024-05-12T04:01:32.236 INFO:tasks.workunit.client.0.smithi032.stdout:6/682: dwrite d7/d9/d48/d67/d25/f7f [0,4194304] 0 2024-05-12T04:01:32.236 INFO:tasks.workunit.client.1.smithi033.stdout:8/907: dread d1/d16/d27/f2d [0,4194304] 0 2024-05-12T04:01:32.237 INFO:tasks.workunit.client.1.smithi033.stdout:8/908: readlink d1/d4/d91/la5 0 2024-05-12T04:01:32.240 INFO:tasks.workunit.client.1.smithi033.stdout:5/838: link d3/d94/d54/d57/d70/da3/dbe/d16/d45/d62/fc0 d3/d94/d54/d57/d70/da3/dbe/d16/d45/d62/f112 0 2024-05-12T04:01:32.241 INFO:tasks.workunit.client.0.smithi032.stdout:3/810: mknod d15/dc2/c11a 0 2024-05-12T04:01:32.246 INFO:tasks.workunit.client.0.smithi032.stdout:1/732: fdatasync d1/d7/d30/d18/d87/fbb 0 2024-05-12T04:01:32.250 INFO:tasks.workunit.client.1.smithi033.stdout:0/979: creat d6/db/dd6/f14a x:0 0 0 2024-05-12T04:01:32.251 INFO:tasks.workunit.client.1.smithi033.stdout:6/938: link d4/d1a/d67/d68/cf8 d4/d1a/d67/d6f/d106/c138 0 2024-05-12T04:01:32.253 INFO:tasks.workunit.client.0.smithi032.stdout:5/858: rename d4/d7/d14/d108/c129 to d4/d7/d14/d108/c12e 0 2024-05-12T04:01:32.256 INFO:tasks.workunit.client.1.smithi033.stdout:1/847: dwrite d11/d17/d27/d36/d3f/d69/d87/da4/ff6 [0,4194304] 0 2024-05-12T04:01:32.257 INFO:tasks.workunit.client.0.smithi032.stdout:4/670: mkdir d0/d3/d12/d1d/d6b/dc2/dc4 0 2024-05-12T04:01:32.259 INFO:tasks.workunit.client.1.smithi033.stdout:1/848: truncate d11/d1c/ddf/d4c/dab/dee/f103 49928 0 2024-05-12T04:01:32.262 INFO:tasks.workunit.client.1.smithi033.stdout:8/909: creat d1/d4/d46/f111 x:0 0 0 2024-05-12T04:01:32.265 INFO:tasks.workunit.client.0.smithi032.stdout:6/683: symlink d7/d9/d48/d67/d8d/dbc/lee 0 2024-05-12T04:01:32.265 INFO:tasks.workunit.client.0.smithi032.stdout:6/684: dread - d7/d33/fd8 zero size 2024-05-12T04:01:32.266 INFO:tasks.workunit.client.0.smithi032.stdout:3/811: mkdir d15/d1b/d43/dd0/d11b 0 2024-05-12T04:01:32.266 INFO:tasks.workunit.client.0.smithi032.stdout:9/624: dwrite d0/d3/d6/d58/f66 [0,4194304] 0 2024-05-12T04:01:32.272 INFO:tasks.workunit.client.0.smithi032.stdout:1/733: creat d1/d7/d30/d18/ff3 x:0 0 0 2024-05-12T04:01:32.274 INFO:tasks.workunit.client.1.smithi033.stdout:6/939: mkdir d4/d1a/d3c/dd9/de2/d139 0 2024-05-12T04:01:32.278 INFO:tasks.workunit.client.1.smithi033.stdout:4/941: getdents dd/d11/d75/dd4 0 2024-05-12T04:01:32.279 INFO:tasks.workunit.client.1.smithi033.stdout:0/980: write d6/db/d2c/fdb [1415625,23626] 0 2024-05-12T04:01:32.281 INFO:tasks.workunit.client.1.smithi033.stdout:4/942: readlink dd/d7a/d87/d7e/da7/d66/d42/le6 0 2024-05-12T04:01:32.281 INFO:tasks.workunit.client.1.smithi033.stdout:6/940: dwrite d4/d1a/d4e/fa0 [4194304,4194304] 0 2024-05-12T04:01:32.287 INFO:tasks.workunit.client.0.smithi032.stdout:2/751: getdents d1/d34/d4a 0 2024-05-12T04:01:32.290 INFO:tasks.workunit.client.1.smithi033.stdout:2/964: mkdir dc/d13/d1c/d98/df0/d129/d135 0 2024-05-12T04:01:32.293 INFO:tasks.workunit.client.0.smithi032.stdout:9/625: fsync d0/d3/d6/d7/d2a/d46/d13/f7c 0 2024-05-12T04:01:32.294 INFO:tasks.workunit.client.0.smithi032.stdout:9/626: stat d0/d3/d4b 0 2024-05-12T04:01:32.296 INFO:tasks.workunit.client.0.smithi032.stdout:3/812: rename d15/d20/d26/d6a/d6e/d7d/fe1 to d15/d20/d22/d4f/de0/f11c 0 2024-05-12T04:01:32.298 INFO:tasks.workunit.client.0.smithi032.stdout:1/734: rmdir d1/d7/d30/d35/d77 39 2024-05-12T04:01:32.300 INFO:tasks.workunit.client.0.smithi032.stdout:5/859: write d4/d7/d14/d26/d33/d4a/d38/d79/fb9 [673802,987] 0 2024-05-12T04:01:32.301 INFO:tasks.workunit.client.0.smithi032.stdout:7/690: sync 2024-05-12T04:01:32.301 INFO:tasks.workunit.client.0.smithi032.stdout:8/783: sync 2024-05-12T04:01:32.302 INFO:tasks.workunit.client.1.smithi033.stdout:1/849: write fd [2247237,75249] 0 2024-05-12T04:01:32.304 INFO:tasks.workunit.client.0.smithi032.stdout:0/763: getdents df/d10/d3b/d75 0 2024-05-12T04:01:32.307 INFO:tasks.workunit.client.0.smithi032.stdout:4/671: mkdir d0/d3/d12/d1d/dc5 0 2024-05-12T04:01:32.309 INFO:tasks.workunit.client.0.smithi032.stdout:7/691: chown dd/d1b/d1e/fd1 870 1 2024-05-12T04:01:32.309 INFO:tasks.workunit.client.0.smithi032.stdout:2/752: unlink d1/d34/d96/d41/f5c 0 2024-05-12T04:01:32.310 INFO:tasks.workunit.client.1.smithi033.stdout:8/910: dwrite d1/d16/d27/d4d/d82/fc2 [0,4194304] 0 2024-05-12T04:01:32.311 INFO:tasks.workunit.client.0.smithi032.stdout:9/627: dwrite d0/d3/d4b/d85/fcd [0,4194304] 0 2024-05-12T04:01:32.314 INFO:tasks.workunit.client.1.smithi033.stdout:5/839: dwrite d3/d12/d29/fcd [0,4194304] 0 2024-05-12T04:01:32.314 INFO:tasks.workunit.client.1.smithi033.stdout:4/943: write dd/d11/d5b/dd1/ff8 [1473597,11139] 0 2024-05-12T04:01:32.320 INFO:tasks.workunit.client.1.smithi033.stdout:0/981: mkdir d6/d11/d57/d14b 0 2024-05-12T04:01:32.321 INFO:tasks.workunit.client.0.smithi032.stdout:8/784: dread - d0/d9/d13/d21/fd3 zero size 2024-05-12T04:01:32.328 INFO:tasks.workunit.client.0.smithi032.stdout:2/753: creat d1/d34/d96/da4/ff8 x:0 0 0 2024-05-12T04:01:32.330 INFO:tasks.workunit.client.1.smithi033.stdout:8/911: rmdir d1/d16/d27/d4d/d82/dd9/d36/d3c/da7 39 2024-05-12T04:01:32.331 INFO:tasks.workunit.client.0.smithi032.stdout:5/860: symlink d4/d7/d14/d26/d33/d4a/d38/d79/l12f 0 2024-05-12T04:01:32.332 INFO:tasks.workunit.client.0.smithi032.stdout:9/628: dread - d0/d3/d4b/f81 zero size 2024-05-12T04:01:32.336 INFO:tasks.workunit.client.0.smithi032.stdout:3/813: dwrite d15/d20/ff2 [0,4194304] 0 2024-05-12T04:01:32.336 INFO:tasks.workunit.client.0.smithi032.stdout:8/785: chown d0/d9/d1b/d27/d4c/d4e/le3 1533 1 2024-05-12T04:01:32.337 INFO:tasks.workunit.client.1.smithi033.stdout:2/965: dread f6 [0,4194304] 0 2024-05-12T04:01:32.338 INFO:tasks.workunit.client.0.smithi032.stdout:6/685: dwrite d7/d9/d48/fbf [0,4194304] 0 2024-05-12T04:01:32.339 INFO:tasks.workunit.client.1.smithi033.stdout:2/966: chown dc/d17/d27/d2f/d49/ddd/d112/d125 21288296 1 2024-05-12T04:01:32.342 INFO:tasks.workunit.client.0.smithi032.stdout:9/629: dread d0/d3/d6/d58/f66 [0,4194304] 0 2024-05-12T04:01:32.345 INFO:tasks.workunit.client.0.smithi032.stdout:9/630: truncate d0/d3/d74/fc0 233267 0 2024-05-12T04:01:32.349 INFO:tasks.workunit.client.1.smithi033.stdout:8/912: rmdir d1/d16/d27/d4d/d82/dd9/d36 39 2024-05-12T04:01:32.351 INFO:tasks.workunit.client.1.smithi033.stdout:6/941: dwrite d4/d5/fab [0,4194304] 0 2024-05-12T04:01:32.352 INFO:tasks.workunit.client.1.smithi033.stdout:4/944: fsync dd/d11/d5b/fee 0 2024-05-12T04:01:32.356 INFO:tasks.workunit.client.1.smithi033.stdout:5/840: symlink d3/d94/d85/d99/l113 0 2024-05-12T04:01:32.361 INFO:tasks.workunit.client.1.smithi033.stdout:3/914: sync 2024-05-12T04:01:32.362 INFO:tasks.workunit.client.1.smithi033.stdout:1/850: truncate d11/d1c/d43/f5e 4185283 0 2024-05-12T04:01:32.363 INFO:tasks.workunit.client.1.smithi033.stdout:0/982: dwrite d6/db/d2c/f83 [0,4194304] 0 2024-05-12T04:01:32.366 INFO:tasks.workunit.client.1.smithi033.stdout:6/942: creat d4/d43/d57/dc4/ddf/f13a x:0 0 0 2024-05-12T04:01:32.368 INFO:tasks.workunit.client.1.smithi033.stdout:4/945: mknod dd/d7a/d87/d7e/da7/d66/d42/d59/dd0/c127 0 2024-05-12T04:01:32.369 INFO:tasks.workunit.client.1.smithi033.stdout:5/841: dwrite d3/d94/d54/d57/d73/f10f [0,4194304] 0 2024-05-12T04:01:32.377 INFO:tasks.workunit.client.1.smithi033.stdout:0/983: dwrite d6/db/d2c/d82/f13c [0,4194304] 0 2024-05-12T04:01:32.383 INFO:tasks.workunit.client.1.smithi033.stdout:1/851: rename d11/d17/d27/d36/d3f/d69 to d11/d17/d27/d36/d3f/d4a/dc6/de1/d114 0 2024-05-12T04:01:32.393 INFO:tasks.workunit.client.1.smithi033.stdout:8/913: dread d1/d16/d27/d4d/d82/dd9/d36/faf [0,4194304] 0 2024-05-12T04:01:32.394 INFO:tasks.workunit.client.1.smithi033.stdout:3/915: mkdir db/d17/d3a/d93/d104/d13d 0 2024-05-12T04:01:32.396 INFO:tasks.workunit.client.1.smithi033.stdout:3/916: truncate db/d17/d3a/d8a/f107 867616 0 2024-05-12T04:01:32.401 INFO:tasks.workunit.client.1.smithi033.stdout:1/852: mkdir d11/d1c/db8/d115 0 2024-05-12T04:01:32.405 INFO:tasks.workunit.client.1.smithi033.stdout:8/914: creat d1/d16/d29/d63/f112 x:0 0 0 2024-05-12T04:01:32.407 INFO:tasks.workunit.client.1.smithi033.stdout:0/984: mknod d6/d11/c14c 0 2024-05-12T04:01:32.410 INFO:tasks.workunit.client.1.smithi033.stdout:5/842: write d3/d12/d29/d2b/d8e/f110 [938955,112637] 0 2024-05-12T04:01:32.414 INFO:tasks.workunit.client.1.smithi033.stdout:4/946: rename dd/d11/l88 to dd/d11/d5b/dd1/l128 0 2024-05-12T04:01:32.414 INFO:tasks.workunit.client.1.smithi033.stdout:4/947: stat dd/c26 0 2024-05-12T04:01:32.419 INFO:tasks.workunit.client.1.smithi033.stdout:0/985: creat d6/dc/d58/dff/f14d x:0 0 0 2024-05-12T04:01:32.420 INFO:tasks.workunit.client.1.smithi033.stdout:0/986: chown d6/dc/ld3 446 1 2024-05-12T04:01:32.422 INFO:tasks.workunit.client.1.smithi033.stdout:3/917: mknod db/d17/c13e 0 2024-05-12T04:01:32.424 INFO:tasks.workunit.client.1.smithi033.stdout:6/943: getdents d4/d1a/d3c/dd9/de2 0 2024-05-12T04:01:32.426 INFO:tasks.workunit.client.1.smithi033.stdout:5/843: mknod d3/d94/d54/d57/d70/da3/dbe/d16/dd7/c114 0 2024-05-12T04:01:32.427 INFO:tasks.workunit.client.1.smithi033.stdout:0/987: dwrite d6/d11/dc7/dc8/f11a [0,4194304] 0 2024-05-12T04:01:32.429 INFO:tasks.workunit.client.1.smithi033.stdout:0/988: chown d6/d11/d98/d143/c128 1590121 1 2024-05-12T04:01:32.430 INFO:tasks.workunit.client.0.smithi032.stdout:4/672: mkdir d0/d3/d12/d1d/d63/d87/dc6 0 2024-05-12T04:01:32.432 INFO:tasks.workunit.client.1.smithi033.stdout:4/948: unlink dd/d7a/d87/d7e/da7/d66/f49 0 2024-05-12T04:01:32.434 INFO:tasks.workunit.client.1.smithi033.stdout:2/967: sync 2024-05-12T04:01:32.436 INFO:tasks.workunit.client.1.smithi033.stdout:8/915: symlink d1/l113 0 2024-05-12T04:01:32.441 INFO:tasks.workunit.client.0.smithi032.stdout:2/754: symlink d1/d34/d4a/lf9 0 2024-05-12T04:01:32.445 INFO:tasks.workunit.client.0.smithi032.stdout:2/755: read d1/d13/f1f [2174410,123480] 0 2024-05-12T04:01:32.448 INFO:tasks.workunit.client.1.smithi033.stdout:0/989: fsync d6/db/d2c/d73/f7a 0 2024-05-12T04:01:32.451 INFO:tasks.workunit.client.1.smithi033.stdout:4/949: creat dd/d11/d5b/d84/f129 x:0 0 0 2024-05-12T04:01:32.456 INFO:tasks.workunit.client.1.smithi033.stdout:2/968: dread - dc/d13/d1c/d98/df0/f132 zero size 2024-05-12T04:01:32.457 INFO:tasks.workunit.client.1.smithi033.stdout:4/950: dwrite dd/d7a/d87/d7e/da7/d38/d104/d109/f10f [0,4194304] 0 2024-05-12T04:01:32.459 INFO:tasks.workunit.client.0.smithi032.stdout:4/673: chown d0/d3/d24/l4a 477 1 2024-05-12T04:01:32.462 INFO:tasks.workunit.client.1.smithi033.stdout:3/918: creat db/d17/d25/d32/d3c/d44/d117/f13f x:0 0 0 2024-05-12T04:01:32.462 INFO:tasks.workunit.client.0.smithi032.stdout:3/814: write d15/d20/d8b/d9b/fa8 [333656,130926] 0 2024-05-12T04:01:32.464 INFO:tasks.workunit.client.0.smithi032.stdout:3/815: chown d15/d1b/d43 3221 1 2024-05-12T04:01:32.465 INFO:tasks.workunit.client.0.smithi032.stdout:7/692: mknod dd/d1b/d1e/d45/cdb 0 2024-05-12T04:01:32.467 INFO:tasks.workunit.client.0.smithi032.stdout:3/816: write d15/d20/d22/d4f/de0/fe7 [744009,5337] 0 2024-05-12T04:01:32.469 INFO:tasks.workunit.client.1.smithi033.stdout:5/844: mkdir d3/d94/d54/d57/d70/da3/dbe/d16/d115 0 2024-05-12T04:01:32.473 INFO:tasks.workunit.client.1.smithi033.stdout:1/853: getdents d11/d17/d27/d36 0 2024-05-12T04:01:32.475 INFO:tasks.workunit.client.0.smithi032.stdout:8/786: dwrite d0/d9/d13/d21/d25/f6e [0,4194304] 0 2024-05-12T04:01:32.477 INFO:tasks.workunit.client.1.smithi033.stdout:8/916: rename d1/d16/d27/d4d/c87 to d1/d16/d27/d7c/c114 0 2024-05-12T04:01:32.478 INFO:tasks.workunit.client.0.smithi032.stdout:6/686: dwrite d7/d9/d5c/d62/fad [0,4194304] 0 2024-05-12T04:01:32.482 INFO:tasks.workunit.client.0.smithi032.stdout:8/787: chown d0/d9/d1b/d33/d73/db7 0 1 2024-05-12T04:01:32.485 INFO:tasks.workunit.client.1.smithi033.stdout:3/919: fdatasync db/d17/d3e/d65/dfb/d11a/d13b/fda 0 2024-05-12T04:01:32.486 INFO:tasks.workunit.client.1.smithi033.stdout:3/920: stat db/d17/d3e/d65/c135 0 2024-05-12T04:01:32.489 INFO:tasks.workunit.client.0.smithi032.stdout:4/674: creat d0/d3/d12/d8d/da1/fc7 x:0 0 0 2024-05-12T04:01:32.493 INFO:tasks.workunit.client.1.smithi033.stdout:3/921: dwrite db/d17/d25/d32/d53/fb6 [0,4194304] 0 2024-05-12T04:01:32.494 INFO:tasks.workunit.client.0.smithi032.stdout:9/631: write d0/d3/d6/d7/da/f53 [2862983,102530] 0 2024-05-12T04:01:32.494 INFO:tasks.workunit.client.0.smithi032.stdout:0/764: link fb df/d10/d3d/d4a/d9a/ffe 0 2024-05-12T04:01:32.495 INFO:tasks.workunit.client.1.smithi033.stdout:0/990: mkdir d6/d11/d131/d149/d14e 0 2024-05-12T04:01:32.495 INFO:tasks.workunit.client.1.smithi033.stdout:3/922: readlink db/d17/d25/l10f 0 2024-05-12T04:01:32.496 INFO:tasks.workunit.client.0.smithi032.stdout:9/632: readlink d0/d84/l9d 0 2024-05-12T04:01:32.497 INFO:tasks.workunit.client.0.smithi032.stdout:3/817: fdatasync d15/d24/f7b 0 2024-05-12T04:01:32.498 INFO:tasks.workunit.client.0.smithi032.stdout:9/633: readlink d0/d3/d6/d7/da/l26 0 2024-05-12T04:01:32.498 INFO:tasks.workunit.client.0.smithi032.stdout:3/818: chown d15/d20/d26/d6a/fb4 0 1 2024-05-12T04:01:32.499 INFO:tasks.workunit.client.1.smithi033.stdout:2/969: mkdir dc/d17/d32/d136 0 2024-05-12T04:01:32.501 INFO:tasks.workunit.client.0.smithi032.stdout:1/735: link d1/d7/d30/d18/d87/f89 d1/d7/d30/d35/d77/ff4 0 2024-05-12T04:01:32.505 INFO:tasks.workunit.client.1.smithi033.stdout:4/951: mknod dd/d7a/d87/d7e/da7/d38/c12a 0 2024-05-12T04:01:32.505 INFO:tasks.workunit.client.1.smithi033.stdout:8/917: readlink d1/d16/d27/d4d/d51/d8e/la8 0 2024-05-12T04:01:32.506 INFO:tasks.workunit.client.0.smithi032.stdout:6/687: symlink d7/d9/d48/d67/d25/lef 0 2024-05-12T04:01:32.507 INFO:tasks.workunit.client.1.smithi033.stdout:8/918: fdatasync d1/d16/d27/d4d/d82/dd9/d36/fce 0 2024-05-12T04:01:32.507 INFO:tasks.workunit.client.0.smithi032.stdout:5/861: link d4/d7/d14/d26/dc6/fc8 d4/d7/d14/d26/d33/d4a/d38/d79/f130 0 2024-05-12T04:01:32.509 INFO:tasks.workunit.client.1.smithi033.stdout:5/845: mknod d3/d94/d54/d57/d70/dfb/c116 0 2024-05-12T04:01:32.511 INFO:tasks.workunit.client.0.smithi032.stdout:7/693: write fa [551817,112410] 0 2024-05-12T04:01:32.512 INFO:tasks.workunit.client.0.smithi032.stdout:8/788: read d0/f68 [946566,75307] 0 2024-05-12T04:01:32.512 INFO:tasks.workunit.client.1.smithi033.stdout:3/923: mknod db/d17/d3a/d8a/c140 0 2024-05-12T04:01:32.514 INFO:tasks.workunit.client.0.smithi032.stdout:0/765: write df/d10/d48/d70/ffb [1996494,52089] 0 2024-05-12T04:01:32.518 INFO:tasks.workunit.client.0.smithi032.stdout:2/756: write d1/d13/d70/d85/f88 [2162841,33130] 0 2024-05-12T04:01:32.521 INFO:tasks.workunit.client.0.smithi032.stdout:4/675: dread d0/d3/d24/f42 [0,4194304] 0 2024-05-12T04:01:32.521 INFO:tasks.workunit.client.0.smithi032.stdout:9/634: rename d0/d3/d4b/d85/dbe/l86 to d0/d3/d6/d7/da/d39/lce 0 2024-05-12T04:01:32.523 INFO:tasks.workunit.client.0.smithi032.stdout:1/736: mknod d1/d7/d30/d18/d93/cf5 0 2024-05-12T04:01:32.529 INFO:tasks.workunit.client.0.smithi032.stdout:5/862: rmdir d4/d7/d14/d26/d33/d4a/d38 39 2024-05-12T04:01:32.534 INFO:tasks.workunit.client.0.smithi032.stdout:8/789: symlink d0/d9/d1b/d27/d88/d56/d63/d8f/db0/l10c 0 2024-05-12T04:01:32.536 INFO:tasks.workunit.client.0.smithi032.stdout:3/819: dread d15/d20/d26/f29 [4194304,4194304] 0 2024-05-12T04:01:32.536 INFO:tasks.workunit.client.0.smithi032.stdout:8/790: write d0/d9/d13/d21/f100 [598544,6287] 0 2024-05-12T04:01:32.538 INFO:tasks.workunit.client.0.smithi032.stdout:3/820: chown d15/d1b/d43/l7f 13 1 2024-05-12T04:01:32.540 INFO:tasks.workunit.client.0.smithi032.stdout:3/821: read d15/d20/d22/f47 [1835795,79359] 0 2024-05-12T04:01:32.541 INFO:tasks.workunit.client.0.smithi032.stdout:2/757: fdatasync d1/d34/d3e/fcd 0 2024-05-12T04:01:32.544 INFO:tasks.workunit.client.0.smithi032.stdout:4/676: fdatasync d0/d3/d12/d1d/f9e 0 2024-05-12T04:01:32.547 INFO:tasks.workunit.client.0.smithi032.stdout:9/635: creat d0/d3/d4b/d88/fcf x:0 0 0 2024-05-12T04:01:32.548 INFO:tasks.workunit.client.0.smithi032.stdout:1/737: rmdir d1/d7/d30/d18/d93/dc1/dc3 39 2024-05-12T04:01:32.554 INFO:tasks.workunit.client.0.smithi032.stdout:6/688: dwrite d7/d9/d48/d67/d25/f74 [0,4194304] 0 2024-05-12T04:01:32.555 INFO:tasks.workunit.client.0.smithi032.stdout:5/863: mkdir d4/d7/d14/d39/d2b/dd6/d131 0 2024-05-12T04:01:32.559 INFO:tasks.workunit.client.0.smithi032.stdout:3/822: dread d15/f19 [0,4194304] 0 2024-05-12T04:01:32.560 INFO:tasks.workunit.client.0.smithi032.stdout:7/694: mknod dd/d6c/dc3/dda/cdc 0 2024-05-12T04:01:32.569 INFO:tasks.workunit.client.0.smithi032.stdout:4/677: creat d0/d3/d12/d1d/d63/d87/d95/fc8 x:0 0 0 2024-05-12T04:01:32.572 INFO:tasks.workunit.client.0.smithi032.stdout:9/636: fsync d0/d3/d4b/d6f/f8b 0 2024-05-12T04:01:32.572 INFO:tasks.workunit.client.0.smithi032.stdout:9/637: fsync d0/d3/d4b/d88/f96 0 2024-05-12T04:01:32.577 INFO:tasks.workunit.client.0.smithi032.stdout:5/864: mkdir d4/d7/d14/d26/d47/d132 0 2024-05-12T04:01:32.579 INFO:tasks.workunit.client.0.smithi032.stdout:5/865: read d4/d7/d14/d26/d33/d72/fa7 [1071325,31144] 0 2024-05-12T04:01:32.581 INFO:tasks.workunit.client.0.smithi032.stdout:5/866: chown d4/d7/d14/d26/d33/d88/c110 7 1 2024-05-12T04:01:32.582 INFO:tasks.workunit.client.0.smithi032.stdout:3/823: creat d15/dec/f11d x:0 0 0 2024-05-12T04:01:32.583 INFO:tasks.workunit.client.0.smithi032.stdout:3/824: stat d15/d24/d2a/dc7 0 2024-05-12T04:01:32.585 INFO:tasks.workunit.client.0.smithi032.stdout:7/695: readlink dd/d1b/d1e/d45/d55/l63 0 2024-05-12T04:01:32.592 INFO:tasks.workunit.client.0.smithi032.stdout:3/825: dwrite d15/ffa [0,4194304] 0 2024-05-12T04:01:32.593 INFO:tasks.workunit.client.0.smithi032.stdout:3/826: dread - d15/d20/d26/d6a/d6e/d7d/d9a/fa3 zero size 2024-05-12T04:01:32.599 INFO:tasks.workunit.client.1.smithi033.stdout:6/944: rename d4/d1a/d67/dc3/c113 to d4/dad/d110/c13b 0 2024-05-12T04:01:32.601 INFO:tasks.workunit.client.1.smithi033.stdout:2/970: mknod dc/d17/d27/d2f/d49/c137 0 2024-05-12T04:01:32.602 INFO:tasks.workunit.client.0.smithi032.stdout:1/738: symlink d1/d25/da5/lf6 0 2024-05-12T04:01:32.604 INFO:tasks.workunit.client.1.smithi033.stdout:4/952: creat dd/d11/d5b/dd1/f12b x:0 0 0 2024-05-12T04:01:32.607 INFO:tasks.workunit.client.1.smithi033.stdout:6/945: dwrite d4/d1a/fed [0,4194304] 0 2024-05-12T04:01:32.609 INFO:tasks.workunit.client.0.smithi032.stdout:9/638: rename d0/d3/f42 to d0/d3/d1b/fd0 0 2024-05-12T04:01:32.618 INFO:tasks.workunit.client.1.smithi033.stdout:1/854: creat d11/d17/d27/d36/d3f/d4a/dc6/f116 x:0 0 0 2024-05-12T04:01:32.620 INFO:tasks.workunit.client.1.smithi033.stdout:0/991: sync 2024-05-12T04:01:32.621 INFO:tasks.workunit.client.1.smithi033.stdout:0/992: stat d6/db/d2c/c4d 0 2024-05-12T04:01:32.621 INFO:tasks.workunit.client.0.smithi032.stdout:0/766: getdents df/d30/d4e/d55 0 2024-05-12T04:01:32.622 INFO:tasks.workunit.client.1.smithi033.stdout:3/924: symlink db/d17/d3a/dc0/l141 0 2024-05-12T04:01:32.623 INFO:tasks.workunit.client.0.smithi032.stdout:4/678: write d0/d3/d31/d80/f8e [461363,55631] 0 2024-05-12T04:01:32.623 INFO:tasks.workunit.client.1.smithi033.stdout:3/925: dread - db/f11c zero size 2024-05-12T04:01:32.624 INFO:tasks.workunit.client.0.smithi032.stdout:7/696: truncate dd/d1b/d1e/d45/d55/d72/da0/dbd/f99 645622 0 2024-05-12T04:01:32.626 INFO:tasks.workunit.client.1.smithi033.stdout:1/855: dread fd [0,4194304] 0 2024-05-12T04:01:32.627 INFO:tasks.workunit.client.1.smithi033.stdout:2/971: mkdir dc/d13/d65/d138 0 2024-05-12T04:01:32.628 INFO:tasks.workunit.client.0.smithi032.stdout:2/758: getdents d1/d4d/d5e 0 2024-05-12T04:01:32.628 INFO:tasks.workunit.client.0.smithi032.stdout:2/759: readlink d1/d13/d6b/l6f 0 2024-05-12T04:01:32.629 INFO:tasks.workunit.client.1.smithi033.stdout:0/993: dwrite d6/db/d2c/f83 [0,4194304] 0 2024-05-12T04:01:32.633 INFO:tasks.workunit.client.1.smithi033.stdout:3/926: dwrite db/f20 [0,4194304] 0 2024-05-12T04:01:32.633 INFO:tasks.workunit.client.1.smithi033.stdout:6/946: creat d4/d1a/d67/d6f/f13c x:0 0 0 2024-05-12T04:01:32.637 INFO:tasks.workunit.client.1.smithi033.stdout:5/846: mknod d3/d94/d54/d57/d70/da3/c117 0 2024-05-12T04:01:32.641 INFO:tasks.workunit.client.1.smithi033.stdout:8/919: write d1/d16/d27/d4d/d51/ff5 [89963,5456] 0 2024-05-12T04:01:32.643 INFO:tasks.workunit.client.1.smithi033.stdout:8/920: stat d1/d16/d27/d4d/d82/dd9/c44 0 2024-05-12T04:01:32.643 INFO:tasks.workunit.client.0.smithi032.stdout:9/639: rmdir d0/d3/d6/d7/da 39 2024-05-12T04:01:32.649 INFO:tasks.workunit.client.0.smithi032.stdout:0/767: mkdir df/d30/d8d/d8f/dff 0 2024-05-12T04:01:32.650 INFO:tasks.workunit.client.0.smithi032.stdout:0/768: chown df/d30/d4e/d55/dd4 0 1 2024-05-12T04:01:32.650 INFO:tasks.workunit.client.1.smithi033.stdout:8/921: dwrite d1/d16/d29/d63/f112 [0,4194304] 0 2024-05-12T04:01:32.651 INFO:tasks.workunit.client.0.smithi032.stdout:0/769: chown df/d30/l3f 3872 1 2024-05-12T04:01:32.653 INFO:tasks.workunit.client.1.smithi033.stdout:5/847: dread d3/d94/d54/d57/d70/da3/dc9/fb2 [0,4194304] 0 2024-05-12T04:01:32.655 INFO:tasks.workunit.client.0.smithi032.stdout:7/697: rmdir dd/d1b/d43/d74 39 2024-05-12T04:01:32.660 INFO:tasks.workunit.client.0.smithi032.stdout:2/760: mknod d1/d13/d70/cfa 0 2024-05-12T04:01:32.664 INFO:tasks.workunit.client.0.smithi032.stdout:3/827: mkdir d15/d33/d3f/d75/d11e 0 2024-05-12T04:01:32.669 INFO:tasks.workunit.client.0.smithi032.stdout:4/679: dwrite d0/d3/d24/f72 [0,4194304] 0 2024-05-12T04:01:32.671 INFO:tasks.workunit.client.1.smithi033.stdout:2/972: mkdir dc/d17/d27/d68/d9f/df7/d139 0 2024-05-12T04:01:32.671 INFO:tasks.workunit.client.1.smithi033.stdout:2/973: dread - dc/f117 zero size 2024-05-12T04:01:32.673 INFO:tasks.workunit.client.0.smithi032.stdout:6/689: getdents d7/d9/d48/d67/d8d/dbc 0 2024-05-12T04:01:32.673 INFO:tasks.workunit.client.1.smithi033.stdout:2/974: write dc/d13/d1f/d6e/f12b [371016,57392] 0 2024-05-12T04:01:32.675 INFO:tasks.workunit.client.1.smithi033.stdout:8/922: write d1/d16/d27/d4d/d82/dd9/ffa [704121,108305] 0 2024-05-12T04:01:32.681 INFO:tasks.workunit.client.1.smithi033.stdout:6/947: dread d4/d5/fa1 [0,4194304] 0 2024-05-12T04:01:32.681 INFO:tasks.workunit.client.1.smithi033.stdout:3/927: dwrite db/d17/d3e/d65/d91/fb3 [0,4194304] 0 2024-05-12T04:01:32.683 INFO:tasks.workunit.client.1.smithi033.stdout:1/856: creat d11/d17/d27/d36/d3f/f117 x:0 0 0 2024-05-12T04:01:32.686 INFO:tasks.workunit.client.1.smithi033.stdout:4/953: sync 2024-05-12T04:01:32.687 INFO:tasks.workunit.client.1.smithi033.stdout:4/954: dread - dd/d7a/d87/d7e/da7/d38/d104/d109/f126 zero size 2024-05-12T04:01:32.687 INFO:tasks.workunit.client.0.smithi032.stdout:4/680: dwrite d0/d3/d12/d1d/d63/f9b [0,4194304] 0 2024-05-12T04:01:32.690 INFO:tasks.workunit.client.1.smithi033.stdout:0/994: link d6/dc/d58/fe8 d6/d11/d131/d149/f14f 0 2024-05-12T04:01:32.691 INFO:tasks.workunit.client.0.smithi032.stdout:6/690: rmdir d7/d9/d48/d67/d8d/dbc/de2/d9d 39 2024-05-12T04:01:32.693 INFO:tasks.workunit.client.1.smithi033.stdout:6/948: mkdir d4/d43/d57/d13d 0 2024-05-12T04:01:32.694 INFO:tasks.workunit.client.0.smithi032.stdout:0/770: write df/d30/d36/f41 [2385038,58473] 0 2024-05-12T04:01:32.697 INFO:tasks.workunit.client.1.smithi033.stdout:2/975: creat dc/d17/d32/d136/f13a x:0 0 0 2024-05-12T04:01:32.697 INFO:tasks.workunit.client.1.smithi033.stdout:2/976: chown dc/d13/dfb/dd7/f114 19829 1 2024-05-12T04:01:32.699 INFO:tasks.workunit.client.1.smithi033.stdout:3/928: symlink db/d17/d3e/d65/d91/l142 0 2024-05-12T04:01:32.702 INFO:tasks.workunit.client.1.smithi033.stdout:5/848: link d3/d94/d54/d57/d73/lc5 d3/d12/d83/l118 0 2024-05-12T04:01:32.706 INFO:tasks.workunit.client.0.smithi032.stdout:3/828: creat d15/d20/f11f x:0 0 0 2024-05-12T04:01:32.707 INFO:tasks.workunit.client.0.smithi032.stdout:1/739: dwrite d1/d7/d30/d18/d87/db7/fc5 [0,4194304] 0 2024-05-12T04:01:32.713 INFO:tasks.workunit.client.1.smithi033.stdout:4/955: symlink dd/d11/d5b/d84/dd7/l12c 0 2024-05-12T04:01:32.714 INFO:tasks.workunit.client.0.smithi032.stdout:3/829: write d15/d20/d8b/d9b/fa8 [182986,19288] 0 2024-05-12T04:01:32.716 INFO:tasks.workunit.client.0.smithi032.stdout:4/681: mkdir d0/d3/d31/d80/d84/dc9 0 2024-05-12T04:01:32.718 INFO:tasks.workunit.client.0.smithi032.stdout:7/698: dwrite dd/d7b/fb5 [0,4194304] 0 2024-05-12T04:01:32.721 INFO:tasks.workunit.client.0.smithi032.stdout:0/771: dread df/d10/d3d/f85 [0,4194304] 0 2024-05-12T04:01:32.724 INFO:tasks.workunit.client.0.smithi032.stdout:7/699: chown dd/d7b/d9a/d9c/dc2/lb0 16426709 1 2024-05-12T04:01:32.725 INFO:tasks.workunit.client.0.smithi032.stdout:9/640: dread d0/d3/d6/d7/d27/f97 [0,4194304] 0 2024-05-12T04:01:32.725 INFO:tasks.workunit.client.1.smithi033.stdout:5/849: dread d3/d12/d25/d26/f9c [0,4194304] 0 2024-05-12T04:01:32.728 INFO:tasks.workunit.client.0.smithi032.stdout:2/761: link d1/f24 d1/d34/d96/ffb 0 2024-05-12T04:01:32.736 INFO:tasks.workunit.client.1.smithi033.stdout:3/929: unlink db/d17/d3a/d93/ld8 0 2024-05-12T04:01:32.738 INFO:tasks.workunit.client.1.smithi033.stdout:6/949: write d4/d1a/d67/f8c [4514552,34255] 0 2024-05-12T04:01:32.739 INFO:tasks.workunit.client.1.smithi033.stdout:3/930: write db/d17/d3a/dc0/fdf [101670,84878] 0 2024-05-12T04:01:32.739 INFO:tasks.workunit.client.1.smithi033.stdout:0/995: dwrite d6/db/d2c/f3e [0,4194304] 0 2024-05-12T04:01:32.742 INFO:tasks.workunit.client.0.smithi032.stdout:3/830: mknod d15/d24/d40/c120 0 2024-05-12T04:01:32.746 INFO:tasks.workunit.client.1.smithi033.stdout:2/977: write dc/d13/d1f/d6e/fb6 [3469822,57459] 0 2024-05-12T04:01:32.746 INFO:tasks.workunit.client.1.smithi033.stdout:4/956: mknod dd/d7a/c12d 0 2024-05-12T04:01:32.749 INFO:tasks.workunit.client.1.smithi033.stdout:8/923: link d1/d4/d46/l6e d1/d16/d27/d4d/d82/dd9/d36/d3c/d64/l115 0 2024-05-12T04:01:32.749 INFO:tasks.workunit.client.0.smithi032.stdout:6/691: creat d7/d9/d48/d67/d1a/d93/dde/ff0 x:0 0 0 2024-05-12T04:01:32.754 INFO:tasks.workunit.client.0.smithi032.stdout:2/762: creat d1/d34/d96/d41/d52/ffc x:0 0 0 2024-05-12T04:01:32.754 INFO:tasks.workunit.client.0.smithi032.stdout:9/641: dread d0/d3/d6/d58/fa7 [0,4194304] 0 2024-05-12T04:01:32.755 INFO:tasks.workunit.client.0.smithi032.stdout:1/740: mknod d1/d6a/d72/da9/ded/cf7 0 2024-05-12T04:01:32.759 INFO:tasks.workunit.client.1.smithi033.stdout:3/931: mknod db/d17/d3a/d93/deb/c143 0 2024-05-12T04:01:32.760 INFO:tasks.workunit.client.0.smithi032.stdout:4/682: mknod d0/d3/d12/d1d/dc5/cca 0 2024-05-12T04:01:32.763 INFO:tasks.workunit.client.0.smithi032.stdout:0/772: creat df/d30/d36/ddd/f100 x:0 0 0 2024-05-12T04:01:32.765 INFO:tasks.workunit.client.0.smithi032.stdout:0/773: read df/d10/d3b/d75/f9b [1545628,40406] 0 2024-05-12T04:01:32.765 INFO:tasks.workunit.client.0.smithi032.stdout:0/774: stat df 0 2024-05-12T04:01:32.766 INFO:tasks.workunit.client.1.smithi033.stdout:5/850: dread d3/d12/f3f [0,4194304] 0 2024-05-12T04:01:32.768 INFO:tasks.workunit.client.1.smithi033.stdout:4/957: truncate dd/d69/dc4/fbb 638317 0 2024-05-12T04:01:32.770 INFO:tasks.workunit.client.1.smithi033.stdout:4/958: read dd/d11/d5b/dd1/f10c [263533,84749] 0 2024-05-12T04:01:32.772 INFO:tasks.workunit.client.1.smithi033.stdout:6/950: write d4/d43/d57/dc4/ffb [410437,110796] 0 2024-05-12T04:01:32.777 INFO:tasks.workunit.client.1.smithi033.stdout:4/959: dwrite dd/d69/da4/f107 [0,4194304] 0 2024-05-12T04:01:32.778 INFO:tasks.workunit.client.1.smithi033.stdout:4/960: readlink dd/d7a/l11c 0 2024-05-12T04:01:32.780 INFO:tasks.workunit.client.0.smithi032.stdout:3/831: dwrite d15/d24/d2a/d5b/fc9 [0,4194304] 0 2024-05-12T04:01:32.782 INFO:tasks.workunit.client.0.smithi032.stdout:7/700: truncate dd/d6c/f77 351072 0 2024-05-12T04:01:32.787 INFO:tasks.workunit.client.1.smithi033.stdout:1/857: getdents d11/d17/d27/d36/d3f/d4a/dc6 0 2024-05-12T04:01:32.787 INFO:tasks.workunit.client.1.smithi033.stdout:1/858: chown d11/d17/d84/da3 157741560 1 2024-05-12T04:01:32.791 INFO:tasks.workunit.client.1.smithi033.stdout:3/932: write db/d17/d3a/d67/d76/da0/fb1 [1880459,89017] 0 2024-05-12T04:01:32.794 INFO:tasks.workunit.client.0.smithi032.stdout:2/763: dwrite d1/d34/d96/d41/d2c/f37 [4194304,4194304] 0 2024-05-12T04:01:32.795 INFO:tasks.workunit.client.1.smithi033.stdout:3/933: write db/d17/d3a/d8a/f107 [980149,55566] 0 2024-05-12T04:01:32.795 INFO:tasks.workunit.client.1.smithi033.stdout:4/961: dread dd/d11/d5b/dd1/ff8 [0,4194304] 0 2024-05-12T04:01:32.795 INFO:tasks.workunit.client.0.smithi032.stdout:1/741: fdatasync d1/f47 0 2024-05-12T04:01:32.798 INFO:tasks.workunit.client.1.smithi033.stdout:8/924: unlink d1/d16/d27/d4d/d82/dd9/d36/d3c/da7/feb 0 2024-05-12T04:01:32.799 INFO:tasks.workunit.client.0.smithi032.stdout:2/764: dread - d1/d13/da0/dae/fc1 zero size 2024-05-12T04:01:32.803 INFO:tasks.workunit.client.0.smithi032.stdout:3/832: mkdir d15/d20/d22/d7a/dbe/d121 0 2024-05-12T04:01:32.806 INFO:tasks.workunit.client.0.smithi032.stdout:1/742: chown d1/d6a/d72/da9/l7b 33876 1 2024-05-12T04:01:32.806 INFO:tasks.workunit.client.0.smithi032.stdout:6/692: dwrite d7/d9/d48/d67/d8d/dbc/de2/f8c [4194304,4194304] 0 2024-05-12T04:01:32.806 INFO:tasks.workunit.client.1.smithi033.stdout:0/996: link d6/d11/dc7/dd9/d10b/dfb/l9f d6/dc/df1/d137/l150 0 2024-05-12T04:01:32.809 INFO:tasks.workunit.client.0.smithi032.stdout:3/833: truncate d15/d20/d22/d7a/fb2 1274051 0 2024-05-12T04:01:32.810 INFO:tasks.workunit.client.0.smithi032.stdout:0/775: fdatasync f5 0 2024-05-12T04:01:32.814 INFO:tasks.workunit.client.0.smithi032.stdout:4/683: truncate d0/d3/d31/d80/d84/f64 308667 0 2024-05-12T04:01:32.814 INFO:tasks.workunit.client.0.smithi032.stdout:4/684: chown d0 8 1 2024-05-12T04:01:32.815 INFO:tasks.workunit.client.0.smithi032.stdout:4/685: fdatasync d0/d3/d12/d8d/da1/fc7 0 2024-05-12T04:01:32.816 INFO:tasks.workunit.client.0.smithi032.stdout:7/701: symlink dd/d1b/d43/ldd 0 2024-05-12T04:01:32.819 INFO:tasks.workunit.client.1.smithi033.stdout:4/962: mkdir dd/d69/dc4/d12e 0 2024-05-12T04:01:32.822 INFO:tasks.workunit.client.1.smithi033.stdout:5/851: write d3/d12/d25/d26/f31 [2886916,54882] 0 2024-05-12T04:01:32.824 INFO:tasks.workunit.client.0.smithi032.stdout:1/743: symlink d1/d25/lf8 0 2024-05-12T04:01:32.828 INFO:tasks.workunit.client.1.smithi033.stdout:2/978: dwrite dc/d17/dcc/fea [0,4194304] 0 2024-05-12T04:01:32.830 INFO:tasks.workunit.client.1.smithi033.stdout:2/979: dread dc/d13/d1f/f9b [0,4194304] 0 2024-05-12T04:01:32.832 INFO:tasks.workunit.client.1.smithi033.stdout:2/980: truncate dc/d17/d32/f131 1035643 0 2024-05-12T04:01:32.833 INFO:tasks.workunit.client.1.smithi033.stdout:8/925: rmdir d1/d16 39 2024-05-12T04:01:32.835 INFO:tasks.workunit.client.0.smithi032.stdout:3/834: dwrite d15/d33/d3f/d75/fcb [0,4194304] 0 2024-05-12T04:01:32.838 INFO:tasks.workunit.client.0.smithi032.stdout:0/776: mknod df/d30/d36/ddd/c101 0 2024-05-12T04:01:32.841 INFO:tasks.workunit.client.1.smithi033.stdout:1/859: mkdir d11/d1c/ddf/d118 0 2024-05-12T04:01:32.843 INFO:tasks.workunit.client.1.smithi033.stdout:4/963: creat dd/d69/f12f x:0 0 0 2024-05-12T04:01:32.846 INFO:tasks.workunit.client.1.smithi033.stdout:4/964: read dd/d11/d5b/dd1/ff9 [461114,109841] 0 2024-05-12T04:01:32.847 INFO:tasks.workunit.client.1.smithi033.stdout:4/965: dread - dd/d11/d75/dd4/fef zero size 2024-05-12T04:01:32.850 INFO:tasks.workunit.client.0.smithi032.stdout:2/765: dwrite d1/f3 [4194304,4194304] 0 2024-05-12T04:01:32.851 INFO:tasks.workunit.client.0.smithi032.stdout:1/744: read d1/d7/d30/d18/f70 [3945726,119082] 0 2024-05-12T04:01:32.852 INFO:tasks.workunit.client.0.smithi032.stdout:1/745: fdatasync d1/dd/f7c 0 2024-05-12T04:01:32.853 INFO:tasks.workunit.client.0.smithi032.stdout:9/642: rename d0/d3/d6/d7/da/d89/cc2 to d0/d3/cd1 0 2024-05-12T04:01:32.855 INFO:tasks.workunit.client.0.smithi032.stdout:7/702: rename dd/d1b/d1e/d45/d55/d72/da0 to dd/d1b/d1e/d45/d55/d72/da0/dbd/db2/dde 22 2024-05-12T04:01:32.858 INFO:tasks.workunit.client.1.smithi033.stdout:6/951: creat d4/dad/f13e x:0 0 0 2024-05-12T04:01:32.859 INFO:tasks.workunit.client.0.smithi032.stdout:0/777: creat df/d30/d36/f102 x:0 0 0 2024-05-12T04:01:32.860 INFO:tasks.workunit.client.0.smithi032.stdout:7/703: chown dd/d1b/c3f 0 1 2024-05-12T04:01:32.861 INFO:tasks.workunit.client.1.smithi033.stdout:8/926: write d1/f6b [576122,8611] 0 2024-05-12T04:01:32.863 INFO:tasks.workunit.client.0.smithi032.stdout:3/835: dread d15/f45 [0,4194304] 0 2024-05-12T04:01:32.863 INFO:tasks.workunit.client.1.smithi033.stdout:0/997: mkdir d6/dc/d58/d6a/d80/d151 0 2024-05-12T04:01:32.863 INFO:tasks.workunit.client.1.smithi033.stdout:0/998: chown d6/dc/d39/l68 1 1 2024-05-12T04:01:32.864 INFO:tasks.workunit.client.0.smithi032.stdout:0/778: write df/f92 [2780143,65079] 0 2024-05-12T04:01:32.867 INFO:tasks.workunit.client.1.smithi033.stdout:1/860: symlink d11/d17/d27/d36/l119 0 2024-05-12T04:01:32.872 INFO:tasks.workunit.client.0.smithi032.stdout:9/643: chown d0/d3/d6/d7/da/d39/c6e 10492449 1 2024-05-12T04:01:32.873 INFO:tasks.workunit.client.0.smithi032.stdout:9/644: write d0/d3/d4b/d88/f96 [1875209,122432] 0 2024-05-12T04:01:32.876 INFO:tasks.workunit.client.0.smithi032.stdout:1/746: rename d1/d6a/d72/da9/ded to d1/d7/d30/d35/d54/d7e/df9 0 2024-05-12T04:01:32.876 INFO:tasks.workunit.client.0.smithi032.stdout:9/645: dread d0/d3/d74/fc0 [0,4194304] 0 2024-05-12T04:01:32.880 INFO:tasks.workunit.client.1.smithi033.stdout:5/852: mknod d3/d94/d54/d57/d70/c119 0 2024-05-12T04:01:32.884 INFO:tasks.workunit.client.1.smithi033.stdout:6/952: truncate d4/d1a/d3c/f65 1899087 0 2024-05-12T04:01:32.885 INFO:tasks.workunit.client.1.smithi033.stdout:6/953: write d4/d5/f2d [3285943,48338] 0 2024-05-12T04:01:32.889 INFO:tasks.workunit.client.0.smithi032.stdout:3/836: mknod d15/d33/d3f/c122 0 2024-05-12T04:01:32.897 INFO:tasks.workunit.client.1.smithi033.stdout:3/934: link db/d17/d3a/c83 db/d17/d25/d32/d3c/d3d/d111/d132/c144 0 2024-05-12T04:01:32.899 INFO:tasks.workunit.client.1.smithi033.stdout:0/999: dread d6/dc/d58/d6a/f89 [0,4194304] 0 2024-05-12T04:01:32.900 INFO:tasks.workunit.client.0.smithi032.stdout:3/837: dwrite d15/dc2/f113 [0,4194304] 0 2024-05-12T04:01:32.903 INFO:tasks.workunit.client.0.smithi032.stdout:7/704: dread dd/d1b/d43/d5e/d37/f54 [0,4194304] 0 2024-05-12T04:01:32.903 INFO:tasks.workunit.client.0.smithi032.stdout:4/686: truncate d0/d75/f83 4097811 0 2024-05-12T04:01:32.904 INFO:tasks.workunit.client.0.smithi032.stdout:9/646: mkdir d0/d3/d1b/dd2 0 2024-05-12T04:01:32.905 INFO:tasks.workunit.client.0.smithi032.stdout:4/687: readlink d0/d3/df/d4e/lc3 0 2024-05-12T04:01:32.909 INFO:tasks.workunit.client.0.smithi032.stdout:4/688: truncate d0/d3/d31/d80/d84/da7/fad 77307 0 2024-05-12T04:01:32.919 INFO:tasks.workunit.client.0.smithi032.stdout:7/705: dread dd/d1b/d1e/f20 [0,4194304] 0 2024-05-12T04:01:32.922 INFO:tasks.workunit.client.0.smithi032.stdout:0/779: write df/d10/f80 [1096327,84622] 0 2024-05-12T04:01:32.922 INFO:tasks.workunit.client.1.smithi033.stdout:8/927: unlink d1/d16/c22 0 2024-05-12T04:01:32.925 INFO:tasks.workunit.client.1.smithi033.stdout:6/954: unlink d4/d8d/ca3 0 2024-05-12T04:01:32.925 INFO:tasks.workunit.client.1.smithi033.stdout:5/853: dread d3/f52 [0,4194304] 0 2024-05-12T04:01:32.926 INFO:tasks.workunit.client.1.smithi033.stdout:1/861: dwrite d11/d1c/ddf/f51 [4194304,4194304] 0 2024-05-12T04:01:32.927 INFO:tasks.workunit.client.1.smithi033.stdout:4/966: creat dd/d7a/d87/d7e/da7/d38/f130 x:0 0 0 2024-05-12T04:01:32.928 INFO:tasks.workunit.client.1.smithi033.stdout:6/955: write d4/d1a/d3c/dd9/de2/def/f128 [940524,97842] 0 2024-05-12T04:01:32.929 INFO:tasks.workunit.client.0.smithi032.stdout:3/838: readlink d15/d24/l62 0 2024-05-12T04:01:32.930 INFO:tasks.workunit.client.1.smithi033.stdout:3/935: symlink db/d59/l145 0 2024-05-12T04:01:32.931 INFO:tasks.workunit.client.1.smithi033.stdout:5/854: dwrite d3/d12/f14 [0,4194304] 0 2024-05-12T04:01:32.938 INFO:tasks.workunit.client.1.smithi033.stdout:2/981: getdents dc/d13/dfb 0 2024-05-12T04:01:32.940 INFO:tasks.workunit.client.0.smithi032.stdout:1/747: dwrite d1/d7/d30/d18/f70 [0,4194304] 0 2024-05-12T04:01:32.940 INFO:tasks.workunit.client.0.smithi032.stdout:4/689: creat d0/d3/d12/d1d/d6b/fcb x:0 0 0 2024-05-12T04:01:32.943 INFO:tasks.workunit.client.0.smithi032.stdout:4/690: chown d0/d3/d12/f5b 4 1 2024-05-12T04:01:32.946 INFO:tasks.workunit.client.0.smithi032.stdout:7/706: mknod dd/d1b/d1e/d45/d55/d72/da0/dbd/cdf 0 2024-05-12T04:01:32.948 INFO:tasks.workunit.client.1.smithi033.stdout:1/862: symlink d11/d1c/d43/l11a 0 2024-05-12T04:01:32.950 INFO:tasks.workunit.client.1.smithi033.stdout:4/967: readlink dd/d11/da5/la9 0 2024-05-12T04:01:32.952 INFO:tasks.workunit.client.1.smithi033.stdout:6/956: mknod d4/d1a/d6c/c13f 0 2024-05-12T04:01:32.957 INFO:tasks.workunit.client.1.smithi033.stdout:5/855: mknod d3/d12/d29/c11a 0 2024-05-12T04:01:32.959 INFO:tasks.workunit.client.0.smithi032.stdout:9/647: dwrite d0/d3/d6/d7/d27/f97 [0,4194304] 0 2024-05-12T04:01:32.961 INFO:tasks.workunit.client.1.smithi033.stdout:2/982: mknod dc/d13/dfb/c13b 0 2024-05-12T04:01:32.963 INFO:tasks.workunit.client.1.smithi033.stdout:2/983: chown dc/d17/d32 4 1 2024-05-12T04:01:32.964 INFO:tasks.workunit.client.1.smithi033.stdout:5/856: dwrite d3/d94/d54/d57/d70/da3/dbe/d16/d78/fe7 [0,4194304] 0 2024-05-12T04:01:32.968 INFO:tasks.workunit.client.0.smithi032.stdout:0/780: write df/d22/f39 [918296,125205] 0 2024-05-12T04:01:32.968 INFO:tasks.workunit.client.0.smithi032.stdout:9/648: chown d0/d3/d6/d7/d2a/d46/c41 516588440 1 2024-05-12T04:01:32.969 INFO:tasks.workunit.client.0.smithi032.stdout:9/649: chown d0/d3/d6/d7/d27/f31 532 1 2024-05-12T04:01:32.969 INFO:tasks.workunit.client.1.smithi033.stdout:1/863: rename d11/fb7 to d11/d1c/db8/f11b 0 2024-05-12T04:01:32.970 INFO:tasks.workunit.client.0.smithi032.stdout:1/748: creat d1/d6a/def/ffa x:0 0 0 2024-05-12T04:01:32.971 INFO:tasks.workunit.client.0.smithi032.stdout:0/781: dwrite fa [0,4194304] 0 2024-05-12T04:01:32.972 INFO:tasks.workunit.client.1.smithi033.stdout:4/968: mkdir dd/d11/d5b/d84/dd7/d131 0 2024-05-12T04:01:32.976 INFO:tasks.workunit.client.1.smithi033.stdout:3/936: mkdir db/d17/d3a/d93/d104/d13d/d146 0 2024-05-12T04:01:32.982 INFO:tasks.workunit.client.0.smithi032.stdout:3/839: creat d15/d20/d26/d6a/d6e/f123 x:0 0 0 2024-05-12T04:01:32.983 INFO:tasks.workunit.client.1.smithi033.stdout:5/857: unlink d3/d12/d29/d2b/c2d 0 2024-05-12T04:01:32.984 INFO:tasks.workunit.client.0.smithi032.stdout:9/650: unlink d0/d3/d6/d7/da/l1f 0 2024-05-12T04:01:32.985 INFO:tasks.workunit.client.1.smithi033.stdout:3/937: dwrite db/d17/d3a/d108/f13a [0,4194304] 0 2024-05-12T04:01:32.986 INFO:tasks.workunit.client.1.smithi033.stdout:3/938: chown db/d59/f8d 0 1 2024-05-12T04:01:32.987 INFO:tasks.workunit.client.0.smithi032.stdout:4/691: dwrite d0/d3/d24/f42 [4194304,4194304] 0 2024-05-12T04:01:32.988 INFO:tasks.workunit.client.1.smithi033.stdout:3/939: read - db/d17/d3a/d67/d76/da0/de7/f136 zero size 2024-05-12T04:01:32.988 INFO:tasks.workunit.client.0.smithi032.stdout:1/749: rename d1/d6a/de6 to d1/d7/d30/d35/d54/d7e/df9/dfb 0 2024-05-12T04:01:32.989 INFO:tasks.workunit.client.1.smithi033.stdout:6/957: mkdir d4/d43/d140 0 2024-05-12T04:01:32.990 INFO:tasks.workunit.client.1.smithi033.stdout:3/940: chown db/d17/d3e/d65/d7d/f86 774191 1 2024-05-12T04:01:32.991 INFO:tasks.workunit.client.1.smithi033.stdout:3/941: read - db/d17/d3a/d67/d76/da0/de7/f11b zero size 2024-05-12T04:01:32.992 INFO:tasks.workunit.client.0.smithi032.stdout:0/782: mknod df/d30/c103 0 2024-05-12T04:01:32.992 INFO:tasks.workunit.client.0.smithi032.stdout:1/750: write d1/d25/fb8 [620906,32112] 0 2024-05-12T04:01:32.997 INFO:tasks.workunit.client.0.smithi032.stdout:7/707: getdents dd/d7b/d9a/d9c/dc2 0 2024-05-12T04:01:32.998 INFO:tasks.workunit.client.1.smithi033.stdout:5/858: rename d3/d94/d54/d57/d70/da3/dc9/d81 to d3/d94/d54/d57/d70/da3/dbe/d16/dd7/d11b 0 2024-05-12T04:01:32.998 INFO:tasks.workunit.client.0.smithi032.stdout:4/692: fdatasync d0/d3/d12/d1d/d63/d77/fa8 0 2024-05-12T04:01:33.001 INFO:tasks.workunit.client.0.smithi032.stdout:3/840: rename d15/d20/d8b/fac to d15/d20/d22/d4f/f124 0 2024-05-12T04:01:33.002 INFO:tasks.workunit.client.1.smithi033.stdout:3/942: mkdir db/d17/d3e/d65/d91/d97/d147 0 2024-05-12T04:01:33.003 INFO:tasks.workunit.client.1.smithi033.stdout:3/943: fdatasync db/d17/d3a/d67/d76/da0/fb1 0 2024-05-12T04:01:33.006 INFO:tasks.workunit.client.1.smithi033.stdout:4/969: creat dd/d11/d5b/f132 x:0 0 0 2024-05-12T04:01:33.007 INFO:tasks.workunit.client.1.smithi033.stdout:6/958: symlink d4/d1a/l141 0 2024-05-12T04:01:33.008 INFO:tasks.workunit.client.0.smithi032.stdout:1/751: truncate d1/dd/f53 1571121 0 2024-05-12T04:01:33.009 INFO:tasks.workunit.client.1.smithi033.stdout:4/970: truncate dd/d11/d5b/dd1/f110 290770 0 2024-05-12T04:01:33.009 INFO:tasks.workunit.client.1.smithi033.stdout:5/859: chown d3/d12/d25/c53 247 1 2024-05-12T04:01:33.010 INFO:tasks.workunit.client.1.smithi033.stdout:5/860: readlink d3/d12/d29/le5 0 2024-05-12T04:01:33.010 INFO:tasks.workunit.client.1.smithi033.stdout:4/971: truncate dd/d11/f11d 593622 0 2024-05-12T04:01:33.010 INFO:tasks.workunit.client.1.smithi033.stdout:4/972: read dd/f2b [3564088,41737] 0 2024-05-12T04:01:33.012 INFO:tasks.workunit.client.1.smithi033.stdout:2/984: link dc/d13/d1f/d6e/ce0 dc/d17/d27/c13c 0 2024-05-12T04:01:33.012 INFO:tasks.workunit.client.0.smithi032.stdout:0/783: rename df/d28/f59 to df/d30/d36/ddd/f104 0 2024-05-12T04:01:33.012 INFO:tasks.workunit.client.1.smithi033.stdout:5/861: dread - d3/d94/d54/d57/d70/da3/dbe/d16/d45/d62/d7f/dcf/dfd/ffe zero size 2024-05-12T04:01:33.013 INFO:tasks.workunit.client.1.smithi033.stdout:5/862: chown d3/d94/d85/ce6 568 1 2024-05-12T04:01:33.013 INFO:tasks.workunit.client.0.smithi032.stdout:4/693: creat d0/d3/d31/d80/d84/fcc x:0 0 0 2024-05-12T04:01:33.014 INFO:tasks.workunit.client.1.smithi033.stdout:5/863: chown d3/d12/d25/d26/f9c 473070 1 2024-05-12T04:01:33.014 INFO:tasks.workunit.client.0.smithi032.stdout:4/694: chown d0/d75 108718241 1 2024-05-12T04:01:33.017 INFO:tasks.workunit.client.0.smithi032.stdout:3/841: truncate d15/d1b/fd8 1289002 0 2024-05-12T04:01:33.019 INFO:tasks.workunit.client.1.smithi033.stdout:6/959: symlink d4/d1a/d67/d68/l142 0 2024-05-12T04:01:33.020 INFO:tasks.workunit.client.0.smithi032.stdout:1/752: stat d1/d6a/d72/cd2 0 2024-05-12T04:01:33.020 INFO:tasks.workunit.client.1.smithi033.stdout:6/960: truncate d4/d1a/d3c/dd9/de2/f10a 963650 0 2024-05-12T04:01:33.031 INFO:tasks.workunit.client.0.smithi032.stdout:9/651: getdents d0/d3/d6/d7/d2a 0 2024-05-12T04:01:33.032 INFO:tasks.workunit.client.1.smithi033.stdout:4/973: symlink dd/d7a/d87/d7e/da7/d38/deb/l133 0 2024-05-12T04:01:33.032 INFO:tasks.workunit.client.1.smithi033.stdout:2/985: fsync dc/d13/d65/f83 0 2024-05-12T04:01:33.033 INFO:tasks.workunit.client.1.smithi033.stdout:2/986: chown l2 1304229 1 2024-05-12T04:01:33.034 INFO:tasks.workunit.client.0.smithi032.stdout:4/695: mkdir d0/d3/d12/d1d/d6b/dc2/dcd 0 2024-05-12T04:01:33.037 INFO:tasks.workunit.client.1.smithi033.stdout:4/974: dwrite dd/d7a/f9f [4194304,4194304] 0 2024-05-12T04:01:33.039 INFO:tasks.workunit.client.1.smithi033.stdout:5/864: rmdir d3/d94/d54/d57/d70/da3/dbe/d16/d45/d62 39 2024-05-12T04:01:33.048 INFO:tasks.workunit.client.1.smithi033.stdout:6/961: rename d4/fd5 to d4/d1a/d67/d6f/d106/d129/f143 0 2024-05-12T04:01:33.048 INFO:tasks.workunit.client.1.smithi033.stdout:6/962: chown d4/d1a/d67/d6f/daf/ff7 65966675 1 2024-05-12T04:01:33.051 INFO:tasks.workunit.client.1.smithi033.stdout:8/928: sync 2024-05-12T04:01:33.052 INFO:tasks.workunit.client.0.smithi032.stdout:0/784: symlink df/d10/d48/l105 0 2024-05-12T04:01:33.060 INFO:tasks.workunit.client.0.smithi032.stdout:1/753: dread d1/d7/d30/d18/f76 [0,4194304] 0 2024-05-12T04:01:33.064 INFO:tasks.workunit.client.0.smithi032.stdout:5/867: sync 2024-05-12T04:01:33.064 INFO:tasks.workunit.client.0.smithi032.stdout:8/791: sync 2024-05-12T04:01:33.065 INFO:tasks.workunit.client.0.smithi032.stdout:8/792: fsync d0/d9/d1b/f101 0 2024-05-12T04:01:33.067 INFO:tasks.workunit.client.0.smithi032.stdout:0/785: rmdir df/d10/d3d/d4a/d84/dc0 39 2024-05-12T04:01:33.077 INFO:tasks.workunit.client.0.smithi032.stdout:1/754: creat d1/dd/d24/ffc x:0 0 0 2024-05-12T04:01:33.079 INFO:tasks.workunit.client.0.smithi032.stdout:1/755: fdatasync d1/d7/d30/f60 0 2024-05-12T04:01:33.081 INFO:tasks.workunit.client.0.smithi032.stdout:5/868: creat d4/d7/d14/d26/d33/d4a/d11d/f133 x:0 0 0 2024-05-12T04:01:33.083 INFO:tasks.workunit.client.1.smithi033.stdout:6/963: stat d4/d1a/d67/d6f/l118 0 2024-05-12T04:01:33.083 INFO:tasks.workunit.client.1.smithi033.stdout:3/944: dwrite db/d59/f8d [4194304,4194304] 0 2024-05-12T04:01:33.085 INFO:tasks.workunit.client.1.smithi033.stdout:3/945: write db/d17/d3a/dc0/fdf [764553,61481] 0 2024-05-12T04:01:33.085 INFO:tasks.workunit.client.1.smithi033.stdout:6/964: read d4/d1a/d67/d6f/daf/ff7 [337562,61554] 0 2024-05-12T04:01:33.087 INFO:tasks.workunit.client.1.smithi033.stdout:6/965: readlink d4/d1a/l141 0 2024-05-12T04:01:33.087 INFO:tasks.workunit.client.1.smithi033.stdout:8/929: fdatasync d1/d16/d27/d4d/d82/dd9/d36/d3c/fdc 0 2024-05-12T04:01:33.088 INFO:tasks.workunit.client.0.smithi032.stdout:0/786: stat df/df2/le9 0 2024-05-12T04:01:33.095 INFO:tasks.workunit.client.1.smithi033.stdout:3/946: dread db/d17/d25/f58 [0,4194304] 0 2024-05-12T04:01:33.097 INFO:tasks.workunit.client.0.smithi032.stdout:4/696: truncate d0/f3d 3998584 0 2024-05-12T04:01:33.108 INFO:tasks.workunit.client.0.smithi032.stdout:3/842: dwrite d15/f45 [0,4194304] 0 2024-05-12T04:01:33.109 INFO:tasks.workunit.client.0.smithi032.stdout:1/756: dread d1/d7/d30/d35/d77/faf [0,4194304] 0 2024-05-12T04:01:33.109 INFO:tasks.workunit.client.0.smithi032.stdout:9/652: write d0/d3/d74/f6c [741542,61421] 0 2024-05-12T04:01:33.111 INFO:tasks.workunit.client.0.smithi032.stdout:0/787: chown df/d10/c1c 295944668 1 2024-05-12T04:01:33.121 INFO:tasks.workunit.client.0.smithi032.stdout:9/653: dwrite d0/d3/d4b/d88/fcf [0,4194304] 0 2024-05-12T04:01:33.121 INFO:tasks.workunit.client.1.smithi033.stdout:3/947: creat db/d17/d25/d32/d3c/d3d/f148 x:0 0 0 2024-05-12T04:01:33.123 INFO:tasks.workunit.client.1.smithi033.stdout:3/948: truncate db/d17/d25/d32/d3c/d44/d117/f13f 560450 0 2024-05-12T04:01:33.125 INFO:tasks.workunit.client.1.smithi033.stdout:4/975: write dd/d7a/d87/d7e/f8f [975315,52511] 0 2024-05-12T04:01:33.125 INFO:tasks.workunit.client.1.smithi033.stdout:3/949: read - db/d17/d3e/dd7/f134 zero size 2024-05-12T04:01:33.127 INFO:tasks.workunit.client.1.smithi033.stdout:3/950: chown db/d17/d25/d32/d3c/la9 49 1 2024-05-12T04:01:33.175 INFO:tasks.workunit.client.0.smithi032.stdout:6/693: sync 2024-05-12T04:01:33.177 INFO:tasks.workunit.client.0.smithi032.stdout:3/843: rename d15/d33/f38 to d15/d20/d22/d7a/f125 0 2024-05-12T04:01:33.177 INFO:tasks.workunit.client.0.smithi032.stdout:6/694: dread - d7/d97/f9b zero size 2024-05-12T04:01:33.177 INFO:tasks.workunit.client.0.smithi032.stdout:8/793: write d0/d9/d1b/d27/d88/d56/d63/d8f/d9a/d28/f49 [274172,79033] 0 2024-05-12T04:01:33.178 INFO:tasks.workunit.client.1.smithi033.stdout:6/966: sync 2024-05-12T04:01:33.179 INFO:tasks.workunit.client.0.smithi032.stdout:3/844: readlink d15/d20/d26/d6a/d6e/d7d/leb 0 2024-05-12T04:01:33.189 INFO:tasks.workunit.client.1.smithi033.stdout:2/987: link dc/d17/d32/d84/dde/lfc dc/d13/d1f/l13d 0 2024-05-12T04:01:33.190 INFO:tasks.workunit.client.1.smithi033.stdout:5/865: dwrite d3/d94/d54/d57/d70/da3/dbe/d16/f42 [0,4194304] 0 2024-05-12T04:01:33.194 INFO:tasks.workunit.client.0.smithi032.stdout:5/869: creat d4/f134 x:0 0 0 2024-05-12T04:01:33.196 INFO:tasks.workunit.client.1.smithi033.stdout:3/951: creat db/d17/d3e/dd7/dd0/f149 x:0 0 0 2024-05-12T04:01:33.198 INFO:tasks.workunit.client.1.smithi033.stdout:3/952: stat db/d17/d3e/d65/d91/la2 0 2024-05-12T04:01:33.201 INFO:tasks.workunit.client.1.smithi033.stdout:8/930: creat d1/f116 x:0 0 0 2024-05-12T04:01:33.203 INFO:tasks.workunit.client.1.smithi033.stdout:1/864: link d11/d17/d27/d76/d93/fb0 d11/d17/d27/da6/dfe/f11c 0 2024-05-12T04:01:33.205 INFO:tasks.workunit.client.1.smithi033.stdout:8/931: write d1/d16/d27/d4d/d82/dd9/d36/d3c/da7/f43 [2819592,106910] 0 2024-05-12T04:01:33.205 INFO:tasks.workunit.client.1.smithi033.stdout:1/865: chown d11/d17/d27/d76/fa7 1886 1 2024-05-12T04:01:33.205 INFO:tasks.workunit.client.0.smithi032.stdout:8/794: rmdir d0/d9/d1b/d33/d73/db7 39 2024-05-12T04:01:33.208 INFO:tasks.workunit.client.1.smithi033.stdout:5/866: mknod d3/d94/d54/d98/c11c 0 2024-05-12T04:01:33.208 INFO:tasks.workunit.client.0.smithi032.stdout:0/788: dwrite df/d30/d4e/d55/fc4 [0,4194304] 0 2024-05-12T04:01:33.208 INFO:tasks.workunit.client.1.smithi033.stdout:4/976: write dd/d69/f97 [4740123,69891] 0 2024-05-12T04:01:33.209 INFO:tasks.workunit.client.1.smithi033.stdout:5/867: dread - d3/d12/d83/f10e zero size 2024-05-12T04:01:33.210 INFO:tasks.workunit.client.1.smithi033.stdout:1/866: dread d11/d17/d84/da3/fd6 [0,4194304] 0 2024-05-12T04:01:33.211 INFO:tasks.workunit.client.1.smithi033.stdout:6/967: mkdir d4/d144 0 2024-05-12T04:01:33.214 INFO:tasks.workunit.client.1.smithi033.stdout:8/932: mkdir d1/d4/db2/d117 0 2024-05-12T04:01:33.218 INFO:tasks.workunit.client.0.smithi032.stdout:9/654: mknod d0/d3/d6/d7/cd3 0 2024-05-12T04:01:33.218 INFO:tasks.workunit.client.1.smithi033.stdout:3/953: dread db/d17/d25/d32/d3c/d44/fbd [0,4194304] 0 2024-05-12T04:01:33.218 INFO:tasks.workunit.client.1.smithi033.stdout:5/868: chown d3/d12/d83/l118 8 1 2024-05-12T04:01:33.219 INFO:tasks.workunit.client.1.smithi033.stdout:8/933: read d1/d16/d27/d4d/d82/dd9/d36/fce [3661629,99377] 0 2024-05-12T04:01:33.219 INFO:tasks.workunit.client.1.smithi033.stdout:3/954: write db/d17/d3a/f129 [874249,88925] 0 2024-05-12T04:01:33.220 INFO:tasks.workunit.client.0.smithi032.stdout:5/870: creat d4/d7/d14/d26/d33/d72/d92/d111/f135 x:0 0 0 2024-05-12T04:01:33.221 INFO:tasks.workunit.client.1.smithi033.stdout:5/869: read d3/d94/d54/d57/d70/da3/dbe/f88 [77376,58228] 0 2024-05-12T04:01:33.226 INFO:tasks.workunit.client.1.smithi033.stdout:5/870: truncate d3/d94/d54/d57/d70/da3/dbe/d16/f44 4422379 0 2024-05-12T04:01:33.227 INFO:tasks.workunit.client.1.smithi033.stdout:8/934: dwrite d1/d16/d27/d4d/d82/dd9/f78 [0,4194304] 0 2024-05-12T04:01:33.228 INFO:tasks.workunit.client.0.smithi032.stdout:3/845: write d15/dc2/fc3 [151703,35122] 0 2024-05-12T04:01:33.229 INFO:tasks.workunit.client.1.smithi033.stdout:4/977: rename dd/d11/d5b/d84 to dd/d69/dc4/d117/d121/d134 0 2024-05-12T04:01:33.231 INFO:tasks.workunit.client.0.smithi032.stdout:8/795: unlink d0/d9/d1b/d66/d75/cf5 0 2024-05-12T04:01:33.234 INFO:tasks.workunit.client.1.smithi033.stdout:6/968: truncate d4/d1a/d67/d68/f8b 4765090 0 2024-05-12T04:01:33.235 INFO:tasks.workunit.client.0.smithi032.stdout:0/789: mkdir df/d10/d3d/d4a/d9a/d9d/d106 0 2024-05-12T04:01:33.236 INFO:tasks.workunit.client.1.smithi033.stdout:5/871: dwrite d3/d12/d25/d26/f31 [4194304,4194304] 0 2024-05-12T04:01:33.240 INFO:tasks.workunit.client.1.smithi033.stdout:3/955: creat db/d17/d3e/d65/dfb/f14a x:0 0 0 2024-05-12T04:01:33.241 INFO:tasks.workunit.client.1.smithi033.stdout:3/956: dread db/d17/d4d/fcd [0,4194304] 0 2024-05-12T04:01:33.243 INFO:tasks.workunit.client.0.smithi032.stdout:4/697: getdents d0/d3/d12/d1d/d63 0 2024-05-12T04:01:33.243 INFO:tasks.workunit.client.1.smithi033.stdout:3/957: truncate db/d17/d3a/f129 1979447 0 2024-05-12T04:01:33.244 INFO:tasks.workunit.client.1.smithi033.stdout:3/958: readlink db/d17/d3e/dd7/la5 0 2024-05-12T04:01:33.249 INFO:tasks.workunit.client.0.smithi032.stdout:1/757: truncate d1/dd/f53 727572 0 2024-05-12T04:01:33.250 INFO:tasks.workunit.client.1.smithi033.stdout:4/978: unlink dd/d7a/d87/d7e/da7/d38/d104/d109/f10f 0 2024-05-12T04:01:33.252 INFO:tasks.workunit.client.1.smithi033.stdout:3/959: dread db/d17/d3a/d67/ffa [4194304,4194304] 0 2024-05-12T04:01:33.257 INFO:tasks.workunit.client.1.smithi033.stdout:6/969: symlink d4/d6e/dbb/l145 0 2024-05-12T04:01:33.258 INFO:tasks.workunit.client.1.smithi033.stdout:5/872: fdatasync d3/d94/d54/d57/d70/da3/dbe/f88 0 2024-05-12T04:01:33.259 INFO:tasks.workunit.client.0.smithi032.stdout:8/796: dread d0/d9/f2d [0,4194304] 0 2024-05-12T04:01:33.260 INFO:tasks.workunit.client.1.smithi033.stdout:3/960: dwrite db/f81 [0,4194304] 0 2024-05-12T04:01:33.260 INFO:tasks.workunit.client.0.smithi032.stdout:0/790: unlink df/d28/la1 0 2024-05-12T04:01:33.260 INFO:tasks.workunit.client.1.smithi033.stdout:3/961: fsync db/d17/d4d/fde 0 2024-05-12T04:01:33.263 INFO:tasks.workunit.client.0.smithi032.stdout:4/698: write d0/d3/f1c [7836117,101232] 0 2024-05-12T04:01:33.266 INFO:tasks.workunit.client.0.smithi032.stdout:6/695: getdents d7/d9/d5c/d8b 0 2024-05-12T04:01:33.270 INFO:tasks.workunit.client.1.smithi033.stdout:1/867: dwrite d11/d17/d27/fc9 [0,4194304] 0 2024-05-12T04:01:33.270 INFO:tasks.workunit.client.0.smithi032.stdout:5/871: symlink d4/d7/d58/d5a/d99/l136 0 2024-05-12T04:01:33.276 INFO:tasks.workunit.client.0.smithi032.stdout:3/846: creat d15/d24/d2a/d8c/db1/dc8/de4/f126 x:0 0 0 2024-05-12T04:01:33.276 INFO:tasks.workunit.client.1.smithi033.stdout:2/988: sync 2024-05-12T04:01:33.278 INFO:tasks.workunit.client.1.smithi033.stdout:2/989: dread - dc/f119 zero size 2024-05-12T04:01:33.278 INFO:tasks.workunit.client.0.smithi032.stdout:8/797: dread d0/d9/d1b/d33/f3a [0,4194304] 0 2024-05-12T04:01:33.281 INFO:tasks.workunit.client.0.smithi032.stdout:9/655: dread d0/d3/d6/d7/d2a/d46/f12 [0,4194304] 0 2024-05-12T04:01:33.282 INFO:tasks.workunit.client.1.smithi033.stdout:6/970: dwrite d4/f82 [4194304,4194304] 0 2024-05-12T04:01:33.284 INFO:tasks.workunit.client.0.smithi032.stdout:9/656: write d0/d3/d6/d7/d2a/f68 [31034,122922] 0 2024-05-12T04:01:33.287 INFO:tasks.workunit.client.1.smithi033.stdout:8/935: rename d1/d16/d27/d4d/d73/f97 to d1/f118 0 2024-05-12T04:01:33.288 INFO:tasks.workunit.client.1.smithi033.stdout:3/962: chown db/d17/lc1 687376 1 2024-05-12T04:01:33.289 INFO:tasks.workunit.client.0.smithi032.stdout:5/872: mknod d4/d7/d14/d26/d33/d72/dc9/c137 0 2024-05-12T04:01:33.291 INFO:tasks.workunit.client.1.smithi033.stdout:1/868: creat d11/d17/d84/f11d x:0 0 0 2024-05-12T04:01:33.296 INFO:tasks.workunit.client.1.smithi033.stdout:4/979: write dd/d11/d5b/dd1/ff8 [2404856,79079] 0 2024-05-12T04:01:33.297 INFO:tasks.workunit.client.1.smithi033.stdout:4/980: fdatasync dd/d7a/d87/d7e/f8f 0 2024-05-12T04:01:33.297 INFO:tasks.workunit.client.0.smithi032.stdout:0/791: rename df/d30/d4d/l63 to df/d30/d4d/l107 0 2024-05-12T04:01:33.298 INFO:tasks.workunit.client.0.smithi032.stdout:1/758: write d1/f47 [264923,52930] 0 2024-05-12T04:01:33.302 INFO:tasks.workunit.client.0.smithi032.stdout:9/657: dread d0/d84/faf [4194304,4194304] 0 2024-05-12T04:01:33.304 INFO:tasks.workunit.client.1.smithi033.stdout:4/981: dwrite dd/d7a/d87/d7e/da7/d38/d104/d109/f126 [0,4194304] 0 2024-05-12T04:01:33.305 INFO:tasks.workunit.client.1.smithi033.stdout:6/971: creat d4/d1a/d67/d6f/dac/f146 x:0 0 0 2024-05-12T04:01:33.305 INFO:tasks.workunit.client.1.smithi033.stdout:4/982: readlink dd/d7a/d87/d7e/da7/ld2 0 2024-05-12T04:01:33.309 INFO:tasks.workunit.client.1.smithi033.stdout:3/963: creat db/d17/d25/d32/d3c/d3d/d111/f14b x:0 0 0 2024-05-12T04:01:33.311 INFO:tasks.workunit.client.0.smithi032.stdout:5/873: dread d4/d7/d14/d26/d33/d72/dc9/dec/f116 [0,4194304] 0 2024-05-12T04:01:33.312 INFO:tasks.workunit.client.1.smithi033.stdout:6/972: dwrite d4/fc8 [4194304,4194304] 0 2024-05-12T04:01:33.312 INFO:tasks.workunit.client.0.smithi032.stdout:6/696: truncate d7/d9/f82 4478146 0 2024-05-12T04:01:33.314 INFO:tasks.workunit.client.0.smithi032.stdout:5/874: dread d4/d7/d14/d26/d33/d72/dc9/fca [0,4194304] 0 2024-05-12T04:01:33.314 INFO:tasks.workunit.client.0.smithi032.stdout:6/697: stat d7/d9/d48/d67/d1a/d20/c88 0 2024-05-12T04:01:33.315 INFO:tasks.workunit.client.1.smithi033.stdout:5/873: creat d3/d12/f11d x:0 0 0 2024-05-12T04:01:33.318 INFO:tasks.workunit.client.0.smithi032.stdout:4/699: dwrite d0/d3/d12/d1d/d6b/f9a [0,4194304] 0 2024-05-12T04:01:33.320 INFO:tasks.workunit.client.0.smithi032.stdout:3/847: symlink d15/d24/d2a/dc7/l127 0 2024-05-12T04:01:33.322 INFO:tasks.workunit.client.1.smithi033.stdout:2/990: symlink dc/d17/d27/d68/d9f/l13e 0 2024-05-12T04:01:33.329 INFO:tasks.workunit.client.1.smithi033.stdout:2/991: dwrite dc/d17/dcc/fea [0,4194304] 0 2024-05-12T04:01:33.330 INFO:tasks.workunit.client.1.smithi033.stdout:2/992: dread dc/d13/d1f/f9b [0,4194304] 0 2024-05-12T04:01:33.331 INFO:tasks.workunit.client.0.smithi032.stdout:1/759: chown d1/d7/d49/f90 1223610328 1 2024-05-12T04:01:33.332 INFO:tasks.workunit.client.0.smithi032.stdout:3/848: fdatasync d15/d20/d8b/d9b/fa8 0 2024-05-12T04:01:33.334 INFO:tasks.workunit.client.0.smithi032.stdout:0/792: rename df/d30/d4e/da5/daf/fda to df/d10/d3d/d83/f108 0 2024-05-12T04:01:33.338 INFO:tasks.workunit.client.0.smithi032.stdout:3/849: write d15/d33/d3f/d75/fcb [4268929,76601] 0 2024-05-12T04:01:33.339 INFO:tasks.workunit.client.1.smithi033.stdout:4/983: chown dd/d11/d5b/c98 1 1 2024-05-12T04:01:33.341 INFO:tasks.workunit.client.0.smithi032.stdout:8/798: dwrite d0/d9/d1b/d27/d4c/f70 [0,4194304] 0 2024-05-12T04:01:33.342 INFO:tasks.workunit.client.0.smithi032.stdout:3/850: chown d15/d20/d22/d7a/fa6 118813639 1 2024-05-12T04:01:33.344 INFO:tasks.workunit.client.0.smithi032.stdout:0/793: dread df/d28/f54 [0,4194304] 0 2024-05-12T04:01:33.344 INFO:tasks.workunit.client.0.smithi032.stdout:3/851: stat d15/d20/d22/d7a/dbe/f101 0 2024-05-12T04:01:33.344 INFO:tasks.workunit.client.1.smithi033.stdout:3/964: unlink db/d17/d3a/f3b 0 2024-05-12T04:01:33.344 INFO:tasks.workunit.client.1.smithi033.stdout:1/869: fdatasync d11/d1c/d59/faa 0 2024-05-12T04:01:33.345 INFO:tasks.workunit.client.1.smithi033.stdout:3/965: write db/d17/d3a/d108/f13a [69982,104586] 0 2024-05-12T04:01:33.346 INFO:tasks.workunit.client.1.smithi033.stdout:3/966: dread - db/d17/d25/d32/d53/f12d zero size 2024-05-12T04:01:33.354 INFO:tasks.workunit.client.1.smithi033.stdout:8/936: creat d1/d16/d27/f119 x:0 0 0 2024-05-12T04:01:33.355 INFO:tasks.workunit.client.0.smithi032.stdout:2/766: sync 2024-05-12T04:01:33.357 INFO:tasks.workunit.client.0.smithi032.stdout:9/658: write d0/d3/d6/d7/da/f1a [10836589,24515] 0 2024-05-12T04:01:33.360 INFO:tasks.workunit.client.1.smithi033.stdout:3/967: dread db/d17/d3a/f63 [0,4194304] 0 2024-05-12T04:01:33.362 INFO:tasks.workunit.client.1.smithi033.stdout:6/973: write d4/d43/d57/fb8 [1316956,17653] 0 2024-05-12T04:01:33.364 INFO:tasks.workunit.client.1.smithi033.stdout:5/874: write d3/d12/f3f [1055078,38176] 0 2024-05-12T04:01:33.371 INFO:tasks.workunit.client.1.smithi033.stdout:8/937: dread d1/d16/d27/d4d/d82/dd9/d36/fce [0,4194304] 0 2024-05-12T04:01:33.376 INFO:tasks.workunit.client.1.smithi033.stdout:4/984: mknod dd/d7a/d87/d7e/da7/d66/c135 0 2024-05-12T04:01:33.380 INFO:tasks.workunit.client.1.smithi033.stdout:1/870: symlink d11/d17/d38/l11e 0 2024-05-12T04:01:33.383 INFO:tasks.workunit.client.1.smithi033.stdout:8/938: dread d1/f13 [0,4194304] 0 2024-05-12T04:01:33.383 INFO:tasks.workunit.client.1.smithi033.stdout:6/974: rename d4/f78 to d4/d8d/f147 0 2024-05-12T04:01:33.388 INFO:tasks.workunit.client.1.smithi033.stdout:3/968: write db/d17/d25/d32/d3c/d3d/f8b [171068,69929] 0 2024-05-12T04:01:33.391 INFO:tasks.workunit.client.1.smithi033.stdout:4/985: symlink dd/d69/da4/l136 0 2024-05-12T04:01:33.392 INFO:tasks.workunit.client.1.smithi033.stdout:4/986: read dd/d69/da4/f107 [2126489,96023] 0 2024-05-12T04:01:33.394 INFO:tasks.workunit.client.1.smithi033.stdout:1/871: symlink d11/d1c/db8/l11f 0 2024-05-12T04:01:33.396 INFO:tasks.workunit.client.1.smithi033.stdout:1/872: chown d11/d1c/d43/fa1 921502 1 2024-05-12T04:01:33.397 INFO:tasks.workunit.client.1.smithi033.stdout:5/875: dwrite d3/d94/d54/d57/d70/da3/dbe/d16/f20 [12582912,4194304] 0 2024-05-12T04:01:33.398 INFO:tasks.workunit.client.1.smithi033.stdout:1/873: fsync d11/d17/d84/de7/f101 0 2024-05-12T04:01:33.398 INFO:tasks.workunit.client.1.smithi033.stdout:8/939: unlink d1/d16/d27/l5f 0 2024-05-12T04:01:33.401 INFO:tasks.workunit.client.1.smithi033.stdout:3/969: mkdir db/d17/d25/d32/d3c/d3d/d47/d14c 0 2024-05-12T04:01:33.403 INFO:tasks.workunit.client.1.smithi033.stdout:1/874: dread d11/d17/d27/fc9 [0,4194304] 0 2024-05-12T04:01:33.404 INFO:tasks.workunit.client.1.smithi033.stdout:4/987: rmdir dd/d11/d75/dd4 39 2024-05-12T04:01:33.408 INFO:tasks.workunit.client.1.smithi033.stdout:2/993: rename dc/d13/dfb/dd7/leb to dc/d17/dcc/df1/d11b/l13f 0 2024-05-12T04:01:33.413 INFO:tasks.workunit.client.1.smithi033.stdout:8/940: mknod d1/d4/d46/dca/c11a 0 2024-05-12T04:01:33.416 INFO:tasks.workunit.client.1.smithi033.stdout:5/876: chown d3/d94/d54/d57/d70/da3/dbe/d16/d45/d62/fad 10053 1 2024-05-12T04:01:33.423 INFO:tasks.workunit.client.1.smithi033.stdout:4/988: creat dd/d7a/d87/d7e/da7/d38/d104/f137 x:0 0 0 2024-05-12T04:01:33.429 INFO:tasks.workunit.client.1.smithi033.stdout:6/975: link d4/d43/d57/c5d d4/d1a/d67/d6f/d106/d129/c148 0 2024-05-12T04:01:33.432 INFO:tasks.workunit.client.1.smithi033.stdout:8/941: creat d1/d16/d27/d4d/d51/d8e/db6/f11b x:0 0 0 2024-05-12T04:01:33.433 INFO:tasks.workunit.client.1.smithi033.stdout:8/942: chown d1/d4/db2/d117 792963 1 2024-05-12T04:01:33.434 INFO:tasks.workunit.client.1.smithi033.stdout:8/943: truncate d1/d16/d29/f10c 27255 0 2024-05-12T04:01:33.437 INFO:tasks.workunit.client.1.smithi033.stdout:4/989: rmdir dd/d69/da4 39 2024-05-12T04:01:33.437 INFO:tasks.workunit.client.1.smithi033.stdout:2/994: dread dc/d3b/d88/fc3 [0,4194304] 0 2024-05-12T04:01:33.440 INFO:tasks.workunit.client.1.smithi033.stdout:5/877: symlink d3/d94/d54/d57/d70/da3/dbe/d5a/l11e 0 2024-05-12T04:01:33.441 INFO:tasks.workunit.client.1.smithi033.stdout:5/878: fsync d3/d94/d54/d57/d70/da3/dbe/d16/f42 0 2024-05-12T04:01:33.444 INFO:tasks.workunit.client.1.smithi033.stdout:3/970: creat db/d17/f14d x:0 0 0 2024-05-12T04:01:33.446 INFO:tasks.workunit.client.1.smithi033.stdout:1/875: link d11/d1c/d59/f86 d11/d1c/ddf/d4c/f120 0 2024-05-12T04:01:33.447 INFO:tasks.workunit.client.1.smithi033.stdout:1/876: fsync d11/d1c/f54 0 2024-05-12T04:01:33.449 INFO:tasks.workunit.client.1.smithi033.stdout:2/995: creat dc/d13/d39/f140 x:0 0 0 2024-05-12T04:01:33.450 INFO:tasks.workunit.client.1.smithi033.stdout:6/976: mkdir d4/d144/d149 0 2024-05-12T04:01:33.450 INFO:tasks.workunit.client.1.smithi033.stdout:3/971: dwrite db/d17/d3a/fb5 [8388608,4194304] 0 2024-05-12T04:01:33.452 INFO:tasks.workunit.client.1.smithi033.stdout:3/972: write db/d59/f8d [7822504,114537] 0 2024-05-12T04:01:33.454 INFO:tasks.workunit.client.1.smithi033.stdout:6/977: truncate d4/d43/d57/dc4/f12a 720962 0 2024-05-12T04:01:33.454 INFO:tasks.workunit.client.1.smithi033.stdout:8/944: mkdir d1/d11c 0 2024-05-12T04:01:33.456 INFO:tasks.workunit.client.1.smithi033.stdout:5/879: creat d3/d94/d54/d57/d70/da3/dbe/d16/d45/d62/d7f/d84/f11f x:0 0 0 2024-05-12T04:01:33.463 INFO:tasks.workunit.client.1.smithi033.stdout:4/990: mknod dd/d69/dff/c138 0 2024-05-12T04:01:33.464 INFO:tasks.workunit.client.1.smithi033.stdout:5/880: dwrite d3/d94/d54/d57/d70/da3/dbe/d16/d45/f68 [4194304,4194304] 0 2024-05-12T04:01:33.465 INFO:tasks.workunit.client.1.smithi033.stdout:1/877: readlink d11/l1a 0 2024-05-12T04:01:33.465 INFO:tasks.workunit.client.1.smithi033.stdout:5/881: write d3/d12/d25/db0/ff7 [699352,71178] 0 2024-05-12T04:01:33.471 INFO:tasks.workunit.client.0.smithi032.stdout:4/700: truncate d0/d3/d12/d1d/d63/f99 1048363 0 2024-05-12T04:01:33.472 INFO:tasks.workunit.client.1.smithi033.stdout:3/973: mknod db/d17/d25/d32/d53/df6/c14e 0 2024-05-12T04:01:33.473 INFO:tasks.workunit.client.1.smithi033.stdout:2/996: dread dc/d17/d27/d2f/f90 [0,4194304] 0 2024-05-12T04:01:33.474 INFO:tasks.workunit.client.1.smithi033.stdout:5/882: dwrite d3/d12/d83/f10e [0,4194304] 0 2024-05-12T04:01:33.474 INFO:tasks.workunit.client.1.smithi033.stdout:6/978: unlink d4/d43/d57/dc4/ddf/lbe 0 2024-05-12T04:01:33.477 INFO:tasks.workunit.client.1.smithi033.stdout:6/979: read - d4/d1a/d3c/dd9/f12f zero size 2024-05-12T04:01:33.478 INFO:tasks.workunit.client.1.smithi033.stdout:6/980: write d4/d43/d57/dc4/ddf/f13a [1006408,54416] 0 2024-05-12T04:01:33.481 INFO:tasks.workunit.client.0.smithi032.stdout:0/794: dread - df/d30/d32/fcf zero size 2024-05-12T04:01:33.482 INFO:tasks.workunit.client.1.smithi033.stdout:1/878: truncate d11/d17/d84/da3/fd6 289586 0 2024-05-12T04:01:33.483 INFO:tasks.workunit.client.0.smithi032.stdout:8/799: truncate d0/d9/d1b/d27/d88/f40 4996983 0 2024-05-12T04:01:33.490 INFO:tasks.workunit.client.1.smithi033.stdout:5/883: truncate d3/d94/f9e 115832 0 2024-05-12T04:01:33.491 INFO:tasks.workunit.client.0.smithi032.stdout:2/767: creat d1/d4d/d5e/d9a/ffd x:0 0 0 2024-05-12T04:01:33.493 INFO:tasks.workunit.client.1.smithi033.stdout:8/945: rename d1/d4/d46/f4f to d1/d16/d27/d4d/d51/d8e/d94/f11d 0 2024-05-12T04:01:33.500 INFO:tasks.workunit.client.0.smithi032.stdout:4/701: creat d0/d3/d12/d1d/d6b/dc2/fce x:0 0 0 2024-05-12T04:01:33.501 INFO:tasks.workunit.client.1.smithi033.stdout:5/884: dread d3/d94/d54/d98/f9f [0,4194304] 0 2024-05-12T04:01:33.502 INFO:tasks.workunit.client.1.smithi033.stdout:5/885: stat d3/d94/d54/d98 0 2024-05-12T04:01:33.502 INFO:tasks.workunit.client.1.smithi033.stdout:3/974: write db/f54 [4827328,27449] 0 2024-05-12T04:01:33.504 INFO:tasks.workunit.client.1.smithi033.stdout:2/997: mknod dc/d17/d27/d68/d9f/c141 0 2024-05-12T04:01:33.505 INFO:tasks.workunit.client.0.smithi032.stdout:8/800: symlink d0/d9/d13/d21/d35/l10d 0 2024-05-12T04:01:33.510 INFO:tasks.workunit.client.0.smithi032.stdout:5/875: dwrite d4/d7/d14/d26/d33/d72/d92/f109 [0,4194304] 0 2024-05-12T04:01:33.516 INFO:tasks.workunit.client.0.smithi032.stdout:2/768: unlink d1/d4d/d5e/fb9 0 2024-05-12T04:01:33.516 INFO:tasks.workunit.client.0.smithi032.stdout:3/852: write d15/d33/f9e [1752194,14118] 0 2024-05-12T04:01:33.518 INFO:tasks.workunit.client.1.smithi033.stdout:3/975: dwrite db/d17/d3e/dd7/dd0/f149 [0,4194304] 0 2024-05-12T04:01:33.519 INFO:tasks.workunit.client.1.smithi033.stdout:5/886: rename d3/d12/d25/d26/f9c to d3/d94/db8/f120 0 2024-05-12T04:01:33.521 INFO:tasks.workunit.client.1.smithi033.stdout:5/887: stat d3/d94/d54/f111 0 2024-05-12T04:01:33.522 INFO:tasks.workunit.client.0.smithi032.stdout:1/760: creat d1/d7/ffd x:0 0 0 2024-05-12T04:01:33.525 INFO:tasks.workunit.client.1.smithi033.stdout:1/879: dwrite d11/d17/fd7 [0,4194304] 0 2024-05-12T04:01:33.525 INFO:tasks.workunit.client.1.smithi033.stdout:6/981: write d4/d5/f41 [1857199,39334] 0 2024-05-12T04:01:33.531 INFO:tasks.workunit.client.0.smithi032.stdout:6/698: getdents d7/d9/d48/d67/d1a/d20 0 2024-05-12T04:01:33.541 INFO:tasks.workunit.client.0.smithi032.stdout:3/853: rename d15/d33/l36 to d15/d20/d26/d6a/d6e/d7d/d9a/l128 0 2024-05-12T04:01:33.549 INFO:tasks.workunit.client.0.smithi032.stdout:6/699: unlink d7/cd 0 2024-05-12T04:01:33.558 INFO:tasks.workunit.client.0.smithi032.stdout:9/659: getdents d0/d3/d6/d7/d2a/d9f 0 2024-05-12T04:01:33.559 INFO:tasks.workunit.client.0.smithi032.stdout:2/769: write d1/d34/d96/d41/d2c/f56 [698050,65443] 0 2024-05-12T04:01:33.565 INFO:tasks.workunit.client.0.smithi032.stdout:8/801: rmdir d0/d109 0 2024-05-12T04:01:33.568 INFO:tasks.workunit.client.0.smithi032.stdout:6/700: read d7/d9/d48/f5d [3105188,71245] 0 2024-05-12T04:01:33.573 INFO:tasks.workunit.client.0.smithi032.stdout:9/660: symlink d0/d3/d4b/ld4 0 2024-05-12T04:01:33.575 INFO:tasks.workunit.client.0.smithi032.stdout:9/661: fdatasync d0/d3/d6/d7/f15 0 2024-05-12T04:01:33.575 INFO:tasks.workunit.client.0.smithi032.stdout:4/702: link d0/d3/df/f86 d0/d3/d12/d1d/dc5/fcf 0 2024-05-12T04:01:33.575 INFO:tasks.workunit.client.0.smithi032.stdout:7/708: sync 2024-05-12T04:01:33.578 INFO:tasks.workunit.client.0.smithi032.stdout:3/854: dwrite d15/d24/d2a/d5b/f71 [0,4194304] 0 2024-05-12T04:01:33.581 INFO:tasks.workunit.client.0.smithi032.stdout:1/761: rename d1/d7/d30/d18/d93/l9a to d1/d7/d30/lfe 0 2024-05-12T04:01:33.583 INFO:tasks.workunit.client.0.smithi032.stdout:6/701: truncate d7/d9/d48/d67/d8d/dbc/de2/f5a 1057932 0 2024-05-12T04:01:33.590 INFO:tasks.workunit.client.0.smithi032.stdout:8/802: mkdir d0/d9/d13/d21/d25/d10e 0 2024-05-12T04:01:33.592 INFO:tasks.workunit.client.0.smithi032.stdout:7/709: creat dd/d1b/d1e/d45/d55/d72/fe0 x:0 0 0 2024-05-12T04:01:33.593 INFO:tasks.workunit.client.0.smithi032.stdout:4/703: mknod d0/d3/d12/d1d/d63/d87/d95/cd0 0 2024-05-12T04:01:33.595 INFO:tasks.workunit.client.0.smithi032.stdout:3/855: truncate d15/d20/d26/d6a/d6e/d7d/d9a/f9c 1018996 0 2024-05-12T04:01:33.596 INFO:tasks.workunit.client.0.smithi032.stdout:3/856: write d15/dc2/fc3 [1149835,63427] 0 2024-05-12T04:01:33.597 INFO:tasks.workunit.client.1.smithi033.stdout:4/991: getdents dd/d11 0 2024-05-12T04:01:33.597 INFO:tasks.workunit.client.0.smithi032.stdout:2/770: creat d1/d4d/ffe x:0 0 0 2024-05-12T04:01:33.598 INFO:tasks.workunit.client.1.smithi033.stdout:4/992: fdatasync dd/d7a/d87/d7e/da7/d38/f130 0 2024-05-12T04:01:33.599 INFO:tasks.workunit.client.1.smithi033.stdout:4/993: fdatasync dd/d7a/d87/d7e/da7/d38/f130 0 2024-05-12T04:01:33.599 INFO:tasks.workunit.client.0.smithi032.stdout:6/702: creat d7/d9/d48/d67/d63/d64/d94/ff1 x:0 0 0 2024-05-12T04:01:33.601 INFO:tasks.workunit.client.0.smithi032.stdout:9/662: mkdir d0/d3/d74/dd5 0 2024-05-12T04:01:33.601 INFO:tasks.workunit.client.0.smithi032.stdout:9/663: stat d0/d3/d4b/d85/d87 0 2024-05-12T04:01:33.602 INFO:tasks.workunit.client.0.smithi032.stdout:5/876: link d4/d7/l43 d4/d7/d14/d26/l138 0 2024-05-12T04:01:33.602 INFO:tasks.workunit.client.0.smithi032.stdout:5/877: chown d4/d7/d14 219533 1 2024-05-12T04:01:33.605 INFO:tasks.workunit.client.0.smithi032.stdout:8/803: mknod d0/d9/d1b/d66/d75/c10f 0 2024-05-12T04:01:33.608 INFO:tasks.workunit.client.1.smithi033.stdout:3/976: creat db/d17/d3e/d65/dfb/f14f x:0 0 0 2024-05-12T04:01:33.610 INFO:tasks.workunit.client.1.smithi033.stdout:6/982: mknod d4/d1a/d67/d6f/daf/d116/c14a 0 2024-05-12T04:01:33.611 INFO:tasks.workunit.client.1.smithi033.stdout:6/983: write d4/d1a/d6c/fa4 [25126,4021] 0 2024-05-12T04:01:33.612 INFO:tasks.workunit.client.0.smithi032.stdout:7/710: dwrite dd/d1b/fa9 [0,4194304] 0 2024-05-12T04:01:33.612 INFO:tasks.workunit.client.0.smithi032.stdout:4/704: mknod d0/d3/d12/d1d/d63/d87/d95/cd1 0 2024-05-12T04:01:33.616 INFO:tasks.workunit.client.1.smithi033.stdout:5/888: unlink d3/d12/d29/d2b/d4f/f9d 0 2024-05-12T04:01:33.620 INFO:tasks.workunit.client.1.smithi033.stdout:8/946: sync 2024-05-12T04:01:33.621 INFO:tasks.workunit.client.0.smithi032.stdout:4/705: dwrite d0/d3/d12/d1d/fba [0,4194304] 0 2024-05-12T04:01:33.625 INFO:tasks.workunit.client.0.smithi032.stdout:6/703: creat d7/d9/d48/d67/d1a/ff2 x:0 0 0 2024-05-12T04:01:33.627 INFO:tasks.workunit.client.1.smithi033.stdout:3/977: mkdir db/d17/d25/d32/d53/d150 0 2024-05-12T04:01:33.629 INFO:tasks.workunit.client.1.smithi033.stdout:3/978: dread - db/d17/d3e/dd7/f134 zero size 2024-05-12T04:01:33.630 INFO:tasks.workunit.client.0.smithi032.stdout:5/878: mknod d4/d7/d14/d26/d33/d88/c139 0 2024-05-12T04:01:33.633 INFO:tasks.workunit.client.0.smithi032.stdout:5/879: write d4/f59 [231173,61449] 0 2024-05-12T04:01:33.633 INFO:tasks.workunit.client.1.smithi033.stdout:1/880: dwrite d11/d17/d27/d36/d3f/ff1 [0,4194304] 0 2024-05-12T04:01:33.633 INFO:tasks.workunit.client.1.smithi033.stdout:6/984: chown d4/d6e/fb3 903515590 1 2024-05-12T04:01:33.634 INFO:tasks.workunit.client.1.smithi033.stdout:6/985: dread - d4/d43/d57/dc4/f120 zero size 2024-05-12T04:01:33.634 INFO:tasks.workunit.client.1.smithi033.stdout:1/881: write d11/d17/d27/d76/d93/fc1 [531988,45480] 0 2024-05-12T04:01:33.635 INFO:tasks.workunit.client.1.smithi033.stdout:6/986: fsync d4/d1a/f10b 0 2024-05-12T04:01:33.637 INFO:tasks.workunit.client.0.smithi032.stdout:4/706: dread d0/d3/d31/d80/f82 [0,4194304] 0 2024-05-12T04:01:33.642 INFO:tasks.workunit.client.1.smithi033.stdout:6/987: dwrite d4/d1a/d3c/dd9/f12f [0,4194304] 0 2024-05-12T04:01:33.650 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:33 smithi032 ceph-mon[40008]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:01:33.650 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:33 smithi032 ceph-mon[40008]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:01:33.650 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:33 smithi032 ceph-mon[40008]: Reconfiguring prometheus.smithi032 (dependencies changed)... 2024-05-12T04:01:33.650 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:33 smithi032 ceph-mon[40008]: Reconfiguring daemon prometheus.smithi032 on smithi032 2024-05-12T04:01:33.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:33 smithi033 ceph-mon[49525]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:01:33.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:33 smithi033 ceph-mon[49525]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:01:33.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:33 smithi033 ceph-mon[49525]: Reconfiguring prometheus.smithi032 (dependencies changed)... 2024-05-12T04:01:33.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:33 smithi033 ceph-mon[49525]: Reconfiguring daemon prometheus.smithi032 on smithi032 2024-05-12T04:01:33.693 INFO:tasks.workunit.client.0.smithi032.stdout:5/880: dwrite d4/d7/d14/f10c [0,4194304] 0 2024-05-12T04:01:33.709 INFO:tasks.workunit.client.0.smithi032.stdout:2/771: dread d1/d34/da5/db3/db7/fc5 [0,4194304] 0 2024-05-12T04:01:33.836 INFO:tasks.workunit.client.0.smithi032.stdout:1/762: creat d1/d7/d49/fff x:0 0 0 2024-05-12T04:01:33.850 INFO:tasks.workunit.client.0.smithi032.stdout:3/857: link d15/d24/d2a/d5b/fc9 d15/d24/d2a/d8c/db1/dc8/de4/f129 0 2024-05-12T04:01:33.852 INFO:tasks.workunit.client.1.smithi033.stdout:3/979: rmdir db/d17/d3e/d65/d7d/d137 39 2024-05-12T04:01:33.867 INFO:tasks.workunit.client.0.smithi032.stdout:3/858: dread d15/d20/d22/d7a/f125 [0,4194304] 0 2024-05-12T04:01:33.867 INFO:tasks.workunit.client.1.smithi033.stdout:6/988: dread d4/d1a/d67/dc3/fd2 [0,4194304] 0 2024-05-12T04:01:33.867 INFO:tasks.workunit.client.1.smithi033.stdout:6/989: readlink d4/dad/l98 0 2024-05-12T04:01:33.870 INFO:tasks.workunit.client.1.smithi033.stdout:5/889: creat d3/d94/d54/d57/d70/da3/dbe/d16/d115/f121 x:0 0 0 2024-05-12T04:01:33.873 INFO:tasks.workunit.client.1.smithi033.stdout:2/998: getdents dc/d17/d27 0 2024-05-12T04:01:33.880 INFO:tasks.workunit.client.1.smithi033.stdout:4/994: write dd/d11/f1d [3589503,95796] 0 2024-05-12T04:01:33.881 INFO:tasks.workunit.client.1.smithi033.stdout:4/995: fsync dd/d7a/d87/d7e/da7/d38/f44 0 2024-05-12T04:01:33.883 INFO:tasks.workunit.client.0.smithi032.stdout:2/772: mkdir d1/d4d/d5e/dff 0 2024-05-12T04:01:33.885 INFO:tasks.workunit.client.0.smithi032.stdout:2/773: write d1/d34/d96/d41/d2c/f90 [332320,80454] 0 2024-05-12T04:01:33.892 INFO:tasks.workunit.client.0.smithi032.stdout:1/763: dread - d1/d6a/fb9 zero size 2024-05-12T04:01:33.893 INFO:tasks.workunit.client.0.smithi032.stdout:5/881: dread d4/d7/d14/d26/d33/d4a/f86 [4194304,4194304] 0 2024-05-12T04:01:33.894 INFO:tasks.workunit.client.1.smithi033.stdout:3/980: dread db/d17/d25/d32/d53/df6/f125 [0,4194304] 0 2024-05-12T04:01:33.896 INFO:tasks.workunit.client.1.smithi033.stdout:3/981: write db/d17/d3a/d67/d76/da0/de7/f11b [1021718,54590] 0 2024-05-12T04:01:33.896 INFO:tasks.workunit.client.1.smithi033.stdout:3/982: stat db/d17/d25/l7f 0 2024-05-12T04:01:33.897 INFO:tasks.workunit.client.1.smithi033.stdout:3/983: dread - db/f11c zero size 2024-05-12T04:01:33.915 INFO:tasks.workunit.client.1.smithi033.stdout:6/990: getdents d4/d43/d140 0 2024-05-12T04:01:33.930 INFO:tasks.workunit.client.0.smithi032.stdout:1/764: unlink d1/d7/d49/d79/f8d 0 2024-05-12T04:01:33.931 INFO:tasks.workunit.client.0.smithi032.stdout:1/765: write d1/d7/d30/d35/d54/f8c [8636632,35175] 0 2024-05-12T04:01:33.938 INFO:tasks.workunit.client.1.smithi033.stdout:8/947: getdents d1/d16/d27/d4d/d82/dd9/d36/d3c/d47 0 2024-05-12T04:01:33.945 INFO:tasks.workunit.client.0.smithi032.stdout:0/795: sync 2024-05-12T04:01:33.946 INFO:tasks.workunit.client.0.smithi032.stdout:0/796: dread - df/d10/d48/d70/ff3 zero size 2024-05-12T04:01:33.950 INFO:tasks.workunit.client.0.smithi032.stdout:2/774: fdatasync d1/d4d/d5e/fdc 0 2024-05-12T04:01:33.950 INFO:tasks.workunit.client.1.smithi033.stdout:2/999: link dc/d17/d27/d2f/f41 dc/d13/dfb/dd7/f142 0 2024-05-12T04:01:33.956 INFO:tasks.workunit.client.0.smithi032.stdout:5/882: link d4/d7/d58/f102 d4/d7/d14/d26/d33/d4a/d11d/d12c/f13a 0 2024-05-12T04:01:33.959 INFO:tasks.workunit.client.0.smithi032.stdout:5/883: truncate d4/d7/d14/d26/d33/d72/d92/f109 4601779 0 2024-05-12T04:01:33.960 INFO:tasks.workunit.client.0.smithi032.stdout:5/884: write d4/d7/d14/d26/f107 [799927,39010] 0 2024-05-12T04:01:33.963 INFO:tasks.workunit.client.0.smithi032.stdout:0/797: creat df/d10/d3b/dba/f109 x:0 0 0 2024-05-12T04:01:33.967 INFO:tasks.workunit.client.0.smithi032.stdout:2/775: mknod d1/d34/d96/d41/d2c/c100 0 2024-05-12T04:01:33.968 INFO:tasks.workunit.client.0.smithi032.stdout:3/859: getdents d15/d20/d26/d6a 0 2024-05-12T04:01:33.970 INFO:tasks.workunit.client.0.smithi032.stdout:5/885: mkdir d4/d7/d14/da9/d13b 0 2024-05-12T04:01:34.010 INFO:tasks.workunit.client.0.smithi032.stdout:3/860: unlink d15/d33/d3f/ff5 0 2024-05-12T04:01:34.014 INFO:tasks.workunit.client.0.smithi032.stdout:3/861: mknod d15/d20/d22/d117/c12a 0 2024-05-12T04:01:34.016 INFO:tasks.workunit.client.0.smithi032.stdout:3/862: write d15/d33/f46 [3766368,32739] 0 2024-05-12T04:01:34.017 INFO:tasks.workunit.client.0.smithi032.stdout:3/863: chown d15/d20/d26/d6a/d6e/d7d 507486 1 2024-05-12T04:01:34.037 INFO:tasks.workunit.client.1.smithi033.stdout:6/991: mkdir d4/d1a/d67/d14b 0 2024-05-12T04:01:34.039 INFO:tasks.workunit.client.1.smithi033.stdout:6/992: mknod d4/d1a/d3c/dd9/de2/def/c14c 0 2024-05-12T04:01:34.042 INFO:tasks.workunit.client.0.smithi032.stdout:6/704: mknod d7/d9/d5c/cf3 0 2024-05-12T04:01:34.044 INFO:tasks.workunit.client.1.smithi033.stdout:6/993: symlink d4/d6e/l14d 0 2024-05-12T04:01:34.044 INFO:tasks.workunit.client.1.smithi033.stdout:6/994: chown d4/d1a/d4e/d4f/d94/cde 27 1 2024-05-12T04:01:34.051 INFO:tasks.workunit.client.0.smithi032.stdout:6/705: getdents d7/d9/d48/d67/d25 0 2024-05-12T04:01:34.058 INFO:tasks.workunit.client.1.smithi033.stdout:6/995: dread d4/d1a/d4e/f66 [0,4194304] 0 2024-05-12T04:01:34.060 INFO:tasks.workunit.client.0.smithi032.stdout:1/766: creat d1/d7/d30/d61/f100 x:0 0 0 2024-05-12T04:01:34.062 INFO:tasks.workunit.client.1.smithi033.stdout:5/890: rmdir d3/d12/d25/d10b 0 2024-05-12T04:01:34.064 INFO:tasks.workunit.client.1.smithi033.stdout:6/996: dwrite d4/dad/d110/f12b [0,4194304] 0 2024-05-12T04:01:34.070 INFO:tasks.workunit.client.0.smithi032.stdout:1/767: unlink d1/d7/c32 0 2024-05-12T04:01:34.070 INFO:tasks.workunit.client.0.smithi032.stdout:1/768: fsync d1/d7/ffd 0 2024-05-12T04:01:34.083 INFO:tasks.workunit.client.1.smithi033.stdout:6/997: rename d4/d1a/d67/ldb to d4/dad/l14e 0 2024-05-12T04:01:34.084 INFO:tasks.workunit.client.0.smithi032.stdout:8/804: rename d0/d9/d13/d21/d25/d10e to d0/d9/d1b/d33/d73/db7/df1/d110 0 2024-05-12T04:01:34.087 INFO:tasks.workunit.client.1.smithi033.stdout:6/998: rename d4/d1a/d67/d68/ca9 to d4/d144/c14f 0 2024-05-12T04:01:34.088 INFO:tasks.workunit.client.1.smithi033.stdout:6/999: chown d4/d1a/d4e/d4f/c5b 90 1 2024-05-12T04:01:34.092 INFO:tasks.workunit.client.0.smithi032.stdout:0/798: rename df/f2b to df/d30/d32/d81/de8/f10a 0 2024-05-12T04:01:34.097 INFO:tasks.workunit.client.0.smithi032.stdout:6/706: rename d7/d9/d5c/d8b/ce3 to d7/d9/d48/d67/d8d/dbc/de2/dd4/cf4 0 2024-05-12T04:01:34.101 INFO:tasks.workunit.client.0.smithi032.stdout:0/799: rename df/fc3 to df/d10/d3b/dba/f10b 0 2024-05-12T04:01:34.104 INFO:tasks.workunit.client.0.smithi032.stdout:0/800: write df/d30/d8d/d8f/f97 [127165,12733] 0 2024-05-12T04:01:34.106 INFO:tasks.workunit.client.0.smithi032.stdout:0/801: read - df/d30/d32/fcf zero size 2024-05-12T04:01:34.107 INFO:tasks.workunit.client.1.smithi033.stdout:1/882: dwrite d11/d17/d27/d36/d3f/d4a/dc6/de1/d114/d87/da4/fd9 [0,4194304] 0 2024-05-12T04:01:34.110 INFO:tasks.workunit.client.0.smithi032.stdout:9/664: dwrite d0/d3/d1b/fd0 [0,4194304] 0 2024-05-12T04:01:34.116 INFO:tasks.workunit.client.0.smithi032.stdout:9/665: truncate d0/d3/d4b/d85/d87/f98 920242 0 2024-05-12T04:01:34.121 INFO:tasks.workunit.client.0.smithi032.stdout:9/666: symlink d0/d3/d4b/d85/dbe/ld6 0 2024-05-12T04:01:34.122 INFO:tasks.workunit.client.1.smithi033.stdout:1/883: link d11/d17/d27/d76/c95 d11/d17/c121 0 2024-05-12T04:01:34.123 INFO:tasks.workunit.client.1.smithi033.stdout:1/884: readlink d11/d17/d27/d36/d3f/d4a/dc6/de1/d114/l8b 0 2024-05-12T04:01:34.123 INFO:tasks.workunit.client.0.smithi032.stdout:6/707: dread d7/d9/d48/d67/d8d/dbc/f9a [0,4194304] 0 2024-05-12T04:01:34.125 INFO:tasks.workunit.client.0.smithi032.stdout:9/667: unlink d0/d3/d4b/d85/dbe/l90 0 2024-05-12T04:01:34.126 INFO:tasks.workunit.client.0.smithi032.stdout:6/708: unlink d7/d33/f7a 0 2024-05-12T04:01:34.128 INFO:tasks.workunit.client.0.smithi032.stdout:6/709: chown d7/d9/d48/d67/f2c 0 1 2024-05-12T04:01:34.131 INFO:tasks.workunit.client.0.smithi032.stdout:6/710: dread - d7/d9/d48/d67/d63/d64/d94/ff1 zero size 2024-05-12T04:01:34.136 INFO:tasks.workunit.client.0.smithi032.stdout:6/711: mkdir d7/d9/d48/d67/d1a/d93/df5 0 2024-05-12T04:01:34.140 INFO:tasks.workunit.client.0.smithi032.stdout:4/707: dwrite d0/d3/d31/d80/d84/f64 [0,4194304] 0 2024-05-12T04:01:34.140 INFO:tasks.workunit.client.0.smithi032.stdout:6/712: symlink d7/d9/d48/d67/d1a/d93/lf6 0 2024-05-12T04:01:34.142 INFO:tasks.workunit.client.0.smithi032.stdout:6/713: readlink d7/d9/d48/d67/d1a/d20/l43 0 2024-05-12T04:01:34.143 INFO:tasks.workunit.client.0.smithi032.stdout:4/708: truncate d0/d3/d31/d80/d84/fcc 722770 0 2024-05-12T04:01:34.147 INFO:tasks.workunit.client.0.smithi032.stdout:6/714: creat d7/d9/d48/d67/d1a/d93/ff7 x:0 0 0 2024-05-12T04:01:34.152 INFO:tasks.workunit.client.0.smithi032.stdout:6/715: creat d7/d9/d48/d67/d25/ff8 x:0 0 0 2024-05-12T04:01:34.157 INFO:tasks.workunit.client.0.smithi032.stdout:6/716: creat d7/d9/d48/d67/d8d/dbc/de2/ff9 x:0 0 0 2024-05-12T04:01:34.162 INFO:tasks.workunit.client.1.smithi033.stdout:4/996: dwrite dd/d69/f7d [0,4194304] 0 2024-05-12T04:01:34.163 INFO:tasks.workunit.client.1.smithi033.stdout:4/997: fsync dd/d7a/d87/d7e/da7/d66/fbf 0 2024-05-12T04:01:34.163 INFO:tasks.workunit.client.1.smithi033.stdout:8/948: symlink d1/d16/d27/d4d/d82/dd9/l11e 0 2024-05-12T04:01:34.166 INFO:tasks.workunit.client.1.smithi033.stdout:4/998: creat dd/d69/dc4/f139 x:0 0 0 2024-05-12T04:01:34.167 INFO:tasks.workunit.client.1.smithi033.stdout:8/949: stat d1/d16/d27/f86 0 2024-05-12T04:01:34.170 INFO:tasks.workunit.client.1.smithi033.stdout:4/999: mknod dd/d7a/d87/d7e/da7/d66/d42/d59/c13a 0 2024-05-12T04:01:34.181 INFO:tasks.workunit.client.0.smithi032.stdout:2/776: dwrite d1/d4d/f98 [0,4194304] 0 2024-05-12T04:01:34.181 INFO:tasks.workunit.client.0.smithi032.stdout:5/886: dwrite d4/d7/d14/d108/f6f [0,4194304] 0 2024-05-12T04:01:34.187 INFO:tasks.workunit.client.0.smithi032.stdout:7/711: creat dd/d7b/d9a/d9c/fe1 x:0 0 0 2024-05-12T04:01:34.189 INFO:tasks.workunit.client.0.smithi032.stdout:2/777: creat d1/d34/d4a/f101 x:0 0 0 2024-05-12T04:01:34.190 INFO:tasks.workunit.client.0.smithi032.stdout:3/864: creat d15/d20/d26/d6a/f12b x:0 0 0 2024-05-12T04:01:34.191 INFO:tasks.workunit.client.0.smithi032.stdout:7/712: truncate dd/d1b/d1e/f68 516025 0 2024-05-12T04:01:34.191 INFO:tasks.workunit.client.0.smithi032.stdout:2/778: dread d1/d4d/d5e/fdc [0,4194304] 0 2024-05-12T04:01:34.192 INFO:tasks.workunit.client.0.smithi032.stdout:7/713: chown dd/d1b/l31 6 1 2024-05-12T04:01:34.193 INFO:tasks.workunit.client.0.smithi032.stdout:3/865: truncate d15/ff6 479688 0 2024-05-12T04:01:34.193 INFO:tasks.workunit.client.0.smithi032.stdout:2/779: read - d1/d4d/d5e/d9a/ff0 zero size 2024-05-12T04:01:34.194 INFO:tasks.workunit.client.0.smithi032.stdout:7/714: fsync dd/d1b/d1e/d45/fd9 0 2024-05-12T04:01:34.196 INFO:tasks.workunit.client.0.smithi032.stdout:3/866: mknod d15/d20/d26/c12c 0 2024-05-12T04:01:34.198 INFO:tasks.workunit.client.0.smithi032.stdout:7/715: creat dd/d6c/dc3/dcd/fe2 x:0 0 0 2024-05-12T04:01:34.202 INFO:tasks.workunit.client.0.smithi032.stdout:7/716: truncate dd/d1b/d1e/d45/d55/f9d 2315131 0 2024-05-12T04:01:34.209 INFO:tasks.workunit.client.0.smithi032.stdout:5/887: rmdir d4/d7/d58/d5a 39 2024-05-12T04:01:34.211 INFO:tasks.workunit.client.0.smithi032.stdout:5/888: chown d4/d7/d14/d26/d33/d4a/d38/d79/l12f 4777 1 2024-05-12T04:01:34.216 INFO:tasks.workunit.client.1.smithi033.stdout:5/891: dwrite d3/d94/f61 [0,4194304] 0 2024-05-12T04:01:34.235 INFO:tasks.workunit.client.0.smithi032.stdout:5/889: creat d4/d7/d14/da9/f13c x:0 0 0 2024-05-12T04:01:34.235 INFO:tasks.workunit.client.1.smithi033.stdout:5/892: dread d3/d94/d54/d57/d70/da3/dbe/d16/d45/d62/f64 [0,4194304] 0 2024-05-12T04:01:34.238 INFO:tasks.workunit.client.0.smithi032.stdout:1/769: write d1/d7/d30/d35/f45 [347962,3573] 0 2024-05-12T04:01:34.240 INFO:tasks.workunit.client.0.smithi032.stdout:1/770: chown d1/f47 0 1 2024-05-12T04:01:34.240 INFO:tasks.workunit.client.1.smithi033.stdout:5/893: truncate d3/d94/fe1 943156 0 2024-05-12T04:01:34.364 INFO:tasks.workunit.client.0.smithi032.stdout:1/771: mkdir d1/dd/d24/d101 0 2024-05-12T04:01:34.365 INFO:tasks.workunit.client.0.smithi032.stdout:1/772: chown d1/d7/d30/d35/d54/f73 357451 1 2024-05-12T04:01:34.366 INFO:tasks.workunit.client.0.smithi032.stdout:1/773: dread - d1/d7/d49/dd1/fee zero size 2024-05-12T04:01:34.367 INFO:tasks.workunit.client.0.smithi032.stdout:8/805: write d0/f5 [4770301,144] 0 2024-05-12T04:01:34.372 INFO:tasks.workunit.client.1.smithi033.stdout:1/885: rename d11/d1c/db8 to d11/d1c/ddf/d4c/dab/d109/d122 0 2024-05-12T04:01:34.379 INFO:tasks.workunit.client.1.smithi033.stdout:8/950: rename d1/d16/d27/d28/c62 to d1/d16/d27/d4d/d82/dd9/d36/d3c/d47/c11f 0 2024-05-12T04:01:34.384 INFO:tasks.workunit.client.1.smithi033.stdout:8/951: dread - d1/d16/d27/d4d/f9d zero size 2024-05-12T04:01:34.393 INFO:tasks.workunit.client.0.smithi032.stdout:9/668: dwrite d0/d84/d8a/da3/fa4 [0,4194304] 0 2024-05-12T04:01:34.402 INFO:tasks.workunit.client.0.smithi032.stdout:4/709: dwrite d0/d3/df/f26 [0,4194304] 0 2024-05-12T04:01:34.405 INFO:tasks.workunit.client.0.smithi032.stdout:6/717: write f6 [4474601,58069] 0 2024-05-12T04:01:34.416 INFO:tasks.workunit.client.1.smithi033.stdout:3/984: rmdir db/d17/d3a/d67 39 2024-05-12T04:01:34.421 INFO:tasks.workunit.client.0.smithi032.stdout:7/717: rename dd/d7b to dd/d1b/d1e/de3 0 2024-05-12T04:01:34.430 INFO:tasks.workunit.client.1.smithi033.stdout:3/985: dread db/d17/d4d/faa [0,4194304] 0 2024-05-12T04:01:34.431 INFO:tasks.workunit.client.1.smithi033.stdout:3/986: fdatasync db/d17/d4d/fde 0 2024-05-12T04:01:34.431 INFO:tasks.workunit.client.0.smithi032.stdout:0/802: fdatasync df/d10/d3b/dba/f10b 0 2024-05-12T04:01:34.433 INFO:tasks.workunit.client.0.smithi032.stdout:5/890: rmdir d4/d7/d14/da9 39 2024-05-12T04:01:34.435 INFO:tasks.workunit.client.0.smithi032.stdout:3/867: dwrite d15/d20/d22/d7a/f88 [0,4194304] 0 2024-05-12T04:01:34.437 INFO:tasks.workunit.client.1.smithi033.stdout:3/987: rename db/f54 to db/d17/d3e/d65/d91/d97/de8/f151 0 2024-05-12T04:01:34.440 INFO:tasks.workunit.client.0.smithi032.stdout:9/669: mkdir d0/d3/d4b/d6f/dd7 0 2024-05-12T04:01:34.442 INFO:tasks.workunit.client.0.smithi032.stdout:9/670: dread - d0/d3/d6/d7/d2a/d46/f49 zero size 2024-05-12T04:01:34.447 INFO:tasks.workunit.client.1.smithi033.stdout:5/894: truncate d3/fc 3951481 0 2024-05-12T04:01:34.448 INFO:tasks.workunit.client.0.smithi032.stdout:4/710: readlink d0/d3/l94 0 2024-05-12T04:01:34.457 INFO:tasks.workunit.client.1.smithi033.stdout:5/895: dwrite d3/d94/d54/d98/f9f [0,4194304] 0 2024-05-12T04:01:34.457 INFO:tasks.workunit.client.0.smithi032.stdout:0/803: dread df/d10/f13 [8388608,4194304] 0 2024-05-12T04:01:34.461 INFO:tasks.workunit.client.0.smithi032.stdout:3/868: write d15/d20/d22/d7a/dbe/ffc [209034,46854] 0 2024-05-12T04:01:34.462 INFO:tasks.workunit.client.1.smithi033.stdout:3/988: getdents db/d17/d3e/d65/d91 0 2024-05-12T04:01:34.462 INFO:tasks.workunit.client.1.smithi033.stdout:5/896: dread - d3/d94/d54/d57/d70/da3/dbe/d16/d45/d62/fad zero size 2024-05-12T04:01:34.464 INFO:tasks.workunit.client.0.smithi032.stdout:1/774: fdatasync d1/d6a/d72/da9/f8a 0 2024-05-12T04:01:34.464 INFO:tasks.workunit.client.1.smithi033.stdout:3/989: dread - db/d17/d25/d32/d53/df6/f109 zero size 2024-05-12T04:01:34.464 INFO:tasks.workunit.client.1.smithi033.stdout:1/886: dwrite d11/f4f [0,4194304] 0 2024-05-12T04:01:34.469 INFO:tasks.workunit.client.1.smithi033.stdout:5/897: readlink d3/d94/d85/d99/l101 0 2024-05-12T04:01:34.471 INFO:tasks.workunit.client.1.smithi033.stdout:1/887: rmdir d11/d1c/ddf/d4c/dab 39 2024-05-12T04:01:34.471 INFO:tasks.workunit.client.1.smithi033.stdout:8/952: dwrite d1/d16/d29/f9a [0,4194304] 0 2024-05-12T04:01:34.473 INFO:tasks.workunit.client.1.smithi033.stdout:3/990: truncate db/d17/d25/f52 484260 0 2024-05-12T04:01:34.474 INFO:tasks.workunit.client.1.smithi033.stdout:3/991: readlink db/d17/d3e/d65/d7d/lae 0 2024-05-12T04:01:34.478 INFO:tasks.workunit.client.1.smithi033.stdout:5/898: unlink d3/d12/d29/d2b/d8e/f110 0 2024-05-12T04:01:34.481 INFO:tasks.workunit.client.1.smithi033.stdout:1/888: readlink d11/l1e 0 2024-05-12T04:01:34.484 INFO:tasks.workunit.client.0.smithi032.stdout:0/804: mkdir df/d30/d4e/da5/d10c 0 2024-05-12T04:01:34.485 INFO:tasks.workunit.client.1.smithi033.stdout:8/953: creat d1/d16/d29/d63/da3/dba/f120 x:0 0 0 2024-05-12T04:01:34.486 INFO:tasks.workunit.client.1.smithi033.stdout:8/954: readlink l0 0 2024-05-12T04:01:34.487 INFO:tasks.workunit.client.0.smithi032.stdout:3/869: dread d15/d33/f9e [0,4194304] 0 2024-05-12T04:01:34.491 INFO:tasks.workunit.client.1.smithi033.stdout:5/899: mkdir d3/d94/d54/d57/d70/da3/dbe/d16/d78/d104/d122 0 2024-05-12T04:01:34.496 INFO:tasks.workunit.client.0.smithi032.stdout:6/718: creat d7/d9/d48/ffa x:0 0 0 2024-05-12T04:01:34.498 INFO:tasks.workunit.client.1.smithi033.stdout:8/955: creat d1/d16/d29/f121 x:0 0 0 2024-05-12T04:01:34.498 INFO:tasks.workunit.client.0.smithi032.stdout:8/806: getdents d0/d9/d1b/d66/d75/d84 0 2024-05-12T04:01:34.500 INFO:tasks.workunit.client.1.smithi033.stdout:3/992: mkdir db/d17/d3e/d65/d7d/d152 0 2024-05-12T04:01:34.502 INFO:tasks.workunit.client.0.smithi032.stdout:1/775: rename d1/d7/d30/d18/f1a to d1/d7/d30/d35/d54/d85/f102 0 2024-05-12T04:01:34.503 INFO:tasks.workunit.client.0.smithi032.stdout:6/719: dwrite d7/d97/db3/fdc [0,4194304] 0 2024-05-12T04:01:34.504 INFO:tasks.workunit.client.1.smithi033.stdout:5/900: mknod d3/d94/d54/d98/c123 0 2024-05-12T04:01:34.506 INFO:tasks.workunit.client.0.smithi032.stdout:1/776: chown d1/d7/d30/d18/c86 10700 1 2024-05-12T04:01:34.507 INFO:tasks.workunit.client.0.smithi032.stdout:0/805: creat df/d10/d3d/d4a/d9a/f10d x:0 0 0 2024-05-12T04:01:34.516 INFO:tasks.workunit.client.1.smithi033.stdout:8/956: rename d1/d16/d27 to d1/d16/d27/d4d/d82/dd9/d36/dea/d122 22 2024-05-12T04:01:34.517 INFO:tasks.workunit.client.0.smithi032.stdout:7/718: write dd/d1b/d1e/de3/d9a/d9c/db9/f6b [85633,34902] 0 2024-05-12T04:01:34.517 INFO:tasks.workunit.client.1.smithi033.stdout:5/901: chown d3/fb 0 1 2024-05-12T04:01:34.517 INFO:tasks.workunit.client.0.smithi032.stdout:6/720: dread d7/d9/d48/faa [0,4194304] 0 2024-05-12T04:01:34.518 INFO:tasks.workunit.client.1.smithi033.stdout:8/957: chown d1/d4/d46/f111 128333447 1 2024-05-12T04:01:34.519 INFO:tasks.workunit.client.1.smithi033.stdout:5/902: fdatasync d3/d94/d54/d57/d70/da3/dbe/d16/d78/fe7 0 2024-05-12T04:01:34.525 INFO:tasks.workunit.client.1.smithi033.stdout:8/958: dwrite d1/d16/d27/d4d/d73/f102 [0,4194304] 0 2024-05-12T04:01:34.527 INFO:tasks.workunit.client.0.smithi032.stdout:8/807: mkdir d0/d9/d1b/d27/d4c/d4e/d7f/da3/d111 0 2024-05-12T04:01:34.529 INFO:tasks.workunit.client.0.smithi032.stdout:5/891: truncate d4/d7/d14/d26/d33/d4a/d38/d79/fc7 2867476 0 2024-05-12T04:01:34.531 INFO:tasks.workunit.client.0.smithi032.stdout:5/892: readlink d4/d7/d14/d26/d33/d4a/d38/dd2/l4d 0 2024-05-12T04:01:34.534 INFO:tasks.workunit.client.0.smithi032.stdout:1/777: write d1/d7/d30/d35/fa3 [1700350,121990] 0 2024-05-12T04:01:34.536 INFO:tasks.workunit.client.0.smithi032.stdout:4/711: write d0/d3/df/f86 [728025,80508] 0 2024-05-12T04:01:34.538 INFO:tasks.workunit.client.1.smithi033.stdout:1/889: write d11/d17/d27/d36/d3f/d4a/dc6/de1/d114/d87/d96/fcb [131852,122020] 0 2024-05-12T04:01:34.541 INFO:tasks.workunit.client.0.smithi032.stdout:9/671: getdents d0/d3/d4b/d85/d87 0 2024-05-12T04:01:34.550 INFO:tasks.workunit.client.0.smithi032.stdout:8/808: mkdir d0/d9/d1b/d66/d112 0 2024-05-12T04:01:34.553 INFO:tasks.workunit.client.0.smithi032.stdout:5/893: rename d4/d7/d14/d108/d126/c127 to d4/d7/d14/d26/d33/d72/dc9/c13d 0 2024-05-12T04:01:34.555 INFO:tasks.workunit.client.0.smithi032.stdout:1/778: mkdir d1/d7/d30/d61/d103 0 2024-05-12T04:01:34.555 INFO:tasks.workunit.client.0.smithi032.stdout:5/894: readlink d4/d7/d14/d26/d33/d72/le6 0 2024-05-12T04:01:34.557 INFO:tasks.workunit.client.0.smithi032.stdout:8/809: dwrite d0/d9/d13/d21/d35/de9/f10a [0,4194304] 0 2024-05-12T04:01:34.569 INFO:tasks.workunit.client.0.smithi032.stdout:8/810: dwrite d0/d9/d1b/d27/d4c/d4e/d90/faf [0,4194304] 0 2024-05-12T04:01:34.570 INFO:tasks.workunit.client.0.smithi032.stdout:8/811: readlink d0/d9/d1b/d27/d4c/ld1 0 2024-05-12T04:01:34.575 INFO:tasks.workunit.client.0.smithi032.stdout:7/719: mkdir dd/d1b/d1e/d45/de4 0 2024-05-12T04:01:34.582 INFO:tasks.workunit.client.0.smithi032.stdout:5/895: unlink d4/d7/d14/d26/d47/f9b 0 2024-05-12T04:01:34.583 INFO:tasks.workunit.client.0.smithi032.stdout:5/896: chown d4/d7/d14/d26/d33/de2 119871888 1 2024-05-12T04:01:34.584 INFO:tasks.workunit.client.0.smithi032.stdout:3/870: getdents d15/d24/d2a/d5b 0 2024-05-12T04:01:34.585 INFO:tasks.workunit.client.0.smithi032.stdout:6/721: dwrite d7/d97/f9b [0,4194304] 0 2024-05-12T04:01:34.592 INFO:tasks.workunit.client.0.smithi032.stdout:8/812: rmdir d0/d9/d1b/d27/d88/d56 39 2024-05-12T04:01:34.594 INFO:tasks.workunit.client.0.smithi032.stdout:0/806: link df/d10/fe1 df/d30/d8d/d8f/f10e 0 2024-05-12T04:01:34.598 INFO:tasks.workunit.client.0.smithi032.stdout:4/712: dwrite d0/d3/d12/d1d/f74 [0,4194304] 0 2024-05-12T04:01:34.600 INFO:tasks.workunit.client.0.smithi032.stdout:7/720: truncate dd/d1b/d1e/d45/d64/f95 1443263 0 2024-05-12T04:01:34.604 INFO:tasks.workunit.client.0.smithi032.stdout:9/672: dwrite d0/d3/d4b/f6d [0,4194304] 0 2024-05-12T04:01:34.618 INFO:tasks.workunit.client.0.smithi032.stdout:6/722: dread d7/d9/d48/d67/f85 [0,4194304] 0 2024-05-12T04:01:34.626 INFO:tasks.workunit.client.0.smithi032.stdout:0/807: rmdir df/d30/d4e 39 2024-05-12T04:01:34.626 INFO:tasks.workunit.client.0.smithi032.stdout:5/897: write d4/d7/d14/d26/d33/d5f/fed [1369522,22312] 0 2024-05-12T04:01:34.628 INFO:tasks.workunit.client.0.smithi032.stdout:4/713: dread - d0/d3/df/d4e/f7e zero size 2024-05-12T04:01:34.629 INFO:tasks.workunit.client.0.smithi032.stdout:8/813: dwrite d0/f68 [4194304,4194304] 0 2024-05-12T04:01:34.632 INFO:tasks.workunit.client.0.smithi032.stdout:1/779: truncate d1/d7/d30/d35/fa3 4391011 0 2024-05-12T04:01:34.643 INFO:tasks.workunit.client.0.smithi032.stdout:1/780: dread d1/f47 [0,4194304] 0 2024-05-12T04:01:34.644 INFO:tasks.workunit.client.0.smithi032.stdout:6/723: truncate d7/d9/d48/d67/d1a/d20/fab 893412 0 2024-05-12T04:01:34.645 INFO:tasks.workunit.client.0.smithi032.stdout:1/781: chown d1/d7/d30/d35/d54/d7e/df9/dfb 1 1 2024-05-12T04:01:34.647 INFO:tasks.workunit.client.0.smithi032.stdout:0/808: mkdir df/d10/d3d/d83/d9f/d10f 0 2024-05-12T04:01:34.654 INFO:tasks.workunit.client.1.smithi033.stdout:8/959: symlink d1/d4/db2/d117/l123 0 2024-05-12T04:01:34.658 INFO:tasks.workunit.client.0.smithi032.stdout:3/871: dwrite d15/d24/f3d [0,4194304] 0 2024-05-12T04:01:34.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:34 smithi033 ceph-mon[49525]: pgmap v45: 65 pgs: 65 active+clean; 3.5 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 73 MiB/s rd, 170 MiB/s wr, 378 op/s 2024-05-12T04:01:34.663 INFO:tasks.workunit.client.1.smithi033.stdout:3/993: creat db/d17/d3a/d67/d76/f153 x:0 0 0 2024-05-12T04:01:34.663 INFO:tasks.workunit.client.0.smithi032.stdout:0/809: dread df/d30/d32/f34 [4194304,4194304] 0 2024-05-12T04:01:34.664 INFO:tasks.workunit.client.1.smithi033.stdout:3/994: write db/f12b [694776,65055] 0 2024-05-12T04:01:34.664 INFO:tasks.workunit.client.0.smithi032.stdout:8/814: mknod d0/d9/d1b/d33/d73/db7/c113 0 2024-05-12T04:01:34.666 INFO:tasks.workunit.client.1.smithi033.stdout:5/903: creat d3/d12/f124 x:0 0 0 2024-05-12T04:01:34.669 INFO:tasks.workunit.client.1.smithi033.stdout:8/960: creat d1/d4/d91/f124 x:0 0 0 2024-05-12T04:01:34.669 INFO:tasks.workunit.client.0.smithi032.stdout:9/673: truncate d0/f75 5071 0 2024-05-12T04:01:34.670 INFO:tasks.workunit.client.1.smithi033.stdout:1/890: link d11/d17/d27/da6/ff3 d11/d17/d27/d36/d3f/d4a/dc6/de1/d114/d87/d96/dd8/f123 0 2024-05-12T04:01:34.670 INFO:tasks.workunit.client.1.smithi033.stdout:8/961: chown d1/d16/d27/d4d/d82/dd9/d36/dea 100 1 2024-05-12T04:01:34.671 INFO:tasks.workunit.client.1.smithi033.stdout:3/995: mknod db/d17/d3e/d65/dfb/c154 0 2024-05-12T04:01:34.671 INFO:tasks.workunit.client.1.smithi033.stdout:8/962: chown d1/d16/d27/d4d/d82/dd9/d36/d3c/l9b 1 1 2024-05-12T04:01:34.673 INFO:tasks.workunit.client.0.smithi032.stdout:1/782: creat d1/d7/d30/d35/d54/f104 x:0 0 0 2024-05-12T04:01:34.676 INFO:tasks.workunit.client.1.smithi033.stdout:3/996: creat db/d17/d3e/d65/dfb/d11a/d13b/f155 x:0 0 0 2024-05-12T04:01:34.680 INFO:tasks.workunit.client.0.smithi032.stdout:5/898: write d4/d7/d14/d39/d2b/d6e/f105 [971099,46613] 0 2024-05-12T04:01:34.681 INFO:tasks.workunit.client.1.smithi033.stdout:8/963: rename d1/d16/d27/d4d/d73/dd0 to d1/d16/d27/d4d/d82/dd9/d36/d3c/d47/dc3/d125 0 2024-05-12T04:01:34.681 INFO:tasks.workunit.client.1.smithi033.stdout:8/964: chown l0 3444837 1 2024-05-12T04:01:34.683 INFO:tasks.workunit.client.1.smithi033.stdout:1/891: mkdir d11/d1c/ddf/d4c/dab/dee/d124 0 2024-05-12T04:01:34.685 INFO:tasks.workunit.client.0.smithi032.stdout:4/714: creat d0/d3/d24/dbd/fd2 x:0 0 0 2024-05-12T04:01:34.687 INFO:tasks.workunit.client.1.smithi033.stdout:3/997: chown db/d17/d25/d32/d3c/d3d/d111/d132/c144 1226 1 2024-05-12T04:01:34.687 INFO:tasks.workunit.client.1.smithi033.stdout:8/965: dwrite d1/d16/d27/d4d/d82/dd9/d36/d3c/da7/f43 [0,4194304] 0 2024-05-12T04:01:34.691 INFO:tasks.workunit.client.0.smithi032.stdout:3/872: mknod d15/d24/d93/c12d 0 2024-05-12T04:01:34.702 INFO:tasks.workunit.client.1.smithi033.stdout:3/998: rename db/d17/d3a/d67/d76/da0/fb1 to db/d42/f156 0 2024-05-12T04:01:34.703 INFO:tasks.workunit.client.0.smithi032.stdout:7/721: link dd/f12 dd/d1b/d1e/d45/de4/fe5 0 2024-05-12T04:01:34.704 INFO:tasks.workunit.client.0.smithi032.stdout:7/722: readlink dd/d1b/d1e/de3/d9a/d9c/dc2/lb0 0 2024-05-12T04:01:34.706 INFO:tasks.workunit.client.0.smithi032.stdout:9/674: unlink d0/d3/d6/d7/d2a/f50 0 2024-05-12T04:01:34.708 INFO:tasks.workunit.client.0.smithi032.stdout:6/724: fdatasync d7/d9/d5c/d8b/fa9 0 2024-05-12T04:01:34.708 INFO:tasks.workunit.client.1.smithi033.stdout:5/904: getdents d3/d94/d85/d99/da2 0 2024-05-12T04:01:34.714 INFO:tasks.workunit.client.1.smithi033.stdout:8/966: symlink d1/d16/d27/d4d/d82/dd9/df4/l126 0 2024-05-12T04:01:34.715 INFO:tasks.workunit.client.1.smithi033.stdout:8/967: read - d1/d16/d27/d7c/f89 zero size 2024-05-12T04:01:34.715 INFO:tasks.workunit.client.1.smithi033.stdout:8/968: chown d1/d16/f96 33136899 1 2024-05-12T04:01:34.716 INFO:tasks.workunit.client.0.smithi032.stdout:5/899: fsync d4/f9a 0 2024-05-12T04:01:34.716 INFO:tasks.workunit.client.1.smithi033.stdout:1/892: write d11/d17/d27/d36/d3f/d4a/dc6/de1/d114/d87/d96/dd8/f123 [143648,69171] 0 2024-05-12T04:01:34.719 INFO:tasks.workunit.client.1.smithi033.stdout:5/905: creat d3/d12/d29/d2b/d8e/f125 x:0 0 0 2024-05-12T04:01:34.724 INFO:tasks.workunit.client.0.smithi032.stdout:3/873: creat d15/d33/d3f/f12e x:0 0 0 2024-05-12T04:01:34.727 INFO:tasks.workunit.client.1.smithi033.stdout:3/999: creat db/d17/d3e/dd7/f157 x:0 0 0 2024-05-12T04:01:34.727 INFO:tasks.workunit.client.0.smithi032.stdout:5/900: dwrite d4/d7/d14/d26/d33/d72/d92/f109 [0,4194304] 0 2024-05-12T04:01:34.729 INFO:tasks.workunit.client.0.smithi032.stdout:4/715: dwrite d0/db5/fbc [0,4194304] 0 2024-05-12T04:01:34.729 INFO:tasks.workunit.client.0.smithi032.stdout:8/815: creat d0/d9/d1b/d66/d112/f114 x:0 0 0 2024-05-12T04:01:34.732 INFO:tasks.workunit.client.1.smithi033.stdout:8/969: dread d1/d16/fbe [0,4194304] 0 2024-05-12T04:01:34.734 INFO:tasks.workunit.client.0.smithi032.stdout:9/675: mkdir d0/d3/d74/d51/dd8 0 2024-05-12T04:01:34.736 INFO:tasks.workunit.client.0.smithi032.stdout:9/676: truncate d0/d3/d74/f6c 1589909 0 2024-05-12T04:01:34.742 INFO:tasks.workunit.client.1.smithi033.stdout:8/970: mknod d1/d16/d27/d4d/d51/d8e/db6/c127 0 2024-05-12T04:01:34.742 INFO:tasks.workunit.client.0.smithi032.stdout:7/723: write dd/d1b/d1e/de3/d9a/d9c/dc2/fce [634750,106547] 0 2024-05-12T04:01:34.745 INFO:tasks.workunit.client.1.smithi033.stdout:5/906: dwrite d3/d94/d54/d57/d70/da3/dbe/d16/d45/d62/d7f/d84/de2/fe8 [0,4194304] 0 2024-05-12T04:01:34.749 INFO:tasks.workunit.client.0.smithi032.stdout:3/874: symlink d15/d24/d2a/dc7/l12f 0 2024-05-12T04:01:34.753 INFO:tasks.workunit.client.0.smithi032.stdout:1/783: rename d1/d7/d30/d61/f68 to d1/d7/d30/f105 0 2024-05-12T04:01:34.754 INFO:tasks.workunit.client.0.smithi032.stdout:7/724: dwrite dd/d1b/d1e/de3/d9a/d9c/dc2/fce [0,4194304] 0 2024-05-12T04:01:34.754 INFO:tasks.workunit.client.1.smithi033.stdout:8/971: truncate d1/d16/d27/f85 941827 0 2024-05-12T04:01:34.755 INFO:tasks.workunit.client.0.smithi032.stdout:1/784: stat d1/d7/d30/d18/ff3 0 2024-05-12T04:01:34.756 INFO:tasks.workunit.client.0.smithi032.stdout:2/780: sync 2024-05-12T04:01:34.757 INFO:tasks.workunit.client.0.smithi032.stdout:0/810: link df/d30/d4e/d55/dd4/fe5 df/d30/d32/d81/de8/f110 0 2024-05-12T04:01:34.757 INFO:tasks.workunit.client.0.smithi032.stdout:1/785: write d1/d7/d30/d18/ff3 [929426,100578] 0 2024-05-12T04:01:34.760 INFO:tasks.workunit.client.0.smithi032.stdout:1/786: write d1/d7/d30/d35/f45 [1850292,7357] 0 2024-05-12T04:01:34.760 INFO:tasks.workunit.client.0.smithi032.stdout:5/901: unlink d4/d7/d14/d26/d33/d4a/d38/d79/fc7 0 2024-05-12T04:01:34.762 INFO:tasks.workunit.client.0.smithi032.stdout:8/816: fdatasync d0/d9/d1b/d27/d88/d56/d63/d8f/d9a/fcf 0 2024-05-12T04:01:34.763 INFO:tasks.workunit.client.0.smithi032.stdout:4/716: rename d0/d3/d12/d1d/d63/d77/f8f to d0/db5/fd3 0 2024-05-12T04:01:34.764 INFO:tasks.workunit.client.0.smithi032.stdout:0/811: mknod df/d30/d32/d81/c111 0 2024-05-12T04:01:34.765 INFO:tasks.workunit.client.0.smithi032.stdout:3/875: chown d15/d20/d26/d6a/d6e/d7d/fd9 191 1 2024-05-12T04:01:34.766 INFO:tasks.workunit.client.0.smithi032.stdout:1/787: symlink d1/dd/d24/d9b/l106 0 2024-05-12T04:01:34.773 INFO:tasks.workunit.client.1.smithi033.stdout:5/907: write d3/d94/d54/d57/d70/da3/dbe/d16/d45/d62/d7f/d84/de2/fe8 [4631275,42123] 0 2024-05-12T04:01:34.774 INFO:tasks.workunit.client.1.smithi033.stdout:5/908: truncate d3/d12/dac/fce 1005910 0 2024-05-12T04:01:34.774 INFO:tasks.workunit.client.1.smithi033.stdout:8/972: write d1/d4/d46/f111 [2723,65920] 0 2024-05-12T04:01:34.779 INFO:tasks.workunit.client.1.smithi033.stdout:8/973: rename d1/d16/d27/d7c/db7/cf0 to d1/d16/d27/d4d/d82/dd9/d36/d3c/d64/c128 0 2024-05-12T04:01:34.781 INFO:tasks.workunit.client.1.smithi033.stdout:5/909: dwrite d3/d12/d29/f3d [4194304,4194304] 0 2024-05-12T04:01:34.785 INFO:tasks.workunit.client.1.smithi033.stdout:8/974: truncate d1/d16/d27/d4d/d82/dd9/d36/fe5 380821 0 2024-05-12T04:01:34.789 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:34 smithi032 ceph-mon[40008]: pgmap v45: 65 pgs: 65 active+clean; 3.5 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 73 MiB/s rd, 170 MiB/s wr, 378 op/s 2024-05-12T04:01:34.794 INFO:tasks.workunit.client.1.smithi033.stdout:8/975: write d1/d16/d27/d4d/fd4 [414633,11703] 0 2024-05-12T04:01:34.795 INFO:tasks.workunit.client.1.smithi033.stdout:8/976: write d1/d16/d29/f10c [448784,7099] 0 2024-05-12T04:01:34.797 INFO:tasks.workunit.client.1.smithi033.stdout:8/977: chown d1/d16/d27/d4d/d82/dd9/d36/d3c/d64/f8d 7 1 2024-05-12T04:01:34.810 INFO:tasks.workunit.client.1.smithi033.stdout:8/978: dread d1/d16/d27/d4d/f5e [0,4194304] 0 2024-05-12T04:01:34.812 INFO:tasks.workunit.client.1.smithi033.stdout:8/979: creat d1/d16/d27/d4d/d51/d8e/f129 x:0 0 0 2024-05-12T04:01:34.818 INFO:tasks.workunit.client.1.smithi033.stdout:8/980: dwrite d1/d16/d27/d4d/d51/fe2 [0,4194304] 0 2024-05-12T04:01:34.819 INFO:tasks.workunit.client.1.smithi033.stdout:8/981: write d1/f6b [846434,99744] 0 2024-05-12T04:01:34.825 INFO:tasks.workunit.client.1.smithi033.stdout:8/982: write d1/d16/f96 [2928393,120442] 0 2024-05-12T04:01:34.831 INFO:tasks.workunit.client.1.smithi033.stdout:8/983: rename d1/d16/d27/d4d/d82/dd9/d36/dea/ff1 to d1/d16/d27/d4d/d82/dd9/d36/d3c/f12a 0 2024-05-12T04:01:34.833 INFO:tasks.workunit.client.1.smithi033.stdout:8/984: truncate d1/d16/d27/d4d/d82/dd9/d36/f59 810619 0 2024-05-12T04:01:34.839 INFO:tasks.workunit.client.1.smithi033.stdout:8/985: dwrite d1/d16/d27/d4d/d82/dd9/d36/f50 [4194304,4194304] 0 2024-05-12T04:01:34.841 INFO:tasks.workunit.client.1.smithi033.stdout:8/986: unlink l0 0 2024-05-12T04:01:34.844 INFO:tasks.workunit.client.1.smithi033.stdout:8/987: fdatasync d1/d16/d27/d7c/fc6 0 2024-05-12T04:01:34.844 INFO:tasks.workunit.client.1.smithi033.stdout:8/988: stat d1/d16/d27/d4d/f9d 0 2024-05-12T04:01:34.845 INFO:tasks.workunit.client.1.smithi033.stdout:8/989: readlink d1/l9 0 2024-05-12T04:01:34.852 INFO:tasks.workunit.client.1.smithi033.stdout:8/990: dwrite d1/d16/d27/d4d/d82/dd9/d36/faf [0,4194304] 0 2024-05-12T04:01:34.859 INFO:tasks.workunit.client.1.smithi033.stdout:8/991: dwrite d1/d16/d29/d63/fbf [0,4194304] 0 2024-05-12T04:01:34.862 INFO:tasks.workunit.client.1.smithi033.stdout:8/992: fdatasync d1/d16/fa6 0 2024-05-12T04:01:34.866 INFO:tasks.workunit.client.1.smithi033.stdout:8/993: mkdir d1/d16/d29/d63/da3/dba/d12b 0 2024-05-12T04:01:34.866 INFO:tasks.workunit.client.1.smithi033.stdout:8/994: dread - d1/d4/db2/fc9 zero size 2024-05-12T04:01:34.868 INFO:tasks.workunit.client.0.smithi032.stdout:0/812: dread - df/d10/fe1 zero size 2024-05-12T04:01:34.869 INFO:tasks.workunit.client.0.smithi032.stdout:9/677: dread d0/d3/d4b/d88/d99/fb1 [0,4194304] 0 2024-05-12T04:01:34.874 INFO:tasks.workunit.client.0.smithi032.stdout:6/725: dwrite d7/d9/d48/d67/f45 [0,4194304] 0 2024-05-12T04:01:34.875 INFO:tasks.workunit.client.0.smithi032.stdout:8/817: write d0/d97/fc6 [347595,109291] 0 2024-05-12T04:01:34.878 INFO:tasks.workunit.client.0.smithi032.stdout:6/726: write d7/d9/d48/ffa [581850,89096] 0 2024-05-12T04:01:34.880 INFO:tasks.workunit.client.0.smithi032.stdout:1/788: unlink d1/d7/d49/dd1/cdb 0 2024-05-12T04:01:34.880 INFO:tasks.workunit.client.0.smithi032.stdout:6/727: write d7/d97/fec [852904,5707] 0 2024-05-12T04:01:34.881 INFO:tasks.workunit.client.1.smithi033.stdout:8/995: link d1/d16/d27/d4d/d82/dd9/d36/d3c/d64/l115 d1/d4/db2/d117/l12c 0 2024-05-12T04:01:34.882 INFO:tasks.workunit.client.0.smithi032.stdout:2/781: link d1/d34/d96/da4/fde d1/d34/d96/d41/d2c/dd4/f102 0 2024-05-12T04:01:34.887 INFO:tasks.workunit.client.0.smithi032.stdout:9/678: mknod d0/d3/d6/d7/d2a/d9f/cd9 0 2024-05-12T04:01:34.888 INFO:tasks.workunit.client.0.smithi032.stdout:0/813: truncate df/d30/d4e/d55/f67 4206367 0 2024-05-12T04:01:34.889 INFO:tasks.workunit.client.0.smithi032.stdout:4/717: link d0/d3/d12/d1d/d63/l96 d0/d3/d12/d1d/d63/d87/dc6/ld4 0 2024-05-12T04:01:34.891 INFO:tasks.workunit.client.0.smithi032.stdout:0/814: read f9 [2577347,64436] 0 2024-05-12T04:01:34.892 INFO:tasks.workunit.client.0.smithi032.stdout:1/789: creat d1/d7/d30/d35/d54/d7e/df9/dfb/f107 x:0 0 0 2024-05-12T04:01:34.895 INFO:tasks.workunit.client.0.smithi032.stdout:2/782: rename d1/d34/d4a/d86/ld7 to d1/d4d/d5e/l103 0 2024-05-12T04:01:34.897 INFO:tasks.workunit.client.0.smithi032.stdout:4/718: dwrite d0/d3/d24/f72 [0,4194304] 0 2024-05-12T04:01:34.897 INFO:tasks.workunit.client.0.smithi032.stdout:5/902: link d4/f12 d4/d7/d14/d26/f13e 0 2024-05-12T04:01:34.907 INFO:tasks.workunit.client.0.smithi032.stdout:5/903: dwrite d4/d7/d14/f10c [4194304,4194304] 0 2024-05-12T04:01:34.910 INFO:tasks.workunit.client.1.smithi033.stdout:8/996: dwrite d1/d4/db2/fbb [0,4194304] 0 2024-05-12T04:01:34.914 INFO:tasks.workunit.client.0.smithi032.stdout:3/876: dread d15/d24/d93/f102 [0,4194304] 0 2024-05-12T04:01:34.915 INFO:tasks.workunit.client.0.smithi032.stdout:4/719: rename d0/d3/d31/d80/d84/f64 to d0/d3/d12/d1d/d6b/dc2/dc4/fd5 0 2024-05-12T04:01:34.917 INFO:tasks.workunit.client.0.smithi032.stdout:3/877: fdatasync d15/d24/d2a/d8c/fd4 0 2024-05-12T04:01:34.921 INFO:tasks.workunit.client.0.smithi032.stdout:5/904: dwrite d4/d7/d14/d26/f107 [0,4194304] 0 2024-05-12T04:01:34.924 INFO:tasks.workunit.client.1.smithi033.stdout:8/997: truncate d1/d16/d27/d4d/d82/dd9/d36/fe5 453787 0 2024-05-12T04:01:34.925 INFO:tasks.workunit.client.0.smithi032.stdout:6/728: write d7/d9/d48/d67/d63/d64/d94/de4/f26 [784424,63979] 0 2024-05-12T04:01:34.928 INFO:tasks.workunit.client.1.smithi033.stdout:8/998: mknod d1/d4/db2/c12d 0 2024-05-12T04:01:34.929 INFO:tasks.workunit.client.1.smithi033.stdout:8/999: rename d1 to d1/d12e 22 2024-05-12T04:01:34.931 INFO:tasks.workunit.client.0.smithi032.stdout:0/815: creat df/d10/d48/f112 x:0 0 0 2024-05-12T04:01:34.933 INFO:tasks.workunit.client.0.smithi032.stdout:2/783: dwrite d1/d13/f2b [0,4194304] 0 2024-05-12T04:01:34.933 INFO:tasks.workunit.client.0.smithi032.stdout:4/720: rename d0/d3/d24/f2d to d0/d3/d24/dbd/fd6 0 2024-05-12T04:01:34.935 INFO:tasks.workunit.client.0.smithi032.stdout:2/784: dread - d1/d34/d96/d41/d52/ffc zero size 2024-05-12T04:01:34.938 INFO:tasks.workunit.client.0.smithi032.stdout:9/679: write d0/f28 [2777261,85891] 0 2024-05-12T04:01:34.940 INFO:tasks.workunit.client.0.smithi032.stdout:2/785: chown d1/d13/f2b 5 1 2024-05-12T04:01:34.943 INFO:tasks.workunit.client.0.smithi032.stdout:5/905: dread d4/f20 [0,4194304] 0 2024-05-12T04:01:34.944 INFO:tasks.workunit.client.1.smithi033.stdout:1/893: sync 2024-05-12T04:01:34.945 INFO:tasks.workunit.client.0.smithi032.stdout:4/721: write d0/f65 [4135524,26332] 0 2024-05-12T04:01:34.946 INFO:tasks.workunit.client.0.smithi032.stdout:3/878: rename d15/d24/d2a/d8c/db1/f112 to d15/d1b/d43/dd0/d11b/f130 0 2024-05-12T04:01:34.950 INFO:tasks.workunit.client.1.smithi033.stdout:1/894: read - d11/d17/d27/d76/d93/fb0 zero size 2024-05-12T04:01:34.951 INFO:tasks.workunit.client.1.smithi033.stdout:1/895: chown d11/d17/d27/da6/dfe/f11c 19573111 1 2024-05-12T04:01:34.952 INFO:tasks.workunit.client.0.smithi032.stdout:9/680: mkdir d0/d3/d4b/dda 0 2024-05-12T04:01:34.957 INFO:tasks.workunit.client.0.smithi032.stdout:2/786: unlink d1/d34/d3e/d67/lb8 0 2024-05-12T04:01:34.958 INFO:tasks.workunit.client.0.smithi032.stdout:0/816: dwrite df/d10/d3d/d4a/f2d [0,4194304] 0 2024-05-12T04:01:34.960 INFO:tasks.workunit.client.0.smithi032.stdout:0/817: dread - df/d10/d3d/de4/dea/ff7 zero size 2024-05-12T04:01:34.961 INFO:tasks.workunit.client.1.smithi033.stdout:1/896: rename d11/d17/d27/d36/d3f/d4a/dc6/de1/f10b to d11/d1c/ddf/d4c/dab/d109/f125 0 2024-05-12T04:01:34.966 INFO:tasks.workunit.client.0.smithi032.stdout:6/729: dwrite d7/d9/d48/d67/d63/d64/d94/de4/f6d [0,4194304] 0 2024-05-12T04:01:34.969 INFO:tasks.workunit.client.0.smithi032.stdout:1/790: link d1/d7/d30/d18/l37 d1/d7/d49/l108 0 2024-05-12T04:01:34.969 INFO:tasks.workunit.client.1.smithi033.stdout:1/897: mknod d11/d10a/c126 0 2024-05-12T04:01:34.970 INFO:tasks.workunit.client.1.smithi033.stdout:1/898: chown d11/d17/d27/d36/d3f/d4a/ce0 234656 1 2024-05-12T04:01:34.972 INFO:tasks.workunit.client.0.smithi032.stdout:9/681: fsync d0/d3/d6/d58/f66 0 2024-05-12T04:01:34.977 INFO:tasks.workunit.client.1.smithi033.stdout:1/899: mkdir d11/d17/d27/d36/d3f/d4a/dc6/de1/d114/d87/d108/d127 0 2024-05-12T04:01:34.980 INFO:tasks.workunit.client.0.smithi032.stdout:3/879: write d15/d20/d26/d6a/d6e/d7d/d9a/fa7 [1467071,48972] 0 2024-05-12T04:01:34.981 INFO:tasks.workunit.client.1.smithi033.stdout:1/900: creat d11/d1c/ddf/d4c/dab/f128 x:0 0 0 2024-05-12T04:01:34.983 INFO:tasks.workunit.client.0.smithi032.stdout:2/787: mkdir d1/d104 0 2024-05-12T04:01:34.985 INFO:tasks.workunit.client.0.smithi032.stdout:5/906: getdents d4/d7/d14/d39 0 2024-05-12T04:01:34.985 INFO:tasks.workunit.client.1.smithi033.stdout:1/901: mknod d11/d17/d27/d36/d3f/d4a/dc6/de1/d114/d87/d108/c129 0 2024-05-12T04:01:34.986 INFO:tasks.workunit.client.0.smithi032.stdout:9/682: write d0/d3/d6/d7/d2a/d46/d13/f3c [3976593,8093] 0 2024-05-12T04:01:34.987 INFO:tasks.workunit.client.0.smithi032.stdout:4/722: creat d0/d3/d12/fd7 x:0 0 0 2024-05-12T04:01:34.988 INFO:tasks.workunit.client.0.smithi032.stdout:6/730: mkdir d7/d9/d5c/d8b/da1/dfb 0 2024-05-12T04:01:34.989 INFO:tasks.workunit.client.1.smithi033.stdout:1/902: fsync d11/d17/d27/d36/d3f/d4a/dc6/de1/d114/d87/da4/fba 0 2024-05-12T04:01:34.991 INFO:tasks.workunit.client.0.smithi032.stdout:1/791: mkdir d1/d7/d30/d18/d93/dc1/d109 0 2024-05-12T04:01:34.999 INFO:tasks.workunit.client.0.smithi032.stdout:3/880: dread d15/d24/f51 [0,4194304] 0 2024-05-12T04:01:35.000 INFO:tasks.workunit.client.0.smithi032.stdout:3/881: stat d15/d1b/d43/dd0 0 2024-05-12T04:01:35.001 INFO:tasks.workunit.client.0.smithi032.stdout:5/907: creat d4/d7/d14/d26/d33/d72/d92/d111/f13f x:0 0 0 2024-05-12T04:01:35.001 INFO:tasks.workunit.client.1.smithi033.stdout:5/910: sync 2024-05-12T04:01:35.004 INFO:tasks.workunit.client.0.smithi032.stdout:4/723: mknod d0/d3/d12/d1d/d63/d77/cd8 0 2024-05-12T04:01:35.004 INFO:tasks.workunit.client.0.smithi032.stdout:3/882: dread d15/d20/d22/d4f/de0/f11c [0,4194304] 0 2024-05-12T04:01:35.005 INFO:tasks.workunit.client.0.smithi032.stdout:9/683: dread d0/d3/d6/d7/d2a/d46/d13/f7c [0,4194304] 0 2024-05-12T04:01:35.006 INFO:tasks.workunit.client.1.smithi033.stdout:5/911: dread d3/d94/d54/d57/d70/da3/dbe/d16/d45/f68 [4194304,4194304] 0 2024-05-12T04:01:35.006 INFO:tasks.workunit.client.0.smithi032.stdout:6/731: rmdir d7/d9/d48/d67/d1a/d20 39 2024-05-12T04:01:35.008 INFO:tasks.workunit.client.0.smithi032.stdout:6/732: write d7/d9/d48/fbf [2258299,66661] 0 2024-05-12T04:01:35.008 INFO:tasks.workunit.client.0.smithi032.stdout:2/788: write d1/d34/da5/db3/db7/fcf [508938,129201] 0 2024-05-12T04:01:35.009 INFO:tasks.workunit.client.0.smithi032.stdout:0/818: getdents df/d30/dc2 0 2024-05-12T04:01:35.009 INFO:tasks.workunit.client.1.smithi033.stdout:1/903: write d11/d1c/f32 [1076053,27150] 0 2024-05-12T04:01:35.010 INFO:tasks.workunit.client.1.smithi033.stdout:1/904: chown d11/d17/d27/d10d 54110 1 2024-05-12T04:01:35.010 INFO:tasks.workunit.client.0.smithi032.stdout:3/883: truncate d15/d24/d2a/d8c/fd4 1128490 0 2024-05-12T04:01:35.012 INFO:tasks.workunit.client.0.smithi032.stdout:1/792: dread d1/d25/da5/db5/fda [0,4194304] 0 2024-05-12T04:01:35.014 INFO:tasks.workunit.client.0.smithi032.stdout:5/908: creat d4/d7/d14/d26/d33/d72/d92/d111/f140 x:0 0 0 2024-05-12T04:01:35.014 INFO:tasks.workunit.client.1.smithi033.stdout:1/905: dread d11/d17/fd7 [0,4194304] 0 2024-05-12T04:01:35.017 INFO:tasks.workunit.client.1.smithi033.stdout:1/906: dread d11/d17/d27/d36/d3f/d4a/dc6/de1/d114/d87/d96/dd8/f123 [0,4194304] 0 2024-05-12T04:01:35.018 INFO:tasks.workunit.client.0.smithi032.stdout:9/684: mkdir d0/d3/d4b/d88/dc4/ddb 0 2024-05-12T04:01:35.021 INFO:tasks.workunit.client.0.smithi032.stdout:6/733: symlink d7/d9/d48/d67/d25/lfc 0 2024-05-12T04:01:35.022 INFO:tasks.workunit.client.1.smithi033.stdout:5/912: dwrite d3/d94/d54/d57/d70/da3/dbe/d16/d45/d4b/f56 [0,4194304] 0 2024-05-12T04:01:35.026 INFO:tasks.workunit.client.0.smithi032.stdout:1/793: unlink d1/dd/l2e 0 2024-05-12T04:01:35.030 INFO:tasks.workunit.client.1.smithi033.stdout:5/913: fdatasync d3/d94/d54/d57/d70/da3/dbe/d16/d45/d62/f64 0 2024-05-12T04:01:35.033 INFO:tasks.workunit.client.1.smithi033.stdout:1/907: mknod d11/d17/d27/d36/d3f/d4a/dc6/de1/d114/d87/d96/dd8/c12a 0 2024-05-12T04:01:35.035 INFO:tasks.workunit.client.0.smithi032.stdout:6/734: mkdir d7/d9/d48/d67/d1a/da0/dc6/dfd 0 2024-05-12T04:01:35.038 INFO:tasks.workunit.client.0.smithi032.stdout:9/685: dread d0/d3/d6/d7/da/d39/f4d [0,4194304] 0 2024-05-12T04:01:35.040 INFO:tasks.workunit.client.0.smithi032.stdout:5/909: dwrite d4/d7/d14/d26/d33/d72/d92/d111/f12d [0,4194304] 0 2024-05-12T04:01:35.045 INFO:tasks.workunit.client.0.smithi032.stdout:5/910: dread - d4/d7/d14/d26/d33/d72/d92/d111/f135 zero size 2024-05-12T04:01:35.046 INFO:tasks.workunit.client.0.smithi032.stdout:1/794: dread d1/d7/d30/d35/f48 [0,4194304] 0 2024-05-12T04:01:35.049 INFO:tasks.workunit.client.0.smithi032.stdout:3/884: dread d15/d24/d93/f78 [0,4194304] 0 2024-05-12T04:01:35.050 INFO:tasks.workunit.client.0.smithi032.stdout:9/686: rmdir d0/d3/d4b/d85 39 2024-05-12T04:01:35.053 INFO:tasks.workunit.client.0.smithi032.stdout:4/724: link d0/d3/c2c d0/d3/d12/d1d/d63/d87/dc6/cd9 0 2024-05-12T04:01:35.060 INFO:tasks.workunit.client.0.smithi032.stdout:2/789: getdents d1/d13/da0/dae 0 2024-05-12T04:01:35.065 INFO:tasks.workunit.client.1.smithi033.stdout:5/914: dwrite d3/d94/d54/d57/d73/d74/f93 [0,4194304] 0 2024-05-12T04:01:35.066 INFO:tasks.workunit.client.0.smithi032.stdout:6/735: rename d7/d9/d48/d67/d8d/dbc/de2/f58 to d7/d9/d48/d67/d8d/dbc/de2/d9d/ffe 0 2024-05-12T04:01:35.067 INFO:tasks.workunit.client.0.smithi032.stdout:3/885: dwrite d15/d20/d22/d7a/f106 [4194304,4194304] 0 2024-05-12T04:01:35.068 INFO:tasks.workunit.client.0.smithi032.stdout:4/725: truncate d0/f93 638644 0 2024-05-12T04:01:35.069 INFO:tasks.workunit.client.0.smithi032.stdout:5/911: creat d4/d7/d58/d5a/d99/f141 x:0 0 0 2024-05-12T04:01:35.070 INFO:tasks.workunit.client.0.smithi032.stdout:1/795: symlink d1/d7/d30/d18/d93/dc1/l10a 0 2024-05-12T04:01:35.072 INFO:tasks.workunit.client.0.smithi032.stdout:1/796: write d1/d7/d30/d35/f45 [2042779,98891] 0 2024-05-12T04:01:35.074 INFO:tasks.workunit.client.0.smithi032.stdout:2/790: symlink d1/d13/l105 0 2024-05-12T04:01:35.075 INFO:tasks.workunit.client.1.smithi033.stdout:5/915: mkdir d3/d94/d54/d57/d70/da3/dbe/d16/d78/d104/d122/d126 0 2024-05-12T04:01:35.077 INFO:tasks.workunit.client.1.smithi033.stdout:5/916: fdatasync d3/d94/d54/d57/d70/da3/dbe/d16/d115/f121 0 2024-05-12T04:01:35.079 INFO:tasks.workunit.client.0.smithi032.stdout:3/886: fdatasync d15/d24/d2a/d8c/db1/fdb 0 2024-05-12T04:01:35.080 INFO:tasks.workunit.client.0.smithi032.stdout:9/687: write d0/d3/d6/f69 [189575,63470] 0 2024-05-12T04:01:35.081 INFO:tasks.workunit.client.0.smithi032.stdout:4/726: symlink d0/d3/d12/d1d/dc5/lda 0 2024-05-12T04:01:35.083 INFO:tasks.workunit.client.0.smithi032.stdout:3/887: write d15/d20/d26/d6a/d6e/d7d/d9a/fa7 [2542990,11716] 0 2024-05-12T04:01:35.087 INFO:tasks.workunit.client.1.smithi033.stdout:5/917: link d3/d94/d54/d57/d70/da3/dbe/d16/d45/d62/f67 d3/d94/d54/d57/d70/da3/dbe/daa/db3/f127 0 2024-05-12T04:01:35.093 INFO:tasks.workunit.client.1.smithi033.stdout:5/918: mkdir d3/d94/d54/d57/d128 0 2024-05-12T04:01:35.095 INFO:tasks.workunit.client.1.smithi033.stdout:5/919: chown d3/d94/d54/d57/d70/da3/dbe/d16/d45/d62/d7f/d84/f91 7 1 2024-05-12T04:01:35.095 INFO:tasks.workunit.client.0.smithi032.stdout:4/727: creat d0/d3/df/d4e/fdb x:0 0 0 2024-05-12T04:01:35.100 INFO:tasks.workunit.client.0.smithi032.stdout:6/736: creat d7/d9/d48/fff x:0 0 0 2024-05-12T04:01:35.103 INFO:tasks.workunit.client.0.smithi032.stdout:2/791: write d1/d34/d96/da4/fc0 [733961,69140] 0 2024-05-12T04:01:35.105 INFO:tasks.workunit.client.1.smithi033.stdout:5/920: creat d3/d94/d54/d57/d70/da3/dbe/f129 x:0 0 0 2024-05-12T04:01:35.106 INFO:tasks.workunit.client.0.smithi032.stdout:3/888: mkdir d15/d33/d3f/d75/d131 0 2024-05-12T04:01:35.108 INFO:tasks.workunit.client.0.smithi032.stdout:4/728: rename d0/d3/d12/faf to d0/d3/d24/dae/fdc 0 2024-05-12T04:01:35.108 INFO:tasks.workunit.client.0.smithi032.stdout:5/912: write d4/d7/d14/d39/d1b/da4/fc4 [342415,10204] 0 2024-05-12T04:01:35.109 INFO:tasks.workunit.client.1.smithi033.stdout:5/921: dread d3/d94/d54/d57/d70/da3/dbe/d16/d45/d4b/f56 [0,4194304] 0 2024-05-12T04:01:35.110 INFO:tasks.workunit.client.1.smithi033.stdout:5/922: fsync d3/d12/d25/db0/ff7 0 2024-05-12T04:01:35.110 INFO:tasks.workunit.client.0.smithi032.stdout:4/729: chown d0/d3/d12/d1d/d63/d77/cd8 13807 1 2024-05-12T04:01:35.110 INFO:tasks.workunit.client.0.smithi032.stdout:1/797: mkdir d1/d6a/d72/da4/d10b 0 2024-05-12T04:01:35.111 INFO:tasks.workunit.client.1.smithi033.stdout:5/923: readlink d3/d94/d54/d57/d73/lc5 0 2024-05-12T04:01:35.111 INFO:tasks.workunit.client.0.smithi032.stdout:9/688: write d0/d3/d6/d7/da/d39/f4d [4383154,74509] 0 2024-05-12T04:01:35.113 INFO:tasks.workunit.client.0.smithi032.stdout:6/737: unlink d7/d9/d48/d67/d63/d64/d94/de4/fcb 0 2024-05-12T04:01:35.116 INFO:tasks.workunit.client.0.smithi032.stdout:5/913: dread d4/d7/d14/d26/d33/d5f/fed [0,4194304] 0 2024-05-12T04:01:35.117 INFO:tasks.workunit.client.0.smithi032.stdout:2/792: creat d1/d34/d96/d41/d58/f106 x:0 0 0 2024-05-12T04:01:35.122 INFO:tasks.workunit.client.0.smithi032.stdout:3/889: creat d15/d33/d3f/f132 x:0 0 0 2024-05-12T04:01:35.125 INFO:tasks.workunit.client.1.smithi033.stdout:5/924: dread d3/d94/dc8/f28 [0,4194304] 0 2024-05-12T04:01:35.126 INFO:tasks.workunit.client.1.smithi033.stdout:5/925: dread - d3/d12/dac/dd9/f109 zero size 2024-05-12T04:01:35.127 INFO:tasks.workunit.client.0.smithi032.stdout:1/798: rmdir d1/d6a/def 39 2024-05-12T04:01:35.128 INFO:tasks.workunit.client.0.smithi032.stdout:9/689: rename d0/d3/d4b/d6f/f8b to d0/d3/d4b/d88/dc4/fdc 0 2024-05-12T04:01:35.132 INFO:tasks.workunit.client.0.smithi032.stdout:5/914: truncate d4/d7/d14/d26/d33/d88/faf 1292511 0 2024-05-12T04:01:35.133 INFO:tasks.workunit.client.1.smithi033.stdout:5/926: dread d3/d12/f80 [0,4194304] 0 2024-05-12T04:01:35.134 INFO:tasks.workunit.client.0.smithi032.stdout:4/730: write d0/d3/d12/f48 [3192011,45854] 0 2024-05-12T04:01:35.137 INFO:tasks.workunit.client.0.smithi032.stdout:4/731: chown d0/d3/d12/d1d/d63/d87/dc6 6648 1 2024-05-12T04:01:35.137 INFO:tasks.workunit.client.1.smithi033.stdout:1/908: sync 2024-05-12T04:01:35.138 INFO:tasks.workunit.client.0.smithi032.stdout:4/732: truncate d0/f93 786653 0 2024-05-12T04:01:35.138 INFO:tasks.workunit.client.1.smithi033.stdout:5/927: mkdir d3/d94/db8/d12a 0 2024-05-12T04:01:35.138 INFO:tasks.workunit.client.0.smithi032.stdout:9/690: dread - d0/d3/d74/d51/f7f zero size 2024-05-12T04:01:35.142 INFO:tasks.workunit.client.0.smithi032.stdout:6/738: mknod d7/d9/d48/d67/d1a/d93/df5/c100 0 2024-05-12T04:01:35.146 INFO:tasks.workunit.client.1.smithi033.stdout:5/928: read d3/d12/d29/f38 [100272,93451] 0 2024-05-12T04:01:35.147 INFO:tasks.workunit.client.0.smithi032.stdout:1/799: dwrite d1/dd/d24/f9c [0,4194304] 0 2024-05-12T04:01:35.148 INFO:tasks.workunit.client.1.smithi033.stdout:1/909: write d11/d17/d27/d36/d3f/d4a/f7a [1560215,32508] 0 2024-05-12T04:01:35.148 INFO:tasks.workunit.client.0.smithi032.stdout:1/800: readlink d1/d7/d30/d35/l97 0 2024-05-12T04:01:35.151 INFO:tasks.workunit.client.1.smithi033.stdout:5/929: creat d3/d94/d54/dc4/f12b x:0 0 0 2024-05-12T04:01:35.153 INFO:tasks.workunit.client.0.smithi032.stdout:6/739: creat d7/d9/d48/d67/d63/d64/d94/f101 x:0 0 0 2024-05-12T04:01:35.155 INFO:tasks.workunit.client.1.smithi033.stdout:1/910: creat d11/d1c/ddf/d4c/dab/dee/d124/f12b x:0 0 0 2024-05-12T04:01:35.156 INFO:tasks.workunit.client.0.smithi032.stdout:3/890: creat d15/d20/d26/d6a/d6e/f133 x:0 0 0 2024-05-12T04:01:35.156 INFO:tasks.workunit.client.1.smithi033.stdout:1/911: write d11/d1c/ddf/f74 [4354875,126332] 0 2024-05-12T04:01:35.157 INFO:tasks.workunit.client.0.smithi032.stdout:6/740: read - d7/d9/d48/d67/d25/ff8 zero size 2024-05-12T04:01:35.157 INFO:tasks.workunit.client.0.smithi032.stdout:5/915: dwrite f1 [0,4194304] 0 2024-05-12T04:01:35.158 INFO:tasks.workunit.client.1.smithi033.stdout:5/930: mknod d3/d94/d54/d57/d70/da3/dbe/d16/d45/d62/d7f/dcf/dfd/c12c 0 2024-05-12T04:01:35.158 INFO:tasks.workunit.client.1.smithi033.stdout:1/912: read d11/d1c/ddf/d4c/dab/dee/f103 [31755,8050] 0 2024-05-12T04:01:35.160 INFO:tasks.workunit.client.1.smithi033.stdout:1/913: write d11/d17/d27/d36/d3f/d4a/dc6/de1/d114/d87/d96/f10c [301577,59454] 0 2024-05-12T04:01:35.163 INFO:tasks.workunit.client.0.smithi032.stdout:9/691: mknod d0/d3/d4b/d85/dbe/cdd 0 2024-05-12T04:01:35.164 INFO:tasks.workunit.client.1.smithi033.stdout:5/931: mkdir d3/d94/db8/d12a/d12d 0 2024-05-12T04:01:35.165 INFO:tasks.workunit.client.0.smithi032.stdout:3/891: mkdir d15/d24/d93/d134 0 2024-05-12T04:01:35.166 INFO:tasks.workunit.client.1.smithi033.stdout:1/914: creat d11/d17/d27/d36/d3f/d4a/dc6/de1/f12c x:0 0 0 2024-05-12T04:01:35.173 INFO:tasks.workunit.client.0.smithi032.stdout:1/801: mkdir d1/d7/d30/d35/d77/de7/d10c 0 2024-05-12T04:01:35.175 INFO:tasks.workunit.client.1.smithi033.stdout:5/932: unlink d3/d12/d29/d2b/d4f/c6f 0 2024-05-12T04:01:35.176 INFO:tasks.workunit.client.0.smithi032.stdout:5/916: dwrite d4/d7/d14/d26/d33/d72/dc9/dec/f116 [0,4194304] 0 2024-05-12T04:01:35.176 INFO:tasks.workunit.client.0.smithi032.stdout:9/692: fsync d0/d3/d6/d7/d2a/d46/f12 0 2024-05-12T04:01:35.182 INFO:tasks.workunit.client.0.smithi032.stdout:4/733: truncate d0/d3/d12/faa 104027 0 2024-05-12T04:01:35.182 INFO:tasks.workunit.client.1.smithi033.stdout:5/933: truncate d3/fb 764290 0 2024-05-12T04:01:35.184 INFO:tasks.workunit.client.1.smithi033.stdout:5/934: write d3/d94/d54/f111 [845588,4099] 0 2024-05-12T04:01:35.185 INFO:tasks.workunit.client.1.smithi033.stdout:5/935: fdatasync d3/d12/d29/d2b/d8e/f125 0 2024-05-12T04:01:35.186 INFO:tasks.workunit.client.0.smithi032.stdout:1/802: stat d1/d7/d30/d18/c55 0 2024-05-12T04:01:35.186 INFO:tasks.workunit.client.0.smithi032.stdout:4/734: dread d0/d3/d12/d1d/d6b/dc2/dc4/fd5 [0,4194304] 0 2024-05-12T04:01:35.187 INFO:tasks.workunit.client.1.smithi033.stdout:5/936: fsync d3/d94/d54/dc4/f12b 0 2024-05-12T04:01:35.187 INFO:tasks.workunit.client.0.smithi032.stdout:5/917: dread d4/d7/d14/d39/d2b/d6e/f105 [0,4194304] 0 2024-05-12T04:01:35.189 INFO:tasks.workunit.client.0.smithi032.stdout:6/741: write d7/d9/d48/d67/d8d/dbc/de2/f81 [3853898,107972] 0 2024-05-12T04:01:35.189 INFO:tasks.workunit.client.0.smithi032.stdout:5/918: readlink d4/d7/d14/d26/d33/d4a/d38/d79/ld1 0 2024-05-12T04:01:35.191 INFO:tasks.workunit.client.1.smithi033.stdout:5/937: mknod d3/d94/d54/dc4/c12e 0 2024-05-12T04:01:35.191 INFO:tasks.workunit.client.0.smithi032.stdout:5/919: read d4/d7/d14/d26/d33/d4a/f51 [168637,66675] 0 2024-05-12T04:01:35.192 INFO:tasks.workunit.client.0.smithi032.stdout:5/920: dread - d4/d7/d14/d26/d33/d4a/d11d/d12c/f13a zero size 2024-05-12T04:01:35.193 INFO:tasks.workunit.client.0.smithi032.stdout:5/921: chown d4/d7/d14/f1d 119 1 2024-05-12T04:01:35.194 INFO:tasks.workunit.client.1.smithi033.stdout:5/938: creat d3/d94/dc8/f12f x:0 0 0 2024-05-12T04:01:35.198 INFO:tasks.workunit.client.0.smithi032.stdout:1/803: truncate d1/d7/d30/f52 4336121 0 2024-05-12T04:01:35.200 INFO:tasks.workunit.client.1.smithi033.stdout:1/915: dwrite d11/d17/d27/d36/fec [0,4194304] 0 2024-05-12T04:01:35.202 INFO:tasks.workunit.client.0.smithi032.stdout:3/892: dwrite d15/d20/d22/d7a/f85 [0,4194304] 0 2024-05-12T04:01:35.205 INFO:tasks.workunit.client.0.smithi032.stdout:3/893: dread - d15/d24/d2a/d8c/db1/dc8/de4/f126 zero size 2024-05-12T04:01:35.206 INFO:tasks.workunit.client.1.smithi033.stdout:1/916: write d11/d17/d27/d36/d3f/d4a/dc6/de1/d114/d87/fdb [1155400,20018] 0 2024-05-12T04:01:35.206 INFO:tasks.workunit.client.1.smithi033.stdout:5/939: dwrite d3/d12/d25/db0/ff7 [0,4194304] 0 2024-05-12T04:01:35.209 INFO:tasks.workunit.client.1.smithi033.stdout:5/940: chown d3/d94/d54/d57/d70/da3/dbe/d16/d78/d8b/fd8 183030 1 2024-05-12T04:01:35.219 INFO:tasks.workunit.client.0.smithi032.stdout:4/735: unlink d0/d3/df/c21 0 2024-05-12T04:01:35.219 INFO:tasks.workunit.client.1.smithi033.stdout:1/917: creat d11/d17/d27/d36/d3f/d4a/dc6/de1/d114/d87/d96/f12d x:0 0 0 2024-05-12T04:01:35.222 INFO:tasks.workunit.client.1.smithi033.stdout:1/918: write d11/d17/d27/d36/d3f/d4a/dc6/de1/d114/d87/d96/f100 [1937857,99819] 0 2024-05-12T04:01:35.224 INFO:tasks.workunit.client.0.smithi032.stdout:5/922: fsync d4/d7/d14/d26/d33/d72/dc9/dec/f12b 0 2024-05-12T04:01:35.224 INFO:tasks.workunit.client.0.smithi032.stdout:6/742: write d7/d9/d48/d67/d25/f29 [39578,6623] 0 2024-05-12T04:01:35.229 INFO:tasks.workunit.client.1.smithi033.stdout:1/919: dwrite d11/d17/d27/d36/d3f/d4a/dc6/de1/d114/d87/fdb [0,4194304] 0 2024-05-12T04:01:35.229 INFO:tasks.workunit.client.0.smithi032.stdout:9/693: rename d0/d3/d4b/d6f/c9c to d0/d3/d6/d7/cde 0 2024-05-12T04:01:35.231 INFO:tasks.workunit.client.0.smithi032.stdout:3/894: mknod d15/d20/d26/d6a/d6e/d7d/d9a/c135 0 2024-05-12T04:01:35.233 INFO:tasks.workunit.client.1.smithi033.stdout:5/941: truncate d3/d94/db8/f120 4307294 0 2024-05-12T04:01:35.235 INFO:tasks.workunit.client.1.smithi033.stdout:1/920: mknod d11/d17/d9e/c12e 0 2024-05-12T04:01:35.239 INFO:tasks.workunit.client.1.smithi033.stdout:5/942: rename d3/d94/d54/d57/d70/da3/dbe/d16/d45/d62/d7f/d84/l9b to d3/d94/d54/d57/d70/da3/dbe/d16/d115/l130 0 2024-05-12T04:01:35.243 INFO:tasks.workunit.client.0.smithi032.stdout:6/743: symlink d7/d97/l102 0 2024-05-12T04:01:35.245 INFO:tasks.workunit.client.0.smithi032.stdout:3/895: dread d15/d1b/d43/f10f [0,4194304] 0 2024-05-12T04:01:35.245 INFO:tasks.workunit.client.0.smithi032.stdout:5/923: fsync d4/d7/d14/d26/dc6/fc8 0 2024-05-12T04:01:35.246 INFO:tasks.workunit.client.1.smithi033.stdout:1/921: fdatasync d11/d17/d27/d36/d3f/d4a/fa5 0 2024-05-12T04:01:35.249 INFO:tasks.workunit.client.0.smithi032.stdout:1/804: unlink d1/d7/d30/d35/c39 0 2024-05-12T04:01:35.253 INFO:tasks.workunit.client.0.smithi032.stdout:8/818: sync 2024-05-12T04:01:35.253 INFO:tasks.workunit.client.0.smithi032.stdout:7/725: sync 2024-05-12T04:01:35.258 INFO:tasks.workunit.client.1.smithi033.stdout:1/922: mkdir d11/d17/d27/d76/dfc/d12f 0 2024-05-12T04:01:35.259 INFO:tasks.workunit.client.1.smithi033.stdout:5/943: write d3/d12/d29/d2b/fd1 [4224558,102255] 0 2024-05-12T04:01:35.263 INFO:tasks.workunit.client.0.smithi032.stdout:9/694: write d0/d3/d6/d58/f5e [4840,17871] 0 2024-05-12T04:01:35.264 INFO:tasks.workunit.client.1.smithi033.stdout:1/923: dwrite d11/d17/d27/d36/d3f/f61 [4194304,4194304] 0 2024-05-12T04:01:35.264 INFO:tasks.workunit.client.0.smithi032.stdout:1/805: read - d1/d7/fdd zero size 2024-05-12T04:01:35.269 INFO:tasks.workunit.client.0.smithi032.stdout:8/819: fdatasync d0/d7/f6c 0 2024-05-12T04:01:35.271 INFO:tasks.workunit.client.1.smithi033.stdout:1/924: symlink d11/d10a/l130 0 2024-05-12T04:01:35.275 INFO:tasks.workunit.client.1.smithi033.stdout:5/944: write d3/f92 [3834971,79365] 0 2024-05-12T04:01:35.276 INFO:tasks.workunit.client.0.smithi032.stdout:4/736: link d0/d3/d12/d1d/f2a d0/d3/df/d4e/fdd 0 2024-05-12T04:01:35.277 INFO:tasks.workunit.client.0.smithi032.stdout:6/744: mknod d7/d9/d48/d67/c103 0 2024-05-12T04:01:35.284 INFO:tasks.workunit.client.1.smithi033.stdout:1/925: rename d11/d17/d84/da3/fd6 to d11/d17/d27/f131 0 2024-05-12T04:01:35.287 INFO:tasks.workunit.client.0.smithi032.stdout:1/806: dread d1/d7/f78 [0,4194304] 0 2024-05-12T04:01:35.287 INFO:tasks.workunit.client.1.smithi033.stdout:5/945: link d3/d94/d54/d57/d70/da3/dbe/d16/d45/d62/fad d3/d12/d29/f131 0 2024-05-12T04:01:35.288 INFO:tasks.workunit.client.0.smithi032.stdout:1/807: read d1/d7/f78 [802617,18267] 0 2024-05-12T04:01:35.289 INFO:tasks.workunit.client.0.smithi032.stdout:1/808: readlink d1/d7/d30/d18/d93/lf0 0 2024-05-12T04:01:35.292 INFO:tasks.workunit.client.1.smithi033.stdout:5/946: dwrite d3/f92 [0,4194304] 0 2024-05-12T04:01:35.299 INFO:tasks.workunit.client.1.smithi033.stdout:5/947: fsync d3/d94/d54/d57/f89 0 2024-05-12T04:01:35.300 INFO:tasks.workunit.client.1.smithi033.stdout:5/948: fdatasync d3/d94/dc8/f12f 0 2024-05-12T04:01:35.300 INFO:tasks.workunit.client.1.smithi033.stdout:1/926: dread d11/d17/d27/da6/fc2 [0,4194304] 0 2024-05-12T04:01:35.301 INFO:tasks.workunit.client.0.smithi032.stdout:5/924: creat d4/d7/d14/d39/d2b/f142 x:0 0 0 2024-05-12T04:01:35.301 INFO:tasks.workunit.client.1.smithi033.stdout:5/949: stat d3/d94/d85 0 2024-05-12T04:01:35.301 INFO:tasks.workunit.client.0.smithi032.stdout:3/896: truncate d15/d1b/d43/f10f 1013011 0 2024-05-12T04:01:35.306 INFO:tasks.workunit.client.0.smithi032.stdout:1/809: dread - d1/d6a/fcf zero size 2024-05-12T04:01:35.309 INFO:tasks.workunit.client.0.smithi032.stdout:7/726: creat dd/d1b/d43/fe6 x:0 0 0 2024-05-12T04:01:35.312 INFO:tasks.workunit.client.0.smithi032.stdout:4/737: mkdir d0/dde 0 2024-05-12T04:01:35.314 INFO:tasks.workunit.client.1.smithi033.stdout:5/950: stat d3/d94/d54/d57/d70/da3/dbe/d8c/cbd 0 2024-05-12T04:01:35.316 INFO:tasks.workunit.client.0.smithi032.stdout:3/897: creat d15/dec/f136 x:0 0 0 2024-05-12T04:01:35.319 INFO:tasks.workunit.client.0.smithi032.stdout:9/695: rename d0/d3/d6/d7/d2a/d46/d13/cc6 to d0/d3/cdf 0 2024-05-12T04:01:35.319 INFO:tasks.workunit.client.1.smithi033.stdout:5/951: dread d3/d94/d54/d57/d70/da3/dbe/d16/f20 [12582912,4194304] 0 2024-05-12T04:01:35.320 INFO:tasks.workunit.client.1.smithi033.stdout:1/927: dwrite d11/d17/d27/d36/d3f/d4a/dc6/de1/d114/d87/d96/dd8/f123 [0,4194304] 0 2024-05-12T04:01:35.321 INFO:tasks.workunit.client.1.smithi033.stdout:5/952: write d3/d94/f49 [4242904,105250] 0 2024-05-12T04:01:35.322 INFO:tasks.workunit.client.1.smithi033.stdout:1/928: dread - d11/d17/d27/d36/d3f/f107 zero size 2024-05-12T04:01:35.323 INFO:tasks.workunit.client.1.smithi033.stdout:5/953: fdatasync d3/d94/dc8/f12f 0 2024-05-12T04:01:35.325 INFO:tasks.workunit.client.0.smithi032.stdout:5/925: mkdir d4/d7/d143 0 2024-05-12T04:01:35.328 INFO:tasks.workunit.client.0.smithi032.stdout:7/727: write dd/f1a [2198646,62285] 0 2024-05-12T04:01:35.328 INFO:tasks.workunit.client.0.smithi032.stdout:7/728: fsync dd/f13 0 2024-05-12T04:01:35.330 INFO:tasks.workunit.client.0.smithi032.stdout:7/729: chown dd/d1b/d1e/c53 224458661 1 2024-05-12T04:01:35.331 INFO:tasks.workunit.client.1.smithi033.stdout:5/954: creat d3/d94/dc8/f132 x:0 0 0 2024-05-12T04:01:35.332 INFO:tasks.workunit.client.1.smithi033.stdout:1/929: dwrite d11/d17/d27/f31 [0,4194304] 0 2024-05-12T04:01:35.340 INFO:tasks.workunit.client.1.smithi033.stdout:5/955: dwrite d3/d94/d54/dc4/f12b [0,4194304] 0 2024-05-12T04:01:35.340 INFO:tasks.workunit.client.0.smithi032.stdout:9/696: creat d0/d3/d4b/d85/dbe/fe0 x:0 0 0 2024-05-12T04:01:35.348 INFO:tasks.workunit.client.1.smithi033.stdout:5/956: dread d3/d94/f61 [0,4194304] 0 2024-05-12T04:01:35.352 INFO:tasks.workunit.client.1.smithi033.stdout:1/930: dread d11/d17/d27/d36/d3f/fd0 [0,4194304] 0 2024-05-12T04:01:35.353 INFO:tasks.workunit.client.1.smithi033.stdout:5/957: readlink d3/d94/d54/d57/d70/da3/dbe/d8c/lf6 0 2024-05-12T04:01:35.356 INFO:tasks.workunit.client.0.smithi032.stdout:4/738: write d0/d3/d12/d1d/d63/f99 [255410,113442] 0 2024-05-12T04:01:35.362 INFO:tasks.workunit.client.1.smithi033.stdout:5/958: unlink d3/d94/d54/d57/d70/c82 0 2024-05-12T04:01:35.368 INFO:tasks.workunit.client.1.smithi033.stdout:5/959: fdatasync d3/d94/d54/d57/d70/da3/dbe/d16/d45/d62/fad 0 2024-05-12T04:01:35.370 INFO:tasks.workunit.client.1.smithi033.stdout:5/960: stat d3/d94/d54/d57/d70/da3/dbe/d16/d78/fe7 0 2024-05-12T04:01:35.376 INFO:tasks.workunit.client.1.smithi033.stdout:1/931: write d11/d1c/d43/f4d [2637975,130325] 0 2024-05-12T04:01:35.377 INFO:tasks.workunit.client.1.smithi033.stdout:5/961: dread d3/fb [0,4194304] 0 2024-05-12T04:01:35.385 INFO:tasks.workunit.client.1.smithi033.stdout:5/962: mkdir d3/d94/d54/d57/d73/d133 0 2024-05-12T04:01:35.393 INFO:tasks.workunit.client.1.smithi033.stdout:5/963: mknod d3/d94/d54/d57/d70/da3/dbe/d16/d45/d62/c134 0 2024-05-12T04:01:35.400 INFO:tasks.workunit.client.1.smithi033.stdout:1/932: dwrite d11/d17/d27/f52 [0,4194304] 0 2024-05-12T04:01:35.405 INFO:tasks.workunit.client.1.smithi033.stdout:1/933: fdatasync d11/d17/d27/d36/d3f/fd0 0 2024-05-12T04:01:35.406 INFO:tasks.workunit.client.0.smithi032.stdout:0/819: sync 2024-05-12T04:01:35.410 INFO:tasks.workunit.client.0.smithi032.stdout:3/898: mkdir d15/d20/d26/d6a/d137 0 2024-05-12T04:01:35.410 INFO:tasks.workunit.client.0.smithi032.stdout:6/745: link d7/d9/d48/d67/d25/f2a d7/d9/d48/d67/f104 0 2024-05-12T04:01:35.415 INFO:tasks.workunit.client.1.smithi033.stdout:1/934: rename d11/d17/d84/de7/f101 to d11/d1c/ddf/d4c/dab/dee/f132 0 2024-05-12T04:01:35.417 INFO:tasks.workunit.client.0.smithi032.stdout:5/926: symlink d4/l144 0 2024-05-12T04:01:35.421 INFO:tasks.workunit.client.1.smithi033.stdout:1/935: rename d11/d17/d27/d36/fec to d11/d17/d27/d36/d3f/d4a/dc6/de1/d114/d87/d108/f133 0 2024-05-12T04:01:35.422 INFO:tasks.workunit.client.0.smithi032.stdout:4/739: dread - d0/d3/d12/d1d/f9e zero size 2024-05-12T04:01:35.425 INFO:tasks.workunit.client.0.smithi032.stdout:1/810: getdents d1/d7/d49/d79 0 2024-05-12T04:01:35.426 INFO:tasks.workunit.client.0.smithi032.stdout:7/730: write dd/d1b/d43/d74/f84 [3113988,20897] 0 2024-05-12T04:01:35.428 INFO:tasks.workunit.client.0.smithi032.stdout:0/820: truncate df/df2/fbd 817627 0 2024-05-12T04:01:35.431 INFO:tasks.workunit.client.1.smithi033.stdout:1/936: rename d11/d17/d27/d76/dfc/d12f to d11/d17/d27/d36/d3f/d4a/dc6/d134 0 2024-05-12T04:01:35.432 INFO:tasks.workunit.client.0.smithi032.stdout:3/899: rename d15/d24/d2a/cc6 to d15/d24/d2a/d8c/c138 0 2024-05-12T04:01:35.434 INFO:tasks.workunit.client.0.smithi032.stdout:5/927: fdatasync d4/d7/d14/d39/d1b/f76 0 2024-05-12T04:01:35.435 INFO:tasks.workunit.client.0.smithi032.stdout:5/928: stat d4/d7/d14/d26/cae 0 2024-05-12T04:01:35.436 INFO:tasks.workunit.client.0.smithi032.stdout:9/697: mkdir d0/d84/da6/de1 0 2024-05-12T04:01:35.440 INFO:tasks.workunit.client.0.smithi032.stdout:4/740: creat d0/d3/d31/d80/fdf x:0 0 0 2024-05-12T04:01:35.441 INFO:tasks.workunit.client.1.smithi033.stdout:1/937: symlink d11/d1c/l135 0 2024-05-12T04:01:35.445 INFO:tasks.workunit.client.0.smithi032.stdout:3/900: symlink d15/d20/d22/d4f/l139 0 2024-05-12T04:01:35.448 INFO:tasks.workunit.client.0.smithi032.stdout:9/698: creat d0/d3/d4b/d6f/fe2 x:0 0 0 2024-05-12T04:01:35.449 INFO:tasks.workunit.client.0.smithi032.stdout:4/741: mkdir d0/d3/d12/d1d/d6b/db7/de0 0 2024-05-12T04:01:35.451 INFO:tasks.workunit.client.0.smithi032.stdout:7/731: creat dd/d1b/d1e/d45/d55/d72/da0/dbd/db2/fe7 x:0 0 0 2024-05-12T04:01:35.452 INFO:tasks.workunit.client.0.smithi032.stdout:0/821: mkdir df/d10/d3d/d4a/d9a/d9d/d106/d113 0 2024-05-12T04:01:35.453 INFO:tasks.workunit.client.0.smithi032.stdout:6/746: creat d7/d9/f105 x:0 0 0 2024-05-12T04:01:35.454 INFO:tasks.workunit.client.1.smithi033.stdout:1/938: getdents d11/d17/d27/d36/d3f/d4a/dc6/de1/d114 0 2024-05-12T04:01:35.454 INFO:tasks.workunit.client.0.smithi032.stdout:6/747: write d7/d9/d5c/d8b/da1/fae [4111684,6117] 0 2024-05-12T04:01:35.456 INFO:tasks.workunit.client.0.smithi032.stdout:6/748: chown d7/d9/d48/d67/d8d/dbc/f80 4 1 2024-05-12T04:01:35.458 INFO:tasks.workunit.client.0.smithi032.stdout:4/742: symlink d0/d3/d12/d1d/dc5/le1 0 2024-05-12T04:01:35.460 INFO:tasks.workunit.client.0.smithi032.stdout:7/732: write dd/d1b/d1e/f20 [979301,46135] 0 2024-05-12T04:01:35.464 INFO:tasks.workunit.client.0.smithi032.stdout:3/901: mknod d15/d20/c13a 0 2024-05-12T04:01:35.465 INFO:tasks.workunit.client.0.smithi032.stdout:9/699: rmdir d0/d3/d4b/d6f 39 2024-05-12T04:01:35.465 INFO:tasks.workunit.client.0.smithi032.stdout:3/902: write d15/d24/f90 [789819,79258] 0 2024-05-12T04:01:35.471 INFO:tasks.workunit.client.0.smithi032.stdout:7/733: dread dd/d1b/d1e/f20 [0,4194304] 0 2024-05-12T04:01:35.473 INFO:tasks.workunit.client.0.smithi032.stdout:5/929: write d4/d7/d14/d39/d2b/f32 [2916248,79302] 0 2024-05-12T04:01:35.475 INFO:tasks.workunit.client.1.smithi033.stdout:1/939: write d11/f19 [3556726,12972] 0 2024-05-12T04:01:35.476 INFO:tasks.workunit.client.0.smithi032.stdout:9/700: dread - d0/d84/d8a/f94 zero size 2024-05-12T04:01:35.477 INFO:tasks.workunit.client.0.smithi032.stdout:6/749: truncate d7/d9/d48/d67/d63/d64/d94/de4/f14 1972456 0 2024-05-12T04:01:35.479 INFO:tasks.workunit.client.0.smithi032.stdout:7/734: creat dd/d1b/d1e/de3/d9a/d9c/fe8 x:0 0 0 2024-05-12T04:01:35.480 INFO:tasks.workunit.client.0.smithi032.stdout:0/822: link df/fc1 df/d10/d3b/f114 0 2024-05-12T04:01:35.480 INFO:tasks.workunit.client.0.smithi032.stdout:0/823: chown df/d22/d58/c5d 883509 1 2024-05-12T04:01:35.483 INFO:tasks.workunit.client.1.smithi033.stdout:1/940: rename d11/d17/d27/f2b to d11/d17/d27/d36/d3f/d4a/dc6/de1/d114/d87/d108/f136 0 2024-05-12T04:01:35.484 INFO:tasks.workunit.client.0.smithi032.stdout:0/824: dread df/d10/f13 [8388608,4194304] 0 2024-05-12T04:01:35.484 INFO:tasks.workunit.client.0.smithi032.stdout:4/743: creat d0/d3/fe2 x:0 0 0 2024-05-12T04:01:35.485 INFO:tasks.workunit.client.0.smithi032.stdout:5/930: symlink d4/d7/d14/d26/d33/d4a/d38/d79/l145 0 2024-05-12T04:01:35.486 INFO:tasks.workunit.client.0.smithi032.stdout:4/744: chown d0/d3/d12/d1d/d63/f97 1655668727 1 2024-05-12T04:01:35.488 INFO:tasks.workunit.client.0.smithi032.stdout:9/701: truncate d0/d3/d6/d7/f7d 185610 0 2024-05-12T04:01:35.491 INFO:tasks.workunit.client.0.smithi032.stdout:7/735: chown dd/d1b/d43/c70 2214 1 2024-05-12T04:01:35.493 INFO:tasks.workunit.client.0.smithi032.stdout:3/903: truncate d15/d20/d22/d7a/fb2 651963 0 2024-05-12T04:01:35.494 INFO:tasks.workunit.client.1.smithi033.stdout:1/941: creat d11/f137 x:0 0 0 2024-05-12T04:01:35.497 INFO:tasks.workunit.client.0.smithi032.stdout:4/745: creat d0/d3/d31/fe3 x:0 0 0 2024-05-12T04:01:35.501 INFO:tasks.workunit.client.0.smithi032.stdout:9/702: symlink d0/d3/d4b/d85/dbe/d82/le3 0 2024-05-12T04:01:35.502 INFO:tasks.workunit.client.0.smithi032.stdout:6/750: dread d7/fa [0,4194304] 0 2024-05-12T04:01:35.507 INFO:tasks.workunit.client.1.smithi033.stdout:1/942: creat d11/d17/d84/da3/df7/f138 x:0 0 0 2024-05-12T04:01:35.507 INFO:tasks.workunit.client.1.smithi033.stdout:1/943: dread - d11/d17/d84/f11d zero size 2024-05-12T04:01:35.508 INFO:tasks.workunit.client.0.smithi032.stdout:3/904: symlink d15/d20/d22/d7a/dbe/l13b 0 2024-05-12T04:01:35.509 INFO:tasks.workunit.client.0.smithi032.stdout:5/931: write d4/d7/d14/d39/d2b/ff6 [6628652,68762] 0 2024-05-12T04:01:35.510 INFO:tasks.workunit.client.0.smithi032.stdout:5/932: stat d4/d7/d14/d26/d33/f61 0 2024-05-12T04:01:35.510 INFO:tasks.workunit.client.0.smithi032.stdout:4/746: creat d0/d3/d12/d1d/dc5/fe4 x:0 0 0 2024-05-12T04:01:35.513 INFO:tasks.workunit.client.0.smithi032.stdout:4/747: write d0/d3/d12/fa2 [950191,41592] 0 2024-05-12T04:01:35.514 INFO:tasks.workunit.client.0.smithi032.stdout:4/748: read d0/f93 [551496,8700] 0 2024-05-12T04:01:35.516 INFO:tasks.workunit.client.0.smithi032.stdout:0/825: write df/d10/d3d/d4a/f38 [2423589,115874] 0 2024-05-12T04:01:35.521 INFO:tasks.workunit.client.0.smithi032.stdout:7/736: write fb [3026913,24448] 0 2024-05-12T04:01:35.525 INFO:tasks.workunit.client.0.smithi032.stdout:3/905: write d15/d20/fd6 [550110,80390] 0 2024-05-12T04:01:35.526 INFO:tasks.workunit.client.0.smithi032.stdout:5/933: truncate d4/d7/d14/d26/d33/d4a/ff1 1024583 0 2024-05-12T04:01:35.528 INFO:tasks.workunit.client.0.smithi032.stdout:6/751: mkdir d7/d9/d48/d67/d1a/d20/d106 0 2024-05-12T04:01:35.529 INFO:tasks.workunit.client.0.smithi032.stdout:5/934: readlink d4/d7/d14/d26/d33/d72/l91 0 2024-05-12T04:01:35.530 INFO:tasks.workunit.client.0.smithi032.stdout:4/749: fdatasync d0/d3/d24/dae/fdc 0 2024-05-12T04:01:35.532 INFO:tasks.workunit.client.0.smithi032.stdout:0/826: creat df/d10/d3d/d4a/d9a/d9d/f115 x:0 0 0 2024-05-12T04:01:35.534 INFO:tasks.workunit.client.0.smithi032.stdout:3/906: dwrite d15/dc2/fc3 [0,4194304] 0 2024-05-12T04:01:35.537 INFO:tasks.workunit.client.0.smithi032.stdout:7/737: read dd/d1b/d1e/d45/d55/f9d [732656,8739] 0 2024-05-12T04:01:35.543 INFO:tasks.workunit.client.0.smithi032.stdout:0/827: chown df/df2/fbd 10187608 1 2024-05-12T04:01:35.546 INFO:tasks.workunit.client.0.smithi032.stdout:3/907: mkdir d15/d20/d22/d117/d13c 0 2024-05-12T04:01:35.550 INFO:tasks.workunit.client.0.smithi032.stdout:7/738: mknod dd/d1b/d1e/de3/d9a/d9c/dc2/daf/ce9 0 2024-05-12T04:01:35.551 INFO:tasks.workunit.client.0.smithi032.stdout:7/739: chown dd/d1b/d43/ldd 2040926 1 2024-05-12T04:01:35.553 INFO:tasks.workunit.client.0.smithi032.stdout:4/750: truncate d0/fb9 789016 0 2024-05-12T04:01:35.556 INFO:tasks.workunit.client.0.smithi032.stdout:0/828: dread - df/d30/d8d/d8f/fb3 zero size 2024-05-12T04:01:35.563 INFO:tasks.workunit.client.0.smithi032.stdout:2/793: sync 2024-05-12T04:01:35.563 INFO:tasks.workunit.client.0.smithi032.stdout:5/935: rmdir d4/d7/d14/d108/d126/d128 0 2024-05-12T04:01:35.565 INFO:tasks.workunit.client.0.smithi032.stdout:6/752: dwrite d7/d9/d48/faa [4194304,4194304] 0 2024-05-12T04:01:35.566 INFO:tasks.workunit.client.0.smithi032.stdout:6/753: fdatasync d7/d9/f105 0 2024-05-12T04:01:35.566 INFO:tasks.workunit.client.0.smithi032.stdout:0/829: mknod df/d30/d4e/c116 0 2024-05-12T04:01:35.574 INFO:tasks.workunit.client.0.smithi032.stdout:6/754: dwrite d7/d9/d48/d67/d1a/da0/dc6/fe5 [0,4194304] 0 2024-05-12T04:01:35.577 INFO:tasks.workunit.client.0.smithi032.stdout:2/794: fdatasync d1/d13/f18 0 2024-05-12T04:01:35.584 INFO:tasks.workunit.client.0.smithi032.stdout:7/740: link dd/d1b/d1e/l8a dd/d1b/d1e/de3/d9a/d9c/dc2/daf/lea 0 2024-05-12T04:01:35.591 INFO:tasks.workunit.client.0.smithi032.stdout:6/755: creat d7/d9/d48/d67/d63/d64/f107 x:0 0 0 2024-05-12T04:01:35.596 INFO:tasks.workunit.client.1.smithi033.stdout:5/964: sync 2024-05-12T04:01:35.597 INFO:tasks.workunit.client.0.smithi032.stdout:7/741: mknod dd/d1b/d1e/d45/d55/d72/da0/dbd/ceb 0 2024-05-12T04:01:35.598 INFO:tasks.workunit.client.0.smithi032.stdout:3/908: truncate d15/d20/d22/d7a/f88 355954 0 2024-05-12T04:01:35.600 INFO:tasks.workunit.client.0.smithi032.stdout:6/756: creat d7/d9/d48/d67/d8d/dbc/f108 x:0 0 0 2024-05-12T04:01:35.601 INFO:tasks.workunit.client.0.smithi032.stdout:8/820: sync 2024-05-12T04:01:35.604 INFO:tasks.workunit.client.0.smithi032.stdout:3/909: mkdir d15/d33/d3f/d13d 0 2024-05-12T04:01:35.607 INFO:tasks.workunit.client.0.smithi032.stdout:6/757: dwrite d7/d9/d5c/d8b/da1/fae [0,4194304] 0 2024-05-12T04:01:35.608 INFO:tasks.workunit.client.0.smithi032.stdout:7/742: rename dd/d1b/d43/ldd to dd/d1b/d1e/d45/d64/lec 0 2024-05-12T04:01:35.612 INFO:tasks.workunit.client.1.smithi033.stdout:5/965: creat d3/d94/d54/d57/d70/da3/dbe/d8c/f135 x:0 0 0 2024-05-12T04:01:35.621 INFO:tasks.workunit.client.1.smithi033.stdout:1/944: sync 2024-05-12T04:01:35.622 INFO:tasks.workunit.client.0.smithi032.stdout:5/936: dwrite d4/d7/d58/d5a/d99/f10b [0,4194304] 0 2024-05-12T04:01:35.623 INFO:tasks.workunit.client.1.smithi033.stdout:5/966: mkdir d3/d94/d54/d57/d70/da3/dbe/d16/dd7/d136 0 2024-05-12T04:01:35.626 INFO:tasks.workunit.client.0.smithi032.stdout:0/830: dwrite df/d30/d8d/d8f/f10e [0,4194304] 0 2024-05-12T04:01:35.630 INFO:tasks.workunit.client.1.smithi033.stdout:5/967: creat d3/d94/d54/d57/d70/da3/dbe/d16/d78/f137 x:0 0 0 2024-05-12T04:01:35.631 INFO:tasks.workunit.client.0.smithi032.stdout:0/831: truncate df/d30/dc2/ff8 567521 0 2024-05-12T04:01:35.632 INFO:tasks.workunit.client.0.smithi032.stdout:2/795: dwrite d1/d34/d3e/d67/fba [0,4194304] 0 2024-05-12T04:01:35.633 INFO:tasks.workunit.client.0.smithi032.stdout:0/832: stat df/d10/d3b/d75/f9b 0 2024-05-12T04:01:35.634 INFO:tasks.workunit.client.0.smithi032.stdout:3/910: creat d15/d20/d26/d6a/f13e x:0 0 0 2024-05-12T04:01:35.638 INFO:tasks.workunit.client.0.smithi032.stdout:5/937: fsync d4/f20 0 2024-05-12T04:01:35.641 INFO:tasks.workunit.client.0.smithi032.stdout:7/743: creat dd/d1b/d1e/d45/fed x:0 0 0 2024-05-12T04:01:35.643 INFO:tasks.workunit.client.1.smithi033.stdout:1/945: creat d11/d1c/ddf/f139 x:0 0 0 2024-05-12T04:01:35.644 INFO:tasks.workunit.client.0.smithi032.stdout:7/744: dread - dd/d1b/d43/d5e/d37/fd8 zero size 2024-05-12T04:01:35.645 INFO:tasks.workunit.client.0.smithi032.stdout:0/833: mkdir df/d30/d32/d81/d117 0 2024-05-12T04:01:35.647 INFO:tasks.workunit.client.0.smithi032.stdout:8/821: dwrite d0/d9/d1b/fdb [0,4194304] 0 2024-05-12T04:01:35.648 INFO:tasks.workunit.client.0.smithi032.stdout:3/911: write d15/d20/d26/d6a/d6e/d7d/d9a/fa7 [924716,8778] 0 2024-05-12T04:01:35.650 INFO:tasks.workunit.client.1.smithi033.stdout:1/946: link d11/d17/f6d d11/d1c/ddf/d118/f13a 0 2024-05-12T04:01:35.653 INFO:tasks.workunit.client.0.smithi032.stdout:6/758: getdents d7/d9/d48/d67/d1a 0 2024-05-12T04:01:35.656 INFO:tasks.workunit.client.0.smithi032.stdout:8/822: stat d0/d9/d1b/d27/d59/la9 0 2024-05-12T04:01:35.656 INFO:tasks.workunit.client.0.smithi032.stdout:3/912: rmdir d15/d20/d8b 39 2024-05-12T04:01:35.658 INFO:tasks.workunit.client.1.smithi033.stdout:1/947: truncate d11/d17/d27/d36/fb6 997589 0 2024-05-12T04:01:35.661 INFO:tasks.workunit.client.0.smithi032.stdout:6/759: symlink d7/d9/d48/d67/d1a/da0/l109 0 2024-05-12T04:01:35.664 INFO:tasks.workunit.client.1.smithi033.stdout:1/948: mknod d11/d17/d27/da6/dfe/c13b 0 2024-05-12T04:01:35.665 INFO:tasks.workunit.client.0.smithi032.stdout:3/913: mkdir d15/d20/d26/d6a/d6e/d7d/d13f 0 2024-05-12T04:01:35.666 INFO:tasks.workunit.client.0.smithi032.stdout:3/914: write d15/d20/d22/d7a/dbe/ffc [68653,18178] 0 2024-05-12T04:01:35.668 INFO:tasks.workunit.client.1.smithi033.stdout:1/949: symlink d11/l13c 0 2024-05-12T04:01:35.669 INFO:tasks.workunit.client.0.smithi032.stdout:8/823: mkdir d0/d4b/dad/d115 0 2024-05-12T04:01:35.673 INFO:tasks.workunit.client.0.smithi032.stdout:3/915: read d15/d33/d3f/fb3 [980751,12805] 0 2024-05-12T04:01:35.677 INFO:tasks.workunit.client.1.smithi033.stdout:1/950: link d11/d17/fd7 d11/d17/d9e/f13d 0 2024-05-12T04:01:35.677 INFO:tasks.workunit.client.0.smithi032.stdout:2/796: write d1/d34/d96/d41/d58/fdd [672010,125982] 0 2024-05-12T04:01:35.678 INFO:tasks.workunit.client.0.smithi032.stdout:6/760: creat d7/d9/d48/f10a x:0 0 0 2024-05-12T04:01:35.679 INFO:tasks.workunit.client.0.smithi032.stdout:3/916: mknod d15/d24/d2a/dc7/c140 0 2024-05-12T04:01:35.681 INFO:tasks.workunit.client.1.smithi033.stdout:1/951: creat d11/d17/d9e/f13e x:0 0 0 2024-05-12T04:01:35.682 INFO:tasks.workunit.client.0.smithi032.stdout:3/917: read d15/d33/d3f/fb7 [2597141,117522] 0 2024-05-12T04:01:35.684 INFO:tasks.workunit.client.0.smithi032.stdout:2/797: fsync d1/d34/d96/fe8 0 2024-05-12T04:01:35.685 INFO:tasks.workunit.client.0.smithi032.stdout:5/938: dwrite d4/d7/d14/d26/d33/d5f/f87 [0,4194304] 0 2024-05-12T04:01:35.685 INFO:tasks.workunit.client.0.smithi032.stdout:2/798: chown d1/f24 1037590 1 2024-05-12T04:01:35.687 INFO:tasks.workunit.client.0.smithi032.stdout:6/761: unlink d7/d9/d48/d67/d63/d64/f107 0 2024-05-12T04:01:35.689 INFO:tasks.workunit.client.1.smithi033.stdout:1/952: symlink d11/d17/d27/d76/d93/l13f 0 2024-05-12T04:01:35.690 INFO:tasks.workunit.client.0.smithi032.stdout:7/745: write dd/d1b/d1e/de3/f7d [1549894,24206] 0 2024-05-12T04:01:35.697 INFO:tasks.workunit.client.0.smithi032.stdout:0/834: dwrite df/d10/f3c [0,4194304] 0 2024-05-12T04:01:35.697 INFO:tasks.workunit.client.1.smithi033.stdout:1/953: mkdir d11/d1c/ddf/d118/d140 0 2024-05-12T04:01:35.698 INFO:tasks.workunit.client.0.smithi032.stdout:0/835: stat df/d22/d58 0 2024-05-12T04:01:35.703 INFO:tasks.workunit.client.0.smithi032.stdout:2/799: fsync d1/d34/d96/d41/f3a 0 2024-05-12T04:01:35.704 INFO:tasks.workunit.client.0.smithi032.stdout:8/824: write d0/d9/d1b/d66/d75/f9b [4143268,82091] 0 2024-05-12T04:01:35.706 INFO:tasks.workunit.client.0.smithi032.stdout:6/762: unlink d7/d9/d5c/f98 0 2024-05-12T04:01:35.708 INFO:tasks.workunit.client.1.smithi033.stdout:1/954: mkdir d11/d17/d27/d141 0 2024-05-12T04:01:35.710 INFO:tasks.workunit.client.0.smithi032.stdout:7/746: rmdir dd/d6c/dc3/dcd 39 2024-05-12T04:01:35.713 INFO:tasks.workunit.client.1.smithi033.stdout:1/955: rmdir d11/d17/d84/de7 39 2024-05-12T04:01:35.715 INFO:tasks.workunit.client.0.smithi032.stdout:0/836: symlink df/d30/d4e/da5/daf/l118 0 2024-05-12T04:01:35.716 INFO:tasks.workunit.client.1.smithi033.stdout:1/956: write d11/d17/d27/d36/d3f/d4a/dc6/de1/d114/d87/d96/dd8/f123 [1886587,98129] 0 2024-05-12T04:01:35.717 INFO:tasks.workunit.client.1.smithi033.stdout:1/957: dread - d11/d17/d27/d36/d3f/d4a/dc6/de1/d114/d87/da4/fda zero size 2024-05-12T04:01:35.721 INFO:tasks.workunit.client.0.smithi032.stdout:2/800: dread - d1/d34/d96/d41/d2c/dd4/f102 zero size 2024-05-12T04:01:35.722 INFO:tasks.workunit.client.1.smithi033.stdout:1/958: mkdir d11/d142 0 2024-05-12T04:01:35.724 INFO:tasks.workunit.client.1.smithi033.stdout:1/959: dread - d11/d1c/ddf/f139 zero size 2024-05-12T04:01:35.724 INFO:tasks.workunit.client.0.smithi032.stdout:8/825: creat d0/d9/d1b/d66/f116 x:0 0 0 2024-05-12T04:01:35.725 INFO:tasks.workunit.client.0.smithi032.stdout:6/763: readlink d7/d9/d48/d67/d63/d64/d94/de4/l75 0 2024-05-12T04:01:35.726 INFO:tasks.workunit.client.0.smithi032.stdout:0/837: mkdir df/d10/d3d/d4a/d9a/d9d/d119 0 2024-05-12T04:01:35.728 INFO:tasks.workunit.client.0.smithi032.stdout:0/838: write df/d10/d3d/d4a/f2d [3346121,118223] 0 2024-05-12T04:01:35.728 INFO:tasks.workunit.client.1.smithi033.stdout:1/960: symlink d11/d17/d27/d36/d3f/d4a/dc6/de1/d114/l143 0 2024-05-12T04:01:35.729 INFO:tasks.workunit.client.0.smithi032.stdout:3/918: link d15/d24/d2a/d8c/db1/dc8/de4/f129 d15/d20/d22/d7a/dbe/d121/f141 0 2024-05-12T04:01:35.732 INFO:tasks.workunit.client.0.smithi032.stdout:2/801: creat d1/d34/d3e/dee/f107 x:0 0 0 2024-05-12T04:01:35.733 INFO:tasks.workunit.client.1.smithi033.stdout:1/961: mkdir d11/d17/d27/d76/dfc/d144 0 2024-05-12T04:01:35.734 INFO:tasks.workunit.client.0.smithi032.stdout:8/826: rmdir d0/d9/d13 39 2024-05-12T04:01:35.735 INFO:tasks.workunit.client.0.smithi032.stdout:3/919: dwrite d15/d24/d2a/d5b/f71 [0,4194304] 0 2024-05-12T04:01:35.737 INFO:tasks.workunit.client.1.smithi033.stdout:1/962: mknod d11/d17/d27/d36/d3f/d4a/dc6/de1/d114/c145 0 2024-05-12T04:01:35.739 INFO:tasks.workunit.client.0.smithi032.stdout:0/839: dread df/d10/d3d/d83/f108 [0,4194304] 0 2024-05-12T04:01:35.746 INFO:tasks.workunit.client.1.smithi033.stdout:5/968: sync 2024-05-12T04:01:35.747 INFO:tasks.workunit.client.0.smithi032.stdout:3/920: creat d15/d24/d2a/d8c/db1/dc8/de4/f142 x:0 0 0 2024-05-12T04:01:35.750 INFO:tasks.workunit.client.0.smithi032.stdout:5/939: link d4/d7/l3a d4/d7/d14/l146 0 2024-05-12T04:01:35.752 INFO:tasks.workunit.client.1.smithi033.stdout:5/969: mknod d3/d94/d54/d57/d73/c138 0 2024-05-12T04:01:35.754 INFO:tasks.workunit.client.0.smithi032.stdout:8/827: fsync d0/d9/d1b/d27/d88/f40 0 2024-05-12T04:01:35.755 INFO:tasks.workunit.client.0.smithi032.stdout:0/840: creat df/d10/d3d/d4a/d9a/d9d/d106/d113/f11a x:0 0 0 2024-05-12T04:01:35.756 INFO:tasks.workunit.client.0.smithi032.stdout:3/921: chown d15/d20/d26/d6a/l6f 948469 1 2024-05-12T04:01:35.757 INFO:tasks.workunit.client.1.smithi033.stdout:1/963: dwrite f8 [0,4194304] 0 2024-05-12T04:01:35.758 INFO:tasks.workunit.client.1.smithi033.stdout:1/964: stat d11/d17/d38/f8f 0 2024-05-12T04:01:35.758 INFO:tasks.workunit.client.1.smithi033.stdout:5/970: unlink d3/d94/d54/d57/d70/da3/dbe/d16/d78/f137 0 2024-05-12T04:01:35.759 INFO:tasks.workunit.client.1.smithi033.stdout:1/965: chown d11/d17/d27/d36/d3f/f50 74 1 2024-05-12T04:01:35.760 INFO:tasks.workunit.client.0.smithi032.stdout:5/940: symlink d4/d7/d14/d108/d126/l147 0 2024-05-12T04:01:35.762 INFO:tasks.workunit.client.0.smithi032.stdout:2/802: dwrite d1/d34/d95/fa1 [4194304,4194304] 0 2024-05-12T04:01:35.763 INFO:tasks.workunit.client.0.smithi032.stdout:6/764: getdents d7/d9/d48/d67/d63/d64/d94/de4 0 2024-05-12T04:01:35.766 INFO:tasks.workunit.client.1.smithi033.stdout:1/966: truncate d11/d17/d27/d36/d3f/d4a/dc6/de1/d114/fe8 2222340 0 2024-05-12T04:01:35.770 INFO:tasks.workunit.client.1.smithi033.stdout:5/971: link d3/d94/d85/fe9 d3/d94/d54/d57/d70/da3/dbe/d16/d78/d104/d122/f139 0 2024-05-12T04:01:35.771 INFO:tasks.workunit.client.1.smithi033.stdout:1/967: read d11/d17/f35 [4012395,27895] 0 2024-05-12T04:01:35.773 INFO:tasks.workunit.client.0.smithi032.stdout:2/803: mkdir d1/d34/d96/d41/d58/d108 0 2024-05-12T04:01:35.774 INFO:tasks.workunit.client.0.smithi032.stdout:6/765: dwrite d7/d9/d48/d67/d1a/d93/dde/ff0 [0,4194304] 0 2024-05-12T04:01:35.776 INFO:tasks.workunit.client.0.smithi032.stdout:0/841: mknod df/d30/d4e/d55/dd4/c11b 0 2024-05-12T04:01:35.776 INFO:tasks.workunit.client.0.smithi032.stdout:2/804: chown d1/d34/d96/d41/d39/l9f 208151429 1 2024-05-12T04:01:35.778 INFO:tasks.workunit.client.0.smithi032.stdout:5/941: rename d4/d7/d14/d26/d33/d4a/d38/d79/ld1 to d4/d7/d14/d39/d2b/d6e/l148 0 2024-05-12T04:01:35.780 INFO:tasks.workunit.client.1.smithi033.stdout:1/968: dread d11/f4f [0,4194304] 0 2024-05-12T04:01:35.785 INFO:tasks.workunit.client.1.smithi033.stdout:5/972: mknod d3/d12/d29/d2b/c13a 0 2024-05-12T04:01:35.789 INFO:tasks.workunit.client.1.smithi033.stdout:1/969: creat d11/d1c/ddf/d4c/dde/f146 x:0 0 0 2024-05-12T04:01:35.789 INFO:tasks.workunit.client.1.smithi033.stdout:1/970: read f8 [3352302,78974] 0 2024-05-12T04:01:35.790 INFO:tasks.workunit.client.0.smithi032.stdout:6/766: symlink d7/d9/d48/d67/d63/d64/d94/l10b 0 2024-05-12T04:01:35.791 INFO:tasks.workunit.client.1.smithi033.stdout:5/973: dread d3/d94/d54/d57/d70/da3/dbe/d16/d45/d62/f112 [0,4194304] 0 2024-05-12T04:01:35.797 INFO:tasks.workunit.client.0.smithi032.stdout:5/942: creat d4/d7/d14/d39/d1b/da4/f149 x:0 0 0 2024-05-12T04:01:35.798 INFO:tasks.workunit.client.0.smithi032.stdout:5/943: chown d4/d7/d14/d26/d33/d4a/d11d/d12c 6724939 1 2024-05-12T04:01:35.799 INFO:tasks.workunit.client.0.smithi032.stdout:3/922: creat d15/d20/f143 x:0 0 0 2024-05-12T04:01:35.800 INFO:tasks.workunit.client.0.smithi032.stdout:8/828: link d0/d4b/fcb d0/d9/f117 0 2024-05-12T04:01:35.802 INFO:tasks.workunit.client.1.smithi033.stdout:1/971: fdatasync d11/d1c/d43/f5e 0 2024-05-12T04:01:35.802 INFO:tasks.workunit.client.0.smithi032.stdout:3/923: chown d15/d20/d26/d6a 29 1 2024-05-12T04:01:35.803 INFO:tasks.workunit.client.1.smithi033.stdout:1/972: chown d11/d1c/ddf/d4c/f10f 1 1 2024-05-12T04:01:35.804 INFO:tasks.workunit.client.0.smithi032.stdout:0/842: dread df/d10/d3d/f56 [0,4194304] 0 2024-05-12T04:01:35.806 INFO:tasks.workunit.client.0.smithi032.stdout:2/805: link d1/d34/d96/da4/fc0 d1/d34/d95/f109 0 2024-05-12T04:01:35.808 INFO:tasks.workunit.client.0.smithi032.stdout:5/944: dwrite d4/f134 [0,4194304] 0 2024-05-12T04:01:35.811 INFO:tasks.workunit.client.0.smithi032.stdout:6/767: creat d7/d9/d48/d67/d63/d64/d94/de4/d57/f10c x:0 0 0 2024-05-12T04:01:35.815 INFO:tasks.workunit.client.0.smithi032.stdout:0/843: dwrite df/d22/f39 [0,4194304] 0 2024-05-12T04:01:35.816 INFO:tasks.workunit.client.0.smithi032.stdout:6/768: chown d7/d9/d48/d67/d8d/dbc/de2/c39 78602960 1 2024-05-12T04:01:35.816 INFO:tasks.workunit.client.1.smithi033.stdout:5/974: creat d3/f13b x:0 0 0 2024-05-12T04:01:35.817 INFO:tasks.workunit.client.0.smithi032.stdout:3/924: symlink d15/d20/d22/d7a/l144 0 2024-05-12T04:01:35.823 INFO:tasks.workunit.client.0.smithi032.stdout:8/829: creat d0/d9/d13/d21/d35/de9/f118 x:0 0 0 2024-05-12T04:01:35.825 INFO:tasks.workunit.client.0.smithi032.stdout:5/945: fsync d4/d7/d14/d26/d33/d72/dc9/dec/fba 0 2024-05-12T04:01:35.827 INFO:tasks.workunit.client.1.smithi033.stdout:1/973: write d11/d17/d27/da6/f106 [383457,37207] 0 2024-05-12T04:01:35.828 INFO:tasks.workunit.client.0.smithi032.stdout:6/769: mknod d7/d9/d48/d67/d63/d64/c10d 0 2024-05-12T04:01:35.830 INFO:tasks.workunit.client.0.smithi032.stdout:2/806: creat d1/d34/d96/d41/d58/d108/f10a x:0 0 0 2024-05-12T04:01:35.831 INFO:tasks.workunit.client.0.smithi032.stdout:2/807: chown d1/d34/d3e/f9e 453824930 1 2024-05-12T04:01:35.831 INFO:tasks.workunit.client.0.smithi032.stdout:1/811: sync 2024-05-12T04:01:35.832 INFO:tasks.workunit.client.0.smithi032.stdout:0/844: symlink df/d30/d8d/l11c 0 2024-05-12T04:01:35.832 INFO:tasks.workunit.client.0.smithi032.stdout:1/812: readlink d1/d7/d30/d18/l37 0 2024-05-12T04:01:35.834 INFO:tasks.workunit.client.0.smithi032.stdout:8/830: rename d0/d9/d1b/d33/d73/db7/ff6 to d0/d9/d1b/d27/d88/d56/d63/d8f/d9a/d28/d3e/f119 0 2024-05-12T04:01:35.836 INFO:tasks.workunit.client.0.smithi032.stdout:8/831: write d0/d9/d13/d21/fd3 [738666,8374] 0 2024-05-12T04:01:35.837 INFO:tasks.workunit.client.1.smithi033.stdout:5/975: write d3/d94/d54/d57/d70/da3/dbe/daa/fc3 [128571,120878] 0 2024-05-12T04:01:35.837 INFO:tasks.workunit.client.0.smithi032.stdout:5/946: write d4/d7/d14/d39/d1b/f23 [3504850,70126] 0 2024-05-12T04:01:35.839 INFO:tasks.workunit.client.0.smithi032.stdout:5/947: write d4/d7/d14/d26/f107 [3009380,118725] 0 2024-05-12T04:01:35.840 INFO:tasks.workunit.client.1.smithi033.stdout:1/974: creat d11/d17/d27/d36/d3f/d4a/dc6/de1/d114/d87/f147 x:0 0 0 2024-05-12T04:01:35.842 INFO:tasks.workunit.client.0.smithi032.stdout:6/770: symlink d7/d9/d5c/d8b/da1/l10e 0 2024-05-12T04:01:35.843 INFO:tasks.workunit.client.1.smithi033.stdout:5/976: mknod d3/d94/d54/d57/d70/da3/dbe/d16/d45/d62/d7f/c13c 0 2024-05-12T04:01:35.844 INFO:tasks.workunit.client.0.smithi032.stdout:2/808: read - d1/d34/d4a/feb zero size 2024-05-12T04:01:35.845 INFO:tasks.workunit.client.0.smithi032.stdout:2/809: chown d1/d34/da5/caf 7049 1 2024-05-12T04:01:35.846 INFO:tasks.workunit.client.0.smithi032.stdout:2/810: stat d1/d13/d70/ddb 0 2024-05-12T04:01:35.849 INFO:tasks.workunit.client.0.smithi032.stdout:3/925: truncate d15/d20/d22/d7a/f125 3766446 0 2024-05-12T04:01:35.855 INFO:tasks.workunit.client.1.smithi033.stdout:1/975: rename d11/d17/d27/d36/d3f/c5a to d11/d17/d27/d36/c148 0 2024-05-12T04:01:35.856 INFO:tasks.workunit.client.0.smithi032.stdout:3/926: dwrite d15/d33/d3f/f132 [0,4194304] 0 2024-05-12T04:01:35.857 INFO:tasks.workunit.client.0.smithi032.stdout:3/927: write d15/d20/f143 [700343,32897] 0 2024-05-12T04:01:35.857 INFO:tasks.workunit.client.1.smithi033.stdout:5/977: dwrite d3/d94/f3e [0,4194304] 0 2024-05-12T04:01:35.861 INFO:tasks.workunit.client.0.smithi032.stdout:6/771: creat d7/d33/f10f x:0 0 0 2024-05-12T04:01:35.863 INFO:tasks.workunit.client.1.smithi033.stdout:1/976: fdatasync d11/d17/d27/d76/d93/ffd 0 2024-05-12T04:01:35.863 INFO:tasks.workunit.client.0.smithi032.stdout:2/811: fsync d1/d34/d3e/d67/f83 0 2024-05-12T04:01:35.864 INFO:tasks.workunit.client.1.smithi033.stdout:1/977: readlink d11/d1c/lb9 0 2024-05-12T04:01:35.865 INFO:tasks.workunit.client.0.smithi032.stdout:2/812: write d1/d34/d96/d41/d2c/f37 [3753919,17120] 0 2024-05-12T04:01:35.866 INFO:tasks.workunit.client.1.smithi033.stdout:5/978: unlink d3/d94/dc8/fa8 0 2024-05-12T04:01:35.867 INFO:tasks.workunit.client.0.smithi032.stdout:1/813: creat d1/dd/d24/d101/f10d x:0 0 0 2024-05-12T04:01:35.868 INFO:tasks.workunit.client.0.smithi032.stdout:8/832: mknod d0/d9/d1b/d27/c11a 0 2024-05-12T04:01:35.869 INFO:tasks.workunit.client.0.smithi032.stdout:1/814: chown d1/d7/d30/d35/d54/f94 573641265 1 2024-05-12T04:01:35.873 INFO:tasks.workunit.client.1.smithi033.stdout:5/979: rename d3/d94/d54/d57/d70/da3/dbe/d16/d45/d62/d7f/c13c to d3/d94/d54/d57/d70/da3/dbe/d16/d78/c13d 0 2024-05-12T04:01:35.879 INFO:tasks.workunit.client.1.smithi033.stdout:5/980: mkdir d3/d94/d54/d57/d70/da3/dbe/d16/dd7/d136/d13e 0 2024-05-12T04:01:35.880 INFO:tasks.workunit.client.1.smithi033.stdout:5/981: readlink d3/d12/d29/le5 0 2024-05-12T04:01:35.881 INFO:tasks.workunit.client.1.smithi033.stdout:5/982: chown d3/d94/d85/ce6 5230 1 2024-05-12T04:01:35.882 INFO:tasks.workunit.client.1.smithi033.stdout:5/983: write d3/f13b [952734,3365] 0 2024-05-12T04:01:35.882 INFO:tasks.workunit.client.0.smithi032.stdout:2/813: rename d1/d34/d96/d41/d2c/f90 to d1/d34/d4a/f10b 0 2024-05-12T04:01:35.884 INFO:tasks.workunit.client.0.smithi032.stdout:5/948: dwrite d4/d7/d14/d108/feb [0,4194304] 0 2024-05-12T04:01:35.884 INFO:tasks.workunit.client.0.smithi032.stdout:1/815: mkdir d1/dd/d24/d101/d10e 0 2024-05-12T04:01:35.887 INFO:tasks.workunit.client.1.smithi033.stdout:5/984: symlink d3/d94/d85/d99/da2/l13f 0 2024-05-12T04:01:35.892 INFO:tasks.workunit.client.0.smithi032.stdout:0/845: dwrite df/d30/d4e/d55/f67 [4194304,4194304] 0 2024-05-12T04:01:35.897 INFO:tasks.workunit.client.0.smithi032.stdout:2/814: rename d1/d34/d96/da4 to d1/d34/d3e/d67/d10c 0 2024-05-12T04:01:35.899 INFO:tasks.workunit.client.0.smithi032.stdout:1/816: fsync d1/d6a/def/ffa 0 2024-05-12T04:01:35.904 INFO:tasks.workunit.client.0.smithi032.stdout:3/928: write d15/d20/d22/f47 [4680384,93748] 0 2024-05-12T04:01:35.905 INFO:tasks.workunit.client.0.smithi032.stdout:1/817: rmdir d1/d6a/def 39 2024-05-12T04:01:35.908 INFO:tasks.workunit.client.0.smithi032.stdout:1/818: truncate d1/d7/d30/f105 4344592 0 2024-05-12T04:01:35.908 INFO:tasks.workunit.client.0.smithi032.stdout:0/846: unlink df/fc1 0 2024-05-12T04:01:35.910 INFO:tasks.workunit.client.1.smithi033.stdout:1/978: dwrite d11/d1c/f94 [0,4194304] 0 2024-05-12T04:01:35.910 INFO:tasks.workunit.client.0.smithi032.stdout:2/815: symlink d1/d4d/d5e/dff/l10d 0 2024-05-12T04:01:35.911 INFO:tasks.workunit.client.0.smithi032.stdout:0/847: readlink df/d10/d48/l105 0 2024-05-12T04:01:35.914 INFO:tasks.workunit.client.0.smithi032.stdout:6/772: dwrite d7/d9/d5c/d8b/da1/fa2 [0,4194304] 0 2024-05-12T04:01:35.919 INFO:tasks.workunit.client.0.smithi032.stdout:1/819: mkdir d1/d7/d30/d35/d77/d10f 0 2024-05-12T04:01:35.919 INFO:tasks.workunit.client.1.smithi033.stdout:5/985: dread d3/d94/d54/d57/d70/fa9 [0,4194304] 0 2024-05-12T04:01:35.920 INFO:tasks.workunit.client.1.smithi033.stdout:5/986: fsync d3/d12/d83/deb/ff9 0 2024-05-12T04:01:35.924 INFO:tasks.workunit.client.0.smithi032.stdout:1/820: truncate d1/d7/d30/d35/d54/d7e/fbd 399522 0 2024-05-12T04:01:35.924 INFO:tasks.workunit.client.0.smithi032.stdout:5/949: write d4/d7/d14/d26/d33/d72/dc9/dec/f48 [4325481,68320] 0 2024-05-12T04:01:35.925 INFO:tasks.workunit.client.0.smithi032.stdout:1/821: chown d1/d7/d49/dd1 10 1 2024-05-12T04:01:35.926 INFO:tasks.workunit.client.0.smithi032.stdout:1/822: write d1/d7/d30/d35/f4e [2951165,58606] 0 2024-05-12T04:01:35.927 INFO:tasks.workunit.client.1.smithi033.stdout:5/987: creat d3/d94/d54/d57/d70/da3/dbe/d16/d45/d62/d7f/d84/de2/f140 x:0 0 0 2024-05-12T04:01:35.929 INFO:tasks.workunit.client.0.smithi032.stdout:0/848: symlink df/d10/d3d/d4a/d8e/l11d 0 2024-05-12T04:01:35.930 INFO:tasks.workunit.client.0.smithi032.stdout:0/849: write df/d30/d8d/d8f/f10e [2320739,105416] 0 2024-05-12T04:01:35.931 INFO:tasks.workunit.client.0.smithi032.stdout:0/850: write df/d10/d48/f52 [3158330,26904] 0 2024-05-12T04:01:35.932 INFO:tasks.workunit.client.1.smithi033.stdout:5/988: fsync d3/d94/dc8/f22 0 2024-05-12T04:01:35.933 INFO:tasks.workunit.client.0.smithi032.stdout:2/816: write d1/d34/d3e/d67/d10c/fc0 [1775740,65280] 0 2024-05-12T04:01:35.935 INFO:tasks.workunit.client.0.smithi032.stdout:3/929: truncate d15/d24/d2a/d8c/f116 2430141 0 2024-05-12T04:01:35.936 INFO:tasks.workunit.client.0.smithi032.stdout:2/817: chown d1/d13/d70/f7d 2 1 2024-05-12T04:01:35.937 INFO:tasks.workunit.client.0.smithi032.stdout:3/930: chown d15/d1b/d43/dd0/ff7 5067 1 2024-05-12T04:01:35.938 INFO:tasks.workunit.client.0.smithi032.stdout:2/818: readlink d1/d34/d96/d41/d39/l9f 0 2024-05-12T04:01:35.943 INFO:tasks.workunit.client.0.smithi032.stdout:5/950: dread d4/d7/d14/d26/d33/d88/faf [0,4194304] 0 2024-05-12T04:01:35.944 INFO:tasks.workunit.client.0.smithi032.stdout:6/773: link d7/d9/d48/d67/d63/d64/d94/f101 d7/d9/d5c/d62/f110 0 2024-05-12T04:01:35.947 INFO:tasks.workunit.client.0.smithi032.stdout:5/951: write d4/d7/d14/d26/d33/d72/d92/d111/f13f [433611,122071] 0 2024-05-12T04:01:35.947 INFO:tasks.workunit.client.1.smithi033.stdout:5/989: dread d3/d12/d83/deb/fff [0,4194304] 0 2024-05-12T04:01:35.948 INFO:tasks.workunit.client.0.smithi032.stdout:0/851: dread df/d30/d32/d81/de8/f110 [0,4194304] 0 2024-05-12T04:01:35.948 INFO:tasks.workunit.client.1.smithi033.stdout:5/990: chown d3/f9 29089735 1 2024-05-12T04:01:35.949 INFO:tasks.workunit.client.0.smithi032.stdout:0/852: chown df/d22/d58/d8a/ff1 52 1 2024-05-12T04:01:35.951 INFO:tasks.workunit.client.0.smithi032.stdout:2/819: creat d1/d34/d96/d8a/f10e x:0 0 0 2024-05-12T04:01:35.953 INFO:tasks.workunit.client.1.smithi033.stdout:5/991: rename d3/d94/d85/d99/da2/l13f to d3/d94/d85/d99/da2/l141 0 2024-05-12T04:01:35.954 INFO:tasks.workunit.client.1.smithi033.stdout:5/992: dread - d3/d12/f124 zero size 2024-05-12T04:01:35.959 INFO:tasks.workunit.client.0.smithi032.stdout:5/952: mkdir d4/d7/d14/d26/d33/d4a/d38/d14a 0 2024-05-12T04:01:35.960 INFO:tasks.workunit.client.0.smithi032.stdout:6/774: write d7/fc7 [2437859,32429] 0 2024-05-12T04:01:35.962 INFO:tasks.workunit.client.0.smithi032.stdout:3/931: mkdir d15/d20/d26/d6a/d6e/d145 0 2024-05-12T04:01:35.964 INFO:tasks.workunit.client.1.smithi033.stdout:5/993: creat d3/d94/d54/d57/f142 x:0 0 0 2024-05-12T04:01:35.965 INFO:tasks.workunit.client.0.smithi032.stdout:1/823: getdents d1/d7/d30 0 2024-05-12T04:01:35.967 INFO:tasks.workunit.client.1.smithi033.stdout:5/994: dread d3/d94/d54/d57/d70/da3/dbe/d16/d45/d62/fc0 [0,4194304] 0 2024-05-12T04:01:35.968 INFO:tasks.workunit.client.0.smithi032.stdout:2/820: dread d1/d34/d3e/f45 [0,4194304] 0 2024-05-12T04:01:35.969 INFO:tasks.workunit.client.0.smithi032.stdout:0/853: rename df/d10/d3d/cec to df/d10/d3d/d4a/d84/c11e 0 2024-05-12T04:01:35.971 INFO:tasks.workunit.client.0.smithi032.stdout:0/854: rename df/d30/d8d/d8f/dff to df/d30/d8d/d8f/dff/d11f 22 2024-05-12T04:01:35.972 INFO:tasks.workunit.client.1.smithi033.stdout:5/995: dread - d3/d94/d54/d57/d70/da3/dbe/d16/d78/d104/d122/f139 zero size 2024-05-12T04:01:35.973 INFO:tasks.workunit.client.1.smithi033.stdout:5/996: fdatasync d3/d12/dac/dd9/f109 0 2024-05-12T04:01:35.974 INFO:tasks.workunit.client.0.smithi032.stdout:3/932: creat d15/d20/d22/d117/d13c/f146 x:0 0 0 2024-05-12T04:01:35.974 INFO:tasks.workunit.client.1.smithi033.stdout:5/997: stat d3/d94/d54/d57/d70/da3/dbe 0 2024-05-12T04:01:35.975 INFO:tasks.workunit.client.0.smithi032.stdout:1/824: truncate d1/d7/d30/f28 2131446 0 2024-05-12T04:01:35.977 INFO:tasks.workunit.client.0.smithi032.stdout:2/821: creat d1/d13/d70/d85/dd5/f10f x:0 0 0 2024-05-12T04:01:35.978 INFO:tasks.workunit.client.0.smithi032.stdout:1/825: fsync d1/d7/d30/d35/d54/ff1 0 2024-05-12T04:01:35.982 INFO:tasks.workunit.client.0.smithi032.stdout:1/826: dread - d1/d6a/fb9 zero size 2024-05-12T04:01:35.982 INFO:tasks.workunit.client.0.smithi032.stdout:5/953: dwrite d4/d7/d14/d26/d33/d4a/d38/f45 [0,4194304] 0 2024-05-12T04:01:35.982 INFO:tasks.workunit.client.0.smithi032.stdout:0/855: dread df/d30/d36/f41 [0,4194304] 0 2024-05-12T04:01:35.984 INFO:tasks.workunit.client.1.smithi033.stdout:5/998: symlink d3/d94/d54/d57/d70/da3/dbe/d16/d45/d4b/l143 0 2024-05-12T04:01:35.986 INFO:tasks.workunit.client.0.smithi032.stdout:6/775: getdents d7/d9/d48/d67/d8d/dbc 0 2024-05-12T04:01:35.990 INFO:tasks.workunit.client.0.smithi032.stdout:2/822: truncate d1/d34/da5/fb5 3453570 0 2024-05-12T04:01:35.993 INFO:tasks.workunit.client.0.smithi032.stdout:9/703: sync 2024-05-12T04:01:35.993 INFO:tasks.workunit.client.0.smithi032.stdout:4/751: sync 2024-05-12T04:01:35.996 INFO:tasks.workunit.client.1.smithi033.stdout:5/999: getdents d3/d12/d25/db0 0 2024-05-12T04:01:36.000 INFO:tasks.workunit.client.0.smithi032.stdout:3/933: link d15/d33/f46 d15/d20/d8b/d9b/f147 0 2024-05-12T04:01:36.005 INFO:tasks.workunit.client.0.smithi032.stdout:0/856: write df/d22/d58/f6a [1815541,102792] 0 2024-05-12T04:01:36.008 INFO:tasks.workunit.client.0.smithi032.stdout:9/704: write d0/d3/d6/d7/d2a/d9f/fa1 [4128987,50136] 0 2024-05-12T04:01:36.008 INFO:tasks.workunit.client.0.smithi032.stdout:6/776: write f3 [1234210,106315] 0 2024-05-12T04:01:36.009 INFO:tasks.workunit.client.0.smithi032.stdout:1/827: dread - d1/d7/d30/d18/d87/dc6/fde zero size 2024-05-12T04:01:36.018 INFO:tasks.workunit.client.0.smithi032.stdout:2/823: rename d1/d34/da5/db3/db7/fc5 to d1/d34/d96/d41/f110 0 2024-05-12T04:01:36.023 INFO:tasks.workunit.client.0.smithi032.stdout:9/705: link d0/d84/faf d0/d3/d1b/dd2/fe4 0 2024-05-12T04:01:36.025 INFO:tasks.workunit.client.0.smithi032.stdout:6/777: getdents d7/d97 0 2024-05-12T04:01:36.027 INFO:tasks.workunit.client.0.smithi032.stdout:3/934: dwrite d15/d33/d3f/fb3 [0,4194304] 0 2024-05-12T04:01:36.027 INFO:tasks.workunit.client.0.smithi032.stdout:4/752: dwrite d0/d3/d12/d1d/f2a [0,4194304] 0 2024-05-12T04:01:36.028 INFO:tasks.workunit.client.0.smithi032.stdout:0/857: rename df/d30/d4d/fe2 to df/d10/d48/d70/dbb/f120 0 2024-05-12T04:01:36.032 INFO:tasks.workunit.client.0.smithi032.stdout:2/824: creat d1/d4d/d5e/d9a/f111 x:0 0 0 2024-05-12T04:01:36.038 INFO:tasks.workunit.client.0.smithi032.stdout:5/954: dwrite d4/d7/d14/da9/faa [0,4194304] 0 2024-05-12T04:01:36.040 INFO:tasks.workunit.client.0.smithi032.stdout:3/935: symlink d15/d33/d3f/l148 0 2024-05-12T04:01:36.043 INFO:tasks.workunit.client.0.smithi032.stdout:5/955: dread d4/d7/d14/d39/d1b/da4/fc4 [0,4194304] 0 2024-05-12T04:01:36.047 INFO:tasks.workunit.client.1.smithi033.stdout:1/979: sync 2024-05-12T04:01:36.050 INFO:tasks.workunit.client.0.smithi032.stdout:3/936: creat d15/d1b/d43/dd0/f149 x:0 0 0 2024-05-12T04:01:36.051 INFO:tasks.workunit.client.0.smithi032.stdout:6/778: write d7/d9/d48/d67/d63/d64/fca [248024,3170] 0 2024-05-12T04:01:36.053 INFO:tasks.workunit.client.0.smithi032.stdout:3/937: chown d15/d20/d22/d7a/fa6 202832655 1 2024-05-12T04:01:36.054 INFO:tasks.workunit.client.1.smithi033.stdout:1/980: rename d11/d1c/ddf/d4c/dab/fff to d11/d17/d9e/f149 0 2024-05-12T04:01:36.054 INFO:tasks.workunit.client.0.smithi032.stdout:9/706: dwrite d0/d3/d6/fa0 [0,4194304] 0 2024-05-12T04:01:36.059 INFO:tasks.workunit.client.0.smithi032.stdout:6/779: write d7/d9/d48/d67/d8d/dbc/f108 [312957,101247] 0 2024-05-12T04:01:36.060 INFO:tasks.workunit.client.0.smithi032.stdout:3/938: chown d15/d20/d22/d7a/f85 9 1 2024-05-12T04:01:36.063 INFO:tasks.workunit.client.1.smithi033.stdout:1/981: fdatasync d11/d1c/f33 0 2024-05-12T04:01:36.064 INFO:tasks.workunit.client.0.smithi032.stdout:0/858: getdents df/d28 0 2024-05-12T04:01:36.066 INFO:tasks.workunit.client.0.smithi032.stdout:4/753: dwrite d0/f3d [0,4194304] 0 2024-05-12T04:01:36.066 INFO:tasks.workunit.client.0.smithi032.stdout:6/780: fdatasync d7/d97/f9e 0 2024-05-12T04:01:36.067 INFO:tasks.workunit.client.0.smithi032.stdout:0/859: chown df/df2 663869464 1 2024-05-12T04:01:36.073 INFO:tasks.workunit.client.0.smithi032.stdout:5/956: rename d4/d7/d14/f10c to d4/d7/d14/d26/d33/d4a/f14b 0 2024-05-12T04:01:36.081 INFO:tasks.workunit.client.0.smithi032.stdout:4/754: dread d0/d3/d12/f5b [0,4194304] 0 2024-05-12T04:01:36.187 INFO:tasks.workunit.client.1.smithi033.stdout:1/982: rename d11/d1c/ddf/d4c/dde/f146 to d11/d17/d84/f14a 0 2024-05-12T04:01:36.188 INFO:tasks.workunit.client.1.smithi033.stdout:1/983: chown d11/d142 222835311 1 2024-05-12T04:01:36.188 INFO:tasks.workunit.client.0.smithi032.stdout:3/939: write d15/d24/f51 [1627605,11587] 0 2024-05-12T04:01:36.188 INFO:tasks.workunit.client.1.smithi033.stdout:1/984: chown d11/d17/d27/da6/c10e 4 1 2024-05-12T04:01:36.192 INFO:tasks.workunit.client.0.smithi032.stdout:9/707: mkdir d0/d3/d6/d7/d2a/d46/d91/dc5/de5 0 2024-05-12T04:01:36.193 INFO:tasks.workunit.client.1.smithi033.stdout:1/985: creat d11/d17/d27/d36/d3f/d4a/dc6/de1/d114/f14b x:0 0 0 2024-05-12T04:01:36.197 INFO:tasks.workunit.client.1.smithi033.stdout:1/986: mknod d11/d17/d27/d36/d3f/d4a/c14c 0 2024-05-12T04:01:36.198 INFO:tasks.workunit.client.0.smithi032.stdout:6/781: dwrite d7/d9/d48/d67/d25/f3e [0,4194304] 0 2024-05-12T04:01:36.200 INFO:tasks.workunit.client.0.smithi032.stdout:6/782: write d7/d9/d5c/d62/fad [1670399,41285] 0 2024-05-12T04:01:36.200 INFO:tasks.workunit.client.0.smithi032.stdout:4/755: unlink d0/d3/d12/d1d/d6b/dc2/dc4/fd5 0 2024-05-12T04:01:36.209 INFO:tasks.workunit.client.0.smithi032.stdout:5/957: dwrite d4/d7/d14/d26/d33/d4a/d38/dd2/fd9 [0,4194304] 0 2024-05-12T04:01:36.216 INFO:tasks.workunit.client.0.smithi032.stdout:5/958: creat d4/d7/d14/d26/d33/d5f/d7f/d119/f14c x:0 0 0 2024-05-12T04:01:36.217 INFO:tasks.workunit.client.1.smithi033.stdout:1/987: dwrite d11/d17/d27/d36/d3f/d4a/dc6/de1/feb [0,4194304] 0 2024-05-12T04:01:36.218 INFO:tasks.workunit.client.0.smithi032.stdout:6/783: getdents d7/d9/d48/d67/d8d/dbc/de2 0 2024-05-12T04:01:36.221 INFO:tasks.workunit.client.0.smithi032.stdout:0/860: dwrite df/d30/d4e/f7e [0,4194304] 0 2024-05-12T04:01:36.224 INFO:tasks.workunit.client.0.smithi032.stdout:5/959: fsync d4/d7/d14/d26/d33/d5f/f87 0 2024-05-12T04:01:36.224 INFO:tasks.workunit.client.1.smithi033.stdout:1/988: unlink d11/d17/d27/f57 0 2024-05-12T04:01:36.225 INFO:tasks.workunit.client.0.smithi032.stdout:6/784: symlink d7/d9/l111 0 2024-05-12T04:01:36.227 INFO:tasks.workunit.client.0.smithi032.stdout:3/940: dwrite d15/d24/d93/f8f [0,4194304] 0 2024-05-12T04:01:36.229 INFO:tasks.workunit.client.0.smithi032.stdout:4/756: dwrite d0/d75/f8a [0,4194304] 0 2024-05-12T04:01:36.232 INFO:tasks.workunit.client.0.smithi032.stdout:3/941: write d15/d20/d22/d117/d13c/f146 [57723,47236] 0 2024-05-12T04:01:36.236 INFO:tasks.workunit.client.0.smithi032.stdout:5/960: symlink d4/d7/d14/l14d 0 2024-05-12T04:01:36.237 INFO:tasks.workunit.client.0.smithi032.stdout:6/785: mkdir d7/d9/d5c/d8b/d112 0 2024-05-12T04:01:36.239 INFO:tasks.workunit.client.0.smithi032.stdout:0/861: getdents df/d30 0 2024-05-12T04:01:36.245 INFO:tasks.workunit.client.0.smithi032.stdout:3/942: truncate d15/d33/d3f/fb7 2082013 0 2024-05-12T04:01:36.246 INFO:tasks.workunit.client.0.smithi032.stdout:6/786: truncate d7/d9/d48/d67/d1a/da0/fcf 176108 0 2024-05-12T04:01:36.246 INFO:tasks.workunit.client.0.smithi032.stdout:4/757: read d0/d3/d24/f72 [558967,28499] 0 2024-05-12T04:01:36.248 INFO:tasks.workunit.client.1.smithi033.stdout:1/989: write d11/d1c/d43/f5e [3786882,84915] 0 2024-05-12T04:01:36.249 INFO:tasks.workunit.client.1.smithi033.stdout:1/990: chown d11/d17/d27/d36/d3f/d4a/ce0 430 1 2024-05-12T04:01:36.250 INFO:tasks.workunit.client.0.smithi032.stdout:5/961: dwrite d4/d7/d14/d26/d33/d4a/d11d/f133 [0,4194304] 0 2024-05-12T04:01:36.250 INFO:tasks.workunit.client.0.smithi032.stdout:3/943: chown d15/d1b/l1d 424514 1 2024-05-12T04:01:36.250 INFO:tasks.workunit.client.1.smithi033.stdout:1/991: stat d11/d1c/ddf/d4c/dab/dee/f132 0 2024-05-12T04:01:36.251 INFO:tasks.workunit.client.0.smithi032.stdout:0/862: creat df/d10/d3d/d4a/d84/f121 x:0 0 0 2024-05-12T04:01:36.253 INFO:tasks.workunit.client.0.smithi032.stdout:6/787: mkdir d7/d9/d113 0 2024-05-12T04:01:36.254 INFO:tasks.workunit.client.0.smithi032.stdout:0/863: fdatasync df/d22/d58/f6a 0 2024-05-12T04:01:36.254 INFO:tasks.workunit.client.0.smithi032.stdout:3/944: chown d15/d20/d22/d7a/dbe 3 1 2024-05-12T04:01:36.258 INFO:tasks.workunit.client.0.smithi032.stdout:9/708: dread d0/d3/d6/d7/da/f72 [0,4194304] 0 2024-05-12T04:01:36.260 INFO:tasks.workunit.client.0.smithi032.stdout:3/945: chown d15/d20/d26/d6a/c94 11 1 2024-05-12T04:01:36.260 INFO:tasks.workunit.client.0.smithi032.stdout:4/758: dread d0/d3/d12/d1d/f2a [0,4194304] 0 2024-05-12T04:01:36.261 INFO:tasks.workunit.client.0.smithi032.stdout:6/788: mknod d7/d9/d48/c114 0 2024-05-12T04:01:36.262 INFO:tasks.workunit.client.0.smithi032.stdout:3/946: write d15/d20/d26/d6a/d6e/f133 [103011,96533] 0 2024-05-12T04:01:36.264 INFO:tasks.workunit.client.0.smithi032.stdout:0/864: rename df/d28/c44 to df/d10/d48/d70/dbb/c122 0 2024-05-12T04:01:36.267 INFO:tasks.workunit.client.0.smithi032.stdout:5/962: link d4/d7/d14/d39/d1b/f23 d4/d7/d14/d39/f14e 0 2024-05-12T04:01:36.269 INFO:tasks.workunit.client.0.smithi032.stdout:9/709: unlink d0/d3/d4b/l8c 0 2024-05-12T04:01:36.270 INFO:tasks.workunit.client.0.smithi032.stdout:9/710: write d0/fc7 [295661,9693] 0 2024-05-12T04:01:36.270 INFO:tasks.workunit.client.0.smithi032.stdout:4/759: creat d0/d3/d31/d80/fe5 x:0 0 0 2024-05-12T04:01:36.272 INFO:tasks.workunit.client.0.smithi032.stdout:9/711: dread - d0/d3/d74/fc8 zero size 2024-05-12T04:01:36.274 INFO:tasks.workunit.client.0.smithi032.stdout:6/789: rename d7/d9/d5c/d8b/da1/fa2 to d7/d9/d48/d67/d1a/da0/dc6/f115 0 2024-05-12T04:01:36.275 INFO:tasks.workunit.client.0.smithi032.stdout:6/790: chown d7/d9/d5c/cf3 25 1 2024-05-12T04:01:36.275 INFO:tasks.workunit.client.0.smithi032.stdout:6/791: chown d7/d97/db3 2 1 2024-05-12T04:01:36.277 INFO:tasks.workunit.client.0.smithi032.stdout:3/947: mkdir d15/d20/d26/de2/d10c/d14a 0 2024-05-12T04:01:36.281 INFO:tasks.workunit.client.0.smithi032.stdout:9/712: stat d0/d84/faf 0 2024-05-12T04:01:36.283 INFO:tasks.workunit.client.0.smithi032.stdout:0/865: mkdir df/d10/d3d/d4a/d84/dc0/d123 0 2024-05-12T04:01:36.286 INFO:tasks.workunit.client.0.smithi032.stdout:6/792: symlink d7/d9/d48/d67/d1a/da0/l116 0 2024-05-12T04:01:36.288 INFO:tasks.workunit.client.0.smithi032.stdout:3/948: unlink d15/d24/d40/l50 0 2024-05-12T04:01:36.292 INFO:tasks.workunit.client.0.smithi032.stdout:5/963: write d4/d7/d14/d26/d33/d5f/fed [2302423,52845] 0 2024-05-12T04:01:36.296 INFO:tasks.workunit.client.0.smithi032.stdout:6/793: mkdir d7/d9/d113/d117 0 2024-05-12T04:01:36.298 INFO:tasks.workunit.client.0.smithi032.stdout:5/964: unlink d4/d7/d14/da9/faa 0 2024-05-12T04:01:36.299 INFO:tasks.workunit.client.0.smithi032.stdout:0/866: link df/d10/c1e df/d10/d3d/de4/c124 0 2024-05-12T04:01:36.300 INFO:tasks.workunit.client.0.smithi032.stdout:5/965: chown d4/d7/d14/d26/l5c 4118 1 2024-05-12T04:01:36.300 INFO:tasks.workunit.client.0.smithi032.stdout:4/760: dwrite d0/d3/d12/d8d/f98 [0,4194304] 0 2024-05-12T04:01:36.303 INFO:tasks.workunit.client.0.smithi032.stdout:3/949: write d15/d20/d26/f5f [101356,47354] 0 2024-05-12T04:01:36.303 INFO:tasks.workunit.client.0.smithi032.stdout:6/794: chown d7/d9/d48/d67/d1a/d20/fab 111141334 1 2024-05-12T04:01:36.305 INFO:tasks.workunit.client.0.smithi032.stdout:4/761: chown d0/d3/d12/d1d/dc5/cca 2986585 1 2024-05-12T04:01:36.305 INFO:tasks.workunit.client.0.smithi032.stdout:5/966: read d4/d7/d14/d26/d33/d4a/d11d/f133 [2268575,12662] 0 2024-05-12T04:01:36.305 INFO:tasks.workunit.client.0.smithi032.stdout:0/867: creat df/d28/f125 x:0 0 0 2024-05-12T04:01:36.305 INFO:tasks.workunit.client.0.smithi032.stdout:4/762: dread - d0/d3/d31/d80/fe5 zero size 2024-05-12T04:01:36.309 INFO:tasks.workunit.client.0.smithi032.stdout:0/868: truncate fb 2363165 0 2024-05-12T04:01:36.311 INFO:tasks.workunit.client.0.smithi032.stdout:6/795: dread d7/d9/d48/fbe [0,4194304] 0 2024-05-12T04:01:36.311 INFO:tasks.workunit.client.0.smithi032.stdout:9/713: dwrite d0/d3/d6/d7/d2a/d46/d13/f56 [4194304,4194304] 0 2024-05-12T04:01:36.312 INFO:tasks.workunit.client.0.smithi032.stdout:4/763: symlink d0/d3/d12/d1d/d6b/db7/le6 0 2024-05-12T04:01:36.313 INFO:tasks.workunit.client.0.smithi032.stdout:6/796: chown d7/d9/d48/d67/d8d/dbc/f68 23150 1 2024-05-12T04:01:36.329 INFO:tasks.workunit.client.1.smithi033.stdout:1/992: sync 2024-05-12T04:01:36.365 INFO:tasks.workunit.client.0.smithi032.stdout:4/764: chown d0/d3/df/d4e/f92 1580 1 2024-05-12T04:01:36.365 INFO:tasks.workunit.client.1.smithi033.stdout:1/993: mkdir d11/d17/d27/d36/d3f/d4a/dc6/de1/d114/d87/d108/d127/d14d 0 2024-05-12T04:01:36.366 INFO:tasks.workunit.client.0.smithi032.stdout:5/967: rename d4/d7/l43 to d4/d7/d14/d26/d33/d4a/d11d/d12c/l14f 0 2024-05-12T04:01:36.366 INFO:tasks.workunit.client.0.smithi032.stdout:9/714: creat d0/d3/d6/d7/d2a/d46/d13/fe6 x:0 0 0 2024-05-12T04:01:36.367 INFO:tasks.workunit.client.0.smithi032.stdout:4/765: chown d0/d3/d24/f76 228 1 2024-05-12T04:01:36.369 INFO:tasks.workunit.client.0.smithi032.stdout:4/766: dread - d0/d3/d12/d8d/da1/fc7 zero size 2024-05-12T04:01:36.373 INFO:tasks.workunit.client.0.smithi032.stdout:9/715: dwrite d0/d3/d4b/d88/f96 [0,4194304] 0 2024-05-12T04:01:36.374 INFO:tasks.workunit.client.0.smithi032.stdout:9/716: chown d0/d3/d4b/d88/dc4/ddb 7600 1 2024-05-12T04:01:36.375 INFO:tasks.workunit.client.1.smithi033.stdout:1/994: rename d11/d17/d27/d36/d3f/d4a/dc6/de1/d114/d87/d96/fcb to d11/d17/d27/d36/d3f/d4a/dc6/f14e 0 2024-05-12T04:01:36.377 INFO:tasks.workunit.client.1.smithi033.stdout:1/995: dread d11/d17/d27/da6/f106 [0,4194304] 0 2024-05-12T04:01:36.382 INFO:tasks.workunit.client.1.smithi033.stdout:1/996: unlink d11/d17/d27/da6/cbe 0 2024-05-12T04:01:36.406 INFO:tasks.workunit.client.0.smithi032.stdout:6/797: dwrite d7/d9/d48/d67/d63/d64/d94/fdf [4194304,4194304] 0 2024-05-12T04:01:36.409 INFO:tasks.workunit.client.0.smithi032.stdout:5/968: symlink d4/d7/d14/d26/d33/d4a/l150 0 2024-05-12T04:01:36.411 INFO:tasks.workunit.client.0.smithi032.stdout:4/767: mknod d0/d3/ce7 0 2024-05-12T04:01:36.413 INFO:tasks.workunit.client.1.smithi033.stdout:1/997: write d11/d1c/d43/faf [411539,56775] 0 2024-05-12T04:01:36.414 INFO:tasks.workunit.client.0.smithi032.stdout:4/768: mknod d0/d3/d12/d8d/ce8 0 2024-05-12T04:01:36.418 INFO:tasks.workunit.client.1.smithi033.stdout:1/998: mknod d11/d1c/ddf/d4c/dab/d109/d122/d115/c14f 0 2024-05-12T04:01:36.423 INFO:tasks.workunit.client.0.smithi032.stdout:5/969: creat d4/f151 x:0 0 0 2024-05-12T04:01:36.423 INFO:tasks.workunit.client.0.smithi032.stdout:6/798: link d7/d9/d48/d67/d8d/dbc/de2/db9/le0 d7/d97/db3/l118 0 2024-05-12T04:01:36.425 INFO:tasks.workunit.client.1.smithi033.stdout:1/999: symlink d11/l150 0 2024-05-12T04:01:36.426 INFO:tasks.workunit.client.0.smithi032.stdout:5/970: write d4/d7/d14/d26/d33/d72/d92/d111/f140 [386056,84602] 0 2024-05-12T04:01:36.428 INFO:tasks.workunit.client.0.smithi032.stdout:6/799: mkdir d7/d9/d48/d67/d63/d119 0 2024-05-12T04:01:36.430 INFO:tasks.workunit.client.1.smithi033.stderr:+ rm -rf -- ./tmp.TCxXXFL3v4 2024-05-12T04:01:36.430 INFO:tasks.workunit.client.0.smithi032.stdout:5/971: creat d4/d7/d14/d26/d47/f152 x:0 0 0 2024-05-12T04:01:36.431 INFO:tasks.workunit.client.0.smithi032.stdout:0/869: write df/d30/d36/ddd/f104 [1210255,12389] 0 2024-05-12T04:01:36.432 INFO:tasks.workunit.client.0.smithi032.stdout:6/800: rmdir d7/d9/d48/d67/d8d/dbc/de2/db9 39 2024-05-12T04:01:36.436 INFO:tasks.workunit.client.0.smithi032.stdout:6/801: chown d7/f51 1 1 2024-05-12T04:01:36.438 INFO:tasks.workunit.client.0.smithi032.stdout:9/717: dwrite d0/f6b [0,4194304] 0 2024-05-12T04:01:36.438 INFO:tasks.workunit.client.0.smithi032.stdout:5/972: mkdir d4/d7/d14/d26/d33/d72/d153 0 2024-05-12T04:01:36.442 INFO:tasks.workunit.client.0.smithi032.stdout:0/870: mkdir df/d30/d4e/da5/d10c/d126 0 2024-05-12T04:01:36.443 INFO:tasks.workunit.client.0.smithi032.stdout:0/871: chown df/d10/d3d/d4a/d9a/d9d/fb5 0 1 2024-05-12T04:01:36.444 INFO:tasks.workunit.client.0.smithi032.stdout:6/802: write d7/d9/d48/d67/d1a/d20/f3d [1450508,851] 0 2024-05-12T04:01:36.445 INFO:tasks.workunit.client.0.smithi032.stdout:4/769: truncate d0/d3/d12/f48 1030092 0 2024-05-12T04:01:36.446 INFO:tasks.workunit.client.0.smithi032.stdout:9/718: chown d0/d3/d4b/d6f 128134 1 2024-05-12T04:01:36.448 INFO:tasks.workunit.client.0.smithi032.stdout:5/973: truncate d4/d7/d14/d26/d33/fe7 585920 0 2024-05-12T04:01:36.448 INFO:tasks.workunit.client.0.smithi032.stdout:9/719: write d0/d3/d74/f6c [66521,13222] 0 2024-05-12T04:01:36.452 INFO:tasks.workunit.client.0.smithi032.stdout:6/803: mkdir d7/d9/d11a 0 2024-05-12T04:01:36.456 INFO:tasks.workunit.client.0.smithi032.stdout:6/804: creat d7/d9/d5c/d62/f11b x:0 0 0 2024-05-12T04:01:36.457 INFO:tasks.workunit.client.0.smithi032.stdout:9/720: mknod d0/d3/d6/d7/d2a/d46/d91/dc5/de5/ce7 0 2024-05-12T04:01:36.460 INFO:tasks.workunit.client.0.smithi032.stdout:6/805: dread - d7/d33/f87 zero size 2024-05-12T04:01:36.467 INFO:tasks.workunit.client.0.smithi032.stdout:6/806: mkdir d7/d9/d48/d67/d63/d64/d94/d11c 0 2024-05-12T04:01:36.468 INFO:tasks.workunit.client.0.smithi032.stdout:6/807: chown d7/d9/d48/d67/d1a/ff2 13539 1 2024-05-12T04:01:36.473 INFO:tasks.workunit.client.0.smithi032.stdout:7/747: sync 2024-05-12T04:01:36.473 INFO:tasks.workunit.client.0.smithi032.stdout:8/833: sync 2024-05-12T04:01:36.473 INFO:tasks.workunit.client.0.smithi032.stdout:9/721: truncate d0/d3/d6/d7/d2a/f68 646598 0 2024-05-12T04:01:36.476 INFO:tasks.workunit.client.0.smithi032.stdout:4/770: dwrite d0/d3/d31/d80/f82 [0,4194304] 0 2024-05-12T04:01:36.480 INFO:tasks.workunit.client.0.smithi032.stdout:5/974: dwrite d4/d7/d14/d39/d1b/f23 [0,4194304] 0 2024-05-12T04:01:36.480 INFO:tasks.workunit.client.0.smithi032.stdout:9/722: chown d0/d3/d6/d7/d2a/d46/d13/fe6 130 1 2024-05-12T04:01:36.484 INFO:tasks.workunit.client.0.smithi032.stdout:0/872: dread df/d10/d3d/d4a/f2e [0,4194304] 0 2024-05-12T04:01:36.484 INFO:tasks.workunit.client.0.smithi032.stdout:4/771: rmdir d0/d3/d31/d80/d84 39 2024-05-12T04:01:36.486 INFO:tasks.workunit.client.0.smithi032.stdout:4/772: dread - d0/d3/d12/d1d/d63/d87/d95/fc8 zero size 2024-05-12T04:01:36.487 INFO:tasks.workunit.client.0.smithi032.stdout:4/773: dread - d0/d3/fe2 zero size 2024-05-12T04:01:36.487 INFO:tasks.workunit.client.0.smithi032.stdout:9/723: creat d0/d3/d6/d7/da/d39/fe8 x:0 0 0 2024-05-12T04:01:36.489 INFO:tasks.workunit.client.0.smithi032.stdout:9/724: chown d0/d3/d6/d58/cad 1042411 1 2024-05-12T04:01:36.491 INFO:tasks.workunit.client.0.smithi032.stdout:6/808: truncate d7/d9/d48/d67/f28 476265 0 2024-05-12T04:01:36.492 INFO:tasks.workunit.client.0.smithi032.stdout:5/975: mknod d4/d113/c154 0 2024-05-12T04:01:36.493 INFO:tasks.workunit.client.0.smithi032.stdout:8/834: write d0/d9/f1c [449508,62323] 0 2024-05-12T04:01:36.497 INFO:tasks.workunit.client.0.smithi032.stdout:4/774: mknod d0/d3/d12/d1d/d63/d87/dc6/ce9 0 2024-05-12T04:01:36.500 INFO:tasks.workunit.client.0.smithi032.stdout:7/748: write dd/d1b/d43/d5e/f6f [718225,78163] 0 2024-05-12T04:01:36.502 INFO:tasks.workunit.client.0.smithi032.stdout:9/725: unlink d0/d3/d6/c59 0 2024-05-12T04:01:36.504 INFO:tasks.workunit.client.0.smithi032.stdout:8/835: fdatasync d0/d9/d1b/d27/d88/d56/d63/d8f/fd9 0 2024-05-12T04:01:36.506 INFO:tasks.workunit.client.0.smithi032.stdout:6/809: mkdir d7/d9/d5c/d8b/da1/dfb/d11d 0 2024-05-12T04:01:36.510 INFO:tasks.workunit.client.0.smithi032.stdout:9/726: dwrite d0/d3/d6/d7/f15 [0,4194304] 0 2024-05-12T04:01:36.511 INFO:tasks.workunit.client.0.smithi032.stdout:9/727: write d0/d3/d74/f20 [3602842,69121] 0 2024-05-12T04:01:36.512 INFO:tasks.workunit.client.0.smithi032.stdout:4/775: rename d0/d3/d12/d1d/d63/d87/d95/c9d to d0/d3/d12/d8d/da1/cea 0 2024-05-12T04:01:36.513 INFO:tasks.workunit.client.0.smithi032.stdout:9/728: dread - d0/d3/d6/d7/da/d39/fe8 zero size 2024-05-12T04:01:36.514 INFO:tasks.workunit.client.0.smithi032.stdout:2/825: sync 2024-05-12T04:01:36.514 INFO:tasks.workunit.client.0.smithi032.stdout:1/828: sync 2024-05-12T04:01:36.516 INFO:tasks.workunit.client.0.smithi032.stdout:7/749: stat dd/d1b/d1e/de3/d9a/d9c/db9/f62 0 2024-05-12T04:01:36.517 INFO:tasks.workunit.client.0.smithi032.stdout:7/750: chown dd/d1b/d1e/l97 8 1 2024-05-12T04:01:36.518 INFO:tasks.workunit.client.0.smithi032.stdout:5/976: write d4/d7/d14/d26/f66 [555999,26534] 0 2024-05-12T04:01:36.520 INFO:tasks.workunit.client.0.smithi032.stdout:8/836: mknod d0/d9/d1b/d27/d88/d56/d63/d8f/db0/c11b 0 2024-05-12T04:01:36.520 INFO:tasks.workunit.client.0.smithi032.stdout:5/977: readlink d4/d7/d14/d26/d33/d4a/d38/d79/l12f 0 2024-05-12T04:01:36.520 INFO:tasks.workunit.client.0.smithi032.stdout:8/837: chown d0/d4b/dad 2336967 1 2024-05-12T04:01:36.521 INFO:tasks.workunit.client.0.smithi032.stdout:6/810: mknod d7/d9/d48/d67/d1a/da0/c11e 0 2024-05-12T04:01:36.572 INFO:tasks.workunit.client.0.smithi032.stdout:4/776: creat d0/d3/d12/d1d/dc5/feb x:0 0 0 2024-05-12T04:01:36.575 INFO:tasks.workunit.client.0.smithi032.stdout:9/729: symlink d0/d3/d4b/d88/d99/le9 0 2024-05-12T04:01:36.575 INFO:tasks.workunit.client.0.smithi032.stdout:9/730: read - d0/d3/d4b/f81 zero size 2024-05-12T04:01:36.577 INFO:tasks.workunit.client.0.smithi032.stdout:1/829: read d1/d7/f7f [6603984,128112] 0 2024-05-12T04:01:36.579 INFO:tasks.workunit.client.0.smithi032.stdout:7/751: creat dd/d1b/d1e/d45/d55/d72/da0/dbd/db2/fee x:0 0 0 2024-05-12T04:01:36.587 INFO:tasks.workunit.client.0.smithi032.stdout:8/838: creat d0/d9/d1b/d27/d4c/d4e/d7f/da3/f11c x:0 0 0 2024-05-12T04:01:36.593 INFO:tasks.workunit.client.0.smithi032.stdout:2/826: write d1/d13/d70/f7d [689290,124527] 0 2024-05-12T04:01:36.596 INFO:tasks.workunit.client.0.smithi032.stdout:9/731: chown d0/d3/ff 1203681247 1 2024-05-12T04:01:36.600 INFO:tasks.workunit.client.0.smithi032.stdout:1/830: rename d1/dd/d24/d101/d10e to d1/d7/d30/d35/d77/d110 0 2024-05-12T04:01:36.601 INFO:tasks.workunit.client.0.smithi032.stdout:5/978: write d4/d7/d14/d26/d33/d72/d92/f97 [2960060,17226] 0 2024-05-12T04:01:36.603 INFO:tasks.workunit.client.0.smithi032.stdout:7/752: mkdir dd/d1b/d1e/de3/d9a/d9c/def 0 2024-05-12T04:01:36.605 INFO:tasks.workunit.client.0.smithi032.stdout:8/839: creat d0/d4b/dad/f11d x:0 0 0 2024-05-12T04:01:36.607 INFO:tasks.workunit.client.0.smithi032.stdout:5/979: dwrite d4/d7/d14/d26/d47/f6d [4194304,4194304] 0 2024-05-12T04:01:36.608 INFO:tasks.workunit.client.0.smithi032.stdout:6/811: link d7/d9/d48/d67/d63/d64/d94/de4/f26 d7/d33/f11f 0 2024-05-12T04:01:36.611 INFO:tasks.workunit.client.0.smithi032.stdout:4/777: mkdir d0/d3/d31/d80/dec 0 2024-05-12T04:01:36.620 INFO:tasks.workunit.client.0.smithi032.stdout:7/753: mkdir dd/d1b/d43/d5e/df0 0 2024-05-12T04:01:36.624 INFO:tasks.workunit.client.0.smithi032.stdout:4/778: stat d0/d3/d31/d80/d84/fcc 0 2024-05-12T04:01:36.626 INFO:tasks.workunit.client.0.smithi032.stdout:2/827: creat d1/d34/d96/d41/da2/f112 x:0 0 0 2024-05-12T04:01:36.628 INFO:tasks.workunit.client.0.smithi032.stdout:1/831: symlink d1/d25/da5/l111 0 2024-05-12T04:01:36.630 INFO:tasks.workunit.client.0.smithi032.stdout:7/754: rmdir dd/d1b/d1e/d45/de4 39 2024-05-12T04:01:36.639 INFO:tasks.workunit.client.0.smithi032.stdout:5/980: dread d4/d7/d14/d39/d1b/f76 [0,4194304] 0 2024-05-12T04:01:36.642 INFO:tasks.workunit.client.0.smithi032.stdout:1/832: write d1/d7/d30/d35/d54/f8c [1636310,101677] 0 2024-05-12T04:01:36.647 INFO:tasks.workunit.client.0.smithi032.stdout:8/840: creat d0/d9/d1b/d27/f11e x:0 0 0 2024-05-12T04:01:36.648 INFO:tasks.workunit.client.0.smithi032.stdout:8/841: write d0/d9/d13/d21/f100 [148553,84066] 0 2024-05-12T04:01:36.651 INFO:tasks.workunit.client.0.smithi032.stdout:6/812: rename d7/d9/d48/d67/d63/d64/d94/de4/d57/fba to d7/d9/d5c/d8b/f120 0 2024-05-12T04:01:36.652 INFO:tasks.workunit.client.0.smithi032.stdout:6/813: readlink d7/d9/d48/d67/d1a/d20/l92 0 2024-05-12T04:01:36.656 INFO:tasks.workunit.client.0.smithi032.stdout:9/732: getdents d0/d84/da6 0 2024-05-12T04:01:36.659 INFO:tasks.workunit.client.0.smithi032.stdout:1/833: creat d1/d59/f112 x:0 0 0 2024-05-12T04:01:36.659 INFO:tasks.workunit.client.0.smithi032.stdout:1/834: chown d1/d25/ce8 14702166 1 2024-05-12T04:01:36.660 INFO:tasks.workunit.client.0.smithi032.stdout:1/835: dread - d1/dd/d24/ffc zero size 2024-05-12T04:01:36.660 INFO:tasks.workunit.client.0.smithi032.stdout:1/836: chown d1/dd/d24 7668011 1 2024-05-12T04:01:36.668 INFO:tasks.workunit.client.0.smithi032.stdout:7/755: rename dd/d1b/d43/d5e/f1c to dd/d1b/d1e/d45/d55/db7/ff1 0 2024-05-12T04:01:36.676 INFO:tasks.workunit.client.0.smithi032.stdout:7/756: dwrite dd/d1b/d1e/d45/d55/d72/da0/dbd/db2/fe7 [0,4194304] 0 2024-05-12T04:01:36.676 INFO:tasks.workunit.client.0.smithi032.stdout:1/837: truncate d1/d7/d49/f6b 1205227 0 2024-05-12T04:01:36.682 INFO:tasks.workunit.client.0.smithi032.stdout:3/950: sync 2024-05-12T04:01:36.683 INFO:tasks.workunit.client.0.smithi032.stdout:6/814: mkdir d7/d9/d48/d121 0 2024-05-12T04:01:36.684 INFO:tasks.workunit.client.0.smithi032.stdout:7/757: dread dd/d1b/d1e/f68 [0,4194304] 0 2024-05-12T04:01:36.697 INFO:tasks.workunit.client.0.smithi032.stdout:2/828: dwrite d1/d13/d70/f91 [0,4194304] 0 2024-05-12T04:01:36.702 INFO:tasks.workunit.client.0.smithi032.stdout:6/815: creat d7/d9/d48/d67/d63/d64/d94/de4/f122 x:0 0 0 2024-05-12T04:01:36.706 INFO:tasks.workunit.client.0.smithi032.stdout:6/816: dread d7/d9/d48/d67/d1a/f73 [0,4194304] 0 2024-05-12T04:01:36.708 INFO:tasks.workunit.client.0.smithi032.stdout:4/779: link d0/d3/d12/f48 d0/d3/d12/fed 0 2024-05-12T04:01:36.716 INFO:tasks.workunit.client.0.smithi032.stdout:5/981: rename d4/d7/d14/d26/d33/d4a/l150 to d4/d7/d58/l155 0 2024-05-12T04:01:36.718 INFO:tasks.workunit.client.0.smithi032.stdout:2/829: creat d1/d13/d6b/f113 x:0 0 0 2024-05-12T04:01:36.720 INFO:tasks.workunit.client.0.smithi032.stdout:8/842: dwrite d0/d9/d1b/d27/d88/d56/d63/d8f/d9a/d28/f2b [0,4194304] 0 2024-05-12T04:01:36.727 INFO:tasks.workunit.client.0.smithi032.stdout:9/733: dwrite d0/d3/d6/d58/f9a [4194304,4194304] 0 2024-05-12T04:01:36.729 INFO:tasks.workunit.client.0.smithi032.stdout:4/780: symlink d0/db5/lee 0 2024-05-12T04:01:36.735 INFO:tasks.workunit.client.0.smithi032.stdout:0/873: sync 2024-05-12T04:01:36.736 INFO:tasks.workunit.client.0.smithi032.stdout:5/982: truncate d4/d7/d14/d26/d33/d72/dc9/dec/fba 4805263 0 2024-05-12T04:01:36.740 INFO:tasks.workunit.client.0.smithi032.stdout:4/781: creat d0/d3/d24/fef x:0 0 0 2024-05-12T04:01:36.740 INFO:tasks.workunit.client.0.smithi032.stdout:3/951: write d15/d24/f7b [419987,40718] 0 2024-05-12T04:01:36.742 INFO:tasks.workunit.client.0.smithi032.stdout:2/830: creat d1/d34/d96/d41/d2c/dd4/f114 x:0 0 0 2024-05-12T04:01:36.744 INFO:tasks.workunit.client.0.smithi032.stdout:8/843: symlink d0/d9/d1b/l11f 0 2024-05-12T04:01:36.753 INFO:tasks.workunit.client.0.smithi032.stdout:1/838: write d1/d7/fba [110857,92938] 0 2024-05-12T04:01:36.753 INFO:tasks.workunit.client.0.smithi032.stdout:7/758: dwrite dd/d1b/d1e/de3/d9a/d9c/dc2/fcf [0,4194304] 0 2024-05-12T04:01:36.754 INFO:tasks.workunit.client.0.smithi032.stdout:1/839: readlink d1/dd/d24/l3d 0 2024-05-12T04:01:36.760 INFO:tasks.workunit.client.0.smithi032.stdout:2/831: symlink d1/d4d/d5e/l115 0 2024-05-12T04:01:36.761 INFO:tasks.workunit.client.0.smithi032.stdout:2/832: readlink d1/d4d/d5e/lb4 0 2024-05-12T04:01:36.763 INFO:tasks.workunit.client.0.smithi032.stdout:2/833: write d1/d34/d96/d41/d2c/f37 [462852,43763] 0 2024-05-12T04:01:36.765 INFO:tasks.workunit.client.0.smithi032.stdout:8/844: symlink d0/d7/l120 0 2024-05-12T04:01:36.766 INFO:tasks.workunit.client.0.smithi032.stdout:5/983: rmdir d4/d7/d14/d39/d2b/d6e 39 2024-05-12T04:01:36.767 INFO:tasks.workunit.client.0.smithi032.stdout:9/734: write d0/d3/d74/fc0 [452026,21751] 0 2024-05-12T04:01:36.768 INFO:tasks.workunit.client.0.smithi032.stdout:9/735: chown d0/d84/d8a 11581782 1 2024-05-12T04:01:36.770 INFO:tasks.workunit.client.0.smithi032.stdout:9/736: read d0/d3/d6/d58/f66 [1866734,61560] 0 2024-05-12T04:01:36.771 INFO:tasks.workunit.client.0.smithi032.stdout:9/737: chown d0/d3/d6/d58/la9 15304102 1 2024-05-12T04:01:36.777 INFO:tasks.workunit.client.0.smithi032.stdout:7/759: dread dd/d6c/f85 [0,4194304] 0 2024-05-12T04:01:36.781 INFO:tasks.workunit.client.0.smithi032.stdout:6/817: mkdir d7/d9/d48/d67/d63/d64/d94/de4/d123 0 2024-05-12T04:01:36.782 INFO:tasks.workunit.client.0.smithi032.stdout:4/782: dwrite d0/d3/d12/d1d/f9e [0,4194304] 0 2024-05-12T04:01:36.792 INFO:tasks.workunit.client.0.smithi032.stdout:8/845: mkdir d0/d9/d1b/d27/d4c/d121 0 2024-05-12T04:01:36.792 INFO:tasks.workunit.client.0.smithi032.stdout:2/834: write d1/d34/d96/ffb [3041364,114788] 0 2024-05-12T04:01:36.793 INFO:tasks.workunit.client.0.smithi032.stdout:3/952: dwrite d15/d20/d26/d6a/f77 [0,4194304] 0 2024-05-12T04:01:36.798 INFO:tasks.workunit.client.0.smithi032.stdout:5/984: write d4/d7/d14/f8c [387763,47338] 0 2024-05-12T04:01:36.804 INFO:tasks.workunit.client.0.smithi032.stdout:1/840: mkdir d1/d7/d113 0 2024-05-12T04:01:36.805 INFO:tasks.workunit.client.0.smithi032.stdout:7/760: fsync dd/f82 0 2024-05-12T04:01:36.807 INFO:tasks.workunit.client.0.smithi032.stdout:7/761: fdatasync dd/f1a 0 2024-05-12T04:01:36.807 INFO:tasks.workunit.client.0.smithi032.stdout:7/762: chown dd/d1b/d1e/d45/d64 1047923 1 2024-05-12T04:01:36.810 INFO:tasks.workunit.client.0.smithi032.stdout:4/783: chown d0/d3/l43 12707495 1 2024-05-12T04:01:36.811 INFO:tasks.workunit.client.0.smithi032.stdout:2/835: dread d1/d13/f93 [0,4194304] 0 2024-05-12T04:01:36.812 INFO:tasks.workunit.client.0.smithi032.stdout:2/836: write d1/f3 [169433,90090] 0 2024-05-12T04:01:36.815 INFO:tasks.workunit.client.0.smithi032.stdout:3/953: rename d15/d20/d26/le5 to d15/dec/l14b 0 2024-05-12T04:01:36.816 INFO:tasks.workunit.client.0.smithi032.stdout:0/874: getdents df/d10/d3d/d4a/d9a/d9d 0 2024-05-12T04:01:36.817 INFO:tasks.workunit.client.0.smithi032.stdout:9/738: getdents d0/d3/d4b/dda 0 2024-05-12T04:01:36.826 INFO:tasks.workunit.client.0.smithi032.stdout:4/784: unlink d0/d3/d12/d1d/d63/lc0 0 2024-05-12T04:01:36.831 INFO:tasks.workunit.client.0.smithi032.stdout:2/837: mkdir d1/d34/d3e/d116 0 2024-05-12T04:01:36.831 INFO:tasks.workunit.client.0.smithi032.stdout:7/763: dread dd/d1b/d1e/f7e [0,4194304] 0 2024-05-12T04:01:36.837 INFO:tasks.workunit.client.0.smithi032.stdout:9/739: chown d0/d3/cdf 24 1 2024-05-12T04:01:36.842 INFO:tasks.workunit.client.0.smithi032.stdout:5/985: unlink d4/d7/d14/d26/d33/d4a/d38/d79/l46 0 2024-05-12T04:01:36.845 INFO:tasks.workunit.client.0.smithi032.stdout:6/818: symlink d7/d9/d48/d67/d8d/dbc/de2/dd4/l124 0 2024-05-12T04:01:36.846 INFO:tasks.workunit.client.0.smithi032.stdout:6/819: truncate d7/d9/d48/faa 9390185 0 2024-05-12T04:01:36.849 INFO:tasks.workunit.client.0.smithi032.stdout:4/785: mknod d0/d3/d12/d1d/d63/d77/cf0 0 2024-05-12T04:01:36.850 INFO:tasks.workunit.client.0.smithi032.stdout:2/838: chown d1/d34/d96/d41/f110 61 1 2024-05-12T04:01:36.854 INFO:tasks.workunit.client.0.smithi032.stdout:4/786: dread d0/d3/d12/fa2 [0,4194304] 0 2024-05-12T04:01:36.855 INFO:tasks.workunit.client.0.smithi032.stdout:7/764: read dd/d6c/f77 [72,17348] 0 2024-05-12T04:01:36.857 INFO:tasks.workunit.client.0.smithi032.stdout:1/841: rename d1/d7/d30/d18/c31 to d1/d7/d30/d18/d87/c114 0 2024-05-12T04:01:36.858 INFO:tasks.workunit.client.0.smithi032.stdout:8/846: truncate d0/d9/d1b/d27/d88/d56/d63/d8f/d9a/d28/d3e/fb6 2235449 0 2024-05-12T04:01:36.861 INFO:tasks.workunit.client.0.smithi032.stdout:1/842: write d1/dd/d24/ffc [376770,30559] 0 2024-05-12T04:01:36.862 INFO:tasks.workunit.client.0.smithi032.stdout:0/875: dwrite df/d10/f1b [0,4194304] 0 2024-05-12T04:01:36.863 INFO:tasks.workunit.client.0.smithi032.stdout:1/843: stat d1/d7/d30/d18/d93/dc1/l10a 0 2024-05-12T04:01:36.866 INFO:tasks.workunit.client.0.smithi032.stdout:5/986: creat d4/d7/d14/d26/d33/d4a/d38/f156 x:0 0 0 2024-05-12T04:01:36.869 INFO:tasks.workunit.client.0.smithi032.stdout:6/820: rmdir d7/d9/d48/d67/d8d 39 2024-05-12T04:01:36.884 INFO:tasks.workunit.client.0.smithi032.stdout:7/765: creat dd/d1b/d43/d74/ff2 x:0 0 0 2024-05-12T04:01:36.884 INFO:tasks.workunit.client.0.smithi032.stdout:2/839: write d1/d34/d96/f8e [41706,60257] 0 2024-05-12T04:01:36.884 INFO:tasks.workunit.client.0.smithi032.stdout:3/954: dwrite d15/d20/d26/d6a/d6e/d7d/d9a/f9c [0,4194304] 0 2024-05-12T04:01:36.888 INFO:tasks.workunit.client.0.smithi032.stdout:8/847: symlink d0/d9/d13/d21/d35/de9/l122 0 2024-05-12T04:01:36.890 INFO:tasks.workunit.client.0.smithi032.stdout:0/876: unlink cc 0 2024-05-12T04:01:36.893 INFO:tasks.workunit.client.0.smithi032.stdout:1/844: truncate d1/d7/d30/d18/d87/f89 146923 0 2024-05-12T04:01:36.894 INFO:tasks.workunit.client.0.smithi032.stdout:5/987: chown d4/d7/d14/d26/d33/d72/dc9/dec/d68/ld4 926 1 2024-05-12T04:01:36.910 INFO:tasks.workunit.client.0.smithi032.stdout:3/955: fdatasync d15/d20/d26/fa5 0 2024-05-12T04:01:36.913 INFO:tasks.workunit.client.0.smithi032.stdout:7/766: rmdir dd/d1b/d1e/de3/d9a 39 2024-05-12T04:01:36.913 INFO:tasks.workunit.client.0.smithi032.stdout:7/767: fdatasync fa 0 2024-05-12T04:01:36.915 INFO:tasks.workunit.client.0.smithi032.stdout:6/821: dread d7/d9/d48/d67/f4c [0,4194304] 0 2024-05-12T04:01:36.921 INFO:tasks.workunit.client.0.smithi032.stdout:0/877: mknod df/d10/d3d/d4a/d9a/d9d/d106/c127 0 2024-05-12T04:01:36.924 INFO:tasks.workunit.client.0.smithi032.stdout:4/787: creat d0/d3/d12/d1d/ff1 x:0 0 0 2024-05-12T04:01:36.926 INFO:tasks.workunit.client.0.smithi032.stdout:2/840: dwrite d1/d13/f1f [0,4194304] 0 2024-05-12T04:01:36.928 INFO:tasks.workunit.client.0.smithi032.stdout:7/768: creat dd/d6c/ff3 x:0 0 0 2024-05-12T04:01:36.929 INFO:tasks.workunit.client.0.smithi032.stdout:9/740: getdents d0/d3/d6/d7/d2a/d46/d91 0 2024-05-12T04:01:36.935 INFO:tasks.workunit.client.0.smithi032.stdout:3/956: creat d15/d20/d26/d6a/d6e/d7d/d13f/f14c x:0 0 0 2024-05-12T04:01:36.941 INFO:tasks.workunit.client.0.smithi032.stdout:7/769: stat dd/d1b/d1e/de3/d9a/d9c/db9/d75 0 2024-05-12T04:01:36.943 INFO:tasks.workunit.client.0.smithi032.stdout:8/848: write d0/d9/d1b/d27/d88/d56/f65 [96974,26681] 0 2024-05-12T04:01:36.944 INFO:tasks.workunit.client.0.smithi032.stdout:1/845: write d1/d7/fe4 [822757,22741] 0 2024-05-12T04:01:36.944 INFO:tasks.workunit.client.0.smithi032.stdout:8/849: stat d0/d9/d13/c41 0 2024-05-12T04:01:36.945 INFO:tasks.workunit.client.0.smithi032.stdout:9/741: mknod d0/d3/d6/d7/d2a/d46/d91/cea 0 2024-05-12T04:01:36.946 INFO:tasks.workunit.client.0.smithi032.stdout:0/878: write df/d30/d32/d60/f90 [3750436,22750] 0 2024-05-12T04:01:36.948 INFO:tasks.workunit.client.0.smithi032.stdout:5/988: rmdir d4/d7/d14/d26/d33/d88 39 2024-05-12T04:01:36.949 INFO:tasks.workunit.client.0.smithi032.stdout:4/788: write d0/d3/d12/fa2 [1905523,8746] 0 2024-05-12T04:01:36.955 INFO:tasks.workunit.client.0.smithi032.stdout:7/770: creat dd/d1b/d43/d5e/ff4 x:0 0 0 2024-05-12T04:01:36.955 INFO:tasks.workunit.client.0.smithi032.stdout:3/957: write d15/d20/d22/fa4 [4318807,6652] 0 2024-05-12T04:01:36.957 INFO:tasks.workunit.client.0.smithi032.stdout:1/846: readlink d1/l5 0 2024-05-12T04:01:36.959 INFO:tasks.workunit.client.0.smithi032.stdout:2/841: dwrite d1/d34/d96/d41/d2c/dd4/f102 [0,4194304] 0 2024-05-12T04:01:36.959 INFO:tasks.workunit.client.0.smithi032.stdout:7/771: fsync dd/d1b/d1e/d45/d55/d72/fe0 0 2024-05-12T04:01:36.962 INFO:tasks.workunit.client.0.smithi032.stdout:5/989: dread d4/d7/d14/d26/d33/d72/d92/fdb [0,4194304] 0 2024-05-12T04:01:36.962 INFO:tasks.workunit.client.0.smithi032.stdout:6/822: getdents d7/d9/d48 0 2024-05-12T04:01:36.965 INFO:tasks.workunit.client.0.smithi032.stdout:2/842: read d1/d34/d3e/fcd [2371196,63052] 0 2024-05-12T04:01:36.966 INFO:tasks.workunit.client.0.smithi032.stdout:0/879: read fb [491807,17970] 0 2024-05-12T04:01:36.969 INFO:tasks.workunit.client.0.smithi032.stdout:9/742: dread d0/d3/d6/d7/d2a/d46/d13/f3c [0,4194304] 0 2024-05-12T04:01:36.973 INFO:tasks.workunit.client.0.smithi032.stdout:7/772: dwrite fa [0,4194304] 0 2024-05-12T04:01:36.974 INFO:tasks.workunit.client.0.smithi032.stdout:0/880: write df/d10/d3d/d4a/d9a/d9d/d106/d113/f11a [827686,15985] 0 2024-05-12T04:01:36.978 INFO:tasks.workunit.client.0.smithi032.stdout:3/958: mkdir d15/d14d 0 2024-05-12T04:01:36.978 INFO:tasks.workunit.client.0.smithi032.stdout:2/843: symlink d1/d13/da0/l117 0 2024-05-12T04:01:36.981 INFO:tasks.workunit.client.0.smithi032.stdout:2/844: chown d1/d13/f93 397539 1 2024-05-12T04:01:36.985 INFO:tasks.workunit.client.0.smithi032.stdout:8/850: dwrite d0/d9/f117 [0,4194304] 0 2024-05-12T04:01:36.986 INFO:tasks.workunit.client.0.smithi032.stdout:9/743: symlink d0/d3/d6/d7/d2a/d46/d91/dc5/leb 0 2024-05-12T04:01:36.992 INFO:tasks.workunit.client.0.smithi032.stdout:6/823: write d7/d9/d48/d67/d8d/dbc/de2/dd4/fb7 [1668623,27097] 0 2024-05-12T04:01:36.994 INFO:tasks.workunit.client.0.smithi032.stdout:3/959: mkdir d15/d20/d8b/d14e 0 2024-05-12T04:01:36.996 INFO:tasks.workunit.client.0.smithi032.stdout:4/789: getdents d0/d3/d31 0 2024-05-12T04:01:36.997 INFO:tasks.workunit.client.0.smithi032.stdout:4/790: stat d0/d3/d12/d1d/d6b/dc2 0 2024-05-12T04:01:36.998 INFO:tasks.workunit.client.0.smithi032.stdout:7/773: write dd/d1b/d1e/f8d [5634708,72898] 0 2024-05-12T04:01:36.999 INFO:tasks.workunit.client.0.smithi032.stdout:4/791: dread - d0/d3/d24/dae/fdc zero size 2024-05-12T04:01:37.000 INFO:tasks.workunit.client.0.smithi032.stdout:4/792: read - d0/d3/d12/fd7 zero size 2024-05-12T04:01:37.001 INFO:tasks.workunit.client.0.smithi032.stdout:8/851: symlink d0/d9/d1b/d66/l123 0 2024-05-12T04:01:37.006 INFO:tasks.workunit.client.0.smithi032.stdout:0/881: mknod df/d10/d3d/d4a/d84/dc0/d123/c128 0 2024-05-12T04:01:37.007 INFO:tasks.workunit.client.0.smithi032.stdout:2/845: truncate d1/f31 594444 0 2024-05-12T04:01:37.009 INFO:tasks.workunit.client.0.smithi032.stdout:7/774: symlink dd/d1b/d43/d74/lf5 0 2024-05-12T04:01:37.009 INFO:tasks.workunit.client.0.smithi032.stdout:1/847: link d1/d7/d30/d35/c6c d1/d7/d49/dd1/c115 0 2024-05-12T04:01:37.013 INFO:tasks.workunit.client.0.smithi032.stdout:6/824: mknod d7/d9/d48/d67/d1a/da0/dc6/dfd/c125 0 2024-05-12T04:01:37.014 INFO:tasks.workunit.client.0.smithi032.stdout:3/960: fdatasync d15/d24/d2a/d8c/f116 0 2024-05-12T04:01:37.018 INFO:tasks.workunit.client.0.smithi032.stdout:0/882: mknod df/d10/d3d/d4a/d84/dc0/d123/c129 0 2024-05-12T04:01:37.027 INFO:tasks.workunit.client.0.smithi032.stdout:8/852: dwrite d0/d9/d13/d21/d25/f52 [0,4194304] 0 2024-05-12T04:01:37.033 INFO:tasks.workunit.client.0.smithi032.stdout:6/825: creat d7/d97/db3/f126 x:0 0 0 2024-05-12T04:01:37.033 INFO:tasks.workunit.client.0.smithi032.stdout:6/826: write d7/d9/d48/d67/d25/f29 [2246938,57249] 0 2024-05-12T04:01:37.035 INFO:tasks.workunit.client.0.smithi032.stdout:0/883: mknod df/d22/c12a 0 2024-05-12T04:01:37.037 INFO:tasks.workunit.client.0.smithi032.stdout:2/846: mknod d1/d4d/c118 0 2024-05-12T04:01:37.037 INFO:tasks.workunit.client.0.smithi032.stdout:0/884: write df/d10/d3d/d4a/d9a/d9d/f115 [830426,110541] 0 2024-05-12T04:01:37.039 INFO:tasks.workunit.client.0.smithi032.stdout:2/847: chown d1/d34/d96/d41/d58/c8d 86 1 2024-05-12T04:01:37.039 INFO:tasks.workunit.client.0.smithi032.stdout:1/848: creat d1/d7/d30/d61/d103/f116 x:0 0 0 2024-05-12T04:01:37.039 INFO:tasks.workunit.client.0.smithi032.stdout:5/990: link d4/d7/d14/d39/d2b/d6e/l83 d4/d7/d14/dfa/l157 0 2024-05-12T04:01:37.042 INFO:tasks.workunit.client.0.smithi032.stdout:5/991: readlink d4/d7/d14/d26/d33/d5f/d7f/lcc 0 2024-05-12T04:01:37.043 INFO:tasks.workunit.client.0.smithi032.stdout:2/848: truncate d1/d13/d6b/f113 815044 0 2024-05-12T04:01:37.045 INFO:tasks.workunit.client.0.smithi032.stdout:5/992: dread - d4/d7/d14/d26/d33/d4a/d38/d79/f130 zero size 2024-05-12T04:01:37.050 INFO:tasks.workunit.client.0.smithi032.stdout:6/827: dread f6 [0,4194304] 0 2024-05-12T04:01:37.052 INFO:tasks.workunit.client.0.smithi032.stdout:0/885: mknod df/d10/d3d/d4a/d84/dc0/c12b 0 2024-05-12T04:01:37.056 INFO:tasks.workunit.client.0.smithi032.stdout:5/993: rename d4/d7/d14/d39/d1b/da4/lfd to d4/d7/d14/d26/d47/d132/l158 0 2024-05-12T04:01:37.058 INFO:tasks.workunit.client.0.smithi032.stdout:3/961: creat d15/d20/f14f x:0 0 0 2024-05-12T04:01:37.059 INFO:tasks.workunit.client.0.smithi032.stdout:8/853: dread d0/d9/d1b/d27/d88/f79 [0,4194304] 0 2024-05-12T04:01:37.062 INFO:tasks.workunit.client.0.smithi032.stdout:1/849: creat d1/d7/d113/f117 x:0 0 0 2024-05-12T04:01:37.064 INFO:tasks.workunit.client.0.smithi032.stdout:2/849: link d1/d34/d95/f109 d1/d34/d96/d41/d2c/dd4/f119 0 2024-05-12T04:01:37.065 INFO:tasks.workunit.client.0.smithi032.stdout:2/850: chown d1/d34/d96/d41/d2c/l65 116 1 2024-05-12T04:01:37.067 INFO:tasks.workunit.client.0.smithi032.stdout:8/854: mkdir d0/d9/d13/d92/d124 0 2024-05-12T04:01:37.070 INFO:tasks.workunit.client.0.smithi032.stdout:3/962: creat d15/d24/d2a/d8c/db1/dc8/de4/f150 x:0 0 0 2024-05-12T04:01:37.072 INFO:tasks.workunit.client.0.smithi032.stdout:1/850: write d1/d25/da5/db5/fda [163335,27873] 0 2024-05-12T04:01:37.075 INFO:tasks.workunit.client.0.smithi032.stdout:5/994: creat d4/d7/d14/d26/d33/d88/f159 x:0 0 0 2024-05-12T04:01:37.076 INFO:tasks.workunit.client.0.smithi032.stdout:8/855: dwrite d0/d9/d1b/d27/d88/d56/d63/d8f/d9a/d28/f49 [0,4194304] 0 2024-05-12T04:01:37.078 INFO:tasks.workunit.client.0.smithi032.stdout:3/963: unlink d15/d20/f11f 0 2024-05-12T04:01:37.078 INFO:tasks.workunit.client.0.smithi032.stdout:8/856: chown d0/d9/d1b/d27/d88/d56/d63/d8f/db0/c11b 93645421 1 2024-05-12T04:01:37.083 INFO:tasks.workunit.client.0.smithi032.stdout:3/964: truncate d15/d20/d26/d6a/d6e/d7d/fd5 334105 0 2024-05-12T04:01:37.094 INFO:tasks.workunit.client.0.smithi032.stdout:3/965: rename d15/c73 to d15/d1b/d43/dd0/c151 0 2024-05-12T04:01:37.094 INFO:tasks.workunit.client.0.smithi032.stdout:8/857: dread d0/d9/d1b/d27/d4c/d4e/d7f/f9c [0,4194304] 0 2024-05-12T04:01:37.095 INFO:tasks.workunit.client.0.smithi032.stdout:3/966: chown d15/d1b/d43/l4a 32113792 1 2024-05-12T04:01:37.096 INFO:tasks.workunit.client.0.smithi032.stdout:5/995: dread d4/f22 [8388608,4194304] 0 2024-05-12T04:01:37.097 INFO:tasks.workunit.client.0.smithi032.stdout:6/828: truncate d7/d9/d48/fbf 198612 0 2024-05-12T04:01:37.103 INFO:tasks.workunit.client.0.smithi032.stdout:3/967: creat d15/d20/d8b/d9b/f152 x:0 0 0 2024-05-12T04:01:37.106 INFO:tasks.workunit.client.0.smithi032.stdout:2/851: dwrite d1/d34/d96/d41/f3a [0,4194304] 0 2024-05-12T04:01:37.108 INFO:tasks.workunit.client.0.smithi032.stdout:8/858: dread d0/d9/d13/d21/d35/de9/f10a [0,4194304] 0 2024-05-12T04:01:37.116 INFO:tasks.workunit.client.0.smithi032.stdout:3/968: read d15/f4e [3901488,90350] 0 2024-05-12T04:01:37.124 INFO:tasks.workunit.client.0.smithi032.stdout:6/829: creat d7/d9/d113/d117/f127 x:0 0 0 2024-05-12T04:01:37.125 INFO:tasks.workunit.client.0.smithi032.stdout:6/830: dread - d7/d97/fa4 zero size 2024-05-12T04:01:37.133 INFO:tasks.workunit.client.0.smithi032.stdout:8/859: dread d0/f2 [0,4194304] 0 2024-05-12T04:01:37.136 INFO:tasks.workunit.client.0.smithi032.stdout:3/969: creat d15/d14d/f153 x:0 0 0 2024-05-12T04:01:37.139 INFO:tasks.workunit.client.0.smithi032.stdout:2/852: dread d1/d13/f2f [0,4194304] 0 2024-05-12T04:01:37.139 INFO:tasks.workunit.client.0.smithi032.stdout:5/996: getdents d4/d7/d14/d26/d33/d72/dc9/dec 0 2024-05-12T04:01:37.142 INFO:tasks.workunit.client.0.smithi032.stdout:6/831: dread d7/d9/d48/d67/d8d/dbc/f68 [0,4194304] 0 2024-05-12T04:01:37.142 INFO:tasks.workunit.client.0.smithi032.stdout:8/860: mknod d0/d97/c125 0 2024-05-12T04:01:37.143 INFO:tasks.workunit.client.0.smithi032.stdout:3/970: dread d15/d20/d26/d6a/f77 [0,4194304] 0 2024-05-12T04:01:37.150 INFO:tasks.workunit.client.0.smithi032.stdout:6/832: mkdir d7/d9/d48/d67/d63/d64/d94/d128 0 2024-05-12T04:01:37.151 INFO:tasks.workunit.client.0.smithi032.stdout:6/833: write d7/d9/d5c/d8b/da1/fae [2836344,8103] 0 2024-05-12T04:01:37.154 INFO:tasks.workunit.client.0.smithi032.stdout:5/997: rename d4/d7/d14/d26/d33/d4a/d38 to d4/d7/d14/d26/d33/d88/dfb/d15a 0 2024-05-12T04:01:37.155 INFO:tasks.workunit.client.0.smithi032.stdout:5/998: chown d4/d7/d14/d39/d1b/da4/dd0/c10f 3435272 1 2024-05-12T04:01:37.159 INFO:tasks.workunit.client.0.smithi032.stdout:3/971: fdatasync d15/d20/d22/d7a/f106 0 2024-05-12T04:01:37.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:36 smithi033 ceph-mon[49525]: pgmap v46: 65 pgs: 65 active+clean; 3.6 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 86 MiB/s rd, 192 MiB/s wr, 470 op/s 2024-05-12T04:01:37.165 INFO:tasks.workunit.client.0.smithi032.stdout:6/834: stat d7/d9/f4a 0 2024-05-12T04:01:37.167 INFO:tasks.workunit.client.0.smithi032.stdout:6/835: chown d7/d9/d48/d67/d63/d64/d94/f101 116925 1 2024-05-12T04:01:37.169 INFO:tasks.workunit.client.0.smithi032.stdout:3/972: dread d15/d24/d2a/d5b/f71 [0,4194304] 0 2024-05-12T04:01:37.175 INFO:tasks.workunit.client.0.smithi032.stdout:3/973: dread d15/d33/d3f/f87 [0,4194304] 0 2024-05-12T04:01:37.175 INFO:tasks.workunit.client.0.smithi032.stdout:3/974: stat d15/d33/d3f/f132 0 2024-05-12T04:01:37.177 INFO:tasks.workunit.client.0.smithi032.stdout:6/836: rename d7/d9/d48/d67/caf to d7/d9/d5c/d8b/da1/dfb/c129 0 2024-05-12T04:01:37.178 INFO:tasks.workunit.client.0.smithi032.stdout:2/853: getdents d1/d34/d96/d41/d58/de1 0 2024-05-12T04:01:37.179 INFO:tasks.workunit.client.0.smithi032.stdout:8/861: getdents d0/d9/d1b/d66 0 2024-05-12T04:01:37.180 INFO:tasks.workunit.client.0.smithi032.stdout:5/999: link d4/d7/d14/d26/d33/d72/d92/fdb d4/d7/d14/d26/d33/d72/d92/d111/f15b 0 2024-05-12T04:01:37.182 INFO:tasks.workunit.client.0.smithi032.stdout:6/837: mknod d7/d9/d48/d67/d1a/c12a 0 2024-05-12T04:01:37.183 INFO:tasks.workunit.client.0.smithi032.stdout:6/838: write d7/fc7 [2713208,66897] 0 2024-05-12T04:01:37.185 INFO:tasks.workunit.client.0.smithi032.stdout:3/975: mkdir d15/d20/d22/d154 0 2024-05-12T04:01:37.187 INFO:tasks.workunit.client.0.smithi032.stdout:2/854: symlink d1/d13/d70/d85/df3/l11a 0 2024-05-12T04:01:37.192 INFO:tasks.workunit.client.0.smithi032.stdout:6/839: truncate d7/d9/d48/d67/f2c 956296 0 2024-05-12T04:01:37.192 INFO:tasks.workunit.client.0.smithi032.stdout:8/862: write d0/d9/d1b/d33/f45 [6136535,118899] 0 2024-05-12T04:01:37.194 INFO:tasks.workunit.client.0.smithi032.stdout:6/840: dread - d7/d9/d5c/d62/f11b zero size 2024-05-12T04:01:37.195 INFO:tasks.workunit.client.0.smithi032.stdout:3/976: mknod d15/d33/c155 0 2024-05-12T04:01:37.195 INFO:tasks.workunit.client.0.smithi032.stdout:2/855: dwrite d1/d13/d6b/f113 [0,4194304] 0 2024-05-12T04:01:37.199 INFO:tasks.workunit.client.0.smithi032.stdout:8/863: rmdir d0/d9/d1b/d27/d59 39 2024-05-12T04:01:37.201 INFO:tasks.workunit.client.0.smithi032.stdout:6/841: chown d7/d9/c3c 0 1 2024-05-12T04:01:37.207 INFO:tasks.workunit.client.0.smithi032.stdout:8/864: dread - d0/d9/d1b/d33/d73/db7/fef zero size 2024-05-12T04:01:37.208 INFO:tasks.workunit.client.0.smithi032.stdout:8/865: stat d0/d9/ce4 0 2024-05-12T04:01:37.210 INFO:tasks.workunit.client.0.smithi032.stdout:3/977: symlink d15/d24/d93/d134/l156 0 2024-05-12T04:01:37.212 INFO:tasks.workunit.client.0.smithi032.stdout:3/978: write d15/d20/d26/d6a/d6e/d7d/d9a/f9c [3264076,46300] 0 2024-05-12T04:01:37.215 INFO:tasks.workunit.client.0.smithi032.stdout:8/866: creat d0/d9/d1b/d33/d73/db7/f126 x:0 0 0 2024-05-12T04:01:37.215 INFO:tasks.workunit.client.0.smithi032.stdout:2/856: getdents d1/d73 0 2024-05-12T04:01:37.217 INFO:tasks.workunit.client.0.smithi032.stdout:6/842: rmdir d7/d9/d48/d67/d63/d64/d94/d11c 0 2024-05-12T04:01:37.221 INFO:tasks.workunit.client.0.smithi032.stdout:8/867: rmdir d0/d9/d13 39 2024-05-12T04:01:37.224 INFO:tasks.workunit.client.0.smithi032.stdout:2/857: symlink d1/d13/l11b 0 2024-05-12T04:01:37.225 INFO:tasks.workunit.client.0.smithi032.stdout:6/843: readlink d7/d9/d48/d67/d63/d64/d94/lb2 0 2024-05-12T04:01:37.231 INFO:tasks.workunit.client.0.smithi032.stdout:3/979: dread d15/d33/d3f/f132 [0,4194304] 0 2024-05-12T04:01:37.232 INFO:tasks.workunit.client.0.smithi032.stdout:6/844: fdatasync d7/d33/f11f 0 2024-05-12T04:01:37.237 INFO:tasks.workunit.client.0.smithi032.stdout:8/868: getdents d0/d4b/dad/d115 0 2024-05-12T04:01:37.240 INFO:tasks.workunit.client.0.smithi032.stdout:2/858: link d1/d34/d96/d41/d2c/f37 d1/d13/da0/f11c 0 2024-05-12T04:01:37.241 INFO:tasks.workunit.client.0.smithi032.stdout:2/859: write d1/d34/d96/d41/f3a [2538016,95585] 0 2024-05-12T04:01:37.248 INFO:tasks.workunit.client.0.smithi032.stdout:3/980: rename d15/d1b/f111 to d15/d33/d3f/d75/f157 0 2024-05-12T04:01:37.248 INFO:tasks.workunit.client.0.smithi032.stdout:8/869: creat d0/d9/d1b/d27/d88/d56/d63/d8f/d9a/f127 x:0 0 0 2024-05-12T04:01:37.250 INFO:tasks.workunit.client.0.smithi032.stdout:8/870: fdatasync d0/d9/d1b/d66/d75/f9b 0 2024-05-12T04:01:37.250 INFO:tasks.workunit.client.0.smithi032.stdout:6/845: mknod d7/d9/d48/d67/d1a/d93/dde/c12b 0 2024-05-12T04:01:37.251 INFO:tasks.workunit.client.0.smithi032.stdout:8/871: chown d0/d4b/dad/d115 5110 1 2024-05-12T04:01:37.255 INFO:tasks.workunit.client.0.smithi032.stdout:6/846: symlink d7/d9/l12c 0 2024-05-12T04:01:37.260 INFO:tasks.workunit.client.0.smithi032.stdout:8/872: creat d0/d9/d1b/d33/f128 x:0 0 0 2024-05-12T04:01:37.261 INFO:tasks.workunit.client.0.smithi032.stdout:3/981: write d15/d20/d26/f110 [681162,55415] 0 2024-05-12T04:01:37.264 INFO:tasks.workunit.client.0.smithi032.stdout:8/873: fdatasync d0/d9/d1b/d27/d88/d56/d63/d8f/d9a/d28/f82 0 2024-05-12T04:01:37.270 INFO:tasks.workunit.client.0.smithi032.stdout:3/982: write d15/d20/d22/d7a/dbe/d121/f141 [4731205,54322] 0 2024-05-12T04:01:37.270 INFO:tasks.workunit.client.0.smithi032.stdout:6/847: getdents d7/d9/d48/d67/d8d/dbc/de2/d9d 0 2024-05-12T04:01:37.273 INFO:tasks.workunit.client.0.smithi032.stdout:3/983: creat d15/d1b/d43/f158 x:0 0 0 2024-05-12T04:01:37.274 INFO:tasks.workunit.client.0.smithi032.stdout:6/848: truncate d7/d9/d48/d67/d8d/dbc/f9a 344252 0 2024-05-12T04:01:37.276 INFO:tasks.workunit.client.0.smithi032.stdout:3/984: read d15/d33/f9e [1579267,126628] 0 2024-05-12T04:01:37.276 INFO:tasks.workunit.client.0.smithi032.stdout:8/874: getdents d0/d9/d1b/d27/d59 0 2024-05-12T04:01:37.279 INFO:tasks.workunit.client.0.smithi032.stdout:3/985: read d15/d20/d26/d6a/d6e/f133 [80183,35699] 0 2024-05-12T04:01:37.280 INFO:tasks.workunit.client.0.smithi032.stdout:6/849: fsync d7/d9/d48/d67/d8d/dbc/de2/f5a 0 2024-05-12T04:01:37.280 INFO:tasks.workunit.client.0.smithi032.stdout:3/986: readlink d15/d24/d40/lea 0 2024-05-12T04:01:37.281 INFO:tasks.workunit.client.0.smithi032.stdout:6/850: chown d7/d97/fc5 5 1 2024-05-12T04:01:37.282 INFO:tasks.workunit.client.0.smithi032.stdout:8/875: write d0/d9/d13/d21/d35/de9/f10a [2917659,23272] 0 2024-05-12T04:01:37.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:36 smithi032 ceph-mon[40008]: pgmap v46: 65 pgs: 65 active+clean; 3.6 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 86 MiB/s rd, 192 MiB/s wr, 470 op/s 2024-05-12T04:01:37.293 INFO:tasks.workunit.client.0.smithi032.stdout:3/987: dread d15/d33/d3f/fb3 [0,4194304] 0 2024-05-12T04:01:37.298 INFO:tasks.workunit.client.0.smithi032.stdout:6/851: write d7/d9/d48/d67/d63/f78 [792559,101740] 0 2024-05-12T04:01:37.301 INFO:tasks.workunit.client.0.smithi032.stdout:8/876: dread d0/f68 [0,4194304] 0 2024-05-12T04:01:37.306 INFO:tasks.workunit.client.0.smithi032.stdout:8/877: fsync d0/d9/d13/d21/d25/f83 0 2024-05-12T04:01:37.307 INFO:tasks.workunit.client.0.smithi032.stdout:3/988: write d15/d20/d22/d4f/f124 [11979,66019] 0 2024-05-12T04:01:37.314 INFO:tasks.workunit.client.0.smithi032.stdout:3/989: mkdir d15/dc2/d159 0 2024-05-12T04:01:37.317 INFO:tasks.workunit.client.0.smithi032.stdout:6/852: dread d7/d9/d48/d67/d1a/d20/f3d [0,4194304] 0 2024-05-12T04:01:37.322 INFO:tasks.workunit.client.0.smithi032.stdout:8/878: write d0/d9/d1b/d33/d73/fe0 [1002207,32891] 0 2024-05-12T04:01:37.324 INFO:tasks.workunit.client.0.smithi032.stdout:3/990: getdents d15/d14d 0 2024-05-12T04:01:37.325 INFO:tasks.workunit.client.0.smithi032.stdout:6/853: dwrite d7/d9/d5c/d8b/da1/fae [0,4194304] 0 2024-05-12T04:01:37.326 INFO:tasks.workunit.client.0.smithi032.stdout:3/991: chown d15/d20/d22/f23 2 1 2024-05-12T04:01:37.329 INFO:tasks.workunit.client.0.smithi032.stdout:8/879: symlink d0/d9/d1b/d27/d88/d56/d63/d8f/d9a/l129 0 2024-05-12T04:01:37.334 INFO:tasks.workunit.client.0.smithi032.stdout:3/992: rename d15/d20/d26/d6a/d6e/f133 to d15/d20/d22/d7a/dbe/d121/f15a 0 2024-05-12T04:01:37.335 INFO:tasks.workunit.client.0.smithi032.stdout:6/854: creat d7/d9/d5c/f12d x:0 0 0 2024-05-12T04:01:37.338 INFO:tasks.workunit.client.0.smithi032.stdout:6/855: fsync d7/d9/d48/d67/d1a/d20/fab 0 2024-05-12T04:01:37.352 INFO:tasks.workunit.client.0.smithi032.stdout:3/993: dwrite d15/d33/f46 [4194304,4194304] 0 2024-05-12T04:01:37.356 INFO:tasks.workunit.client.0.smithi032.stdout:6/856: dwrite d7/d97/f99 [0,4194304] 0 2024-05-12T04:01:37.372 INFO:tasks.workunit.client.0.smithi032.stdout:3/994: mkdir d15/d24/d2a/d5b/d15b 0 2024-05-12T04:01:37.373 INFO:tasks.workunit.client.0.smithi032.stdout:6/857: mkdir d7/d9/d48/d67/d25/d12e 0 2024-05-12T04:01:37.409 INFO:tasks.workunit.client.0.smithi032.stdout:3/995: mkdir d15/d33/d3f/d75/d11e/d15c 0 2024-05-12T04:01:37.412 INFO:tasks.workunit.client.0.smithi032.stdout:3/996: dread - d15/d24/f105 zero size 2024-05-12T04:01:37.412 INFO:tasks.workunit.client.0.smithi032.stdout:3/997: chown d15/d20/d22/d154 1952060 1 2024-05-12T04:01:37.414 INFO:tasks.workunit.client.0.smithi032.stdout:6/858: link d7/d9/d48/d67/d8d/dbc/de2/db9/lbb d7/d9/d48/d67/d25/d12e/l12f 0 2024-05-12T04:01:37.416 INFO:tasks.workunit.client.0.smithi032.stdout:6/859: stat d7/d9/d48/d67/d8d/dbc/de2/dd4/fb7 0 2024-05-12T04:01:37.419 INFO:tasks.workunit.client.0.smithi032.stdout:6/860: chown d7/d9/d48/d67/d63/d64/d94/de4/f14 26192286 1 2024-05-12T04:01:37.420 INFO:tasks.workunit.client.0.smithi032.stdout:6/861: chown d7/d9/d48/d67/d63/d119 1127614861 1 2024-05-12T04:01:37.423 INFO:tasks.workunit.client.0.smithi032.stdout:3/998: truncate d15/d20/d26/d6a/d6e/d7d/fa1 2991165 0 2024-05-12T04:01:37.424 INFO:tasks.workunit.client.0.smithi032.stdout:6/862: symlink d7/d9/d113/l130 0 2024-05-12T04:01:37.441 INFO:tasks.workunit.client.0.smithi032.stdout:6/863: getdents d7/d9/d48/d67/d1a/da0 0 2024-05-12T04:01:37.448 INFO:tasks.workunit.client.0.smithi032.stdout:6/864: dwrite d7/d9/d48/d67/d63/d64/d94/fce [0,4194304] 0 2024-05-12T04:01:37.449 INFO:tasks.workunit.client.0.smithi032.stdout:6/865: fdatasync d7/d9/d48/d67/d25/f74 0 2024-05-12T04:01:37.457 INFO:tasks.workunit.client.0.smithi032.stdout:3/999: write d15/d20/d26/d6a/d6e/d7d/fa1 [632525,26771] 0 2024-05-12T04:01:37.458 INFO:tasks.workunit.client.0.smithi032.stdout:6/866: mkdir d7/d9/d11a/d131 0 2024-05-12T04:01:37.461 INFO:tasks.workunit.client.0.smithi032.stdout:6/867: dread d7/d9/d48/faa [8388608,4194304] 0 2024-05-12T04:01:37.463 INFO:tasks.workunit.client.0.smithi032.stdout:6/868: write d7/d9/d5c/d8b/f120 [2452228,23486] 0 2024-05-12T04:01:37.467 INFO:tasks.workunit.client.0.smithi032.stdout:6/869: dread - d7/d9/fd0 zero size 2024-05-12T04:01:37.473 INFO:tasks.workunit.client.0.smithi032.stdout:6/870: write d7/d9/d48/d67/d8d/dbc/f68 [1033876,18436] 0 2024-05-12T04:01:37.477 INFO:tasks.workunit.client.0.smithi032.stdout:6/871: getdents d7/d9/d5c 0 2024-05-12T04:01:37.480 INFO:tasks.workunit.client.0.smithi032.stdout:6/872: mknod d7/d9/d48/d67/d1a/d93/df5/c132 0 2024-05-12T04:01:37.486 INFO:tasks.workunit.client.0.smithi032.stdout:6/873: symlink d7/d9/d48/d67/d8d/l133 0 2024-05-12T04:01:37.486 INFO:tasks.workunit.client.0.smithi032.stdout:6/874: chown d7/d9/d5c/d62/cdb 1643395 1 2024-05-12T04:01:37.502 INFO:tasks.workunit.client.0.smithi032.stdout:6/875: dwrite d7/d97/f9e [0,4194304] 0 2024-05-12T04:01:37.508 INFO:tasks.workunit.client.0.smithi032.stdout:6/876: write d7/d9/d48/fbe [920870,17483] 0 2024-05-12T04:01:37.534 INFO:tasks.workunit.client.0.smithi032.stdout:6/877: dread d7/fa [0,4194304] 0 2024-05-12T04:01:37.535 INFO:tasks.workunit.client.0.smithi032.stdout:6/878: chown d7/d9/d48/d67/d1a/da0/dc6/dfd 1 1 2024-05-12T04:01:37.535 INFO:tasks.workunit.client.0.smithi032.stdout:6/879: fdatasync d7/d9/d48/fff 0 2024-05-12T04:01:37.539 INFO:tasks.workunit.client.0.smithi032.stdout:6/880: mknod d7/d9/d5c/d8b/d112/c134 0 2024-05-12T04:01:37.543 INFO:tasks.workunit.client.0.smithi032.stdout:6/881: mkdir d7/d9/d48/d67/d63/d119/d135 0 2024-05-12T04:01:37.571 INFO:tasks.workunit.client.0.smithi032.stdout:6/882: dwrite d7/d9/d48/d67/d25/f7f [0,4194304] 0 2024-05-12T04:01:37.572 INFO:tasks.workunit.client.0.smithi032.stdout:6/883: dread - d7/d97/fa4 zero size 2024-05-12T04:01:37.581 INFO:tasks.workunit.client.0.smithi032.stdout:6/884: truncate d7/d9/d48/d67/d8d/dbc/de2/dd4/fb7 1070963 0 2024-05-12T04:01:37.583 INFO:tasks.workunit.client.0.smithi032.stdout:6/885: truncate d7/d9/d5c/d8b/fa9 4682585 0 2024-05-12T04:01:37.586 INFO:tasks.workunit.client.0.smithi032.stdout:6/886: rename d7/d9/d5c/d62/f11b to d7/d97/f136 0 2024-05-12T04:01:37.589 INFO:tasks.workunit.client.0.smithi032.stdout:6/887: mkdir d7/d9/d48/d67/d63/d64/d94/d137 0 2024-05-12T04:01:37.591 INFO:tasks.workunit.client.0.smithi032.stdout:6/888: symlink d7/d9/d48/d67/d63/d64/d94/l138 0 2024-05-12T04:01:37.602 INFO:tasks.workunit.client.0.smithi032.stdout:6/889: write d7/d97/fc5 [294412,15581] 0 2024-05-12T04:01:37.609 INFO:tasks.workunit.client.0.smithi032.stdout:6/890: write d7/d9/d48/d67/d8d/fb5 [200965,83575] 0 2024-05-12T04:01:37.614 INFO:tasks.workunit.client.0.smithi032.stdout:6/891: write d7/d9/d5c/fbd [1248758,71354] 0 2024-05-12T04:01:37.616 INFO:tasks.workunit.client.0.smithi032.stdout:6/892: chown d7/d9/d48/d67/d1a/c91 46 1 2024-05-12T04:01:37.617 INFO:tasks.workunit.client.0.smithi032.stdout:6/893: read d7/d9/d48/d67/d1a/d20/f3d [2770456,77331] 0 2024-05-12T04:01:37.617 INFO:tasks.workunit.client.0.smithi032.stdout:6/894: read - d7/d9/fd0 zero size 2024-05-12T04:01:37.619 INFO:tasks.workunit.client.0.smithi032.stdout:6/895: chown d7/d9/d48/d67/d8d/dbc/f108 3461 1 2024-05-12T04:01:37.624 INFO:tasks.workunit.client.0.smithi032.stdout:6/896: truncate d7/d9/d48/d67/d63/d64/d94/de4/fb4 142485 0 2024-05-12T04:01:37.626 INFO:tasks.workunit.client.0.smithi032.stdout:6/897: truncate d7/d9/d48/f5d 1460584 0 2024-05-12T04:01:37.638 INFO:tasks.workunit.client.0.smithi032.stdout:6/898: dwrite d7/d9/d48/d67/d1a/da0/fcf [0,4194304] 0 2024-05-12T04:01:37.642 INFO:tasks.workunit.client.0.smithi032.stdout:6/899: mkdir d7/d9/d48/d67/d8d/dbc/de2/dd4/d139 0 2024-05-12T04:01:37.648 INFO:tasks.workunit.client.0.smithi032.stdout:6/900: rename d7/d9/d48/d67/d1a/c12a to d7/d9/d48/d67/d63/d64/c13a 0 2024-05-12T04:01:37.648 INFO:tasks.workunit.client.0.smithi032.stdout:6/901: readlink d7/d9/l111 0 2024-05-12T04:01:37.654 INFO:tasks.workunit.client.0.smithi032.stdout:6/902: link d7/d9/d5c/d62/c77 d7/d9/d5c/d8b/d112/c13b 0 2024-05-12T04:01:37.657 INFO:tasks.workunit.client.0.smithi032.stdout:6/903: mknod d7/d9/d5c/d8b/da1/dfb/c13c 0 2024-05-12T04:01:37.662 INFO:tasks.workunit.client.0.smithi032.stdout:6/904: creat d7/d9/d48/d67/d8d/dbc/de2/f13d x:0 0 0 2024-05-12T04:01:37.666 INFO:tasks.workunit.client.0.smithi032.stdout:6/905: link d7/d9/d48/d67/d63/d64/d94/de4/f122 d7/d9/d48/d67/d63/d64/d94/d128/f13e 0 2024-05-12T04:01:37.668 INFO:tasks.workunit.client.0.smithi032.stdout:6/906: symlink d7/d9/d5c/l13f 0 2024-05-12T04:01:37.682 INFO:tasks.workunit.client.0.smithi032.stdout:9/744: sync 2024-05-12T04:01:37.683 INFO:tasks.workunit.client.0.smithi032.stdout:6/907: dread d7/d9/d48/d67/f45 [0,4194304] 0 2024-05-12T04:01:37.688 INFO:tasks.workunit.client.0.smithi032.stdout:6/908: rename d7/d9/d48/d67/d8d/dbc/de2/dd4/d139 to d7/d9/d48/d121/d140 0 2024-05-12T04:01:37.689 INFO:tasks.workunit.client.0.smithi032.stdout:9/745: dwrite d0/d3/d74/f20 [0,4194304] 0 2024-05-12T04:01:37.699 INFO:tasks.workunit.client.0.smithi032.stdout:6/909: write d7/d9/d48/d67/d8d/dbc/de2/dd4/fdd [472899,93375] 0 2024-05-12T04:01:37.700 INFO:tasks.workunit.client.0.smithi032.stdout:9/746: write d0/d3/d6/d7/d2a/d46/d13/fb6 [804516,38931] 0 2024-05-12T04:01:37.712 INFO:tasks.workunit.client.0.smithi032.stdout:9/747: rename d0/d3/d6/d7/da/d89/cac to d0/d3/d6/d7/d2a/d46/d91/cec 0 2024-05-12T04:01:37.714 INFO:tasks.workunit.client.0.smithi032.stdout:9/748: stat d0/d3/d4b/d88/f96 0 2024-05-12T04:01:37.718 INFO:tasks.workunit.client.0.smithi032.stdout:6/910: dwrite d7/d9/d48/d67/d1a/f73 [0,4194304] 0 2024-05-12T04:01:37.718 INFO:tasks.workunit.client.0.smithi032.stdout:9/749: dread - d0/d3/d6/d7/d2a/fb3 zero size 2024-05-12T04:01:37.723 INFO:tasks.workunit.client.0.smithi032.stdout:9/750: rename d0/d3/d4b/l5f to d0/led 0 2024-05-12T04:01:37.730 INFO:tasks.workunit.client.0.smithi032.stdout:6/911: rename d7/d9/f6c to d7/d9/d48/d67/d25/d12e/f141 0 2024-05-12T04:01:37.731 INFO:tasks.workunit.client.0.smithi032.stdout:9/751: truncate d0/d3/d6/d7/d2a/f68 1220612 0 2024-05-12T04:01:37.734 INFO:tasks.workunit.client.0.smithi032.stdout:6/912: write d7/d9/d48/d67/d63/d64/d94/de4/f6d [1031881,58524] 0 2024-05-12T04:01:37.735 INFO:tasks.workunit.client.0.smithi032.stdout:9/752: rename d0/d3/d6/d58/cad to d0/d3/d4b/d85/cee 0 2024-05-12T04:01:37.738 INFO:tasks.workunit.client.0.smithi032.stdout:6/913: creat d7/d9/d48/d67/d1a/d93/dde/f142 x:0 0 0 2024-05-12T04:01:37.742 INFO:tasks.workunit.client.0.smithi032.stdout:9/753: dwrite d0/d84/d8a/da3/fa4 [0,4194304] 0 2024-05-12T04:01:37.743 INFO:tasks.workunit.client.0.smithi032.stdout:9/754: fdatasync d0/d3/d74/f6c 0 2024-05-12T04:01:37.749 INFO:tasks.workunit.client.0.smithi032.stdout:9/755: mkdir d0/d3/d6/d7/da/d89/def 0 2024-05-12T04:01:37.750 INFO:tasks.workunit.client.0.smithi032.stdout:6/914: dwrite d7/d9/d48/d67/d8d/dbc/de2/ff9 [0,4194304] 0 2024-05-12T04:01:37.764 INFO:tasks.workunit.client.0.smithi032.stdout:9/756: link d0/d3/d4b/d88/cba d0/d3/d6/cf0 0 2024-05-12T04:01:37.767 INFO:tasks.workunit.client.0.smithi032.stdout:9/757: dread - d0/d3/d6/d7/d2a/fc1 zero size 2024-05-12T04:01:37.771 INFO:tasks.workunit.client.0.smithi032.stdout:6/915: getdents d7/d9/d48/d67/d1a/da0/dc6 0 2024-05-12T04:01:37.780 INFO:tasks.workunit.client.0.smithi032.stdout:6/916: mkdir d7/d9/d48/d67/d1a/d143 0 2024-05-12T04:01:37.784 INFO:tasks.workunit.client.0.smithi032.stdout:6/917: chown d7/d9/d48/f5d 2 1 2024-05-12T04:01:37.784 INFO:tasks.workunit.client.0.smithi032.stdout:9/758: dwrite d0/d3/d6/d7/da/f72 [4194304,4194304] 0 2024-05-12T04:01:37.789 INFO:tasks.workunit.client.0.smithi032.stdout:9/759: mkdir d0/d3/d4b/d88/d99/df1 0 2024-05-12T04:01:37.792 INFO:tasks.workunit.client.0.smithi032.stdout:6/918: truncate d7/d9/f4a 3049383 0 2024-05-12T04:01:37.795 INFO:tasks.workunit.client.0.smithi032.stdout:9/760: creat d0/d3/d6/d7/d2a/d46/d91/dc5/ff2 x:0 0 0 2024-05-12T04:01:37.800 INFO:tasks.workunit.client.0.smithi032.stdout:9/761: symlink d0/d3/d74/lf3 0 2024-05-12T04:01:37.801 INFO:tasks.workunit.client.0.smithi032.stdout:6/919: getdents d7/d9/d48/d121 0 2024-05-12T04:01:37.807 INFO:tasks.workunit.client.0.smithi032.stdout:6/920: rename d7/d9/d48/d67/f52 to d7/d9/d48/d67/d1a/d93/dde/f144 0 2024-05-12T04:01:37.813 INFO:tasks.workunit.client.0.smithi032.stdout:9/762: dread d0/d3/d4b/d6f/f71 [0,4194304] 0 2024-05-12T04:01:37.828 INFO:tasks.workunit.client.0.smithi032.stdout:6/921: write d7/d9/d48/d67/d1a/d93/dde/f144 [408746,60155] 0 2024-05-12T04:01:37.831 INFO:tasks.workunit.client.0.smithi032.stdout:6/922: creat d7/d9/d5c/d8b/da1/dfb/f145 x:0 0 0 2024-05-12T04:01:37.833 INFO:tasks.workunit.client.0.smithi032.stdout:6/923: mknod d7/d9/d5c/d8b/da1/c146 0 2024-05-12T04:01:37.837 INFO:tasks.workunit.client.0.smithi032.stdout:6/924: mkdir d7/d9/d48/d67/d63/d64/d94/d147 0 2024-05-12T04:01:37.837 INFO:tasks.workunit.client.0.smithi032.stdout:9/763: dread d0/d3/d4b/fcc [0,4194304] 0 2024-05-12T04:01:37.842 INFO:tasks.workunit.client.0.smithi032.stdout:9/764: rmdir d0/d3/d6/d7/da/d9e 0 2024-05-12T04:01:37.853 INFO:tasks.workunit.client.0.smithi032.stdout:6/925: dread d7/d9/d48/d67/f6e [0,4194304] 0 2024-05-12T04:01:37.856 INFO:tasks.workunit.client.0.smithi032.stdout:9/765: dwrite d0/d3/d74/d51/f7f [0,4194304] 0 2024-05-12T04:01:37.858 INFO:tasks.workunit.client.0.smithi032.stdout:6/926: creat d7/d9/d48/d67/d1a/da0/f148 x:0 0 0 2024-05-12T04:01:37.867 INFO:tasks.workunit.client.0.smithi032.stdout:9/766: dread d0/d3/d6/d7/da/f53 [0,4194304] 0 2024-05-12T04:01:37.869 INFO:tasks.workunit.client.0.smithi032.stdout:6/927: creat d7/d9/d48/d67/d1a/f149 x:0 0 0 2024-05-12T04:01:37.871 INFO:tasks.workunit.client.0.smithi032.stdout:6/928: rmdir d7/d9/d48/d121 39 2024-05-12T04:01:37.874 INFO:tasks.workunit.client.0.smithi032.stdout:9/767: rename d0/d3/d6/d7/da/d39/c43 to d0/d3/d4b/d88/d99/cf4 0 2024-05-12T04:01:37.877 INFO:tasks.workunit.client.0.smithi032.stdout:9/768: unlink d0/f6b 0 2024-05-12T04:01:37.879 INFO:tasks.workunit.client.0.smithi032.stdout:7/775: sync 2024-05-12T04:01:37.879 INFO:tasks.workunit.client.0.smithi032.stdout:4/793: sync 2024-05-12T04:01:37.883 INFO:tasks.workunit.client.0.smithi032.stdout:6/929: creat d7/d9/f14a x:0 0 0 2024-05-12T04:01:37.892 INFO:tasks.workunit.client.0.smithi032.stdout:9/769: dread d0/d3/d1b/fd0 [0,4194304] 0 2024-05-12T04:01:37.895 INFO:tasks.workunit.client.0.smithi032.stdout:6/930: rename d7/d9/d5c/d8b to d7/d9/d48/d67/d63/d64/d94/de4/d57/d14b 0 2024-05-12T04:01:37.902 INFO:tasks.workunit.client.0.smithi032.stdout:7/776: write dd/d1b/d1e/d45/d55/d72/da0/dbd/f99 [1282704,27873] 0 2024-05-12T04:01:37.902 INFO:tasks.workunit.client.0.smithi032.stdout:7/777: chown dd/d1b/d1e/fd1 37508 1 2024-05-12T04:01:37.902 INFO:tasks.workunit.client.0.smithi032.stdout:4/794: dread d0/d3/df/d4e/f92 [0,4194304] 0 2024-05-12T04:01:37.903 INFO:tasks.workunit.client.0.smithi032.stdout:9/770: rename d0/fc7 to d0/d3/d4b/d88/d99/df1/ff5 0 2024-05-12T04:01:37.907 INFO:tasks.workunit.client.0.smithi032.stdout:6/931: rmdir d7/d9/d48/d67/d1a/d93 39 2024-05-12T04:01:37.910 INFO:tasks.workunit.client.0.smithi032.stdout:4/795: dwrite d0/d3/d31/d80/d84/da7/fad [0,4194304] 0 2024-05-12T04:01:37.910 INFO:tasks.workunit.client.0.smithi032.stdout:6/932: readlink d7/d9/d48/d67/d63/d64/d94/l138 0 2024-05-12T04:01:37.911 INFO:tasks.workunit.client.0.smithi032.stdout:4/796: chown d0/d3/d24/dbd/fd2 925090 1 2024-05-12T04:01:37.917 INFO:tasks.workunit.client.0.smithi032.stdout:7/778: mkdir dd/d1b/d1e/d45/d55/db7/df6 0 2024-05-12T04:01:37.922 INFO:tasks.workunit.client.0.smithi032.stdout:4/797: creat d0/d3/d31/ff2 x:0 0 0 2024-05-12T04:01:37.925 INFO:tasks.workunit.client.0.smithi032.stdout:6/933: rename d7/d97/f9e to d7/d9/d48/d67/d8d/dbc/de2/db9/f14c 0 2024-05-12T04:01:37.927 INFO:tasks.workunit.client.0.smithi032.stdout:6/934: write d7/d9/d48/d67/d63/d64/fca [787443,35565] 0 2024-05-12T04:01:37.928 INFO:tasks.workunit.client.0.smithi032.stdout:4/798: stat d0/d3/d12/fed 0 2024-05-12T04:01:37.928 INFO:tasks.workunit.client.0.smithi032.stdout:6/935: dread - d7/d97/fa4 zero size 2024-05-12T04:01:37.935 INFO:tasks.workunit.client.0.smithi032.stdout:7/779: rename dd/d1b/d1e/de3/d9a/d9c/db9/d75/c81 to dd/d1b/d1e/d45/d55/d72/da0/cf7 0 2024-05-12T04:01:37.938 INFO:tasks.workunit.client.0.smithi032.stdout:9/771: getdents d0/d3/d4b/d88 0 2024-05-12T04:01:37.938 INFO:tasks.workunit.client.0.smithi032.stdout:6/936: dread d7/d9/d48/d67/d8d/dbc/de2/d9d/ffe [0,4194304] 0 2024-05-12T04:01:37.939 INFO:tasks.workunit.client.0.smithi032.stdout:9/772: readlink d0/d3/d6/d7/d2a/lae 0 2024-05-12T04:01:37.939 INFO:tasks.workunit.client.0.smithi032.stdout:9/773: chown d0/d3/d4b/d85 161961164 1 2024-05-12T04:01:37.940 INFO:tasks.workunit.client.0.smithi032.stdout:4/799: rename d0/d3/d12/d1d/d63/f70 to d0/d3/d12/d1d/d6b/dc2/ff3 0 2024-05-12T04:01:37.941 INFO:tasks.workunit.client.0.smithi032.stdout:9/774: fsync d0/d3/d6/d7/d2a/d46/d13/fe6 0 2024-05-12T04:01:37.975 INFO:tasks.workunit.client.0.smithi032.stdout:6/937: mkdir d7/d9/d48/d67/d63/d64/d94/de4/d123/d14d 0 2024-05-12T04:01:37.976 INFO:tasks.workunit.client.0.smithi032.stdout:9/775: symlink d0/d3/d4b/d6f/dd7/lf6 0 2024-05-12T04:01:37.976 INFO:tasks.workunit.client.0.smithi032.stdout:6/938: readlink d7/d9/d48/d67/d63/d64/d94/de4/d57/d14b/da1/l10e 0 2024-05-12T04:01:37.977 INFO:tasks.workunit.client.0.smithi032.stdout:9/776: chown d0/d84/d8a 91 1 2024-05-12T04:01:38.037 INFO:tasks.workunit.client.0.smithi032.stdout:0/886: sync 2024-05-12T04:01:38.037 INFO:tasks.workunit.client.0.smithi032.stdout:1/851: sync 2024-05-12T04:01:38.054 INFO:tasks.workunit.client.0.smithi032.stdout:9/777: unlink d0/d3/d74/c83 0 2024-05-12T04:01:38.066 INFO:tasks.workunit.client.0.smithi032.stdout:9/778: truncate d0/d3/d6/f62 3214519 0 2024-05-12T04:01:38.075 INFO:tasks.workunit.client.0.smithi032.stdout:6/939: write d7/d9/d48/d67/d63/d64/f65 [5003317,545] 0 2024-05-12T04:01:38.077 INFO:tasks.workunit.client.0.smithi032.stdout:9/779: symlink d0/d84/da6/lf7 0 2024-05-12T04:01:38.078 INFO:tasks.workunit.client.0.smithi032.stdout:9/780: read - d0/d84/d8a/f94 zero size 2024-05-12T04:01:38.078 INFO:tasks.workunit.client.0.smithi032.stdout:0/887: write df/d10/fed [4049060,117501] 0 2024-05-12T04:01:38.085 INFO:tasks.workunit.client.0.smithi032.stdout:9/781: unlink d0/d3/d6/d7/d2a/d46/d13/f3c 0 2024-05-12T04:01:38.086 INFO:tasks.workunit.client.0.smithi032.stdout:9/782: fdatasync d0/d3/d6/f69 0 2024-05-12T04:01:38.087 INFO:tasks.workunit.client.0.smithi032.stdout:6/940: unlink d7/d9/d48/d67/d63/d64/d94/de4/fb4 0 2024-05-12T04:01:38.089 INFO:tasks.workunit.client.0.smithi032.stdout:6/941: chown d7/d9/f82 671320 1 2024-05-12T04:01:38.091 INFO:tasks.workunit.client.0.smithi032.stdout:9/783: unlink d0/d3/d4b/d88/fcf 0 2024-05-12T04:01:38.094 INFO:tasks.workunit.client.0.smithi032.stdout:1/852: write d1/d7/d30/d18/d87/dc6/fd0 [753949,101646] 0 2024-05-12T04:01:38.095 INFO:tasks.workunit.client.0.smithi032.stdout:9/784: mkdir d0/d3/d4b/d88/d99/df8 0 2024-05-12T04:01:38.096 INFO:tasks.workunit.client.0.smithi032.stdout:6/942: truncate d7/d9/f4a 1372346 0 2024-05-12T04:01:38.097 INFO:tasks.workunit.client.0.smithi032.stdout:6/943: chown d7/d9/d48/d67/d63/d119/d135 259806703 1 2024-05-12T04:01:38.099 INFO:tasks.workunit.client.0.smithi032.stdout:1/853: symlink d1/d6a/d72/l118 0 2024-05-12T04:01:38.102 INFO:tasks.workunit.client.0.smithi032.stdout:9/785: truncate d0/d3/d6/d7/d27/f48 561565 0 2024-05-12T04:01:38.103 INFO:tasks.workunit.client.0.smithi032.stdout:1/854: write d1/d7/d30/d61/f100 [693082,121705] 0 2024-05-12T04:01:38.107 INFO:tasks.workunit.client.0.smithi032.stdout:6/944: dwrite d7/d9/d48/d67/d1a/f149 [0,4194304] 0 2024-05-12T04:01:38.109 INFO:tasks.workunit.client.0.smithi032.stdout:9/786: chown d0/d3/cb8 49931 1 2024-05-12T04:01:38.110 INFO:tasks.workunit.client.0.smithi032.stdout:0/888: dwrite df/d10/d3d/d4a/f65 [0,4194304] 0 2024-05-12T04:01:38.113 INFO:tasks.workunit.client.0.smithi032.stdout:2/860: sync 2024-05-12T04:01:38.113 INFO:tasks.workunit.client.0.smithi032.stdout:8/880: sync 2024-05-12T04:01:38.117 INFO:tasks.workunit.client.0.smithi032.stdout:8/881: chown d0/d9/d1b/d33/d73/ca8 13731 1 2024-05-12T04:01:38.119 INFO:tasks.workunit.client.0.smithi032.stdout:8/882: chown d0/d9/d1b/d27/lc8 603 1 2024-05-12T04:01:38.121 INFO:tasks.workunit.client.0.smithi032.stdout:6/945: rename d7/d9/d48/d67/d1a/d20/la6 to d7/d9/d48/d67/d63/d64/l14e 0 2024-05-12T04:01:38.123 INFO:tasks.workunit.client.0.smithi032.stdout:1/855: dread d1/d7/d30/d18/f76 [0,4194304] 0 2024-05-12T04:01:38.126 INFO:tasks.workunit.client.0.smithi032.stdout:9/787: link d0/d3/d4b/d6f/fe2 d0/d3/d4b/d6f/dd7/ff9 0 2024-05-12T04:01:38.128 INFO:tasks.workunit.client.0.smithi032.stdout:2/861: mknod d1/d104/c11d 0 2024-05-12T04:01:38.130 INFO:tasks.workunit.client.0.smithi032.stdout:6/946: creat d7/d9/d48/d67/d63/d119/f14f x:0 0 0 2024-05-12T04:01:38.131 INFO:tasks.workunit.client.0.smithi032.stdout:6/947: write d7/d9/d5c/d62/fad [2364707,92881] 0 2024-05-12T04:01:38.133 INFO:tasks.workunit.client.0.smithi032.stdout:9/788: unlink d0/l6a 0 2024-05-12T04:01:38.136 INFO:tasks.workunit.client.0.smithi032.stdout:9/789: chown d0/d3/d6/d7/d2a/d46/d91/cec 14408277 1 2024-05-12T04:01:38.139 INFO:tasks.workunit.client.0.smithi032.stdout:0/889: dwrite fb [0,4194304] 0 2024-05-12T04:01:38.141 INFO:tasks.workunit.client.0.smithi032.stdout:0/890: chown df/d30/d36 3541709 1 2024-05-12T04:01:38.149 INFO:tasks.workunit.client.0.smithi032.stdout:8/883: creat d0/d9/d1b/d33/d73/f12a x:0 0 0 2024-05-12T04:01:38.150 INFO:tasks.workunit.client.0.smithi032.stdout:8/884: chown d0/d9/d13/d21/d25 58 1 2024-05-12T04:01:38.152 INFO:tasks.workunit.client.0.smithi032.stdout:9/790: symlink d0/d3/d6/d58/lfa 0 2024-05-12T04:01:38.153 INFO:tasks.workunit.client.0.smithi032.stdout:9/791: chown d0/d3/d74/l44 23990094 1 2024-05-12T04:01:38.154 INFO:tasks.workunit.client.0.smithi032.stdout:9/792: chown d0/d3/d6/d7/da/d89 1739224 1 2024-05-12T04:01:38.155 INFO:tasks.workunit.client.0.smithi032.stdout:0/891: creat df/d30/d32/d60/dc7/f12c x:0 0 0 2024-05-12T04:01:38.156 INFO:tasks.workunit.client.0.smithi032.stdout:9/793: stat d0/d3/d4b/d85/dbe/cdd 0 2024-05-12T04:01:38.159 INFO:tasks.workunit.client.0.smithi032.stdout:2/862: dread d1/d34/d96/d41/d2c/f59 [0,4194304] 0 2024-05-12T04:01:38.172 INFO:tasks.workunit.client.0.smithi032.stdout:6/948: dread d7/d9/d48/d67/d8d/dbc/de2/f81 [0,4194304] 0 2024-05-12T04:01:38.173 INFO:tasks.workunit.client.0.smithi032.stdout:1/856: dwrite d1/d7/f5d [0,4194304] 0 2024-05-12T04:01:38.174 INFO:tasks.workunit.client.0.smithi032.stdout:8/885: mkdir d0/d9/d13/d92/d124/d12b 0 2024-05-12T04:01:38.179 INFO:tasks.workunit.client.0.smithi032.stdout:4/800: sync 2024-05-12T04:01:38.179 INFO:tasks.workunit.client.0.smithi032.stdout:7/780: sync 2024-05-12T04:01:38.179 INFO:tasks.workunit.client.0.smithi032.stdout:1/857: mkdir d1/d119 0 2024-05-12T04:01:38.181 INFO:tasks.workunit.client.0.smithi032.stdout:1/858: chown d1/d6a/d72/da4/cdf 4403 1 2024-05-12T04:01:38.185 INFO:tasks.workunit.client.0.smithi032.stdout:6/949: mknod d7/d9/d48/d67/d63/d119/d135/c150 0 2024-05-12T04:01:38.187 INFO:tasks.workunit.client.0.smithi032.stdout:6/950: read d7/d9/d48/d67/d1a/da0/fcf [1043352,129908] 0 2024-05-12T04:01:38.190 INFO:tasks.workunit.client.0.smithi032.stdout:0/892: dwrite df/d10/d3d/f5c [0,4194304] 0 2024-05-12T04:01:38.199 INFO:tasks.workunit.client.0.smithi032.stdout:8/886: mkdir d0/d9/d1b/d66/d75/de2/d103/d108/d12c 0 2024-05-12T04:01:38.202 INFO:tasks.workunit.client.0.smithi032.stdout:7/781: dread dd/d1b/d1e/de3/d9a/d9c/db9/d75/fad [0,4194304] 0 2024-05-12T04:01:38.204 INFO:tasks.workunit.client.0.smithi032.stdout:6/951: mknod d7/d9/d48/d67/d1a/da0/dc6/c151 0 2024-05-12T04:01:38.206 INFO:tasks.workunit.client.0.smithi032.stdout:6/952: read - d7/d9/d48/d67/d63/d64/d94/ff1 zero size 2024-05-12T04:01:38.210 INFO:tasks.workunit.client.0.smithi032.stdout:6/953: dread d7/d97/f99 [0,4194304] 0 2024-05-12T04:01:38.219 INFO:tasks.workunit.client.0.smithi032.stdout:4/801: creat d0/d3/d12/d1d/d6b/db7/de0/ff4 x:0 0 0 2024-05-12T04:01:38.219 INFO:tasks.workunit.client.0.smithi032.stdout:8/887: creat d0/d7/f12d x:0 0 0 2024-05-12T04:01:38.219 INFO:tasks.workunit.client.0.smithi032.stdout:8/888: chown d0/d9/d1b/d66/d75/de2/d103/d108/d12c 42 1 2024-05-12T04:01:38.219 INFO:tasks.workunit.client.0.smithi032.stdout:7/782: rmdir dd/d1b/d1e 39 2024-05-12T04:01:38.219 INFO:tasks.workunit.client.0.smithi032.stdout:6/954: dwrite d7/d9/d48/d67/d8d/dbc/de2/ff9 [0,4194304] 0 2024-05-12T04:01:38.246 INFO:tasks.workunit.client.0.smithi032.stdout:0/893: truncate df/d10/d3d/d4a/f2d 2737687 0 2024-05-12T04:01:38.247 INFO:tasks.workunit.client.0.smithi032.stdout:4/802: creat d0/d3/d12/d1d/d6b/ff5 x:0 0 0 2024-05-12T04:01:38.255 INFO:tasks.workunit.client.0.smithi032.stdout:7/783: mkdir dd/d6c/df8 0 2024-05-12T04:01:38.261 INFO:tasks.workunit.client.0.smithi032.stdout:8/889: write d0/d9/d13/d21/d25/db5/fdc [5328835,38900] 0 2024-05-12T04:01:38.265 INFO:tasks.workunit.client.0.smithi032.stdout:1/859: link d1/d7/d49/d79/c9d d1/d7/d30/d35/dbe/c11a 0 2024-05-12T04:01:38.275 INFO:tasks.workunit.client.0.smithi032.stdout:8/890: write d0/d9/d1b/d33/f3b [1024466,69629] 0 2024-05-12T04:01:38.284 INFO:tasks.workunit.client.0.smithi032.stdout:1/860: rename d1/d7/d30/d18/d87/f89 to d1/d7/d30/d18/d93/f11b 0 2024-05-12T04:01:38.288 INFO:tasks.workunit.client.0.smithi032.stdout:6/955: link d7/d97/fa4 d7/d9/d48/d67/d8d/dbc/de2/f152 0 2024-05-12T04:01:38.288 INFO:tasks.workunit.client.0.smithi032.stdout:0/894: link df/d30/d36/l68 df/d10/d3d/d6c/l12d 0 2024-05-12T04:01:38.290 INFO:tasks.workunit.client.0.smithi032.stdout:4/803: getdents d0/d3/d31 0 2024-05-12T04:01:38.290 INFO:tasks.workunit.client.0.smithi032.stdout:0/895: write df/d22/f39 [1052031,92584] 0 2024-05-12T04:01:38.291 INFO:tasks.workunit.client.0.smithi032.stdout:7/784: rename dd/d1b to dd/d1b/d1e/de3/df9 22 2024-05-12T04:01:38.292 INFO:tasks.workunit.client.0.smithi032.stdout:4/804: fdatasync d0/d3/d12/d1d/d6b/db7/de0/ff4 0 2024-05-12T04:01:38.294 INFO:tasks.workunit.client.0.smithi032.stdout:8/891: link d0/d9/d1b/f101 d0/d9/d1b/d66/f12e 0 2024-05-12T04:01:38.294 INFO:tasks.workunit.client.0.smithi032.stdout:4/805: stat d0/db5/lee 0 2024-05-12T04:01:38.298 INFO:tasks.workunit.client.0.smithi032.stdout:7/785: chown dd/d1b/d1e/de3/d9a/d9c/db9/d75/f86 459 1 2024-05-12T04:01:38.298 INFO:tasks.workunit.client.0.smithi032.stdout:6/956: creat d7/d9/d48/d67/d1a/da0/f153 x:0 0 0 2024-05-12T04:01:38.299 INFO:tasks.workunit.client.0.smithi032.stdout:1/861: dread d1/d7/d49/f6b [0,4194304] 0 2024-05-12T04:01:38.299 INFO:tasks.workunit.client.0.smithi032.stdout:8/892: fdatasync d0/d9/d1b/d33/d73/fe0 0 2024-05-12T04:01:38.301 INFO:tasks.workunit.client.0.smithi032.stdout:0/896: dwrite df/d30/d36/ddd/f104 [0,4194304] 0 2024-05-12T04:01:38.303 INFO:tasks.workunit.client.0.smithi032.stdout:8/893: chown d0/d9/d13/d21/d25/db5/ld2 1 1 2024-05-12T04:01:38.303 INFO:tasks.workunit.client.0.smithi032.stdout:0/897: write df/d22/f39 [3018630,67887] 0 2024-05-12T04:01:38.317 INFO:tasks.workunit.client.0.smithi032.stdout:7/786: mknod dd/d1b/d1e/de3/d9a/d9c/db9/d75/cfa 0 2024-05-12T04:01:38.317 INFO:tasks.workunit.client.0.smithi032.stdout:4/806: write d0/d3/d12/d1d/fa4 [888270,46434] 0 2024-05-12T04:01:38.321 INFO:tasks.workunit.client.0.smithi032.stdout:8/894: fsync d0/d9/d1b/d27/d4c/d4e/ff0 0 2024-05-12T04:01:38.323 INFO:tasks.workunit.client.0.smithi032.stdout:6/957: creat d7/d9/d48/d121/d140/f154 x:0 0 0 2024-05-12T04:01:38.326 INFO:tasks.workunit.client.0.smithi032.stdout:4/807: dread d0/d3/d12/fa2 [0,4194304] 0 2024-05-12T04:01:38.328 INFO:tasks.workunit.client.0.smithi032.stdout:7/787: dwrite dd/d1b/d1e/d45/fed [0,4194304] 0 2024-05-12T04:01:38.330 INFO:tasks.workunit.client.0.smithi032.stdout:8/895: read d0/d9/d1b/d27/d88/d56/d63/d8f/d9a/d28/f82 [4107214,111060] 0 2024-05-12T04:01:38.333 INFO:tasks.workunit.client.0.smithi032.stdout:1/862: creat d1/dd/f11c x:0 0 0 2024-05-12T04:01:38.335 INFO:tasks.workunit.client.0.smithi032.stdout:0/898: truncate df/d10/fe1 2280387 0 2024-05-12T04:01:38.338 INFO:tasks.workunit.client.0.smithi032.stdout:6/958: creat d7/d9/d48/d67/d63/d64/d94/de4/d57/d14b/da1/dfb/f155 x:0 0 0 2024-05-12T04:01:38.340 INFO:tasks.workunit.client.0.smithi032.stdout:4/808: creat d0/d3/d31/d80/d84/ff6 x:0 0 0 2024-05-12T04:01:38.342 INFO:tasks.workunit.client.0.smithi032.stdout:7/788: fdatasync dd/d1b/d43/d5e/d37/f46 0 2024-05-12T04:01:38.344 INFO:tasks.workunit.client.0.smithi032.stdout:7/789: chown dd/d1b/d43/d5e/f6f 118753 1 2024-05-12T04:01:38.348 INFO:tasks.workunit.client.0.smithi032.stdout:6/959: write d7/d9/d48/d67/d1a/da0/fcf [4562621,69199] 0 2024-05-12T04:01:38.349 INFO:tasks.workunit.client.0.smithi032.stdout:4/809: mkdir d0/d3/d12/d8d/df7 0 2024-05-12T04:01:38.353 INFO:tasks.workunit.client.0.smithi032.stdout:8/896: mkdir d0/d9/d12f 0 2024-05-12T04:01:38.354 INFO:tasks.workunit.client.0.smithi032.stdout:1/863: dread d1/d7/d30/d35/d54/f8c [0,4194304] 0 2024-05-12T04:01:38.355 INFO:tasks.workunit.client.0.smithi032.stdout:8/897: dread - d0/d9/d1b/d33/f128 zero size 2024-05-12T04:01:38.356 INFO:tasks.workunit.client.0.smithi032.stdout:2/863: sync 2024-05-12T04:01:38.356 INFO:tasks.workunit.client.0.smithi032.stdout:9/794: sync 2024-05-12T04:01:38.357 INFO:tasks.workunit.client.0.smithi032.stdout:2/864: fdatasync d1/d13/f2b 0 2024-05-12T04:01:38.357 INFO:tasks.workunit.client.0.smithi032.stdout:7/790: mknod dd/d1b/d1e/d45/d55/db7/cfb 0 2024-05-12T04:01:38.364 INFO:tasks.workunit.client.0.smithi032.stdout:1/864: unlink d1/d7/d49/l92 0 2024-05-12T04:01:38.371 INFO:tasks.workunit.client.0.smithi032.stdout:6/960: dread d7/d9/d48/d67/d25/f2a [0,4194304] 0 2024-05-12T04:01:38.378 INFO:tasks.workunit.client.0.smithi032.stdout:2/865: creat d1/d13/d70/ddb/f11e x:0 0 0 2024-05-12T04:01:38.378 INFO:tasks.workunit.client.0.smithi032.stdout:4/810: mknod d0/d3/d12/d1d/cf8 0 2024-05-12T04:01:38.379 INFO:tasks.workunit.client.0.smithi032.stdout:1/865: rename d1/d7/d30/d61/dc8 to d1/d7/d30/d35/d77/d110/d11d 0 2024-05-12T04:01:38.380 INFO:tasks.workunit.client.0.smithi032.stdout:0/899: dwrite df/d10/d3d/d4a/f2d [0,4194304] 0 2024-05-12T04:01:38.382 INFO:tasks.workunit.client.0.smithi032.stdout:6/961: fdatasync d7/d9/d48/d67/f4c 0 2024-05-12T04:01:38.384 INFO:tasks.workunit.client.0.smithi032.stdout:6/962: write d7/fc7 [2198405,43574] 0 2024-05-12T04:01:38.385 INFO:tasks.workunit.client.0.smithi032.stdout:7/791: dread dd/d1b/d1e/de3/f7d [0,4194304] 0 2024-05-12T04:01:38.430 INFO:tasks.workunit.client.0.smithi032.stdout:8/898: creat d0/d9/d1b/d33/f130 x:0 0 0 2024-05-12T04:01:38.432 INFO:tasks.workunit.client.0.smithi032.stdout:2/866: creat d1/d4d/d5e/f11f x:0 0 0 2024-05-12T04:01:38.435 INFO:tasks.workunit.client.0.smithi032.stdout:1/866: creat d1/d7/d30/d61/f11e x:0 0 0 2024-05-12T04:01:38.440 INFO:tasks.workunit.client.0.smithi032.stdout:6/963: fsync d7/d9/d48/f5d 0 2024-05-12T04:01:38.536 INFO:tasks.workunit.client.0.smithi032.stdout:1/867: mkdir d1/d7/d30/d35/d77/d110/d11d/d11f 0 2024-05-12T04:01:38.536 INFO:tasks.workunit.client.0.smithi032.stdout:1/868: truncate d1/d7/f78 3202607 0 2024-05-12T04:01:38.546 INFO:tasks.workunit.client.0.smithi032.stdout:6/964: creat d7/d9/d48/d67/d1a/d20/f156 x:0 0 0 2024-05-12T04:01:38.546 INFO:tasks.workunit.client.0.smithi032.stdout:6/965: stat d7/d9/d113/l130 0 2024-05-12T04:01:38.554 INFO:tasks.workunit.client.0.smithi032.stdout:0/900: dwrite df/d10/f64 [0,4194304] 0 2024-05-12T04:01:38.562 INFO:tasks.workunit.client.0.smithi032.stdout:2/867: write d1/d34/d3e/d67/d10c/fc0 [1881848,126706] 0 2024-05-12T04:01:38.566 INFO:tasks.workunit.client.0.smithi032.stdout:8/899: dwrite d0/d9/d1b/f4d [4194304,4194304] 0 2024-05-12T04:01:38.566 INFO:tasks.workunit.client.0.smithi032.stdout:7/792: dwrite dd/d1b/d1e/f68 [0,4194304] 0 2024-05-12T04:01:38.567 INFO:tasks.workunit.client.0.smithi032.stdout:9/795: truncate d0/d3/d6/d7/da/f53 1889863 0 2024-05-12T04:01:38.570 INFO:tasks.workunit.client.0.smithi032.stdout:7/793: write dd/d1b/d1e/de3/d9a/d9c/dc2/fce [3456067,73119] 0 2024-05-12T04:01:38.574 INFO:tasks.workunit.client.0.smithi032.stdout:7/794: chown dd/d1b/d1e/d45/d64 13 1 2024-05-12T04:01:38.575 INFO:tasks.workunit.client.0.smithi032.stdout:7/795: chown dd/d1b/d43/d74/l80 25382616 1 2024-05-12T04:01:38.575 INFO:tasks.workunit.client.0.smithi032.stdout:7/796: fdatasync f5 0 2024-05-12T04:01:38.583 INFO:tasks.workunit.client.0.smithi032.stdout:8/900: write d0/d9/d1b/d27/d88/d56/d63/d8f/db0/fc0 [1986022,93264] 0 2024-05-12T04:01:38.585 INFO:tasks.workunit.client.0.smithi032.stdout:6/966: write d7/d97/f99 [707465,82717] 0 2024-05-12T04:01:38.586 INFO:tasks.workunit.client.0.smithi032.stdout:8/901: fsync d0/d9/d1b/d27/f11e 0 2024-05-12T04:01:38.587 INFO:tasks.workunit.client.0.smithi032.stdout:1/869: dwrite d1/d25/f64 [0,4194304] 0 2024-05-12T04:01:38.589 INFO:tasks.workunit.client.0.smithi032.stdout:7/797: rmdir dd/d1b 39 2024-05-12T04:01:38.595 INFO:tasks.workunit.client.0.smithi032.stdout:9/796: rename d0/d84/l9d to d0/d3/d4b/d88/dc4/ddb/lfb 0 2024-05-12T04:01:38.596 INFO:tasks.workunit.client.0.smithi032.stdout:2/868: dread d1/d34/fce [0,4194304] 0 2024-05-12T04:01:38.599 INFO:tasks.workunit.client.0.smithi032.stdout:8/902: dwrite d0/d4b/fcb [0,4194304] 0 2024-05-12T04:01:38.602 INFO:tasks.workunit.client.0.smithi032.stdout:7/798: truncate dd/d1b/d43/fe6 124608 0 2024-05-12T04:01:38.607 INFO:tasks.workunit.client.0.smithi032.stdout:1/870: creat d1/d7/d30/d18/d87/f120 x:0 0 0 2024-05-12T04:01:38.611 INFO:tasks.workunit.client.0.smithi032.stdout:6/967: unlink d7/d9/d48/d67/d8d/dbc/de2/d9d/ffe 0 2024-05-12T04:01:38.616 INFO:tasks.workunit.client.0.smithi032.stdout:2/869: mkdir d1/d13/da0/dae/d120 0 2024-05-12T04:01:38.620 INFO:tasks.workunit.client.0.smithi032.stdout:8/903: creat d0/d9/d1b/d27/d4c/d4e/d7f/f131 x:0 0 0 2024-05-12T04:01:38.622 INFO:tasks.workunit.client.0.smithi032.stdout:7/799: truncate dd/d1b/d43/d5e/d37/f41 245485 0 2024-05-12T04:01:38.624 INFO:tasks.workunit.client.0.smithi032.stdout:1/871: truncate d1/d7/d30/d35/f42 471460 0 2024-05-12T04:01:38.634 INFO:tasks.workunit.client.0.smithi032.stdout:2/870: unlink d1/d34/d96/d41/d58/d108/f10a 0 2024-05-12T04:01:38.640 INFO:tasks.workunit.client.0.smithi032.stdout:6/968: mknod d7/d9/d48/d67/d8d/c157 0 2024-05-12T04:01:38.647 INFO:tasks.workunit.client.0.smithi032.stdout:8/904: rename d0/d9/d1b/d27/d88/d56/fe5 to d0/d9/d1b/d27/d88/dd7/ddf/f132 0 2024-05-12T04:01:38.653 INFO:tasks.workunit.client.0.smithi032.stdout:6/969: dread - d7/d9/d48/d67/d1a/d93/ff7 zero size 2024-05-12T04:01:38.659 INFO:tasks.workunit.client.0.smithi032.stdout:7/800: creat dd/d1b/d1e/de3/d9a/ffc x:0 0 0 2024-05-12T04:01:38.660 INFO:tasks.workunit.client.0.smithi032.stdout:2/871: dread d1/d34/d96/d41/d2c/f99 [0,4194304] 0 2024-05-12T04:01:38.663 INFO:tasks.workunit.client.0.smithi032.stdout:8/905: dread d0/d9/d13/f29 [0,4194304] 0 2024-05-12T04:01:38.667 INFO:tasks.workunit.client.0.smithi032.stdout:1/872: rename d1/d25/c66 to d1/d25/da5/db5/c121 0 2024-05-12T04:01:38.670 INFO:tasks.workunit.client.0.smithi032.stdout:8/906: truncate d0/d9/d1b/d27/d88/f40 1969516 0 2024-05-12T04:01:38.674 INFO:tasks.workunit.client.0.smithi032.stdout:2/872: rename d1/d34/d3e/d67/d10c/fc0 to d1/d13/f121 0 2024-05-12T04:01:38.676 INFO:tasks.workunit.client.0.smithi032.stdout:2/873: write d1/d34/d96/d41/d2c/f56 [826675,4774] 0 2024-05-12T04:01:38.680 INFO:tasks.workunit.client.0.smithi032.stdout:9/797: truncate d0/f28 1523679 0 2024-05-12T04:01:38.681 INFO:tasks.workunit.client.0.smithi032.stdout:9/798: readlink d0/d3/d74/lf3 0 2024-05-12T04:01:38.685 INFO:tasks.workunit.client.0.smithi032.stdout:2/874: mkdir d1/d34/d3e/d67/d122 0 2024-05-12T04:01:38.688 INFO:tasks.workunit.client.0.smithi032.stdout:8/907: mkdir d0/d9/d1b/d133 0 2024-05-12T04:01:38.690 INFO:tasks.workunit.client.0.smithi032.stdout:1/873: dread d1/d7/d30/d18/d87/db7/fc5 [0,4194304] 0 2024-05-12T04:01:38.696 INFO:tasks.workunit.client.0.smithi032.stdout:2/875: dread d1/d13/f93 [0,4194304] 0 2024-05-12T04:01:38.778 INFO:tasks.workunit.client.0.smithi032.stdout:8/908: fdatasync d0/d7/f11 0 2024-05-12T04:01:38.781 INFO:tasks.workunit.client.0.smithi032.stdout:4/811: sync 2024-05-12T04:01:38.791 INFO:tasks.workunit.client.0.smithi032.stdout:9/799: link d0/d3/d4b/d85/d87/f98 d0/d3/d4b/d88/d99/df8/ffc 0 2024-05-12T04:01:38.791 INFO:tasks.workunit.client.0.smithi032.stdout:6/970: dwrite d7/d9/f82 [0,4194304] 0 2024-05-12T04:01:38.800 INFO:tasks.workunit.client.0.smithi032.stdout:8/909: dread d0/d9/d1b/d27/d88/f79 [0,4194304] 0 2024-05-12T04:01:38.801 INFO:tasks.workunit.client.0.smithi032.stdout:8/910: stat d0/d9/d1b/d33/d73/db7/fef 0 2024-05-12T04:01:38.807 INFO:tasks.workunit.client.0.smithi032.stdout:0/901: sync 2024-05-12T04:01:38.807 INFO:tasks.workunit.client.0.smithi032.stdout:7/801: sync 2024-05-12T04:01:38.814 INFO:tasks.workunit.client.0.smithi032.stdout:6/971: mknod d7/d9/d48/d67/d63/d64/d94/de4/d57/d14b/da1/c158 0 2024-05-12T04:01:38.817 INFO:tasks.workunit.client.0.smithi032.stdout:9/800: dread d0/d3/d6/d7/f7d [0,4194304] 0 2024-05-12T04:01:38.821 INFO:tasks.workunit.client.0.smithi032.stdout:4/812: dread d0/d3/d24/f71 [0,4194304] 0 2024-05-12T04:01:38.826 INFO:tasks.workunit.client.0.smithi032.stdout:2/876: getdents d1 0 2024-05-12T04:01:38.829 INFO:tasks.workunit.client.0.smithi032.stdout:8/911: dread d0/d7/f11 [0,4194304] 0 2024-05-12T04:01:38.840 INFO:tasks.workunit.client.0.smithi032.stdout:4/813: dread d0/d3/d24/f72 [0,4194304] 0 2024-05-12T04:01:38.841 INFO:tasks.workunit.client.0.smithi032.stdout:8/912: dread d0/d9/d1b/d27/d88/d56/d63/d8f/d9a/f55 [4194304,4194304] 0 2024-05-12T04:01:38.842 INFO:tasks.workunit.client.0.smithi032.stdout:8/913: fsync d0/d9/d13/d21/d35/de9/f118 0 2024-05-12T04:01:38.856 INFO:tasks.workunit.client.0.smithi032.stdout:6/972: creat d7/d9/d48/d67/d8d/f159 x:0 0 0 2024-05-12T04:01:38.864 INFO:tasks.workunit.client.0.smithi032.stdout:9/801: creat d0/d3/d6/d7/da/ffd x:0 0 0 2024-05-12T04:01:38.868 INFO:tasks.workunit.client.0.smithi032.stdout:4/814: creat d0/dde/ff9 x:0 0 0 2024-05-12T04:01:38.869 INFO:tasks.workunit.client.0.smithi032.stdout:6/973: dread - d7/d9/d48/d67/d1a/d20/fc2 zero size 2024-05-12T04:01:38.869 INFO:tasks.workunit.client.0.smithi032.stdout:4/815: chown d0/d3/d31/fe3 1794266901 1 2024-05-12T04:01:38.870 INFO:tasks.workunit.client.0.smithi032.stdout:6/974: readlink d7/d9/d48/d67/d1a/da0/l116 0 2024-05-12T04:01:38.872 INFO:tasks.workunit.client.0.smithi032.stdout:8/914: creat d0/d9/d1b/d27/d4c/d4e/d7f/da3/d111/f134 x:0 0 0 2024-05-12T04:01:38.873 INFO:tasks.workunit.client.0.smithi032.stdout:9/802: rmdir d0/d3/d4b 39 2024-05-12T04:01:38.873 INFO:tasks.workunit.client.0.smithi032.stdout:8/915: chown d0/d9/d1b/d33/d73/fe7 34248 1 2024-05-12T04:01:38.876 INFO:tasks.workunit.client.0.smithi032.stdout:6/975: creat d7/d9/d48/d67/d1a/d20/f15a x:0 0 0 2024-05-12T04:01:38.882 INFO:tasks.workunit.client.0.smithi032.stdout:6/976: fsync d7/d9/d48/d67/f85 0 2024-05-12T04:01:38.883 INFO:tasks.workunit.client.0.smithi032.stdout:9/803: mkdir d0/d3/d74/dfe 0 2024-05-12T04:01:38.885 INFO:tasks.workunit.client.0.smithi032.stdout:8/916: mknod d0/d9/d13/d92/d124/d12b/c135 0 2024-05-12T04:01:38.891 INFO:tasks.workunit.client.0.smithi032.stdout:8/917: rename d0/d4b/c6f to d0/d9/d1b/d66/d75/de2/d103/d108/d12c/c136 0 2024-05-12T04:01:38.891 INFO:tasks.workunit.client.0.smithi032.stdout:8/918: chown d0/d97/fc6 124 1 2024-05-12T04:01:38.896 INFO:tasks.workunit.client.0.smithi032.stdout:8/919: symlink d0/d9/d1b/d27/d4c/l137 0 2024-05-12T04:01:38.902 INFO:tasks.workunit.client.0.smithi032.stdout:8/920: truncate d0/d9/d13/f102 372759 0 2024-05-12T04:01:38.926 INFO:tasks.workunit.client.0.smithi032.stdout:4/816: unlink d0/d3/d12/d1d/d6b/cbe 0 2024-05-12T04:01:38.931 INFO:tasks.workunit.client.0.smithi032.stdout:4/817: getdents d0/d3/d24 0 2024-05-12T04:01:38.944 INFO:tasks.workunit.client.0.smithi032.stdout:1/874: dwrite d1/d7/d30/d18/d87/fc7 [0,4194304] 0 2024-05-12T04:01:38.944 INFO:tasks.workunit.client.0.smithi032.stdout:1/875: chown d1/d59 14869 1 2024-05-12T04:01:38.947 INFO:tasks.workunit.client.0.smithi032.stdout:1/876: symlink d1/d7/d113/l122 0 2024-05-12T04:01:38.952 INFO:tasks.workunit.client.0.smithi032.stdout:1/877: creat d1/d7/d113/f123 x:0 0 0 2024-05-12T04:01:38.953 INFO:tasks.workunit.client.0.smithi032.stdout:1/878: chown d1/d7/d30/d18/d93/dc1/d109 261040220 1 2024-05-12T04:01:38.956 INFO:tasks.workunit.client.0.smithi032.stdout:1/879: dread d1/d7/d30/d35/d77/ff4 [0,4194304] 0 2024-05-12T04:01:38.958 INFO:tasks.workunit.client.0.smithi032.stdout:1/880: truncate d1/dd/f11c 467389 0 2024-05-12T04:01:38.960 INFO:tasks.workunit.client.0.smithi032.stdout:1/881: rmdir d1/d7 39 2024-05-12T04:01:38.970 INFO:tasks.workunit.client.0.smithi032.stdout:2/877: write d1/d13/d70/d85/f9c [2747734,58181] 0 2024-05-12T04:01:38.976 INFO:tasks.workunit.client.0.smithi032.stdout:0/902: write df/d10/d3b/d75/f9b [3615312,19206] 0 2024-05-12T04:01:38.979 INFO:tasks.workunit.client.0.smithi032.stdout:0/903: read df/d30/d36/ddd/f104 [242608,2528] 0 2024-05-12T04:01:38.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:38.980+0000 7f713c20c700 1 -- 172.21.15.32:0/1182938255 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71340fed80 msgr2=0x7f71340faec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:01:38.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:38.980+0000 7f713c20c700 1 --2- 172.21.15.32:0/1182938255 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71340fed80 0x7f71340faec0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7130004940 tx=0x7f7130009e30 comp rx=0 tx=0).stop 2024-05-12T04:01:38.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:38.980+0000 7f713c20c700 1 -- 172.21.15.32:0/1182938255 shutdown_connections 2024-05-12T04:01:38.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:38.980+0000 7f713c20c700 1 --2- 172.21.15.32:0/1182938255 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71340fed80 0x7f71340faec0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:38.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:38.980+0000 7f713c20c700 1 --2- 172.21.15.32:0/1182938255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71340fe430 0x7f71340fe840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:38.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:38.980+0000 7f713c20c700 1 -- 172.21.15.32:0/1182938255 >> 172.21.15.32:0/1182938255 conn(0x7f71340f6bd0 msgr2=0x7f71340f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:01:38.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:38.980+0000 7f713c20c700 1 -- 172.21.15.32:0/1182938255 shutdown_connections 2024-05-12T04:01:38.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:38.980+0000 7f713c20c700 1 -- 172.21.15.32:0/1182938255 wait complete. 2024-05-12T04:01:38.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:38.981+0000 7f713c20c700 1 Processor -- start 2024-05-12T04:01:38.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:38.981+0000 7f713c20c700 1 -- start start 2024-05-12T04:01:38.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:38.981+0000 7f713c20c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71340fe430 0x7f713410d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:01:38.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:38.981+0000 7f713c20c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71340fed80 0x7f713410dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:01:38.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:38.981+0000 7f713c20c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f713410e300 con 0x7f71340fe430 2024-05-12T04:01:38.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:38.981+0000 7f713c20c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f713410e470 con 0x7f71340fed80 2024-05-12T04:01:38.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:38.982+0000 7f71397a7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71340fed80 0x7f713410dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:01:38.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:38.982+0000 7f71397a7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71340fed80 0x7f713410dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.32:51300/0 (socket says 172.21.15.32:51300) 2024-05-12T04:01:38.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:38.982+0000 7f71397a7700 1 -- 172.21.15.32:0/1134040522 learned_addr learned my addr 172.21.15.32:0/1134040522 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:01:38.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:38.982+0000 7f71397a7700 1 -- 172.21.15.32:0/1134040522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71340fe430 msgr2=0x7f713410d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:01:38.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:38.982+0000 7f71397a7700 1 --2- 172.21.15.32:0/1134040522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71340fe430 0x7f713410d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:38.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:38.982+0000 7f71397a7700 1 -- 172.21.15.32:0/1134040522 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7130005040 con 0x7f71340fed80 2024-05-12T04:01:38.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:38.982+0000 7f71397a7700 1 --2- 172.21.15.32:0/1134040522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71340fed80 0x7f713410dd00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f71300137b0 tx=0x7f7130009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:01:38.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:38.982+0000 7f712a7fc700 1 -- 172.21.15.32:0/1134040522 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f713000e070 con 0x7f71340fed80 2024-05-12T04:01:38.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:38.983+0000 7f713c20c700 1 -- 172.21.15.32:0/1134040522 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7134112e70 con 0x7f71340fed80 2024-05-12T04:01:38.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:38.983+0000 7f712a7fc700 1 -- 172.21.15.32:0/1134040522 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f71300093f0 con 0x7f71340fed80 2024-05-12T04:01:38.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:38.983+0000 7f712a7fc700 1 -- 172.21.15.32:0/1134040522 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f713000b410 con 0x7f71340fed80 2024-05-12T04:01:38.986 INFO:tasks.workunit.client.0.smithi032.stdout:1/882: dread d1/d7/d30/d35/d54/d85/f102 [0,4194304] 0 2024-05-12T04:01:38.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:38.983+0000 7f713c20c700 1 -- 172.21.15.32:0/1134040522 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71341133a0 con 0x7f71340fed80 2024-05-12T04:01:38.987 INFO:tasks.workunit.client.0.smithi032.stdout:1/883: stat d1/d7/d30/d35/d77/ff4 0 2024-05-12T04:01:38.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:38.984+0000 7f713c20c700 1 -- 172.21.15.32:0/1134040522 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f713404c710 con 0x7f71340fed80 2024-05-12T04:01:38.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:38.984+0000 7f712a7fc700 1 -- 172.21.15.32:0/1134040522 <== mon.1 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) v1 ==== 88772+0+0 (secure 0 0 0) 0x7f713001c400 con 0x7f71340fed80 2024-05-12T04:01:38.987 INFO:tasks.workunit.client.0.smithi032.stdout:2/878: rmdir d1/d34/d96 39 2024-05-12T04:01:38.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:38.985+0000 7f712a7fc700 1 --2- 172.21.15.32:0/1134040522 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f712006aed0 0x7f712006d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:01:38.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:38.985+0000 7f712a7fc700 1 -- 172.21.15.32:0/1134040522 <== mon.1 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f713008afe0 con 0x7f71340fed80 2024-05-12T04:01:38.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:38.985+0000 7f7139fa8700 1 --2- 172.21.15.32:0/1134040522 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f712006aed0 0x7f712006d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:01:38.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:38.985+0000 7f7139fa8700 1 --2- 172.21.15.32:0/1134040522 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f712006aed0 0x7f712006d380 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7124009fd0 tx=0x7f712400b040 comp rx=0 tx=0).ready entity=mgr.24389 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:01:38.989 INFO:tasks.workunit.client.0.smithi032.stdout:1/884: creat d1/d25/f124 x:0 0 0 2024-05-12T04:01:38.991 INFO:tasks.workunit.client.0.smithi032.stdout:1/885: rmdir d1/d7/d30/d61 39 2024-05-12T04:01:38.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:38.990+0000 7f712a7fc700 1 -- 172.21.15.32:0/1134040522 <== mon.1 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181777 (secure 0 0 0) 0x7f7130058e60 con 0x7f71340fed80 2024-05-12T04:01:38.998 INFO:tasks.workunit.client.0.smithi032.stdout:1/886: symlink d1/d7/d30/d35/d77/de7/l125 0 2024-05-12T04:01:39.002 INFO:tasks.workunit.client.0.smithi032.stdout:1/887: mknod d1/d7/d30/d18/d87/db7/c126 0 2024-05-12T04:01:39.003 INFO:tasks.workunit.client.0.smithi032.stdout:1/888: chown d1/d7/d30/d18/l37 49972843 1 2024-05-12T04:01:39.008 INFO:tasks.workunit.client.0.smithi032.stdout:6/977: dwrite d7/d9/d48/d67/f6e [0,4194304] 0 2024-05-12T04:01:39.011 INFO:tasks.workunit.client.0.smithi032.stdout:6/978: mknod d7/d9/d11a/c15b 0 2024-05-12T04:01:39.031 INFO:tasks.workunit.client.0.smithi032.stdout:4/818: write d0/d3/d12/d1d/d6b/fcb [522277,125609] 0 2024-05-12T04:01:39.035 INFO:tasks.workunit.client.0.smithi032.stdout:8/921: rename d0/d9/d13/d21/d35/de9/l122 to d0/d9/d1b/d27/l138 0 2024-05-12T04:01:39.038 INFO:tasks.workunit.client.0.smithi032.stdout:4/819: dwrite d0/d3/d31/d80/fdf [0,4194304] 0 2024-05-12T04:01:39.039 INFO:tasks.workunit.client.0.smithi032.stdout:4/820: stat d0/d3/d31/d80/f8c 0 2024-05-12T04:01:39.040 INFO:tasks.workunit.client.0.smithi032.stdout:8/922: creat d0/d9/d1b/d27/d4c/d4e/d90/f139 x:0 0 0 2024-05-12T04:01:39.042 INFO:tasks.workunit.client.0.smithi032.stdout:8/923: write d0/d9/d1b/d27/f11e [422869,27694] 0 2024-05-12T04:01:39.045 INFO:tasks.workunit.client.0.smithi032.stdout:4/821: rename d0/d3/d24/f71 to d0/d3/d12/d1d/d63/d87/ffa 0 2024-05-12T04:01:39.050 INFO:tasks.workunit.client.0.smithi032.stdout:4/822: creat d0/ffb x:0 0 0 2024-05-12T04:01:39.052 INFO:tasks.workunit.client.0.smithi032.stdout:4/823: fdatasync d0/d3/d12/d1d/d6b/ff5 0 2024-05-12T04:01:39.054 INFO:tasks.workunit.client.0.smithi032.stdout:6/979: write d7/d9/d48/d67/d25/f7e [328190,37188] 0 2024-05-12T04:01:39.062 INFO:tasks.workunit.client.0.smithi032.stdout:0/904: dwrite df/d30/d8d/d8f/f10e [0,4194304] 0 2024-05-12T04:01:39.062 INFO:tasks.workunit.client.0.smithi032.stdout:6/980: chown d7/d9/d48/d67/d63/d64/d94/f101 6 1 2024-05-12T04:01:39.063 INFO:tasks.workunit.client.0.smithi032.stdout:6/981: readlink d7/d9/d48/d67/lcc 0 2024-05-12T04:01:39.066 INFO:tasks.workunit.client.0.smithi032.stdout:1/889: dwrite d1/d6a/f7a [4194304,4194304] 0 2024-05-12T04:01:39.071 INFO:tasks.workunit.client.0.smithi032.stdout:6/982: creat d7/d9/d48/d67/d8d/dbc/de2/d9d/f15c x:0 0 0 2024-05-12T04:01:39.073 INFO:tasks.workunit.client.0.smithi032.stdout:1/890: read d1/d7/d30/d35/d54/d7e/fca [397320,50213] 0 2024-05-12T04:01:39.081 INFO:tasks.workunit.client.0.smithi032.stdout:8/924: rename d0/d9/d1b/d27/l138 to d0/d9/d1b/d27/d88/d56/d63/d8f/db0/l13a 0 2024-05-12T04:01:39.085 INFO:tasks.workunit.client.0.smithi032.stdout:6/983: dwrite d7/d9/d48/d67/d63/d64/d94/de4/d57/d14b/da1/dfb/f145 [0,4194304] 0 2024-05-12T04:01:39.086 INFO:tasks.workunit.client.0.smithi032.stdout:4/824: truncate d0/d3/df/d4e/fdd 1731808 0 2024-05-12T04:01:39.087 INFO:tasks.workunit.client.0.smithi032.stdout:1/891: mknod d1/d7/d30/d18/d87/dc6/c127 0 2024-05-12T04:01:39.089 INFO:tasks.workunit.client.0.smithi032.stdout:8/925: mknod d0/d9/d1b/d66/c13b 0 2024-05-12T04:01:39.091 INFO:tasks.workunit.client.0.smithi032.stdout:4/825: symlink d0/d3/d12/d1d/d63/d87/dc6/lfc 0 2024-05-12T04:01:39.092 INFO:tasks.workunit.client.0.smithi032.stdout:8/926: truncate d0/d9/d13/d21/d25/f52 4979937 0 2024-05-12T04:01:39.093 INFO:tasks.workunit.client.0.smithi032.stdout:4/826: write d0/d3/d12/d1d/fba [2383662,39149] 0 2024-05-12T04:01:39.095 INFO:tasks.workunit.client.0.smithi032.stdout:6/984: mkdir d7/d9/d48/d67/d63/d64/d94/de4/d57/d14b/da1/dfb/d15d 0 2024-05-12T04:01:39.105 INFO:tasks.workunit.client.0.smithi032.stdout:4/827: dwrite d0/d3/d31/d80/d84/fa9 [4194304,4194304] 0 2024-05-12T04:01:39.110 INFO:tasks.workunit.client.0.smithi032.stdout:6/985: creat d7/d9/d48/d121/f15e x:0 0 0 2024-05-12T04:01:39.111 INFO:tasks.workunit.client.0.smithi032.stdout:4/828: dread d0/d3/df/d4e/f92 [0,4194304] 0 2024-05-12T04:01:39.113 INFO:tasks.workunit.client.0.smithi032.stdout:6/986: fsync d7/f59 0 2024-05-12T04:01:39.122 INFO:tasks.workunit.client.0.smithi032.stdout:6/987: dwrite d7/d9/d48/d67/d63/d64/d94/de4/d57/d14b/da1/fae [0,4194304] 0 2024-05-12T04:01:39.133 INFO:tasks.workunit.client.0.smithi032.stdout:8/927: write d0/d9/d1b/d27/d88/d56/d63/d8f/d9a/d28/f82 [2041406,14070] 0 2024-05-12T04:01:39.137 INFO:tasks.workunit.client.0.smithi032.stdout:8/928: fdatasync d0/d9/d13/d21/d25/f2c 0 2024-05-12T04:01:39.138 INFO:tasks.workunit.client.0.smithi032.stdout:1/892: truncate d1/d7/f78 2411697 0 2024-05-12T04:01:39.140 INFO:tasks.workunit.client.0.smithi032.stdout:6/988: rename d7/d9/d48/d67/d1a/d93/lf6 to d7/d9/d48/d67/d1a/l15f 0 2024-05-12T04:01:39.143 INFO:tasks.workunit.client.0.smithi032.stdout:1/893: fsync d1/d7/d30/d35/dbe/fe0 0 2024-05-12T04:01:39.146 INFO:tasks.workunit.client.0.smithi032.stdout:8/929: mknod d0/d9/d1b/d27/c13c 0 2024-05-12T04:01:39.151 INFO:tasks.workunit.client.0.smithi032.stdout:1/894: creat d1/d7/d30/d35/dbe/f128 x:0 0 0 2024-05-12T04:01:39.154 INFO:tasks.workunit.client.0.smithi032.stdout:1/895: chown d1/d119 14046934 1 2024-05-12T04:01:39.154 INFO:tasks.workunit.client.0.smithi032.stdout:6/989: truncate d7/d9/d48/fbf 520632 0 2024-05-12T04:01:39.156 INFO:tasks.workunit.client.0.smithi032.stdout:6/990: fsync d7/d9/d48/d67/d25/f74 0 2024-05-12T04:01:39.160 INFO:tasks.workunit.client.0.smithi032.stdout:4/829: write d0/d3/d12/f48 [2024404,2417] 0 2024-05-12T04:01:39.160 INFO:tasks.workunit.client.0.smithi032.stdout:6/991: chown d7/d9/d48/d67/d8d/dbc/f68 60867100 1 2024-05-12T04:01:39.160 INFO:tasks.workunit.client.0.smithi032.stdout:6/992: stat d7/d97/fa4 0 2024-05-12T04:01:39.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:38 smithi033 ceph-mon[49525]: pgmap v47: 65 pgs: 65 active+clean; 3.9 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 98 MiB/s rd, 216 MiB/s wr, 518 op/s 2024-05-12T04:01:39.162 INFO:tasks.workunit.client.0.smithi032.stdout:1/896: mknod d1/d25/da5/db5/c129 0 2024-05-12T04:01:39.163 INFO:tasks.workunit.client.0.smithi032.stdout:6/993: dread - d7/d9/d48/d67/d63/d64/d94/de4/d57/f10c zero size 2024-05-12T04:01:39.163 INFO:tasks.workunit.client.0.smithi032.stdout:8/930: mkdir d0/d9/d1b/d13d 0 2024-05-12T04:01:39.166 INFO:tasks.workunit.client.0.smithi032.stdout:4/830: dread d0/d3/d31/d80/d84/fcc [0,4194304] 0 2024-05-12T04:01:39.169 INFO:tasks.workunit.client.0.smithi032.stdout:7/802: sync 2024-05-12T04:01:39.178 INFO:tasks.workunit.client.0.smithi032.stdout:7/803: chown dd/d1b/d1e/de3/d9a/d9c/fe8 11618754 1 2024-05-12T04:01:39.179 INFO:tasks.workunit.client.0.smithi032.stdout:1/897: creat d1/d7/d113/f12a x:0 0 0 2024-05-12T04:01:39.184 INFO:tasks.workunit.client.0.smithi032.stdout:6/994: dwrite d7/d97/db3/fdc [0,4194304] 0 2024-05-12T04:01:39.189 INFO:tasks.workunit.client.0.smithi032.stdout:6/995: truncate d7/d9/d48/d67/d1a/d93/ff7 766018 0 2024-05-12T04:01:39.190 INFO:tasks.workunit.client.0.smithi032.stdout:1/898: read - d1/d7/d30/d35/d54/d7e/f99 zero size 2024-05-12T04:01:39.192 INFO:tasks.workunit.client.0.smithi032.stdout:1/899: write d1/d59/f112 [671905,125731] 0 2024-05-12T04:01:39.193 INFO:tasks.workunit.client.0.smithi032.stdout:4/831: mknod d0/d3/cfd 0 2024-05-12T04:01:39.197 INFO:tasks.workunit.client.0.smithi032.stdout:6/996: creat d7/d9/d48/d67/d63/d64/d94/de4/d57/d14b/d112/f160 x:0 0 0 2024-05-12T04:01:39.198 INFO:tasks.workunit.client.0.smithi032.stdout:1/900: mknod d1/d7/d30/d18/d87/db7/c12b 0 2024-05-12T04:01:39.200 INFO:tasks.workunit.client.0.smithi032.stdout:1/901: chown d1/d7/d30/d18/f8b 16063 1 2024-05-12T04:01:39.200 INFO:tasks.workunit.client.0.smithi032.stdout:4/832: fdatasync d0/d3/d12/d1d/d63/d77/fa8 0 2024-05-12T04:01:39.202 INFO:tasks.workunit.client.0.smithi032.stdout:4/833: write d0/d3/d12/d1d/fa4 [674222,95156] 0 2024-05-12T04:01:39.203 INFO:tasks.workunit.client.0.smithi032.stdout:4/834: chown d0/d3/d12/d8d/f98 47004 1 2024-05-12T04:01:39.205 INFO:tasks.workunit.client.0.smithi032.stdout:1/902: creat d1/d6a/d72/da4/f12c x:0 0 0 2024-05-12T04:01:39.206 INFO:tasks.workunit.client.0.smithi032.stdout:1/903: dread - d1/d7/d113/f117 zero size 2024-05-12T04:01:39.207 INFO:tasks.workunit.client.0.smithi032.stdout:6/997: creat d7/d9/d11a/d131/f161 x:0 0 0 2024-05-12T04:01:39.213 INFO:tasks.workunit.client.0.smithi032.stdout:8/931: dwrite d0/d9/d1b/d66/fe8 [0,4194304] 0 2024-05-12T04:01:39.215 INFO:tasks.workunit.client.0.smithi032.stdout:1/904: fsync d1/dd/f7c 0 2024-05-12T04:01:39.216 INFO:tasks.workunit.client.0.smithi032.stdout:7/804: truncate dd/d1b/d43/d74/f84 3633792 0 2024-05-12T04:01:39.221 INFO:tasks.workunit.client.0.smithi032.stdout:1/905: dread d1/d7/d30/d35/f48 [0,4194304] 0 2024-05-12T04:01:39.221 INFO:tasks.workunit.client.0.smithi032.stdout:6/998: creat d7/d9/d113/f162 x:0 0 0 2024-05-12T04:01:39.224 INFO:tasks.workunit.client.0.smithi032.stdout:4/835: link d0/db5/fbc d0/d3/d12/d1d/d63/d87/d95/ffe 0 2024-05-12T04:01:39.228 INFO:tasks.workunit.client.0.smithi032.stdout:1/906: dwrite d1/d7/d30/d35/f4e [0,4194304] 0 2024-05-12T04:01:39.228 INFO:tasks.workunit.client.0.smithi032.stdout:7/805: readlink dd/l17 0 2024-05-12T04:01:39.232 INFO:tasks.workunit.client.0.smithi032.stdout:8/932: symlink d0/d9/d1b/d27/d88/d56/d63/d8f/d9a/d28/l13e 0 2024-05-12T04:01:39.236 INFO:tasks.workunit.client.0.smithi032.stdout:6/999: mkdir d7/d9/d113/d117/d163 0 2024-05-12T04:01:39.246 INFO:tasks.workunit.client.0.smithi032.stdout:1/907: rename d1/d7/d30/d18/d87/db7/c12b to d1/d25/da5/c12d 0 2024-05-12T04:01:39.250 INFO:tasks.workunit.client.0.smithi032.stdout:1/908: creat d1/d7/d30/d18/d93/f12e x:0 0 0 2024-05-12T04:01:39.252 INFO:tasks.workunit.client.0.smithi032.stdout:4/836: getdents d0/d3/d12/d1d/d63/d87/d95 0 2024-05-12T04:01:39.253 INFO:tasks.workunit.client.0.smithi032.stdout:7/806: dread dd/d1b/d1e/d45/d64/f8b [4194304,4194304] 0 2024-05-12T04:01:39.254 INFO:tasks.workunit.client.0.smithi032.stdout:7/807: chown dd/d1b/d1e/de3/d9a/d9c/def 10663 1 2024-05-12T04:01:39.256 INFO:tasks.workunit.client.0.smithi032.stdout:7/808: dread dd/d1b/d43/fe6 [0,4194304] 0 2024-05-12T04:01:39.257 INFO:tasks.workunit.client.0.smithi032.stdout:7/809: fdatasync dd/d1b/d43/fe6 0 2024-05-12T04:01:39.257 INFO:tasks.workunit.client.0.smithi032.stdout:4/837: fsync d0/d3/d12/d1d/f5d 0 2024-05-12T04:01:39.258 INFO:tasks.workunit.client.0.smithi032.stdout:4/838: write d0/d3/df/f86 [184081,116073] 0 2024-05-12T04:01:39.264 INFO:tasks.workunit.client.0.smithi032.stdout:7/810: rmdir dd/d1b/d1e/d45/d64 39 2024-05-12T04:01:39.269 INFO:tasks.workunit.client.0.smithi032.stdout:4/839: dwrite d0/d3/d12/d1d/f74 [0,4194304] 0 2024-05-12T04:01:39.273 INFO:tasks.workunit.client.0.smithi032.stdout:4/840: write d0/dde/ff9 [358117,107198] 0 2024-05-12T04:01:39.273 INFO:tasks.workunit.client.0.smithi032.stdout:8/933: dwrite d0/d9/d1b/d27/d59/ffc [0,4194304] 0 2024-05-12T04:01:39.273 INFO:tasks.workunit.client.0.smithi032.stdout:1/909: write d1/d25/fb8 [57287,121438] 0 2024-05-12T04:01:39.278 INFO:tasks.workunit.client.0.smithi032.stdout:1/910: write d1/d7/d30/d18/d87/dc6/fd0 [192324,88578] 0 2024-05-12T04:01:39.284 INFO:tasks.workunit.client.0.smithi032.stdout:1/911: chown d1/d7/d30/d18/l37 0 1 2024-05-12T04:01:39.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:38 smithi032 ceph-mon[40008]: pgmap v47: 65 pgs: 65 active+clean; 3.9 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 98 MiB/s rd, 216 MiB/s wr, 518 op/s 2024-05-12T04:01:39.291 INFO:tasks.workunit.client.0.smithi032.stdout:7/811: dread dd/d1b/d43/d5e/f92 [0,4194304] 0 2024-05-12T04:01:39.304 INFO:tasks.workunit.client.0.smithi032.stdout:1/912: getdents d1/d7/d30/d18/d93 0 2024-05-12T04:01:39.312 INFO:tasks.workunit.client.0.smithi032.stdout:7/812: write dd/f1d [5144138,126394] 0 2024-05-12T04:01:39.313 INFO:tasks.workunit.client.0.smithi032.stdout:1/913: symlink d1/d7/d30/d18/d87/dc6/l12f 0 2024-05-12T04:01:39.318 INFO:tasks.workunit.client.0.smithi032.stdout:7/813: dread dd/d1b/d43/d5e/f67 [0,4194304] 0 2024-05-12T04:01:39.324 INFO:tasks.workunit.client.0.smithi032.stdout:7/814: mkdir dd/d1b/d1e/de3/dfd 0 2024-05-12T04:01:39.324 INFO:tasks.workunit.client.0.smithi032.stdout:7/815: fdatasync f5 0 2024-05-12T04:01:39.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.362+0000 7f713c20c700 1 -- 172.21.15.32:0/1134040522 --> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7134113590 con 0x7f712006aed0 2024-05-12T04:01:39.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.363+0000 7f712a7fc700 1 -- 172.21.15.32:0/1134040522 <== mgr.24389 v2:172.21.15.33:6828/1598726962 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7f7134113590 con 0x7f712006aed0 2024-05-12T04:01:39.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.365+0000 7f713c20c700 1 -- 172.21.15.32:0/1134040522 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f712006aed0 msgr2=0x7f712006d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:01:39.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.365+0000 7f713c20c700 1 --2- 172.21.15.32:0/1134040522 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f712006aed0 0x7f712006d380 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7124009fd0 tx=0x7f712400b040 comp rx=0 tx=0).stop 2024-05-12T04:01:39.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.366+0000 7f713c20c700 1 -- 172.21.15.32:0/1134040522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71340fed80 msgr2=0x7f713410dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:01:39.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.366+0000 7f713c20c700 1 --2- 172.21.15.32:0/1134040522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71340fed80 0x7f713410dd00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f71300137b0 tx=0x7f7130009e30 comp rx=0 tx=0).stop 2024-05-12T04:01:39.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.366+0000 7f713c20c700 1 -- 172.21.15.32:0/1134040522 shutdown_connections 2024-05-12T04:01:39.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.366+0000 7f713c20c700 1 --2- 172.21.15.32:0/1134040522 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f712006aed0 0x7f712006d380 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:39.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.366+0000 7f713c20c700 1 --2- 172.21.15.32:0/1134040522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71340fe430 0x7f713410d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:39.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.366+0000 7f713c20c700 1 --2- 172.21.15.32:0/1134040522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71340fed80 0x7f713410dd00 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:39.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.366+0000 7f713c20c700 1 -- 172.21.15.32:0/1134040522 >> 172.21.15.32:0/1134040522 conn(0x7f71340f6bd0 msgr2=0x7f71340f8310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:01:39.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.366+0000 7f713c20c700 1 -- 172.21.15.32:0/1134040522 shutdown_connections 2024-05-12T04:01:39.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.366+0000 7f713c20c700 1 -- 172.21.15.32:0/1134040522 wait complete. 2024-05-12T04:01:39.375 INFO:tasks.workunit.client.0.smithi032.stdout:1/914: truncate d1/dd/f11c 298160 0 2024-05-12T04:01:39.375 INFO:tasks.workunit.client.0.smithi032.stdout:9/804: sync 2024-05-12T04:01:39.380 INFO:teuthology.orchestra.run.smithi032.stdout:true 2024-05-12T04:01:39.384 INFO:tasks.workunit.client.0.smithi032.stdout:1/915: mkdir d1/d6a/d72/da4/d130 0 2024-05-12T04:01:39.386 INFO:tasks.workunit.client.0.smithi032.stdout:7/816: dwrite fb [0,4194304] 0 2024-05-12T04:01:39.400 INFO:tasks.workunit.client.0.smithi032.stdout:7/817: dread dd/d1b/d43/d5e/d37/f39 [0,4194304] 0 2024-05-12T04:01:39.404 INFO:tasks.workunit.client.0.smithi032.stdout:9/805: creat d0/d3/d4b/d85/dbe/d82/fff x:0 0 0 2024-05-12T04:01:39.409 INFO:tasks.workunit.client.0.smithi032.stdout:9/806: dread d0/d3/d6/f69 [0,4194304] 0 2024-05-12T04:01:39.411 INFO:tasks.workunit.client.0.smithi032.stdout:9/807: write d0/d3/d4b/f6d [284498,49000] 0 2024-05-12T04:01:39.419 INFO:tasks.workunit.client.0.smithi032.stdout:9/808: dwrite d0/d3/d74/fc0 [0,4194304] 0 2024-05-12T04:01:39.431 INFO:tasks.workunit.client.0.smithi032.stdout:9/809: getdents d0/d3/d6/d7 0 2024-05-12T04:01:39.435 INFO:tasks.workunit.client.0.smithi032.stdout:9/810: mkdir d0/d3/d6/d7/d2a/d46/d91/dc5/de5/d100 0 2024-05-12T04:01:39.438 INFO:tasks.workunit.client.0.smithi032.stdout:1/916: write d1/d7/d30/fac [787121,31978] 0 2024-05-12T04:01:39.442 INFO:tasks.workunit.client.0.smithi032.stdout:9/811: link d0/d3/d6/d58/f5e d0/d3/d4b/d88/d99/f101 0 2024-05-12T04:01:39.445 INFO:tasks.workunit.client.0.smithi032.stdout:7/818: dwrite dd/d1b/d43/d5e/f34 [0,4194304] 0 2024-05-12T04:01:39.447 INFO:tasks.workunit.client.0.smithi032.stdout:9/812: chown d0/d3/d6/d7/d2a/f68 5767 1 2024-05-12T04:01:39.448 INFO:tasks.workunit.client.0.smithi032.stdout:9/813: readlink d0/d3/d4b/d6f/dd7/lf6 0 2024-05-12T04:01:39.451 INFO:tasks.workunit.client.0.smithi032.stdout:9/814: write d0/d3/d6/d7/da/d39/f4d [5060003,33113] 0 2024-05-12T04:01:39.453 INFO:tasks.workunit.client.0.smithi032.stdout:9/815: truncate d0/d3/d6/d7/d2a/d46/d13/fe6 847446 0 2024-05-12T04:01:39.457 INFO:tasks.workunit.client.0.smithi032.stdout:9/816: truncate d0/d3/d4b/d85/dbe/d82/fff 702892 0 2024-05-12T04:01:39.458 INFO:tasks.workunit.client.0.smithi032.stdout:9/817: write d0/d3/d6/d7/da/ffd [960759,15448] 0 2024-05-12T04:01:39.460 INFO:tasks.workunit.client.0.smithi032.stdout:9/818: chown d0/d3/d6/d7/d2a/d46/d91/dc5/ff2 295149 1 2024-05-12T04:01:39.462 INFO:tasks.workunit.client.0.smithi032.stdout:1/917: getdents d1/d7/d30/d18/d93/dc1/dc3 0 2024-05-12T04:01:39.462 INFO:tasks.workunit.client.0.smithi032.stdout:9/819: truncate d0/d3/d4b/d6f/fe2 305371 0 2024-05-12T04:01:39.464 INFO:tasks.workunit.client.0.smithi032.stdout:1/918: stat d1/d6a/d72/cd2 0 2024-05-12T04:01:39.465 INFO:tasks.workunit.client.0.smithi032.stdout:7/819: getdents dd/d6c 0 2024-05-12T04:01:39.469 INFO:tasks.workunit.client.0.smithi032.stdout:1/919: symlink d1/d7/d30/d35/d54/d7e/l131 0 2024-05-12T04:01:39.476 INFO:tasks.workunit.client.0.smithi032.stdout:7/820: truncate dd/d1b/d1e/d45/d64/f91 554305 0 2024-05-12T04:01:39.479 INFO:tasks.workunit.client.0.smithi032.stdout:7/821: read fa [2393638,50711] 0 2024-05-12T04:01:39.482 INFO:tasks.workunit.client.0.smithi032.stdout:7/822: write f5 [291047,43269] 0 2024-05-12T04:01:39.482 INFO:tasks.workunit.client.0.smithi032.stdout:9/820: dwrite d0/d3/d4b/d88/d99/df8/ffc [0,4194304] 0 2024-05-12T04:01:39.498 INFO:tasks.workunit.client.0.smithi032.stdout:1/920: dread d1/d25/da5/fc0 [0,4194304] 0 2024-05-12T04:01:39.503 INFO:tasks.workunit.client.0.smithi032.stdout:1/921: symlink d1/l132 0 2024-05-12T04:01:39.505 INFO:tasks.workunit.client.0.smithi032.stdout:1/922: write d1/d25/da5/db5/fda [2266974,85495] 0 2024-05-12T04:01:39.511 INFO:tasks.workunit.client.0.smithi032.stdout:9/821: dread d0/d3/d6/d7/d2a/d46/f12 [0,4194304] 0 2024-05-12T04:01:39.513 INFO:tasks.workunit.client.0.smithi032.stdout:7/823: dwrite dd/d1b/d1e/fd1 [0,4194304] 0 2024-05-12T04:01:39.525 INFO:tasks.workunit.client.0.smithi032.stdout:7/824: link dd/d1b/d1e/de3/d9a/d9c/db9/d75/f86 dd/d1b/d43/d5e/ffe 0 2024-05-12T04:01:39.527 INFO:tasks.workunit.client.0.smithi032.stdout:1/923: dwrite d1/d7/d30/d18/f70 [0,4194304] 0 2024-05-12T04:01:39.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.528+0000 7f07733dd700 1 -- 172.21.15.32:0/1461471252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f076c0fbcb0 msgr2=0x7f076c0fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:01:39.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.528+0000 7f07733dd700 1 --2- 172.21.15.32:0/1461471252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f076c0fbcb0 0x7f076c0fc0c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f075c004940 tx=0x7f075c009e30 comp rx=0 tx=0).stop 2024-05-12T04:01:39.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.528+0000 7f07733dd700 1 -- 172.21.15.32:0/1461471252 shutdown_connections 2024-05-12T04:01:39.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.528+0000 7f07733dd700 1 --2- 172.21.15.32:0/1461471252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f076c0fcf60 0x7f076c0fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:39.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.528+0000 7f07733dd700 1 --2- 172.21.15.32:0/1461471252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f076c0fbcb0 0x7f076c0fc0c0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:39.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.528+0000 7f07733dd700 1 -- 172.21.15.32:0/1461471252 >> 172.21.15.32:0/1461471252 conn(0x7f076c0f7430 msgr2=0x7f076c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:01:39.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.528+0000 7f07733dd700 1 -- 172.21.15.32:0/1461471252 shutdown_connections 2024-05-12T04:01:39.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.528+0000 7f07733dd700 1 -- 172.21.15.32:0/1461471252 wait complete. 2024-05-12T04:01:39.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.529+0000 7f07733dd700 1 Processor -- start 2024-05-12T04:01:39.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.529+0000 7f07733dd700 1 -- start start 2024-05-12T04:01:39.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.529+0000 7f07733dd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f076c0fbcb0 0x7f076c10c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:01:39.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.529+0000 7f07733dd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f076c0fcf60 0x7f076c10cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:01:39.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.529+0000 7f07733dd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f076c10d2e0 con 0x7f076c0fcf60 2024-05-12T04:01:39.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.529+0000 7f07733dd700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f076c10db20 con 0x7f076c0fbcb0 2024-05-12T04:01:39.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.529+0000 7f0770978700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f076c0fcf60 0x7f076c10cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:01:39.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.529+0000 7f0770978700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f076c0fcf60 0x7f076c10cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38502/0 (socket says 172.21.15.32:38502) 2024-05-12T04:01:39.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.529+0000 7f0770978700 1 -- 172.21.15.32:0/1530311171 learned_addr learned my addr 172.21.15.32:0/1530311171 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:01:39.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.529+0000 7f0771179700 1 --2- 172.21.15.32:0/1530311171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f076c0fbcb0 0x7f076c10c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:01:39.531 INFO:tasks.workunit.client.0.smithi032.stdout:1/924: rmdir d1/d25 39 2024-05-12T04:01:39.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.530+0000 7f0771179700 1 -- 172.21.15.32:0/1530311171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f076c0fcf60 msgr2=0x7f076c10cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:01:39.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.530+0000 7f0771179700 1 --2- 172.21.15.32:0/1530311171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f076c0fcf60 0x7f076c10cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:39.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.530+0000 7f0771179700 1 -- 172.21.15.32:0/1530311171 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f075c005040 con 0x7f076c0fbcb0 2024-05-12T04:01:39.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.530+0000 7f0771179700 1 --2- 172.21.15.32:0/1530311171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f076c0fbcb0 0x7f076c10c750 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f075c000c00 tx=0x7f075c009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:01:39.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.531+0000 7f0761ffb700 1 -- 172.21.15.32:0/1530311171 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f075c00e040 con 0x7f076c0fbcb0 2024-05-12T04:01:39.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.531+0000 7f0761ffb700 1 -- 172.21.15.32:0/1530311171 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f075c00bc50 con 0x7f076c0fbcb0 2024-05-12T04:01:39.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.531+0000 7f0761ffb700 1 -- 172.21.15.32:0/1530311171 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f075c01c650 con 0x7f076c0fbcb0 2024-05-12T04:01:39.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.531+0000 7f07733dd700 1 -- 172.21.15.32:0/1530311171 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f076c10dda0 con 0x7f076c0fbcb0 2024-05-12T04:01:39.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.531+0000 7f07733dd700 1 -- 172.21.15.32:0/1530311171 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f076c10e360 con 0x7f076c0fbcb0 2024-05-12T04:01:39.536 INFO:tasks.workunit.client.0.smithi032.stdout:9/822: dwrite d0/d3/ff [8388608,4194304] 0 2024-05-12T04:01:39.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.533+0000 7f07733dd700 1 -- 172.21.15.32:0/1530311171 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f076c05f5c0 con 0x7f076c0fbcb0 2024-05-12T04:01:39.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.533+0000 7f0761ffb700 1 -- 172.21.15.32:0/1530311171 <== mon.1 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) v1 ==== 88772+0+0 (secure 0 0 0) 0x7f075c01c7b0 con 0x7f076c0fbcb0 2024-05-12T04:01:39.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.534+0000 7f0761ffb700 1 --2- 172.21.15.32:0/1530311171 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f075806af90 0x7f075806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:01:39.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.534+0000 7f0761ffb700 1 -- 172.21.15.32:0/1530311171 <== mon.1 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f075c08bab0 con 0x7f076c0fbcb0 2024-05-12T04:01:39.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.535+0000 7f0770978700 1 --2- 172.21.15.32:0/1530311171 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f075806af90 0x7f075806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:01:39.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.537+0000 7f0770978700 1 --2- 172.21.15.32:0/1530311171 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f075806af90 0x7f075806d440 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f076c10d760 tx=0x7f076800b040 comp rx=0 tx=0).ready entity=mgr.24389 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:01:39.544 INFO:tasks.workunit.client.0.smithi032.stdout:2/879: sync 2024-05-12T04:01:39.544 INFO:tasks.workunit.client.0.smithi032.stdout:0/905: sync 2024-05-12T04:01:39.544 INFO:tasks.workunit.client.0.smithi032.stdout:2/880: read d1/d34/fce [2025103,37385] 0 2024-05-12T04:01:39.547 INFO:tasks.workunit.client.0.smithi032.stdout:1/925: dread d1/d7/d30/d61/f100 [0,4194304] 0 2024-05-12T04:01:39.547 INFO:tasks.workunit.client.0.smithi032.stdout:0/906: dread - df/d30/d8d/d8f/f94 zero size 2024-05-12T04:01:39.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.547+0000 7f0761ffb700 1 -- 172.21.15.32:0/1530311171 <== mon.1 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181777 (secure 0 0 0) 0x7f075c055f00 con 0x7f076c0fbcb0 2024-05-12T04:01:39.552 INFO:tasks.workunit.client.0.smithi032.stdout:0/907: write df/d10/d48/f52 [2425549,60631] 0 2024-05-12T04:01:39.552 INFO:tasks.workunit.client.0.smithi032.stdout:2/881: symlink d1/d13/d70/ddb/l123 0 2024-05-12T04:01:39.553 INFO:tasks.workunit.client.0.smithi032.stdout:1/926: dread - d1/d7/d30/d18/d87/fbb zero size 2024-05-12T04:01:39.557 INFO:tasks.workunit.client.0.smithi032.stdout:2/882: fsync d1/d4d/ffe 0 2024-05-12T04:01:39.563 INFO:tasks.workunit.client.0.smithi032.stdout:2/883: truncate d1/d34/d96/d41/d52/ffc 832073 0 2024-05-12T04:01:39.564 INFO:tasks.workunit.client.0.smithi032.stdout:1/927: creat d1/f133 x:0 0 0 2024-05-12T04:01:39.564 INFO:tasks.workunit.client.0.smithi032.stdout:2/884: dread - d1/d34/d4a/feb zero size 2024-05-12T04:01:39.567 INFO:tasks.workunit.client.0.smithi032.stdout:1/928: rmdir d1/d6a 39 2024-05-12T04:01:39.569 INFO:tasks.workunit.client.0.smithi032.stdout:2/885: creat d1/d34/da5/db3/f124 x:0 0 0 2024-05-12T04:01:39.570 INFO:tasks.workunit.client.0.smithi032.stdout:2/886: write d1/d13/f18 [504131,17432] 0 2024-05-12T04:01:39.571 INFO:tasks.workunit.client.0.smithi032.stdout:2/887: stat d1/d34/d96/d41/c26 0 2024-05-12T04:01:39.573 INFO:tasks.workunit.client.0.smithi032.stdout:1/929: unlink d1/d7/d113/l122 0 2024-05-12T04:01:39.574 INFO:tasks.workunit.client.0.smithi032.stdout:0/908: dread df/d30/f8b [0,4194304] 0 2024-05-12T04:01:39.580 INFO:tasks.workunit.client.0.smithi032.stdout:2/888: mknod d1/d34/d3e/d67/d122/c125 0 2024-05-12T04:01:39.583 INFO:tasks.workunit.client.0.smithi032.stdout:0/909: getdents df/d30/d4e/d55 0 2024-05-12T04:01:39.583 INFO:tasks.workunit.client.0.smithi032.stdout:0/910: readlink df/df2/lbc 0 2024-05-12T04:01:39.586 INFO:tasks.workunit.client.0.smithi032.stdout:2/889: getdents d1/d34/d96/d41/da2 0 2024-05-12T04:01:39.589 INFO:tasks.workunit.client.0.smithi032.stdout:0/911: dwrite df/d10/d48/f52 [0,4194304] 0 2024-05-12T04:01:39.595 INFO:tasks.workunit.client.0.smithi032.stdout:2/890: getdents d1/d4d 0 2024-05-12T04:01:39.597 INFO:tasks.workunit.client.0.smithi032.stdout:1/930: write d1/d7/d30/d18/f76 [2522263,83721] 0 2024-05-12T04:01:39.601 INFO:tasks.workunit.client.0.smithi032.stdout:0/912: dwrite df/d10/d3d/de4/dea/ff7 [0,4194304] 0 2024-05-12T04:01:39.603 INFO:tasks.workunit.client.0.smithi032.stdout:1/931: truncate d1/d7/d30/d35/d54/ff1 584477 0 2024-05-12T04:01:39.606 INFO:tasks.workunit.client.0.smithi032.stdout:0/913: creat df/d10/d3d/d83/d9f/d10f/f12e x:0 0 0 2024-05-12T04:01:39.611 INFO:tasks.workunit.client.0.smithi032.stdout:0/914: rename df/d10/d3d/f85 to df/d30/d36/f12f 0 2024-05-12T04:01:39.616 INFO:tasks.workunit.client.0.smithi032.stdout:0/915: read df/d10/d48/d70/ffb [1360596,80418] 0 2024-05-12T04:01:39.618 INFO:tasks.workunit.client.0.smithi032.stdout:0/916: rmdir df/d30 39 2024-05-12T04:01:39.624 INFO:tasks.workunit.client.0.smithi032.stdout:1/932: dwrite d1/d7/d49/f4f [0,4194304] 0 2024-05-12T04:01:39.625 INFO:tasks.workunit.client.0.smithi032.stdout:2/891: dread d1/d34/d3e/fcd [0,4194304] 0 2024-05-12T04:01:39.629 INFO:tasks.workunit.client.0.smithi032.stdout:0/917: write df/d10/d48/d70/ffb [210315,41694] 0 2024-05-12T04:01:39.631 INFO:tasks.workunit.client.0.smithi032.stdout:0/918: write df/d10/d48/f52 [2705394,97436] 0 2024-05-12T04:01:39.637 INFO:tasks.workunit.client.0.smithi032.stdout:2/892: rename d1/d34/d96/d41/cc4 to d1/d34/d96/d8a/c126 0 2024-05-12T04:01:39.643 INFO:tasks.workunit.client.0.smithi032.stdout:1/933: creat d1/d7/d30/d35/f134 x:0 0 0 2024-05-12T04:01:39.647 INFO:tasks.workunit.client.0.smithi032.stdout:2/893: creat d1/d104/f127 x:0 0 0 2024-05-12T04:01:39.648 INFO:tasks.workunit.client.0.smithi032.stdout:2/894: stat d1/d34/d96/d41/d52 0 2024-05-12T04:01:39.650 INFO:tasks.workunit.client.0.smithi032.stdout:1/934: stat d1/d6a/d72/cd2 0 2024-05-12T04:01:39.653 INFO:tasks.workunit.client.0.smithi032.stdout:2/895: mkdir d1/d73/d128 0 2024-05-12T04:01:39.654 INFO:tasks.workunit.client.0.smithi032.stdout:0/919: dwrite df/d10/d3b/d75/f78 [0,4194304] 0 2024-05-12T04:01:39.656 INFO:tasks.workunit.client.0.smithi032.stdout:1/935: creat d1/d7/d30/d35/d77/d110/f135 x:0 0 0 2024-05-12T04:01:39.662 INFO:tasks.workunit.client.0.smithi032.stdout:0/920: mkdir df/d10/d3d/d4a/d8e/d130 0 2024-05-12T04:01:39.666 INFO:tasks.workunit.client.0.smithi032.stdout:1/936: link d1/d25/da5/db5/c129 d1/d6a/d72/c136 0 2024-05-12T04:01:39.667 INFO:tasks.workunit.client.0.smithi032.stdout:1/937: rmdir d1/d25/da5 39 2024-05-12T04:01:39.774 INFO:tasks.workunit.client.0.smithi032.stdout:1/938: creat d1/d7/d30/d35/d77/d110/d11d/d11f/f137 x:0 0 0 2024-05-12T04:01:39.789 INFO:tasks.workunit.client.0.smithi032.stdout:1/939: dwrite d1/d7/d30/f60 [4194304,4194304] 0 2024-05-12T04:01:39.794 INFO:tasks.workunit.client.0.smithi032.stdout:1/940: mknod d1/d7/d30/d18/d93/dc1/dc3/c138 0 2024-05-12T04:01:39.798 INFO:tasks.workunit.client.0.smithi032.stdout:1/941: mknod d1/d6a/d72/da9/c139 0 2024-05-12T04:01:39.800 INFO:tasks.workunit.client.0.smithi032.stdout:1/942: creat d1/dd/d24/f13a x:0 0 0 2024-05-12T04:01:39.806 INFO:tasks.workunit.client.0.smithi032.stdout:1/943: dread d1/d7/f78 [0,4194304] 0 2024-05-12T04:01:39.810 INFO:tasks.workunit.client.0.smithi032.stdout:1/944: truncate d1/d7/d30/d35/d77/d110/d11d/fd4 616229 0 2024-05-12T04:01:39.812 INFO:tasks.workunit.client.0.smithi032.stdout:1/945: mkdir d1/d6a/d72/da4/d10b/d13b 0 2024-05-12T04:01:39.827 INFO:tasks.workunit.client.0.smithi032.stdout:1/946: dwrite d1/d7/d30/d35/d77/d110/d11d/fd3 [0,4194304] 0 2024-05-12T04:01:39.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.827+0000 7f07733dd700 1 -- 172.21.15.32:0/1530311171 --> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f076c10e6e0 con 0x7f075806af90 2024-05-12T04:01:39.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.829+0000 7f0761ffb700 1 -- 172.21.15.32:0/1530311171 <== mgr.24389 v2:172.21.15.33:6828/1598726962 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7f076c10e6e0 con 0x7f075806af90 2024-05-12T04:01:39.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.831+0000 7f07733dd700 1 -- 172.21.15.32:0/1530311171 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f075806af90 msgr2=0x7f075806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:01:39.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.831+0000 7f07733dd700 1 --2- 172.21.15.32:0/1530311171 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f075806af90 0x7f075806d440 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f076c10d760 tx=0x7f076800b040 comp rx=0 tx=0).stop 2024-05-12T04:01:39.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.831+0000 7f07733dd700 1 -- 172.21.15.32:0/1530311171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f076c0fbcb0 msgr2=0x7f076c10c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:01:39.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.831+0000 7f07733dd700 1 --2- 172.21.15.32:0/1530311171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f076c0fbcb0 0x7f076c10c750 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f075c000c00 tx=0x7f075c009e30 comp rx=0 tx=0).stop 2024-05-12T04:01:39.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.831+0000 7f07733dd700 1 -- 172.21.15.32:0/1530311171 shutdown_connections 2024-05-12T04:01:39.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.831+0000 7f07733dd700 1 --2- 172.21.15.32:0/1530311171 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f075806af90 0x7f075806d440 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:39.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.831+0000 7f07733dd700 1 --2- 172.21.15.32:0/1530311171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f076c0fbcb0 0x7f076c10c750 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:39.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.831+0000 7f07733dd700 1 --2- 172.21.15.32:0/1530311171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f076c0fcf60 0x7f076c10cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:39.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.832+0000 7f07733dd700 1 -- 172.21.15.32:0/1530311171 >> 172.21.15.32:0/1530311171 conn(0x7f076c0f7430 msgr2=0x7f076c0f8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:01:39.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.832+0000 7f07733dd700 1 -- 172.21.15.32:0/1530311171 shutdown_connections 2024-05-12T04:01:39.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.832+0000 7f07733dd700 1 -- 172.21.15.32:0/1530311171 wait complete. 2024-05-12T04:01:39.837 INFO:tasks.workunit.client.0.smithi032.stdout:1/947: symlink d1/l13c 0 2024-05-12T04:01:39.841 INFO:tasks.workunit.client.0.smithi032.stdout:1/948: unlink d1/d7/d30/d18/d87/db7/c126 0 2024-05-12T04:01:39.846 INFO:tasks.workunit.client.0.smithi032.stdout:1/949: dread d1/d7/d30/d18/fab [0,4194304] 0 2024-05-12T04:01:39.853 INFO:tasks.workunit.client.0.smithi032.stdout:1/950: mknod d1/d7/d30/d18/d93/dc1/d109/c13d 0 2024-05-12T04:01:39.931 INFO:tasks.workunit.client.0.smithi032.stdout:4/841: sync 2024-05-12T04:01:39.931 INFO:tasks.workunit.client.0.smithi032.stdout:8/934: sync 2024-05-12T04:01:39.937 INFO:tasks.workunit.client.0.smithi032.stdout:8/935: creat d0/d9/d13/d21/f13f x:0 0 0 2024-05-12T04:01:39.938 INFO:tasks.workunit.client.0.smithi032.stdout:4/842: rename d0/fb9 to d0/d3/d12/d1d/d6b/db7/fff 0 2024-05-12T04:01:39.939 INFO:tasks.workunit.client.0.smithi032.stdout:4/843: stat d0/d3/d31/d80/d84/l59 0 2024-05-12T04:01:39.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.942+0000 7f5c6c6fe700 1 -- 172.21.15.32:0/132484313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c640fdcd0 msgr2=0x7f5c64100150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:01:39.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.942+0000 7f5c6c6fe700 1 --2- 172.21.15.32:0/132484313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c640fdcd0 0x7f5c64100150 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5c54004940 tx=0x7f5c54009e30 comp rx=0 tx=0).stop 2024-05-12T04:01:39.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.942+0000 7f5c6c6fe700 1 -- 172.21.15.32:0/132484313 shutdown_connections 2024-05-12T04:01:39.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.942+0000 7f5c6c6fe700 1 --2- 172.21.15.32:0/132484313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c640fdcd0 0x7f5c64100150 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:39.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.942+0000 7f5c6c6fe700 1 --2- 172.21.15.32:0/132484313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c640662c0 0x7f5c640fd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:39.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.942+0000 7f5c6c6fe700 1 -- 172.21.15.32:0/132484313 >> 172.21.15.32:0/132484313 conn(0x7f5c640f7430 msgr2=0x7f5c640f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:01:39.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.942+0000 7f5c6c6fe700 1 -- 172.21.15.32:0/132484313 shutdown_connections 2024-05-12T04:01:39.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.942+0000 7f5c6c6fe700 1 -- 172.21.15.32:0/132484313 wait complete. 2024-05-12T04:01:39.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.943+0000 7f5c6c6fe700 1 Processor -- start 2024-05-12T04:01:39.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.943+0000 7f5c6c6fe700 1 -- start start 2024-05-12T04:01:39.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.943+0000 7f5c6c6fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c640662c0 0x7f5c64113f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:01:39.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.943+0000 7f5c6c6fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c640fdcd0 0x7f5c641144b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:01:39.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.943+0000 7f5c6c6fe700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c64114ab0 con 0x7f5c640fdcd0 2024-05-12T04:01:39.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.943+0000 7f5c6c6fe700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c641152b0 con 0x7f5c640662c0 2024-05-12T04:01:39.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.943+0000 7f5c69c99700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c640fdcd0 0x7f5c641144b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:01:39.945 INFO:tasks.workunit.client.0.smithi032.stdout:4/844: dread d0/d3/d12/d1d/d63/f99 [0,4194304] 0 2024-05-12T04:01:39.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.943+0000 7f5c69c99700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c640fdcd0 0x7f5c641144b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38524/0 (socket says 172.21.15.32:38524) 2024-05-12T04:01:39.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.943+0000 7f5c69c99700 1 -- 172.21.15.32:0/546636193 learned_addr learned my addr 172.21.15.32:0/546636193 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:01:39.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.944+0000 7f5c6a49a700 1 --2- 172.21.15.32:0/546636193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c640662c0 0x7f5c64113f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:01:39.946 INFO:tasks.workunit.client.0.smithi032.stdout:8/936: truncate d0/d9/d1b/d27/d88/d56/d63/d8f/d9a/d28/d3e/fb6 857833 0 2024-05-12T04:01:39.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.944+0000 7f5c69c99700 1 -- 172.21.15.32:0/546636193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c640662c0 msgr2=0x7f5c64113f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:01:39.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.944+0000 7f5c69c99700 1 --2- 172.21.15.32:0/546636193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c640662c0 0x7f5c64113f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:39.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.944+0000 7f5c69c99700 1 -- 172.21.15.32:0/546636193 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c54005040 con 0x7f5c640fdcd0 2024-05-12T04:01:39.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.944+0000 7f5c6a49a700 1 --2- 172.21.15.32:0/546636193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c640662c0 0x7f5c64113f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-12T04:01:39.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.944+0000 7f5c69c99700 1 --2- 172.21.15.32:0/546636193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c640fdcd0 0x7f5c641144b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5c5400b3b0 tx=0x7f5c54009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:01:39.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.944+0000 7f5c5affd700 1 -- 172.21.15.32:0/546636193 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c5400e040 con 0x7f5c640fdcd0 2024-05-12T04:01:39.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.944+0000 7f5c6c6fe700 1 -- 172.21.15.32:0/546636193 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c64115530 con 0x7f5c640fdcd0 2024-05-12T04:01:39.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.944+0000 7f5c5affd700 1 -- 172.21.15.32:0/546636193 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5c5400bc50 con 0x7f5c640fdcd0 2024-05-12T04:01:39.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.944+0000 7f5c6c6fe700 1 -- 172.21.15.32:0/546636193 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c64115a70 con 0x7f5c640fdcd0 2024-05-12T04:01:39.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.944+0000 7f5c5affd700 1 -- 172.21.15.32:0/546636193 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c5401c630 con 0x7f5c640fdcd0 2024-05-12T04:01:39.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.946+0000 7f5c6c6fe700 1 -- 172.21.15.32:0/546636193 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c480052e0 con 0x7f5c640fdcd0 2024-05-12T04:01:39.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.946+0000 7f5c5affd700 1 -- 172.21.15.32:0/546636193 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 25) v1 ==== 88772+0+0 (secure 0 0 0) 0x7f5c54026440 con 0x7f5c640fdcd0 2024-05-12T04:01:39.949 INFO:tasks.workunit.client.0.smithi032.stdout:8/937: stat d0/d9/d1b/d27/d88/d56/d63/d8f/db0/c11b 0 2024-05-12T04:01:39.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.947+0000 7f5c5affd700 1 --2- 172.21.15.32:0/546636193 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f5c5006aec0 0x7f5c5006d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:01:39.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.947+0000 7f5c5affd700 1 -- 172.21.15.32:0/546636193 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f5c5408a420 con 0x7f5c640fdcd0 2024-05-12T04:01:39.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.947+0000 7f5c6a49a700 1 --2- 172.21.15.32:0/546636193 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f5c5006aec0 0x7f5c5006d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:01:39.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.947+0000 7f5c6a49a700 1 --2- 172.21.15.32:0/546636193 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f5c5006aec0 0x7f5c5006d370 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5c60004fd0 tx=0x7f5c600043c0 comp rx=0 tx=0).ready entity=mgr.24389 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:01:39.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:39.953+0000 7f5c5affd700 1 -- 172.21.15.32:0/546636193 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181777 (secure 0 0 0) 0x7f5c54058420 con 0x7f5c640fdcd0 2024-05-12T04:01:39.954 INFO:tasks.workunit.client.0.smithi032.stdout:4/845: link d0/d3/df/d4e/f7e d0/d3/d12/f100 0 2024-05-12T04:01:39.959 INFO:tasks.workunit.client.0.smithi032.stdout:4/846: symlink d0/d3/d31/d80/d84/dc9/l101 0 2024-05-12T04:01:39.965 INFO:tasks.workunit.client.0.smithi032.stdout:8/938: dread d0/d9/d1b/d27/d4c/d4e/d90/faf [0,4194304] 0 2024-05-12T04:01:39.966 INFO:tasks.workunit.client.0.smithi032.stdout:4/847: dwrite d0/d3/d31/ff2 [0,4194304] 0 2024-05-12T04:01:39.968 INFO:tasks.workunit.client.0.smithi032.stdout:8/939: stat d0/d9/d1b/d27/d88/d56/f65 0 2024-05-12T04:01:39.974 INFO:tasks.workunit.client.0.smithi032.stdout:4/848: creat d0/d3/d31/d80/d84/dc9/f102 x:0 0 0 2024-05-12T04:01:39.976 INFO:tasks.workunit.client.0.smithi032.stdout:4/849: write d0/d3/d12/d1d/fa4 [672181,61373] 0 2024-05-12T04:01:39.985 INFO:tasks.workunit.client.0.smithi032.stdout:4/850: creat d0/d3/d12/d1d/d63/d77/f103 x:0 0 0 2024-05-12T04:01:39.987 INFO:tasks.workunit.client.0.smithi032.stdout:8/940: dread d0/d9/d1b/d33/f3a [0,4194304] 0 2024-05-12T04:01:39.992 INFO:tasks.workunit.client.0.smithi032.stdout:4/851: dwrite d0/d3/d24/dae/fdc [0,4194304] 0 2024-05-12T04:01:39.994 INFO:tasks.workunit.client.0.smithi032.stdout:4/852: read d0/dde/ff9 [101504,71275] 0 2024-05-12T04:01:40.005 INFO:tasks.workunit.client.0.smithi032.stdout:4/853: unlink d0/d3/d12/d8d/f98 0 2024-05-12T04:01:40.005 INFO:tasks.workunit.client.0.smithi032.stdout:4/854: fsync d0/d3/df/f86 0 2024-05-12T04:01:40.018 INFO:tasks.workunit.client.0.smithi032.stdout:8/941: dwrite d0/d9/d1b/d27/d88/f39 [0,4194304] 0 2024-05-12T04:01:40.022 INFO:tasks.workunit.client.0.smithi032.stdout:4/855: dwrite d0/d3/d24/f72 [0,4194304] 0 2024-05-12T04:01:40.036 INFO:tasks.workunit.client.0.smithi032.stdout:4/856: write d0/d3/d31/f85 [2047259,19315] 0 2024-05-12T04:01:40.041 INFO:tasks.workunit.client.0.smithi032.stdout:8/942: dwrite d0/d9/d1b/d66/f12e [0,4194304] 0 2024-05-12T04:01:40.046 INFO:tasks.workunit.client.0.smithi032.stdout:4/857: dwrite d0/d3/d31/d80/fdf [0,4194304] 0 2024-05-12T04:01:40.068 INFO:tasks.workunit.client.0.smithi032.stdout:8/943: write d0/d7/f11 [1483047,30507] 0 2024-05-12T04:01:40.070 INFO:tasks.workunit.client.0.smithi032.stdout:4/858: write d0/f6f [1918611,30496] 0 2024-05-12T04:01:40.071 INFO:tasks.workunit.client.0.smithi032.stdout:4/859: chown d0/d3/d31/d80 11 1 2024-05-12T04:01:40.072 INFO:tasks.workunit.client.0.smithi032.stdout:4/860: write d0/f6f [1040408,71913] 0 2024-05-12T04:01:40.073 INFO:tasks.workunit.client.0.smithi032.stdout:4/861: fdatasync d0/d3/d12/d1d/dc5/fe4 0 2024-05-12T04:01:40.079 INFO:tasks.workunit.client.0.smithi032.stdout:4/862: mknod d0/d3/d12/d1d/d63/d87/c104 0 2024-05-12T04:01:40.080 INFO:tasks.workunit.client.0.smithi032.stdout:8/944: link d0/d4b/dad/ld4 d0/d9/d13/d92/l140 0 2024-05-12T04:01:40.083 INFO:tasks.workunit.client.0.smithi032.stdout:4/863: rename d0/d3/d12/d1d/d6b/f9a to d0/d3/df/dac/f105 0 2024-05-12T04:01:40.085 INFO:tasks.workunit.client.0.smithi032.stdout:9/823: sync 2024-05-12T04:01:40.085 INFO:tasks.workunit.client.0.smithi032.stdout:7/825: sync 2024-05-12T04:01:40.087 INFO:tasks.workunit.client.0.smithi032.stdout:4/864: read d0/d3/d12/faa [32563,127524] 0 2024-05-12T04:01:40.088 INFO:tasks.workunit.client.0.smithi032.stdout:8/945: unlink d0/d9/d13/d21/l43 0 2024-05-12T04:01:40.092 INFO:tasks.workunit.client.0.smithi032.stdout:9/824: creat d0/d3/d74/d51/f102 x:0 0 0 2024-05-12T04:01:40.093 INFO:tasks.workunit.client.0.smithi032.stdout:4/865: chown d0/d3/df/d4e/fdd 51957 1 2024-05-12T04:01:40.093 INFO:tasks.workunit.client.0.smithi032.stdout:8/946: dread d0/d9/d1b/d27/d88/dd7/ff4 [0,4194304] 0 2024-05-12T04:01:40.096 INFO:tasks.workunit.client.0.smithi032.stdout:8/947: write d0/d9/d13/d21/d35/de9/f10a [1356646,95664] 0 2024-05-12T04:01:40.096 INFO:tasks.workunit.client.0.smithi032.stdout:9/825: rmdir d0/d3/d6/d7/d27 39 2024-05-12T04:01:40.102 INFO:tasks.workunit.client.0.smithi032.stdout:8/948: creat d0/d9/d1b/d27/d59/f141 x:0 0 0 2024-05-12T04:01:40.105 INFO:tasks.workunit.client.0.smithi032.stdout:4/866: creat d0/d3/d12/f106 x:0 0 0 2024-05-12T04:01:40.108 INFO:tasks.workunit.client.0.smithi032.stdout:4/867: chown d0/d3/df/f29 6277 1 2024-05-12T04:01:40.110 INFO:tasks.workunit.client.0.smithi032.stdout:4/868: chown d0/d3/d31/f85 9 1 2024-05-12T04:01:40.111 INFO:tasks.workunit.client.0.smithi032.stdout:8/949: dwrite d0/d9/d13/d21/d35/de9/f118 [0,4194304] 0 2024-05-12T04:01:40.114 INFO:tasks.workunit.client.0.smithi032.stdout:8/950: symlink d0/d9/d13/d92/d124/d12b/l142 0 2024-05-12T04:01:40.120 INFO:tasks.workunit.client.0.smithi032.stdout:4/869: rmdir d0/d3/d12/d8d/df7 0 2024-05-12T04:01:40.127 INFO:tasks.workunit.client.0.smithi032.stdout:4/870: dwrite d0/d3/d31/d80/d84/fa9 [4194304,4194304] 0 2024-05-12T04:01:40.128 INFO:tasks.workunit.client.0.smithi032.stdout:8/951: creat d0/d9/d1b/d27/d88/d56/d63/d8f/d9a/f143 x:0 0 0 2024-05-12T04:01:40.130 INFO:tasks.workunit.client.0.smithi032.stdout:8/952: fsync d0/d9/d1b/d27/d88/d56/d63/d8f/d9a/d28/f49 0 2024-05-12T04:01:40.149 INFO:tasks.workunit.client.0.smithi032.stdout:9/826: write d0/d3/d6/d7/d27/f31 [838006,124001] 0 2024-05-12T04:01:40.151 INFO:tasks.workunit.client.0.smithi032.stdout:7/826: dwrite dd/f82 [0,4194304] 0 2024-05-12T04:01:40.152 INFO:tasks.workunit.client.0.smithi032.stdout:7/827: stat f5 0 2024-05-12T04:01:40.163 INFO:tasks.workunit.client.0.smithi032.stdout:4/871: dread d0/db5/fbc [0,4194304] 0 2024-05-12T04:01:40.165 INFO:tasks.workunit.client.0.smithi032.stdout:7/828: mknod dd/d1b/d1e/de3/d9a/d9c/cff 0 2024-05-12T04:01:40.171 INFO:tasks.workunit.client.0.smithi032.stdout:7/829: dread dd/d1b/d43/d5e/f6f [0,4194304] 0 2024-05-12T04:01:40.172 INFO:tasks.workunit.client.0.smithi032.stdout:9/827: unlink d0/d3/d6/d7/d2a/fb3 0 2024-05-12T04:01:40.178 INFO:tasks.workunit.client.0.smithi032.stdout:7/830: dread - dd/d1b/d1e/de3/d9a/d9c/fe1 zero size 2024-05-12T04:01:40.180 INFO:tasks.workunit.client.0.smithi032.stdout:8/953: write d0/d9/d1b/d27/d88/f79 [2323150,17967] 0 2024-05-12T04:01:40.194 INFO:tasks.workunit.client.0.smithi032.stdout:7/831: write dd/d1b/d43/d5e/ffe [653059,81784] 0 2024-05-12T04:01:40.196 INFO:tasks.workunit.client.0.smithi032.stdout:9/828: dwrite d0/d3/d4b/d88/dc4/fdc [0,4194304] 0 2024-05-12T04:01:40.197 INFO:tasks.workunit.client.0.smithi032.stdout:7/832: stat dd/d1b/d1e/de3/d9a/d9c/dc2/daf/ce9 0 2024-05-12T04:01:40.199 INFO:tasks.workunit.client.0.smithi032.stdout:8/954: truncate d0/d9/d13/f29 2409418 0 2024-05-12T04:01:40.204 INFO:tasks.workunit.client.0.smithi032.stdout:9/829: symlink d0/d3/d6/l103 0 2024-05-12T04:01:40.204 INFO:tasks.workunit.client.0.smithi032.stdout:7/833: truncate dd/d1b/f4b 323016 0 2024-05-12T04:01:40.207 INFO:tasks.workunit.client.0.smithi032.stdout:9/830: write d0/d3/d6/d7/da/ffd [1168801,116913] 0 2024-05-12T04:01:40.207 INFO:tasks.workunit.client.0.smithi032.stdout:7/834: chown dd/d1b/d1e/de3/d9a/d9c/db9/fcc 19 1 2024-05-12T04:01:40.209 INFO:tasks.workunit.client.0.smithi032.stdout:8/955: dwrite d0/d9/d1b/d27/d88/f39 [4194304,4194304] 0 2024-05-12T04:01:40.217 INFO:tasks.workunit.client.0.smithi032.stdout:8/956: fdatasync d0/d9/d13/d21/d25/db5/fdc 0 2024-05-12T04:01:40.219 INFO:tasks.workunit.client.0.smithi032.stdout:8/957: dread - d0/d9/d1b/d33/f130 zero size 2024-05-12T04:01:40.222 INFO:tasks.workunit.client.0.smithi032.stdout:8/958: rmdir d0/d9/d13/d21 39 2024-05-12T04:01:40.224 INFO:tasks.workunit.client.0.smithi032.stdout:8/959: rmdir d0/d9/d1b/d27/d59 39 2024-05-12T04:01:40.226 INFO:tasks.workunit.client.0.smithi032.stdout:8/960: write d0/d9/d1b/d27/d88/dd7/ff4 [2035303,88374] 0 2024-05-12T04:01:40.231 INFO:tasks.workunit.client.0.smithi032.stdout:9/831: write d0/d3/d6/d7/d2a/fc1 [303510,108033] 0 2024-05-12T04:01:40.233 INFO:tasks.workunit.client.0.smithi032.stdout:9/832: write d0/d3/d6/f69 [242867,121070] 0 2024-05-12T04:01:40.233 INFO:tasks.workunit.client.0.smithi032.stdout:9/833: readlink d0/d3/d74/lf3 0 2024-05-12T04:01:40.238 INFO:tasks.workunit.client.0.smithi032.stdout:9/834: symlink d0/d3/d6/l104 0 2024-05-12T04:01:40.241 INFO:tasks.workunit.client.0.smithi032.stdout:9/835: creat d0/d84/f105 x:0 0 0 2024-05-12T04:01:40.242 INFO:tasks.workunit.client.0.smithi032.stdout:9/836: fdatasync d0/d3/d6/d7/d2a/d9f/fa1 0 2024-05-12T04:01:40.243 INFO:tasks.workunit.client.0.smithi032.stdout:9/837: chown d0/la2 56 1 2024-05-12T04:01:40.248 INFO:tasks.workunit.client.0.smithi032.stdout:9/838: rmdir d0/d3/d4b/dda 0 2024-05-12T04:01:40.249 INFO:tasks.workunit.client.0.smithi032.stdout:9/839: fsync d0/d3/d4b/d88/dc4/fdc 0 2024-05-12T04:01:40.250 INFO:tasks.workunit.client.0.smithi032.stdout:9/840: read - d0/d3/d74/d51/f102 zero size 2024-05-12T04:01:40.256 INFO:tasks.workunit.client.0.smithi032.stdout:9/841: mkdir d0/d3/d6/d7/d106 0 2024-05-12T04:01:40.269 INFO:tasks.workunit.client.0.smithi032.stdout:9/842: write d0/d3/d4b/d85/d87/fb0 [433074,115549] 0 2024-05-12T04:01:40.273 INFO:tasks.workunit.client.0.smithi032.stdout:9/843: symlink d0/d3/d74/dbf/l107 0 2024-05-12T04:01:40.277 INFO:tasks.workunit.client.0.smithi032.stdout:9/844: dread d0/d3/d6/d7/da/ffd [0,4194304] 0 2024-05-12T04:01:40.284 INFO:tasks.workunit.client.0.smithi032.stdout:2/896: sync 2024-05-12T04:01:40.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.283+0000 7f5c6c6fe700 1 -- 172.21.15.32:0/546636193 --> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5c48000bf0 con 0x7f5c5006aec0 2024-05-12T04:01:40.286 INFO:tasks.workunit.client.0.smithi032.stdout:2/897: write d1/d13/d70/d85/dd5/f10f [8749,2840] 0 2024-05-12T04:01:40.296 INFO:tasks.workunit.client.0.smithi032.stdout:9/845: dwrite d0/d3/d4b/d88/d99/f101 [0,4194304] 0 2024-05-12T04:01:40.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.295+0000 7f5c5affd700 1 -- 172.21.15.32:0/546636193 <== mgr.24389 v2:172.21.15.33:6828/1598726962 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3410 (secure 0 0 0) 0x7f5c48000bf0 con 0x7f5c5006aec0 2024-05-12T04:01:40.297 INFO:teuthology.orchestra.run.smithi032.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-12T04:01:40.297 INFO:teuthology.orchestra.run.smithi032.stdout:alertmanager.smithi032 smithi032 *:9093,9094 starting - - - - 2024-05-12T04:01:40.297 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi032 smithi032 running (9m) 82s ago 9m 6651k - 17.2.7-939-gc190bdee 284803f7d9cf a478874e9dfa 2024-05-12T04:01:40.297 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi033 smithi033 running (8m) 82s ago 8m 6651k - 17.2.7-939-gc190bdee 284803f7d9cf 34c62d004c7e 2024-05-12T04:01:40.297 INFO:teuthology.orchestra.run.smithi032.stdout:grafana.smithi032 smithi032 *:3000 starting - - - - 2024-05-12T04:01:40.297 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.ialtqb smithi032 running (3m) 82s ago 3m 11.6M - 17.2.7-939-gc190bdee 284803f7d9cf 0683abf00e3e 2024-05-12T04:01:40.297 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.qpsawh smithi032 running (3m) 82s ago 3m 104M - 17.2.7-939-gc190bdee 284803f7d9cf 95a7db5c4fcd 2024-05-12T04:01:40.297 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi033.hglwml smithi033 running (3m) 82s ago 3m 11.6M - 17.2.7-939-gc190bdee 284803f7d9cf c97902b05025 2024-05-12T04:01:40.297 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi033.jgbjhk smithi033 running (3m) 82s ago 3m 82.6M - 17.2.7-939-gc190bdee 284803f7d9cf af97052cb97a 2024-05-12T04:01:40.298 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi032.ukatkq smithi032 *:9283 running (11m) 82s ago 11m 158M - 17.2.7-939-gc190bdee 284803f7d9cf d9763dc47404 2024-05-12T04:01:40.298 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi033.pcvxwy smithi033 *:8443,9283 running (102s) 82s ago 8m 484M - 18.2.2-1272-g58249552 ddcb0a9d6fde b0bff3ab143f 2024-05-12T04:01:40.298 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi032 smithi032 running (11m) 82s ago 11m 49.9M 2048M 17.2.7-939-gc190bdee 284803f7d9cf d7fea430c6e2 2024-05-12T04:01:40.298 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi033 smithi033 running (8m) 82s ago 8m 40.1M 2048M 17.2.7-939-gc190bdee 284803f7d9cf 707654d85748 2024-05-12T04:01:40.298 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi032 smithi032 *:9100 starting - - - - 2024-05-12T04:01:40.298 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi033 smithi033 *:9100 running (8m) 82s ago 8m 17.4M - 1.5.0 0da6a335fe13 1144caed00e9 2024-05-12T04:01:40.298 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0 smithi032 running (7m) 82s ago 7m 66.2M 1570M 17.2.7-939-gc190bdee 284803f7d9cf 75c78a47f8e5 2024-05-12T04:01:40.298 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1 smithi032 running (7m) 82s ago 7m 67.5M 1570M 17.2.7-939-gc190bdee 284803f7d9cf 87cfd247d8d6 2024-05-12T04:01:40.298 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2 smithi032 running (6m) 82s ago 6m 57.7M 1570M 17.2.7-939-gc190bdee 284803f7d9cf 7aef7affb8ba 2024-05-12T04:01:40.298 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3 smithi033 running (6m) 82s ago 6m 77.0M 2594M 17.2.7-939-gc190bdee 284803f7d9cf 3a92b5e3348e 2024-05-12T04:01:40.298 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4 smithi033 running (5m) 82s ago 5m 63.5M 2594M 17.2.7-939-gc190bdee 284803f7d9cf 0896268b4591 2024-05-12T04:01:40.298 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5 smithi033 running (5m) 82s ago 5m 64.4M 2594M 17.2.7-939-gc190bdee 284803f7d9cf 57e65a64c821 2024-05-12T04:01:40.298 INFO:teuthology.orchestra.run.smithi032.stdout:prometheus.smithi032 smithi032 *:9095 running (7m) 82s ago 9m 57.7M - 2.43.0 a07b618ecd1d 0b3ce3323a49 2024-05-12T04:01:40.299 INFO:tasks.workunit.client.0.smithi032.stdout:2/898: dwrite d1/d4d/d5e/d9a/ff0 [0,4194304] 0 2024-05-12T04:01:40.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.298+0000 7f5c6c6fe700 1 -- 172.21.15.32:0/546636193 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f5c5006aec0 msgr2=0x7f5c5006d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:01:40.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.298+0000 7f5c6c6fe700 1 --2- 172.21.15.32:0/546636193 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f5c5006aec0 0x7f5c5006d370 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5c60004fd0 tx=0x7f5c600043c0 comp rx=0 tx=0).stop 2024-05-12T04:01:40.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.298+0000 7f5c6c6fe700 1 -- 172.21.15.32:0/546636193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c640fdcd0 msgr2=0x7f5c641144b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:01:40.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.298+0000 7f5c6c6fe700 1 --2- 172.21.15.32:0/546636193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c640fdcd0 0x7f5c641144b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5c5400b3b0 tx=0x7f5c54009e30 comp rx=0 tx=0).stop 2024-05-12T04:01:40.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.298+0000 7f5c6c6fe700 1 -- 172.21.15.32:0/546636193 shutdown_connections 2024-05-12T04:01:40.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.298+0000 7f5c6c6fe700 1 --2- 172.21.15.32:0/546636193 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f5c5006aec0 0x7f5c5006d370 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:40.301 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.298+0000 7f5c6c6fe700 1 --2- 172.21.15.32:0/546636193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c640662c0 0x7f5c64113f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:40.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.298+0000 7f5c6c6fe700 1 --2- 172.21.15.32:0/546636193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c640fdcd0 0x7f5c641144b0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:40.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.298+0000 7f5c6c6fe700 1 -- 172.21.15.32:0/546636193 >> 172.21.15.32:0/546636193 conn(0x7f5c640f7430 msgr2=0x7f5c640f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:01:40.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.299+0000 7f5c6c6fe700 1 -- 172.21.15.32:0/546636193 shutdown_connections 2024-05-12T04:01:40.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.299+0000 7f5c6c6fe700 1 -- 172.21.15.32:0/546636193 wait complete. 2024-05-12T04:01:40.306 INFO:tasks.workunit.client.0.smithi032.stdout:2/899: dread d1/d34/da5/fb5 [0,4194304] 0 2024-05-12T04:01:40.313 INFO:tasks.workunit.client.0.smithi032.stdout:9/846: dwrite d0/d84/d8a/f94 [0,4194304] 0 2024-05-12T04:01:40.315 INFO:tasks.workunit.client.0.smithi032.stdout:2/900: symlink d1/d34/d96/d41/d52/df7/l129 0 2024-05-12T04:01:40.315 INFO:tasks.workunit.client.0.smithi032.stdout:2/901: chown d1/d73/cab 0 1 2024-05-12T04:01:40.319 INFO:tasks.workunit.client.0.smithi032.stdout:2/902: creat d1/d34/d96/d41/d39/f12a x:0 0 0 2024-05-12T04:01:40.321 INFO:tasks.workunit.client.0.smithi032.stdout:9/847: mknod d0/d3/d6/d7/da/c108 0 2024-05-12T04:01:40.323 INFO:tasks.workunit.client.0.smithi032.stdout:9/848: symlink d0/d3/d6/d7/d2a/l109 0 2024-05-12T04:01:40.325 INFO:tasks.workunit.client.0.smithi032.stdout:9/849: mkdir d0/d10a 0 2024-05-12T04:01:40.329 INFO:tasks.workunit.client.0.smithi032.stdout:9/850: mkdir d0/d3/d74/d51/dd8/d10b 0 2024-05-12T04:01:40.331 INFO:tasks.workunit.client.0.smithi032.stdout:9/851: mkdir d0/d84/d8a/da3/db7/d10c 0 2024-05-12T04:01:40.334 INFO:tasks.workunit.client.0.smithi032.stdout:9/852: truncate d0/d3/d6/d7/da/f1a 10029504 0 2024-05-12T04:01:40.346 INFO:tasks.workunit.client.0.smithi032.stdout:9/853: write d0/d3/d4b/d6f/dd7/ff9 [873464,26786] 0 2024-05-12T04:01:40.347 INFO:tasks.workunit.client.0.smithi032.stdout:9/854: chown d0/d3/c8f 18 1 2024-05-12T04:01:40.355 INFO:tasks.workunit.client.0.smithi032.stdout:9/855: getdents d0/d84/d8a/da3/db7 0 2024-05-12T04:01:40.356 INFO:tasks.workunit.client.0.smithi032.stdout:9/856: write d0/d84/d8a/da3/fa4 [2923694,33311] 0 2024-05-12T04:01:40.357 INFO:tasks.workunit.client.0.smithi032.stdout:9/857: read d0/d84/d8a/f94 [1318281,47130] 0 2024-05-12T04:01:40.369 INFO:tasks.workunit.client.0.smithi032.stdout:9/858: dread d0/d84/faf [4194304,4194304] 0 2024-05-12T04:01:40.371 INFO:tasks.workunit.client.0.smithi032.stdout:9/859: write d0/d3/d4b/d88/d99/df8/ffc [2014585,30006] 0 2024-05-12T04:01:40.379 INFO:tasks.workunit.client.0.smithi032.stdout:9/860: symlink d0/d84/da6/de1/l10d 0 2024-05-12T04:01:40.388 INFO:tasks.workunit.client.0.smithi032.stdout:9/861: rename d0/d3/d6/d7/d27/f31 to d0/d84/da6/f10e 0 2024-05-12T04:01:40.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.447+0000 7fae4472e700 1 -- 172.21.15.32:0/896386091 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae3c0fbcb0 msgr2=0x7fae3c0fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:01:40.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.447+0000 7fae4472e700 1 --2- 172.21.15.32:0/896386091 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae3c0fbcb0 0x7fae3c0fc0c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fae38004940 tx=0x7fae38009e30 comp rx=0 tx=0).stop 2024-05-12T04:01:40.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.448+0000 7fae4472e700 1 -- 172.21.15.32:0/896386091 shutdown_connections 2024-05-12T04:01:40.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.448+0000 7fae4472e700 1 --2- 172.21.15.32:0/896386091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae3c0fcf60 0x7fae3c0fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:40.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.448+0000 7fae4472e700 1 --2- 172.21.15.32:0/896386091 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae3c0fbcb0 0x7fae3c0fc0c0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:40.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.448+0000 7fae4472e700 1 -- 172.21.15.32:0/896386091 >> 172.21.15.32:0/896386091 conn(0x7fae3c0f7430 msgr2=0x7fae3c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:01:40.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.448+0000 7fae4472e700 1 -- 172.21.15.32:0/896386091 shutdown_connections 2024-05-12T04:01:40.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.448+0000 7fae4472e700 1 -- 172.21.15.32:0/896386091 wait complete. 2024-05-12T04:01:40.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.448+0000 7fae4472e700 1 Processor -- start 2024-05-12T04:01:40.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.448+0000 7fae4472e700 1 -- start start 2024-05-12T04:01:40.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.449+0000 7fae4472e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae3c0fbcb0 0x7fae3c10c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:01:40.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.449+0000 7fae4472e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae3c0fcf60 0x7fae3c10cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:01:40.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.449+0000 7fae4472e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3c10d290 con 0x7fae3c0fbcb0 2024-05-12T04:01:40.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.449+0000 7fae4472e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3c10dae0 con 0x7fae3c0fcf60 2024-05-12T04:01:40.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.449+0000 7fae424ca700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae3c0fbcb0 0x7fae3c10c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:01:40.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.449+0000 7fae424ca700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae3c0fbcb0 0x7fae3c10c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38540/0 (socket says 172.21.15.32:38540) 2024-05-12T04:01:40.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.449+0000 7fae424ca700 1 -- 172.21.15.32:0/1218846095 learned_addr learned my addr 172.21.15.32:0/1218846095 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:01:40.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.449+0000 7fae41cc9700 1 --2- 172.21.15.32:0/1218846095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae3c0fcf60 0x7fae3c10cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:01:40.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.449+0000 7fae424ca700 1 -- 172.21.15.32:0/1218846095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae3c0fcf60 msgr2=0x7fae3c10cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:01:40.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.449+0000 7fae424ca700 1 --2- 172.21.15.32:0/1218846095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae3c0fcf60 0x7fae3c10cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:40.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.449+0000 7fae424ca700 1 -- 172.21.15.32:0/1218846095 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae38005040 con 0x7fae3c0fbcb0 2024-05-12T04:01:40.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.449+0000 7fae424ca700 1 --2- 172.21.15.32:0/1218846095 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae3c0fbcb0 0x7fae3c10c750 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fae380137b0 tx=0x7fae38009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:01:40.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.449+0000 7fae32ffd700 1 -- 172.21.15.32:0/1218846095 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae3800e070 con 0x7fae3c0fbcb0 2024-05-12T04:01:40.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.449+0000 7fae4472e700 1 -- 172.21.15.32:0/1218846095 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae3c10dce0 con 0x7fae3c0fbcb0 2024-05-12T04:01:40.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.450+0000 7fae32ffd700 1 -- 172.21.15.32:0/1218846095 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fae3801c400 con 0x7fae3c0fbcb0 2024-05-12T04:01:40.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.450+0000 7fae32ffd700 1 -- 172.21.15.32:0/1218846095 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae3800b650 con 0x7fae3c0fbcb0 2024-05-12T04:01:40.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.450+0000 7fae4472e700 1 -- 172.21.15.32:0/1218846095 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae3c10e240 con 0x7fae3c0fbcb0 2024-05-12T04:01:40.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.451+0000 7fae4472e700 1 -- 172.21.15.32:0/1218846095 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae3c05f5c0 con 0x7fae3c0fbcb0 2024-05-12T04:01:40.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.451+0000 7fae32ffd700 1 -- 172.21.15.32:0/1218846095 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 25) v1 ==== 88772+0+0 (secure 0 0 0) 0x7fae38013a30 con 0x7fae3c0fbcb0 2024-05-12T04:01:40.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.452+0000 7fae32ffd700 1 --2- 172.21.15.32:0/1218846095 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7fae2806aec0 0x7fae2806d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:01:40.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.452+0000 7fae32ffd700 1 -- 172.21.15.32:0/1218846095 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fae3808af30 con 0x7fae3c0fbcb0 2024-05-12T04:01:40.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.452+0000 7fae41cc9700 1 --2- 172.21.15.32:0/1218846095 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7fae2806aec0 0x7fae2806d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:01:40.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.452+0000 7fae41cc9700 1 --2- 172.21.15.32:0/1218846095 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7fae2806aec0 0x7fae2806d370 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fae2c009b80 tx=0x7fae2c00b040 comp rx=0 tx=0).ready entity=mgr.24389 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:01:40.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.456+0000 7fae32ffd700 1 -- 172.21.15.32:0/1218846095 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181777 (secure 0 0 0) 0x7fae38058e10 con 0x7fae3c0fbcb0 2024-05-12T04:01:40.467 INFO:tasks.workunit.client.0.smithi032.stdout:9/862: truncate d0/d3/d6/d58/f9a 4328747 0 2024-05-12T04:01:40.474 INFO:tasks.workunit.client.0.smithi032.stdout:9/863: write d0/d3/d74/fc0 [4426243,104030] 0 2024-05-12T04:01:40.477 INFO:tasks.workunit.client.0.smithi032.stdout:9/864: fsync d0/d3/d4b/f81 0 2024-05-12T04:01:40.478 INFO:tasks.workunit.client.0.smithi032.stdout:9/865: fsync d0/d3/d74/f20 0 2024-05-12T04:01:40.482 INFO:tasks.workunit.client.0.smithi032.stdout:9/866: creat d0/d3/d6/d7/da/f10f x:0 0 0 2024-05-12T04:01:40.484 INFO:tasks.workunit.client.0.smithi032.stdout:9/867: fsync d0/d3/d6/d7/d2a/d9f/fa1 0 2024-05-12T04:01:40.484 INFO:tasks.workunit.client.0.smithi032.stdout:9/868: chown d0/d3/d6/d58/f66 84207 1 2024-05-12T04:01:40.485 INFO:tasks.workunit.client.0.smithi032.stdout:9/869: creat d0/d3/d1b/f110 x:0 0 0 2024-05-12T04:01:40.486 INFO:tasks.workunit.client.0.smithi032.stdout:9/870: fdatasync d0/d3/d6/f69 0 2024-05-12T04:01:40.495 INFO:tasks.workunit.client.0.smithi032.stdout:9/871: rename d0/d3/d6/d7/da to d0/d84/d8a/da3/db7/d10c/d111 0 2024-05-12T04:01:40.498 INFO:tasks.workunit.client.0.smithi032.stdout:1/951: sync 2024-05-12T04:01:40.498 INFO:tasks.workunit.client.0.smithi032.stdout:0/921: sync 2024-05-12T04:01:40.503 INFO:tasks.workunit.client.0.smithi032.stdout:1/952: mkdir d1/d7/d30/d35/d54/d7e/df9/d13e 0 2024-05-12T04:01:40.505 INFO:tasks.workunit.client.0.smithi032.stdout:9/872: dwrite d0/d84/d8a/da3/db7/d10c/d111/ffd [0,4194304] 0 2024-05-12T04:01:40.513 INFO:tasks.workunit.client.0.smithi032.stdout:1/953: creat d1/d7/d49/dd1/f13f x:0 0 0 2024-05-12T04:01:40.518 INFO:tasks.workunit.client.0.smithi032.stdout:9/873: dwrite d0/d3/d74/d51/f7f [0,4194304] 0 2024-05-12T04:01:40.519 INFO:tasks.workunit.client.0.smithi032.stdout:0/922: dwrite df/d10/d3b/dba/f109 [0,4194304] 0 2024-05-12T04:01:40.528 INFO:tasks.workunit.client.0.smithi032.stdout:9/874: creat d0/d84/d8a/da3/db7/d10c/d111/f112 x:0 0 0 2024-05-12T04:01:40.529 INFO:tasks.workunit.client.0.smithi032.stdout:0/923: rmdir df/d10/d3d/d4a/d8e/d130 0 2024-05-12T04:01:40.536 INFO:tasks.workunit.client.0.smithi032.stdout:9/875: rmdir d0/d84 39 2024-05-12T04:01:40.537 INFO:tasks.workunit.client.0.smithi032.stdout:9/876: dread - d0/d3/d74/fc8 zero size 2024-05-12T04:01:40.542 INFO:tasks.workunit.client.0.smithi032.stdout:0/924: rename df/d30/d8d/d8f/dff to df/d10/d3d/d131 0 2024-05-12T04:01:40.544 INFO:tasks.workunit.client.0.smithi032.stdout:9/877: symlink d0/d3/d4b/d85/dbe/d82/l113 0 2024-05-12T04:01:40.546 INFO:tasks.workunit.client.0.smithi032.stdout:9/878: write d0/d3/d6/d7/d2a/d9f/fa1 [1022949,37588] 0 2024-05-12T04:01:40.549 INFO:tasks.workunit.client.0.smithi032.stdout:0/925: creat df/d10/d3d/d83/d9f/dd3/f132 x:0 0 0 2024-05-12T04:01:40.549 INFO:tasks.workunit.client.0.smithi032.stdout:9/879: write d0/d3/d6/d7/d2a/d46/d91/dc5/ff2 [306776,53762] 0 2024-05-12T04:01:40.552 INFO:tasks.workunit.client.0.smithi032.stdout:0/926: dread df/d10/d3d/d4a/d9a/d9d/fb5 [0,4194304] 0 2024-05-12T04:01:40.556 INFO:tasks.workunit.client.0.smithi032.stdout:9/880: fsync d0/d84/d8a/da3/db7/d10c/d111/f72 0 2024-05-12T04:01:40.557 INFO:tasks.workunit.client.0.smithi032.stdout:9/881: write d0/d3/d74/fc8 [58919,21324] 0 2024-05-12T04:01:40.558 INFO:tasks.workunit.client.0.smithi032.stdout:0/927: truncate df/d10/d48/d70/dbb/fc9 309629 0 2024-05-12T04:01:40.562 INFO:tasks.workunit.client.0.smithi032.stdout:9/882: unlink d0/d84/d8a/da3/db7/d10c/d111/f33 0 2024-05-12T04:01:40.562 INFO:tasks.workunit.client.0.smithi032.stdout:9/883: stat d0/d3/c8f 0 2024-05-12T04:01:40.566 INFO:tasks.workunit.client.0.smithi032.stdout:0/928: mkdir df/d10/d3d/d4a/d84/d133 0 2024-05-12T04:01:40.566 INFO:tasks.workunit.client.0.smithi032.stdout:9/884: dread d0/d3/d4b/d88/d99/df1/ff5 [0,4194304] 0 2024-05-12T04:01:40.571 INFO:tasks.workunit.client.0.smithi032.stdout:0/929: truncate df/d30/f8b 2300842 0 2024-05-12T04:01:40.572 INFO:tasks.workunit.client.0.smithi032.stdout:0/930: write df/d10/d3d/d4a/d84/f8c [5410296,120793] 0 2024-05-12T04:01:40.574 INFO:tasks.workunit.client.0.smithi032.stdout:9/885: creat d0/d3/d74/f114 x:0 0 0 2024-05-12T04:01:40.591 INFO:tasks.workunit.client.0.smithi032.stdout:9/886: dread d0/d3/d4b/d85/fcd [0,4194304] 0 2024-05-12T04:01:40.621 INFO:tasks.workunit.client.0.smithi032.stdout:0/931: dwrite df/d30/d4e/d55/fc4 [4194304,4194304] 0 2024-05-12T04:01:40.626 INFO:tasks.workunit.client.0.smithi032.stdout:0/932: read - df/df2/ffc zero size 2024-05-12T04:01:40.630 INFO:tasks.workunit.client.0.smithi032.stdout:0/933: creat df/d30/d4e/d55/f134 x:0 0 0 2024-05-12T04:01:40.631 INFO:tasks.workunit.client.0.smithi032.stdout:0/934: readlink df/d30/d32/l69 0 2024-05-12T04:01:40.644 INFO:tasks.workunit.client.0.smithi032.stdout:4/872: sync 2024-05-12T04:01:40.692 INFO:tasks.workunit.client.0.smithi032.stdout:4/873: readlink d0/d3/d31/d80/d84/l46 0 2024-05-12T04:01:40.693 INFO:tasks.workunit.client.0.smithi032.stdout:9/887: dwrite d0/d84/d8a/da3/db7/d10c/d111/f53 [0,4194304] 0 2024-05-12T04:01:40.698 INFO:tasks.workunit.client.0.smithi032.stdout:4/874: fdatasync d0/f65 0 2024-05-12T04:01:40.702 INFO:tasks.workunit.client.0.smithi032.stdout:0/935: truncate fb 3241501 0 2024-05-12T04:01:40.708 INFO:tasks.workunit.client.0.smithi032.stdout:4/875: dwrite d0/d3/d31/d80/d84/dc9/f102 [0,4194304] 0 2024-05-12T04:01:40.712 INFO:tasks.workunit.client.0.smithi032.stdout:9/888: dwrite d0/d3/d6/d7/f7d [0,4194304] 0 2024-05-12T04:01:40.715 INFO:tasks.workunit.client.0.smithi032.stdout:9/889: mknod d0/d3/d6/d7/d27/c115 0 2024-05-12T04:01:40.718 INFO:tasks.workunit.client.0.smithi032.stdout:4/876: write d0/d3/df/f26 [591146,28114] 0 2024-05-12T04:01:40.721 INFO:tasks.workunit.client.0.smithi032.stdout:9/890: fsync d0/d3/d6/d7/fc 0 2024-05-12T04:01:40.725 INFO:tasks.workunit.client.0.smithi032.stdout:9/891: unlink d0/d3/d74/l44 0 2024-05-12T04:01:40.730 INFO:tasks.workunit.client.0.smithi032.stdout:9/892: dread d0/d3/d6/d58/f5e [0,4194304] 0 2024-05-12T04:01:40.731 INFO:tasks.workunit.client.0.smithi032.stdout:4/877: rename d0/d3/d31/d80/d84/l67 to d0/d3/d12/l107 0 2024-05-12T04:01:40.732 INFO:tasks.workunit.client.0.smithi032.stdout:4/878: chown d0/d3/d31/d80/d84/c73 103336 1 2024-05-12T04:01:40.733 INFO:tasks.workunit.client.0.smithi032.stdout:9/893: creat d0/d3/d4b/d6f/dd7/f116 x:0 0 0 2024-05-12T04:01:40.734 INFO:tasks.workunit.client.0.smithi032.stdout:4/879: mkdir d0/d3/d24/dbd/d108 0 2024-05-12T04:01:40.735 INFO:tasks.workunit.client.0.smithi032.stdout:9/894: write d0/d3/d6/d7/d2a/d46/d13/fe6 [1646114,48521] 0 2024-05-12T04:01:40.738 INFO:tasks.workunit.client.0.smithi032.stdout:9/895: mknod d0/d3/d74/d51/c117 0 2024-05-12T04:01:40.802 INFO:tasks.workunit.client.0.smithi032.stdout:7/835: sync 2024-05-12T04:01:40.820 INFO:tasks.workunit.client.0.smithi032.stdout:4/880: write d0/d3/fd [3493186,111663] 0 2024-05-12T04:01:40.821 INFO:tasks.workunit.client.0.smithi032.stdout:7/836: mknod dd/d1b/d1e/de3/d9a/c100 0 2024-05-12T04:01:40.822 INFO:tasks.workunit.client.0.smithi032.stdout:4/881: write d0/d3/d31/d80/fe5 [912569,55999] 0 2024-05-12T04:01:40.824 INFO:tasks.workunit.client.0.smithi032.stdout:9/896: write d0/d3/d6/d7/d2a/d46/d13/f7c [1145523,21511] 0 2024-05-12T04:01:40.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.826+0000 7fae4472e700 1 -- 172.21.15.32:0/1218846095 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fae3c10e5c0 con 0x7fae3c0fbcb0 2024-05-12T04:01:40.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.827+0000 7fae32ffd700 1 -- 172.21.15.32:0/1218846095 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+850 (secure 0 0 0) 0x7fae38004490 con 0x7fae3c0fbcb0 2024-05-12T04:01:40.828 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-05-12T04:01:40.828 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": { 2024-05-12T04:01:40.829 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 2 2024-05-12T04:01:40.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:01:40.829 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": { 2024-05-12T04:01:40.829 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 1, 2024-05-12T04:01:40.829 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 1 2024-05-12T04:01:40.829 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:01:40.830 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": { 2024-05-12T04:01:40.830 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 6 2024-05-12T04:01:40.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:01:40.830 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": { 2024-05-12T04:01:40.830 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 4 2024-05-12T04:01:40.830 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:01:40.830 INFO:teuthology.orchestra.run.smithi032.stdout: "overall": { 2024-05-12T04:01:40.830 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 13, 2024-05-12T04:01:40.830 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 1 2024-05-12T04:01:40.830 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-05-12T04:01:40.830 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-05-12T04:01:40.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.830+0000 7fae4472e700 1 -- 172.21.15.32:0/1218846095 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7fae2806aec0 msgr2=0x7fae2806d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:01:40.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.830+0000 7fae4472e700 1 --2- 172.21.15.32:0/1218846095 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7fae2806aec0 0x7fae2806d370 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fae2c009b80 tx=0x7fae2c00b040 comp rx=0 tx=0).stop 2024-05-12T04:01:40.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.830+0000 7fae4472e700 1 -- 172.21.15.32:0/1218846095 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae3c0fbcb0 msgr2=0x7fae3c10c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:01:40.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.830+0000 7fae4472e700 1 --2- 172.21.15.32:0/1218846095 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae3c0fbcb0 0x7fae3c10c750 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fae380137b0 tx=0x7fae38009e30 comp rx=0 tx=0).stop 2024-05-12T04:01:40.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.830+0000 7fae4472e700 1 -- 172.21.15.32:0/1218846095 shutdown_connections 2024-05-12T04:01:40.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.830+0000 7fae4472e700 1 --2- 172.21.15.32:0/1218846095 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7fae2806aec0 0x7fae2806d370 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:40.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.830+0000 7fae4472e700 1 --2- 172.21.15.32:0/1218846095 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae3c0fbcb0 0x7fae3c10c750 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:40.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.830+0000 7fae4472e700 1 --2- 172.21.15.32:0/1218846095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae3c0fcf60 0x7fae3c10cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:40.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.830+0000 7fae4472e700 1 -- 172.21.15.32:0/1218846095 >> 172.21.15.32:0/1218846095 conn(0x7fae3c0f7430 msgr2=0x7fae3c0f8b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:01:40.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.830+0000 7fae4472e700 1 -- 172.21.15.32:0/1218846095 shutdown_connections 2024-05-12T04:01:40.832 INFO:tasks.workunit.client.0.smithi032.stdout:9/897: unlink d0/d84/d8a/da3/db7/d10c/d111/f1a 0 2024-05-12T04:01:40.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.831+0000 7fae4472e700 1 -- 172.21.15.32:0/1218846095 wait complete. 2024-05-12T04:01:40.834 INFO:tasks.workunit.client.0.smithi032.stdout:7/837: dread dd/d1b/d43/d5e/d37/f46 [0,4194304] 0 2024-05-12T04:01:40.838 INFO:tasks.workunit.client.0.smithi032.stdout:9/898: getdents d0/d3/d6/d7/d27 0 2024-05-12T04:01:40.839 INFO:tasks.workunit.client.0.smithi032.stdout:9/899: chown d0/d84/d8a/da3/db7/d10c/d111/f10f 2065907 1 2024-05-12T04:01:40.844 INFO:tasks.workunit.client.0.smithi032.stdout:9/900: dread d0/d84/da6/f10e [0,4194304] 0 2024-05-12T04:01:40.844 INFO:tasks.workunit.client.0.smithi032.stdout:7/838: truncate fb 537728 0 2024-05-12T04:01:40.848 INFO:tasks.workunit.client.0.smithi032.stdout:9/901: mknod d0/d3/d4b/d88/dc4/ddb/c118 0 2024-05-12T04:01:40.852 INFO:tasks.workunit.client.0.smithi032.stdout:9/902: mkdir d0/d3/d4b/d88/d119 0 2024-05-12T04:01:40.853 INFO:tasks.workunit.client.0.smithi032.stdout:7/839: link dd/d1b/cbb dd/d6c/df8/c101 0 2024-05-12T04:01:40.854 INFO:tasks.workunit.client.0.smithi032.stdout:9/903: fsync d0/d3/d4b/fcc 0 2024-05-12T04:01:40.856 INFO:tasks.workunit.client.0.smithi032.stdout:9/904: fsync d0/d3/d4b/d88/dc4/fdc 0 2024-05-12T04:01:40.860 INFO:tasks.workunit.client.0.smithi032.stdout:9/905: fdatasync d0/d3/d4b/fcc 0 2024-05-12T04:01:40.862 INFO:tasks.workunit.client.0.smithi032.stdout:7/840: creat dd/f102 x:0 0 0 2024-05-12T04:01:40.863 INFO:tasks.workunit.client.0.smithi032.stdout:7/841: readlink dd/d1b/d43/d74/l8e 0 2024-05-12T04:01:40.867 INFO:tasks.workunit.client.0.smithi032.stdout:7/842: mknod dd/d6c/c103 0 2024-05-12T04:01:40.871 INFO:tasks.workunit.client.0.smithi032.stdout:7/843: rmdir dd/d1b/d1e/d45 39 2024-05-12T04:01:40.878 INFO:tasks.workunit.client.0.smithi032.stdout:7/844: creat dd/d1b/d1e/d45/d55/d72/da0/dbd/f104 x:0 0 0 2024-05-12T04:01:40.882 INFO:tasks.workunit.client.0.smithi032.stdout:7/845: creat dd/d6c/f105 x:0 0 0 2024-05-12T04:01:40.884 INFO:tasks.workunit.client.0.smithi032.stdout:9/906: dwrite d0/d3/d6/f62 [0,4194304] 0 2024-05-12T04:01:40.886 INFO:tasks.workunit.client.0.smithi032.stdout:7/846: dread - dd/d6c/dc3/dcd/fe2 zero size 2024-05-12T04:01:40.894 INFO:tasks.workunit.client.0.smithi032.stdout:7/847: mkdir dd/d1b/d106 0 2024-05-12T04:01:40.909 INFO:tasks.workunit.client.0.smithi032.stdout:9/907: dwrite d0/f75 [0,4194304] 0 2024-05-12T04:01:40.910 INFO:tasks.workunit.client.0.smithi032.stdout:7/848: write dd/d1b/d1e/de3/d9a/d9c/db9/d75/fa3 [208237,15199] 0 2024-05-12T04:01:40.917 INFO:tasks.workunit.client.0.smithi032.stdout:9/908: chown d0/d3/d6/d58/f9a 240095780 1 2024-05-12T04:01:40.919 INFO:tasks.workunit.client.0.smithi032.stdout:9/909: truncate d0/d3/d74/f6c 2628985 0 2024-05-12T04:01:40.920 INFO:tasks.workunit.client.0.smithi032.stdout:7/849: stat dd/d1b/d1e/d45/d55/fa5 0 2024-05-12T04:01:40.931 INFO:tasks.workunit.client.0.smithi032.stdout:9/910: truncate d0/d3/d4b/d88/f96 3079263 0 2024-05-12T04:01:40.934 INFO:tasks.workunit.client.0.smithi032.stdout:9/911: mkdir d0/d3/d6/d7/d27/d11a 0 2024-05-12T04:01:40.939 INFO:tasks.workunit.client.0.smithi032.stdout:7/850: dread dd/d1b/d1e/d45/d64/f8b [0,4194304] 0 2024-05-12T04:01:40.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.966+0000 7f4847eb6700 1 -- 172.21.15.32:0/2558807333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48400fff40 msgr2=0x7f4840102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:01:40.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.966+0000 7f4847eb6700 1 --2- 172.21.15.32:0/2558807333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48400fff40 0x7f4840102320 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4830004940 tx=0x7f4830009e30 comp rx=0 tx=0).stop 2024-05-12T04:01:40.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.966+0000 7f4847eb6700 1 -- 172.21.15.32:0/2558807333 shutdown_connections 2024-05-12T04:01:40.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.966+0000 7f4847eb6700 1 --2- 172.21.15.32:0/2558807333 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48400fff40 0x7f4840102320 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:40.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.966+0000 7f4847eb6700 1 --2- 172.21.15.32:0/2558807333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4840066260 0x7f48400ffa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:40.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.966+0000 7f4847eb6700 1 -- 172.21.15.32:0/2558807333 >> 172.21.15.32:0/2558807333 conn(0x7f48400f7430 msgr2=0x7f48400f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:01:40.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.966+0000 7f4847eb6700 1 -- 172.21.15.32:0/2558807333 shutdown_connections 2024-05-12T04:01:40.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.966+0000 7f4847eb6700 1 -- 172.21.15.32:0/2558807333 wait complete. 2024-05-12T04:01:40.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.967+0000 7f4847eb6700 1 Processor -- start 2024-05-12T04:01:40.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.967+0000 7f4847eb6700 1 -- start start 2024-05-12T04:01:40.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.967+0000 7f4847eb6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4840066260 0x7f484010d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:01:40.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.967+0000 7f4847eb6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48400fff40 0x7f484010de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:01:40.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.967+0000 7f4847eb6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f484010e400 con 0x7f4840066260 2024-05-12T04:01:40.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.967+0000 7f4847eb6700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f484010ec50 con 0x7f48400fff40 2024-05-12T04:01:40.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.967+0000 7f4845c52700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4840066260 0x7f484010d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:01:40.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.967+0000 7f4845c52700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4840066260 0x7f484010d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38560/0 (socket says 172.21.15.32:38560) 2024-05-12T04:01:40.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.967+0000 7f4845c52700 1 -- 172.21.15.32:0/3317299972 learned_addr learned my addr 172.21.15.32:0/3317299972 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:01:40.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.967+0000 7f4845c52700 1 -- 172.21.15.32:0/3317299972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48400fff40 msgr2=0x7f484010de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:01:40.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.967+0000 7f4845451700 1 --2- 172.21.15.32:0/3317299972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48400fff40 0x7f484010de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:01:40.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.967+0000 7f4845c52700 1 --2- 172.21.15.32:0/3317299972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48400fff40 0x7f484010de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:40.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.968+0000 7f4845c52700 1 -- 172.21.15.32:0/3317299972 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4830005040 con 0x7f4840066260 2024-05-12T04:01:40.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.968+0000 7f4845451700 1 --2- 172.21.15.32:0/3317299972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48400fff40 0x7f484010de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-12T04:01:40.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.968+0000 7f4845c52700 1 --2- 172.21.15.32:0/3317299972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4840066260 0x7f484010d8c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f483c00eb70 tx=0x7f483c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:01:40.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.968+0000 7f48367fc700 1 -- 172.21.15.32:0/3317299972 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f483c007cc0 con 0x7f4840066260 2024-05-12T04:01:40.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.968+0000 7f48367fc700 1 -- 172.21.15.32:0/3317299972 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f483c00f040 con 0x7f4840066260 2024-05-12T04:01:40.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.968+0000 7f4847eb6700 1 -- 172.21.15.32:0/3317299972 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f484010eed0 con 0x7f4840066260 2024-05-12T04:01:40.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.968+0000 7f48367fc700 1 -- 172.21.15.32:0/3317299972 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f483c00cad0 con 0x7f4840066260 2024-05-12T04:01:40.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.968+0000 7f4847eb6700 1 -- 172.21.15.32:0/3317299972 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f484010f410 con 0x7f4840066260 2024-05-12T04:01:40.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.969+0000 7f4847eb6700 1 -- 172.21.15.32:0/3317299972 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f484005f5c0 con 0x7f4840066260 2024-05-12T04:01:40.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.969+0000 7f48367fc700 1 -- 172.21.15.32:0/3317299972 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 25) v1 ==== 88772+0+0 (secure 0 0 0) 0x7f483c00cc30 con 0x7f4840066260 2024-05-12T04:01:40.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.970+0000 7f48367fc700 1 --2- 172.21.15.32:0/3317299972 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f482c06aec0 0x7f482c06d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:01:40.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.970+0000 7f48367fc700 1 -- 172.21.15.32:0/3317299972 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f483c0893d0 con 0x7f4840066260 2024-05-12T04:01:40.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.970+0000 7f4845451700 1 --2- 172.21.15.32:0/3317299972 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f482c06aec0 0x7f482c06d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:01:40.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.970+0000 7f4845451700 1 --2- 172.21.15.32:0/3317299972 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f482c06aec0 0x7f482c06d370 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f48300137b0 tx=0x7f4830009e30 comp rx=0 tx=0).ready entity=mgr.24389 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:01:40.982 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:40.974+0000 7f48367fc700 1 -- 172.21.15.32:0/3317299972 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181777 (secure 0 0 0) 0x7f483c04f630 con 0x7f4840066260 2024-05-12T04:01:40.982 INFO:tasks.workunit.client.0.smithi032.stdout:8/961: sync 2024-05-12T04:01:40.991 INFO:tasks.workunit.client.0.smithi032.stdout:8/962: write d0/d9/d13/d21/d25/f50 [3173440,71445] 0 2024-05-12T04:01:40.995 INFO:tasks.workunit.client.0.smithi032.stdout:8/963: mkdir d0/d9/d1b/d66/d75/de2/d103/d108/d12c/d144 0 2024-05-12T04:01:40.998 INFO:tasks.workunit.client.0.smithi032.stdout:8/964: symlink d0/d9/d1b/d66/d75/de2/d103/d108/d12c/d144/l145 0 2024-05-12T04:01:41.000 INFO:tasks.workunit.client.0.smithi032.stdout:8/965: fdatasync d0/d7/f6c 0 2024-05-12T04:01:41.008 INFO:tasks.workunit.client.0.smithi032.stdout:8/966: link d0/d7/f11 d0/d9/d1b/d27/d88/d56/d63/d8f/d9a/d28/f146 0 2024-05-12T04:01:41.011 INFO:tasks.workunit.client.0.smithi032.stdout:8/967: read d0/f68 [3142175,130792] 0 2024-05-12T04:01:41.016 INFO:tasks.workunit.client.0.smithi032.stdout:8/968: rename d0/d9/d1b/d27/d88/dd7/ff4 to d0/d9/d1b/d27/d88/d56/f147 0 2024-05-12T04:01:41.017 INFO:tasks.workunit.client.0.smithi032.stdout:8/969: read - d0/d7/f12d zero size 2024-05-12T04:01:41.019 INFO:tasks.workunit.client.0.smithi032.stdout:8/970: truncate d0/f2 4400836 0 2024-05-12T04:01:41.024 INFO:tasks.workunit.client.0.smithi032.stdout:8/971: creat d0/d9/d1b/d133/f148 x:0 0 0 2024-05-12T04:01:41.027 INFO:tasks.workunit.client.0.smithi032.stdout:8/972: dread d0/d9/d1b/d33/f3a [4194304,4194304] 0 2024-05-12T04:01:41.028 INFO:tasks.workunit.client.0.smithi032.stdout:8/973: fdatasync d0/d9/d1b/d27/d88/d56/d63/d8f/d9a/d28/f49 0 2024-05-12T04:01:41.031 INFO:tasks.workunit.client.0.smithi032.stdout:8/974: mkdir d0/d149 0 2024-05-12T04:01:41.035 INFO:tasks.workunit.client.0.smithi032.stdout:8/975: creat d0/d9/d13/d21/d35/f14a x:0 0 0 2024-05-12T04:01:41.040 INFO:tasks.workunit.client.0.smithi032.stdout:8/976: creat d0/d9/d1b/d27/d88/dd7/ddf/df3/f14b x:0 0 0 2024-05-12T04:01:41.045 INFO:tasks.workunit.client.0.smithi032.stdout:8/977: creat d0/d9/f14c x:0 0 0 2024-05-12T04:01:41.047 INFO:tasks.workunit.client.0.smithi032.stdout:8/978: truncate d0/d9/d13/d21/f13f 910558 0 2024-05-12T04:01:41.050 INFO:tasks.workunit.client.0.smithi032.stdout:8/979: mknod d0/d9/d1b/d66/d75/de2/d103/d108/d12c/d144/c14d 0 2024-05-12T04:01:41.054 INFO:tasks.workunit.client.0.smithi032.stdout:8/980: mknod d0/d9/d1b/d27/d4c/d4e/d90/c14e 0 2024-05-12T04:01:41.126 INFO:tasks.workunit.client.0.smithi032.stdout:2/903: sync 2024-05-12T04:01:41.126 INFO:tasks.workunit.client.0.smithi032.stdout:0/936: sync 2024-05-12T04:01:41.126 INFO:tasks.workunit.client.0.smithi032.stdout:1/954: sync 2024-05-12T04:01:41.127 INFO:tasks.workunit.client.0.smithi032.stdout:1/955: chown d1/d7/d30/d35/f48 1534 1 2024-05-12T04:01:41.133 INFO:tasks.workunit.client.0.smithi032.stdout:1/956: truncate d1/dd/d24/f9c 2574609 0 2024-05-12T04:01:41.134 INFO:tasks.workunit.client.0.smithi032.stdout:2/904: dread d1/d34/d3e/f7e [4194304,4194304] 0 2024-05-12T04:01:41.134 INFO:tasks.workunit.client.0.smithi032.stdout:1/957: fsync d1/d7/d30/d35/d77/d110/d11d/d11f/f137 0 2024-05-12T04:01:41.140 INFO:tasks.workunit.client.0.smithi032.stdout:1/958: chown d1/d7/d30/d35/fa3 13 1 2024-05-12T04:01:41.140 INFO:tasks.workunit.client.0.smithi032.stdout:0/937: dwrite df/d30/d8d/d8f/fcc [0,4194304] 0 2024-05-12T04:01:41.141 INFO:tasks.workunit.client.0.smithi032.stdout:0/938: chown df/f92 0 1 2024-05-12T04:01:41.142 INFO:tasks.workunit.client.0.smithi032.stdout:0/939: write df/d22/f39 [2072352,130111] 0 2024-05-12T04:01:41.154 INFO:tasks.workunit.client.0.smithi032.stdout:0/940: symlink df/d30/d8d/d8f/l135 0 2024-05-12T04:01:41.155 INFO:tasks.workunit.client.0.smithi032.stdout:2/905: dwrite d1/d13/d70/d85/f88 [0,4194304] 0 2024-05-12T04:01:41.156 INFO:tasks.workunit.client.0.smithi032.stdout:0/941: readlink df/d28/lcd 0 2024-05-12T04:01:41.158 INFO:tasks.workunit.client.0.smithi032.stdout:1/959: symlink d1/d6a/def/l140 0 2024-05-12T04:01:41.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:40 smithi033 ceph-mon[49525]: from='client.24479 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:01:41.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:40 smithi033 ceph-mon[49525]: pgmap v48: 65 pgs: 65 active+clean; 3.9 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 69 MiB/s rd, 161 MiB/s wr, 367 op/s 2024-05-12T04:01:41.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:40 smithi033 ceph-mon[49525]: from='client.24483 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:01:41.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:40 smithi033 ceph-mon[49525]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:01:41.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:40 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/1218846095' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:01:41.163 INFO:tasks.workunit.client.0.smithi032.stdout:2/906: dwrite d1/d34/d95/fa1 [4194304,4194304] 0 2024-05-12T04:01:41.169 INFO:tasks.workunit.client.0.smithi032.stdout:0/942: mknod df/d10/d3d/d4a/c136 0 2024-05-12T04:01:41.174 INFO:tasks.workunit.client.0.smithi032.stdout:2/907: unlink d1/d13/f18 0 2024-05-12T04:01:41.174 INFO:tasks.workunit.client.0.smithi032.stdout:0/943: truncate df/d10/d48/d70/ff3 638328 0 2024-05-12T04:01:41.188 INFO:tasks.workunit.client.0.smithi032.stdout:0/944: dwrite df/d30/d32/d81/de8/f110 [4194304,4194304] 0 2024-05-12T04:01:41.190 INFO:tasks.workunit.client.0.smithi032.stdout:1/960: dwrite d1/d7/d30/d18/d87/db7/fc5 [0,4194304] 0 2024-05-12T04:01:41.201 INFO:tasks.workunit.client.0.smithi032.stdout:2/908: dwrite d1/d34/da5/fbd [0,4194304] 0 2024-05-12T04:01:41.203 INFO:tasks.workunit.client.0.smithi032.stdout:1/961: mknod d1/d7/d49/c141 0 2024-05-12T04:01:41.204 INFO:tasks.workunit.client.0.smithi032.stdout:1/962: stat d1/d7/d113/f123 0 2024-05-12T04:01:41.208 INFO:tasks.workunit.client.0.smithi032.stdout:0/945: write df/d10/d3b/dba/f10b [122414,15619] 0 2024-05-12T04:01:41.210 INFO:tasks.workunit.client.0.smithi032.stdout:0/946: chown df/d30/d32/d81/de8/f110 363 1 2024-05-12T04:01:41.211 INFO:tasks.workunit.client.0.smithi032.stdout:1/963: mkdir d1/d25/da5/d142 0 2024-05-12T04:01:41.217 INFO:tasks.workunit.client.0.smithi032.stdout:0/947: link df/d10/d48/d70/ffb df/d30/f137 0 2024-05-12T04:01:41.220 INFO:tasks.workunit.client.0.smithi032.stdout:4/882: sync 2024-05-12T04:01:41.222 INFO:tasks.workunit.client.0.smithi032.stdout:0/948: unlink df/d22/d58/lfd 0 2024-05-12T04:01:41.223 INFO:tasks.workunit.client.0.smithi032.stdout:4/883: write d0/d3/d12/d1d/d63/d77/f103 [141401,50031] 0 2024-05-12T04:01:41.228 INFO:tasks.workunit.client.0.smithi032.stdout:2/909: chown d1/d34/d96/d41/d2c/dd4 21 1 2024-05-12T04:01:41.229 INFO:tasks.workunit.client.0.smithi032.stdout:4/884: dread d0/d3/d24/f72 [0,4194304] 0 2024-05-12T04:01:41.230 INFO:tasks.workunit.client.0.smithi032.stdout:4/885: stat d0/d3/d12/d1d/d63/d77/cf0 0 2024-05-12T04:01:41.238 INFO:tasks.workunit.client.0.smithi032.stdout:1/964: dread d1/d7/d30/d35/fa3 [0,4194304] 0 2024-05-12T04:01:41.239 INFO:tasks.workunit.client.0.smithi032.stdout:4/886: readlink d0/d3/d12/d1d/d63/l96 0 2024-05-12T04:01:41.239 INFO:tasks.workunit.client.0.smithi032.stdout:0/949: dwrite df/d30/dc2/ff0 [4194304,4194304] 0 2024-05-12T04:01:41.241 INFO:tasks.workunit.client.0.smithi032.stdout:1/965: read d1/d25/f4d [2096393,83837] 0 2024-05-12T04:01:41.242 INFO:tasks.workunit.client.0.smithi032.stdout:1/966: chown d1/d7/d30/d35/d54/d7e/la0 825674041 1 2024-05-12T04:01:41.250 INFO:tasks.workunit.client.0.smithi032.stdout:2/910: mknod d1/d13/da0/dae/d120/c12b 0 2024-05-12T04:01:41.258 INFO:tasks.workunit.client.0.smithi032.stdout:4/887: creat d0/d3/d12/d8d/f109 x:0 0 0 2024-05-12T04:01:41.262 INFO:tasks.workunit.client.0.smithi032.stdout:0/950: dread df/d10/d3d/d4a/f2d [0,4194304] 0 2024-05-12T04:01:41.262 INFO:tasks.workunit.client.0.smithi032.stdout:2/911: mkdir d1/d13/d70/d85/df3/d12c 0 2024-05-12T04:01:41.263 INFO:tasks.workunit.client.0.smithi032.stdout:0/951: chown df/d10/d3d/d4a/c136 316760430 1 2024-05-12T04:01:41.264 INFO:tasks.workunit.client.0.smithi032.stdout:4/888: mknod d0/d3/d31/d80/d84/dc9/c10a 0 2024-05-12T04:01:41.268 INFO:tasks.workunit.client.0.smithi032.stdout:2/912: truncate d1/d34/d96/d41/d2c/f56 1388993 0 2024-05-12T04:01:41.272 INFO:tasks.workunit.client.0.smithi032.stdout:1/967: write d1/d7/d30/f28 [1886339,15182] 0 2024-05-12T04:01:41.273 INFO:tasks.workunit.client.0.smithi032.stdout:0/952: write df/d10/f80 [1341736,15309] 0 2024-05-12T04:01:41.275 INFO:tasks.workunit.client.0.smithi032.stdout:2/913: creat d1/d34/da5/f12d x:0 0 0 2024-05-12T04:01:41.276 INFO:tasks.workunit.client.0.smithi032.stdout:2/914: chown d1/d34/d4a/f10b 80424 1 2024-05-12T04:01:41.276 INFO:tasks.workunit.client.0.smithi032.stdout:4/889: fsync d0/d3/d12/d1d/d6b/db7/fff 0 2024-05-12T04:01:41.278 INFO:tasks.workunit.client.0.smithi032.stdout:1/968: mkdir d1/d7/d30/d61/d103/d143 0 2024-05-12T04:01:41.280 INFO:tasks.workunit.client.0.smithi032.stdout:2/915: chown d1/d34/d96/d41/f40 201061281 1 2024-05-12T04:01:41.284 INFO:tasks.workunit.client.0.smithi032.stdout:1/969: truncate d1/d7/f78 622675 0 2024-05-12T04:01:41.284 INFO:tasks.workunit.client.0.smithi032.stdout:0/953: dwrite df/d28/f125 [0,4194304] 0 2024-05-12T04:01:41.289 INFO:tasks.workunit.client.0.smithi032.stdout:4/890: creat d0/d3/df/f10b x:0 0 0 2024-05-12T04:01:41.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:40 smithi032 ceph-mon[40008]: from='client.24479 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:01:41.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:40 smithi032 ceph-mon[40008]: pgmap v48: 65 pgs: 65 active+clean; 3.9 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 69 MiB/s rd, 161 MiB/s wr, 367 op/s 2024-05-12T04:01:41.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:40 smithi032 ceph-mon[40008]: from='client.24483 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:01:41.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:40 smithi032 ceph-mon[40008]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:01:41.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:40 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/1218846095' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:01:41.293 INFO:tasks.workunit.client.0.smithi032.stdout:1/970: mkdir d1/d7/d30/d18/d93/dc1/dc3/d144 0 2024-05-12T04:01:41.294 INFO:tasks.workunit.client.0.smithi032.stdout:1/971: read d1/d7/fba [125794,48784] 0 2024-05-12T04:01:41.294 INFO:tasks.workunit.client.0.smithi032.stdout:2/916: truncate d1/d34/d96/d8a/fbb 721402 0 2024-05-12T04:01:41.299 INFO:tasks.workunit.client.0.smithi032.stdout:2/917: truncate d1/d34/d3e/f7e 3658889 0 2024-05-12T04:01:41.307 INFO:tasks.workunit.client.0.smithi032.stdout:0/954: dread df/d10/d3d/d4a/d9a/ffe [0,4194304] 0 2024-05-12T04:01:41.311 INFO:tasks.workunit.client.0.smithi032.stdout:2/918: rename d1/d13/d70/f91 to d1/d4d/d5e/dff/f12e 0 2024-05-12T04:01:41.311 INFO:tasks.workunit.client.0.smithi032.stdout:4/891: dwrite d0/d3/d12/d1d/d63/d87/ffa [0,4194304] 0 2024-05-12T04:01:41.312 INFO:tasks.workunit.client.0.smithi032.stdout:2/919: stat d1/d4d/ffe 0 2024-05-12T04:01:41.314 INFO:tasks.workunit.client.0.smithi032.stdout:2/920: write d1/d13/d6b/f113 [4121124,12756] 0 2024-05-12T04:01:41.314 INFO:tasks.workunit.client.0.smithi032.stdout:4/892: chown d0/d3/d12/d1d/d63/d77/cb0 0 1 2024-05-12T04:01:41.315 INFO:tasks.workunit.client.0.smithi032.stdout:2/921: fdatasync d1/d34/da5/fbd 0 2024-05-12T04:01:41.317 INFO:tasks.workunit.client.0.smithi032.stdout:1/972: link d1/d7/d30/d35/d54/f8c d1/d7/d30/d18/d87/dc6/f145 0 2024-05-12T04:01:41.320 INFO:tasks.workunit.client.0.smithi032.stdout:9/912: sync 2024-05-12T04:01:41.320 INFO:tasks.workunit.client.0.smithi032.stdout:8/981: sync 2024-05-12T04:01:41.320 INFO:tasks.workunit.client.0.smithi032.stdout:7/851: sync 2024-05-12T04:01:41.324 INFO:tasks.workunit.client.0.smithi032.stdout:7/852: fdatasync dd/d1b/d1e/de3/d9a/d9c/db9/d75/f86 0 2024-05-12T04:01:41.334 INFO:tasks.workunit.client.0.smithi032.stdout:2/922: dread d1/d34/da5/db3/db7/fcf [0,4194304] 0 2024-05-12T04:01:41.338 INFO:tasks.workunit.client.0.smithi032.stdout:9/913: truncate d0/d84/d8a/f94 2848851 0 2024-05-12T04:01:41.344 INFO:tasks.workunit.client.0.smithi032.stdout:0/955: dwrite df/d30/d4d/ff5 [0,4194304] 0 2024-05-12T04:01:41.351 INFO:tasks.workunit.client.0.smithi032.stdout:1/973: truncate d1/d7/d30/d35/d77/d110/d11d/fd3 419314 0 2024-05-12T04:01:41.352 INFO:tasks.workunit.client.0.smithi032.stdout:8/982: dwrite d0/d9/d1b/d27/d59/f6d [8388608,4194304] 0 2024-05-12T04:01:41.355 INFO:tasks.workunit.client.0.smithi032.stdout:1/974: write d1/d7/d30/d18/d87/db7/fc5 [1042661,66909] 0 2024-05-12T04:01:41.355 INFO:tasks.workunit.client.0.smithi032.stdout:9/914: chown d0/d3/d6/d7/la5 51979977 1 2024-05-12T04:01:41.363 INFO:tasks.workunit.client.0.smithi032.stdout:4/893: getdents d0/d3/d12 0 2024-05-12T04:01:41.363 INFO:tasks.workunit.client.0.smithi032.stdout:7/853: creat dd/d1b/d1e/de3/f107 x:0 0 0 2024-05-12T04:01:41.364 INFO:tasks.workunit.client.0.smithi032.stdout:4/894: chown d0/d3/d12/d1d/d6b/dc2 5 1 2024-05-12T04:01:41.369 INFO:tasks.workunit.client.0.smithi032.stdout:2/923: link d1/d34/d4a/lc6 d1/d13/d70/l12f 0 2024-05-12T04:01:41.373 INFO:tasks.workunit.client.0.smithi032.stdout:2/924: readlink d1/d13/d70/l12f 0 2024-05-12T04:01:41.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.373+0000 7f4847eb6700 1 -- 172.21.15.32:0/3317299972 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f484004bed0 con 0x7f4840066260 2024-05-12T04:01:41.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.373+0000 7f48367fc700 1 -- 172.21.15.32:0/3317299972 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1828 (secure 0 0 0) 0x7f483c0572e0 con 0x7f4840066260 2024-05-12T04:01:41.375 INFO:teuthology.orchestra.run.smithi032.stdout:e18 2024-05-12T04:01:41.375 INFO:teuthology.orchestra.run.smithi032.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-12T04:01:41.375 INFO:teuthology.orchestra.run.smithi032.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-12T04:01:41.375 INFO:teuthology.orchestra.run.smithi032.stdout:legacy client fscid: 1 2024-05-12T04:01:41.375 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:01:41.375 INFO:teuthology.orchestra.run.smithi032.stdout:Filesystem 'cephfs' (1) 2024-05-12T04:01:41.375 INFO:teuthology.orchestra.run.smithi032.stdout:fs_name cephfs 2024-05-12T04:01:41.375 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 18 2024-05-12T04:01:41.375 INFO:teuthology.orchestra.run.smithi032.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-12T04:01:41.376 INFO:teuthology.orchestra.run.smithi032.stdout:created 2024-05-12T03:57:48.904614+0000 2024-05-12T04:01:41.376 INFO:teuthology.orchestra.run.smithi032.stdout:modified 2024-05-12T03:58:06.821591+0000 2024-05-12T04:01:41.376 INFO:teuthology.orchestra.run.smithi032.stdout:tableserver 0 2024-05-12T04:01:41.376 INFO:teuthology.orchestra.run.smithi032.stdout:root 0 2024-05-12T04:01:41.376 INFO:teuthology.orchestra.run.smithi032.stdout:session_timeout 60 2024-05-12T04:01:41.376 INFO:teuthology.orchestra.run.smithi032.stdout:session_autoclose 300 2024-05-12T04:01:41.376 INFO:teuthology.orchestra.run.smithi032.stdout:max_file_size 1099511627776 2024-05-12T04:01:41.376 INFO:teuthology.orchestra.run.smithi032.stdout:required_client_features {} 2024-05-12T04:01:41.376 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure 0 2024-05-12T04:01:41.376 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure_osd_epoch 46 2024-05-12T04:01:41.376 INFO:teuthology.orchestra.run.smithi032.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-12T04:01:41.376 INFO:teuthology.orchestra.run.smithi032.stdout:max_mds 1 2024-05-12T04:01:41.376 INFO:teuthology.orchestra.run.smithi032.stdout:in 0 2024-05-12T04:01:41.376 INFO:teuthology.orchestra.run.smithi032.stdout:up {0=14474} 2024-05-12T04:01:41.376 INFO:teuthology.orchestra.run.smithi032.stdout:failed 2024-05-12T04:01:41.377 INFO:teuthology.orchestra.run.smithi032.stdout:damaged 2024-05-12T04:01:41.377 INFO:teuthology.orchestra.run.smithi032.stdout:stopped 2024-05-12T04:01:41.377 INFO:teuthology.orchestra.run.smithi032.stdout:data_pools [3] 2024-05-12T04:01:41.377 INFO:teuthology.orchestra.run.smithi032.stdout:metadata_pool 2 2024-05-12T04:01:41.377 INFO:teuthology.orchestra.run.smithi032.stdout:inline_data enabled 2024-05-12T04:01:41.377 INFO:teuthology.orchestra.run.smithi032.stdout:balancer 2024-05-12T04:01:41.377 INFO:teuthology.orchestra.run.smithi032.stdout:standby_count_wanted 1 2024-05-12T04:01:41.377 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.qpsawh{0:14474} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.32:6828/3100155296,v1:172.21.15.32:6829/3100155296] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:01:41.377 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi033.jgbjhk{0:24267} state up:standby-replay seq 3 join_fscid=1 addr [v2:172.21.15.33:6824/55417471,v1:172.21.15.33:6825/55417471] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:01:41.377 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:01:41.377 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:01:41.377 INFO:teuthology.orchestra.run.smithi032.stdout:Standby daemons: 2024-05-12T04:01:41.377 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:01:41.377 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi033.hglwml{-1:24283} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/433858238,v1:172.21.15.33:6827/433858238] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:01:41.378 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.ialtqb{-1:24287} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.32:6826/3703132329,v1:172.21.15.32:6827/3703132329] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:01:41.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.376+0000 7f4847eb6700 1 -- 172.21.15.32:0/3317299972 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f482c06aec0 msgr2=0x7f482c06d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:01:41.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.376+0000 7f4847eb6700 1 --2- 172.21.15.32:0/3317299972 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f482c06aec0 0x7f482c06d370 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f48300137b0 tx=0x7f4830009e30 comp rx=0 tx=0).stop 2024-05-12T04:01:41.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.376+0000 7f4847eb6700 1 -- 172.21.15.32:0/3317299972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4840066260 msgr2=0x7f484010d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:01:41.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.376+0000 7f4847eb6700 1 --2- 172.21.15.32:0/3317299972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4840066260 0x7f484010d8c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f483c00eb70 tx=0x7f483c00b040 comp rx=0 tx=0).stop 2024-05-12T04:01:41.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.376+0000 7f4847eb6700 1 -- 172.21.15.32:0/3317299972 shutdown_connections 2024-05-12T04:01:41.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.376+0000 7f4847eb6700 1 --2- 172.21.15.32:0/3317299972 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f482c06aec0 0x7f482c06d370 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:41.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.376+0000 7f4847eb6700 1 --2- 172.21.15.32:0/3317299972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4840066260 0x7f484010d8c0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:41.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.376+0000 7f4847eb6700 1 --2- 172.21.15.32:0/3317299972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48400fff40 0x7f484010de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:41.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.376+0000 7f4847eb6700 1 -- 172.21.15.32:0/3317299972 >> 172.21.15.32:0/3317299972 conn(0x7f48400f7430 msgr2=0x7f48400f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:01:41.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.376+0000 7f4847eb6700 1 -- 172.21.15.32:0/3317299972 shutdown_connections 2024-05-12T04:01:41.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.376+0000 7f4847eb6700 1 -- 172.21.15.32:0/3317299972 wait complete. 2024-05-12T04:01:41.379 INFO:tasks.workunit.client.0.smithi032.stdout:8/983: link d0/d9/d1b/d27/d59/ffc d0/d9/d13/d21/f14f 0 2024-05-12T04:01:41.379 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 18 2024-05-12T04:01:41.380 INFO:tasks.workunit.client.0.smithi032.stdout:9/915: dread d0/d3/d6/d7/f15 [0,4194304] 0 2024-05-12T04:01:41.383 INFO:tasks.workunit.client.0.smithi032.stdout:8/984: read - d0/d9/d1b/d66/d112/f114 zero size 2024-05-12T04:01:41.384 INFO:tasks.workunit.client.0.smithi032.stdout:1/975: getdents d1/dd 0 2024-05-12T04:01:41.396 INFO:tasks.workunit.client.0.smithi032.stdout:9/916: truncate d0/d3/d74/f78 241984 0 2024-05-12T04:01:41.396 INFO:tasks.workunit.client.0.smithi032.stdout:0/956: write df/d30/d32/d60/dc7/fd7 [838975,12348] 0 2024-05-12T04:01:41.396 INFO:tasks.workunit.client.0.smithi032.stdout:7/854: write dd/d1b/d1e/de3/fbf [1434863,58484] 0 2024-05-12T04:01:41.399 INFO:tasks.workunit.client.0.smithi032.stdout:9/917: fdatasync d0/f75 0 2024-05-12T04:01:41.404 INFO:tasks.workunit.client.0.smithi032.stdout:8/985: write d0/d9/d1b/d27/d88/dd7/fff [386761,5585] 0 2024-05-12T04:01:41.412 INFO:tasks.workunit.client.0.smithi032.stdout:2/925: dwrite d1/d34/d96/d41/f110 [0,4194304] 0 2024-05-12T04:01:41.418 INFO:tasks.workunit.client.0.smithi032.stdout:1/976: dwrite d1/d7/d30/d35/d54/f73 [0,4194304] 0 2024-05-12T04:01:41.419 INFO:tasks.workunit.client.0.smithi032.stdout:7/855: dread dd/d1b/d1e/d45/d55/d72/da0/dbd/db2/fe7 [0,4194304] 0 2024-05-12T04:01:41.420 INFO:tasks.workunit.client.0.smithi032.stdout:1/977: chown d1/d7/d30/d35/fa3 56761 1 2024-05-12T04:01:41.474 INFO:tasks.workunit.client.0.smithi032.stdout:0/957: mkdir df/d30/d32/d60/d138 0 2024-05-12T04:01:41.476 INFO:tasks.workunit.client.0.smithi032.stdout:9/918: truncate d0/d3/d4b/f81 820190 0 2024-05-12T04:01:41.479 INFO:tasks.workunit.client.0.smithi032.stdout:2/926: write d1/d34/d95/f109 [2477655,41614] 0 2024-05-12T04:01:41.491 INFO:tasks.workunit.client.0.smithi032.stdout:8/986: rename d0/d9/f31 to d0/d9/d1b/d33/d73/db7/df1/d110/f150 0 2024-05-12T04:01:41.492 INFO:tasks.workunit.client.0.smithi032.stdout:8/987: write d0/d9/f1c [1523019,95759] 0 2024-05-12T04:01:41.494 INFO:tasks.workunit.client.0.smithi032.stdout:9/919: creat d0/d84/d8a/da3/db7/d10c/d111/d89/f11b x:0 0 0 2024-05-12T04:01:41.495 INFO:tasks.workunit.client.0.smithi032.stdout:9/920: chown d0/d3/d6/d7/d106 297558110 1 2024-05-12T04:01:41.495 INFO:tasks.workunit.client.0.smithi032.stdout:1/978: fsync d1/d6a/d72/f9e 0 2024-05-12T04:01:41.499 INFO:tasks.workunit.client.0.smithi032.stdout:7/856: creat dd/d1b/d1e/d45/d55/db7/df6/f108 x:0 0 0 2024-05-12T04:01:41.501 INFO:tasks.workunit.client.0.smithi032.stdout:0/958: mkdir df/d30/d36/ddd/d139 0 2024-05-12T04:01:41.503 INFO:tasks.workunit.client.0.smithi032.stdout:8/988: fdatasync d0/d9/d1b/fdb 0 2024-05-12T04:01:41.504 INFO:tasks.workunit.client.0.smithi032.stdout:8/989: stat d0/d9/d1b/d27/d4c/d4e/d7f/da3 0 2024-05-12T04:01:41.508 INFO:tasks.workunit.client.0.smithi032.stdout:1/979: symlink d1/d7/d30/d18/d93/dc1/dc3/l146 0 2024-05-12T04:01:41.514 INFO:tasks.workunit.client.0.smithi032.stdout:0/959: mknod df/d30/d36/c13a 0 2024-05-12T04:01:41.514 INFO:tasks.workunit.client.0.smithi032.stdout:0/960: read - df/d28/fc6 zero size 2024-05-12T04:01:41.515 INFO:tasks.workunit.client.0.smithi032.stdout:0/961: chown df/d10/d3d/d4a/c136 303987 1 2024-05-12T04:01:41.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.515+0000 7f30b0331700 1 -- 172.21.15.32:0/2011546895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30a80fbcb0 msgr2=0x7f30a80fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:01:41.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.515+0000 7f30b0331700 1 --2- 172.21.15.32:0/2011546895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30a80fbcb0 0x7f30a80fc0c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f30a4004940 tx=0x7f30a4009e30 comp rx=0 tx=0).stop 2024-05-12T04:01:41.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.516+0000 7f30b0331700 1 -- 172.21.15.32:0/2011546895 shutdown_connections 2024-05-12T04:01:41.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.516+0000 7f30b0331700 1 --2- 172.21.15.32:0/2011546895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30a80fcf60 0x7f30a80fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:41.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.516+0000 7f30b0331700 1 --2- 172.21.15.32:0/2011546895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30a80fbcb0 0x7f30a80fc0c0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:41.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.516+0000 7f30b0331700 1 -- 172.21.15.32:0/2011546895 >> 172.21.15.32:0/2011546895 conn(0x7f30a80f7430 msgr2=0x7f30a80f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:01:41.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.516+0000 7f30b0331700 1 -- 172.21.15.32:0/2011546895 shutdown_connections 2024-05-12T04:01:41.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.516+0000 7f30b0331700 1 -- 172.21.15.32:0/2011546895 wait complete. 2024-05-12T04:01:41.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.516+0000 7f30b0331700 1 Processor -- start 2024-05-12T04:01:41.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.516+0000 7f30b0331700 1 -- start start 2024-05-12T04:01:41.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.517+0000 7f30b0331700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30a80fbcb0 0x7f30a810fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:01:41.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.517+0000 7f30b0331700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30a80fcf60 0x7f30a8110040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:01:41.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.517+0000 7f30b0331700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a8110640 con 0x7f30a80fcf60 2024-05-12T04:01:41.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.517+0000 7f30b0331700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a8110dd0 con 0x7f30a80fbcb0 2024-05-12T04:01:41.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.517+0000 7f30ae0cd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30a80fbcb0 0x7f30a810fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:01:41.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.517+0000 7f30ad8cc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30a80fcf60 0x7f30a8110040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:01:41.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.517+0000 7f30ae0cd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30a80fbcb0 0x7f30a810fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.32:51400/0 (socket says 172.21.15.32:51400) 2024-05-12T04:01:41.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.517+0000 7f30ae0cd700 1 -- 172.21.15.32:0/2423261712 learned_addr learned my addr 172.21.15.32:0/2423261712 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:01:41.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.517+0000 7f30ae0cd700 1 -- 172.21.15.32:0/2423261712 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30a80fcf60 msgr2=0x7f30a8110040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:01:41.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.517+0000 7f30ae0cd700 1 --2- 172.21.15.32:0/2423261712 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30a80fcf60 0x7f30a8110040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:41.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.517+0000 7f30ae0cd700 1 -- 172.21.15.32:0/2423261712 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30a4005040 con 0x7f30a80fbcb0 2024-05-12T04:01:41.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.517+0000 7f30ad8cc700 1 --2- 172.21.15.32:0/2423261712 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30a80fcf60 0x7f30a8110040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-12T04:01:41.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.517+0000 7f30ae0cd700 1 --2- 172.21.15.32:0/2423261712 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30a80fbcb0 0x7f30a810fb00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f30a40137b0 tx=0x7f30a4009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:01:41.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.518+0000 7f309effd700 1 -- 172.21.15.32:0/2423261712 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f30a400e070 con 0x7f30a80fbcb0 2024-05-12T04:01:41.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.518+0000 7f30b0331700 1 -- 172.21.15.32:0/2423261712 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30a8111050 con 0x7f30a80fbcb0 2024-05-12T04:01:41.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.518+0000 7f309effd700 1 -- 172.21.15.32:0/2423261712 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f30a401c400 con 0x7f30a80fbcb0 2024-05-12T04:01:41.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.518+0000 7f309effd700 1 -- 172.21.15.32:0/2423261712 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f30a400b650 con 0x7f30a80fbcb0 2024-05-12T04:01:41.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.518+0000 7f30b0331700 1 -- 172.21.15.32:0/2423261712 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30a8111530 con 0x7f30a80fbcb0 2024-05-12T04:01:41.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.519+0000 7f30b0331700 1 -- 172.21.15.32:0/2423261712 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30a805f5c0 con 0x7f30a80fbcb0 2024-05-12T04:01:41.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.519+0000 7f309effd700 1 -- 172.21.15.32:0/2423261712 <== mon.1 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) v1 ==== 88772+0+0 (secure 0 0 0) 0x7f30a401c9e0 con 0x7f30a80fbcb0 2024-05-12T04:01:41.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.519+0000 7f309effd700 1 --2- 172.21.15.32:0/2423261712 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f309406aec0 0x7f309406d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:01:41.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.520+0000 7f309effd700 1 -- 172.21.15.32:0/2423261712 <== mon.1 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f30a408af60 con 0x7f30a80fbcb0 2024-05-12T04:01:41.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.520+0000 7f30ad8cc700 1 --2- 172.21.15.32:0/2423261712 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f309406aec0 0x7f309406d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:01:41.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.520+0000 7f30ad8cc700 1 --2- 172.21.15.32:0/2423261712 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f309406aec0 0x7f309406d370 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3098009b80 tx=0x7f309800b040 comp rx=0 tx=0).ready entity=mgr.24389 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:01:41.525 INFO:tasks.workunit.client.0.smithi032.stdout:8/990: creat d0/d149/f151 x:0 0 0 2024-05-12T04:01:41.526 INFO:tasks.workunit.client.0.smithi032.stdout:8/991: fsync d0/d9/d1b/d66/fe8 0 2024-05-12T04:01:41.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.525+0000 7f309effd700 1 -- 172.21.15.32:0/2423261712 <== mon.1 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181777 (secure 0 0 0) 0x7f30a4058dc0 con 0x7f30a80fbcb0 2024-05-12T04:01:41.527 INFO:tasks.workunit.client.0.smithi032.stdout:1/980: creat d1/d7/d30/d35/d54/d7e/df9/d13e/f147 x:0 0 0 2024-05-12T04:01:41.528 INFO:tasks.workunit.client.0.smithi032.stdout:1/981: dread - d1/d7/d30/d18/f8b zero size 2024-05-12T04:01:41.529 INFO:tasks.workunit.client.0.smithi032.stdout:9/921: getdents d0/d84/d8a/da3 0 2024-05-12T04:01:41.533 INFO:tasks.workunit.client.0.smithi032.stdout:8/992: write d0/d9/d1b/d27/d88/d56/f147 [4536772,19418] 0 2024-05-12T04:01:41.534 INFO:tasks.workunit.client.0.smithi032.stdout:1/982: mknod d1/d7/d30/d35/dbe/c148 0 2024-05-12T04:01:41.534 INFO:tasks.workunit.client.0.smithi032.stdout:2/927: write d1/d34/d96/d41/d58/f106 [329671,1177] 0 2024-05-12T04:01:41.535 INFO:tasks.workunit.client.0.smithi032.stdout:9/922: fdatasync d0/d3/d6/d7/fc 0 2024-05-12T04:01:41.539 INFO:tasks.workunit.client.0.smithi032.stdout:9/923: chown d0/d84/d8a/da3/db7/d10c/d111/d89/f11b 1109258 1 2024-05-12T04:01:41.544 INFO:tasks.workunit.client.0.smithi032.stdout:8/993: rename d0/d9/d1b/l11f to d0/d9/d1b/d66/d75/de2/d103/d108/d12c/l152 0 2024-05-12T04:01:41.545 INFO:tasks.workunit.client.0.smithi032.stdout:7/857: write dd/d1b/d43/d5e/f69 [2404656,62109] 0 2024-05-12T04:01:41.548 INFO:tasks.workunit.client.0.smithi032.stdout:2/928: dwrite d1/d34/d4a/f10b [0,4194304] 0 2024-05-12T04:01:41.550 INFO:tasks.workunit.client.0.smithi032.stdout:8/994: chown d0/d9/d13/d21/d25/df9 247503424 1 2024-05-12T04:01:41.550 INFO:tasks.workunit.client.0.smithi032.stdout:7/858: read - dd/d6c/dc3/dcd/fe2 zero size 2024-05-12T04:01:41.551 INFO:tasks.workunit.client.0.smithi032.stdout:7/859: chown dd/d1b/d1e/de3/f7d 13642 1 2024-05-12T04:01:41.552 INFO:tasks.workunit.client.0.smithi032.stdout:7/860: readlink dd/d1b/d1e/d45/d64/la1 0 2024-05-12T04:01:41.553 INFO:tasks.workunit.client.0.smithi032.stdout:1/983: dread d1/d7/d49/f4f [0,4194304] 0 2024-05-12T04:01:41.556 INFO:tasks.workunit.client.0.smithi032.stdout:9/924: mkdir d0/d3/d4b/d11c 0 2024-05-12T04:01:41.557 INFO:tasks.workunit.client.0.smithi032.stdout:9/925: write d0/d3/d6/d7/f15 [5060157,40841] 0 2024-05-12T04:01:41.562 INFO:tasks.workunit.client.0.smithi032.stdout:8/995: fdatasync d0/d7/ffe 0 2024-05-12T04:01:41.565 INFO:tasks.workunit.client.0.smithi032.stdout:0/962: dwrite df/d10/d48/d70/ff3 [0,4194304] 0 2024-05-12T04:01:41.567 INFO:tasks.workunit.client.0.smithi032.stdout:0/963: chown df/d22/d58/d8a/ff1 135675017 1 2024-05-12T04:01:41.578 INFO:tasks.workunit.client.0.smithi032.stdout:1/984: truncate d1/d7/d30/d35/f42 1407632 0 2024-05-12T04:01:41.580 INFO:tasks.workunit.client.0.smithi032.stdout:4/895: sync 2024-05-12T04:01:41.581 INFO:tasks.workunit.client.0.smithi032.stdout:4/896: dread - d0/d3/d31/d80/d84/ff6 zero size 2024-05-12T04:01:41.583 INFO:tasks.workunit.client.0.smithi032.stdout:2/929: write d1/d34/d3e/f9e [2940208,100336] 0 2024-05-12T04:01:41.584 INFO:tasks.workunit.client.0.smithi032.stdout:0/964: dread - df/d30/d4e/d55/f134 zero size 2024-05-12T04:01:41.589 INFO:tasks.workunit.client.0.smithi032.stdout:9/926: link d0/d3/d4b/d88/d99/df1/ff5 d0/d84/f11d 0 2024-05-12T04:01:41.591 INFO:tasks.workunit.client.0.smithi032.stdout:1/985: chown d1/d6a/d72/c136 131152 1 2024-05-12T04:01:41.595 INFO:tasks.workunit.client.0.smithi032.stdout:7/861: link dd/d1b/d1e/d45/d64/l89 dd/d1b/d43/d5e/df0/l109 0 2024-05-12T04:01:41.599 INFO:tasks.workunit.client.0.smithi032.stdout:2/930: rename d1/d13/da0/dae to d1/d34/d3e/d67/dda/d130 0 2024-05-12T04:01:41.603 INFO:tasks.workunit.client.0.smithi032.stdout:7/862: dwrite dd/d1b/d43/d74/ff2 [0,4194304] 0 2024-05-12T04:01:41.606 INFO:tasks.workunit.client.0.smithi032.stdout:1/986: dread - d1/d7/d30/d35/d77/fd5 zero size 2024-05-12T04:01:41.609 INFO:tasks.workunit.client.0.smithi032.stdout:8/996: link d0/d9/d1b/d27/d88/d56/d63/d8f/db0/l10c d0/d4b/l153 0 2024-05-12T04:01:41.611 INFO:tasks.workunit.client.0.smithi032.stdout:2/931: readlink d1/ld 0 2024-05-12T04:01:41.613 INFO:tasks.workunit.client.0.smithi032.stdout:2/932: chown d1/d34/d96/d41/f40 40612 1 2024-05-12T04:01:41.613 INFO:tasks.workunit.client.0.smithi032.stdout:2/933: chown d1/d4d/l66 893552 1 2024-05-12T04:01:41.619 INFO:tasks.workunit.client.0.smithi032.stdout:7/863: rename dd/d1b/d1e/de3/d9a/fb8 to dd/d1b/d1e/d45/d55/d72/da0/dbd/f10a 0 2024-05-12T04:01:41.621 INFO:tasks.workunit.client.0.smithi032.stdout:4/897: write d0/d3/d24/f76 [918804,29108] 0 2024-05-12T04:01:41.622 INFO:tasks.workunit.client.0.smithi032.stdout:1/987: dread d1/d25/f5b [0,4194304] 0 2024-05-12T04:01:41.624 INFO:tasks.workunit.client.0.smithi032.stdout:4/898: fdatasync d0/d3/d12/d1d/d6b/db7/de0/ff4 0 2024-05-12T04:01:41.626 INFO:tasks.workunit.client.0.smithi032.stdout:0/965: creat df/d10/d3d/d4a/d84/f13b x:0 0 0 2024-05-12T04:01:41.627 INFO:tasks.workunit.client.0.smithi032.stdout:9/927: link d0/d3/d6/d7/d2a/lae d0/d84/d8a/l11e 0 2024-05-12T04:01:41.632 INFO:tasks.workunit.client.0.smithi032.stdout:8/997: rename d0/d9/d1b/d27/d88/d56/d63/d8f/d9a/d28/f49 to d0/d9/d13/d92/f154 0 2024-05-12T04:01:41.636 INFO:tasks.workunit.client.0.smithi032.stdout:2/934: mknod d1/d13/d70/d85/df3/d12c/c131 0 2024-05-12T04:01:41.637 INFO:tasks.workunit.client.0.smithi032.stdout:7/864: write dd/d1b/d1e/d45/d55/db7/fd5 [827272,7510] 0 2024-05-12T04:01:41.639 INFO:tasks.workunit.client.0.smithi032.stdout:2/935: dread - d1/d34/d96/d41/d39/f12a zero size 2024-05-12T04:01:41.641 INFO:tasks.workunit.client.0.smithi032.stdout:2/936: readlink d1/d13/da0/l117 0 2024-05-12T04:01:41.651 INFO:tasks.workunit.client.0.smithi032.stdout:4/899: dwrite d0/d75/f78 [0,4194304] 0 2024-05-12T04:01:41.656 INFO:tasks.workunit.client.0.smithi032.stdout:9/928: dwrite d0/d3/d4b/d88/d99/fbd [0,4194304] 0 2024-05-12T04:01:41.656 INFO:tasks.workunit.client.0.smithi032.stdout:2/937: rmdir d1/d34/d96/d41 39 2024-05-12T04:01:41.656 INFO:tasks.workunit.client.0.smithi032.stdout:7/865: dread dd/d1b/d1e/f7e [0,4194304] 0 2024-05-12T04:01:41.659 INFO:tasks.workunit.client.0.smithi032.stdout:1/988: dwrite d1/d25/f4d [0,4194304] 0 2024-05-12T04:01:41.660 INFO:tasks.workunit.client.0.smithi032.stdout:8/998: mkdir d0/d9/d1b/d27/d4c/d121/d155 0 2024-05-12T04:01:41.663 INFO:tasks.workunit.client.0.smithi032.stdout:0/966: rmdir df/d30/d4e/de0 0 2024-05-12T04:01:41.672 INFO:tasks.workunit.client.0.smithi032.stdout:2/938: rmdir d1/d34/da5 39 2024-05-12T04:01:41.672 INFO:tasks.workunit.client.0.smithi032.stdout:4/900: dread - d0/db5/fd3 zero size 2024-05-12T04:01:41.676 INFO:tasks.workunit.client.0.smithi032.stdout:2/939: write d1/d13/d70/d85/f9c [3609694,26360] 0 2024-05-12T04:01:41.677 INFO:tasks.workunit.client.0.smithi032.stdout:4/901: fsync d0/d3/d31/f85 0 2024-05-12T04:01:41.679 INFO:tasks.workunit.client.0.smithi032.stdout:1/989: symlink d1/d7/d30/d35/dbe/l149 0 2024-05-12T04:01:41.680 INFO:tasks.workunit.client.0.smithi032.stdout:9/929: dwrite d0/d84/d8a/da3/db7/d10c/d111/d89/f11b [0,4194304] 0 2024-05-12T04:01:41.680 INFO:tasks.workunit.client.0.smithi032.stdout:8/999: symlink d0/d9/d1b/d66/d75/de2/d105/l156 0 2024-05-12T04:01:41.684 INFO:tasks.workunit.client.0.smithi032.stdout:7/866: write dd/d1b/d1e/d45/d64/f91 [65118,51362] 0 2024-05-12T04:01:41.690 INFO:tasks.workunit.client.0.smithi032.stdout:2/940: creat d1/d4d/d5e/f132 x:0 0 0 2024-05-12T04:01:41.695 INFO:tasks.workunit.client.0.smithi032.stdout:1/990: creat d1/dd/d24/d9b/f14a x:0 0 0 2024-05-12T04:01:41.696 INFO:tasks.workunit.client.0.smithi032.stdout:0/967: mknod df/d30/d4e/da5/d10c/d126/c13c 0 2024-05-12T04:01:41.697 INFO:tasks.workunit.client.0.smithi032.stdout:0/968: fdatasync df/d10/d3d/d4a/f65 0 2024-05-12T04:01:41.698 INFO:tasks.workunit.client.0.smithi032.stdout:9/930: dread d0/f28 [0,4194304] 0 2024-05-12T04:01:41.699 INFO:tasks.workunit.client.0.smithi032.stdout:4/902: rename d0/d3/c28 to d0/d3/d12/d1d/d6b/c10c 0 2024-05-12T04:01:41.701 INFO:tasks.workunit.client.0.smithi032.stdout:2/941: fdatasync d1/d34/d96/d41/d2c/f59 0 2024-05-12T04:01:41.704 INFO:tasks.workunit.client.0.smithi032.stdout:9/931: mknod d0/d3/d6/d7/d27/c11f 0 2024-05-12T04:01:41.704 INFO:tasks.workunit.client.0.smithi032.stdout:9/932: readlink d0/d3/d4b/l60 0 2024-05-12T04:01:41.717 INFO:tasks.workunit.client.0.smithi032.stdout:2/942: dread d1/d34/d96/d41/f3a [0,4194304] 0 2024-05-12T04:01:41.718 INFO:tasks.workunit.client.0.smithi032.stdout:0/969: dwrite df/d10/d3d/d83/f108 [0,4194304] 0 2024-05-12T04:01:41.720 INFO:tasks.workunit.client.0.smithi032.stdout:9/933: mknod d0/d84/d8a/da3/db7/d10c/d111/c120 0 2024-05-12T04:01:41.721 INFO:tasks.workunit.client.0.smithi032.stdout:4/903: getdents d0/d3/d24 0 2024-05-12T04:01:41.724 INFO:tasks.workunit.client.0.smithi032.stdout:2/943: creat d1/d34/d96/d41/d52/f133 x:0 0 0 2024-05-12T04:01:41.726 INFO:tasks.workunit.client.0.smithi032.stdout:9/934: chown d0/d3/d6/d7/c7b 63647 1 2024-05-12T04:01:41.726 INFO:tasks.workunit.client.0.smithi032.stdout:9/935: readlink d0/d3/d4b/ld4 0 2024-05-12T04:01:41.727 INFO:tasks.workunit.client.0.smithi032.stdout:4/904: unlink d0/d3/d31/f53 0 2024-05-12T04:01:41.727 INFO:tasks.workunit.client.0.smithi032.stdout:9/936: chown d0/d3/d6/d7/d27 588 1 2024-05-12T04:01:41.728 INFO:tasks.workunit.client.0.smithi032.stdout:4/905: read - d0/d3/df/f10b zero size 2024-05-12T04:01:41.731 INFO:tasks.workunit.client.0.smithi032.stdout:0/970: rename df/d30/d4e/d55/dd4/fe5 to df/d10/d48/d70/dbb/f13d 0 2024-05-12T04:01:41.736 INFO:tasks.workunit.client.0.smithi032.stdout:4/906: symlink d0/d3/d12/d1d/d63/d87/dc6/l10d 0 2024-05-12T04:01:41.737 INFO:tasks.workunit.client.0.smithi032.stdout:4/907: chown d0/d3/d12/d1d/cf8 952 1 2024-05-12T04:01:41.739 INFO:tasks.workunit.client.0.smithi032.stdout:2/944: mknod d1/d34/da5/db3/db7/c134 0 2024-05-12T04:01:41.740 INFO:tasks.workunit.client.0.smithi032.stdout:9/937: creat d0/d3/d6/d7/d106/f121 x:0 0 0 2024-05-12T04:01:41.741 INFO:tasks.workunit.client.0.smithi032.stdout:9/938: dread - d0/d84/d8a/da3/db7/d10c/d111/d39/fe8 zero size 2024-05-12T04:01:41.745 INFO:tasks.workunit.client.0.smithi032.stdout:4/908: creat d0/d3/d12/d1d/d63/d77/f10e x:0 0 0 2024-05-12T04:01:41.745 INFO:tasks.workunit.client.0.smithi032.stdout:2/945: rmdir d1/d34/d4a 39 2024-05-12T04:01:41.746 INFO:tasks.workunit.client.0.smithi032.stdout:0/971: link df/d30/d36/f12f df/d10/d3d/f13e 0 2024-05-12T04:01:41.831 INFO:tasks.workunit.client.0.smithi032.stdout:4/909: fsync d0/f65 0 2024-05-12T04:01:41.831 INFO:tasks.workunit.client.0.smithi032.stdout:2/946: unlink d1/d34/d96/d41/d2c/dd4/f114 0 2024-05-12T04:01:41.836 INFO:tasks.workunit.client.0.smithi032.stdout:9/939: creat d0/d3/f122 x:0 0 0 2024-05-12T04:01:41.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.836+0000 7f30b0331700 1 -- 172.21.15.32:0/2423261712 --> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f30a81118b0 con 0x7f309406aec0 2024-05-12T04:01:41.837 INFO:tasks.workunit.client.0.smithi032.stdout:4/910: dread d0/d3/d12/d1d/fa4 [0,4194304] 0 2024-05-12T04:01:41.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.837+0000 7f309effd700 1 -- 172.21.15.32:0/2423261712 <== mgr.24389 v2:172.21.15.33:6828/1598726962 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7f30a81118b0 con 0x7f309406aec0 2024-05-12T04:01:41.838 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-05-12T04:01:41.838 INFO:teuthology.orchestra.run.smithi032.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:19c9734256aa43a60e203feb242f50bf8b7f58666d520536af43cc4fee96173a", 2024-05-12T04:01:41.839 INFO:teuthology.orchestra.run.smithi032.stdout: "in_progress": true, 2024-05-12T04:01:41.839 INFO:teuthology.orchestra.run.smithi032.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-12T04:01:41.839 INFO:teuthology.orchestra.run.smithi032.stdout: "services_complete": [], 2024-05-12T04:01:41.839 INFO:teuthology.orchestra.run.smithi032.stdout: "progress": "1/21 daemons upgraded", 2024-05-12T04:01:41.839 INFO:teuthology.orchestra.run.smithi032.stdout: "message": "", 2024-05-12T04:01:41.839 INFO:teuthology.orchestra.run.smithi032.stdout: "is_paused": false 2024-05-12T04:01:41.839 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-05-12T04:01:41.841 INFO:tasks.workunit.client.0.smithi032.stdout:2/947: rename d1/d34/d96/d41/f110 to d1/d34/d96/d41/d52/df7/f135 0 2024-05-12T04:01:41.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.841+0000 7f30b0331700 1 -- 172.21.15.32:0/2423261712 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f309406aec0 msgr2=0x7f309406d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:01:41.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.841+0000 7f30b0331700 1 --2- 172.21.15.32:0/2423261712 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f309406aec0 0x7f309406d370 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3098009b80 tx=0x7f309800b040 comp rx=0 tx=0).stop 2024-05-12T04:01:41.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.841+0000 7f30b0331700 1 -- 172.21.15.32:0/2423261712 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30a80fbcb0 msgr2=0x7f30a810fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:01:41.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.841+0000 7f30b0331700 1 --2- 172.21.15.32:0/2423261712 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30a80fbcb0 0x7f30a810fb00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f30a40137b0 tx=0x7f30a4009e30 comp rx=0 tx=0).stop 2024-05-12T04:01:41.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.841+0000 7f30b0331700 1 -- 172.21.15.32:0/2423261712 shutdown_connections 2024-05-12T04:01:41.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.841+0000 7f30b0331700 1 --2- 172.21.15.32:0/2423261712 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f309406aec0 0x7f309406d370 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:41.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.841+0000 7f30b0331700 1 --2- 172.21.15.32:0/2423261712 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30a80fbcb0 0x7f30a810fb00 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:41.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.841+0000 7f30b0331700 1 --2- 172.21.15.32:0/2423261712 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30a80fcf60 0x7f30a8110040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:41.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.841+0000 7f30b0331700 1 -- 172.21.15.32:0/2423261712 >> 172.21.15.32:0/2423261712 conn(0x7f30a80f7430 msgr2=0x7f30a80f9810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:01:41.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.841+0000 7f30b0331700 1 -- 172.21.15.32:0/2423261712 shutdown_connections 2024-05-12T04:01:41.843 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.841+0000 7f30b0331700 1 -- 172.21.15.32:0/2423261712 wait complete. 2024-05-12T04:01:41.843 INFO:tasks.workunit.client.0.smithi032.stdout:0/972: mkdir df/d10/d3d/d6c/df4/d13f 0 2024-05-12T04:01:41.844 INFO:tasks.workunit.client.0.smithi032.stdout:9/940: mkdir d0/d3/d4b/d85/d123 0 2024-05-12T04:01:41.850 INFO:tasks.workunit.client.0.smithi032.stdout:2/948: dwrite d1/d4d/d5e/f132 [0,4194304] 0 2024-05-12T04:01:41.860 INFO:tasks.workunit.client.0.smithi032.stdout:4/911: rename d0/d3/df/l40 to d0/d3/d12/d8d/da1/l10f 0 2024-05-12T04:01:41.865 INFO:tasks.workunit.client.0.smithi032.stdout:0/973: link df/d28/f125 df/d22/f140 0 2024-05-12T04:01:41.867 INFO:tasks.workunit.client.0.smithi032.stdout:2/949: mkdir d1/d34/d96/d41/da2/d136 0 2024-05-12T04:01:41.873 INFO:tasks.workunit.client.0.smithi032.stdout:0/974: mkdir df/d30/d4e/d55/d141 0 2024-05-12T04:01:41.874 INFO:tasks.workunit.client.0.smithi032.stdout:0/975: chown df/d10/d3d/d4a/d84/f13b 731 1 2024-05-12T04:01:41.875 INFO:tasks.workunit.client.0.smithi032.stdout:0/976: write df/d10/d3d/d4a/d84/f13b [823631,88464] 0 2024-05-12T04:01:41.879 INFO:tasks.workunit.client.0.smithi032.stdout:2/950: creat d1/d73/d128/f137 x:0 0 0 2024-05-12T04:01:41.880 INFO:tasks.workunit.client.0.smithi032.stdout:9/941: dwrite d0/d3/d4b/d6f/f71 [0,4194304] 0 2024-05-12T04:01:41.883 INFO:tasks.workunit.client.0.smithi032.stdout:2/951: chown d1/d34/d96/d41/d52/ffc 96 1 2024-05-12T04:01:41.898 INFO:tasks.workunit.client.0.smithi032.stdout:4/912: dwrite d0/d3/df/d4e/f9f [0,4194304] 0 2024-05-12T04:01:41.901 INFO:tasks.workunit.client.0.smithi032.stdout:9/942: dwrite d0/d3/d1b/fd0 [4194304,4194304] 0 2024-05-12T04:01:41.902 INFO:tasks.workunit.client.0.smithi032.stdout:4/913: dread - d0/d3/d12/f106 zero size 2024-05-12T04:01:41.902 INFO:tasks.workunit.client.0.smithi032.stdout:0/977: truncate df/d10/f1b 1659137 0 2024-05-12T04:01:41.904 INFO:tasks.workunit.client.0.smithi032.stdout:9/943: read d0/d3/d4b/d85/dbe/d82/fff [60247,119413] 0 2024-05-12T04:01:41.907 INFO:tasks.workunit.client.0.smithi032.stdout:4/914: unlink d0/d3/d12/d1d/ff1 0 2024-05-12T04:01:41.908 INFO:tasks.workunit.client.0.smithi032.stdout:2/952: dread d1/d34/d96/d41/d2c/dd4/f102 [0,4194304] 0 2024-05-12T04:01:41.916 INFO:tasks.workunit.client.0.smithi032.stdout:2/953: symlink d1/d13/d70/l138 0 2024-05-12T04:01:41.922 INFO:tasks.workunit.client.0.smithi032.stdout:2/954: rename d1/d13/d6b/l6f to d1/d104/l139 0 2024-05-12T04:01:41.923 INFO:tasks.workunit.client.0.smithi032.stdout:0/978: link df/d28/f125 df/d10/d3d/d4a/d84/d133/f142 0 2024-05-12T04:01:41.925 INFO:tasks.workunit.client.0.smithi032.stdout:2/955: creat d1/d13/d70/d85/f13a x:0 0 0 2024-05-12T04:01:41.930 INFO:tasks.workunit.client.0.smithi032.stdout:2/956: readlink d1/d4d/d5e/l103 0 2024-05-12T04:01:41.933 INFO:tasks.workunit.client.0.smithi032.stdout:9/944: dread d0/d3/f25 [0,4194304] 0 2024-05-12T04:01:41.934 INFO:tasks.workunit.client.0.smithi032.stdout:9/945: chown d0/d3/d74/d51 98947 1 2024-05-12T04:01:41.939 INFO:tasks.workunit.client.0.smithi032.stdout:0/979: dread df/d30/d36/ddd/f104 [0,4194304] 0 2024-05-12T04:01:41.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.985+0000 7f47e42f9700 1 -- 172.21.15.32:0/390050610 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47dc0fcf60 msgr2=0x7f47dc0fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:01:41.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.985+0000 7f47e42f9700 1 --2- 172.21.15.32:0/390050610 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47dc0fcf60 0x7f47dc0fd3d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f47d0004940 tx=0x7f47d0009e30 comp rx=0 tx=0).stop 2024-05-12T04:01:41.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.985+0000 7f47e42f9700 1 -- 172.21.15.32:0/390050610 shutdown_connections 2024-05-12T04:01:41.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.985+0000 7f47e42f9700 1 --2- 172.21.15.32:0/390050610 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47dc0fcf60 0x7f47dc0fd3d0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:41.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.985+0000 7f47e42f9700 1 --2- 172.21.15.32:0/390050610 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47dc0fbcb0 0x7f47dc0fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:41.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.985+0000 7f47e42f9700 1 -- 172.21.15.32:0/390050610 >> 172.21.15.32:0/390050610 conn(0x7f47dc0f7430 msgr2=0x7f47dc0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:01:41.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.985+0000 7f47e42f9700 1 -- 172.21.15.32:0/390050610 shutdown_connections 2024-05-12T04:01:41.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.985+0000 7f47e42f9700 1 -- 172.21.15.32:0/390050610 wait complete. 2024-05-12T04:01:41.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.986+0000 7f47e42f9700 1 Processor -- start 2024-05-12T04:01:41.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.986+0000 7f47e42f9700 1 -- start start 2024-05-12T04:01:41.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.986+0000 7f47e42f9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47dc0fbcb0 0x7f47dc066c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:01:41.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.986+0000 7f47e42f9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47dc0fcf60 0x7f47dc1125f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:01:41.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.986+0000 7f47e42f9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47dc112b30 con 0x7f47dc0fcf60 2024-05-12T04:01:41.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.986+0000 7f47e42f9700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47dc112c70 con 0x7f47dc0fbcb0 2024-05-12T04:01:41.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.986+0000 7f47e1894700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47dc0fcf60 0x7f47dc1125f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:01:41.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.986+0000 7f47e1894700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47dc0fcf60 0x7f47dc1125f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46004/0 (socket says 172.21.15.32:46004) 2024-05-12T04:01:41.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.986+0000 7f47e1894700 1 -- 172.21.15.32:0/2808451881 learned_addr learned my addr 172.21.15.32:0/2808451881 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:01:41.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.986+0000 7f47e2095700 1 --2- 172.21.15.32:0/2808451881 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47dc0fbcb0 0x7f47dc066c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:01:41.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.986+0000 7f47e1894700 1 -- 172.21.15.32:0/2808451881 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47dc0fbcb0 msgr2=0x7f47dc066c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:01:41.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.986+0000 7f47e1894700 1 --2- 172.21.15.32:0/2808451881 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47dc0fbcb0 0x7f47dc066c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:41.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.986+0000 7f47e1894700 1 -- 172.21.15.32:0/2808451881 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47d0005040 con 0x7f47dc0fcf60 2024-05-12T04:01:41.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.986+0000 7f47e1894700 1 --2- 172.21.15.32:0/2808451881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47dc0fcf60 0x7f47dc1125f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f47d0009380 tx=0x7f47d0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:01:41.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.987+0000 7f47ceffd700 1 -- 172.21.15.32:0/2808451881 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47d000e040 con 0x7f47dc0fcf60 2024-05-12T04:01:41.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.987+0000 7f47ceffd700 1 -- 172.21.15.32:0/2808451881 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f47d000bd20 con 0x7f47dc0fcf60 2024-05-12T04:01:41.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.987+0000 7f47e42f9700 1 -- 172.21.15.32:0/2808451881 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47dc112ea0 con 0x7f47dc0fcf60 2024-05-12T04:01:41.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.987+0000 7f47ceffd700 1 -- 172.21.15.32:0/2808451881 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47d001c630 con 0x7f47dc0fcf60 2024-05-12T04:01:41.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.987+0000 7f47e42f9700 1 -- 172.21.15.32:0/2808451881 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47dc113430 con 0x7f47dc0fcf60 2024-05-12T04:01:41.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.988+0000 7f47e42f9700 1 -- 172.21.15.32:0/2808451881 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47dc04c710 con 0x7f47dc0fcf60 2024-05-12T04:01:41.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.988+0000 7f47ceffd700 1 -- 172.21.15.32:0/2808451881 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) v1 ==== 88670+0+0 (secure 0 0 0) 0x7f47d0013420 con 0x7f47dc0fcf60 2024-05-12T04:01:41.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.989+0000 7f47ceffd700 1 --2- 172.21.15.32:0/2808451881 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f47c806ada0 0x7f47c806d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:01:41.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.989+0000 7f47ceffd700 1 -- 172.21.15.32:0/2808451881 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f47d008a250 con 0x7f47dc0fcf60 2024-05-12T04:01:41.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.989+0000 7f47e2095700 1 --2- 172.21.15.32:0/2808451881 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f47c806ada0 0x7f47c806d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:01:41.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.989+0000 7f47e2095700 1 --2- 172.21.15.32:0/2808451881 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f47c806ada0 0x7f47c806d250 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f47d80041d0 tx=0x7f47d80043c0 comp rx=0 tx=0).ready entity=mgr.24389 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:01:41.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:41.993+0000 7f47ceffd700 1 -- 172.21.15.32:0/2808451881 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181777 (secure 0 0 0) 0x7f47d0058470 con 0x7f47dc0fcf60 2024-05-12T04:01:42.085 INFO:tasks.workunit.client.0.smithi032.stdout:2/957: creat d1/d13/d6b/f13b x:0 0 0 2024-05-12T04:01:42.085 INFO:tasks.workunit.client.0.smithi032.stdout:2/958: chown d1/d13/da0 3 1 2024-05-12T04:01:42.087 INFO:tasks.workunit.client.0.smithi032.stdout:2/959: read - d1/d34/da5/db3/f124 zero size 2024-05-12T04:01:42.091 INFO:tasks.workunit.client.0.smithi032.stdout:9/946: getdents d0/d84/d8a/da3/db7/d10c/d111/d89/def 0 2024-05-12T04:01:42.100 INFO:tasks.workunit.client.0.smithi032.stdout:9/947: symlink d0/d3/d1b/l124 0 2024-05-12T04:01:42.101 INFO:tasks.workunit.client.0.smithi032.stdout:2/960: dread d1/d34/d3e/f9e [0,4194304] 0 2024-05-12T04:01:42.101 INFO:tasks.workunit.client.0.smithi032.stdout:9/948: chown d0/d3/d4b/f6d 3056609 1 2024-05-12T04:01:42.103 INFO:tasks.workunit.client.0.smithi032.stdout:2/961: truncate d1/d34/d95/f109 3029161 0 2024-05-12T04:01:42.103 INFO:tasks.workunit.client.0.smithi032.stdout:9/949: write d0/d3/ff [10296148,129251] 0 2024-05-12T04:01:42.106 INFO:tasks.workunit.client.0.smithi032.stdout:0/980: write df/d30/d4e/d55/fa7 [1718518,52712] 0 2024-05-12T04:01:42.109 INFO:tasks.workunit.client.0.smithi032.stdout:9/950: dread d0/d3/d6/d58/f5e [0,4194304] 0 2024-05-12T04:01:42.109 INFO:tasks.workunit.client.0.smithi032.stdout:2/962: creat d1/d34/da5/f13c x:0 0 0 2024-05-12T04:01:42.114 INFO:tasks.workunit.client.0.smithi032.stdout:2/963: mkdir d1/d34/d96/d41/d58/de1/d13d 0 2024-05-12T04:01:42.116 INFO:tasks.workunit.client.0.smithi032.stdout:2/964: write d1/d104/f127 [392624,101395] 0 2024-05-12T04:01:42.119 INFO:tasks.workunit.client.0.smithi032.stdout:2/965: write d1/d34/d4a/f10b [89160,74763] 0 2024-05-12T04:01:42.125 INFO:tasks.workunit.client.0.smithi032.stdout:2/966: rename d1/d34/d95 to d1/d73/d13e 0 2024-05-12T04:01:42.128 INFO:tasks.workunit.client.0.smithi032.stdout:0/981: dread fa [4194304,4194304] 0 2024-05-12T04:01:42.131 INFO:tasks.workunit.client.0.smithi032.stdout:0/982: read df/d30/d32/f34 [657123,50737] 0 2024-05-12T04:01:42.137 INFO:tasks.workunit.client.0.smithi032.stdout:0/983: truncate df/d10/d48/d70/dbb/fc9 923630 0 2024-05-12T04:01:42.140 INFO:tasks.workunit.client.0.smithi032.stdout:0/984: creat df/d22/d58/f143 x:0 0 0 2024-05-12T04:01:42.141 INFO:tasks.workunit.client.0.smithi032.stdout:0/985: chown df/d10/d3d/d131 18581 1 2024-05-12T04:01:42.143 INFO:tasks.workunit.client.0.smithi032.stdout:0/986: truncate df/d10/f13 594931 0 2024-05-12T04:01:42.144 INFO:tasks.workunit.client.0.smithi032.stdout:0/987: chown df/d10/d48/d70/f76 333 1 2024-05-12T04:01:42.148 INFO:tasks.workunit.client.0.smithi032.stdout:0/988: symlink df/df2/dd5/l144 0 2024-05-12T04:01:42.151 INFO:tasks.workunit.client.0.smithi032.stdout:0/989: write df/d30/d8d/d8f/f10e [4950405,42826] 0 2024-05-12T04:01:42.152 INFO:tasks.workunit.client.0.smithi032.stdout:0/990: dread - df/d30/d32/d60/dc7/f12c zero size 2024-05-12T04:01:42.155 INFO:tasks.workunit.client.0.smithi032.stdout:0/991: symlink df/d30/d4e/da5/d10c/d126/l145 0 2024-05-12T04:01:42.161 INFO:tasks.workunit.client.0.smithi032.stdout:0/992: read df/d30/d8d/d8f/fcb [664065,30563] 0 2024-05-12T04:01:42.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:41 smithi033 ceph-mon[49525]: from='client.14732 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:01:42.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:41 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/3317299972' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-12T04:01:42.162 INFO:tasks.workunit.client.0.smithi032.stdout:7/867: sync 2024-05-12T04:01:42.163 INFO:tasks.workunit.client.0.smithi032.stdout:0/993: mkdir df/d22/d58/d146 0 2024-05-12T04:01:42.168 INFO:tasks.workunit.client.0.smithi032.stdout:0/994: fdatasync df/d30/d4e/d55/f7f 0 2024-05-12T04:01:42.196 INFO:tasks.workunit.client.0.smithi032.stdout:7/868: write dd/d1b/d1e/de3/d9a/d9c/db9/d75/fc4 [207554,27439] 0 2024-05-12T04:01:42.207 INFO:tasks.workunit.client.0.smithi032.stdout:7/869: dread - dd/d1b/d1e/de3/d9a/d9c/db9/fcc zero size 2024-05-12T04:01:42.207 INFO:tasks.workunit.client.0.smithi032.stdout:0/995: dwrite df/d30/d36/ddd/f104 [0,4194304] 0 2024-05-12T04:01:42.226 INFO:tasks.workunit.client.0.smithi032.stdout:7/870: mkdir dd/d1b/d1e/de3/d10b 0 2024-05-12T04:01:42.230 INFO:tasks.workunit.client.0.smithi032.stdout:7/871: mkdir dd/d1b/d1e/d45/d55/d72/d10c 0 2024-05-12T04:01:42.234 INFO:tasks.workunit.client.0.smithi032.stdout:0/996: write df/d10/d3d/f56 [5104257,58218] 0 2024-05-12T04:01:42.238 INFO:tasks.workunit.client.0.smithi032.stdout:7/872: getdents dd/d1b/d1e/d45/d55/d72/d10c 0 2024-05-12T04:01:42.243 INFO:tasks.workunit.client.0.smithi032.stdout:0/997: dwrite df/d10/d3d/d4a/d9a/d9d/d106/d113/f11a [0,4194304] 0 2024-05-12T04:01:42.257 INFO:tasks.workunit.client.0.smithi032.stdout:7/873: dwrite dd/d1b/d1e/de3/d9a/d9c/db9/f6b [0,4194304] 0 2024-05-12T04:01:42.260 INFO:tasks.workunit.client.0.smithi032.stdout:7/874: mknod dd/d1b/d43/d5e/c10d 0 2024-05-12T04:01:42.270 INFO:tasks.workunit.client.0.smithi032.stdout:0/998: dread df/d10/d3b/dba/f109 [0,4194304] 0 2024-05-12T04:01:42.274 INFO:tasks.workunit.client.0.smithi032.stdout:7/875: dwrite dd/d1b/d1e/d45/fd9 [0,4194304] 0 2024-05-12T04:01:42.279 INFO:tasks.workunit.client.0.smithi032.stdout:0/999: truncate df/d10/d3d/d83/d9f/faa 157956 0 2024-05-12T04:01:42.282 INFO:tasks.workunit.client.0.smithi032.stdout:7/876: creat dd/d1b/d1e/d45/d64/f10e x:0 0 0 2024-05-12T04:01:42.288 INFO:tasks.workunit.client.0.smithi032.stdout:7/877: unlink dd/d1b/d1e/de3/d9a/d9c/db9/d75/f76 0 2024-05-12T04:01:42.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:41 smithi032 ceph-mon[40008]: from='client.14732 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:01:42.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:41 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/3317299972' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-12T04:01:42.290 INFO:tasks.workunit.client.0.smithi032.stdout:7/878: symlink dd/d1b/d43/d74/l10f 0 2024-05-12T04:01:42.294 INFO:tasks.workunit.client.0.smithi032.stdout:7/879: unlink dd/d1b/d43/c70 0 2024-05-12T04:01:42.295 INFO:tasks.workunit.client.0.smithi032.stdout:7/880: write dd/f102 [547777,105683] 0 2024-05-12T04:01:42.303 INFO:tasks.workunit.client.0.smithi032.stdout:7/881: dwrite dd/d6c/ff3 [0,4194304] 0 2024-05-12T04:01:42.304 INFO:tasks.workunit.client.0.smithi032.stdout:7/882: write dd/f13 [7151649,66317] 0 2024-05-12T04:01:42.304 INFO:tasks.workunit.client.0.smithi032.stdout:7/883: stat dd/d1b/d1e/c53 0 2024-05-12T04:01:42.314 INFO:tasks.workunit.client.0.smithi032.stdout:7/884: link dd/d1b/d43/d5e/f5a dd/d1b/d1e/d45/d55/d72/da0/dbd/db2/f110 0 2024-05-12T04:01:42.318 INFO:tasks.workunit.client.0.smithi032.stdout:7/885: creat dd/d1b/d1e/d45/d55/d72/da0/dbd/db2/f111 x:0 0 0 2024-05-12T04:01:42.323 INFO:tasks.workunit.client.0.smithi032.stdout:7/886: getdents dd/d1b 0 2024-05-12T04:01:42.325 INFO:tasks.workunit.client.0.smithi032.stdout:7/887: chown dd/l2e 312 1 2024-05-12T04:01:42.330 INFO:tasks.workunit.client.0.smithi032.stdout:7/888: fsync dd/d1b/d1e/d45/d55/db7/ff1 0 2024-05-12T04:01:42.345 INFO:tasks.workunit.client.0.smithi032.stdout:7/889: dread dd/d1b/d43/d5e/f5a [0,4194304] 0 2024-05-12T04:01:42.361 INFO:tasks.workunit.client.0.smithi032.stdout:7/890: dread dd/d1b/d1e/d45/de4/fe5 [0,4194304] 0 2024-05-12T04:01:42.363 INFO:tasks.workunit.client.0.smithi032.stdout:7/891: mknod dd/d1b/d43/d5e/c112 0 2024-05-12T04:01:42.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:42.390+0000 7f47e42f9700 1 -- 172.21.15.32:0/2808451881 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f47dc104660 con 0x7f47dc0fcf60 2024-05-12T04:01:42.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:42.390+0000 7f47ceffd700 1 -- 172.21.15.32:0/2808451881 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f47d0058000 con 0x7f47dc0fcf60 2024-05-12T04:01:42.391 INFO:teuthology.orchestra.run.smithi032.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-12T04:01:42.391 INFO:teuthology.orchestra.run.smithi032.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-12T04:01:42.391 INFO:teuthology.orchestra.run.smithi032.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-12T04:01:42.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:42.392+0000 7f47e42f9700 1 -- 172.21.15.32:0/2808451881 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f47c806ada0 msgr2=0x7f47c806d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:01:42.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:42.392+0000 7f47e42f9700 1 --2- 172.21.15.32:0/2808451881 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f47c806ada0 0x7f47c806d250 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f47d80041d0 tx=0x7f47d80043c0 comp rx=0 tx=0).stop 2024-05-12T04:01:42.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:42.392+0000 7f47e42f9700 1 -- 172.21.15.32:0/2808451881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47dc0fcf60 msgr2=0x7f47dc1125f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:01:42.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:42.392+0000 7f47e42f9700 1 --2- 172.21.15.32:0/2808451881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47dc0fcf60 0x7f47dc1125f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f47d0009380 tx=0x7f47d0009e30 comp rx=0 tx=0).stop 2024-05-12T04:01:42.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:42.392+0000 7f47e42f9700 1 -- 172.21.15.32:0/2808451881 shutdown_connections 2024-05-12T04:01:42.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:42.392+0000 7f47e42f9700 1 --2- 172.21.15.32:0/2808451881 >> [v2:172.21.15.33:6828/1598726962,v1:172.21.15.33:6829/1598726962] conn(0x7f47c806ada0 0x7f47c806d250 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:42.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:42.392+0000 7f47e42f9700 1 --2- 172.21.15.32:0/2808451881 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47dc0fbcb0 0x7f47dc066c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:42.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:42.392+0000 7f47e42f9700 1 --2- 172.21.15.32:0/2808451881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47dc0fcf60 0x7f47dc1125f0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:01:42.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:42.392+0000 7f47e42f9700 1 -- 172.21.15.32:0/2808451881 >> 172.21.15.32:0/2808451881 conn(0x7f47dc0f7430 msgr2=0x7f47dc0f8f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:01:42.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:42.392+0000 7f47e42f9700 1 -- 172.21.15.32:0/2808451881 shutdown_connections 2024-05-12T04:01:42.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:01:42.393+0000 7f47e42f9700 1 -- 172.21.15.32:0/2808451881 wait complete. 2024-05-12T04:01:42.462 INFO:tasks.workunit.client.0.smithi032.stdout:1/991: sync 2024-05-12T04:01:42.463 INFO:tasks.workunit.client.0.smithi032.stdout:1/992: stat d1/d7/d30/d18/d87/dc6/f145 0 2024-05-12T04:01:42.466 INFO:tasks.workunit.client.0.smithi032.stdout:1/993: symlink d1/d25/da5/d142/l14b 0 2024-05-12T04:01:42.466 INFO:tasks.workunit.client.0.smithi032.stdout:1/994: chown d1/d7/d49/c65 8 1 2024-05-12T04:01:42.469 INFO:tasks.workunit.client.0.smithi032.stdout:1/995: mknod d1/d7/d30/d18/c14c 0 2024-05-12T04:01:42.470 INFO:tasks.workunit.client.0.smithi032.stdout:1/996: rmdir d1/d7/d30/d35/d54/d85 39 2024-05-12T04:01:42.472 INFO:tasks.workunit.client.0.smithi032.stdout:1/997: fdatasync d1/dd/d24/d101/f10d 0 2024-05-12T04:01:42.477 INFO:tasks.workunit.client.0.smithi032.stdout:1/998: link d1/d7/d30/d18/d93/dc1/dc3/ld6 d1/d7/d30/d61/d103/l14d 0 2024-05-12T04:01:42.485 INFO:tasks.workunit.client.0.smithi032.stdout:1/999: dwrite d1/d7/d30/d18/d87/f120 [0,4194304] 0 2024-05-12T04:01:42.621 INFO:tasks.workunit.client.0.smithi032.stdout:9/951: sync 2024-05-12T04:01:42.621 INFO:tasks.workunit.client.0.smithi032.stdout:4/915: sync 2024-05-12T04:01:42.621 INFO:tasks.workunit.client.0.smithi032.stdout:2/967: sync 2024-05-12T04:01:42.623 INFO:tasks.workunit.client.0.smithi032.stdout:4/916: read d0/f3d [7488994,45843] 0 2024-05-12T04:01:42.624 INFO:tasks.workunit.client.0.smithi032.stdout:2/968: mknod d1/d34/d4a/d86/c13f 0 2024-05-12T04:01:42.625 INFO:tasks.workunit.client.0.smithi032.stdout:2/969: dread - d1/d13/f8f zero size 2024-05-12T04:01:42.625 INFO:tasks.workunit.client.0.smithi032.stdout:9/952: mknod d0/d3/d4b/d6f/c125 0 2024-05-12T04:01:42.626 INFO:tasks.workunit.client.0.smithi032.stdout:9/953: read - d0/d3/d1b/f110 zero size 2024-05-12T04:01:42.627 INFO:tasks.workunit.client.0.smithi032.stdout:2/970: truncate d1/d13/d6b/f13b 489547 0 2024-05-12T04:01:42.631 INFO:tasks.workunit.client.0.smithi032.stdout:9/954: creat d0/d3/d74/d51/dd8/f126 x:0 0 0 2024-05-12T04:01:42.633 INFO:tasks.workunit.client.0.smithi032.stdout:2/971: symlink d1/d4d/d5e/dff/l140 0 2024-05-12T04:01:42.634 INFO:tasks.workunit.client.0.smithi032.stdout:4/917: creat d0/d3/d31/d80/f110 x:0 0 0 2024-05-12T04:01:42.638 INFO:tasks.workunit.client.0.smithi032.stdout:2/972: mknod d1/d34/d96/d41/da2/c141 0 2024-05-12T04:01:42.640 INFO:tasks.workunit.client.0.smithi032.stdout:2/973: write d1/d34/d96/d41/d58/f106 [211258,34929] 0 2024-05-12T04:01:42.643 INFO:tasks.workunit.client.0.smithi032.stdout:9/955: dread d0/d84/d8a/da3/db7/d10c/d111/f53 [0,4194304] 0 2024-05-12T04:01:42.645 INFO:tasks.workunit.client.0.smithi032.stdout:2/974: creat d1/d34/d3e/d67/dda/d130/d120/f142 x:0 0 0 2024-05-12T04:01:42.648 INFO:tasks.workunit.client.0.smithi032.stdout:9/956: dread d0/d3/d74/f20 [0,4194304] 0 2024-05-12T04:01:42.650 INFO:tasks.workunit.client.0.smithi032.stdout:2/975: creat d1/d34/d96/d41/d52/df7/f143 x:0 0 0 2024-05-12T04:01:42.650 INFO:tasks.workunit.client.0.smithi032.stdout:9/957: stat d0/d3/d74/d51/dd8 0 2024-05-12T04:01:42.650 INFO:tasks.workunit.client.0.smithi032.stdout:4/918: dread d0/d3/df/d4e/f92 [0,4194304] 0 2024-05-12T04:01:42.653 INFO:tasks.workunit.client.0.smithi032.stdout:9/958: read d0/d3/d6/d7/d2a/d46/d13/fb6 [725629,94731] 0 2024-05-12T04:01:42.658 INFO:tasks.workunit.client.0.smithi032.stdout:9/959: write d0/d3/d6/d58/fa7 [1480950,86757] 0 2024-05-12T04:01:42.660 INFO:tasks.workunit.client.0.smithi032.stdout:9/960: chown d0/d3/d4b/d6f/dd7/lf6 274026308 1 2024-05-12T04:01:42.664 INFO:tasks.workunit.client.0.smithi032.stdout:4/919: symlink d0/d3/d31/d80/dec/l111 0 2024-05-12T04:01:42.668 INFO:tasks.workunit.client.0.smithi032.stdout:4/920: dread d0/d3/d31/d80/fe5 [0,4194304] 0 2024-05-12T04:01:42.669 INFO:tasks.workunit.client.0.smithi032.stdout:4/921: readlink d0/db5/lee 0 2024-05-12T04:01:42.677 INFO:tasks.workunit.client.0.smithi032.stdout:9/961: mkdir d0/d3/d4b/d88/d99/df8/d127 0 2024-05-12T04:01:42.679 INFO:tasks.workunit.client.0.smithi032.stdout:9/962: truncate d0/d3/f122 965795 0 2024-05-12T04:01:42.682 INFO:tasks.workunit.client.0.smithi032.stdout:9/963: unlink d0/d3/d74/d51/dd8/f126 0 2024-05-12T04:01:42.691 INFO:tasks.workunit.client.0.smithi032.stdout:2/976: write d1/d34/d3e/dee/f107 [557547,117509] 0 2024-05-12T04:01:42.693 INFO:tasks.workunit.client.0.smithi032.stdout:4/922: dread d0/d3/d12/d1d/f4c [0,4194304] 0 2024-05-12T04:01:42.696 INFO:tasks.workunit.client.0.smithi032.stdout:2/977: rmdir d1/d34/d96/d41/d58 39 2024-05-12T04:01:42.700 INFO:tasks.workunit.client.0.smithi032.stdout:4/923: truncate d0/d3/d31/d80/d84/fcc 1085751 0 2024-05-12T04:01:42.704 INFO:tasks.workunit.client.0.smithi032.stdout:4/924: creat d0/dde/f112 x:0 0 0 2024-05-12T04:01:42.708 INFO:tasks.workunit.client.0.smithi032.stdout:2/978: write d1/d34/da5/db3/db7/fcf [630686,72522] 0 2024-05-12T04:01:42.709 INFO:tasks.workunit.client.0.smithi032.stdout:4/925: read - d0/d3/d12/d1d/dc5/feb zero size 2024-05-12T04:01:42.710 INFO:tasks.workunit.client.0.smithi032.stdout:2/979: dread d1/d13/d70/d85/dd5/f10f [0,4194304] 0 2024-05-12T04:01:42.714 INFO:tasks.workunit.client.0.smithi032.stdout:2/980: creat d1/d4d/d5e/d9a/f144 x:0 0 0 2024-05-12T04:01:42.715 INFO:tasks.workunit.client.0.smithi032.stdout:2/981: chown d1/d34/d96/d41/d2c/dd4/f119 126 1 2024-05-12T04:01:42.721 INFO:tasks.workunit.client.0.smithi032.stdout:2/982: creat d1/d13/da0/ded/f145 x:0 0 0 2024-05-12T04:01:42.726 INFO:tasks.workunit.client.0.smithi032.stdout:2/983: read d1/d34/d96/d41/d58/f89 [111445,34248] 0 2024-05-12T04:01:42.727 INFO:tasks.workunit.client.0.smithi032.stdout:2/984: readlink d1/d34/d96/d41/d52/df7/l129 0 2024-05-12T04:01:42.728 INFO:tasks.workunit.client.0.smithi032.stdout:4/926: write d0/d3/f5e [2894062,17195] 0 2024-05-12T04:01:42.732 INFO:tasks.workunit.client.0.smithi032.stdout:4/927: mknod d0/d3/df/c113 0 2024-05-12T04:01:42.734 INFO:tasks.workunit.client.0.smithi032.stdout:2/985: getdents d1/d34/d3e/d67/dda/d130/d120 0 2024-05-12T04:01:42.735 INFO:tasks.workunit.client.0.smithi032.stdout:2/986: readlink d1/d34/d3e/l7b 0 2024-05-12T04:01:42.737 INFO:tasks.workunit.client.0.smithi032.stdout:4/928: fsync d0/d3/d12/f100 0 2024-05-12T04:01:42.738 INFO:tasks.workunit.client.0.smithi032.stdout:2/987: dread d1/f31 [0,4194304] 0 2024-05-12T04:01:42.739 INFO:tasks.workunit.client.0.smithi032.stdout:2/988: read d1/d34/da5/db3/db7/fcf [656697,86252] 0 2024-05-12T04:01:42.740 INFO:tasks.workunit.client.0.smithi032.stdout:2/989: dread d1/f31 [0,4194304] 0 2024-05-12T04:01:42.741 INFO:tasks.workunit.client.0.smithi032.stdout:4/929: readlink d0/l3a 0 2024-05-12T04:01:42.745 INFO:tasks.workunit.client.0.smithi032.stdout:4/930: dread d0/d3/d24/f72 [0,4194304] 0 2024-05-12T04:01:42.748 INFO:tasks.workunit.client.0.smithi032.stdout:2/990: link d1/d34/d3e/f7e d1/d34/d96/d41/d52/f146 0 2024-05-12T04:01:42.749 INFO:tasks.workunit.client.0.smithi032.stdout:4/931: mkdir d0/d3/d12/d1d/d6b/dc2/d114 0 2024-05-12T04:01:42.755 INFO:tasks.workunit.client.0.smithi032.stdout:4/932: creat d0/d3/d12/d1d/d6b/dc2/d114/f115 x:0 0 0 2024-05-12T04:01:42.764 INFO:tasks.workunit.client.0.smithi032.stdout:2/991: dwrite d1/f1e [0,4194304] 0 2024-05-12T04:01:42.765 INFO:tasks.workunit.client.0.smithi032.stdout:2/992: dread - d1/d13/d70/d85/f13a zero size 2024-05-12T04:01:42.773 INFO:tasks.workunit.client.0.smithi032.stdout:4/933: dwrite d0/d3/d31/fe3 [0,4194304] 0 2024-05-12T04:01:42.775 INFO:tasks.workunit.client.0.smithi032.stdout:4/934: stat d0/d3/d12/d1d/d63/d87/d95/fc8 0 2024-05-12T04:01:42.776 INFO:tasks.workunit.client.0.smithi032.stdout:4/935: readlink d0/d3/d12/d1d/d6b/db7/le6 0 2024-05-12T04:01:42.781 INFO:tasks.workunit.client.0.smithi032.stdout:2/993: link d1/c79 d1/d34/d3e/d67/dda/d130/c147 0 2024-05-12T04:01:42.782 INFO:tasks.workunit.client.0.smithi032.stdout:2/994: chown d1/ca9 5102779 1 2024-05-12T04:01:42.785 INFO:tasks.workunit.client.0.smithi032.stdout:2/995: creat d1/d34/d3e/d67/dda/d130/f148 x:0 0 0 2024-05-12T04:01:42.801 INFO:tasks.workunit.client.0.smithi032.stdout:2/996: dwrite d1/d34/d4a/d86/fb1 [0,4194304] 0 2024-05-12T04:01:42.803 INFO:tasks.workunit.client.0.smithi032.stdout:4/936: dread d0/db5/fbc [0,4194304] 0 2024-05-12T04:01:42.805 INFO:tasks.workunit.client.0.smithi032.stdout:4/937: creat d0/d3/d12/d1d/d6b/dc2/d114/f116 x:0 0 0 2024-05-12T04:01:42.807 INFO:tasks.workunit.client.0.smithi032.stdout:4/938: symlink d0/d3/d12/d8d/l117 0 2024-05-12T04:01:42.853 INFO:tasks.workunit.client.0.smithi032.stdout:4/939: mkdir d0/dde/d118 0 2024-05-12T04:01:42.854 INFO:tasks.workunit.client.0.smithi032.stdout:4/940: creat d0/d3/d12/d8d/da1/f119 x:0 0 0 2024-05-12T04:01:42.857 INFO:tasks.workunit.client.0.smithi032.stdout:4/941: mkdir d0/d3/d12/d1d/d63/d87/dc6/d11a 0 2024-05-12T04:01:42.867 INFO:tasks.workunit.client.0.smithi032.stdout:4/942: mkdir d0/d3/d12/d1d/d6b/db7/d11b 0 2024-05-12T04:01:42.877 INFO:tasks.workunit.client.0.smithi032.stdout:4/943: write d0/d3/d12/d1d/dc5/feb [432753,15173] 0 2024-05-12T04:01:42.885 INFO:tasks.workunit.client.0.smithi032.stdout:4/944: getdents d0/d3/d31 0 2024-05-12T04:01:42.887 INFO:tasks.workunit.client.0.smithi032.stdout:4/945: creat d0/d3/d12/d1d/d6b/dc2/d114/f11c x:0 0 0 2024-05-12T04:01:42.893 INFO:tasks.workunit.client.0.smithi032.stdout:4/946: unlink d0/d3/d12/d8d/da1/l10f 0 2024-05-12T04:01:42.893 INFO:tasks.workunit.client.0.smithi032.stdout:4/947: dread - d0/d3/d12/d1d/d63/f97 zero size 2024-05-12T04:01:42.896 INFO:tasks.workunit.client.0.smithi032.stdout:4/948: symlink d0/d3/d12/d1d/d63/d87/dc6/l11d 0 2024-05-12T04:01:42.898 INFO:tasks.workunit.client.0.smithi032.stdout:4/949: creat d0/d3/d12/d1d/d63/d87/dc6/d11a/f11e x:0 0 0 2024-05-12T04:01:42.903 INFO:tasks.workunit.client.0.smithi032.stdout:4/950: truncate d0/d3/d31/f8b 463525 0 2024-05-12T04:01:42.909 INFO:tasks.workunit.client.0.smithi032.stdout:4/951: dread d0/d3/d12/d1d/f5d [0,4194304] 0 2024-05-12T04:01:42.912 INFO:tasks.workunit.client.0.smithi032.stdout:4/952: mkdir d0/d3/d12/d1d/d11f 0 2024-05-12T04:01:42.916 INFO:tasks.workunit.client.0.smithi032.stdout:4/953: dread d0/dde/ff9 [0,4194304] 0 2024-05-12T04:01:42.934 INFO:tasks.workunit.client.0.smithi032.stdout:4/954: dwrite d0/db5/fbc [0,4194304] 0 2024-05-12T04:01:42.940 INFO:tasks.workunit.client.0.smithi032.stdout:4/955: dread d0/db5/fbc [0,4194304] 0 2024-05-12T04:01:42.941 INFO:tasks.workunit.client.0.smithi032.stdout:4/956: chown d0/d3/d31/f35 4316 1 2024-05-12T04:01:42.952 INFO:tasks.workunit.client.0.smithi032.stdout:4/957: write d0/d3/d24/f72 [1583400,2033] 0 2024-05-12T04:01:42.969 INFO:tasks.workunit.client.0.smithi032.stdout:4/958: dread d0/d3/d31/d80/d84/fa9 [0,4194304] 0 2024-05-12T04:01:42.986 INFO:tasks.workunit.client.0.smithi032.stdout:7/892: sync 2024-05-12T04:01:42.986 INFO:tasks.workunit.client.0.smithi032.stdout:9/964: sync 2024-05-12T04:01:42.994 INFO:tasks.workunit.client.0.smithi032.stdout:7/893: creat dd/d1b/d1e/d45/d55/f113 x:0 0 0 2024-05-12T04:01:43.000 INFO:tasks.workunit.client.0.smithi032.stdout:7/894: creat dd/d1b/d1e/de3/d9a/d9c/dc2/f114 x:0 0 0 2024-05-12T04:01:43.001 INFO:tasks.workunit.client.0.smithi032.stdout:9/965: dread d0/d3/d4b/d88/d99/df8/ffc [0,4194304] 0 2024-05-12T04:01:43.008 INFO:tasks.workunit.client.0.smithi032.stdout:9/966: truncate d0/d3/d6/d7/d2a/f68 1662314 0 2024-05-12T04:01:43.011 INFO:tasks.workunit.client.0.smithi032.stdout:9/967: creat d0/d84/d8a/da3/f128 x:0 0 0 2024-05-12T04:01:43.012 INFO:tasks.workunit.client.0.smithi032.stdout:7/895: getdents dd/d6c/dc3 0 2024-05-12T04:01:43.012 INFO:tasks.workunit.client.0.smithi032.stdout:9/968: fsync d0/d84/f105 0 2024-05-12T04:01:43.014 INFO:tasks.workunit.client.0.smithi032.stdout:7/896: chown dd/d6c/df8/c101 168240 1 2024-05-12T04:01:43.015 INFO:tasks.workunit.client.0.smithi032.stdout:7/897: stat dd/d1b/d1e/d45/d55/d72/da0/dbd 0 2024-05-12T04:01:43.017 INFO:tasks.workunit.client.0.smithi032.stdout:9/969: creat d0/d3/d4b/d85/d123/f129 x:0 0 0 2024-05-12T04:01:43.018 INFO:tasks.workunit.client.0.smithi032.stdout:7/898: creat dd/d1b/d43/d5e/f115 x:0 0 0 2024-05-12T04:01:43.020 INFO:tasks.workunit.client.0.smithi032.stdout:7/899: creat dd/d1b/f116 x:0 0 0 2024-05-12T04:01:43.040 INFO:tasks.workunit.client.0.smithi032.stdout:7/900: link dd/d1b/d1e/f20 dd/d6c/f117 0 2024-05-12T04:01:43.045 INFO:tasks.workunit.client.0.smithi032.stdout:7/901: rename dd/d1b/d43/c51 to dd/d6c/df8/c118 0 2024-05-12T04:01:43.048 INFO:tasks.workunit.client.0.smithi032.stdout:7/902: unlink dd/d6c/c103 0 2024-05-12T04:01:43.051 INFO:tasks.workunit.client.0.smithi032.stdout:7/903: creat dd/d1b/d1e/d45/d55/d72/da0/dbd/f119 x:0 0 0 2024-05-12T04:01:43.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:42 smithi033 ceph-mon[49525]: pgmap v49: 65 pgs: 65 active+clean; 4.0 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 93 MiB/s rd, 204 MiB/s wr, 506 op/s 2024-05-12T04:01:43.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:42 smithi033 ceph-mon[49525]: from='client.24495 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:01:43.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:42 smithi033 ceph-mon[49525]: mgrmap e26: smithi033.pcvxwy(active, since 92s), standbys: smithi032.ukatkq 2024-05-12T04:01:43.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:42 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/2808451881' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-12T04:01:43.250 INFO:tasks.workunit.client.0.smithi032.stdout:2/997: sync 2024-05-12T04:01:43.250 INFO:tasks.workunit.client.0.smithi032.stdout:2/998: stat d1/l35 0 2024-05-12T04:01:43.251 INFO:tasks.workunit.client.0.smithi032.stdout:2/999: write d1/d73/d128/f137 [303691,52081] 0 2024-05-12T04:01:43.284 INFO:tasks.workunit.client.0.smithi032.stdout:9/970: sync 2024-05-12T04:01:43.284 INFO:tasks.workunit.client.0.smithi032.stdout:7/904: sync 2024-05-12T04:01:43.284 INFO:tasks.workunit.client.0.smithi032.stdout:4/959: sync 2024-05-12T04:01:43.286 INFO:tasks.workunit.client.0.smithi032.stdout:4/960: write d0/d3/d12/d8d/f109 [644596,2079] 0 2024-05-12T04:01:43.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:42 smithi032 ceph-mon[40008]: pgmap v49: 65 pgs: 65 active+clean; 4.0 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 93 MiB/s rd, 204 MiB/s wr, 506 op/s 2024-05-12T04:01:43.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:42 smithi032 ceph-mon[40008]: from='client.24495 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:01:43.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:42 smithi032 ceph-mon[40008]: mgrmap e26: smithi033.pcvxwy(active, since 92s), standbys: smithi032.ukatkq 2024-05-12T04:01:43.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:42 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/2808451881' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-12T04:01:43.291 INFO:tasks.workunit.client.0.smithi032.stdout:9/971: mknod d0/d3/d6/d7/d2a/d46/d91/dc5/de5/d100/c12a 0 2024-05-12T04:01:43.292 INFO:tasks.workunit.client.0.smithi032.stdout:4/961: creat d0/dde/d118/f120 x:0 0 0 2024-05-12T04:01:43.293 INFO:tasks.workunit.client.0.smithi032.stdout:9/972: chown d0/d3/d6/d7/d2a/lae 2279508 1 2024-05-12T04:01:43.295 INFO:tasks.workunit.client.0.smithi032.stdout:7/905: link dd/d1b/d1e/d45/d55/d72/da0/dbd/f104 dd/d1b/d1e/f11a 0 2024-05-12T04:01:43.342 INFO:tasks.workunit.client.0.smithi032.stdout:9/973: symlink d0/d3/d4b/d88/d99/df8/d127/l12b 0 2024-05-12T04:01:43.346 INFO:tasks.workunit.client.0.smithi032.stdout:9/974: mkdir d0/d3/d4b/d88/d12c 0 2024-05-12T04:01:43.347 INFO:tasks.workunit.client.0.smithi032.stdout:9/975: dread d0/d3/d74/fc8 [0,4194304] 0 2024-05-12T04:01:43.452 INFO:tasks.workunit.client.0.smithi032.stdout:7/906: dwrite dd/d1b/d1e/d45/d55/f9d [0,4194304] 0 2024-05-12T04:01:43.506 INFO:tasks.workunit.client.0.smithi032.stdout:7/907: creat dd/d1b/d1e/d45/d55/db7/f11b x:0 0 0 2024-05-12T04:01:43.508 INFO:tasks.workunit.client.0.smithi032.stdout:9/976: getdents d0/d3/d4b/d85/dbe 0 2024-05-12T04:01:43.510 INFO:tasks.workunit.client.0.smithi032.stdout:9/977: chown d0/d3/d4b/d88/d99/df8/ffc 6079 1 2024-05-12T04:01:43.511 INFO:tasks.workunit.client.0.smithi032.stdout:9/978: fsync d0/d3/d74/d51/f102 0 2024-05-12T04:01:43.513 INFO:tasks.workunit.client.0.smithi032.stdout:9/979: write d0/d84/d8a/da3/db7/d10c/d111/d89/f11b [3677018,44820] 0 2024-05-12T04:01:43.514 INFO:tasks.workunit.client.0.smithi032.stdout:9/980: chown d0/d3/d6/d7/d106 5256214 1 2024-05-12T04:01:43.516 INFO:tasks.workunit.client.0.smithi032.stdout:9/981: truncate d0/d3/d6/d7/d2a/d9f/fa1 4429636 0 2024-05-12T04:01:43.518 INFO:tasks.workunit.client.0.smithi032.stdout:9/982: chown d0/d3/d6/d7/d2a/d46/d91/dc5/leb 21160 1 2024-05-12T04:01:43.519 INFO:tasks.workunit.client.0.smithi032.stdout:4/962: sync 2024-05-12T04:01:43.521 INFO:tasks.workunit.client.0.smithi032.stdout:4/963: dread - d0/d3/fe2 zero size 2024-05-12T04:01:43.526 INFO:tasks.workunit.client.0.smithi032.stdout:4/964: creat d0/d3/d31/d80/d84/f121 x:0 0 0 2024-05-12T04:01:43.537 INFO:tasks.workunit.client.0.smithi032.stdout:9/983: dread d0/d3/d6/d7/d2a/d46/d13/f56 [4194304,4194304] 0 2024-05-12T04:01:43.542 INFO:tasks.workunit.client.0.smithi032.stdout:9/984: creat d0/d3/d4b/d85/dbe/f12d x:0 0 0 2024-05-12T04:01:43.546 INFO:tasks.workunit.client.0.smithi032.stdout:7/908: dwrite dd/d6c/dc3/fc6 [0,4194304] 0 2024-05-12T04:01:43.553 INFO:tasks.workunit.client.0.smithi032.stdout:9/985: mknod d0/d3/d4b/c12e 0 2024-05-12T04:01:43.553 INFO:tasks.workunit.client.0.smithi032.stdout:4/965: dwrite d0/d3/d12/faa [0,4194304] 0 2024-05-12T04:01:43.556 INFO:tasks.workunit.client.0.smithi032.stdout:7/909: dread - dd/d1b/d43/fa8 zero size 2024-05-12T04:01:43.570 INFO:tasks.workunit.client.0.smithi032.stdout:4/966: write d0/d3/d24/dbd/fd2 [1029715,107204] 0 2024-05-12T04:01:43.573 INFO:tasks.workunit.client.0.smithi032.stdout:7/910: dwrite fa [4194304,4194304] 0 2024-05-12T04:01:43.574 INFO:tasks.workunit.client.0.smithi032.stdout:9/986: dwrite d0/d3/d74/d51/f77 [0,4194304] 0 2024-05-12T04:01:43.578 INFO:tasks.workunit.client.0.smithi032.stdout:7/911: readlink dd/d1b/d43/d74/l8e 0 2024-05-12T04:01:43.582 INFO:tasks.workunit.client.0.smithi032.stdout:4/967: rmdir d0/d3/d24 39 2024-05-12T04:01:43.587 INFO:tasks.workunit.client.0.smithi032.stdout:7/912: rename dd/d1b/d43/d5e/c112 to dd/d1b/d1e/de3/dfd/c11c 0 2024-05-12T04:01:43.594 INFO:tasks.workunit.client.0.smithi032.stdout:4/968: link d0/d3/d12/f5b d0/d3/d24/f122 0 2024-05-12T04:01:43.598 INFO:tasks.workunit.client.0.smithi032.stdout:9/987: truncate d0/d3/d6/d7/f15 756087 0 2024-05-12T04:01:43.602 INFO:tasks.workunit.client.0.smithi032.stdout:7/913: link dd/d1b/d43/d5e/d37/f41 dd/d1b/d1e/de3/d9a/d9c/db9/d75/f11d 0 2024-05-12T04:01:43.602 INFO:tasks.workunit.client.0.smithi032.stdout:4/969: creat d0/d3/d12/d1d/f123 x:0 0 0 2024-05-12T04:01:43.605 INFO:tasks.workunit.client.0.smithi032.stdout:4/970: truncate d0/d75/f8a 5136299 0 2024-05-12T04:01:43.616 INFO:tasks.workunit.client.0.smithi032.stdout:9/988: dwrite d0/d3/d74/f78 [0,4194304] 0 2024-05-12T04:01:43.617 INFO:tasks.workunit.client.0.smithi032.stdout:4/971: link d0/f3d d0/d3/d12/d1d/f124 0 2024-05-12T04:01:43.624 INFO:tasks.workunit.client.0.smithi032.stdout:9/989: dread d0/d3/d74/fc8 [0,4194304] 0 2024-05-12T04:01:43.629 INFO:tasks.workunit.client.0.smithi032.stdout:4/972: unlink d0/d3/d12/d1d/d6b/dc2/d114/f116 0 2024-05-12T04:01:43.630 INFO:tasks.workunit.client.0.smithi032.stdout:4/973: write d0/d3/d12/d8d/da1/f119 [514217,36559] 0 2024-05-12T04:01:43.635 INFO:tasks.workunit.client.0.smithi032.stdout:9/990: rename d0/d84 to d0/d3/d4b/d88/d119/d12f 0 2024-05-12T04:01:43.640 INFO:tasks.workunit.client.0.smithi032.stdout:4/974: stat d0/f62 0 2024-05-12T04:01:43.644 INFO:tasks.workunit.client.0.smithi032.stdout:9/991: unlink d0/d3/d4b/d85/dbe/d82/l113 0 2024-05-12T04:01:43.645 INFO:tasks.workunit.client.0.smithi032.stdout:4/975: fdatasync d0/d3/df/d4e/f69 0 2024-05-12T04:01:43.646 INFO:tasks.workunit.client.0.smithi032.stdout:4/976: stat d0/d3/d12/d1d/d63/d87/dc6/l10d 0 2024-05-12T04:01:43.651 INFO:tasks.workunit.client.0.smithi032.stdout:4/977: mknod d0/d3/d12/d8d/da1/c125 0 2024-05-12T04:01:43.653 INFO:tasks.workunit.client.0.smithi032.stdout:9/992: creat d0/d3/d74/dfe/f130 x:0 0 0 2024-05-12T04:01:43.654 INFO:tasks.workunit.client.0.smithi032.stdout:9/993: chown d0/d3/d4b/d85/dbe 6716 1 2024-05-12T04:01:43.658 INFO:tasks.workunit.client.0.smithi032.stdout:9/994: symlink d0/d3/d4b/d6f/dd7/l131 0 2024-05-12T04:01:43.659 INFO:tasks.workunit.client.0.smithi032.stdout:9/995: chown d0/d3/d4b/d88/d119 8207455 1 2024-05-12T04:01:43.665 INFO:tasks.workunit.client.0.smithi032.stdout:9/996: creat d0/d3/d4b/d85/dc9/f132 x:0 0 0 2024-05-12T04:01:43.668 INFO:tasks.workunit.client.0.smithi032.stdout:9/997: fsync d0/d3/d4b/d88/d99/df1/ff5 0 2024-05-12T04:01:43.673 INFO:tasks.workunit.client.0.smithi032.stdout:4/978: dwrite d0/d75/f8a [0,4194304] 0 2024-05-12T04:01:43.673 INFO:tasks.workunit.client.0.smithi032.stdout:9/998: mknod d0/d3/d4b/d85/dbe/c133 0 2024-05-12T04:01:43.677 INFO:tasks.workunit.client.0.smithi032.stdout:4/979: dread - d0/d3/d24/fef zero size 2024-05-12T04:01:43.677 INFO:tasks.workunit.client.0.smithi032.stdout:4/980: chown d0/d3/df/d4e/f69 14151 1 2024-05-12T04:01:43.683 INFO:tasks.workunit.client.0.smithi032.stdout:9/999: write d0/d3/d6/d7/d2a/d46/d13/fb6 [1881369,72287] 0 2024-05-12T04:01:43.684 INFO:tasks.workunit.client.0.smithi032.stdout:4/981: write d0/d3/d31/f35 [2890882,62071] 0 2024-05-12T04:01:43.686 INFO:tasks.workunit.client.0.smithi032.stdout:4/982: creat d0/d3/d12/d1d/d6b/db7/de0/f126 x:0 0 0 2024-05-12T04:01:43.689 INFO:tasks.workunit.client.0.smithi032.stdout:4/983: fsync d0/d3/d12/fa2 0 2024-05-12T04:01:43.692 INFO:tasks.workunit.client.0.smithi032.stdout:4/984: creat d0/d3/d12/d1d/d6b/f127 x:0 0 0 2024-05-12T04:01:43.845 INFO:tasks.workunit.client.0.smithi032.stdout:4/985: chown d0/d3/f88 1019143015 1 2024-05-12T04:01:43.865 INFO:tasks.workunit.client.0.smithi032.stdout:4/986: dread d0/d3/d12/d1d/fba [0,4194304] 0 2024-05-12T04:01:43.870 INFO:tasks.workunit.client.0.smithi032.stdout:4/987: rename d0/d3/d12/d1d/d63/d87/d95/cd1 to d0/d3/d12/d1d/d6b/dc2/dc4/c128 0 2024-05-12T04:01:43.876 INFO:tasks.workunit.client.0.smithi032.stdout:4/988: rename d0/d3/d31/d80/d84/l59 to d0/dde/d118/l129 0 2024-05-12T04:01:43.879 INFO:tasks.workunit.client.0.smithi032.stdout:7/914: sync 2024-05-12T04:01:43.885 INFO:tasks.workunit.client.0.smithi032.stdout:4/989: dwrite d0/d3/d12/d1d/d63/d87/dc6/d11a/f11e [0,4194304] 0 2024-05-12T04:01:43.892 INFO:tasks.workunit.client.0.smithi032.stdout:4/990: rmdir d0/d3/d31/d80/d84/da7 39 2024-05-12T04:01:43.900 INFO:tasks.workunit.client.0.smithi032.stdout:4/991: dwrite d0/d3/d12/d1d/d63/d87/ffa [0,4194304] 0 2024-05-12T04:01:43.906 INFO:tasks.workunit.client.0.smithi032.stdout:4/992: dread d0/d3/d24/dbd/fd2 [0,4194304] 0 2024-05-12T04:01:43.919 INFO:tasks.workunit.client.0.smithi032.stdout:4/993: getdents d0/d3/d12 0 2024-05-12T04:01:43.920 INFO:tasks.workunit.client.0.smithi032.stdout:4/994: truncate d0/d3/d24/f76 1076585 0 2024-05-12T04:01:43.930 INFO:tasks.workunit.client.0.smithi032.stdout:4/995: creat d0/d3/d31/d80/d84/f12a x:0 0 0 2024-05-12T04:01:43.934 INFO:tasks.workunit.client.0.smithi032.stdout:4/996: mkdir d0/d3/df/dac/d12b 0 2024-05-12T04:01:43.938 INFO:tasks.workunit.client.0.smithi032.stdout:4/997: dread d0/d3/d12/d1d/dc5/fcf [0,4194304] 0 2024-05-12T04:01:43.941 INFO:tasks.workunit.client.0.smithi032.stdout:4/998: mknod d0/d3/d12/d1d/d63/d87/dc6/c12c 0 2024-05-12T04:01:43.943 INFO:tasks.workunit.client.0.smithi032.stdout:4/999: truncate d0/d3/d24/dae/fdc 5096060 0 2024-05-12T04:01:44.038 INFO:tasks.workunit.client.0.smithi032.stdout:7/915: dwrite dd/f12 [0,4194304] 0 2024-05-12T04:01:44.040 INFO:tasks.workunit.client.0.smithi032.stdout:7/916: write fa [5059757,88493] 0 2024-05-12T04:01:44.046 INFO:tasks.workunit.client.0.smithi032.stdout:7/917: fdatasync dd/d1b/d43/d74/f7f 0 2024-05-12T04:01:44.050 INFO:tasks.workunit.client.0.smithi032.stdout:7/918: truncate dd/d1b/d1e/faa 1357841 0 2024-05-12T04:01:44.053 INFO:tasks.workunit.client.0.smithi032.stdout:7/919: write dd/d1b/d1e/d45/d55/db7/df6/f108 [827168,119936] 0 2024-05-12T04:01:44.054 INFO:tasks.workunit.client.0.smithi032.stdout:7/920: dread - dd/d1b/d1e/de3/d9a/ffc zero size 2024-05-12T04:01:44.057 INFO:tasks.workunit.client.0.smithi032.stdout:7/921: chown dd/d1b/d1e/d45/d64/f95 6286 1 2024-05-12T04:01:44.117 INFO:tasks.workunit.client.0.smithi032.stdout:7/922: dwrite dd/d1b/d1e/de3/fb5 [0,4194304] 0 2024-05-12T04:01:44.154 INFO:tasks.workunit.client.0.smithi032.stdout:7/923: dwrite dd/d6c/dc3/dcd/fe2 [0,4194304] 0 2024-05-12T04:01:44.160 INFO:tasks.workunit.client.0.smithi032.stdout:7/924: mknod dd/d1b/d1e/de3/d9a/d9c/dc2/c11e 0 2024-05-12T04:01:44.165 INFO:tasks.workunit.client.0.smithi032.stdout:7/925: write dd/d1b/d1e/d45/d55/db7/df6/f108 [611622,78060] 0 2024-05-12T04:01:44.167 INFO:tasks.workunit.client.0.smithi032.stdout:7/926: write dd/d1b/d1e/fd1 [1155646,20170] 0 2024-05-12T04:01:44.786 INFO:tasks.workunit.client.0.smithi032.stdout:7/927: sync 2024-05-12T04:01:44.789 INFO:tasks.workunit.client.0.smithi032.stdout:7/928: symlink dd/d1b/d1e/d45/l11f 0 2024-05-12T04:01:45.083 INFO:tasks.workunit.client.0.smithi032.stdout:7/929: sync 2024-05-12T04:01:45.106 INFO:tasks.workunit.client.0.smithi032.stdout:7/930: write dd/d1b/d1e/de3/d9a/d9c/db9/f88 [680752,26044] 0 2024-05-12T04:01:45.111 INFO:tasks.workunit.client.0.smithi032.stdout:7/931: mkdir dd/d1b/d1e/d45/d55/d72/da0/dbd/db2/d120 0 2024-05-12T04:01:45.114 INFO:tasks.workunit.client.0.smithi032.stdout:7/932: creat dd/d1b/d1e/d45/d55/d72/da0/dbd/f121 x:0 0 0 2024-05-12T04:01:45.117 INFO:tasks.workunit.client.0.smithi032.stdout:7/933: unlink dd/d1b/d1e/d45/cdb 0 2024-05-12T04:01:45.134 INFO:tasks.workunit.client.0.smithi032.stdout:7/934: write dd/f1d [542330,99146] 0 2024-05-12T04:01:45.134 INFO:tasks.workunit.client.0.smithi032.stdout:7/935: dread - dd/d1b/d1e/de3/f107 zero size 2024-05-12T04:01:45.141 INFO:tasks.workunit.client.0.smithi032.stdout:7/936: rename dd/d1b/d1e/d45/d55/db7/df6 to dd/d1b/d1e/de3/d9a/d9c/db9/d75/d122 0 2024-05-12T04:01:45.146 INFO:tasks.workunit.client.0.smithi032.stdout:7/937: dread dd/d1b/d1e/de3/d9a/d9c/db9/d75/d122/f108 [0,4194304] 0 2024-05-12T04:01:45.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:44 smithi033 ceph-mon[49525]: pgmap v50: 65 pgs: 65 active+clean; 4.0 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 59 MiB/s rd, 115 MiB/s wr, 316 op/s 2024-05-12T04:01:45.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:44 smithi032 ceph-mon[40008]: pgmap v50: 65 pgs: 65 active+clean; 4.0 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 59 MiB/s rd, 115 MiB/s wr, 316 op/s 2024-05-12T04:01:45.409 INFO:tasks.workunit.client.0.smithi032.stdout:7/938: sync 2024-05-12T04:01:45.413 INFO:tasks.workunit.client.0.smithi032.stdout:7/939: read dd/f82 [332245,109868] 0 2024-05-12T04:01:45.415 INFO:tasks.workunit.client.0.smithi032.stdout:7/940: write dd/d1b/d1e/de3/d9a/ffc [1026719,14132] 0 2024-05-12T04:01:45.479 INFO:tasks.workunit.client.0.smithi032.stdout:7/941: dread dd/d1b/d43/d5e/d37/f41 [0,4194304] 0 2024-05-12T04:01:45.493 INFO:tasks.workunit.client.0.smithi032.stdout:7/942: dread dd/d1b/d43/d5e/f4c [0,4194304] 0 2024-05-12T04:01:45.494 INFO:tasks.workunit.client.0.smithi032.stdout:7/943: chown dd/d1b/d1e/de3/d9a/d9c/dc2 2895421 1 2024-05-12T04:01:45.498 INFO:tasks.workunit.client.0.smithi032.stdout:7/944: dread dd/d1b/d1e/d45/d55/d72/da0/dbd/db2/f110 [0,4194304] 0 2024-05-12T04:01:45.500 INFO:tasks.workunit.client.0.smithi032.stdout:7/945: creat dd/d1b/d1e/d45/d55/d72/f123 x:0 0 0 2024-05-12T04:01:45.505 INFO:tasks.workunit.client.0.smithi032.stdout:7/946: rename dd/d1b/d43/cc5 to dd/d1b/d1e/de3/c124 0 2024-05-12T04:01:45.562 INFO:tasks.workunit.client.0.smithi032.stdout:7/947: write dd/d1b/d1e/d45/d55/d72/da0/dbd/db2/fe7 [3495920,55811] 0 2024-05-12T04:01:45.582 INFO:tasks.workunit.client.0.smithi032.stdout:7/948: write dd/d1b/d1e/f38 [1672485,37465] 0 2024-05-12T04:01:45.583 INFO:tasks.workunit.client.0.smithi032.stdout:7/949: stat dd/d1b/d1e/de3/d9a/c100 0 2024-05-12T04:01:45.590 INFO:tasks.workunit.client.0.smithi032.stdout:7/950: dwrite dd/d1b/d1e/d45/d55/db7/fd5 [0,4194304] 0 2024-05-12T04:01:45.600 INFO:tasks.workunit.client.0.smithi032.stdout:7/951: dwrite dd/d1b/d1e/d45/d55/db7/fd5 [0,4194304] 0 2024-05-12T04:01:45.611 INFO:tasks.workunit.client.0.smithi032.stdout:7/952: unlink dd/d1b/d1e/d45/fd9 0 2024-05-12T04:01:45.620 INFO:tasks.workunit.client.0.smithi032.stdout:7/953: write dd/d1b/d43/d5e/f34 [3961638,54187] 0 2024-05-12T04:01:45.623 INFO:tasks.workunit.client.0.smithi032.stdout:7/954: mknod dd/d1b/d43/d5e/c125 0 2024-05-12T04:01:45.624 INFO:tasks.workunit.client.0.smithi032.stdout:7/955: dread - dd/d1b/d1e/d45/d55/d72/da0/dbd/f104 zero size 2024-05-12T04:01:45.625 INFO:tasks.workunit.client.0.smithi032.stdout:7/956: chown dd/d1b/d1e/de3/d9a/d9c/dc2/c11e 3954539 1 2024-05-12T04:01:45.626 INFO:tasks.workunit.client.0.smithi032.stdout:7/957: chown dd/d1b/d1e/de3/d9a/d9c/db9/d75/d122 8276 1 2024-05-12T04:01:45.629 INFO:tasks.workunit.client.0.smithi032.stdout:7/958: truncate dd/d1b/d43/fe6 260536 0 2024-05-12T04:01:45.634 INFO:tasks.workunit.client.0.smithi032.stdout:7/959: link dd/d1b/d1e/de3/d9a/d9c/db9/d75/fad dd/f126 0 2024-05-12T04:01:45.637 INFO:tasks.workunit.client.0.smithi032.stdout:7/960: mknod dd/d1b/d1e/d45/d55/d72/d10c/c127 0 2024-05-12T04:01:45.642 INFO:tasks.workunit.client.0.smithi032.stdout:7/961: dread dd/d1b/d1e/d45/d64/f95 [0,4194304] 0 2024-05-12T04:01:45.657 INFO:tasks.workunit.client.0.smithi032.stdout:7/962: dwrite dd/d1b/d43/d5e/f67 [0,4194304] 0 2024-05-12T04:01:45.669 INFO:tasks.workunit.client.0.smithi032.stdout:7/963: write dd/d1b/d1e/d45/d55/f5b [132719,26104] 0 2024-05-12T04:01:45.682 INFO:tasks.workunit.client.0.smithi032.stdout:7/964: write dd/d1b/f7a [972001,67888] 0 2024-05-12T04:01:45.686 INFO:tasks.workunit.client.0.smithi032.stdout:7/965: rename dd/d1b/d1e/d45/d64/f10e to dd/d1b/d43/d5e/f128 0 2024-05-12T04:01:46.204 INFO:tasks.workunit.client.0.smithi032.stdout:7/966: sync 2024-05-12T04:01:46.204 INFO:tasks.workunit.client.0.smithi032.stdout:7/967: readlink dd/l2e 0 2024-05-12T04:01:46.205 INFO:tasks.workunit.client.0.smithi032.stdout:7/968: write dd/d1b/d1e/d45/d64/f8b [2935727,59099] 0 2024-05-12T04:01:46.212 INFO:tasks.workunit.client.0.smithi032.stdout:7/969: link dd/d1b/d1e/d45/ld7 dd/d1b/d43/d5e/l129 0 2024-05-12T04:01:46.213 INFO:tasks.workunit.client.0.smithi032.stdout:7/970: fdatasync dd/d1b/d1e/d45/d55/f113 0 2024-05-12T04:01:46.214 INFO:tasks.workunit.client.0.smithi032.stdout:7/971: dread dd/d1b/d43/d5e/d37/f41 [0,4194304] 0 2024-05-12T04:01:46.218 INFO:tasks.workunit.client.0.smithi032.stdout:7/972: dread dd/d1b/d43/d5e/f67 [0,4194304] 0 2024-05-12T04:01:46.222 INFO:tasks.workunit.client.0.smithi032.stdout:7/973: creat dd/d1b/d1e/de3/d9a/d9c/dc2/f12a x:0 0 0 2024-05-12T04:01:46.223 INFO:tasks.workunit.client.0.smithi032.stdout:7/974: chown dd/d1b/d43/d5e/d37/f41 0 1 2024-05-12T04:01:46.229 INFO:tasks.workunit.client.0.smithi032.stdout:7/975: creat dd/d1b/d106/f12b x:0 0 0 2024-05-12T04:01:46.230 INFO:tasks.workunit.client.0.smithi032.stdout:7/976: chown dd/d1b/d1e/de3/d9a/d9c/db9/f6b 8052 1 2024-05-12T04:01:46.233 INFO:tasks.workunit.client.0.smithi032.stdout:7/977: mknod dd/d1b/d43/d5e/c12c 0 2024-05-12T04:01:46.237 INFO:tasks.workunit.client.0.smithi032.stdout:7/978: creat dd/d1b/d1e/de3/d9a/d9c/db9/d75/d122/f12d x:0 0 0 2024-05-12T04:01:46.242 INFO:tasks.workunit.client.0.smithi032.stdout:7/979: truncate fb 253785 0 2024-05-12T04:01:46.248 INFO:tasks.workunit.client.0.smithi032.stdout:7/980: dwrite dd/d6c/dc3/fc6 [0,4194304] 0 2024-05-12T04:01:46.257 INFO:tasks.workunit.client.0.smithi032.stdout:7/981: write dd/d1b/d1e/f28 [3779362,21378] 0 2024-05-12T04:01:46.260 INFO:tasks.workunit.client.0.smithi032.stdout:7/982: creat dd/d6c/dc3/dcd/f12e x:0 0 0 2024-05-12T04:01:46.262 INFO:tasks.workunit.client.0.smithi032.stdout:7/983: write dd/d1b/d1e/d45/d55/db7/fd5 [2180416,65392] 0 2024-05-12T04:01:46.266 INFO:tasks.workunit.client.0.smithi032.stdout:7/984: mkdir dd/d1b/d43/d5e/d37/d12f 0 2024-05-12T04:01:46.270 INFO:tasks.workunit.client.0.smithi032.stdout:7/985: mkdir dd/d1b/d1e/d130 0 2024-05-12T04:01:46.272 INFO:tasks.workunit.client.0.smithi032.stdout:7/986: read dd/d1b/d43/d5e/f69 [406535,112057] 0 2024-05-12T04:01:46.274 INFO:tasks.workunit.client.0.smithi032.stdout:7/987: symlink dd/d1b/d43/l131 0 2024-05-12T04:01:46.276 INFO:tasks.workunit.client.0.smithi032.stdout:7/988: mknod dd/d6c/df8/c132 0 2024-05-12T04:01:46.278 INFO:tasks.workunit.client.0.smithi032.stdout:7/989: mknod dd/d1b/d1e/d45/d55/d72/da0/c133 0 2024-05-12T04:01:46.306 INFO:tasks.workunit.client.0.smithi032.stdout:7/990: dwrite f5 [8388608,4194304] 0 2024-05-12T04:01:46.310 INFO:tasks.workunit.client.0.smithi032.stdout:7/991: rename dd/d1b/d1e/d45/d55/d72/cbe to dd/d1b/d1e/de3/d9a/c134 0 2024-05-12T04:01:46.312 INFO:tasks.workunit.client.0.smithi032.stdout:7/992: mkdir dd/d6c/dc3/dda/d135 0 2024-05-12T04:01:46.324 INFO:tasks.workunit.client.0.smithi032.stdout:7/993: write dd/d1b/d43/d5e/f5a [3831101,109280] 0 2024-05-12T04:01:46.328 INFO:tasks.workunit.client.0.smithi032.stdout:7/994: mknod dd/d1b/d1e/de3/d9a/d9c/def/c136 0 2024-05-12T04:01:46.646 INFO:tasks.workunit.client.0.smithi032.stdout:7/995: sync 2024-05-12T04:01:46.654 INFO:tasks.workunit.client.0.smithi032.stdout:7/996: dwrite dd/d1b/d43/d5e/f6f [0,4194304] 0 2024-05-12T04:01:46.662 INFO:tasks.workunit.client.0.smithi032.stdout:7/997: getdents dd/d1b/d43/d5e/df0 0 2024-05-12T04:01:46.675 INFO:tasks.workunit.client.0.smithi032.stdout:7/998: dwrite dd/d6c/f105 [0,4194304] 0 2024-05-12T04:01:46.679 INFO:tasks.workunit.client.0.smithi032.stdout:7/999: symlink dd/d1b/d1e/de3/d9a/l137 0 2024-05-12T04:01:46.683 INFO:tasks.workunit.client.0.smithi032.stderr:+ rm -rf -- ./tmp.P7RwIzMTK8 2024-05-12T04:01:47.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:46 smithi033 ceph-mon[49525]: pgmap v51: 65 pgs: 65 active+clean; 3.9 GiB data, 13 GiB used, 524 GiB / 536 GiB avail; 61 MiB/s rd, 122 MiB/s wr, 330 op/s 2024-05-12T04:01:47.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:46 smithi032 ceph-mon[40008]: pgmap v51: 65 pgs: 65 active+clean; 3.9 GiB data, 13 GiB used, 524 GiB / 536 GiB avail; 61 MiB/s rd, 122 MiB/s wr, 330 op/s 2024-05-12T04:01:49.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:48 smithi032 ceph-mon[40008]: pgmap v52: 65 pgs: 65 active+clean; 3.7 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 57 MiB/s rd, 111 MiB/s wr, 324 op/s 2024-05-12T04:01:49.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:48 smithi033 ceph-mon[49525]: pgmap v52: 65 pgs: 65 active+clean; 3.7 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 57 MiB/s rd, 111 MiB/s wr, 324 op/s 2024-05-12T04:01:50.690 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:50 smithi033 ceph-mon[49525]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:01:50.690 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:50 smithi033 ceph-mon[49525]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:01:50.690 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:50 smithi033 ceph-mon[49525]: Reconfiguring node-exporter.smithi033 (dependencies changed)... 2024-05-12T04:01:50.690 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:50 smithi033 ceph-mon[49525]: Reconfiguring daemon node-exporter.smithi033 on smithi033 2024-05-12T04:01:50.690 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:50 smithi033 ceph-mon[49525]: pgmap v53: 65 pgs: 65 active+clean; 3.7 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 35 MiB/s rd, 60 MiB/s wr, 237 op/s 2024-05-12T04:01:50.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:50 smithi032 ceph-mon[40008]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:01:50.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:50 smithi032 ceph-mon[40008]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:01:50.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:50 smithi032 ceph-mon[40008]: Reconfiguring node-exporter.smithi033 (dependencies changed)... 2024-05-12T04:01:50.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:50 smithi032 ceph-mon[40008]: Reconfiguring daemon node-exporter.smithi033 on smithi033 2024-05-12T04:01:50.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:50 smithi032 ceph-mon[40008]: pgmap v53: 65 pgs: 65 active+clean; 3.7 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 35 MiB/s rd, 60 MiB/s wr, 237 op/s 2024-05-12T04:01:53.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:52 smithi033 ceph-mon[49525]: pgmap v54: 65 pgs: 65 active+clean; 3.0 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 35 MiB/s rd, 62 MiB/s wr, 303 op/s 2024-05-12T04:01:53.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:52 smithi033 ceph-mon[49525]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:01:53.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:52 smithi033 ceph-mon[49525]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:01:53.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:52 smithi033 ceph-mon[49525]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-12T04:01:53.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:52 smithi033 ceph-mon[49525]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-12T04:01:53.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:52 smithi033 ceph-mon[49525]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-12T04:01:53.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:52 smithi033 ceph-mon[49525]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:01:53.268 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:52 smithi032 ceph-mon[40008]: pgmap v54: 65 pgs: 65 active+clean; 3.0 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 35 MiB/s rd, 62 MiB/s wr, 303 op/s 2024-05-12T04:01:53.268 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:52 smithi032 ceph-mon[40008]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:01:53.268 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:52 smithi032 ceph-mon[40008]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:01:53.268 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:52 smithi032 ceph-mon[40008]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-12T04:01:53.269 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:52 smithi032 ceph-mon[40008]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-12T04:01:53.269 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:52 smithi032 ceph-mon[40008]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-12T04:01:53.269 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:52 smithi032 ceph-mon[40008]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:01:54.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:53 smithi032 ceph-mon[40008]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-12T04:01:54.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:53 smithi032 ceph-mon[40008]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-12T04:01:54.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:53 smithi032 ceph-mon[40008]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-12T04:01:54.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:53 smithi032 ceph-mon[40008]: Upgrade: Need to upgrade myself (mgr.smithi033.pcvxwy) 2024-05-12T04:01:54.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:53 smithi032 ceph-mon[40008]: Upgrade: Need to upgrade myself (mgr.smithi033.pcvxwy) 2024-05-12T04:01:54.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:53 smithi032 ceph-mon[40008]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:01:54.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:53 smithi032 ceph-mon[40008]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi032.ukatkq", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-12T04:01:54.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:53 smithi032 ceph-mon[40008]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi032.ukatkq", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-12T04:01:54.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:53 smithi032 ceph-mon[40008]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-12T04:01:54.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:53 smithi032 ceph-mon[40008]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:01:54.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:53 smithi033 ceph-mon[49525]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-12T04:01:54.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:53 smithi033 ceph-mon[49525]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-12T04:01:54.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:53 smithi033 ceph-mon[49525]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-12T04:01:54.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:53 smithi033 ceph-mon[49525]: Upgrade: Need to upgrade myself (mgr.smithi033.pcvxwy) 2024-05-12T04:01:54.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:53 smithi033 ceph-mon[49525]: Upgrade: Need to upgrade myself (mgr.smithi033.pcvxwy) 2024-05-12T04:01:54.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:53 smithi033 ceph-mon[49525]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:01:54.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:53 smithi033 ceph-mon[49525]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi032.ukatkq", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-12T04:01:54.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:53 smithi033 ceph-mon[49525]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi032.ukatkq", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-12T04:01:54.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:53 smithi033 ceph-mon[49525]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-12T04:01:54.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:53 smithi033 ceph-mon[49525]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:01:55.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:54 smithi033 ceph-mon[49525]: Upgrade: Updating mgr.smithi032.ukatkq 2024-05-12T04:01:55.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:54 smithi033 ceph-mon[49525]: pgmap v55: 65 pgs: 65 active+clean; 3.0 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 12 MiB/s rd, 20 MiB/s wr, 165 op/s 2024-05-12T04:01:55.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:54 smithi033 ceph-mon[49525]: Deploying daemon mgr.smithi032.ukatkq on smithi032 2024-05-12T04:01:55.200 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:54 smithi032 ceph-mon[40008]: Upgrade: Updating mgr.smithi032.ukatkq 2024-05-12T04:01:55.200 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:54 smithi032 ceph-mon[40008]: pgmap v55: 65 pgs: 65 active+clean; 3.0 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 12 MiB/s rd, 20 MiB/s wr, 165 op/s 2024-05-12T04:01:55.200 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:54 smithi032 ceph-mon[40008]: Deploying daemon mgr.smithi032.ukatkq on smithi032 2024-05-12T04:01:56.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:55 smithi032 ceph-mon[40008]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:01:56.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:55 smithi033 ceph-mon[49525]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:01:57.161 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:56 smithi032 ceph-mon[40008]: pgmap v56: 65 pgs: 65 active+clean; 2.7 GiB data, 9.5 GiB used, 527 GiB / 536 GiB avail; 12 MiB/s rd, 20 MiB/s wr, 187 op/s 2024-05-12T04:01:57.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:56 smithi033 ceph-mon[49525]: pgmap v56: 65 pgs: 65 active+clean; 2.7 GiB data, 9.5 GiB used, 527 GiB / 536 GiB avail; 12 MiB/s rd, 20 MiB/s wr, 187 op/s 2024-05-12T04:01:58.385 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-12T04:01:58.385 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-05-12T04:01:58.858 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:58 smithi033 ceph-mon[49525]: pgmap v57: 65 pgs: 65 active+clean; 2.0 GiB data, 7.6 GiB used, 529 GiB / 536 GiB avail; 9.9 MiB/s rd, 14 MiB/s wr, 219 op/s 2024-05-12T04:01:58.858 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:58 smithi033 ceph-mon[49525]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:01:58.858 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:58 smithi033 ceph-mon[49525]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:01:58.858 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:58 smithi033 ceph-mon[49525]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:01:58.858 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:58 smithi033 ceph-mon[49525]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:01:58.858 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:01:58 smithi033 ceph-mon[49525]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:01:59.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:58 smithi032 ceph-mon[40008]: pgmap v57: 65 pgs: 65 active+clean; 2.0 GiB data, 7.6 GiB used, 529 GiB / 536 GiB avail; 9.9 MiB/s rd, 14 MiB/s wr, 219 op/s 2024-05-12T04:01:59.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:58 smithi032 ceph-mon[40008]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:01:59.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:58 smithi032 ceph-mon[40008]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:01:59.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:58 smithi032 ceph-mon[40008]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:01:59.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:58 smithi032 ceph-mon[40008]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:01:59.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:01:58 smithi032 ceph-mon[40008]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:02:01.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:00 smithi033 ceph-mon[49525]: pgmap v58: 65 pgs: 65 active+clean; 2.0 GiB data, 7.6 GiB used, 529 GiB / 536 GiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 134 op/s 2024-05-12T04:02:01.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:00 smithi032 ceph-mon[40008]: pgmap v58: 65 pgs: 65 active+clean; 2.0 GiB data, 7.6 GiB used, 529 GiB / 536 GiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 134 op/s 2024-05-12T04:02:02.789 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:02 smithi032 ceph-mon[40008]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:02:02.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:02 smithi032 ceph-mon[40008]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:02:02.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:02 smithi032 ceph-mon[40008]: pgmap v59: 65 pgs: 65 active+clean; 1.2 GiB data, 5.3 GiB used, 531 GiB / 536 GiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 194 op/s 2024-05-12T04:02:03.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:02 smithi033 ceph-mon[49525]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:02:03.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:02 smithi033 ceph-mon[49525]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:02:03.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:02 smithi033 ceph-mon[49525]: pgmap v59: 65 pgs: 65 active+clean; 1.2 GiB data, 5.3 GiB used, 531 GiB / 536 GiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 194 op/s 2024-05-12T04:02:05.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:04 smithi033 ceph-mon[49525]: pgmap v60: 65 pgs: 65 active+clean; 1.2 GiB data, 5.3 GiB used, 531 GiB / 536 GiB avail; 888 KiB/s rd, 946 KiB/s wr, 128 op/s 2024-05-12T04:02:05.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:04 smithi033 ceph-mon[49525]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:02:05.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:04 smithi033 ceph-mon[49525]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:02:05.193 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:04 smithi032 ceph-mon[40008]: pgmap v60: 65 pgs: 65 active+clean; 1.2 GiB data, 5.3 GiB used, 531 GiB / 536 GiB avail; 888 KiB/s rd, 946 KiB/s wr, 128 op/s 2024-05-12T04:02:05.193 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:04 smithi032 ceph-mon[40008]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:02:05.194 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:04 smithi032 ceph-mon[40008]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:02:06.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:06 smithi033 ceph-mon[49525]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:02:06.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:06 smithi033 ceph-mon[49525]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:02:06.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:06 smithi033 ceph-mon[49525]: pgmap v61: 65 pgs: 65 active+clean; 1.0 GiB data, 4.9 GiB used, 532 GiB / 536 GiB avail; 888 KiB/s rd, 947 KiB/s wr, 137 op/s 2024-05-12T04:02:06.763 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:06 smithi032 ceph-mon[40008]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:02:06.763 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:06 smithi032 ceph-mon[40008]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:02:06.763 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:06 smithi032 ceph-mon[40008]: pgmap v61: 65 pgs: 65 active+clean; 1.0 GiB data, 4.9 GiB used, 532 GiB / 536 GiB avail; 888 KiB/s rd, 947 KiB/s wr, 137 op/s 2024-05-12T04:02:07.528 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:07 smithi032 ceph-mon[40008]: Standby manager daemon smithi032.ukatkq restarted 2024-05-12T04:02:07.528 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:07 smithi032 ceph-mon[40008]: Standby manager daemon smithi032.ukatkq started 2024-05-12T04:02:07.528 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:07 smithi032 ceph-mon[40008]: from='mgr.? 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi032.ukatkq/crt"}]: dispatch 2024-05-12T04:02:07.528 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:07 smithi032 ceph-mon[40008]: from='mgr.? 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-12T04:02:07.528 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:07 smithi032 ceph-mon[40008]: from='mgr.? 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi032.ukatkq/key"}]: dispatch 2024-05-12T04:02:07.528 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:07 smithi032 ceph-mon[40008]: from='mgr.? 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-12T04:02:07.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:07 smithi033 ceph-mon[49525]: Standby manager daemon smithi032.ukatkq restarted 2024-05-12T04:02:07.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:07 smithi033 ceph-mon[49525]: Standby manager daemon smithi032.ukatkq started 2024-05-12T04:02:07.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:07 smithi033 ceph-mon[49525]: from='mgr.? 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi032.ukatkq/crt"}]: dispatch 2024-05-12T04:02:07.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:07 smithi033 ceph-mon[49525]: from='mgr.? 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-12T04:02:07.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:07 smithi033 ceph-mon[49525]: from='mgr.? 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi032.ukatkq/key"}]: dispatch 2024-05-12T04:02:07.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:07 smithi033 ceph-mon[49525]: from='mgr.? 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-12T04:02:07.981 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-12T04:02:07.982 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-12T04:02:08.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:08 smithi032 ceph-mon[40008]: mgrmap e27: smithi033.pcvxwy(active, since 117s), standbys: smithi032.ukatkq 2024-05-12T04:02:08.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:08 smithi032 ceph-mon[40008]: pgmap v62: 65 pgs: 65 active+clean; 762 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 158 op/s 2024-05-12T04:02:08.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:08 smithi033 ceph-mon[49525]: mgrmap e27: smithi033.pcvxwy(active, since 117s), standbys: smithi032.ukatkq 2024-05-12T04:02:08.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:08 smithi033 ceph-mon[49525]: pgmap v62: 65 pgs: 65 active+clean; 762 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 158 op/s 2024-05-12T04:02:11.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:10 smithi033 ceph-mon[49525]: pgmap v63: 65 pgs: 65 active+clean; 762 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 896 KiB/s rd, 963 KiB/s wr, 112 op/s 2024-05-12T04:02:11.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:10 smithi033 ceph-mon[49525]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:02:11.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:10 smithi033 ceph-mon[49525]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:02:11.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:10 smithi033 ceph-mon[49525]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:02:11.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:10 smithi033 ceph-mon[49525]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:02:11.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:10 smithi033 ceph-mon[49525]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T04:02:11.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:10 smithi033 ceph-mon[49525]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:02:11.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:10 smithi033 ceph-mon[49525]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:02:11.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:10 smithi033 ceph-mon[49525]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "mgr fail", "who": "smithi033.pcvxwy"}]: dispatch 2024-05-12T04:02:11.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:10 smithi033 ceph-mon[49525]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "mgr fail", "who": "smithi033.pcvxwy"}]: dispatch 2024-05-12T04:02:11.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:10 smithi033 ceph-mon[49525]: Activating manager daemon smithi032.ukatkq 2024-05-12T04:02:11.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:10 smithi033 ceph-mon[49525]: osdmap e48: 6 total, 6 up, 6 in 2024-05-12T04:02:11.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:10 smithi033 ceph-mon[49525]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' cmd='[{"prefix": "mgr fail", "who": "smithi033.pcvxwy"}]': finished 2024-05-12T04:02:11.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:10 smithi033 ceph-mon[49525]: mgrmap e28: smithi032.ukatkq(active, starting, since 0.00532075s) 2024-05-12T04:02:11.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:10 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mon metadata", "id": "smithi032"}]: dispatch 2024-05-12T04:02:11.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:10 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mon metadata", "id": "smithi033"}]: dispatch 2024-05-12T04:02:11.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:10 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi032.qpsawh"}]: dispatch 2024-05-12T04:02:11.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:10 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi033.jgbjhk"}]: dispatch 2024-05-12T04:02:11.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:10 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi033.hglwml"}]: dispatch 2024-05-12T04:02:11.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:10 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi032.ialtqb"}]: dispatch 2024-05-12T04:02:11.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:10 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mgr metadata", "who": "smithi032.ukatkq", "id": "smithi032.ukatkq"}]: dispatch 2024-05-12T04:02:11.164 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:10 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-12T04:02:11.164 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:10 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-12T04:02:11.164 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:10 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-12T04:02:11.164 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:10 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-12T04:02:11.164 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:10 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-12T04:02:11.164 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:10 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-12T04:02:11.164 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:10 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-12T04:02:11.164 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:10 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-12T04:02:11.164 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:10 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-12T04:02:11.164 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:10 smithi033 ceph-mon[49525]: Manager daemon smithi032.ukatkq is now available 2024-05-12T04:02:11.164 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:10 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:02:11.164 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:10 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:02:11.164 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:10 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi032.ukatkq/mirror_snapshot_schedule"}]: dispatch 2024-05-12T04:02:11.164 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:10 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi032.ukatkq/trash_purge_schedule"}]: dispatch 2024-05-12T04:02:11.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:10 smithi032 ceph-mon[40008]: pgmap v63: 65 pgs: 65 active+clean; 762 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 896 KiB/s rd, 963 KiB/s wr, 112 op/s 2024-05-12T04:02:11.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:10 smithi032 ceph-mon[40008]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:02:11.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:10 smithi032 ceph-mon[40008]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:02:11.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:10 smithi032 ceph-mon[40008]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:02:11.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:10 smithi032 ceph-mon[40008]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:02:11.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:10 smithi032 ceph-mon[40008]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T04:02:11.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:10 smithi032 ceph-mon[40008]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' 2024-05-12T04:02:11.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:10 smithi032 ceph-mon[40008]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:02:11.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:10 smithi032 ceph-mon[40008]: from='mgr.24389 172.21.15.33:0/4154008027' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "mgr fail", "who": "smithi033.pcvxwy"}]: dispatch 2024-05-12T04:02:11.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:10 smithi032 ceph-mon[40008]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "mgr fail", "who": "smithi033.pcvxwy"}]: dispatch 2024-05-12T04:02:11.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:10 smithi032 ceph-mon[40008]: Activating manager daemon smithi032.ukatkq 2024-05-12T04:02:11.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:10 smithi032 ceph-mon[40008]: osdmap e48: 6 total, 6 up, 6 in 2024-05-12T04:02:11.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:10 smithi032 ceph-mon[40008]: from='mgr.24389 ' entity='mgr.smithi033.pcvxwy' cmd='[{"prefix": "mgr fail", "who": "smithi033.pcvxwy"}]': finished 2024-05-12T04:02:11.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:10 smithi032 ceph-mon[40008]: mgrmap e28: smithi032.ukatkq(active, starting, since 0.00532075s) 2024-05-12T04:02:11.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:10 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mon metadata", "id": "smithi032"}]: dispatch 2024-05-12T04:02:11.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:10 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mon metadata", "id": "smithi033"}]: dispatch 2024-05-12T04:02:11.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:10 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi032.qpsawh"}]: dispatch 2024-05-12T04:02:11.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:10 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi033.jgbjhk"}]: dispatch 2024-05-12T04:02:11.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:10 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi033.hglwml"}]: dispatch 2024-05-12T04:02:11.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:10 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi032.ialtqb"}]: dispatch 2024-05-12T04:02:11.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:10 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mgr metadata", "who": "smithi032.ukatkq", "id": "smithi032.ukatkq"}]: dispatch 2024-05-12T04:02:11.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:10 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-12T04:02:11.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:10 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-12T04:02:11.292 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:10 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-12T04:02:11.292 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:10 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-12T04:02:11.292 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:10 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-12T04:02:11.292 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:10 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-12T04:02:11.292 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:10 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-12T04:02:11.292 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:10 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-12T04:02:11.292 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:10 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-12T04:02:11.292 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:10 smithi032 ceph-mon[40008]: Manager daemon smithi032.ukatkq is now available 2024-05-12T04:02:11.292 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:10 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:02:11.292 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:10 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:02:11.292 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:10 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi032.ukatkq/mirror_snapshot_schedule"}]: dispatch 2024-05-12T04:02:11.292 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:10 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi032.ukatkq/trash_purge_schedule"}]: dispatch 2024-05-12T04:02:12.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:11 smithi033 ceph-mon[49525]: [12/May/2024:04:02:10] ENGINE Bus STARTING 2024-05-12T04:02:12.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:11 smithi033 ceph-mon[49525]: [12/May/2024:04:02:10] ENGINE Serving on https://172.21.15.32:7150 2024-05-12T04:02:12.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:11 smithi033 ceph-mon[49525]: [12/May/2024:04:02:10] ENGINE Serving on http://172.21.15.32:8765 2024-05-12T04:02:12.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:11 smithi033 ceph-mon[49525]: [12/May/2024:04:02:10] ENGINE Bus STARTED 2024-05-12T04:02:12.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:11 smithi033 ceph-mon[49525]: mgrmap e29: smithi032.ukatkq(active, since 1.00803s) 2024-05-12T04:02:12.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:11 smithi032 ceph-mon[40008]: [12/May/2024:04:02:10] ENGINE Bus STARTING 2024-05-12T04:02:12.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:11 smithi032 ceph-mon[40008]: [12/May/2024:04:02:10] ENGINE Serving on https://172.21.15.32:7150 2024-05-12T04:02:12.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:11 smithi032 ceph-mon[40008]: [12/May/2024:04:02:10] ENGINE Serving on http://172.21.15.32:8765 2024-05-12T04:02:12.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:11 smithi032 ceph-mon[40008]: [12/May/2024:04:02:10] ENGINE Bus STARTED 2024-05-12T04:02:12.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:11 smithi032 ceph-mon[40008]: mgrmap e29: smithi032.ukatkq(active, since 1.00803s) 2024-05-12T04:02:12.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:12.528+0000 7f514ac74700 1 -- 172.21.15.32:0/3942665561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51440fe080 msgr2=0x7f51440fe4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:02:12.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:12.528+0000 7f514ac74700 1 --2- 172.21.15.32:0/3942665561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51440fe080 0x7f51440fe4f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f5130004940 tx=0x7f5130009e30 comp rx=0 tx=0).stop 2024-05-12T04:02:12.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:12.528+0000 7f514ac74700 1 -- 172.21.15.32:0/3942665561 shutdown_connections 2024-05-12T04:02:12.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:12.528+0000 7f514ac74700 1 --2- 172.21.15.32:0/3942665561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51440fe080 0x7f51440fe4f0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:12.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:12.528+0000 7f514ac74700 1 --2- 172.21.15.32:0/3942665561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51440fce80 0x7f51440fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:12.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:12.528+0000 7f514ac74700 1 -- 172.21.15.32:0/3942665561 >> 172.21.15.32:0/3942665561 conn(0x7f51440f8590 msgr2=0x7f51440fa9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:02:12.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:12.528+0000 7f514ac74700 1 -- 172.21.15.32:0/3942665561 shutdown_connections 2024-05-12T04:02:12.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:12.528+0000 7f514ac74700 1 -- 172.21.15.32:0/3942665561 wait complete. 2024-05-12T04:02:12.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:12.529+0000 7f514ac74700 1 Processor -- start 2024-05-12T04:02:12.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:12.529+0000 7f514ac74700 1 -- start start 2024-05-12T04:02:12.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:12.529+0000 7f514ac74700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51440fce80 0x7f514406e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:02:12.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:12.529+0000 7f514ac74700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51440fe080 0x7f514406ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:02:12.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:12.529+0000 7f514ac74700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514406f240 con 0x7f51440fce80 2024-05-12T04:02:12.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:12.529+0000 7f514ac74700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514411bb60 con 0x7f51440fe080 2024-05-12T04:02:12.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:12.530+0000 7f5143fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51440fe080 0x7f514406ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:02:12.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:12.530+0000 7f5148a10700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51440fce80 0x7f514406e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:02:12.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:12.530+0000 7f5143fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51440fe080 0x7f514406ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.32:46452/0 (socket says 172.21.15.32:46452) 2024-05-12T04:02:12.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:12.530+0000 7f5143fff700 1 -- 172.21.15.32:0/532608963 learned_addr learned my addr 172.21.15.32:0/532608963 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:02:12.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:12.530+0000 7f5148a10700 1 -- 172.21.15.32:0/532608963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51440fe080 msgr2=0x7f514406ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:02:12.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:12.530+0000 7f5148a10700 1 --2- 172.21.15.32:0/532608963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51440fe080 0x7f514406ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:12.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:12.530+0000 7f5148a10700 1 -- 172.21.15.32:0/532608963 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5130005040 con 0x7f51440fce80 2024-05-12T04:02:12.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:12.530+0000 7f5143fff700 1 --2- 172.21.15.32:0/532608963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51440fe080 0x7f514406ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-12T04:02:12.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:12.530+0000 7f5148a10700 1 --2- 172.21.15.32:0/532608963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51440fce80 0x7f514406e6b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f51440fdbf0 tx=0x7f513800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:02:12.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:12.530+0000 7f51417fa700 1 -- 172.21.15.32:0/532608963 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5138004e90 con 0x7f51440fce80 2024-05-12T04:02:12.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:12.530+0000 7f51417fa700 1 -- 172.21.15.32:0/532608963 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f513800f040 con 0x7f51440fce80 2024-05-12T04:02:12.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:12.530+0000 7f514ac74700 1 -- 172.21.15.32:0/532608963 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f514411bde0 con 0x7f51440fce80 2024-05-12T04:02:12.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:12.530+0000 7f51417fa700 1 -- 172.21.15.32:0/532608963 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f513800c890 con 0x7f51440fce80 2024-05-12T04:02:12.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:12.530+0000 7f514ac74700 1 -- 172.21.15.32:0/532608963 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f514411c3a0 con 0x7f51440fce80 2024-05-12T04:02:12.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:12.531+0000 7f514ac74700 1 -- 172.21.15.32:0/532608963 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f514405f5c0 con 0x7f51440fce80 2024-05-12T04:02:12.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:12.532+0000 7f51417fa700 1 -- 172.21.15.32:0/532608963 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 30) v1 ==== 45632+0+0 (secure 0 0 0) 0x7f513800cb00 con 0x7f51440fce80 2024-05-12T04:02:12.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:12.532+0000 7f51417fa700 1 --2- 172.21.15.32:0/532608963 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f51340385b0 0x7f513403aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:02:12.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:12.532+0000 7f51417fa700 1 -- 172.21.15.32:0/532608963 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f513804bca0 con 0x7f51440fce80 2024-05-12T04:02:12.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:12.532+0000 7f5143fff700 1 --2- 172.21.15.32:0/532608963 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f51340385b0 0x7f513403aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:02:12.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:12.533+0000 7f5143fff700 1 --2- 172.21.15.32:0/532608963 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f51340385b0 0x7f513403aa60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5130013680 tx=0x7f5130009e30 comp rx=0 tx=0).ready entity=mgr.24501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:02:12.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:12.538+0000 7f51417fa700 1 -- 172.21.15.32:0/532608963 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181777 (secure 0 0 0) 0x7f513801c360 con 0x7f51440fce80 2024-05-12T04:02:12.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:12.919+0000 7f514ac74700 1 -- 172.21.15.32:0/532608963 --> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5144102420 con 0x7f51340385b0 2024-05-12T04:02:12.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:12.921+0000 7f51417fa700 1 -- 172.21.15.32:0/532608963 <== mgr.24501 v2:172.21.15.32:6800/3729453095 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f5144102420 con 0x7f51340385b0 2024-05-12T04:02:12.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:12.925+0000 7f514ac74700 1 -- 172.21.15.32:0/532608963 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f51340385b0 msgr2=0x7f513403aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:02:12.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:12.925+0000 7f514ac74700 1 --2- 172.21.15.32:0/532608963 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f51340385b0 0x7f513403aa60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5130013680 tx=0x7f5130009e30 comp rx=0 tx=0).stop 2024-05-12T04:02:12.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:12.925+0000 7f514ac74700 1 -- 172.21.15.32:0/532608963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51440fce80 msgr2=0x7f514406e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:02:12.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:12.925+0000 7f514ac74700 1 --2- 172.21.15.32:0/532608963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51440fce80 0x7f514406e6b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f51440fdbf0 tx=0x7f513800b040 comp rx=0 tx=0).stop 2024-05-12T04:02:12.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:12.925+0000 7f514ac74700 1 -- 172.21.15.32:0/532608963 shutdown_connections 2024-05-12T04:02:12.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:12.925+0000 7f514ac74700 1 --2- 172.21.15.32:0/532608963 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f51340385b0 0x7f513403aa60 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:12.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:12.925+0000 7f514ac74700 1 --2- 172.21.15.32:0/532608963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51440fce80 0x7f514406e6b0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:12.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:12.925+0000 7f514ac74700 1 --2- 172.21.15.32:0/532608963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51440fe080 0x7f514406ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:12.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:12.925+0000 7f514ac74700 1 -- 172.21.15.32:0/532608963 >> 172.21.15.32:0/532608963 conn(0x7f51440f8590 msgr2=0x7f51440f9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:02:12.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:12.925+0000 7f514ac74700 1 -- 172.21.15.32:0/532608963 shutdown_connections 2024-05-12T04:02:12.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:12.925+0000 7f514ac74700 1 -- 172.21.15.32:0/532608963 wait complete. 2024-05-12T04:02:12.936 INFO:teuthology.orchestra.run.smithi032.stdout:true 2024-05-12T04:02:13.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.054+0000 7f715b59e700 1 -- 172.21.15.32:0/846596697 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f715c0fbaf0 msgr2=0x7f715c0fbf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:02:13.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.054+0000 7f715b59e700 1 --2- 172.21.15.32:0/846596697 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f715c0fbaf0 0x7f715c0fbf00 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f714c004940 tx=0x7f714c009e30 comp rx=0 tx=0).stop 2024-05-12T04:02:13.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.054+0000 7f715b59e700 1 -- 172.21.15.32:0/846596697 shutdown_connections 2024-05-12T04:02:13.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.054+0000 7f715b59e700 1 --2- 172.21.15.32:0/846596697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f715c0fcda0 0x7f715c0fd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:13.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.054+0000 7f715b59e700 1 --2- 172.21.15.32:0/846596697 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f715c0fbaf0 0x7f715c0fbf00 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:13.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.054+0000 7f715b59e700 1 -- 172.21.15.32:0/846596697 >> 172.21.15.32:0/846596697 conn(0x7f715c0f7250 msgr2=0x7f715c0f9680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:02:13.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.054+0000 7f715b59e700 1 -- 172.21.15.32:0/846596697 shutdown_connections 2024-05-12T04:02:13.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.054+0000 7f715b59e700 1 -- 172.21.15.32:0/846596697 wait complete. 2024-05-12T04:02:13.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.055+0000 7f715b59e700 1 Processor -- start 2024-05-12T04:02:13.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.055+0000 7f715b59e700 1 -- start start 2024-05-12T04:02:13.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.055+0000 7f715b59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f715c0fbaf0 0x7f715c0ffc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:02:13.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.055+0000 7f715b59e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f715c0fcda0 0x7f715c0fe140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:02:13.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.055+0000 7f715b59e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715c111360 con 0x7f715c0fcda0 2024-05-12T04:02:13.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.055+0000 7f715b59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715c1114d0 con 0x7f715c0fbaf0 2024-05-12T04:02:13.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.055+0000 7f7159d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f715c0fcda0 0x7f715c0fe140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:02:13.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.055+0000 7f7159d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f715c0fcda0 0x7f715c0fe140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47582/0 (socket says 172.21.15.32:47582) 2024-05-12T04:02:13.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.055+0000 7f7159d9b700 1 -- 172.21.15.32:0/2540616469 learned_addr learned my addr 172.21.15.32:0/2540616469 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:02:13.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.055+0000 7f715a59c700 1 --2- 172.21.15.32:0/2540616469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f715c0fbaf0 0x7f715c0ffc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:02:13.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.055+0000 7f7159d9b700 1 -- 172.21.15.32:0/2540616469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f715c0fbaf0 msgr2=0x7f715c0ffc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:02:13.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.055+0000 7f7159d9b700 1 --2- 172.21.15.32:0/2540616469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f715c0fbaf0 0x7f715c0ffc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:13.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.055+0000 7f7159d9b700 1 -- 172.21.15.32:0/2540616469 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f714c005040 con 0x7f715c0fcda0 2024-05-12T04:02:13.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.056+0000 7f7159d9b700 1 --2- 172.21.15.32:0/2540616469 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f715c0fcda0 0x7f715c0fe140 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f715000eb40 tx=0x7f715000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:02:13.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.056+0000 7f714affd700 1 -- 172.21.15.32:0/2540616469 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7150007cc0 con 0x7f715c0fcda0 2024-05-12T04:02:13.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.056+0000 7f714affd700 1 -- 172.21.15.32:0/2540616469 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f715000f040 con 0x7f715c0fcda0 2024-05-12T04:02:13.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.056+0000 7f715b59e700 1 -- 172.21.15.32:0/2540616469 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f715c111750 con 0x7f715c0fcda0 2024-05-12T04:02:13.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.056+0000 7f714affd700 1 -- 172.21.15.32:0/2540616469 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7150009570 con 0x7f715c0fcda0 2024-05-12T04:02:13.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.056+0000 7f715b59e700 1 -- 172.21.15.32:0/2540616469 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f715c111d10 con 0x7f715c0fcda0 2024-05-12T04:02:13.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.057+0000 7f715b59e700 1 -- 172.21.15.32:0/2540616469 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f715c05f5c0 con 0x7f715c0fcda0 2024-05-12T04:02:13.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.057+0000 7f714affd700 1 -- 172.21.15.32:0/2540616469 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 30) v1 ==== 45632+0+0 (secure 0 0 0) 0x7f71500096d0 con 0x7f715c0fcda0 2024-05-12T04:02:13.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.057+0000 7f714affd700 1 --2- 172.21.15.32:0/2540616469 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f7144038610 0x7f714403aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:02:13.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.057+0000 7f714affd700 1 -- 172.21.15.32:0/2540616469 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f715004c030 con 0x7f715c0fcda0 2024-05-12T04:02:13.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.058+0000 7f715a59c700 1 --2- 172.21.15.32:0/2540616469 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f7144038610 0x7f714403aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:02:13.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.059+0000 7f715a59c700 1 --2- 172.21.15.32:0/2540616469 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f7144038610 0x7f714403aac0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f714c0137b0 tx=0x7f714c009e30 comp rx=0 tx=0).ready entity=mgr.24501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:02:13.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.063+0000 7f714affd700 1 -- 172.21.15.32:0/2540616469 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181777 (secure 0 0 0) 0x7f715007f630 con 0x7f715c0fcda0 2024-05-12T04:02:13.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:12 smithi033 ceph-mon[49525]: pgmap v3: 65 pgs: 65 active+clean; 287 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail 2024-05-12T04:02:13.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:12 smithi033 ceph-mon[49525]: mgrmap e30: smithi032.ukatkq(active, since 2s) 2024-05-12T04:02:13.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:12 smithi032 ceph-mon[40008]: pgmap v3: 65 pgs: 65 active+clean; 287 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail 2024-05-12T04:02:13.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:12 smithi032 ceph-mon[40008]: mgrmap e30: smithi032.ukatkq(active, since 2s) 2024-05-12T04:02:13.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.339+0000 7f715b59e700 1 -- 172.21.15.32:0/2540616469 --> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f715c0fef60 con 0x7f7144038610 2024-05-12T04:02:13.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.340+0000 7f714affd700 1 -- 172.21.15.32:0/2540616469 <== mgr.24501 v2:172.21.15.32:6800/3729453095 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f715c0fef60 con 0x7f7144038610 2024-05-12T04:02:13.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.343+0000 7f715b59e700 1 -- 172.21.15.32:0/2540616469 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f7144038610 msgr2=0x7f714403aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:02:13.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.343+0000 7f715b59e700 1 --2- 172.21.15.32:0/2540616469 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f7144038610 0x7f714403aac0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f714c0137b0 tx=0x7f714c009e30 comp rx=0 tx=0).stop 2024-05-12T04:02:13.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.343+0000 7f715b59e700 1 -- 172.21.15.32:0/2540616469 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f715c0fcda0 msgr2=0x7f715c0fe140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:02:13.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.343+0000 7f715b59e700 1 --2- 172.21.15.32:0/2540616469 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f715c0fcda0 0x7f715c0fe140 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f715000eb40 tx=0x7f715000b040 comp rx=0 tx=0).stop 2024-05-12T04:02:13.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.343+0000 7f715b59e700 1 -- 172.21.15.32:0/2540616469 shutdown_connections 2024-05-12T04:02:13.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.343+0000 7f715b59e700 1 --2- 172.21.15.32:0/2540616469 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f7144038610 0x7f714403aac0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:13.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.343+0000 7f715b59e700 1 --2- 172.21.15.32:0/2540616469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f715c0fbaf0 0x7f715c0ffc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:13.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.343+0000 7f715b59e700 1 --2- 172.21.15.32:0/2540616469 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f715c0fcda0 0x7f715c0fe140 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:13.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.343+0000 7f715b59e700 1 -- 172.21.15.32:0/2540616469 >> 172.21.15.32:0/2540616469 conn(0x7f715c0f7250 msgr2=0x7f715c0f9680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:02:13.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.343+0000 7f715b59e700 1 -- 172.21.15.32:0/2540616469 shutdown_connections 2024-05-12T04:02:13.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.343+0000 7f715b59e700 1 -- 172.21.15.32:0/2540616469 wait complete. 2024-05-12T04:02:13.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.464+0000 7f65abb13700 1 -- 172.21.15.32:0/2041760962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65a40fe080 msgr2=0x7f65a40fe4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:02:13.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.464+0000 7f65abb13700 1 --2- 172.21.15.32:0/2041760962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65a40fe080 0x7f65a40fe4f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6594004940 tx=0x7f6594009e30 comp rx=0 tx=0).stop 2024-05-12T04:02:13.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.464+0000 7f65abb13700 1 -- 172.21.15.32:0/2041760962 shutdown_connections 2024-05-12T04:02:13.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.464+0000 7f65abb13700 1 --2- 172.21.15.32:0/2041760962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65a40fe080 0x7f65a40fe4f0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:13.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.464+0000 7f65abb13700 1 --2- 172.21.15.32:0/2041760962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65a40fcf10 0x7f65a40fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:13.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.464+0000 7f65abb13700 1 -- 172.21.15.32:0/2041760962 >> 172.21.15.32:0/2041760962 conn(0x7f65a40f8560 msgr2=0x7f65a40fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:02:13.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.464+0000 7f65abb13700 1 -- 172.21.15.32:0/2041760962 shutdown_connections 2024-05-12T04:02:13.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.464+0000 7f65abb13700 1 -- 172.21.15.32:0/2041760962 wait complete. 2024-05-12T04:02:13.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.465+0000 7f65abb13700 1 Processor -- start 2024-05-12T04:02:13.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.465+0000 7f65abb13700 1 -- start start 2024-05-12T04:02:13.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.465+0000 7f65abb13700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65a40fcf10 0x7f65a410d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:02:13.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.465+0000 7f65abb13700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65a40fe080 0x7f65a410ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:02:13.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.465+0000 7f65abb13700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a410e3d0 con 0x7f65a40fcf10 2024-05-12T04:02:13.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.465+0000 7f65abb13700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a410ec20 con 0x7f65a40fe080 2024-05-12T04:02:13.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.465+0000 7f65a90ae700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65a40fe080 0x7f65a410ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:02:13.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.465+0000 7f65a98af700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65a40fcf10 0x7f65a410d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:02:13.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.465+0000 7f65a90ae700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65a40fe080 0x7f65a410ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.32:46508/0 (socket says 172.21.15.32:46508) 2024-05-12T04:02:13.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.465+0000 7f65a90ae700 1 -- 172.21.15.32:0/2107106179 learned_addr learned my addr 172.21.15.32:0/2107106179 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:02:13.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.465+0000 7f65a98af700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65a40fcf10 0x7f65a410d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47600/0 (socket says 172.21.15.32:47600) 2024-05-12T04:02:13.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.466+0000 7f65a90ae700 1 -- 172.21.15.32:0/2107106179 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65a40fcf10 msgr2=0x7f65a410d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:02:13.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.466+0000 7f65a90ae700 1 --2- 172.21.15.32:0/2107106179 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65a40fcf10 0x7f65a410d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:13.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.466+0000 7f65a90ae700 1 -- 172.21.15.32:0/2107106179 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6594005040 con 0x7f65a40fe080 2024-05-12T04:02:13.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.466+0000 7f65a90ae700 1 --2- 172.21.15.32:0/2107106179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65a40fe080 0x7f65a410ddd0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f65940137b0 tx=0x7f6594009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:02:13.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.466+0000 7f659a7fc700 1 -- 172.21.15.32:0/2107106179 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6594021070 con 0x7f65a40fe080 2024-05-12T04:02:13.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.466+0000 7f659a7fc700 1 -- 172.21.15.32:0/2107106179 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f659401c400 con 0x7f65a40fe080 2024-05-12T04:02:13.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.466+0000 7f65abb13700 1 -- 172.21.15.32:0/2107106179 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65a410eea0 con 0x7f65a40fe080 2024-05-12T04:02:13.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.466+0000 7f659a7fc700 1 -- 172.21.15.32:0/2107106179 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f659400e070 con 0x7f65a40fe080 2024-05-12T04:02:13.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.466+0000 7f65abb13700 1 -- 172.21.15.32:0/2107106179 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65a410f380 con 0x7f65a40fe080 2024-05-12T04:02:13.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.467+0000 7f659a7fc700 1 -- 172.21.15.32:0/2107106179 <== mon.1 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 30) v1 ==== 45632+0+0 (secure 0 0 0) 0x7f659400b760 con 0x7f65a40fe080 2024-05-12T04:02:13.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.467+0000 7f659a7fc700 1 --2- 172.21.15.32:0/2107106179 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f65900385b0 0x7f659003aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:02:13.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.467+0000 7f659a7fc700 1 -- 172.21.15.32:0/2107106179 <== mon.1 v2:172.21.15.33:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f659404d170 con 0x7f65a40fe080 2024-05-12T04:02:13.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.467+0000 7f65abb13700 1 -- 172.21.15.32:0/2107106179 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65a405f5c0 con 0x7f65a40fe080 2024-05-12T04:02:13.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.468+0000 7f65a98af700 1 --2- 172.21.15.32:0/2107106179 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f65900385b0 0x7f659003aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:02:13.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.468+0000 7f65a98af700 1 --2- 172.21.15.32:0/2107106179 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f65900385b0 0x7f659003aa60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f65a0009b80 tx=0x7f65a000b040 comp rx=0 tx=0).ready entity=mgr.24501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:02:13.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.473+0000 7f659a7fc700 1 -- 172.21.15.32:0/2107106179 <== mon.1 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181777 (secure 0 0 0) 0x7f659400bb00 con 0x7f65a40fe080 2024-05-12T04:02:13.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.805+0000 7f65abb13700 1 -- 172.21.15.32:0/2107106179 --> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f65a410f690 con 0x7f65900385b0 2024-05-12T04:02:13.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.816+0000 7f659a7fc700 1 -- 172.21.15.32:0/2107106179 <== mgr.24501 v2:172.21.15.32:6800/3729453095 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f65a410f690 con 0x7f65900385b0 2024-05-12T04:02:13.816 INFO:teuthology.orchestra.run.smithi032.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-12T04:02:13.816 INFO:teuthology.orchestra.run.smithi032.stdout:alertmanager.smithi032 smithi032 *:9093,9094 running (79s) 9s ago 10m 17.8M - 0.25.0 c8568f914cd2 a37fbab7af4c 2024-05-12T04:02:13.816 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi032 smithi032 running (10m) 9s ago 10m 6656k - 17.2.7-939-gc190bdee 284803f7d9cf a478874e9dfa 2024-05-12T04:02:13.816 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi033 smithi033 running (9m) 12s ago 9m 6651k - 17.2.7-939-gc190bdee 284803f7d9cf 34c62d004c7e 2024-05-12T04:02:13.816 INFO:teuthology.orchestra.run.smithi032.stdout:grafana.smithi032 smithi032 *:3000 running (44s) 9s ago 9m 79.4M - 9.4.7 954c08fa6188 83055e58d23c 2024-05-12T04:02:13.816 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.ialtqb smithi032 running (4m) 9s ago 4m 12.2M - 17.2.7-939-gc190bdee 284803f7d9cf 0683abf00e3e 2024-05-12T04:02:13.817 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.qpsawh smithi032 running (4m) 9s ago 4m 253M - 17.2.7-939-gc190bdee 284803f7d9cf 95a7db5c4fcd 2024-05-12T04:02:13.817 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi033.hglwml smithi033 running (4m) 12s ago 4m 12.2M - 17.2.7-939-gc190bdee 284803f7d9cf c97902b05025 2024-05-12T04:02:13.817 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi033.jgbjhk smithi033 running (4m) 12s ago 4m 264M - 17.2.7-939-gc190bdee 284803f7d9cf af97052cb97a 2024-05-12T04:02:13.817 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi032.ukatkq smithi032 *:8443,9283,8765 running (16s) 9s ago 11m 67.8M - 18.2.2-1272-g58249552 ddcb0a9d6fde 0d88ab9a5292 2024-05-12T04:02:13.817 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi033.pcvxwy smithi033 *:8443,9283 running (2m) 12s ago 9m 500M - 18.2.2-1272-g58249552 ddcb0a9d6fde b0bff3ab143f 2024-05-12T04:02:13.817 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi032 smithi032 running (11m) 9s ago 11m 48.7M 2048M 17.2.7-939-gc190bdee 284803f7d9cf d7fea430c6e2 2024-05-12T04:02:13.817 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi033 smithi033 running (9m) 12s ago 9m 39.4M 2048M 17.2.7-939-gc190bdee 284803f7d9cf 707654d85748 2024-05-12T04:02:13.817 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi032 smithi032 *:9100 running (95s) 9s ago 10m 9290k - 1.5.0 0da6a335fe13 cb75a7c0c829 2024-05-12T04:02:13.817 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi033 smithi033 *:9100 running (22s) 12s ago 9m 3288k - 1.5.0 0da6a335fe13 7888c36c659f 2024-05-12T04:02:13.817 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0 smithi032 running (8m) 9s ago 8m 177M 1570M 17.2.7-939-gc190bdee 284803f7d9cf 75c78a47f8e5 2024-05-12T04:02:13.817 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1 smithi032 running (7m) 9s ago 7m 181M 1570M 17.2.7-939-gc190bdee 284803f7d9cf 87cfd247d8d6 2024-05-12T04:02:13.817 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2 smithi032 running (7m) 9s ago 7m 150M 1570M 17.2.7-939-gc190bdee 284803f7d9cf 7aef7affb8ba 2024-05-12T04:02:13.817 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3 smithi033 running (6m) 12s ago 6m 202M 2594M 17.2.7-939-gc190bdee 284803f7d9cf 3a92b5e3348e 2024-05-12T04:02:13.817 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4 smithi033 running (6m) 12s ago 6m 180M 2594M 17.2.7-939-gc190bdee 284803f7d9cf 0896268b4591 2024-05-12T04:02:13.817 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5 smithi033 running (5m) 12s ago 5m 178M 2594M 17.2.7-939-gc190bdee 284803f7d9cf 57e65a64c821 2024-05-12T04:02:13.817 INFO:teuthology.orchestra.run.smithi032.stdout:prometheus.smithi032 smithi032 *:9095 running (24s) 9s ago 9m 38.2M - 2.43.0 a07b618ecd1d bd6d82ae14e3 2024-05-12T04:02:13.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.818+0000 7f65abb13700 1 -- 172.21.15.32:0/2107106179 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f65900385b0 msgr2=0x7f659003aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:02:13.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.818+0000 7f65abb13700 1 --2- 172.21.15.32:0/2107106179 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f65900385b0 0x7f659003aa60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f65a0009b80 tx=0x7f65a000b040 comp rx=0 tx=0).stop 2024-05-12T04:02:13.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.818+0000 7f65abb13700 1 -- 172.21.15.32:0/2107106179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65a40fe080 msgr2=0x7f65a410ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:02:13.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.818+0000 7f65abb13700 1 --2- 172.21.15.32:0/2107106179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65a40fe080 0x7f65a410ddd0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f65940137b0 tx=0x7f6594009e30 comp rx=0 tx=0).stop 2024-05-12T04:02:13.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.819+0000 7f65abb13700 1 -- 172.21.15.32:0/2107106179 shutdown_connections 2024-05-12T04:02:13.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.819+0000 7f65abb13700 1 --2- 172.21.15.32:0/2107106179 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f65900385b0 0x7f659003aa60 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:13.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.819+0000 7f65abb13700 1 --2- 172.21.15.32:0/2107106179 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65a40fcf10 0x7f65a410d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:13.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.819+0000 7f65abb13700 1 --2- 172.21.15.32:0/2107106179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65a40fe080 0x7f65a410ddd0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:13.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.819+0000 7f65abb13700 1 -- 172.21.15.32:0/2107106179 >> 172.21.15.32:0/2107106179 conn(0x7f65a40f8560 msgr2=0x7f65a40fa040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:02:13.819 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.819+0000 7f65abb13700 1 -- 172.21.15.32:0/2107106179 shutdown_connections 2024-05-12T04:02:13.820 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.819+0000 7f65abb13700 1 -- 172.21.15.32:0/2107106179 wait complete. 2024-05-12T04:02:13.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.945+0000 7f96c8496700 1 -- 172.21.15.32:0/3154268362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96c00fff70 msgr2=0x7f96c0102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:02:13.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.945+0000 7f96c8496700 1 --2- 172.21.15.32:0/3154268362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96c00fff70 0x7f96c0102390 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f96b0004940 tx=0x7f96b0009e30 comp rx=0 tx=0).stop 2024-05-12T04:02:13.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.945+0000 7f96c8496700 1 -- 172.21.15.32:0/3154268362 shutdown_connections 2024-05-12T04:02:13.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.945+0000 7f96c8496700 1 --2- 172.21.15.32:0/3154268362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96c00fff70 0x7f96c0102390 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:13.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.945+0000 7f96c8496700 1 --2- 172.21.15.32:0/3154268362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96c0066b50 0x7f96c00ffa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:13.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.945+0000 7f96c8496700 1 -- 172.21.15.32:0/3154268362 >> 172.21.15.32:0/3154268362 conn(0x7f96c00f74c0 msgr2=0x7f96c00f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:02:13.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.945+0000 7f96c8496700 1 -- 172.21.15.32:0/3154268362 shutdown_connections 2024-05-12T04:02:13.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.945+0000 7f96c8496700 1 -- 172.21.15.32:0/3154268362 wait complete. 2024-05-12T04:02:13.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.946+0000 7f96c8496700 1 Processor -- start 2024-05-12T04:02:13.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.946+0000 7f96c8496700 1 -- start start 2024-05-12T04:02:13.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.946+0000 7f96c8496700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96c0066b50 0x7f96c010d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:02:13.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.946+0000 7f96c8496700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96c00fff70 0x7f96c010de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:02:13.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.946+0000 7f96c8496700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c010e440 con 0x7f96c00fff70 2024-05-12T04:02:13.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.946+0000 7f96c8496700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c010ec90 con 0x7f96c0066b50 2024-05-12T04:02:13.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.946+0000 7f96c5a31700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96c00fff70 0x7f96c010de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:02:13.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.946+0000 7f96c5a31700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96c00fff70 0x7f96c010de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47612/0 (socket says 172.21.15.32:47612) 2024-05-12T04:02:13.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.946+0000 7f96c5a31700 1 -- 172.21.15.32:0/3011967120 learned_addr learned my addr 172.21.15.32:0/3011967120 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:02:13.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.946+0000 7f96c6232700 1 --2- 172.21.15.32:0/3011967120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96c0066b50 0x7f96c010d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:02:13.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.947+0000 7f96c5a31700 1 -- 172.21.15.32:0/3011967120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96c0066b50 msgr2=0x7f96c010d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:02:13.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.947+0000 7f96c5a31700 1 --2- 172.21.15.32:0/3011967120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96c0066b50 0x7f96c010d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:13.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.947+0000 7f96c5a31700 1 -- 172.21.15.32:0/3011967120 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b0005040 con 0x7f96c00fff70 2024-05-12T04:02:13.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.947+0000 7f96c5a31700 1 --2- 172.21.15.32:0/3011967120 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96c00fff70 0x7f96c010de40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f96b00137b0 tx=0x7f96b0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:02:13.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.947+0000 7f96b6ffd700 1 -- 172.21.15.32:0/3011967120 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f96b000e070 con 0x7f96c00fff70 2024-05-12T04:02:13.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.947+0000 7f96b6ffd700 1 -- 172.21.15.32:0/3011967120 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f96b001c400 con 0x7f96c00fff70 2024-05-12T04:02:13.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.947+0000 7f96c8496700 1 -- 172.21.15.32:0/3011967120 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96c010ef10 con 0x7f96c00fff70 2024-05-12T04:02:13.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.947+0000 7f96c8496700 1 -- 172.21.15.32:0/3011967120 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96c010f3f0 con 0x7f96c00fff70 2024-05-12T04:02:13.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.947+0000 7f96b6ffd700 1 -- 172.21.15.32:0/3011967120 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f96b000b650 con 0x7f96c00fff70 2024-05-12T04:02:13.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.948+0000 7f96b6ffd700 1 -- 172.21.15.32:0/3011967120 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 30) v1 ==== 45632+0+0 (secure 0 0 0) 0x7f96b000b910 con 0x7f96c00fff70 2024-05-12T04:02:13.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.948+0000 7f96b6ffd700 1 --2- 172.21.15.32:0/3011967120 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f96ac0385c0 0x7f96ac03aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:02:13.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.948+0000 7f96c8496700 1 -- 172.21.15.32:0/3011967120 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96a40052e0 con 0x7f96c00fff70 2024-05-12T04:02:13.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.948+0000 7f96b6ffd700 1 -- 172.21.15.32:0/3011967120 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f96b004d530 con 0x7f96c00fff70 2024-05-12T04:02:13.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.949+0000 7f96c6232700 1 --2- 172.21.15.32:0/3011967120 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f96ac0385c0 0x7f96ac03aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:02:13.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.949+0000 7f96c6232700 1 --2- 172.21.15.32:0/3011967120 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f96ac0385c0 0x7f96ac03aa70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f96bc009b80 tx=0x7f96bc00b040 comp rx=0 tx=0).ready entity=mgr.24501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:02:13.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:13.953+0000 7f96b6ffd700 1 -- 172.21.15.32:0/3011967120 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181777 (secure 0 0 0) 0x7f96b001bb90 con 0x7f96c00fff70 2024-05-12T04:02:14.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.147+0000 7f96b6ffd700 1 -- 172.21.15.32:0/3011967120 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 31) v1 ==== 45688+0+0 (secure 0 0 0) 0x7f96b0029d40 con 0x7f96c00fff70 2024-05-12T04:02:14.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:13 smithi033 ceph-mon[49525]: pgmap v4: 65 pgs: 65 active+clean; 287 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail 2024-05-12T04:02:14.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:13 smithi033 ceph-mon[49525]: from='client.14782 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:02:14.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:13 smithi032 ceph-mon[40008]: pgmap v4: 65 pgs: 65 active+clean; 287 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail 2024-05-12T04:02:14.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:13 smithi032 ceph-mon[40008]: from='client.14782 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:02:14.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.339+0000 7f96c8496700 1 -- 172.21.15.32:0/3011967120 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f96a4005860 con 0x7f96c00fff70 2024-05-12T04:02:14.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.339+0000 7f96b6ffd700 1 -- 172.21.15.32:0/3011967120 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+743 (secure 0 0 0) 0x7f96b001bb90 con 0x7f96c00fff70 2024-05-12T04:02:14.340 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-05-12T04:02:14.340 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": { 2024-05-12T04:02:14.340 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 2 2024-05-12T04:02:14.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:02:14.340 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": { 2024-05-12T04:02:14.340 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 1 2024-05-12T04:02:14.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:02:14.340 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": { 2024-05-12T04:02:14.340 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 6 2024-05-12T04:02:14.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:02:14.340 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": { 2024-05-12T04:02:14.340 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 4 2024-05-12T04:02:14.340 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:02:14.341 INFO:teuthology.orchestra.run.smithi032.stdout: "overall": { 2024-05-12T04:02:14.341 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 12, 2024-05-12T04:02:14.341 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 1 2024-05-12T04:02:14.341 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-05-12T04:02:14.341 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-05-12T04:02:14.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.342+0000 7f96c8496700 1 -- 172.21.15.32:0/3011967120 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f96ac0385c0 msgr2=0x7f96ac03aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:02:14.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.342+0000 7f96c8496700 1 --2- 172.21.15.32:0/3011967120 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f96ac0385c0 0x7f96ac03aa70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f96bc009b80 tx=0x7f96bc00b040 comp rx=0 tx=0).stop 2024-05-12T04:02:14.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.342+0000 7f96c8496700 1 -- 172.21.15.32:0/3011967120 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96c00fff70 msgr2=0x7f96c010de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:02:14.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.342+0000 7f96c8496700 1 --2- 172.21.15.32:0/3011967120 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96c00fff70 0x7f96c010de40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f96b00137b0 tx=0x7f96b0009e30 comp rx=0 tx=0).stop 2024-05-12T04:02:14.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.342+0000 7f96c8496700 1 -- 172.21.15.32:0/3011967120 shutdown_connections 2024-05-12T04:02:14.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.342+0000 7f96c8496700 1 --2- 172.21.15.32:0/3011967120 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f96ac0385c0 0x7f96ac03aa70 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:14.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.342+0000 7f96c8496700 1 --2- 172.21.15.32:0/3011967120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96c0066b50 0x7f96c010d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:14.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.342+0000 7f96c8496700 1 --2- 172.21.15.32:0/3011967120 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96c00fff70 0x7f96c010de40 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:14.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.342+0000 7f96c8496700 1 -- 172.21.15.32:0/3011967120 >> 172.21.15.32:0/3011967120 conn(0x7f96c00f74c0 msgr2=0x7f96c00f7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:02:14.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.342+0000 7f96c8496700 1 -- 172.21.15.32:0/3011967120 shutdown_connections 2024-05-12T04:02:14.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.342+0000 7f96c8496700 1 -- 172.21.15.32:0/3011967120 wait complete. 2024-05-12T04:02:14.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.477+0000 7f5ea0367700 1 -- 172.21.15.32:0/4172644010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e98066af0 msgr2=0x7f5e980ffa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:02:14.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.477+0000 7f5ea0367700 1 --2- 172.21.15.32:0/4172644010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e98066af0 0x7f5e980ffa90 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f5e94004940 tx=0x7f5e94009e30 comp rx=0 tx=0).stop 2024-05-12T04:02:14.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.477+0000 7f5ea0367700 1 -- 172.21.15.32:0/4172644010 shutdown_connections 2024-05-12T04:02:14.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.477+0000 7f5ea0367700 1 --2- 172.21.15.32:0/4172644010 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e980fffd0 0x7f5e981023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:14.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.477+0000 7f5ea0367700 1 --2- 172.21.15.32:0/4172644010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e98066af0 0x7f5e980ffa90 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:14.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.477+0000 7f5ea0367700 1 -- 172.21.15.32:0/4172644010 >> 172.21.15.32:0/4172644010 conn(0x7f5e980f74c0 msgr2=0x7f5e980f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:02:14.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.477+0000 7f5ea0367700 1 -- 172.21.15.32:0/4172644010 shutdown_connections 2024-05-12T04:02:14.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.477+0000 7f5ea0367700 1 -- 172.21.15.32:0/4172644010 wait complete. 2024-05-12T04:02:14.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.478+0000 7f5ea0367700 1 Processor -- start 2024-05-12T04:02:14.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.478+0000 7f5ea0367700 1 -- start start 2024-05-12T04:02:14.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.478+0000 7f5ea0367700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e98066af0 0x7f5e980fd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:02:14.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.478+0000 7f5ea0367700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e980fffd0 0x7f5e980fb990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:02:14.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.478+0000 7f5ea0367700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e980fbf40 con 0x7f5e98066af0 2024-05-12T04:02:14.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.478+0000 7f5ea0367700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e980fc0b0 con 0x7f5e980fffd0 2024-05-12T04:02:14.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.478+0000 7f5e9d902700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e980fffd0 0x7f5e980fb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:02:14.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.478+0000 7f5e9d902700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e980fffd0 0x7f5e980fb990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.32:46562/0 (socket says 172.21.15.32:46562) 2024-05-12T04:02:14.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.478+0000 7f5e9d902700 1 -- 172.21.15.32:0/2678224826 learned_addr learned my addr 172.21.15.32:0/2678224826 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:02:14.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.479+0000 7f5e9e103700 1 --2- 172.21.15.32:0/2678224826 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e98066af0 0x7f5e980fd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:02:14.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.479+0000 7f5e9d902700 1 -- 172.21.15.32:0/2678224826 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e98066af0 msgr2=0x7f5e980fd4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:02:14.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.479+0000 7f5e9d902700 1 --2- 172.21.15.32:0/2678224826 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e98066af0 0x7f5e980fd4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:14.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.479+0000 7f5e9d902700 1 -- 172.21.15.32:0/2678224826 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e94005040 con 0x7f5e980fffd0 2024-05-12T04:02:14.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.479+0000 7f5e9d902700 1 --2- 172.21.15.32:0/2678224826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e980fffd0 0x7f5e980fb990 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5e8800eb40 tx=0x7f5e8800b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:02:14.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.479+0000 7f5e8effd700 1 -- 172.21.15.32:0/2678224826 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e88007cc0 con 0x7f5e980fffd0 2024-05-12T04:02:14.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.479+0000 7f5ea0367700 1 -- 172.21.15.32:0/2678224826 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e980fc330 con 0x7f5e980fffd0 2024-05-12T04:02:14.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.479+0000 7f5e8effd700 1 -- 172.21.15.32:0/2678224826 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5e8800f040 con 0x7f5e980fffd0 2024-05-12T04:02:14.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.479+0000 7f5e8effd700 1 -- 172.21.15.32:0/2678224826 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e88009570 con 0x7f5e980fffd0 2024-05-12T04:02:14.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.479+0000 7f5ea0367700 1 -- 172.21.15.32:0/2678224826 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e98115930 con 0x7f5e980fffd0 2024-05-12T04:02:14.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.480+0000 7f5e8effd700 1 -- 172.21.15.32:0/2678224826 <== mon.1 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 31) v1 ==== 45688+0+0 (secure 0 0 0) 0x7f5e880096d0 con 0x7f5e980fffd0 2024-05-12T04:02:14.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.481+0000 7f5ea0367700 1 -- 172.21.15.32:0/2678224826 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e7c0052e0 con 0x7f5e980fffd0 2024-05-12T04:02:14.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.481+0000 7f5e8effd700 1 --2- 172.21.15.32:0/2678224826 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f5e84038640 0x7f5e8403aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:02:14.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.481+0000 7f5e8effd700 1 -- 172.21.15.32:0/2678224826 <== mon.1 v2:172.21.15.33:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f5e8804bcf0 con 0x7f5e980fffd0 2024-05-12T04:02:14.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.481+0000 7f5e9e103700 1 --2- 172.21.15.32:0/2678224826 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f5e84038640 0x7f5e8403aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:02:14.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.481+0000 7f5e9e103700 1 --2- 172.21.15.32:0/2678224826 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f5e84038640 0x7f5e8403aaf0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5e940137d0 tx=0x7f5e94009e30 comp rx=0 tx=0).ready entity=mgr.24501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:02:14.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.486+0000 7f5e8effd700 1 -- 172.21.15.32:0/2678224826 <== mon.1 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181777 (secure 0 0 0) 0x7f5e8801b2c0 con 0x7f5e980fffd0 2024-05-12T04:02:14.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.832+0000 7f5ea0367700 1 -- 172.21.15.32:0/2678224826 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5e7c005860 con 0x7f5e980fffd0 2024-05-12T04:02:14.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.832+0000 7f5e8effd700 1 -- 172.21.15.32:0/2678224826 <== mon.1 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1828 (secure 0 0 0) 0x7f5e88019070 con 0x7f5e980fffd0 2024-05-12T04:02:14.833 INFO:teuthology.orchestra.run.smithi032.stdout:e18 2024-05-12T04:02:14.833 INFO:teuthology.orchestra.run.smithi032.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-12T04:02:14.833 INFO:teuthology.orchestra.run.smithi032.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-12T04:02:14.833 INFO:teuthology.orchestra.run.smithi032.stdout:legacy client fscid: 1 2024-05-12T04:02:14.833 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:02:14.833 INFO:teuthology.orchestra.run.smithi032.stdout:Filesystem 'cephfs' (1) 2024-05-12T04:02:14.833 INFO:teuthology.orchestra.run.smithi032.stdout:fs_name cephfs 2024-05-12T04:02:14.833 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 18 2024-05-12T04:02:14.834 INFO:teuthology.orchestra.run.smithi032.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-12T04:02:14.834 INFO:teuthology.orchestra.run.smithi032.stdout:created 2024-05-12T03:57:48.904614+0000 2024-05-12T04:02:14.834 INFO:teuthology.orchestra.run.smithi032.stdout:modified 2024-05-12T03:58:06.821591+0000 2024-05-12T04:02:14.834 INFO:teuthology.orchestra.run.smithi032.stdout:tableserver 0 2024-05-12T04:02:14.834 INFO:teuthology.orchestra.run.smithi032.stdout:root 0 2024-05-12T04:02:14.834 INFO:teuthology.orchestra.run.smithi032.stdout:session_timeout 60 2024-05-12T04:02:14.834 INFO:teuthology.orchestra.run.smithi032.stdout:session_autoclose 300 2024-05-12T04:02:14.834 INFO:teuthology.orchestra.run.smithi032.stdout:max_file_size 1099511627776 2024-05-12T04:02:14.834 INFO:teuthology.orchestra.run.smithi032.stdout:required_client_features {} 2024-05-12T04:02:14.834 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure 0 2024-05-12T04:02:14.834 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure_osd_epoch 46 2024-05-12T04:02:14.834 INFO:teuthology.orchestra.run.smithi032.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-12T04:02:14.834 INFO:teuthology.orchestra.run.smithi032.stdout:max_mds 1 2024-05-12T04:02:14.835 INFO:teuthology.orchestra.run.smithi032.stdout:in 0 2024-05-12T04:02:14.835 INFO:teuthology.orchestra.run.smithi032.stdout:up {0=14474} 2024-05-12T04:02:14.835 INFO:teuthology.orchestra.run.smithi032.stdout:failed 2024-05-12T04:02:14.835 INFO:teuthology.orchestra.run.smithi032.stdout:damaged 2024-05-12T04:02:14.835 INFO:teuthology.orchestra.run.smithi032.stdout:stopped 2024-05-12T04:02:14.835 INFO:teuthology.orchestra.run.smithi032.stdout:data_pools [3] 2024-05-12T04:02:14.835 INFO:teuthology.orchestra.run.smithi032.stdout:metadata_pool 2 2024-05-12T04:02:14.835 INFO:teuthology.orchestra.run.smithi032.stdout:inline_data enabled 2024-05-12T04:02:14.835 INFO:teuthology.orchestra.run.smithi032.stdout:balancer 2024-05-12T04:02:14.835 INFO:teuthology.orchestra.run.smithi032.stdout:standby_count_wanted 1 2024-05-12T04:02:14.835 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.qpsawh{0:14474} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.32:6828/3100155296,v1:172.21.15.32:6829/3100155296] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:02:14.835 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi033.jgbjhk{0:24267} state up:standby-replay seq 3 join_fscid=1 addr [v2:172.21.15.33:6824/55417471,v1:172.21.15.33:6825/55417471] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:02:14.835 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:02:14.835 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:02:14.835 INFO:teuthology.orchestra.run.smithi032.stdout:Standby daemons: 2024-05-12T04:02:14.836 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:02:14.836 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi033.hglwml{-1:24283} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/433858238,v1:172.21.15.33:6827/433858238] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:02:14.836 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.ialtqb{-1:24287} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.32:6826/3703132329,v1:172.21.15.32:6827/3703132329] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:02:14.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.835+0000 7f5ea0367700 1 -- 172.21.15.32:0/2678224826 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f5e84038640 msgr2=0x7f5e8403aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:02:14.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.835+0000 7f5ea0367700 1 --2- 172.21.15.32:0/2678224826 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f5e84038640 0x7f5e8403aaf0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5e940137d0 tx=0x7f5e94009e30 comp rx=0 tx=0).stop 2024-05-12T04:02:14.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.835+0000 7f5ea0367700 1 -- 172.21.15.32:0/2678224826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e980fffd0 msgr2=0x7f5e980fb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:02:14.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.835+0000 7f5ea0367700 1 --2- 172.21.15.32:0/2678224826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e980fffd0 0x7f5e980fb990 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5e8800eb40 tx=0x7f5e8800b040 comp rx=0 tx=0).stop 2024-05-12T04:02:14.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.835+0000 7f5ea0367700 1 -- 172.21.15.32:0/2678224826 shutdown_connections 2024-05-12T04:02:14.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.835+0000 7f5ea0367700 1 --2- 172.21.15.32:0/2678224826 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f5e84038640 0x7f5e8403aaf0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:14.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.835+0000 7f5ea0367700 1 --2- 172.21.15.32:0/2678224826 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e98066af0 0x7f5e980fd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:14.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.835+0000 7f5ea0367700 1 --2- 172.21.15.32:0/2678224826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e980fffd0 0x7f5e980fb990 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:14.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.835+0000 7f5ea0367700 1 -- 172.21.15.32:0/2678224826 >> 172.21.15.32:0/2678224826 conn(0x7f5e980f74c0 msgr2=0x7f5e980f7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:02:14.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.835+0000 7f5ea0367700 1 -- 172.21.15.32:0/2678224826 shutdown_connections 2024-05-12T04:02:14.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.835+0000 7f5ea0367700 1 -- 172.21.15.32:0/2678224826 wait complete. 2024-05-12T04:02:14.837 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 18 2024-05-12T04:02:14.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.967+0000 7fcc30f2c700 1 -- 172.21.15.32:0/641801282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc2c0fce80 msgr2=0x7fcc2c0fd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:02:14.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.967+0000 7fcc30f2c700 1 --2- 172.21.15.32:0/641801282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc2c0fce80 0x7fcc2c0fd290 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c004940 tx=0x7fcc1c009e30 comp rx=0 tx=0).stop 2024-05-12T04:02:14.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.968+0000 7fcc30f2c700 1 -- 172.21.15.32:0/641801282 shutdown_connections 2024-05-12T04:02:14.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.968+0000 7fcc30f2c700 1 --2- 172.21.15.32:0/641801282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc2c0fdff0 0x7fcc2c0fe460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:14.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.968+0000 7fcc30f2c700 1 --2- 172.21.15.32:0/641801282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc2c0fce80 0x7fcc2c0fd290 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:14.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.968+0000 7fcc30f2c700 1 -- 172.21.15.32:0/641801282 >> 172.21.15.32:0/641801282 conn(0x7fcc2c0f8540 msgr2=0x7fcc2c0fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:02:14.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.968+0000 7fcc30f2c700 1 -- 172.21.15.32:0/641801282 shutdown_connections 2024-05-12T04:02:14.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.968+0000 7fcc30f2c700 1 -- 172.21.15.32:0/641801282 wait complete. 2024-05-12T04:02:14.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.968+0000 7fcc30f2c700 1 Processor -- start 2024-05-12T04:02:14.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.968+0000 7fcc30f2c700 1 -- start start 2024-05-12T04:02:14.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.969+0000 7fcc30f2c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc2c0fce80 0x7fcc2c071c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:02:14.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.969+0000 7fcc30f2c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc2c0fdff0 0x7fcc2c0702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:02:14.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.969+0000 7fcc30f2c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2c072200 con 0x7fcc2c0fce80 2024-05-12T04:02:14.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.969+0000 7fcc30f2c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2c0707f0 con 0x7fcc2c0fdff0 2024-05-12T04:02:14.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.969+0000 7fcc2a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc2c0fce80 0x7fcc2c071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:02:14.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.969+0000 7fcc29d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc2c0fdff0 0x7fcc2c0702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:02:14.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.969+0000 7fcc2a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc2c0fce80 0x7fcc2c071c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47640/0 (socket says 172.21.15.32:47640) 2024-05-12T04:02:14.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.969+0000 7fcc29d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc2c0fdff0 0x7fcc2c0702b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.32:46576/0 (socket says 172.21.15.32:46576) 2024-05-12T04:02:14.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.969+0000 7fcc2a59c700 1 -- 172.21.15.32:0/2585526106 learned_addr learned my addr 172.21.15.32:0/2585526106 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:02:14.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.969+0000 7fcc29d9b700 1 -- 172.21.15.32:0/2585526106 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc2c0fce80 msgr2=0x7fcc2c071c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:02:14.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.969+0000 7fcc29d9b700 1 --2- 172.21.15.32:0/2585526106 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc2c0fce80 0x7fcc2c071c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:14.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.969+0000 7fcc29d9b700 1 -- 172.21.15.32:0/2585526106 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc20005040 con 0x7fcc2c0fdff0 2024-05-12T04:02:14.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.969+0000 7fcc29d9b700 1 --2- 172.21.15.32:0/2585526106 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc2c0fdff0 0x7fcc2c0702b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcc200094a0 tx=0x7fcc20009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:02:14.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.969+0000 7fcc1affd700 1 -- 172.21.15.32:0/2585526106 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc2000e600 con 0x7fcc2c0fdff0 2024-05-12T04:02:14.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.969+0000 7fcc1affd700 1 -- 172.21.15.32:0/2585526106 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcc2000f040 con 0x7fcc2c0fdff0 2024-05-12T04:02:14.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.970+0000 7fcc30f2c700 1 -- 172.21.15.32:0/2585526106 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc1c005040 con 0x7fcc2c0fdff0 2024-05-12T04:02:14.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.970+0000 7fcc1affd700 1 -- 172.21.15.32:0/2585526106 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc20013b90 con 0x7fcc2c0fdff0 2024-05-12T04:02:14.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.970+0000 7fcc30f2c700 1 -- 172.21.15.32:0/2585526106 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc2c070e70 con 0x7fcc2c0fdff0 2024-05-12T04:02:14.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.970+0000 7fcc1affd700 1 -- 172.21.15.32:0/2585526106 <== mon.1 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 31) v1 ==== 45688+0+0 (secure 0 0 0) 0x7fcc2001c410 con 0x7fcc2c0fdff0 2024-05-12T04:02:14.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.970+0000 7fcc30f2c700 1 -- 172.21.15.32:0/2585526106 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc2c04c710 con 0x7fcc2c0fdff0 2024-05-12T04:02:14.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.971+0000 7fcc1affd700 1 --2- 172.21.15.32:0/2585526106 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7fcc14038640 0x7fcc1403aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:02:14.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.971+0000 7fcc1affd700 1 -- 172.21.15.32:0/2585526106 <== mon.1 v2:172.21.15.33:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fcc2004cdb0 con 0x7fcc2c0fdff0 2024-05-12T04:02:14.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.971+0000 7fcc2a59c700 1 --2- 172.21.15.32:0/2585526106 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7fcc14038640 0x7fcc1403aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:02:14.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.972+0000 7fcc2a59c700 1 --2- 172.21.15.32:0/2585526106 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7fcc14038640 0x7fcc1403aaf0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c013600 tx=0x7fcc1c009e30 comp rx=0 tx=0).ready entity=mgr.24501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:02:14.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:14.976+0000 7fcc1affd700 1 -- 172.21.15.32:0/2585526106 <== mon.1 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181777 (secure 0 0 0) 0x7fcc20022630 con 0x7fcc2c0fdff0 2024-05-12T04:02:15.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:14 smithi033 ceph-mon[49525]: from='client.14786 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:02:15.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:14 smithi033 ceph-mon[49525]: from='client.24539 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:02:15.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:14 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' 2024-05-12T04:02:15.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:14 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' 2024-05-12T04:02:15.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:14 smithi033 ceph-mon[49525]: mgrmap e31: smithi032.ukatkq(active, since 4s) 2024-05-12T04:02:15.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:14 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/3011967120' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:02:15.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:14 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' 2024-05-12T04:02:15.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:14 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' 2024-05-12T04:02:15.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:14 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/2678224826' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-12T04:02:15.167 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:14 smithi032 ceph-mon[40008]: from='client.14786 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:02:15.167 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:14 smithi032 ceph-mon[40008]: from='client.24539 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:02:15.167 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:14 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' 2024-05-12T04:02:15.167 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:14 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' 2024-05-12T04:02:15.167 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:14 smithi032 ceph-mon[40008]: mgrmap e31: smithi032.ukatkq(active, since 4s) 2024-05-12T04:02:15.167 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:14 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/3011967120' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:02:15.167 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:14 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' 2024-05-12T04:02:15.168 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:14 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' 2024-05-12T04:02:15.168 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:14 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/2678224826' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-12T04:02:15.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:15.274+0000 7fcc30f2c700 1 -- 172.21.15.32:0/2585526106 --> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcc2c102360 con 0x7fcc14038640 2024-05-12T04:02:15.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:15.275+0000 7fcc1affd700 1 -- 172.21.15.32:0/2585526106 <== mgr.24501 v2:172.21.15.32:6800/3729453095 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7fcc2c102360 con 0x7fcc14038640 2024-05-12T04:02:15.275 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-05-12T04:02:15.275 INFO:teuthology.orchestra.run.smithi032.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:19c9734256aa43a60e203feb242f50bf8b7f58666d520536af43cc4fee96173a", 2024-05-12T04:02:15.275 INFO:teuthology.orchestra.run.smithi032.stdout: "in_progress": true, 2024-05-12T04:02:15.275 INFO:teuthology.orchestra.run.smithi032.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-12T04:02:15.275 INFO:teuthology.orchestra.run.smithi032.stdout: "services_complete": [ 2024-05-12T04:02:15.275 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr" 2024-05-12T04:02:15.275 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-05-12T04:02:15.276 INFO:teuthology.orchestra.run.smithi032.stdout: "progress": "2/21 daemons upgraded", 2024-05-12T04:02:15.276 INFO:teuthology.orchestra.run.smithi032.stdout: "message": "", 2024-05-12T04:02:15.276 INFO:teuthology.orchestra.run.smithi032.stdout: "is_paused": false 2024-05-12T04:02:15.276 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-05-12T04:02:15.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:15.277+0000 7fcc30f2c700 1 -- 172.21.15.32:0/2585526106 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7fcc14038640 msgr2=0x7fcc1403aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:02:15.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:15.277+0000 7fcc30f2c700 1 --2- 172.21.15.32:0/2585526106 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7fcc14038640 0x7fcc1403aaf0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c013600 tx=0x7fcc1c009e30 comp rx=0 tx=0).stop 2024-05-12T04:02:15.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:15.277+0000 7fcc30f2c700 1 -- 172.21.15.32:0/2585526106 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc2c0fdff0 msgr2=0x7fcc2c0702b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:02:15.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:15.277+0000 7fcc30f2c700 1 --2- 172.21.15.32:0/2585526106 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc2c0fdff0 0x7fcc2c0702b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcc200094a0 tx=0x7fcc20009e30 comp rx=0 tx=0).stop 2024-05-12T04:02:15.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:15.277+0000 7fcc30f2c700 1 -- 172.21.15.32:0/2585526106 shutdown_connections 2024-05-12T04:02:15.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:15.277+0000 7fcc30f2c700 1 --2- 172.21.15.32:0/2585526106 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7fcc14038640 0x7fcc1403aaf0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:15.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:15.277+0000 7fcc30f2c700 1 --2- 172.21.15.32:0/2585526106 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc2c0fce80 0x7fcc2c071c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:15.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:15.277+0000 7fcc30f2c700 1 --2- 172.21.15.32:0/2585526106 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc2c0fdff0 0x7fcc2c0702b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:15.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:15.277+0000 7fcc30f2c700 1 -- 172.21.15.32:0/2585526106 >> 172.21.15.32:0/2585526106 conn(0x7fcc2c0f8540 msgr2=0x7fcc2c0f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:02:15.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:15.278+0000 7fcc30f2c700 1 -- 172.21.15.32:0/2585526106 shutdown_connections 2024-05-12T04:02:15.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:15.278+0000 7fcc30f2c700 1 -- 172.21.15.32:0/2585526106 wait complete. 2024-05-12T04:02:15.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:15.407+0000 7f2314991700 1 -- 172.21.15.32:0/2938418806 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23100fe3e0 msgr2=0x7f23100fe7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:02:15.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:15.407+0000 7f2314991700 1 --2- 172.21.15.32:0/2938418806 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23100fe3e0 0x7f23100fe7f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2300004940 tx=0x7f2300009e30 comp rx=0 tx=0).stop 2024-05-12T04:02:15.407 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:15.407+0000 7f2314991700 1 -- 172.21.15.32:0/2938418806 shutdown_connections 2024-05-12T04:02:15.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:15.407+0000 7f2314991700 1 --2- 172.21.15.32:0/2938418806 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23100fed30 0x7f23100fae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:15.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:15.407+0000 7f2314991700 1 --2- 172.21.15.32:0/2938418806 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23100fe3e0 0x7f23100fe7f0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:15.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:15.407+0000 7f2314991700 1 -- 172.21.15.32:0/2938418806 >> 172.21.15.32:0/2938418806 conn(0x7f23100f6b80 msgr2=0x7f23100f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:02:15.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:15.407+0000 7f2314991700 1 -- 172.21.15.32:0/2938418806 shutdown_connections 2024-05-12T04:02:15.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:15.407+0000 7f2314991700 1 -- 172.21.15.32:0/2938418806 wait complete. 2024-05-12T04:02:15.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:15.407+0000 7f2314991700 1 Processor -- start 2024-05-12T04:02:15.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:15.408+0000 7f2314991700 1 -- start start 2024-05-12T04:02:15.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:15.408+0000 7f2314991700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23100fe3e0 0x7f23100fdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:02:15.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:15.408+0000 7f2314991700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23100fed30 0x7f23100fc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:02:15.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:15.408+0000 7f2314991700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23101114e0 con 0x7f23100fed30 2024-05-12T04:02:15.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:15.408+0000 7f2314991700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2310111650 con 0x7f23100fe3e0 2024-05-12T04:02:15.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:15.408+0000 7f230dd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23100fed30 0x7f23100fc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:02:15.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:15.408+0000 7f230dd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23100fed30 0x7f23100fc0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47652/0 (socket says 172.21.15.32:47652) 2024-05-12T04:02:15.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:15.408+0000 7f230dd9b700 1 -- 172.21.15.32:0/3402281903 learned_addr learned my addr 172.21.15.32:0/3402281903 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:02:15.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:15.408+0000 7f230e59c700 1 --2- 172.21.15.32:0/3402281903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23100fe3e0 0x7f23100fdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:02:15.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:15.408+0000 7f230dd9b700 1 -- 172.21.15.32:0/3402281903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23100fe3e0 msgr2=0x7f23100fdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:02:15.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:15.408+0000 7f230dd9b700 1 --2- 172.21.15.32:0/3402281903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23100fe3e0 0x7f23100fdbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:15.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:15.408+0000 7f230dd9b700 1 -- 172.21.15.32:0/3402281903 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2304005040 con 0x7f23100fed30 2024-05-12T04:02:15.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:15.408+0000 7f230dd9b700 1 --2- 172.21.15.32:0/3402281903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23100fed30 0x7f23100fc0a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f23040096d0 tx=0x7f2304009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:02:15.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:15.409+0000 7f22feffd700 1 -- 172.21.15.32:0/3402281903 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f230400e840 con 0x7f23100fed30 2024-05-12T04:02:15.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:15.409+0000 7f22feffd700 1 -- 172.21.15.32:0/3402281903 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f230400f040 con 0x7f23100fed30 2024-05-12T04:02:15.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:15.409+0000 7f22feffd700 1 -- 172.21.15.32:0/3402281903 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2304013d30 con 0x7f23100fed30 2024-05-12T04:02:15.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:15.409+0000 7f2314991700 1 -- 172.21.15.32:0/3402281903 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2300005040 con 0x7f23100fed30 2024-05-12T04:02:15.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:15.409+0000 7f2314991700 1 -- 172.21.15.32:0/3402281903 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2310111d00 con 0x7f23100fed30 2024-05-12T04:02:15.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:15.410+0000 7f2314991700 1 -- 172.21.15.32:0/3402281903 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f231005f5c0 con 0x7f23100fed30 2024-05-12T04:02:15.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:15.411+0000 7f22feffd700 1 -- 172.21.15.32:0/3402281903 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 31) v1 ==== 45688+0+0 (secure 0 0 0) 0x7f230400e9a0 con 0x7f23100fed30 2024-05-12T04:02:15.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:15.411+0000 7f22feffd700 1 --2- 172.21.15.32:0/3402281903 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f22f8038640 0x7f22f803aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:02:15.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:15.411+0000 7f22feffd700 1 -- 172.21.15.32:0/3402281903 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f230404bf20 con 0x7f23100fed30 2024-05-12T04:02:15.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:15.411+0000 7f230e59c700 1 --2- 172.21.15.32:0/3402281903 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f22f8038640 0x7f22f803aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:02:15.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:15.412+0000 7f230e59c700 1 --2- 172.21.15.32:0/3402281903 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f22f8038640 0x7f22f803aaf0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f23000137d0 tx=0x7f2300009e30 comp rx=0 tx=0).ready entity=mgr.24501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:02:15.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:15.417+0000 7f22feffd700 1 -- 172.21.15.32:0/3402281903 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181777 (secure 0 0 0) 0x7f2304011c90 con 0x7f23100fed30 2024-05-12T04:02:15.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:15.774+0000 7f2314991700 1 -- 172.21.15.32:0/3402281903 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f231004bed0 con 0x7f23100fed30 2024-05-12T04:02:15.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:15.774+0000 7f22feffd700 1 -- 172.21.15.32:0/3402281903 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f2304011ab0 con 0x7f23100fed30 2024-05-12T04:02:15.775 INFO:teuthology.orchestra.run.smithi032.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-12T04:02:15.775 INFO:teuthology.orchestra.run.smithi032.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-12T04:02:15.775 INFO:teuthology.orchestra.run.smithi032.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-12T04:02:15.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:15.777+0000 7f2314991700 1 -- 172.21.15.32:0/3402281903 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f22f8038640 msgr2=0x7f22f803aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:02:15.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:15.777+0000 7f2314991700 1 --2- 172.21.15.32:0/3402281903 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f22f8038640 0x7f22f803aaf0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f23000137d0 tx=0x7f2300009e30 comp rx=0 tx=0).stop 2024-05-12T04:02:15.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:15.777+0000 7f2314991700 1 -- 172.21.15.32:0/3402281903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23100fed30 msgr2=0x7f23100fc0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:02:15.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:15.777+0000 7f2314991700 1 --2- 172.21.15.32:0/3402281903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23100fed30 0x7f23100fc0a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f23040096d0 tx=0x7f2304009e30 comp rx=0 tx=0).stop 2024-05-12T04:02:15.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:15.777+0000 7f2314991700 1 -- 172.21.15.32:0/3402281903 shutdown_connections 2024-05-12T04:02:15.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:15.777+0000 7f2314991700 1 --2- 172.21.15.32:0/3402281903 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f22f8038640 0x7f22f803aaf0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:15.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:15.777+0000 7f2314991700 1 --2- 172.21.15.32:0/3402281903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23100fe3e0 0x7f23100fdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:15.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:15.777+0000 7f2314991700 1 --2- 172.21.15.32:0/3402281903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23100fed30 0x7f23100fc0a0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:15.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:15.777+0000 7f2314991700 1 -- 172.21.15.32:0/3402281903 >> 172.21.15.32:0/3402281903 conn(0x7f23100f6b80 msgr2=0x7f23100f82b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:02:15.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:15.777+0000 7f2314991700 1 -- 172.21.15.32:0/3402281903 shutdown_connections 2024-05-12T04:02:15.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:15.778+0000 7f2314991700 1 -- 172.21.15.32:0/3402281903 wait complete. 2024-05-12T04:02:16.151 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:15 smithi032 ceph-mon[40008]: pgmap v5: 65 pgs: 65 active+clean; 287 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail 2024-05-12T04:02:16.151 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:15 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/3402281903' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-12T04:02:16.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:15 smithi033 ceph-mon[49525]: pgmap v5: 65 pgs: 65 active+clean; 287 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail 2024-05-12T04:02:16.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:15 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/3402281903' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-12T04:02:17.153 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:16 smithi032 ceph-mon[40008]: from='client.24551 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:02:17.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:16 smithi033 ceph-mon[49525]: from='client.24551 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:02:18.145 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:17 smithi033 ceph-mon[49525]: pgmap v6: 65 pgs: 65 active+clean; 287 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 820 B/s rd, 1.4 KiB/s wr, 8 op/s 2024-05-12T04:02:18.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:17 smithi032 ceph-mon[40008]: pgmap v6: 65 pgs: 65 active+clean; 287 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 820 B/s rd, 1.4 KiB/s wr, 8 op/s 2024-05-12T04:02:19.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:18 smithi033 ceph-mon[49525]: Standby manager daemon smithi033.pcvxwy started 2024-05-12T04:02:19.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:18 smithi033 ceph-mon[49525]: from='mgr.? 172.21.15.33:0/1779004703' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi033.pcvxwy/crt"}]: dispatch 2024-05-12T04:02:19.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:18 smithi033 ceph-mon[49525]: from='mgr.? 172.21.15.33:0/1779004703' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-12T04:02:19.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:18 smithi033 ceph-mon[49525]: from='mgr.? 172.21.15.33:0/1779004703' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi033.pcvxwy/key"}]: dispatch 2024-05-12T04:02:19.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:18 smithi033 ceph-mon[49525]: from='mgr.? 172.21.15.33:0/1779004703' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-12T04:02:19.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:18 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' 2024-05-12T04:02:19.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:18 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' 2024-05-12T04:02:19.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:18 smithi032 ceph-mon[40008]: Standby manager daemon smithi033.pcvxwy started 2024-05-12T04:02:19.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:18 smithi032 ceph-mon[40008]: from='mgr.? 172.21.15.33:0/1779004703' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi033.pcvxwy/crt"}]: dispatch 2024-05-12T04:02:19.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:18 smithi032 ceph-mon[40008]: from='mgr.? 172.21.15.33:0/1779004703' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-12T04:02:19.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:18 smithi032 ceph-mon[40008]: from='mgr.? 172.21.15.33:0/1779004703' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi033.pcvxwy/key"}]: dispatch 2024-05-12T04:02:19.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:18 smithi032 ceph-mon[40008]: from='mgr.? 172.21.15.33:0/1779004703' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-12T04:02:19.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:18 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' 2024-05-12T04:02:19.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:18 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' 2024-05-12T04:02:20.141 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:19 smithi032 ceph-mon[40008]: pgmap v7: 65 pgs: 65 active+clean; 292 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 85 op/s 2024-05-12T04:02:20.141 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:19 smithi032 ceph-mon[40008]: mgrmap e32: smithi032.ukatkq(active, since 8s), standbys: smithi033.pcvxwy 2024-05-12T04:02:20.141 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:19 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mgr metadata", "who": "smithi033.pcvxwy", "id": "smithi033.pcvxwy"}]: dispatch 2024-05-12T04:02:20.142 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:19 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' 2024-05-12T04:02:20.142 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:19 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' 2024-05-12T04:02:20.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:19 smithi033 ceph-mon[49525]: pgmap v7: 65 pgs: 65 active+clean; 292 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 85 op/s 2024-05-12T04:02:20.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:19 smithi033 ceph-mon[49525]: mgrmap e32: smithi032.ukatkq(active, since 8s), standbys: smithi033.pcvxwy 2024-05-12T04:02:20.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:19 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mgr metadata", "who": "smithi033.pcvxwy", "id": "smithi033.pcvxwy"}]: dispatch 2024-05-12T04:02:20.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:19 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' 2024-05-12T04:02:20.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:19 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' 2024-05-12T04:02:22.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:21 smithi033 ceph-mon[49525]: pgmap v8: 65 pgs: 65 active+clean; 292 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 901 KiB/s rd, 816 KiB/s wr, 66 op/s 2024-05-12T04:02:22.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:21 smithi032 ceph-mon[40008]: pgmap v8: 65 pgs: 65 active+clean; 292 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 901 KiB/s rd, 816 KiB/s wr, 66 op/s 2024-05-12T04:02:24.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:23 smithi033 ceph-mon[49525]: pgmap v9: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 102 op/s 2024-05-12T04:02:24.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:23 smithi032 ceph-mon[40008]: pgmap v9: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 102 op/s 2024-05-12T04:02:25.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:25 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' 2024-05-12T04:02:25.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:25 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' 2024-05-12T04:02:25.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:25 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:02:25.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:25 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T04:02:25.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:25 smithi032 ceph-mon[40008]: Updating smithi032:/etc/ceph/ceph.conf 2024-05-12T04:02:25.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:25 smithi032 ceph-mon[40008]: Updating smithi033:/etc/ceph/ceph.conf 2024-05-12T04:02:25.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:25 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' 2024-05-12T04:02:25.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:25 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' 2024-05-12T04:02:25.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:25 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:02:25.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:25 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T04:02:25.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:25 smithi033 ceph-mon[49525]: Updating smithi032:/etc/ceph/ceph.conf 2024-05-12T04:02:25.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:25 smithi033 ceph-mon[49525]: Updating smithi033:/etc/ceph/ceph.conf 2024-05-12T04:02:26.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:26 smithi033 ceph-mon[49525]: pgmap v10: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 93 op/s 2024-05-12T04:02:26.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:26 smithi033 ceph-mon[49525]: Updating smithi033:/var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-12T04:02:26.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:26 smithi033 ceph-mon[49525]: Updating smithi032:/var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-12T04:02:26.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:26 smithi033 ceph-mon[49525]: Updating smithi033:/etc/ceph/ceph.client.admin.keyring 2024-05-12T04:02:26.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:26 smithi033 ceph-mon[49525]: Updating smithi032:/etc/ceph/ceph.client.admin.keyring 2024-05-12T04:02:26.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:26 smithi033 ceph-mon[49525]: Updating smithi033:/var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-12T04:02:26.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:26 smithi033 ceph-mon[49525]: Updating smithi032:/var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-12T04:02:26.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:26 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' 2024-05-12T04:02:26.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:26 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' 2024-05-12T04:02:26.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:26 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:02:26.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:26 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' 2024-05-12T04:02:26.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:26 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' 2024-05-12T04:02:26.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:26 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' 2024-05-12T04:02:26.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:26 smithi032 ceph-mon[40008]: pgmap v10: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 93 op/s 2024-05-12T04:02:26.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:26 smithi032 ceph-mon[40008]: Updating smithi033:/var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-12T04:02:26.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:26 smithi032 ceph-mon[40008]: Updating smithi032:/var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-12T04:02:26.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:26 smithi032 ceph-mon[40008]: Updating smithi033:/etc/ceph/ceph.client.admin.keyring 2024-05-12T04:02:26.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:26 smithi032 ceph-mon[40008]: Updating smithi032:/etc/ceph/ceph.client.admin.keyring 2024-05-12T04:02:26.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:26 smithi032 ceph-mon[40008]: Updating smithi033:/var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-12T04:02:26.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:26 smithi032 ceph-mon[40008]: Updating smithi032:/var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-12T04:02:26.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:26 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' 2024-05-12T04:02:26.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:26 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' 2024-05-12T04:02:26.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:26 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:02:26.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:26 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' 2024-05-12T04:02:26.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:26 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' 2024-05-12T04:02:26.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:26 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' 2024-05-12T04:02:27.284 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:27 smithi032 ceph-mon[40008]: Reconfiguring prometheus.smithi032 (dependencies changed)... 2024-05-12T04:02:27.284 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:27 smithi032 ceph-mon[40008]: Reconfiguring daemon prometheus.smithi032 on smithi032 2024-05-12T04:02:27.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:27 smithi033 ceph-mon[49525]: Reconfiguring prometheus.smithi032 (dependencies changed)... 2024-05-12T04:02:27.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:27 smithi033 ceph-mon[49525]: Reconfiguring daemon prometheus.smithi032 on smithi032 2024-05-12T04:02:28.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:28 smithi032 ceph-mon[40008]: pgmap v11: 65 pgs: 65 active+clean; 305 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 114 op/s 2024-05-12T04:02:28.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:28 smithi033 ceph-mon[49525]: pgmap v11: 65 pgs: 65 active+clean; 305 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 114 op/s 2024-05-12T04:02:30.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:30 smithi032 ceph-mon[40008]: pgmap v12: 65 pgs: 65 active+clean; 301 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 130 op/s 2024-05-12T04:02:30.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:30 smithi033 ceph-mon[49525]: pgmap v12: 65 pgs: 65 active+clean; 301 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 130 op/s 2024-05-12T04:02:32.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:32 smithi032 ceph-mon[40008]: pgmap v13: 65 pgs: 65 active+clean; 301 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 84 op/s 2024-05-12T04:02:32.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:32 smithi033 ceph-mon[49525]: pgmap v13: 65 pgs: 65 active+clean; 301 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 84 op/s 2024-05-12T04:02:34.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:34 smithi033 ceph-mon[49525]: pgmap v14: 65 pgs: 65 active+clean; 302 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 132 op/s 2024-05-12T04:02:34.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:34 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' 2024-05-12T04:02:34.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:34 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' 2024-05-12T04:02:34.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:34 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-12T04:02:34.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:34 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:02:34.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:34 smithi032 ceph-mon[40008]: pgmap v14: 65 pgs: 65 active+clean; 302 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 132 op/s 2024-05-12T04:02:34.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:34 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' 2024-05-12T04:02:34.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:34 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' 2024-05-12T04:02:34.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:34 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-12T04:02:34.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:34 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:02:35.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:35 smithi032 ceph-mon[40008]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-12T04:02:35.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:35 smithi033 ceph-mon[49525]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-12T04:02:36.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:36 smithi033 ceph-mon[49525]: pgmap v15: 65 pgs: 65 active+clean; 302 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 89 op/s 2024-05-12T04:02:36.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:36 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi033.pcvxwy", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-12T04:02:36.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:36 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-12T04:02:36.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:36 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:02:36.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:36 smithi032 ceph-mon[40008]: pgmap v15: 65 pgs: 65 active+clean; 302 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 89 op/s 2024-05-12T04:02:36.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:36 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi033.pcvxwy", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-12T04:02:36.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:36 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-12T04:02:36.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:36 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:02:37.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:37 smithi033 ceph-mon[49525]: Upgrade: Updating mgr.smithi033.pcvxwy 2024-05-12T04:02:37.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:37 smithi033 ceph-mon[49525]: Deploying daemon mgr.smithi033.pcvxwy on smithi033 2024-05-12T04:02:37.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:37 smithi032 ceph-mon[40008]: Upgrade: Updating mgr.smithi033.pcvxwy 2024-05-12T04:02:37.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:37 smithi032 ceph-mon[40008]: Deploying daemon mgr.smithi033.pcvxwy on smithi033 2024-05-12T04:02:38.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:38 smithi033 ceph-mon[49525]: pgmap v16: 65 pgs: 65 active+clean; 305 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 103 op/s 2024-05-12T04:02:38.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:38 smithi032 ceph-mon[40008]: pgmap v16: 65 pgs: 65 active+clean; 305 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 103 op/s 2024-05-12T04:02:39.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:39 smithi033 ceph-mon[49525]: pgmap v17: 65 pgs: 65 active+clean; 307 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 101 op/s 2024-05-12T04:02:39.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:39 smithi032 ceph-mon[40008]: pgmap v17: 65 pgs: 65 active+clean; 307 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 101 op/s 2024-05-12T04:02:40.792 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:40 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' 2024-05-12T04:02:40.792 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:40 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' 2024-05-12T04:02:40.792 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:40 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:02:40.792 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:40 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:02:40.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:40 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' 2024-05-12T04:02:40.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:40 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' 2024-05-12T04:02:40.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:40 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:02:40.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:40 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:02:41.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:41 smithi033 ceph-mon[49525]: pgmap v18: 65 pgs: 65 active+clean; 307 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 81 op/s 2024-05-12T04:02:42.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:41 smithi032 ceph-mon[40008]: pgmap v18: 65 pgs: 65 active+clean; 307 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 81 op/s 2024-05-12T04:02:43.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:43 smithi032 ceph-mon[40008]: pgmap v19: 65 pgs: 65 active+clean; 307 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 131 op/s 2024-05-12T04:02:43.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:43 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' 2024-05-12T04:02:43.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:43 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' 2024-05-12T04:02:43.891 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:43 smithi033 ceph-mon[49525]: pgmap v19: 65 pgs: 65 active+clean; 307 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 131 op/s 2024-05-12T04:02:43.891 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:43 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' 2024-05-12T04:02:43.891 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:43 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' 2024-05-12T04:02:45.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:45 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' 2024-05-12T04:02:45.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:45 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' 2024-05-12T04:02:45.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:45 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' 2024-05-12T04:02:45.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:45 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' 2024-05-12T04:02:45.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:45.920+0000 7f6399f0d700 1 -- 172.21.15.32:0/4024556250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63940fcf60 msgr2=0x7f63940fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:02:45.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:45.920+0000 7f6399f0d700 1 --2- 172.21.15.32:0/4024556250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63940fcf60 0x7f63940fd3d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6388004940 tx=0x7f6388009e30 comp rx=0 tx=0).stop 2024-05-12T04:02:45.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:45.920+0000 7f6399f0d700 1 -- 172.21.15.32:0/4024556250 shutdown_connections 2024-05-12T04:02:45.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:45.920+0000 7f6399f0d700 1 --2- 172.21.15.32:0/4024556250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63940fcf60 0x7f63940fd3d0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:45.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:45.920+0000 7f6399f0d700 1 --2- 172.21.15.32:0/4024556250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63940fbcb0 0x7f63940fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:45.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:45.920+0000 7f6399f0d700 1 -- 172.21.15.32:0/4024556250 >> 172.21.15.32:0/4024556250 conn(0x7f63940f7430 msgr2=0x7f63940f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:02:45.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:45.920+0000 7f6399f0d700 1 -- 172.21.15.32:0/4024556250 shutdown_connections 2024-05-12T04:02:45.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:45.920+0000 7f6399f0d700 1 -- 172.21.15.32:0/4024556250 wait complete. 2024-05-12T04:02:45.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:45.921+0000 7f6399f0d700 1 Processor -- start 2024-05-12T04:02:45.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:45.921+0000 7f6399f0d700 1 -- start start 2024-05-12T04:02:45.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:45.921+0000 7f6399f0d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63940fbcb0 0x7f63940ffd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:02:45.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:45.921+0000 7f6399f0d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63940fcf60 0x7f63940fe2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:02:45.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:45.921+0000 7f6399f0d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63940fe830 con 0x7f63940fbcb0 2024-05-12T04:02:45.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:45.921+0000 7f6399f0d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63940fe970 con 0x7f63940fcf60 2024-05-12T04:02:45.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:45.921+0000 7f63937fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63940fbcb0 0x7f63940ffd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:02:45.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:45.921+0000 7f63937fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63940fbcb0 0x7f63940ffd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36886/0 (socket says 172.21.15.32:36886) 2024-05-12T04:02:45.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:45.921+0000 7f63937fe700 1 -- 172.21.15.32:0/4141297916 learned_addr learned my addr 172.21.15.32:0/4141297916 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:02:45.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:45.921+0000 7f6392ffd700 1 --2- 172.21.15.32:0/4141297916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63940fcf60 0x7f63940fe2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:02:45.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:45.922+0000 7f63937fe700 1 -- 172.21.15.32:0/4141297916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63940fcf60 msgr2=0x7f63940fe2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:02:45.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:45.922+0000 7f63937fe700 1 --2- 172.21.15.32:0/4141297916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63940fcf60 0x7f63940fe2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:45.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:45.922+0000 7f63937fe700 1 -- 172.21.15.32:0/4141297916 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6388005040 con 0x7f63940fbcb0 2024-05-12T04:02:45.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:45.922+0000 7f6392ffd700 1 --2- 172.21.15.32:0/4141297916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63940fcf60 0x7f63940fe2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-12T04:02:45.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:45.922+0000 7f63937fe700 1 --2- 172.21.15.32:0/4141297916 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63940fbcb0 0x7f63940ffd80 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f638400eb40 tx=0x7f638400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:02:45.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:45.922+0000 7f637bfff700 1 -- 172.21.15.32:0/4141297916 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6384007cc0 con 0x7f63940fbcb0 2024-05-12T04:02:45.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:45.922+0000 7f637bfff700 1 -- 172.21.15.32:0/4141297916 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f638400f040 con 0x7f63940fbcb0 2024-05-12T04:02:45.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:45.922+0000 7f637bfff700 1 -- 172.21.15.32:0/4141297916 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f638400c990 con 0x7f63940fbcb0 2024-05-12T04:02:45.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:45.922+0000 7f6399f0d700 1 -- 172.21.15.32:0/4141297916 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63940febf0 con 0x7f63940fbcb0 2024-05-12T04:02:45.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:45.922+0000 7f6399f0d700 1 -- 172.21.15.32:0/4141297916 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6394115900 con 0x7f63940fbcb0 2024-05-12T04:02:45.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:45.924+0000 7f6399f0d700 1 -- 172.21.15.32:0/4141297916 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6394101af0 con 0x7f63940fbcb0 2024-05-12T04:02:45.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:45.927+0000 7f637bfff700 1 -- 172.21.15.32:0/4141297916 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 32) v1 ==== 90813+0+0 (secure 0 0 0) 0x7f638400caf0 con 0x7f63940fbcb0 2024-05-12T04:02:45.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:45.927+0000 7f637bfff700 1 --2- 172.21.15.32:0/4141297916 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f637c06ca90 0x7f637c06ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:02:45.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:45.927+0000 7f637bfff700 1 -- 172.21.15.32:0/4141297916 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f638408bb30 con 0x7f63940fbcb0 2024-05-12T04:02:45.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:45.927+0000 7f6392ffd700 1 --2- 172.21.15.32:0/4141297916 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f637c06ca90 0x7f637c06ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:02:45.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:45.928+0000 7f6392ffd700 1 --2- 172.21.15.32:0/4141297916 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f637c06ca90 0x7f637c06ef40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6394100360 tx=0x7f6388009e30 comp rx=0 tx=0).ready entity=mgr.24501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:02:45.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:45.932+0000 7f637bfff700 1 -- 172.21.15.32:0/4141297916 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181777 (secure 0 0 0) 0x7f6384055e50 con 0x7f63940fbcb0 2024-05-12T04:02:46.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.274+0000 7f6399f0d700 1 -- 172.21.15.32:0/4141297916 --> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f639405e3a0 con 0x7f637c06ca90 2024-05-12T04:02:46.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.275+0000 7f637bfff700 1 -- 172.21.15.32:0/4141297916 <== mgr.24501 v2:172.21.15.32:6800/3729453095 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f639405e3a0 con 0x7f637c06ca90 2024-05-12T04:02:46.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.278+0000 7f6399f0d700 1 -- 172.21.15.32:0/4141297916 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f637c06ca90 msgr2=0x7f637c06ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:02:46.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.278+0000 7f6399f0d700 1 --2- 172.21.15.32:0/4141297916 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f637c06ca90 0x7f637c06ef40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6394100360 tx=0x7f6388009e30 comp rx=0 tx=0).stop 2024-05-12T04:02:46.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.278+0000 7f6399f0d700 1 -- 172.21.15.32:0/4141297916 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63940fbcb0 msgr2=0x7f63940ffd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:02:46.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.278+0000 7f6399f0d700 1 --2- 172.21.15.32:0/4141297916 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63940fbcb0 0x7f63940ffd80 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f638400eb40 tx=0x7f638400b040 comp rx=0 tx=0).stop 2024-05-12T04:02:46.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.278+0000 7f6399f0d700 1 -- 172.21.15.32:0/4141297916 shutdown_connections 2024-05-12T04:02:46.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.278+0000 7f6399f0d700 1 --2- 172.21.15.32:0/4141297916 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f637c06ca90 0x7f637c06ef40 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:46.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.278+0000 7f6399f0d700 1 --2- 172.21.15.32:0/4141297916 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63940fbcb0 0x7f63940ffd80 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:46.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.278+0000 7f6399f0d700 1 --2- 172.21.15.32:0/4141297916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63940fcf60 0x7f63940fe2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:46.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.278+0000 7f6399f0d700 1 -- 172.21.15.32:0/4141297916 >> 172.21.15.32:0/4141297916 conn(0x7f63940f7430 msgr2=0x7f63940f8b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:02:46.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.278+0000 7f6399f0d700 1 -- 172.21.15.32:0/4141297916 shutdown_connections 2024-05-12T04:02:46.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.278+0000 7f6399f0d700 1 -- 172.21.15.32:0/4141297916 wait complete. 2024-05-12T04:02:46.289 INFO:teuthology.orchestra.run.smithi032.stdout:true 2024-05-12T04:02:46.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:46 smithi033 ceph-mon[49525]: pgmap v20: 65 pgs: 65 active+clean; 307 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 82 op/s 2024-05-12T04:02:46.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.416+0000 7f32a0d0b700 1 -- 172.21.15.32:0/2735094157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f329c0ffee0 msgr2=0x7f329c102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:02:46.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.416+0000 7f32a0d0b700 1 --2- 172.21.15.32:0/2735094157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f329c0ffee0 0x7f329c102300 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f3284004940 tx=0x7f3284009e30 comp rx=0 tx=0).stop 2024-05-12T04:02:46.416 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.416+0000 7f32a0d0b700 1 -- 172.21.15.32:0/2735094157 shutdown_connections 2024-05-12T04:02:46.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.416+0000 7f32a0d0b700 1 --2- 172.21.15.32:0/2735094157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f329c0ffee0 0x7f329c102300 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:46.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.416+0000 7f32a0d0b700 1 --2- 172.21.15.32:0/2735094157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f329c0662c0 0x7f329c0ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:46.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.416+0000 7f32a0d0b700 1 -- 172.21.15.32:0/2735094157 >> 172.21.15.32:0/2735094157 conn(0x7f329c0f7430 msgr2=0x7f329c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:02:46.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.416+0000 7f32a0d0b700 1 -- 172.21.15.32:0/2735094157 shutdown_connections 2024-05-12T04:02:46.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.416+0000 7f32a0d0b700 1 -- 172.21.15.32:0/2735094157 wait complete. 2024-05-12T04:02:46.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.417+0000 7f32a0d0b700 1 Processor -- start 2024-05-12T04:02:46.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.417+0000 7f32a0d0b700 1 -- start start 2024-05-12T04:02:46.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.417+0000 7f32a0d0b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f329c0662c0 0x7f329c0fd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:02:46.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.417+0000 7f32a0d0b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f329c0ffee0 0x7f329c0fb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:02:46.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.417+0000 7f32a0d0b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f329c0fbde0 con 0x7f329c0662c0 2024-05-12T04:02:46.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.417+0000 7f32a0d0b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f329c0fbf20 con 0x7f329c0ffee0 2024-05-12T04:02:46.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.417+0000 7f329a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f329c0662c0 0x7f329c0fd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:02:46.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.417+0000 7f329a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f329c0662c0 0x7f329c0fd330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36904/0 (socket says 172.21.15.32:36904) 2024-05-12T04:02:46.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.417+0000 7f329a59c700 1 -- 172.21.15.32:0/2625357077 learned_addr learned my addr 172.21.15.32:0/2625357077 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:02:46.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.418+0000 7f3299d9b700 1 --2- 172.21.15.32:0/2625357077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f329c0ffee0 0x7f329c0fb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:02:46.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.418+0000 7f329a59c700 1 -- 172.21.15.32:0/2625357077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f329c0ffee0 msgr2=0x7f329c0fb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:02:46.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.418+0000 7f329a59c700 1 --2- 172.21.15.32:0/2625357077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f329c0ffee0 0x7f329c0fb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:46.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.418+0000 7f329a59c700 1 -- 172.21.15.32:0/2625357077 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3284005040 con 0x7f329c0662c0 2024-05-12T04:02:46.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.418+0000 7f3299d9b700 1 --2- 172.21.15.32:0/2625357077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f329c0ffee0 0x7f329c0fb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-12T04:02:46.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.418+0000 7f329a59c700 1 --2- 172.21.15.32:0/2625357077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f329c0662c0 0x7f329c0fd330 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f328c00eb40 tx=0x7f328c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:02:46.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.418+0000 7f3292ffd700 1 -- 172.21.15.32:0/2625357077 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f328c007cc0 con 0x7f329c0662c0 2024-05-12T04:02:46.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.418+0000 7f32a0d0b700 1 -- 172.21.15.32:0/2625357077 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f329c0fc1a0 con 0x7f329c0662c0 2024-05-12T04:02:46.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.418+0000 7f32a0d0b700 1 -- 172.21.15.32:0/2625357077 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f329c1159a0 con 0x7f329c0662c0 2024-05-12T04:02:46.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.419+0000 7f3292ffd700 1 -- 172.21.15.32:0/2625357077 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f328c00f040 con 0x7f329c0662c0 2024-05-12T04:02:46.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.419+0000 7f3292ffd700 1 -- 172.21.15.32:0/2625357077 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f328c00c8f0 con 0x7f329c0662c0 2024-05-12T04:02:46.420 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.420+0000 7f32a0d0b700 1 -- 172.21.15.32:0/2625357077 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f327c0052e0 con 0x7f329c0662c0 2024-05-12T04:02:46.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.421+0000 7f3292ffd700 1 -- 172.21.15.32:0/2625357077 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 32) v1 ==== 90813+0+0 (secure 0 0 0) 0x7f328c009610 con 0x7f329c0662c0 2024-05-12T04:02:46.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.422+0000 7f3292ffd700 1 --2- 172.21.15.32:0/2625357077 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f328806ca90 0x7f328806ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:02:46.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.422+0000 7f3292ffd700 1 -- 172.21.15.32:0/2625357077 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f328c08b6b0 con 0x7f329c0662c0 2024-05-12T04:02:46.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.422+0000 7f3299d9b700 1 --2- 172.21.15.32:0/2625357077 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f328806ca90 0x7f328806ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:02:46.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.422+0000 7f3299d9b700 1 --2- 172.21.15.32:0/2625357077 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f328806ca90 0x7f328806ef40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3284009fd0 tx=0x7f3284009e30 comp rx=0 tx=0).ready entity=mgr.24501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:02:46.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.427+0000 7f3292ffd700 1 -- 172.21.15.32:0/2625357077 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181777 (secure 0 0 0) 0x7f328c055a00 con 0x7f329c0662c0 2024-05-12T04:02:46.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:46 smithi032 ceph-mon[40008]: pgmap v20: 65 pgs: 65 active+clean; 307 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 82 op/s 2024-05-12T04:02:46.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.705+0000 7f32a0d0b700 1 -- 172.21.15.32:0/2625357077 --> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f327c000bf0 con 0x7f328806ca90 2024-05-12T04:02:46.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.706+0000 7f3292ffd700 1 -- 172.21.15.32:0/2625357077 <== mgr.24501 v2:172.21.15.32:6800/3729453095 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f327c000bf0 con 0x7f328806ca90 2024-05-12T04:02:46.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.708+0000 7f32a0d0b700 1 -- 172.21.15.32:0/2625357077 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f328806ca90 msgr2=0x7f328806ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:02:46.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.708+0000 7f32a0d0b700 1 --2- 172.21.15.32:0/2625357077 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f328806ca90 0x7f328806ef40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3284009fd0 tx=0x7f3284009e30 comp rx=0 tx=0).stop 2024-05-12T04:02:46.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.708+0000 7f32a0d0b700 1 -- 172.21.15.32:0/2625357077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f329c0662c0 msgr2=0x7f329c0fd330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:02:46.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.708+0000 7f32a0d0b700 1 --2- 172.21.15.32:0/2625357077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f329c0662c0 0x7f329c0fd330 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f328c00eb40 tx=0x7f328c00b040 comp rx=0 tx=0).stop 2024-05-12T04:02:46.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.708+0000 7f32a0d0b700 1 -- 172.21.15.32:0/2625357077 shutdown_connections 2024-05-12T04:02:46.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.708+0000 7f32a0d0b700 1 --2- 172.21.15.32:0/2625357077 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f328806ca90 0x7f328806ef40 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:46.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.708+0000 7f32a0d0b700 1 --2- 172.21.15.32:0/2625357077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f329c0662c0 0x7f329c0fd330 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:46.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.708+0000 7f32a0d0b700 1 --2- 172.21.15.32:0/2625357077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f329c0ffee0 0x7f329c0fb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:46.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.708+0000 7f32a0d0b700 1 -- 172.21.15.32:0/2625357077 >> 172.21.15.32:0/2625357077 conn(0x7f329c0f7430 msgr2=0x7f329c0f9810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:02:46.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.708+0000 7f32a0d0b700 1 -- 172.21.15.32:0/2625357077 shutdown_connections 2024-05-12T04:02:46.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.708+0000 7f32a0d0b700 1 -- 172.21.15.32:0/2625357077 wait complete. 2024-05-12T04:02:46.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.830+0000 7f71075c7700 1 -- 172.21.15.32:0/1966947509 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71000fce60 msgr2=0x7f71000fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:02:46.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.830+0000 7f71075c7700 1 --2- 172.21.15.32:0/1966947509 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71000fce60 0x7f71000fd270 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f70fc004940 tx=0x7f70fc009e30 comp rx=0 tx=0).stop 2024-05-12T04:02:46.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.830+0000 7f71075c7700 1 -- 172.21.15.32:0/1966947509 shutdown_connections 2024-05-12T04:02:46.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.830+0000 7f71075c7700 1 --2- 172.21.15.32:0/1966947509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71000fdfd0 0x7f71000fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:46.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.830+0000 7f71075c7700 1 --2- 172.21.15.32:0/1966947509 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71000fce60 0x7f71000fd270 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:46.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.830+0000 7f71075c7700 1 -- 172.21.15.32:0/1966947509 >> 172.21.15.32:0/1966947509 conn(0x7f71000f8540 msgr2=0x7f71000fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:02:46.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.830+0000 7f71075c7700 1 -- 172.21.15.32:0/1966947509 shutdown_connections 2024-05-12T04:02:46.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.830+0000 7f71075c7700 1 -- 172.21.15.32:0/1966947509 wait complete. 2024-05-12T04:02:46.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.831+0000 7f71075c7700 1 Processor -- start 2024-05-12T04:02:46.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.831+0000 7f71075c7700 1 -- start start 2024-05-12T04:02:46.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.831+0000 7f71075c7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71000fce60 0x7f710010d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:02:46.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.831+0000 7f71075c7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71000fdfd0 0x7f710010ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:02:46.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.831+0000 7f71075c7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f710010e3b0 con 0x7f71000fdfd0 2024-05-12T04:02:46.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.831+0000 7f71075c7700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f710010ec00 con 0x7f71000fce60 2024-05-12T04:02:46.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.831+0000 7f7104b62700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71000fdfd0 0x7f710010ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:02:46.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.831+0000 7f7104b62700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71000fdfd0 0x7f710010ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36910/0 (socket says 172.21.15.32:36910) 2024-05-12T04:02:46.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.831+0000 7f7104b62700 1 -- 172.21.15.32:0/2076626305 learned_addr learned my addr 172.21.15.32:0/2076626305 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:02:46.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.831+0000 7f7105363700 1 --2- 172.21.15.32:0/2076626305 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71000fce60 0x7f710010d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:02:46.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.831+0000 7f7104b62700 1 -- 172.21.15.32:0/2076626305 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71000fce60 msgr2=0x7f710010d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:02:46.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.831+0000 7f7104b62700 1 --2- 172.21.15.32:0/2076626305 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71000fce60 0x7f710010d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:46.832 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.831+0000 7f7104b62700 1 -- 172.21.15.32:0/2076626305 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70fc005040 con 0x7f71000fdfd0 2024-05-12T04:02:46.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.831+0000 7f7105363700 1 --2- 172.21.15.32:0/2076626305 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71000fce60 0x7f710010d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-12T04:02:46.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.832+0000 7f7104b62700 1 --2- 172.21.15.32:0/2076626305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71000fdfd0 0x7f710010ddb0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f70f000eb40 tx=0x7f70f000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:02:46.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.832+0000 7f70f5ffb700 1 -- 172.21.15.32:0/2076626305 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70f0007cc0 con 0x7f71000fdfd0 2024-05-12T04:02:46.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.832+0000 7f71075c7700 1 -- 172.21.15.32:0/2076626305 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f710010ee80 con 0x7f71000fdfd0 2024-05-12T04:02:46.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.832+0000 7f70f5ffb700 1 -- 172.21.15.32:0/2076626305 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f70f000f040 con 0x7f71000fdfd0 2024-05-12T04:02:46.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.832+0000 7f70f5ffb700 1 -- 172.21.15.32:0/2076626305 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70f0009570 con 0x7f71000fdfd0 2024-05-12T04:02:46.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.832+0000 7f71075c7700 1 -- 172.21.15.32:0/2076626305 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f710010f3c0 con 0x7f71000fdfd0 2024-05-12T04:02:46.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.833+0000 7f71075c7700 1 -- 172.21.15.32:0/2076626305 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f710005f5c0 con 0x7f71000fdfd0 2024-05-12T04:02:46.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.834+0000 7f70f5ffb700 1 -- 172.21.15.32:0/2076626305 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 32) v1 ==== 90813+0+0 (secure 0 0 0) 0x7f70f00096d0 con 0x7f71000fdfd0 2024-05-12T04:02:46.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.834+0000 7f70f5ffb700 1 --2- 172.21.15.32:0/2076626305 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f70ec06c9c0 0x7f70ec06ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:02:46.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.834+0000 7f70f5ffb700 1 -- 172.21.15.32:0/2076626305 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f70f008ba00 con 0x7f71000fdfd0 2024-05-12T04:02:46.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.834+0000 7f7105363700 1 --2- 172.21.15.32:0/2076626305 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f70ec06c9c0 0x7f70ec06ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:02:46.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.835+0000 7f7105363700 1 --2- 172.21.15.32:0/2076626305 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f70ec06c9c0 0x7f70ec06ee70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f70fc0137b0 tx=0x7f70fc009e30 comp rx=0 tx=0).ready entity=mgr.24501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:02:46.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:46.839+0000 7f70f5ffb700 1 -- 172.21.15.32:0/2076626305 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181777 (secure 0 0 0) 0x7f70f0059800 con 0x7f71000fdfd0 2024-05-12T04:02:47.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.153+0000 7f71075c7700 1 -- 172.21.15.32:0/2076626305 --> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f710010f750 con 0x7f70ec06c9c0 2024-05-12T04:02:47.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.164+0000 7f70f5ffb700 1 -- 172.21.15.32:0/2076626305 <== mgr.24501 v2:172.21.15.32:6800/3729453095 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f710010f750 con 0x7f70ec06c9c0 2024-05-12T04:02:47.164 INFO:teuthology.orchestra.run.smithi032.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-12T04:02:47.164 INFO:teuthology.orchestra.run.smithi032.stdout:alertmanager.smithi032 smithi032 *:9093,9094 running (113s) 28s ago 10m 17.8M - 0.25.0 c8568f914cd2 a37fbab7af4c 2024-05-12T04:02:47.164 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi032 smithi032 running (11m) 28s ago 11m 6656k - 17.2.7-939-gc190bdee 284803f7d9cf a478874e9dfa 2024-05-12T04:02:47.164 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi033 smithi033 running (9m) 3s ago 9m 6651k - 17.2.7-939-gc190bdee 284803f7d9cf 34c62d004c7e 2024-05-12T04:02:47.164 INFO:teuthology.orchestra.run.smithi032.stdout:grafana.smithi032 smithi032 *:3000 running (78s) 28s ago 10m 79.5M - 9.4.7 954c08fa6188 83055e58d23c 2024-05-12T04:02:47.164 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.ialtqb smithi032 running (4m) 28s ago 4m 12.4M - 17.2.7-939-gc190bdee 284803f7d9cf 0683abf00e3e 2024-05-12T04:02:47.164 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.qpsawh smithi032 running (4m) 28s ago 4m 252M - 17.2.7-939-gc190bdee 284803f7d9cf 95a7db5c4fcd 2024-05-12T04:02:47.164 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi033.hglwml smithi033 running (4m) 3s ago 4m 12.6M - 17.2.7-939-gc190bdee 284803f7d9cf c97902b05025 2024-05-12T04:02:47.164 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi033.jgbjhk smithi033 running (4m) 3s ago 4m 233M - 17.2.7-939-gc190bdee 284803f7d9cf af97052cb97a 2024-05-12T04:02:47.165 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi032.ukatkq smithi032 *:8443,9283,8765 running (49s) 28s ago 12m 473M - 18.2.2-1272-g58249552 ddcb0a9d6fde 0d88ab9a5292 2024-05-12T04:02:47.165 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi033.pcvxwy smithi033 *:8443,9283,8765 running (8s) 3s ago 9m 52.9M - 18.2.2-1272-g58249552 ddcb0a9d6fde d42dcb6bd307 2024-05-12T04:02:47.165 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi032 smithi032 running (12m) 28s ago 12m 52.1M 2048M 17.2.7-939-gc190bdee 284803f7d9cf d7fea430c6e2 2024-05-12T04:02:47.165 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi033 smithi033 running (9m) 3s ago 9m 42.5M 2048M 17.2.7-939-gc190bdee 284803f7d9cf 707654d85748 2024-05-12T04:02:47.165 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi032 smithi032 *:9100 running (2m) 28s ago 10m 9076k - 1.5.0 0da6a335fe13 cb75a7c0c829 2024-05-12T04:02:47.165 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi033 smithi033 *:9100 running (55s) 3s ago 9m 16.6M - 1.5.0 0da6a335fe13 7888c36c659f 2024-05-12T04:02:47.165 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0 smithi032 running (8m) 28s ago 8m 173M 1570M 17.2.7-939-gc190bdee 284803f7d9cf 75c78a47f8e5 2024-05-12T04:02:47.165 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1 smithi032 running (8m) 28s ago 8m 186M 1570M 17.2.7-939-gc190bdee 284803f7d9cf 87cfd247d8d6 2024-05-12T04:02:47.165 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2 smithi032 running (7m) 28s ago 7m 149M 1570M 17.2.7-939-gc190bdee 284803f7d9cf 7aef7affb8ba 2024-05-12T04:02:47.165 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3 smithi033 running (7m) 3s ago 7m 213M 2594M 17.2.7-939-gc190bdee 284803f7d9cf 3a92b5e3348e 2024-05-12T04:02:47.165 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4 smithi033 running (7m) 3s ago 6m 187M 2594M 17.2.7-939-gc190bdee 284803f7d9cf 0896268b4591 2024-05-12T04:02:47.165 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5 smithi033 running (6m) 3s ago 6m 175M 2594M 17.2.7-939-gc190bdee 284803f7d9cf 57e65a64c821 2024-05-12T04:02:47.165 INFO:teuthology.orchestra.run.smithi032.stdout:prometheus.smithi032 smithi032 *:9095 starting - - - - 2024-05-12T04:02:47.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.166+0000 7f71075c7700 1 -- 172.21.15.32:0/2076626305 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f70ec06c9c0 msgr2=0x7f70ec06ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:02:47.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.166+0000 7f71075c7700 1 --2- 172.21.15.32:0/2076626305 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f70ec06c9c0 0x7f70ec06ee70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f70fc0137b0 tx=0x7f70fc009e30 comp rx=0 tx=0).stop 2024-05-12T04:02:47.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.166+0000 7f71075c7700 1 -- 172.21.15.32:0/2076626305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71000fdfd0 msgr2=0x7f710010ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:02:47.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.166+0000 7f71075c7700 1 --2- 172.21.15.32:0/2076626305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71000fdfd0 0x7f710010ddb0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f70f000eb40 tx=0x7f70f000b040 comp rx=0 tx=0).stop 2024-05-12T04:02:47.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.166+0000 7f71075c7700 1 -- 172.21.15.32:0/2076626305 shutdown_connections 2024-05-12T04:02:47.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.166+0000 7f71075c7700 1 --2- 172.21.15.32:0/2076626305 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f70ec06c9c0 0x7f70ec06ee70 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:47.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.166+0000 7f71075c7700 1 --2- 172.21.15.32:0/2076626305 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71000fce60 0x7f710010d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:47.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.166+0000 7f71075c7700 1 --2- 172.21.15.32:0/2076626305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71000fdfd0 0x7f710010ddb0 secure :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f70f000eb40 tx=0x7f70f000b040 comp rx=0 tx=0).stop 2024-05-12T04:02:47.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.166+0000 7f71075c7700 1 -- 172.21.15.32:0/2076626305 >> 172.21.15.32:0/2076626305 conn(0x7f71000f8540 msgr2=0x7f71000f9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:02:47.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.166+0000 7f71075c7700 1 -- 172.21.15.32:0/2076626305 shutdown_connections 2024-05-12T04:02:47.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.167+0000 7f71075c7700 1 -- 172.21.15.32:0/2076626305 wait complete. 2024-05-12T04:02:47.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.305+0000 7f6630702700 1 -- 172.21.15.32:0/1514851060 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f662806e780 msgr2=0x7f662806eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:02:47.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.305+0000 7f6630702700 1 --2- 172.21.15.32:0/1514851060 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f662806e780 0x7f662806eb90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6624004a50 tx=0x7f662400af40 comp rx=0 tx=0).stop 2024-05-12T04:02:47.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.306+0000 7f6630702700 1 -- 172.21.15.32:0/1514851060 shutdown_connections 2024-05-12T04:02:47.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.306+0000 7f6630702700 1 --2- 172.21.15.32:0/1514851060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f662806f0d0 0x7f6628108f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:47.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.306+0000 7f6630702700 1 --2- 172.21.15.32:0/1514851060 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f662806e780 0x7f662806eb90 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:47.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.306+0000 7f6630702700 1 -- 172.21.15.32:0/1514851060 >> 172.21.15.32:0/1514851060 conn(0x7f662806a360 msgr2=0x7f662806c790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:02:47.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.307+0000 7f6630702700 1 -- 172.21.15.32:0/1514851060 shutdown_connections 2024-05-12T04:02:47.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.307+0000 7f6630702700 1 -- 172.21.15.32:0/1514851060 wait complete. 2024-05-12T04:02:47.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.308+0000 7f6630702700 1 Processor -- start 2024-05-12T04:02:47.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.308+0000 7f6630702700 1 -- start start 2024-05-12T04:02:47.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.308+0000 7f6630702700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f662806e780 0x7f662810d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:02:47.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.308+0000 7f6630702700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f662806f0d0 0x7f662810d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:02:47.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.308+0000 7f6630702700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662810df80 con 0x7f662806f0d0 2024-05-12T04:02:47.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.308+0000 7f6630702700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662810e0f0 con 0x7f662806e780 2024-05-12T04:02:47.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.309+0000 7f662dc9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f662806f0d0 0x7f662810d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:02:47.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.309+0000 7f662dc9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f662806f0d0 0x7f662810d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36938/0 (socket says 172.21.15.32:36938) 2024-05-12T04:02:47.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.309+0000 7f662dc9d700 1 -- 172.21.15.32:0/1530585460 learned_addr learned my addr 172.21.15.32:0/1530585460 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:02:47.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.310+0000 7f662e49e700 1 --2- 172.21.15.32:0/1530585460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f662806e780 0x7f662810d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:02:47.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.310+0000 7f662dc9d700 1 -- 172.21.15.32:0/1530585460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f662806e780 msgr2=0x7f662810d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:02:47.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.310+0000 7f662dc9d700 1 --2- 172.21.15.32:0/1530585460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f662806e780 0x7f662810d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:47.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.310+0000 7f662dc9d700 1 -- 172.21.15.32:0/1530585460 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6624005040 con 0x7f662806f0d0 2024-05-12T04:02:47.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.310+0000 7f662dc9d700 1 --2- 172.21.15.32:0/1530585460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f662806f0d0 0x7f662810d980 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6618002a50 tx=0x7f66180046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:02:47.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.311+0000 7f661effd700 1 -- 172.21.15.32:0/1530585460 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6618007cc0 con 0x7f662806f0d0 2024-05-12T04:02:47.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.311+0000 7f661effd700 1 -- 172.21.15.32:0/1530585460 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f661800f040 con 0x7f662806f0d0 2024-05-12T04:02:47.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.311+0000 7f6630702700 1 -- 172.21.15.32:0/1530585460 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6628112b50 con 0x7f662806f0d0 2024-05-12T04:02:47.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.311+0000 7f661effd700 1 -- 172.21.15.32:0/1530585460 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f661800d9f0 con 0x7f662806f0d0 2024-05-12T04:02:47.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.311+0000 7f6630702700 1 -- 172.21.15.32:0/1530585460 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6628113110 con 0x7f662806f0d0 2024-05-12T04:02:47.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.312+0000 7f6630702700 1 -- 172.21.15.32:0/1530585460 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f662804c710 con 0x7f662806f0d0 2024-05-12T04:02:47.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.315+0000 7f661effd700 1 -- 172.21.15.32:0/1530585460 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 32) v1 ==== 90813+0+0 (secure 0 0 0) 0x7f661800db50 con 0x7f662806f0d0 2024-05-12T04:02:47.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.315+0000 7f661effd700 1 --2- 172.21.15.32:0/1530585460 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f661406cad0 0x7f661406ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:02:47.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.315+0000 7f661effd700 1 -- 172.21.15.32:0/1530585460 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f661808b8f0 con 0x7f662806f0d0 2024-05-12T04:02:47.316 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.316+0000 7f662e49e700 1 --2- 172.21.15.32:0/1530585460 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f661406cad0 0x7f661406ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:02:47.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.318+0000 7f662e49e700 1 --2- 172.21.15.32:0/1530585460 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f661406cad0 0x7f661406ef80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f662400afd0 tx=0x7f662400af40 comp rx=0 tx=0).ready entity=mgr.24501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:02:47.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.321+0000 7f661effd700 1 -- 172.21.15.32:0/1530585460 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181777 (secure 0 0 0) 0x7f66180596a0 con 0x7f662806f0d0 2024-05-12T04:02:47.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.748+0000 7f6630702700 1 -- 172.21.15.32:0/1530585460 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f662810bd50 con 0x7f662806f0d0 2024-05-12T04:02:47.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.748+0000 7f661effd700 1 -- 172.21.15.32:0/1530585460 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+743 (secure 0 0 0) 0x7f6618059230 con 0x7f662806f0d0 2024-05-12T04:02:47.749 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-05-12T04:02:47.749 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": { 2024-05-12T04:02:47.749 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 2 2024-05-12T04:02:47.749 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:02:47.749 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": { 2024-05-12T04:02:47.749 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 2 2024-05-12T04:02:47.749 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:02:47.749 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": { 2024-05-12T04:02:47.749 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 6 2024-05-12T04:02:47.749 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:02:47.749 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": { 2024-05-12T04:02:47.750 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 4 2024-05-12T04:02:47.750 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:02:47.750 INFO:teuthology.orchestra.run.smithi032.stdout: "overall": { 2024-05-12T04:02:47.750 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 12, 2024-05-12T04:02:47.750 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 2 2024-05-12T04:02:47.750 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-05-12T04:02:47.750 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-05-12T04:02:47.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.752+0000 7f6630702700 1 -- 172.21.15.32:0/1530585460 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f661406cad0 msgr2=0x7f661406ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:02:47.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.752+0000 7f6630702700 1 --2- 172.21.15.32:0/1530585460 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f661406cad0 0x7f661406ef80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f662400afd0 tx=0x7f662400af40 comp rx=0 tx=0).stop 2024-05-12T04:02:47.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.752+0000 7f6630702700 1 -- 172.21.15.32:0/1530585460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f662806f0d0 msgr2=0x7f662810d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:02:47.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.752+0000 7f6630702700 1 --2- 172.21.15.32:0/1530585460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f662806f0d0 0x7f662810d980 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6618002a50 tx=0x7f66180046b0 comp rx=0 tx=0).stop 2024-05-12T04:02:47.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.752+0000 7f6630702700 1 -- 172.21.15.32:0/1530585460 shutdown_connections 2024-05-12T04:02:47.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.752+0000 7f6630702700 1 --2- 172.21.15.32:0/1530585460 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f661406cad0 0x7f661406ef80 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:47.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.752+0000 7f6630702700 1 --2- 172.21.15.32:0/1530585460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f662806e780 0x7f662810d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:47.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.752+0000 7f6630702700 1 --2- 172.21.15.32:0/1530585460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f662806f0d0 0x7f662810d980 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:47.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.752+0000 7f6630702700 1 -- 172.21.15.32:0/1530585460 >> 172.21.15.32:0/1530585460 conn(0x7f662806a360 msgr2=0x7f66281070c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:02:47.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.752+0000 7f6630702700 1 -- 172.21.15.32:0/1530585460 shutdown_connections 2024-05-12T04:02:47.753 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.752+0000 7f6630702700 1 -- 172.21.15.32:0/1530585460 wait complete. 2024-05-12T04:02:47.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.871+0000 7fa666dcc700 1 -- 172.21.15.32:0/1582726656 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6600fbcb0 msgr2=0x7fa6600fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:02:47.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.871+0000 7fa666dcc700 1 --2- 172.21.15.32:0/1582726656 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6600fbcb0 0x7fa6600fc0c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa654004940 tx=0x7fa654009e30 comp rx=0 tx=0).stop 2024-05-12T04:02:47.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.872+0000 7fa666dcc700 1 -- 172.21.15.32:0/1582726656 shutdown_connections 2024-05-12T04:02:47.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.872+0000 7fa666dcc700 1 --2- 172.21.15.32:0/1582726656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6600fcf60 0x7fa6600fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:47.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.872+0000 7fa666dcc700 1 --2- 172.21.15.32:0/1582726656 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6600fbcb0 0x7fa6600fc0c0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:47.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.872+0000 7fa666dcc700 1 -- 172.21.15.32:0/1582726656 >> 172.21.15.32:0/1582726656 conn(0x7fa6600f7430 msgr2=0x7fa6600f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:02:47.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.872+0000 7fa666dcc700 1 -- 172.21.15.32:0/1582726656 shutdown_connections 2024-05-12T04:02:47.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.872+0000 7fa666dcc700 1 -- 172.21.15.32:0/1582726656 wait complete. 2024-05-12T04:02:47.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.872+0000 7fa666dcc700 1 Processor -- start 2024-05-12T04:02:47.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.872+0000 7fa666dcc700 1 -- start start 2024-05-12T04:02:47.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.873+0000 7fa666dcc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6600fbcb0 0x7fa66010d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:02:47.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.873+0000 7fa666dcc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6600fcf60 0x7fa66010de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:02:47.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.873+0000 7fa666dcc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa66010e470 con 0x7fa6600fbcb0 2024-05-12T04:02:47.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.873+0000 7fa666dcc700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa66010ec20 con 0x7fa6600fcf60 2024-05-12T04:02:47.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.873+0000 7fa65ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6600fcf60 0x7fa66010de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:02:47.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.873+0000 7fa65ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6600fcf60 0x7fa66010de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.32:58664/0 (socket says 172.21.15.32:58664) 2024-05-12T04:02:47.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.873+0000 7fa65ffff700 1 -- 172.21.15.32:0/3796854843 learned_addr learned my addr 172.21.15.32:0/3796854843 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:02:47.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.873+0000 7fa664b68700 1 --2- 172.21.15.32:0/3796854843 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6600fbcb0 0x7fa66010d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:02:47.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.873+0000 7fa65ffff700 1 -- 172.21.15.32:0/3796854843 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6600fbcb0 msgr2=0x7fa66010d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:02:47.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.873+0000 7fa65ffff700 1 --2- 172.21.15.32:0/3796854843 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6600fbcb0 0x7fa66010d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:47.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.873+0000 7fa65ffff700 1 -- 172.21.15.32:0/3796854843 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa654005040 con 0x7fa6600fcf60 2024-05-12T04:02:47.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.873+0000 7fa65ffff700 1 --2- 172.21.15.32:0/3796854843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6600fcf60 0x7fa66010de70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa64c00eb40 tx=0x7fa64c00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:02:47.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.873+0000 7fa65d7fa700 1 -- 172.21.15.32:0/3796854843 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa64c007cc0 con 0x7fa6600fcf60 2024-05-12T04:02:47.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.874+0000 7fa65d7fa700 1 -- 172.21.15.32:0/3796854843 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa64c00f040 con 0x7fa6600fcf60 2024-05-12T04:02:47.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.874+0000 7fa666dcc700 1 -- 172.21.15.32:0/3796854843 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa66010eea0 con 0x7fa6600fcf60 2024-05-12T04:02:47.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.874+0000 7fa65d7fa700 1 -- 172.21.15.32:0/3796854843 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa64c00c990 con 0x7fa6600fcf60 2024-05-12T04:02:47.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.874+0000 7fa666dcc700 1 -- 172.21.15.32:0/3796854843 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa66010f3e0 con 0x7fa6600fcf60 2024-05-12T04:02:47.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.875+0000 7fa666dcc700 1 -- 172.21.15.32:0/3796854843 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa660101af0 con 0x7fa6600fcf60 2024-05-12T04:02:47.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.875+0000 7fa65d7fa700 1 -- 172.21.15.32:0/3796854843 <== mon.1 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 32) v1 ==== 90813+0+0 (secure 0 0 0) 0x7fa64c00cc00 con 0x7fa6600fcf60 2024-05-12T04:02:47.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.876+0000 7fa65d7fa700 1 --2- 172.21.15.32:0/3796854843 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7fa65006c9c0 0x7fa65006ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:02:47.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.876+0000 7fa65d7fa700 1 -- 172.21.15.32:0/3796854843 <== mon.1 v2:172.21.15.33:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fa64c08b9d0 con 0x7fa6600fcf60 2024-05-12T04:02:47.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.876+0000 7fa664b68700 1 --2- 172.21.15.32:0/3796854843 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7fa65006c9c0 0x7fa65006ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:02:47.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.876+0000 7fa664b68700 1 --2- 172.21.15.32:0/3796854843 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7fa65006c9c0 0x7fa65006ee70 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa654004890 tx=0x7fa654009e30 comp rx=0 tx=0).ready entity=mgr.24501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:02:47.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:47.880+0000 7fa65d7fa700 1 -- 172.21.15.32:0/3796854843 <== mon.1 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181777 (secure 0 0 0) 0x7fa64c059730 con 0x7fa6600fcf60 2024-05-12T04:02:48.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.056+0000 7fa65d7fa700 1 -- 172.21.15.32:0/3796854843 <== mon.1 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 33) v1 ==== 90813+0+0 (secure 0 0 0) 0x7fa64c056070 con 0x7fa6600fcf60 2024-05-12T04:02:48.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:48 smithi033 ceph-mon[49525]: pgmap v21: 65 pgs: 65 active+clean; 310 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 93 op/s 2024-05-12T04:02:48.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:48 smithi033 ceph-mon[49525]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:02:48.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:48 smithi033 ceph-mon[49525]: from='client.14818 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:02:48.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:48 smithi033 ceph-mon[49525]: Standby manager daemon smithi033.pcvxwy restarted 2024-05-12T04:02:48.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:48 smithi033 ceph-mon[49525]: Standby manager daemon smithi033.pcvxwy started 2024-05-12T04:02:48.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:48 smithi033 ceph-mon[49525]: from='mgr.? 172.21.15.33:0/4247242247' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi033.pcvxwy/crt"}]: dispatch 2024-05-12T04:02:48.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:48 smithi033 ceph-mon[49525]: from='mgr.? 172.21.15.33:0/4247242247' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-12T04:02:48.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:48 smithi033 ceph-mon[49525]: from='mgr.? 172.21.15.33:0/4247242247' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi033.pcvxwy/key"}]: dispatch 2024-05-12T04:02:48.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:48 smithi033 ceph-mon[49525]: from='mgr.? 172.21.15.33:0/4247242247' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-12T04:02:48.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:48 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/1530585460' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:02:48.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.179+0000 7fa666dcc700 1 -- 172.21.15.32:0/3796854843 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa66004bed0 con 0x7fa6600fcf60 2024-05-12T04:02:48.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.179+0000 7fa65d7fa700 1 -- 172.21.15.32:0/3796854843 <== mon.1 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1828 (secure 0 0 0) 0x7fa64c0771b0 con 0x7fa6600fcf60 2024-05-12T04:02:48.179 INFO:teuthology.orchestra.run.smithi032.stdout:e18 2024-05-12T04:02:48.179 INFO:teuthology.orchestra.run.smithi032.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-12T04:02:48.179 INFO:teuthology.orchestra.run.smithi032.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-12T04:02:48.180 INFO:teuthology.orchestra.run.smithi032.stdout:legacy client fscid: 1 2024-05-12T04:02:48.180 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:02:48.180 INFO:teuthology.orchestra.run.smithi032.stdout:Filesystem 'cephfs' (1) 2024-05-12T04:02:48.180 INFO:teuthology.orchestra.run.smithi032.stdout:fs_name cephfs 2024-05-12T04:02:48.180 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 18 2024-05-12T04:02:48.180 INFO:teuthology.orchestra.run.smithi032.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-12T04:02:48.180 INFO:teuthology.orchestra.run.smithi032.stdout:created 2024-05-12T03:57:48.904614+0000 2024-05-12T04:02:48.180 INFO:teuthology.orchestra.run.smithi032.stdout:modified 2024-05-12T03:58:06.821591+0000 2024-05-12T04:02:48.180 INFO:teuthology.orchestra.run.smithi032.stdout:tableserver 0 2024-05-12T04:02:48.180 INFO:teuthology.orchestra.run.smithi032.stdout:root 0 2024-05-12T04:02:48.180 INFO:teuthology.orchestra.run.smithi032.stdout:session_timeout 60 2024-05-12T04:02:48.180 INFO:teuthology.orchestra.run.smithi032.stdout:session_autoclose 300 2024-05-12T04:02:48.180 INFO:teuthology.orchestra.run.smithi032.stdout:max_file_size 1099511627776 2024-05-12T04:02:48.180 INFO:teuthology.orchestra.run.smithi032.stdout:required_client_features {} 2024-05-12T04:02:48.181 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure 0 2024-05-12T04:02:48.181 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure_osd_epoch 46 2024-05-12T04:02:48.181 INFO:teuthology.orchestra.run.smithi032.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-12T04:02:48.181 INFO:teuthology.orchestra.run.smithi032.stdout:max_mds 1 2024-05-12T04:02:48.181 INFO:teuthology.orchestra.run.smithi032.stdout:in 0 2024-05-12T04:02:48.181 INFO:teuthology.orchestra.run.smithi032.stdout:up {0=14474} 2024-05-12T04:02:48.181 INFO:teuthology.orchestra.run.smithi032.stdout:failed 2024-05-12T04:02:48.181 INFO:teuthology.orchestra.run.smithi032.stdout:damaged 2024-05-12T04:02:48.181 INFO:teuthology.orchestra.run.smithi032.stdout:stopped 2024-05-12T04:02:48.181 INFO:teuthology.orchestra.run.smithi032.stdout:data_pools [3] 2024-05-12T04:02:48.181 INFO:teuthology.orchestra.run.smithi032.stdout:metadata_pool 2 2024-05-12T04:02:48.181 INFO:teuthology.orchestra.run.smithi032.stdout:inline_data enabled 2024-05-12T04:02:48.181 INFO:teuthology.orchestra.run.smithi032.stdout:balancer 2024-05-12T04:02:48.181 INFO:teuthology.orchestra.run.smithi032.stdout:standby_count_wanted 1 2024-05-12T04:02:48.181 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.qpsawh{0:14474} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.32:6828/3100155296,v1:172.21.15.32:6829/3100155296] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:02:48.182 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi033.jgbjhk{0:24267} state up:standby-replay seq 3 join_fscid=1 addr [v2:172.21.15.33:6824/55417471,v1:172.21.15.33:6825/55417471] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:02:48.182 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:02:48.182 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:02:48.182 INFO:teuthology.orchestra.run.smithi032.stdout:Standby daemons: 2024-05-12T04:02:48.182 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:02:48.182 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi033.hglwml{-1:24283} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/433858238,v1:172.21.15.33:6827/433858238] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:02:48.182 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.ialtqb{-1:24287} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.32:6826/3703132329,v1:172.21.15.32:6827/3703132329] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:02:48.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.181+0000 7fa666dcc700 1 -- 172.21.15.32:0/3796854843 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7fa65006c9c0 msgr2=0x7fa65006ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:02:48.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.181+0000 7fa666dcc700 1 --2- 172.21.15.32:0/3796854843 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7fa65006c9c0 0x7fa65006ee70 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa654004890 tx=0x7fa654009e30 comp rx=0 tx=0).stop 2024-05-12T04:02:48.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.181+0000 7fa666dcc700 1 -- 172.21.15.32:0/3796854843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6600fcf60 msgr2=0x7fa66010de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:02:48.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.181+0000 7fa666dcc700 1 --2- 172.21.15.32:0/3796854843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6600fcf60 0x7fa66010de70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa64c00eb40 tx=0x7fa64c00b040 comp rx=0 tx=0).stop 2024-05-12T04:02:48.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.181+0000 7fa666dcc700 1 -- 172.21.15.32:0/3796854843 shutdown_connections 2024-05-12T04:02:48.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.181+0000 7fa666dcc700 1 --2- 172.21.15.32:0/3796854843 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7fa65006c9c0 0x7fa65006ee70 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:48.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.181+0000 7fa666dcc700 1 --2- 172.21.15.32:0/3796854843 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6600fbcb0 0x7fa66010d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:48.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.181+0000 7fa666dcc700 1 --2- 172.21.15.32:0/3796854843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6600fcf60 0x7fa66010de70 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:48.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.181+0000 7fa666dcc700 1 -- 172.21.15.32:0/3796854843 >> 172.21.15.32:0/3796854843 conn(0x7fa6600f7430 msgr2=0x7fa660065b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:02:48.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.182+0000 7fa666dcc700 1 -- 172.21.15.32:0/3796854843 shutdown_connections 2024-05-12T04:02:48.183 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.182+0000 7fa666dcc700 1 -- 172.21.15.32:0/3796854843 wait complete. 2024-05-12T04:02:48.185 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 18 2024-05-12T04:02:48.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.337+0000 7f58e9582700 1 -- 172.21.15.32:0/646938886 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58e40fe4a0 msgr2=0x7f58e40fe8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:02:48.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.337+0000 7f58e9582700 1 --2- 172.21.15.32:0/646938886 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58e40fe4a0 0x7f58e40fe8b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f58d4004940 tx=0x7f58d4009e30 comp rx=0 tx=0).stop 2024-05-12T04:02:48.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.337+0000 7f58e9582700 1 -- 172.21.15.32:0/646938886 shutdown_connections 2024-05-12T04:02:48.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.337+0000 7f58e9582700 1 --2- 172.21.15.32:0/646938886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58e40fedf0 0x7f58e40faf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:48.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.337+0000 7f58e9582700 1 --2- 172.21.15.32:0/646938886 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58e40fe4a0 0x7f58e40fe8b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:48.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.337+0000 7f58e9582700 1 -- 172.21.15.32:0/646938886 >> 172.21.15.32:0/646938886 conn(0x7f58e40f6b80 msgr2=0x7f58e40f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:02:48.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.337+0000 7f58e9582700 1 -- 172.21.15.32:0/646938886 shutdown_connections 2024-05-12T04:02:48.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.337+0000 7f58e9582700 1 -- 172.21.15.32:0/646938886 wait complete. 2024-05-12T04:02:48.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.338+0000 7f58e9582700 1 Processor -- start 2024-05-12T04:02:48.338 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.338+0000 7f58e9582700 1 -- start start 2024-05-12T04:02:48.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.338+0000 7f58e9582700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58e40fe4a0 0x7f58e40fdc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:02:48.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.338+0000 7f58e9582700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58e40fedf0 0x7f58e40fc160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:02:48.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.338+0000 7f58e9582700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e40fc710 con 0x7f58e40fedf0 2024-05-12T04:02:48.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.338+0000 7f58e9582700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e40fc880 con 0x7f58e40fe4a0 2024-05-12T04:02:48.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.338+0000 7f58e2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58e40fe4a0 0x7f58e40fdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:02:48.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.338+0000 7f58e27fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58e40fedf0 0x7f58e40fc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:02:48.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.338+0000 7f58e2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58e40fe4a0 0x7f58e40fdc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.32:58682/0 (socket says 172.21.15.32:58682) 2024-05-12T04:02:48.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.338+0000 7f58e2ffd700 1 -- 172.21.15.32:0/1231918354 learned_addr learned my addr 172.21.15.32:0/1231918354 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:02:48.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.339+0000 7f58e2ffd700 1 -- 172.21.15.32:0/1231918354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58e40fedf0 msgr2=0x7f58e40fc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:02:48.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.339+0000 7f58e2ffd700 1 --2- 172.21.15.32:0/1231918354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58e40fedf0 0x7f58e40fc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:48.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.339+0000 7f58e2ffd700 1 -- 172.21.15.32:0/1231918354 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58d4005040 con 0x7f58e40fe4a0 2024-05-12T04:02:48.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.339+0000 7f58e2ffd700 1 --2- 172.21.15.32:0/1231918354 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58e40fe4a0 0x7f58e40fdc90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f58d4007d40 tx=0x7f58d4009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:02:48.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.339+0000 7f58cb7fe700 1 -- 172.21.15.32:0/1231918354 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58d400e030 con 0x7f58e40fe4a0 2024-05-12T04:02:48.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.339+0000 7f58cb7fe700 1 -- 172.21.15.32:0/1231918354 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f58d401c400 con 0x7f58e40fe4a0 2024-05-12T04:02:48.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.339+0000 7f58e9582700 1 -- 172.21.15.32:0/1231918354 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58e40fcb00 con 0x7f58e40fe4a0 2024-05-12T04:02:48.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.339+0000 7f58e9582700 1 -- 172.21.15.32:0/1231918354 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58e41158d0 con 0x7f58e40fe4a0 2024-05-12T04:02:48.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.339+0000 7f58cb7fe700 1 -- 172.21.15.32:0/1231918354 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58d400b650 con 0x7f58e40fe4a0 2024-05-12T04:02:48.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.341+0000 7f58e9582700 1 -- 172.21.15.32:0/1231918354 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58e405f5c0 con 0x7f58e40fe4a0 2024-05-12T04:02:48.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.341+0000 7f58cb7fe700 1 -- 172.21.15.32:0/1231918354 <== mon.1 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 33) v1 ==== 90813+0+0 (secure 0 0 0) 0x7f58d400b7b0 con 0x7f58e40fe4a0 2024-05-12T04:02:48.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.341+0000 7f58cb7fe700 1 --2- 172.21.15.32:0/1231918354 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f58cc06c9c0 0x7f58cc06ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:02:48.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.342+0000 7f58cb7fe700 1 -- 172.21.15.32:0/1231918354 <== mon.1 v2:172.21.15.33:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f58d408d0e0 con 0x7f58e40fe4a0 2024-05-12T04:02:48.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.342+0000 7f58e27fc700 1 --2- 172.21.15.32:0/1231918354 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f58cc06c9c0 0x7f58cc06ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:02:48.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.342+0000 7f58e27fc700 1 --2- 172.21.15.32:0/1231918354 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f58cc06c9c0 0x7f58cc06ee70 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f58d8004fd0 tx=0x7f58d800b040 comp rx=0 tx=0).ready entity=mgr.24501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:02:48.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.347+0000 7f58cb7fe700 1 -- 172.21.15.32:0/1231918354 <== mon.1 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181777 (secure 0 0 0) 0x7f58d4057530 con 0x7f58e40fe4a0 2024-05-12T04:02:48.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:48 smithi032 ceph-mon[40008]: pgmap v21: 65 pgs: 65 active+clean; 310 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 93 op/s 2024-05-12T04:02:48.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:48 smithi032 ceph-mon[40008]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:02:48.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:48 smithi032 ceph-mon[40008]: from='client.14818 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:02:48.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:48 smithi032 ceph-mon[40008]: Standby manager daemon smithi033.pcvxwy restarted 2024-05-12T04:02:48.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:48 smithi032 ceph-mon[40008]: Standby manager daemon smithi033.pcvxwy started 2024-05-12T04:02:48.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:48 smithi032 ceph-mon[40008]: from='mgr.? 172.21.15.33:0/4247242247' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi033.pcvxwy/crt"}]: dispatch 2024-05-12T04:02:48.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:48 smithi032 ceph-mon[40008]: from='mgr.? 172.21.15.33:0/4247242247' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-12T04:02:48.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:48 smithi032 ceph-mon[40008]: from='mgr.? 172.21.15.33:0/4247242247' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi033.pcvxwy/key"}]: dispatch 2024-05-12T04:02:48.542 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:48 smithi032 ceph-mon[40008]: from='mgr.? 172.21.15.33:0/4247242247' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-12T04:02:48.542 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:48 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/1530585460' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:02:48.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.616+0000 7f58e9582700 1 -- 172.21.15.32:0/1231918354 --> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f58e41025b0 con 0x7f58cc06c9c0 2024-05-12T04:02:48.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.617+0000 7f58cb7fe700 1 -- 172.21.15.32:0/1231918354 <== mgr.24501 v2:172.21.15.32:6800/3729453095 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f58e41025b0 con 0x7f58cc06c9c0 2024-05-12T04:02:48.618 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-05-12T04:02:48.618 INFO:teuthology.orchestra.run.smithi032.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:19c9734256aa43a60e203feb242f50bf8b7f58666d520536af43cc4fee96173a", 2024-05-12T04:02:48.618 INFO:teuthology.orchestra.run.smithi032.stdout: "in_progress": true, 2024-05-12T04:02:48.618 INFO:teuthology.orchestra.run.smithi032.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-12T04:02:48.618 INFO:teuthology.orchestra.run.smithi032.stdout: "services_complete": [ 2024-05-12T04:02:48.618 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr" 2024-05-12T04:02:48.618 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-05-12T04:02:48.618 INFO:teuthology.orchestra.run.smithi032.stdout: "progress": "2/21 daemons upgraded", 2024-05-12T04:02:48.618 INFO:teuthology.orchestra.run.smithi032.stdout: "message": "Currently upgrading mgr daemons", 2024-05-12T04:02:48.618 INFO:teuthology.orchestra.run.smithi032.stdout: "is_paused": false 2024-05-12T04:02:48.618 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-05-12T04:02:48.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.620+0000 7f58e9582700 1 -- 172.21.15.32:0/1231918354 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f58cc06c9c0 msgr2=0x7f58cc06ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:02:48.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.620+0000 7f58e9582700 1 --2- 172.21.15.32:0/1231918354 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f58cc06c9c0 0x7f58cc06ee70 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f58d8004fd0 tx=0x7f58d800b040 comp rx=0 tx=0).stop 2024-05-12T04:02:48.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.620+0000 7f58e9582700 1 -- 172.21.15.32:0/1231918354 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58e40fe4a0 msgr2=0x7f58e40fdc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:02:48.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.620+0000 7f58e9582700 1 --2- 172.21.15.32:0/1231918354 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58e40fe4a0 0x7f58e40fdc90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f58d4007d40 tx=0x7f58d4009e30 comp rx=0 tx=0).stop 2024-05-12T04:02:48.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.620+0000 7f58e9582700 1 -- 172.21.15.32:0/1231918354 shutdown_connections 2024-05-12T04:02:48.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.620+0000 7f58e9582700 1 --2- 172.21.15.32:0/1231918354 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f58cc06c9c0 0x7f58cc06ee70 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:48.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.620+0000 7f58e9582700 1 --2- 172.21.15.32:0/1231918354 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58e40fe4a0 0x7f58e40fdc90 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:48.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.620+0000 7f58e9582700 1 --2- 172.21.15.32:0/1231918354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58e40fedf0 0x7f58e40fc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:48.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.620+0000 7f58e9582700 1 -- 172.21.15.32:0/1231918354 >> 172.21.15.32:0/1231918354 conn(0x7f58e40f6b80 msgr2=0x7f58e40f82d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:02:48.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.620+0000 7f58e9582700 1 -- 172.21.15.32:0/1231918354 shutdown_connections 2024-05-12T04:02:48.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.620+0000 7f58e9582700 1 -- 172.21.15.32:0/1231918354 wait complete. 2024-05-12T04:02:48.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.746+0000 7f68370fd700 1 -- 172.21.15.32:0/193610004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68300fcf60 msgr2=0x7f68300fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:02:48.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.746+0000 7f68370fd700 1 --2- 172.21.15.32:0/193610004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68300fcf60 0x7f68300fd3d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f681c004940 tx=0x7f681c009e30 comp rx=0 tx=0).stop 2024-05-12T04:02:48.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.747+0000 7f68370fd700 1 -- 172.21.15.32:0/193610004 shutdown_connections 2024-05-12T04:02:48.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.747+0000 7f68370fd700 1 --2- 172.21.15.32:0/193610004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68300fcf60 0x7f68300fd3d0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:48.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.747+0000 7f68370fd700 1 --2- 172.21.15.32:0/193610004 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68300fbcb0 0x7f68300fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:48.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.747+0000 7f68370fd700 1 -- 172.21.15.32:0/193610004 >> 172.21.15.32:0/193610004 conn(0x7f68300f7430 msgr2=0x7f68300f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:02:48.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.747+0000 7f68370fd700 1 -- 172.21.15.32:0/193610004 shutdown_connections 2024-05-12T04:02:48.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.747+0000 7f68370fd700 1 -- 172.21.15.32:0/193610004 wait complete. 2024-05-12T04:02:48.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.747+0000 7f68370fd700 1 Processor -- start 2024-05-12T04:02:48.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.747+0000 7f68370fd700 1 -- start start 2024-05-12T04:02:48.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.748+0000 7f68370fd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68300fbcb0 0x7f683010fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:02:48.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.748+0000 7f68370fd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68300fcf60 0x7f6830110040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:02:48.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.748+0000 7f68370fd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6830110640 con 0x7f68300fbcb0 2024-05-12T04:02:48.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.748+0000 7f68370fd700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6830110e40 con 0x7f68300fcf60 2024-05-12T04:02:48.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.748+0000 7f6834e99700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68300fbcb0 0x7f683010fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:02:48.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.748+0000 7f6834e99700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68300fbcb0 0x7f683010fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37006/0 (socket says 172.21.15.32:37006) 2024-05-12T04:02:48.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.748+0000 7f6834e99700 1 -- 172.21.15.32:0/1400489571 learned_addr learned my addr 172.21.15.32:0/1400489571 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:02:48.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.748+0000 7f682ffff700 1 --2- 172.21.15.32:0/1400489571 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68300fcf60 0x7f6830110040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:02:48.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.748+0000 7f682ffff700 1 -- 172.21.15.32:0/1400489571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68300fbcb0 msgr2=0x7f683010fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:02:48.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.748+0000 7f682ffff700 1 --2- 172.21.15.32:0/1400489571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68300fbcb0 0x7f683010fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:48.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.748+0000 7f682ffff700 1 -- 172.21.15.32:0/1400489571 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f681c005040 con 0x7f68300fcf60 2024-05-12T04:02:48.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.748+0000 7f682ffff700 1 --2- 172.21.15.32:0/1400489571 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68300fcf60 0x7f6830110040 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f681c000c00 tx=0x7f681c009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:02:48.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.748+0000 7f682d7fa700 1 -- 172.21.15.32:0/1400489571 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f681c00e070 con 0x7f68300fcf60 2024-05-12T04:02:48.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.749+0000 7f682d7fa700 1 -- 172.21.15.32:0/1400489571 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f681c0093b0 con 0x7f68300fcf60 2024-05-12T04:02:48.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.749+0000 7f682d7fa700 1 -- 172.21.15.32:0/1400489571 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f681c01cdc0 con 0x7f68300fcf60 2024-05-12T04:02:48.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.749+0000 7f68370fd700 1 -- 172.21.15.32:0/1400489571 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68301110c0 con 0x7f68300fcf60 2024-05-12T04:02:48.751 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.749+0000 7f68370fd700 1 -- 172.21.15.32:0/1400489571 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68301115a0 con 0x7f68300fcf60 2024-05-12T04:02:48.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.750+0000 7f68370fd700 1 -- 172.21.15.32:0/1400489571 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68301018f0 con 0x7f68300fcf60 2024-05-12T04:02:48.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.751+0000 7f682d7fa700 1 -- 172.21.15.32:0/1400489571 <== mon.1 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 33) v1 ==== 90813+0+0 (secure 0 0 0) 0x7f681c01c600 con 0x7f68300fcf60 2024-05-12T04:02:48.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.751+0000 7f682d7fa700 1 --2- 172.21.15.32:0/1400489571 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f682006c9c0 0x7f682006ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:02:48.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.751+0000 7f682d7fa700 1 -- 172.21.15.32:0/1400489571 <== mon.1 v2:172.21.15.33:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f681c08d270 con 0x7f68300fcf60 2024-05-12T04:02:48.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.751+0000 7f6834e99700 1 --2- 172.21.15.32:0/1400489571 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f682006c9c0 0x7f682006ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:02:48.752 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.752+0000 7f6834e99700 1 --2- 172.21.15.32:0/1400489571 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f682006c9c0 0x7f682006ee70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6824008fd0 tx=0x7f682400b040 comp rx=0 tx=0).ready entity=mgr.24501 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:02:48.756 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:48.756+0000 7f682d7fa700 1 -- 172.21.15.32:0/1400489571 <== mon.1 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181777 (secure 0 0 0) 0x7f681c05b040 con 0x7f68300fcf60 2024-05-12T04:02:49.109 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:49 smithi033 ceph-mon[49525]: from='client.14822 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:02:49.109 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:49 smithi033 ceph-mon[49525]: mgrmap e33: smithi032.ukatkq(active, since 37s), standbys: smithi033.pcvxwy 2024-05-12T04:02:49.109 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:49 smithi033 ceph-mon[49525]: pgmap v22: 65 pgs: 65 active+clean; 305 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 118 op/s 2024-05-12T04:02:49.109 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:49 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/3796854843' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-12T04:02:49.109 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:49 smithi033 ceph-mon[49525]: from='client.24581 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:02:49.109 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:49 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' 2024-05-12T04:02:49.110 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:49 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' 2024-05-12T04:02:49.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:49.265+0000 7f68370fd700 1 -- 172.21.15.32:0/1400489571 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f683005f5c0 con 0x7f68300fcf60 2024-05-12T04:02:49.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:49.267+0000 7f682d7fa700 1 -- 172.21.15.32:0/1400489571 <== mon.1 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f681c05abd0 con 0x7f68300fcf60 2024-05-12T04:02:49.269 INFO:teuthology.orchestra.run.smithi032.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-12T04:02:49.269 INFO:teuthology.orchestra.run.smithi032.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-12T04:02:49.269 INFO:teuthology.orchestra.run.smithi032.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-12T04:02:49.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:49.278+0000 7f68370fd700 1 -- 172.21.15.32:0/1400489571 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f682006c9c0 msgr2=0x7f682006ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:02:49.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:49.278+0000 7f68370fd700 1 --2- 172.21.15.32:0/1400489571 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f682006c9c0 0x7f682006ee70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6824008fd0 tx=0x7f682400b040 comp rx=0 tx=0).stop 2024-05-12T04:02:49.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:49.278+0000 7f68370fd700 1 -- 172.21.15.32:0/1400489571 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68300fcf60 msgr2=0x7f6830110040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:02:49.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:49.278+0000 7f68370fd700 1 --2- 172.21.15.32:0/1400489571 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68300fcf60 0x7f6830110040 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f681c000c00 tx=0x7f681c009e30 comp rx=0 tx=0).stop 2024-05-12T04:02:49.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:49.278+0000 7f68370fd700 1 -- 172.21.15.32:0/1400489571 shutdown_connections 2024-05-12T04:02:49.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:49.278+0000 7f68370fd700 1 --2- 172.21.15.32:0/1400489571 >> [v2:172.21.15.32:6800/3729453095,v1:172.21.15.32:6801/3729453095] conn(0x7f682006c9c0 0x7f682006ee70 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:49.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:49.278+0000 7f68370fd700 1 --2- 172.21.15.32:0/1400489571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68300fbcb0 0x7f683010fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:49.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:49.278+0000 7f68370fd700 1 --2- 172.21.15.32:0/1400489571 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68300fcf60 0x7f6830110040 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:02:49.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:49.278+0000 7f68370fd700 1 -- 172.21.15.32:0/1400489571 >> 172.21.15.32:0/1400489571 conn(0x7f68300f7430 msgr2=0x7f6830065bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:02:49.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:49.279+0000 7f68370fd700 1 -- 172.21.15.32:0/1400489571 shutdown_connections 2024-05-12T04:02:49.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:02:49.280+0000 7f68370fd700 1 -- 172.21.15.32:0/1400489571 wait complete. 2024-05-12T04:02:49.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:49 smithi032 ceph-mon[40008]: from='client.14822 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:02:49.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:49 smithi032 ceph-mon[40008]: mgrmap e33: smithi032.ukatkq(active, since 37s), standbys: smithi033.pcvxwy 2024-05-12T04:02:49.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:49 smithi032 ceph-mon[40008]: pgmap v22: 65 pgs: 65 active+clean; 305 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 118 op/s 2024-05-12T04:02:49.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:49 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/3796854843' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-12T04:02:49.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:49 smithi032 ceph-mon[40008]: from='client.24581 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:02:49.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:49 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' 2024-05-12T04:02:49.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:49 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' 2024-05-12T04:02:50.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:50 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/1400489571' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-12T04:02:50.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:50 smithi032 ceph-mon[40008]: from='client.? 172.21.15.32:0/1400489571' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-12T04:02:51.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:51 smithi033 ceph-mon[49525]: pgmap v23: 65 pgs: 65 active+clean; 305 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 99 op/s 2024-05-12T04:02:51.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:51 smithi032 ceph-mon[40008]: pgmap v23: 65 pgs: 65 active+clean; 305 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 99 op/s 2024-05-12T04:02:54.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:53 smithi033 ceph-mon[49525]: pgmap v24: 65 pgs: 65 active+clean; 308 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 454 op/s 2024-05-12T04:02:54.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:53 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' 2024-05-12T04:02:54.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:53 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' 2024-05-12T04:02:54.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:53 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:02:54.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:53 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T04:02:54.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:53 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' 2024-05-12T04:02:54.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:53 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:02:54.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:53 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:02:54.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:53 smithi033 ceph-mon[49525]: Upgrade: Setting container_image for all mgr 2024-05-12T04:02:54.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:53 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' 2024-05-12T04:02:54.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:53 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi032.ukatkq"}]: dispatch 2024-05-12T04:02:54.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:53 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi032.ukatkq"}]': finished 2024-05-12T04:02:54.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:53 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi033.pcvxwy"}]: dispatch 2024-05-12T04:02:54.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:53 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi033.pcvxwy"}]': finished 2024-05-12T04:02:54.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:53 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-05-12T04:02:54.292 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:53 smithi032 ceph-mon[40008]: pgmap v24: 65 pgs: 65 active+clean; 308 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 454 op/s 2024-05-12T04:02:54.292 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:53 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' 2024-05-12T04:02:54.292 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:53 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' 2024-05-12T04:02:54.292 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:53 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:02:54.292 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:53 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T04:02:54.292 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:53 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' 2024-05-12T04:02:54.292 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:53 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:02:54.292 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:53 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:02:54.292 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:53 smithi032 ceph-mon[40008]: Upgrade: Setting container_image for all mgr 2024-05-12T04:02:54.292 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:53 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' 2024-05-12T04:02:54.292 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:53 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi032.ukatkq"}]: dispatch 2024-05-12T04:02:54.293 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:53 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi032.ukatkq"}]': finished 2024-05-12T04:02:54.293 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:53 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi033.pcvxwy"}]: dispatch 2024-05-12T04:02:54.293 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:53 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi033.pcvxwy"}]': finished 2024-05-12T04:02:54.293 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:53 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-05-12T04:02:56.034 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:55 smithi032 ceph-mon[40008]: pgmap v25: 65 pgs: 65 active+clean; 308 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 404 op/s 2024-05-12T04:02:56.034 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:55 smithi032 ceph-mon[40008]: Upgrade: Updating mon.smithi032 2024-05-12T04:02:56.034 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:55 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' 2024-05-12T04:02:56.034 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:55 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-12T04:02:56.034 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:55 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-12T04:02:56.034 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:55 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:02:56.035 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:55 smithi032 ceph-mon[40008]: Deploying daemon mon.smithi032 on smithi032 2024-05-12T04:02:56.035 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:55 smithi032 ceph-mon[40008]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:02:56.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:55 smithi033 ceph-mon[49525]: pgmap v25: 65 pgs: 65 active+clean; 308 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 404 op/s 2024-05-12T04:02:56.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:55 smithi033 ceph-mon[49525]: Upgrade: Updating mon.smithi032 2024-05-12T04:02:56.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:55 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' 2024-05-12T04:02:56.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:55 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-12T04:02:56.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:55 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-12T04:02:56.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:55 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:02:56.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:55 smithi033 ceph-mon[49525]: Deploying daemon mon.smithi032 on smithi032 2024-05-12T04:02:56.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:02:55 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:02:56.793 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:56 smithi032 systemd[1]: Stopping Ceph mon.smithi032 for a3e178f8-1012-11ef-bc98-c7b262605968... 2024-05-12T04:02:57.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:57 smithi032 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-mon-smithi032[39985]: 2024-05-12T04:02:57.003+0000 7f54f04d0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi032 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-12T04:02:57.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:57 smithi032 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-mon-smithi032[39985]: 2024-05-12T04:02:57.003+0000 7f54f04d0700 -1 mon.smithi032@0(leader) e2 *** Got Signal Terminated *** 2024-05-12T04:02:59.263 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:58 smithi032 bash[120199]: ceph-a3e178f8-1012-11ef-bc98-c7b262605968-mon-smithi032 2024-05-12T04:02:59.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:59 smithi032 systemd[1]: ceph-a3e178f8-1012-11ef-bc98-c7b262605968@mon.smithi032.service: Succeeded. 2024-05-12T04:02:59.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:59 smithi032 systemd[1]: Stopped Ceph mon.smithi032 for a3e178f8-1012-11ef-bc98-c7b262605968. 2024-05-12T04:02:59.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:02:59 smithi032 systemd[1]: Starting Ceph mon.smithi032 for a3e178f8-1012-11ef-bc98-c7b262605968... 2024-05-12T04:03:00.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: set uid:gid to 167:167 (ceph:ceph) 2024-05-12T04:03:00.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable), process ceph-mon, pid 7 2024-05-12T04:03:00.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: pidfile_write: ignore empty --pid-file 2024-05-12T04:03:00.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: load: jerasure load: lrc 2024-05-12T04:03:00.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: RocksDB version: 7.9.2 2024-05-12T04:03:00.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Git sha 0 2024-05-12T04:03:00.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Compile date 2024-04-30 20:57:58 2024-05-12T04:03:00.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: DB SUMMARY 2024-05-12T04:03:00.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: DB Session ID: IGYDPKGT180QAJHWRE62 2024-05-12T04:03:00.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: CURRENT file: CURRENT 2024-05-12T04:03:00.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: IDENTITY file: IDENTITY 2024-05-12T04:03:00.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: MANIFEST file: MANIFEST-000015 size: 1165 Bytes 2024-05-12T04:03:00.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi032/store.db dir, Total Num: 1, files: 000033.sst 2024-05-12T04:03:00.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi032/store.db: 000031.log size: 132158 ; 2024-05-12T04:03:00.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.error_if_exists: 0 2024-05-12T04:03:00.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.create_if_missing: 0 2024-05-12T04:03:00.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.paranoid_checks: 1 2024-05-12T04:03:00.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-12T04:03:00.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-12T04:03:00.542 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-12T04:03:00.542 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.env: 0x55e585081820 2024-05-12T04:03:00.542 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.fs: PosixFileSystem 2024-05-12T04:03:00.542 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.info_log: 0x55e5869b93c0 2024-05-12T04:03:00.542 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.max_file_opening_threads: 16 2024-05-12T04:03:00.542 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.statistics: (nil) 2024-05-12T04:03:00.542 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.use_fsync: 0 2024-05-12T04:03:00.542 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.max_log_file_size: 0 2024-05-12T04:03:00.542 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-12T04:03:00.542 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-12T04:03:00.542 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.keep_log_file_num: 1000 2024-05-12T04:03:00.542 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.recycle_log_file_num: 0 2024-05-12T04:03:00.542 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.allow_fallocate: 1 2024-05-12T04:03:00.542 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.allow_mmap_reads: 0 2024-05-12T04:03:00.543 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.allow_mmap_writes: 0 2024-05-12T04:03:00.543 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.use_direct_reads: 0 2024-05-12T04:03:00.543 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-12T04:03:00.543 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.create_missing_column_families: 0 2024-05-12T04:03:00.543 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.db_log_dir: 2024-05-12T04:03:00.543 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.wal_dir: 2024-05-12T04:03:00.543 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-12T04:03:00.543 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-12T04:03:00.543 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-12T04:03:00.543 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-12T04:03:00.543 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-12T04:03:00.543 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-12T04:03:00.544 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.advise_random_on_open: 1 2024-05-12T04:03:00.544 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.db_write_buffer_size: 0 2024-05-12T04:03:00.544 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.write_buffer_manager: 0x55e585c2e140 2024-05-12T04:03:00.544 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-12T04:03:00.544 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-12T04:03:00.544 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-12T04:03:00.544 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.rate_limiter: (nil) 2024-05-12T04:03:00.544 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-12T04:03:00.544 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.wal_recovery_mode: 2 2024-05-12T04:03:00.544 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.enable_thread_tracking: 0 2024-05-12T04:03:00.544 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.enable_pipelined_write: 0 2024-05-12T04:03:00.544 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.unordered_write: 0 2024-05-12T04:03:00.544 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-12T04:03:00.544 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-12T04:03:00.544 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-12T04:03:00.545 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-12T04:03:00.545 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.row_cache: None 2024-05-12T04:03:00.545 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.wal_filter: None 2024-05-12T04:03:00.545 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-12T04:03:00.545 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.allow_ingest_behind: 0 2024-05-12T04:03:00.545 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.two_write_queues: 0 2024-05-12T04:03:00.545 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.manual_wal_flush: 0 2024-05-12T04:03:00.545 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.wal_compression: 0 2024-05-12T04:03:00.545 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.atomic_flush: 0 2024-05-12T04:03:00.545 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-12T04:03:00.545 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-12T04:03:00.545 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-12T04:03:00.545 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.log_readahead_size: 0 2024-05-12T04:03:00.545 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-12T04:03:00.545 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.best_efforts_recovery: 0 2024-05-12T04:03:00.546 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-12T04:03:00.546 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-12T04:03:00.546 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.allow_data_in_errors: 0 2024-05-12T04:03:00.546 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.db_host_id: __hostname__ 2024-05-12T04:03:00.546 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-12T04:03:00.546 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.max_background_jobs: 2 2024-05-12T04:03:00.546 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.max_background_compactions: -1 2024-05-12T04:03:00.546 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.max_subcompactions: 1 2024-05-12T04:03:00.546 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-12T04:03:00.546 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-12T04:03:00.546 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-12T04:03:00.546 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.max_total_wal_size: 0 2024-05-12T04:03:00.546 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-12T04:03:00.547 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-12T04:03:00.547 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-12T04:03:00.547 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-12T04:03:00.547 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.max_open_files: -1 2024-05-12T04:03:00.547 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.bytes_per_sync: 0 2024-05-12T04:03:00.547 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-12T04:03:00.547 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-12T04:03:00.547 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.compaction_readahead_size: 0 2024-05-12T04:03:00.547 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.max_background_flushes: -1 2024-05-12T04:03:00.547 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Compression algorithms supported: 2024-05-12T04:03:00.547 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-12T04:03:00.547 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: kZSTD supported: 0 2024-05-12T04:03:00.547 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: kXpressCompression supported: 0 2024-05-12T04:03:00.547 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: kLZ4HCCompression supported: 1 2024-05-12T04:03:00.547 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: kZlibCompression supported: 1 2024-05-12T04:03:00.547 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: kSnappyCompression supported: 1 2024-05-12T04:03:00.548 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: kLZ4Compression supported: 1 2024-05-12T04:03:00.548 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: kBZip2Compression supported: 0 2024-05-12T04:03:00.548 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-12T04:03:00.548 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-12T04:03:00.548 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi032/store.db/MANIFEST-000015 2024-05-12T04:03:00.548 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-12T04:03:00.548 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-12T04:03:00.548 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.merge_operator: 2024-05-12T04:03:00.548 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.compaction_filter: None 2024-05-12T04:03:00.548 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.compaction_filter_factory: None 2024-05-12T04:03:00.548 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.sst_partitioner_factory: None 2024-05-12T04:03:00.548 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-12T04:03:00.548 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-12T04:03:00.548 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55e5869b94e0) 2024-05-12T04:03:00.548 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: cache_index_and_filter_blocks: 1 2024-05-12T04:03:00.549 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-12T04:03:00.549 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-12T04:03:00.549 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: pin_top_level_index_and_filter: 1 2024-05-12T04:03:00.549 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: index_type: 0 2024-05-12T04:03:00.549 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: data_block_index_type: 0 2024-05-12T04:03:00.549 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: index_shortening: 1 2024-05-12T04:03:00.549 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-12T04:03:00.549 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: checksum: 4 2024-05-12T04:03:00.549 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: no_block_cache: 0 2024-05-12T04:03:00.549 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: block_cache: 0x55e585ccf1f0 2024-05-12T04:03:00.549 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: block_cache_name: BinnedLRUCache 2024-05-12T04:03:00.549 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: block_cache_options: 2024-05-12T04:03:00.549 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: capacity : 536870912 2024-05-12T04:03:00.549 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: num_shard_bits : 4 2024-05-12T04:03:00.549 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: strict_capacity_limit : 0 2024-05-12T04:03:00.549 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: high_pri_pool_ratio: 0.000 2024-05-12T04:03:00.550 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: block_cache_compressed: (nil) 2024-05-12T04:03:00.550 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: persistent_cache: (nil) 2024-05-12T04:03:00.550 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: block_size: 4096 2024-05-12T04:03:00.550 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: block_size_deviation: 10 2024-05-12T04:03:00.550 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: block_restart_interval: 16 2024-05-12T04:03:00.550 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: index_block_restart_interval: 1 2024-05-12T04:03:00.550 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: metadata_block_size: 4096 2024-05-12T04:03:00.550 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: partition_filters: 0 2024-05-12T04:03:00.550 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: use_delta_encoding: 1 2024-05-12T04:03:00.550 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: filter_policy: bloomfilter 2024-05-12T04:03:00.550 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: whole_key_filtering: 1 2024-05-12T04:03:00.550 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: verify_compression: 0 2024-05-12T04:03:00.550 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: read_amp_bytes_per_bit: 0 2024-05-12T04:03:00.550 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: format_version: 5 2024-05-12T04:03:00.551 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: enable_index_compression: 1 2024-05-12T04:03:00.551 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: block_align: 0 2024-05-12T04:03:00.551 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: max_auto_readahead_size: 262144 2024-05-12T04:03:00.551 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: prepopulate_block_cache: 0 2024-05-12T04:03:00.551 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: initial_auto_readahead_size: 8192 2024-05-12T04:03:00.551 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: num_file_reads_for_auto_readahead: 2 2024-05-12T04:03:00.551 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.write_buffer_size: 33554432 2024-05-12T04:03:00.551 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.max_write_buffer_number: 2 2024-05-12T04:03:00.551 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.compression: NoCompression 2024-05-12T04:03:00.551 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.bottommost_compression: Disabled 2024-05-12T04:03:00.551 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.prefix_extractor: nullptr 2024-05-12T04:03:00.551 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-12T04:03:00.551 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.num_levels: 7 2024-05-12T04:03:00.551 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-12T04:03:00.551 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-12T04:03:00.552 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-12T04:03:00.552 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-12T04:03:00.552 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-12T04:03:00.552 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-12T04:03:00.552 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-12T04:03:00.553 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-12T04:03:00.553 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-12T04:03:00.553 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-12T04:03:00.553 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-12T04:03:00.553 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-12T04:03:00.553 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-12T04:03:00.553 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.compression_opts.level: 32767 2024-05-12T04:03:00.553 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.compression_opts.strategy: 0 2024-05-12T04:03:00.554 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-12T04:03:00.554 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-12T04:03:00.554 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-12T04:03:00.554 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-12T04:03:00.554 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.compression_opts.enabled: false 2024-05-12T04:03:00.554 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-12T04:03:00.554 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-12T04:03:00.554 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-12T04:03:00.554 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-12T04:03:00.554 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.target_file_size_base: 67108864 2024-05-12T04:03:00.554 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-12T04:03:00.554 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-12T04:03:00.554 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-12T04:03:00.554 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-12T04:03:00.555 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-12T04:03:00.555 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-12T04:03:00.555 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-12T04:03:00.555 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-12T04:03:00.555 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-12T04:03:00.555 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-12T04:03:00.555 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-12T04:03:00.555 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-12T04:03:00.555 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-12T04:03:00.555 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-12T04:03:00.555 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.arena_block_size: 1048576 2024-05-12T04:03:00.555 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-12T04:03:00.555 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-12T04:03:00.555 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.disable_auto_compactions: 0 2024-05-12T04:03:00.555 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-12T04:03:00.555 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-12T04:03:00.556 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-12T04:03:00.556 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-12T04:03:00.556 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-12T04:03:00.556 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-12T04:03:00.556 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-12T04:03:00.556 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-12T04:03:00.556 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-12T04:03:00.556 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-12T04:03:00.556 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.table_properties_collectors: 2024-05-12T04:03:00.556 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.inplace_update_support: 0 2024-05-12T04:03:00.556 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-12T04:03:00.556 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-12T04:03:00.556 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-12T04:03:00.556 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-12T04:03:00.556 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.bloom_locality: 0 2024-05-12T04:03:00.557 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.max_successive_merges: 0 2024-05-12T04:03:00.557 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-12T04:03:00.557 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.paranoid_file_checks: 0 2024-05-12T04:03:00.557 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.force_consistency_checks: 1 2024-05-12T04:03:00.557 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.report_bg_io_stats: 0 2024-05-12T04:03:00.557 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.ttl: 2592000 2024-05-12T04:03:00.557 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-12T04:03:00.557 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-12T04:03:00.557 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-12T04:03:00.557 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.enable_blob_files: false 2024-05-12T04:03:00.557 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.min_blob_size: 0 2024-05-12T04:03:00.557 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.blob_file_size: 268435456 2024-05-12T04:03:00.557 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-12T04:03:00.557 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-12T04:03:00.558 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-12T04:03:00.558 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-12T04:03:00.558 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-12T04:03:00.558 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.blob_file_starting_level: 0 2024-05-12T04:03:00.558 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-12T04:03:00.558 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 33.sst 2024-05-12T04:03:00.558 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-05-12T04:03:00.558 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi032/store.db/MANIFEST-000015 succeeded,manifest_file_number is 15, next_file_number is 35, last_sequence is 11911, log_number is 31,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-12T04:03:00.558 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 31 2024-05-12T04:03:00.558 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 6b5adf2d-d544-4e40-89cd-4cfcd89bf7d6 2024-05-12T04:03:00.558 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715486580058941, "job": 1, "event": "recovery_started", "wal_files": [31]} 2024-05-12T04:03:00.558 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #31 mode 2 2024-05-12T04:03:00.558 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715486580060277, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 36, "file_size": 131470, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 11912, "largest_seqno": 12050, "table_properties": {"data_size": 129793, "index_size": 409, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 261, "raw_key_size": 2289, "raw_average_key_size": 28, "raw_value_size": 127881, "raw_average_value_size": 1618, "num_data_blocks": 17, "num_entries": 79, "num_filter_entries": 79, "num_deletions": 2, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715486580, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "6b5adf2d-d544-4e40-89cd-4cfcd89bf7d6", "db_session_id": "IGYDPKGT180QAJHWRE62", "orig_file_number": 36, "seqno_to_time_mapping": "N/A"}} 2024-05-12T04:03:00.558 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715486580060362, "job": 1, "event": "recovery_finished"} 2024-05-12T04:03:00.559 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: [db/version_set.cc:5047] Creating manifest 38 2024-05-12T04:03:00.559 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-05-12T04:03:00.559 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi032/store.db/000031.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-12T04:03:00.559 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55e585d6e000 2024-05-12T04:03:00.559 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: DB pointer 0x55e585d5c000 2024-05-12T04:03:00.559 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-12T04:03:00.559 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-12T04:03:00.559 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: ** DB Stats ** 2024-05-12T04:03:00.559 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-12T04:03:00.559 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-12T04:03:00.559 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-12T04:03:00.559 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-12T04:03:00.559 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-12T04:03:00.559 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-12T04:03:00.559 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-12T04:03:00.559 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: 2024-05-12T04:03:00.560 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: ** Compaction Stats [default] ** 2024-05-12T04:03:00.560 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-12T04:03:00.560 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-12T04:03:00.560 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: L0 1/0 128.39 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 131.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-12T04:03:00.560 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: L6 1/0 6.83 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-05-12T04:03:00.560 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Sum 2/0 6.96 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 131.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-12T04:03:00.560 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 131.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-12T04:03:00.560 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: 2024-05-12T04:03:00.560 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: ** Compaction Stats [default] ** 2024-05-12T04:03:00.560 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-12T04:03:00.560 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-12T04:03:00.560 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 131.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-12T04:03:00.560 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: 2024-05-12T04:03:00.560 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-12T04:03:00.560 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: 2024-05-12T04:03:00.561 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-12T04:03:00.561 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-12T04:03:00.561 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-12T04:03:00.561 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-12T04:03:00.561 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-12T04:03:00.561 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-12T04:03:00.561 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Cumulative compaction: 0.00 GB write, 19.60 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-12T04:03:00.561 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Interval compaction: 0.00 GB write, 19.60 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-12T04:03:00.561 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-12T04:03:00.561 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Block cache BinnedLRUCache@0x55e585ccf1f0#7 capacity: 512.00 MB usage: 0.80 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-05-12T04:03:00.561 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.31 KB,5.96046e-05%) IndexBlock(1,0.48 KB,9.23872e-05%) Misc(1,0.00 KB,0%) 2024-05-12T04:03:00.561 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: 2024-05-12T04:03:00.561 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-12T04:03:00.561 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: starting mon.smithi032 rank 0 at public addrs [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] at bind addrs [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi032 fsid a3e178f8-1012-11ef-bc98-c7b262605968 2024-05-12T04:03:00.561 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: mon.smithi032@-1(???) e2 preinit fsid a3e178f8-1012-11ef-bc98-c7b262605968 2024-05-12T04:03:00.562 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: mon.smithi032@-1(???).mds e18 new map 2024-05-12T04:03:00.562 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: mon.smithi032@-1(???).mds e18 print_map 2024-05-12T04:03:00.562 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: e18 2024-05-12T04:03:00.562 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-12T04:03:00.562 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-12T04:03:00.562 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: legacy client fscid: 1 2024-05-12T04:03:00.562 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: 2024-05-12T04:03:00.562 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Filesystem 'cephfs' (1) 2024-05-12T04:03:00.562 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: fs_name cephfs 2024-05-12T04:03:00.562 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: epoch 18 2024-05-12T04:03:00.562 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-12T04:03:00.562 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: created 2024-05-12T03:57:48.904614+0000 2024-05-12T04:03:00.562 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: modified 2024-05-12T03:58:06.821591+0000 2024-05-12T04:03:00.562 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: tableserver 0 2024-05-12T04:03:00.563 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: root 0 2024-05-12T04:03:00.563 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: session_timeout 60 2024-05-12T04:03:00.563 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: session_autoclose 300 2024-05-12T04:03:00.563 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: max_file_size 1099511627776 2024-05-12T04:03:00.563 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: max_xattr_size 65536 2024-05-12T04:03:00.563 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: required_client_features {} 2024-05-12T04:03:00.563 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: last_failure 0 2024-05-12T04:03:00.563 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: last_failure_osd_epoch 46 2024-05-12T04:03:00.563 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-12T04:03:00.563 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: max_mds 1 2024-05-12T04:03:00.563 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: in 0 2024-05-12T04:03:00.563 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: up {0=14474} 2024-05-12T04:03:00.563 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: failed 2024-05-12T04:03:00.563 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: damaged 2024-05-12T04:03:00.563 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: stopped 2024-05-12T04:03:00.564 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: data_pools [3] 2024-05-12T04:03:00.564 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: metadata_pool 2 2024-05-12T04:03:00.564 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: inline_data enabled 2024-05-12T04:03:00.564 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: balancer 2024-05-12T04:03:00.564 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: bal_rank_mask -1 2024-05-12T04:03:00.564 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: standby_count_wanted 1 2024-05-12T04:03:00.564 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: [mds.cephfs.smithi032.qpsawh{0:14474} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.32:6828/3100155296,v1:172.21.15.32:6829/3100155296] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:03:00.564 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: [mds.cephfs.smithi033.jgbjhk{0:24267} state up:standby-replay seq 3 join_fscid=1 addr [v2:172.21.15.33:6824/55417471,v1:172.21.15.33:6825/55417471] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:03:00.564 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: 2024-05-12T04:03:00.564 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: 2024-05-12T04:03:00.564 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: Standby daemons: 2024-05-12T04:03:00.564 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: 2024-05-12T04:03:00.564 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: [mds.cephfs.smithi033.hglwml{-1:24283} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/433858238,v1:172.21.15.33:6827/433858238] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:03:00.564 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout: [mds.cephfs.smithi032.ialtqb{-1:24287} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.32:6826/3703132329,v1:172.21.15.32:6827/3703132329] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:03:00.564 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: mon.smithi032@-1(???).osd e48 crush map has features 3314933000852226048, adjusting msgr requires 2024-05-12T04:03:00.564 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: mon.smithi032@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-05-12T04:03:00.565 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: mon.smithi032@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-05-12T04:03:00.565 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: mon.smithi032@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-05-12T04:03:00.565 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 ceph-mon[120392]: mon.smithi032@-1(???).paxosservice(auth 1..21) refresh upgraded, format 0 -> 3 2024-05-12T04:03:00.565 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 bash[120356]: 81f94171c5dc453160e5b465b251d95b0a1ca8c1678da632898a3902daae8ae5 2024-05-12T04:03:00.565 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:00 smithi032 systemd[1]: Started Ceph mon.smithi032 for a3e178f8-1012-11ef-bc98-c7b262605968. 2024-05-12T04:03:06.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:06 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mon metadata", "id": "smithi032"}]: dispatch 2024-05-12T04:03:06.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:06 smithi033 ceph-mon[49525]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mon metadata", "id": "smithi033"}]: dispatch 2024-05-12T04:03:06.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:06 smithi033 ceph-mon[49525]: pgmap v27: 65 pgs: 65 active+clean; 309 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 532 op/s 2024-05-12T04:03:06.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:06 smithi033 ceph-mon[49525]: pgmap v28: 65 pgs: 65 active+clean; 309 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 493 op/s 2024-05-12T04:03:06.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:06 smithi033 ceph-mon[49525]: pgmap v29: 65 pgs: 65 active+clean; 305 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 586 op/s 2024-05-12T04:03:06.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:06 smithi033 ceph-mon[49525]: pgmap v30: 65 pgs: 65 active+clean; 305 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 231 op/s 2024-05-12T04:03:06.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:06 smithi033 ceph-mon[49525]: mon.smithi032 calling monitor election 2024-05-12T04:03:06.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:06 smithi033 ceph-mon[49525]: mon.smithi032 is new leader, mons smithi032,smithi033 in quorum (ranks 0,1) 2024-05-12T04:03:06.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:06 smithi033 ceph-mon[49525]: monmap e2: 2 mons at {smithi032=[v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0],smithi033=[v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-12T04:03:06.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:06 smithi033 ceph-mon[49525]: fsmap cephfs:1 {0=cephfs.smithi032.qpsawh=up:active} 1 up:standby-replay 2 up:standby 2024-05-12T04:03:06.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:06 smithi033 ceph-mon[49525]: osdmap e48: 6 total, 6 up, 6 in 2024-05-12T04:03:06.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:06 smithi033 ceph-mon[49525]: mgrmap e33: smithi032.ukatkq(active, since 56s), standbys: smithi033.pcvxwy 2024-05-12T04:03:06.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:06 smithi033 ceph-mon[49525]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-12T04:03:06.664 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:06 smithi033 ceph-mon[49525]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-12T04:03:06.664 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:06 smithi033 ceph-mon[49525]: fs cephfs has deprecated feature inline_data enabled. 2024-05-12T04:03:06.664 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:06 smithi033 ceph-mon[49525]: mgrmap e34: smithi032.ukatkq(active, since 56s), standbys: smithi033.pcvxwy 2024-05-12T04:03:06.664 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:06 smithi033 ceph-mon[49525]: from='mgr.24501 ' entity='' 2024-05-12T04:03:06.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:06 smithi032 ceph-mon[120392]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mon metadata", "id": "smithi032"}]: dispatch 2024-05-12T04:03:06.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:06 smithi032 ceph-mon[120392]: from='mgr.24501 172.21.15.32:0/4236795275' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mon metadata", "id": "smithi033"}]: dispatch 2024-05-12T04:03:06.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:06 smithi032 ceph-mon[120392]: pgmap v27: 65 pgs: 65 active+clean; 309 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 532 op/s 2024-05-12T04:03:06.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:06 smithi032 ceph-mon[120392]: pgmap v28: 65 pgs: 65 active+clean; 309 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 493 op/s 2024-05-12T04:03:06.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:06 smithi032 ceph-mon[120392]: pgmap v29: 65 pgs: 65 active+clean; 305 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 586 op/s 2024-05-12T04:03:06.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:06 smithi032 ceph-mon[120392]: pgmap v30: 65 pgs: 65 active+clean; 305 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 231 op/s 2024-05-12T04:03:06.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:06 smithi032 ceph-mon[120392]: mon.smithi032 calling monitor election 2024-05-12T04:03:06.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:06 smithi032 ceph-mon[120392]: mon.smithi032 is new leader, mons smithi032,smithi033 in quorum (ranks 0,1) 2024-05-12T04:03:06.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:06 smithi032 ceph-mon[120392]: monmap e2: 2 mons at {smithi032=[v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0],smithi033=[v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-12T04:03:06.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:06 smithi032 ceph-mon[120392]: fsmap cephfs:1 {0=cephfs.smithi032.qpsawh=up:active} 1 up:standby-replay 2 up:standby 2024-05-12T04:03:06.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:06 smithi032 ceph-mon[120392]: osdmap e48: 6 total, 6 up, 6 in 2024-05-12T04:03:06.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:06 smithi032 ceph-mon[120392]: mgrmap e33: smithi032.ukatkq(active, since 56s), standbys: smithi033.pcvxwy 2024-05-12T04:03:06.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:06 smithi032 ceph-mon[120392]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-12T04:03:06.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:06 smithi032 ceph-mon[120392]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-12T04:03:06.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:06 smithi032 ceph-mon[120392]: fs cephfs has deprecated feature inline_data enabled. 2024-05-12T04:03:06.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:06 smithi032 ceph-mon[120392]: mgrmap e34: smithi032.ukatkq(active, since 56s), standbys: smithi033.pcvxwy 2024-05-12T04:03:06.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:06 smithi032 ceph-mon[120392]: from='mgr.24501 ' entity='' 2024-05-12T04:03:14.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:14 smithi033 ceph-mon[49525]: Standby manager daemon smithi033.pcvxwy restarted 2024-05-12T04:03:14.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:14 smithi033 ceph-mon[49525]: Standby manager daemon smithi033.pcvxwy started 2024-05-12T04:03:14.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:14 smithi033 ceph-mon[49525]: from='mgr.? 172.21.15.33:0/272879505' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi033.pcvxwy/crt"}]: dispatch 2024-05-12T04:03:14.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:14 smithi033 ceph-mon[49525]: from='mgr.? 172.21.15.33:0/272879505' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-12T04:03:14.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:14 smithi033 ceph-mon[49525]: from='mgr.? 172.21.15.33:0/272879505' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi033.pcvxwy/key"}]: dispatch 2024-05-12T04:03:14.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:14 smithi033 ceph-mon[49525]: from='mgr.? 172.21.15.33:0/272879505' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-12T04:03:14.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:14 smithi032 ceph-mon[120392]: Standby manager daemon smithi033.pcvxwy restarted 2024-05-12T04:03:14.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:14 smithi032 ceph-mon[120392]: Standby manager daemon smithi033.pcvxwy started 2024-05-12T04:03:14.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:14 smithi032 ceph-mon[120392]: from='mgr.? 172.21.15.33:0/272879505' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi033.pcvxwy/crt"}]: dispatch 2024-05-12T04:03:14.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:14 smithi032 ceph-mon[120392]: from='mgr.? 172.21.15.33:0/272879505' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-12T04:03:14.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:14 smithi032 ceph-mon[120392]: from='mgr.? 172.21.15.33:0/272879505' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi033.pcvxwy/key"}]: dispatch 2024-05-12T04:03:14.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:14 smithi032 ceph-mon[120392]: from='mgr.? 172.21.15.33:0/272879505' entity='mgr.smithi033.pcvxwy' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-12T04:03:15.237 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:15 smithi032 ceph-mon[120392]: mgrmap e35: smithi032.ukatkq(active, since 63s), standbys: smithi033.pcvxwy 2024-05-12T04:03:15.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:15 smithi033 ceph-mon[49525]: mgrmap e35: smithi032.ukatkq(active, since 63s), standbys: smithi033.pcvxwy 2024-05-12T04:03:16.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:16 smithi033 ceph-mon[49525]: Active manager daemon smithi032.ukatkq restarted 2024-05-12T04:03:16.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:16 smithi033 ceph-mon[49525]: Activating manager daemon smithi032.ukatkq 2024-05-12T04:03:16.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:16 smithi033 ceph-mon[49525]: osdmap e49: 6 total, 6 up, 6 in 2024-05-12T04:03:16.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:16 smithi033 ceph-mon[49525]: mgrmap e36: smithi032.ukatkq(active, starting, since 0.00579111s), standbys: smithi033.pcvxwy 2024-05-12T04:03:16.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:16 smithi033 ceph-mon[49525]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mon metadata", "id": "smithi032"}]: dispatch 2024-05-12T04:03:16.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:16 smithi033 ceph-mon[49525]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mon metadata", "id": "smithi033"}]: dispatch 2024-05-12T04:03:16.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:16 smithi033 ceph-mon[49525]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi032.qpsawh"}]: dispatch 2024-05-12T04:03:16.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:16 smithi033 ceph-mon[49525]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi033.jgbjhk"}]: dispatch 2024-05-12T04:03:16.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:16 smithi033 ceph-mon[49525]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi033.hglwml"}]: dispatch 2024-05-12T04:03:16.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:16 smithi033 ceph-mon[49525]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi032.ialtqb"}]: dispatch 2024-05-12T04:03:16.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:16 smithi033 ceph-mon[49525]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mgr metadata", "who": "smithi032.ukatkq", "id": "smithi032.ukatkq"}]: dispatch 2024-05-12T04:03:16.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:16 smithi033 ceph-mon[49525]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mgr metadata", "who": "smithi033.pcvxwy", "id": "smithi033.pcvxwy"}]: dispatch 2024-05-12T04:03:16.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:16 smithi033 ceph-mon[49525]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-12T04:03:16.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:16 smithi033 ceph-mon[49525]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-12T04:03:16.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:16 smithi033 ceph-mon[49525]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-12T04:03:16.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:16 smithi033 ceph-mon[49525]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-12T04:03:16.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:16 smithi033 ceph-mon[49525]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-12T04:03:16.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:16 smithi033 ceph-mon[49525]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-12T04:03:16.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:16 smithi033 ceph-mon[49525]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-12T04:03:16.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:16 smithi033 ceph-mon[49525]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-12T04:03:16.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:16 smithi033 ceph-mon[49525]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-12T04:03:16.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:16 smithi033 ceph-mon[49525]: Manager daemon smithi032.ukatkq is now available 2024-05-12T04:03:16.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:16 smithi033 ceph-mon[49525]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:03:16.164 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:16 smithi033 ceph-mon[49525]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:03:16.164 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:16 smithi033 ceph-mon[49525]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi032.ukatkq/mirror_snapshot_schedule"}]: dispatch 2024-05-12T04:03:16.164 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:16 smithi033 ceph-mon[49525]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi032.ukatkq/trash_purge_schedule"}]: dispatch 2024-05-12T04:03:16.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:16 smithi032 ceph-mon[120392]: Active manager daemon smithi032.ukatkq restarted 2024-05-12T04:03:16.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:16 smithi032 ceph-mon[120392]: Activating manager daemon smithi032.ukatkq 2024-05-12T04:03:16.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:16 smithi032 ceph-mon[120392]: osdmap e49: 6 total, 6 up, 6 in 2024-05-12T04:03:16.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:16 smithi032 ceph-mon[120392]: mgrmap e36: smithi032.ukatkq(active, starting, since 0.00579111s), standbys: smithi033.pcvxwy 2024-05-12T04:03:16.542 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:16 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mon metadata", "id": "smithi032"}]: dispatch 2024-05-12T04:03:16.542 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:16 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mon metadata", "id": "smithi033"}]: dispatch 2024-05-12T04:03:16.542 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:16 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi032.qpsawh"}]: dispatch 2024-05-12T04:03:16.542 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:16 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi033.jgbjhk"}]: dispatch 2024-05-12T04:03:16.542 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:16 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi033.hglwml"}]: dispatch 2024-05-12T04:03:16.542 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:16 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi032.ialtqb"}]: dispatch 2024-05-12T04:03:16.543 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:16 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mgr metadata", "who": "smithi032.ukatkq", "id": "smithi032.ukatkq"}]: dispatch 2024-05-12T04:03:16.543 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:16 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mgr metadata", "who": "smithi033.pcvxwy", "id": "smithi033.pcvxwy"}]: dispatch 2024-05-12T04:03:16.543 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:16 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-12T04:03:16.543 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:16 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-12T04:03:16.543 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:16 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-12T04:03:16.543 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:16 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-12T04:03:16.544 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:16 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-12T04:03:16.544 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:16 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-12T04:03:16.544 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:16 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-12T04:03:16.544 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:16 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-12T04:03:16.544 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:16 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-12T04:03:16.544 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:16 smithi032 ceph-mon[120392]: Manager daemon smithi032.ukatkq is now available 2024-05-12T04:03:16.545 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:16 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:03:16.545 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:16 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:03:16.545 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:16 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi032.ukatkq/mirror_snapshot_schedule"}]: dispatch 2024-05-12T04:03:16.545 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:16 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi032.ukatkq/trash_purge_schedule"}]: dispatch 2024-05-12T04:03:17.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:17 smithi033 ceph-mon[49525]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:17.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:17 smithi033 ceph-mon[49525]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:17.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:17 smithi033 ceph-mon[49525]: mgrmap e37: smithi032.ukatkq(active, since 1.01205s), standbys: smithi033.pcvxwy 2024-05-12T04:03:17.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:17 smithi033 ceph-mon[49525]: [12/May/2024:04:03:17] ENGINE Bus STARTING 2024-05-12T04:03:17.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:17 smithi033 ceph-mon[49525]: [12/May/2024:04:03:17] ENGINE Serving on http://172.21.15.32:8765 2024-05-12T04:03:17.789 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:17 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:17.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:17 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:17.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:17 smithi032 ceph-mon[120392]: mgrmap e37: smithi032.ukatkq(active, since 1.01205s), standbys: smithi033.pcvxwy 2024-05-12T04:03:17.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:17 smithi032 ceph-mon[120392]: [12/May/2024:04:03:17] ENGINE Bus STARTING 2024-05-12T04:03:17.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:17 smithi032 ceph-mon[120392]: [12/May/2024:04:03:17] ENGINE Serving on http://172.21.15.32:8765 2024-05-12T04:03:19.039 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:18 smithi032 ceph-mon[120392]: [12/May/2024:04:03:17] ENGINE Serving on https://172.21.15.32:7150 2024-05-12T04:03:19.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:18 smithi032 ceph-mon[120392]: [12/May/2024:04:03:17] ENGINE Bus STARTED 2024-05-12T04:03:19.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:18 smithi032 ceph-mon[120392]: pgmap v4: 65 pgs: 65 active+clean; 279 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-05-12T04:03:19.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:18 smithi033 ceph-mon[49525]: [12/May/2024:04:03:17] ENGINE Serving on https://172.21.15.32:7150 2024-05-12T04:03:19.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:18 smithi033 ceph-mon[49525]: [12/May/2024:04:03:17] ENGINE Bus STARTED 2024-05-12T04:03:19.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:18 smithi033 ceph-mon[49525]: pgmap v4: 65 pgs: 65 active+clean; 279 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-05-12T04:03:19.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.420+0000 7f3027fcd700 1 -- 172.21.15.32:0/3595642946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30200fcf60 msgr2=0x7f30200fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:03:19.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.420+0000 7f3027fcd700 1 --2- 172.21.15.32:0/3595642946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30200fcf60 0x7f30200fd3d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3010004940 tx=0x7f3010009e30 comp rx=0 tx=0).stop 2024-05-12T04:03:19.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.420+0000 7f3027fcd700 1 -- 172.21.15.32:0/3595642946 shutdown_connections 2024-05-12T04:03:19.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.420+0000 7f3027fcd700 1 --2- 172.21.15.32:0/3595642946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30200fcf60 0x7f30200fd3d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:19.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.420+0000 7f3027fcd700 1 --2- 172.21.15.32:0/3595642946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30200fbcb0 0x7f30200fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:19.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.420+0000 7f3027fcd700 1 -- 172.21.15.32:0/3595642946 >> 172.21.15.32:0/3595642946 conn(0x7f30200f7430 msgr2=0x7f30200f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:03:19.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.420+0000 7f3027fcd700 1 -- 172.21.15.32:0/3595642946 shutdown_connections 2024-05-12T04:03:19.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.420+0000 7f3027fcd700 1 -- 172.21.15.32:0/3595642946 wait complete. 2024-05-12T04:03:19.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.421+0000 7f3027fcd700 1 Processor -- start 2024-05-12T04:03:19.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.421+0000 7f3027fcd700 1 -- start start 2024-05-12T04:03:19.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.421+0000 7f3027fcd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30200fbcb0 0x7f302010fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:03:19.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.421+0000 7f3027fcd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30200fcf60 0x7f3020110000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:03:19.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.421+0000 7f3027fcd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3020110600 con 0x7f30200fcf60 2024-05-12T04:03:19.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.421+0000 7f3027fcd700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3020110e00 con 0x7f30200fbcb0 2024-05-12T04:03:19.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.421+0000 7f3025568700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30200fcf60 0x7f3020110000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:03:19.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.421+0000 7f3025568700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30200fcf60 0x7f3020110000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57132/0 (socket says 172.21.15.32:57132) 2024-05-12T04:03:19.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.421+0000 7f3025568700 1 -- 172.21.15.32:0/4285172792 learned_addr learned my addr 172.21.15.32:0/4285172792 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:03:19.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.421+0000 7f3025d69700 1 --2- 172.21.15.32:0/4285172792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30200fbcb0 0x7f302010fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:03:19.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.422+0000 7f3025568700 1 -- 172.21.15.32:0/4285172792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30200fbcb0 msgr2=0x7f302010fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:03:19.424 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.422+0000 7f3025568700 1 --2- 172.21.15.32:0/4285172792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30200fbcb0 0x7f302010fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:19.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.422+0000 7f3025568700 1 -- 172.21.15.32:0/4285172792 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3010005040 con 0x7f30200fcf60 2024-05-12T04:03:19.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.422+0000 7f3025d69700 1 --2- 172.21.15.32:0/4285172792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30200fbcb0 0x7f302010fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-12T04:03:19.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.422+0000 7f3025568700 1 --2- 172.21.15.32:0/4285172792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30200fcf60 0x7f3020110000 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f30100137b0 tx=0x7f3010009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:03:19.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.422+0000 7f30167fc700 1 -- 172.21.15.32:0/4285172792 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f301000e070 con 0x7f30200fcf60 2024-05-12T04:03:19.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.422+0000 7f30167fc700 1 -- 172.21.15.32:0/4285172792 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f301001c400 con 0x7f30200fcf60 2024-05-12T04:03:19.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.422+0000 7f3027fcd700 1 -- 172.21.15.32:0/4285172792 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3020111080 con 0x7f30200fcf60 2024-05-12T04:03:19.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.422+0000 7f3027fcd700 1 -- 172.21.15.32:0/4285172792 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3020111560 con 0x7f30200fcf60 2024-05-12T04:03:19.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.422+0000 7f30167fc700 1 -- 172.21.15.32:0/4285172792 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f301000b650 con 0x7f30200fcf60 2024-05-12T04:03:19.425 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.423+0000 7f3027fcd700 1 -- 172.21.15.32:0/4285172792 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30040052e0 con 0x7f30200fcf60 2024-05-12T04:03:19.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.425+0000 7f30167fc700 1 -- 172.21.15.32:0/4285172792 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 90896+0+0 (secure 0 0 0) 0x7f3010013a30 con 0x7f30200fcf60 2024-05-12T04:03:19.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.425+0000 7f30167fc700 1 --2- 172.21.15.32:0/4285172792 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f300c06cd90 0x7f300c06f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:03:19.426 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.425+0000 7f30167fc700 1 -- 172.21.15.32:0/4285172792 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f301008e170 con 0x7f30200fcf60 2024-05-12T04:03:19.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.426+0000 7f3025d69700 1 --2- 172.21.15.32:0/4285172792 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f300c06cd90 0x7f300c06f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:03:19.427 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.426+0000 7f3025d69700 1 --2- 172.21.15.32:0/4285172792 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f300c06cd90 0x7f300c06f240 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f301c009b80 tx=0x7f301c00b040 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:03:19.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.431+0000 7f30167fc700 1 -- 172.21.15.32:0/4285172792 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f301005b880 con 0x7f30200fcf60 2024-05-12T04:03:19.730 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.729+0000 7f3027fcd700 1 -- 172.21.15.32:0/4285172792 --> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3004000bf0 con 0x7f300c06cd90 2024-05-12T04:03:19.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.730+0000 7f30167fc700 1 -- 172.21.15.32:0/4285172792 <== mgr.24593 v2:172.21.15.32:6800/2957111591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f3004000bf0 con 0x7f300c06cd90 2024-05-12T04:03:19.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.733+0000 7f3027fcd700 1 -- 172.21.15.32:0/4285172792 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f300c06cd90 msgr2=0x7f300c06f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:03:19.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.733+0000 7f3027fcd700 1 --2- 172.21.15.32:0/4285172792 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f300c06cd90 0x7f300c06f240 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f301c009b80 tx=0x7f301c00b040 comp rx=0 tx=0).stop 2024-05-12T04:03:19.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.733+0000 7f3027fcd700 1 -- 172.21.15.32:0/4285172792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30200fcf60 msgr2=0x7f3020110000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:03:19.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.733+0000 7f3027fcd700 1 --2- 172.21.15.32:0/4285172792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30200fcf60 0x7f3020110000 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f30100137b0 tx=0x7f3010009e30 comp rx=0 tx=0).stop 2024-05-12T04:03:19.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.733+0000 7f3027fcd700 1 -- 172.21.15.32:0/4285172792 shutdown_connections 2024-05-12T04:03:19.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.733+0000 7f3027fcd700 1 --2- 172.21.15.32:0/4285172792 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f300c06cd90 0x7f300c06f240 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:19.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.733+0000 7f3027fcd700 1 --2- 172.21.15.32:0/4285172792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30200fbcb0 0x7f302010fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:19.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.733+0000 7f3027fcd700 1 --2- 172.21.15.32:0/4285172792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30200fcf60 0x7f3020110000 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:19.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.733+0000 7f3027fcd700 1 -- 172.21.15.32:0/4285172792 >> 172.21.15.32:0/4285172792 conn(0x7f30200f7430 msgr2=0x7f3020065b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:03:19.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.733+0000 7f3027fcd700 1 -- 172.21.15.32:0/4285172792 shutdown_connections 2024-05-12T04:03:19.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.734+0000 7f3027fcd700 1 -- 172.21.15.32:0/4285172792 wait complete. 2024-05-12T04:03:19.746 INFO:teuthology.orchestra.run.smithi032.stdout:true 2024-05-12T04:03:19.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.873+0000 7f4ca828b700 1 -- 172.21.15.32:0/2227143389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ca00fbce0 msgr2=0x7f4ca00fc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:03:19.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.873+0000 7f4ca828b700 1 --2- 172.21.15.32:0/2227143389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ca00fbce0 0x7f4ca00fc0f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c004880 tx=0x7f4c9c009e30 comp rx=0 tx=0).stop 2024-05-12T04:03:19.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.873+0000 7f4ca828b700 1 -- 172.21.15.32:0/2227143389 shutdown_connections 2024-05-12T04:03:19.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.873+0000 7f4ca828b700 1 --2- 172.21.15.32:0/2227143389 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ca00fcf90 0x7f4ca00fd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:19.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.873+0000 7f4ca828b700 1 --2- 172.21.15.32:0/2227143389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ca00fbce0 0x7f4ca00fc0f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:19.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.873+0000 7f4ca828b700 1 -- 172.21.15.32:0/2227143389 >> 172.21.15.32:0/2227143389 conn(0x7f4ca00f7440 msgr2=0x7f4ca00f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:03:19.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.874+0000 7f4ca828b700 1 -- 172.21.15.32:0/2227143389 shutdown_connections 2024-05-12T04:03:19.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.874+0000 7f4ca828b700 1 -- 172.21.15.32:0/2227143389 wait complete. 2024-05-12T04:03:19.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.875+0000 7f4ca828b700 1 Processor -- start 2024-05-12T04:03:19.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.875+0000 7f4ca828b700 1 -- start start 2024-05-12T04:03:19.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.875+0000 7f4ca828b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ca00fbce0 0x7f4ca0113db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:03:19.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.875+0000 7f4ca828b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ca00fcf90 0x7f4ca01142f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:03:19.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.875+0000 7f4ca828b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca01148f0 con 0x7f4ca00fbce0 2024-05-12T04:03:19.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.875+0000 7f4ca828b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca0114a60 con 0x7f4ca00fcf90 2024-05-12T04:03:19.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.875+0000 7f4ca6027700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ca00fbce0 0x7f4ca0113db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:03:19.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.875+0000 7f4ca6027700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ca00fbce0 0x7f4ca0113db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57140/0 (socket says 172.21.15.32:57140) 2024-05-12T04:03:19.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.875+0000 7f4ca6027700 1 -- 172.21.15.32:0/3716533920 learned_addr learned my addr 172.21.15.32:0/3716533920 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:03:19.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.876+0000 7f4ca6027700 1 -- 172.21.15.32:0/3716533920 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ca00fcf90 msgr2=0x7f4ca01142f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-12T04:03:19.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.876+0000 7f4ca6027700 1 --2- 172.21.15.32:0/3716533920 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ca00fcf90 0x7f4ca01142f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:19.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.876+0000 7f4ca6027700 1 -- 172.21.15.32:0/3716533920 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c90005040 con 0x7f4ca00fbce0 2024-05-12T04:03:19.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.876+0000 7f4ca6027700 1 --2- 172.21.15.32:0/3716533920 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ca00fbce0 0x7f4ca0113db0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c0137a0 tx=0x7f4c9c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:03:19.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.876+0000 7f4c96ffd700 1 -- 172.21.15.32:0/3716533920 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c9c00e030 con 0x7f4ca00fbce0 2024-05-12T04:03:19.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.876+0000 7f4c96ffd700 1 -- 172.21.15.32:0/3716533920 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4c9c01c400 con 0x7f4ca00fbce0 2024-05-12T04:03:19.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.876+0000 7f4ca828b700 1 -- 172.21.15.32:0/3716533920 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c9c005040 con 0x7f4ca00fbce0 2024-05-12T04:03:19.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.876+0000 7f4c96ffd700 1 -- 172.21.15.32:0/3716533920 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c9c012600 con 0x7f4ca00fbce0 2024-05-12T04:03:19.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.876+0000 7f4ca828b700 1 -- 172.21.15.32:0/3716533920 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ca0119880 con 0x7f4ca00fbce0 2024-05-12T04:03:19.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.878+0000 7f4ca828b700 1 -- 172.21.15.32:0/3716533920 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ca004c710 con 0x7f4ca00fbce0 2024-05-12T04:03:19.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.879+0000 7f4c96ffd700 1 -- 172.21.15.32:0/3716533920 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 90896+0+0 (secure 0 0 0) 0x7f4c9c01c570 con 0x7f4ca00fbce0 2024-05-12T04:03:19.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.879+0000 7f4c96ffd700 1 --2- 172.21.15.32:0/3716533920 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f4c8c06ccb0 0x7f4c8c06f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:03:19.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.879+0000 7f4c96ffd700 1 -- 172.21.15.32:0/3716533920 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f4c9c08dbb0 con 0x7f4ca00fbce0 2024-05-12T04:03:19.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.880+0000 7f4ca5826700 1 --2- 172.21.15.32:0/3716533920 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f4c8c06ccb0 0x7f4c8c06f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:03:19.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.881+0000 7f4ca5826700 1 --2- 172.21.15.32:0/3716533920 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f4c8c06ccb0 0x7f4c8c06f160 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4c90009fd0 tx=0x7f4c90009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:03:19.886 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:19.885+0000 7f4c96ffd700 1 -- 172.21.15.32:0/3716533920 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f4c9c094080 con 0x7f4ca00fbce0 2024-05-12T04:03:20.039 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:19 smithi032 ceph-mon[120392]: mgrmap e38: smithi032.ukatkq(active, since 3s), standbys: smithi033.pcvxwy 2024-05-12T04:03:20.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:19 smithi033 ceph-mon[49525]: mgrmap e38: smithi032.ukatkq(active, since 3s), standbys: smithi033.pcvxwy 2024-05-12T04:03:20.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.238+0000 7f4ca828b700 1 -- 172.21.15.32:0/3716533920 --> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4ca0109c90 con 0x7f4c8c06ccb0 2024-05-12T04:03:20.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.239+0000 7f4c96ffd700 1 -- 172.21.15.32:0/3716533920 <== mgr.24593 v2:172.21.15.32:6800/2957111591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f4ca0109c90 con 0x7f4c8c06ccb0 2024-05-12T04:03:20.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.242+0000 7f4ca828b700 1 -- 172.21.15.32:0/3716533920 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f4c8c06ccb0 msgr2=0x7f4c8c06f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:03:20.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.242+0000 7f4ca828b700 1 --2- 172.21.15.32:0/3716533920 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f4c8c06ccb0 0x7f4c8c06f160 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4c90009fd0 tx=0x7f4c90009e30 comp rx=0 tx=0).stop 2024-05-12T04:03:20.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.242+0000 7f4ca828b700 1 -- 172.21.15.32:0/3716533920 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ca00fbce0 msgr2=0x7f4ca0113db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:03:20.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.242+0000 7f4ca828b700 1 --2- 172.21.15.32:0/3716533920 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ca00fbce0 0x7f4ca0113db0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c0137a0 tx=0x7f4c9c009e30 comp rx=0 tx=0).stop 2024-05-12T04:03:20.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.244+0000 7f4ca828b700 1 -- 172.21.15.32:0/3716533920 shutdown_connections 2024-05-12T04:03:20.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.244+0000 7f4ca828b700 1 --2- 172.21.15.32:0/3716533920 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f4c8c06ccb0 0x7f4c8c06f160 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:20.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.244+0000 7f4ca828b700 1 --2- 172.21.15.32:0/3716533920 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ca00fbce0 0x7f4ca0113db0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:20.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.244+0000 7f4ca828b700 1 --2- 172.21.15.32:0/3716533920 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ca00fcf90 0x7f4ca01142f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:20.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.244+0000 7f4ca828b700 1 -- 172.21.15.32:0/3716533920 >> 172.21.15.32:0/3716533920 conn(0x7f4ca00f7440 msgr2=0x7f4ca00f8ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:03:20.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.244+0000 7f4ca828b700 1 -- 172.21.15.32:0/3716533920 shutdown_connections 2024-05-12T04:03:20.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.244+0000 7f4ca828b700 1 -- 172.21.15.32:0/3716533920 wait complete. 2024-05-12T04:03:20.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.385+0000 7ff8d1751700 1 -- 172.21.15.32:0/796391681 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8cc0fcf60 msgr2=0x7ff8cc0fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:03:20.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.385+0000 7ff8d1751700 1 --2- 172.21.15.32:0/796391681 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8cc0fcf60 0x7ff8cc0fd3d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff8c0004940 tx=0x7ff8c0009e30 comp rx=0 tx=0).stop 2024-05-12T04:03:20.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.386+0000 7ff8d1751700 1 -- 172.21.15.32:0/796391681 shutdown_connections 2024-05-12T04:03:20.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.386+0000 7ff8d1751700 1 --2- 172.21.15.32:0/796391681 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8cc0fcf60 0x7ff8cc0fd3d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:20.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.386+0000 7ff8d1751700 1 --2- 172.21.15.32:0/796391681 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8cc0fbcb0 0x7ff8cc0fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:20.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.386+0000 7ff8d1751700 1 -- 172.21.15.32:0/796391681 >> 172.21.15.32:0/796391681 conn(0x7ff8cc0f7430 msgr2=0x7ff8cc0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:03:20.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.386+0000 7ff8d1751700 1 -- 172.21.15.32:0/796391681 shutdown_connections 2024-05-12T04:03:20.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.386+0000 7ff8d1751700 1 -- 172.21.15.32:0/796391681 wait complete. 2024-05-12T04:03:20.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.386+0000 7ff8d1751700 1 Processor -- start 2024-05-12T04:03:20.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.386+0000 7ff8d1751700 1 -- start start 2024-05-12T04:03:20.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.387+0000 7ff8d1751700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8cc0fbcb0 0x7ff8cc10e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:03:20.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.387+0000 7ff8d1751700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8cc0fcf60 0x7ff8cc110910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:03:20.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.387+0000 7ff8d1751700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8cc066390 con 0x7ff8cc0fbcb0 2024-05-12T04:03:20.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.387+0000 7ff8d1751700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8cc0664d0 con 0x7ff8cc0fcf60 2024-05-12T04:03:20.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.387+0000 7ff8caffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8cc0fbcb0 0x7ff8cc10e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:03:20.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.387+0000 7ff8ca7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8cc0fcf60 0x7ff8cc110910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:03:20.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.387+0000 7ff8caffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8cc0fbcb0 0x7ff8cc10e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57156/0 (socket says 172.21.15.32:57156) 2024-05-12T04:03:20.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.387+0000 7ff8caffd700 1 -- 172.21.15.32:0/390449441 learned_addr learned my addr 172.21.15.32:0/390449441 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:03:20.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.387+0000 7ff8caffd700 1 -- 172.21.15.32:0/390449441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8cc0fcf60 msgr2=0x7ff8cc110910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:03:20.389 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.387+0000 7ff8caffd700 1 --2- 172.21.15.32:0/390449441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8cc0fcf60 0x7ff8cc110910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:20.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.387+0000 7ff8caffd700 1 -- 172.21.15.32:0/390449441 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c0005040 con 0x7ff8cc0fbcb0 2024-05-12T04:03:20.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.387+0000 7ff8caffd700 1 --2- 172.21.15.32:0/390449441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8cc0fbcb0 0x7ff8cc10e3c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc00c8c0 tx=0x7ff8bc0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:03:20.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.387+0000 7ff8b37fe700 1 -- 172.21.15.32:0/390449441 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8bc00a760 con 0x7ff8cc0fbcb0 2024-05-12T04:03:20.390 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.387+0000 7ff8d1751700 1 -- 172.21.15.32:0/390449441 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8cc110f10 con 0x7ff8cc0fbcb0 2024-05-12T04:03:20.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.388+0000 7ff8b37fe700 1 -- 172.21.15.32:0/390449441 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff8bc00ad70 con 0x7ff8cc0fbcb0 2024-05-12T04:03:20.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.388+0000 7ff8b37fe700 1 -- 172.21.15.32:0/390449441 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8bc0113a0 con 0x7ff8cc0fbcb0 2024-05-12T04:03:20.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.388+0000 7ff8d1751700 1 -- 172.21.15.32:0/390449441 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8cc1114d0 con 0x7ff8cc0fbcb0 2024-05-12T04:03:20.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.389+0000 7ff8d1751700 1 -- 172.21.15.32:0/390449441 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8cc05f5c0 con 0x7ff8cc0fbcb0 2024-05-12T04:03:20.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.389+0000 7ff8b37fe700 1 -- 172.21.15.32:0/390449441 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 90896+0+0 (secure 0 0 0) 0x7ff8bc012400 con 0x7ff8cc0fbcb0 2024-05-12T04:03:20.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.390+0000 7ff8b37fe700 1 --2- 172.21.15.32:0/390449441 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7ff8b406ccc0 0x7ff8b406f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:03:20.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.390+0000 7ff8b37fe700 1 -- 172.21.15.32:0/390449441 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7ff8bc08c0c0 con 0x7ff8cc0fbcb0 2024-05-12T04:03:20.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.390+0000 7ff8ca7fc700 1 --2- 172.21.15.32:0/390449441 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7ff8b406ccc0 0x7ff8b406f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:03:20.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.390+0000 7ff8ca7fc700 1 --2- 172.21.15.32:0/390449441 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7ff8b406ccc0 0x7ff8b406f170 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff8c000b3b0 tx=0x7ff8c0009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:03:20.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.395+0000 7ff8b37fe700 1 -- 172.21.15.32:0/390449441 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7ff8bc0560e0 con 0x7ff8cc0fbcb0 2024-05-12T04:03:20.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.705+0000 7ff8d1751700 1 -- 172.21.15.32:0/390449441 --> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff8cc1011a0 con 0x7ff8b406ccc0 2024-05-12T04:03:20.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.709+0000 7ff8b37fe700 1 -- 172.21.15.32:0/390449441 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 39) v1 ==== 90952+0+0 (secure 0 0 0) 0x7ff8bc0564b0 con 0x7ff8cc0fbcb0 2024-05-12T04:03:20.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.719+0000 7ff8b37fe700 1 -- 172.21.15.32:0/390449441 <== mgr.24593 v2:172.21.15.32:6800/2957111591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7ff8cc1011a0 con 0x7ff8b406ccc0 2024-05-12T04:03:20.720 INFO:teuthology.orchestra.run.smithi032.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-12T04:03:20.720 INFO:teuthology.orchestra.run.smithi032.stdout:alertmanager.smithi032 smithi032 *:9093,9094 running (2m) 27s ago 11m 17.8M - 0.25.0 c8568f914cd2 a37fbab7af4c 2024-05-12T04:03:20.720 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi032 smithi032 running (11m) 27s ago 11m 6656k - 17.2.7-939-gc190bdee 284803f7d9cf a478874e9dfa 2024-05-12T04:03:20.720 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi033 smithi033 running (10m) 0s ago 10m 6651k - 17.2.7-939-gc190bdee 284803f7d9cf 34c62d004c7e 2024-05-12T04:03:20.720 INFO:teuthology.orchestra.run.smithi032.stdout:grafana.smithi032 smithi032 *:3000 running (111s) 27s ago 11m 79.5M - 9.4.7 954c08fa6188 83055e58d23c 2024-05-12T04:03:20.720 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.ialtqb smithi032 running (5m) 27s ago 5m 12.6M - 17.2.7-939-gc190bdee 284803f7d9cf 0683abf00e3e 2024-05-12T04:03:20.721 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.qpsawh smithi032 running (5m) 27s ago 5m 236M - 17.2.7-939-gc190bdee 284803f7d9cf 95a7db5c4fcd 2024-05-12T04:03:20.721 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi033.hglwml smithi033 running (5m) 0s ago 5m 12.9M - 17.2.7-939-gc190bdee 284803f7d9cf c97902b05025 2024-05-12T04:03:20.721 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi033.jgbjhk smithi033 running (5m) 0s ago 5m 163M - 17.2.7-939-gc190bdee 284803f7d9cf af97052cb97a 2024-05-12T04:03:20.721 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi032.ukatkq smithi032 *:8443,9283,8765 running (83s) 27s ago 12m 492M - 18.2.2-1272-g58249552 ddcb0a9d6fde 0d88ab9a5292 2024-05-12T04:03:20.721 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi033.pcvxwy smithi033 *:8443,9283,8765 running (41s) 0s ago 10m 446M - 18.2.2-1272-g58249552 ddcb0a9d6fde d42dcb6bd307 2024-05-12T04:03:20.721 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi032 smithi032 running (12m) 27s ago 13m 52.5M 2048M 17.2.7-939-gc190bdee 284803f7d9cf d7fea430c6e2 2024-05-12T04:03:20.721 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi033 smithi033 running (10m) 0s ago 10m 48.3M 2048M 17.2.7-939-gc190bdee 284803f7d9cf 707654d85748 2024-05-12T04:03:20.721 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi032 smithi032 *:9100 running (2m) 27s ago 11m 9512k - 1.5.0 0da6a335fe13 cb75a7c0c829 2024-05-12T04:03:20.721 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi033 smithi033 *:9100 running (89s) 0s ago 10m 17.1M - 1.5.0 0da6a335fe13 7888c36c659f 2024-05-12T04:03:20.721 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0 smithi032 running (9m) 27s ago 9m 178M 1570M 17.2.7-939-gc190bdee 284803f7d9cf 75c78a47f8e5 2024-05-12T04:03:20.721 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1 smithi032 running (9m) 27s ago 9m 196M 1570M 17.2.7-939-gc190bdee 284803f7d9cf 87cfd247d8d6 2024-05-12T04:03:20.721 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2 smithi032 running (8m) 27s ago 8m 160M 1570M 17.2.7-939-gc190bdee 284803f7d9cf 7aef7affb8ba 2024-05-12T04:03:20.721 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3 smithi033 running (8m) 0s ago 8m 238M 2594M 17.2.7-939-gc190bdee 284803f7d9cf 3a92b5e3348e 2024-05-12T04:03:20.721 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4 smithi033 running (7m) 0s ago 7m 209M 2594M 17.2.7-939-gc190bdee 284803f7d9cf 0896268b4591 2024-05-12T04:03:20.721 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5 smithi033 running (7m) 0s ago 7m 193M 2594M 17.2.7-939-gc190bdee 284803f7d9cf 57e65a64c821 2024-05-12T04:03:20.722 INFO:teuthology.orchestra.run.smithi032.stdout:prometheus.smithi032 smithi032 *:9095 running (49s) 27s ago 10m 39.7M - 2.43.0 a07b618ecd1d 46eafb25e176 2024-05-12T04:03:20.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.721+0000 7ff8d1751700 1 -- 172.21.15.32:0/390449441 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7ff8b406ccc0 msgr2=0x7ff8b406f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:03:20.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.721+0000 7ff8d1751700 1 --2- 172.21.15.32:0/390449441 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7ff8b406ccc0 0x7ff8b406f170 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff8c000b3b0 tx=0x7ff8c0009e30 comp rx=0 tx=0).stop 2024-05-12T04:03:20.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.721+0000 7ff8d1751700 1 -- 172.21.15.32:0/390449441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8cc0fbcb0 msgr2=0x7ff8cc10e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:03:20.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.721+0000 7ff8d1751700 1 --2- 172.21.15.32:0/390449441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8cc0fbcb0 0x7ff8cc10e3c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc00c8c0 tx=0x7ff8bc0044c0 comp rx=0 tx=0).stop 2024-05-12T04:03:20.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.721+0000 7ff8d1751700 1 -- 172.21.15.32:0/390449441 shutdown_connections 2024-05-12T04:03:20.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.721+0000 7ff8d1751700 1 --2- 172.21.15.32:0/390449441 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7ff8b406ccc0 0x7ff8b406f170 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:20.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.721+0000 7ff8d1751700 1 --2- 172.21.15.32:0/390449441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8cc0fbcb0 0x7ff8cc10e3c0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:20.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.721+0000 7ff8d1751700 1 --2- 172.21.15.32:0/390449441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8cc0fcf60 0x7ff8cc110910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:20.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.721+0000 7ff8d1751700 1 -- 172.21.15.32:0/390449441 >> 172.21.15.32:0/390449441 conn(0x7ff8cc0f7430 msgr2=0x7ff8cc0f8f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:03:20.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.721+0000 7ff8d1751700 1 -- 172.21.15.32:0/390449441 shutdown_connections 2024-05-12T04:03:20.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.721+0000 7ff8d1751700 1 -- 172.21.15.32:0/390449441 wait complete. 2024-05-12T04:03:20.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.870+0000 7fada3b56700 1 -- 172.21.15.32:0/1241761053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad9c0fcf90 msgr2=0x7fad9c0fd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:03:20.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.870+0000 7fada3b56700 1 --2- 172.21.15.32:0/1241761053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad9c0fcf90 0x7fad9c0fd400 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fad98004940 tx=0x7fad98009e30 comp rx=0 tx=0).stop 2024-05-12T04:03:20.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.871+0000 7fada3b56700 1 -- 172.21.15.32:0/1241761053 shutdown_connections 2024-05-12T04:03:20.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.871+0000 7fada3b56700 1 --2- 172.21.15.32:0/1241761053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad9c0fcf90 0x7fad9c0fd400 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:20.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.871+0000 7fada3b56700 1 --2- 172.21.15.32:0/1241761053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad9c0fbce0 0x7fad9c0fc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:20.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.871+0000 7fada3b56700 1 -- 172.21.15.32:0/1241761053 >> 172.21.15.32:0/1241761053 conn(0x7fad9c0f7440 msgr2=0x7fad9c0f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:03:20.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.871+0000 7fada3b56700 1 -- 172.21.15.32:0/1241761053 shutdown_connections 2024-05-12T04:03:20.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.871+0000 7fada3b56700 1 -- 172.21.15.32:0/1241761053 wait complete. 2024-05-12T04:03:20.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.872+0000 7fada3b56700 1 Processor -- start 2024-05-12T04:03:20.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.872+0000 7fada3b56700 1 -- start start 2024-05-12T04:03:20.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.872+0000 7fada3b56700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad9c0fbce0 0x7fad9c10e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:03:20.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.872+0000 7fada3b56700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad9c0fcf90 0x7fad9c1108f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:03:20.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.872+0000 7fada3b56700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad9c066be0 con 0x7fad9c0fbce0 2024-05-12T04:03:20.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.872+0000 7fada3b56700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad9c066d20 con 0x7fad9c0fcf90 2024-05-12T04:03:20.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.872+0000 7fada10f1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad9c0fcf90 0x7fad9c1108f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:03:20.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.872+0000 7fada10f1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad9c0fcf90 0x7fad9c1108f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.32:48262/0 (socket says 172.21.15.32:48262) 2024-05-12T04:03:20.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.872+0000 7fada10f1700 1 -- 172.21.15.32:0/4263303757 learned_addr learned my addr 172.21.15.32:0/4263303757 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:03:20.874 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.872+0000 7fada18f2700 1 --2- 172.21.15.32:0/4263303757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad9c0fbce0 0x7fad9c10e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:03:20.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.873+0000 7fada10f1700 1 -- 172.21.15.32:0/4263303757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad9c0fbce0 msgr2=0x7fad9c10e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:03:20.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.873+0000 7fada10f1700 1 --2- 172.21.15.32:0/4263303757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad9c0fbce0 0x7fad9c10e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:20.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.873+0000 7fada10f1700 1 -- 172.21.15.32:0/4263303757 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad98005040 con 0x7fad9c0fcf90 2024-05-12T04:03:20.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.873+0000 7fada18f2700 1 --2- 172.21.15.32:0/4263303757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad9c0fbce0 0x7fad9c10e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-12T04:03:20.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.873+0000 7fada10f1700 1 --2- 172.21.15.32:0/4263303757 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad9c0fcf90 0x7fad9c1108f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fad980137b0 tx=0x7fad98009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:03:20.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.873+0000 7fad927fc700 1 -- 172.21.15.32:0/4263303757 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad9800e070 con 0x7fad9c0fcf90 2024-05-12T04:03:20.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.873+0000 7fada3b56700 1 -- 172.21.15.32:0/4263303757 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad9c110e90 con 0x7fad9c0fcf90 2024-05-12T04:03:20.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.873+0000 7fad927fc700 1 -- 172.21.15.32:0/4263303757 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fad980093f0 con 0x7fad9c0fcf90 2024-05-12T04:03:20.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.873+0000 7fad927fc700 1 -- 172.21.15.32:0/4263303757 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad9800b410 con 0x7fad9c0fcf90 2024-05-12T04:03:20.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.873+0000 7fada3b56700 1 -- 172.21.15.32:0/4263303757 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad9c1113f0 con 0x7fad9c0fcf90 2024-05-12T04:03:20.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.874+0000 7fad87fff700 1 -- 172.21.15.32:0/4263303757 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad7c0052b0 con 0x7fad9c0fcf90 2024-05-12T04:03:20.877 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.876+0000 7fad927fc700 1 -- 172.21.15.32:0/4263303757 <== mon.1 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 39) v1 ==== 90952+0+0 (secure 0 0 0) 0x7fad98013c40 con 0x7fad9c0fcf90 2024-05-12T04:03:20.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.876+0000 7fad927fc700 1 --2- 172.21.15.32:0/4263303757 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fad8806ce10 0x7fad8806f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:03:20.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.876+0000 7fad927fc700 1 -- 172.21.15.32:0/4263303757 <== mon.1 v2:172.21.15.33:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fad9808e4e0 con 0x7fad9c0fcf90 2024-05-12T04:03:20.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.876+0000 7fada18f2700 1 --2- 172.21.15.32:0/4263303757 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fad8806ce10 0x7fad8806f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:03:20.878 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.877+0000 7fada18f2700 1 --2- 172.21.15.32:0/4263303757 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fad8806ce10 0x7fad8806f2c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fad9c066860 tx=0x7fad8c00b040 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:03:20.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:20.880+0000 7fad927fc700 1 -- 172.21.15.32:0/4263303757 <== mon.1 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fad98058450 con 0x7fad9c0fcf90 2024-05-12T04:03:21.043 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.1... 2024-05-12T04:03:21.044 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-05-12T04:03:21.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.309+0000 7fad87fff700 1 -- 172.21.15.32:0/4263303757 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fad7c005830 con 0x7fad9c0fcf90 2024-05-12T04:03:21.312 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-05-12T04:03:21.312 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": { 2024-05-12T04:03:21.312 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 1, 2024-05-12T04:03:21.312 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 1 2024-05-12T04:03:21.312 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:03:21.312 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": { 2024-05-12T04:03:21.312 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 2 2024-05-12T04:03:21.312 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:03:21.312 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": { 2024-05-12T04:03:21.312 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 6 2024-05-12T04:03:21.312 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:03:21.313 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": { 2024-05-12T04:03:21.313 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 4 2024-05-12T04:03:21.313 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:03:21.313 INFO:teuthology.orchestra.run.smithi032.stdout: "overall": { 2024-05-12T04:03:21.313 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 11, 2024-05-12T04:03:21.313 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 3 2024-05-12T04:03:21.313 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-05-12T04:03:21.313 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-05-12T04:03:21.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.310+0000 7fad927fc700 1 -- 172.21.15.32:0/4263303757 <== mon.1 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+849 (secure 0 0 0) 0x7fad98060e60 con 0x7fad9c0fcf90 2024-05-12T04:03:21.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.313+0000 7fad87fff700 1 -- 172.21.15.32:0/4263303757 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fad8806ce10 msgr2=0x7fad8806f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:03:21.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.313+0000 7fad87fff700 1 --2- 172.21.15.32:0/4263303757 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fad8806ce10 0x7fad8806f2c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fad9c066860 tx=0x7fad8c00b040 comp rx=0 tx=0).stop 2024-05-12T04:03:21.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.313+0000 7fad87fff700 1 -- 172.21.15.32:0/4263303757 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad9c0fcf90 msgr2=0x7fad9c1108f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:03:21.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.313+0000 7fad87fff700 1 --2- 172.21.15.32:0/4263303757 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad9c0fcf90 0x7fad9c1108f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fad980137b0 tx=0x7fad98009e30 comp rx=0 tx=0).stop 2024-05-12T04:03:21.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.313+0000 7fad87fff700 1 -- 172.21.15.32:0/4263303757 shutdown_connections 2024-05-12T04:03:21.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.313+0000 7fad87fff700 1 --2- 172.21.15.32:0/4263303757 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fad8806ce10 0x7fad8806f2c0 secure :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fad9c066860 tx=0x7fad8c00b040 comp rx=0 tx=0).stop 2024-05-12T04:03:21.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.313+0000 7fad87fff700 1 --2- 172.21.15.32:0/4263303757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad9c0fbce0 0x7fad9c10e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:21.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.313+0000 7fad87fff700 1 --2- 172.21.15.32:0/4263303757 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad9c0fcf90 0x7fad9c1108f0 secure :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fad980137b0 tx=0x7fad98009e30 comp rx=0 tx=0).stop 2024-05-12T04:03:21.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.313+0000 7fad87fff700 1 -- 172.21.15.32:0/4263303757 >> 172.21.15.32:0/4263303757 conn(0x7fad9c0f7440 msgr2=0x7fad9c0f8b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:03:21.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.313+0000 7fad87fff700 1 -- 172.21.15.32:0/4263303757 shutdown_connections 2024-05-12T04:03:21.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.313+0000 7fad87fff700 1 -- 172.21.15.32:0/4263303757 wait complete. 2024-05-12T04:03:21.355 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:21 smithi033 ceph-mon[49525]: pgmap v5: 65 pgs: 65 active+clean; 279 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-05-12T04:03:21.355 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:21 smithi033 ceph-mon[49525]: from='client.34128 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:03:21.355 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:21 smithi033 ceph-mon[49525]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:21.355 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:21 smithi033 ceph-mon[49525]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:21.355 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:21 smithi033 ceph-mon[49525]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:03:21.355 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:21 smithi033 ceph-mon[49525]: mgrmap e39: smithi032.ukatkq(active, since 5s), standbys: smithi033.pcvxwy 2024-05-12T04:03:21.356 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:21 smithi033 ceph-mon[49525]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:21.356 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:21 smithi033 ceph-mon[49525]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:21.357 DEBUG:teuthology.parallel:result is None 2024-05-12T04:03:21.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.438+0000 7f1b65809700 1 -- 172.21.15.32:0/2994134527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b60066260 msgr2=0x7f1b600ffa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:03:21.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.438+0000 7f1b65809700 1 --2- 172.21.15.32:0/2994134527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b60066260 0x7f1b600ffa00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1b50004940 tx=0x7f1b50009e30 comp rx=0 tx=0).stop 2024-05-12T04:03:21.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.438+0000 7f1b65809700 1 -- 172.21.15.32:0/2994134527 shutdown_connections 2024-05-12T04:03:21.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.438+0000 7f1b65809700 1 --2- 172.21.15.32:0/2994134527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b600fff40 0x7f1b60102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:21.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.438+0000 7f1b65809700 1 --2- 172.21.15.32:0/2994134527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b60066260 0x7f1b600ffa00 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:21.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.438+0000 7f1b65809700 1 -- 172.21.15.32:0/2994134527 >> 172.21.15.32:0/2994134527 conn(0x7f1b600f7430 msgr2=0x7f1b600f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:03:21.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.438+0000 7f1b65809700 1 -- 172.21.15.32:0/2994134527 shutdown_connections 2024-05-12T04:03:21.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.438+0000 7f1b65809700 1 -- 172.21.15.32:0/2994134527 wait complete. 2024-05-12T04:03:21.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.439+0000 7f1b65809700 1 Processor -- start 2024-05-12T04:03:21.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.439+0000 7f1b65809700 1 -- start start 2024-05-12T04:03:21.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.439+0000 7f1b65809700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b60066260 0x7f1b6010b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:03:21.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.439+0000 7f1b65809700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b600fff40 0x7f1b6010bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:03:21.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.439+0000 7f1b65809700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6010c350 con 0x7f1b60066260 2024-05-12T04:03:21.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.439+0000 7f1b65809700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6010cb10 con 0x7f1b600fff40 2024-05-12T04:03:21.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.440+0000 7f1b5effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b60066260 0x7f1b6010b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:03:21.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.440+0000 7f1b5effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b60066260 0x7f1b6010b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57194/0 (socket says 172.21.15.32:57194) 2024-05-12T04:03:21.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.440+0000 7f1b5effd700 1 -- 172.21.15.32:0/1460012200 learned_addr learned my addr 172.21.15.32:0/1460012200 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:03:21.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.440+0000 7f1b5e7fc700 1 --2- 172.21.15.32:0/1460012200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b600fff40 0x7f1b6010bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:03:21.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.440+0000 7f1b5effd700 1 -- 172.21.15.32:0/1460012200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b600fff40 msgr2=0x7f1b6010bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:03:21.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.440+0000 7f1b5effd700 1 --2- 172.21.15.32:0/1460012200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b600fff40 0x7f1b6010bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:21.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.440+0000 7f1b5effd700 1 -- 172.21.15.32:0/1460012200 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b50005040 con 0x7f1b60066260 2024-05-12T04:03:21.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.440+0000 7f1b5effd700 1 --2- 172.21.15.32:0/1460012200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b60066260 0x7f1b6010b730 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1b5000b3b0 tx=0x7f1b50009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:03:21.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.441+0000 7f1b477fe700 1 -- 172.21.15.32:0/1460012200 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b5000e040 con 0x7f1b60066260 2024-05-12T04:03:21.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.441+0000 7f1b477fe700 1 -- 172.21.15.32:0/1460012200 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1b50004290 con 0x7f1b60066260 2024-05-12T04:03:21.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.441+0000 7f1b477fe700 1 -- 172.21.15.32:0/1460012200 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b5001c3f0 con 0x7f1b60066260 2024-05-12T04:03:21.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.441+0000 7f1b65809700 1 -- 172.21.15.32:0/1460012200 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b6010cd90 con 0x7f1b60066260 2024-05-12T04:03:21.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.441+0000 7f1b65809700 1 -- 172.21.15.32:0/1460012200 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b6010d350 con 0x7f1b60066260 2024-05-12T04:03:21.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.443+0000 7f1b477fe700 1 -- 172.21.15.32:0/1460012200 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 39) v1 ==== 90952+0+0 (secure 0 0 0) 0x7f1b50013820 con 0x7f1b60066260 2024-05-12T04:03:21.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.443+0000 7f1b65809700 1 -- 172.21.15.32:0/1460012200 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b600f8f60 con 0x7f1b60066260 2024-05-12T04:03:21.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.444+0000 7f1b477fe700 1 --2- 172.21.15.32:0/1460012200 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f1b4806ce00 0x7f1b4806f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:03:21.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.444+0000 7f1b477fe700 1 -- 172.21.15.32:0/1460012200 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f1b5008d490 con 0x7f1b60066260 2024-05-12T04:03:21.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.444+0000 7f1b5e7fc700 1 --2- 172.21.15.32:0/1460012200 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f1b4806ce00 0x7f1b4806f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:03:21.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.445+0000 7f1b5e7fc700 1 --2- 172.21.15.32:0/1460012200 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f1b4806ce00 0x7f1b4806f2b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1b54009bf0 tx=0x7f1b540091a0 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:03:21.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.449+0000 7f1b477fe700 1 -- 172.21.15.32:0/1460012200 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f1b50057450 con 0x7f1b60066260 2024-05-12T04:03:21.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:21 smithi032 ceph-mon[120392]: pgmap v5: 65 pgs: 65 active+clean; 279 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-05-12T04:03:21.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:21 smithi032 ceph-mon[120392]: from='client.34128 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:03:21.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:21 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:21.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:21 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:21.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:21 smithi032 ceph-mon[120392]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:03:21.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:21 smithi032 ceph-mon[120392]: mgrmap e39: smithi032.ukatkq(active, since 5s), standbys: smithi033.pcvxwy 2024-05-12T04:03:21.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:21 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:21.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:21 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:21.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.816+0000 7f1b65809700 1 -- 172.21.15.32:0/1460012200 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f1b601058d0 con 0x7f1b60066260 2024-05-12T04:03:21.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.816+0000 7f1b477fe700 1 -- 172.21.15.32:0/1460012200 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1866 (secure 0 0 0) 0x7f1b5005aa70 con 0x7f1b60066260 2024-05-12T04:03:21.817 INFO:teuthology.orchestra.run.smithi032.stdout:e18 2024-05-12T04:03:21.817 INFO:teuthology.orchestra.run.smithi032.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-12T04:03:21.818 INFO:teuthology.orchestra.run.smithi032.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-12T04:03:21.818 INFO:teuthology.orchestra.run.smithi032.stdout:legacy client fscid: 1 2024-05-12T04:03:21.818 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:03:21.818 INFO:teuthology.orchestra.run.smithi032.stdout:Filesystem 'cephfs' (1) 2024-05-12T04:03:21.818 INFO:teuthology.orchestra.run.smithi032.stdout:fs_name cephfs 2024-05-12T04:03:21.818 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 18 2024-05-12T04:03:21.818 INFO:teuthology.orchestra.run.smithi032.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-12T04:03:21.818 INFO:teuthology.orchestra.run.smithi032.stdout:created 2024-05-12T03:57:48.904614+0000 2024-05-12T04:03:21.818 INFO:teuthology.orchestra.run.smithi032.stdout:modified 2024-05-12T03:58:06.821591+0000 2024-05-12T04:03:21.818 INFO:teuthology.orchestra.run.smithi032.stdout:tableserver 0 2024-05-12T04:03:21.818 INFO:teuthology.orchestra.run.smithi032.stdout:root 0 2024-05-12T04:03:21.818 INFO:teuthology.orchestra.run.smithi032.stdout:session_timeout 60 2024-05-12T04:03:21.818 INFO:teuthology.orchestra.run.smithi032.stdout:session_autoclose 300 2024-05-12T04:03:21.818 INFO:teuthology.orchestra.run.smithi032.stdout:max_file_size 1099511627776 2024-05-12T04:03:21.819 INFO:teuthology.orchestra.run.smithi032.stdout:max_xattr_size 65536 2024-05-12T04:03:21.819 INFO:teuthology.orchestra.run.smithi032.stdout:required_client_features {} 2024-05-12T04:03:21.819 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure 0 2024-05-12T04:03:21.819 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure_osd_epoch 46 2024-05-12T04:03:21.819 INFO:teuthology.orchestra.run.smithi032.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-12T04:03:21.819 INFO:teuthology.orchestra.run.smithi032.stdout:max_mds 1 2024-05-12T04:03:21.819 INFO:teuthology.orchestra.run.smithi032.stdout:in 0 2024-05-12T04:03:21.819 INFO:teuthology.orchestra.run.smithi032.stdout:up {0=14474} 2024-05-12T04:03:21.819 INFO:teuthology.orchestra.run.smithi032.stdout:failed 2024-05-12T04:03:21.819 INFO:teuthology.orchestra.run.smithi032.stdout:damaged 2024-05-12T04:03:21.819 INFO:teuthology.orchestra.run.smithi032.stdout:stopped 2024-05-12T04:03:21.819 INFO:teuthology.orchestra.run.smithi032.stdout:data_pools [3] 2024-05-12T04:03:21.819 INFO:teuthology.orchestra.run.smithi032.stdout:metadata_pool 2 2024-05-12T04:03:21.819 INFO:teuthology.orchestra.run.smithi032.stdout:inline_data enabled 2024-05-12T04:03:21.819 INFO:teuthology.orchestra.run.smithi032.stdout:balancer 2024-05-12T04:03:21.819 INFO:teuthology.orchestra.run.smithi032.stdout:bal_rank_mask -1 2024-05-12T04:03:21.820 INFO:teuthology.orchestra.run.smithi032.stdout:standby_count_wanted 1 2024-05-12T04:03:21.820 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.qpsawh{0:14474} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.32:6828/3100155296,v1:172.21.15.32:6829/3100155296] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:03:21.820 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi033.jgbjhk{0:24267} state up:standby-replay seq 3 join_fscid=1 addr [v2:172.21.15.33:6824/55417471,v1:172.21.15.33:6825/55417471] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:03:21.820 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:03:21.820 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:03:21.820 INFO:teuthology.orchestra.run.smithi032.stdout:Standby daemons: 2024-05-12T04:03:21.820 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:03:21.820 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi033.hglwml{-1:24283} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/433858238,v1:172.21.15.33:6827/433858238] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:03:21.820 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.ialtqb{-1:24287} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.32:6826/3703132329,v1:172.21.15.32:6827/3703132329] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:03:21.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.818+0000 7f1b65809700 1 -- 172.21.15.32:0/1460012200 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f1b4806ce00 msgr2=0x7f1b4806f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:03:21.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.818+0000 7f1b65809700 1 --2- 172.21.15.32:0/1460012200 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f1b4806ce00 0x7f1b4806f2b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1b54009bf0 tx=0x7f1b540091a0 comp rx=0 tx=0).stop 2024-05-12T04:03:21.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.818+0000 7f1b65809700 1 -- 172.21.15.32:0/1460012200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b60066260 msgr2=0x7f1b6010b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:03:21.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.818+0000 7f1b65809700 1 --2- 172.21.15.32:0/1460012200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b60066260 0x7f1b6010b730 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1b5000b3b0 tx=0x7f1b50009e30 comp rx=0 tx=0).stop 2024-05-12T04:03:21.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.819+0000 7f1b65809700 1 -- 172.21.15.32:0/1460012200 shutdown_connections 2024-05-12T04:03:21.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.819+0000 7f1b65809700 1 --2- 172.21.15.32:0/1460012200 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f1b4806ce00 0x7f1b4806f2b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:21.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.819+0000 7f1b65809700 1 --2- 172.21.15.32:0/1460012200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b60066260 0x7f1b6010b730 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:21.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.819+0000 7f1b65809700 1 --2- 172.21.15.32:0/1460012200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b600fff40 0x7f1b6010bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:21.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.819+0000 7f1b65809700 1 -- 172.21.15.32:0/1460012200 >> 172.21.15.32:0/1460012200 conn(0x7f1b600f7430 msgr2=0x7f1b600fbb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:03:21.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.819+0000 7f1b65809700 1 -- 172.21.15.32:0/1460012200 shutdown_connections 2024-05-12T04:03:21.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.819+0000 7f1b65809700 1 -- 172.21.15.32:0/1460012200 wait complete. 2024-05-12T04:03:21.822 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 18 2024-05-12T04:03:21.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.934+0000 7f7517384700 1 -- 172.21.15.32:0/2473154447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7510066af0 msgr2=0x7f75100ffa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:03:21.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.934+0000 7f7517384700 1 --2- 172.21.15.32:0/2473154447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7510066af0 0x7f75100ffa90 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f750c004940 tx=0x7f750c009e30 comp rx=0 tx=0).stop 2024-05-12T04:03:21.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.934+0000 7f7517384700 1 -- 172.21.15.32:0/2473154447 shutdown_connections 2024-05-12T04:03:21.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.934+0000 7f7517384700 1 --2- 172.21.15.32:0/2473154447 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75100fffd0 0x7f75101023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:21.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.934+0000 7f7517384700 1 --2- 172.21.15.32:0/2473154447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7510066af0 0x7f75100ffa90 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:21.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.934+0000 7f7517384700 1 -- 172.21.15.32:0/2473154447 >> 172.21.15.32:0/2473154447 conn(0x7f75100f74c0 msgr2=0x7f75100f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:03:21.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.934+0000 7f7517384700 1 -- 172.21.15.32:0/2473154447 shutdown_connections 2024-05-12T04:03:21.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.934+0000 7f7517384700 1 -- 172.21.15.32:0/2473154447 wait complete. 2024-05-12T04:03:21.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.935+0000 7f7517384700 1 Processor -- start 2024-05-12T04:03:21.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.935+0000 7f7517384700 1 -- start start 2024-05-12T04:03:21.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.935+0000 7f7517384700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7510066af0 0x7f751010da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:03:21.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.935+0000 7f7517384700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75100fffd0 0x7f751010df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:03:21.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.935+0000 7f7517384700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f751010e540 con 0x7f7510066af0 2024-05-12T04:03:21.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.935+0000 7f7517384700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f751010ed40 con 0x7f75100fffd0 2024-05-12T04:03:21.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.935+0000 7f7515120700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7510066af0 0x7f751010da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:03:21.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.935+0000 7f7515120700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7510066af0 0x7f751010da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55888/0 (socket says 172.21.15.32:55888) 2024-05-12T04:03:21.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.935+0000 7f7515120700 1 -- 172.21.15.32:0/922203349 learned_addr learned my addr 172.21.15.32:0/922203349 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:03:21.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.935+0000 7f751491f700 1 --2- 172.21.15.32:0/922203349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75100fffd0 0x7f751010df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:03:21.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.936+0000 7f751491f700 1 -- 172.21.15.32:0/922203349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7510066af0 msgr2=0x7f751010da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:03:21.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.936+0000 7f751491f700 1 --2- 172.21.15.32:0/922203349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7510066af0 0x7f751010da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:21.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.936+0000 7f751491f700 1 -- 172.21.15.32:0/922203349 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f750c005040 con 0x7f75100fffd0 2024-05-12T04:03:21.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.936+0000 7f7515120700 1 --2- 172.21.15.32:0/922203349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7510066af0 0x7f751010da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-12T04:03:21.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.936+0000 7f751491f700 1 --2- 172.21.15.32:0/922203349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75100fffd0 0x7f751010df40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7500002820 tx=0x7f7500004570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:03:21.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.936+0000 7f7505ffb700 1 -- 172.21.15.32:0/922203349 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7500007cc0 con 0x7f75100fffd0 2024-05-12T04:03:21.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.936+0000 7f7517384700 1 -- 172.21.15.32:0/922203349 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f751010f020 con 0x7f75100fffd0 2024-05-12T04:03:21.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.936+0000 7f7505ffb700 1 -- 172.21.15.32:0/922203349 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f750000f040 con 0x7f75100fffd0 2024-05-12T04:03:21.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.936+0000 7f7505ffb700 1 -- 172.21.15.32:0/922203349 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f75000137d0 con 0x7f75100fffd0 2024-05-12T04:03:21.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.936+0000 7f7517384700 1 -- 172.21.15.32:0/922203349 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f751010f560 con 0x7f75100fffd0 2024-05-12T04:03:21.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.938+0000 7f7505ffb700 1 -- 172.21.15.32:0/922203349 <== mon.1 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 39) v1 ==== 90952+0+0 (secure 0 0 0) 0x7f7500013a00 con 0x7f75100fffd0 2024-05-12T04:03:21.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.939+0000 7f7505ffb700 1 --2- 172.21.15.32:0/922203349 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f74fc06cd40 0x7f74fc06f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:03:21.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.939+0000 7f7505ffb700 1 -- 172.21.15.32:0/922203349 <== mon.1 v2:172.21.15.33:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f750008bf00 con 0x7f75100fffd0 2024-05-12T04:03:21.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.939+0000 7f7517384700 1 -- 172.21.15.32:0/922203349 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f751005f5c0 con 0x7f75100fffd0 2024-05-12T04:03:21.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.939+0000 7f7515120700 1 --2- 172.21.15.32:0/922203349 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f74fc06cd40 0x7f74fc06f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:03:21.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.939+0000 7f7515120700 1 --2- 172.21.15.32:0/922203349 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f74fc06cd40 0x7f74fc06f1f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f750c009fd0 tx=0x7f750c009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:03:21.945 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:21.944+0000 7f7505ffb700 1 -- 172.21.15.32:0/922203349 <== mon.1 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f7500059870 con 0x7f75100fffd0 2024-05-12T04:03:22.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:22 smithi033 ceph-mon[49525]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:03:22.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:22 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/4263303757' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:03:22.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:22 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/1460012200' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-12T04:03:22.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:22.260+0000 7f7517384700 1 -- 172.21.15.32:0/922203349 --> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f751010f8e0 con 0x7f74fc06cd40 2024-05-12T04:03:22.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:22.262+0000 7f7505ffb700 1 -- 172.21.15.32:0/922203349 <== mgr.24593 v2:172.21.15.32:6800/2957111591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f751010f8e0 con 0x7f74fc06cd40 2024-05-12T04:03:22.263 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-05-12T04:03:22.263 INFO:teuthology.orchestra.run.smithi032.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:19c9734256aa43a60e203feb242f50bf8b7f58666d520536af43cc4fee96173a", 2024-05-12T04:03:22.263 INFO:teuthology.orchestra.run.smithi032.stdout: "in_progress": true, 2024-05-12T04:03:22.263 INFO:teuthology.orchestra.run.smithi032.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-12T04:03:22.263 INFO:teuthology.orchestra.run.smithi032.stdout: "services_complete": [ 2024-05-12T04:03:22.263 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr" 2024-05-12T04:03:22.263 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-05-12T04:03:22.263 INFO:teuthology.orchestra.run.smithi032.stdout: "progress": "2/21 daemons upgraded", 2024-05-12T04:03:22.264 INFO:teuthology.orchestra.run.smithi032.stdout: "message": "", 2024-05-12T04:03:22.264 INFO:teuthology.orchestra.run.smithi032.stdout: "is_paused": false 2024-05-12T04:03:22.264 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-05-12T04:03:22.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:22.265+0000 7f7517384700 1 -- 172.21.15.32:0/922203349 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f74fc06cd40 msgr2=0x7f74fc06f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:03:22.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:22.265+0000 7f7517384700 1 --2- 172.21.15.32:0/922203349 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f74fc06cd40 0x7f74fc06f1f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f750c009fd0 tx=0x7f750c009e30 comp rx=0 tx=0).stop 2024-05-12T04:03:22.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:22.265+0000 7f7517384700 1 -- 172.21.15.32:0/922203349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75100fffd0 msgr2=0x7f751010df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:03:22.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:22.265+0000 7f7517384700 1 --2- 172.21.15.32:0/922203349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75100fffd0 0x7f751010df40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7500002820 tx=0x7f7500004570 comp rx=0 tx=0).stop 2024-05-12T04:03:22.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:22.265+0000 7f7517384700 1 -- 172.21.15.32:0/922203349 shutdown_connections 2024-05-12T04:03:22.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:22.265+0000 7f7517384700 1 --2- 172.21.15.32:0/922203349 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f74fc06cd40 0x7f74fc06f1f0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:22.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:22.265+0000 7f7517384700 1 --2- 172.21.15.32:0/922203349 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7510066af0 0x7f751010da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:22.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:22.265+0000 7f7517384700 1 --2- 172.21.15.32:0/922203349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75100fffd0 0x7f751010df40 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:22.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:22.265+0000 7f7517384700 1 -- 172.21.15.32:0/922203349 >> 172.21.15.32:0/922203349 conn(0x7f75100f74c0 msgr2=0x7f75100f7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:03:22.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:22.265+0000 7f7517384700 1 -- 172.21.15.32:0/922203349 shutdown_connections 2024-05-12T04:03:22.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:22.265+0000 7f7517384700 1 -- 172.21.15.32:0/922203349 wait complete. 2024-05-12T04:03:22.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:22 smithi032 ceph-mon[120392]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:03:22.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:22 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/4263303757' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:03:22.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:22 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/1460012200' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-12T04:03:22.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:22.378+0000 7f8dbba22700 1 -- 172.21.15.32:0/800453322 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8db40fcf60 msgr2=0x7f8db40fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:03:22.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:22.378+0000 7f8dbba22700 1 --2- 172.21.15.32:0/800453322 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8db40fcf60 0x7f8db40fd3d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8da4004940 tx=0x7f8da4009e30 comp rx=0 tx=0).stop 2024-05-12T04:03:22.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:22.379+0000 7f8dbba22700 1 -- 172.21.15.32:0/800453322 shutdown_connections 2024-05-12T04:03:22.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:22.379+0000 7f8dbba22700 1 --2- 172.21.15.32:0/800453322 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8db40fcf60 0x7f8db40fd3d0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:22.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:22.379+0000 7f8dbba22700 1 --2- 172.21.15.32:0/800453322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8db40fbcb0 0x7f8db40fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:22.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:22.379+0000 7f8dbba22700 1 -- 172.21.15.32:0/800453322 >> 172.21.15.32:0/800453322 conn(0x7f8db40f7430 msgr2=0x7f8db40f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:03:22.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:22.379+0000 7f8dbba22700 1 -- 172.21.15.32:0/800453322 shutdown_connections 2024-05-12T04:03:22.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:22.379+0000 7f8dbba22700 1 -- 172.21.15.32:0/800453322 wait complete. 2024-05-12T04:03:22.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:22.379+0000 7f8dbba22700 1 Processor -- start 2024-05-12T04:03:22.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:22.379+0000 7f8dbba22700 1 -- start start 2024-05-12T04:03:22.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:22.380+0000 7f8dbba22700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8db40fbcb0 0x7f8db4110580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:03:22.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:22.380+0000 7f8dbba22700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8db40fcf60 0x7f8db4112ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:03:22.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:22.380+0000 7f8dbba22700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db4066350 con 0x7f8db40fcf60 2024-05-12T04:03:22.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:22.380+0000 7f8dbba22700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db4066490 con 0x7f8db40fbcb0 2024-05-12T04:03:22.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:22.380+0000 7f8db8fbd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8db40fcf60 0x7f8db4112ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:03:22.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:22.380+0000 7f8db8fbd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8db40fcf60 0x7f8db4112ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55904/0 (socket says 172.21.15.32:55904) 2024-05-12T04:03:22.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:22.380+0000 7f8db8fbd700 1 -- 172.21.15.32:0/4161425700 learned_addr learned my addr 172.21.15.32:0/4161425700 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:03:22.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:22.380+0000 7f8db97be700 1 --2- 172.21.15.32:0/4161425700 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8db40fbcb0 0x7f8db4110580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:03:22.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:22.380+0000 7f8db8fbd700 1 -- 172.21.15.32:0/4161425700 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8db40fbcb0 msgr2=0x7f8db4110580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:03:22.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:22.380+0000 7f8db8fbd700 1 --2- 172.21.15.32:0/4161425700 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8db40fbcb0 0x7f8db4110580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:22.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:22.380+0000 7f8db8fbd700 1 -- 172.21.15.32:0/4161425700 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8da4005040 con 0x7f8db40fcf60 2024-05-12T04:03:22.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:22.380+0000 7f8db97be700 1 --2- 172.21.15.32:0/4161425700 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8db40fbcb0 0x7f8db4110580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-12T04:03:22.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:22.380+0000 7f8db8fbd700 1 --2- 172.21.15.32:0/4161425700 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8db40fcf60 0x7f8db4112ad0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8da4009380 tx=0x7f8da4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:03:22.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:22.381+0000 7f8da9ffb700 1 -- 172.21.15.32:0/4161425700 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8da400e040 con 0x7f8db40fcf60 2024-05-12T04:03:22.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:22.381+0000 7f8dbba22700 1 -- 172.21.15.32:0/4161425700 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8db4113070 con 0x7f8db40fcf60 2024-05-12T04:03:22.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:22.381+0000 7f8da9ffb700 1 -- 172.21.15.32:0/4161425700 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8da400bd20 con 0x7f8db40fcf60 2024-05-12T04:03:22.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:22.381+0000 7f8da9ffb700 1 -- 172.21.15.32:0/4161425700 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8da401c630 con 0x7f8db40fcf60 2024-05-12T04:03:22.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:22.381+0000 7f8dbba22700 1 -- 172.21.15.32:0/4161425700 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8db4113600 con 0x7f8db40fcf60 2024-05-12T04:03:22.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:22.382+0000 7f8dbba22700 1 -- 172.21.15.32:0/4161425700 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8db405f5c0 con 0x7f8db40fcf60 2024-05-12T04:03:22.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:22.383+0000 7f8da9ffb700 1 -- 172.21.15.32:0/4161425700 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 39) v1 ==== 90952+0+0 (secure 0 0 0) 0x7f8da4013420 con 0x7f8db40fcf60 2024-05-12T04:03:22.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:22.383+0000 7f8da9ffb700 1 --2- 172.21.15.32:0/4161425700 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f8da006cd40 0x7f8da006f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:03:22.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:22.383+0000 7f8da9ffb700 1 -- 172.21.15.32:0/4161425700 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f8da408cef0 con 0x7f8db40fcf60 2024-05-12T04:03:22.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:22.383+0000 7f8db97be700 1 --2- 172.21.15.32:0/4161425700 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f8da006cd40 0x7f8da006f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:03:22.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:22.384+0000 7f8db97be700 1 --2- 172.21.15.32:0/4161425700 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f8da006cd40 0x7f8da006f1f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8db0004fd0 tx=0x7f8db000b040 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:03:22.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:22.391+0000 7f8da9ffb700 1 -- 172.21.15.32:0/4161425700 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f8da4056d60 con 0x7f8db40fcf60 2024-05-12T04:03:22.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:22.787+0000 7f8dbba22700 1 -- 172.21.15.32:0/4161425700 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f8db404bed0 con 0x7f8db40fcf60 2024-05-12T04:03:22.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:22.787+0000 7f8da9ffb700 1 -- 172.21.15.32:0/4161425700 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f8da405a380 con 0x7f8db40fcf60 2024-05-12T04:03:22.789 INFO:teuthology.orchestra.run.smithi032.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-12T04:03:22.789 INFO:teuthology.orchestra.run.smithi032.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-12T04:03:22.789 INFO:teuthology.orchestra.run.smithi032.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-12T04:03:22.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:22.790+0000 7f8dbba22700 1 -- 172.21.15.32:0/4161425700 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f8da006cd40 msgr2=0x7f8da006f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:03:22.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:22.790+0000 7f8dbba22700 1 --2- 172.21.15.32:0/4161425700 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f8da006cd40 0x7f8da006f1f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8db0004fd0 tx=0x7f8db000b040 comp rx=0 tx=0).stop 2024-05-12T04:03:22.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:22.790+0000 7f8dbba22700 1 -- 172.21.15.32:0/4161425700 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8db40fcf60 msgr2=0x7f8db4112ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:03:22.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:22.790+0000 7f8dbba22700 1 --2- 172.21.15.32:0/4161425700 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8db40fcf60 0x7f8db4112ad0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8da4009380 tx=0x7f8da4009e30 comp rx=0 tx=0).stop 2024-05-12T04:03:22.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:22.790+0000 7f8dbba22700 1 -- 172.21.15.32:0/4161425700 shutdown_connections 2024-05-12T04:03:22.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:22.790+0000 7f8dbba22700 1 --2- 172.21.15.32:0/4161425700 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f8da006cd40 0x7f8da006f1f0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:22.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:22.790+0000 7f8dbba22700 1 --2- 172.21.15.32:0/4161425700 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8db40fbcb0 0x7f8db4110580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:22.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:22.790+0000 7f8dbba22700 1 --2- 172.21.15.32:0/4161425700 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8db40fcf60 0x7f8db4112ad0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:22.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:22.790+0000 7f8dbba22700 1 -- 172.21.15.32:0/4161425700 >> 172.21.15.32:0/4161425700 conn(0x7f8db40f7430 msgr2=0x7f8db40f8b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:03:22.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:22.790+0000 7f8dbba22700 1 -- 172.21.15.32:0/4161425700 shutdown_connections 2024-05-12T04:03:22.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:22.790+0000 7f8dbba22700 1 -- 172.21.15.32:0/4161425700 wait complete. 2024-05-12T04:03:23.256 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:23 smithi033 ceph-mon[49525]: pgmap v6: 65 pgs: 65 active+clean; 279 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-05-12T04:03:23.257 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:23 smithi033 ceph-mon[49525]: from='client.24631 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:03:23.257 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:23 smithi033 ceph-mon[49525]: from='client.? 172.21.15.32:0/4161425700' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-12T04:03:23.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:23 smithi032 ceph-mon[120392]: pgmap v6: 65 pgs: 65 active+clean; 279 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-05-12T04:03:23.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:23 smithi032 ceph-mon[120392]: from='client.24631 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:03:23.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:23 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/4161425700' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-12T04:03:25.392 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:25 smithi033 ceph-mon[49525]: pgmap v7: 65 pgs: 65 active+clean; 273 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 739 KiB/s rd, 826 KiB/s wr, 161 op/s 2024-05-12T04:03:25.392 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:25 smithi033 ceph-mon[49525]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:25.392 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:25 smithi033 ceph-mon[49525]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:25.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:25 smithi032 ceph-mon[120392]: pgmap v7: 65 pgs: 65 active+clean; 273 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 739 KiB/s rd, 826 KiB/s wr, 161 op/s 2024-05-12T04:03:25.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:25 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:25.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:25 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:26.051 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-05-12T04:03:26.052 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-12T04:03:26.398 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:26 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:26.398 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:26 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:26.398 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:26 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:26.398 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:26 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:26.398 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:26 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:03:26.398 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:26 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T04:03:26.399 DEBUG:teuthology.parallel:result is None 2024-05-12T04:03:26.400 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-05-12T04:03:26.440 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-05-12T04:03:26.440 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-05-12T04:03:26.479 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-05-12T04:03:26.480 DEBUG:teuthology.parallel:result is None 2024-05-12T04:03:26.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:26 smithi033 ceph-mon[49525]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:26.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:26 smithi033 ceph-mon[49525]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:26.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:26 smithi033 ceph-mon[49525]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:26.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:26 smithi033 ceph-mon[49525]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:26.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:26 smithi033 ceph-mon[49525]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:03:26.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:26 smithi033 ceph-mon[49525]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T04:03:27.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:27 smithi033 ceph-mon[49525]: pgmap v8: 65 pgs: 65 active+clean; 273 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 574 KiB/s rd, 642 KiB/s wr, 125 op/s 2024-05-12T04:03:27.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:27 smithi033 ceph-mon[49525]: Updating smithi032:/etc/ceph/ceph.conf 2024-05-12T04:03:27.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:27 smithi033 ceph-mon[49525]: Updating smithi033:/etc/ceph/ceph.conf 2024-05-12T04:03:27.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:27 smithi033 ceph-mon[49525]: Updating smithi033:/var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-12T04:03:27.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:27 smithi033 ceph-mon[49525]: Updating smithi032:/var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-12T04:03:27.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:27 smithi033 ceph-mon[49525]: Updating smithi033:/etc/ceph/ceph.client.admin.keyring 2024-05-12T04:03:27.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:27 smithi033 ceph-mon[49525]: Updating smithi032:/etc/ceph/ceph.client.admin.keyring 2024-05-12T04:03:27.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:27 smithi033 ceph-mon[49525]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:27.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:27 smithi033 ceph-mon[49525]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:27.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:27 smithi033 ceph-mon[49525]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:27.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:27 smithi033 ceph-mon[49525]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:27.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:27 smithi033 ceph-mon[49525]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:27.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:27 smithi033 ceph-mon[49525]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:03:27.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:27 smithi033 ceph-mon[49525]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:03:27.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:27 smithi033 ceph-mon[49525]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-05-12T04:03:27.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:27 smithi032 ceph-mon[120392]: pgmap v8: 65 pgs: 65 active+clean; 273 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 574 KiB/s rd, 642 KiB/s wr, 125 op/s 2024-05-12T04:03:27.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:27 smithi032 ceph-mon[120392]: Updating smithi032:/etc/ceph/ceph.conf 2024-05-12T04:03:27.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:27 smithi032 ceph-mon[120392]: Updating smithi033:/etc/ceph/ceph.conf 2024-05-12T04:03:27.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:27 smithi032 ceph-mon[120392]: Updating smithi033:/var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-12T04:03:27.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:27 smithi032 ceph-mon[120392]: Updating smithi032:/var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-12T04:03:27.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:27 smithi032 ceph-mon[120392]: Updating smithi033:/etc/ceph/ceph.client.admin.keyring 2024-05-12T04:03:27.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:27 smithi032 ceph-mon[120392]: Updating smithi032:/etc/ceph/ceph.client.admin.keyring 2024-05-12T04:03:27.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:27 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:27.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:27 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:27.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:27 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:27.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:27 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:27.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:27 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:27.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:27 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:03:27.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:27 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:03:27.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:27 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-05-12T04:03:28.660 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:28 smithi032 ceph-mon[120392]: Updating smithi033:/var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-12T04:03:28.660 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:28 smithi032 ceph-mon[120392]: Updating smithi032:/var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-12T04:03:28.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:28 smithi033 ceph-mon[49525]: Updating smithi033:/var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-12T04:03:28.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:28 smithi033 ceph-mon[49525]: Updating smithi032:/var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-12T04:03:29.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:29 smithi033 ceph-mon[49525]: pgmap v9: 65 pgs: 65 active+clean; 262 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 962 KiB/s rd, 960 KiB/s wr, 178 op/s 2024-05-12T04:03:29.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:29 smithi033 ceph-mon[49525]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:29.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:29 smithi033 ceph-mon[49525]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-12T04:03:29.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:29 smithi033 ceph-mon[49525]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-12T04:03:29.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:29 smithi033 ceph-mon[49525]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:03:29.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:29 smithi032 ceph-mon[120392]: pgmap v9: 65 pgs: 65 active+clean; 262 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 962 KiB/s rd, 960 KiB/s wr, 178 op/s 2024-05-12T04:03:29.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:29 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:29.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:29 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-12T04:03:29.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:29 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-12T04:03:29.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:29 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:03:30.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:30 smithi033 ceph-mon[49525]: Upgrade: Updating mon.smithi033 2024-05-12T04:03:30.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:30 smithi033 ceph-mon[49525]: Deploying daemon mon.smithi033 on smithi033 2024-05-12T04:03:30.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:30 smithi032 ceph-mon[120392]: Upgrade: Updating mon.smithi033 2024-05-12T04:03:30.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:30 smithi032 ceph-mon[120392]: Deploying daemon mon.smithi033 on smithi033 2024-05-12T04:03:31.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:30 smithi033 systemd[1]: Stopping Ceph mon.smithi033 for a3e178f8-1012-11ef-bc98-c7b262605968... 2024-05-12T04:03:31.630 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:31 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-mon-smithi033[49501]: 2024-05-12T04:03:31.219+0000 7f5133374700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi033 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-12T04:03:31.630 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:31 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-mon-smithi033[49501]: 2024-05-12T04:03:31.219+0000 7f5133374700 -1 mon.smithi033@1(peon) e2 *** Got Signal Terminated *** 2024-05-12T04:03:31.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:31 smithi033 bash[107114]: ceph-a3e178f8-1012-11ef-bc98-c7b262605968-mon-smithi033 2024-05-12T04:03:32.657 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 systemd[1]: ceph-a3e178f8-1012-11ef-bc98-c7b262605968@mon.smithi033.service: Succeeded. 2024-05-12T04:03:32.658 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 systemd[1]: Stopped Ceph mon.smithi033 for a3e178f8-1012-11ef-bc98-c7b262605968. 2024-05-12T04:03:32.658 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 systemd[1]: Starting Ceph mon.smithi033 for a3e178f8-1012-11ef-bc98-c7b262605968... 2024-05-12T04:03:33.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: set uid:gid to 167:167 (ceph:ceph) 2024-05-12T04:03:33.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable), process ceph-mon, pid 7 2024-05-12T04:03:33.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: pidfile_write: ignore empty --pid-file 2024-05-12T04:03:33.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: load: jerasure load: lrc 2024-05-12T04:03:33.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: RocksDB version: 7.9.2 2024-05-12T04:03:33.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Git sha 0 2024-05-12T04:03:33.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Compile date 2024-04-30 20:57:58 2024-05-12T04:03:33.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: DB SUMMARY 2024-05-12T04:03:33.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: DB Session ID: F7JQB6MJ1KUMDS2GYBBG 2024-05-12T04:03:33.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: CURRENT file: CURRENT 2024-05-12T04:03:33.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: IDENTITY file: IDENTITY 2024-05-12T04:03:33.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: MANIFEST file: MANIFEST-000009 size: 1090 Bytes 2024-05-12T04:03:33.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi033/store.db dir, Total Num: 1, files: 000027.sst 2024-05-12T04:03:33.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi033/store.db: 000025.log size: 3096117 ; 2024-05-12T04:03:33.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.error_if_exists: 0 2024-05-12T04:03:33.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.create_if_missing: 0 2024-05-12T04:03:33.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.paranoid_checks: 1 2024-05-12T04:03:33.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-12T04:03:33.164 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-12T04:03:33.164 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-12T04:03:33.164 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.env: 0x561daac8f820 2024-05-12T04:03:33.164 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.fs: PosixFileSystem 2024-05-12T04:03:33.164 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.info_log: 0x561dadd0f3c0 2024-05-12T04:03:33.164 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.max_file_opening_threads: 16 2024-05-12T04:03:33.164 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.statistics: (nil) 2024-05-12T04:03:33.164 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.use_fsync: 0 2024-05-12T04:03:33.164 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.max_log_file_size: 0 2024-05-12T04:03:33.164 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-12T04:03:33.164 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-12T04:03:33.164 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.keep_log_file_num: 1000 2024-05-12T04:03:33.164 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.recycle_log_file_num: 0 2024-05-12T04:03:33.164 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.allow_fallocate: 1 2024-05-12T04:03:33.164 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.allow_mmap_reads: 0 2024-05-12T04:03:33.164 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.allow_mmap_writes: 0 2024-05-12T04:03:33.164 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.use_direct_reads: 0 2024-05-12T04:03:33.165 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-12T04:03:33.165 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.create_missing_column_families: 0 2024-05-12T04:03:33.165 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.db_log_dir: 2024-05-12T04:03:33.165 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.wal_dir: 2024-05-12T04:03:33.165 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-12T04:03:33.165 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-12T04:03:33.165 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-12T04:03:33.165 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-12T04:03:33.165 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-12T04:03:33.165 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-12T04:03:33.165 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.advise_random_on_open: 1 2024-05-12T04:03:33.165 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.db_write_buffer_size: 0 2024-05-12T04:03:33.165 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.write_buffer_manager: 0x561dacf84140 2024-05-12T04:03:33.165 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-12T04:03:33.165 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-12T04:03:33.165 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-12T04:03:33.165 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.rate_limiter: (nil) 2024-05-12T04:03:33.166 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-12T04:03:33.166 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.wal_recovery_mode: 2 2024-05-12T04:03:33.166 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.enable_thread_tracking: 0 2024-05-12T04:03:33.166 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.enable_pipelined_write: 0 2024-05-12T04:03:33.166 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.unordered_write: 0 2024-05-12T04:03:33.166 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-12T04:03:33.166 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-12T04:03:33.166 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-12T04:03:33.166 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-12T04:03:33.166 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.row_cache: None 2024-05-12T04:03:33.166 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.wal_filter: None 2024-05-12T04:03:33.166 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-12T04:03:33.166 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.allow_ingest_behind: 0 2024-05-12T04:03:33.166 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.two_write_queues: 0 2024-05-12T04:03:33.166 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.manual_wal_flush: 0 2024-05-12T04:03:33.166 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.wal_compression: 0 2024-05-12T04:03:33.167 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.atomic_flush: 0 2024-05-12T04:03:33.167 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-12T04:03:33.167 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-12T04:03:33.167 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-12T04:03:33.167 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.log_readahead_size: 0 2024-05-12T04:03:33.167 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-12T04:03:33.167 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.best_efforts_recovery: 0 2024-05-12T04:03:33.167 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-12T04:03:33.167 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-12T04:03:33.167 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.allow_data_in_errors: 0 2024-05-12T04:03:33.167 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.db_host_id: __hostname__ 2024-05-12T04:03:33.167 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-12T04:03:33.167 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.max_background_jobs: 2 2024-05-12T04:03:33.167 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.max_background_compactions: -1 2024-05-12T04:03:33.167 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.max_subcompactions: 1 2024-05-12T04:03:33.167 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-12T04:03:33.168 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-12T04:03:33.168 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-12T04:03:33.168 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.max_total_wal_size: 0 2024-05-12T04:03:33.168 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-12T04:03:33.168 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-12T04:03:33.168 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-12T04:03:33.168 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-12T04:03:33.168 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.max_open_files: -1 2024-05-12T04:03:33.168 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.bytes_per_sync: 0 2024-05-12T04:03:33.168 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-12T04:03:33.168 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-12T04:03:33.168 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.compaction_readahead_size: 0 2024-05-12T04:03:33.168 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.max_background_flushes: -1 2024-05-12T04:03:33.168 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Compression algorithms supported: 2024-05-12T04:03:33.168 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-12T04:03:33.168 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: kZSTD supported: 0 2024-05-12T04:03:33.169 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: kXpressCompression supported: 0 2024-05-12T04:03:33.169 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: kLZ4HCCompression supported: 1 2024-05-12T04:03:33.169 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: kZlibCompression supported: 1 2024-05-12T04:03:33.169 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: kSnappyCompression supported: 1 2024-05-12T04:03:33.169 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: kLZ4Compression supported: 1 2024-05-12T04:03:33.169 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: kBZip2Compression supported: 0 2024-05-12T04:03:33.169 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-12T04:03:33.169 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-12T04:03:33.169 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi033/store.db/MANIFEST-000009 2024-05-12T04:03:33.169 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-12T04:03:33.169 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-12T04:03:33.169 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.merge_operator: 2024-05-12T04:03:33.169 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.compaction_filter: None 2024-05-12T04:03:33.169 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.compaction_filter_factory: None 2024-05-12T04:03:33.169 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.sst_partitioner_factory: None 2024-05-12T04:03:33.169 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-12T04:03:33.170 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-12T04:03:33.170 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x561dadd0f4e0) 2024-05-12T04:03:33.170 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: cache_index_and_filter_blocks: 1 2024-05-12T04:03:33.170 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-12T04:03:33.170 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-12T04:03:33.170 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: pin_top_level_index_and_filter: 1 2024-05-12T04:03:33.170 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: index_type: 0 2024-05-12T04:03:33.170 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: data_block_index_type: 0 2024-05-12T04:03:33.170 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: index_shortening: 1 2024-05-12T04:03:33.170 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-12T04:03:33.170 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: checksum: 4 2024-05-12T04:03:33.170 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: no_block_cache: 0 2024-05-12T04:03:33.170 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: block_cache: 0x561dad0251f0 2024-05-12T04:03:33.170 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: block_cache_name: BinnedLRUCache 2024-05-12T04:03:33.170 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: block_cache_options: 2024-05-12T04:03:33.171 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: capacity : 536870912 2024-05-12T04:03:33.171 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: num_shard_bits : 4 2024-05-12T04:03:33.171 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: strict_capacity_limit : 0 2024-05-12T04:03:33.171 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: high_pri_pool_ratio: 0.000 2024-05-12T04:03:33.171 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: block_cache_compressed: (nil) 2024-05-12T04:03:33.171 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: persistent_cache: (nil) 2024-05-12T04:03:33.171 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: block_size: 4096 2024-05-12T04:03:33.171 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: block_size_deviation: 10 2024-05-12T04:03:33.171 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: block_restart_interval: 16 2024-05-12T04:03:33.171 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: index_block_restart_interval: 1 2024-05-12T04:03:33.171 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: metadata_block_size: 4096 2024-05-12T04:03:33.171 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: partition_filters: 0 2024-05-12T04:03:33.171 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: use_delta_encoding: 1 2024-05-12T04:03:33.171 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: filter_policy: bloomfilter 2024-05-12T04:03:33.171 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: whole_key_filtering: 1 2024-05-12T04:03:33.172 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: verify_compression: 0 2024-05-12T04:03:33.172 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: read_amp_bytes_per_bit: 0 2024-05-12T04:03:33.172 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: format_version: 5 2024-05-12T04:03:33.172 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: enable_index_compression: 1 2024-05-12T04:03:33.172 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: block_align: 0 2024-05-12T04:03:33.172 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: max_auto_readahead_size: 262144 2024-05-12T04:03:33.172 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: prepopulate_block_cache: 0 2024-05-12T04:03:33.172 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: initial_auto_readahead_size: 8192 2024-05-12T04:03:33.172 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: num_file_reads_for_auto_readahead: 2 2024-05-12T04:03:33.172 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.write_buffer_size: 33554432 2024-05-12T04:03:33.172 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.max_write_buffer_number: 2 2024-05-12T04:03:33.172 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.compression: NoCompression 2024-05-12T04:03:33.172 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.bottommost_compression: Disabled 2024-05-12T04:03:33.172 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.prefix_extractor: nullptr 2024-05-12T04:03:33.172 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-12T04:03:33.172 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.num_levels: 7 2024-05-12T04:03:33.173 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-12T04:03:33.173 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-12T04:03:33.173 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-12T04:03:33.173 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-12T04:03:33.173 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-12T04:03:33.173 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-12T04:03:33.173 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-12T04:03:33.173 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-12T04:03:33.173 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-12T04:03:33.173 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-12T04:03:33.173 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-12T04:03:33.173 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-12T04:03:33.173 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-12T04:03:33.173 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.compression_opts.level: 32767 2024-05-12T04:03:33.173 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.compression_opts.strategy: 0 2024-05-12T04:03:33.174 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-12T04:03:33.174 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-12T04:03:33.174 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-12T04:03:33.174 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-12T04:03:33.174 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.compression_opts.enabled: false 2024-05-12T04:03:33.174 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-12T04:03:33.174 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-12T04:03:33.174 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-12T04:03:33.174 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-12T04:03:33.174 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.target_file_size_base: 67108864 2024-05-12T04:03:33.174 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-12T04:03:33.174 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-12T04:03:33.174 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-12T04:03:33.174 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-12T04:03:33.175 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-12T04:03:33.175 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-12T04:03:33.175 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-12T04:03:33.175 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-12T04:03:33.175 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-12T04:03:33.175 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-12T04:03:33.175 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-12T04:03:33.175 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-12T04:03:33.175 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-12T04:03:33.175 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-12T04:03:33.175 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.arena_block_size: 1048576 2024-05-12T04:03:33.175 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-12T04:03:33.175 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-12T04:03:33.175 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.disable_auto_compactions: 0 2024-05-12T04:03:33.176 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-12T04:03:33.176 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-12T04:03:33.176 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-12T04:03:33.177 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-12T04:03:33.177 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-12T04:03:33.177 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-12T04:03:33.177 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-12T04:03:33.177 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-12T04:03:33.177 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-12T04:03:33.177 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-12T04:03:33.177 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.table_properties_collectors: 2024-05-12T04:03:33.177 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.inplace_update_support: 0 2024-05-12T04:03:33.177 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-12T04:03:33.177 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-12T04:03:33.177 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-12T04:03:33.177 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-12T04:03:33.177 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.bloom_locality: 0 2024-05-12T04:03:33.177 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.max_successive_merges: 0 2024-05-12T04:03:33.177 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-12T04:03:33.178 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.paranoid_file_checks: 0 2024-05-12T04:03:33.178 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.force_consistency_checks: 1 2024-05-12T04:03:33.178 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.report_bg_io_stats: 0 2024-05-12T04:03:33.178 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.ttl: 2592000 2024-05-12T04:03:33.178 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-12T04:03:33.178 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-12T04:03:33.178 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-12T04:03:33.178 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.enable_blob_files: false 2024-05-12T04:03:33.178 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.min_blob_size: 0 2024-05-12T04:03:33.178 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.blob_file_size: 268435456 2024-05-12T04:03:33.178 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-12T04:03:33.178 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-12T04:03:33.178 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-12T04:03:33.178 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-12T04:03:33.178 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-12T04:03:33.179 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.blob_file_starting_level: 0 2024-05-12T04:03:33.179 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-12T04:03:33.179 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 27.sst 2024-05-12T04:03:33.179 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-05-12T04:03:33.179 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi033/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 29, last_sequence is 12101, log_number is 25,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-12T04:03:33.179 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 25 2024-05-12T04:03:33.179 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 3c4959c9-cbf5-4cf4-92f0-0bee4772d0b1 2024-05-12T04:03:33.179 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715486612785887, "job": 1, "event": "recovery_started", "wal_files": [25]} 2024-05-12T04:03:33.179 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #25 mode 2 2024-05-12T04:03:33.179 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715486612793366, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 30, "file_size": 1965839, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12106, "largest_seqno": 12775, "table_properties": {"data_size": 1962008, "index_size": 1985, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 837, "raw_key_size": 7768, "raw_average_key_size": 24, "raw_value_size": 1955057, "raw_average_value_size": 6266, "num_data_blocks": 90, "num_entries": 312, "num_filter_entries": 312, "num_deletions": 2, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715486612, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "3c4959c9-cbf5-4cf4-92f0-0bee4772d0b1", "db_session_id": "F7JQB6MJ1KUMDS2GYBBG", "orig_file_number": 30, "seqno_to_time_mapping": "N/A"}} 2024-05-12T04:03:33.179 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715486612793465, "job": 1, "event": "recovery_finished"} 2024-05-12T04:03:33.179 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: [db/version_set.cc:5047] Creating manifest 32 2024-05-12T04:03:33.179 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-05-12T04:03:33.179 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi033/store.db/000025.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-12T04:03:33.179 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x561dad0c4000 2024-05-12T04:03:33.179 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: DB pointer 0x561dad0b2000 2024-05-12T04:03:33.179 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-12T04:03:33.180 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-12T04:03:33.180 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: ** DB Stats ** 2024-05-12T04:03:33.180 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-12T04:03:33.180 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-12T04:03:33.180 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-12T04:03:33.180 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-12T04:03:33.180 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-12T04:03:33.180 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-12T04:03:33.180 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-12T04:03:33.180 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: 2024-05-12T04:03:33.180 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: ** Compaction Stats [default] ** 2024-05-12T04:03:33.180 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-12T04:03:33.180 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-12T04:03:33.180 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: L0 1/0 1.87 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 618.3 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-05-12T04:03:33.180 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: L6 1/0 6.83 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-05-12T04:03:33.180 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Sum 2/0 8.71 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 618.3 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-05-12T04:03:33.180 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 618.3 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-05-12T04:03:33.181 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: 2024-05-12T04:03:33.181 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: ** Compaction Stats [default] ** 2024-05-12T04:03:33.181 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-12T04:03:33.181 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-12T04:03:33.181 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 618.3 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-05-12T04:03:33.181 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: 2024-05-12T04:03:33.181 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-12T04:03:33.181 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: 2024-05-12T04:03:33.181 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-12T04:03:33.181 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Flush(GB): cumulative 0.002, interval 0.002 2024-05-12T04:03:33.181 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-12T04:03:33.181 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-12T04:03:33.181 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-12T04:03:33.181 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-12T04:03:33.181 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Cumulative compaction: 0.00 GB write, 153.48 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-12T04:03:33.181 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Interval compaction: 0.00 GB write, 153.48 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-12T04:03:33.182 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-12T04:03:33.182 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Block cache BinnedLRUCache@0x561dad0251f0#7 capacity: 512.00 MB usage: 2.98 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-05-12T04:03:33.182 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.91 KB,0.000172853%) IndexBlock(1,2.08 KB,0.000396371%) Misc(1,0.00 KB,0%) 2024-05-12T04:03:33.182 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: 2024-05-12T04:03:33.182 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-12T04:03:33.182 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: starting mon.smithi033 rank 1 at public addrs [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] at bind addrs [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi033 fsid a3e178f8-1012-11ef-bc98-c7b262605968 2024-05-12T04:03:33.182 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: mon.smithi033@-1(???) e2 preinit fsid a3e178f8-1012-11ef-bc98-c7b262605968 2024-05-12T04:03:33.182 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: mon.smithi033@-1(???).mds e18 new map 2024-05-12T04:03:33.182 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: mon.smithi033@-1(???).mds e18 print_map 2024-05-12T04:03:33.182 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: e18 2024-05-12T04:03:33.182 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-12T04:03:33.182 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-12T04:03:33.182 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: legacy client fscid: 1 2024-05-12T04:03:33.182 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: 2024-05-12T04:03:33.183 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Filesystem 'cephfs' (1) 2024-05-12T04:03:33.183 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: fs_name cephfs 2024-05-12T04:03:33.183 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: epoch 18 2024-05-12T04:03:33.183 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-12T04:03:33.183 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: created 2024-05-12T03:57:48.904614+0000 2024-05-12T04:03:33.183 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: modified 2024-05-12T03:58:06.821591+0000 2024-05-12T04:03:33.183 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: tableserver 0 2024-05-12T04:03:33.183 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: root 0 2024-05-12T04:03:33.183 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: session_timeout 60 2024-05-12T04:03:33.183 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: session_autoclose 300 2024-05-12T04:03:33.183 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: max_file_size 1099511627776 2024-05-12T04:03:33.183 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: max_xattr_size 65536 2024-05-12T04:03:33.183 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: required_client_features {} 2024-05-12T04:03:33.183 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: last_failure 0 2024-05-12T04:03:33.183 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: last_failure_osd_epoch 46 2024-05-12T04:03:33.183 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-12T04:03:33.184 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: max_mds 1 2024-05-12T04:03:33.184 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: in 0 2024-05-12T04:03:33.184 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: up {0=14474} 2024-05-12T04:03:33.184 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: failed 2024-05-12T04:03:33.184 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: damaged 2024-05-12T04:03:33.184 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: stopped 2024-05-12T04:03:33.184 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: data_pools [3] 2024-05-12T04:03:33.184 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: metadata_pool 2 2024-05-12T04:03:33.184 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: inline_data enabled 2024-05-12T04:03:33.184 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: balancer 2024-05-12T04:03:33.184 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: bal_rank_mask -1 2024-05-12T04:03:33.184 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: standby_count_wanted 1 2024-05-12T04:03:33.184 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: [mds.cephfs.smithi032.qpsawh{0:14474} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.32:6828/3100155296,v1:172.21.15.32:6829/3100155296] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:03:33.184 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: [mds.cephfs.smithi033.jgbjhk{0:24267} state up:standby-replay seq 3 join_fscid=1 addr [v2:172.21.15.33:6824/55417471,v1:172.21.15.33:6825/55417471] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:03:33.184 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: 2024-05-12T04:03:33.185 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: 2024-05-12T04:03:33.185 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Standby daemons: 2024-05-12T04:03:33.185 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: 2024-05-12T04:03:33.185 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: [mds.cephfs.smithi033.hglwml{-1:24283} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/433858238,v1:172.21.15.33:6827/433858238] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:03:33.185 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: [mds.cephfs.smithi032.ialtqb{-1:24287} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.32:6826/3703132329,v1:172.21.15.32:6827/3703132329] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:03:33.185 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: mon.smithi033@-1(???).osd e49 crush map has features 3314933000852226048, adjusting msgr requires 2024-05-12T04:03:33.185 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: mon.smithi033@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-05-12T04:03:33.185 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: mon.smithi033@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-05-12T04:03:33.185 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: mon.smithi033@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-05-12T04:03:33.185 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 ceph-mon[107305]: mon.smithi033@-1(???).paxosservice(auth 1..22) refresh upgraded, format 0 -> 3 2024-05-12T04:03:33.185 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 bash[107267]: d648ba108f3e35313f0fae2eb7200ee71689fcae4f7ed6a6b45de3a48dacce83 2024-05-12T04:03:33.185 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:32 smithi033 systemd[1]: Started Ceph mon.smithi033 for a3e178f8-1012-11ef-bc98-c7b262605968. 2024-05-12T04:03:34.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:34 smithi033 ceph-mon[107305]: pgmap v10: 65 pgs: 65 active+clean; 262 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 880 KiB/s rd, 879 KiB/s wr, 163 op/s 2024-05-12T04:03:34.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:34 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:03:34.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:34 smithi032 ceph-mon[120392]: pgmap v10: 65 pgs: 65 active+clean; 262 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 880 KiB/s rd, 879 KiB/s wr, 163 op/s 2024-05-12T04:03:34.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:34 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:03:35.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:35 smithi033 ceph-mon[107305]: pgmap v12: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 914 KiB/s rd, 913 KiB/s wr, 177 op/s 2024-05-12T04:03:35.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:35 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mon metadata", "id": "smithi032"}]: dispatch 2024-05-12T04:03:35.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:35 smithi033 ceph-mon[107305]: mon.smithi032 calling monitor election 2024-05-12T04:03:35.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:35 smithi033 ceph-mon[107305]: mon.smithi033 calling monitor election 2024-05-12T04:03:35.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:35 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mon metadata", "id": "smithi033"}]: dispatch 2024-05-12T04:03:35.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:35 smithi033 ceph-mon[107305]: mon.smithi032 is new leader, mons smithi032,smithi033 in quorum (ranks 0,1) 2024-05-12T04:03:35.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:35 smithi033 ceph-mon[107305]: monmap e3: 2 mons at {smithi032=[v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0],smithi033=[v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-12T04:03:35.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:35 smithi033 ceph-mon[107305]: fsmap cephfs:1 {0=cephfs.smithi032.qpsawh=up:active} 1 up:standby-replay 2 up:standby 2024-05-12T04:03:35.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:35 smithi033 ceph-mon[107305]: osdmap e49: 6 total, 6 up, 6 in 2024-05-12T04:03:35.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:35 smithi033 ceph-mon[107305]: mgrmap e39: smithi032.ukatkq(active, since 18s), standbys: smithi033.pcvxwy 2024-05-12T04:03:35.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:35 smithi033 ceph-mon[107305]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-12T04:03:35.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:35 smithi033 ceph-mon[107305]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-12T04:03:35.413 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:35 smithi033 ceph-mon[107305]: fs cephfs has deprecated feature inline_data enabled. 2024-05-12T04:03:35.413 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:35 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:35.413 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:35 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:35.413 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:35 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:03:35.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:35 smithi032 ceph-mon[120392]: pgmap v12: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 914 KiB/s rd, 913 KiB/s wr, 177 op/s 2024-05-12T04:03:35.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:35 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mon metadata", "id": "smithi032"}]: dispatch 2024-05-12T04:03:35.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:35 smithi032 ceph-mon[120392]: mon.smithi032 calling monitor election 2024-05-12T04:03:35.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:35 smithi032 ceph-mon[120392]: mon.smithi033 calling monitor election 2024-05-12T04:03:35.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:35 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mon metadata", "id": "smithi033"}]: dispatch 2024-05-12T04:03:35.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:35 smithi032 ceph-mon[120392]: mon.smithi032 is new leader, mons smithi032,smithi033 in quorum (ranks 0,1) 2024-05-12T04:03:35.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:35 smithi032 ceph-mon[120392]: monmap e3: 2 mons at {smithi032=[v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0],smithi033=[v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-12T04:03:35.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:35 smithi032 ceph-mon[120392]: fsmap cephfs:1 {0=cephfs.smithi032.qpsawh=up:active} 1 up:standby-replay 2 up:standby 2024-05-12T04:03:35.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:35 smithi032 ceph-mon[120392]: osdmap e49: 6 total, 6 up, 6 in 2024-05-12T04:03:35.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:35 smithi032 ceph-mon[120392]: mgrmap e39: smithi032.ukatkq(active, since 18s), standbys: smithi033.pcvxwy 2024-05-12T04:03:35.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:35 smithi032 ceph-mon[120392]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-12T04:03:35.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:35 smithi032 ceph-mon[120392]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-12T04:03:35.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:35 smithi032 ceph-mon[120392]: fs cephfs has deprecated feature inline_data enabled. 2024-05-12T04:03:35.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:35 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:35.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:35 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:35.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:35 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:03:37.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:37 smithi033 ceph-mon[107305]: pgmap v13: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 484 KiB/s rd, 432 KiB/s wr, 83 op/s 2024-05-12T04:03:37.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:37 smithi032 ceph-mon[120392]: pgmap v13: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 484 KiB/s rd, 432 KiB/s wr, 83 op/s 2024-05-12T04:03:39.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:39 smithi033 ceph-mon[107305]: pgmap v14: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 484 KiB/s rd, 432 KiB/s wr, 84 op/s 2024-05-12T04:03:39.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:39 smithi032 ceph-mon[120392]: pgmap v14: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 484 KiB/s rd, 432 KiB/s wr, 84 op/s 2024-05-12T04:03:41.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:41 smithi033 ceph-mon[107305]: pgmap v15: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 34 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-05-12T04:03:41.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:41 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:41.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:41 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:41.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:41 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:41.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:41 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:41.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:41 smithi032 ceph-mon[120392]: pgmap v15: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 34 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-05-12T04:03:41.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:41 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:41.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:41 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:41.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:41 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:41.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:41 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:43.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:43 smithi032 ceph-mon[120392]: pgmap v16: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 34 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-05-12T04:03:43.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:43 smithi033 ceph-mon[107305]: pgmap v16: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 34 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-05-12T04:03:45.512 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:45 smithi032 ceph-mon[120392]: pgmap v17: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 34 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-05-12T04:03:45.513 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:45 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:45.513 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:45 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:45.513 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:45 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:03:45.513 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:45 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T04:03:45.513 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:45 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:45.513 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:45 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-12T04:03:45.513 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:45 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-12T04:03:45.513 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:45 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:03:45.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:45 smithi033 ceph-mon[107305]: pgmap v17: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 34 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-05-12T04:03:45.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:45 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:45.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:45 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:45.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:45 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:03:45.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:45 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T04:03:45.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:45 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:45.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:45 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-12T04:03:45.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:45 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-12T04:03:45.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:45 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:03:46.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:46 smithi032 ceph-mon[120392]: Reconfiguring mon.smithi032 (monmap changed)... 2024-05-12T04:03:46.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:46 smithi032 ceph-mon[120392]: Reconfiguring daemon mon.smithi032 on smithi032 2024-05-12T04:03:46.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:46 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:03:46.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:46 smithi033 ceph-mon[107305]: Reconfiguring mon.smithi032 (monmap changed)... 2024-05-12T04:03:46.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:46 smithi033 ceph-mon[107305]: Reconfiguring daemon mon.smithi032 on smithi032 2024-05-12T04:03:46.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:46 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:03:47.516 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:47 smithi032 ceph-mon[120392]: pgmap v18: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-05-12T04:03:47.517 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:47 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:47.517 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:47 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:47.517 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:47 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi032.ukatkq", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-12T04:03:47.517 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:47 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-12T04:03:47.517 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:47 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:03:47.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:47 smithi033 ceph-mon[107305]: pgmap v18: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-05-12T04:03:47.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:47 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:47.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:47 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:47.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:47 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi032.ukatkq", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-12T04:03:47.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:47 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-12T04:03:47.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:47 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:03:48.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:48 smithi032 ceph-mon[120392]: Reconfiguring mgr.smithi032.ukatkq (monmap changed)... 2024-05-12T04:03:48.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:48 smithi032 ceph-mon[120392]: Reconfiguring daemon mgr.smithi032.ukatkq on smithi032 2024-05-12T04:03:48.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:48 smithi033 ceph-mon[107305]: Reconfiguring mgr.smithi032.ukatkq (monmap changed)... 2024-05-12T04:03:48.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:48 smithi033 ceph-mon[107305]: Reconfiguring daemon mgr.smithi032.ukatkq on smithi032 2024-05-12T04:03:49.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:49 smithi033 ceph-mon[107305]: pgmap v19: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-05-12T04:03:49.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:49 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:49.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:49 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:49.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:49 smithi033 ceph-mon[107305]: Reconfiguring crash.smithi032 (monmap changed)... 2024-05-12T04:03:49.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:49 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi032", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-12T04:03:49.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:49 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:03:49.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:49 smithi033 ceph-mon[107305]: Reconfiguring daemon crash.smithi032 on smithi032 2024-05-12T04:03:49.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:49 smithi032 ceph-mon[120392]: pgmap v19: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-05-12T04:03:49.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:49 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:49.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:49 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:49.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:49 smithi032 ceph-mon[120392]: Reconfiguring crash.smithi032 (monmap changed)... 2024-05-12T04:03:49.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:49 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi032", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-12T04:03:49.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:49 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:03:49.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:49 smithi032 ceph-mon[120392]: Reconfiguring daemon crash.smithi032 on smithi032 2024-05-12T04:03:51.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:50 smithi033 ceph-mon[107305]: pgmap v20: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:03:51.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:50 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:51.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:50 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:51.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:50 smithi033 ceph-mon[107305]: Reconfiguring osd.0 (monmap changed)... 2024-05-12T04:03:51.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:50 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-12T04:03:51.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:50 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:03:51.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:50 smithi033 ceph-mon[107305]: Reconfiguring daemon osd.0 on smithi032 2024-05-12T04:03:51.169 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:50 smithi032 ceph-mon[120392]: pgmap v20: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:03:51.169 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:50 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:51.169 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:50 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:51.169 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:50 smithi032 ceph-mon[120392]: Reconfiguring osd.0 (monmap changed)... 2024-05-12T04:03:51.169 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:50 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-12T04:03:51.169 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:50 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:03:51.169 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:50 smithi032 ceph-mon[120392]: Reconfiguring daemon osd.0 on smithi032 2024-05-12T04:03:52.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:52 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:52.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:52 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:52.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:52 smithi032 ceph-mon[120392]: Reconfiguring osd.1 (monmap changed)... 2024-05-12T04:03:52.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:52 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-12T04:03:52.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:52 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:03:52.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:52 smithi032 ceph-mon[120392]: Reconfiguring daemon osd.1 on smithi032 2024-05-12T04:03:52.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:52 smithi032 ceph-mon[120392]: pgmap v21: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:03:52.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:52.904+0000 7f4add1fb700 1 -- 172.21.15.32:0/3855036955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ad80ffee0 msgr2=0x7f4ad8102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:03:52.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:52.904+0000 7f4add1fb700 1 --2- 172.21.15.32:0/3855036955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ad80ffee0 0x7f4ad8102300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4acc004940 tx=0x7f4acc009e30 comp rx=0 tx=0).stop 2024-05-12T04:03:52.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:52.904+0000 7f4add1fb700 1 -- 172.21.15.32:0/3855036955 shutdown_connections 2024-05-12T04:03:52.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:52.904+0000 7f4add1fb700 1 --2- 172.21.15.32:0/3855036955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ad80ffee0 0x7f4ad8102300 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:52.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:52.904+0000 7f4add1fb700 1 --2- 172.21.15.32:0/3855036955 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ad80662c0 0x7f4ad80ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:52.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:52.904+0000 7f4add1fb700 1 -- 172.21.15.32:0/3855036955 >> 172.21.15.32:0/3855036955 conn(0x7f4ad80f7430 msgr2=0x7f4ad80f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:03:52.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:52.904+0000 7f4add1fb700 1 -- 172.21.15.32:0/3855036955 shutdown_connections 2024-05-12T04:03:52.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:52.904+0000 7f4add1fb700 1 -- 172.21.15.32:0/3855036955 wait complete. 2024-05-12T04:03:52.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:52.905+0000 7f4add1fb700 1 Processor -- start 2024-05-12T04:03:52.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:52.905+0000 7f4add1fb700 1 -- start start 2024-05-12T04:03:52.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:52.905+0000 7f4add1fb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ad80662c0 0x7f4ad80fd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:03:52.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:52.905+0000 7f4add1fb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ad80ffee0 0x7f4ad80fb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:03:52.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:52.905+0000 7f4add1fb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad80fbde0 con 0x7f4ad80ffee0 2024-05-12T04:03:52.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:52.905+0000 7f4add1fb700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad80fbf20 con 0x7f4ad80662c0 2024-05-12T04:03:52.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:52.905+0000 7f4ad659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ad80ffee0 0x7f4ad80fb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:03:52.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:52.905+0000 7f4ad659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ad80ffee0 0x7f4ad80fb8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51958/0 (socket says 172.21.15.32:51958) 2024-05-12T04:03:52.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:52.905+0000 7f4ad659c700 1 -- 172.21.15.32:0/2389924072 learned_addr learned my addr 172.21.15.32:0/2389924072 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:03:52.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:52.906+0000 7f4ad6d9d700 1 --2- 172.21.15.32:0/2389924072 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ad80662c0 0x7f4ad80fd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:03:52.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:52.906+0000 7f4ad659c700 1 -- 172.21.15.32:0/2389924072 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ad80662c0 msgr2=0x7f4ad80fd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:03:52.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:52.906+0000 7f4ad659c700 1 --2- 172.21.15.32:0/2389924072 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ad80662c0 0x7f4ad80fd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:52.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:52.906+0000 7f4ad659c700 1 -- 172.21.15.32:0/2389924072 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4acc005040 con 0x7f4ad80ffee0 2024-05-12T04:03:52.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:52.906+0000 7f4ad659c700 1 --2- 172.21.15.32:0/2389924072 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ad80ffee0 0x7f4ad80fb8a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4acc00b3b0 tx=0x7f4acc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:03:52.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:52.906+0000 7f4abf7fe700 1 -- 172.21.15.32:0/2389924072 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4acc00e040 con 0x7f4ad80ffee0 2024-05-12T04:03:52.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:52.906+0000 7f4add1fb700 1 -- 172.21.15.32:0/2389924072 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ad80fc1a0 con 0x7f4ad80ffee0 2024-05-12T04:03:52.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:52.906+0000 7f4abf7fe700 1 -- 172.21.15.32:0/2389924072 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4acc004290 con 0x7f4ad80ffee0 2024-05-12T04:03:52.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:52.906+0000 7f4abf7fe700 1 -- 172.21.15.32:0/2389924072 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4acc01c3f0 con 0x7f4ad80ffee0 2024-05-12T04:03:52.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:52.906+0000 7f4add1fb700 1 -- 172.21.15.32:0/2389924072 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ad8115a10 con 0x7f4ad80ffee0 2024-05-12T04:03:52.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:52.907+0000 7f4add1fb700 1 -- 172.21.15.32:0/2389924072 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ad805f5c0 con 0x7f4ad80ffee0 2024-05-12T04:03:52.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:52.907+0000 7f4abf7fe700 1 -- 172.21.15.32:0/2389924072 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 39) v1 ==== 90952+0+0 (secure 0 0 0) 0x7f4acc013420 con 0x7f4ad80ffee0 2024-05-12T04:03:52.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:52.908+0000 7f4abf7fe700 1 --2- 172.21.15.32:0/2389924072 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f4ac006cd40 0x7f4ac006f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:03:52.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:52.908+0000 7f4abf7fe700 1 -- 172.21.15.32:0/2389924072 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f4acc08d600 con 0x7f4ad80ffee0 2024-05-12T04:03:52.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:52.908+0000 7f4ad6d9d700 1 --2- 172.21.15.32:0/2389924072 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f4ac006cd40 0x7f4ac006f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:03:52.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:52.908+0000 7f4ad6d9d700 1 --2- 172.21.15.32:0/2389924072 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f4ac006cd40 0x7f4ac006f1f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4ac80041d0 tx=0x7f4ac8009040 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:03:52.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:52 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:52.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:52 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:52.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:52 smithi033 ceph-mon[107305]: Reconfiguring osd.1 (monmap changed)... 2024-05-12T04:03:52.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:52 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-12T04:03:52.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:52 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:03:52.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:52 smithi033 ceph-mon[107305]: Reconfiguring daemon osd.1 on smithi032 2024-05-12T04:03:52.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:52 smithi033 ceph-mon[107305]: pgmap v21: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:03:52.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:52.913+0000 7f4abf7fe700 1 -- 172.21.15.32:0/2389924072 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f4acc057470 con 0x7f4ad80ffee0 2024-05-12T04:03:53.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.135+0000 7f4add1fb700 1 -- 172.21.15.32:0/2389924072 --> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4ad8101b50 con 0x7f4ac006cd40 2024-05-12T04:03:53.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.136+0000 7f4abf7fe700 1 -- 172.21.15.32:0/2389924072 <== mgr.24593 v2:172.21.15.32:6800/2957111591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f4ad8101b50 con 0x7f4ac006cd40 2024-05-12T04:03:53.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.138+0000 7f4add1fb700 1 -- 172.21.15.32:0/2389924072 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f4ac006cd40 msgr2=0x7f4ac006f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:03:53.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.138+0000 7f4add1fb700 1 --2- 172.21.15.32:0/2389924072 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f4ac006cd40 0x7f4ac006f1f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4ac80041d0 tx=0x7f4ac8009040 comp rx=0 tx=0).stop 2024-05-12T04:03:53.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.138+0000 7f4add1fb700 1 -- 172.21.15.32:0/2389924072 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ad80ffee0 msgr2=0x7f4ad80fb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:03:53.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.138+0000 7f4add1fb700 1 --2- 172.21.15.32:0/2389924072 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ad80ffee0 0x7f4ad80fb8a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4acc00b3b0 tx=0x7f4acc009e30 comp rx=0 tx=0).stop 2024-05-12T04:03:53.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.139+0000 7f4add1fb700 1 -- 172.21.15.32:0/2389924072 shutdown_connections 2024-05-12T04:03:53.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.139+0000 7f4add1fb700 1 --2- 172.21.15.32:0/2389924072 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f4ac006cd40 0x7f4ac006f1f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:53.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.139+0000 7f4add1fb700 1 --2- 172.21.15.32:0/2389924072 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ad80662c0 0x7f4ad80fd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:53.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.139+0000 7f4add1fb700 1 --2- 172.21.15.32:0/2389924072 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ad80ffee0 0x7f4ad80fb8a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:53.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.139+0000 7f4add1fb700 1 -- 172.21.15.32:0/2389924072 >> 172.21.15.32:0/2389924072 conn(0x7f4ad80f7430 msgr2=0x7f4ad80f9810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:03:53.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.139+0000 7f4add1fb700 1 -- 172.21.15.32:0/2389924072 shutdown_connections 2024-05-12T04:03:53.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.139+0000 7f4add1fb700 1 -- 172.21.15.32:0/2389924072 wait complete. 2024-05-12T04:03:53.149 INFO:teuthology.orchestra.run.smithi032.stdout:true 2024-05-12T04:03:53.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.249+0000 7ff342e99700 1 -- 172.21.15.32:0/267734497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff33c0fcf60 msgr2=0x7ff33c0fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:03:53.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.249+0000 7ff342e99700 1 --2- 172.21.15.32:0/267734497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff33c0fcf60 0x7ff33c0fd3d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff328004940 tx=0x7ff328009e30 comp rx=0 tx=0).stop 2024-05-12T04:03:53.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.249+0000 7ff342e99700 1 -- 172.21.15.32:0/267734497 shutdown_connections 2024-05-12T04:03:53.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.249+0000 7ff342e99700 1 --2- 172.21.15.32:0/267734497 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff33c0fcf60 0x7ff33c0fd3d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:53.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.249+0000 7ff342e99700 1 --2- 172.21.15.32:0/267734497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff33c0fbcb0 0x7ff33c0fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:53.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.249+0000 7ff342e99700 1 -- 172.21.15.32:0/267734497 >> 172.21.15.32:0/267734497 conn(0x7ff33c0f7430 msgr2=0x7ff33c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:03:53.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.249+0000 7ff342e99700 1 -- 172.21.15.32:0/267734497 shutdown_connections 2024-05-12T04:03:53.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.249+0000 7ff342e99700 1 -- 172.21.15.32:0/267734497 wait complete. 2024-05-12T04:03:53.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.250+0000 7ff342e99700 1 Processor -- start 2024-05-12T04:03:53.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.250+0000 7ff342e99700 1 -- start start 2024-05-12T04:03:53.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.250+0000 7ff342e99700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff33c0fbcb0 0x7ff33c10c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:03:53.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.250+0000 7ff342e99700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff33c0fcf60 0x7ff33c10cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:03:53.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.251+0000 7ff342e99700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff33c10d400 con 0x7ff33c0fcf60 2024-05-12T04:03:53.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.251+0000 7ff342e99700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff33c10dbf0 con 0x7ff33c0fbcb0 2024-05-12T04:03:53.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.251+0000 7ff33bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff33c0fcf60 0x7ff33c10cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:03:53.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.251+0000 7ff33bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff33c0fcf60 0x7ff33c10cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51976/0 (socket says 172.21.15.32:51976) 2024-05-12T04:03:53.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.251+0000 7ff33bfff700 1 -- 172.21.15.32:0/718930771 learned_addr learned my addr 172.21.15.32:0/718930771 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:03:53.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.251+0000 7ff340c35700 1 --2- 172.21.15.32:0/718930771 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff33c0fbcb0 0x7ff33c10c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:03:53.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.251+0000 7ff33bfff700 1 -- 172.21.15.32:0/718930771 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff33c0fbcb0 msgr2=0x7ff33c10c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:03:53.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.251+0000 7ff33bfff700 1 --2- 172.21.15.32:0/718930771 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff33c0fbcb0 0x7ff33c10c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:53.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.251+0000 7ff33bfff700 1 -- 172.21.15.32:0/718930771 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff328005040 con 0x7ff33c0fcf60 2024-05-12T04:03:53.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.251+0000 7ff33bfff700 1 --2- 172.21.15.32:0/718930771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff33c0fcf60 0x7ff33c10cd70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff32800b3b0 tx=0x7ff328009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:03:53.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.251+0000 7ff3397fa700 1 -- 172.21.15.32:0/718930771 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff32800e040 con 0x7ff33c0fcf60 2024-05-12T04:03:53.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.251+0000 7ff342e99700 1 -- 172.21.15.32:0/718930771 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff33c10de70 con 0x7ff33c0fcf60 2024-05-12T04:03:53.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.251+0000 7ff3397fa700 1 -- 172.21.15.32:0/718930771 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff328004290 con 0x7ff33c0fcf60 2024-05-12T04:03:53.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.251+0000 7ff3397fa700 1 -- 172.21.15.32:0/718930771 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff32801c3f0 con 0x7ff33c0fcf60 2024-05-12T04:03:53.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.252+0000 7ff342e99700 1 -- 172.21.15.32:0/718930771 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff33c10e430 con 0x7ff33c0fcf60 2024-05-12T04:03:53.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.253+0000 7ff342e99700 1 -- 172.21.15.32:0/718930771 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff33c05f5c0 con 0x7ff33c0fcf60 2024-05-12T04:03:53.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.253+0000 7ff3397fa700 1 -- 172.21.15.32:0/718930771 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 39) v1 ==== 90952+0+0 (secure 0 0 0) 0x7ff32801c6b0 con 0x7ff33c0fcf60 2024-05-12T04:03:53.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.253+0000 7ff3397fa700 1 --2- 172.21.15.32:0/718930771 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7ff32c06cd40 0x7ff32c06f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:03:53.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.254+0000 7ff3397fa700 1 -- 172.21.15.32:0/718930771 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7ff32808d670 con 0x7ff33c0fcf60 2024-05-12T04:03:53.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.254+0000 7ff340c35700 1 --2- 172.21.15.32:0/718930771 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7ff32c06cd40 0x7ff32c06f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:03:53.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.254+0000 7ff340c35700 1 --2- 172.21.15.32:0/718930771 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7ff32c06cd40 0x7ff32c06f1f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff330006fd0 tx=0x7ff330008040 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:03:53.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.259+0000 7ff3397fa700 1 -- 172.21.15.32:0/718930771 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7ff32805b0c0 con 0x7ff33c0fcf60 2024-05-12T04:03:53.498 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.498+0000 7ff342e99700 1 -- 172.21.15.32:0/718930771 --> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff33c10e710 con 0x7ff32c06cd40 2024-05-12T04:03:53.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.499+0000 7ff3397fa700 1 -- 172.21.15.32:0/718930771 <== mgr.24593 v2:172.21.15.32:6800/2957111591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7ff33c10e710 con 0x7ff32c06cd40 2024-05-12T04:03:53.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.501+0000 7ff342e99700 1 -- 172.21.15.32:0/718930771 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7ff32c06cd40 msgr2=0x7ff32c06f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:03:53.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.501+0000 7ff342e99700 1 --2- 172.21.15.32:0/718930771 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7ff32c06cd40 0x7ff32c06f1f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff330006fd0 tx=0x7ff330008040 comp rx=0 tx=0).stop 2024-05-12T04:03:53.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.501+0000 7ff342e99700 1 -- 172.21.15.32:0/718930771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff33c0fcf60 msgr2=0x7ff33c10cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:03:53.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.501+0000 7ff342e99700 1 --2- 172.21.15.32:0/718930771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff33c0fcf60 0x7ff33c10cd70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff32800b3b0 tx=0x7ff328009e30 comp rx=0 tx=0).stop 2024-05-12T04:03:53.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.501+0000 7ff342e99700 1 -- 172.21.15.32:0/718930771 shutdown_connections 2024-05-12T04:03:53.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.501+0000 7ff342e99700 1 --2- 172.21.15.32:0/718930771 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7ff32c06cd40 0x7ff32c06f1f0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:53.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.501+0000 7ff342e99700 1 --2- 172.21.15.32:0/718930771 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff33c0fbcb0 0x7ff33c10c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:53.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.501+0000 7ff342e99700 1 --2- 172.21.15.32:0/718930771 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff33c0fcf60 0x7ff33c10cd70 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:53.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.501+0000 7ff342e99700 1 -- 172.21.15.32:0/718930771 >> 172.21.15.32:0/718930771 conn(0x7ff33c0f7430 msgr2=0x7ff33c0f8b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:03:53.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.501+0000 7ff342e99700 1 -- 172.21.15.32:0/718930771 shutdown_connections 2024-05-12T04:03:53.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.501+0000 7ff342e99700 1 -- 172.21.15.32:0/718930771 wait complete. 2024-05-12T04:03:53.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.624+0000 7f6ff2918700 1 -- 172.21.15.32:0/2412914341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fec0ffee0 msgr2=0x7f6fec102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:03:53.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.624+0000 7f6ff2918700 1 --2- 172.21.15.32:0/2412914341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fec0ffee0 0x7f6fec102300 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6fd4004940 tx=0x7f6fd4009e30 comp rx=0 tx=0).stop 2024-05-12T04:03:53.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.624+0000 7f6ff2918700 1 -- 172.21.15.32:0/2412914341 shutdown_connections 2024-05-12T04:03:53.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.624+0000 7f6ff2918700 1 --2- 172.21.15.32:0/2412914341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fec0ffee0 0x7f6fec102300 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:53.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.624+0000 7f6ff2918700 1 --2- 172.21.15.32:0/2412914341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fec0662c0 0x7f6fec0ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:53.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.624+0000 7f6ff2918700 1 -- 172.21.15.32:0/2412914341 >> 172.21.15.32:0/2412914341 conn(0x7f6fec0f7430 msgr2=0x7f6fec0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:03:53.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.624+0000 7f6ff2918700 1 -- 172.21.15.32:0/2412914341 shutdown_connections 2024-05-12T04:03:53.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.624+0000 7f6ff2918700 1 -- 172.21.15.32:0/2412914341 wait complete. 2024-05-12T04:03:53.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.624+0000 7f6ff2918700 1 Processor -- start 2024-05-12T04:03:53.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.625+0000 7f6ff2918700 1 -- start start 2024-05-12T04:03:53.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.625+0000 7f6ff2918700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fec0662c0 0x7f6fec10d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:03:53.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.625+0000 7f6ff2918700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fec0ffee0 0x7f6fec10deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:03:53.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.625+0000 7f6ff2918700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fec10e4b0 con 0x7f6fec0662c0 2024-05-12T04:03:53.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.625+0000 7f6ff2918700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fec10ed00 con 0x7f6fec0ffee0 2024-05-12T04:03:53.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.625+0000 7f6febfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fec0662c0 0x7f6fec10d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:03:53.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.625+0000 7f6febfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fec0662c0 0x7f6fec10d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51986/0 (socket says 172.21.15.32:51986) 2024-05-12T04:03:53.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.625+0000 7f6febfff700 1 -- 172.21.15.32:0/2079780215 learned_addr learned my addr 172.21.15.32:0/2079780215 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:03:53.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.625+0000 7f6feb7fe700 1 --2- 172.21.15.32:0/2079780215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fec0ffee0 0x7f6fec10deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:03:53.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.625+0000 7f6febfff700 1 -- 172.21.15.32:0/2079780215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fec0ffee0 msgr2=0x7f6fec10deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:03:53.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.625+0000 7f6febfff700 1 --2- 172.21.15.32:0/2079780215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fec0ffee0 0x7f6fec10deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:53.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.625+0000 7f6febfff700 1 -- 172.21.15.32:0/2079780215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd4005040 con 0x7f6fec0662c0 2024-05-12T04:03:53.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.626+0000 7f6febfff700 1 --2- 172.21.15.32:0/2079780215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fec0662c0 0x7f6fec10d970 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc00c890 tx=0x7f6fdc0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:03:53.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.626+0000 7f6fe8ff9700 1 -- 172.21.15.32:0/2079780215 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6fdc00a7d0 con 0x7f6fec0662c0 2024-05-12T04:03:53.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.626+0000 7f6fe8ff9700 1 -- 172.21.15.32:0/2079780215 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6fdc00ade0 con 0x7f6fec0662c0 2024-05-12T04:03:53.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.626+0000 7f6ff2918700 1 -- 172.21.15.32:0/2079780215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fec10efe0 con 0x7f6fec0662c0 2024-05-12T04:03:53.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.626+0000 7f6fe8ff9700 1 -- 172.21.15.32:0/2079780215 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6fdc0113a0 con 0x7f6fec0662c0 2024-05-12T04:03:53.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.626+0000 7f6ff2918700 1 -- 172.21.15.32:0/2079780215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fec10f520 con 0x7f6fec0662c0 2024-05-12T04:03:53.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.627+0000 7f6ff2918700 1 -- 172.21.15.32:0/2079780215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fec05f5c0 con 0x7f6fec0662c0 2024-05-12T04:03:53.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.627+0000 7f6fe8ff9700 1 -- 172.21.15.32:0/2079780215 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 39) v1 ==== 90952+0+0 (secure 0 0 0) 0x7f6fdc012400 con 0x7f6fec0662c0 2024-05-12T04:03:53.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.628+0000 7f6fe8ff9700 1 --2- 172.21.15.32:0/2079780215 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f6fd806cd40 0x7f6fd806f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:03:53.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.628+0000 7f6fe8ff9700 1 -- 172.21.15.32:0/2079780215 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f6fdc08c2a0 con 0x7f6fec0662c0 2024-05-12T04:03:53.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.628+0000 7f6feb7fe700 1 --2- 172.21.15.32:0/2079780215 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f6fd806cd40 0x7f6fd806f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:03:53.629 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.628+0000 7f6feb7fe700 1 --2- 172.21.15.32:0/2079780215 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f6fd806cd40 0x7f6fd806f1f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6fd400b3b0 tx=0x7f6fd4009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:03:53.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.632+0000 7f6fe8ff9700 1 -- 172.21.15.32:0/2079780215 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f6fdc059cb0 con 0x7f6fec0662c0 2024-05-12T04:03:53.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.856+0000 7f6ff2918700 1 -- 172.21.15.32:0/2079780215 --> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6fec10f810 con 0x7f6fd806cd40 2024-05-12T04:03:53.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.868+0000 7f6fe8ff9700 1 -- 172.21.15.32:0/2079780215 <== mgr.24593 v2:172.21.15.32:6800/2957111591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f6fec10f810 con 0x7f6fd806cd40 2024-05-12T04:03:53.868 INFO:teuthology.orchestra.run.smithi032.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-12T04:03:53.868 INFO:teuthology.orchestra.run.smithi032.stdout:alertmanager.smithi032 smithi032 *:9093,9094 running (3m) 29s ago 11m 17.8M - 0.25.0 c8568f914cd2 a37fbab7af4c 2024-05-12T04:03:53.868 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi032 smithi032 running (12m) 29s ago 12m 6656k - 17.2.7-939-gc190bdee 284803f7d9cf a478874e9dfa 2024-05-12T04:03:53.868 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi033 smithi033 running (11m) 13s ago 11m 6651k - 17.2.7-939-gc190bdee 284803f7d9cf 34c62d004c7e 2024-05-12T04:03:53.868 INFO:teuthology.orchestra.run.smithi032.stdout:grafana.smithi032 smithi032 *:3000 running (2m) 29s ago 11m 81.7M - 9.4.7 954c08fa6188 83055e58d23c 2024-05-12T04:03:53.868 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.ialtqb smithi032 running (6m) 29s ago 6m 12.9M - 17.2.7-939-gc190bdee 284803f7d9cf 0683abf00e3e 2024-05-12T04:03:53.869 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.qpsawh smithi032 running (5m) 29s ago 5m 190M - 17.2.7-939-gc190bdee 284803f7d9cf 95a7db5c4fcd 2024-05-12T04:03:53.869 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi033.hglwml smithi033 running (5m) 13s ago 5m 13.0M - 17.2.7-939-gc190bdee 284803f7d9cf c97902b05025 2024-05-12T04:03:53.869 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi033.jgbjhk smithi033 running (5m) 13s ago 5m 102M - 17.2.7-939-gc190bdee 284803f7d9cf af97052cb97a 2024-05-12T04:03:53.869 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi032.ukatkq smithi032 *:8443,9283,8765 running (116s) 29s ago 13m 468M - 18.2.2-1272-g58249552 ddcb0a9d6fde 0d88ab9a5292 2024-05-12T04:03:53.869 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi033.pcvxwy smithi033 *:8443,9283,8765 running (74s) 13s ago 10m 447M - 18.2.2-1272-g58249552 ddcb0a9d6fde d42dcb6bd307 2024-05-12T04:03:53.869 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi032 smithi032 running (54s) 29s ago 13m 29.1M 2048M 18.2.2-1272-g58249552 ddcb0a9d6fde 81f94171c5dc 2024-05-12T04:03:53.869 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi033 smithi033 running (21s) 13s ago 10m 21.0M 2048M 18.2.2-1272-g58249552 ddcb0a9d6fde d648ba108f3e 2024-05-12T04:03:53.869 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi032 smithi032 *:9100 running (3m) 29s ago 12m 14.3M - 1.5.0 0da6a335fe13 cb75a7c0c829 2024-05-12T04:03:53.869 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi033 smithi033 *:9100 running (2m) 13s ago 10m 17.7M - 1.5.0 0da6a335fe13 7888c36c659f 2024-05-12T04:03:53.870 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0 smithi032 running (10m) 29s ago 10m 206M 1570M 17.2.7-939-gc190bdee 284803f7d9cf 75c78a47f8e5 2024-05-12T04:03:53.870 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1 smithi032 running (9m) 29s ago 9m 225M 1570M 17.2.7-939-gc190bdee 284803f7d9cf 87cfd247d8d6 2024-05-12T04:03:53.870 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2 smithi032 running (9m) 29s ago 9m 180M 1570M 17.2.7-939-gc190bdee 284803f7d9cf 7aef7affb8ba 2024-05-12T04:03:53.870 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3 smithi033 running (8m) 13s ago 8m 239M 2594M 17.2.7-939-gc190bdee 284803f7d9cf 3a92b5e3348e 2024-05-12T04:03:53.870 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4 smithi033 running (8m) 13s ago 8m 213M 2594M 17.2.7-939-gc190bdee 284803f7d9cf 0896268b4591 2024-05-12T04:03:53.870 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5 smithi033 running (7m) 13s ago 7m 196M 2594M 17.2.7-939-gc190bdee 284803f7d9cf 57e65a64c821 2024-05-12T04:03:53.870 INFO:teuthology.orchestra.run.smithi032.stdout:prometheus.smithi032 smithi032 *:9095 running (82s) 29s ago 11m 48.0M - 2.43.0 a07b618ecd1d 46eafb25e176 2024-05-12T04:03:53.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.870+0000 7f6ff2918700 1 -- 172.21.15.32:0/2079780215 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f6fd806cd40 msgr2=0x7f6fd806f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:03:53.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.870+0000 7f6ff2918700 1 --2- 172.21.15.32:0/2079780215 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f6fd806cd40 0x7f6fd806f1f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6fd400b3b0 tx=0x7f6fd4009e30 comp rx=0 tx=0).stop 2024-05-12T04:03:53.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.870+0000 7f6ff2918700 1 -- 172.21.15.32:0/2079780215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fec0662c0 msgr2=0x7f6fec10d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:03:53.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.870+0000 7f6ff2918700 1 --2- 172.21.15.32:0/2079780215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fec0662c0 0x7f6fec10d970 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc00c890 tx=0x7f6fdc0044c0 comp rx=0 tx=0).stop 2024-05-12T04:03:53.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.870+0000 7f6ff2918700 1 -- 172.21.15.32:0/2079780215 shutdown_connections 2024-05-12T04:03:53.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.870+0000 7f6ff2918700 1 --2- 172.21.15.32:0/2079780215 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f6fd806cd40 0x7f6fd806f1f0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:53.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.870+0000 7f6ff2918700 1 --2- 172.21.15.32:0/2079780215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6fec0662c0 0x7f6fec10d970 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:53.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.870+0000 7f6ff2918700 1 --2- 172.21.15.32:0/2079780215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fec0ffee0 0x7f6fec10deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:53.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.870+0000 7f6ff2918700 1 -- 172.21.15.32:0/2079780215 >> 172.21.15.32:0/2079780215 conn(0x7f6fec0f7430 msgr2=0x7f6fec0f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:03:53.871 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.870+0000 7f6ff2918700 1 -- 172.21.15.32:0/2079780215 shutdown_connections 2024-05-12T04:03:53.872 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.871+0000 7f6ff2918700 1 -- 172.21.15.32:0/2079780215 wait complete. 2024-05-12T04:03:53.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.976+0000 7f81e595d700 1 -- 172.21.15.32:0/2119303689 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81e0066e10 msgr2=0x7f81e00ffa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:03:53.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.976+0000 7f81e595d700 1 --2- 172.21.15.32:0/2119303689 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81e0066e10 0x7f81e00ffa60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f81d0004940 tx=0x7f81d0009e30 comp rx=0 tx=0).stop 2024-05-12T04:03:53.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.976+0000 7f81e595d700 1 -- 172.21.15.32:0/2119303689 shutdown_connections 2024-05-12T04:03:53.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.976+0000 7f81e595d700 1 --2- 172.21.15.32:0/2119303689 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81e00fffa0 0x7f81e0102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:53.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.976+0000 7f81e595d700 1 --2- 172.21.15.32:0/2119303689 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81e0066e10 0x7f81e00ffa60 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:53.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.976+0000 7f81e595d700 1 -- 172.21.15.32:0/2119303689 >> 172.21.15.32:0/2119303689 conn(0x7f81e00f7430 msgr2=0x7f81e00f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:03:53.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.976+0000 7f81e595d700 1 -- 172.21.15.32:0/2119303689 shutdown_connections 2024-05-12T04:03:53.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.976+0000 7f81e595d700 1 -- 172.21.15.32:0/2119303689 wait complete. 2024-05-12T04:03:53.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.976+0000 7f81e595d700 1 Processor -- start 2024-05-12T04:03:53.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.976+0000 7f81e595d700 1 -- start start 2024-05-12T04:03:53.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.977+0000 7f81e595d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81e0066e10 0x7f81e010d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:03:53.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.977+0000 7f81e595d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81e00fffa0 0x7f81e010de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:03:53.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.977+0000 7f81e595d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e010e470 con 0x7f81e0066e10 2024-05-12T04:03:53.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.977+0000 7f81e595d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e010e5e0 con 0x7f81e00fffa0 2024-05-12T04:03:53.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.977+0000 7f81deffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81e0066e10 0x7f81e010d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:03:53.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.977+0000 7f81deffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81e0066e10 0x7f81e010d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51996/0 (socket says 172.21.15.32:51996) 2024-05-12T04:03:53.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.977+0000 7f81deffd700 1 -- 172.21.15.32:0/954146210 learned_addr learned my addr 172.21.15.32:0/954146210 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:03:53.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.977+0000 7f81de7fc700 1 --2- 172.21.15.32:0/954146210 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81e00fffa0 0x7f81e010de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:03:53.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.977+0000 7f81deffd700 1 -- 172.21.15.32:0/954146210 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81e00fffa0 msgr2=0x7f81e010de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:03:53.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.977+0000 7f81deffd700 1 --2- 172.21.15.32:0/954146210 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81e00fffa0 0x7f81e010de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:53.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.977+0000 7f81deffd700 1 -- 172.21.15.32:0/954146210 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81d0005040 con 0x7f81e0066e10 2024-05-12T04:03:53.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.977+0000 7f81deffd700 1 --2- 172.21.15.32:0/954146210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81e0066e10 0x7f81e010d930 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f81d0000c00 tx=0x7f81d0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:03:53.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.977+0000 7f81c77fe700 1 -- 172.21.15.32:0/954146210 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81d000e040 con 0x7f81e0066e10 2024-05-12T04:03:53.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.978+0000 7f81e595d700 1 -- 172.21.15.32:0/954146210 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81e0112f90 con 0x7f81e0066e10 2024-05-12T04:03:53.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.978+0000 7f81c77fe700 1 -- 172.21.15.32:0/954146210 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f81d000bc50 con 0x7f81e0066e10 2024-05-12T04:03:53.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.978+0000 7f81c77fe700 1 -- 172.21.15.32:0/954146210 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81d001c630 con 0x7f81e0066e10 2024-05-12T04:03:53.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.978+0000 7f81e595d700 1 -- 172.21.15.32:0/954146210 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81e01134c0 con 0x7f81e0066e10 2024-05-12T04:03:53.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.979+0000 7f81e595d700 1 -- 172.21.15.32:0/954146210 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81e004c710 con 0x7f81e0066e10 2024-05-12T04:03:53.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.979+0000 7f81c77fe700 1 -- 172.21.15.32:0/954146210 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 39) v1 ==== 90952+0+0 (secure 0 0 0) 0x7f81d0013420 con 0x7f81e0066e10 2024-05-12T04:03:53.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.979+0000 7f81c77fe700 1 --2- 172.21.15.32:0/954146210 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f81c806cd50 0x7f81c806f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:03:53.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.980+0000 7f81c77fe700 1 -- 172.21.15.32:0/954146210 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f81d008d570 con 0x7f81e0066e10 2024-05-12T04:03:53.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.980+0000 7f81de7fc700 1 --2- 172.21.15.32:0/954146210 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f81c806cd50 0x7f81c806f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:03:53.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.980+0000 7f81de7fc700 1 --2- 172.21.15.32:0/954146210 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f81c806cd50 0x7f81c806f200 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f81d4007fd0 tx=0x7f81d4009040 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:03:53.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:53.984+0000 7f81c77fe700 1 -- 172.21.15.32:0/954146210 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f81d005af20 con 0x7f81e0066e10 2024-05-12T04:03:54.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.283+0000 7f81e595d700 1 -- 172.21.15.32:0/954146210 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f81e0105780 con 0x7f81e0066e10 2024-05-12T04:03:54.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.284+0000 7f81c77fe700 1 -- 172.21.15.32:0/954146210 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+742 (secure 0 0 0) 0x7f81d000bdc0 con 0x7f81e0066e10 2024-05-12T04:03:54.284 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-05-12T04:03:54.284 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": { 2024-05-12T04:03:54.284 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 2 2024-05-12T04:03:54.284 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:03:54.284 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": { 2024-05-12T04:03:54.285 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 2 2024-05-12T04:03:54.285 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:03:54.285 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": { 2024-05-12T04:03:54.285 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 6 2024-05-12T04:03:54.285 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:03:54.285 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": { 2024-05-12T04:03:54.285 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 4 2024-05-12T04:03:54.285 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:03:54.285 INFO:teuthology.orchestra.run.smithi032.stdout: "overall": { 2024-05-12T04:03:54.285 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 10, 2024-05-12T04:03:54.285 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 4 2024-05-12T04:03:54.285 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-05-12T04:03:54.285 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-05-12T04:03:54.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.286+0000 7f81e595d700 1 -- 172.21.15.32:0/954146210 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f81c806cd50 msgr2=0x7f81c806f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:03:54.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.286+0000 7f81e595d700 1 --2- 172.21.15.32:0/954146210 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f81c806cd50 0x7f81c806f200 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f81d4007fd0 tx=0x7f81d4009040 comp rx=0 tx=0).stop 2024-05-12T04:03:54.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.286+0000 7f81e595d700 1 -- 172.21.15.32:0/954146210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81e0066e10 msgr2=0x7f81e010d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:03:54.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.286+0000 7f81e595d700 1 --2- 172.21.15.32:0/954146210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81e0066e10 0x7f81e010d930 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f81d0000c00 tx=0x7f81d0009e30 comp rx=0 tx=0).stop 2024-05-12T04:03:54.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.286+0000 7f81e595d700 1 -- 172.21.15.32:0/954146210 shutdown_connections 2024-05-12T04:03:54.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.286+0000 7f81e595d700 1 --2- 172.21.15.32:0/954146210 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f81c806cd50 0x7f81c806f200 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:54.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.286+0000 7f81e595d700 1 --2- 172.21.15.32:0/954146210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81e0066e10 0x7f81e010d930 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:54.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.286+0000 7f81e595d700 1 --2- 172.21.15.32:0/954146210 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81e00fffa0 0x7f81e010de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:54.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.286+0000 7f81e595d700 1 -- 172.21.15.32:0/954146210 >> 172.21.15.32:0/954146210 conn(0x7f81e00f7430 msgr2=0x7f81e00f9810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:03:54.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.286+0000 7f81e595d700 1 -- 172.21.15.32:0/954146210 shutdown_connections 2024-05-12T04:03:54.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.286+0000 7f81e595d700 1 -- 172.21.15.32:0/954146210 wait complete. 2024-05-12T04:03:54.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:54 smithi032 ceph-mon[120392]: from='client.34156 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:03:54.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:54 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:54.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:54 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:54.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:54 smithi032 ceph-mon[120392]: Reconfiguring osd.2 (monmap changed)... 2024-05-12T04:03:54.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:54 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-12T04:03:54.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:54 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:03:54.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:54 smithi032 ceph-mon[120392]: Reconfiguring daemon osd.2 on smithi032 2024-05-12T04:03:54.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.390+0000 7f8b1de5c700 1 -- 172.21.15.32:0/3950742935 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b180fdff0 msgr2=0x7f8b180fe460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:03:54.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.390+0000 7f8b1de5c700 1 --2- 172.21.15.32:0/3950742935 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b180fdff0 0x7f8b180fe460 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c004940 tx=0x7f8b0c009e30 comp rx=0 tx=0).stop 2024-05-12T04:03:54.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.390+0000 7f8b1de5c700 1 -- 172.21.15.32:0/3950742935 shutdown_connections 2024-05-12T04:03:54.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.390+0000 7f8b1de5c700 1 --2- 172.21.15.32:0/3950742935 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b180fdff0 0x7f8b180fe460 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:54.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.390+0000 7f8b1de5c700 1 --2- 172.21.15.32:0/3950742935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b180fce80 0x7f8b180fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:54.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.390+0000 7f8b1de5c700 1 -- 172.21.15.32:0/3950742935 >> 172.21.15.32:0/3950742935 conn(0x7f8b180f8520 msgr2=0x7f8b180fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:03:54.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.391+0000 7f8b1de5c700 1 -- 172.21.15.32:0/3950742935 shutdown_connections 2024-05-12T04:03:54.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.391+0000 7f8b1de5c700 1 -- 172.21.15.32:0/3950742935 wait complete. 2024-05-12T04:03:54.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.391+0000 7f8b1de5c700 1 Processor -- start 2024-05-12T04:03:54.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.391+0000 7f8b1de5c700 1 -- start start 2024-05-12T04:03:54.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.391+0000 7f8b1de5c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b180fce80 0x7f8b1810d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:03:54.393 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.391+0000 7f8b1de5c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b180fdff0 0x7f8b1810de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:03:54.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.391+0000 7f8b1de5c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1810e490 con 0x7f8b180fdff0 2024-05-12T04:03:54.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.392+0000 7f8b1de5c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1810ece0 con 0x7f8b180fce80 2024-05-12T04:03:54.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.392+0000 7f8b16ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b180fdff0 0x7f8b1810de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:03:54.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.392+0000 7f8b16ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b180fdff0 0x7f8b1810de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52006/0 (socket says 172.21.15.32:52006) 2024-05-12T04:03:54.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.392+0000 7f8b16ffd700 1 -- 172.21.15.32:0/2765969518 learned_addr learned my addr 172.21.15.32:0/2765969518 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:03:54.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.392+0000 7f8b177fe700 1 --2- 172.21.15.32:0/2765969518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b180fce80 0x7f8b1810d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:03:54.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.392+0000 7f8b16ffd700 1 -- 172.21.15.32:0/2765969518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b180fce80 msgr2=0x7f8b1810d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:03:54.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.392+0000 7f8b16ffd700 1 --2- 172.21.15.32:0/2765969518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b180fce80 0x7f8b1810d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:54.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.392+0000 7f8b16ffd700 1 -- 172.21.15.32:0/2765969518 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b0c005040 con 0x7f8b180fdff0 2024-05-12T04:03:54.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.392+0000 7f8b177fe700 1 --2- 172.21.15.32:0/2765969518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b180fce80 0x7f8b1810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-12T04:03:54.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.392+0000 7f8b16ffd700 1 --2- 172.21.15.32:0/2765969518 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b180fdff0 0x7f8b1810de90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c00b3b0 tx=0x7f8b0c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:03:54.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.392+0000 7f8afffff700 1 -- 172.21.15.32:0/2765969518 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b0c00e040 con 0x7f8b180fdff0 2024-05-12T04:03:54.394 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.392+0000 7f8afffff700 1 -- 172.21.15.32:0/2765969518 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8b0c00bd20 con 0x7f8b180fdff0 2024-05-12T04:03:54.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.392+0000 7f8b1de5c700 1 -- 172.21.15.32:0/2765969518 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b1810ef60 con 0x7f8b180fdff0 2024-05-12T04:03:54.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.392+0000 7f8afffff700 1 -- 172.21.15.32:0/2765969518 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b0c01c630 con 0x7f8b180fdff0 2024-05-12T04:03:54.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.392+0000 7f8b1de5c700 1 -- 172.21.15.32:0/2765969518 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b1810f4a0 con 0x7f8b180fdff0 2024-05-12T04:03:54.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.393+0000 7f8b1de5c700 1 -- 172.21.15.32:0/2765969518 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b1805f5c0 con 0x7f8b180fdff0 2024-05-12T04:03:54.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.394+0000 7f8afffff700 1 -- 172.21.15.32:0/2765969518 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 39) v1 ==== 90952+0+0 (secure 0 0 0) 0x7f8b0c013420 con 0x7f8b180fdff0 2024-05-12T04:03:54.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.394+0000 7f8afffff700 1 --2- 172.21.15.32:0/2765969518 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f8b0006cd40 0x7f8b0006f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:03:54.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.394+0000 7f8afffff700 1 -- 172.21.15.32:0/2765969518 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f8b0c08cf10 con 0x7f8b180fdff0 2024-05-12T04:03:54.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.394+0000 7f8b177fe700 1 --2- 172.21.15.32:0/2765969518 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f8b0006cd40 0x7f8b0006f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:03:54.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.395+0000 7f8b177fe700 1 --2- 172.21.15.32:0/2765969518 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f8b0006cd40 0x7f8b0006f1f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8b08007fd0 tx=0x7f8b08009040 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:03:54.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.399+0000 7f8afffff700 1 -- 172.21.15.32:0/2765969518 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f8b0c05a770 con 0x7f8b180fdff0 2024-05-12T04:03:54.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.648+0000 7f8b1de5c700 1 -- 172.21.15.32:0/2765969518 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f8b181057a0 con 0x7f8b180fdff0 2024-05-12T04:03:54.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.649+0000 7f8afffff700 1 -- 172.21.15.32:0/2765969518 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1866 (secure 0 0 0) 0x7f8b0c05a300 con 0x7f8b180fdff0 2024-05-12T04:03:54.649 INFO:teuthology.orchestra.run.smithi032.stdout:e18 2024-05-12T04:03:54.649 INFO:teuthology.orchestra.run.smithi032.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-12T04:03:54.649 INFO:teuthology.orchestra.run.smithi032.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-12T04:03:54.649 INFO:teuthology.orchestra.run.smithi032.stdout:legacy client fscid: 1 2024-05-12T04:03:54.650 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:03:54.650 INFO:teuthology.orchestra.run.smithi032.stdout:Filesystem 'cephfs' (1) 2024-05-12T04:03:54.650 INFO:teuthology.orchestra.run.smithi032.stdout:fs_name cephfs 2024-05-12T04:03:54.650 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 18 2024-05-12T04:03:54.650 INFO:teuthology.orchestra.run.smithi032.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-12T04:03:54.650 INFO:teuthology.orchestra.run.smithi032.stdout:created 2024-05-12T03:57:48.904614+0000 2024-05-12T04:03:54.650 INFO:teuthology.orchestra.run.smithi032.stdout:modified 2024-05-12T03:58:06.821591+0000 2024-05-12T04:03:54.650 INFO:teuthology.orchestra.run.smithi032.stdout:tableserver 0 2024-05-12T04:03:54.650 INFO:teuthology.orchestra.run.smithi032.stdout:root 0 2024-05-12T04:03:54.650 INFO:teuthology.orchestra.run.smithi032.stdout:session_timeout 60 2024-05-12T04:03:54.650 INFO:teuthology.orchestra.run.smithi032.stdout:session_autoclose 300 2024-05-12T04:03:54.650 INFO:teuthology.orchestra.run.smithi032.stdout:max_file_size 1099511627776 2024-05-12T04:03:54.650 INFO:teuthology.orchestra.run.smithi032.stdout:max_xattr_size 65536 2024-05-12T04:03:54.650 INFO:teuthology.orchestra.run.smithi032.stdout:required_client_features {} 2024-05-12T04:03:54.650 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure 0 2024-05-12T04:03:54.651 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure_osd_epoch 46 2024-05-12T04:03:54.651 INFO:teuthology.orchestra.run.smithi032.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-12T04:03:54.651 INFO:teuthology.orchestra.run.smithi032.stdout:max_mds 1 2024-05-12T04:03:54.651 INFO:teuthology.orchestra.run.smithi032.stdout:in 0 2024-05-12T04:03:54.651 INFO:teuthology.orchestra.run.smithi032.stdout:up {0=14474} 2024-05-12T04:03:54.651 INFO:teuthology.orchestra.run.smithi032.stdout:failed 2024-05-12T04:03:54.651 INFO:teuthology.orchestra.run.smithi032.stdout:damaged 2024-05-12T04:03:54.651 INFO:teuthology.orchestra.run.smithi032.stdout:stopped 2024-05-12T04:03:54.651 INFO:teuthology.orchestra.run.smithi032.stdout:data_pools [3] 2024-05-12T04:03:54.651 INFO:teuthology.orchestra.run.smithi032.stdout:metadata_pool 2 2024-05-12T04:03:54.651 INFO:teuthology.orchestra.run.smithi032.stdout:inline_data enabled 2024-05-12T04:03:54.651 INFO:teuthology.orchestra.run.smithi032.stdout:balancer 2024-05-12T04:03:54.651 INFO:teuthology.orchestra.run.smithi032.stdout:bal_rank_mask -1 2024-05-12T04:03:54.651 INFO:teuthology.orchestra.run.smithi032.stdout:standby_count_wanted 1 2024-05-12T04:03:54.652 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.qpsawh{0:14474} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.32:6828/3100155296,v1:172.21.15.32:6829/3100155296] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:03:54.652 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi033.jgbjhk{0:24267} state up:standby-replay seq 3 join_fscid=1 addr [v2:172.21.15.33:6824/55417471,v1:172.21.15.33:6825/55417471] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:03:54.652 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:03:54.652 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:03:54.652 INFO:teuthology.orchestra.run.smithi032.stdout:Standby daemons: 2024-05-12T04:03:54.652 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:03:54.652 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi033.hglwml{-1:24283} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/433858238,v1:172.21.15.33:6827/433858238] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:03:54.652 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.ialtqb{-1:24287} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.32:6826/3703132329,v1:172.21.15.32:6827/3703132329] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:03:54.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.651+0000 7f8b1de5c700 1 -- 172.21.15.32:0/2765969518 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f8b0006cd40 msgr2=0x7f8b0006f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:03:54.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.651+0000 7f8b1de5c700 1 --2- 172.21.15.32:0/2765969518 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f8b0006cd40 0x7f8b0006f1f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8b08007fd0 tx=0x7f8b08009040 comp rx=0 tx=0).stop 2024-05-12T04:03:54.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.651+0000 7f8b1de5c700 1 -- 172.21.15.32:0/2765969518 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b180fdff0 msgr2=0x7f8b1810de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:03:54.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.651+0000 7f8b1de5c700 1 --2- 172.21.15.32:0/2765969518 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b180fdff0 0x7f8b1810de90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c00b3b0 tx=0x7f8b0c009e30 comp rx=0 tx=0).stop 2024-05-12T04:03:54.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.651+0000 7f8b1de5c700 1 -- 172.21.15.32:0/2765969518 shutdown_connections 2024-05-12T04:03:54.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.651+0000 7f8b1de5c700 1 --2- 172.21.15.32:0/2765969518 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f8b0006cd40 0x7f8b0006f1f0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:54.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.651+0000 7f8b1de5c700 1 --2- 172.21.15.32:0/2765969518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b180fce80 0x7f8b1810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:54.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.651+0000 7f8b1de5c700 1 --2- 172.21.15.32:0/2765969518 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b180fdff0 0x7f8b1810de90 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:54.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.651+0000 7f8b1de5c700 1 -- 172.21.15.32:0/2765969518 >> 172.21.15.32:0/2765969518 conn(0x7f8b180f8520 msgr2=0x7f8b180f9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:03:54.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.651+0000 7f8b1de5c700 1 -- 172.21.15.32:0/2765969518 shutdown_connections 2024-05-12T04:03:54.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.651+0000 7f8b1de5c700 1 -- 172.21.15.32:0/2765969518 wait complete. 2024-05-12T04:03:54.654 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 18 2024-05-12T04:03:54.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:54 smithi033 ceph-mon[107305]: from='client.34156 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:03:54.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:54 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:54.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:54 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:54.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:54 smithi033 ceph-mon[107305]: Reconfiguring osd.2 (monmap changed)... 2024-05-12T04:03:54.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:54 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-12T04:03:54.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:54 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:03:54.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:54 smithi033 ceph-mon[107305]: Reconfiguring daemon osd.2 on smithi032 2024-05-12T04:03:54.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.762+0000 7fd19e295700 1 -- 172.21.15.32:0/1188435489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1980ffee0 msgr2=0x7fd198102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:03:54.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.762+0000 7fd19e295700 1 --2- 172.21.15.32:0/1188435489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1980ffee0 0x7fd198102300 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd18c004940 tx=0x7fd18c009e30 comp rx=0 tx=0).stop 2024-05-12T04:03:54.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.762+0000 7fd19e295700 1 -- 172.21.15.32:0/1188435489 shutdown_connections 2024-05-12T04:03:54.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.762+0000 7fd19e295700 1 --2- 172.21.15.32:0/1188435489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1980ffee0 0x7fd198102300 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:54.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.762+0000 7fd19e295700 1 --2- 172.21.15.32:0/1188435489 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1980662c0 0x7fd1980ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:54.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.762+0000 7fd19e295700 1 -- 172.21.15.32:0/1188435489 >> 172.21.15.32:0/1188435489 conn(0x7fd1980f7430 msgr2=0x7fd1980f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:03:54.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.762+0000 7fd19e295700 1 -- 172.21.15.32:0/1188435489 shutdown_connections 2024-05-12T04:03:54.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.762+0000 7fd19e295700 1 -- 172.21.15.32:0/1188435489 wait complete. 2024-05-12T04:03:54.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.763+0000 7fd19e295700 1 Processor -- start 2024-05-12T04:03:54.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.763+0000 7fd19e295700 1 -- start start 2024-05-12T04:03:54.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.763+0000 7fd19e295700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1980662c0 0x7fd19810d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:03:54.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.763+0000 7fd19e295700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1980ffee0 0x7fd19810de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:03:54.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.763+0000 7fd19e295700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19810e410 con 0x7fd1980662c0 2024-05-12T04:03:54.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.763+0000 7fd19e295700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19810ec60 con 0x7fd1980ffee0 2024-05-12T04:03:54.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.763+0000 7fd197fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1980662c0 0x7fd19810d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:03:54.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.763+0000 7fd197fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1980662c0 0x7fd19810d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52028/0 (socket says 172.21.15.32:52028) 2024-05-12T04:03:54.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.763+0000 7fd197fff700 1 -- 172.21.15.32:0/514917269 learned_addr learned my addr 172.21.15.32:0/514917269 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:03:54.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.764+0000 7fd1977fe700 1 --2- 172.21.15.32:0/514917269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1980ffee0 0x7fd19810de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:03:54.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.764+0000 7fd197fff700 1 -- 172.21.15.32:0/514917269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1980ffee0 msgr2=0x7fd19810de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:03:54.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.764+0000 7fd197fff700 1 --2- 172.21.15.32:0/514917269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1980ffee0 0x7fd19810de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:54.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.764+0000 7fd197fff700 1 -- 172.21.15.32:0/514917269 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd18c005040 con 0x7fd1980662c0 2024-05-12T04:03:54.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.764+0000 7fd197fff700 1 --2- 172.21.15.32:0/514917269 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1980662c0 0x7fd19810d8d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd18800eb40 tx=0x7fd18800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:03:54.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.764+0000 7fd194ff9700 1 -- 172.21.15.32:0/514917269 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd188007cc0 con 0x7fd1980662c0 2024-05-12T04:03:54.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.764+0000 7fd19e295700 1 -- 172.21.15.32:0/514917269 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd19810eee0 con 0x7fd1980662c0 2024-05-12T04:03:54.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.764+0000 7fd194ff9700 1 -- 172.21.15.32:0/514917269 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd18800f040 con 0x7fd1980662c0 2024-05-12T04:03:54.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.764+0000 7fd194ff9700 1 -- 172.21.15.32:0/514917269 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd18800c8f0 con 0x7fd1980662c0 2024-05-12T04:03:54.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.764+0000 7fd19e295700 1 -- 172.21.15.32:0/514917269 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd19810f420 con 0x7fd1980662c0 2024-05-12T04:03:54.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.766+0000 7fd194ff9700 1 -- 172.21.15.32:0/514917269 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 39) v1 ==== 90952+0+0 (secure 0 0 0) 0x7fd18800cb60 con 0x7fd1980662c0 2024-05-12T04:03:54.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.766+0000 7fd19e295700 1 -- 172.21.15.32:0/514917269 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd19805f5c0 con 0x7fd1980662c0 2024-05-12T04:03:54.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.766+0000 7fd194ff9700 1 --2- 172.21.15.32:0/514917269 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fd18006cd40 0x7fd18006f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:03:54.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.766+0000 7fd194ff9700 1 -- 172.21.15.32:0/514917269 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fd18808bce0 con 0x7fd1980662c0 2024-05-12T04:03:54.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.766+0000 7fd1977fe700 1 --2- 172.21.15.32:0/514917269 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fd18006cd40 0x7fd18006f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:03:54.767 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.767+0000 7fd1977fe700 1 --2- 172.21.15.32:0/514917269 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fd18006cd40 0x7fd18006f1f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd18c013680 tx=0x7fd18c009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:03:54.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:54.771+0000 7fd194ff9700 1 -- 172.21.15.32:0/514917269 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fd188059610 con 0x7fd1980662c0 2024-05-12T04:03:55.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:55.001+0000 7fd19e295700 1 -- 172.21.15.32:0/514917269 --> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd19810f7a0 con 0x7fd18006cd40 2024-05-12T04:03:55.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:55.002+0000 7fd194ff9700 1 -- 172.21.15.32:0/514917269 <== mgr.24593 v2:172.21.15.32:6800/2957111591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7fd19810f7a0 con 0x7fd18006cd40 2024-05-12T04:03:55.003 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-05-12T04:03:55.003 INFO:teuthology.orchestra.run.smithi032.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:19c9734256aa43a60e203feb242f50bf8b7f58666d520536af43cc4fee96173a", 2024-05-12T04:03:55.003 INFO:teuthology.orchestra.run.smithi032.stdout: "in_progress": true, 2024-05-12T04:03:55.003 INFO:teuthology.orchestra.run.smithi032.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-12T04:03:55.003 INFO:teuthology.orchestra.run.smithi032.stdout: "services_complete": [ 2024-05-12T04:03:55.003 INFO:teuthology.orchestra.run.smithi032.stdout: "mon", 2024-05-12T04:03:55.003 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr" 2024-05-12T04:03:55.003 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-05-12T04:03:55.003 INFO:teuthology.orchestra.run.smithi032.stdout: "progress": "4/21 daemons upgraded", 2024-05-12T04:03:55.003 INFO:teuthology.orchestra.run.smithi032.stdout: "message": "Currently upgrading mon daemons", 2024-05-12T04:03:55.003 INFO:teuthology.orchestra.run.smithi032.stdout: "is_paused": false 2024-05-12T04:03:55.003 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-05-12T04:03:55.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:55.005+0000 7fd19e295700 1 -- 172.21.15.32:0/514917269 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fd18006cd40 msgr2=0x7fd18006f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:03:55.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:55.005+0000 7fd19e295700 1 --2- 172.21.15.32:0/514917269 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fd18006cd40 0x7fd18006f1f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd18c013680 tx=0x7fd18c009e30 comp rx=0 tx=0).stop 2024-05-12T04:03:55.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:55.005+0000 7fd19e295700 1 -- 172.21.15.32:0/514917269 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1980662c0 msgr2=0x7fd19810d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:03:55.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:55.005+0000 7fd19e295700 1 --2- 172.21.15.32:0/514917269 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1980662c0 0x7fd19810d8d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd18800eb40 tx=0x7fd18800b040 comp rx=0 tx=0).stop 2024-05-12T04:03:55.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:55.005+0000 7fd19e295700 1 -- 172.21.15.32:0/514917269 shutdown_connections 2024-05-12T04:03:55.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:55.005+0000 7fd19e295700 1 --2- 172.21.15.32:0/514917269 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fd18006cd40 0x7fd18006f1f0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:55.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:55.005+0000 7fd19e295700 1 --2- 172.21.15.32:0/514917269 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1980662c0 0x7fd19810d8d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:55.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:55.005+0000 7fd19e295700 1 --2- 172.21.15.32:0/514917269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1980ffee0 0x7fd19810de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:55.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:55.005+0000 7fd19e295700 1 -- 172.21.15.32:0/514917269 >> 172.21.15.32:0/514917269 conn(0x7fd1980f7430 msgr2=0x7fd1980f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:03:55.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:55.005+0000 7fd19e295700 1 -- 172.21.15.32:0/514917269 shutdown_connections 2024-05-12T04:03:55.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:55.005+0000 7fd19e295700 1 -- 172.21.15.32:0/514917269 wait complete. 2024-05-12T04:03:55.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:55.115+0000 7f1cf4cae700 1 -- 172.21.15.32:0/269584234 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cf00fdfd0 msgr2=0x7f1cf00fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:03:55.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:55.115+0000 7f1cf4cae700 1 --2- 172.21.15.32:0/269584234 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cf00fdfd0 0x7f1cf00fe440 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1ce4004940 tx=0x7f1ce4009e30 comp rx=0 tx=0).stop 2024-05-12T04:03:55.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:55.115+0000 7f1cf4cae700 1 -- 172.21.15.32:0/269584234 shutdown_connections 2024-05-12T04:03:55.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:55.115+0000 7f1cf4cae700 1 --2- 172.21.15.32:0/269584234 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cf00fdfd0 0x7f1cf00fe440 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:55.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:55.115+0000 7f1cf4cae700 1 --2- 172.21.15.32:0/269584234 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cf00fce60 0x7f1cf00fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:55.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:55.115+0000 7f1cf4cae700 1 -- 172.21.15.32:0/269584234 >> 172.21.15.32:0/269584234 conn(0x7f1cf00f8560 msgr2=0x7f1cf00fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:03:55.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:55.115+0000 7f1cf4cae700 1 -- 172.21.15.32:0/269584234 shutdown_connections 2024-05-12T04:03:55.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:55.115+0000 7f1cf4cae700 1 -- 172.21.15.32:0/269584234 wait complete. 2024-05-12T04:03:55.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:55.116+0000 7f1cf4cae700 1 Processor -- start 2024-05-12T04:03:55.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:55.116+0000 7f1cf4cae700 1 -- start start 2024-05-12T04:03:55.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:55.116+0000 7f1cf4cae700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cf00fce60 0x7f1cf0071de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:03:55.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:55.116+0000 7f1cf4cae700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cf00fdfd0 0x7f1cf00702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:03:55.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:55.116+0000 7f1cf4cae700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf0070860 con 0x7f1cf00fce60 2024-05-12T04:03:55.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:55.116+0000 7f1cf4cae700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf00709d0 con 0x7f1cf00fdfd0 2024-05-12T04:03:55.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:55.116+0000 7f1cee59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cf00fce60 0x7f1cf0071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:03:55.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:55.116+0000 7f1cee59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cf00fce60 0x7f1cf0071de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52046/0 (socket says 172.21.15.32:52046) 2024-05-12T04:03:55.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:55.116+0000 7f1cee59c700 1 -- 172.21.15.32:0/3176258429 learned_addr learned my addr 172.21.15.32:0/3176258429 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:03:55.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:55.116+0000 7f1cee59c700 1 -- 172.21.15.32:0/3176258429 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cf00fdfd0 msgr2=0x7f1cf00702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:03:55.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:55.116+0000 7f1cee59c700 1 --2- 172.21.15.32:0/3176258429 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cf00fdfd0 0x7f1cf00702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:55.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:55.116+0000 7f1cee59c700 1 -- 172.21.15.32:0/3176258429 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ce4005040 con 0x7f1cf00fce60 2024-05-12T04:03:55.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:55.116+0000 7f1cee59c700 1 --2- 172.21.15.32:0/3176258429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cf00fce60 0x7f1cf0071de0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1ce0001d80 tx=0x7f1ce0009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:03:55.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:55.117+0000 7f1cdeffd700 1 -- 172.21.15.32:0/3176258429 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ce0007d10 con 0x7f1cf00fce60 2024-05-12T04:03:55.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:55.117+0000 7f1cdeffd700 1 -- 172.21.15.32:0/3176258429 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1ce0007e70 con 0x7f1cf00fce60 2024-05-12T04:03:55.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:55.117+0000 7f1cf4cae700 1 -- 172.21.15.32:0/3176258429 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cf0070cb0 con 0x7f1cf00fce60 2024-05-12T04:03:55.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:55.117+0000 7f1cdeffd700 1 -- 172.21.15.32:0/3176258429 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ce0014640 con 0x7f1cf00fce60 2024-05-12T04:03:55.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:55.117+0000 7f1cf4cae700 1 -- 172.21.15.32:0/3176258429 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cf0115950 con 0x7f1cf00fce60 2024-05-12T04:03:55.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:55.118+0000 7f1cf4cae700 1 -- 172.21.15.32:0/3176258429 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cf005f5c0 con 0x7f1cf00fce60 2024-05-12T04:03:55.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:55.119+0000 7f1cdeffd700 1 -- 172.21.15.32:0/3176258429 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 39) v1 ==== 90952+0+0 (secure 0 0 0) 0x7f1ce000e960 con 0x7f1cf00fce60 2024-05-12T04:03:55.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:55.119+0000 7f1cdeffd700 1 --2- 172.21.15.32:0/3176258429 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f1cd806cd40 0x7f1cd806f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:03:55.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:55.119+0000 7f1cdeffd700 1 -- 172.21.15.32:0/3176258429 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f1ce008ca80 con 0x7f1cf00fce60 2024-05-12T04:03:55.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:55.119+0000 7f1cedd9b700 1 --2- 172.21.15.32:0/3176258429 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f1cd806cd40 0x7f1cd806f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:03:55.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:55.119+0000 7f1cedd9b700 1 --2- 172.21.15.32:0/3176258429 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f1cd806cd40 0x7f1cd806f1f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1ce400b900 tx=0x7f1ce4009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:03:55.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:55.123+0000 7f1cdeffd700 1 -- 172.21.15.32:0/3176258429 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f1ce0056aa0 con 0x7f1cf00fce60 2024-05-12T04:03:55.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:55.418+0000 7f1cf4cae700 1 -- 172.21.15.32:0/3176258429 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f1cf0105780 con 0x7f1cf00fce60 2024-05-12T04:03:55.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:55.418+0000 7f1cdeffd700 1 -- 172.21.15.32:0/3176258429 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f1ce005a0c0 con 0x7f1cf00fce60 2024-05-12T04:03:55.419 INFO:teuthology.orchestra.run.smithi032.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-12T04:03:55.419 INFO:teuthology.orchestra.run.smithi032.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-12T04:03:55.419 INFO:teuthology.orchestra.run.smithi032.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-12T04:03:55.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:55.420+0000 7f1cf4cae700 1 -- 172.21.15.32:0/3176258429 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f1cd806cd40 msgr2=0x7f1cd806f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:03:55.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:55.420+0000 7f1cf4cae700 1 --2- 172.21.15.32:0/3176258429 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f1cd806cd40 0x7f1cd806f1f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1ce400b900 tx=0x7f1ce4009e30 comp rx=0 tx=0).stop 2024-05-12T04:03:55.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:55.420+0000 7f1cf4cae700 1 -- 172.21.15.32:0/3176258429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cf00fce60 msgr2=0x7f1cf0071de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:03:55.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:55.420+0000 7f1cf4cae700 1 --2- 172.21.15.32:0/3176258429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cf00fce60 0x7f1cf0071de0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1ce0001d80 tx=0x7f1ce0009ec0 comp rx=0 tx=0).stop 2024-05-12T04:03:55.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:55.420+0000 7f1cf4cae700 1 -- 172.21.15.32:0/3176258429 shutdown_connections 2024-05-12T04:03:55.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:55.420+0000 7f1cf4cae700 1 --2- 172.21.15.32:0/3176258429 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f1cd806cd40 0x7f1cd806f1f0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:55.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:55.420+0000 7f1cf4cae700 1 --2- 172.21.15.32:0/3176258429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cf00fce60 0x7f1cf0071de0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:55.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:55.420+0000 7f1cf4cae700 1 --2- 172.21.15.32:0/3176258429 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cf00fdfd0 0x7f1cf00702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:03:55.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:55.420+0000 7f1cf4cae700 1 -- 172.21.15.32:0/3176258429 >> 172.21.15.32:0/3176258429 conn(0x7f1cf00f8560 msgr2=0x7f1cf00f9d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:03:55.421 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:55.421+0000 7f1cf4cae700 1 -- 172.21.15.32:0/3176258429 shutdown_connections 2024-05-12T04:03:55.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:03:55.421+0000 7f1cf4cae700 1 -- 172.21.15.32:0/3176258429 wait complete. 2024-05-12T04:03:55.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:55 smithi032 ceph-mon[120392]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:03:55.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:55 smithi032 ceph-mon[120392]: pgmap v22: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:03:55.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:55 smithi032 ceph-mon[120392]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:03:55.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:55 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/954146210' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:03:55.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:55 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/2765969518' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-12T04:03:55.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:55 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:55.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:55 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:55.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:55 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi032.ialtqb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-12T04:03:55.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:55 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:03:55.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:55 smithi033 ceph-mon[107305]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:03:55.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:55 smithi033 ceph-mon[107305]: pgmap v22: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:03:55.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:55 smithi033 ceph-mon[107305]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:03:55.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:55 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/954146210' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:03:55.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:55 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/2765969518' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-12T04:03:55.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:55 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:55.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:55 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:55.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:55 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi032.ialtqb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-12T04:03:55.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:55 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:03:56.389 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:56 smithi032 ceph-mon[120392]: Reconfiguring mds.cephfs.smithi032.ialtqb (monmap changed)... 2024-05-12T04:03:56.389 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:56 smithi032 ceph-mon[120392]: Reconfiguring daemon mds.cephfs.smithi032.ialtqb on smithi032 2024-05-12T04:03:56.389 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:56 smithi032 ceph-mon[120392]: from='client.34176 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:03:56.389 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:56 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/3176258429' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-12T04:03:56.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:56 smithi033 ceph-mon[107305]: Reconfiguring mds.cephfs.smithi032.ialtqb (monmap changed)... 2024-05-12T04:03:56.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:56 smithi033 ceph-mon[107305]: Reconfiguring daemon mds.cephfs.smithi032.ialtqb on smithi032 2024-05-12T04:03:56.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:56 smithi033 ceph-mon[107305]: from='client.34176 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:03:56.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:56 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/3176258429' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-12T04:03:57.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:57 smithi032 ceph-mon[120392]: pgmap v23: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:03:57.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:57 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:57.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:57 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:57.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:57 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi032.qpsawh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-12T04:03:57.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:57 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:03:57.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:57 smithi033 ceph-mon[107305]: pgmap v23: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:03:57.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:57 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:57.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:57 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:57.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:57 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi032.qpsawh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-12T04:03:57.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:57 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:03:58.561 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:58 smithi033 ceph-mon[107305]: Reconfiguring mds.cephfs.smithi032.qpsawh (monmap changed)... 2024-05-12T04:03:58.561 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:58 smithi033 ceph-mon[107305]: Reconfiguring daemon mds.cephfs.smithi032.qpsawh on smithi032 2024-05-12T04:03:58.561 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:58 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:58.562 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:58 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:58.562 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:58 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi033", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-12T04:03:58.562 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:58 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:03:58.662 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:58 smithi032 ceph-mon[120392]: Reconfiguring mds.cephfs.smithi032.qpsawh (monmap changed)... 2024-05-12T04:03:58.662 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:58 smithi032 ceph-mon[120392]: Reconfiguring daemon mds.cephfs.smithi032.qpsawh on smithi032 2024-05-12T04:03:58.662 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:58 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:58.662 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:58 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:03:58.662 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:58 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi033", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-12T04:03:58.662 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:58 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:03:59.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:59 smithi033 ceph-mon[107305]: pgmap v24: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:03:59.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:59 smithi033 ceph-mon[107305]: Reconfiguring crash.smithi033 (monmap changed)... 2024-05-12T04:03:59.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:03:59 smithi033 ceph-mon[107305]: Reconfiguring daemon crash.smithi033 on smithi033 2024-05-12T04:03:59.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:59 smithi032 ceph-mon[120392]: pgmap v24: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:03:59.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:59 smithi032 ceph-mon[120392]: Reconfiguring crash.smithi033 (monmap changed)... 2024-05-12T04:03:59.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:03:59 smithi032 ceph-mon[120392]: Reconfiguring daemon crash.smithi033 on smithi033 2024-05-12T04:04:01.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:00 smithi033 ceph-mon[107305]: pgmap v25: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:04:01.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:00 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:01.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:00 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:01.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:00 smithi033 ceph-mon[107305]: Reconfiguring mgr.smithi033.pcvxwy (monmap changed)... 2024-05-12T04:04:01.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:00 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi033.pcvxwy", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-12T04:04:01.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:00 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-12T04:04:01.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:00 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:04:01.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:00 smithi033 ceph-mon[107305]: Reconfiguring daemon mgr.smithi033.pcvxwy on smithi033 2024-05-12T04:04:01.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:00 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:04:01.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:00 smithi032 ceph-mon[120392]: pgmap v25: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:04:01.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:00 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:01.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:00 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:01.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:00 smithi032 ceph-mon[120392]: Reconfiguring mgr.smithi033.pcvxwy (monmap changed)... 2024-05-12T04:04:01.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:00 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi033.pcvxwy", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-12T04:04:01.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:00 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-12T04:04:01.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:00 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:04:01.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:00 smithi032 ceph-mon[120392]: Reconfiguring daemon mgr.smithi033.pcvxwy on smithi033 2024-05-12T04:04:01.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:00 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:04:02.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:02 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:02.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:02 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:02.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:02 smithi033 ceph-mon[107305]: Reconfiguring mon.smithi033 (monmap changed)... 2024-05-12T04:04:02.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:02 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-12T04:04:02.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:02 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-12T04:04:02.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:02 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:04:02.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:02 smithi033 ceph-mon[107305]: Reconfiguring daemon mon.smithi033 on smithi033 2024-05-12T04:04:02.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:02 smithi033 ceph-mon[107305]: pgmap v26: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:04:02.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:02 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:02.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:02 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:02.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:02 smithi032 ceph-mon[120392]: Reconfiguring mon.smithi033 (monmap changed)... 2024-05-12T04:04:02.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:02 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-12T04:04:02.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:02 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-12T04:04:02.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:02 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:04:02.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:02 smithi032 ceph-mon[120392]: Reconfiguring daemon mon.smithi033 on smithi033 2024-05-12T04:04:02.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:02 smithi032 ceph-mon[120392]: pgmap v26: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:04:04.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:04 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:04.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:04 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:04.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:04 smithi033 ceph-mon[107305]: Reconfiguring osd.3 (monmap changed)... 2024-05-12T04:04:04.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:04 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-12T04:04:04.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:04 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:04:04.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:04 smithi033 ceph-mon[107305]: Reconfiguring daemon osd.3 on smithi033 2024-05-12T04:04:04.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:04 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:04.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:04 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:04.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:04 smithi032 ceph-mon[120392]: Reconfiguring osd.3 (monmap changed)... 2024-05-12T04:04:04.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:04 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-12T04:04:04.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:04 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:04:04.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:04 smithi032 ceph-mon[120392]: Reconfiguring daemon osd.3 on smithi033 2024-05-12T04:04:05.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:05 smithi033 ceph-mon[107305]: pgmap v27: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:04:05.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:05 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:05.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:05 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:05.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:05 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-12T04:04:05.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:05 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:04:05.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:05 smithi032 ceph-mon[120392]: pgmap v27: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:04:05.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:05 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:05.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:05 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:05.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:05 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-12T04:04:05.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:05 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:04:06.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:06 smithi033 ceph-mon[107305]: Reconfiguring osd.4 (monmap changed)... 2024-05-12T04:04:06.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:06 smithi033 ceph-mon[107305]: Reconfiguring daemon osd.4 on smithi033 2024-05-12T04:04:06.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:06 smithi032 ceph-mon[120392]: Reconfiguring osd.4 (monmap changed)... 2024-05-12T04:04:06.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:06 smithi032 ceph-mon[120392]: Reconfiguring daemon osd.4 on smithi033 2024-05-12T04:04:07.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:07 smithi033 ceph-mon[107305]: pgmap v28: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:04:07.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:07 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:07.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:07 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:07.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:07 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-12T04:04:07.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:07 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:04:07.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:07 smithi032 ceph-mon[120392]: pgmap v28: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:04:07.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:07 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:07.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:07 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:07.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:07 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-12T04:04:07.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:07 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:04:08.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:08 smithi033 ceph-mon[107305]: Reconfiguring osd.5 (monmap changed)... 2024-05-12T04:04:08.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:08 smithi033 ceph-mon[107305]: Reconfiguring daemon osd.5 on smithi033 2024-05-12T04:04:08.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:08 smithi032 ceph-mon[120392]: Reconfiguring osd.5 (monmap changed)... 2024-05-12T04:04:08.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:08 smithi032 ceph-mon[120392]: Reconfiguring daemon osd.5 on smithi033 2024-05-12T04:04:09.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:09 smithi033 ceph-mon[107305]: pgmap v29: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:04:09.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:09 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:09.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:09 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:09.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:09 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi033.jgbjhk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-12T04:04:09.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:09 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:04:09.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:09 smithi032 ceph-mon[120392]: pgmap v29: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:04:09.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:09 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:09.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:09 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:09.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:09 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi033.jgbjhk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-12T04:04:09.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:09 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:04:10.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:10 smithi032 ceph-mon[120392]: Reconfiguring mds.cephfs.smithi033.jgbjhk (monmap changed)... 2024-05-12T04:04:10.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:10 smithi032 ceph-mon[120392]: Reconfiguring daemon mds.cephfs.smithi033.jgbjhk on smithi033 2024-05-12T04:04:10.569 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:10 smithi033 ceph-mon[107305]: Reconfiguring mds.cephfs.smithi033.jgbjhk (monmap changed)... 2024-05-12T04:04:10.569 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:10 smithi033 ceph-mon[107305]: Reconfiguring daemon mds.cephfs.smithi033.jgbjhk on smithi033 2024-05-12T04:04:11.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:11 smithi032 ceph-mon[120392]: pgmap v30: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:04:11.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:11 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:11.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:11 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:11.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:11 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi033.hglwml", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-12T04:04:11.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:11 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:04:11.567 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:11 smithi033 ceph-mon[107305]: pgmap v30: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:04:11.567 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:11 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:11.567 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:11 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:11.568 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:11 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi033.hglwml", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-12T04:04:11.568 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:11 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:04:12.268 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:12 smithi033 ceph-mon[107305]: Reconfiguring mds.cephfs.smithi033.hglwml (monmap changed)... 2024-05-12T04:04:12.268 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:12 smithi033 ceph-mon[107305]: Reconfiguring daemon mds.cephfs.smithi033.hglwml on smithi033 2024-05-12T04:04:12.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:12 smithi032 ceph-mon[120392]: Reconfiguring mds.cephfs.smithi033.hglwml (monmap changed)... 2024-05-12T04:04:12.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:12 smithi032 ceph-mon[120392]: Reconfiguring daemon mds.cephfs.smithi033.hglwml on smithi033 2024-05-12T04:04:13.503 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:13 smithi032 ceph-mon[120392]: pgmap v31: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:04:13.503 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:13 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:13.503 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:13 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:13.503 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:13 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:04:13.503 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:13 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:04:13.503 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:13 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:04:13.503 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:13 smithi032 ceph-mon[120392]: Upgrade: Setting container_image for all mon 2024-05-12T04:04:13.504 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:13 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:13.504 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:13 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi032"}]: dispatch 2024-05-12T04:04:13.504 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:13 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi032"}]': finished 2024-05-12T04:04:13.504 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:13 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi033"}]: dispatch 2024-05-12T04:04:13.504 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:13 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi033"}]': finished 2024-05-12T04:04:13.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:13 smithi033 ceph-mon[107305]: pgmap v31: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:04:13.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:13 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:13.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:13 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:13.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:13 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:04:13.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:13 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:04:13.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:13 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:04:13.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:13 smithi033 ceph-mon[107305]: Upgrade: Setting container_image for all mon 2024-05-12T04:04:13.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:13 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:13.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:13 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi032"}]: dispatch 2024-05-12T04:04:13.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:13 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi032"}]': finished 2024-05-12T04:04:13.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:13 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi033"}]: dispatch 2024-05-12T04:04:13.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:13 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi033"}]': finished 2024-05-12T04:04:15.076 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:14 smithi032 ceph-mon[120392]: pgmap v32: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:04:15.076 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:14 smithi032 ceph-mon[120392]: Upgrade: Updating crash.smithi032 (1/2) 2024-05-12T04:04:15.076 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:14 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:15.076 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:14 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi032", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-12T04:04:15.076 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:14 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:04:15.076 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:14 smithi032 ceph-mon[120392]: Deploying daemon crash.smithi032 on smithi032 2024-05-12T04:04:15.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:14 smithi033 ceph-mon[107305]: pgmap v32: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:04:15.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:14 smithi033 ceph-mon[107305]: Upgrade: Updating crash.smithi032 (1/2) 2024-05-12T04:04:15.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:14 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:15.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:14 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi032", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-12T04:04:15.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:14 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:04:15.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:14 smithi033 ceph-mon[107305]: Deploying daemon crash.smithi032 on smithi032 2024-05-12T04:04:16.155 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:15 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:04:16.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:15 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:04:17.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:16 smithi032 ceph-mon[120392]: pgmap v33: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:04:17.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:16 smithi033 ceph-mon[107305]: pgmap v33: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:04:18.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:18.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:18.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:18 smithi032 ceph-mon[120392]: pgmap v34: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:04:18.835 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:18.835 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:18.835 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:18 smithi033 ceph-mon[107305]: pgmap v34: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:04:20.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:20 smithi033 ceph-mon[107305]: Upgrade: Updating crash.smithi033 (2/2) 2024-05-12T04:04:20.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:20 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:20.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:20 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi033", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-12T04:04:20.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:20 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:04:20.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:20 smithi033 ceph-mon[107305]: Deploying daemon crash.smithi033 on smithi033 2024-05-12T04:04:20.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:20 smithi032 ceph-mon[120392]: Upgrade: Updating crash.smithi033 (2/2) 2024-05-12T04:04:20.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:20 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:20.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:20 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi033", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-05-12T04:04:20.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:20 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:04:20.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:20 smithi032 ceph-mon[120392]: Deploying daemon crash.smithi033 on smithi033 2024-05-12T04:04:21.368 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:21 smithi033 ceph-mon[107305]: pgmap v35: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:04:21.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:21 smithi032 ceph-mon[120392]: pgmap v35: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:04:23.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:23 smithi032 ceph-mon[120392]: pgmap v36: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:04:23.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:23 smithi033 ceph-mon[107305]: pgmap v36: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:04:24.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:24 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:24.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:24 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:24.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:24 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:04:24.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:24 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:24.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:24 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:24.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:24 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:04:25.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.542+0000 7fa9fbc23700 1 -- 172.21.15.32:0/2163725872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9f40fbcb0 msgr2=0x7fa9f40fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:04:25.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.542+0000 7fa9fbc23700 1 --2- 172.21.15.32:0/2163725872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9f40fbcb0 0x7fa9f40fc0c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0004940 tx=0x7fa9f0009e30 comp rx=0 tx=0).stop 2024-05-12T04:04:25.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.542+0000 7fa9fbc23700 1 -- 172.21.15.32:0/2163725872 shutdown_connections 2024-05-12T04:04:25.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.542+0000 7fa9fbc23700 1 --2- 172.21.15.32:0/2163725872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9f40fcf60 0x7fa9f40fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:25.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.542+0000 7fa9fbc23700 1 --2- 172.21.15.32:0/2163725872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9f40fbcb0 0x7fa9f40fc0c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:25.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.542+0000 7fa9fbc23700 1 -- 172.21.15.32:0/2163725872 >> 172.21.15.32:0/2163725872 conn(0x7fa9f40f7430 msgr2=0x7fa9f40f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:04:25.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.542+0000 7fa9fbc23700 1 -- 172.21.15.32:0/2163725872 shutdown_connections 2024-05-12T04:04:25.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.542+0000 7fa9fbc23700 1 -- 172.21.15.32:0/2163725872 wait complete. 2024-05-12T04:04:25.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.542+0000 7fa9fbc23700 1 Processor -- start 2024-05-12T04:04:25.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.542+0000 7fa9fbc23700 1 -- start start 2024-05-12T04:04:25.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.543+0000 7fa9fbc23700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9f40fbcb0 0x7fa9f41105b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:04:25.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.543+0000 7fa9fbc23700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9f40fcf60 0x7fa9f4112b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:04:25.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.543+0000 7fa9fbc23700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f4066330 con 0x7fa9f40fcf60 2024-05-12T04:04:25.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.543+0000 7fa9fbc23700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f4066470 con 0x7fa9f40fbcb0 2024-05-12T04:04:25.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.543+0000 7fa9f91be700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9f40fcf60 0x7fa9f4112b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:04:25.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.543+0000 7fa9f91be700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9f40fcf60 0x7fa9f4112b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38546/0 (socket says 172.21.15.32:38546) 2024-05-12T04:04:25.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.543+0000 7fa9f91be700 1 -- 172.21.15.32:0/1308776708 learned_addr learned my addr 172.21.15.32:0/1308776708 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:04:25.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.543+0000 7fa9f99bf700 1 --2- 172.21.15.32:0/1308776708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9f40fbcb0 0x7fa9f41105b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:04:25.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.543+0000 7fa9f91be700 1 -- 172.21.15.32:0/1308776708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9f40fbcb0 msgr2=0x7fa9f41105b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:04:25.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.543+0000 7fa9f91be700 1 --2- 172.21.15.32:0/1308776708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9f40fbcb0 0x7fa9f41105b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:25.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.543+0000 7fa9f91be700 1 -- 172.21.15.32:0/1308776708 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f0005040 con 0x7fa9f40fcf60 2024-05-12T04:04:25.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.543+0000 7fa9f91be700 1 --2- 172.21.15.32:0/1308776708 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9f40fcf60 0x7fa9f4112b00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa9e800eb40 tx=0x7fa9e800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:04:25.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.544+0000 7fa9e67fc700 1 -- 172.21.15.32:0/1308776708 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa9e8007cc0 con 0x7fa9f40fcf60 2024-05-12T04:04:25.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.544+0000 7fa9fbc23700 1 -- 172.21.15.32:0/1308776708 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9f4113040 con 0x7fa9f40fcf60 2024-05-12T04:04:25.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.544+0000 7fa9e67fc700 1 -- 172.21.15.32:0/1308776708 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa9e800f040 con 0x7fa9f40fcf60 2024-05-12T04:04:25.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.544+0000 7fa9e67fc700 1 -- 172.21.15.32:0/1308776708 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa9e800c990 con 0x7fa9f40fcf60 2024-05-12T04:04:25.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.544+0000 7fa9fbc23700 1 -- 172.21.15.32:0/1308776708 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9f4113600 con 0x7fa9f40fcf60 2024-05-12T04:04:25.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.545+0000 7fa9fbc23700 1 -- 172.21.15.32:0/1308776708 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9f405f5c0 con 0x7fa9f40fcf60 2024-05-12T04:04:25.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.545+0000 7fa9e67fc700 1 -- 172.21.15.32:0/1308776708 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 39) v1 ==== 90952+0+0 (secure 0 0 0) 0x7fa9e800cc00 con 0x7fa9f40fcf60 2024-05-12T04:04:25.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.546+0000 7fa9e67fc700 1 --2- 172.21.15.32:0/1308776708 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fa9e006cd40 0x7fa9e006f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:04:25.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.546+0000 7fa9e67fc700 1 -- 172.21.15.32:0/1308776708 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fa9e808be90 con 0x7fa9f40fcf60 2024-05-12T04:04:25.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.546+0000 7fa9f99bf700 1 --2- 172.21.15.32:0/1308776708 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fa9e006cd40 0x7fa9e006f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:04:25.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.546+0000 7fa9f99bf700 1 --2- 172.21.15.32:0/1308776708 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fa9e006cd40 0x7fa9e006f1f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0013680 tx=0x7fa9f0009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:04:25.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.551+0000 7fa9e67fc700 1 -- 172.21.15.32:0/1308776708 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fa9e8055d30 con 0x7fa9f40fcf60 2024-05-12T04:04:25.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:25 smithi033 ceph-mon[107305]: pgmap v37: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:04:25.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.775+0000 7fa9fbc23700 1 -- 172.21.15.32:0/1308776708 --> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa9f4101320 con 0x7fa9e006cd40 2024-05-12T04:04:25.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.776+0000 7fa9e67fc700 1 -- 172.21.15.32:0/1308776708 <== mgr.24593 v2:172.21.15.32:6800/2957111591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+385 (secure 0 0 0) 0x7fa9f4101320 con 0x7fa9e006cd40 2024-05-12T04:04:25.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.778+0000 7fa9fbc23700 1 -- 172.21.15.32:0/1308776708 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fa9e006cd40 msgr2=0x7fa9e006f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:04:25.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.778+0000 7fa9fbc23700 1 --2- 172.21.15.32:0/1308776708 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fa9e006cd40 0x7fa9e006f1f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0013680 tx=0x7fa9f0009e30 comp rx=0 tx=0).stop 2024-05-12T04:04:25.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.778+0000 7fa9fbc23700 1 -- 172.21.15.32:0/1308776708 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9f40fcf60 msgr2=0x7fa9f4112b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:04:25.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.778+0000 7fa9fbc23700 1 --2- 172.21.15.32:0/1308776708 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9f40fcf60 0x7fa9f4112b00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa9e800eb40 tx=0x7fa9e800b040 comp rx=0 tx=0).stop 2024-05-12T04:04:25.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.778+0000 7fa9fbc23700 1 -- 172.21.15.32:0/1308776708 shutdown_connections 2024-05-12T04:04:25.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.778+0000 7fa9fbc23700 1 --2- 172.21.15.32:0/1308776708 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fa9e006cd40 0x7fa9e006f1f0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:25.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.778+0000 7fa9fbc23700 1 --2- 172.21.15.32:0/1308776708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9f40fbcb0 0x7fa9f41105b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:25.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.778+0000 7fa9fbc23700 1 --2- 172.21.15.32:0/1308776708 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9f40fcf60 0x7fa9f4112b00 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:25.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.778+0000 7fa9fbc23700 1 -- 172.21.15.32:0/1308776708 >> 172.21.15.32:0/1308776708 conn(0x7fa9f40f7430 msgr2=0x7fa9f40f8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:04:25.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.778+0000 7fa9fbc23700 1 -- 172.21.15.32:0/1308776708 shutdown_connections 2024-05-12T04:04:25.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.778+0000 7fa9fbc23700 1 -- 172.21.15.32:0/1308776708 wait complete. 2024-05-12T04:04:25.789 INFO:teuthology.orchestra.run.smithi032.stdout:true 2024-05-12T04:04:25.789 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:25 smithi032 ceph-mon[120392]: pgmap v37: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:04:25.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.883+0000 7fb4b1d20700 1 -- 172.21.15.32:0/1518918262 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4ac0fdfd0 msgr2=0x7fb4ac0fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:04:25.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.883+0000 7fb4b1d20700 1 --2- 172.21.15.32:0/1518918262 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4ac0fdfd0 0x7fb4ac0fe440 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb4a0004940 tx=0x7fb4a0009e30 comp rx=0 tx=0).stop 2024-05-12T04:04:25.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.884+0000 7fb4b1d20700 1 -- 172.21.15.32:0/1518918262 shutdown_connections 2024-05-12T04:04:25.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.884+0000 7fb4b1d20700 1 --2- 172.21.15.32:0/1518918262 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4ac0fdfd0 0x7fb4ac0fe440 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:25.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.884+0000 7fb4b1d20700 1 --2- 172.21.15.32:0/1518918262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4ac0fce60 0x7fb4ac0fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:25.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.884+0000 7fb4b1d20700 1 -- 172.21.15.32:0/1518918262 >> 172.21.15.32:0/1518918262 conn(0x7fb4ac0f8540 msgr2=0x7fb4ac0fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:04:25.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.884+0000 7fb4b1d20700 1 -- 172.21.15.32:0/1518918262 shutdown_connections 2024-05-12T04:04:25.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.884+0000 7fb4b1d20700 1 -- 172.21.15.32:0/1518918262 wait complete. 2024-05-12T04:04:25.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.884+0000 7fb4b1d20700 1 Processor -- start 2024-05-12T04:04:25.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.884+0000 7fb4b1d20700 1 -- start start 2024-05-12T04:04:25.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.884+0000 7fb4b1d20700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4ac0fce60 0x7fb4ac10d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:04:25.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.885+0000 7fb4b1d20700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4ac0fdfd0 0x7fb4ac10de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:04:25.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.885+0000 7fb4b1d20700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4ac10e480 con 0x7fb4ac0fce60 2024-05-12T04:04:25.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.885+0000 7fb4b1d20700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4ac10ecd0 con 0x7fb4ac0fdfd0 2024-05-12T04:04:25.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.885+0000 7fb4ab7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4ac0fce60 0x7fb4ac10d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:04:25.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.885+0000 7fb4ab7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4ac0fce60 0x7fb4ac10d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38568/0 (socket says 172.21.15.32:38568) 2024-05-12T04:04:25.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.885+0000 7fb4ab7fe700 1 -- 172.21.15.32:0/360681442 learned_addr learned my addr 172.21.15.32:0/360681442 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:04:25.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.885+0000 7fb4aaffd700 1 --2- 172.21.15.32:0/360681442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4ac0fdfd0 0x7fb4ac10de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:04:25.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.885+0000 7fb4ab7fe700 1 -- 172.21.15.32:0/360681442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4ac0fdfd0 msgr2=0x7fb4ac10de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:04:25.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.885+0000 7fb4ab7fe700 1 --2- 172.21.15.32:0/360681442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4ac0fdfd0 0x7fb4ac10de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:25.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.885+0000 7fb4ab7fe700 1 -- 172.21.15.32:0/360681442 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4a0005040 con 0x7fb4ac0fce60 2024-05-12T04:04:25.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.885+0000 7fb4ab7fe700 1 --2- 172.21.15.32:0/360681442 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4ac0fce60 0x7fb4ac10d940 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb49c00c830 tx=0x7fb49c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:04:25.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.885+0000 7fb493fff700 1 -- 172.21.15.32:0/360681442 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb49c0106f0 con 0x7fb4ac0fce60 2024-05-12T04:04:25.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.885+0000 7fb4b1d20700 1 -- 172.21.15.32:0/360681442 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4ac10efb0 con 0x7fb4ac0fce60 2024-05-12T04:04:25.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.885+0000 7fb493fff700 1 -- 172.21.15.32:0/360681442 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb49c010d00 con 0x7fb4ac0fce60 2024-05-12T04:04:25.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.886+0000 7fb493fff700 1 -- 172.21.15.32:0/360681442 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb49c012e90 con 0x7fb4ac0fce60 2024-05-12T04:04:25.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.886+0000 7fb4b1d20700 1 -- 172.21.15.32:0/360681442 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4ac10f4f0 con 0x7fb4ac0fce60 2024-05-12T04:04:25.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.887+0000 7fb4b1d20700 1 -- 172.21.15.32:0/360681442 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4ac05f5c0 con 0x7fb4ac0fce60 2024-05-12T04:04:25.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.887+0000 7fb493fff700 1 -- 172.21.15.32:0/360681442 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 39) v1 ==== 90952+0+0 (secure 0 0 0) 0x7fb49c010850 con 0x7fb4ac0fce60 2024-05-12T04:04:25.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.887+0000 7fb493fff700 1 --2- 172.21.15.32:0/360681442 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fb49406cd40 0x7fb49406f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:04:25.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.888+0000 7fb493fff700 1 -- 172.21.15.32:0/360681442 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fb49c08c130 con 0x7fb4ac0fce60 2024-05-12T04:04:25.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.888+0000 7fb4aaffd700 1 --2- 172.21.15.32:0/360681442 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fb49406cd40 0x7fb49406f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:04:25.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.888+0000 7fb4aaffd700 1 --2- 172.21.15.32:0/360681442 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fb49406cd40 0x7fb49406f1f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb4a0009fd0 tx=0x7fb4a0009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:04:25.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:25.892+0000 7fb493fff700 1 -- 172.21.15.32:0/360681442 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fb49c059b40 con 0x7fb4ac0fce60 2024-05-12T04:04:26.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.118+0000 7fb4b1d20700 1 -- 172.21.15.32:0/360681442 --> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb4ac10f8d0 con 0x7fb49406cd40 2024-05-12T04:04:26.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.120+0000 7fb493fff700 1 -- 172.21.15.32:0/360681442 <== mgr.24593 v2:172.21.15.32:6800/2957111591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+385 (secure 0 0 0) 0x7fb4ac10f8d0 con 0x7fb49406cd40 2024-05-12T04:04:26.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.122+0000 7fb4b1d20700 1 -- 172.21.15.32:0/360681442 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fb49406cd40 msgr2=0x7fb49406f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:04:26.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.122+0000 7fb4b1d20700 1 --2- 172.21.15.32:0/360681442 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fb49406cd40 0x7fb49406f1f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb4a0009fd0 tx=0x7fb4a0009e30 comp rx=0 tx=0).stop 2024-05-12T04:04:26.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.122+0000 7fb4b1d20700 1 -- 172.21.15.32:0/360681442 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4ac0fce60 msgr2=0x7fb4ac10d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:04:26.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.122+0000 7fb4b1d20700 1 --2- 172.21.15.32:0/360681442 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4ac0fce60 0x7fb4ac10d940 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb49c00c830 tx=0x7fb49c004470 comp rx=0 tx=0).stop 2024-05-12T04:04:26.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.122+0000 7fb4b1d20700 1 -- 172.21.15.32:0/360681442 shutdown_connections 2024-05-12T04:04:26.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.122+0000 7fb4b1d20700 1 --2- 172.21.15.32:0/360681442 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fb49406cd40 0x7fb49406f1f0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:26.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.122+0000 7fb4b1d20700 1 --2- 172.21.15.32:0/360681442 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4ac0fce60 0x7fb4ac10d940 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:26.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.122+0000 7fb4b1d20700 1 --2- 172.21.15.32:0/360681442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4ac0fdfd0 0x7fb4ac10de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:26.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.122+0000 7fb4b1d20700 1 -- 172.21.15.32:0/360681442 >> 172.21.15.32:0/360681442 conn(0x7fb4ac0f8540 msgr2=0x7fb4ac0fa050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:04:26.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.122+0000 7fb4b1d20700 1 -- 172.21.15.32:0/360681442 shutdown_connections 2024-05-12T04:04:26.123 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.122+0000 7fb4b1d20700 1 -- 172.21.15.32:0/360681442 wait complete. 2024-05-12T04:04:26.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.227+0000 7fcdbb962700 1 -- 172.21.15.32:0/2655270967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdb4070940 msgr2=0x7fcdb4070db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:04:26.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.227+0000 7fcdbb962700 1 --2- 172.21.15.32:0/2655270967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdb4070940 0x7fcdb4070db0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0004940 tx=0x7fcdb0009e30 comp rx=0 tx=0).stop 2024-05-12T04:04:26.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.227+0000 7fcdbb962700 1 -- 172.21.15.32:0/2655270967 shutdown_connections 2024-05-12T04:04:26.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.227+0000 7fcdbb962700 1 --2- 172.21.15.32:0/2655270967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdb4070940 0x7fcdb4070db0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:26.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.227+0000 7fcdbb962700 1 --2- 172.21.15.32:0/2655270967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdb4071ef0 0x7fcdb4070400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:26.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.227+0000 7fcdbb962700 1 -- 172.21.15.32:0/2655270967 >> 172.21.15.32:0/2655270967 conn(0x7fcdb40f8560 msgr2=0x7fcdb40fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:04:26.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.227+0000 7fcdbb962700 1 -- 172.21.15.32:0/2655270967 shutdown_connections 2024-05-12T04:04:26.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.227+0000 7fcdbb962700 1 -- 172.21.15.32:0/2655270967 wait complete. 2024-05-12T04:04:26.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.227+0000 7fcdbb962700 1 Processor -- start 2024-05-12T04:04:26.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.227+0000 7fcdbb962700 1 -- start start 2024-05-12T04:04:26.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.228+0000 7fcdbb962700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdb4070940 0x7fcdb410fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:04:26.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.228+0000 7fcdbb962700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdb4071ef0 0x7fcdb4110090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:04:26.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.228+0000 7fcdbb962700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb4110720 con 0x7fcdb4070940 2024-05-12T04:04:26.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.228+0000 7fcdbb962700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb4110f10 con 0x7fcdb4071ef0 2024-05-12T04:04:26.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.228+0000 7fcdb96fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdb4070940 0x7fcdb410fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:04:26.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.228+0000 7fcdb96fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdb4070940 0x7fcdb410fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38584/0 (socket says 172.21.15.32:38584) 2024-05-12T04:04:26.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.228+0000 7fcdb96fe700 1 -- 172.21.15.32:0/2627433141 learned_addr learned my addr 172.21.15.32:0/2627433141 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:04:26.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.228+0000 7fcdb8efd700 1 --2- 172.21.15.32:0/2627433141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdb4071ef0 0x7fcdb4110090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:04:26.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.228+0000 7fcdb96fe700 1 -- 172.21.15.32:0/2627433141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdb4071ef0 msgr2=0x7fcdb4110090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:04:26.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.228+0000 7fcdb96fe700 1 --2- 172.21.15.32:0/2627433141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdb4071ef0 0x7fcdb4110090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:26.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.228+0000 7fcdb96fe700 1 -- 172.21.15.32:0/2627433141 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdb0005040 con 0x7fcdb4070940 2024-05-12T04:04:26.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.228+0000 7fcdb96fe700 1 --2- 172.21.15.32:0/2627433141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdb4070940 0x7fcdb410fb50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcda400b790 tx=0x7fcda4009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:04:26.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.228+0000 7fcda9ffb700 1 -- 172.21.15.32:0/2627433141 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcda400f6f0 con 0x7fcdb4070940 2024-05-12T04:04:26.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.228+0000 7fcdbb962700 1 -- 172.21.15.32:0/2627433141 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdb41111f0 con 0x7fcdb4070940 2024-05-12T04:04:26.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.228+0000 7fcda9ffb700 1 -- 172.21.15.32:0/2627433141 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcda400fd00 con 0x7fcdb4070940 2024-05-12T04:04:26.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.228+0000 7fcda9ffb700 1 -- 172.21.15.32:0/2627433141 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcda401add0 con 0x7fcdb4070940 2024-05-12T04:04:26.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.229+0000 7fcdbb962700 1 -- 172.21.15.32:0/2627433141 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdb41117b0 con 0x7fcdb4070940 2024-05-12T04:04:26.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.229+0000 7fcdbb962700 1 -- 172.21.15.32:0/2627433141 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdb405f5c0 con 0x7fcdb4070940 2024-05-12T04:04:26.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.230+0000 7fcda9ffb700 1 -- 172.21.15.32:0/2627433141 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 39) v1 ==== 90952+0+0 (secure 0 0 0) 0x7fcda401a600 con 0x7fcdb4070940 2024-05-12T04:04:26.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.230+0000 7fcda9ffb700 1 --2- 172.21.15.32:0/2627433141 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fcda006cd40 0x7fcda006f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:04:26.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.230+0000 7fcda9ffb700 1 -- 172.21.15.32:0/2627433141 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fcda408bfe0 con 0x7fcdb4070940 2024-05-12T04:04:26.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.230+0000 7fcdb8efd700 1 --2- 172.21.15.32:0/2627433141 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fcda006cd40 0x7fcda006f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:04:26.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.231+0000 7fcdb8efd700 1 --2- 172.21.15.32:0/2627433141 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fcda006cd40 0x7fcda006f1f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcdb000b3b0 tx=0x7fcdb0009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:04:26.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.235+0000 7fcda9ffb700 1 -- 172.21.15.32:0/2627433141 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fcda4059970 con 0x7fcdb4070940 2024-05-12T04:04:26.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.450+0000 7fcdbb962700 1 -- 172.21.15.32:0/2627433141 --> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fcdb4111ad0 con 0x7fcda006cd40 2024-05-12T04:04:26.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.460+0000 7fcda9ffb700 1 -- 172.21.15.32:0/2627433141 <== mgr.24593 v2:172.21.15.32:6800/2957111591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fcdb4111ad0 con 0x7fcda006cd40 2024-05-12T04:04:26.460 INFO:teuthology.orchestra.run.smithi032.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-12T04:04:26.461 INFO:teuthology.orchestra.run.smithi032.stdout:alertmanager.smithi032 smithi032 *:9093,9094 running (3m) 62s ago 12m 17.8M - 0.25.0 c8568f914cd2 a37fbab7af4c 2024-05-12T04:04:26.461 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi032 smithi032 starting - - - - 2024-05-12T04:04:26.461 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi033 smithi033 starting - - - - 2024-05-12T04:04:26.461 INFO:teuthology.orchestra.run.smithi032.stdout:grafana.smithi032 smithi032 *:3000 running (2m) 62s ago 12m 81.7M - 9.4.7 954c08fa6188 83055e58d23c 2024-05-12T04:04:26.461 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.ialtqb smithi032 running (6m) 62s ago 6m 12.9M - 17.2.7-939-gc190bdee 284803f7d9cf 0683abf00e3e 2024-05-12T04:04:26.461 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.qpsawh smithi032 running (6m) 62s ago 6m 190M - 17.2.7-939-gc190bdee 284803f7d9cf 95a7db5c4fcd 2024-05-12T04:04:26.461 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi033.hglwml smithi033 running (6m) 46s ago 6m 13.0M - 17.2.7-939-gc190bdee 284803f7d9cf c97902b05025 2024-05-12T04:04:26.461 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi033.jgbjhk smithi033 running (6m) 46s ago 6m 102M - 17.2.7-939-gc190bdee 284803f7d9cf af97052cb97a 2024-05-12T04:04:26.461 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi032.ukatkq smithi032 *:8443,9283,8765 running (2m) 62s ago 13m 468M - 18.2.2-1272-g58249552 ddcb0a9d6fde 0d88ab9a5292 2024-05-12T04:04:26.461 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi033.pcvxwy smithi033 *:8443,9283,8765 running (107s) 46s ago 11m 447M - 18.2.2-1272-g58249552 ddcb0a9d6fde d42dcb6bd307 2024-05-12T04:04:26.461 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi032 smithi032 running (86s) 62s ago 14m 29.1M 2048M 18.2.2-1272-g58249552 ddcb0a9d6fde 81f94171c5dc 2024-05-12T04:04:26.461 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi033 smithi033 running (54s) 46s ago 11m 21.0M 2048M 18.2.2-1272-g58249552 ddcb0a9d6fde d648ba108f3e 2024-05-12T04:04:26.461 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi032 smithi032 *:9100 running (3m) 62s ago 12m 14.3M - 1.5.0 0da6a335fe13 cb75a7c0c829 2024-05-12T04:04:26.462 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi033 smithi033 *:9100 running (2m) 46s ago 11m 17.7M - 1.5.0 0da6a335fe13 7888c36c659f 2024-05-12T04:04:26.462 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0 smithi032 running (10m) 62s ago 10m 206M 1570M 17.2.7-939-gc190bdee 284803f7d9cf 75c78a47f8e5 2024-05-12T04:04:26.462 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1 smithi032 running (10m) 62s ago 10m 225M 1570M 17.2.7-939-gc190bdee 284803f7d9cf 87cfd247d8d6 2024-05-12T04:04:26.462 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2 smithi032 running (9m) 62s ago 9m 180M 1570M 17.2.7-939-gc190bdee 284803f7d9cf 7aef7affb8ba 2024-05-12T04:04:26.462 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3 smithi033 running (9m) 46s ago 9m 239M 2594M 17.2.7-939-gc190bdee 284803f7d9cf 3a92b5e3348e 2024-05-12T04:04:26.462 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4 smithi033 running (8m) 46s ago 8m 213M 2594M 17.2.7-939-gc190bdee 284803f7d9cf 0896268b4591 2024-05-12T04:04:26.462 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5 smithi033 running (8m) 46s ago 8m 196M 2594M 17.2.7-939-gc190bdee 284803f7d9cf 57e65a64c821 2024-05-12T04:04:26.462 INFO:teuthology.orchestra.run.smithi032.stdout:prometheus.smithi032 smithi032 *:9095 running (115s) 62s ago 11m 48.0M - 2.43.0 a07b618ecd1d 46eafb25e176 2024-05-12T04:04:26.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.462+0000 7fcdbb962700 1 -- 172.21.15.32:0/2627433141 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fcda006cd40 msgr2=0x7fcda006f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:04:26.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.462+0000 7fcdbb962700 1 --2- 172.21.15.32:0/2627433141 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fcda006cd40 0x7fcda006f1f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcdb000b3b0 tx=0x7fcdb0009e30 comp rx=0 tx=0).stop 2024-05-12T04:04:26.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.462+0000 7fcdbb962700 1 -- 172.21.15.32:0/2627433141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdb4070940 msgr2=0x7fcdb410fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:04:26.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.462+0000 7fcdbb962700 1 --2- 172.21.15.32:0/2627433141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdb4070940 0x7fcdb410fb50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcda400b790 tx=0x7fcda4009220 comp rx=0 tx=0).stop 2024-05-12T04:04:26.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.463+0000 7fcdbb962700 1 -- 172.21.15.32:0/2627433141 shutdown_connections 2024-05-12T04:04:26.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.463+0000 7fcdbb962700 1 --2- 172.21.15.32:0/2627433141 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fcda006cd40 0x7fcda006f1f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:26.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.463+0000 7fcdbb962700 1 --2- 172.21.15.32:0/2627433141 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdb4070940 0x7fcdb410fb50 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:26.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.463+0000 7fcdbb962700 1 --2- 172.21.15.32:0/2627433141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdb4071ef0 0x7fcdb4110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:26.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.463+0000 7fcdbb962700 1 -- 172.21.15.32:0/2627433141 >> 172.21.15.32:0/2627433141 conn(0x7fcdb40f8560 msgr2=0x7fcdb40f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:04:26.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.463+0000 7fcdbb962700 1 -- 172.21.15.32:0/2627433141 shutdown_connections 2024-05-12T04:04:26.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.463+0000 7fcdbb962700 1 -- 172.21.15.32:0/2627433141 wait complete. 2024-05-12T04:04:26.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.568+0000 7fa9ad56b700 1 -- 172.21.15.32:0/1357896984 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9a80fe3e0 msgr2=0x7fa9a80fe7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:04:26.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.568+0000 7fa9ad56b700 1 --2- 172.21.15.32:0/1357896984 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9a80fe3e0 0x7fa9a80fe7f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa998004940 tx=0x7fa998009e30 comp rx=0 tx=0).stop 2024-05-12T04:04:26.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.569+0000 7fa9ad56b700 1 -- 172.21.15.32:0/1357896984 shutdown_connections 2024-05-12T04:04:26.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.569+0000 7fa9ad56b700 1 --2- 172.21.15.32:0/1357896984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9a80fed30 0x7fa9a80fae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:26.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.569+0000 7fa9ad56b700 1 --2- 172.21.15.32:0/1357896984 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9a80fe3e0 0x7fa9a80fe7f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:26.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.569+0000 7fa9ad56b700 1 -- 172.21.15.32:0/1357896984 >> 172.21.15.32:0/1357896984 conn(0x7fa9a80f6b80 msgr2=0x7fa9a80f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:04:26.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.569+0000 7fa9ad56b700 1 -- 172.21.15.32:0/1357896984 shutdown_connections 2024-05-12T04:04:26.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.569+0000 7fa9ad56b700 1 -- 172.21.15.32:0/1357896984 wait complete. 2024-05-12T04:04:26.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.570+0000 7fa9ad56b700 1 Processor -- start 2024-05-12T04:04:26.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.570+0000 7fa9ad56b700 1 -- start start 2024-05-12T04:04:26.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.570+0000 7fa9ad56b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9a80fe3e0 0x7fa9a810d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:04:26.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.570+0000 7fa9ad56b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9a80fed30 0x7fa9a810de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:04:26.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.570+0000 7fa9ad56b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a810e430 con 0x7fa9a80fed30 2024-05-12T04:04:26.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.570+0000 7fa9ad56b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a810ec80 con 0x7fa9a80fe3e0 2024-05-12T04:04:26.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.570+0000 7fa9a67fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9a80fed30 0x7fa9a810de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:04:26.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.570+0000 7fa9a67fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9a80fed30 0x7fa9a810de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38592/0 (socket says 172.21.15.32:38592) 2024-05-12T04:04:26.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.570+0000 7fa9a67fc700 1 -- 172.21.15.32:0/3273858953 learned_addr learned my addr 172.21.15.32:0/3273858953 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:04:26.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.570+0000 7fa9a6ffd700 1 --2- 172.21.15.32:0/3273858953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9a80fe3e0 0x7fa9a810d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:04:26.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.570+0000 7fa9a67fc700 1 -- 172.21.15.32:0/3273858953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9a80fe3e0 msgr2=0x7fa9a810d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:04:26.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.570+0000 7fa9a67fc700 1 --2- 172.21.15.32:0/3273858953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9a80fe3e0 0x7fa9a810d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:26.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.570+0000 7fa9a67fc700 1 -- 172.21.15.32:0/3273858953 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa998005040 con 0x7fa9a80fed30 2024-05-12T04:04:26.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.571+0000 7fa9a67fc700 1 --2- 172.21.15.32:0/3273858953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9a80fed30 0x7fa9a810de30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa99c00fe60 tx=0x7fa99c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:04:26.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.571+0000 7fa98f7fe700 1 -- 172.21.15.32:0/3273858953 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa99c00e070 con 0x7fa9a80fed30 2024-05-12T04:04:26.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.571+0000 7fa9ad56b700 1 -- 172.21.15.32:0/3273858953 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9a810ef00 con 0x7fa9a80fed30 2024-05-12T04:04:26.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.571+0000 7fa98f7fe700 1 -- 172.21.15.32:0/3273858953 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa99c002d80 con 0x7fa9a80fed30 2024-05-12T04:04:26.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.571+0000 7fa98f7fe700 1 -- 172.21.15.32:0/3273858953 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa99c0090d0 con 0x7fa9a80fed30 2024-05-12T04:04:26.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.571+0000 7fa9ad56b700 1 -- 172.21.15.32:0/3273858953 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9a810f4c0 con 0x7fa9a80fed30 2024-05-12T04:04:26.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.572+0000 7fa9ad56b700 1 -- 172.21.15.32:0/3273858953 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9a805f5c0 con 0x7fa9a80fed30 2024-05-12T04:04:26.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.573+0000 7fa98f7fe700 1 -- 172.21.15.32:0/3273858953 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 39) v1 ==== 90952+0+0 (secure 0 0 0) 0x7fa99c0028d0 con 0x7fa9a80fed30 2024-05-12T04:04:26.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.573+0000 7fa98f7fe700 1 --2- 172.21.15.32:0/3273858953 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fa99006cd40 0x7fa99006f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:04:26.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.574+0000 7fa98f7fe700 1 -- 172.21.15.32:0/3273858953 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fa99c094770 con 0x7fa9a80fed30 2024-05-12T04:04:26.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.574+0000 7fa9a6ffd700 1 --2- 172.21.15.32:0/3273858953 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fa99006cd40 0x7fa99006f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:04:26.574 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.574+0000 7fa9a6ffd700 1 --2- 172.21.15.32:0/3273858953 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fa99006cd40 0x7fa99006f1f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa998009fd0 tx=0x7fa998009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:04:26.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.580+0000 7fa98f7fe700 1 -- 172.21.15.32:0/3273858953 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fa99c061ff0 con 0x7fa9a80fed30 2024-05-12T04:04:26.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.891+0000 7fa9ad56b700 1 -- 172.21.15.32:0/3273858953 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa9a804bed0 con 0x7fa9a80fed30 2024-05-12T04:04:26.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.891+0000 7fa98f7fe700 1 -- 172.21.15.32:0/3273858953 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+742 (secure 0 0 0) 0x7fa99c061b80 con 0x7fa9a80fed30 2024-05-12T04:04:26.892 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-05-12T04:04:26.892 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": { 2024-05-12T04:04:26.892 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 2 2024-05-12T04:04:26.892 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:04:26.892 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": { 2024-05-12T04:04:26.892 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 2 2024-05-12T04:04:26.892 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:04:26.892 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": { 2024-05-12T04:04:26.892 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 6 2024-05-12T04:04:26.892 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:04:26.892 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": { 2024-05-12T04:04:26.893 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 4 2024-05-12T04:04:26.893 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:04:26.893 INFO:teuthology.orchestra.run.smithi032.stdout: "overall": { 2024-05-12T04:04:26.893 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 10, 2024-05-12T04:04:26.893 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 4 2024-05-12T04:04:26.893 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-05-12T04:04:26.893 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-05-12T04:04:26.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.893+0000 7fa9ad56b700 1 -- 172.21.15.32:0/3273858953 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fa99006cd40 msgr2=0x7fa99006f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:04:26.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.893+0000 7fa9ad56b700 1 --2- 172.21.15.32:0/3273858953 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fa99006cd40 0x7fa99006f1f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa998009fd0 tx=0x7fa998009e30 comp rx=0 tx=0).stop 2024-05-12T04:04:26.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.893+0000 7fa9ad56b700 1 -- 172.21.15.32:0/3273858953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9a80fed30 msgr2=0x7fa9a810de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:04:26.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.893+0000 7fa9ad56b700 1 --2- 172.21.15.32:0/3273858953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9a80fed30 0x7fa9a810de30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa99c00fe60 tx=0x7fa99c0046b0 comp rx=0 tx=0).stop 2024-05-12T04:04:26.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.894+0000 7fa9ad56b700 1 -- 172.21.15.32:0/3273858953 shutdown_connections 2024-05-12T04:04:26.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.894+0000 7fa9ad56b700 1 --2- 172.21.15.32:0/3273858953 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fa99006cd40 0x7fa99006f1f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:26.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.894+0000 7fa9ad56b700 1 --2- 172.21.15.32:0/3273858953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9a80fe3e0 0x7fa9a810d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:26.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.894+0000 7fa9ad56b700 1 --2- 172.21.15.32:0/3273858953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9a80fed30 0x7fa9a810de30 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:26.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.894+0000 7fa9ad56b700 1 -- 172.21.15.32:0/3273858953 >> 172.21.15.32:0/3273858953 conn(0x7fa9a80f6b80 msgr2=0x7fa9a80f8310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:04:26.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.894+0000 7fa9ad56b700 1 -- 172.21.15.32:0/3273858953 shutdown_connections 2024-05-12T04:04:26.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:26.894+0000 7fa9ad56b700 1 -- 172.21.15.32:0/3273858953 wait complete. 2024-05-12T04:04:27.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.001+0000 7f9d5359e700 1 -- 172.21.15.32:0/2194491726 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d4c066ad0 msgr2=0x7f9d4c101eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:04:27.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.001+0000 7f9d5359e700 1 --2- 172.21.15.32:0/2194491726 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d4c066ad0 0x7f9d4c101eb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9d48004940 tx=0x7f9d48009e30 comp rx=0 tx=0).stop 2024-05-12T04:04:27.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.001+0000 7f9d5359e700 1 -- 172.21.15.32:0/2194491726 shutdown_connections 2024-05-12T04:04:27.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.001+0000 7f9d5359e700 1 --2- 172.21.15.32:0/2194491726 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d4c1023f0 0x7f9d4c1047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:27.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.001+0000 7f9d5359e700 1 --2- 172.21.15.32:0/2194491726 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d4c066ad0 0x7f9d4c101eb0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:27.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.001+0000 7f9d5359e700 1 -- 172.21.15.32:0/2194491726 >> 172.21.15.32:0/2194491726 conn(0x7f9d4c0f74c0 msgr2=0x7f9d4c0f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:04:27.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.001+0000 7f9d5359e700 1 -- 172.21.15.32:0/2194491726 shutdown_connections 2024-05-12T04:04:27.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.001+0000 7f9d5359e700 1 -- 172.21.15.32:0/2194491726 wait complete. 2024-05-12T04:04:27.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.001+0000 7f9d5359e700 1 Processor -- start 2024-05-12T04:04:27.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.002+0000 7f9d5359e700 1 -- start start 2024-05-12T04:04:27.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.002+0000 7f9d5359e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d4c066ad0 0x7f9d4c06e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:04:27.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.002+0000 7f9d5359e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d4c1023f0 0x7f9d4c06ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:04:27.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.002+0000 7f9d5359e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4c101ac0 con 0x7f9d4c1023f0 2024-05-12T04:04:27.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.002+0000 7f9d5359e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4c101c00 con 0x7f9d4c066ad0 2024-05-12T04:04:27.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.002+0000 7f9d50b39700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d4c1023f0 0x7f9d4c06ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:04:27.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.002+0000 7f9d5133a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d4c066ad0 0x7f9d4c06e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:04:27.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.002+0000 7f9d50b39700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d4c1023f0 0x7f9d4c06ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38612/0 (socket says 172.21.15.32:38612) 2024-05-12T04:04:27.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.002+0000 7f9d5133a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d4c066ad0 0x7f9d4c06e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.32:52112/0 (socket says 172.21.15.32:52112) 2024-05-12T04:04:27.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.002+0000 7f9d50b39700 1 -- 172.21.15.32:0/263091133 learned_addr learned my addr 172.21.15.32:0/263091133 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:04:27.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.002+0000 7f9d5133a700 1 -- 172.21.15.32:0/263091133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d4c1023f0 msgr2=0x7f9d4c06ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:04:27.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.002+0000 7f9d5133a700 1 --2- 172.21.15.32:0/263091133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d4c1023f0 0x7f9d4c06ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:27.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.002+0000 7f9d5133a700 1 -- 172.21.15.32:0/263091133 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d48005040 con 0x7f9d4c066ad0 2024-05-12T04:04:27.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.002+0000 7f9d50b39700 1 --2- 172.21.15.32:0/263091133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d4c1023f0 0x7f9d4c06ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-12T04:04:27.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.003+0000 7f9d5133a700 1 --2- 172.21.15.32:0/263091133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d4c066ad0 0x7f9d4c06e520 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9d48000c00 tx=0x7f9d48009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:04:27.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.003+0000 7f9d41ffb700 1 -- 172.21.15.32:0/263091133 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d4800e040 con 0x7f9d4c066ad0 2024-05-12T04:04:27.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.003+0000 7f9d5359e700 1 -- 172.21.15.32:0/263091133 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d4c06efa0 con 0x7f9d4c066ad0 2024-05-12T04:04:27.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.003+0000 7f9d5359e700 1 -- 172.21.15.32:0/263091133 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d4c122580 con 0x7f9d4c066ad0 2024-05-12T04:04:27.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.003+0000 7f9d41ffb700 1 -- 172.21.15.32:0/263091133 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9d4801d440 con 0x7f9d4c066ad0 2024-05-12T04:04:27.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.003+0000 7f9d41ffb700 1 -- 172.21.15.32:0/263091133 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d48013660 con 0x7f9d4c066ad0 2024-05-12T04:04:27.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.004+0000 7f9d5359e700 1 -- 172.21.15.32:0/263091133 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d300052e0 con 0x7f9d4c066ad0 2024-05-12T04:04:27.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.005+0000 7f9d41ffb700 1 -- 172.21.15.32:0/263091133 <== mon.1 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 39) v1 ==== 90952+0+0 (secure 0 0 0) 0x7f9d4800bb20 con 0x7f9d4c066ad0 2024-05-12T04:04:27.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.005+0000 7f9d41ffb700 1 --2- 172.21.15.32:0/263091133 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f9d3806cd40 0x7f9d3806f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:04:27.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.005+0000 7f9d41ffb700 1 -- 172.21.15.32:0/263091133 <== mon.1 v2:172.21.15.33:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f9d4808d470 con 0x7f9d4c066ad0 2024-05-12T04:04:27.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.006+0000 7f9d50b39700 1 --2- 172.21.15.32:0/263091133 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f9d3806cd40 0x7f9d3806f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:04:27.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.006+0000 7f9d50b39700 1 --2- 172.21.15.32:0/263091133 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f9d3806cd40 0x7f9d3806f1f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c0041d0 tx=0x7f9d3c00b040 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:04:27.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.011+0000 7f9d41ffb700 1 -- 172.21.15.32:0/263091133 <== mon.1 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f9d480572c0 con 0x7f9d4c066ad0 2024-05-12T04:04:27.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.264+0000 7f9d5359e700 1 -- 172.21.15.32:0/263091133 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9d30005860 con 0x7f9d4c066ad0 2024-05-12T04:04:27.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.265+0000 7f9d41ffb700 1 -- 172.21.15.32:0/263091133 <== mon.1 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1866 (secure 0 0 0) 0x7f9d4805a8e0 con 0x7f9d4c066ad0 2024-05-12T04:04:27.265 INFO:teuthology.orchestra.run.smithi032.stdout:e18 2024-05-12T04:04:27.265 INFO:teuthology.orchestra.run.smithi032.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-12T04:04:27.265 INFO:teuthology.orchestra.run.smithi032.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-12T04:04:27.265 INFO:teuthology.orchestra.run.smithi032.stdout:legacy client fscid: 1 2024-05-12T04:04:27.265 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:04:27.265 INFO:teuthology.orchestra.run.smithi032.stdout:Filesystem 'cephfs' (1) 2024-05-12T04:04:27.265 INFO:teuthology.orchestra.run.smithi032.stdout:fs_name cephfs 2024-05-12T04:04:27.266 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 18 2024-05-12T04:04:27.266 INFO:teuthology.orchestra.run.smithi032.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-12T04:04:27.266 INFO:teuthology.orchestra.run.smithi032.stdout:created 2024-05-12T03:57:48.904614+0000 2024-05-12T04:04:27.266 INFO:teuthology.orchestra.run.smithi032.stdout:modified 2024-05-12T03:58:06.821591+0000 2024-05-12T04:04:27.266 INFO:teuthology.orchestra.run.smithi032.stdout:tableserver 0 2024-05-12T04:04:27.266 INFO:teuthology.orchestra.run.smithi032.stdout:root 0 2024-05-12T04:04:27.266 INFO:teuthology.orchestra.run.smithi032.stdout:session_timeout 60 2024-05-12T04:04:27.266 INFO:teuthology.orchestra.run.smithi032.stdout:session_autoclose 300 2024-05-12T04:04:27.266 INFO:teuthology.orchestra.run.smithi032.stdout:max_file_size 1099511627776 2024-05-12T04:04:27.266 INFO:teuthology.orchestra.run.smithi032.stdout:max_xattr_size 65536 2024-05-12T04:04:27.266 INFO:teuthology.orchestra.run.smithi032.stdout:required_client_features {} 2024-05-12T04:04:27.266 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure 0 2024-05-12T04:04:27.266 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure_osd_epoch 46 2024-05-12T04:04:27.266 INFO:teuthology.orchestra.run.smithi032.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-12T04:04:27.266 INFO:teuthology.orchestra.run.smithi032.stdout:max_mds 1 2024-05-12T04:04:27.267 INFO:teuthology.orchestra.run.smithi032.stdout:in 0 2024-05-12T04:04:27.267 INFO:teuthology.orchestra.run.smithi032.stdout:up {0=14474} 2024-05-12T04:04:27.267 INFO:teuthology.orchestra.run.smithi032.stdout:failed 2024-05-12T04:04:27.267 INFO:teuthology.orchestra.run.smithi032.stdout:damaged 2024-05-12T04:04:27.267 INFO:teuthology.orchestra.run.smithi032.stdout:stopped 2024-05-12T04:04:27.267 INFO:teuthology.orchestra.run.smithi032.stdout:data_pools [3] 2024-05-12T04:04:27.267 INFO:teuthology.orchestra.run.smithi032.stdout:metadata_pool 2 2024-05-12T04:04:27.267 INFO:teuthology.orchestra.run.smithi032.stdout:inline_data enabled 2024-05-12T04:04:27.267 INFO:teuthology.orchestra.run.smithi032.stdout:balancer 2024-05-12T04:04:27.267 INFO:teuthology.orchestra.run.smithi032.stdout:bal_rank_mask -1 2024-05-12T04:04:27.267 INFO:teuthology.orchestra.run.smithi032.stdout:standby_count_wanted 1 2024-05-12T04:04:27.267 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.qpsawh{0:14474} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.32:6828/3100155296,v1:172.21.15.32:6829/3100155296] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:04:27.267 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi033.jgbjhk{0:24267} state up:standby-replay seq 3 join_fscid=1 addr [v2:172.21.15.33:6824/55417471,v1:172.21.15.33:6825/55417471] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:04:27.267 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:04:27.268 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:04:27.268 INFO:teuthology.orchestra.run.smithi032.stdout:Standby daemons: 2024-05-12T04:04:27.268 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:04:27.268 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi033.hglwml{-1:24283} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/433858238,v1:172.21.15.33:6827/433858238] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:04:27.268 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.ialtqb{-1:24287} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.32:6826/3703132329,v1:172.21.15.32:6827/3703132329] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:04:27.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.267+0000 7f9d5359e700 1 -- 172.21.15.32:0/263091133 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f9d3806cd40 msgr2=0x7f9d3806f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:04:27.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.267+0000 7f9d5359e700 1 --2- 172.21.15.32:0/263091133 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f9d3806cd40 0x7f9d3806f1f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c0041d0 tx=0x7f9d3c00b040 comp rx=0 tx=0).stop 2024-05-12T04:04:27.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.267+0000 7f9d5359e700 1 -- 172.21.15.32:0/263091133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d4c066ad0 msgr2=0x7f9d4c06e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:04:27.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.267+0000 7f9d5359e700 1 --2- 172.21.15.32:0/263091133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d4c066ad0 0x7f9d4c06e520 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9d48000c00 tx=0x7f9d48009e30 comp rx=0 tx=0).stop 2024-05-12T04:04:27.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.267+0000 7f9d5359e700 1 -- 172.21.15.32:0/263091133 shutdown_connections 2024-05-12T04:04:27.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.267+0000 7f9d5359e700 1 --2- 172.21.15.32:0/263091133 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f9d3806cd40 0x7f9d3806f1f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:27.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.267+0000 7f9d5359e700 1 --2- 172.21.15.32:0/263091133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d4c066ad0 0x7f9d4c06e520 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:27.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.267+0000 7f9d5359e700 1 --2- 172.21.15.32:0/263091133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d4c1023f0 0x7f9d4c06ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:27.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.267+0000 7f9d5359e700 1 -- 172.21.15.32:0/263091133 >> 172.21.15.32:0/263091133 conn(0x7f9d4c0f74c0 msgr2=0x7f9d4c0f7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:04:27.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.267+0000 7f9d5359e700 1 -- 172.21.15.32:0/263091133 shutdown_connections 2024-05-12T04:04:27.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.267+0000 7f9d5359e700 1 -- 172.21.15.32:0/263091133 wait complete. 2024-05-12T04:04:27.269 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 18 2024-05-12T04:04:27.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.374+0000 7f2279439700 1 -- 172.21.15.32:0/2272481237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22740fbcb0 msgr2=0x7f22740fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:04:27.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.374+0000 7f2279439700 1 --2- 172.21.15.32:0/2272481237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22740fbcb0 0x7f22740fc0c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f225c004940 tx=0x7f225c009e30 comp rx=0 tx=0).stop 2024-05-12T04:04:27.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.374+0000 7f2279439700 1 -- 172.21.15.32:0/2272481237 shutdown_connections 2024-05-12T04:04:27.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.374+0000 7f2279439700 1 --2- 172.21.15.32:0/2272481237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f22740fcf60 0x7f22740fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:27.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.374+0000 7f2279439700 1 --2- 172.21.15.32:0/2272481237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22740fbcb0 0x7f22740fc0c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:27.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.374+0000 7f2279439700 1 -- 172.21.15.32:0/2272481237 >> 172.21.15.32:0/2272481237 conn(0x7f22740f7430 msgr2=0x7f22740f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:04:27.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.374+0000 7f2279439700 1 -- 172.21.15.32:0/2272481237 shutdown_connections 2024-05-12T04:04:27.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.374+0000 7f2279439700 1 -- 172.21.15.32:0/2272481237 wait complete. 2024-05-12T04:04:27.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.375+0000 7f2279439700 1 Processor -- start 2024-05-12T04:04:27.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.375+0000 7f2279439700 1 -- start start 2024-05-12T04:04:27.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.375+0000 7f2279439700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f22740fbcb0 0x7f22741105b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:04:27.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.375+0000 7f2279439700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22740fcf60 0x7f2274112b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:04:27.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.375+0000 7f2279439700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2274066330 con 0x7f22740fcf60 2024-05-12T04:04:27.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.375+0000 7f2279439700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2274066470 con 0x7f22740fbcb0 2024-05-12T04:04:27.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.375+0000 7f22727fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22740fcf60 0x7f2274112b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:04:27.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.375+0000 7f2272ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f22740fbcb0 0x7f22741105b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:04:27.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.375+0000 7f22727fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22740fcf60 0x7f2274112b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38622/0 (socket says 172.21.15.32:38622) 2024-05-12T04:04:27.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.375+0000 7f22727fc700 1 -- 172.21.15.32:0/1523638462 learned_addr learned my addr 172.21.15.32:0/1523638462 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:04:27.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.376+0000 7f22727fc700 1 -- 172.21.15.32:0/1523638462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f22740fbcb0 msgr2=0x7f22741105b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:04:27.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.376+0000 7f22727fc700 1 --2- 172.21.15.32:0/1523638462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f22740fbcb0 0x7f22741105b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:27.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.376+0000 7f22727fc700 1 -- 172.21.15.32:0/1523638462 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f225c005040 con 0x7f22740fcf60 2024-05-12T04:04:27.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.376+0000 7f2272ffd700 1 --2- 172.21.15.32:0/1523638462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f22740fbcb0 0x7f22741105b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-12T04:04:27.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.376+0000 7f22727fc700 1 --2- 172.21.15.32:0/1523638462 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22740fcf60 0x7f2274112b00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f226400eb40 tx=0x7f226400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:04:27.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.376+0000 7f226b7fe700 1 -- 172.21.15.32:0/1523638462 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2264007cc0 con 0x7f22740fcf60 2024-05-12T04:04:27.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.376+0000 7f2279439700 1 -- 172.21.15.32:0/1523638462 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2274113040 con 0x7f22740fcf60 2024-05-12T04:04:27.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.376+0000 7f226b7fe700 1 -- 172.21.15.32:0/1523638462 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f226400f040 con 0x7f22740fcf60 2024-05-12T04:04:27.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.376+0000 7f226b7fe700 1 -- 172.21.15.32:0/1523638462 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f226400c990 con 0x7f22740fcf60 2024-05-12T04:04:27.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.376+0000 7f2279439700 1 -- 172.21.15.32:0/1523638462 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2274113600 con 0x7f22740fcf60 2024-05-12T04:04:27.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.377+0000 7f2279439700 1 -- 172.21.15.32:0/1523638462 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f227405f5c0 con 0x7f22740fcf60 2024-05-12T04:04:27.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.378+0000 7f226b7fe700 1 -- 172.21.15.32:0/1523638462 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 39) v1 ==== 90952+0+0 (secure 0 0 0) 0x7f226400cc00 con 0x7f22740fcf60 2024-05-12T04:04:27.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.379+0000 7f226b7fe700 1 --2- 172.21.15.32:0/1523638462 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f226006cd40 0x7f226006f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:04:27.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.379+0000 7f2272ffd700 1 --2- 172.21.15.32:0/1523638462 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f226006cd40 0x7f226006f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:04:27.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.379+0000 7f226b7fe700 1 -- 172.21.15.32:0/1523638462 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f226408c0d0 con 0x7f22740fcf60 2024-05-12T04:04:27.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.379+0000 7f2272ffd700 1 --2- 172.21.15.32:0/1523638462 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f226006cd40 0x7f226006f1f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f225c013680 tx=0x7f225c009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:04:27.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.383+0000 7f226b7fe700 1 -- 172.21.15.32:0/1523638462 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f2264055ff0 con 0x7f22740fcf60 2024-05-12T04:04:27.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.624+0000 7f2279439700 1 -- 172.21.15.32:0/1523638462 --> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2274101320 con 0x7f226006cd40 2024-05-12T04:04:27.625 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.625+0000 7f226b7fe700 1 -- 172.21.15.32:0/1523638462 <== mgr.24593 v2:172.21.15.32:6800/2957111591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+385 (secure 0 0 0) 0x7f2274101320 con 0x7f226006cd40 2024-05-12T04:04:27.625 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-05-12T04:04:27.625 INFO:teuthology.orchestra.run.smithi032.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:19c9734256aa43a60e203feb242f50bf8b7f58666d520536af43cc4fee96173a", 2024-05-12T04:04:27.625 INFO:teuthology.orchestra.run.smithi032.stdout: "in_progress": true, 2024-05-12T04:04:27.625 INFO:teuthology.orchestra.run.smithi032.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-12T04:04:27.626 INFO:teuthology.orchestra.run.smithi032.stdout: "services_complete": [ 2024-05-12T04:04:27.626 INFO:teuthology.orchestra.run.smithi032.stdout: "mon", 2024-05-12T04:04:27.626 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr" 2024-05-12T04:04:27.626 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-05-12T04:04:27.626 INFO:teuthology.orchestra.run.smithi032.stdout: "progress": "5/21 daemons upgraded", 2024-05-12T04:04:27.626 INFO:teuthology.orchestra.run.smithi032.stdout: "message": "Currently upgrading crash daemons", 2024-05-12T04:04:27.626 INFO:teuthology.orchestra.run.smithi032.stdout: "is_paused": false 2024-05-12T04:04:27.626 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-05-12T04:04:27.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.627+0000 7f2279439700 1 -- 172.21.15.32:0/1523638462 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f226006cd40 msgr2=0x7f226006f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:04:27.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.627+0000 7f2279439700 1 --2- 172.21.15.32:0/1523638462 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f226006cd40 0x7f226006f1f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f225c013680 tx=0x7f225c009e30 comp rx=0 tx=0).stop 2024-05-12T04:04:27.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.627+0000 7f2279439700 1 -- 172.21.15.32:0/1523638462 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22740fcf60 msgr2=0x7f2274112b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:04:27.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.627+0000 7f2279439700 1 --2- 172.21.15.32:0/1523638462 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22740fcf60 0x7f2274112b00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f226400eb40 tx=0x7f226400b040 comp rx=0 tx=0).stop 2024-05-12T04:04:27.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.627+0000 7f2279439700 1 -- 172.21.15.32:0/1523638462 shutdown_connections 2024-05-12T04:04:27.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.627+0000 7f2279439700 1 --2- 172.21.15.32:0/1523638462 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f226006cd40 0x7f226006f1f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:27.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.627+0000 7f2279439700 1 --2- 172.21.15.32:0/1523638462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f22740fbcb0 0x7f22741105b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:27.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.627+0000 7f2279439700 1 --2- 172.21.15.32:0/1523638462 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22740fcf60 0x7f2274112b00 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:27.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.627+0000 7f2279439700 1 -- 172.21.15.32:0/1523638462 >> 172.21.15.32:0/1523638462 conn(0x7f22740f7430 msgr2=0x7f22740f8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:04:27.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.628+0000 7f2279439700 1 -- 172.21.15.32:0/1523638462 shutdown_connections 2024-05-12T04:04:27.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.628+0000 7f2279439700 1 -- 172.21.15.32:0/1523638462 wait complete. 2024-05-12T04:04:27.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:27 smithi033 ceph-mon[107305]: pgmap v38: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:04:27.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:27 smithi033 ceph-mon[107305]: from='client.34184 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:04:27.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:27 smithi033 ceph-mon[107305]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:04:27.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:27 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:27.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:27 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:27.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:27 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/3273858953' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:04:27.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:27 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:27.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:27 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:27.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:27 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/263091133' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-12T04:04:27.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.732+0000 7fb9bcb16700 1 -- 172.21.15.32:0/474571959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9b80fdbc0 msgr2=0x7fb9b80fbfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:04:27.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.732+0000 7fb9bcb16700 1 --2- 172.21.15.32:0/474571959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9b80fdbc0 0x7fb9b80fbfe0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb9a0004940 tx=0x7fb9a0009e30 comp rx=0 tx=0).stop 2024-05-12T04:04:27.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.732+0000 7fb9bcb16700 1 -- 172.21.15.32:0/474571959 shutdown_connections 2024-05-12T04:04:27.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.732+0000 7fb9bcb16700 1 --2- 172.21.15.32:0/474571959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9b80fdbc0 0x7fb9b80fbfe0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:27.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.732+0000 7fb9bcb16700 1 --2- 172.21.15.32:0/474571959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9b80fd270 0x7fb9b80fd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:27.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.732+0000 7fb9bcb16700 1 -- 172.21.15.32:0/474571959 >> 172.21.15.32:0/474571959 conn(0x7fb9b80f6b80 msgr2=0x7fb9b80f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:04:27.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.732+0000 7fb9bcb16700 1 -- 172.21.15.32:0/474571959 shutdown_connections 2024-05-12T04:04:27.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.732+0000 7fb9bcb16700 1 -- 172.21.15.32:0/474571959 wait complete. 2024-05-12T04:04:27.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.733+0000 7fb9bcb16700 1 Processor -- start 2024-05-12T04:04:27.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.733+0000 7fb9bcb16700 1 -- start start 2024-05-12T04:04:27.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.733+0000 7fb9bcb16700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9b80fd270 0x7fb9b810d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:04:27.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.733+0000 7fb9bcb16700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9b810de70 0x7fb9b810ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:04:27.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.733+0000 7fb9bcb16700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b810e3c0 con 0x7fb9b80fd270 2024-05-12T04:04:27.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.733+0000 7fb9bcb16700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b810e500 con 0x7fb9b810de70 2024-05-12T04:04:27.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.733+0000 7fb9b659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9b80fd270 0x7fb9b810d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:04:27.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.733+0000 7fb9b659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9b80fd270 0x7fb9b810d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38642/0 (socket says 172.21.15.32:38642) 2024-05-12T04:04:27.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.733+0000 7fb9b659c700 1 -- 172.21.15.32:0/492456188 learned_addr learned my addr 172.21.15.32:0/492456188 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:04:27.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.733+0000 7fb9b5d9b700 1 --2- 172.21.15.32:0/492456188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9b810de70 0x7fb9b810ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:04:27.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.733+0000 7fb9b659c700 1 -- 172.21.15.32:0/492456188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9b810de70 msgr2=0x7fb9b810ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:04:27.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.733+0000 7fb9b659c700 1 --2- 172.21.15.32:0/492456188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9b810de70 0x7fb9b810ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:27.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.733+0000 7fb9b659c700 1 -- 172.21.15.32:0/492456188 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9a0005040 con 0x7fb9b80fd270 2024-05-12T04:04:27.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.733+0000 7fb9b659c700 1 --2- 172.21.15.32:0/492456188 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9b80fd270 0x7fb9b810d930 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb9a800b790 tx=0x7fb9a8009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:04:27.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.734+0000 7fb9aeffd700 1 -- 172.21.15.32:0/492456188 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9a800f6f0 con 0x7fb9b80fd270 2024-05-12T04:04:27.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.734+0000 7fb9aeffd700 1 -- 172.21.15.32:0/492456188 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb9a800fd00 con 0x7fb9b80fd270 2024-05-12T04:04:27.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.734+0000 7fb9bcb16700 1 -- 172.21.15.32:0/492456188 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9b810f3a0 con 0x7fb9b80fd270 2024-05-12T04:04:27.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.734+0000 7fb9aeffd700 1 -- 172.21.15.32:0/492456188 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9a8011e90 con 0x7fb9b80fd270 2024-05-12T04:04:27.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.734+0000 7fb9bcb16700 1 -- 172.21.15.32:0/492456188 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9b810f960 con 0x7fb9b80fd270 2024-05-12T04:04:27.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.735+0000 7fb9bcb16700 1 -- 172.21.15.32:0/492456188 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9b8101b20 con 0x7fb9b80fd270 2024-05-12T04:04:27.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.736+0000 7fb9aeffd700 1 -- 172.21.15.32:0/492456188 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 39) v1 ==== 90952+0+0 (secure 0 0 0) 0x7fb9a800f850 con 0x7fb9b80fd270 2024-05-12T04:04:27.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.736+0000 7fb9aeffd700 1 --2- 172.21.15.32:0/492456188 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fb9a406cd40 0x7fb9a406f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:04:27.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.736+0000 7fb9aeffd700 1 -- 172.21.15.32:0/492456188 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fb9a808c990 con 0x7fb9b80fd270 2024-05-12T04:04:27.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.736+0000 7fb9b5d9b700 1 --2- 172.21.15.32:0/492456188 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fb9a406cd40 0x7fb9a406f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:04:27.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.736+0000 7fb9b5d9b700 1 --2- 172.21.15.32:0/492456188 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fb9a406cd40 0x7fb9a406f1f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb9a000b900 tx=0x7fb9a0009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:04:27.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:27.740+0000 7fb9aeffd700 1 -- 172.21.15.32:0/492456188 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fb9a805a440 con 0x7fb9b80fd270 2024-05-12T04:04:27.789 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:27 smithi032 ceph-mon[120392]: pgmap v38: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:04:27.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:27 smithi032 ceph-mon[120392]: from='client.34184 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:04:27.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:27 smithi032 ceph-mon[120392]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:04:27.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:27 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:27.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:27 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:27.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:27 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/3273858953' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:04:27.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:27 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:27.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:27 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:27.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:27 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/263091133' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-12T04:04:28.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:28.044+0000 7fb9bcb16700 1 -- 172.21.15.32:0/492456188 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb9b804bed0 con 0x7fb9b80fd270 2024-05-12T04:04:28.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:28.044+0000 7fb9aeffd700 1 -- 172.21.15.32:0/492456188 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fb9a8059fd0 con 0x7fb9b80fd270 2024-05-12T04:04:28.045 INFO:teuthology.orchestra.run.smithi032.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-12T04:04:28.045 INFO:teuthology.orchestra.run.smithi032.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-12T04:04:28.045 INFO:teuthology.orchestra.run.smithi032.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-12T04:04:28.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:28.047+0000 7fb9bcb16700 1 -- 172.21.15.32:0/492456188 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fb9a406cd40 msgr2=0x7fb9a406f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:04:28.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:28.047+0000 7fb9bcb16700 1 --2- 172.21.15.32:0/492456188 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fb9a406cd40 0x7fb9a406f1f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb9a000b900 tx=0x7fb9a0009e30 comp rx=0 tx=0).stop 2024-05-12T04:04:28.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:28.047+0000 7fb9bcb16700 1 -- 172.21.15.32:0/492456188 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9b80fd270 msgr2=0x7fb9b810d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:04:28.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:28.047+0000 7fb9bcb16700 1 --2- 172.21.15.32:0/492456188 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9b80fd270 0x7fb9b810d930 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb9a800b790 tx=0x7fb9a8009220 comp rx=0 tx=0).stop 2024-05-12T04:04:28.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:28.047+0000 7fb9bcb16700 1 -- 172.21.15.32:0/492456188 shutdown_connections 2024-05-12T04:04:28.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:28.047+0000 7fb9bcb16700 1 --2- 172.21.15.32:0/492456188 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fb9a406cd40 0x7fb9a406f1f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:28.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:28.047+0000 7fb9bcb16700 1 --2- 172.21.15.32:0/492456188 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9b80fd270 0x7fb9b810d930 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:28.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:28.047+0000 7fb9bcb16700 1 --2- 172.21.15.32:0/492456188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9b810de70 0x7fb9b810ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:28.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:28.047+0000 7fb9bcb16700 1 -- 172.21.15.32:0/492456188 >> 172.21.15.32:0/492456188 conn(0x7fb9b80f6b80 msgr2=0x7fb9b80fb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:04:28.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:28.047+0000 7fb9bcb16700 1 -- 172.21.15.32:0/492456188 shutdown_connections 2024-05-12T04:04:28.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:28.047+0000 7fb9bcb16700 1 -- 172.21.15.32:0/492456188 wait complete. 2024-05-12T04:04:28.660 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:28 smithi032 ceph-mon[120392]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:04:28.661 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:28 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/492456188' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-12T04:04:28.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:28 smithi033 ceph-mon[107305]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:04:28.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:28 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/492456188' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-12T04:04:29.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:29 smithi033 ceph-mon[107305]: from='client.34204 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:04:29.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:29 smithi033 ceph-mon[107305]: pgmap v39: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:04:29.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:29 smithi032 ceph-mon[120392]: from='client.34204 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:04:29.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:29 smithi032 ceph-mon[120392]: pgmap v39: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:04:31.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:31 smithi033 ceph-mon[107305]: pgmap v40: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:04:31.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:31 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:31.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:31 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:31.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:31 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:04:31.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:31 smithi032 ceph-mon[120392]: pgmap v40: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:04:31.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:31 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:31.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:31 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:31.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:31 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:04:32.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:32 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:32.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:32 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:32.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:32 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:32.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:32 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:32.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:32 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:32.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:32 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:32.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:32 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:32.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:32 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:33.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:33 smithi032 ceph-mon[120392]: pgmap v41: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:04:33.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:33 smithi033 ceph-mon[107305]: pgmap v41: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:04:35.620 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:35 smithi032 ceph-mon[120392]: pgmap v42: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:04:35.621 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:35 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:35.621 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:35 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:35.621 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:35 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:04:35.621 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:35 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T04:04:35.621 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:35 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:35.621 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:35 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:04:35.621 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:35 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:04:35.621 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:35 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:04:35.621 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:35 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:04:35.621 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:35 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:35.621 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:35 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi032"}]: dispatch 2024-05-12T04:04:35.621 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:35 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi032"}]': finished 2024-05-12T04:04:35.621 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:35 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi033"}]: dispatch 2024-05-12T04:04:35.621 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:35 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi033"}]': finished 2024-05-12T04:04:35.622 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:35 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-05-12T04:04:35.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:35 smithi033 ceph-mon[107305]: pgmap v42: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:04:35.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:35 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:35.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:35 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:35.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:35 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:04:35.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:35 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T04:04:35.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:35 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:35.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:35 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:04:35.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:35 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:04:35.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:35 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:04:35.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:35 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:04:35.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:35 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:35.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:35 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi032"}]: dispatch 2024-05-12T04:04:35.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:35 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi032"}]': finished 2024-05-12T04:04:35.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:35 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi033"}]: dispatch 2024-05-12T04:04:35.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:35 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi033"}]': finished 2024-05-12T04:04:35.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:35 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-05-12T04:04:36.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:36 smithi033 ceph-mon[107305]: Upgrade: Setting container_image for all crash 2024-05-12T04:04:36.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:36 smithi033 ceph-mon[107305]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-05-12T04:04:36.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:36 smithi033 ceph-mon[107305]: Upgrade: osd.0 is safe to restart 2024-05-12T04:04:36.713 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:36 smithi032 ceph-mon[120392]: Upgrade: Setting container_image for all crash 2024-05-12T04:04:36.713 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:36 smithi032 ceph-mon[120392]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-05-12T04:04:36.713 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:36 smithi032 ceph-mon[120392]: Upgrade: osd.0 is safe to restart 2024-05-12T04:04:37.507 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:37 smithi032 ceph-mon[120392]: pgmap v43: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:04:37.507 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:37 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:37.507 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:37 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-12T04:04:37.508 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:37 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:04:37.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:37 smithi033 ceph-mon[107305]: pgmap v43: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:04:37.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:37 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:37.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:37 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-12T04:04:37.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:37 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:04:38.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:38 smithi032 ceph-mon[120392]: Upgrade: Updating osd.0 2024-05-12T04:04:38.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:38 smithi032 ceph-mon[120392]: Deploying daemon osd.0 on smithi032 2024-05-12T04:04:38.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:38 smithi033 ceph-mon[107305]: Upgrade: Updating osd.0 2024-05-12T04:04:38.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:38 smithi033 ceph-mon[107305]: Deploying daemon osd.0 on smithi032 2024-05-12T04:04:39.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:39 smithi033 ceph-mon[107305]: pgmap v44: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:04:39.702 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:39 smithi032 ceph-mon[120392]: pgmap v44: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:04:40.375 INFO:journalctl@ceph.osd.0.smithi032.stdout:May 12 04:04:40 smithi032 systemd[1]: Stopping Ceph osd.0 for a3e178f8-1012-11ef-bc98-c7b262605968... 2024-05-12T04:04:40.376 INFO:journalctl@ceph.osd.0.smithi032.stdout:May 12 04:04:40 smithi032 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-0[64381]: 2024-05-12T04:04:40.293+0000 7fa07fdc6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-12T04:04:40.376 INFO:journalctl@ceph.osd.0.smithi032.stdout:May 12 04:04:40 smithi032 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-0[64381]: 2024-05-12T04:04:40.293+0000 7fa07fdc6700 -1 osd.0 49 *** Got signal Terminated *** 2024-05-12T04:04:40.376 INFO:journalctl@ceph.osd.0.smithi032.stdout:May 12 04:04:40 smithi032 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-0[64381]: 2024-05-12T04:04:40.293+0000 7fa07fdc6700 -1 osd.0 49 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-12T04:04:40.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:40 smithi033 ceph-mon[107305]: osd.0 marked itself down and dead 2024-05-12T04:04:40.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:40 smithi032 ceph-mon[120392]: osd.0 marked itself down and dead 2024-05-12T04:04:41.376 INFO:journalctl@ceph.osd.0.smithi032.stdout:May 12 04:04:41 smithi032 bash[130739]: ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-0 2024-05-12T04:04:41.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:41 smithi033 ceph-mon[107305]: pgmap v45: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:04:41.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:41 smithi033 ceph-mon[107305]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-12T04:04:41.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:41 smithi033 ceph-mon[107305]: osdmap e50: 6 total, 5 up, 6 in 2024-05-12T04:04:41.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:41 smithi032 ceph-mon[120392]: pgmap v45: 65 pgs: 65 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:04:41.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:41 smithi032 ceph-mon[120392]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-12T04:04:41.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:41 smithi032 ceph-mon[120392]: osdmap e50: 6 total, 5 up, 6 in 2024-05-12T04:04:42.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:42 smithi033 ceph-mon[107305]: osdmap e51: 6 total, 5 up, 6 in 2024-05-12T04:04:42.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:42 smithi032 ceph-mon[120392]: osdmap e51: 6 total, 5 up, 6 in 2024-05-12T04:04:43.434 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:43 smithi032 ceph-mon[120392]: pgmap v48: 65 pgs: 16 active+undersized+degraded, 12 active+undersized, 33 active+clean, 4 stale+active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 45/276 objects degraded (16.304%) 2024-05-12T04:04:43.435 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:43 smithi032 ceph-mon[120392]: Health check failed: Degraded data redundancy: 45/276 objects degraded (16.304%), 16 pgs degraded (PG_DEGRADED) 2024-05-12T04:04:43.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:43 smithi033 ceph-mon[107305]: pgmap v48: 65 pgs: 16 active+undersized+degraded, 12 active+undersized, 33 active+clean, 4 stale+active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 45/276 objects degraded (16.304%) 2024-05-12T04:04:43.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:43 smithi033 ceph-mon[107305]: Health check failed: Degraded data redundancy: 45/276 objects degraded (16.304%), 16 pgs degraded (PG_DEGRADED) 2024-05-12T04:04:43.790 INFO:journalctl@ceph.osd.0.smithi032.stdout:May 12 04:04:43 smithi032 systemd[1]: ceph-a3e178f8-1012-11ef-bc98-c7b262605968@osd.0.service: Succeeded. 2024-05-12T04:04:43.790 INFO:journalctl@ceph.osd.0.smithi032.stdout:May 12 04:04:43 smithi032 systemd[1]: Stopped Ceph osd.0 for a3e178f8-1012-11ef-bc98-c7b262605968. 2024-05-12T04:04:43.790 INFO:journalctl@ceph.osd.0.smithi032.stdout:May 12 04:04:43 smithi032 systemd[1]: Starting Ceph osd.0 for a3e178f8-1012-11ef-bc98-c7b262605968... 2024-05-12T04:04:44.913 INFO:journalctl@ceph.osd.0.smithi032.stdout:May 12 04:04:44 smithi032 bash[131011]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-12T04:04:44.913 INFO:journalctl@ceph.osd.0.smithi032.stdout:May 12 04:04:44 smithi032 bash[131011]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-05-12T04:04:44.913 INFO:journalctl@ceph.osd.0.smithi032.stdout:May 12 04:04:44 smithi032 bash[131011]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-05-12T04:04:44.913 INFO:journalctl@ceph.osd.0.smithi032.stdout:May 12 04:04:44 smithi032 bash[131011]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-05-12T04:04:44.914 INFO:journalctl@ceph.osd.0.smithi032.stdout:May 12 04:04:44 smithi032 bash[131011]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-05-12T04:04:44.914 INFO:journalctl@ceph.osd.0.smithi032.stdout:May 12 04:04:44 smithi032 bash[131011]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-12T04:04:44.914 INFO:journalctl@ceph.osd.0.smithi032.stdout:May 12 04:04:44 smithi032 bash[131011]: --> ceph-volume raw activate successful for osd ID: 0 2024-05-12T04:04:45.644 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:45 smithi032 ceph-mon[120392]: pgmap v49: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 53/276 objects degraded (19.203%) 2024-05-12T04:04:45.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:45 smithi033 ceph-mon[107305]: pgmap v49: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 53/276 objects degraded (19.203%) 2024-05-12T04:04:46.040 INFO:journalctl@ceph.osd.0.smithi032.stdout:May 12 04:04:45 smithi032 ceph-osd[131330]: -- 172.21.15.32:0/571834116 <== mon.0 v2:172.21.15.32:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d25e2791e0 con 0x55d25e249000 2024-05-12T04:04:46.040 INFO:journalctl@ceph.osd.0.smithi032.stdout:May 12 04:04:45 smithi032 bash[131292]: 07f66c4deb3b45f9c27bd5373a7ac66f9899e0fe3d89de25f9a739b8fa57d7c1 2024-05-12T04:04:46.040 INFO:journalctl@ceph.osd.0.smithi032.stdout:May 12 04:04:45 smithi032 systemd[1]: Started Ceph osd.0 for a3e178f8-1012-11ef-bc98-c7b262605968. 2024-05-12T04:04:46.540 INFO:journalctl@ceph.osd.0.smithi032.stdout:May 12 04:04:46 smithi032 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-0[131307]: 2024-05-12T04:04:46.279+0000 7f55adf0a7c0 -1 Falling back to public interface 2024-05-12T04:04:47.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:46 smithi032 ceph-mon[120392]: pgmap v50: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 53/276 objects degraded (19.203%) 2024-05-12T04:04:47.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:46 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:47.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:46 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:04:47.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:46 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:47.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:46 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:47.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:46 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:04:47.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:46 smithi033 ceph-mon[107305]: pgmap v50: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 53/276 objects degraded (19.203%) 2024-05-12T04:04:47.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:46 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:47.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:46 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:04:47.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:46 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:47.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:46 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:47.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:46 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:04:48.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:47 smithi033 ceph-mon[107305]: mgrmap e40: smithi032.ukatkq(active, since 91s), standbys: smithi033.pcvxwy 2024-05-12T04:04:48.167 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:47 smithi032 ceph-mon[120392]: mgrmap e40: smithi032.ukatkq(active, since 91s), standbys: smithi033.pcvxwy 2024-05-12T04:04:49.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:48 smithi032 ceph-mon[120392]: pgmap v51: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 383 B/s wr, 0 op/s; 53/276 objects degraded (19.203%) 2024-05-12T04:04:49.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:48 smithi033 ceph-mon[107305]: pgmap v51: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 383 B/s wr, 0 op/s; 53/276 objects degraded (19.203%) 2024-05-12T04:04:50.910 INFO:journalctl@ceph.osd.0.smithi032.stdout:May 12 04:04:50 smithi032 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-0[131307]: 2024-05-12T04:04:50.588+0000 7f55adf0a7c0 -1 osd.0 49 log_to_monitors true 2024-05-12T04:04:51.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:50 smithi033 ceph-mon[107305]: pgmap v52: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 328 B/s wr, 0 op/s; 53/276 objects degraded (19.203%) 2024-05-12T04:04:51.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:50 smithi033 ceph-mon[107305]: Health check update: Degraded data redundancy: 53/276 objects degraded (19.203%), 19 pgs degraded (PG_DEGRADED) 2024-05-12T04:04:51.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:50 smithi033 ceph-mon[107305]: from='osd.0 [v2:172.21.15.32:6802/1676344197,v1:172.21.15.32:6803/1676344197]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-12T04:04:51.290 INFO:journalctl@ceph.osd.0.smithi032.stdout:May 12 04:04:50 smithi032 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-0[131307]: 2024-05-12T04:04:50.928+0000 7f559f363700 -1 osd.0 49 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-12T04:04:51.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:50 smithi032 ceph-mon[120392]: pgmap v52: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 260 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 328 B/s wr, 0 op/s; 53/276 objects degraded (19.203%) 2024-05-12T04:04:51.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:50 smithi032 ceph-mon[120392]: Health check update: Degraded data redundancy: 53/276 objects degraded (19.203%), 19 pgs degraded (PG_DEGRADED) 2024-05-12T04:04:51.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:50 smithi032 ceph-mon[120392]: from='osd.0 [v2:172.21.15.32:6802/1676344197,v1:172.21.15.32:6803/1676344197]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-12T04:04:52.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:51 smithi032 ceph-mon[120392]: from='osd.0 [v2:172.21.15.32:6802/1676344197,v1:172.21.15.32:6803/1676344197]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-12T04:04:52.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:51 smithi032 ceph-mon[120392]: osdmap e52: 6 total, 5 up, 6 in 2024-05-12T04:04:52.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:51 smithi032 ceph-mon[120392]: from='osd.0 [v2:172.21.15.32:6802/1676344197,v1:172.21.15.32:6803/1676344197]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2024-05-12T04:04:52.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:51 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:52.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:51 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:52.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:51 smithi033 ceph-mon[107305]: from='osd.0 [v2:172.21.15.32:6802/1676344197,v1:172.21.15.32:6803/1676344197]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-12T04:04:52.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:51 smithi033 ceph-mon[107305]: osdmap e52: 6 total, 5 up, 6 in 2024-05-12T04:04:52.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:51 smithi033 ceph-mon[107305]: from='osd.0 [v2:172.21.15.32:6802/1676344197,v1:172.21.15.32:6803/1676344197]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2024-05-12T04:04:52.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:51 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:52.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:51 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:53.211 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:52 smithi032 ceph-mon[120392]: pgmap v54: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 260 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 307 B/s wr, 0 op/s; 53/276 objects degraded (19.203%) 2024-05-12T04:04:53.211 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:52 smithi032 ceph-mon[120392]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-12T04:04:53.211 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:52 smithi032 ceph-mon[120392]: osd.0 [v2:172.21.15.32:6802/1676344197,v1:172.21.15.32:6803/1676344197] boot 2024-05-12T04:04:53.211 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:52 smithi032 ceph-mon[120392]: osdmap e53: 6 total, 6 up, 6 in 2024-05-12T04:04:53.211 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:52 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-12T04:04:53.212 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:52 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:53.212 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:52 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:53.212 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:52 smithi032 ceph-mon[120392]: osdmap e54: 6 total, 6 up, 6 in 2024-05-12T04:04:53.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:52 smithi033 ceph-mon[107305]: pgmap v54: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 260 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 307 B/s wr, 0 op/s; 53/276 objects degraded (19.203%) 2024-05-12T04:04:53.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:52 smithi033 ceph-mon[107305]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-12T04:04:53.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:52 smithi033 ceph-mon[107305]: osd.0 [v2:172.21.15.32:6802/1676344197,v1:172.21.15.32:6803/1676344197] boot 2024-05-12T04:04:53.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:52 smithi033 ceph-mon[107305]: osdmap e53: 6 total, 6 up, 6 in 2024-05-12T04:04:53.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:52 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-12T04:04:53.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:52 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:53.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:52 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:53.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:52 smithi033 ceph-mon[107305]: osdmap e54: 6 total, 6 up, 6 in 2024-05-12T04:04:55.192 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:54 smithi032 ceph-mon[120392]: pgmap v57: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 260 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 53/276 objects degraded (19.203%) 2024-05-12T04:04:55.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:54 smithi033 ceph-mon[107305]: pgmap v57: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 260 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 53/276 objects degraded (19.203%) 2024-05-12T04:04:56.913 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:56 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:56.913 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:56 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:56.913 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:56 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:04:56.913 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:56 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T04:04:56.913 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:56 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:56.913 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:56 smithi033 ceph-mon[107305]: pgmap v58: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 260 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 53/276 objects degraded (19.203%) 2024-05-12T04:04:56.913 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:56 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:04:56.914 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:56 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:04:56.914 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:56 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:04:56.914 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:56 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:04:56.914 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:56 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-05-12T04:04:56.914 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:56 smithi033 ceph-mon[107305]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-05-12T04:04:56.914 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:56 smithi033 ceph-mon[107305]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-05-12T04:04:57.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:56 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:57.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:56 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:57.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:56 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:04:57.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:56 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T04:04:57.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:56 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:04:57.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:56 smithi032 ceph-mon[120392]: pgmap v58: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 260 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 53/276 objects degraded (19.203%) 2024-05-12T04:04:57.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:56 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:04:57.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:56 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:04:57.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:56 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:04:57.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:56 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:04:57.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:56 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-05-12T04:04:57.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:56 smithi032 ceph-mon[120392]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-05-12T04:04:57.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:56 smithi032 ceph-mon[120392]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-05-12T04:04:58.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:57 smithi033 ceph-mon[107305]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 53/276 objects degraded (19.203%), 19 pgs degraded) 2024-05-12T04:04:58.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.166+0000 7f8c30e7c700 1 -- 172.21.15.32:0/3246155528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c2c066220 msgr2=0x7f8c2c0ff9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:04:58.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.166+0000 7f8c30e7c700 1 --2- 172.21.15.32:0/3246155528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c2c066220 0x7f8c2c0ff9b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8c14004940 tx=0x7f8c14009e30 comp rx=0 tx=0).stop 2024-05-12T04:04:58.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.166+0000 7f8c30e7c700 1 -- 172.21.15.32:0/3246155528 shutdown_connections 2024-05-12T04:04:58.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.166+0000 7f8c30e7c700 1 --2- 172.21.15.32:0/3246155528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c2c0ffef0 0x7f8c2c1022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:58.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.166+0000 7f8c30e7c700 1 --2- 172.21.15.32:0/3246155528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c2c066220 0x7f8c2c0ff9b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:58.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.166+0000 7f8c30e7c700 1 -- 172.21.15.32:0/3246155528 >> 172.21.15.32:0/3246155528 conn(0x7f8c2c0f7360 msgr2=0x7f8c2c0f97b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:04:58.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.166+0000 7f8c30e7c700 1 -- 172.21.15.32:0/3246155528 shutdown_connections 2024-05-12T04:04:58.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.166+0000 7f8c30e7c700 1 -- 172.21.15.32:0/3246155528 wait complete. 2024-05-12T04:04:58.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.167+0000 7f8c30e7c700 1 Processor -- start 2024-05-12T04:04:58.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.167+0000 7f8c30e7c700 1 -- start start 2024-05-12T04:04:58.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.167+0000 7f8c30e7c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c2c066220 0x7f8c2c10d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:04:58.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.167+0000 7f8c30e7c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c2c0ffef0 0x7f8c2c10dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:04:58.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.167+0000 7f8c30e7c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2c10e3a0 con 0x7f8c2c066220 2024-05-12T04:04:58.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.167+0000 7f8c30e7c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2c10ebf0 con 0x7f8c2c0ffef0 2024-05-12T04:04:58.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.167+0000 7f8c2a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c2c066220 0x7f8c2c10d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:04:58.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.167+0000 7f8c2a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c2c066220 0x7f8c2c10d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48104/0 (socket says 172.21.15.32:48104) 2024-05-12T04:04:58.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.167+0000 7f8c2a59c700 1 -- 172.21.15.32:0/158001073 learned_addr learned my addr 172.21.15.32:0/158001073 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:04:58.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.168+0000 7f8c29d9b700 1 --2- 172.21.15.32:0/158001073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c2c0ffef0 0x7f8c2c10dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:04:58.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.168+0000 7f8c2a59c700 1 -- 172.21.15.32:0/158001073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c2c0ffef0 msgr2=0x7f8c2c10dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:04:58.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.168+0000 7f8c2a59c700 1 --2- 172.21.15.32:0/158001073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c2c0ffef0 0x7f8c2c10dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:58.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.168+0000 7f8c2a59c700 1 -- 172.21.15.32:0/158001073 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c14005040 con 0x7f8c2c066220 2024-05-12T04:04:58.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.168+0000 7f8c2a59c700 1 --2- 172.21.15.32:0/158001073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c2c066220 0x7f8c2c10d860 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8c14000c00 tx=0x7f8c14009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:04:58.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.168+0000 7f8c22ffd700 1 -- 172.21.15.32:0/158001073 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c1400e070 con 0x7f8c2c066220 2024-05-12T04:04:58.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.168+0000 7f8c22ffd700 1 -- 172.21.15.32:0/158001073 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8c1401c400 con 0x7f8c2c066220 2024-05-12T04:04:58.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.168+0000 7f8c30e7c700 1 -- 172.21.15.32:0/158001073 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c2c10ee70 con 0x7f8c2c066220 2024-05-12T04:04:58.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.168+0000 7f8c22ffd700 1 -- 172.21.15.32:0/158001073 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c1400b650 con 0x7f8c2c066220 2024-05-12T04:04:58.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.168+0000 7f8c30e7c700 1 -- 172.21.15.32:0/158001073 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c2c10f350 con 0x7f8c2c066220 2024-05-12T04:04:58.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.169+0000 7f8c30e7c700 1 -- 172.21.15.32:0/158001073 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c2c05f5c0 con 0x7f8c2c066220 2024-05-12T04:04:58.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.170+0000 7f8c22ffd700 1 -- 172.21.15.32:0/158001073 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f8c1401c570 con 0x7f8c2c066220 2024-05-12T04:04:58.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.170+0000 7f8c22ffd700 1 --2- 172.21.15.32:0/158001073 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f8c1806cc30 0x7f8c1806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:04:58.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.170+0000 7f8c22ffd700 1 -- 172.21.15.32:0/158001073 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f8c1408d670 con 0x7f8c2c066220 2024-05-12T04:04:58.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.170+0000 7f8c29d9b700 1 --2- 172.21.15.32:0/158001073 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f8c1806cc30 0x7f8c1806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:04:58.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.171+0000 7f8c29d9b700 1 --2- 172.21.15.32:0/158001073 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f8c1806cc30 0x7f8c1806f0e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c004600 tx=0x7f8c1c00b040 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:04:58.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.175+0000 7f8c22ffd700 1 -- 172.21.15.32:0/158001073 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f8c1405b230 con 0x7f8c2c066220 2024-05-12T04:04:58.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:57 smithi032 ceph-mon[120392]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 53/276 objects degraded (19.203%), 19 pgs degraded) 2024-05-12T04:04:58.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.401+0000 7f8c30e7c700 1 -- 172.21.15.32:0/158001073 --> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8c2c10f6d0 con 0x7f8c1806cc30 2024-05-12T04:04:58.403 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.402+0000 7f8c22ffd700 1 -- 172.21.15.32:0/158001073 <== mgr.24593 v2:172.21.15.32:6800/2957111591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f8c2c10f6d0 con 0x7f8c1806cc30 2024-05-12T04:04:58.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.404+0000 7f8c30e7c700 1 -- 172.21.15.32:0/158001073 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f8c1806cc30 msgr2=0x7f8c1806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:04:58.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.404+0000 7f8c30e7c700 1 --2- 172.21.15.32:0/158001073 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f8c1806cc30 0x7f8c1806f0e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c004600 tx=0x7f8c1c00b040 comp rx=0 tx=0).stop 2024-05-12T04:04:58.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.404+0000 7f8c30e7c700 1 -- 172.21.15.32:0/158001073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c2c066220 msgr2=0x7f8c2c10d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:04:58.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.404+0000 7f8c30e7c700 1 --2- 172.21.15.32:0/158001073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c2c066220 0x7f8c2c10d860 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8c14000c00 tx=0x7f8c14009e30 comp rx=0 tx=0).stop 2024-05-12T04:04:58.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.404+0000 7f8c30e7c700 1 -- 172.21.15.32:0/158001073 shutdown_connections 2024-05-12T04:04:58.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.404+0000 7f8c30e7c700 1 --2- 172.21.15.32:0/158001073 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f8c1806cc30 0x7f8c1806f0e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:58.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.404+0000 7f8c30e7c700 1 --2- 172.21.15.32:0/158001073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c2c066220 0x7f8c2c10d860 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:58.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.404+0000 7f8c30e7c700 1 --2- 172.21.15.32:0/158001073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c2c0ffef0 0x7f8c2c10dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:58.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.404+0000 7f8c30e7c700 1 -- 172.21.15.32:0/158001073 >> 172.21.15.32:0/158001073 conn(0x7f8c2c0f7360 msgr2=0x7f8c2c0f77a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:04:58.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.404+0000 7f8c30e7c700 1 -- 172.21.15.32:0/158001073 shutdown_connections 2024-05-12T04:04:58.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.404+0000 7f8c30e7c700 1 -- 172.21.15.32:0/158001073 wait complete. 2024-05-12T04:04:58.416 INFO:teuthology.orchestra.run.smithi032.stdout:true 2024-05-12T04:04:58.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.509+0000 7f3190c53700 1 -- 172.21.15.32:0/1001637084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f318c0ffee0 msgr2=0x7f318c102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:04:58.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.509+0000 7f3190c53700 1 --2- 172.21.15.32:0/1001637084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f318c0ffee0 0x7f318c102300 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3180004940 tx=0x7f3180009e30 comp rx=0 tx=0).stop 2024-05-12T04:04:58.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.509+0000 7f3190c53700 1 -- 172.21.15.32:0/1001637084 shutdown_connections 2024-05-12T04:04:58.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.509+0000 7f3190c53700 1 --2- 172.21.15.32:0/1001637084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f318c0ffee0 0x7f318c102300 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:58.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.509+0000 7f3190c53700 1 --2- 172.21.15.32:0/1001637084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f318c0662c0 0x7f318c0ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:58.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.509+0000 7f3190c53700 1 -- 172.21.15.32:0/1001637084 >> 172.21.15.32:0/1001637084 conn(0x7f318c0f7430 msgr2=0x7f318c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:04:58.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.509+0000 7f3190c53700 1 -- 172.21.15.32:0/1001637084 shutdown_connections 2024-05-12T04:04:58.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.509+0000 7f3190c53700 1 -- 172.21.15.32:0/1001637084 wait complete. 2024-05-12T04:04:58.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.510+0000 7f3190c53700 1 Processor -- start 2024-05-12T04:04:58.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.510+0000 7f3190c53700 1 -- start start 2024-05-12T04:04:58.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.510+0000 7f3190c53700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f318c0662c0 0x7f318c10d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:04:58.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.510+0000 7f3190c53700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f318c0ffee0 0x7f318c10df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:04:58.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.510+0000 7f3190c53700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f318c10e520 con 0x7f318c0662c0 2024-05-12T04:04:58.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.510+0000 7f3190c53700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f318c10ecd0 con 0x7f318c0ffee0 2024-05-12T04:04:58.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.510+0000 7f318a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f318c0662c0 0x7f318c10d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:04:58.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.510+0000 7f318a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f318c0662c0 0x7f318c10d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48114/0 (socket says 172.21.15.32:48114) 2024-05-12T04:04:58.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.510+0000 7f318a59c700 1 -- 172.21.15.32:0/2720120947 learned_addr learned my addr 172.21.15.32:0/2720120947 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:04:58.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.510+0000 7f3189d9b700 1 --2- 172.21.15.32:0/2720120947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f318c0ffee0 0x7f318c10df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:04:58.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.510+0000 7f318a59c700 1 -- 172.21.15.32:0/2720120947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f318c0ffee0 msgr2=0x7f318c10df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:04:58.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.510+0000 7f318a59c700 1 --2- 172.21.15.32:0/2720120947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f318c0ffee0 0x7f318c10df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:58.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.510+0000 7f318a59c700 1 -- 172.21.15.32:0/2720120947 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3180005040 con 0x7f318c0662c0 2024-05-12T04:04:58.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.511+0000 7f318a59c700 1 --2- 172.21.15.32:0/2720120947 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f318c0662c0 0x7f318c10d9e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f317c002a50 tx=0x7f317c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:04:58.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.511+0000 7f317affd700 1 -- 172.21.15.32:0/2720120947 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f317c007cc0 con 0x7f318c0662c0 2024-05-12T04:04:58.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.511+0000 7f317affd700 1 -- 172.21.15.32:0/2720120947 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f317c00f040 con 0x7f318c0662c0 2024-05-12T04:04:58.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.511+0000 7f3190c53700 1 -- 172.21.15.32:0/2720120947 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f318c10efb0 con 0x7f318c0662c0 2024-05-12T04:04:58.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.511+0000 7f317affd700 1 -- 172.21.15.32:0/2720120947 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f317c00d920 con 0x7f318c0662c0 2024-05-12T04:04:58.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.511+0000 7f3190c53700 1 -- 172.21.15.32:0/2720120947 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f318c10f4f0 con 0x7f318c0662c0 2024-05-12T04:04:58.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.512+0000 7f3190c53700 1 -- 172.21.15.32:0/2720120947 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f318c05f5c0 con 0x7f318c0662c0 2024-05-12T04:04:58.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.512+0000 7f317affd700 1 -- 172.21.15.32:0/2720120947 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f317c009680 con 0x7f318c0662c0 2024-05-12T04:04:58.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.513+0000 7f317affd700 1 --2- 172.21.15.32:0/2720120947 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f317406cc20 0x7f317406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:04:58.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.513+0000 7f317affd700 1 -- 172.21.15.32:0/2720120947 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f317c08bcc0 con 0x7f318c0662c0 2024-05-12T04:04:58.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.513+0000 7f3189d9b700 1 --2- 172.21.15.32:0/2720120947 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f317406cc20 0x7f317406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:04:58.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.513+0000 7f3189d9b700 1 --2- 172.21.15.32:0/2720120947 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f317406cc20 0x7f317406f0d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f318000b900 tx=0x7f3180009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:04:58.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.517+0000 7f317affd700 1 -- 172.21.15.32:0/2720120947 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f317c0597e0 con 0x7f318c0662c0 2024-05-12T04:04:58.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.742+0000 7f3190c53700 1 -- 172.21.15.32:0/2720120947 --> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f318c10f8e0 con 0x7f317406cc20 2024-05-12T04:04:58.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.743+0000 7f317affd700 1 -- 172.21.15.32:0/2720120947 <== mgr.24593 v2:172.21.15.32:6800/2957111591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f318c10f8e0 con 0x7f317406cc20 2024-05-12T04:04:58.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.745+0000 7f3190c53700 1 -- 172.21.15.32:0/2720120947 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f317406cc20 msgr2=0x7f317406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:04:58.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.745+0000 7f3190c53700 1 --2- 172.21.15.32:0/2720120947 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f317406cc20 0x7f317406f0d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f318000b900 tx=0x7f3180009e30 comp rx=0 tx=0).stop 2024-05-12T04:04:58.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.745+0000 7f3190c53700 1 -- 172.21.15.32:0/2720120947 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f318c0662c0 msgr2=0x7f318c10d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:04:58.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.745+0000 7f3190c53700 1 --2- 172.21.15.32:0/2720120947 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f318c0662c0 0x7f318c10d9e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f317c002a50 tx=0x7f317c004600 comp rx=0 tx=0).stop 2024-05-12T04:04:58.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.746+0000 7f3190c53700 1 -- 172.21.15.32:0/2720120947 shutdown_connections 2024-05-12T04:04:58.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.746+0000 7f3190c53700 1 --2- 172.21.15.32:0/2720120947 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f317406cc20 0x7f317406f0d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:58.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.746+0000 7f3190c53700 1 --2- 172.21.15.32:0/2720120947 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f318c0662c0 0x7f318c10d9e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:58.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.746+0000 7f3190c53700 1 --2- 172.21.15.32:0/2720120947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f318c0ffee0 0x7f318c10df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:58.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.746+0000 7f3190c53700 1 -- 172.21.15.32:0/2720120947 >> 172.21.15.32:0/2720120947 conn(0x7f318c0f7430 msgr2=0x7f318c0f9810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:04:58.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.746+0000 7f3190c53700 1 -- 172.21.15.32:0/2720120947 shutdown_connections 2024-05-12T04:04:58.748 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.746+0000 7f3190c53700 1 -- 172.21.15.32:0/2720120947 wait complete. 2024-05-12T04:04:58.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.850+0000 7f2aa177e700 1 -- 172.21.15.32:0/1015090568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a9c0fb460 msgr2=0x7f2a9c0fb870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:04:58.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.850+0000 7f2aa177e700 1 --2- 172.21.15.32:0/1015090568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a9c0fb460 0x7f2a9c0fb870 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c004a50 tx=0x7f2a8c00af40 comp rx=0 tx=0).stop 2024-05-12T04:04:58.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.850+0000 7f2aa177e700 1 -- 172.21.15.32:0/1015090568 shutdown_connections 2024-05-12T04:04:58.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.850+0000 7f2aa177e700 1 --2- 172.21.15.32:0/1015090568 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a9c0fc710 0x7f2a9c0fcb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:58.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.850+0000 7f2aa177e700 1 --2- 172.21.15.32:0/1015090568 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a9c0fb460 0x7f2a9c0fb870 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:58.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.850+0000 7f2aa177e700 1 -- 172.21.15.32:0/1015090568 >> 172.21.15.32:0/1015090568 conn(0x7f2a9c0f6be0 msgr2=0x7f2a9c0f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:04:58.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.850+0000 7f2aa177e700 1 -- 172.21.15.32:0/1015090568 shutdown_connections 2024-05-12T04:04:58.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.850+0000 7f2aa177e700 1 -- 172.21.15.32:0/1015090568 wait complete. 2024-05-12T04:04:58.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.851+0000 7f2aa177e700 1 Processor -- start 2024-05-12T04:04:58.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.851+0000 7f2aa177e700 1 -- start start 2024-05-12T04:04:58.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.851+0000 7f2aa177e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a9c0fb460 0x7f2a9c10f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:04:58.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.851+0000 7f2aa177e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a9c0fc710 0x7f2a9c10ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:04:58.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.851+0000 7f2aa177e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9c110500 con 0x7f2a9c0fb460 2024-05-12T04:04:58.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.851+0000 7f2aa177e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9c110670 con 0x7f2a9c0fc710 2024-05-12T04:04:58.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.851+0000 7f2a9affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a9c0fb460 0x7f2a9c10f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:04:58.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.851+0000 7f2a9affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a9c0fb460 0x7f2a9c10f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48132/0 (socket says 172.21.15.32:48132) 2024-05-12T04:04:58.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.851+0000 7f2a9affd700 1 -- 172.21.15.32:0/1807332057 learned_addr learned my addr 172.21.15.32:0/1807332057 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:04:58.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.851+0000 7f2a9a7fc700 1 --2- 172.21.15.32:0/1807332057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a9c0fc710 0x7f2a9c10ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:04:58.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.851+0000 7f2a9affd700 1 -- 172.21.15.32:0/1807332057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a9c0fc710 msgr2=0x7f2a9c10ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:04:58.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.851+0000 7f2a9affd700 1 --2- 172.21.15.32:0/1807332057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a9c0fc710 0x7f2a9c10ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:58.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.852+0000 7f2a9affd700 1 -- 172.21.15.32:0/1807332057 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a8c005040 con 0x7f2a9c0fb460 2024-05-12T04:04:58.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.852+0000 7f2a9affd700 1 --2- 172.21.15.32:0/1807332057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a9c0fb460 0x7f2a9c10f9c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c004710 tx=0x7f2a8c00af40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:04:58.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.852+0000 7f2a937fe700 1 -- 172.21.15.32:0/1807332057 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a8c00f030 con 0x7f2a9c0fb460 2024-05-12T04:04:58.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.852+0000 7f2a937fe700 1 -- 172.21.15.32:0/1807332057 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2a8c00cdc0 con 0x7f2a9c0fb460 2024-05-12T04:04:58.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.852+0000 7f2aa177e700 1 -- 172.21.15.32:0/1807332057 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a9c115070 con 0x7f2a9c0fb460 2024-05-12T04:04:58.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.852+0000 7f2a937fe700 1 -- 172.21.15.32:0/1807332057 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a8c01d6c0 con 0x7f2a9c0fb460 2024-05-12T04:04:58.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.852+0000 7f2aa177e700 1 -- 172.21.15.32:0/1807332057 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a9c115550 con 0x7f2a9c0fb460 2024-05-12T04:04:58.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.853+0000 7f2aa177e700 1 -- 172.21.15.32:0/1807332057 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a9c04c710 con 0x7f2a9c0fb460 2024-05-12T04:04:58.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.854+0000 7f2a937fe700 1 -- 172.21.15.32:0/1807332057 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f2a8c008590 con 0x7f2a9c0fb460 2024-05-12T04:04:58.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.854+0000 7f2a937fe700 1 --2- 172.21.15.32:0/1807332057 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f2a8806cc20 0x7f2a8806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:04:58.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.854+0000 7f2a937fe700 1 -- 172.21.15.32:0/1807332057 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f2a8c08d890 con 0x7f2a9c0fb460 2024-05-12T04:04:58.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.854+0000 7f2a9a7fc700 1 --2- 172.21.15.32:0/1807332057 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f2a8806cc20 0x7f2a8806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:04:58.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.855+0000 7f2a9a7fc700 1 --2- 172.21.15.32:0/1807332057 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f2a8806cc20 0x7f2a8806f0d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2a84007fd0 tx=0x7f2a84009040 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:04:58.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:58.858+0000 7f2a937fe700 1 -- 172.21.15.32:0/1807332057 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f2a8c05b450 con 0x7f2a9c0fb460 2024-05-12T04:04:59.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:58 smithi032 ceph-mon[120392]: pgmap v59: 65 pgs: 65 active+clean; 260 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 41 B/s, 0 objects/s recovering 2024-05-12T04:04:59.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.075+0000 7f2aa177e700 1 -- 172.21.15.32:0/1807332057 --> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2a9c100a20 con 0x7f2a8806cc20 2024-05-12T04:04:59.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.085+0000 7f2a937fe700 1 -- 172.21.15.32:0/1807332057 <== mgr.24593 v2:172.21.15.32:6800/2957111591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f2a9c100a20 con 0x7f2a8806cc20 2024-05-12T04:04:59.086 INFO:teuthology.orchestra.run.smithi032.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-12T04:04:59.086 INFO:teuthology.orchestra.run.smithi032.stdout:alertmanager.smithi032 smithi032 *:9093,9094 running (4m) 7s ago 12m 25.5M - 0.25.0 c8568f914cd2 a37fbab7af4c 2024-05-12T04:04:59.086 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi032 smithi032 running (42s) 7s ago 13m 6660k - 18.2.2-1272-g58249552 ddcb0a9d6fde 8c53f1fc7b06 2024-05-12T04:04:59.086 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi033 smithi033 running (36s) 32s ago 12m 6651k - 18.2.2-1272-g58249552 ddcb0a9d6fde cf4fe9ca070b 2024-05-12T04:04:59.086 INFO:teuthology.orchestra.run.smithi032.stdout:grafana.smithi032 smithi032 *:3000 running (3m) 7s ago 12m 82.7M - 9.4.7 954c08fa6188 83055e58d23c 2024-05-12T04:04:59.086 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.ialtqb smithi032 running (7m) 7s ago 7m 13.6M - 17.2.7-939-gc190bdee 284803f7d9cf 0683abf00e3e 2024-05-12T04:04:59.087 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.qpsawh smithi032 running (7m) 7s ago 7m 155M - 17.2.7-939-gc190bdee 284803f7d9cf 95a7db5c4fcd 2024-05-12T04:04:59.087 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi033.hglwml smithi033 running (6m) 32s ago 6m 13.2M - 17.2.7-939-gc190bdee 284803f7d9cf c97902b05025 2024-05-12T04:04:59.087 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi033.jgbjhk smithi033 running (7m) 32s ago 7m 102M - 17.2.7-939-gc190bdee 284803f7d9cf af97052cb97a 2024-05-12T04:04:59.087 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi032.ukatkq smithi032 *:8443,9283,8765 running (3m) 7s ago 14m 498M - 18.2.2-1272-g58249552 ddcb0a9d6fde 0d88ab9a5292 2024-05-12T04:04:59.087 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi033.pcvxwy smithi033 *:8443,9283,8765 running (2m) 32s ago 11m 448M - 18.2.2-1272-g58249552 ddcb0a9d6fde d42dcb6bd307 2024-05-12T04:04:59.087 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi032 smithi032 running (119s) 7s ago 14m 41.8M 2048M 18.2.2-1272-g58249552 ddcb0a9d6fde 81f94171c5dc 2024-05-12T04:04:59.087 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi033 smithi033 running (86s) 32s ago 11m 32.0M 2048M 18.2.2-1272-g58249552 ddcb0a9d6fde d648ba108f3e 2024-05-12T04:04:59.087 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi032 smithi032 *:9100 running (4m) 7s ago 13m 16.7M - 1.5.0 0da6a335fe13 cb75a7c0c829 2024-05-12T04:04:59.087 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi033 smithi033 *:9100 running (3m) 32s ago 12m 18.3M - 1.5.0 0da6a335fe13 7888c36c659f 2024-05-12T04:04:59.087 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0 smithi032 running (13s) 7s ago 11m 11.1M 1570M 18.2.2-1272-g58249552 ddcb0a9d6fde 07f66c4deb3b 2024-05-12T04:04:59.087 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1 smithi032 running (10m) 7s ago 10m 228M 1570M 17.2.7-939-gc190bdee 284803f7d9cf 87cfd247d8d6 2024-05-12T04:04:59.087 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2 smithi032 running (10m) 7s ago 10m 182M 1570M 17.2.7-939-gc190bdee 284803f7d9cf 7aef7affb8ba 2024-05-12T04:04:59.087 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3 smithi033 running (9m) 32s ago 9m 240M 2594M 17.2.7-939-gc190bdee 284803f7d9cf 3a92b5e3348e 2024-05-12T04:04:59.087 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4 smithi033 running (9m) 32s ago 9m 213M 2594M 17.2.7-939-gc190bdee 284803f7d9cf 0896268b4591 2024-05-12T04:04:59.087 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5 smithi033 running (8m) 32s ago 8m 196M 2594M 17.2.7-939-gc190bdee 284803f7d9cf 57e65a64c821 2024-05-12T04:04:59.088 INFO:teuthology.orchestra.run.smithi032.stdout:prometheus.smithi032 smithi032 *:9095 running (2m) 7s ago 12m 51.8M - 2.43.0 a07b618ecd1d 46eafb25e176 2024-05-12T04:04:59.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.087+0000 7f2aa177e700 1 -- 172.21.15.32:0/1807332057 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f2a8806cc20 msgr2=0x7f2a8806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:04:59.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.087+0000 7f2aa177e700 1 --2- 172.21.15.32:0/1807332057 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f2a8806cc20 0x7f2a8806f0d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2a84007fd0 tx=0x7f2a84009040 comp rx=0 tx=0).stop 2024-05-12T04:04:59.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.087+0000 7f2aa177e700 1 -- 172.21.15.32:0/1807332057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a9c0fb460 msgr2=0x7f2a9c10f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:04:59.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.087+0000 7f2aa177e700 1 --2- 172.21.15.32:0/1807332057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a9c0fb460 0x7f2a9c10f9c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c004710 tx=0x7f2a8c00af40 comp rx=0 tx=0).stop 2024-05-12T04:04:59.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.087+0000 7f2aa177e700 1 -- 172.21.15.32:0/1807332057 shutdown_connections 2024-05-12T04:04:59.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.087+0000 7f2aa177e700 1 --2- 172.21.15.32:0/1807332057 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f2a8806cc20 0x7f2a8806f0d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:59.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.087+0000 7f2aa177e700 1 --2- 172.21.15.32:0/1807332057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a9c0fb460 0x7f2a9c10f9c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:59.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.087+0000 7f2aa177e700 1 --2- 172.21.15.32:0/1807332057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a9c0fc710 0x7f2a9c10ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:59.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.087+0000 7f2aa177e700 1 -- 172.21.15.32:0/1807332057 >> 172.21.15.32:0/1807332057 conn(0x7f2a9c0f6be0 msgr2=0x7f2a9c0f8220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:04:59.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.087+0000 7f2aa177e700 1 -- 172.21.15.32:0/1807332057 shutdown_connections 2024-05-12T04:04:59.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.087+0000 7f2aa177e700 1 -- 172.21.15.32:0/1807332057 wait complete. 2024-05-12T04:04:59.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:58 smithi033 ceph-mon[107305]: pgmap v59: 65 pgs: 65 active+clean; 260 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 41 B/s, 0 objects/s recovering 2024-05-12T04:04:59.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.190+0000 7f1005ab7700 1 -- 172.21.15.32:0/1125540390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1000070810 msgr2=0x7f1000070c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:04:59.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.190+0000 7f1005ab7700 1 --2- 172.21.15.32:0/1125540390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1000070810 0x7f1000070c80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4004940 tx=0x7f0ff4009e30 comp rx=0 tx=0).stop 2024-05-12T04:04:59.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.190+0000 7f1005ab7700 1 -- 172.21.15.32:0/1125540390 shutdown_connections 2024-05-12T04:04:59.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.190+0000 7f1005ab7700 1 --2- 172.21.15.32:0/1125540390 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1000070810 0x7f1000070c80 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:59.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.190+0000 7f1005ab7700 1 --2- 172.21.15.32:0/1125540390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1000071dc0 0x7f10000702d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:59.191 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.190+0000 7f1005ab7700 1 -- 172.21.15.32:0/1125540390 >> 172.21.15.32:0/1125540390 conn(0x7f10000f8590 msgr2=0x7f10000fa9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:04:59.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.190+0000 7f1005ab7700 1 -- 172.21.15.32:0/1125540390 shutdown_connections 2024-05-12T04:04:59.192 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.190+0000 7f1005ab7700 1 -- 172.21.15.32:0/1125540390 wait complete. 2024-05-12T04:04:59.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.190+0000 7f1005ab7700 1 Processor -- start 2024-05-12T04:04:59.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.191+0000 7f1005ab7700 1 -- start start 2024-05-12T04:04:59.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.191+0000 7f1005ab7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1000070810 0x7f100010fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:04:59.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.191+0000 7f1005ab7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1000071dc0 0x7f1000110070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:04:59.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.191+0000 7f1005ab7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1000110670 con 0x7f1000070810 2024-05-12T04:04:59.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.191+0000 7f1005ab7700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1000110ec0 con 0x7f1000071dc0 2024-05-12T04:04:59.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.191+0000 7f0fff7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1000070810 0x7f100010fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:04:59.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.191+0000 7f0fff7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1000070810 0x7f100010fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48150/0 (socket says 172.21.15.32:48150) 2024-05-12T04:04:59.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.191+0000 7f0fff7fe700 1 -- 172.21.15.32:0/539977783 learned_addr learned my addr 172.21.15.32:0/539977783 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:04:59.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.191+0000 7f0ffeffd700 1 --2- 172.21.15.32:0/539977783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1000071dc0 0x7f1000110070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:04:59.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.191+0000 7f0fff7fe700 1 -- 172.21.15.32:0/539977783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1000071dc0 msgr2=0x7f1000110070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:04:59.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.191+0000 7f0fff7fe700 1 --2- 172.21.15.32:0/539977783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1000071dc0 0x7f1000110070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:59.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.191+0000 7f0fff7fe700 1 -- 172.21.15.32:0/539977783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff4005040 con 0x7f1000070810 2024-05-12T04:04:59.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.191+0000 7f0fff7fe700 1 --2- 172.21.15.32:0/539977783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1000070810 0x7f100010fb30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0ff000c8e0 tx=0x7f0ff00044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:04:59.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.192+0000 7f0fe7fff700 1 -- 172.21.15.32:0/539977783 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ff000a760 con 0x7f1000070810 2024-05-12T04:04:59.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.192+0000 7f1005ab7700 1 -- 172.21.15.32:0/539977783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10001111a0 con 0x7f1000070810 2024-05-12T04:04:59.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.192+0000 7f0fe7fff700 1 -- 172.21.15.32:0/539977783 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0ff000ad70 con 0x7f1000070810 2024-05-12T04:04:59.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.192+0000 7f0fe7fff700 1 -- 172.21.15.32:0/539977783 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ff00113a0 con 0x7f1000070810 2024-05-12T04:04:59.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.192+0000 7f1005ab7700 1 -- 172.21.15.32:0/539977783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10001116e0 con 0x7f1000070810 2024-05-12T04:04:59.195 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.193+0000 7f1005ab7700 1 -- 172.21.15.32:0/539977783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f100005f5c0 con 0x7f1000070810 2024-05-12T04:04:59.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.193+0000 7f0fe7fff700 1 -- 172.21.15.32:0/539977783 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f0ff0012400 con 0x7f1000070810 2024-05-12T04:04:59.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.194+0000 7f0fe7fff700 1 --2- 172.21.15.32:0/539977783 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f0fe806cc30 0x7f0fe806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:04:59.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.194+0000 7f0fe7fff700 1 -- 172.21.15.32:0/539977783 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f0ff008c030 con 0x7f1000070810 2024-05-12T04:04:59.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.194+0000 7f0ffeffd700 1 --2- 172.21.15.32:0/539977783 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f0fe806cc30 0x7f0fe806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:04:59.196 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.194+0000 7f0ffeffd700 1 --2- 172.21.15.32:0/539977783 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f0fe806cc30 0x7f0fe806f0e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4009fd0 tx=0x7f0ff4009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:04:59.199 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.198+0000 7f0fe7fff700 1 -- 172.21.15.32:0/539977783 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f0ff0059b50 con 0x7f1000070810 2024-05-12T04:04:59.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.485+0000 7f1005ab7700 1 -- 172.21.15.32:0/539977783 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1000111990 con 0x7f1000070810 2024-05-12T04:04:59.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.486+0000 7f0fe7fff700 1 -- 172.21.15.32:0/539977783 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+847 (secure 0 0 0) 0x7f0ff00596e0 con 0x7f1000070810 2024-05-12T04:04:59.488 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-05-12T04:04:59.488 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": { 2024-05-12T04:04:59.488 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 2 2024-05-12T04:04:59.488 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:04:59.488 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": { 2024-05-12T04:04:59.489 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 2 2024-05-12T04:04:59.489 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:04:59.489 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": { 2024-05-12T04:04:59.489 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 5, 2024-05-12T04:04:59.489 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 1 2024-05-12T04:04:59.490 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:04:59.490 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": { 2024-05-12T04:04:59.490 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 4 2024-05-12T04:04:59.490 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:04:59.490 INFO:teuthology.orchestra.run.smithi032.stdout: "overall": { 2024-05-12T04:04:59.490 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 9, 2024-05-12T04:04:59.491 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 5 2024-05-12T04:04:59.491 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-05-12T04:04:59.491 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-05-12T04:04:59.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.488+0000 7f1005ab7700 1 -- 172.21.15.32:0/539977783 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f0fe806cc30 msgr2=0x7f0fe806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:04:59.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.488+0000 7f1005ab7700 1 --2- 172.21.15.32:0/539977783 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f0fe806cc30 0x7f0fe806f0e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4009fd0 tx=0x7f0ff4009e30 comp rx=0 tx=0).stop 2024-05-12T04:04:59.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.488+0000 7f1005ab7700 1 -- 172.21.15.32:0/539977783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1000070810 msgr2=0x7f100010fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:04:59.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.488+0000 7f1005ab7700 1 --2- 172.21.15.32:0/539977783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1000070810 0x7f100010fb30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0ff000c8e0 tx=0x7f0ff00044c0 comp rx=0 tx=0).stop 2024-05-12T04:04:59.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.488+0000 7f1005ab7700 1 -- 172.21.15.32:0/539977783 shutdown_connections 2024-05-12T04:04:59.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.488+0000 7f1005ab7700 1 --2- 172.21.15.32:0/539977783 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f0fe806cc30 0x7f0fe806f0e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:59.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.488+0000 7f1005ab7700 1 --2- 172.21.15.32:0/539977783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1000070810 0x7f100010fb30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:59.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.488+0000 7f1005ab7700 1 --2- 172.21.15.32:0/539977783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1000071dc0 0x7f1000110070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:59.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.488+0000 7f1005ab7700 1 -- 172.21.15.32:0/539977783 >> 172.21.15.32:0/539977783 conn(0x7f10000f8590 msgr2=0x7f10000f9d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:04:59.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.488+0000 7f1005ab7700 1 -- 172.21.15.32:0/539977783 shutdown_connections 2024-05-12T04:04:59.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.488+0000 7f1005ab7700 1 -- 172.21.15.32:0/539977783 wait complete. 2024-05-12T04:04:59.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.592+0000 7fd2c9dcf700 1 -- 172.21.15.32:0/347770878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2c40662c0 msgr2=0x7fd2c40ff9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:04:59.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.592+0000 7fd2c9dcf700 1 --2- 172.21.15.32:0/347770878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2c40662c0 0x7fd2c40ff9a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd2b4004940 tx=0x7fd2b4009e30 comp rx=0 tx=0).stop 2024-05-12T04:04:59.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.592+0000 7fd2c9dcf700 1 -- 172.21.15.32:0/347770878 shutdown_connections 2024-05-12T04:04:59.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.592+0000 7fd2c9dcf700 1 --2- 172.21.15.32:0/347770878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2c40ffee0 0x7fd2c4102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:59.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.592+0000 7fd2c9dcf700 1 --2- 172.21.15.32:0/347770878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2c40662c0 0x7fd2c40ff9a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:59.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.592+0000 7fd2c9dcf700 1 -- 172.21.15.32:0/347770878 >> 172.21.15.32:0/347770878 conn(0x7fd2c40f7430 msgr2=0x7fd2c40f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:04:59.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.592+0000 7fd2c9dcf700 1 -- 172.21.15.32:0/347770878 shutdown_connections 2024-05-12T04:04:59.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.592+0000 7fd2c9dcf700 1 -- 172.21.15.32:0/347770878 wait complete. 2024-05-12T04:04:59.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.593+0000 7fd2c9dcf700 1 Processor -- start 2024-05-12T04:04:59.595 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.593+0000 7fd2c9dcf700 1 -- start start 2024-05-12T04:04:59.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.593+0000 7fd2c9dcf700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2c40662c0 0x7fd2c410d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:04:59.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.593+0000 7fd2c9dcf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2c40ffee0 0x7fd2c410dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:04:59.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.593+0000 7fd2c9dcf700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c410e4a0 con 0x7fd2c40ffee0 2024-05-12T04:04:59.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.593+0000 7fd2c9dcf700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c410ec50 con 0x7fd2c40662c0 2024-05-12T04:04:59.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.593+0000 7fd2c2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2c40ffee0 0x7fd2c410dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:04:59.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.593+0000 7fd2c2ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2c40ffee0 0x7fd2c410dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48160/0 (socket says 172.21.15.32:48160) 2024-05-12T04:04:59.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.593+0000 7fd2c2ffd700 1 -- 172.21.15.32:0/4267393091 learned_addr learned my addr 172.21.15.32:0/4267393091 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:04:59.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.593+0000 7fd2c37fe700 1 --2- 172.21.15.32:0/4267393091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2c40662c0 0x7fd2c410d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:04:59.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.593+0000 7fd2c2ffd700 1 -- 172.21.15.32:0/4267393091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2c40662c0 msgr2=0x7fd2c410d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:04:59.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.593+0000 7fd2c2ffd700 1 --2- 172.21.15.32:0/4267393091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2c40662c0 0x7fd2c410d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:59.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.593+0000 7fd2c2ffd700 1 -- 172.21.15.32:0/4267393091 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b4005040 con 0x7fd2c40ffee0 2024-05-12T04:04:59.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.594+0000 7fd2c2ffd700 1 --2- 172.21.15.32:0/4267393091 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2c40ffee0 0x7fd2c410dea0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd2b800eb40 tx=0x7fd2b800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:04:59.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.594+0000 7fd2abfff700 1 -- 172.21.15.32:0/4267393091 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2b8007cc0 con 0x7fd2c40ffee0 2024-05-12T04:04:59.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.594+0000 7fd2c9dcf700 1 -- 172.21.15.32:0/4267393091 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2c410eed0 con 0x7fd2c40ffee0 2024-05-12T04:04:59.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.594+0000 7fd2abfff700 1 -- 172.21.15.32:0/4267393091 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd2b800f040 con 0x7fd2c40ffee0 2024-05-12T04:04:59.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.594+0000 7fd2abfff700 1 -- 172.21.15.32:0/4267393091 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2b800c990 con 0x7fd2c40ffee0 2024-05-12T04:04:59.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.594+0000 7fd2c9dcf700 1 -- 172.21.15.32:0/4267393091 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2c410f410 con 0x7fd2c40ffee0 2024-05-12T04:04:59.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.595+0000 7fd2c9dcf700 1 -- 172.21.15.32:0/4267393091 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2c405f5c0 con 0x7fd2c40ffee0 2024-05-12T04:04:59.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.595+0000 7fd2abfff700 1 -- 172.21.15.32:0/4267393091 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fd2b800cc00 con 0x7fd2c40ffee0 2024-05-12T04:04:59.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.596+0000 7fd2abfff700 1 --2- 172.21.15.32:0/4267393091 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fd2ac06cc20 0x7fd2ac06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:04:59.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.596+0000 7fd2abfff700 1 -- 172.21.15.32:0/4267393091 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fd2b808bc10 con 0x7fd2c40ffee0 2024-05-12T04:04:59.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.596+0000 7fd2c37fe700 1 --2- 172.21.15.32:0/4267393091 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fd2ac06cc20 0x7fd2ac06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:04:59.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.596+0000 7fd2c37fe700 1 --2- 172.21.15.32:0/4267393091 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fd2ac06cc20 0x7fd2ac06f0d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd2b4013680 tx=0x7fd2b4009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:04:59.601 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.600+0000 7fd2abfff700 1 -- 172.21.15.32:0/4267393091 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fd2b8059760 con 0x7fd2c40ffee0 2024-05-12T04:04:59.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.855+0000 7fd2c9dcf700 1 -- 172.21.15.32:0/4267393091 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd2c410f6d0 con 0x7fd2c40ffee0 2024-05-12T04:04:59.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.856+0000 7fd2abfff700 1 -- 172.21.15.32:0/4267393091 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1866 (secure 0 0 0) 0x7fd2b80592f0 con 0x7fd2c40ffee0 2024-05-12T04:04:59.857 INFO:teuthology.orchestra.run.smithi032.stdout:e18 2024-05-12T04:04:59.857 INFO:teuthology.orchestra.run.smithi032.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-12T04:04:59.857 INFO:teuthology.orchestra.run.smithi032.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-12T04:04:59.857 INFO:teuthology.orchestra.run.smithi032.stdout:legacy client fscid: 1 2024-05-12T04:04:59.857 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:04:59.858 INFO:teuthology.orchestra.run.smithi032.stdout:Filesystem 'cephfs' (1) 2024-05-12T04:04:59.858 INFO:teuthology.orchestra.run.smithi032.stdout:fs_name cephfs 2024-05-12T04:04:59.858 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 18 2024-05-12T04:04:59.858 INFO:teuthology.orchestra.run.smithi032.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-12T04:04:59.858 INFO:teuthology.orchestra.run.smithi032.stdout:created 2024-05-12T03:57:48.904614+0000 2024-05-12T04:04:59.858 INFO:teuthology.orchestra.run.smithi032.stdout:modified 2024-05-12T03:58:06.821591+0000 2024-05-12T04:04:59.858 INFO:teuthology.orchestra.run.smithi032.stdout:tableserver 0 2024-05-12T04:04:59.858 INFO:teuthology.orchestra.run.smithi032.stdout:root 0 2024-05-12T04:04:59.858 INFO:teuthology.orchestra.run.smithi032.stdout:session_timeout 60 2024-05-12T04:04:59.858 INFO:teuthology.orchestra.run.smithi032.stdout:session_autoclose 300 2024-05-12T04:04:59.858 INFO:teuthology.orchestra.run.smithi032.stdout:max_file_size 1099511627776 2024-05-12T04:04:59.858 INFO:teuthology.orchestra.run.smithi032.stdout:max_xattr_size 65536 2024-05-12T04:04:59.858 INFO:teuthology.orchestra.run.smithi032.stdout:required_client_features {} 2024-05-12T04:04:59.858 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure 0 2024-05-12T04:04:59.859 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure_osd_epoch 46 2024-05-12T04:04:59.859 INFO:teuthology.orchestra.run.smithi032.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-12T04:04:59.859 INFO:teuthology.orchestra.run.smithi032.stdout:max_mds 1 2024-05-12T04:04:59.859 INFO:teuthology.orchestra.run.smithi032.stdout:in 0 2024-05-12T04:04:59.859 INFO:teuthology.orchestra.run.smithi032.stdout:up {0=14474} 2024-05-12T04:04:59.859 INFO:teuthology.orchestra.run.smithi032.stdout:failed 2024-05-12T04:04:59.859 INFO:teuthology.orchestra.run.smithi032.stdout:damaged 2024-05-12T04:04:59.859 INFO:teuthology.orchestra.run.smithi032.stdout:stopped 2024-05-12T04:04:59.859 INFO:teuthology.orchestra.run.smithi032.stdout:data_pools [3] 2024-05-12T04:04:59.859 INFO:teuthology.orchestra.run.smithi032.stdout:metadata_pool 2 2024-05-12T04:04:59.859 INFO:teuthology.orchestra.run.smithi032.stdout:inline_data enabled 2024-05-12T04:04:59.859 INFO:teuthology.orchestra.run.smithi032.stdout:balancer 2024-05-12T04:04:59.859 INFO:teuthology.orchestra.run.smithi032.stdout:bal_rank_mask -1 2024-05-12T04:04:59.859 INFO:teuthology.orchestra.run.smithi032.stdout:standby_count_wanted 1 2024-05-12T04:04:59.860 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.qpsawh{0:14474} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.32:6828/3100155296,v1:172.21.15.32:6829/3100155296] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:04:59.860 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi033.jgbjhk{0:24267} state up:standby-replay seq 3 join_fscid=1 addr [v2:172.21.15.33:6824/55417471,v1:172.21.15.33:6825/55417471] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:04:59.860 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:04:59.860 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:04:59.860 INFO:teuthology.orchestra.run.smithi032.stdout:Standby daemons: 2024-05-12T04:04:59.860 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:04:59.860 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi033.hglwml{-1:24283} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/433858238,v1:172.21.15.33:6827/433858238] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:04:59.860 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.ialtqb{-1:24287} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.32:6826/3703132329,v1:172.21.15.32:6827/3703132329] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:04:59.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.858+0000 7fd2c9dcf700 1 -- 172.21.15.32:0/4267393091 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fd2ac06cc20 msgr2=0x7fd2ac06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:04:59.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.858+0000 7fd2c9dcf700 1 --2- 172.21.15.32:0/4267393091 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fd2ac06cc20 0x7fd2ac06f0d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd2b4013680 tx=0x7fd2b4009e30 comp rx=0 tx=0).stop 2024-05-12T04:04:59.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.858+0000 7fd2c9dcf700 1 -- 172.21.15.32:0/4267393091 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2c40ffee0 msgr2=0x7fd2c410dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:04:59.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.858+0000 7fd2c9dcf700 1 --2- 172.21.15.32:0/4267393091 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2c40ffee0 0x7fd2c410dea0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd2b800eb40 tx=0x7fd2b800b040 comp rx=0 tx=0).stop 2024-05-12T04:04:59.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.858+0000 7fd2c9dcf700 1 -- 172.21.15.32:0/4267393091 shutdown_connections 2024-05-12T04:04:59.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.858+0000 7fd2c9dcf700 1 --2- 172.21.15.32:0/4267393091 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fd2ac06cc20 0x7fd2ac06f0d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:59.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.858+0000 7fd2c9dcf700 1 --2- 172.21.15.32:0/4267393091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2c40662c0 0x7fd2c410d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:59.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.858+0000 7fd2c9dcf700 1 --2- 172.21.15.32:0/4267393091 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2c40ffee0 0x7fd2c410dea0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:59.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.858+0000 7fd2c9dcf700 1 -- 172.21.15.32:0/4267393091 >> 172.21.15.32:0/4267393091 conn(0x7fd2c40f7430 msgr2=0x7fd2c40f9810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:04:59.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.858+0000 7fd2c9dcf700 1 -- 172.21.15.32:0/4267393091 shutdown_connections 2024-05-12T04:04:59.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.858+0000 7fd2c9dcf700 1 -- 172.21.15.32:0/4267393091 wait complete. 2024-05-12T04:04:59.861 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 18 2024-05-12T04:04:59.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.963+0000 7f26c0bfd700 1 -- 172.21.15.32:0/1800325461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26bc0fce80 msgr2=0x7f26bc0fd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:04:59.964 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.963+0000 7f26c0bfd700 1 --2- 172.21.15.32:0/1800325461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26bc0fce80 0x7f26bc0fd290 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f26ac004940 tx=0x7f26ac009e30 comp rx=0 tx=0).stop 2024-05-12T04:04:59.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.963+0000 7f26c0bfd700 1 -- 172.21.15.32:0/1800325461 shutdown_connections 2024-05-12T04:04:59.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.963+0000 7f26c0bfd700 1 --2- 172.21.15.32:0/1800325461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26bc0fe080 0x7f26bc0fe4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:59.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.963+0000 7f26c0bfd700 1 --2- 172.21.15.32:0/1800325461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26bc0fce80 0x7f26bc0fd290 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:59.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.963+0000 7f26c0bfd700 1 -- 172.21.15.32:0/1800325461 >> 172.21.15.32:0/1800325461 conn(0x7f26bc0f8590 msgr2=0x7f26bc0fa9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:04:59.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.963+0000 7f26c0bfd700 1 -- 172.21.15.32:0/1800325461 shutdown_connections 2024-05-12T04:04:59.965 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.963+0000 7f26c0bfd700 1 -- 172.21.15.32:0/1800325461 wait complete. 2024-05-12T04:04:59.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.964+0000 7f26c0bfd700 1 Processor -- start 2024-05-12T04:04:59.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.964+0000 7f26c0bfd700 1 -- start start 2024-05-12T04:04:59.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.964+0000 7f26c0bfd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26bc0fce80 0x7f26bc10d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:04:59.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.964+0000 7f26c0bfd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26bc0fe080 0x7f26bc10de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:04:59.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.964+0000 7f26c0bfd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26bc10e470 con 0x7f26bc0fe080 2024-05-12T04:04:59.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.964+0000 7f26c0bfd700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26bc10ecc0 con 0x7f26bc0fce80 2024-05-12T04:04:59.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.964+0000 7f26b9d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26bc0fe080 0x7f26bc10de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:04:59.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.964+0000 7f26b9d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26bc0fe080 0x7f26bc10de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48176/0 (socket says 172.21.15.32:48176) 2024-05-12T04:04:59.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.964+0000 7f26b9d9b700 1 -- 172.21.15.32:0/1033357957 learned_addr learned my addr 172.21.15.32:0/1033357957 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:04:59.967 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.964+0000 7f26ba59c700 1 --2- 172.21.15.32:0/1033357957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26bc0fce80 0x7f26bc10d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:04:59.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.964+0000 7f26b9d9b700 1 -- 172.21.15.32:0/1033357957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26bc0fce80 msgr2=0x7f26bc10d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:04:59.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.964+0000 7f26b9d9b700 1 --2- 172.21.15.32:0/1033357957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26bc0fce80 0x7f26bc10d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:04:59.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.965+0000 7f26b9d9b700 1 -- 172.21.15.32:0/1033357957 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26ac005040 con 0x7f26bc0fe080 2024-05-12T04:04:59.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.965+0000 7f26b9d9b700 1 --2- 172.21.15.32:0/1033357957 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26bc0fe080 0x7f26bc10de70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f26a400c8c0 tx=0x7f26a40044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:04:59.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.965+0000 7f26b2ffd700 1 -- 172.21.15.32:0/1033357957 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f26a400a980 con 0x7f26bc0fe080 2024-05-12T04:04:59.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.965+0000 7f26c0bfd700 1 -- 172.21.15.32:0/1033357957 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26bc10efa0 con 0x7f26bc0fe080 2024-05-12T04:04:59.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.965+0000 7f26b2ffd700 1 -- 172.21.15.32:0/1033357957 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f26a400ce60 con 0x7f26bc0fe080 2024-05-12T04:04:59.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.965+0000 7f26b2ffd700 1 -- 172.21.15.32:0/1033357957 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f26a40113a0 con 0x7f26bc0fe080 2024-05-12T04:04:59.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.965+0000 7f26c0bfd700 1 -- 172.21.15.32:0/1033357957 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26bc10f4e0 con 0x7f26bc0fe080 2024-05-12T04:04:59.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.966+0000 7f26c0bfd700 1 -- 172.21.15.32:0/1033357957 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26bc05f5c0 con 0x7f26bc0fe080 2024-05-12T04:04:59.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.966+0000 7f26b2ffd700 1 -- 172.21.15.32:0/1033357957 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f26a4012400 con 0x7f26bc0fe080 2024-05-12T04:04:59.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.967+0000 7f26b2ffd700 1 --2- 172.21.15.32:0/1033357957 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f26a806cc20 0x7f26a806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:04:59.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.967+0000 7f26b2ffd700 1 -- 172.21.15.32:0/1033357957 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f26a408c1b0 con 0x7f26bc0fe080 2024-05-12T04:04:59.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.967+0000 7f26ba59c700 1 --2- 172.21.15.32:0/1033357957 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f26a806cc20 0x7f26a806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:04:59.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.967+0000 7f26ba59c700 1 --2- 172.21.15.32:0/1033357957 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f26a806cc20 0x7f26a806f0d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f26ac00b900 tx=0x7f26ac009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:04:59.973 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:04:59.971+0000 7f26b2ffd700 1 -- 172.21.15.32:0/1033357957 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f26a4059c30 con 0x7f26bc0fe080 2024-05-12T04:05:00.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:59 smithi033 ceph-mon[107305]: from='client.34216 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:05:00.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:59 smithi033 ceph-mon[107305]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:05:00.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:59 smithi033 ceph-mon[107305]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:05:00.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:04:59 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/539977783' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:05:00.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:00.194+0000 7f26c0bfd700 1 -- 172.21.15.32:0/1033357957 --> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f26bc10f800 con 0x7f26a806cc20 2024-05-12T04:05:00.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:00.195+0000 7f26b2ffd700 1 -- 172.21.15.32:0/1033357957 <== mgr.24593 v2:172.21.15.32:6800/2957111591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f26bc10f800 con 0x7f26a806cc20 2024-05-12T04:05:00.199 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-05-12T04:05:00.199 INFO:teuthology.orchestra.run.smithi032.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:19c9734256aa43a60e203feb242f50bf8b7f58666d520536af43cc4fee96173a", 2024-05-12T04:05:00.199 INFO:teuthology.orchestra.run.smithi032.stdout: "in_progress": true, 2024-05-12T04:05:00.199 INFO:teuthology.orchestra.run.smithi032.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-12T04:05:00.199 INFO:teuthology.orchestra.run.smithi032.stdout: "services_complete": [ 2024-05-12T04:05:00.199 INFO:teuthology.orchestra.run.smithi032.stdout: "mon", 2024-05-12T04:05:00.199 INFO:teuthology.orchestra.run.smithi032.stdout: "crash", 2024-05-12T04:05:00.200 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr" 2024-05-12T04:05:00.200 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-05-12T04:05:00.200 INFO:teuthology.orchestra.run.smithi032.stdout: "progress": "7/21 daemons upgraded", 2024-05-12T04:05:00.200 INFO:teuthology.orchestra.run.smithi032.stdout: "message": "Currently upgrading osd daemons", 2024-05-12T04:05:00.200 INFO:teuthology.orchestra.run.smithi032.stdout: "is_paused": false 2024-05-12T04:05:00.200 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-05-12T04:05:00.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:00.197+0000 7f26c0bfd700 1 -- 172.21.15.32:0/1033357957 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f26a806cc20 msgr2=0x7f26a806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:05:00.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:00.197+0000 7f26c0bfd700 1 --2- 172.21.15.32:0/1033357957 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f26a806cc20 0x7f26a806f0d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f26ac00b900 tx=0x7f26ac009e30 comp rx=0 tx=0).stop 2024-05-12T04:05:00.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:00.197+0000 7f26c0bfd700 1 -- 172.21.15.32:0/1033357957 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26bc0fe080 msgr2=0x7f26bc10de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:05:00.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:00.197+0000 7f26c0bfd700 1 --2- 172.21.15.32:0/1033357957 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26bc0fe080 0x7f26bc10de70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f26a400c8c0 tx=0x7f26a40044c0 comp rx=0 tx=0).stop 2024-05-12T04:05:00.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:00.197+0000 7f26c0bfd700 1 -- 172.21.15.32:0/1033357957 shutdown_connections 2024-05-12T04:05:00.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:00.197+0000 7f26c0bfd700 1 --2- 172.21.15.32:0/1033357957 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f26a806cc20 0x7f26a806f0d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:05:00.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:00.197+0000 7f26c0bfd700 1 --2- 172.21.15.32:0/1033357957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26bc0fce80 0x7f26bc10d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:05:00.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:00.197+0000 7f26c0bfd700 1 --2- 172.21.15.32:0/1033357957 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26bc0fe080 0x7f26bc10de70 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:05:00.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:00.197+0000 7f26c0bfd700 1 -- 172.21.15.32:0/1033357957 >> 172.21.15.32:0/1033357957 conn(0x7f26bc0f8590 msgr2=0x7f26bc0f9da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:05:00.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:00.197+0000 7f26c0bfd700 1 -- 172.21.15.32:0/1033357957 shutdown_connections 2024-05-12T04:05:00.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:00.197+0000 7f26c0bfd700 1 -- 172.21.15.32:0/1033357957 wait complete. 2024-05-12T04:05:00.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:59 smithi032 ceph-mon[120392]: from='client.34216 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:05:00.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:59 smithi032 ceph-mon[120392]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:05:00.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:59 smithi032 ceph-mon[120392]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:05:00.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:04:59 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/539977783' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:05:00.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:00.302+0000 7f03c15c0700 1 -- 172.21.15.32:0/2208663366 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03bc0fcf60 msgr2=0x7f03bc0fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:05:00.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:00.302+0000 7f03c15c0700 1 --2- 172.21.15.32:0/2208663366 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03bc0fcf60 0x7f03bc0fd3d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f03a4004940 tx=0x7f03a4009e30 comp rx=0 tx=0).stop 2024-05-12T04:05:00.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:00.302+0000 7f03c15c0700 1 -- 172.21.15.32:0/2208663366 shutdown_connections 2024-05-12T04:05:00.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:00.302+0000 7f03c15c0700 1 --2- 172.21.15.32:0/2208663366 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03bc0fcf60 0x7f03bc0fd3d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:05:00.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:00.302+0000 7f03c15c0700 1 --2- 172.21.15.32:0/2208663366 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03bc0fbcb0 0x7f03bc0fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:05:00.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:00.302+0000 7f03c15c0700 1 -- 172.21.15.32:0/2208663366 >> 172.21.15.32:0/2208663366 conn(0x7f03bc0f7430 msgr2=0x7f03bc0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:05:00.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:00.302+0000 7f03c15c0700 1 -- 172.21.15.32:0/2208663366 shutdown_connections 2024-05-12T04:05:00.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:00.302+0000 7f03c15c0700 1 -- 172.21.15.32:0/2208663366 wait complete. 2024-05-12T04:05:00.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:00.303+0000 7f03c15c0700 1 Processor -- start 2024-05-12T04:05:00.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:00.303+0000 7f03c15c0700 1 -- start start 2024-05-12T04:05:00.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:00.303+0000 7f03c15c0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03bc0fbcb0 0x7f03bc10d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:05:00.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:00.303+0000 7f03c15c0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03bc0fcf60 0x7f03bc10de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:05:00.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:00.303+0000 7f03c15c0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03bc10e470 con 0x7f03bc0fbcb0 2024-05-12T04:05:00.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:00.303+0000 7f03c15c0700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03bc10ecc0 con 0x7f03bc0fcf60 2024-05-12T04:05:00.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:00.303+0000 7f03baffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03bc0fbcb0 0x7f03bc10d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:05:00.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:00.303+0000 7f03baffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03bc0fbcb0 0x7f03bc10d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48208/0 (socket says 172.21.15.32:48208) 2024-05-12T04:05:00.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:00.303+0000 7f03baffd700 1 -- 172.21.15.32:0/2094093030 learned_addr learned my addr 172.21.15.32:0/2094093030 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:05:00.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:00.303+0000 7f03ba7fc700 1 --2- 172.21.15.32:0/2094093030 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03bc0fcf60 0x7f03bc10de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:05:00.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:00.303+0000 7f03baffd700 1 -- 172.21.15.32:0/2094093030 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03bc0fcf60 msgr2=0x7f03bc10de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:05:00.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:00.303+0000 7f03baffd700 1 --2- 172.21.15.32:0/2094093030 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03bc0fcf60 0x7f03bc10de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:05:00.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:00.303+0000 7f03baffd700 1 -- 172.21.15.32:0/2094093030 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a4005040 con 0x7f03bc0fbcb0 2024-05-12T04:05:00.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:00.303+0000 7f03baffd700 1 --2- 172.21.15.32:0/2094093030 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03bc0fbcb0 0x7f03bc10d930 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f03ac002a50 tx=0x7f03ac0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:05:00.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:00.304+0000 7f03b37fe700 1 -- 172.21.15.32:0/2094093030 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03ac007cc0 con 0x7f03bc0fbcb0 2024-05-12T04:05:00.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:00.304+0000 7f03b37fe700 1 -- 172.21.15.32:0/2094093030 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f03ac00f040 con 0x7f03bc0fbcb0 2024-05-12T04:05:00.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:00.304+0000 7f03c15c0700 1 -- 172.21.15.32:0/2094093030 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03bc10efa0 con 0x7f03bc0fbcb0 2024-05-12T04:05:00.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:00.304+0000 7f03b37fe700 1 -- 172.21.15.32:0/2094093030 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03ac00d9c0 con 0x7f03bc0fbcb0 2024-05-12T04:05:00.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:00.304+0000 7f03c15c0700 1 -- 172.21.15.32:0/2094093030 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03bc10f4e0 con 0x7f03bc0fbcb0 2024-05-12T04:05:00.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:00.305+0000 7f03c15c0700 1 -- 172.21.15.32:0/2094093030 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03bc101af0 con 0x7f03bc0fbcb0 2024-05-12T04:05:00.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:00.305+0000 7f03b37fe700 1 -- 172.21.15.32:0/2094093030 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f03ac00db20 con 0x7f03bc0fbcb0 2024-05-12T04:05:00.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:00.306+0000 7f03b37fe700 1 --2- 172.21.15.32:0/2094093030 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f03a806cc20 0x7f03a806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:05:00.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:00.306+0000 7f03b37fe700 1 -- 172.21.15.32:0/2094093030 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f03ac08bc60 con 0x7f03bc0fbcb0 2024-05-12T04:05:00.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:00.306+0000 7f03ba7fc700 1 --2- 172.21.15.32:0/2094093030 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f03a806cc20 0x7f03a806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:05:00.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:00.306+0000 7f03ba7fc700 1 --2- 172.21.15.32:0/2094093030 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f03a806cc20 0x7f03a806f0d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f03a400b900 tx=0x7f03a4009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:05:00.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:00.310+0000 7f03b37fe700 1 -- 172.21.15.32:0/2094093030 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f03ac059780 con 0x7f03bc0fbcb0 2024-05-12T04:05:00.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:00.594+0000 7f03c15c0700 1 -- 172.21.15.32:0/2094093030 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f03bc104780 con 0x7f03bc0fbcb0 2024-05-12T04:05:00.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:00.595+0000 7f03b37fe700 1 -- 172.21.15.32:0/2094093030 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f03ac059310 con 0x7f03bc0fbcb0 2024-05-12T04:05:00.596 INFO:teuthology.orchestra.run.smithi032.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-12T04:05:00.597 INFO:teuthology.orchestra.run.smithi032.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-12T04:05:00.597 INFO:teuthology.orchestra.run.smithi032.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-12T04:05:00.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:00.597+0000 7f03c15c0700 1 -- 172.21.15.32:0/2094093030 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f03a806cc20 msgr2=0x7f03a806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:05:00.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:00.597+0000 7f03c15c0700 1 --2- 172.21.15.32:0/2094093030 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f03a806cc20 0x7f03a806f0d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f03a400b900 tx=0x7f03a4009e30 comp rx=0 tx=0).stop 2024-05-12T04:05:00.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:00.597+0000 7f03c15c0700 1 -- 172.21.15.32:0/2094093030 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03bc0fbcb0 msgr2=0x7f03bc10d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:05:00.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:00.597+0000 7f03c15c0700 1 --2- 172.21.15.32:0/2094093030 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03bc0fbcb0 0x7f03bc10d930 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f03ac002a50 tx=0x7f03ac0046b0 comp rx=0 tx=0).stop 2024-05-12T04:05:00.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:00.597+0000 7f03c15c0700 1 -- 172.21.15.32:0/2094093030 shutdown_connections 2024-05-12T04:05:00.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:00.597+0000 7f03c15c0700 1 --2- 172.21.15.32:0/2094093030 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f03a806cc20 0x7f03a806f0d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:05:00.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:00.597+0000 7f03c15c0700 1 --2- 172.21.15.32:0/2094093030 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03bc0fbcb0 0x7f03bc10d930 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:05:00.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:00.597+0000 7f03c15c0700 1 --2- 172.21.15.32:0/2094093030 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03bc0fcf60 0x7f03bc10de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:05:00.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:00.597+0000 7f03c15c0700 1 -- 172.21.15.32:0/2094093030 >> 172.21.15.32:0/2094093030 conn(0x7f03bc0f7430 msgr2=0x7f03bc0f8f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:05:00.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:00.597+0000 7f03c15c0700 1 -- 172.21.15.32:0/2094093030 shutdown_connections 2024-05-12T04:05:00.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:00.597+0000 7f03c15c0700 1 -- 172.21.15.32:0/2094093030 wait complete. 2024-05-12T04:05:01.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:00 smithi033 ceph-mon[107305]: pgmap v60: 65 pgs: 65 active+clean; 260 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 35 B/s, 0 objects/s recovering 2024-05-12T04:05:01.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:00 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/4267393091' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-12T04:05:01.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:00 smithi033 ceph-mon[107305]: from='client.34236 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:05:01.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:00 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/2094093030' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-12T04:05:01.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:00 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:05:01.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:00 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:05:01.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:00 smithi032 ceph-mon[120392]: pgmap v60: 65 pgs: 65 active+clean; 260 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 35 B/s, 0 objects/s recovering 2024-05-12T04:05:01.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:00 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/4267393091' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-12T04:05:01.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:00 smithi032 ceph-mon[120392]: from='client.34236 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:05:01.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:00 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/2094093030' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-12T04:05:01.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:00 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:05:01.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:00 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:05:03.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:02 smithi033 ceph-mon[107305]: pgmap v61: 65 pgs: 65 active+clean; 260 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 834 B/s rd, 1 op/s; 28 B/s, 0 objects/s recovering 2024-05-12T04:05:03.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:02 smithi032 ceph-mon[120392]: pgmap v61: 65 pgs: 65 active+clean; 260 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 834 B/s rd, 1 op/s; 28 B/s, 0 objects/s recovering 2024-05-12T04:05:05.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:04 smithi033 ceph-mon[107305]: pgmap v62: 65 pgs: 65 active+clean; 260 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 757 B/s rd, 0 op/s; 26 B/s, 0 objects/s recovering 2024-05-12T04:05:05.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:04 smithi032 ceph-mon[120392]: pgmap v62: 65 pgs: 65 active+clean; 260 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 757 B/s rd, 0 op/s; 26 B/s, 0 objects/s recovering 2024-05-12T04:05:07.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:06 smithi033 ceph-mon[107305]: pgmap v63: 65 pgs: 65 active+clean; 260 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 682 B/s rd, 0 op/s; 23 B/s, 0 objects/s recovering 2024-05-12T04:05:07.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:06 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:05:07.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:06 smithi032 ceph-mon[120392]: pgmap v63: 65 pgs: 65 active+clean; 260 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 682 B/s rd, 0 op/s; 23 B/s, 0 objects/s recovering 2024-05-12T04:05:07.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:06 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:05:09.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:08 smithi032 ceph-mon[120392]: pgmap v64: 65 pgs: 65 active+clean; 260 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s; 23 B/s, 0 objects/s recovering 2024-05-12T04:05:09.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:08 smithi033 ceph-mon[107305]: pgmap v64: 65 pgs: 65 active+clean; 260 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s; 23 B/s, 0 objects/s recovering 2024-05-12T04:05:11.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:10 smithi033 ceph-mon[107305]: pgmap v65: 65 pgs: 65 active+clean; 260 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:05:11.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:10 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-05-12T04:05:11.277 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:10 smithi032 ceph-mon[120392]: pgmap v65: 65 pgs: 65 active+clean; 260 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:05:11.277 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:10 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-05-12T04:05:11.990 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:11 smithi032 ceph-mon[120392]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-05-12T04:05:11.990 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:11 smithi032 ceph-mon[120392]: Upgrade: osd.1 is safe to restart 2024-05-12T04:05:12.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:11 smithi033 ceph-mon[107305]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-05-12T04:05:12.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:11 smithi033 ceph-mon[107305]: Upgrade: osd.1 is safe to restart 2024-05-12T04:05:13.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:13 smithi033 ceph-mon[107305]: pgmap v66: 65 pgs: 65 active+clean; 260 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:05:13.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:13 smithi033 ceph-mon[107305]: Upgrade: Updating osd.1 2024-05-12T04:05:13.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:13 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:05:13.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:13 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-12T04:05:13.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:13 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:05:13.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:13 smithi033 ceph-mon[107305]: Deploying daemon osd.1 on smithi032 2024-05-12T04:05:13.747 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:13 smithi032 ceph-mon[120392]: pgmap v66: 65 pgs: 65 active+clean; 260 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:05:13.747 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:13 smithi032 ceph-mon[120392]: Upgrade: Updating osd.1 2024-05-12T04:05:13.747 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:13 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:05:13.747 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:13 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-12T04:05:13.747 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:13 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:05:13.747 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:13 smithi032 ceph-mon[120392]: Deploying daemon osd.1 on smithi032 2024-05-12T04:05:15.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:15 smithi032 ceph-mon[120392]: pgmap v67: 65 pgs: 65 active+clean; 260 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:05:15.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:15 smithi033 ceph-mon[107305]: pgmap v67: 65 pgs: 65 active+clean; 260 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:05:16.290 INFO:journalctl@ceph.osd.1.smithi032.stdout:May 12 04:05:15 smithi032 systemd[1]: Stopping Ceph osd.1 for a3e178f8-1012-11ef-bc98-c7b262605968... 2024-05-12T04:05:16.290 INFO:journalctl@ceph.osd.1.smithi032.stdout:May 12 04:05:16 smithi032 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-1[73194]: 2024-05-12T04:05:16.138+0000 7fb3f51fe700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-12T04:05:16.290 INFO:journalctl@ceph.osd.1.smithi032.stdout:May 12 04:05:16 smithi032 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-1[73194]: 2024-05-12T04:05:16.138+0000 7fb3f51fe700 -1 osd.1 54 *** Got signal Terminated *** 2024-05-12T04:05:16.290 INFO:journalctl@ceph.osd.1.smithi032.stdout:May 12 04:05:16 smithi032 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-1[73194]: 2024-05-12T04:05:16.138+0000 7fb3f51fe700 -1 osd.1 54 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-12T04:05:16.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:16 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:05:16.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:16 smithi033 ceph-mon[107305]: osd.1 marked itself down and dead 2024-05-12T04:05:16.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:16 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:05:16.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:16 smithi032 ceph-mon[120392]: osd.1 marked itself down and dead 2024-05-12T04:05:17.394 INFO:journalctl@ceph.osd.1.smithi032.stdout:May 12 04:05:17 smithi032 bash[138010]: ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-1 2024-05-12T04:05:17.394 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:17 smithi032 ceph-mon[120392]: pgmap v68: 65 pgs: 65 active+clean; 260 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:05:17.394 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:17 smithi032 ceph-mon[120392]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-12T04:05:17.394 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:17 smithi032 ceph-mon[120392]: osdmap e55: 6 total, 5 up, 6 in 2024-05-12T04:05:17.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:17 smithi033 ceph-mon[107305]: pgmap v68: 65 pgs: 65 active+clean; 260 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:05:17.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:17 smithi033 ceph-mon[107305]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-12T04:05:17.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:17 smithi033 ceph-mon[107305]: osdmap e55: 6 total, 5 up, 6 in 2024-05-12T04:05:18.659 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:18 smithi032 ceph-mon[120392]: osdmap e56: 6 total, 5 up, 6 in 2024-05-12T04:05:18.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:18 smithi033 ceph-mon[107305]: osdmap e56: 6 total, 5 up, 6 in 2024-05-12T04:05:19.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:19 smithi032 ceph-mon[120392]: pgmap v71: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 260 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 2 op/s 2024-05-12T04:05:19.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:19 smithi032 ceph-mon[120392]: Health check failed: Reduced data availability: 9 pgs peering (PG_AVAILABILITY) 2024-05-12T04:05:19.541 INFO:journalctl@ceph.osd.1.smithi032.stdout:May 12 04:05:19 smithi032 systemd[1]: ceph-a3e178f8-1012-11ef-bc98-c7b262605968@osd.1.service: Succeeded. 2024-05-12T04:05:19.541 INFO:journalctl@ceph.osd.1.smithi032.stdout:May 12 04:05:19 smithi032 systemd[1]: Stopped Ceph osd.1 for a3e178f8-1012-11ef-bc98-c7b262605968. 2024-05-12T04:05:19.541 INFO:journalctl@ceph.osd.1.smithi032.stdout:May 12 04:05:19 smithi032 systemd[1]: Starting Ceph osd.1 for a3e178f8-1012-11ef-bc98-c7b262605968... 2024-05-12T04:05:19.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:19 smithi033 ceph-mon[107305]: pgmap v71: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 260 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 2 op/s 2024-05-12T04:05:19.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:19 smithi033 ceph-mon[107305]: Health check failed: Reduced data availability: 9 pgs peering (PG_AVAILABILITY) 2024-05-12T04:05:20.739 INFO:journalctl@ceph.osd.1.smithi032.stdout:May 12 04:05:20 smithi032 bash[138283]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-12T04:05:20.740 INFO:journalctl@ceph.osd.1.smithi032.stdout:May 12 04:05:20 smithi032 bash[138283]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-05-12T04:05:20.740 INFO:journalctl@ceph.osd.1.smithi032.stdout:May 12 04:05:20 smithi032 bash[138283]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-05-12T04:05:20.740 INFO:journalctl@ceph.osd.1.smithi032.stdout:May 12 04:05:20 smithi032 bash[138283]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-05-12T04:05:20.740 INFO:journalctl@ceph.osd.1.smithi032.stdout:May 12 04:05:20 smithi032 bash[138283]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-05-12T04:05:20.740 INFO:journalctl@ceph.osd.1.smithi032.stdout:May 12 04:05:20 smithi032 bash[138283]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-12T04:05:20.740 INFO:journalctl@ceph.osd.1.smithi032.stdout:May 12 04:05:20 smithi032 bash[138283]: --> ceph-volume raw activate successful for osd ID: 1 2024-05-12T04:05:21.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:21 smithi033 ceph-mon[107305]: pgmap v72: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 260 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-05-12T04:05:21.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:21 smithi032 ceph-mon[120392]: pgmap v72: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 260 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-05-12T04:05:21.790 INFO:journalctl@ceph.osd.1.smithi032.stdout:May 12 04:05:21 smithi032 ceph-osd[138583]: -- 172.21.15.32:0/3738929182 <== mon.0 v2:172.21.15.32:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557a4db531e0 con 0x557a4db23000 2024-05-12T04:05:21.790 INFO:journalctl@ceph.osd.1.smithi032.stdout:May 12 04:05:21 smithi032 bash[138546]: 67a87807d654f7d55cbd00ca9f0058e8ea89858dbd8a5b7411078b9c3d0e51dd 2024-05-12T04:05:21.790 INFO:journalctl@ceph.osd.1.smithi032.stdout:May 12 04:05:21 smithi032 systemd[1]: Started Ceph osd.1 for a3e178f8-1012-11ef-bc98-c7b262605968. 2024-05-12T04:05:22.360 INFO:journalctl@ceph.osd.1.smithi032.stdout:May 12 04:05:22 smithi032 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-1[138560]: 2024-05-12T04:05:22.063+0000 7fef0a1027c0 -1 Falling back to public interface 2024-05-12T04:05:22.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:22 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:05:22.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:22 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:05:22.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:22 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:05:22.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:22 smithi033 ceph-mon[107305]: pgmap v73: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 260 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 36/276 objects degraded (13.043%) 2024-05-12T04:05:23.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:22 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:05:23.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:22 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:05:23.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:22 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:05:23.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:22 smithi032 ceph-mon[120392]: pgmap v73: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 260 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 36/276 objects degraded (13.043%) 2024-05-12T04:05:23.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:23 smithi032 ceph-mon[120392]: Health check failed: Degraded data redundancy: 36/276 objects degraded (13.043%), 13 pgs degraded (PG_DEGRADED) 2024-05-12T04:05:23.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:23 smithi032 ceph-mon[120392]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 9 pgs peering) 2024-05-12T04:05:23.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:23 smithi033 ceph-mon[107305]: Health check failed: Degraded data redundancy: 36/276 objects degraded (13.043%), 13 pgs degraded (PG_DEGRADED) 2024-05-12T04:05:23.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:23 smithi033 ceph-mon[107305]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 9 pgs peering) 2024-05-12T04:05:24.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:24 smithi033 ceph-mon[107305]: pgmap v74: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 260 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 36/276 objects degraded (13.043%) 2024-05-12T04:05:25.039 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:24 smithi032 ceph-mon[120392]: pgmap v74: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 260 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 36/276 objects degraded (13.043%) 2024-05-12T04:05:27.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:26 smithi032 ceph-mon[120392]: pgmap v75: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 260 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 36/276 objects degraded (13.043%) 2024-05-12T04:05:27.040 INFO:journalctl@ceph.osd.1.smithi032.stdout:May 12 04:05:26 smithi032 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-1[138560]: 2024-05-12T04:05:26.957+0000 7fef0a1027c0 -1 osd.1 54 log_to_monitors true 2024-05-12T04:05:27.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:26 smithi033 ceph-mon[107305]: pgmap v75: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 260 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 36/276 objects degraded (13.043%) 2024-05-12T04:05:28.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:27 smithi033 ceph-mon[107305]: from='osd.1 [v2:172.21.15.32:6810/954580115,v1:172.21.15.32:6811/954580115]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-12T04:05:28.181 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:27 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:05:28.181 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:27 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:05:28.181 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:27 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:05:28.181 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:27 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:05:28.181 INFO:journalctl@ceph.osd.1.smithi032.stdout:May 12 04:05:27 smithi032 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-1[138560]: 2024-05-12T04:05:27.812+0000 7feefb55b700 -1 osd.1 54 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-12T04:05:28.182 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:27 smithi032 ceph-mon[120392]: from='osd.1 [v2:172.21.15.32:6810/954580115,v1:172.21.15.32:6811/954580115]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-12T04:05:28.182 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:27 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:05:28.182 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:27 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:05:28.182 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:27 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:05:28.182 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:27 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:05:29.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:28 smithi032 ceph-mon[120392]: pgmap v76: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 260 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 36/276 objects degraded (13.043%) 2024-05-12T04:05:29.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:28 smithi032 ceph-mon[120392]: from='osd.1 [v2:172.21.15.32:6810/954580115,v1:172.21.15.32:6811/954580115]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-12T04:05:29.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:28 smithi032 ceph-mon[120392]: osdmap e57: 6 total, 5 up, 6 in 2024-05-12T04:05:29.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:28 smithi032 ceph-mon[120392]: from='osd.1 [v2:172.21.15.32:6810/954580115,v1:172.21.15.32:6811/954580115]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2024-05-12T04:05:29.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:28 smithi033 ceph-mon[107305]: pgmap v76: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 260 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 36/276 objects degraded (13.043%) 2024-05-12T04:05:29.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:28 smithi033 ceph-mon[107305]: from='osd.1 [v2:172.21.15.32:6810/954580115,v1:172.21.15.32:6811/954580115]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-12T04:05:29.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:28 smithi033 ceph-mon[107305]: osdmap e57: 6 total, 5 up, 6 in 2024-05-12T04:05:29.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:28 smithi033 ceph-mon[107305]: from='osd.1 [v2:172.21.15.32:6810/954580115,v1:172.21.15.32:6811/954580115]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2024-05-12T04:05:30.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:29 smithi033 ceph-mon[107305]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-12T04:05:30.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:29 smithi033 ceph-mon[107305]: osd.1 [v2:172.21.15.32:6810/954580115,v1:172.21.15.32:6811/954580115] boot 2024-05-12T04:05:30.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:29 smithi033 ceph-mon[107305]: osdmap e58: 6 total, 6 up, 6 in 2024-05-12T04:05:30.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:29 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-12T04:05:30.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:29 smithi033 ceph-mon[107305]: osdmap e59: 6 total, 6 up, 6 in 2024-05-12T04:05:30.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:29 smithi032 ceph-mon[120392]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-12T04:05:30.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:29 smithi032 ceph-mon[120392]: osd.1 [v2:172.21.15.32:6810/954580115,v1:172.21.15.32:6811/954580115] boot 2024-05-12T04:05:30.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:29 smithi032 ceph-mon[120392]: osdmap e58: 6 total, 6 up, 6 in 2024-05-12T04:05:30.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:29 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-12T04:05:30.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:29 smithi032 ceph-mon[120392]: osdmap e59: 6 total, 6 up, 6 in 2024-05-12T04:05:30.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:30.707+0000 7fc158f11700 1 -- 172.21.15.32:0/1442122046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1540fcf60 msgr2=0x7fc1540fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:05:30.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:30.707+0000 7fc158f11700 1 --2- 172.21.15.32:0/1442122046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1540fcf60 0x7fc1540fd3d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc148004940 tx=0x7fc148009e30 comp rx=0 tx=0).stop 2024-05-12T04:05:30.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:30.707+0000 7fc158f11700 1 -- 172.21.15.32:0/1442122046 shutdown_connections 2024-05-12T04:05:30.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:30.707+0000 7fc158f11700 1 --2- 172.21.15.32:0/1442122046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1540fcf60 0x7fc1540fd3d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:05:30.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:30.707+0000 7fc158f11700 1 --2- 172.21.15.32:0/1442122046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1540fbcb0 0x7fc1540fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:05:30.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:30.707+0000 7fc158f11700 1 -- 172.21.15.32:0/1442122046 >> 172.21.15.32:0/1442122046 conn(0x7fc1540f7430 msgr2=0x7fc1540f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:05:30.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:30.707+0000 7fc158f11700 1 -- 172.21.15.32:0/1442122046 shutdown_connections 2024-05-12T04:05:30.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:30.707+0000 7fc158f11700 1 -- 172.21.15.32:0/1442122046 wait complete. 2024-05-12T04:05:30.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:30.708+0000 7fc158f11700 1 Processor -- start 2024-05-12T04:05:30.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:30.708+0000 7fc158f11700 1 -- start start 2024-05-12T04:05:30.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:30.708+0000 7fc158f11700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1540fbcb0 0x7fc15410d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:05:30.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:30.708+0000 7fc158f11700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1540fcf60 0x7fc15410de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:05:30.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:30.708+0000 7fc158f11700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc15410e410 con 0x7fc1540fcf60 2024-05-12T04:05:30.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:30.708+0000 7fc158f11700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc15410ec60 con 0x7fc1540fbcb0 2024-05-12T04:05:30.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:30.708+0000 7fc151d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1540fcf60 0x7fc15410de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:05:30.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:30.708+0000 7fc151d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1540fcf60 0x7fc15410de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39868/0 (socket says 172.21.15.32:39868) 2024-05-12T04:05:30.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:30.708+0000 7fc151d9b700 1 -- 172.21.15.32:0/1982456560 learned_addr learned my addr 172.21.15.32:0/1982456560 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:05:30.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:30.708+0000 7fc15259c700 1 --2- 172.21.15.32:0/1982456560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1540fbcb0 0x7fc15410d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:05:30.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:30.708+0000 7fc151d9b700 1 -- 172.21.15.32:0/1982456560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1540fbcb0 msgr2=0x7fc15410d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:05:30.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:30.708+0000 7fc151d9b700 1 --2- 172.21.15.32:0/1982456560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1540fbcb0 0x7fc15410d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:05:30.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:30.708+0000 7fc151d9b700 1 -- 172.21.15.32:0/1982456560 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc148005040 con 0x7fc1540fcf60 2024-05-12T04:05:30.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:30.708+0000 7fc151d9b700 1 --2- 172.21.15.32:0/1982456560 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1540fcf60 0x7fc15410de10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc1480137b0 tx=0x7fc148009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:05:30.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:30.709+0000 7fc142ffd700 1 -- 172.21.15.32:0/1982456560 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc14800e070 con 0x7fc1540fcf60 2024-05-12T04:05:30.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:30.709+0000 7fc142ffd700 1 -- 172.21.15.32:0/1982456560 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc14801c400 con 0x7fc1540fcf60 2024-05-12T04:05:30.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:30.709+0000 7fc158f11700 1 -- 172.21.15.32:0/1982456560 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc15410eee0 con 0x7fc1540fcf60 2024-05-12T04:05:30.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:30.709+0000 7fc142ffd700 1 -- 172.21.15.32:0/1982456560 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc14800b650 con 0x7fc1540fcf60 2024-05-12T04:05:30.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:30.709+0000 7fc158f11700 1 -- 172.21.15.32:0/1982456560 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc15410f3c0 con 0x7fc1540fcf60 2024-05-12T04:05:30.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:30.710+0000 7fc158f11700 1 -- 172.21.15.32:0/1982456560 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc154101af0 con 0x7fc1540fcf60 2024-05-12T04:05:30.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:30.710+0000 7fc142ffd700 1 -- 172.21.15.32:0/1982456560 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fc14801c9e0 con 0x7fc1540fcf60 2024-05-12T04:05:30.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:30.711+0000 7fc142ffd700 1 --2- 172.21.15.32:0/1982456560 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fc13c06cc20 0x7fc13c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:05:30.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:30.711+0000 7fc15259c700 1 --2- 172.21.15.32:0/1982456560 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fc13c06cc20 0x7fc13c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:05:30.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:30.711+0000 7fc142ffd700 1 -- 172.21.15.32:0/1982456560 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fc14808d530 con 0x7fc1540fcf60 2024-05-12T04:05:30.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:30.711+0000 7fc15259c700 1 --2- 172.21.15.32:0/1982456560 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fc13c06cc20 0x7fc13c06f0d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc144009b80 tx=0x7fc14400b040 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:05:30.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:30.715+0000 7fc142ffd700 1 -- 172.21.15.32:0/1982456560 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fc14805b0f0 con 0x7fc1540fcf60 2024-05-12T04:05:30.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:30.945+0000 7fc158f11700 1 -- 172.21.15.32:0/1982456560 --> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc154104840 con 0x7fc13c06cc20 2024-05-12T04:05:30.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:30.947+0000 7fc142ffd700 1 -- 172.21.15.32:0/1982456560 <== mgr.24593 v2:172.21.15.32:6800/2957111591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fc154104840 con 0x7fc13c06cc20 2024-05-12T04:05:30.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:30.949+0000 7fc158f11700 1 -- 172.21.15.32:0/1982456560 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fc13c06cc20 msgr2=0x7fc13c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:05:30.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:30.949+0000 7fc158f11700 1 --2- 172.21.15.32:0/1982456560 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fc13c06cc20 0x7fc13c06f0d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc144009b80 tx=0x7fc14400b040 comp rx=0 tx=0).stop 2024-05-12T04:05:30.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:30.949+0000 7fc158f11700 1 -- 172.21.15.32:0/1982456560 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1540fcf60 msgr2=0x7fc15410de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:05:30.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:30.949+0000 7fc158f11700 1 --2- 172.21.15.32:0/1982456560 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1540fcf60 0x7fc15410de10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc1480137b0 tx=0x7fc148009e30 comp rx=0 tx=0).stop 2024-05-12T04:05:30.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:30.949+0000 7fc158f11700 1 -- 172.21.15.32:0/1982456560 shutdown_connections 2024-05-12T04:05:30.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:30.949+0000 7fc158f11700 1 --2- 172.21.15.32:0/1982456560 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fc13c06cc20 0x7fc13c06f0d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:05:30.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:30.949+0000 7fc158f11700 1 --2- 172.21.15.32:0/1982456560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1540fbcb0 0x7fc15410d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:05:30.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:30.949+0000 7fc158f11700 1 --2- 172.21.15.32:0/1982456560 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1540fcf60 0x7fc15410de10 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:05:30.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:30.949+0000 7fc158f11700 1 -- 172.21.15.32:0/1982456560 >> 172.21.15.32:0/1982456560 conn(0x7fc1540f7430 msgr2=0x7fc154065ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:05:30.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:30.949+0000 7fc158f11700 1 -- 172.21.15.32:0/1982456560 shutdown_connections 2024-05-12T04:05:30.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:30.949+0000 7fc158f11700 1 -- 172.21.15.32:0/1982456560 wait complete. 2024-05-12T04:05:30.960 INFO:teuthology.orchestra.run.smithi032.stdout:true 2024-05-12T04:05:31.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.054+0000 7fa0cdeac700 1 -- 172.21.15.32:0/3425815983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0c80fcf60 msgr2=0x7fa0c80fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:05:31.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.054+0000 7fa0cdeac700 1 --2- 172.21.15.32:0/3425815983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0c80fcf60 0x7fa0c80fd3d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc004940 tx=0x7fa0bc009e30 comp rx=0 tx=0).stop 2024-05-12T04:05:31.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.054+0000 7fa0cdeac700 1 -- 172.21.15.32:0/3425815983 shutdown_connections 2024-05-12T04:05:31.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.054+0000 7fa0cdeac700 1 --2- 172.21.15.32:0/3425815983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0c80fcf60 0x7fa0c80fd3d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:05:31.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.054+0000 7fa0cdeac700 1 --2- 172.21.15.32:0/3425815983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0c80fbcb0 0x7fa0c80fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:05:31.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.054+0000 7fa0cdeac700 1 -- 172.21.15.32:0/3425815983 >> 172.21.15.32:0/3425815983 conn(0x7fa0c80f7430 msgr2=0x7fa0c80f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:05:31.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.054+0000 7fa0cdeac700 1 -- 172.21.15.32:0/3425815983 shutdown_connections 2024-05-12T04:05:31.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.054+0000 7fa0cdeac700 1 -- 172.21.15.32:0/3425815983 wait complete. 2024-05-12T04:05:31.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.055+0000 7fa0cdeac700 1 Processor -- start 2024-05-12T04:05:31.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.055+0000 7fa0cdeac700 1 -- start start 2024-05-12T04:05:31.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.055+0000 7fa0cdeac700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0c80fbcb0 0x7fa0c8110600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:05:31.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.055+0000 7fa0cdeac700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0c80fcf60 0x7fa0c8112b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:05:31.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.055+0000 7fa0cdeac700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c8066350 con 0x7fa0c80fcf60 2024-05-12T04:05:31.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.055+0000 7fa0cdeac700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c8066490 con 0x7fa0c80fbcb0 2024-05-12T04:05:31.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.055+0000 7fa0c6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0c80fcf60 0x7fa0c8112b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:05:31.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.055+0000 7fa0c6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0c80fcf60 0x7fa0c8112b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39886/0 (socket says 172.21.15.32:39886) 2024-05-12T04:05:31.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.055+0000 7fa0c6ffd700 1 -- 172.21.15.32:0/2896512006 learned_addr learned my addr 172.21.15.32:0/2896512006 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:05:31.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.055+0000 7fa0c77fe700 1 --2- 172.21.15.32:0/2896512006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0c80fbcb0 0x7fa0c8110600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:05:31.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.056+0000 7fa0c6ffd700 1 -- 172.21.15.32:0/2896512006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0c80fbcb0 msgr2=0x7fa0c8110600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:05:31.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.056+0000 7fa0c6ffd700 1 --2- 172.21.15.32:0/2896512006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0c80fbcb0 0x7fa0c8110600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:05:31.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.056+0000 7fa0c6ffd700 1 -- 172.21.15.32:0/2896512006 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0bc005040 con 0x7fa0c80fcf60 2024-05-12T04:05:31.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.056+0000 7fa0c6ffd700 1 --2- 172.21.15.32:0/2896512006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0c80fcf60 0x7fa0c8112b50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc00b3b0 tx=0x7fa0bc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:05:31.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.056+0000 7fa0affff700 1 -- 172.21.15.32:0/2896512006 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa0bc00e040 con 0x7fa0c80fcf60 2024-05-12T04:05:31.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.056+0000 7fa0cdeac700 1 -- 172.21.15.32:0/2896512006 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0c81130f0 con 0x7fa0c80fcf60 2024-05-12T04:05:31.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.056+0000 7fa0affff700 1 -- 172.21.15.32:0/2896512006 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa0bc00bc50 con 0x7fa0c80fcf60 2024-05-12T04:05:31.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.056+0000 7fa0affff700 1 -- 172.21.15.32:0/2896512006 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa0bc01c630 con 0x7fa0c80fcf60 2024-05-12T04:05:31.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.056+0000 7fa0cdeac700 1 -- 172.21.15.32:0/2896512006 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0c8113680 con 0x7fa0c80fcf60 2024-05-12T04:05:31.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.057+0000 7fa0cdeac700 1 -- 172.21.15.32:0/2896512006 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0c805f5c0 con 0x7fa0c80fcf60 2024-05-12T04:05:31.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.058+0000 7fa0affff700 1 -- 172.21.15.32:0/2896512006 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fa0bc013420 con 0x7fa0c80fcf60 2024-05-12T04:05:31.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.058+0000 7fa0affff700 1 --2- 172.21.15.32:0/2896512006 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fa0b006cc20 0x7fa0b006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:05:31.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.058+0000 7fa0affff700 1 -- 172.21.15.32:0/2896512006 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fa0bc08d2e0 con 0x7fa0c80fcf60 2024-05-12T04:05:31.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.058+0000 7fa0c77fe700 1 --2- 172.21.15.32:0/2896512006 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fa0b006cc20 0x7fa0b006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:05:31.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.059+0000 7fa0c77fe700 1 --2- 172.21.15.32:0/2896512006 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fa0b006cc20 0x7fa0b006f0d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa0c8065fd0 tx=0x7fa0b800b040 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:05:31.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.062+0000 7fa0affff700 1 -- 172.21.15.32:0/2896512006 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fa0bc057410 con 0x7fa0c80fcf60 2024-05-12T04:05:31.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:30 smithi033 ceph-mon[107305]: pgmap v79: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 260 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 36/276 objects degraded (13.043%) 2024-05-12T04:05:31.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:30 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:05:31.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:30 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:05:31.223 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:30 smithi032 ceph-mon[120392]: pgmap v79: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 260 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 36/276 objects degraded (13.043%) 2024-05-12T04:05:31.223 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:30 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:05:31.223 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:30 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:05:31.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.298+0000 7fa0cdeac700 1 -- 172.21.15.32:0/2896512006 --> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa0c8101410 con 0x7fa0b006cc20 2024-05-12T04:05:31.300 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.300+0000 7fa0affff700 1 -- 172.21.15.32:0/2896512006 <== mgr.24593 v2:172.21.15.32:6800/2957111591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fa0c8101410 con 0x7fa0b006cc20 2024-05-12T04:05:31.302 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.302+0000 7fa0cdeac700 1 -- 172.21.15.32:0/2896512006 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fa0b006cc20 msgr2=0x7fa0b006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:05:31.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.302+0000 7fa0cdeac700 1 --2- 172.21.15.32:0/2896512006 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fa0b006cc20 0x7fa0b006f0d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa0c8065fd0 tx=0x7fa0b800b040 comp rx=0 tx=0).stop 2024-05-12T04:05:31.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.302+0000 7fa0cdeac700 1 -- 172.21.15.32:0/2896512006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0c80fcf60 msgr2=0x7fa0c8112b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:05:31.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.302+0000 7fa0cdeac700 1 --2- 172.21.15.32:0/2896512006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0c80fcf60 0x7fa0c8112b50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc00b3b0 tx=0x7fa0bc009e30 comp rx=0 tx=0).stop 2024-05-12T04:05:31.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.302+0000 7fa0cdeac700 1 -- 172.21.15.32:0/2896512006 shutdown_connections 2024-05-12T04:05:31.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.302+0000 7fa0cdeac700 1 --2- 172.21.15.32:0/2896512006 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fa0b006cc20 0x7fa0b006f0d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:05:31.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.302+0000 7fa0cdeac700 1 --2- 172.21.15.32:0/2896512006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0c80fbcb0 0x7fa0c8110600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:05:31.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.302+0000 7fa0cdeac700 1 --2- 172.21.15.32:0/2896512006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa0c80fcf60 0x7fa0c8112b50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:05:31.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.302+0000 7fa0cdeac700 1 -- 172.21.15.32:0/2896512006 >> 172.21.15.32:0/2896512006 conn(0x7fa0c80f7430 msgr2=0x7fa0c80f8b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:05:31.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.302+0000 7fa0cdeac700 1 -- 172.21.15.32:0/2896512006 shutdown_connections 2024-05-12T04:05:31.303 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.302+0000 7fa0cdeac700 1 -- 172.21.15.32:0/2896512006 wait complete. 2024-05-12T04:05:31.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.408+0000 7fc24c822700 1 -- 172.21.15.32:0/26021197 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc248066260 msgr2=0x7fc248101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:05:31.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.408+0000 7fc24c822700 1 --2- 172.21.15.32:0/26021197 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc248066260 0x7fc248101e00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc238004940 tx=0x7fc238009e30 comp rx=0 tx=0).stop 2024-05-12T04:05:31.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.408+0000 7fc24c822700 1 -- 172.21.15.32:0/26021197 shutdown_connections 2024-05-12T04:05:31.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.408+0000 7fc24c822700 1 --2- 172.21.15.32:0/26021197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc248102340 0x7fc248104720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:05:31.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.408+0000 7fc24c822700 1 --2- 172.21.15.32:0/26021197 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc248066260 0x7fc248101e00 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:05:31.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.408+0000 7fc24c822700 1 -- 172.21.15.32:0/26021197 >> 172.21.15.32:0/26021197 conn(0x7fc2480f7430 msgr2=0x7fc2480f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:05:31.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.409+0000 7fc24c822700 1 -- 172.21.15.32:0/26021197 shutdown_connections 2024-05-12T04:05:31.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.409+0000 7fc24c822700 1 -- 172.21.15.32:0/26021197 wait complete. 2024-05-12T04:05:31.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.409+0000 7fc24c822700 1 Processor -- start 2024-05-12T04:05:31.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.409+0000 7fc24c822700 1 -- start start 2024-05-12T04:05:31.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.409+0000 7fc24c822700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc248066260 0x7fc24810fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:05:31.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.409+0000 7fc24c822700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc248102340 0x7fc248110040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:05:31.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.409+0000 7fc24c822700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc248110640 con 0x7fc248066260 2024-05-12T04:05:31.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.409+0000 7fc24c822700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc248110e90 con 0x7fc248102340 2024-05-12T04:05:31.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.409+0000 7fc24659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc248066260 0x7fc24810fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:05:31.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.409+0000 7fc24659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc248066260 0x7fc24810fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39910/0 (socket says 172.21.15.32:39910) 2024-05-12T04:05:31.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.410+0000 7fc24659c700 1 -- 172.21.15.32:0/2405646071 learned_addr learned my addr 172.21.15.32:0/2405646071 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:05:31.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.410+0000 7fc245d9b700 1 --2- 172.21.15.32:0/2405646071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc248102340 0x7fc248110040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:05:31.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.410+0000 7fc24659c700 1 -- 172.21.15.32:0/2405646071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc248102340 msgr2=0x7fc248110040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:05:31.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.410+0000 7fc24659c700 1 --2- 172.21.15.32:0/2405646071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc248102340 0x7fc248110040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:05:31.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.410+0000 7fc24659c700 1 -- 172.21.15.32:0/2405646071 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc238005040 con 0x7fc248066260 2024-05-12T04:05:31.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.410+0000 7fc245d9b700 1 --2- 172.21.15.32:0/2405646071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc248102340 0x7fc248110040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-12T04:05:31.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.410+0000 7fc24659c700 1 --2- 172.21.15.32:0/2405646071 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc248066260 0x7fc24810fb00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc238000c00 tx=0x7fc238009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:05:31.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.410+0000 7fc236ffd700 1 -- 172.21.15.32:0/2405646071 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc23800e070 con 0x7fc248066260 2024-05-12T04:05:31.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.410+0000 7fc236ffd700 1 -- 172.21.15.32:0/2405646071 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc23801c400 con 0x7fc248066260 2024-05-12T04:05:31.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.410+0000 7fc24c822700 1 -- 172.21.15.32:0/2405646071 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc248111110 con 0x7fc248066260 2024-05-12T04:05:31.412 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.410+0000 7fc236ffd700 1 -- 172.21.15.32:0/2405646071 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc23800b650 con 0x7fc248066260 2024-05-12T04:05:31.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.410+0000 7fc24c822700 1 -- 172.21.15.32:0/2405646071 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2481115f0 con 0x7fc248066260 2024-05-12T04:05:31.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.411+0000 7fc24c822700 1 -- 172.21.15.32:0/2405646071 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc24805f5c0 con 0x7fc248066260 2024-05-12T04:05:31.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.412+0000 7fc236ffd700 1 -- 172.21.15.32:0/2405646071 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fc23801c570 con 0x7fc248066260 2024-05-12T04:05:31.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.412+0000 7fc236ffd700 1 --2- 172.21.15.32:0/2405646071 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fc23006cc30 0x7fc23006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:05:31.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.412+0000 7fc236ffd700 1 -- 172.21.15.32:0/2405646071 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fc23808d6c0 con 0x7fc248066260 2024-05-12T04:05:31.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.412+0000 7fc245d9b700 1 --2- 172.21.15.32:0/2405646071 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fc23006cc30 0x7fc23006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:05:31.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.413+0000 7fc245d9b700 1 --2- 172.21.15.32:0/2405646071 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fc23006cc30 0x7fc23006f0e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc23c004600 tx=0x7fc23c00b040 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:05:31.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.417+0000 7fc236ffd700 1 -- 172.21.15.32:0/2405646071 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fc23805b1e0 con 0x7fc248066260 2024-05-12T04:05:31.627 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.627+0000 7fc24c822700 1 -- 172.21.15.32:0/2405646071 --> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc248111920 con 0x7fc23006cc30 2024-05-12T04:05:31.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.637+0000 7fc236ffd700 1 -- 172.21.15.32:0/2405646071 <== mgr.24593 v2:172.21.15.32:6800/2957111591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fc248111920 con 0x7fc23006cc30 2024-05-12T04:05:31.637 INFO:teuthology.orchestra.run.smithi032.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-12T04:05:31.637 INFO:teuthology.orchestra.run.smithi032.stdout:alertmanager.smithi032 smithi032 *:9093,9094 running (4m) 4s ago 13m 25.5M - 0.25.0 c8568f914cd2 a37fbab7af4c 2024-05-12T04:05:31.637 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi032 smithi032 running (74s) 4s ago 13m 6660k - 18.2.2-1272-g58249552 ddcb0a9d6fde 8c53f1fc7b06 2024-05-12T04:05:31.637 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi033 smithi033 running (69s) 65s ago 12m 6651k - 18.2.2-1272-g58249552 ddcb0a9d6fde cf4fe9ca070b 2024-05-12T04:05:31.637 INFO:teuthology.orchestra.run.smithi032.stdout:grafana.smithi032 smithi032 *:3000 running (4m) 4s ago 13m 83.5M - 9.4.7 954c08fa6188 83055e58d23c 2024-05-12T04:05:31.637 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.ialtqb smithi032 running (7m) 4s ago 7m 13.7M - 17.2.7-939-gc190bdee 284803f7d9cf 0683abf00e3e 2024-05-12T04:05:31.637 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.qpsawh smithi032 running (7m) 4s ago 7m 155M - 17.2.7-939-gc190bdee 284803f7d9cf 95a7db5c4fcd 2024-05-12T04:05:31.638 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi033.hglwml smithi033 running (7m) 65s ago 7m 13.2M - 17.2.7-939-gc190bdee 284803f7d9cf c97902b05025 2024-05-12T04:05:31.638 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi033.jgbjhk smithi033 running (7m) 65s ago 7m 102M - 17.2.7-939-gc190bdee 284803f7d9cf af97052cb97a 2024-05-12T04:05:31.638 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi032.ukatkq smithi032 *:8443,9283,8765 running (3m) 4s ago 15m 500M - 18.2.2-1272-g58249552 ddcb0a9d6fde 0d88ab9a5292 2024-05-12T04:05:31.638 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi033.pcvxwy smithi033 *:8443,9283,8765 running (2m) 65s ago 12m 448M - 18.2.2-1272-g58249552 ddcb0a9d6fde d42dcb6bd307 2024-05-12T04:05:31.638 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi032 smithi032 running (2m) 4s ago 15m 44.5M 2048M 18.2.2-1272-g58249552 ddcb0a9d6fde 81f94171c5dc 2024-05-12T04:05:31.638 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi033 smithi033 running (119s) 65s ago 12m 32.0M 2048M 18.2.2-1272-g58249552 ddcb0a9d6fde d648ba108f3e 2024-05-12T04:05:31.638 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi032 smithi032 *:9100 running (4m) 4s ago 13m 16.7M - 1.5.0 0da6a335fe13 cb75a7c0c829 2024-05-12T04:05:31.638 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi033 smithi033 *:9100 running (3m) 65s ago 12m 18.3M - 1.5.0 0da6a335fe13 7888c36c659f 2024-05-12T04:05:31.638 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0 smithi032 running (46s) 4s ago 11m 80.3M 1570M 18.2.2-1272-g58249552 ddcb0a9d6fde 07f66c4deb3b 2024-05-12T04:05:31.638 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1 smithi032 running (10s) 4s ago 11m 11.5M 1570M 18.2.2-1272-g58249552 ddcb0a9d6fde 67a87807d654 2024-05-12T04:05:31.638 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2 smithi032 running (10m) 4s ago 10m 183M 1570M 17.2.7-939-gc190bdee 284803f7d9cf 7aef7affb8ba 2024-05-12T04:05:31.639 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3 smithi033 running (10m) 65s ago 10m 240M 2594M 17.2.7-939-gc190bdee 284803f7d9cf 3a92b5e3348e 2024-05-12T04:05:31.639 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4 smithi033 running (9m) 65s ago 9m 213M 2594M 17.2.7-939-gc190bdee 284803f7d9cf 0896268b4591 2024-05-12T04:05:31.639 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5 smithi033 running (9m) 65s ago 9m 196M 2594M 17.2.7-939-gc190bdee 284803f7d9cf 57e65a64c821 2024-05-12T04:05:31.639 INFO:teuthology.orchestra.run.smithi032.stdout:prometheus.smithi032 smithi032 *:9095 running (3m) 4s ago 12m 51.9M - 2.43.0 a07b618ecd1d 46eafb25e176 2024-05-12T04:05:31.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.639+0000 7fc24c822700 1 -- 172.21.15.32:0/2405646071 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fc23006cc30 msgr2=0x7fc23006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:05:31.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.639+0000 7fc24c822700 1 --2- 172.21.15.32:0/2405646071 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fc23006cc30 0x7fc23006f0e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc23c004600 tx=0x7fc23c00b040 comp rx=0 tx=0).stop 2024-05-12T04:05:31.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.639+0000 7fc24c822700 1 -- 172.21.15.32:0/2405646071 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc248066260 msgr2=0x7fc24810fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:05:31.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.639+0000 7fc24c822700 1 --2- 172.21.15.32:0/2405646071 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc248066260 0x7fc24810fb00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc238000c00 tx=0x7fc238009e30 comp rx=0 tx=0).stop 2024-05-12T04:05:31.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.639+0000 7fc24c822700 1 -- 172.21.15.32:0/2405646071 shutdown_connections 2024-05-12T04:05:31.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.639+0000 7fc24c822700 1 --2- 172.21.15.32:0/2405646071 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fc23006cc30 0x7fc23006f0e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:05:31.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.639+0000 7fc24c822700 1 --2- 172.21.15.32:0/2405646071 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc248066260 0x7fc24810fb00 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:05:31.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.639+0000 7fc24c822700 1 --2- 172.21.15.32:0/2405646071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc248102340 0x7fc248110040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:05:31.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.639+0000 7fc24c822700 1 -- 172.21.15.32:0/2405646071 >> 172.21.15.32:0/2405646071 conn(0x7fc2480f7430 msgr2=0x7fc2480f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:05:31.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.639+0000 7fc24c822700 1 -- 172.21.15.32:0/2405646071 shutdown_connections 2024-05-12T04:05:31.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.639+0000 7fc24c822700 1 -- 172.21.15.32:0/2405646071 wait complete. 2024-05-12T04:05:31.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.742+0000 7feb668cd700 1 -- 172.21.15.32:0/4244827290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb600fbcb0 msgr2=0x7feb600fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:05:31.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.742+0000 7feb668cd700 1 --2- 172.21.15.32:0/4244827290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb600fbcb0 0x7feb600fc0c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7feb48004940 tx=0x7feb48009e30 comp rx=0 tx=0).stop 2024-05-12T04:05:31.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.742+0000 7feb668cd700 1 -- 172.21.15.32:0/4244827290 shutdown_connections 2024-05-12T04:05:31.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.742+0000 7feb668cd700 1 --2- 172.21.15.32:0/4244827290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb600fcf60 0x7feb600fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:05:31.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.742+0000 7feb668cd700 1 --2- 172.21.15.32:0/4244827290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb600fbcb0 0x7feb600fc0c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:05:31.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.742+0000 7feb668cd700 1 -- 172.21.15.32:0/4244827290 >> 172.21.15.32:0/4244827290 conn(0x7feb600f7430 msgr2=0x7feb600f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:05:31.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.742+0000 7feb668cd700 1 -- 172.21.15.32:0/4244827290 shutdown_connections 2024-05-12T04:05:31.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.742+0000 7feb668cd700 1 -- 172.21.15.32:0/4244827290 wait complete. 2024-05-12T04:05:31.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.742+0000 7feb668cd700 1 Processor -- start 2024-05-12T04:05:31.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.743+0000 7feb668cd700 1 -- start start 2024-05-12T04:05:31.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.743+0000 7feb668cd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb600fbcb0 0x7feb6010c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:05:31.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.743+0000 7feb668cd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb600fcf60 0x7feb6010cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:05:31.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.743+0000 7feb668cd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb6010d290 con 0x7feb600fbcb0 2024-05-12T04:05:31.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.743+0000 7feb668cd700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb6010dae0 con 0x7feb600fcf60 2024-05-12T04:05:31.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.743+0000 7feb5ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb600fbcb0 0x7feb6010c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:05:31.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.743+0000 7feb5ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb600fbcb0 0x7feb6010c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39936/0 (socket says 172.21.15.32:39936) 2024-05-12T04:05:31.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.743+0000 7feb5ffff700 1 -- 172.21.15.32:0/974991809 learned_addr learned my addr 172.21.15.32:0/974991809 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:05:31.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.743+0000 7feb5f7fe700 1 --2- 172.21.15.32:0/974991809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb600fcf60 0x7feb6010cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:05:31.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.743+0000 7feb5ffff700 1 -- 172.21.15.32:0/974991809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb600fcf60 msgr2=0x7feb6010cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:05:31.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.743+0000 7feb5ffff700 1 --2- 172.21.15.32:0/974991809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb600fcf60 0x7feb6010cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:05:31.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.743+0000 7feb5ffff700 1 -- 172.21.15.32:0/974991809 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb48005040 con 0x7feb600fbcb0 2024-05-12T04:05:31.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.743+0000 7feb5f7fe700 1 --2- 172.21.15.32:0/974991809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb600fcf60 0x7feb6010cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-12T04:05:31.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.743+0000 7feb5ffff700 1 --2- 172.21.15.32:0/974991809 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb600fbcb0 0x7feb6010c750 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7feb480137b0 tx=0x7feb48009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:05:31.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.744+0000 7feb5cff9700 1 -- 172.21.15.32:0/974991809 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb4800e070 con 0x7feb600fbcb0 2024-05-12T04:05:31.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.744+0000 7feb5cff9700 1 -- 172.21.15.32:0/974991809 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7feb480093f0 con 0x7feb600fbcb0 2024-05-12T04:05:31.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.744+0000 7feb668cd700 1 -- 172.21.15.32:0/974991809 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb6010dd60 con 0x7feb600fbcb0 2024-05-12T04:05:31.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.744+0000 7feb5cff9700 1 -- 172.21.15.32:0/974991809 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb4800b410 con 0x7feb600fbcb0 2024-05-12T04:05:31.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.744+0000 7feb668cd700 1 -- 172.21.15.32:0/974991809 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb6010e240 con 0x7feb600fbcb0 2024-05-12T04:05:31.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.745+0000 7feb668cd700 1 -- 172.21.15.32:0/974991809 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb6005f5c0 con 0x7feb600fbcb0 2024-05-12T04:05:31.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.745+0000 7feb5cff9700 1 -- 172.21.15.32:0/974991809 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7feb48013c40 con 0x7feb600fbcb0 2024-05-12T04:05:31.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.745+0000 7feb5cff9700 1 --2- 172.21.15.32:0/974991809 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7feb4c06cc20 0x7feb4c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:05:31.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.746+0000 7feb5cff9700 1 -- 172.21.15.32:0/974991809 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6410+0+0 (secure 0 0 0) 0x7feb4808d700 con 0x7feb600fbcb0 2024-05-12T04:05:31.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.746+0000 7feb5f7fe700 1 --2- 172.21.15.32:0/974991809 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7feb4c06cc20 0x7feb4c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:05:31.747 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.746+0000 7feb5f7fe700 1 --2- 172.21.15.32:0/974991809 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7feb4c06cc20 0x7feb4c06f0d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7feb50004600 tx=0x7feb5000b040 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:05:31.750 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:31.750+0000 7feb5cff9700 1 -- 172.21.15.32:0/974991809 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7feb4805b220 con 0x7feb600fbcb0 2024-05-12T04:05:32.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.038+0000 7feb668cd700 1 -- 172.21.15.32:0/974991809 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7feb60104660 con 0x7feb600fbcb0 2024-05-12T04:05:32.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.038+0000 7feb5cff9700 1 -- 172.21.15.32:0/974991809 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+847 (secure 0 0 0) 0x7feb48004490 con 0x7feb600fbcb0 2024-05-12T04:05:32.039 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-05-12T04:05:32.039 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": { 2024-05-12T04:05:32.039 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 2 2024-05-12T04:05:32.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:05:32.040 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": { 2024-05-12T04:05:32.040 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 2 2024-05-12T04:05:32.040 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:05:32.040 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": { 2024-05-12T04:05:32.040 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 4, 2024-05-12T04:05:32.040 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 2 2024-05-12T04:05:32.040 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:05:32.040 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": { 2024-05-12T04:05:32.040 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 4 2024-05-12T04:05:32.040 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:05:32.040 INFO:teuthology.orchestra.run.smithi032.stdout: "overall": { 2024-05-12T04:05:32.040 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 8, 2024-05-12T04:05:32.040 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 6 2024-05-12T04:05:32.041 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-05-12T04:05:32.041 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-05-12T04:05:32.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.040+0000 7feb668cd700 1 -- 172.21.15.32:0/974991809 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7feb4c06cc20 msgr2=0x7feb4c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:05:32.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.040+0000 7feb668cd700 1 --2- 172.21.15.32:0/974991809 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7feb4c06cc20 0x7feb4c06f0d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7feb50004600 tx=0x7feb5000b040 comp rx=0 tx=0).stop 2024-05-12T04:05:32.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.040+0000 7feb668cd700 1 -- 172.21.15.32:0/974991809 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb600fbcb0 msgr2=0x7feb6010c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:05:32.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.040+0000 7feb668cd700 1 --2- 172.21.15.32:0/974991809 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb600fbcb0 0x7feb6010c750 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7feb480137b0 tx=0x7feb48009e30 comp rx=0 tx=0).stop 2024-05-12T04:05:32.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.041+0000 7feb668cd700 1 -- 172.21.15.32:0/974991809 shutdown_connections 2024-05-12T04:05:32.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.041+0000 7feb668cd700 1 --2- 172.21.15.32:0/974991809 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7feb4c06cc20 0x7feb4c06f0d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:05:32.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.041+0000 7feb668cd700 1 --2- 172.21.15.32:0/974991809 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb600fbcb0 0x7feb6010c750 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:05:32.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.041+0000 7feb668cd700 1 --2- 172.21.15.32:0/974991809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb600fcf60 0x7feb6010cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:05:32.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.041+0000 7feb668cd700 1 -- 172.21.15.32:0/974991809 >> 172.21.15.32:0/974991809 conn(0x7feb600f7430 msgr2=0x7feb600f8b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:05:32.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.041+0000 7feb668cd700 1 -- 172.21.15.32:0/974991809 shutdown_connections 2024-05-12T04:05:32.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.041+0000 7feb668cd700 1 -- 172.21.15.32:0/974991809 wait complete. 2024-05-12T04:05:32.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.144+0000 7f4d6e5bb700 1 -- 172.21.15.32:0/2002011716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d680fcf60 msgr2=0x7f4d680fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:05:32.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.144+0000 7f4d6e5bb700 1 --2- 172.21.15.32:0/2002011716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d680fcf60 0x7f4d680fd3d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c004940 tx=0x7f4d5c009e30 comp rx=0 tx=0).stop 2024-05-12T04:05:32.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.144+0000 7f4d6e5bb700 1 -- 172.21.15.32:0/2002011716 shutdown_connections 2024-05-12T04:05:32.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.144+0000 7f4d6e5bb700 1 --2- 172.21.15.32:0/2002011716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d680fcf60 0x7f4d680fd3d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:05:32.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.144+0000 7f4d6e5bb700 1 --2- 172.21.15.32:0/2002011716 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d680fbcb0 0x7f4d680fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:05:32.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.144+0000 7f4d6e5bb700 1 -- 172.21.15.32:0/2002011716 >> 172.21.15.32:0/2002011716 conn(0x7f4d680f7430 msgr2=0x7f4d680f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:05:32.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.144+0000 7f4d6e5bb700 1 -- 172.21.15.32:0/2002011716 shutdown_connections 2024-05-12T04:05:32.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.144+0000 7f4d6e5bb700 1 -- 172.21.15.32:0/2002011716 wait complete. 2024-05-12T04:05:32.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.145+0000 7f4d6e5bb700 1 Processor -- start 2024-05-12T04:05:32.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.145+0000 7f4d6e5bb700 1 -- start start 2024-05-12T04:05:32.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.145+0000 7f4d6e5bb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d680fbcb0 0x7f4d680ffd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:05:32.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.145+0000 7f4d6e5bb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d680fcf60 0x7f4d680fe2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:05:32.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.145+0000 7f4d6e5bb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d680fe830 con 0x7f4d680fbcb0 2024-05-12T04:05:32.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.145+0000 7f4d6e5bb700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d680fe970 con 0x7f4d680fcf60 2024-05-12T04:05:32.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.145+0000 7f4d67fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d680fbcb0 0x7f4d680ffd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:05:32.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.145+0000 7f4d67fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d680fbcb0 0x7f4d680ffd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38558/0 (socket says 172.21.15.32:38558) 2024-05-12T04:05:32.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.145+0000 7f4d67fff700 1 -- 172.21.15.32:0/322891644 learned_addr learned my addr 172.21.15.32:0/322891644 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:05:32.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.146+0000 7f4d677fe700 1 --2- 172.21.15.32:0/322891644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d680fcf60 0x7f4d680fe2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:05:32.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.146+0000 7f4d67fff700 1 -- 172.21.15.32:0/322891644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d680fcf60 msgr2=0x7f4d680fe2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:05:32.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.146+0000 7f4d67fff700 1 --2- 172.21.15.32:0/322891644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d680fcf60 0x7f4d680fe2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:05:32.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.146+0000 7f4d67fff700 1 -- 172.21.15.32:0/322891644 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d5c005040 con 0x7f4d680fbcb0 2024-05-12T04:05:32.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.146+0000 7f4d67fff700 1 --2- 172.21.15.32:0/322891644 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d680fbcb0 0x7f4d680ffd80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4d58002a50 tx=0x7f4d580046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:05:32.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.146+0000 7f4d64ff9700 1 -- 172.21.15.32:0/322891644 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d58007cc0 con 0x7f4d680fbcb0 2024-05-12T04:05:32.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.146+0000 7f4d6e5bb700 1 -- 172.21.15.32:0/322891644 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d680fec50 con 0x7f4d680fbcb0 2024-05-12T04:05:32.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.146+0000 7f4d64ff9700 1 -- 172.21.15.32:0/322891644 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4d5800f040 con 0x7f4d680fbcb0 2024-05-12T04:05:32.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.146+0000 7f4d64ff9700 1 -- 172.21.15.32:0/322891644 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d58009540 con 0x7f4d680fbcb0 2024-05-12T04:05:32.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.146+0000 7f4d6e5bb700 1 -- 172.21.15.32:0/322891644 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d681159d0 con 0x7f4d680fbcb0 2024-05-12T04:05:32.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.147+0000 7f4d6e5bb700 1 -- 172.21.15.32:0/322891644 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d68101af0 con 0x7f4d680fbcb0 2024-05-12T04:05:32.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.148+0000 7f4d64ff9700 1 -- 172.21.15.32:0/322891644 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f4d580096a0 con 0x7f4d680fbcb0 2024-05-12T04:05:32.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.148+0000 7f4d64ff9700 1 --2- 172.21.15.32:0/322891644 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f4d5006cc20 0x7f4d5006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:05:32.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.148+0000 7f4d64ff9700 1 -- 172.21.15.32:0/322891644 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f4d5808be20 con 0x7f4d680fbcb0 2024-05-12T04:05:32.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.148+0000 7f4d677fe700 1 --2- 172.21.15.32:0/322891644 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f4d5006cc20 0x7f4d5006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:05:32.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.149+0000 7f4d677fe700 1 --2- 172.21.15.32:0/322891644 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f4d5006cc20 0x7f4d5006f0d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4d68100360 tx=0x7f4d5c009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:05:32.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.152+0000 7f4d64ff9700 1 -- 172.21.15.32:0/322891644 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f4d58055e30 con 0x7f4d680fbcb0 2024-05-12T04:05:32.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.400+0000 7f4d6e5bb700 1 -- 172.21.15.32:0/322891644 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4d6805f5c0 con 0x7f4d680fbcb0 2024-05-12T04:05:32.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.401+0000 7f4d64ff9700 1 -- 172.21.15.32:0/322891644 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1866 (secure 0 0 0) 0x7f4d58059450 con 0x7f4d680fbcb0 2024-05-12T04:05:32.401 INFO:teuthology.orchestra.run.smithi032.stdout:e18 2024-05-12T04:05:32.401 INFO:teuthology.orchestra.run.smithi032.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-12T04:05:32.401 INFO:teuthology.orchestra.run.smithi032.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-12T04:05:32.401 INFO:teuthology.orchestra.run.smithi032.stdout:legacy client fscid: 1 2024-05-12T04:05:32.401 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:05:32.401 INFO:teuthology.orchestra.run.smithi032.stdout:Filesystem 'cephfs' (1) 2024-05-12T04:05:32.402 INFO:teuthology.orchestra.run.smithi032.stdout:fs_name cephfs 2024-05-12T04:05:32.402 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 18 2024-05-12T04:05:32.402 INFO:teuthology.orchestra.run.smithi032.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-12T04:05:32.402 INFO:teuthology.orchestra.run.smithi032.stdout:created 2024-05-12T03:57:48.904614+0000 2024-05-12T04:05:32.402 INFO:teuthology.orchestra.run.smithi032.stdout:modified 2024-05-12T03:58:06.821591+0000 2024-05-12T04:05:32.402 INFO:teuthology.orchestra.run.smithi032.stdout:tableserver 0 2024-05-12T04:05:32.402 INFO:teuthology.orchestra.run.smithi032.stdout:root 0 2024-05-12T04:05:32.402 INFO:teuthology.orchestra.run.smithi032.stdout:session_timeout 60 2024-05-12T04:05:32.402 INFO:teuthology.orchestra.run.smithi032.stdout:session_autoclose 300 2024-05-12T04:05:32.402 INFO:teuthology.orchestra.run.smithi032.stdout:max_file_size 1099511627776 2024-05-12T04:05:32.402 INFO:teuthology.orchestra.run.smithi032.stdout:max_xattr_size 65536 2024-05-12T04:05:32.402 INFO:teuthology.orchestra.run.smithi032.stdout:required_client_features {} 2024-05-12T04:05:32.402 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure 0 2024-05-12T04:05:32.403 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure_osd_epoch 46 2024-05-12T04:05:32.403 INFO:teuthology.orchestra.run.smithi032.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-12T04:05:32.403 INFO:teuthology.orchestra.run.smithi032.stdout:max_mds 1 2024-05-12T04:05:32.403 INFO:teuthology.orchestra.run.smithi032.stdout:in 0 2024-05-12T04:05:32.403 INFO:teuthology.orchestra.run.smithi032.stdout:up {0=14474} 2024-05-12T04:05:32.403 INFO:teuthology.orchestra.run.smithi032.stdout:failed 2024-05-12T04:05:32.403 INFO:teuthology.orchestra.run.smithi032.stdout:damaged 2024-05-12T04:05:32.403 INFO:teuthology.orchestra.run.smithi032.stdout:stopped 2024-05-12T04:05:32.403 INFO:teuthology.orchestra.run.smithi032.stdout:data_pools [3] 2024-05-12T04:05:32.403 INFO:teuthology.orchestra.run.smithi032.stdout:metadata_pool 2 2024-05-12T04:05:32.403 INFO:teuthology.orchestra.run.smithi032.stdout:inline_data enabled 2024-05-12T04:05:32.403 INFO:teuthology.orchestra.run.smithi032.stdout:balancer 2024-05-12T04:05:32.403 INFO:teuthology.orchestra.run.smithi032.stdout:bal_rank_mask -1 2024-05-12T04:05:32.403 INFO:teuthology.orchestra.run.smithi032.stdout:standby_count_wanted 1 2024-05-12T04:05:32.403 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.qpsawh{0:14474} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.32:6828/3100155296,v1:172.21.15.32:6829/3100155296] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:05:32.404 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi033.jgbjhk{0:24267} state up:standby-replay seq 3 join_fscid=1 addr [v2:172.21.15.33:6824/55417471,v1:172.21.15.33:6825/55417471] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:05:32.404 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:05:32.404 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:05:32.404 INFO:teuthology.orchestra.run.smithi032.stdout:Standby daemons: 2024-05-12T04:05:32.404 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:05:32.404 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi033.hglwml{-1:24283} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/433858238,v1:172.21.15.33:6827/433858238] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:05:32.404 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.ialtqb{-1:24287} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.32:6826/3703132329,v1:172.21.15.32:6827/3703132329] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:05:32.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.403+0000 7f4d6e5bb700 1 -- 172.21.15.32:0/322891644 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f4d5006cc20 msgr2=0x7f4d5006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:05:32.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.403+0000 7f4d6e5bb700 1 --2- 172.21.15.32:0/322891644 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f4d5006cc20 0x7f4d5006f0d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4d68100360 tx=0x7f4d5c009e30 comp rx=0 tx=0).stop 2024-05-12T04:05:32.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.403+0000 7f4d6e5bb700 1 -- 172.21.15.32:0/322891644 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d680fbcb0 msgr2=0x7f4d680ffd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:05:32.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.403+0000 7f4d6e5bb700 1 --2- 172.21.15.32:0/322891644 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d680fbcb0 0x7f4d680ffd80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4d58002a50 tx=0x7f4d580046b0 comp rx=0 tx=0).stop 2024-05-12T04:05:32.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.403+0000 7f4d6e5bb700 1 -- 172.21.15.32:0/322891644 shutdown_connections 2024-05-12T04:05:32.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.403+0000 7f4d6e5bb700 1 --2- 172.21.15.32:0/322891644 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f4d5006cc20 0x7f4d5006f0d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:05:32.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.403+0000 7f4d6e5bb700 1 --2- 172.21.15.32:0/322891644 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d680fbcb0 0x7f4d680ffd80 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:05:32.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.403+0000 7f4d6e5bb700 1 --2- 172.21.15.32:0/322891644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d680fcf60 0x7f4d680fe2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:05:32.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.403+0000 7f4d6e5bb700 1 -- 172.21.15.32:0/322891644 >> 172.21.15.32:0/322891644 conn(0x7f4d680f7430 msgr2=0x7f4d68065ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:05:32.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.403+0000 7f4d6e5bb700 1 -- 172.21.15.32:0/322891644 shutdown_connections 2024-05-12T04:05:32.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.403+0000 7f4d6e5bb700 1 -- 172.21.15.32:0/322891644 wait complete. 2024-05-12T04:05:32.405 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 18 2024-05-12T04:05:32.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.508+0000 7f7255349700 1 -- 172.21.15.32:0/3726771429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72500fce60 msgr2=0x7f72500fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:05:32.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.508+0000 7f7255349700 1 --2- 172.21.15.32:0/3726771429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72500fce60 0x7f72500fd270 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7240004940 tx=0x7f7240009e30 comp rx=0 tx=0).stop 2024-05-12T04:05:32.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.508+0000 7f7255349700 1 -- 172.21.15.32:0/3726771429 shutdown_connections 2024-05-12T04:05:32.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.508+0000 7f7255349700 1 --2- 172.21.15.32:0/3726771429 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72500fdfd0 0x7f72500fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:05:32.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.508+0000 7f7255349700 1 --2- 172.21.15.32:0/3726771429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72500fce60 0x7f72500fd270 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:05:32.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.508+0000 7f7255349700 1 -- 172.21.15.32:0/3726771429 >> 172.21.15.32:0/3726771429 conn(0x7f72500f8540 msgr2=0x7f72500fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:05:32.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.508+0000 7f7255349700 1 -- 172.21.15.32:0/3726771429 shutdown_connections 2024-05-12T04:05:32.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.508+0000 7f7255349700 1 -- 172.21.15.32:0/3726771429 wait complete. 2024-05-12T04:05:32.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.509+0000 7f7255349700 1 Processor -- start 2024-05-12T04:05:32.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.509+0000 7f7255349700 1 -- start start 2024-05-12T04:05:32.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.509+0000 7f7255349700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72500fce60 0x7f7250071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:05:32.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.509+0000 7f7255349700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72500fdfd0 0x7f72500702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:05:32.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.509+0000 7f7255349700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72500721b0 con 0x7f72500fce60 2024-05-12T04:05:32.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.509+0000 7f7255349700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72500707f0 con 0x7f72500fdfd0 2024-05-12T04:05:32.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.509+0000 7f724effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72500fce60 0x7f7250071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:05:32.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.509+0000 7f724effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72500fce60 0x7f7250071bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38574/0 (socket says 172.21.15.32:38574) 2024-05-12T04:05:32.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.509+0000 7f724effd700 1 -- 172.21.15.32:0/2817384351 learned_addr learned my addr 172.21.15.32:0/2817384351 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:05:32.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.509+0000 7f724e7fc700 1 --2- 172.21.15.32:0/2817384351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72500fdfd0 0x7f72500702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:05:32.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.510+0000 7f724effd700 1 -- 172.21.15.32:0/2817384351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72500fdfd0 msgr2=0x7f72500702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:05:32.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.510+0000 7f724effd700 1 --2- 172.21.15.32:0/2817384351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72500fdfd0 0x7f72500702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:05:32.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.510+0000 7f724effd700 1 -- 172.21.15.32:0/2817384351 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7240005040 con 0x7f72500fce60 2024-05-12T04:05:32.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.510+0000 7f724effd700 1 --2- 172.21.15.32:0/2817384351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72500fce60 0x7f7250071bb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7240009fd0 tx=0x7f7240009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:05:32.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.510+0000 7f72377fe700 1 -- 172.21.15.32:0/2817384351 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f724000e040 con 0x7f72500fce60 2024-05-12T04:05:32.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.510+0000 7f72377fe700 1 -- 172.21.15.32:0/2817384351 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7240004290 con 0x7f72500fce60 2024-05-12T04:05:32.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.510+0000 7f7255349700 1 -- 172.21.15.32:0/2817384351 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7250070a70 con 0x7f72500fce60 2024-05-12T04:05:32.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.510+0000 7f72377fe700 1 -- 172.21.15.32:0/2817384351 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f724001c3f0 con 0x7f72500fce60 2024-05-12T04:05:32.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.510+0000 7f7255349700 1 -- 172.21.15.32:0/2817384351 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7250070fb0 con 0x7f72500fce60 2024-05-12T04:05:32.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.511+0000 7f7255349700 1 -- 172.21.15.32:0/2817384351 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f725004c710 con 0x7f72500fce60 2024-05-12T04:05:32.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.512+0000 7f72377fe700 1 -- 172.21.15.32:0/2817384351 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f7240013790 con 0x7f72500fce60 2024-05-12T04:05:32.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.512+0000 7f72377fe700 1 --2- 172.21.15.32:0/2817384351 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f723806cc20 0x7f723806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:05:32.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.512+0000 7f72377fe700 1 -- 172.21.15.32:0/2817384351 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f724008d400 con 0x7f72500fce60 2024-05-12T04:05:32.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.512+0000 7f724e7fc700 1 --2- 172.21.15.32:0/2817384351 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f723806cc20 0x7f723806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:05:32.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.513+0000 7f724e7fc700 1 --2- 172.21.15.32:0/2817384351 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f723806cc20 0x7f723806f0d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f72440041d0 tx=0x7f72440043c0 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:05:32.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.516+0000 7f72377fe700 1 -- 172.21.15.32:0/2817384351 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f724005af20 con 0x7f72500fce60 2024-05-12T04:05:32.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:32 smithi032 ceph-mon[120392]: from='client.34248 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:05:32.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:32 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:05:32.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:32 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:05:32.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:32 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:05:32.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:32 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T04:05:32.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:32 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:05:32.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:32 smithi032 ceph-mon[120392]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:05:32.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:32 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:05:32.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:32 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:05:32.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:32 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:05:32.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:32 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:05:32.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:32 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-05-12T04:05:32.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:32 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/974991809' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:05:32.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:32 smithi033 ceph-mon[107305]: from='client.34248 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:05:32.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:32 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:05:32.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:32 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:05:32.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:32 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:05:32.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:32 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T04:05:32.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:32 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:05:32.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:32 smithi033 ceph-mon[107305]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:05:32.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:32 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:05:32.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:32 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:05:32.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:32 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:05:32.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:32 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:05:32.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:32 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-05-12T04:05:32.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:32 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/974991809' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:05:32.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.739+0000 7f7255349700 1 -- 172.21.15.32:0/2817384351 --> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f72501023e0 con 0x7f723806cc20 2024-05-12T04:05:32.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.741+0000 7f72377fe700 1 -- 172.21.15.32:0/2817384351 <== mgr.24593 v2:172.21.15.32:6800/2957111591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f72501023e0 con 0x7f723806cc20 2024-05-12T04:05:32.741 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-05-12T04:05:32.741 INFO:teuthology.orchestra.run.smithi032.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:19c9734256aa43a60e203feb242f50bf8b7f58666d520536af43cc4fee96173a", 2024-05-12T04:05:32.741 INFO:teuthology.orchestra.run.smithi032.stdout: "in_progress": true, 2024-05-12T04:05:32.741 INFO:teuthology.orchestra.run.smithi032.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-12T04:05:32.741 INFO:teuthology.orchestra.run.smithi032.stdout: "services_complete": [ 2024-05-12T04:05:32.741 INFO:teuthology.orchestra.run.smithi032.stdout: "mon", 2024-05-12T04:05:32.742 INFO:teuthology.orchestra.run.smithi032.stdout: "crash", 2024-05-12T04:05:32.742 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr" 2024-05-12T04:05:32.742 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-05-12T04:05:32.742 INFO:teuthology.orchestra.run.smithi032.stdout: "progress": "8/21 daemons upgraded", 2024-05-12T04:05:32.742 INFO:teuthology.orchestra.run.smithi032.stdout: "message": "Currently upgrading osd daemons", 2024-05-12T04:05:32.742 INFO:teuthology.orchestra.run.smithi032.stdout: "is_paused": false 2024-05-12T04:05:32.742 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-05-12T04:05:32.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.743+0000 7f7255349700 1 -- 172.21.15.32:0/2817384351 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f723806cc20 msgr2=0x7f723806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:05:32.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.743+0000 7f7255349700 1 --2- 172.21.15.32:0/2817384351 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f723806cc20 0x7f723806f0d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f72440041d0 tx=0x7f72440043c0 comp rx=0 tx=0).stop 2024-05-12T04:05:32.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.743+0000 7f7255349700 1 -- 172.21.15.32:0/2817384351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72500fce60 msgr2=0x7f7250071bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:05:32.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.743+0000 7f7255349700 1 --2- 172.21.15.32:0/2817384351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72500fce60 0x7f7250071bb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7240009fd0 tx=0x7f7240009e30 comp rx=0 tx=0).stop 2024-05-12T04:05:32.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.743+0000 7f7255349700 1 -- 172.21.15.32:0/2817384351 shutdown_connections 2024-05-12T04:05:32.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.743+0000 7f7255349700 1 --2- 172.21.15.32:0/2817384351 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f723806cc20 0x7f723806f0d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:05:32.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.743+0000 7f7255349700 1 --2- 172.21.15.32:0/2817384351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72500fce60 0x7f7250071bb0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:05:32.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.743+0000 7f7255349700 1 --2- 172.21.15.32:0/2817384351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72500fdfd0 0x7f72500702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:05:32.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.743+0000 7f7255349700 1 -- 172.21.15.32:0/2817384351 >> 172.21.15.32:0/2817384351 conn(0x7f72500f8540 msgr2=0x7f72500f9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:05:32.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.743+0000 7f7255349700 1 -- 172.21.15.32:0/2817384351 shutdown_connections 2024-05-12T04:05:32.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.743+0000 7f7255349700 1 -- 172.21.15.32:0/2817384351 wait complete. 2024-05-12T04:05:32.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.847+0000 7fa641c23700 1 -- 172.21.15.32:0/2422188816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa63c0fff40 msgr2=0x7fa63c102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:05:32.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.847+0000 7fa641c23700 1 --2- 172.21.15.32:0/2422188816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa63c0fff40 0x7fa63c102320 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa624004940 tx=0x7fa624009e30 comp rx=0 tx=0).stop 2024-05-12T04:05:32.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.847+0000 7fa641c23700 1 -- 172.21.15.32:0/2422188816 shutdown_connections 2024-05-12T04:05:32.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.847+0000 7fa641c23700 1 --2- 172.21.15.32:0/2422188816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa63c0fff40 0x7fa63c102320 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:05:32.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.847+0000 7fa641c23700 1 --2- 172.21.15.32:0/2422188816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa63c066260 0x7fa63c0ffa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:05:32.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.847+0000 7fa641c23700 1 -- 172.21.15.32:0/2422188816 >> 172.21.15.32:0/2422188816 conn(0x7fa63c0f7430 msgr2=0x7fa63c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:05:32.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.847+0000 7fa641c23700 1 -- 172.21.15.32:0/2422188816 shutdown_connections 2024-05-12T04:05:32.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.847+0000 7fa641c23700 1 -- 172.21.15.32:0/2422188816 wait complete. 2024-05-12T04:05:32.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.848+0000 7fa641c23700 1 Processor -- start 2024-05-12T04:05:32.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.848+0000 7fa641c23700 1 -- start start 2024-05-12T04:05:32.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.848+0000 7fa641c23700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa63c066260 0x7fa63c10d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:05:32.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.848+0000 7fa641c23700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa63c0fff40 0x7fa63c10deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:05:32.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.848+0000 7fa641c23700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63c10e4b0 con 0x7fa63c0fff40 2024-05-12T04:05:32.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.848+0000 7fa641c23700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63c10ecb0 con 0x7fa63c066260 2024-05-12T04:05:32.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.848+0000 7fa632ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa63c0fff40 0x7fa63c10deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:05:32.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.848+0000 7fa632ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa63c0fff40 0x7fa63c10deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38584/0 (socket says 172.21.15.32:38584) 2024-05-12T04:05:32.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.848+0000 7fa632ffd700 1 -- 172.21.15.32:0/4293817348 learned_addr learned my addr 172.21.15.32:0/4293817348 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:05:32.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.848+0000 7fa63b7fe700 1 --2- 172.21.15.32:0/4293817348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa63c066260 0x7fa63c10d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:05:32.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.849+0000 7fa632ffd700 1 -- 172.21.15.32:0/4293817348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa63c066260 msgr2=0x7fa63c10d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:05:32.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.849+0000 7fa632ffd700 1 --2- 172.21.15.32:0/4293817348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa63c066260 0x7fa63c10d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:05:32.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.849+0000 7fa632ffd700 1 -- 172.21.15.32:0/4293817348 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa624005040 con 0x7fa63c0fff40 2024-05-12T04:05:32.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.849+0000 7fa632ffd700 1 --2- 172.21.15.32:0/4293817348 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa63c0fff40 0x7fa63c10deb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa62400b3b0 tx=0x7fa624009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:05:32.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.849+0000 7fa638ff9700 1 -- 172.21.15.32:0/4293817348 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa62400e040 con 0x7fa63c0fff40 2024-05-12T04:05:32.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.849+0000 7fa638ff9700 1 -- 172.21.15.32:0/4293817348 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa62400bc50 con 0x7fa63c0fff40 2024-05-12T04:05:32.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.849+0000 7fa638ff9700 1 -- 172.21.15.32:0/4293817348 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa62401c630 con 0x7fa63c0fff40 2024-05-12T04:05:32.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.849+0000 7fa641c23700 1 -- 172.21.15.32:0/4293817348 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa63c10ef30 con 0x7fa63c0fff40 2024-05-12T04:05:32.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.849+0000 7fa641c23700 1 -- 172.21.15.32:0/4293817348 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa63c10f470 con 0x7fa63c0fff40 2024-05-12T04:05:32.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.850+0000 7fa641c23700 1 -- 172.21.15.32:0/4293817348 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa63c05f5c0 con 0x7fa63c0fff40 2024-05-12T04:05:32.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.851+0000 7fa638ff9700 1 -- 172.21.15.32:0/4293817348 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fa624013420 con 0x7fa63c0fff40 2024-05-12T04:05:32.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.851+0000 7fa638ff9700 1 --2- 172.21.15.32:0/4293817348 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fa62806cc20 0x7fa62806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:05:32.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.851+0000 7fa638ff9700 1 -- 172.21.15.32:0/4293817348 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fa62408cb00 con 0x7fa63c0fff40 2024-05-12T04:05:32.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.851+0000 7fa63b7fe700 1 --2- 172.21.15.32:0/4293817348 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fa62806cc20 0x7fa62806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:05:32.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.852+0000 7fa63b7fe700 1 --2- 172.21.15.32:0/4293817348 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fa62806cc20 0x7fa62806f0d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa63c065fa0 tx=0x7fa62c00b040 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:05:32.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:32.855+0000 7fa638ff9700 1 -- 172.21.15.32:0/4293817348 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fa62405a6c0 con 0x7fa63c0fff40 2024-05-12T04:05:33.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:33.142+0000 7fa641c23700 1 -- 172.21.15.32:0/4293817348 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa63c10f780 con 0x7fa63c0fff40 2024-05-12T04:05:33.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:33.142+0000 7fa638ff9700 1 -- 172.21.15.32:0/4293817348 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+758 (secure 0 0 0) 0x7fa62405a250 con 0x7fa63c0fff40 2024-05-12T04:05:33.143 INFO:teuthology.orchestra.run.smithi032.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; Degraded data redundancy: 19/276 objects degraded (6.884%), 7 pgs degraded 2024-05-12T04:05:33.143 INFO:teuthology.orchestra.run.smithi032.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-12T04:05:33.143 INFO:teuthology.orchestra.run.smithi032.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-12T04:05:33.143 INFO:teuthology.orchestra.run.smithi032.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 19/276 objects degraded (6.884%), 7 pgs degraded 2024-05-12T04:05:33.143 INFO:teuthology.orchestra.run.smithi032.stdout: pg 2.4 is active+undersized+degraded, acting [0,4] 2024-05-12T04:05:33.143 INFO:teuthology.orchestra.run.smithi032.stdout: pg 2.6 is active+undersized+degraded, acting [3,4] 2024-05-12T04:05:33.143 INFO:teuthology.orchestra.run.smithi032.stdout: pg 2.9 is active+undersized+degraded, acting [4,0] 2024-05-12T04:05:33.143 INFO:teuthology.orchestra.run.smithi032.stdout: pg 2.a is active+undersized+degraded, acting [4,3] 2024-05-12T04:05:33.144 INFO:teuthology.orchestra.run.smithi032.stdout: pg 2.d is active+undersized+degraded, acting [3,2] 2024-05-12T04:05:33.144 INFO:teuthology.orchestra.run.smithi032.stdout: pg 2.15 is active+undersized+degraded, acting [3,0] 2024-05-12T04:05:33.144 INFO:teuthology.orchestra.run.smithi032.stdout: pg 2.1b is active+undersized+degraded, acting [0,5] 2024-05-12T04:05:33.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:33.145+0000 7fa641c23700 1 -- 172.21.15.32:0/4293817348 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fa62806cc20 msgr2=0x7fa62806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:05:33.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:33.145+0000 7fa641c23700 1 --2- 172.21.15.32:0/4293817348 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fa62806cc20 0x7fa62806f0d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa63c065fa0 tx=0x7fa62c00b040 comp rx=0 tx=0).stop 2024-05-12T04:05:33.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:33.145+0000 7fa641c23700 1 -- 172.21.15.32:0/4293817348 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa63c0fff40 msgr2=0x7fa63c10deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:05:33.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:33.145+0000 7fa641c23700 1 --2- 172.21.15.32:0/4293817348 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa63c0fff40 0x7fa63c10deb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa62400b3b0 tx=0x7fa624009e30 comp rx=0 tx=0).stop 2024-05-12T04:05:33.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:33.145+0000 7fa641c23700 1 -- 172.21.15.32:0/4293817348 shutdown_connections 2024-05-12T04:05:33.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:33.145+0000 7fa641c23700 1 --2- 172.21.15.32:0/4293817348 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fa62806cc20 0x7fa62806f0d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:05:33.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:33.145+0000 7fa641c23700 1 --2- 172.21.15.32:0/4293817348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa63c066260 0x7fa63c10d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:05:33.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:33.145+0000 7fa641c23700 1 --2- 172.21.15.32:0/4293817348 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa63c0fff40 0x7fa63c10deb0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:05:33.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:33.145+0000 7fa641c23700 1 -- 172.21.15.32:0/4293817348 >> 172.21.15.32:0/4293817348 conn(0x7fa63c0f7430 msgr2=0x7fa63c0f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:05:33.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:33.145+0000 7fa641c23700 1 -- 172.21.15.32:0/4293817348 shutdown_connections 2024-05-12T04:05:33.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:05:33.145+0000 7fa641c23700 1 -- 172.21.15.32:0/4293817348 wait complete. 2024-05-12T04:05:33.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:33 smithi032 ceph-mon[120392]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-05-12T04:05:33.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:33 smithi032 ceph-mon[120392]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-05-12T04:05:33.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:33 smithi032 ceph-mon[120392]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:05:33.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:33 smithi032 ceph-mon[120392]: pgmap v81: 65 pgs: 7 active+undersized+degraded, 5 active+undersized, 53 active+clean; 260 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 19/276 objects degraded (6.884%) 2024-05-12T04:05:33.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:33 smithi032 ceph-mon[120392]: Health check update: Degraded data redundancy: 19/276 objects degraded (6.884%), 7 pgs degraded (PG_DEGRADED) 2024-05-12T04:05:33.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:33 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/322891644' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-12T04:05:33.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:33 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/4293817348' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-12T04:05:33.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:33 smithi033 ceph-mon[107305]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-05-12T04:05:33.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:33 smithi033 ceph-mon[107305]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-05-12T04:05:33.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:33 smithi033 ceph-mon[107305]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:05:33.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:33 smithi033 ceph-mon[107305]: pgmap v81: 65 pgs: 7 active+undersized+degraded, 5 active+undersized, 53 active+clean; 260 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 19/276 objects degraded (6.884%) 2024-05-12T04:05:33.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:33 smithi033 ceph-mon[107305]: Health check update: Degraded data redundancy: 19/276 objects degraded (6.884%), 7 pgs degraded (PG_DEGRADED) 2024-05-12T04:05:33.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:33 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/322891644' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-12T04:05:33.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:33 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/4293817348' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-12T04:05:34.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:34 smithi032 ceph-mon[120392]: from='client.34268 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:05:34.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:34 smithi033 ceph-mon[107305]: from='client.34268 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:05:35.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:35 smithi032 ceph-mon[120392]: pgmap v82: 65 pgs: 65 active+clean; 260 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:05:35.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:35 smithi032 ceph-mon[120392]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 19/276 objects degraded (6.884%), 7 pgs degraded) 2024-05-12T04:05:35.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:35 smithi033 ceph-mon[107305]: pgmap v82: 65 pgs: 65 active+clean; 260 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:05:35.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:35 smithi033 ceph-mon[107305]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 19/276 objects degraded (6.884%), 7 pgs degraded) 2024-05-12T04:05:37.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:37 smithi032 ceph-mon[120392]: pgmap v83: 65 pgs: 65 active+clean; 260 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 644 B/s rd, 1 op/s 2024-05-12T04:05:37.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:37 smithi033 ceph-mon[107305]: pgmap v83: 65 pgs: 65 active+clean; 260 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 644 B/s rd, 1 op/s 2024-05-12T04:05:39.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:39 smithi032 ceph-mon[120392]: pgmap v84: 65 pgs: 65 active+clean; 260 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s 2024-05-12T04:05:39.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:39 smithi033 ceph-mon[107305]: pgmap v84: 65 pgs: 65 active+clean; 260 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s 2024-05-12T04:05:41.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:41 smithi032 ceph-mon[120392]: pgmap v85: 65 pgs: 65 active+clean; 260 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-05-12T04:05:41.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:41 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:05:41.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:41 smithi033 ceph-mon[107305]: pgmap v85: 65 pgs: 65 active+clean; 260 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-05-12T04:05:41.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:41 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:05:43.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:43 smithi032 ceph-mon[120392]: pgmap v86: 65 pgs: 65 active+clean; 260 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s 2024-05-12T04:05:43.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:43 smithi033 ceph-mon[107305]: pgmap v86: 65 pgs: 65 active+clean; 260 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s 2024-05-12T04:05:45.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:45 smithi032 ceph-mon[120392]: pgmap v87: 65 pgs: 65 active+clean; 260 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:05:45.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:45 smithi033 ceph-mon[107305]: pgmap v87: 65 pgs: 65 active+clean; 260 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:05:47.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:46 smithi033 ceph-mon[107305]: pgmap v88: 65 pgs: 65 active+clean; 260 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:05:47.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:46 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:05:47.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:46 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:05:47.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:46 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-05-12T04:05:47.263 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:46 smithi032 ceph-mon[120392]: pgmap v88: 65 pgs: 65 active+clean; 260 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:05:47.263 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:46 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:05:47.263 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:46 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:05:47.263 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:46 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-05-12T04:05:48.038 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:47 smithi032 ceph-mon[120392]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-05-12T04:05:48.039 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:47 smithi032 ceph-mon[120392]: Upgrade: osd.2 is safe to restart 2024-05-12T04:05:48.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:47 smithi033 ceph-mon[107305]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-05-12T04:05:48.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:47 smithi033 ceph-mon[107305]: Upgrade: osd.2 is safe to restart 2024-05-12T04:05:49.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:49 smithi032 ceph-mon[120392]: pgmap v89: 65 pgs: 65 active+clean; 260 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:05:49.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:49 smithi032 ceph-mon[120392]: Upgrade: Updating osd.2 2024-05-12T04:05:49.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:49 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:05:49.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:49 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-12T04:05:49.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:49 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:05:49.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:49 smithi032 ceph-mon[120392]: Deploying daemon osd.2 on smithi032 2024-05-12T04:05:49.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:49 smithi033 ceph-mon[107305]: pgmap v89: 65 pgs: 65 active+clean; 260 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:05:49.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:49 smithi033 ceph-mon[107305]: Upgrade: Updating osd.2 2024-05-12T04:05:49.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:49 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:05:49.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:49 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-12T04:05:49.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:49 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:05:49.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:49 smithi033 ceph-mon[107305]: Deploying daemon osd.2 on smithi032 2024-05-12T04:05:51.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:51 smithi032 ceph-mon[120392]: pgmap v90: 65 pgs: 65 active+clean; 260 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:05:51.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:51 smithi033 ceph-mon[107305]: pgmap v90: 65 pgs: 65 active+clean; 260 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:05:51.790 INFO:journalctl@ceph.osd.2.smithi032.stdout:May 12 04:05:51 smithi032 systemd[1]: Stopping Ceph osd.2 for a3e178f8-1012-11ef-bc98-c7b262605968... 2024-05-12T04:05:51.790 INFO:journalctl@ceph.osd.2.smithi032.stdout:May 12 04:05:51 smithi032 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-2[82340]: 2024-05-12T04:05:51.573+0000 7fc35b381700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-12T04:05:51.790 INFO:journalctl@ceph.osd.2.smithi032.stdout:May 12 04:05:51 smithi032 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-2[82340]: 2024-05-12T04:05:51.573+0000 7fc35b381700 -1 osd.2 59 *** Got signal Terminated *** 2024-05-12T04:05:51.790 INFO:journalctl@ceph.osd.2.smithi032.stdout:May 12 04:05:51 smithi032 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-2[82340]: 2024-05-12T04:05:51.573+0000 7fc35b381700 -1 osd.2 59 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-12T04:05:52.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:52 smithi033 ceph-mon[107305]: osd.2 marked itself down and dead 2024-05-12T04:05:52.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:52 smithi032 ceph-mon[120392]: osd.2 marked itself down and dead 2024-05-12T04:05:53.129 INFO:journalctl@ceph.osd.2.smithi032.stdout:May 12 04:05:52 smithi032 bash[145409]: ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-2 2024-05-12T04:05:53.398 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:53 smithi032 ceph-mon[120392]: pgmap v91: 65 pgs: 65 active+clean; 260 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:05:53.398 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:53 smithi032 ceph-mon[120392]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-12T04:05:53.399 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:53 smithi032 ceph-mon[120392]: osdmap e60: 6 total, 5 up, 6 in 2024-05-12T04:05:53.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:53 smithi033 ceph-mon[107305]: pgmap v91: 65 pgs: 65 active+clean; 260 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:05:53.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:53 smithi033 ceph-mon[107305]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-12T04:05:53.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:53 smithi033 ceph-mon[107305]: osdmap e60: 6 total, 5 up, 6 in 2024-05-12T04:05:54.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:54 smithi033 ceph-mon[107305]: osdmap e61: 6 total, 5 up, 6 in 2024-05-12T04:05:54.502 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:54 smithi032 ceph-mon[120392]: osdmap e61: 6 total, 5 up, 6 in 2024-05-12T04:05:55.135 INFO:journalctl@ceph.osd.2.smithi032.stdout:May 12 04:05:54 smithi032 systemd[1]: ceph-a3e178f8-1012-11ef-bc98-c7b262605968@osd.2.service: Succeeded. 2024-05-12T04:05:55.135 INFO:journalctl@ceph.osd.2.smithi032.stdout:May 12 04:05:54 smithi032 systemd[1]: Stopped Ceph osd.2 for a3e178f8-1012-11ef-bc98-c7b262605968. 2024-05-12T04:05:55.135 INFO:journalctl@ceph.osd.2.smithi032.stdout:May 12 04:05:54 smithi032 systemd[1]: Starting Ceph osd.2 for a3e178f8-1012-11ef-bc98-c7b262605968... 2024-05-12T04:05:55.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:55 smithi033 ceph-mon[107305]: pgmap v94: 65 pgs: 7 peering, 5 stale+active+clean, 53 active+clean; 260 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 383 B/s rd, 2 op/s 2024-05-12T04:05:55.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:55 smithi033 ceph-mon[107305]: Health check failed: Reduced data availability: 4 pgs peering (PG_AVAILABILITY) 2024-05-12T04:05:55.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:55 smithi032 ceph-mon[120392]: pgmap v94: 65 pgs: 7 peering, 5 stale+active+clean, 53 active+clean; 260 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 383 B/s rd, 2 op/s 2024-05-12T04:05:55.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:55 smithi032 ceph-mon[120392]: Health check failed: Reduced data availability: 4 pgs peering (PG_AVAILABILITY) 2024-05-12T04:05:56.249 INFO:journalctl@ceph.osd.2.smithi032.stdout:May 12 04:05:55 smithi032 bash[145679]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-12T04:05:56.249 INFO:journalctl@ceph.osd.2.smithi032.stdout:May 12 04:05:55 smithi032 bash[145679]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-05-12T04:05:56.249 INFO:journalctl@ceph.osd.2.smithi032.stdout:May 12 04:05:55 smithi032 bash[145679]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-05-12T04:05:56.249 INFO:journalctl@ceph.osd.2.smithi032.stdout:May 12 04:05:55 smithi032 bash[145679]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-05-12T04:05:56.249 INFO:journalctl@ceph.osd.2.smithi032.stdout:May 12 04:05:55 smithi032 bash[145679]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-05-12T04:05:56.249 INFO:journalctl@ceph.osd.2.smithi032.stdout:May 12 04:05:55 smithi032 bash[145679]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-12T04:05:56.249 INFO:journalctl@ceph.osd.2.smithi032.stdout:May 12 04:05:55 smithi032 bash[145679]: --> ceph-volume raw activate successful for osd ID: 2 2024-05-12T04:05:57.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:57 smithi033 ceph-mon[107305]: pgmap v95: 65 pgs: 7 peering, 5 stale+active+clean, 53 active+clean; 260 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-05-12T04:05:57.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:57 smithi032 ceph-mon[120392]: pgmap v95: 65 pgs: 7 peering, 5 stale+active+clean, 53 active+clean; 260 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-05-12T04:05:57.540 INFO:journalctl@ceph.osd.2.smithi032.stdout:May 12 04:05:57 smithi032 ceph-osd[145998]: -- 172.21.15.32:0/2802372885 <== mon.0 v2:172.21.15.32:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c6cae351e0 con 0x55c6cae04c00 2024-05-12T04:05:57.540 INFO:journalctl@ceph.osd.2.smithi032.stdout:May 12 04:05:57 smithi032 bash[145961]: 4ac48d74ee8bbc8bebdea2a10c5075cc623fe4edac9b3e30271ed50489561a67 2024-05-12T04:05:57.541 INFO:journalctl@ceph.osd.2.smithi032.stdout:May 12 04:05:57 smithi032 systemd[1]: Started Ceph osd.2 for a3e178f8-1012-11ef-bc98-c7b262605968. 2024-05-12T04:05:58.040 INFO:journalctl@ceph.osd.2.smithi032.stdout:May 12 04:05:57 smithi032 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-2[145975]: 2024-05-12T04:05:57.659+0000 7f9fe486c7c0 -1 Falling back to public interface 2024-05-12T04:05:58.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:58 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:05:58.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:58 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:05:58.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:58 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:05:58.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:58 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:05:58.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:58 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:05:58.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:58 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:05:59.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:59 smithi033 ceph-mon[107305]: pgmap v96: 65 pgs: 9 active+undersized+degraded, 7 peering, 11 active+undersized, 38 active+clean; 260 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 383 B/s rd, 1 op/s; 21/276 objects degraded (7.609%) 2024-05-12T04:05:59.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:05:59 smithi033 ceph-mon[107305]: Health check failed: Degraded data redundancy: 21/276 objects degraded (7.609%), 9 pgs degraded (PG_DEGRADED) 2024-05-12T04:05:59.695 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:59 smithi032 ceph-mon[120392]: pgmap v96: 65 pgs: 9 active+undersized+degraded, 7 peering, 11 active+undersized, 38 active+clean; 260 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 383 B/s rd, 1 op/s; 21/276 objects degraded (7.609%) 2024-05-12T04:05:59.695 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:05:59 smithi032 ceph-mon[120392]: Health check failed: Degraded data redundancy: 21/276 objects degraded (7.609%), 9 pgs degraded (PG_DEGRADED) 2024-05-12T04:06:00.635 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:00 smithi032 ceph-mon[120392]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 4 pgs peering) 2024-05-12T04:06:00.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:00 smithi033 ceph-mon[107305]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 4 pgs peering) 2024-05-12T04:06:01.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:01 smithi033 ceph-mon[107305]: pgmap v97: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 260 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 36/276 objects degraded (13.043%) 2024-05-12T04:06:01.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:01 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:06:01.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:01 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:06:01.789 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:01 smithi032 ceph-mon[120392]: pgmap v97: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 260 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 36/276 objects degraded (13.043%) 2024-05-12T04:06:01.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:01 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:06:01.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:01 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:06:02.269 INFO:journalctl@ceph.osd.2.smithi032.stdout:May 12 04:06:01 smithi032 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-2[145975]: 2024-05-12T04:06:01.860+0000 7f9fe486c7c0 -1 osd.2 59 log_to_monitors true 2024-05-12T04:06:02.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:02 smithi032 ceph-mon[120392]: from='osd.2 [v2:172.21.15.32:6818/793261683,v1:172.21.15.32:6819/793261683]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-12T04:06:02.540 INFO:journalctl@ceph.osd.2.smithi032.stdout:May 12 04:06:02 smithi032 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-2[145975]: 2024-05-12T04:06:02.368+0000 7f9fd5cc5700 -1 osd.2 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-12T04:06:02.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:02 smithi033 ceph-mon[107305]: from='osd.2 [v2:172.21.15.32:6818/793261683,v1:172.21.15.32:6819/793261683]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-12T04:06:03.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.254+0000 7f263876e700 1 -- 172.21.15.32:0/3396093492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26300fe3e0 msgr2=0x7f26300fe7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:06:03.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.254+0000 7f263876e700 1 --2- 172.21.15.32:0/3396093492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26300fe3e0 0x7f26300fe7f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f262c004940 tx=0x7f262c009e30 comp rx=0 tx=0).stop 2024-05-12T04:06:03.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.255+0000 7f263876e700 1 -- 172.21.15.32:0/3396093492 shutdown_connections 2024-05-12T04:06:03.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.255+0000 7f263876e700 1 --2- 172.21.15.32:0/3396093492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26300fed30 0x7f26300fae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:03.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.255+0000 7f263876e700 1 --2- 172.21.15.32:0/3396093492 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26300fe3e0 0x7f26300fe7f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:03.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.255+0000 7f263876e700 1 -- 172.21.15.32:0/3396093492 >> 172.21.15.32:0/3396093492 conn(0x7f26300f6b80 msgr2=0x7f26300f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:06:03.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.255+0000 7f263876e700 1 -- 172.21.15.32:0/3396093492 shutdown_connections 2024-05-12T04:06:03.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.255+0000 7f263876e700 1 -- 172.21.15.32:0/3396093492 wait complete. 2024-05-12T04:06:03.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.255+0000 7f263876e700 1 Processor -- start 2024-05-12T04:06:03.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.255+0000 7f263876e700 1 -- start start 2024-05-12T04:06:03.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.255+0000 7f263876e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26300fe3e0 0x7f26300fdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:06:03.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.255+0000 7f263876e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26300fed30 0x7f26300fc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:06:03.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.255+0000 7f263876e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26300fc650 con 0x7f26300fed30 2024-05-12T04:06:03.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.255+0000 7f263876e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26300fc7c0 con 0x7f26300fe3e0 2024-05-12T04:06:03.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.255+0000 7f2635d09700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26300fed30 0x7f26300fc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:06:03.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.256+0000 7f2635d09700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26300fed30 0x7f26300fc0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55332/0 (socket says 172.21.15.32:55332) 2024-05-12T04:06:03.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.256+0000 7f2635d09700 1 -- 172.21.15.32:0/4006876360 learned_addr learned my addr 172.21.15.32:0/4006876360 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:06:03.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.256+0000 7f263650a700 1 --2- 172.21.15.32:0/4006876360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26300fe3e0 0x7f26300fdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:06:03.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.256+0000 7f2635d09700 1 -- 172.21.15.32:0/4006876360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26300fe3e0 msgr2=0x7f26300fdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:06:03.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.256+0000 7f2635d09700 1 --2- 172.21.15.32:0/4006876360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26300fe3e0 0x7f26300fdbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:03.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.256+0000 7f2635d09700 1 -- 172.21.15.32:0/4006876360 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f262c005040 con 0x7f26300fed30 2024-05-12T04:06:03.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.256+0000 7f2635d09700 1 --2- 172.21.15.32:0/4006876360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26300fed30 0x7f26300fc0a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f262000c8c0 tx=0x7f26200044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:06:03.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.256+0000 7f2626ffd700 1 -- 172.21.15.32:0/4006876360 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f262000a760 con 0x7f26300fed30 2024-05-12T04:06:03.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.256+0000 7f263876e700 1 -- 172.21.15.32:0/4006876360 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26300fcaa0 con 0x7f26300fed30 2024-05-12T04:06:03.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.256+0000 7f263876e700 1 -- 172.21.15.32:0/4006876360 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2630115940 con 0x7f26300fed30 2024-05-12T04:06:03.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.256+0000 7f2626ffd700 1 -- 172.21.15.32:0/4006876360 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f262000ad70 con 0x7f26300fed30 2024-05-12T04:06:03.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.257+0000 7f2626ffd700 1 -- 172.21.15.32:0/4006876360 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f262001bdb0 con 0x7f26300fed30 2024-05-12T04:06:03.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.257+0000 7f263876e700 1 -- 172.21.15.32:0/4006876360 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26140052e0 con 0x7f26300fed30 2024-05-12T04:06:03.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.258+0000 7f2626ffd700 1 -- 172.21.15.32:0/4006876360 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f262000a8c0 con 0x7f26300fed30 2024-05-12T04:06:03.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.258+0000 7f2626ffd700 1 --2- 172.21.15.32:0/4006876360 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f261c06cc30 0x7f261c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:06:03.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.258+0000 7f2626ffd700 1 -- 172.21.15.32:0/4006876360 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f262008bff0 con 0x7f26300fed30 2024-05-12T04:06:03.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.259+0000 7f263650a700 1 --2- 172.21.15.32:0/4006876360 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f261c06cc30 0x7f261c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:06:03.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.259+0000 7f263650a700 1 --2- 172.21.15.32:0/4006876360 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f261c06cc30 0x7f261c06f0e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f262c00b3b0 tx=0x7f262c009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:06:03.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.264+0000 7f2626ffd700 1 -- 172.21.15.32:0/4006876360 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f2620056080 con 0x7f26300fed30 2024-05-12T04:06:03.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.523+0000 7f263876e700 1 -- 172.21.15.32:0/4006876360 --> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2614000bf0 con 0x7f261c06cc30 2024-05-12T04:06:03.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.524+0000 7f2626ffd700 1 -- 172.21.15.32:0/4006876360 <== mgr.24593 v2:172.21.15.32:6800/2957111591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f2614000bf0 con 0x7f261c06cc30 2024-05-12T04:06:03.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.527+0000 7f263876e700 1 -- 172.21.15.32:0/4006876360 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f261c06cc30 msgr2=0x7f261c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:06:03.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.527+0000 7f263876e700 1 --2- 172.21.15.32:0/4006876360 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f261c06cc30 0x7f261c06f0e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f262c00b3b0 tx=0x7f262c009e30 comp rx=0 tx=0).stop 2024-05-12T04:06:03.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.527+0000 7f263876e700 1 -- 172.21.15.32:0/4006876360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26300fed30 msgr2=0x7f26300fc0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:06:03.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.527+0000 7f263876e700 1 --2- 172.21.15.32:0/4006876360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26300fed30 0x7f26300fc0a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f262000c8c0 tx=0x7f26200044c0 comp rx=0 tx=0).stop 2024-05-12T04:06:03.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.527+0000 7f263876e700 1 -- 172.21.15.32:0/4006876360 shutdown_connections 2024-05-12T04:06:03.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.527+0000 7f263876e700 1 --2- 172.21.15.32:0/4006876360 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f261c06cc30 0x7f261c06f0e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:03.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.527+0000 7f263876e700 1 --2- 172.21.15.32:0/4006876360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26300fe3e0 0x7f26300fdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:03.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.527+0000 7f263876e700 1 --2- 172.21.15.32:0/4006876360 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26300fed30 0x7f26300fc0a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:03.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.527+0000 7f263876e700 1 -- 172.21.15.32:0/4006876360 >> 172.21.15.32:0/4006876360 conn(0x7f26300f6b80 msgr2=0x7f26300f8370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:06:03.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.527+0000 7f263876e700 1 -- 172.21.15.32:0/4006876360 shutdown_connections 2024-05-12T04:06:03.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.527+0000 7f263876e700 1 -- 172.21.15.32:0/4006876360 wait complete. 2024-05-12T04:06:03.538 INFO:teuthology.orchestra.run.smithi032.stdout:true 2024-05-12T04:06:03.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.645+0000 7f6a48669700 1 -- 172.21.15.32:0/3725555287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a400662c0 msgr2=0x7f6a400ff9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:06:03.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.645+0000 7f6a48669700 1 --2- 172.21.15.32:0/3725555287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a400662c0 0x7f6a400ff9a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c004940 tx=0x7f6a3c009e30 comp rx=0 tx=0).stop 2024-05-12T04:06:03.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.645+0000 7f6a48669700 1 -- 172.21.15.32:0/3725555287 shutdown_connections 2024-05-12T04:06:03.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.645+0000 7f6a48669700 1 --2- 172.21.15.32:0/3725555287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a400ffee0 0x7f6a40102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:03.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.645+0000 7f6a48669700 1 --2- 172.21.15.32:0/3725555287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a400662c0 0x7f6a400ff9a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:03.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.645+0000 7f6a48669700 1 -- 172.21.15.32:0/3725555287 >> 172.21.15.32:0/3725555287 conn(0x7f6a400f7430 msgr2=0x7f6a400f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:06:03.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.645+0000 7f6a48669700 1 -- 172.21.15.32:0/3725555287 shutdown_connections 2024-05-12T04:06:03.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.646+0000 7f6a48669700 1 -- 172.21.15.32:0/3725555287 wait complete. 2024-05-12T04:06:03.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.646+0000 7f6a48669700 1 Processor -- start 2024-05-12T04:06:03.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.646+0000 7f6a48669700 1 -- start start 2024-05-12T04:06:03.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.646+0000 7f6a48669700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a400662c0 0x7f6a4010d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:06:03.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.646+0000 7f6a48669700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a400ffee0 0x7f6a4010de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:06:03.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.646+0000 7f6a48669700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4010e460 con 0x7f6a400ffee0 2024-05-12T04:06:03.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.646+0000 7f6a48669700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4010ecb0 con 0x7f6a400662c0 2024-05-12T04:06:03.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.646+0000 7f6a45c04700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a400ffee0 0x7f6a4010de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:06:03.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.646+0000 7f6a45c04700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a400ffee0 0x7f6a4010de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55338/0 (socket says 172.21.15.32:55338) 2024-05-12T04:06:03.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.646+0000 7f6a45c04700 1 -- 172.21.15.32:0/1454675245 learned_addr learned my addr 172.21.15.32:0/1454675245 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:06:03.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.647+0000 7f6a46405700 1 --2- 172.21.15.32:0/1454675245 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a400662c0 0x7f6a4010d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:06:03.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.647+0000 7f6a45c04700 1 -- 172.21.15.32:0/1454675245 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a400662c0 msgr2=0x7f6a4010d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:06:03.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.647+0000 7f6a45c04700 1 --2- 172.21.15.32:0/1454675245 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a400662c0 0x7f6a4010d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:03.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.647+0000 7f6a45c04700 1 -- 172.21.15.32:0/1454675245 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a3c005040 con 0x7f6a400ffee0 2024-05-12T04:06:03.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.647+0000 7f6a45c04700 1 --2- 172.21.15.32:0/1454675245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a400ffee0 0x7f6a4010de60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6a30002a50 tx=0x7f6a300046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:06:03.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.647+0000 7f6a36ffd700 1 -- 172.21.15.32:0/1454675245 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a30007cc0 con 0x7f6a400ffee0 2024-05-12T04:06:03.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.647+0000 7f6a36ffd700 1 -- 172.21.15.32:0/1454675245 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6a3000f040 con 0x7f6a400ffee0 2024-05-12T04:06:03.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.647+0000 7f6a48669700 1 -- 172.21.15.32:0/1454675245 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a4010ef90 con 0x7f6a400ffee0 2024-05-12T04:06:03.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.647+0000 7f6a36ffd700 1 -- 172.21.15.32:0/1454675245 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a3000d9c0 con 0x7f6a400ffee0 2024-05-12T04:06:03.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.647+0000 7f6a48669700 1 -- 172.21.15.32:0/1454675245 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a4010f4d0 con 0x7f6a400ffee0 2024-05-12T04:06:03.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.648+0000 7f6a48669700 1 -- 172.21.15.32:0/1454675245 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a4005f5c0 con 0x7f6a400ffee0 2024-05-12T04:06:03.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.650+0000 7f6a36ffd700 1 -- 172.21.15.32:0/1454675245 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f6a3000db20 con 0x7f6a400ffee0 2024-05-12T04:06:03.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.650+0000 7f6a36ffd700 1 --2- 172.21.15.32:0/1454675245 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f6a2c06cc20 0x7f6a2c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:06:03.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.650+0000 7f6a36ffd700 1 -- 172.21.15.32:0/1454675245 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f6a3008c2f0 con 0x7f6a400ffee0 2024-05-12T04:06:03.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.650+0000 7f6a46405700 1 --2- 172.21.15.32:0/1454675245 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f6a2c06cc20 0x7f6a2c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:06:03.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.650+0000 7f6a46405700 1 --2- 172.21.15.32:0/1454675245 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f6a2c06cc20 0x7f6a2c06f0d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c00b900 tx=0x7f6a3c009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:06:03.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.655+0000 7f6a36ffd700 1 -- 172.21.15.32:0/1454675245 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f6a30059f60 con 0x7f6a400ffee0 2024-05-12T04:06:03.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:03 smithi033 ceph-mon[107305]: pgmap v98: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 260 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 319 B/s rd, 1 op/s; 36/276 objects degraded (13.043%) 2024-05-12T04:06:03.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:03 smithi033 ceph-mon[107305]: from='osd.2 [v2:172.21.15.32:6818/793261683,v1:172.21.15.32:6819/793261683]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-12T04:06:03.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:03 smithi033 ceph-mon[107305]: osdmap e62: 6 total, 5 up, 6 in 2024-05-12T04:06:03.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:03 smithi033 ceph-mon[107305]: from='osd.2 [v2:172.21.15.32:6818/793261683,v1:172.21.15.32:6819/793261683]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2024-05-12T04:06:03.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:03 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:06:03.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:03 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:06:03.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:03 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:06:03.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:03 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:06:03.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:03 smithi032 ceph-mon[120392]: pgmap v98: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 260 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 319 B/s rd, 1 op/s; 36/276 objects degraded (13.043%) 2024-05-12T04:06:03.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:03 smithi032 ceph-mon[120392]: from='osd.2 [v2:172.21.15.32:6818/793261683,v1:172.21.15.32:6819/793261683]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-12T04:06:03.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:03 smithi032 ceph-mon[120392]: osdmap e62: 6 total, 5 up, 6 in 2024-05-12T04:06:03.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:03 smithi032 ceph-mon[120392]: from='osd.2 [v2:172.21.15.32:6818/793261683,v1:172.21.15.32:6819/793261683]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2024-05-12T04:06:03.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:03 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:06:03.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:03 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:06:03.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:03 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:06:03.792 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:03 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:06:03.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.881+0000 7f6a48669700 1 -- 172.21.15.32:0/1454675245 --> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6a4010f8c0 con 0x7f6a2c06cc20 2024-05-12T04:06:03.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.882+0000 7f6a36ffd700 1 -- 172.21.15.32:0/1454675245 <== mgr.24593 v2:172.21.15.32:6800/2957111591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f6a4010f8c0 con 0x7f6a2c06cc20 2024-05-12T04:06:03.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.884+0000 7f6a48669700 1 -- 172.21.15.32:0/1454675245 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f6a2c06cc20 msgr2=0x7f6a2c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:06:03.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.884+0000 7f6a48669700 1 --2- 172.21.15.32:0/1454675245 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f6a2c06cc20 0x7f6a2c06f0d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c00b900 tx=0x7f6a3c009e30 comp rx=0 tx=0).stop 2024-05-12T04:06:03.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.884+0000 7f6a48669700 1 -- 172.21.15.32:0/1454675245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a400ffee0 msgr2=0x7f6a4010de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:06:03.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.884+0000 7f6a48669700 1 --2- 172.21.15.32:0/1454675245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a400ffee0 0x7f6a4010de60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6a30002a50 tx=0x7f6a300046b0 comp rx=0 tx=0).stop 2024-05-12T04:06:03.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.884+0000 7f6a48669700 1 -- 172.21.15.32:0/1454675245 shutdown_connections 2024-05-12T04:06:03.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.884+0000 7f6a48669700 1 --2- 172.21.15.32:0/1454675245 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f6a2c06cc20 0x7f6a2c06f0d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:03.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.884+0000 7f6a48669700 1 --2- 172.21.15.32:0/1454675245 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a400662c0 0x7f6a4010d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:03.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.884+0000 7f6a48669700 1 --2- 172.21.15.32:0/1454675245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a400ffee0 0x7f6a4010de60 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:03.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.884+0000 7f6a48669700 1 -- 172.21.15.32:0/1454675245 >> 172.21.15.32:0/1454675245 conn(0x7f6a400f7430 msgr2=0x7f6a400f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:06:03.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.884+0000 7f6a48669700 1 -- 172.21.15.32:0/1454675245 shutdown_connections 2024-05-12T04:06:03.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.884+0000 7f6a48669700 1 -- 172.21.15.32:0/1454675245 wait complete. 2024-05-12T04:06:03.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.996+0000 7f9a567de700 1 -- 172.21.15.32:0/2378659366 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a500fce60 msgr2=0x7f9a500fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:06:03.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.996+0000 7f9a567de700 1 --2- 172.21.15.32:0/2378659366 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a500fce60 0x7f9a500fd270 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9a40004940 tx=0x7f9a40009e30 comp rx=0 tx=0).stop 2024-05-12T04:06:03.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.996+0000 7f9a567de700 1 -- 172.21.15.32:0/2378659366 shutdown_connections 2024-05-12T04:06:03.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.996+0000 7f9a567de700 1 --2- 172.21.15.32:0/2378659366 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a500fdfd0 0x7f9a500fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:03.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.996+0000 7f9a567de700 1 --2- 172.21.15.32:0/2378659366 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a500fce60 0x7f9a500fd270 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:03.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.996+0000 7f9a567de700 1 -- 172.21.15.32:0/2378659366 >> 172.21.15.32:0/2378659366 conn(0x7f9a500f8500 msgr2=0x7f9a500fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:06:03.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.996+0000 7f9a567de700 1 -- 172.21.15.32:0/2378659366 shutdown_connections 2024-05-12T04:06:03.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.996+0000 7f9a567de700 1 -- 172.21.15.32:0/2378659366 wait complete. 2024-05-12T04:06:03.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.996+0000 7f9a567de700 1 Processor -- start 2024-05-12T04:06:03.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.996+0000 7f9a567de700 1 -- start start 2024-05-12T04:06:03.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.997+0000 7f9a567de700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a500fce60 0x7f9a5010d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:06:03.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.997+0000 7f9a567de700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a500fdfd0 0x7f9a5010de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:06:03.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.997+0000 7f9a567de700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5010e470 con 0x7f9a500fce60 2024-05-12T04:06:03.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.997+0000 7f9a567de700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5010ecc0 con 0x7f9a500fdfd0 2024-05-12T04:06:03.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.997+0000 7f9a4ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a500fce60 0x7f9a5010d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:06:03.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.997+0000 7f9a4ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a500fce60 0x7f9a5010d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55356/0 (socket says 172.21.15.32:55356) 2024-05-12T04:06:03.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.997+0000 7f9a4ffff700 1 -- 172.21.15.32:0/2808827673 learned_addr learned my addr 172.21.15.32:0/2808827673 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:06:03.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.997+0000 7f9a4f7fe700 1 --2- 172.21.15.32:0/2808827673 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a500fdfd0 0x7f9a5010de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:06:03.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.997+0000 7f9a4ffff700 1 -- 172.21.15.32:0/2808827673 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a500fdfd0 msgr2=0x7f9a5010de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:06:03.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.997+0000 7f9a4ffff700 1 --2- 172.21.15.32:0/2808827673 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a500fdfd0 0x7f9a5010de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:03.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.997+0000 7f9a4ffff700 1 -- 172.21.15.32:0/2808827673 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a40005040 con 0x7f9a500fce60 2024-05-12T04:06:04.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.997+0000 7f9a4ffff700 1 --2- 172.21.15.32:0/2808827673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a500fce60 0x7f9a5010d930 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9a40009380 tx=0x7f9a40009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:06:04.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.997+0000 7f9a4cff9700 1 -- 172.21.15.32:0/2808827673 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a4000e040 con 0x7f9a500fce60 2024-05-12T04:06:04.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.998+0000 7f9a4cff9700 1 -- 172.21.15.32:0/2808827673 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9a4000bd20 con 0x7f9a500fce60 2024-05-12T04:06:04.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.998+0000 7f9a567de700 1 -- 172.21.15.32:0/2808827673 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a5010ef40 con 0x7f9a500fce60 2024-05-12T04:06:04.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.998+0000 7f9a4cff9700 1 -- 172.21.15.32:0/2808827673 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a4001c630 con 0x7f9a500fce60 2024-05-12T04:06:04.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.998+0000 7f9a567de700 1 -- 172.21.15.32:0/2808827673 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a5010f480 con 0x7f9a500fce60 2024-05-12T04:06:04.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.999+0000 7f9a567de700 1 -- 172.21.15.32:0/2808827673 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a5005f5c0 con 0x7f9a500fce60 2024-05-12T04:06:04.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.999+0000 7f9a4cff9700 1 -- 172.21.15.32:0/2808827673 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f9a40013420 con 0x7f9a500fce60 2024-05-12T04:06:04.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:03.999+0000 7f9a4cff9700 1 --2- 172.21.15.32:0/2808827673 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f9a3806cc20 0x7f9a3806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:06:04.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.000+0000 7f9a4cff9700 1 -- 172.21.15.32:0/2808827673 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f9a4008cb30 con 0x7f9a500fce60 2024-05-12T04:06:04.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.000+0000 7f9a4f7fe700 1 --2- 172.21.15.32:0/2808827673 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f9a3806cc20 0x7f9a3806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:06:04.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.000+0000 7f9a4f7fe700 1 --2- 172.21.15.32:0/2808827673 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f9a3806cc20 0x7f9a3806f0d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9a44007fd0 tx=0x7f9a44009040 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:06:04.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.004+0000 7f9a4cff9700 1 -- 172.21.15.32:0/2808827673 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f9a4005a6f0 con 0x7f9a500fce60 2024-05-12T04:06:04.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.230+0000 7f9a567de700 1 -- 172.21.15.32:0/2808827673 --> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9a5010f750 con 0x7f9a3806cc20 2024-05-12T04:06:04.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.241+0000 7f9a4cff9700 1 -- 172.21.15.32:0/2808827673 <== mgr.24593 v2:172.21.15.32:6800/2957111591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f9a5010f750 con 0x7f9a3806cc20 2024-05-12T04:06:04.242 INFO:teuthology.orchestra.run.smithi032.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-12T04:06:04.242 INFO:teuthology.orchestra.run.smithi032.stdout:alertmanager.smithi032 smithi032 *:9093,9094 running (5m) 1s ago 14m 25.5M - 0.25.0 c8568f914cd2 a37fbab7af4c 2024-05-12T04:06:04.242 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi032 smithi032 running (107s) 1s ago 14m 6660k - 18.2.2-1272-g58249552 ddcb0a9d6fde 8c53f1fc7b06 2024-05-12T04:06:04.242 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi033 smithi033 running (101s) 97s ago 13m 6651k - 18.2.2-1272-g58249552 ddcb0a9d6fde cf4fe9ca070b 2024-05-12T04:06:04.242 INFO:teuthology.orchestra.run.smithi032.stdout:grafana.smithi032 smithi032 *:3000 running (4m) 1s ago 13m 89.7M - 9.4.7 954c08fa6188 83055e58d23c 2024-05-12T04:06:04.242 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.ialtqb smithi032 running (8m) 1s ago 8m 13.9M - 17.2.7-939-gc190bdee 284803f7d9cf 0683abf00e3e 2024-05-12T04:06:04.242 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.qpsawh smithi032 running (8m) 1s ago 8m 155M - 17.2.7-939-gc190bdee 284803f7d9cf 95a7db5c4fcd 2024-05-12T04:06:04.242 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi033.hglwml smithi033 running (8m) 97s ago 8m 13.2M - 17.2.7-939-gc190bdee 284803f7d9cf c97902b05025 2024-05-12T04:06:04.242 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi033.jgbjhk smithi033 running (8m) 97s ago 8m 102M - 17.2.7-939-gc190bdee 284803f7d9cf af97052cb97a 2024-05-12T04:06:04.242 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi032.ukatkq smithi032 *:8443,9283,8765 running (4m) 1s ago 15m 501M - 18.2.2-1272-g58249552 ddcb0a9d6fde 0d88ab9a5292 2024-05-12T04:06:04.243 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi033.pcvxwy smithi033 *:8443,9283,8765 running (3m) 97s ago 13m 448M - 18.2.2-1272-g58249552 ddcb0a9d6fde d42dcb6bd307 2024-05-12T04:06:04.243 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi032 smithi032 running (3m) 1s ago 15m 45.5M 2048M 18.2.2-1272-g58249552 ddcb0a9d6fde 81f94171c5dc 2024-05-12T04:06:04.243 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi033 smithi033 running (2m) 97s ago 12m 32.0M 2048M 18.2.2-1272-g58249552 ddcb0a9d6fde d648ba108f3e 2024-05-12T04:06:04.243 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi032 smithi032 *:9100 running (5m) 1s ago 14m 16.5M - 1.5.0 0da6a335fe13 cb75a7c0c829 2024-05-12T04:06:04.243 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi033 smithi033 *:9100 running (4m) 97s ago 13m 18.3M - 1.5.0 0da6a335fe13 7888c36c659f 2024-05-12T04:06:04.243 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0 smithi032 running (78s) 1s ago 12m 80.8M 1570M 18.2.2-1272-g58249552 ddcb0a9d6fde 07f66c4deb3b 2024-05-12T04:06:04.243 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1 smithi032 running (43s) 1s ago 11m 84.5M 1570M 18.2.2-1272-g58249552 ddcb0a9d6fde 67a87807d654 2024-05-12T04:06:04.243 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2 smithi032 running (7s) 1s ago 11m 11.9M 1570M 18.2.2-1272-g58249552 ddcb0a9d6fde 4ac48d74ee8b 2024-05-12T04:06:04.243 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3 smithi033 running (10m) 97s ago 10m 240M 2594M 17.2.7-939-gc190bdee 284803f7d9cf 3a92b5e3348e 2024-05-12T04:06:04.243 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4 smithi033 running (10m) 97s ago 10m 213M 2594M 17.2.7-939-gc190bdee 284803f7d9cf 0896268b4591 2024-05-12T04:06:04.243 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5 smithi033 running (9m) 97s ago 9m 196M 2594M 17.2.7-939-gc190bdee 284803f7d9cf 57e65a64c821 2024-05-12T04:06:04.243 INFO:teuthology.orchestra.run.smithi032.stdout:prometheus.smithi032 smithi032 *:9095 running (3m) 1s ago 13m 54.2M - 2.43.0 a07b618ecd1d 46eafb25e176 2024-05-12T04:06:04.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.243+0000 7f9a567de700 1 -- 172.21.15.32:0/2808827673 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f9a3806cc20 msgr2=0x7f9a3806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:06:04.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.243+0000 7f9a567de700 1 --2- 172.21.15.32:0/2808827673 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f9a3806cc20 0x7f9a3806f0d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9a44007fd0 tx=0x7f9a44009040 comp rx=0 tx=0).stop 2024-05-12T04:06:04.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.243+0000 7f9a567de700 1 -- 172.21.15.32:0/2808827673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a500fce60 msgr2=0x7f9a5010d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:06:04.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.243+0000 7f9a567de700 1 --2- 172.21.15.32:0/2808827673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a500fce60 0x7f9a5010d930 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9a40009380 tx=0x7f9a40009e30 comp rx=0 tx=0).stop 2024-05-12T04:06:04.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.243+0000 7f9a567de700 1 -- 172.21.15.32:0/2808827673 shutdown_connections 2024-05-12T04:06:04.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.243+0000 7f9a567de700 1 --2- 172.21.15.32:0/2808827673 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f9a3806cc20 0x7f9a3806f0d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:04.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.243+0000 7f9a567de700 1 --2- 172.21.15.32:0/2808827673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a500fce60 0x7f9a5010d930 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:04.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.243+0000 7f9a567de700 1 --2- 172.21.15.32:0/2808827673 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a500fdfd0 0x7f9a5010de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:04.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.243+0000 7f9a567de700 1 -- 172.21.15.32:0/2808827673 >> 172.21.15.32:0/2808827673 conn(0x7f9a500f8500 msgr2=0x7f9a500f9d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:06:04.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.244+0000 7f9a567de700 1 -- 172.21.15.32:0/2808827673 shutdown_connections 2024-05-12T04:06:04.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.244+0000 7f9a567de700 1 -- 172.21.15.32:0/2808827673 wait complete. 2024-05-12T04:06:04.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.348+0000 7f8149337700 1 -- 172.21.15.32:0/1937044983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81440fed70 msgr2=0x7f81440faeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:06:04.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.348+0000 7f8149337700 1 --2- 172.21.15.32:0/1937044983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81440fed70 0x7f81440faeb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f812c004940 tx=0x7f812c009e30 comp rx=0 tx=0).stop 2024-05-12T04:06:04.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.348+0000 7f8149337700 1 -- 172.21.15.32:0/1937044983 shutdown_connections 2024-05-12T04:06:04.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.348+0000 7f8149337700 1 --2- 172.21.15.32:0/1937044983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81440fed70 0x7f81440faeb0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:04.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.348+0000 7f8149337700 1 --2- 172.21.15.32:0/1937044983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81440fe420 0x7f81440fe830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:04.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.348+0000 7f8149337700 1 -- 172.21.15.32:0/1937044983 >> 172.21.15.32:0/1937044983 conn(0x7f81440f6be0 msgr2=0x7f81440f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:06:04.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.348+0000 7f8149337700 1 -- 172.21.15.32:0/1937044983 shutdown_connections 2024-05-12T04:06:04.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.348+0000 7f8149337700 1 -- 172.21.15.32:0/1937044983 wait complete. 2024-05-12T04:06:04.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.349+0000 7f8149337700 1 Processor -- start 2024-05-12T04:06:04.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.349+0000 7f8149337700 1 -- start start 2024-05-12T04:06:04.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.349+0000 7f8149337700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81440fe420 0x7f814410d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:06:04.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.349+0000 7f8149337700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81440fed70 0x7f814410dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:06:04.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.349+0000 7f8149337700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f814410e320 con 0x7f81440fed70 2024-05-12T04:06:04.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.349+0000 7f8149337700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f814410e490 con 0x7f81440fe420 2024-05-12T04:06:04.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.349+0000 7f81427fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81440fed70 0x7f814410dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:06:04.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.349+0000 7f81427fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81440fed70 0x7f814410dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55372/0 (socket says 172.21.15.32:55372) 2024-05-12T04:06:04.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.349+0000 7f81427fc700 1 -- 172.21.15.32:0/470667634 learned_addr learned my addr 172.21.15.32:0/470667634 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:06:04.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.349+0000 7f8142ffd700 1 --2- 172.21.15.32:0/470667634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81440fe420 0x7f814410d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:06:04.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.349+0000 7f81427fc700 1 -- 172.21.15.32:0/470667634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81440fe420 msgr2=0x7f814410d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:06:04.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.349+0000 7f81427fc700 1 --2- 172.21.15.32:0/470667634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81440fe420 0x7f814410d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:04.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.349+0000 7f81427fc700 1 -- 172.21.15.32:0/470667634 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f812c005040 con 0x7f81440fed70 2024-05-12T04:06:04.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.350+0000 7f81427fc700 1 --2- 172.21.15.32:0/470667634 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81440fed70 0x7f814410dd20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f812c0137b0 tx=0x7f812c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:06:04.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.350+0000 7f813b7fe700 1 -- 172.21.15.32:0/470667634 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f812c00e070 con 0x7f81440fed70 2024-05-12T04:06:04.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.350+0000 7f813b7fe700 1 -- 172.21.15.32:0/470667634 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f812c01c400 con 0x7f81440fed70 2024-05-12T04:06:04.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.350+0000 7f813b7fe700 1 -- 172.21.15.32:0/470667634 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f812c00b650 con 0x7f81440fed70 2024-05-12T04:06:04.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.350+0000 7f8149337700 1 -- 172.21.15.32:0/470667634 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8144112e40 con 0x7f81440fed70 2024-05-12T04:06:04.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.350+0000 7f8149337700 1 -- 172.21.15.32:0/470667634 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8144113320 con 0x7f81440fed70 2024-05-12T04:06:04.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.351+0000 7f8149337700 1 -- 172.21.15.32:0/470667634 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f814404c710 con 0x7f81440fed70 2024-05-12T04:06:04.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.351+0000 7f813b7fe700 1 -- 172.21.15.32:0/470667634 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f812c01c9e0 con 0x7f81440fed70 2024-05-12T04:06:04.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.352+0000 7f813b7fe700 1 --2- 172.21.15.32:0/470667634 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f813006cc20 0x7f813006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:06:04.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.352+0000 7f813b7fe700 1 -- 172.21.15.32:0/470667634 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f812c08d610 con 0x7f81440fed70 2024-05-12T04:06:04.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.352+0000 7f8142ffd700 1 --2- 172.21.15.32:0/470667634 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f813006cc20 0x7f813006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:06:04.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.352+0000 7f8142ffd700 1 --2- 172.21.15.32:0/470667634 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f813006cc20 0x7f813006f0d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8134009b80 tx=0x7f813400b040 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:06:04.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.356+0000 7f813b7fe700 1 -- 172.21.15.32:0/470667634 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f812c05b130 con 0x7f81440fed70 2024-05-12T04:06:04.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:04 smithi033 ceph-mon[107305]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-12T04:06:04.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:04 smithi033 ceph-mon[107305]: osd.2 [v2:172.21.15.32:6818/793261683,v1:172.21.15.32:6819/793261683] boot 2024-05-12T04:06:04.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:04 smithi033 ceph-mon[107305]: osdmap e63: 6 total, 6 up, 6 in 2024-05-12T04:06:04.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:04 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-12T04:06:04.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:04 smithi033 ceph-mon[107305]: osdmap e64: 6 total, 6 up, 6 in 2024-05-12T04:06:04.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.665+0000 7f8149337700 1 -- 172.21.15.32:0/470667634 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8144113820 con 0x7f81440fed70 2024-05-12T04:06:04.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.665+0000 7f813b7fe700 1 -- 172.21.15.32:0/470667634 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+847 (secure 0 0 0) 0x7f812c004490 con 0x7f81440fed70 2024-05-12T04:06:04.666 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-05-12T04:06:04.666 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": { 2024-05-12T04:06:04.666 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 2 2024-05-12T04:06:04.666 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:06:04.667 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": { 2024-05-12T04:06:04.667 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 2 2024-05-12T04:06:04.667 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:06:04.667 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": { 2024-05-12T04:06:04.667 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 3, 2024-05-12T04:06:04.667 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 3 2024-05-12T04:06:04.668 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:06:04.668 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": { 2024-05-12T04:06:04.668 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 4 2024-05-12T04:06:04.668 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:06:04.668 INFO:teuthology.orchestra.run.smithi032.stdout: "overall": { 2024-05-12T04:06:04.668 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 7, 2024-05-12T04:06:04.669 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 7 2024-05-12T04:06:04.669 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-05-12T04:06:04.669 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-05-12T04:06:04.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.667+0000 7f8149337700 1 -- 172.21.15.32:0/470667634 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f813006cc20 msgr2=0x7f813006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:06:04.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.667+0000 7f8149337700 1 --2- 172.21.15.32:0/470667634 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f813006cc20 0x7f813006f0d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8134009b80 tx=0x7f813400b040 comp rx=0 tx=0).stop 2024-05-12T04:06:04.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.667+0000 7f8149337700 1 -- 172.21.15.32:0/470667634 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81440fed70 msgr2=0x7f814410dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:06:04.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.667+0000 7f8149337700 1 --2- 172.21.15.32:0/470667634 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81440fed70 0x7f814410dd20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f812c0137b0 tx=0x7f812c009e30 comp rx=0 tx=0).stop 2024-05-12T04:06:04.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.667+0000 7f8149337700 1 -- 172.21.15.32:0/470667634 shutdown_connections 2024-05-12T04:06:04.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.667+0000 7f8149337700 1 --2- 172.21.15.32:0/470667634 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f813006cc20 0x7f813006f0d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:04.671 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.667+0000 7f8149337700 1 --2- 172.21.15.32:0/470667634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81440fe420 0x7f814410d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:04.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.667+0000 7f8149337700 1 --2- 172.21.15.32:0/470667634 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81440fed70 0x7f814410dd20 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:04.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.667+0000 7f8149337700 1 -- 172.21.15.32:0/470667634 >> 172.21.15.32:0/470667634 conn(0x7f81440f6be0 msgr2=0x7f81440f82b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:06:04.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.667+0000 7f8149337700 1 -- 172.21.15.32:0/470667634 shutdown_connections 2024-05-12T04:06:04.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.667+0000 7f8149337700 1 -- 172.21.15.32:0/470667634 wait complete. 2024-05-12T04:06:04.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.771+0000 7f02fc66a700 1 -- 172.21.15.32:0/3161316443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02f40ffee0 msgr2=0x7f02f4102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:06:04.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.771+0000 7f02fc66a700 1 --2- 172.21.15.32:0/3161316443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02f40ffee0 0x7f02f4102300 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f02e4004940 tx=0x7f02e4009e30 comp rx=0 tx=0).stop 2024-05-12T04:06:04.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.772+0000 7f02fc66a700 1 -- 172.21.15.32:0/3161316443 shutdown_connections 2024-05-12T04:06:04.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.772+0000 7f02fc66a700 1 --2- 172.21.15.32:0/3161316443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02f40ffee0 0x7f02f4102300 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:04.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.772+0000 7f02fc66a700 1 --2- 172.21.15.32:0/3161316443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02f40662c0 0x7f02f40ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:04.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.772+0000 7f02fc66a700 1 -- 172.21.15.32:0/3161316443 >> 172.21.15.32:0/3161316443 conn(0x7f02f40f7430 msgr2=0x7f02f40f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:06:04.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.772+0000 7f02fc66a700 1 -- 172.21.15.32:0/3161316443 shutdown_connections 2024-05-12T04:06:04.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.772+0000 7f02fc66a700 1 -- 172.21.15.32:0/3161316443 wait complete. 2024-05-12T04:06:04.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.772+0000 7f02fc66a700 1 Processor -- start 2024-05-12T04:06:04.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.772+0000 7f02fc66a700 1 -- start start 2024-05-12T04:06:04.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.772+0000 7f02fc66a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02f40662c0 0x7f02f410d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:06:04.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.772+0000 7f02fc66a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02f40ffee0 0x7f02f410dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:06:04.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.772+0000 7f02fc66a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02f410e550 con 0x7f02f40ffee0 2024-05-12T04:06:04.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.772+0000 7f02fc66a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02f410ed40 con 0x7f02f40662c0 2024-05-12T04:06:04.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.773+0000 7f02f9c05700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02f40ffee0 0x7f02f410dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:06:04.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.773+0000 7f02f9c05700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02f40ffee0 0x7f02f410dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55396/0 (socket says 172.21.15.32:55396) 2024-05-12T04:06:04.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.773+0000 7f02f9c05700 1 -- 172.21.15.32:0/2066439234 learned_addr learned my addr 172.21.15.32:0/2066439234 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:06:04.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.773+0000 7f02fa406700 1 --2- 172.21.15.32:0/2066439234 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02f40662c0 0x7f02f410d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:06:04.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.773+0000 7f02f9c05700 1 -- 172.21.15.32:0/2066439234 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02f40662c0 msgr2=0x7f02f410d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:06:04.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.773+0000 7f02f9c05700 1 --2- 172.21.15.32:0/2066439234 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02f40662c0 0x7f02f410d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:04.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.773+0000 7f02f9c05700 1 -- 172.21.15.32:0/2066439234 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02e4005040 con 0x7f02f40ffee0 2024-05-12T04:06:04.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.773+0000 7f02f9c05700 1 --2- 172.21.15.32:0/2066439234 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02f40ffee0 0x7f02f410dec0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f02e400b3b0 tx=0x7f02e4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:06:04.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.773+0000 7f02eaffd700 1 -- 172.21.15.32:0/2066439234 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02e400e040 con 0x7f02f40ffee0 2024-05-12T04:06:04.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.773+0000 7f02fc66a700 1 -- 172.21.15.32:0/2066439234 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02f410efc0 con 0x7f02f40ffee0 2024-05-12T04:06:04.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.773+0000 7f02eaffd700 1 -- 172.21.15.32:0/2066439234 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f02e4004290 con 0x7f02f40ffee0 2024-05-12T04:06:04.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.773+0000 7f02eaffd700 1 -- 172.21.15.32:0/2066439234 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02e401c3f0 con 0x7f02f40ffee0 2024-05-12T04:06:04.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.773+0000 7f02fc66a700 1 -- 172.21.15.32:0/2066439234 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02f410f580 con 0x7f02f40ffee0 2024-05-12T04:06:04.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.774+0000 7f02fc66a700 1 -- 172.21.15.32:0/2066439234 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02f405f5c0 con 0x7f02f40ffee0 2024-05-12T04:06:04.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.775+0000 7f02eaffd700 1 -- 172.21.15.32:0/2066439234 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f02e4013420 con 0x7f02f40ffee0 2024-05-12T04:06:04.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.776+0000 7f02eaffd700 1 --2- 172.21.15.32:0/2066439234 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f02e006cc30 0x7f02e006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:06:04.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.776+0000 7f02eaffd700 1 -- 172.21.15.32:0/2066439234 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f02e408de60 con 0x7f02f40ffee0 2024-05-12T04:06:04.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.776+0000 7f02fa406700 1 --2- 172.21.15.32:0/2066439234 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f02e006cc30 0x7f02e006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:06:04.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.776+0000 7f02fa406700 1 --2- 172.21.15.32:0/2066439234 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f02e006cc30 0x7f02e006f0e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f02f4066000 tx=0x7f02f00091a0 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:06:04.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:04.779+0000 7f02eaffd700 1 -- 172.21.15.32:0/2066439234 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f02e405b960 con 0x7f02f40ffee0 2024-05-12T04:06:04.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:04 smithi032 ceph-mon[120392]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-12T04:06:04.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:04 smithi032 ceph-mon[120392]: osd.2 [v2:172.21.15.32:6818/793261683,v1:172.21.15.32:6819/793261683] boot 2024-05-12T04:06:04.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:04 smithi032 ceph-mon[120392]: osdmap e63: 6 total, 6 up, 6 in 2024-05-12T04:06:04.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:04 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-12T04:06:04.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:04 smithi032 ceph-mon[120392]: osdmap e64: 6 total, 6 up, 6 in 2024-05-12T04:06:05.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.050+0000 7f02fc66a700 1 -- 172.21.15.32:0/2066439234 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f02f41057f0 con 0x7f02f40ffee0 2024-05-12T04:06:05.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.050+0000 7f02eaffd700 1 -- 172.21.15.32:0/2066439234 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1866 (secure 0 0 0) 0x7f02e405b4f0 con 0x7f02f40ffee0 2024-05-12T04:06:05.051 INFO:teuthology.orchestra.run.smithi032.stdout:e18 2024-05-12T04:06:05.051 INFO:teuthology.orchestra.run.smithi032.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-12T04:06:05.051 INFO:teuthology.orchestra.run.smithi032.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-12T04:06:05.051 INFO:teuthology.orchestra.run.smithi032.stdout:legacy client fscid: 1 2024-05-12T04:06:05.051 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:06:05.051 INFO:teuthology.orchestra.run.smithi032.stdout:Filesystem 'cephfs' (1) 2024-05-12T04:06:05.051 INFO:teuthology.orchestra.run.smithi032.stdout:fs_name cephfs 2024-05-12T04:06:05.051 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 18 2024-05-12T04:06:05.051 INFO:teuthology.orchestra.run.smithi032.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-12T04:06:05.051 INFO:teuthology.orchestra.run.smithi032.stdout:created 2024-05-12T03:57:48.904614+0000 2024-05-12T04:06:05.052 INFO:teuthology.orchestra.run.smithi032.stdout:modified 2024-05-12T03:58:06.821591+0000 2024-05-12T04:06:05.052 INFO:teuthology.orchestra.run.smithi032.stdout:tableserver 0 2024-05-12T04:06:05.052 INFO:teuthology.orchestra.run.smithi032.stdout:root 0 2024-05-12T04:06:05.052 INFO:teuthology.orchestra.run.smithi032.stdout:session_timeout 60 2024-05-12T04:06:05.052 INFO:teuthology.orchestra.run.smithi032.stdout:session_autoclose 300 2024-05-12T04:06:05.052 INFO:teuthology.orchestra.run.smithi032.stdout:max_file_size 1099511627776 2024-05-12T04:06:05.052 INFO:teuthology.orchestra.run.smithi032.stdout:max_xattr_size 65536 2024-05-12T04:06:05.052 INFO:teuthology.orchestra.run.smithi032.stdout:required_client_features {} 2024-05-12T04:06:05.052 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure 0 2024-05-12T04:06:05.052 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure_osd_epoch 46 2024-05-12T04:06:05.052 INFO:teuthology.orchestra.run.smithi032.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-12T04:06:05.052 INFO:teuthology.orchestra.run.smithi032.stdout:max_mds 1 2024-05-12T04:06:05.052 INFO:teuthology.orchestra.run.smithi032.stdout:in 0 2024-05-12T04:06:05.052 INFO:teuthology.orchestra.run.smithi032.stdout:up {0=14474} 2024-05-12T04:06:05.052 INFO:teuthology.orchestra.run.smithi032.stdout:failed 2024-05-12T04:06:05.052 INFO:teuthology.orchestra.run.smithi032.stdout:damaged 2024-05-12T04:06:05.053 INFO:teuthology.orchestra.run.smithi032.stdout:stopped 2024-05-12T04:06:05.053 INFO:teuthology.orchestra.run.smithi032.stdout:data_pools [3] 2024-05-12T04:06:05.053 INFO:teuthology.orchestra.run.smithi032.stdout:metadata_pool 2 2024-05-12T04:06:05.053 INFO:teuthology.orchestra.run.smithi032.stdout:inline_data enabled 2024-05-12T04:06:05.053 INFO:teuthology.orchestra.run.smithi032.stdout:balancer 2024-05-12T04:06:05.053 INFO:teuthology.orchestra.run.smithi032.stdout:bal_rank_mask -1 2024-05-12T04:06:05.053 INFO:teuthology.orchestra.run.smithi032.stdout:standby_count_wanted 1 2024-05-12T04:06:05.053 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.qpsawh{0:14474} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.32:6828/3100155296,v1:172.21.15.32:6829/3100155296] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:06:05.053 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi033.jgbjhk{0:24267} state up:standby-replay seq 3 join_fscid=1 addr [v2:172.21.15.33:6824/55417471,v1:172.21.15.33:6825/55417471] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:06:05.053 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:06:05.053 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:06:05.053 INFO:teuthology.orchestra.run.smithi032.stdout:Standby daemons: 2024-05-12T04:06:05.053 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:06:05.053 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi033.hglwml{-1:24283} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/433858238,v1:172.21.15.33:6827/433858238] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:06:05.053 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.ialtqb{-1:24287} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.32:6826/3703132329,v1:172.21.15.32:6827/3703132329] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:06:05.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.052+0000 7f02fc66a700 1 -- 172.21.15.32:0/2066439234 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f02e006cc30 msgr2=0x7f02e006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:06:05.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.052+0000 7f02fc66a700 1 --2- 172.21.15.32:0/2066439234 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f02e006cc30 0x7f02e006f0e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f02f4066000 tx=0x7f02f00091a0 comp rx=0 tx=0).stop 2024-05-12T04:06:05.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.052+0000 7f02fc66a700 1 -- 172.21.15.32:0/2066439234 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02f40ffee0 msgr2=0x7f02f410dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:06:05.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.052+0000 7f02fc66a700 1 --2- 172.21.15.32:0/2066439234 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02f40ffee0 0x7f02f410dec0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f02e400b3b0 tx=0x7f02e4009e30 comp rx=0 tx=0).stop 2024-05-12T04:06:05.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.052+0000 7f02fc66a700 1 -- 172.21.15.32:0/2066439234 shutdown_connections 2024-05-12T04:06:05.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.052+0000 7f02fc66a700 1 --2- 172.21.15.32:0/2066439234 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f02e006cc30 0x7f02e006f0e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:05.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.052+0000 7f02fc66a700 1 --2- 172.21.15.32:0/2066439234 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02f40662c0 0x7f02f410d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:05.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.052+0000 7f02fc66a700 1 --2- 172.21.15.32:0/2066439234 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02f40ffee0 0x7f02f410dec0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:05.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.052+0000 7f02fc66a700 1 -- 172.21.15.32:0/2066439234 >> 172.21.15.32:0/2066439234 conn(0x7f02f40f7430 msgr2=0x7f02f40f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:06:05.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.052+0000 7f02fc66a700 1 -- 172.21.15.32:0/2066439234 shutdown_connections 2024-05-12T04:06:05.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.052+0000 7f02fc66a700 1 -- 172.21.15.32:0/2066439234 wait complete. 2024-05-12T04:06:05.055 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 18 2024-05-12T04:06:05.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.158+0000 7f89af334700 1 -- 172.21.15.32:0/544668368 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89a80662c0 msgr2=0x7f89a80ff9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:06:05.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.158+0000 7f89af334700 1 --2- 172.21.15.32:0/544668368 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89a80662c0 0x7f89a80ff9a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f89a4004940 tx=0x7f89a4009e30 comp rx=0 tx=0).stop 2024-05-12T04:06:05.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.158+0000 7f89af334700 1 -- 172.21.15.32:0/544668368 shutdown_connections 2024-05-12T04:06:05.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.158+0000 7f89af334700 1 --2- 172.21.15.32:0/544668368 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89a80ffee0 0x7f89a8102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:05.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.158+0000 7f89af334700 1 --2- 172.21.15.32:0/544668368 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89a80662c0 0x7f89a80ff9a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:05.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.158+0000 7f89af334700 1 -- 172.21.15.32:0/544668368 >> 172.21.15.32:0/544668368 conn(0x7f89a80f7430 msgr2=0x7f89a80f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:06:05.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.158+0000 7f89af334700 1 -- 172.21.15.32:0/544668368 shutdown_connections 2024-05-12T04:06:05.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.158+0000 7f89af334700 1 -- 172.21.15.32:0/544668368 wait complete. 2024-05-12T04:06:05.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.159+0000 7f89af334700 1 Processor -- start 2024-05-12T04:06:05.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.159+0000 7f89af334700 1 -- start start 2024-05-12T04:06:05.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.159+0000 7f89af334700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89a80662c0 0x7f89a810d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:06:05.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.159+0000 7f89af334700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89a80ffee0 0x7f89a810de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:06:05.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.159+0000 7f89af334700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a810e460 con 0x7f89a80ffee0 2024-05-12T04:06:05.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.159+0000 7f89af334700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a810ecb0 con 0x7f89a80662c0 2024-05-12T04:06:05.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.159+0000 7f89ac8cf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89a80ffee0 0x7f89a810de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:06:05.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.159+0000 7f89ac8cf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89a80ffee0 0x7f89a810de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55402/0 (socket says 172.21.15.32:55402) 2024-05-12T04:06:05.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.159+0000 7f89ac8cf700 1 -- 172.21.15.32:0/2535903928 learned_addr learned my addr 172.21.15.32:0/2535903928 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:06:05.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.159+0000 7f89ad0d0700 1 --2- 172.21.15.32:0/2535903928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89a80662c0 0x7f89a810d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:06:05.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.159+0000 7f89ac8cf700 1 -- 172.21.15.32:0/2535903928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89a80662c0 msgr2=0x7f89a810d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:06:05.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.159+0000 7f89ac8cf700 1 --2- 172.21.15.32:0/2535903928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89a80662c0 0x7f89a810d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:05.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.159+0000 7f89ac8cf700 1 -- 172.21.15.32:0/2535903928 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89a4005040 con 0x7f89a80ffee0 2024-05-12T04:06:05.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.159+0000 7f89ac8cf700 1 --2- 172.21.15.32:0/2535903928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89a80ffee0 0x7f89a810de60 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f899c002a50 tx=0x7f899c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:06:05.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.160+0000 7f8999ffb700 1 -- 172.21.15.32:0/2535903928 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f899c007cc0 con 0x7f89a80ffee0 2024-05-12T04:06:05.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.160+0000 7f8999ffb700 1 -- 172.21.15.32:0/2535903928 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f899c00f040 con 0x7f89a80ffee0 2024-05-12T04:06:05.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.160+0000 7f89af334700 1 -- 172.21.15.32:0/2535903928 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89a810ef90 con 0x7f89a80ffee0 2024-05-12T04:06:05.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.160+0000 7f8999ffb700 1 -- 172.21.15.32:0/2535903928 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f899c00d9c0 con 0x7f89a80ffee0 2024-05-12T04:06:05.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.160+0000 7f89af334700 1 -- 172.21.15.32:0/2535903928 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89a810f4d0 con 0x7f89a80ffee0 2024-05-12T04:06:05.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.161+0000 7f89af334700 1 -- 172.21.15.32:0/2535903928 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89a805f5c0 con 0x7f89a80ffee0 2024-05-12T04:06:05.162 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.161+0000 7f8999ffb700 1 -- 172.21.15.32:0/2535903928 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f899c00db20 con 0x7f89a80ffee0 2024-05-12T04:06:05.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.162+0000 7f8999ffb700 1 --2- 172.21.15.32:0/2535903928 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f899406cc20 0x7f899406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:06:05.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.162+0000 7f8999ffb700 1 -- 172.21.15.32:0/2535903928 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f899c08bc60 con 0x7f89a80ffee0 2024-05-12T04:06:05.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.162+0000 7f89ad0d0700 1 --2- 172.21.15.32:0/2535903928 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f899406cc20 0x7f899406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:06:05.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.162+0000 7f89ad0d0700 1 --2- 172.21.15.32:0/2535903928 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f899406cc20 0x7f899406f0d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f89a400b900 tx=0x7f89a4009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:06:05.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.166+0000 7f8999ffb700 1 -- 172.21.15.32:0/2535903928 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f899c059780 con 0x7f89a80ffee0 2024-05-12T04:06:05.397 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.396+0000 7f89af334700 1 -- 172.21.15.32:0/2535903928 --> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f89a810f850 con 0x7f899406cc20 2024-05-12T04:06:05.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.398+0000 7f8999ffb700 1 -- 172.21.15.32:0/2535903928 <== mgr.24593 v2:172.21.15.32:6800/2957111591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f89a810f850 con 0x7f899406cc20 2024-05-12T04:06:05.398 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-05-12T04:06:05.398 INFO:teuthology.orchestra.run.smithi032.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:19c9734256aa43a60e203feb242f50bf8b7f58666d520536af43cc4fee96173a", 2024-05-12T04:06:05.398 INFO:teuthology.orchestra.run.smithi032.stdout: "in_progress": true, 2024-05-12T04:06:05.398 INFO:teuthology.orchestra.run.smithi032.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-12T04:06:05.398 INFO:teuthology.orchestra.run.smithi032.stdout: "services_complete": [ 2024-05-12T04:06:05.398 INFO:teuthology.orchestra.run.smithi032.stdout: "mon", 2024-05-12T04:06:05.398 INFO:teuthology.orchestra.run.smithi032.stdout: "crash", 2024-05-12T04:06:05.399 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr" 2024-05-12T04:06:05.399 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-05-12T04:06:05.399 INFO:teuthology.orchestra.run.smithi032.stdout: "progress": "9/21 daemons upgraded", 2024-05-12T04:06:05.399 INFO:teuthology.orchestra.run.smithi032.stdout: "message": "Currently upgrading osd daemons", 2024-05-12T04:06:05.399 INFO:teuthology.orchestra.run.smithi032.stdout: "is_paused": false 2024-05-12T04:06:05.399 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-05-12T04:06:05.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.400+0000 7f89af334700 1 -- 172.21.15.32:0/2535903928 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f899406cc20 msgr2=0x7f899406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:06:05.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.400+0000 7f89af334700 1 --2- 172.21.15.32:0/2535903928 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f899406cc20 0x7f899406f0d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f89a400b900 tx=0x7f89a4009e30 comp rx=0 tx=0).stop 2024-05-12T04:06:05.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.400+0000 7f89af334700 1 -- 172.21.15.32:0/2535903928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89a80ffee0 msgr2=0x7f89a810de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:06:05.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.400+0000 7f89af334700 1 --2- 172.21.15.32:0/2535903928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89a80ffee0 0x7f89a810de60 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f899c002a50 tx=0x7f899c0046b0 comp rx=0 tx=0).stop 2024-05-12T04:06:05.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.400+0000 7f89af334700 1 -- 172.21.15.32:0/2535903928 shutdown_connections 2024-05-12T04:06:05.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.400+0000 7f89af334700 1 --2- 172.21.15.32:0/2535903928 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f899406cc20 0x7f899406f0d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:05.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.400+0000 7f89af334700 1 --2- 172.21.15.32:0/2535903928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89a80662c0 0x7f89a810d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:05.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.400+0000 7f89af334700 1 --2- 172.21.15.32:0/2535903928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89a80ffee0 0x7f89a810de60 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:05.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.400+0000 7f89af334700 1 -- 172.21.15.32:0/2535903928 >> 172.21.15.32:0/2535903928 conn(0x7f89a80f7430 msgr2=0x7f89a80f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:06:05.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.400+0000 7f89af334700 1 -- 172.21.15.32:0/2535903928 shutdown_connections 2024-05-12T04:06:05.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.400+0000 7f89af334700 1 -- 172.21.15.32:0/2535903928 wait complete. 2024-05-12T04:06:05.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.505+0000 7f851689b700 1 -- 172.21.15.32:0/665604233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8510066240 msgr2=0x7f8510101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:06:05.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.505+0000 7f851689b700 1 --2- 172.21.15.32:0/665604233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8510066240 0x7f8510101e20 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8500004940 tx=0x7f8500009e30 comp rx=0 tx=0).stop 2024-05-12T04:06:05.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.505+0000 7f851689b700 1 -- 172.21.15.32:0/665604233 shutdown_connections 2024-05-12T04:06:05.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.505+0000 7f851689b700 1 --2- 172.21.15.32:0/665604233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8510102360 0x7f8510104740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:05.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.505+0000 7f851689b700 1 --2- 172.21.15.32:0/665604233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8510066240 0x7f8510101e20 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:05.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.505+0000 7f851689b700 1 -- 172.21.15.32:0/665604233 >> 172.21.15.32:0/665604233 conn(0x7f85100f7430 msgr2=0x7f85100f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:06:05.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.505+0000 7f851689b700 1 -- 172.21.15.32:0/665604233 shutdown_connections 2024-05-12T04:06:05.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.505+0000 7f851689b700 1 -- 172.21.15.32:0/665604233 wait complete. 2024-05-12T04:06:05.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.506+0000 7f851689b700 1 Processor -- start 2024-05-12T04:06:05.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.506+0000 7f851689b700 1 -- start start 2024-05-12T04:06:05.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.506+0000 7f851689b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8510066240 0x7f851010fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:06:05.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.506+0000 7f851689b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8510102360 0x7f85101100d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:06:05.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.506+0000 7f851689b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85101106d0 con 0x7f8510066240 2024-05-12T04:06:05.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.506+0000 7f851689b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8510110f20 con 0x7f8510102360 2024-05-12T04:06:05.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.507+0000 7f850ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8510066240 0x7f851010fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:06:05.508 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.507+0000 7f850ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8510066240 0x7f851010fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55426/0 (socket says 172.21.15.32:55426) 2024-05-12T04:06:05.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.507+0000 7f850ffff700 1 -- 172.21.15.32:0/1212933535 learned_addr learned my addr 172.21.15.32:0/1212933535 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:06:05.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.507+0000 7f850f7fe700 1 --2- 172.21.15.32:0/1212933535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8510102360 0x7f85101100d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:06:05.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.507+0000 7f850ffff700 1 -- 172.21.15.32:0/1212933535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8510102360 msgr2=0x7f85101100d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:06:05.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.507+0000 7f850ffff700 1 --2- 172.21.15.32:0/1212933535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8510102360 0x7f85101100d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:05.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.507+0000 7f850ffff700 1 -- 172.21.15.32:0/1212933535 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8500005040 con 0x7f8510066240 2024-05-12T04:06:05.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.507+0000 7f850ffff700 1 --2- 172.21.15.32:0/1212933535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8510066240 0x7f851010fb90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f850000b8a0 tx=0x7f8500009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:06:05.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.507+0000 7f850cff9700 1 -- 172.21.15.32:0/1212933535 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f850000e040 con 0x7f8510066240 2024-05-12T04:06:05.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.507+0000 7f851689b700 1 -- 172.21.15.32:0/1212933535 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85101111a0 con 0x7f8510066240 2024-05-12T04:06:05.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.507+0000 7f850cff9700 1 -- 172.21.15.32:0/1212933535 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8500009af0 con 0x7f8510066240 2024-05-12T04:06:05.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.507+0000 7f850cff9700 1 -- 172.21.15.32:0/1212933535 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f850001c3f0 con 0x7f8510066240 2024-05-12T04:06:05.509 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.507+0000 7f851689b700 1 -- 172.21.15.32:0/1212933535 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85101116e0 con 0x7f8510066240 2024-05-12T04:06:05.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.508+0000 7f851689b700 1 -- 172.21.15.32:0/1212933535 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f851005f5c0 con 0x7f8510066240 2024-05-12T04:06:05.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.509+0000 7f850cff9700 1 -- 172.21.15.32:0/1212933535 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f8500013690 con 0x7f8510066240 2024-05-12T04:06:05.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.509+0000 7f850cff9700 1 --2- 172.21.15.32:0/1212933535 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f84f806cc20 0x7f84f806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:06:05.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.509+0000 7f850cff9700 1 -- 172.21.15.32:0/1212933535 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f850008d370 con 0x7f8510066240 2024-05-12T04:06:05.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.509+0000 7f850f7fe700 1 --2- 172.21.15.32:0/1212933535 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f84f806cc20 0x7f84f806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:06:05.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.510+0000 7f850f7fe700 1 --2- 172.21.15.32:0/1212933535 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f84f806cc20 0x7f84f806f0d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8504004f40 tx=0x7f850400b040 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:06:05.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.514+0000 7f850cff9700 1 -- 172.21.15.32:0/1212933535 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f850005af40 con 0x7f8510066240 2024-05-12T04:06:05.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:05 smithi032 ceph-mon[120392]: from='client.34280 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:06:05.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:05 smithi032 ceph-mon[120392]: pgmap v101: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 260 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 36/276 objects degraded (13.043%) 2024-05-12T04:06:05.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:05 smithi032 ceph-mon[120392]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:06:05.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:05 smithi032 ceph-mon[120392]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:06:05.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:05 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/470667634' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:06:05.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:05 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/2066439234' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-12T04:06:05.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:05 smithi032 ceph-mon[120392]: Health check update: Degraded data redundancy: 36/276 objects degraded (13.043%), 12 pgs degraded (PG_DEGRADED) 2024-05-12T04:06:05.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:05 smithi033 ceph-mon[107305]: from='client.34280 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:06:05.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:05 smithi033 ceph-mon[107305]: pgmap v101: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 260 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 36/276 objects degraded (13.043%) 2024-05-12T04:06:05.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:05 smithi033 ceph-mon[107305]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:06:05.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:05 smithi033 ceph-mon[107305]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:06:05.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:05 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/470667634' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:06:05.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:05 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/2066439234' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-12T04:06:05.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:05 smithi033 ceph-mon[107305]: Health check update: Degraded data redundancy: 36/276 objects degraded (13.043%), 12 pgs degraded (PG_DEGRADED) 2024-05-12T04:06:05.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.804+0000 7f851689b700 1 -- 172.21.15.32:0/1212933535 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f851004bed0 con 0x7f8510066240 2024-05-12T04:06:05.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.804+0000 7f850cff9700 1 -- 172.21.15.32:0/1212933535 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1041 (secure 0 0 0) 0x7f8500013420 con 0x7f8510066240 2024-05-12T04:06:05.805 INFO:teuthology.orchestra.run.smithi032.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; Degraded data redundancy: 36/276 objects degraded (13.043%), 12 pgs degraded 2024-05-12T04:06:05.805 INFO:teuthology.orchestra.run.smithi032.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-12T04:06:05.805 INFO:teuthology.orchestra.run.smithi032.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-12T04:06:05.805 INFO:teuthology.orchestra.run.smithi032.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 36/276 objects degraded (13.043%), 12 pgs degraded 2024-05-12T04:06:05.805 INFO:teuthology.orchestra.run.smithi032.stdout: pg 2.1 is active+undersized+degraded, acting [1,0] 2024-05-12T04:06:05.805 INFO:teuthology.orchestra.run.smithi032.stdout: pg 2.3 is active+undersized+degraded, acting [5,1] 2024-05-12T04:06:05.805 INFO:teuthology.orchestra.run.smithi032.stdout: pg 2.7 is active+undersized+degraded, acting [3,4] 2024-05-12T04:06:05.805 INFO:teuthology.orchestra.run.smithi032.stdout: pg 2.c is active+undersized+degraded, acting [3,0] 2024-05-12T04:06:05.806 INFO:teuthology.orchestra.run.smithi032.stdout: pg 2.d is active+undersized+degraded, acting [1,3] 2024-05-12T04:06:05.806 INFO:teuthology.orchestra.run.smithi032.stdout: pg 2.e is active+undersized+degraded, acting [0,3] 2024-05-12T04:06:05.806 INFO:teuthology.orchestra.run.smithi032.stdout: pg 2.10 is active+undersized+degraded, acting [1,0] 2024-05-12T04:06:05.806 INFO:teuthology.orchestra.run.smithi032.stdout: pg 2.13 is active+undersized+degraded, acting [0,4] 2024-05-12T04:06:05.806 INFO:teuthology.orchestra.run.smithi032.stdout: pg 2.16 is active+undersized+degraded, acting [5,3] 2024-05-12T04:06:05.806 INFO:teuthology.orchestra.run.smithi032.stdout: pg 2.19 is active+undersized+degraded, acting [0,4] 2024-05-12T04:06:05.806 INFO:teuthology.orchestra.run.smithi032.stdout: pg 2.1c is active+undersized+degraded, acting [4,5] 2024-05-12T04:06:05.806 INFO:teuthology.orchestra.run.smithi032.stdout: pg 2.1e is active+undersized+degraded, acting [0,5] 2024-05-12T04:06:05.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.806+0000 7f851689b700 1 -- 172.21.15.32:0/1212933535 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f84f806cc20 msgr2=0x7f84f806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:06:05.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.806+0000 7f851689b700 1 --2- 172.21.15.32:0/1212933535 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f84f806cc20 0x7f84f806f0d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8504004f40 tx=0x7f850400b040 comp rx=0 tx=0).stop 2024-05-12T04:06:05.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.806+0000 7f851689b700 1 -- 172.21.15.32:0/1212933535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8510066240 msgr2=0x7f851010fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:06:05.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.806+0000 7f851689b700 1 --2- 172.21.15.32:0/1212933535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8510066240 0x7f851010fb90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f850000b8a0 tx=0x7f8500009e30 comp rx=0 tx=0).stop 2024-05-12T04:06:05.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.806+0000 7f851689b700 1 -- 172.21.15.32:0/1212933535 shutdown_connections 2024-05-12T04:06:05.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.806+0000 7f851689b700 1 --2- 172.21.15.32:0/1212933535 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f84f806cc20 0x7f84f806f0d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:05.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.806+0000 7f851689b700 1 --2- 172.21.15.32:0/1212933535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8510066240 0x7f851010fb90 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:05.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.806+0000 7f851689b700 1 --2- 172.21.15.32:0/1212933535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8510102360 0x7f85101100d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:05.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.807+0000 7f851689b700 1 -- 172.21.15.32:0/1212933535 >> 172.21.15.32:0/1212933535 conn(0x7f85100f7430 msgr2=0x7f85100f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:06:05.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.807+0000 7f851689b700 1 -- 172.21.15.32:0/1212933535 shutdown_connections 2024-05-12T04:06:05.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:05.807+0000 7f851689b700 1 -- 172.21.15.32:0/1212933535 wait complete. 2024-05-12T04:06:06.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:06 smithi032 ceph-mon[120392]: from='client.34300 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:06:06.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:06 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/1212933535' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-12T04:06:06.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:06 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:06:06.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:06 smithi033 ceph-mon[107305]: from='client.34300 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:06:06.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:06 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/1212933535' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-12T04:06:06.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:06 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:06:07.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:07 smithi033 ceph-mon[107305]: pgmap v103: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 260 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 36/276 objects degraded (13.043%) 2024-05-12T04:06:07.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:07 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:06:07.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:07 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:06:07.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:07 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:06:07.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:07 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T04:06:07.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:07 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:06:07.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:07 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:06:07.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:07 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:06:07.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:07 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:06:07.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:07 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:06:07.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:07 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-05-12T04:06:07.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:07 smithi032 ceph-mon[120392]: pgmap v103: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 260 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 36/276 objects degraded (13.043%) 2024-05-12T04:06:07.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:07 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:06:07.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:07 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:06:07.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:07 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:06:07.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:07 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T04:06:07.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:07 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:06:07.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:07 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:06:07.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:07 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:06:07.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:07 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:06:07.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:07 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:06:07.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:07 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-05-12T04:06:08.659 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:08 smithi032 ceph-mon[120392]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-05-12T04:06:08.659 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:08 smithi032 ceph-mon[120392]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-05-12T04:06:08.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:08 smithi033 ceph-mon[107305]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-05-12T04:06:08.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:08 smithi033 ceph-mon[107305]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-05-12T04:06:09.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:09 smithi033 ceph-mon[107305]: pgmap v104: 65 pgs: 1 active+undersized+degraded, 2 active+undersized, 62 active+clean; 260 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 4/276 objects degraded (1.449%) 2024-05-12T04:06:09.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:09 smithi032 ceph-mon[120392]: pgmap v104: 65 pgs: 1 active+undersized+degraded, 2 active+undersized, 62 active+clean; 260 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 4/276 objects degraded (1.449%) 2024-05-12T04:06:10.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:10 smithi033 ceph-mon[107305]: Health check update: Degraded data redundancy: 4/276 objects degraded (1.449%), 1 pg degraded (PG_DEGRADED) 2024-05-12T04:06:10.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:10 smithi032 ceph-mon[120392]: Health check update: Degraded data redundancy: 4/276 objects degraded (1.449%), 1 pg degraded (PG_DEGRADED) 2024-05-12T04:06:11.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:11 smithi033 ceph-mon[107305]: pgmap v105: 65 pgs: 65 active+clean; 260 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 692 B/s rd, 1 op/s 2024-05-12T04:06:11.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:11 smithi033 ceph-mon[107305]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 4/276 objects degraded (1.449%), 1 pg degraded) 2024-05-12T04:06:11.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:11 smithi032 ceph-mon[120392]: pgmap v105: 65 pgs: 65 active+clean; 260 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 692 B/s rd, 1 op/s 2024-05-12T04:06:11.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:11 smithi032 ceph-mon[120392]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 4/276 objects degraded (1.449%), 1 pg degraded) 2024-05-12T04:06:13.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:13 smithi033 ceph-mon[107305]: pgmap v106: 65 pgs: 65 active+clean; 260 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:06:13.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:13 smithi032 ceph-mon[120392]: pgmap v106: 65 pgs: 65 active+clean; 260 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:06:15.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:15 smithi032 ceph-mon[120392]: pgmap v107: 65 pgs: 65 active+clean; 260 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-05-12T04:06:15.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:15 smithi033 ceph-mon[107305]: pgmap v107: 65 pgs: 65 active+clean; 260 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-05-12T04:06:17.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:16 smithi033 ceph-mon[107305]: pgmap v108: 65 pgs: 65 active+clean; 260 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 899 B/s rd, 1 op/s 2024-05-12T04:06:17.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:16 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:06:17.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:16 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:06:17.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:16 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:06:17.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:16 smithi032 ceph-mon[120392]: pgmap v108: 65 pgs: 65 active+clean; 260 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 899 B/s rd, 1 op/s 2024-05-12T04:06:17.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:16 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:06:17.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:16 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:06:17.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:16 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:06:19.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:18 smithi032 ceph-mon[120392]: pgmap v109: 65 pgs: 65 active+clean; 260 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:06:19.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:18 smithi033 ceph-mon[107305]: pgmap v109: 65 pgs: 65 active+clean; 260 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:06:21.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:20 smithi033 ceph-mon[107305]: pgmap v110: 65 pgs: 65 active+clean; 260 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:06:21.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:20 smithi032 ceph-mon[120392]: pgmap v110: 65 pgs: 65 active+clean; 260 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:06:22.885 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:22 smithi033 ceph-mon[107305]: pgmap v111: 65 pgs: 65 active+clean; 260 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:06:22.885 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:22 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-05-12T04:06:22.885 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:22 smithi033 ceph-mon[107305]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-05-12T04:06:22.885 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:22 smithi033 ceph-mon[107305]: Upgrade: osd.3 is safe to restart 2024-05-12T04:06:23.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:22 smithi032 ceph-mon[120392]: pgmap v111: 65 pgs: 65 active+clean; 260 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:06:23.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:22 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-05-12T04:06:23.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:22 smithi032 ceph-mon[120392]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-05-12T04:06:23.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:22 smithi032 ceph-mon[120392]: Upgrade: osd.3 is safe to restart 2024-05-12T04:06:24.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:24 smithi033 ceph-mon[107305]: Upgrade: Updating osd.3 2024-05-12T04:06:24.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:24 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:06:24.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:24 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-12T04:06:24.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:24 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:06:24.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:24 smithi033 ceph-mon[107305]: Deploying daemon osd.3 on smithi033 2024-05-12T04:06:24.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:24 smithi033 ceph-mon[107305]: pgmap v112: 65 pgs: 65 active+clean; 260 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:06:25.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:24 smithi032 ceph-mon[120392]: Upgrade: Updating osd.3 2024-05-12T04:06:25.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:24 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:06:25.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:24 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-12T04:06:25.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:24 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:06:25.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:24 smithi032 ceph-mon[120392]: Deploying daemon osd.3 on smithi033 2024-05-12T04:06:25.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:24 smithi032 ceph-mon[120392]: pgmap v112: 65 pgs: 65 active+clean; 260 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:06:27.115 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 12 04:06:26 smithi033 systemd[1]: Stopping Ceph osd.3 for a3e178f8-1012-11ef-bc98-c7b262605968... 2024-05-12T04:06:27.115 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 12 04:06:26 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-3[55966]: 2024-05-12T04:06:26.970+0000 7fb7ce51a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-12T04:06:27.115 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 12 04:06:26 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-3[55966]: 2024-05-12T04:06:26.970+0000 7fb7ce51a700 -1 osd.3 64 *** Got signal Terminated *** 2024-05-12T04:06:27.115 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 12 04:06:26 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-3[55966]: 2024-05-12T04:06:26.970+0000 7fb7ce51a700 -1 osd.3 64 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-12T04:06:27.115 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:27 smithi033 ceph-mon[107305]: pgmap v113: 65 pgs: 65 active+clean; 260 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:06:27.115 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:27 smithi033 ceph-mon[107305]: osd.3 marked itself down and dead 2024-05-12T04:06:27.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:27 smithi032 ceph-mon[120392]: pgmap v113: 65 pgs: 65 active+clean; 260 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:06:27.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:27 smithi032 ceph-mon[120392]: osd.3 marked itself down and dead 2024-05-12T04:06:28.141 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 12 04:06:27 smithi033 bash[119652]: ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-3 2024-05-12T04:06:28.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:28 smithi033 ceph-mon[107305]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-12T04:06:28.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:28 smithi033 ceph-mon[107305]: osdmap e65: 6 total, 5 up, 6 in 2024-05-12T04:06:28.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:28 smithi032 ceph-mon[120392]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-12T04:06:28.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:28 smithi032 ceph-mon[120392]: osdmap e65: 6 total, 5 up, 6 in 2024-05-12T04:06:29.355 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:29 smithi033 ceph-mon[107305]: pgmap v115: 65 pgs: 4 peering, 15 stale+active+clean, 46 active+clean; 260 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-05-12T04:06:29.355 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:29 smithi033 ceph-mon[107305]: osdmap e66: 6 total, 5 up, 6 in 2024-05-12T04:06:29.355 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:29 smithi033 ceph-mon[107305]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-05-12T04:06:29.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:29 smithi032 ceph-mon[120392]: pgmap v115: 65 pgs: 4 peering, 15 stale+active+clean, 46 active+clean; 260 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-05-12T04:06:29.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:29 smithi032 ceph-mon[120392]: osdmap e66: 6 total, 5 up, 6 in 2024-05-12T04:06:29.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:29 smithi032 ceph-mon[120392]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-05-12T04:06:29.912 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 12 04:06:29 smithi033 systemd[1]: ceph-a3e178f8-1012-11ef-bc98-c7b262605968@osd.3.service: Succeeded. 2024-05-12T04:06:29.912 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 12 04:06:29 smithi033 systemd[1]: Stopped Ceph osd.3 for a3e178f8-1012-11ef-bc98-c7b262605968. 2024-05-12T04:06:29.912 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 12 04:06:29 smithi033 systemd[1]: Starting Ceph osd.3 for a3e178f8-1012-11ef-bc98-c7b262605968... 2024-05-12T04:06:31.079 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 12 04:06:30 smithi033 bash[119926]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-12T04:06:31.079 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 12 04:06:30 smithi033 bash[119926]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-05-12T04:06:31.079 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 12 04:06:30 smithi033 bash[119926]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-05-12T04:06:31.079 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 12 04:06:30 smithi033 bash[119926]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-05-12T04:06:31.079 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 12 04:06:30 smithi033 bash[119926]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-05-12T04:06:31.079 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 12 04:06:30 smithi033 bash[119926]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-12T04:06:31.080 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 12 04:06:30 smithi033 bash[119926]: --> ceph-volume raw activate successful for osd ID: 3 2024-05-12T04:06:31.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:31 smithi033 ceph-mon[107305]: pgmap v117: 65 pgs: 13 peering, 13 stale+active+clean, 39 active+clean; 260 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s 2024-05-12T04:06:31.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:31 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:06:31.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:31 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:06:31.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:31 smithi032 ceph-mon[120392]: pgmap v117: 65 pgs: 13 peering, 13 stale+active+clean, 39 active+clean; 260 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s 2024-05-12T04:06:31.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:31 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:06:31.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:31 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:06:32.162 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 12 04:06:31 smithi033 ceph-osd[120247]: -- 172.21.15.33:0/3375734388 <== mon.1 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5651c09e51e0 con 0x5651c09b5000 2024-05-12T04:06:32.162 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 12 04:06:31 smithi033 bash[120209]: 11f097a455bb57c82385ee9bc48140e482d30e55f7cfa1b167b94978806012ea 2024-05-12T04:06:32.162 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 12 04:06:31 smithi033 systemd[1]: Started Ceph osd.3 for a3e178f8-1012-11ef-bc98-c7b262605968. 2024-05-12T04:06:32.661 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 12 04:06:32 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-3[120224]: 2024-05-12T04:06:32.374+0000 7fd2a73747c0 -1 Falling back to public interface 2024-05-12T04:06:33.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:32 smithi033 ceph-mon[107305]: pgmap v118: 65 pgs: 11 active+undersized+degraded, 13 peering, 14 active+undersized, 27 active+clean; 260 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 41/276 objects degraded (14.855%) 2024-05-12T04:06:33.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:32 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:06:33.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:32 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:06:33.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:32 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:06:33.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:32 smithi033 ceph-mon[107305]: Health check failed: Degraded data redundancy: 41/276 objects degraded (14.855%), 11 pgs degraded (PG_DEGRADED) 2024-05-12T04:06:33.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:32 smithi032 ceph-mon[120392]: pgmap v118: 65 pgs: 11 active+undersized+degraded, 13 peering, 14 active+undersized, 27 active+clean; 260 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 41/276 objects degraded (14.855%) 2024-05-12T04:06:33.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:32 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:06:33.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:32 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:06:33.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:32 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:06:33.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:32 smithi032 ceph-mon[120392]: Health check failed: Degraded data redundancy: 41/276 objects degraded (14.855%), 11 pgs degraded (PG_DEGRADED) 2024-05-12T04:06:34.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:34 smithi032 ceph-mon[120392]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-05-12T04:06:34.364 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:34 smithi033 ceph-mon[107305]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-05-12T04:06:35.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:35 smithi032 ceph-mon[120392]: pgmap v119: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 260 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 60/276 objects degraded (21.739%) 2024-05-12T04:06:35.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:35 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:06:35.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:35 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:06:35.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:35 smithi033 ceph-mon[107305]: pgmap v119: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 260 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 60/276 objects degraded (21.739%) 2024-05-12T04:06:35.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:35 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:06:35.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:35 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:06:35.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:35.928+0000 7fc5a9ba6700 1 -- 172.21.15.32:0/1492968718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5a40fe3e0 msgr2=0x7fc5a40fe7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:06:35.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:35.928+0000 7fc5a9ba6700 1 --2- 172.21.15.32:0/1492968718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5a40fe3e0 0x7fc5a40fe7f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc594004940 tx=0x7fc594009e30 comp rx=0 tx=0).stop 2024-05-12T04:06:35.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:35.928+0000 7fc5a9ba6700 1 -- 172.21.15.32:0/1492968718 shutdown_connections 2024-05-12T04:06:35.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:35.928+0000 7fc5a9ba6700 1 --2- 172.21.15.32:0/1492968718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5a40fed30 0x7fc5a40fae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:35.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:35.928+0000 7fc5a9ba6700 1 --2- 172.21.15.32:0/1492968718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5a40fe3e0 0x7fc5a40fe7f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:35.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:35.929+0000 7fc5a9ba6700 1 -- 172.21.15.32:0/1492968718 >> 172.21.15.32:0/1492968718 conn(0x7fc5a40f6b80 msgr2=0x7fc5a40f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:06:35.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:35.929+0000 7fc5a9ba6700 1 -- 172.21.15.32:0/1492968718 shutdown_connections 2024-05-12T04:06:35.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:35.929+0000 7fc5a9ba6700 1 -- 172.21.15.32:0/1492968718 wait complete. 2024-05-12T04:06:35.931 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:35.929+0000 7fc5a9ba6700 1 Processor -- start 2024-05-12T04:06:35.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:35.929+0000 7fc5a9ba6700 1 -- start start 2024-05-12T04:06:35.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:35.929+0000 7fc5a9ba6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5a40fe3e0 0x7fc5a410d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:06:35.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:35.930+0000 7fc5a9ba6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5a40fed30 0x7fc5a410ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:06:35.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:35.930+0000 7fc5a9ba6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a410e3c0 con 0x7fc5a40fe3e0 2024-05-12T04:06:35.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:35.930+0000 7fc5a9ba6700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a410ec10 con 0x7fc5a40fed30 2024-05-12T04:06:35.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:35.930+0000 7fc5a37fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5a40fe3e0 0x7fc5a410d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:06:35.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:35.930+0000 7fc5a37fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5a40fe3e0 0x7fc5a410d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35182/0 (socket says 172.21.15.32:35182) 2024-05-12T04:06:35.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:35.930+0000 7fc5a37fe700 1 -- 172.21.15.32:0/3463936636 learned_addr learned my addr 172.21.15.32:0/3463936636 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:06:35.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:35.930+0000 7fc5a2ffd700 1 --2- 172.21.15.32:0/3463936636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5a40fed30 0x7fc5a410ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:06:35.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:35.930+0000 7fc5a37fe700 1 -- 172.21.15.32:0/3463936636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5a40fed30 msgr2=0x7fc5a410ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:06:35.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:35.930+0000 7fc5a37fe700 1 --2- 172.21.15.32:0/3463936636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5a40fed30 0x7fc5a410ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:35.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:35.930+0000 7fc5a37fe700 1 -- 172.21.15.32:0/3463936636 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc594005040 con 0x7fc5a40fe3e0 2024-05-12T04:06:35.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:35.930+0000 7fc5a2ffd700 1 --2- 172.21.15.32:0/3463936636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5a40fed30 0x7fc5a410ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-12T04:06:35.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:35.930+0000 7fc5a37fe700 1 --2- 172.21.15.32:0/3463936636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5a40fe3e0 0x7fc5a410d880 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc5940137b0 tx=0x7fc594009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:06:35.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:35.930+0000 7fc58bfff700 1 -- 172.21.15.32:0/3463936636 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc594021070 con 0x7fc5a40fe3e0 2024-05-12T04:06:35.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:35.930+0000 7fc58bfff700 1 -- 172.21.15.32:0/3463936636 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc59401c400 con 0x7fc5a40fe3e0 2024-05-12T04:06:35.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:35.930+0000 7fc5a9ba6700 1 -- 172.21.15.32:0/3463936636 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5a410ee90 con 0x7fc5a40fe3e0 2024-05-12T04:06:35.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:35.930+0000 7fc58bfff700 1 -- 172.21.15.32:0/3463936636 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc59400e070 con 0x7fc5a40fe3e0 2024-05-12T04:06:35.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:35.930+0000 7fc5a9ba6700 1 -- 172.21.15.32:0/3463936636 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5a410f370 con 0x7fc5a40fe3e0 2024-05-12T04:06:35.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:35.931+0000 7fc5a9ba6700 1 -- 172.21.15.32:0/3463936636 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5a405f5c0 con 0x7fc5a40fe3e0 2024-05-12T04:06:35.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:35.932+0000 7fc58bfff700 1 -- 172.21.15.32:0/3463936636 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fc59401c570 con 0x7fc5a40fe3e0 2024-05-12T04:06:35.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:35.932+0000 7fc58bfff700 1 --2- 172.21.15.32:0/3463936636 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fc58c06cc30 0x7fc58c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:06:35.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:35.933+0000 7fc58bfff700 1 -- 172.21.15.32:0/3463936636 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fc59408d1d0 con 0x7fc5a40fe3e0 2024-05-12T04:06:35.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:35.933+0000 7fc5a2ffd700 1 --2- 172.21.15.32:0/3463936636 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fc58c06cc30 0x7fc58c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:06:35.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:35.933+0000 7fc5a2ffd700 1 --2- 172.21.15.32:0/3463936636 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fc58c06cc30 0x7fc58c06f0e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc598009b80 tx=0x7fc59800b040 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:06:35.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:35.937+0000 7fc58bfff700 1 -- 172.21.15.32:0/3463936636 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fc59405acf0 con 0x7fc5a40fe3e0 2024-05-12T04:06:36.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.160+0000 7fc5a9ba6700 1 -- 172.21.15.32:0/3463936636 --> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc5a410f760 con 0x7fc58c06cc30 2024-05-12T04:06:36.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.161+0000 7fc58bfff700 1 -- 172.21.15.32:0/3463936636 <== mgr.24593 v2:172.21.15.32:6800/2957111591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fc5a410f760 con 0x7fc58c06cc30 2024-05-12T04:06:36.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.163+0000 7fc5a9ba6700 1 -- 172.21.15.32:0/3463936636 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fc58c06cc30 msgr2=0x7fc58c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:06:36.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.163+0000 7fc5a9ba6700 1 --2- 172.21.15.32:0/3463936636 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fc58c06cc30 0x7fc58c06f0e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc598009b80 tx=0x7fc59800b040 comp rx=0 tx=0).stop 2024-05-12T04:06:36.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.163+0000 7fc5a9ba6700 1 -- 172.21.15.32:0/3463936636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5a40fe3e0 msgr2=0x7fc5a410d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:06:36.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.163+0000 7fc5a9ba6700 1 --2- 172.21.15.32:0/3463936636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5a40fe3e0 0x7fc5a410d880 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc5940137b0 tx=0x7fc594009e30 comp rx=0 tx=0).stop 2024-05-12T04:06:36.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.163+0000 7fc5a9ba6700 1 -- 172.21.15.32:0/3463936636 shutdown_connections 2024-05-12T04:06:36.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.163+0000 7fc5a9ba6700 1 --2- 172.21.15.32:0/3463936636 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fc58c06cc30 0x7fc58c06f0e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:36.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.163+0000 7fc5a9ba6700 1 --2- 172.21.15.32:0/3463936636 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5a40fe3e0 0x7fc5a410d880 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:36.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.163+0000 7fc5a9ba6700 1 --2- 172.21.15.32:0/3463936636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5a40fed30 0x7fc5a410ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:36.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.163+0000 7fc5a9ba6700 1 -- 172.21.15.32:0/3463936636 >> 172.21.15.32:0/3463936636 conn(0x7fc5a40f6b80 msgr2=0x7fc5a40f82d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:06:36.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.163+0000 7fc5a9ba6700 1 -- 172.21.15.32:0/3463936636 shutdown_connections 2024-05-12T04:06:36.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.163+0000 7fc5a9ba6700 1 -- 172.21.15.32:0/3463936636 wait complete. 2024-05-12T04:06:36.174 INFO:teuthology.orchestra.run.smithi032.stdout:true 2024-05-12T04:06:36.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.269+0000 7f4c04da2700 1 -- 172.21.15.32:0/192204791 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c000fdec0 msgr2=0x7f4c000fe330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:06:36.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.269+0000 7f4c04da2700 1 --2- 172.21.15.32:0/192204791 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c000fdec0 0x7f4c000fe330 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4004940 tx=0x7f4bf4009e30 comp rx=0 tx=0).stop 2024-05-12T04:06:36.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.269+0000 7f4c04da2700 1 -- 172.21.15.32:0/192204791 shutdown_connections 2024-05-12T04:06:36.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.269+0000 7f4c04da2700 1 --2- 172.21.15.32:0/192204791 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c000fdec0 0x7f4c000fe330 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:36.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.269+0000 7f4c04da2700 1 --2- 172.21.15.32:0/192204791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c000fcc10 0x7f4c000fd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:36.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.269+0000 7f4c04da2700 1 -- 172.21.15.32:0/192204791 >> 172.21.15.32:0/192204791 conn(0x7f4c000f8370 msgr2=0x7f4c000fa7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:06:36.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.269+0000 7f4c04da2700 1 -- 172.21.15.32:0/192204791 shutdown_connections 2024-05-12T04:06:36.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.269+0000 7f4c04da2700 1 -- 172.21.15.32:0/192204791 wait complete. 2024-05-12T04:06:36.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.270+0000 7f4c04da2700 1 Processor -- start 2024-05-12T04:06:36.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.270+0000 7f4c04da2700 1 -- start start 2024-05-12T04:06:36.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.270+0000 7f4c04da2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c000fcc10 0x7f4c0010d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:06:36.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.270+0000 7f4c04da2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c000fdec0 0x7f4c0010dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:06:36.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.270+0000 7f4c04da2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0010e1f0 con 0x7f4c000fdec0 2024-05-12T04:06:36.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.270+0000 7f4c04da2700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0010ea40 con 0x7f4c000fcc10 2024-05-12T04:06:36.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.270+0000 7f4bfeffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c000fdec0 0x7f4c0010dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:06:36.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.270+0000 7f4bfeffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c000fdec0 0x7f4c0010dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35194/0 (socket says 172.21.15.32:35194) 2024-05-12T04:06:36.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.270+0000 7f4bfeffd700 1 -- 172.21.15.32:0/1019906143 learned_addr learned my addr 172.21.15.32:0/1019906143 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:06:36.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.270+0000 7f4bff7fe700 1 --2- 172.21.15.32:0/1019906143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c000fcc10 0x7f4c0010d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:06:36.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.270+0000 7f4bfeffd700 1 -- 172.21.15.32:0/1019906143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c000fcc10 msgr2=0x7f4c0010d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:06:36.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.270+0000 7f4bfeffd700 1 --2- 172.21.15.32:0/1019906143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c000fcc10 0x7f4c0010d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:36.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.271+0000 7f4bfeffd700 1 -- 172.21.15.32:0/1019906143 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf4005040 con 0x7f4c000fdec0 2024-05-12T04:06:36.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.271+0000 7f4bfeffd700 1 --2- 172.21.15.32:0/1019906143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c000fdec0 0x7f4c0010dbf0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4bf40137b0 tx=0x7f4bf4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:06:36.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.271+0000 7f4be7fff700 1 -- 172.21.15.32:0/1019906143 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4bf400e070 con 0x7f4c000fdec0 2024-05-12T04:06:36.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.271+0000 7f4be7fff700 1 -- 172.21.15.32:0/1019906143 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4bf401c400 con 0x7f4c000fdec0 2024-05-12T04:06:36.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.271+0000 7f4c04da2700 1 -- 172.21.15.32:0/1019906143 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c0010ecc0 con 0x7f4c000fdec0 2024-05-12T04:06:36.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.271+0000 7f4be7fff700 1 -- 172.21.15.32:0/1019906143 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4bf400b650 con 0x7f4c000fdec0 2024-05-12T04:06:36.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.271+0000 7f4c04da2700 1 -- 172.21.15.32:0/1019906143 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c0010f1a0 con 0x7f4c000fdec0 2024-05-12T04:06:36.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.272+0000 7f4c04da2700 1 -- 172.21.15.32:0/1019906143 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c0005f5c0 con 0x7f4c000fdec0 2024-05-12T04:06:36.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.272+0000 7f4be7fff700 1 -- 172.21.15.32:0/1019906143 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f4bf401c9e0 con 0x7f4c000fdec0 2024-05-12T04:06:36.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.273+0000 7f4be7fff700 1 --2- 172.21.15.32:0/1019906143 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f4be806cc20 0x7f4be806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:06:36.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.273+0000 7f4be7fff700 1 -- 172.21.15.32:0/1019906143 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f4bf408d6b0 con 0x7f4c000fdec0 2024-05-12T04:06:36.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.273+0000 7f4bff7fe700 1 --2- 172.21.15.32:0/1019906143 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f4be806cc20 0x7f4be806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:06:36.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.273+0000 7f4bff7fe700 1 --2- 172.21.15.32:0/1019906143 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f4be806cc20 0x7f4be806f0d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4bf0009b80 tx=0x7f4bf000b040 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:06:36.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.278+0000 7f4be7fff700 1 -- 172.21.15.32:0/1019906143 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f4bf405b130 con 0x7f4c000fdec0 2024-05-12T04:06:36.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.500+0000 7f4c04da2700 1 -- 172.21.15.32:0/1019906143 --> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4c0010f4d0 con 0x7f4be806cc20 2024-05-12T04:06:36.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.501+0000 7f4be7fff700 1 -- 172.21.15.32:0/1019906143 <== mgr.24593 v2:172.21.15.32:6800/2957111591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f4c0010f4d0 con 0x7f4be806cc20 2024-05-12T04:06:36.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.503+0000 7f4c04da2700 1 -- 172.21.15.32:0/1019906143 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f4be806cc20 msgr2=0x7f4be806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:06:36.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.503+0000 7f4c04da2700 1 --2- 172.21.15.32:0/1019906143 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f4be806cc20 0x7f4be806f0d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4bf0009b80 tx=0x7f4bf000b040 comp rx=0 tx=0).stop 2024-05-12T04:06:36.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.503+0000 7f4c04da2700 1 -- 172.21.15.32:0/1019906143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c000fdec0 msgr2=0x7f4c0010dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:06:36.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.503+0000 7f4c04da2700 1 --2- 172.21.15.32:0/1019906143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c000fdec0 0x7f4c0010dbf0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4bf40137b0 tx=0x7f4bf4009e30 comp rx=0 tx=0).stop 2024-05-12T04:06:36.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.504+0000 7f4c04da2700 1 -- 172.21.15.32:0/1019906143 shutdown_connections 2024-05-12T04:06:36.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.504+0000 7f4c04da2700 1 --2- 172.21.15.32:0/1019906143 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f4be806cc20 0x7f4be806f0d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:36.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.504+0000 7f4c04da2700 1 --2- 172.21.15.32:0/1019906143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c000fcc10 0x7f4c0010d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:36.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.504+0000 7f4c04da2700 1 --2- 172.21.15.32:0/1019906143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c000fdec0 0x7f4c0010dbf0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:36.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.504+0000 7f4c04da2700 1 -- 172.21.15.32:0/1019906143 >> 172.21.15.32:0/1019906143 conn(0x7f4c000f8370 msgr2=0x7f4c000f9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:06:36.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.504+0000 7f4c04da2700 1 -- 172.21.15.32:0/1019906143 shutdown_connections 2024-05-12T04:06:36.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.504+0000 7f4c04da2700 1 -- 172.21.15.32:0/1019906143 wait complete. 2024-05-12T04:06:36.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.608+0000 7f4188433700 1 -- 172.21.15.32:0/2129321322 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41800fdff0 msgr2=0x7f41800fe460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:06:36.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.608+0000 7f4188433700 1 --2- 172.21.15.32:0/2129321322 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41800fdff0 0x7f41800fe460 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4170004940 tx=0x7f4170009e30 comp rx=0 tx=0).stop 2024-05-12T04:06:36.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.608+0000 7f4188433700 1 -- 172.21.15.32:0/2129321322 shutdown_connections 2024-05-12T04:06:36.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.608+0000 7f4188433700 1 --2- 172.21.15.32:0/2129321322 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41800fdff0 0x7f41800fe460 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:36.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.608+0000 7f4188433700 1 --2- 172.21.15.32:0/2129321322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41800fce80 0x7f41800fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:36.608 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.608+0000 7f4188433700 1 -- 172.21.15.32:0/2129321322 >> 172.21.15.32:0/2129321322 conn(0x7f41800f8540 msgr2=0x7f41800fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:06:36.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.608+0000 7f4188433700 1 -- 172.21.15.32:0/2129321322 shutdown_connections 2024-05-12T04:06:36.609 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.608+0000 7f4188433700 1 -- 172.21.15.32:0/2129321322 wait complete. 2024-05-12T04:06:36.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.608+0000 7f4188433700 1 Processor -- start 2024-05-12T04:06:36.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.609+0000 7f4188433700 1 -- start start 2024-05-12T04:06:36.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.609+0000 7f4188433700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41800fce80 0x7f418010d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:06:36.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.609+0000 7f4188433700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41800fdff0 0x7f418010de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:06:36.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.609+0000 7f4188433700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f418010e430 con 0x7f41800fdff0 2024-05-12T04:06:36.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.609+0000 7f4188433700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f418010ec80 con 0x7f41800fce80 2024-05-12T04:06:36.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.609+0000 7f41859ce700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41800fdff0 0x7f418010de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:06:36.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.609+0000 7f41859ce700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41800fdff0 0x7f418010de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35214/0 (socket says 172.21.15.32:35214) 2024-05-12T04:06:36.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.609+0000 7f41859ce700 1 -- 172.21.15.32:0/2791187853 learned_addr learned my addr 172.21.15.32:0/2791187853 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:06:36.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.609+0000 7f41861cf700 1 --2- 172.21.15.32:0/2791187853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41800fce80 0x7f418010d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:06:36.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.609+0000 7f41859ce700 1 -- 172.21.15.32:0/2791187853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41800fce80 msgr2=0x7f418010d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:06:36.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.609+0000 7f41859ce700 1 --2- 172.21.15.32:0/2791187853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41800fce80 0x7f418010d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:36.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.609+0000 7f41859ce700 1 -- 172.21.15.32:0/2791187853 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4170005040 con 0x7f41800fdff0 2024-05-12T04:06:36.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.609+0000 7f41861cf700 1 --2- 172.21.15.32:0/2791187853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41800fce80 0x7f418010d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-12T04:06:36.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.609+0000 7f41859ce700 1 --2- 172.21.15.32:0/2791187853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41800fdff0 0x7f418010de30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f417000b0d0 tx=0x7f4170009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:06:36.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.611+0000 7f4176ffd700 1 -- 172.21.15.32:0/2791187853 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f417000e070 con 0x7f41800fdff0 2024-05-12T04:06:36.611 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.611+0000 7f4176ffd700 1 -- 172.21.15.32:0/2791187853 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f417000b410 con 0x7f41800fdff0 2024-05-12T04:06:36.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.611+0000 7f4176ffd700 1 -- 172.21.15.32:0/2791187853 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4170012600 con 0x7f41800fdff0 2024-05-12T04:06:36.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.611+0000 7f4188433700 1 -- 172.21.15.32:0/2791187853 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f418010ef00 con 0x7f41800fdff0 2024-05-12T04:06:36.612 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.611+0000 7f4188433700 1 -- 172.21.15.32:0/2791187853 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f418010f440 con 0x7f41800fdff0 2024-05-12T04:06:36.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.612+0000 7f4176ffd700 1 -- 172.21.15.32:0/2791187853 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f417000b9f0 con 0x7f41800fdff0 2024-05-12T04:06:36.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.613+0000 7f4188433700 1 -- 172.21.15.32:0/2791187853 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f418005f5c0 con 0x7f41800fdff0 2024-05-12T04:06:36.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.613+0000 7f4176ffd700 1 --2- 172.21.15.32:0/2791187853 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f416c06cc30 0x7f416c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:06:36.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.613+0000 7f4176ffd700 1 -- 172.21.15.32:0/2791187853 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f417008cad0 con 0x7f41800fdff0 2024-05-12T04:06:36.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.613+0000 7f41861cf700 1 --2- 172.21.15.32:0/2791187853 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f416c06cc30 0x7f416c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:06:36.614 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.613+0000 7f41861cf700 1 --2- 172.21.15.32:0/2791187853 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f416c06cc30 0x7f416c06f0e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f417c009b80 tx=0x7f417c00b040 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:06:36.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.618+0000 7f4176ffd700 1 -- 172.21.15.32:0/2791187853 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f417005a740 con 0x7f41800fdff0 2024-05-12T04:06:36.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.831+0000 7f4188433700 1 -- 172.21.15.32:0/2791187853 --> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f418010f7c0 con 0x7f416c06cc30 2024-05-12T04:06:36.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.841+0000 7f4176ffd700 1 -- 172.21.15.32:0/2791187853 <== mgr.24593 v2:172.21.15.32:6800/2957111591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f418010f7c0 con 0x7f416c06cc30 2024-05-12T04:06:36.841 INFO:teuthology.orchestra.run.smithi032.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-12T04:06:36.841 INFO:teuthology.orchestra.run.smithi032.stdout:alertmanager.smithi032 smithi032 *:9093,9094 running (5m) 34s ago 14m 25.5M - 0.25.0 c8568f914cd2 a37fbab7af4c 2024-05-12T04:06:36.841 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi032 smithi032 running (2m) 34s ago 14m 6660k - 18.2.2-1272-g58249552 ddcb0a9d6fde 8c53f1fc7b06 2024-05-12T04:06:36.841 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi033 smithi033 running (2m) 1s ago 13m 6651k - 18.2.2-1272-g58249552 ddcb0a9d6fde cf4fe9ca070b 2024-05-12T04:06:36.841 INFO:teuthology.orchestra.run.smithi032.stdout:grafana.smithi032 smithi032 *:3000 running (5m) 34s ago 14m 89.7M - 9.4.7 954c08fa6188 83055e58d23c 2024-05-12T04:06:36.842 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.ialtqb smithi032 running (8m) 34s ago 8m 13.9M - 17.2.7-939-gc190bdee 284803f7d9cf 0683abf00e3e 2024-05-12T04:06:36.842 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.qpsawh smithi032 running (8m) 34s ago 8m 155M - 17.2.7-939-gc190bdee 284803f7d9cf 95a7db5c4fcd 2024-05-12T04:06:36.842 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi033.hglwml smithi033 running (8m) 1s ago 8m 13.8M - 17.2.7-939-gc190bdee 284803f7d9cf c97902b05025 2024-05-12T04:06:36.842 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi033.jgbjhk smithi033 running (8m) 1s ago 8m 102M - 17.2.7-939-gc190bdee 284803f7d9cf af97052cb97a 2024-05-12T04:06:36.842 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi032.ukatkq smithi032 *:8443,9283,8765 running (4m) 34s ago 16m 501M - 18.2.2-1272-g58249552 ddcb0a9d6fde 0d88ab9a5292 2024-05-12T04:06:36.842 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi033.pcvxwy smithi033 *:8443,9283,8765 running (3m) 1s ago 13m 449M - 18.2.2-1272-g58249552 ddcb0a9d6fde d42dcb6bd307 2024-05-12T04:06:36.842 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi032 smithi032 running (3m) 34s ago 16m 45.5M 2048M 18.2.2-1272-g58249552 ddcb0a9d6fde 81f94171c5dc 2024-05-12T04:06:36.842 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi033 smithi033 running (3m) 1s ago 13m 37.4M 2048M 18.2.2-1272-g58249552 ddcb0a9d6fde d648ba108f3e 2024-05-12T04:06:36.842 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi032 smithi032 *:9100 running (5m) 34s ago 14m 16.5M - 1.5.0 0da6a335fe13 cb75a7c0c829 2024-05-12T04:06:36.842 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi033 smithi033 *:9100 running (4m) 1s ago 13m 18.9M - 1.5.0 0da6a335fe13 7888c36c659f 2024-05-12T04:06:36.842 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0 smithi032 running (111s) 34s ago 12m 80.8M 1570M 18.2.2-1272-g58249552 ddcb0a9d6fde 07f66c4deb3b 2024-05-12T04:06:36.842 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1 smithi032 running (75s) 34s ago 12m 84.5M 1570M 18.2.2-1272-g58249552 ddcb0a9d6fde 67a87807d654 2024-05-12T04:06:36.842 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2 smithi032 running (40s) 34s ago 11m 11.9M 1570M 18.2.2-1272-g58249552 ddcb0a9d6fde 4ac48d74ee8b 2024-05-12T04:06:36.842 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3 smithi033 running (5s) 1s ago 11m 13.7M 2594M 18.2.2-1272-g58249552 ddcb0a9d6fde 11f097a455bb 2024-05-12T04:06:36.843 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4 smithi033 running (10m) 1s ago 10m 215M 2594M 17.2.7-939-gc190bdee 284803f7d9cf 0896268b4591 2024-05-12T04:06:36.843 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5 smithi033 running (10m) 1s ago 10m 197M 2594M 17.2.7-939-gc190bdee 284803f7d9cf 57e65a64c821 2024-05-12T04:06:36.843 INFO:teuthology.orchestra.run.smithi032.stdout:prometheus.smithi032 smithi032 *:9095 running (4m) 34s ago 14m 54.2M - 2.43.0 a07b618ecd1d 46eafb25e176 2024-05-12T04:06:36.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.843+0000 7f4188433700 1 -- 172.21.15.32:0/2791187853 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f416c06cc30 msgr2=0x7f416c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:06:36.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.843+0000 7f4188433700 1 --2- 172.21.15.32:0/2791187853 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f416c06cc30 0x7f416c06f0e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f417c009b80 tx=0x7f417c00b040 comp rx=0 tx=0).stop 2024-05-12T04:06:36.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.843+0000 7f4188433700 1 -- 172.21.15.32:0/2791187853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41800fdff0 msgr2=0x7f418010de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:06:36.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.843+0000 7f4188433700 1 --2- 172.21.15.32:0/2791187853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41800fdff0 0x7f418010de30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f417000b0d0 tx=0x7f4170009e30 comp rx=0 tx=0).stop 2024-05-12T04:06:36.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.843+0000 7f4188433700 1 -- 172.21.15.32:0/2791187853 shutdown_connections 2024-05-12T04:06:36.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.843+0000 7f4188433700 1 --2- 172.21.15.32:0/2791187853 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f416c06cc30 0x7f416c06f0e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:36.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.843+0000 7f4188433700 1 --2- 172.21.15.32:0/2791187853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41800fce80 0x7f418010d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:36.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.843+0000 7f4188433700 1 --2- 172.21.15.32:0/2791187853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41800fdff0 0x7f418010de30 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:36.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.843+0000 7f4188433700 1 -- 172.21.15.32:0/2791187853 >> 172.21.15.32:0/2791187853 conn(0x7f41800f8540 msgr2=0x7f41800f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:06:36.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.843+0000 7f4188433700 1 -- 172.21.15.32:0/2791187853 shutdown_connections 2024-05-12T04:06:36.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.843+0000 7f4188433700 1 -- 172.21.15.32:0/2791187853 wait complete. 2024-05-12T04:06:36.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:36 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:06:36.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:36 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:06:36.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:36 smithi033 ceph-mon[107305]: pgmap v120: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 260 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 60/276 objects degraded (21.739%) 2024-05-12T04:06:36.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:36 smithi033 ceph-mon[107305]: from='client.34308 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:06:36.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.946+0000 7f586b66a700 1 -- 172.21.15.32:0/3431098990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58640fce60 msgr2=0x7f58640fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:06:36.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.946+0000 7f586b66a700 1 --2- 172.21.15.32:0/3431098990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58640fce60 0x7f58640fd270 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5860004940 tx=0x7f5860009e30 comp rx=0 tx=0).stop 2024-05-12T04:06:36.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.946+0000 7f586b66a700 1 -- 172.21.15.32:0/3431098990 shutdown_connections 2024-05-12T04:06:36.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.946+0000 7f586b66a700 1 --2- 172.21.15.32:0/3431098990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58640fdfd0 0x7f58640fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:36.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.946+0000 7f586b66a700 1 --2- 172.21.15.32:0/3431098990 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58640fce60 0x7f58640fd270 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:36.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.946+0000 7f586b66a700 1 -- 172.21.15.32:0/3431098990 >> 172.21.15.32:0/3431098990 conn(0x7f58640f8560 msgr2=0x7f58640fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:06:36.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.946+0000 7f586b66a700 1 -- 172.21.15.32:0/3431098990 shutdown_connections 2024-05-12T04:06:36.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.946+0000 7f586b66a700 1 -- 172.21.15.32:0/3431098990 wait complete. 2024-05-12T04:06:36.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.947+0000 7f586b66a700 1 Processor -- start 2024-05-12T04:06:36.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.947+0000 7f586b66a700 1 -- start start 2024-05-12T04:06:36.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.947+0000 7f586b66a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58640fce60 0x7f5864071d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:06:36.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.947+0000 7f586b66a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58640fdfd0 0x7f58640702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:06:36.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.947+0000 7f586b66a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5864070840 con 0x7f58640fdfd0 2024-05-12T04:06:36.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.947+0000 7f586b66a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5864070980 con 0x7f58640fce60 2024-05-12T04:06:36.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.947+0000 7f5868c05700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58640fdfd0 0x7f58640702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:06:36.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.947+0000 7f5868c05700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58640fdfd0 0x7f58640702b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35240/0 (socket says 172.21.15.32:35240) 2024-05-12T04:06:36.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.947+0000 7f5868c05700 1 -- 172.21.15.32:0/1095956574 learned_addr learned my addr 172.21.15.32:0/1095956574 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:06:36.949 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.947+0000 7f5869406700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58640fce60 0x7f5864071d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:06:36.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.947+0000 7f5868c05700 1 -- 172.21.15.32:0/1095956574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58640fce60 msgr2=0x7f5864071d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:06:36.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.947+0000 7f5868c05700 1 --2- 172.21.15.32:0/1095956574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58640fce60 0x7f5864071d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:36.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.947+0000 7f5868c05700 1 -- 172.21.15.32:0/1095956574 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5860005040 con 0x7f58640fdfd0 2024-05-12T04:06:36.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.947+0000 7f5869406700 1 --2- 172.21.15.32:0/1095956574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58640fce60 0x7f5864071d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-12T04:06:36.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.948+0000 7f5868c05700 1 --2- 172.21.15.32:0/1095956574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58640fdfd0 0x7f58640702b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f585400eb40 tx=0x7f585400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:06:36.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.948+0000 7f5859ffb700 1 -- 172.21.15.32:0/1095956574 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5854007cc0 con 0x7f58640fdfd0 2024-05-12T04:06:36.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.948+0000 7f586b66a700 1 -- 172.21.15.32:0/1095956574 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5864070c00 con 0x7f58640fdfd0 2024-05-12T04:06:36.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.948+0000 7f5859ffb700 1 -- 172.21.15.32:0/1095956574 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f585400f040 con 0x7f58640fdfd0 2024-05-12T04:06:36.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.948+0000 7f5859ffb700 1 -- 172.21.15.32:0/1095956574 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f585400c990 con 0x7f58640fdfd0 2024-05-12T04:06:36.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.948+0000 7f586b66a700 1 -- 172.21.15.32:0/1095956574 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58641158e0 con 0x7f58640fdfd0 2024-05-12T04:06:36.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.949+0000 7f586b66a700 1 -- 172.21.15.32:0/1095956574 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f586405f5c0 con 0x7f58640fdfd0 2024-05-12T04:06:36.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.949+0000 7f5859ffb700 1 -- 172.21.15.32:0/1095956574 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f585400caf0 con 0x7f58640fdfd0 2024-05-12T04:06:36.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.950+0000 7f5859ffb700 1 --2- 172.21.15.32:0/1095956574 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f585006cc20 0x7f585006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:06:36.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.950+0000 7f5859ffb700 1 -- 172.21.15.32:0/1095956574 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f585408bb40 con 0x7f58640fdfd0 2024-05-12T04:06:36.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.950+0000 7f5869406700 1 --2- 172.21.15.32:0/1095956574 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f585006cc20 0x7f585006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:06:36.952 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.950+0000 7f5869406700 1 --2- 172.21.15.32:0/1095956574 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f585006cc20 0x7f585006f0d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5860013680 tx=0x7f5860009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:06:36.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:36.955+0000 7f5859ffb700 1 -- 172.21.15.32:0/1095956574 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f5854055c80 con 0x7f58640fdfd0 2024-05-12T04:06:37.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:36 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:06:37.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:36 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:06:37.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:36 smithi032 ceph-mon[120392]: pgmap v120: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 260 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 60/276 objects degraded (21.739%) 2024-05-12T04:06:37.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:36 smithi032 ceph-mon[120392]: from='client.34308 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:06:37.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.241+0000 7f586b66a700 1 -- 172.21.15.32:0/1095956574 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5864115aa0 con 0x7f58640fdfd0 2024-05-12T04:06:37.242 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.241+0000 7f5859ffb700 1 -- 172.21.15.32:0/1095956574 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+847 (secure 0 0 0) 0x7f58540592a0 con 0x7f58640fdfd0 2024-05-12T04:06:37.242 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-05-12T04:06:37.242 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": { 2024-05-12T04:06:37.242 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 2 2024-05-12T04:06:37.242 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:06:37.242 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": { 2024-05-12T04:06:37.242 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 2 2024-05-12T04:06:37.242 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:06:37.242 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": { 2024-05-12T04:06:37.242 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 2, 2024-05-12T04:06:37.243 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 3 2024-05-12T04:06:37.243 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:06:37.243 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": { 2024-05-12T04:06:37.243 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 4 2024-05-12T04:06:37.243 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:06:37.243 INFO:teuthology.orchestra.run.smithi032.stdout: "overall": { 2024-05-12T04:06:37.243 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 6, 2024-05-12T04:06:37.243 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 7 2024-05-12T04:06:37.243 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-05-12T04:06:37.243 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-05-12T04:06:37.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.243+0000 7f586b66a700 1 -- 172.21.15.32:0/1095956574 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f585006cc20 msgr2=0x7f585006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:06:37.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.243+0000 7f586b66a700 1 --2- 172.21.15.32:0/1095956574 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f585006cc20 0x7f585006f0d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5860013680 tx=0x7f5860009e30 comp rx=0 tx=0).stop 2024-05-12T04:06:37.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.243+0000 7f586b66a700 1 -- 172.21.15.32:0/1095956574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58640fdfd0 msgr2=0x7f58640702b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:06:37.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.243+0000 7f586b66a700 1 --2- 172.21.15.32:0/1095956574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58640fdfd0 0x7f58640702b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f585400eb40 tx=0x7f585400b040 comp rx=0 tx=0).stop 2024-05-12T04:06:37.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.243+0000 7f586b66a700 1 -- 172.21.15.32:0/1095956574 shutdown_connections 2024-05-12T04:06:37.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.243+0000 7f586b66a700 1 --2- 172.21.15.32:0/1095956574 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f585006cc20 0x7f585006f0d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:37.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.243+0000 7f586b66a700 1 --2- 172.21.15.32:0/1095956574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58640fce60 0x7f5864071d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:37.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.243+0000 7f586b66a700 1 --2- 172.21.15.32:0/1095956574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58640fdfd0 0x7f58640702b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:37.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.243+0000 7f586b66a700 1 -- 172.21.15.32:0/1095956574 >> 172.21.15.32:0/1095956574 conn(0x7f58640f8560 msgr2=0x7f58640f9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:06:37.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.244+0000 7f586b66a700 1 -- 172.21.15.32:0/1095956574 shutdown_connections 2024-05-12T04:06:37.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.244+0000 7f586b66a700 1 -- 172.21.15.32:0/1095956574 wait complete. 2024-05-12T04:06:37.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.347+0000 7f9664575700 1 -- 172.21.15.32:0/3770067634 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f965c066260 msgr2=0x7f965c0ffa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:06:37.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.347+0000 7f9664575700 1 --2- 172.21.15.32:0/3770067634 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f965c066260 0x7f965c0ffa00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9658004940 tx=0x7f9658009e30 comp rx=0 tx=0).stop 2024-05-12T04:06:37.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.347+0000 7f9664575700 1 -- 172.21.15.32:0/3770067634 shutdown_connections 2024-05-12T04:06:37.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.347+0000 7f9664575700 1 --2- 172.21.15.32:0/3770067634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f965c0fff40 0x7f965c102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:37.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.347+0000 7f9664575700 1 --2- 172.21.15.32:0/3770067634 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f965c066260 0x7f965c0ffa00 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:37.348 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.347+0000 7f9664575700 1 -- 172.21.15.32:0/3770067634 >> 172.21.15.32:0/3770067634 conn(0x7f965c0f7430 msgr2=0x7f965c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:06:37.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.347+0000 7f9664575700 1 -- 172.21.15.32:0/3770067634 shutdown_connections 2024-05-12T04:06:37.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.347+0000 7f9664575700 1 -- 172.21.15.32:0/3770067634 wait complete. 2024-05-12T04:06:37.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.347+0000 7f9664575700 1 Processor -- start 2024-05-12T04:06:37.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.348+0000 7f9664575700 1 -- start start 2024-05-12T04:06:37.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.348+0000 7f9664575700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f965c066260 0x7f965c10d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:06:37.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.348+0000 7f9664575700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f965c0fff40 0x7f965c10de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:06:37.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.348+0000 7f9664575700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f965c10e460 con 0x7f965c0fff40 2024-05-12T04:06:37.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.348+0000 7f9664575700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f965c10ecb0 con 0x7f965c066260 2024-05-12T04:06:37.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.348+0000 7f9662311700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f965c066260 0x7f965c10d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:06:37.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.348+0000 7f9661b10700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f965c0fff40 0x7f965c10de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:06:37.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.348+0000 7f9662311700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f965c066260 0x7f965c10d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.32:56548/0 (socket says 172.21.15.32:56548) 2024-05-12T04:06:37.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.348+0000 7f9662311700 1 -- 172.21.15.32:0/2645065361 learned_addr learned my addr 172.21.15.32:0/2645065361 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:06:37.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.348+0000 7f9662311700 1 -- 172.21.15.32:0/2645065361 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f965c0fff40 msgr2=0x7f965c10de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:06:37.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.348+0000 7f9662311700 1 --2- 172.21.15.32:0/2645065361 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f965c0fff40 0x7f965c10de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:37.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.348+0000 7f9662311700 1 -- 172.21.15.32:0/2645065361 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9658005040 con 0x7f965c066260 2024-05-12T04:06:37.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.348+0000 7f9661b10700 1 --2- 172.21.15.32:0/2645065361 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f965c0fff40 0x7f965c10de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-12T04:06:37.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.348+0000 7f9662311700 1 --2- 172.21.15.32:0/2645065361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f965c066260 0x7f965c10d920 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9658000c00 tx=0x7f9658009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:06:37.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.348+0000 7f9652ffd700 1 -- 172.21.15.32:0/2645065361 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f965800e070 con 0x7f965c066260 2024-05-12T04:06:37.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.349+0000 7f9664575700 1 -- 172.21.15.32:0/2645065361 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f965c10ef30 con 0x7f965c066260 2024-05-12T04:06:37.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.349+0000 7f9652ffd700 1 -- 172.21.15.32:0/2645065361 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f96580093b0 con 0x7f965c066260 2024-05-12T04:06:37.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.349+0000 7f9652ffd700 1 -- 172.21.15.32:0/2645065361 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f965801cdc0 con 0x7f965c066260 2024-05-12T04:06:37.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.349+0000 7f9664575700 1 -- 172.21.15.32:0/2645065361 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f965c10f410 con 0x7f965c066260 2024-05-12T04:06:37.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.349+0000 7f9664575700 1 -- 172.21.15.32:0/2645065361 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f965c05f5c0 con 0x7f965c066260 2024-05-12T04:06:37.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.350+0000 7f9652ffd700 1 -- 172.21.15.32:0/2645065361 <== mon.1 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f965801c600 con 0x7f965c066260 2024-05-12T04:06:37.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.350+0000 7f9652ffd700 1 --2- 172.21.15.32:0/2645065361 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f964806cc30 0x7f964806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:06:37.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.351+0000 7f9652ffd700 1 -- 172.21.15.32:0/2645065361 <== mon.1 v2:172.21.15.33:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f965808d5c0 con 0x7f965c066260 2024-05-12T04:06:37.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.351+0000 7f9661b10700 1 --2- 172.21.15.32:0/2645065361 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f964806cc30 0x7f964806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:06:37.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.351+0000 7f9661b10700 1 --2- 172.21.15.32:0/2645065361 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f964806cc30 0x7f964806f0e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f964c009b80 tx=0x7f964c00b040 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:06:37.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.355+0000 7f9652ffd700 1 -- 172.21.15.32:0/2645065361 <== mon.1 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f965805b0c0 con 0x7f965c066260 2024-05-12T04:06:37.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.602+0000 7f9664575700 1 -- 172.21.15.32:0/2645065361 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f965c10f800 con 0x7f965c066260 2024-05-12T04:06:37.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.602+0000 7f9652ffd700 1 -- 172.21.15.32:0/2645065361 <== mon.1 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1866 (secure 0 0 0) 0x7f965805ac50 con 0x7f965c066260 2024-05-12T04:06:37.603 INFO:teuthology.orchestra.run.smithi032.stdout:e18 2024-05-12T04:06:37.603 INFO:teuthology.orchestra.run.smithi032.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-12T04:06:37.603 INFO:teuthology.orchestra.run.smithi032.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-12T04:06:37.603 INFO:teuthology.orchestra.run.smithi032.stdout:legacy client fscid: 1 2024-05-12T04:06:37.603 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:06:37.603 INFO:teuthology.orchestra.run.smithi032.stdout:Filesystem 'cephfs' (1) 2024-05-12T04:06:37.603 INFO:teuthology.orchestra.run.smithi032.stdout:fs_name cephfs 2024-05-12T04:06:37.603 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 18 2024-05-12T04:06:37.603 INFO:teuthology.orchestra.run.smithi032.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-12T04:06:37.603 INFO:teuthology.orchestra.run.smithi032.stdout:created 2024-05-12T03:57:48.904614+0000 2024-05-12T04:06:37.603 INFO:teuthology.orchestra.run.smithi032.stdout:modified 2024-05-12T03:58:06.821591+0000 2024-05-12T04:06:37.604 INFO:teuthology.orchestra.run.smithi032.stdout:tableserver 0 2024-05-12T04:06:37.604 INFO:teuthology.orchestra.run.smithi032.stdout:root 0 2024-05-12T04:06:37.604 INFO:teuthology.orchestra.run.smithi032.stdout:session_timeout 60 2024-05-12T04:06:37.604 INFO:teuthology.orchestra.run.smithi032.stdout:session_autoclose 300 2024-05-12T04:06:37.604 INFO:teuthology.orchestra.run.smithi032.stdout:max_file_size 1099511627776 2024-05-12T04:06:37.604 INFO:teuthology.orchestra.run.smithi032.stdout:max_xattr_size 65536 2024-05-12T04:06:37.604 INFO:teuthology.orchestra.run.smithi032.stdout:required_client_features {} 2024-05-12T04:06:37.604 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure 0 2024-05-12T04:06:37.604 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure_osd_epoch 46 2024-05-12T04:06:37.604 INFO:teuthology.orchestra.run.smithi032.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-12T04:06:37.604 INFO:teuthology.orchestra.run.smithi032.stdout:max_mds 1 2024-05-12T04:06:37.604 INFO:teuthology.orchestra.run.smithi032.stdout:in 0 2024-05-12T04:06:37.604 INFO:teuthology.orchestra.run.smithi032.stdout:up {0=14474} 2024-05-12T04:06:37.604 INFO:teuthology.orchestra.run.smithi032.stdout:failed 2024-05-12T04:06:37.604 INFO:teuthology.orchestra.run.smithi032.stdout:damaged 2024-05-12T04:06:37.605 INFO:teuthology.orchestra.run.smithi032.stdout:stopped 2024-05-12T04:06:37.605 INFO:teuthology.orchestra.run.smithi032.stdout:data_pools [3] 2024-05-12T04:06:37.605 INFO:teuthology.orchestra.run.smithi032.stdout:metadata_pool 2 2024-05-12T04:06:37.605 INFO:teuthology.orchestra.run.smithi032.stdout:inline_data enabled 2024-05-12T04:06:37.605 INFO:teuthology.orchestra.run.smithi032.stdout:balancer 2024-05-12T04:06:37.605 INFO:teuthology.orchestra.run.smithi032.stdout:bal_rank_mask -1 2024-05-12T04:06:37.605 INFO:teuthology.orchestra.run.smithi032.stdout:standby_count_wanted 1 2024-05-12T04:06:37.605 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.qpsawh{0:14474} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.32:6828/3100155296,v1:172.21.15.32:6829/3100155296] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:06:37.605 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi033.jgbjhk{0:24267} state up:standby-replay seq 3 join_fscid=1 addr [v2:172.21.15.33:6824/55417471,v1:172.21.15.33:6825/55417471] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:06:37.605 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:06:37.605 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:06:37.605 INFO:teuthology.orchestra.run.smithi032.stdout:Standby daemons: 2024-05-12T04:06:37.605 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:06:37.605 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi033.hglwml{-1:24283} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/433858238,v1:172.21.15.33:6827/433858238] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:06:37.605 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.ialtqb{-1:24287} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.32:6826/3703132329,v1:172.21.15.32:6827/3703132329] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:06:37.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.605+0000 7f9664575700 1 -- 172.21.15.32:0/2645065361 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f964806cc30 msgr2=0x7f964806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:06:37.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.605+0000 7f9664575700 1 --2- 172.21.15.32:0/2645065361 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f964806cc30 0x7f964806f0e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f964c009b80 tx=0x7f964c00b040 comp rx=0 tx=0).stop 2024-05-12T04:06:37.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.605+0000 7f9664575700 1 -- 172.21.15.32:0/2645065361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f965c066260 msgr2=0x7f965c10d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:06:37.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.605+0000 7f9664575700 1 --2- 172.21.15.32:0/2645065361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f965c066260 0x7f965c10d920 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9658000c00 tx=0x7f9658009e30 comp rx=0 tx=0).stop 2024-05-12T04:06:37.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.605+0000 7f9664575700 1 -- 172.21.15.32:0/2645065361 shutdown_connections 2024-05-12T04:06:37.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.605+0000 7f9664575700 1 --2- 172.21.15.32:0/2645065361 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f964806cc30 0x7f964806f0e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:37.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.605+0000 7f9664575700 1 --2- 172.21.15.32:0/2645065361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f965c066260 0x7f965c10d920 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:37.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.605+0000 7f9664575700 1 --2- 172.21.15.32:0/2645065361 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f965c0fff40 0x7f965c10de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:37.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.605+0000 7f9664575700 1 -- 172.21.15.32:0/2645065361 >> 172.21.15.32:0/2645065361 conn(0x7f965c0f7430 msgr2=0x7f965c0f9810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:06:37.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.605+0000 7f9664575700 1 -- 172.21.15.32:0/2645065361 shutdown_connections 2024-05-12T04:06:37.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.605+0000 7f9664575700 1 -- 172.21.15.32:0/2645065361 wait complete. 2024-05-12T04:06:37.607 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 18 2024-05-12T04:06:37.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.711+0000 7f36cd208700 1 -- 172.21.15.32:0/1347590508 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36c80fbcb0 msgr2=0x7f36c80fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:06:37.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.711+0000 7f36cd208700 1 --2- 172.21.15.32:0/1347590508 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36c80fbcb0 0x7f36c80fc0c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f36b8004940 tx=0x7f36b8009e30 comp rx=0 tx=0).stop 2024-05-12T04:06:37.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.711+0000 7f36cd208700 1 -- 172.21.15.32:0/1347590508 shutdown_connections 2024-05-12T04:06:37.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.711+0000 7f36cd208700 1 --2- 172.21.15.32:0/1347590508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36c80fcf60 0x7f36c80fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:37.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.711+0000 7f36cd208700 1 --2- 172.21.15.32:0/1347590508 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36c80fbcb0 0x7f36c80fc0c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:37.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.711+0000 7f36cd208700 1 -- 172.21.15.32:0/1347590508 >> 172.21.15.32:0/1347590508 conn(0x7f36c80f7430 msgr2=0x7f36c80f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:06:37.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.711+0000 7f36cd208700 1 -- 172.21.15.32:0/1347590508 shutdown_connections 2024-05-12T04:06:37.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.711+0000 7f36cd208700 1 -- 172.21.15.32:0/1347590508 wait complete. 2024-05-12T04:06:37.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.712+0000 7f36cd208700 1 Processor -- start 2024-05-12T04:06:37.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.712+0000 7f36cd208700 1 -- start start 2024-05-12T04:06:37.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.712+0000 7f36cd208700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36c80fbcb0 0x7f36c81105b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:06:37.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.712+0000 7f36cd208700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36c80fcf60 0x7f36c8112b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:06:37.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.712+0000 7f36cd208700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c8066330 con 0x7f36c80fcf60 2024-05-12T04:06:37.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.712+0000 7f36cd208700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c8066470 con 0x7f36c80fbcb0 2024-05-12T04:06:37.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.712+0000 7f36c659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36c80fcf60 0x7f36c8112b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:06:37.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.712+0000 7f36c659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36c80fcf60 0x7f36c8112b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35272/0 (socket says 172.21.15.32:35272) 2024-05-12T04:06:37.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.712+0000 7f36c659c700 1 -- 172.21.15.32:0/254505092 learned_addr learned my addr 172.21.15.32:0/254505092 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:06:37.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.712+0000 7f36c6d9d700 1 --2- 172.21.15.32:0/254505092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36c80fbcb0 0x7f36c81105b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:06:37.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.713+0000 7f36c659c700 1 -- 172.21.15.32:0/254505092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36c80fbcb0 msgr2=0x7f36c81105b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:06:37.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.713+0000 7f36c659c700 1 --2- 172.21.15.32:0/254505092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36c80fbcb0 0x7f36c81105b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:37.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.713+0000 7f36c659c700 1 -- 172.21.15.32:0/254505092 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36b8005040 con 0x7f36c80fcf60 2024-05-12T04:06:37.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.713+0000 7f36c659c700 1 --2- 172.21.15.32:0/254505092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36c80fcf60 0x7f36c8112b00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f36b000eb40 tx=0x7f36b000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:06:37.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.713+0000 7f36bf7fe700 1 -- 172.21.15.32:0/254505092 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36b0007cc0 con 0x7f36c80fcf60 2024-05-12T04:06:37.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.713+0000 7f36bf7fe700 1 -- 172.21.15.32:0/254505092 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f36b000f040 con 0x7f36c80fcf60 2024-05-12T04:06:37.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.713+0000 7f36cd208700 1 -- 172.21.15.32:0/254505092 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36c8113040 con 0x7f36c80fcf60 2024-05-12T04:06:37.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.713+0000 7f36bf7fe700 1 -- 172.21.15.32:0/254505092 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36b000c990 con 0x7f36c80fcf60 2024-05-12T04:06:37.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.713+0000 7f36cd208700 1 -- 172.21.15.32:0/254505092 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36c8113600 con 0x7f36c80fcf60 2024-05-12T04:06:37.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.714+0000 7f36cd208700 1 -- 172.21.15.32:0/254505092 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36c805f5c0 con 0x7f36c80fcf60 2024-05-12T04:06:37.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.715+0000 7f36bf7fe700 1 -- 172.21.15.32:0/254505092 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f36b000caf0 con 0x7f36c80fcf60 2024-05-12T04:06:37.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.715+0000 7f36bf7fe700 1 --2- 172.21.15.32:0/254505092 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f36b406cc20 0x7f36b406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:06:37.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.715+0000 7f36bf7fe700 1 -- 172.21.15.32:0/254505092 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f36b008bb50 con 0x7f36c80fcf60 2024-05-12T04:06:37.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.715+0000 7f36c6d9d700 1 --2- 172.21.15.32:0/254505092 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f36b406cc20 0x7f36b406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:06:37.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.716+0000 7f36c6d9d700 1 --2- 172.21.15.32:0/254505092 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f36b406cc20 0x7f36b406f0d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f36b8013680 tx=0x7f36b8009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:06:37.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.720+0000 7f36bf7fe700 1 -- 172.21.15.32:0/254505092 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f36b0055c90 con 0x7f36c80fcf60 2024-05-12T04:06:37.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:37 smithi033 ceph-mon[107305]: from='client.34312 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:06:37.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:37 smithi033 ceph-mon[107305]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:06:37.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:37 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/1095956574' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:06:37.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:37 smithi033 ceph-mon[107305]: from='osd.3 [v2:172.21.15.33:6800/3436738456,v1:172.21.15.33:6801/3436738456]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-12T04:06:37.913 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:37 smithi033 ceph-mon[107305]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-12T04:06:37.913 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:37 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/2645065361' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-12T04:06:37.913 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 12 04:06:37 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-3[120224]: 2024-05-12T04:06:37.438+0000 7fd2a73747c0 -1 osd.3 64 log_to_monitors true 2024-05-12T04:06:37.913 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 12 04:06:37 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-3[120224]: 2024-05-12T04:06:37.636+0000 7fd2987cd700 -1 osd.3 64 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-12T04:06:37.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.942+0000 7f36cd208700 1 -- 172.21.15.32:0/254505092 --> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f36c8101320 con 0x7f36b406cc20 2024-05-12T04:06:37.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.943+0000 7f36bf7fe700 1 -- 172.21.15.32:0/254505092 <== mgr.24593 v2:172.21.15.32:6800/2957111591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f36c8101320 con 0x7f36b406cc20 2024-05-12T04:06:37.945 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-05-12T04:06:37.945 INFO:teuthology.orchestra.run.smithi032.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:19c9734256aa43a60e203feb242f50bf8b7f58666d520536af43cc4fee96173a", 2024-05-12T04:06:37.945 INFO:teuthology.orchestra.run.smithi032.stdout: "in_progress": true, 2024-05-12T04:06:37.945 INFO:teuthology.orchestra.run.smithi032.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-12T04:06:37.945 INFO:teuthology.orchestra.run.smithi032.stdout: "services_complete": [ 2024-05-12T04:06:37.946 INFO:teuthology.orchestra.run.smithi032.stdout: "mon", 2024-05-12T04:06:37.946 INFO:teuthology.orchestra.run.smithi032.stdout: "crash", 2024-05-12T04:06:37.946 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr" 2024-05-12T04:06:37.946 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-05-12T04:06:37.946 INFO:teuthology.orchestra.run.smithi032.stdout: "progress": "10/21 daemons upgraded", 2024-05-12T04:06:37.946 INFO:teuthology.orchestra.run.smithi032.stdout: "message": "Currently upgrading osd daemons", 2024-05-12T04:06:37.946 INFO:teuthology.orchestra.run.smithi032.stdout: "is_paused": false 2024-05-12T04:06:37.946 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-05-12T04:06:37.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.946+0000 7f36cd208700 1 -- 172.21.15.32:0/254505092 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f36b406cc20 msgr2=0x7f36b406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:06:37.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.946+0000 7f36cd208700 1 --2- 172.21.15.32:0/254505092 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f36b406cc20 0x7f36b406f0d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f36b8013680 tx=0x7f36b8009e30 comp rx=0 tx=0).stop 2024-05-12T04:06:37.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.946+0000 7f36cd208700 1 -- 172.21.15.32:0/254505092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36c80fcf60 msgr2=0x7f36c8112b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:06:37.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.946+0000 7f36cd208700 1 --2- 172.21.15.32:0/254505092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36c80fcf60 0x7f36c8112b00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f36b000eb40 tx=0x7f36b000b040 comp rx=0 tx=0).stop 2024-05-12T04:06:37.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.946+0000 7f36cd208700 1 -- 172.21.15.32:0/254505092 shutdown_connections 2024-05-12T04:06:37.947 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.946+0000 7f36cd208700 1 --2- 172.21.15.32:0/254505092 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f36b406cc20 0x7f36b406f0d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:37.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.946+0000 7f36cd208700 1 --2- 172.21.15.32:0/254505092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36c80fbcb0 0x7f36c81105b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:37.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.946+0000 7f36cd208700 1 --2- 172.21.15.32:0/254505092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36c80fcf60 0x7f36c8112b00 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:37.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.946+0000 7f36cd208700 1 -- 172.21.15.32:0/254505092 >> 172.21.15.32:0/254505092 conn(0x7f36c80f7430 msgr2=0x7f36c80f8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:06:37.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.946+0000 7f36cd208700 1 -- 172.21.15.32:0/254505092 shutdown_connections 2024-05-12T04:06:37.948 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:37.946+0000 7f36cd208700 1 -- 172.21.15.32:0/254505092 wait complete. 2024-05-12T04:06:38.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:37 smithi032 ceph-mon[120392]: from='client.34312 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:06:38.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:37 smithi032 ceph-mon[120392]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:06:38.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:37 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/1095956574' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:06:38.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:37 smithi032 ceph-mon[120392]: from='osd.3 [v2:172.21.15.33:6800/3436738456,v1:172.21.15.33:6801/3436738456]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-12T04:06:38.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:37 smithi032 ceph-mon[120392]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-12T04:06:38.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:37 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/2645065361' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-12T04:06:38.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:38.050+0000 7f325ad96700 1 -- 172.21.15.32:0/2239952893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32540fbcb0 msgr2=0x7f32540fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:06:38.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:38.050+0000 7f325ad96700 1 --2- 172.21.15.32:0/2239952893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32540fbcb0 0x7f32540fc0c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3248004940 tx=0x7f3248009e30 comp rx=0 tx=0).stop 2024-05-12T04:06:38.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:38.050+0000 7f325ad96700 1 -- 172.21.15.32:0/2239952893 shutdown_connections 2024-05-12T04:06:38.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:38.050+0000 7f325ad96700 1 --2- 172.21.15.32:0/2239952893 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32540fcf60 0x7f32540fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:38.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:38.050+0000 7f325ad96700 1 --2- 172.21.15.32:0/2239952893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32540fbcb0 0x7f32540fc0c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:38.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:38.050+0000 7f325ad96700 1 -- 172.21.15.32:0/2239952893 >> 172.21.15.32:0/2239952893 conn(0x7f32540f7430 msgr2=0x7f32540f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:06:38.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:38.050+0000 7f325ad96700 1 -- 172.21.15.32:0/2239952893 shutdown_connections 2024-05-12T04:06:38.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:38.050+0000 7f325ad96700 1 -- 172.21.15.32:0/2239952893 wait complete. 2024-05-12T04:06:38.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:38.050+0000 7f325ad96700 1 Processor -- start 2024-05-12T04:06:38.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:38.050+0000 7f325ad96700 1 -- start start 2024-05-12T04:06:38.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:38.051+0000 7f325ad96700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32540fbcb0 0x7f32540ffd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:06:38.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:38.051+0000 7f325ad96700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32540fcf60 0x7f32540fe2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:06:38.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:38.051+0000 7f325ad96700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32540fe8c0 con 0x7f32540fbcb0 2024-05-12T04:06:38.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:38.051+0000 7f325ad96700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32540fea00 con 0x7f32540fcf60 2024-05-12T04:06:38.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:38.051+0000 7f3258b32700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32540fbcb0 0x7f32540ffd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:06:38.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:38.051+0000 7f3258b32700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32540fbcb0 0x7f32540ffd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35296/0 (socket says 172.21.15.32:35296) 2024-05-12T04:06:38.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:38.051+0000 7f3258b32700 1 -- 172.21.15.32:0/177987643 learned_addr learned my addr 172.21.15.32:0/177987643 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:06:38.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:38.051+0000 7f3253fff700 1 --2- 172.21.15.32:0/177987643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32540fcf60 0x7f32540fe2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:06:38.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:38.051+0000 7f3258b32700 1 -- 172.21.15.32:0/177987643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32540fcf60 msgr2=0x7f32540fe2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:06:38.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:38.051+0000 7f3258b32700 1 --2- 172.21.15.32:0/177987643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32540fcf60 0x7f32540fe2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:38.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:38.051+0000 7f3258b32700 1 -- 172.21.15.32:0/177987643 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3248005040 con 0x7f32540fbcb0 2024-05-12T04:06:38.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:38.051+0000 7f3253fff700 1 --2- 172.21.15.32:0/177987643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32540fcf60 0x7f32540fe2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-12T04:06:38.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:38.051+0000 7f3258b32700 1 --2- 172.21.15.32:0/177987643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32540fbcb0 0x7f32540ffd80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3248009fd0 tx=0x7f3248009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:06:38.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:38.051+0000 7f32517fa700 1 -- 172.21.15.32:0/177987643 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f324800e040 con 0x7f32540fbcb0 2024-05-12T04:06:38.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:38.051+0000 7f32517fa700 1 -- 172.21.15.32:0/177987643 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3248004290 con 0x7f32540fbcb0 2024-05-12T04:06:38.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:38.051+0000 7f325ad96700 1 -- 172.21.15.32:0/177987643 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32540fec80 con 0x7f32540fbcb0 2024-05-12T04:06:38.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:38.051+0000 7f32517fa700 1 -- 172.21.15.32:0/177987643 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f324801c3f0 con 0x7f32540fbcb0 2024-05-12T04:06:38.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:38.052+0000 7f325ad96700 1 -- 172.21.15.32:0/177987643 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3254115950 con 0x7f32540fbcb0 2024-05-12T04:06:38.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:38.052+0000 7f325ad96700 1 -- 172.21.15.32:0/177987643 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3254101af0 con 0x7f32540fbcb0 2024-05-12T04:06:38.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:38.053+0000 7f32517fa700 1 -- 172.21.15.32:0/177987643 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f324800bc50 con 0x7f32540fbcb0 2024-05-12T04:06:38.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:38.053+0000 7f32517fa700 1 --2- 172.21.15.32:0/177987643 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f324406cc30 0x7f324406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:06:38.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:38.053+0000 7f32517fa700 1 -- 172.21.15.32:0/177987643 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f324808d410 con 0x7f32540fbcb0 2024-05-12T04:06:38.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:38.053+0000 7f3253fff700 1 --2- 172.21.15.32:0/177987643 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f324406cc30 0x7f324406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:06:38.056 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:38.054+0000 7f3253fff700 1 --2- 172.21.15.32:0/177987643 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f324406cc30 0x7f324406f0e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3254100360 tx=0x7f3240008040 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:06:38.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:38.058+0000 7f32517fa700 1 -- 172.21.15.32:0/177987643 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f32480575f0 con 0x7f32540fbcb0 2024-05-12T04:06:38.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:38.342+0000 7f325ad96700 1 -- 172.21.15.32:0/177987643 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f325405f5c0 con 0x7f32540fbcb0 2024-05-12T04:06:38.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:38.342+0000 7f32517fa700 1 -- 172.21.15.32:0/177987643 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1461 (secure 0 0 0) 0x7f324805ac10 con 0x7f32540fbcb0 2024-05-12T04:06:38.347 INFO:teuthology.orchestra.run.smithi032.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down; Degraded data redundancy: 60/276 objects degraded (21.739%), 18 pgs degraded 2024-05-12T04:06:38.347 INFO:teuthology.orchestra.run.smithi032.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-12T04:06:38.347 INFO:teuthology.orchestra.run.smithi032.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-12T04:06:38.347 INFO:teuthology.orchestra.run.smithi032.stdout:[WRN] OSD_DOWN: 1 osds down 2024-05-12T04:06:38.347 INFO:teuthology.orchestra.run.smithi032.stdout: osd.3 (root=default,host=smithi033) is down 2024-05-12T04:06:38.347 INFO:teuthology.orchestra.run.smithi032.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 60/276 objects degraded (21.739%), 18 pgs degraded 2024-05-12T04:06:38.347 INFO:teuthology.orchestra.run.smithi032.stdout: pg 1.0 is active+undersized+degraded, acting [0,1] 2024-05-12T04:06:38.347 INFO:teuthology.orchestra.run.smithi032.stdout: pg 2.0 is active+undersized+degraded, acting [1,0] 2024-05-12T04:06:38.347 INFO:teuthology.orchestra.run.smithi032.stdout: pg 2.5 is active+undersized+degraded, acting [0,4] 2024-05-12T04:06:38.347 INFO:teuthology.orchestra.run.smithi032.stdout: pg 2.6 is active+undersized+degraded, acting [1,4] 2024-05-12T04:06:38.347 INFO:teuthology.orchestra.run.smithi032.stdout: pg 2.7 is active+undersized+degraded, acting [4,2] 2024-05-12T04:06:38.347 INFO:teuthology.orchestra.run.smithi032.stdout: pg 2.8 is active+undersized+degraded, acting [5,0] 2024-05-12T04:06:38.347 INFO:teuthology.orchestra.run.smithi032.stdout: pg 2.a is active+undersized+degraded, acting [1,4] 2024-05-12T04:06:38.348 INFO:teuthology.orchestra.run.smithi032.stdout: pg 2.b is active+undersized+degraded, acting [4,5] 2024-05-12T04:06:38.348 INFO:teuthology.orchestra.run.smithi032.stdout: pg 2.c is active+undersized+degraded, acting [2,0] 2024-05-12T04:06:38.348 INFO:teuthology.orchestra.run.smithi032.stdout: pg 2.d is active+undersized+degraded, acting [1,2] 2024-05-12T04:06:38.348 INFO:teuthology.orchestra.run.smithi032.stdout: pg 2.e is active+undersized+degraded, acting [2,0] 2024-05-12T04:06:38.348 INFO:teuthology.orchestra.run.smithi032.stdout: pg 2.14 is active+undersized+degraded, acting [4,5] 2024-05-12T04:06:38.348 INFO:teuthology.orchestra.run.smithi032.stdout: pg 2.15 is active+undersized+degraded, acting [1,0] 2024-05-12T04:06:38.348 INFO:teuthology.orchestra.run.smithi032.stdout: pg 2.16 is active+undersized+degraded, acting [5,2] 2024-05-12T04:06:38.348 INFO:teuthology.orchestra.run.smithi032.stdout: pg 2.18 is active+undersized+degraded, acting [5,4] 2024-05-12T04:06:38.348 INFO:teuthology.orchestra.run.smithi032.stdout: pg 2.1a is active+undersized+degraded, acting [4,5] 2024-05-12T04:06:38.348 INFO:teuthology.orchestra.run.smithi032.stdout: pg 2.1d is active+undersized+degraded, acting [5,0] 2024-05-12T04:06:38.348 INFO:teuthology.orchestra.run.smithi032.stdout: pg 2.1f is active+undersized+degraded, acting [0,4] 2024-05-12T04:06:38.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:38.344+0000 7f325ad96700 1 -- 172.21.15.32:0/177987643 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f324406cc30 msgr2=0x7f324406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:06:38.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:38.344+0000 7f325ad96700 1 --2- 172.21.15.32:0/177987643 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f324406cc30 0x7f324406f0e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3254100360 tx=0x7f3240008040 comp rx=0 tx=0).stop 2024-05-12T04:06:38.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:38.344+0000 7f325ad96700 1 -- 172.21.15.32:0/177987643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32540fbcb0 msgr2=0x7f32540ffd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:06:38.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:38.344+0000 7f325ad96700 1 --2- 172.21.15.32:0/177987643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32540fbcb0 0x7f32540ffd80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3248009fd0 tx=0x7f3248009e30 comp rx=0 tx=0).stop 2024-05-12T04:06:38.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:38.344+0000 7f325ad96700 1 -- 172.21.15.32:0/177987643 shutdown_connections 2024-05-12T04:06:38.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:38.344+0000 7f325ad96700 1 --2- 172.21.15.32:0/177987643 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f324406cc30 0x7f324406f0e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:38.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:38.344+0000 7f325ad96700 1 --2- 172.21.15.32:0/177987643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32540fbcb0 0x7f32540ffd80 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:38.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:38.344+0000 7f325ad96700 1 --2- 172.21.15.32:0/177987643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32540fcf60 0x7f32540fe2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:06:38.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:38.344+0000 7f325ad96700 1 -- 172.21.15.32:0/177987643 >> 172.21.15.32:0/177987643 conn(0x7f32540f7430 msgr2=0x7f3254065c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:06:38.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:38.344+0000 7f325ad96700 1 -- 172.21.15.32:0/177987643 shutdown_connections 2024-05-12T04:06:38.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:06:38.344+0000 7f325ad96700 1 -- 172.21.15.32:0/177987643 wait complete. 2024-05-12T04:06:38.753 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:38 smithi033 ceph-mon[107305]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-12T04:06:38.754 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:38 smithi033 ceph-mon[107305]: osdmap e67: 6 total, 5 up, 6 in 2024-05-12T04:06:38.754 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:38 smithi033 ceph-mon[107305]: from='osd.3 [v2:172.21.15.33:6800/3436738456,v1:172.21.15.33:6801/3436738456]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-05-12T04:06:38.754 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:38 smithi033 ceph-mon[107305]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-05-12T04:06:38.754 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:38 smithi033 ceph-mon[107305]: pgmap v122: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 260 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 60/276 objects degraded (21.739%) 2024-05-12T04:06:38.754 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:38 smithi033 ceph-mon[107305]: from='client.34330 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:06:38.754 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:38 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/177987643' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-12T04:06:39.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:38 smithi032 ceph-mon[120392]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-12T04:06:39.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:38 smithi032 ceph-mon[120392]: osdmap e67: 6 total, 5 up, 6 in 2024-05-12T04:06:39.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:38 smithi032 ceph-mon[120392]: from='osd.3 [v2:172.21.15.33:6800/3436738456,v1:172.21.15.33:6801/3436738456]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-05-12T04:06:39.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:38 smithi032 ceph-mon[120392]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-05-12T04:06:39.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:38 smithi032 ceph-mon[120392]: pgmap v122: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 260 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 60/276 objects degraded (21.739%) 2024-05-12T04:06:39.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:38 smithi032 ceph-mon[120392]: from='client.34330 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:06:39.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:38 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/177987643' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-12T04:06:39.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:39 smithi033 ceph-mon[107305]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-12T04:06:39.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:39 smithi033 ceph-mon[107305]: osd.3 [v2:172.21.15.33:6800/3436738456,v1:172.21.15.33:6801/3436738456] boot 2024-05-12T04:06:39.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:39 smithi033 ceph-mon[107305]: osdmap e68: 6 total, 6 up, 6 in 2024-05-12T04:06:39.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:39 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-12T04:06:39.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:39 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:06:39.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:39 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:06:39.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:39 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:06:39.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:39 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T04:06:39.913 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:39 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:06:39.913 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:39 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:06:39.913 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:39 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:06:39.913 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:39 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:06:39.913 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:39 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:06:39.913 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:39 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-05-12T04:06:39.913 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:39 smithi033 ceph-mon[107305]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-05-12T04:06:39.913 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:39 smithi033 ceph-mon[107305]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-05-12T04:06:40.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:39 smithi032 ceph-mon[120392]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-12T04:06:40.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:39 smithi032 ceph-mon[120392]: osd.3 [v2:172.21.15.33:6800/3436738456,v1:172.21.15.33:6801/3436738456] boot 2024-05-12T04:06:40.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:39 smithi032 ceph-mon[120392]: osdmap e68: 6 total, 6 up, 6 in 2024-05-12T04:06:40.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:39 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-12T04:06:40.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:39 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:06:40.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:39 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:06:40.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:39 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:06:40.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:39 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T04:06:40.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:39 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:06:40.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:39 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:06:40.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:39 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:06:40.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:39 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:06:40.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:39 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:06:40.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:39 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-05-12T04:06:40.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:39 smithi032 ceph-mon[120392]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-05-12T04:06:40.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:39 smithi032 ceph-mon[120392]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-05-12T04:06:40.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:40 smithi033 ceph-mon[107305]: osdmap e69: 6 total, 6 up, 6 in 2024-05-12T04:06:40.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:40 smithi033 ceph-mon[107305]: pgmap v125: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 260 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 60/276 objects degraded (21.739%) 2024-05-12T04:06:40.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:40 smithi033 ceph-mon[107305]: Health check update: Degraded data redundancy: 60/276 objects degraded (21.739%), 18 pgs degraded (PG_DEGRADED) 2024-05-12T04:06:41.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:40 smithi032 ceph-mon[120392]: osdmap e69: 6 total, 6 up, 6 in 2024-05-12T04:06:41.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:40 smithi032 ceph-mon[120392]: pgmap v125: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 260 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 60/276 objects degraded (21.739%) 2024-05-12T04:06:41.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:40 smithi032 ceph-mon[120392]: Health check update: Degraded data redundancy: 60/276 objects degraded (21.739%), 18 pgs degraded (PG_DEGRADED) 2024-05-12T04:06:43.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:42 smithi033 ceph-mon[107305]: pgmap v126: 65 pgs: 15 active+undersized+degraded, 11 active+undersized, 39 active+clean; 260 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s; 47/276 objects degraded (17.029%) 2024-05-12T04:06:43.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:42 smithi032 ceph-mon[120392]: pgmap v126: 65 pgs: 15 active+undersized+degraded, 11 active+undersized, 39 active+clean; 260 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s; 47/276 objects degraded (17.029%) 2024-05-12T04:06:44.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:43 smithi033 ceph-mon[107305]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 47/276 objects degraded (17.029%), 15 pgs degraded) 2024-05-12T04:06:44.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:43 smithi032 ceph-mon[120392]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 47/276 objects degraded (17.029%), 15 pgs degraded) 2024-05-12T04:06:45.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:44 smithi033 ceph-mon[107305]: pgmap v127: 65 pgs: 65 active+clean; 260 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 667 B/s rd, 1 op/s 2024-05-12T04:06:45.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:44 smithi032 ceph-mon[120392]: pgmap v127: 65 pgs: 65 active+clean; 260 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 667 B/s rd, 1 op/s 2024-05-12T04:06:47.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:46 smithi033 ceph-mon[107305]: pgmap v128: 65 pgs: 65 active+clean; 260 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 511 B/s rd, 1 op/s 2024-05-12T04:06:47.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:46 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:06:47.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:46 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:06:47.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:46 smithi032 ceph-mon[120392]: pgmap v128: 65 pgs: 65 active+clean; 260 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 511 B/s rd, 1 op/s 2024-05-12T04:06:47.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:46 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:06:47.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:46 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:06:49.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:48 smithi033 ceph-mon[107305]: pgmap v129: 65 pgs: 65 active+clean; 260 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1008 B/s rd, 2 op/s 2024-05-12T04:06:49.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:48 smithi032 ceph-mon[120392]: pgmap v129: 65 pgs: 65 active+clean; 260 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1008 B/s rd, 2 op/s 2024-05-12T04:06:51.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:50 smithi033 ceph-mon[107305]: pgmap v130: 65 pgs: 65 active+clean; 260 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 909 B/s rd, 1 op/s 2024-05-12T04:06:51.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:50 smithi032 ceph-mon[120392]: pgmap v130: 65 pgs: 65 active+clean; 260 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 909 B/s rd, 1 op/s 2024-05-12T04:06:52.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:51 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:06:52.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:51 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:06:53.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:52 smithi033 ceph-mon[107305]: pgmap v131: 65 pgs: 65 active+clean; 260 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-05-12T04:06:53.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:52 smithi032 ceph-mon[120392]: pgmap v131: 65 pgs: 65 active+clean; 260 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-05-12T04:06:55.088 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:54 smithi033 ceph-mon[107305]: pgmap v132: 65 pgs: 65 active+clean; 260 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-05-12T04:06:55.088 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:54 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-05-12T04:06:55.088 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:54 smithi033 ceph-mon[107305]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-05-12T04:06:55.088 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:54 smithi033 ceph-mon[107305]: Upgrade: osd.4 is safe to restart 2024-05-12T04:06:55.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:54 smithi032 ceph-mon[120392]: pgmap v132: 65 pgs: 65 active+clean; 260 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-05-12T04:06:55.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:54 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-05-12T04:06:55.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:54 smithi032 ceph-mon[120392]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-05-12T04:06:55.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:54 smithi032 ceph-mon[120392]: Upgrade: osd.4 is safe to restart 2024-05-12T04:06:57.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:56 smithi033 ceph-mon[107305]: pgmap v133: 65 pgs: 65 active+clean; 260 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:06:57.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:56 smithi033 ceph-mon[107305]: Upgrade: Updating osd.4 2024-05-12T04:06:57.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:56 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:06:57.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:56 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-12T04:06:57.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:56 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:06:57.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:56 smithi033 ceph-mon[107305]: Deploying daemon osd.4 on smithi033 2024-05-12T04:06:57.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:56 smithi032 ceph-mon[120392]: pgmap v133: 65 pgs: 65 active+clean; 260 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:06:57.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:56 smithi032 ceph-mon[120392]: Upgrade: Updating osd.4 2024-05-12T04:06:57.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:56 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:06:57.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:56 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-12T04:06:57.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:56 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:06:57.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:56 smithi032 ceph-mon[120392]: Deploying daemon osd.4 on smithi033 2024-05-12T04:06:59.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:58 smithi033 ceph-mon[107305]: pgmap v134: 65 pgs: 65 active+clean; 260 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:06:59.162 INFO:journalctl@ceph.osd.4.smithi033.stdout:May 12 04:06:58 smithi033 systemd[1]: Stopping Ceph osd.4 for a3e178f8-1012-11ef-bc98-c7b262605968... 2024-05-12T04:06:59.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:58 smithi032 ceph-mon[120392]: pgmap v134: 65 pgs: 65 active+clean; 260 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:06:59.662 INFO:journalctl@ceph.osd.4.smithi033.stdout:May 12 04:06:59 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-4[64116]: 2024-05-12T04:06:59.168+0000 7fbb5d0e7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-12T04:06:59.662 INFO:journalctl@ceph.osd.4.smithi033.stdout:May 12 04:06:59 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-4[64116]: 2024-05-12T04:06:59.168+0000 7fbb5d0e7700 -1 osd.4 69 *** Got signal Terminated *** 2024-05-12T04:06:59.662 INFO:journalctl@ceph.osd.4.smithi033.stdout:May 12 04:06:59 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-4[64116]: 2024-05-12T04:06:59.168+0000 7fbb5d0e7700 -1 osd.4 69 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-12T04:07:00.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:06:59 smithi033 ceph-mon[107305]: osd.4 marked itself down and dead 2024-05-12T04:07:00.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:06:59 smithi032 ceph-mon[120392]: osd.4 marked itself down and dead 2024-05-12T04:07:00.894 INFO:journalctl@ceph.osd.4.smithi033.stdout:May 12 04:07:00 smithi033 bash[125782]: ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-4 2024-05-12T04:07:01.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:00 smithi033 ceph-mon[107305]: pgmap v135: 65 pgs: 65 active+clean; 260 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:07:01.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:00 smithi033 ceph-mon[107305]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-12T04:07:01.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:00 smithi033 ceph-mon[107305]: osdmap e70: 6 total, 5 up, 6 in 2024-05-12T04:07:01.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:00 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:07:01.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:00 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:07:01.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:00 smithi032 ceph-mon[120392]: pgmap v135: 65 pgs: 65 active+clean; 260 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:07:01.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:00 smithi032 ceph-mon[120392]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-12T04:07:01.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:00 smithi032 ceph-mon[120392]: osdmap e70: 6 total, 5 up, 6 in 2024-05-12T04:07:01.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:00 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:07:01.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:00 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:07:02.116 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:01 smithi033 ceph-mon[107305]: osdmap e71: 6 total, 5 up, 6 in 2024-05-12T04:07:02.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:01 smithi032 ceph-mon[120392]: osdmap e71: 6 total, 5 up, 6 in 2024-05-12T04:07:02.662 INFO:journalctl@ceph.osd.4.smithi033.stdout:May 12 04:07:02 smithi033 systemd[1]: ceph-a3e178f8-1012-11ef-bc98-c7b262605968@osd.4.service: Succeeded. 2024-05-12T04:07:02.662 INFO:journalctl@ceph.osd.4.smithi033.stdout:May 12 04:07:02 smithi033 systemd[1]: Stopped Ceph osd.4 for a3e178f8-1012-11ef-bc98-c7b262605968. 2024-05-12T04:07:02.662 INFO:journalctl@ceph.osd.4.smithi033.stdout:May 12 04:07:02 smithi033 systemd[1]: Starting Ceph osd.4 for a3e178f8-1012-11ef-bc98-c7b262605968... 2024-05-12T04:07:03.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:02 smithi033 ceph-mon[107305]: pgmap v138: 65 pgs: 6 active+undersized+degraded, 3 stale+active+clean, 7 active+undersized, 49 active+clean; 260 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 0 op/s; 19/276 objects degraded (6.884%) 2024-05-12T04:07:03.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:02 smithi033 ceph-mon[107305]: Health check failed: Degraded data redundancy: 19/276 objects degraded (6.884%), 6 pgs degraded (PG_DEGRADED) 2024-05-12T04:07:03.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:02 smithi032 ceph-mon[120392]: pgmap v138: 65 pgs: 6 active+undersized+degraded, 3 stale+active+clean, 7 active+undersized, 49 active+clean; 260 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 0 op/s; 19/276 objects degraded (6.884%) 2024-05-12T04:07:03.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:02 smithi032 ceph-mon[120392]: Health check failed: Degraded data redundancy: 19/276 objects degraded (6.884%), 6 pgs degraded (PG_DEGRADED) 2024-05-12T04:07:03.740 INFO:journalctl@ceph.osd.4.smithi033.stdout:May 12 04:07:03 smithi033 bash[126054]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-12T04:07:03.741 INFO:journalctl@ceph.osd.4.smithi033.stdout:May 12 04:07:03 smithi033 bash[126054]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-05-12T04:07:03.741 INFO:journalctl@ceph.osd.4.smithi033.stdout:May 12 04:07:03 smithi033 bash[126054]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-05-12T04:07:03.741 INFO:journalctl@ceph.osd.4.smithi033.stdout:May 12 04:07:03 smithi033 bash[126054]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-05-12T04:07:03.741 INFO:journalctl@ceph.osd.4.smithi033.stdout:May 12 04:07:03 smithi033 bash[126054]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-05-12T04:07:03.741 INFO:journalctl@ceph.osd.4.smithi033.stdout:May 12 04:07:03 smithi033 bash[126054]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-12T04:07:03.741 INFO:journalctl@ceph.osd.4.smithi033.stdout:May 12 04:07:03 smithi033 bash[126054]: --> ceph-volume raw activate successful for osd ID: 4 2024-05-12T04:07:04.906 INFO:journalctl@ceph.osd.4.smithi033.stdout:May 12 04:07:04 smithi033 ceph-osd[126374]: -- 172.21.15.33:0/1048412536 <== mon.1 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f9b61591e0 con 0x55f9b6129000 2024-05-12T04:07:04.906 INFO:journalctl@ceph.osd.4.smithi033.stdout:May 12 04:07:04 smithi033 bash[126335]: db6fda7d52de3b809fe305c3827cdf642b3d6e73f56c7407011935698aeebf8f 2024-05-12T04:07:04.906 INFO:journalctl@ceph.osd.4.smithi033.stdout:May 12 04:07:04 smithi033 systemd[1]: Started Ceph osd.4 for a3e178f8-1012-11ef-bc98-c7b262605968. 2024-05-12T04:07:05.162 INFO:journalctl@ceph.osd.4.smithi033.stdout:May 12 04:07:05 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-4[126351]: 2024-05-12T04:07:05.119+0000 7fbd0dfc07c0 -1 Falling back to public interface 2024-05-12T04:07:05.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:04 smithi033 ceph-mon[107305]: pgmap v139: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 260 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 0 op/s; 49/276 objects degraded (17.754%) 2024-05-12T04:07:05.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:04 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:07:05.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:04 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:07:05.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:04 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:07:05.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:04 smithi032 ceph-mon[120392]: pgmap v139: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 260 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 0 op/s; 49/276 objects degraded (17.754%) 2024-05-12T04:07:05.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:04 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:07:05.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:04 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:07:05.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:04 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:07:07.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:06 smithi033 ceph-mon[107305]: pgmap v140: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 260 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 49/276 objects degraded (17.754%) 2024-05-12T04:07:07.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:06 smithi032 ceph-mon[120392]: pgmap v140: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 260 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 49/276 objects degraded (17.754%) 2024-05-12T04:07:08.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.465+0000 7f68183f7700 1 -- 172.21.15.32:0/3981398147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68100fbcb0 msgr2=0x7f68100fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:07:08.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.465+0000 7f68183f7700 1 --2- 172.21.15.32:0/3981398147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68100fbcb0 0x7f68100fc0c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f680c004940 tx=0x7f680c009e30 comp rx=0 tx=0).stop 2024-05-12T04:07:08.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.465+0000 7f68183f7700 1 -- 172.21.15.32:0/3981398147 shutdown_connections 2024-05-12T04:07:08.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.466+0000 7f68183f7700 1 --2- 172.21.15.32:0/3981398147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68100fcf60 0x7f68100fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:08.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.466+0000 7f68183f7700 1 --2- 172.21.15.32:0/3981398147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68100fbcb0 0x7f68100fc0c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:08.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.466+0000 7f68183f7700 1 -- 172.21.15.32:0/3981398147 >> 172.21.15.32:0/3981398147 conn(0x7f68100f7430 msgr2=0x7f68100f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:07:08.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.466+0000 7f68183f7700 1 -- 172.21.15.32:0/3981398147 shutdown_connections 2024-05-12T04:07:08.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.466+0000 7f68183f7700 1 -- 172.21.15.32:0/3981398147 wait complete. 2024-05-12T04:07:08.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.466+0000 7f68183f7700 1 Processor -- start 2024-05-12T04:07:08.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.466+0000 7f68183f7700 1 -- start start 2024-05-12T04:07:08.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.467+0000 7f68183f7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68100fbcb0 0x7f681010d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:07:08.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.467+0000 7f68183f7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68100fcf60 0x7f681010de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:07:08.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.467+0000 7f68183f7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f681010e410 con 0x7f68100fbcb0 2024-05-12T04:07:08.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.467+0000 7f68183f7700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f681010ec10 con 0x7f68100fcf60 2024-05-12T04:07:08.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.467+0000 7f6816193700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68100fbcb0 0x7f681010d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:07:08.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.467+0000 7f6816193700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68100fbcb0 0x7f681010d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44312/0 (socket says 172.21.15.32:44312) 2024-05-12T04:07:08.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.467+0000 7f6816193700 1 -- 172.21.15.32:0/2923597149 learned_addr learned my addr 172.21.15.32:0/2923597149 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:07:08.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.467+0000 7f6815992700 1 --2- 172.21.15.32:0/2923597149 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68100fcf60 0x7f681010de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:07:08.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.467+0000 7f6816193700 1 -- 172.21.15.32:0/2923597149 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68100fcf60 msgr2=0x7f681010de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:07:08.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.467+0000 7f6816193700 1 --2- 172.21.15.32:0/2923597149 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68100fcf60 0x7f681010de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:08.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.467+0000 7f6816193700 1 -- 172.21.15.32:0/2923597149 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f680c005040 con 0x7f68100fbcb0 2024-05-12T04:07:08.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.467+0000 7f6816193700 1 --2- 172.21.15.32:0/2923597149 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68100fbcb0 0x7f681010d8d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f680c0137b0 tx=0x7f680c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:07:08.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.467+0000 7f6802ffd700 1 -- 172.21.15.32:0/2923597149 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f680c00e070 con 0x7f68100fbcb0 2024-05-12T04:07:08.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.468+0000 7f68183f7700 1 -- 172.21.15.32:0/2923597149 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f681010ee90 con 0x7f68100fbcb0 2024-05-12T04:07:08.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.468+0000 7f6802ffd700 1 -- 172.21.15.32:0/2923597149 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f680c01c400 con 0x7f68100fbcb0 2024-05-12T04:07:08.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.468+0000 7f6802ffd700 1 -- 172.21.15.32:0/2923597149 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f680c00b650 con 0x7f68100fbcb0 2024-05-12T04:07:08.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.468+0000 7f68183f7700 1 -- 172.21.15.32:0/2923597149 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f681010f370 con 0x7f68100fbcb0 2024-05-12T04:07:08.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.469+0000 7f68183f7700 1 -- 172.21.15.32:0/2923597149 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6810101af0 con 0x7f68100fbcb0 2024-05-12T04:07:08.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.469+0000 7f6802ffd700 1 -- 172.21.15.32:0/2923597149 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f680c01c9e0 con 0x7f68100fbcb0 2024-05-12T04:07:08.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.469+0000 7f6802ffd700 1 --2- 172.21.15.32:0/2923597149 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f67fc06cc20 0x7f67fc06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:07:08.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.470+0000 7f6802ffd700 1 -- 172.21.15.32:0/2923597149 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f680c08d610 con 0x7f68100fbcb0 2024-05-12T04:07:08.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.470+0000 7f6815992700 1 --2- 172.21.15.32:0/2923597149 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f67fc06cc20 0x7f67fc06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:07:08.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.470+0000 7f6815992700 1 --2- 172.21.15.32:0/2923597149 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f67fc06cc20 0x7f67fc06f0d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6804009b80 tx=0x7f680400b040 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:07:08.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.474+0000 7f6802ffd700 1 -- 172.21.15.32:0/2923597149 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f680c05b130 con 0x7f68100fbcb0 2024-05-12T04:07:08.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.704+0000 7f68183f7700 1 -- 172.21.15.32:0/2923597149 --> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f681010f590 con 0x7f67fc06cc20 2024-05-12T04:07:08.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.705+0000 7f6802ffd700 1 -- 172.21.15.32:0/2923597149 <== mgr.24593 v2:172.21.15.32:6800/2957111591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f681010f590 con 0x7f67fc06cc20 2024-05-12T04:07:08.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.707+0000 7f68183f7700 1 -- 172.21.15.32:0/2923597149 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f67fc06cc20 msgr2=0x7f67fc06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:07:08.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.707+0000 7f68183f7700 1 --2- 172.21.15.32:0/2923597149 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f67fc06cc20 0x7f67fc06f0d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6804009b80 tx=0x7f680400b040 comp rx=0 tx=0).stop 2024-05-12T04:07:08.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.707+0000 7f68183f7700 1 -- 172.21.15.32:0/2923597149 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68100fbcb0 msgr2=0x7f681010d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:07:08.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.707+0000 7f68183f7700 1 --2- 172.21.15.32:0/2923597149 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68100fbcb0 0x7f681010d8d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f680c0137b0 tx=0x7f680c009e30 comp rx=0 tx=0).stop 2024-05-12T04:07:08.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.707+0000 7f68183f7700 1 -- 172.21.15.32:0/2923597149 shutdown_connections 2024-05-12T04:07:08.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.707+0000 7f68183f7700 1 --2- 172.21.15.32:0/2923597149 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f67fc06cc20 0x7f67fc06f0d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:08.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.707+0000 7f68183f7700 1 --2- 172.21.15.32:0/2923597149 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68100fbcb0 0x7f681010d8d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:08.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.707+0000 7f68183f7700 1 --2- 172.21.15.32:0/2923597149 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68100fcf60 0x7f681010de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:08.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.707+0000 7f68183f7700 1 -- 172.21.15.32:0/2923597149 >> 172.21.15.32:0/2923597149 conn(0x7f68100f7430 msgr2=0x7f6810065c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:07:08.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.707+0000 7f68183f7700 1 -- 172.21.15.32:0/2923597149 shutdown_connections 2024-05-12T04:07:08.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.707+0000 7f68183f7700 1 -- 172.21.15.32:0/2923597149 wait complete. 2024-05-12T04:07:08.718 INFO:teuthology.orchestra.run.smithi032.stdout:true 2024-05-12T04:07:08.810 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:08 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:07:08.810 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:08 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:07:08.810 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:08 smithi033 ceph-mon[107305]: pgmap v141: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 260 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 0 op/s; 49/276 objects degraded (17.754%) 2024-05-12T04:07:08.810 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:08 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:07:08.811 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:08 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:07:08.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.812+0000 7f7ae1ed6700 1 -- 172.21.15.32:0/671974987 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7adc0fbcb0 msgr2=0x7f7adc0fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:07:08.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.812+0000 7f7ae1ed6700 1 --2- 172.21.15.32:0/671974987 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7adc0fbcb0 0x7f7adc0fc0c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7acc004940 tx=0x7f7acc009e30 comp rx=0 tx=0).stop 2024-05-12T04:07:08.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.812+0000 7f7ae1ed6700 1 -- 172.21.15.32:0/671974987 shutdown_connections 2024-05-12T04:07:08.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.812+0000 7f7ae1ed6700 1 --2- 172.21.15.32:0/671974987 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7adc0fcf60 0x7f7adc0fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:08.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.812+0000 7f7ae1ed6700 1 --2- 172.21.15.32:0/671974987 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7adc0fbcb0 0x7f7adc0fc0c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:08.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.812+0000 7f7ae1ed6700 1 -- 172.21.15.32:0/671974987 >> 172.21.15.32:0/671974987 conn(0x7f7adc0f7430 msgr2=0x7f7adc0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:07:08.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.812+0000 7f7ae1ed6700 1 -- 172.21.15.32:0/671974987 shutdown_connections 2024-05-12T04:07:08.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.812+0000 7f7ae1ed6700 1 -- 172.21.15.32:0/671974987 wait complete. 2024-05-12T04:07:08.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.813+0000 7f7ae1ed6700 1 Processor -- start 2024-05-12T04:07:08.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.813+0000 7f7ae1ed6700 1 -- start start 2024-05-12T04:07:08.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.813+0000 7f7ae1ed6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7adc0fbcb0 0x7f7adc110640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:07:08.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.813+0000 7f7ae1ed6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7adc0fcf60 0x7f7adc112b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:07:08.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.813+0000 7f7ae1ed6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7adc066330 con 0x7f7adc0fcf60 2024-05-12T04:07:08.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.813+0000 7f7ae1ed6700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7adc066470 con 0x7f7adc0fbcb0 2024-05-12T04:07:08.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.813+0000 7f7adaffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7adc0fcf60 0x7f7adc112b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:07:08.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.813+0000 7f7adaffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7adc0fcf60 0x7f7adc112b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44342/0 (socket says 172.21.15.32:44342) 2024-05-12T04:07:08.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.813+0000 7f7adaffd700 1 -- 172.21.15.32:0/3873057516 learned_addr learned my addr 172.21.15.32:0/3873057516 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:07:08.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.813+0000 7f7adb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7adc0fbcb0 0x7f7adc110640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:07:08.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.814+0000 7f7adaffd700 1 -- 172.21.15.32:0/3873057516 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7adc0fbcb0 msgr2=0x7f7adc110640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:07:08.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.814+0000 7f7adaffd700 1 --2- 172.21.15.32:0/3873057516 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7adc0fbcb0 0x7f7adc110640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:08.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.814+0000 7f7adaffd700 1 -- 172.21.15.32:0/3873057516 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7acc005040 con 0x7f7adc0fcf60 2024-05-12T04:07:08.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.814+0000 7f7adaffd700 1 --2- 172.21.15.32:0/3873057516 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7adc0fcf60 0x7f7adc112b90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7ad0002a50 tx=0x7f7ad00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:07:08.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.814+0000 7f7ac3fff700 1 -- 172.21.15.32:0/3873057516 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ad0007cc0 con 0x7f7adc0fcf60 2024-05-12T04:07:08.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.814+0000 7f7ae1ed6700 1 -- 172.21.15.32:0/3873057516 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7adc113130 con 0x7f7adc0fcf60 2024-05-12T04:07:08.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.814+0000 7f7ac3fff700 1 -- 172.21.15.32:0/3873057516 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7ad000f040 con 0x7f7adc0fcf60 2024-05-12T04:07:08.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.814+0000 7f7ac3fff700 1 -- 172.21.15.32:0/3873057516 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ad000d9c0 con 0x7f7adc0fcf60 2024-05-12T04:07:08.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.814+0000 7f7ae1ed6700 1 -- 172.21.15.32:0/3873057516 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7adc1136f0 con 0x7f7adc0fcf60 2024-05-12T04:07:08.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.815+0000 7f7ae1ed6700 1 -- 172.21.15.32:0/3873057516 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7adc05f5c0 con 0x7f7adc0fcf60 2024-05-12T04:07:08.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.816+0000 7f7ac3fff700 1 -- 172.21.15.32:0/3873057516 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f7ad000db20 con 0x7f7adc0fcf60 2024-05-12T04:07:08.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.816+0000 7f7ac3fff700 1 --2- 172.21.15.32:0/3873057516 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f7ac406cc20 0x7f7ac406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:07:08.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.816+0000 7f7ac3fff700 1 -- 172.21.15.32:0/3873057516 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f7ad008bd20 con 0x7f7adc0fcf60 2024-05-12T04:07:08.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.816+0000 7f7adb7fe700 1 --2- 172.21.15.32:0/3873057516 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f7ac406cc20 0x7f7ac406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:07:08.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.816+0000 7f7adb7fe700 1 --2- 172.21.15.32:0/3873057516 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f7ac406cc20 0x7f7ac406f0d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7acc004890 tx=0x7f7acc009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:07:08.821 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:08.820+0000 7f7ac3fff700 1 -- 172.21.15.32:0/3873057516 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f7ad0055d30 con 0x7f7adc0fcf60 2024-05-12T04:07:09.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:08 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:07:09.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:08 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:07:09.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:08 smithi032 ceph-mon[120392]: pgmap v141: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 260 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 0 op/s; 49/276 objects degraded (17.754%) 2024-05-12T04:07:09.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:08 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:07:09.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:08 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:07:09.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.045+0000 7f7ae1ed6700 1 -- 172.21.15.32:0/3873057516 --> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7adc1013d0 con 0x7f7ac406cc20 2024-05-12T04:07:09.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.047+0000 7f7ac3fff700 1 -- 172.21.15.32:0/3873057516 <== mgr.24593 v2:172.21.15.32:6800/2957111591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f7adc1013d0 con 0x7f7ac406cc20 2024-05-12T04:07:09.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.049+0000 7f7ae1ed6700 1 -- 172.21.15.32:0/3873057516 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f7ac406cc20 msgr2=0x7f7ac406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:07:09.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.049+0000 7f7ae1ed6700 1 --2- 172.21.15.32:0/3873057516 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f7ac406cc20 0x7f7ac406f0d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7acc004890 tx=0x7f7acc009e30 comp rx=0 tx=0).stop 2024-05-12T04:07:09.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.049+0000 7f7ae1ed6700 1 -- 172.21.15.32:0/3873057516 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7adc0fcf60 msgr2=0x7f7adc112b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:07:09.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.049+0000 7f7ae1ed6700 1 --2- 172.21.15.32:0/3873057516 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7adc0fcf60 0x7f7adc112b90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7ad0002a50 tx=0x7f7ad00046b0 comp rx=0 tx=0).stop 2024-05-12T04:07:09.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.049+0000 7f7ae1ed6700 1 -- 172.21.15.32:0/3873057516 shutdown_connections 2024-05-12T04:07:09.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.049+0000 7f7ae1ed6700 1 --2- 172.21.15.32:0/3873057516 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f7ac406cc20 0x7f7ac406f0d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:09.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.049+0000 7f7ae1ed6700 1 --2- 172.21.15.32:0/3873057516 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7adc0fbcb0 0x7f7adc110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:09.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.049+0000 7f7ae1ed6700 1 --2- 172.21.15.32:0/3873057516 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7adc0fcf60 0x7f7adc112b90 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:09.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.049+0000 7f7ae1ed6700 1 -- 172.21.15.32:0/3873057516 >> 172.21.15.32:0/3873057516 conn(0x7f7adc0f7430 msgr2=0x7f7adc0f8c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:07:09.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.049+0000 7f7ae1ed6700 1 -- 172.21.15.32:0/3873057516 shutdown_connections 2024-05-12T04:07:09.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.049+0000 7f7ae1ed6700 1 -- 172.21.15.32:0/3873057516 wait complete. 2024-05-12T04:07:09.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.153+0000 7f2d8b39d700 1 -- 172.21.15.32:0/3017089070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d840fbcb0 msgr2=0x7f2d840fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:07:09.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.153+0000 7f2d8b39d700 1 --2- 172.21.15.32:0/3017089070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d840fbcb0 0x7f2d840fc0c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2d74004940 tx=0x7f2d74009e30 comp rx=0 tx=0).stop 2024-05-12T04:07:09.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.153+0000 7f2d8b39d700 1 -- 172.21.15.32:0/3017089070 shutdown_connections 2024-05-12T04:07:09.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.153+0000 7f2d8b39d700 1 --2- 172.21.15.32:0/3017089070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d840fcf60 0x7f2d840fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:09.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.153+0000 7f2d8b39d700 1 --2- 172.21.15.32:0/3017089070 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d840fbcb0 0x7f2d840fc0c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:09.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.153+0000 7f2d8b39d700 1 -- 172.21.15.32:0/3017089070 >> 172.21.15.32:0/3017089070 conn(0x7f2d840f7430 msgr2=0x7f2d840f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:07:09.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.153+0000 7f2d8b39d700 1 -- 172.21.15.32:0/3017089070 shutdown_connections 2024-05-12T04:07:09.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.153+0000 7f2d8b39d700 1 -- 172.21.15.32:0/3017089070 wait complete. 2024-05-12T04:07:09.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.153+0000 7f2d8b39d700 1 Processor -- start 2024-05-12T04:07:09.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.153+0000 7f2d8b39d700 1 -- start start 2024-05-12T04:07:09.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.154+0000 7f2d8b39d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d840fbcb0 0x7f2d84110620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:07:09.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.154+0000 7f2d8b39d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d840fcf60 0x7f2d84112b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:07:09.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.154+0000 7f2d8b39d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d84066330 con 0x7f2d840fbcb0 2024-05-12T04:07:09.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.154+0000 7f2d8b39d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d84066470 con 0x7f2d840fcf60 2024-05-12T04:07:09.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.154+0000 7f2d89139700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d840fbcb0 0x7f2d84110620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:07:09.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.154+0000 7f2d88938700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d840fcf60 0x7f2d84112b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:07:09.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.154+0000 7f2d89139700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d840fbcb0 0x7f2d84110620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44352/0 (socket says 172.21.15.32:44352) 2024-05-12T04:07:09.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.154+0000 7f2d89139700 1 -- 172.21.15.32:0/1682812395 learned_addr learned my addr 172.21.15.32:0/1682812395 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:07:09.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.154+0000 7f2d89139700 1 -- 172.21.15.32:0/1682812395 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d840fcf60 msgr2=0x7f2d84112b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:07:09.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.154+0000 7f2d89139700 1 --2- 172.21.15.32:0/1682812395 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d840fcf60 0x7f2d84112b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:09.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.154+0000 7f2d89139700 1 -- 172.21.15.32:0/1682812395 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d74005040 con 0x7f2d840fbcb0 2024-05-12T04:07:09.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.154+0000 7f2d89139700 1 --2- 172.21.15.32:0/1682812395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d840fbcb0 0x7f2d84110620 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2d74009380 tx=0x7f2d74009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:07:09.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.154+0000 7f2d79ffb700 1 -- 172.21.15.32:0/1682812395 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d7400e040 con 0x7f2d840fbcb0 2024-05-12T04:07:09.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.154+0000 7f2d79ffb700 1 -- 172.21.15.32:0/1682812395 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2d7400bd20 con 0x7f2d840fbcb0 2024-05-12T04:07:09.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.154+0000 7f2d8b39d700 1 -- 172.21.15.32:0/1682812395 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d841130b0 con 0x7f2d840fbcb0 2024-05-12T04:07:09.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.154+0000 7f2d79ffb700 1 -- 172.21.15.32:0/1682812395 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d7401c630 con 0x7f2d840fbcb0 2024-05-12T04:07:09.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.155+0000 7f2d8b39d700 1 -- 172.21.15.32:0/1682812395 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d84113670 con 0x7f2d840fbcb0 2024-05-12T04:07:09.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.155+0000 7f2d8b39d700 1 -- 172.21.15.32:0/1682812395 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d8405f5c0 con 0x7f2d840fbcb0 2024-05-12T04:07:09.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.156+0000 7f2d79ffb700 1 -- 172.21.15.32:0/1682812395 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f2d74013420 con 0x7f2d840fbcb0 2024-05-12T04:07:09.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.156+0000 7f2d79ffb700 1 --2- 172.21.15.32:0/1682812395 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f2d7006cc20 0x7f2d7006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:07:09.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.156+0000 7f2d79ffb700 1 -- 172.21.15.32:0/1682812395 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f2d7408cbe0 con 0x7f2d840fbcb0 2024-05-12T04:07:09.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.157+0000 7f2d88938700 1 --2- 172.21.15.32:0/1682812395 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f2d7006cc20 0x7f2d7006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:07:09.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.157+0000 7f2d88938700 1 --2- 172.21.15.32:0/1682812395 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f2d7006cc20 0x7f2d7006f0d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2d80007fd0 tx=0x7f2d80009040 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:07:09.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.161+0000 7f2d79ffb700 1 -- 172.21.15.32:0/1682812395 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f2d74056c50 con 0x7f2d840fbcb0 2024-05-12T04:07:09.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.373+0000 7f2d8b39d700 1 -- 172.21.15.32:0/1682812395 --> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2d841013b0 con 0x7f2d7006cc20 2024-05-12T04:07:09.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.382+0000 7f2d79ffb700 1 -- 172.21.15.32:0/1682812395 <== mgr.24593 v2:172.21.15.32:6800/2957111591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f2d841013b0 con 0x7f2d7006cc20 2024-05-12T04:07:09.383 INFO:teuthology.orchestra.run.smithi032.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-12T04:07:09.383 INFO:teuthology.orchestra.run.smithi032.stdout:alertmanager.smithi032 smithi032 *:9093,9094 running (6m) 66s ago 15m 25.5M - 0.25.0 c8568f914cd2 a37fbab7af4c 2024-05-12T04:07:09.383 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi032 smithi032 running (2m) 66s ago 15m 6660k - 18.2.2-1272-g58249552 ddcb0a9d6fde 8c53f1fc7b06 2024-05-12T04:07:09.383 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi033 smithi033 running (2m) 1s ago 14m 6651k - 18.2.2-1272-g58249552 ddcb0a9d6fde cf4fe9ca070b 2024-05-12T04:07:09.383 INFO:teuthology.orchestra.run.smithi032.stdout:grafana.smithi032 smithi032 *:3000 running (5m) 66s ago 14m 89.7M - 9.4.7 954c08fa6188 83055e58d23c 2024-05-12T04:07:09.383 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.ialtqb smithi032 running (9m) 66s ago 9m 13.9M - 17.2.7-939-gc190bdee 284803f7d9cf 0683abf00e3e 2024-05-12T04:07:09.383 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.qpsawh smithi032 running (9m) 66s ago 9m 155M - 17.2.7-939-gc190bdee 284803f7d9cf 95a7db5c4fcd 2024-05-12T04:07:09.383 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi033.hglwml smithi033 running (9m) 1s ago 9m 13.9M - 17.2.7-939-gc190bdee 284803f7d9cf c97902b05025 2024-05-12T04:07:09.383 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi033.jgbjhk smithi033 running (9m) 1s ago 9m 102M - 17.2.7-939-gc190bdee 284803f7d9cf af97052cb97a 2024-05-12T04:07:09.383 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi032.ukatkq smithi032 *:8443,9283,8765 running (5m) 66s ago 16m 501M - 18.2.2-1272-g58249552 ddcb0a9d6fde 0d88ab9a5292 2024-05-12T04:07:09.383 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi033.pcvxwy smithi033 *:8443,9283,8765 running (4m) 1s ago 14m 450M - 18.2.2-1272-g58249552 ddcb0a9d6fde d42dcb6bd307 2024-05-12T04:07:09.383 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi032 smithi032 running (4m) 66s ago 16m 45.5M 2048M 18.2.2-1272-g58249552 ddcb0a9d6fde 81f94171c5dc 2024-05-12T04:07:09.383 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi033 smithi033 running (3m) 1s ago 14m 38.1M 2048M 18.2.2-1272-g58249552 ddcb0a9d6fde d648ba108f3e 2024-05-12T04:07:09.384 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi032 smithi032 *:9100 running (6m) 66s ago 15m 16.5M - 1.5.0 0da6a335fe13 cb75a7c0c829 2024-05-12T04:07:09.384 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi033 smithi033 *:9100 running (5m) 1s ago 14m 18.6M - 1.5.0 0da6a335fe13 7888c36c659f 2024-05-12T04:07:09.384 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0 smithi032 running (2m) 66s ago 13m 80.8M 1570M 18.2.2-1272-g58249552 ddcb0a9d6fde 07f66c4deb3b 2024-05-12T04:07:09.384 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1 smithi032 running (108s) 66s ago 12m 84.5M 1570M 18.2.2-1272-g58249552 ddcb0a9d6fde 67a87807d654 2024-05-12T04:07:09.384 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2 smithi032 running (72s) 66s ago 12m 11.9M 1570M 18.2.2-1272-g58249552 ddcb0a9d6fde 4ac48d74ee8b 2024-05-12T04:07:09.384 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3 smithi033 running (37s) 1s ago 11m 90.5M 2594M 18.2.2-1272-g58249552 ddcb0a9d6fde 11f097a455bb 2024-05-12T04:07:09.384 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4 smithi033 running (5s) 1s ago 11m 11.5M 2594M 18.2.2-1272-g58249552 ddcb0a9d6fde db6fda7d52de 2024-05-12T04:07:09.384 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5 smithi033 running (10m) 1s ago 10m 197M 2594M 17.2.7-939-gc190bdee 284803f7d9cf 57e65a64c821 2024-05-12T04:07:09.384 INFO:teuthology.orchestra.run.smithi032.stdout:prometheus.smithi032 smithi032 *:9095 running (4m) 66s ago 14m 54.2M - 2.43.0 a07b618ecd1d 46eafb25e176 2024-05-12T04:07:09.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.385+0000 7f2d8b39d700 1 -- 172.21.15.32:0/1682812395 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f2d7006cc20 msgr2=0x7f2d7006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:07:09.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.385+0000 7f2d8b39d700 1 --2- 172.21.15.32:0/1682812395 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f2d7006cc20 0x7f2d7006f0d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2d80007fd0 tx=0x7f2d80009040 comp rx=0 tx=0).stop 2024-05-12T04:07:09.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.385+0000 7f2d8b39d700 1 -- 172.21.15.32:0/1682812395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d840fbcb0 msgr2=0x7f2d84110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:07:09.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.385+0000 7f2d8b39d700 1 --2- 172.21.15.32:0/1682812395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d840fbcb0 0x7f2d84110620 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2d74009380 tx=0x7f2d74009e30 comp rx=0 tx=0).stop 2024-05-12T04:07:09.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.385+0000 7f2d8b39d700 1 -- 172.21.15.32:0/1682812395 shutdown_connections 2024-05-12T04:07:09.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.385+0000 7f2d8b39d700 1 --2- 172.21.15.32:0/1682812395 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f2d7006cc20 0x7f2d7006f0d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:09.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.385+0000 7f2d8b39d700 1 --2- 172.21.15.32:0/1682812395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d840fbcb0 0x7f2d84110620 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:09.385 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.385+0000 7f2d8b39d700 1 --2- 172.21.15.32:0/1682812395 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d840fcf60 0x7f2d84112b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:09.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.385+0000 7f2d8b39d700 1 -- 172.21.15.32:0/1682812395 >> 172.21.15.32:0/1682812395 conn(0x7f2d840f7430 msgr2=0x7f2d840f8bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:07:09.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.385+0000 7f2d8b39d700 1 -- 172.21.15.32:0/1682812395 shutdown_connections 2024-05-12T04:07:09.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.385+0000 7f2d8b39d700 1 -- 172.21.15.32:0/1682812395 wait complete. 2024-05-12T04:07:09.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.487+0000 7f9f65409700 1 -- 172.21.15.32:0/3141687747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f600fb1f0 msgr2=0x7f9f600fb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:07:09.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.487+0000 7f9f65409700 1 --2- 172.21.15.32:0/3141687747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f600fb1f0 0x7f9f600fb600 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9f50004940 tx=0x7f9f50009e30 comp rx=0 tx=0).stop 2024-05-12T04:07:09.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.487+0000 7f9f65409700 1 -- 172.21.15.32:0/3141687747 shutdown_connections 2024-05-12T04:07:09.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.487+0000 7f9f65409700 1 --2- 172.21.15.32:0/3141687747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f600fbb40 0x7f9f600fe3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:09.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.487+0000 7f9f65409700 1 --2- 172.21.15.32:0/3141687747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f600fb1f0 0x7f9f600fb600 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:09.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.487+0000 7f9f65409700 1 -- 172.21.15.32:0/3141687747 >> 172.21.15.32:0/3141687747 conn(0x7f9f600f6bd0 msgr2=0x7f9f600f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:07:09.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.487+0000 7f9f65409700 1 -- 172.21.15.32:0/3141687747 shutdown_connections 2024-05-12T04:07:09.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.487+0000 7f9f65409700 1 -- 172.21.15.32:0/3141687747 wait complete. 2024-05-12T04:07:09.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.488+0000 7f9f65409700 1 Processor -- start 2024-05-12T04:07:09.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.488+0000 7f9f65409700 1 -- start start 2024-05-12T04:07:09.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.488+0000 7f9f65409700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f600fb1f0 0x7f9f6010da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:07:09.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.488+0000 7f9f65409700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f600fbb40 0x7f9f6010df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:07:09.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.488+0000 7f9f65409700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f6010e550 con 0x7f9f600fbb40 2024-05-12T04:07:09.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.488+0000 7f9f65409700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f6010e6c0 con 0x7f9f600fb1f0 2024-05-12T04:07:09.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.488+0000 7f9f5e7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f600fbb40 0x7f9f6010df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:07:09.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.488+0000 7f9f5e7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f600fbb40 0x7f9f6010df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44368/0 (socket says 172.21.15.32:44368) 2024-05-12T04:07:09.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.488+0000 7f9f5e7fc700 1 -- 172.21.15.32:0/517241129 learned_addr learned my addr 172.21.15.32:0/517241129 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:07:09.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.488+0000 7f9f5effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f600fb1f0 0x7f9f6010da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:07:09.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.489+0000 7f9f5e7fc700 1 -- 172.21.15.32:0/517241129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f600fb1f0 msgr2=0x7f9f6010da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:07:09.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.489+0000 7f9f5e7fc700 1 --2- 172.21.15.32:0/517241129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f600fb1f0 0x7f9f6010da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:09.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.489+0000 7f9f5e7fc700 1 -- 172.21.15.32:0/517241129 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f50005040 con 0x7f9f600fbb40 2024-05-12T04:07:09.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.489+0000 7f9f5effd700 1 --2- 172.21.15.32:0/517241129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f600fb1f0 0x7f9f6010da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-12T04:07:09.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.489+0000 7f9f5e7fc700 1 --2- 172.21.15.32:0/517241129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f600fbb40 0x7f9f6010df50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9f4800eb40 tx=0x7f9f4800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:07:09.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.489+0000 7f9f577fe700 1 -- 172.21.15.32:0/517241129 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f48007cc0 con 0x7f9f600fbb40 2024-05-12T04:07:09.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.489+0000 7f9f577fe700 1 -- 172.21.15.32:0/517241129 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9f4800f040 con 0x7f9f600fbb40 2024-05-12T04:07:09.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.489+0000 7f9f65409700 1 -- 172.21.15.32:0/517241129 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f601130c0 con 0x7f9f600fbb40 2024-05-12T04:07:09.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.489+0000 7f9f577fe700 1 -- 172.21.15.32:0/517241129 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f4800c990 con 0x7f9f600fbb40 2024-05-12T04:07:09.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.489+0000 7f9f65409700 1 -- 172.21.15.32:0/517241129 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f60113680 con 0x7f9f600fbb40 2024-05-12T04:07:09.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.490+0000 7f9f65409700 1 -- 172.21.15.32:0/517241129 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f6004c710 con 0x7f9f600fbb40 2024-05-12T04:07:09.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.491+0000 7f9f577fe700 1 -- 172.21.15.32:0/517241129 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f9f4800cc00 con 0x7f9f600fbb40 2024-05-12T04:07:09.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.491+0000 7f9f577fe700 1 --2- 172.21.15.32:0/517241129 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f9f4c06cc20 0x7f9f4c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:07:09.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.491+0000 7f9f577fe700 1 -- 172.21.15.32:0/517241129 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f9f4808bc10 con 0x7f9f600fbb40 2024-05-12T04:07:09.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.491+0000 7f9f5effd700 1 --2- 172.21.15.32:0/517241129 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f9f4c06cc20 0x7f9f4c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:07:09.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.492+0000 7f9f5effd700 1 --2- 172.21.15.32:0/517241129 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f9f4c06cc20 0x7f9f4c06f0d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9f50013680 tx=0x7f9f50009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:07:09.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.496+0000 7f9f577fe700 1 -- 172.21.15.32:0/517241129 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f9f48059760 con 0x7f9f600fbb40 2024-05-12T04:07:09.768 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:09 smithi033 ceph-mon[107305]: from='client.34340 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:07:09.769 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:09 smithi033 ceph-mon[107305]: from='client.34344 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:07:09.769 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:09 smithi033 ceph-mon[107305]: from='client.34348 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:07:09.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.782+0000 7f9f65409700 1 -- 172.21.15.32:0/517241129 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9f601138d0 con 0x7f9f600fbb40 2024-05-12T04:07:09.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.782+0000 7f9f577fe700 1 -- 172.21.15.32:0/517241129 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+847 (secure 0 0 0) 0x7f9f480592f0 con 0x7f9f600fbb40 2024-05-12T04:07:09.783 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-05-12T04:07:09.783 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": { 2024-05-12T04:07:09.783 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 2 2024-05-12T04:07:09.783 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:07:09.783 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": { 2024-05-12T04:07:09.783 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 2 2024-05-12T04:07:09.783 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:07:09.783 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": { 2024-05-12T04:07:09.783 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 1, 2024-05-12T04:07:09.783 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 4 2024-05-12T04:07:09.783 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:07:09.783 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": { 2024-05-12T04:07:09.784 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 4 2024-05-12T04:07:09.784 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:07:09.784 INFO:teuthology.orchestra.run.smithi032.stdout: "overall": { 2024-05-12T04:07:09.784 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 5, 2024-05-12T04:07:09.784 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 8 2024-05-12T04:07:09.784 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-05-12T04:07:09.784 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-05-12T04:07:09.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.784+0000 7f9f65409700 1 -- 172.21.15.32:0/517241129 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f9f4c06cc20 msgr2=0x7f9f4c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:07:09.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.784+0000 7f9f65409700 1 --2- 172.21.15.32:0/517241129 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f9f4c06cc20 0x7f9f4c06f0d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9f50013680 tx=0x7f9f50009e30 comp rx=0 tx=0).stop 2024-05-12T04:07:09.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.784+0000 7f9f65409700 1 -- 172.21.15.32:0/517241129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f600fbb40 msgr2=0x7f9f6010df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:07:09.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.784+0000 7f9f65409700 1 --2- 172.21.15.32:0/517241129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f600fbb40 0x7f9f6010df50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9f4800eb40 tx=0x7f9f4800b040 comp rx=0 tx=0).stop 2024-05-12T04:07:09.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.785+0000 7f9f65409700 1 -- 172.21.15.32:0/517241129 shutdown_connections 2024-05-12T04:07:09.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.785+0000 7f9f65409700 1 --2- 172.21.15.32:0/517241129 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f9f4c06cc20 0x7f9f4c06f0d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:09.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.785+0000 7f9f65409700 1 --2- 172.21.15.32:0/517241129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f600fb1f0 0x7f9f6010da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:09.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.785+0000 7f9f65409700 1 --2- 172.21.15.32:0/517241129 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f600fbb40 0x7f9f6010df50 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:09.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.785+0000 7f9f65409700 1 -- 172.21.15.32:0/517241129 >> 172.21.15.32:0/517241129 conn(0x7f9f600f6bd0 msgr2=0x7f9f600f8320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:07:09.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.785+0000 7f9f65409700 1 -- 172.21.15.32:0/517241129 shutdown_connections 2024-05-12T04:07:09.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.785+0000 7f9f65409700 1 -- 172.21.15.32:0/517241129 wait complete. 2024-05-12T04:07:09.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.888+0000 7fc615ae5700 1 -- 172.21.15.32:0/1677014960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc610102340 msgr2=0x7fc610104720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:07:09.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.888+0000 7fc615ae5700 1 --2- 172.21.15.32:0/1677014960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc610102340 0x7fc610104720 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc604004940 tx=0x7fc604009e30 comp rx=0 tx=0).stop 2024-05-12T04:07:09.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.888+0000 7fc615ae5700 1 -- 172.21.15.32:0/1677014960 shutdown_connections 2024-05-12T04:07:09.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.888+0000 7fc615ae5700 1 --2- 172.21.15.32:0/1677014960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc610102340 0x7fc610104720 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:09.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.888+0000 7fc615ae5700 1 --2- 172.21.15.32:0/1677014960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc610066260 0x7fc610101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:09.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.888+0000 7fc615ae5700 1 -- 172.21.15.32:0/1677014960 >> 172.21.15.32:0/1677014960 conn(0x7fc6100f7430 msgr2=0x7fc6100f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:07:09.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.888+0000 7fc615ae5700 1 -- 172.21.15.32:0/1677014960 shutdown_connections 2024-05-12T04:07:09.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.888+0000 7fc615ae5700 1 -- 172.21.15.32:0/1677014960 wait complete. 2024-05-12T04:07:09.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.889+0000 7fc615ae5700 1 Processor -- start 2024-05-12T04:07:09.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.889+0000 7fc615ae5700 1 -- start start 2024-05-12T04:07:09.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.889+0000 7fc615ae5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc610066260 0x7fc61010faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:07:09.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.889+0000 7fc615ae5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc610102340 0x7fc61010ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:07:09.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.889+0000 7fc615ae5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6101105e0 con 0x7fc610066260 2024-05-12T04:07:09.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.889+0000 7fc615ae5700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc610110e30 con 0x7fc610102340 2024-05-12T04:07:09.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.889+0000 7fc60f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc610066260 0x7fc61010faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:07:09.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.889+0000 7fc60f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc610066260 0x7fc61010faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44382/0 (socket says 172.21.15.32:44382) 2024-05-12T04:07:09.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.889+0000 7fc60f7fe700 1 -- 172.21.15.32:0/1284659613 learned_addr learned my addr 172.21.15.32:0/1284659613 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:07:09.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.889+0000 7fc60effd700 1 --2- 172.21.15.32:0/1284659613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc610102340 0x7fc61010ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:07:09.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.889+0000 7fc60f7fe700 1 -- 172.21.15.32:0/1284659613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc610102340 msgr2=0x7fc61010ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:07:09.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.889+0000 7fc60f7fe700 1 --2- 172.21.15.32:0/1284659613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc610102340 0x7fc61010ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:09.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.889+0000 7fc60f7fe700 1 -- 172.21.15.32:0/1284659613 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc604005040 con 0x7fc610066260 2024-05-12T04:07:09.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.890+0000 7fc60f7fe700 1 --2- 172.21.15.32:0/1284659613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc610066260 0x7fc61010faa0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc60000eb70 tx=0x7fc60000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:07:09.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.890+0000 7fc5f7fff700 1 -- 172.21.15.32:0/1284659613 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc600007cc0 con 0x7fc610066260 2024-05-12T04:07:09.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.890+0000 7fc5f7fff700 1 -- 172.21.15.32:0/1284659613 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc60000f040 con 0x7fc610066260 2024-05-12T04:07:09.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.890+0000 7fc615ae5700 1 -- 172.21.15.32:0/1284659613 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6101110b0 con 0x7fc610066260 2024-05-12T04:07:09.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.890+0000 7fc5f7fff700 1 -- 172.21.15.32:0/1284659613 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc60000c9d0 con 0x7fc610066260 2024-05-12T04:07:09.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.890+0000 7fc615ae5700 1 -- 172.21.15.32:0/1284659613 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6101115f0 con 0x7fc610066260 2024-05-12T04:07:09.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.891+0000 7fc615ae5700 1 -- 172.21.15.32:0/1284659613 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc61005f5c0 con 0x7fc610066260 2024-05-12T04:07:09.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.891+0000 7fc5f7fff700 1 -- 172.21.15.32:0/1284659613 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fc60000cc40 con 0x7fc610066260 2024-05-12T04:07:09.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.892+0000 7fc5f7fff700 1 --2- 172.21.15.32:0/1284659613 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fc5f806cc30 0x7fc5f806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:07:09.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.892+0000 7fc60effd700 1 --2- 172.21.15.32:0/1284659613 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fc5f806cc30 0x7fc5f806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:07:09.894 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.892+0000 7fc5f7fff700 1 -- 172.21.15.32:0/1284659613 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fc60008bab0 con 0x7fc610066260 2024-05-12T04:07:09.895 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.892+0000 7fc60effd700 1 --2- 172.21.15.32:0/1284659613 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fc5f806cc30 0x7fc5f806f0e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc604013680 tx=0x7fc604009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:07:09.896 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:09.896+0000 7fc5f7fff700 1 -- 172.21.15.32:0/1284659613 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fc600059600 con 0x7fc610066260 2024-05-12T04:07:10.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:09 smithi032 ceph-mon[120392]: from='client.34340 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:07:10.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:09 smithi032 ceph-mon[120392]: from='client.34344 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:07:10.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:09 smithi032 ceph-mon[120392]: from='client.34348 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:07:10.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.148+0000 7fc615ae5700 1 -- 172.21.15.32:0/1284659613 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc610111910 con 0x7fc610066260 2024-05-12T04:07:10.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.148+0000 7fc5f7fff700 1 -- 172.21.15.32:0/1284659613 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1866 (secure 0 0 0) 0x7fc600059190 con 0x7fc610066260 2024-05-12T04:07:10.149 INFO:teuthology.orchestra.run.smithi032.stdout:e18 2024-05-12T04:07:10.149 INFO:teuthology.orchestra.run.smithi032.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-12T04:07:10.149 INFO:teuthology.orchestra.run.smithi032.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-12T04:07:10.149 INFO:teuthology.orchestra.run.smithi032.stdout:legacy client fscid: 1 2024-05-12T04:07:10.149 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:07:10.149 INFO:teuthology.orchestra.run.smithi032.stdout:Filesystem 'cephfs' (1) 2024-05-12T04:07:10.150 INFO:teuthology.orchestra.run.smithi032.stdout:fs_name cephfs 2024-05-12T04:07:10.150 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 18 2024-05-12T04:07:10.150 INFO:teuthology.orchestra.run.smithi032.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-12T04:07:10.150 INFO:teuthology.orchestra.run.smithi032.stdout:created 2024-05-12T03:57:48.904614+0000 2024-05-12T04:07:10.150 INFO:teuthology.orchestra.run.smithi032.stdout:modified 2024-05-12T03:58:06.821591+0000 2024-05-12T04:07:10.150 INFO:teuthology.orchestra.run.smithi032.stdout:tableserver 0 2024-05-12T04:07:10.150 INFO:teuthology.orchestra.run.smithi032.stdout:root 0 2024-05-12T04:07:10.150 INFO:teuthology.orchestra.run.smithi032.stdout:session_timeout 60 2024-05-12T04:07:10.150 INFO:teuthology.orchestra.run.smithi032.stdout:session_autoclose 300 2024-05-12T04:07:10.150 INFO:teuthology.orchestra.run.smithi032.stdout:max_file_size 1099511627776 2024-05-12T04:07:10.150 INFO:teuthology.orchestra.run.smithi032.stdout:max_xattr_size 65536 2024-05-12T04:07:10.150 INFO:teuthology.orchestra.run.smithi032.stdout:required_client_features {} 2024-05-12T04:07:10.151 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure 0 2024-05-12T04:07:10.151 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure_osd_epoch 46 2024-05-12T04:07:10.151 INFO:teuthology.orchestra.run.smithi032.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-12T04:07:10.151 INFO:teuthology.orchestra.run.smithi032.stdout:max_mds 1 2024-05-12T04:07:10.151 INFO:teuthology.orchestra.run.smithi032.stdout:in 0 2024-05-12T04:07:10.151 INFO:teuthology.orchestra.run.smithi032.stdout:up {0=14474} 2024-05-12T04:07:10.151 INFO:teuthology.orchestra.run.smithi032.stdout:failed 2024-05-12T04:07:10.151 INFO:teuthology.orchestra.run.smithi032.stdout:damaged 2024-05-12T04:07:10.151 INFO:teuthology.orchestra.run.smithi032.stdout:stopped 2024-05-12T04:07:10.151 INFO:teuthology.orchestra.run.smithi032.stdout:data_pools [3] 2024-05-12T04:07:10.151 INFO:teuthology.orchestra.run.smithi032.stdout:metadata_pool 2 2024-05-12T04:07:10.151 INFO:teuthology.orchestra.run.smithi032.stdout:inline_data enabled 2024-05-12T04:07:10.151 INFO:teuthology.orchestra.run.smithi032.stdout:balancer 2024-05-12T04:07:10.151 INFO:teuthology.orchestra.run.smithi032.stdout:bal_rank_mask -1 2024-05-12T04:07:10.151 INFO:teuthology.orchestra.run.smithi032.stdout:standby_count_wanted 1 2024-05-12T04:07:10.151 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.qpsawh{0:14474} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.32:6828/3100155296,v1:172.21.15.32:6829/3100155296] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:07:10.152 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi033.jgbjhk{0:24267} state up:standby-replay seq 3 join_fscid=1 addr [v2:172.21.15.33:6824/55417471,v1:172.21.15.33:6825/55417471] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:07:10.152 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:07:10.152 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:07:10.152 INFO:teuthology.orchestra.run.smithi032.stdout:Standby daemons: 2024-05-12T04:07:10.152 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:07:10.152 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi033.hglwml{-1:24283} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/433858238,v1:172.21.15.33:6827/433858238] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:07:10.152 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.ialtqb{-1:24287} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.32:6826/3703132329,v1:172.21.15.32:6827/3703132329] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:07:10.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.150+0000 7fc615ae5700 1 -- 172.21.15.32:0/1284659613 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fc5f806cc30 msgr2=0x7fc5f806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:07:10.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.150+0000 7fc615ae5700 1 --2- 172.21.15.32:0/1284659613 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fc5f806cc30 0x7fc5f806f0e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc604013680 tx=0x7fc604009e30 comp rx=0 tx=0).stop 2024-05-12T04:07:10.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.150+0000 7fc615ae5700 1 -- 172.21.15.32:0/1284659613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc610066260 msgr2=0x7fc61010faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:07:10.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.150+0000 7fc615ae5700 1 --2- 172.21.15.32:0/1284659613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc610066260 0x7fc61010faa0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc60000eb70 tx=0x7fc60000b040 comp rx=0 tx=0).stop 2024-05-12T04:07:10.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.151+0000 7fc615ae5700 1 -- 172.21.15.32:0/1284659613 shutdown_connections 2024-05-12T04:07:10.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.151+0000 7fc615ae5700 1 --2- 172.21.15.32:0/1284659613 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fc5f806cc30 0x7fc5f806f0e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:10.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.151+0000 7fc615ae5700 1 --2- 172.21.15.32:0/1284659613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc610066260 0x7fc61010faa0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:10.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.151+0000 7fc615ae5700 1 --2- 172.21.15.32:0/1284659613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc610102340 0x7fc61010ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:10.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.151+0000 7fc615ae5700 1 -- 172.21.15.32:0/1284659613 >> 172.21.15.32:0/1284659613 conn(0x7fc6100f7430 msgr2=0x7fc6100f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:07:10.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.151+0000 7fc615ae5700 1 -- 172.21.15.32:0/1284659613 shutdown_connections 2024-05-12T04:07:10.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.151+0000 7fc615ae5700 1 -- 172.21.15.32:0/1284659613 wait complete. 2024-05-12T04:07:10.153 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 18 2024-05-12T04:07:10.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.255+0000 7f45fcc16700 1 -- 172.21.15.32:0/1143142322 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45f80fed30 msgr2=0x7f45f80fae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:07:10.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.255+0000 7f45fcc16700 1 --2- 172.21.15.32:0/1143142322 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45f80fed30 0x7f45f80fae70 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f45e0004940 tx=0x7f45e0009e30 comp rx=0 tx=0).stop 2024-05-12T04:07:10.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.255+0000 7f45fcc16700 1 -- 172.21.15.32:0/1143142322 shutdown_connections 2024-05-12T04:07:10.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.255+0000 7f45fcc16700 1 --2- 172.21.15.32:0/1143142322 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45f80fed30 0x7f45f80fae70 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:10.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.255+0000 7f45fcc16700 1 --2- 172.21.15.32:0/1143142322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45f80fe3e0 0x7f45f80fe7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:10.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.255+0000 7f45fcc16700 1 -- 172.21.15.32:0/1143142322 >> 172.21.15.32:0/1143142322 conn(0x7f45f80f6b80 msgr2=0x7f45f80f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:07:10.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.255+0000 7f45fcc16700 1 -- 172.21.15.32:0/1143142322 shutdown_connections 2024-05-12T04:07:10.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.255+0000 7f45fcc16700 1 -- 172.21.15.32:0/1143142322 wait complete. 2024-05-12T04:07:10.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.256+0000 7f45fcc16700 1 Processor -- start 2024-05-12T04:07:10.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.256+0000 7f45fcc16700 1 -- start start 2024-05-12T04:07:10.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.256+0000 7f45fcc16700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45f80fe3e0 0x7f45f810d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:07:10.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.256+0000 7f45fcc16700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45f80fed30 0x7f45f810ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:07:10.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.256+0000 7f45fcc16700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f810e3c0 con 0x7f45f80fe3e0 2024-05-12T04:07:10.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.256+0000 7f45fcc16700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f810ec10 con 0x7f45f80fed30 2024-05-12T04:07:10.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.256+0000 7f45f659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45f80fe3e0 0x7f45f810d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:07:10.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.256+0000 7f45f659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45f80fe3e0 0x7f45f810d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44396/0 (socket says 172.21.15.32:44396) 2024-05-12T04:07:10.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.256+0000 7f45f659c700 1 -- 172.21.15.32:0/1145038325 learned_addr learned my addr 172.21.15.32:0/1145038325 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:07:10.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.256+0000 7f45f5d9b700 1 --2- 172.21.15.32:0/1145038325 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45f80fed30 0x7f45f810ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:07:10.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.257+0000 7f45f659c700 1 -- 172.21.15.32:0/1145038325 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45f80fed30 msgr2=0x7f45f810ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:07:10.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.257+0000 7f45f659c700 1 --2- 172.21.15.32:0/1145038325 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45f80fed30 0x7f45f810ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:10.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.257+0000 7f45f659c700 1 -- 172.21.15.32:0/1145038325 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45e0005040 con 0x7f45f80fe3e0 2024-05-12T04:07:10.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.257+0000 7f45f659c700 1 --2- 172.21.15.32:0/1145038325 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45f80fe3e0 0x7f45f810d880 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f45e8002a80 tx=0x7f45e80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:07:10.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.257+0000 7f45eeffd700 1 -- 172.21.15.32:0/1145038325 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45e8007cc0 con 0x7f45f80fe3e0 2024-05-12T04:07:10.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.257+0000 7f45eeffd700 1 -- 172.21.15.32:0/1145038325 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f45e800f040 con 0x7f45f80fe3e0 2024-05-12T04:07:10.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.257+0000 7f45fcc16700 1 -- 172.21.15.32:0/1145038325 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45f810eef0 con 0x7f45f80fe3e0 2024-05-12T04:07:10.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.257+0000 7f45eeffd700 1 -- 172.21.15.32:0/1145038325 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45e800da60 con 0x7f45f80fe3e0 2024-05-12T04:07:10.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.257+0000 7f45fcc16700 1 -- 172.21.15.32:0/1145038325 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45f810f430 con 0x7f45f80fe3e0 2024-05-12T04:07:10.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.258+0000 7f45fcc16700 1 -- 172.21.15.32:0/1145038325 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45f805f5c0 con 0x7f45f80fe3e0 2024-05-12T04:07:10.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.258+0000 7f45eeffd700 1 -- 172.21.15.32:0/1145038325 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f45e800dbc0 con 0x7f45f80fe3e0 2024-05-12T04:07:10.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.259+0000 7f45eeffd700 1 --2- 172.21.15.32:0/1145038325 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f45e406cc20 0x7f45e406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:07:10.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.259+0000 7f45eeffd700 1 -- 172.21.15.32:0/1145038325 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f45e808bbb0 con 0x7f45f80fe3e0 2024-05-12T04:07:10.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.259+0000 7f45f5d9b700 1 --2- 172.21.15.32:0/1145038325 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f45e406cc20 0x7f45e406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:07:10.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.260+0000 7f45f5d9b700 1 --2- 172.21.15.32:0/1145038325 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f45e406cc20 0x7f45e406f0d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f45e000b900 tx=0x7f45e0009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:07:10.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.264+0000 7f45eeffd700 1 -- 172.21.15.32:0/1145038325 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f45e8092080 con 0x7f45f80fe3e0 2024-05-12T04:07:10.412 INFO:journalctl@ceph.osd.4.smithi033.stdout:May 12 04:07:10 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-4[126351]: 2024-05-12T04:07:10.017+0000 7fbd0dfc07c0 -1 osd.4 69 log_to_monitors true 2024-05-12T04:07:10.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.487+0000 7f45fcc16700 1 -- 172.21.15.32:0/1145038325 --> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f45f810f800 con 0x7f45e406cc20 2024-05-12T04:07:10.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.488+0000 7f45eeffd700 1 -- 172.21.15.32:0/1145038325 <== mgr.24593 v2:172.21.15.32:6800/2957111591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f45f810f800 con 0x7f45e406cc20 2024-05-12T04:07:10.488 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-05-12T04:07:10.488 INFO:teuthology.orchestra.run.smithi032.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:19c9734256aa43a60e203feb242f50bf8b7f58666d520536af43cc4fee96173a", 2024-05-12T04:07:10.488 INFO:teuthology.orchestra.run.smithi032.stdout: "in_progress": true, 2024-05-12T04:07:10.488 INFO:teuthology.orchestra.run.smithi032.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-12T04:07:10.489 INFO:teuthology.orchestra.run.smithi032.stdout: "services_complete": [ 2024-05-12T04:07:10.489 INFO:teuthology.orchestra.run.smithi032.stdout: "mon", 2024-05-12T04:07:10.489 INFO:teuthology.orchestra.run.smithi032.stdout: "crash", 2024-05-12T04:07:10.489 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr" 2024-05-12T04:07:10.489 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-05-12T04:07:10.489 INFO:teuthology.orchestra.run.smithi032.stdout: "progress": "11/21 daemons upgraded", 2024-05-12T04:07:10.489 INFO:teuthology.orchestra.run.smithi032.stdout: "message": "Currently upgrading osd daemons", 2024-05-12T04:07:10.489 INFO:teuthology.orchestra.run.smithi032.stdout: "is_paused": false 2024-05-12T04:07:10.489 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-05-12T04:07:10.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.490+0000 7f45fcc16700 1 -- 172.21.15.32:0/1145038325 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f45e406cc20 msgr2=0x7f45e406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:07:10.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.490+0000 7f45fcc16700 1 --2- 172.21.15.32:0/1145038325 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f45e406cc20 0x7f45e406f0d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f45e000b900 tx=0x7f45e0009e30 comp rx=0 tx=0).stop 2024-05-12T04:07:10.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.490+0000 7f45fcc16700 1 -- 172.21.15.32:0/1145038325 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45f80fe3e0 msgr2=0x7f45f810d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:07:10.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.490+0000 7f45fcc16700 1 --2- 172.21.15.32:0/1145038325 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45f80fe3e0 0x7f45f810d880 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f45e8002a80 tx=0x7f45e80046b0 comp rx=0 tx=0).stop 2024-05-12T04:07:10.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.490+0000 7f45fcc16700 1 -- 172.21.15.32:0/1145038325 shutdown_connections 2024-05-12T04:07:10.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.490+0000 7f45fcc16700 1 --2- 172.21.15.32:0/1145038325 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f45e406cc20 0x7f45e406f0d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:10.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.490+0000 7f45fcc16700 1 --2- 172.21.15.32:0/1145038325 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45f80fe3e0 0x7f45f810d880 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:10.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.490+0000 7f45fcc16700 1 --2- 172.21.15.32:0/1145038325 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45f80fed30 0x7f45f810ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:10.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.490+0000 7f45fcc16700 1 -- 172.21.15.32:0/1145038325 >> 172.21.15.32:0/1145038325 conn(0x7f45f80f6b80 msgr2=0x7f45f80f82a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:07:10.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.490+0000 7f45fcc16700 1 -- 172.21.15.32:0/1145038325 shutdown_connections 2024-05-12T04:07:10.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.490+0000 7f45fcc16700 1 -- 172.21.15.32:0/1145038325 wait complete. 2024-05-12T04:07:10.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.595+0000 7f7a43d26700 1 -- 172.21.15.32:0/175592298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a3c0fce60 msgr2=0x7f7a3c0fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:07:10.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.595+0000 7f7a43d26700 1 --2- 172.21.15.32:0/175592298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a3c0fce60 0x7f7a3c0fd270 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7a38004940 tx=0x7f7a38009e30 comp rx=0 tx=0).stop 2024-05-12T04:07:10.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.595+0000 7f7a43d26700 1 -- 172.21.15.32:0/175592298 shutdown_connections 2024-05-12T04:07:10.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.595+0000 7f7a43d26700 1 --2- 172.21.15.32:0/175592298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a3c0fdfd0 0x7f7a3c0fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:10.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.595+0000 7f7a43d26700 1 --2- 172.21.15.32:0/175592298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a3c0fce60 0x7f7a3c0fd270 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:10.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.595+0000 7f7a43d26700 1 -- 172.21.15.32:0/175592298 >> 172.21.15.32:0/175592298 conn(0x7f7a3c0f8540 msgr2=0x7f7a3c0fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:07:10.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.596+0000 7f7a43d26700 1 -- 172.21.15.32:0/175592298 shutdown_connections 2024-05-12T04:07:10.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.596+0000 7f7a43d26700 1 -- 172.21.15.32:0/175592298 wait complete. 2024-05-12T04:07:10.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.596+0000 7f7a43d26700 1 Processor -- start 2024-05-12T04:07:10.597 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.596+0000 7f7a43d26700 1 -- start start 2024-05-12T04:07:10.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.596+0000 7f7a43d26700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a3c0fce60 0x7f7a3c071de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:07:10.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.596+0000 7f7a43d26700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a3c0fdfd0 0x7f7a3c0702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:07:10.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.596+0000 7f7a43d26700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3c070860 con 0x7f7a3c0fce60 2024-05-12T04:07:10.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.596+0000 7f7a43d26700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3c0709d0 con 0x7f7a3c0fdfd0 2024-05-12T04:07:10.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.596+0000 7f7a41ac2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a3c0fce60 0x7f7a3c071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:07:10.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.596+0000 7f7a41ac2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a3c0fce60 0x7f7a3c071de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44406/0 (socket says 172.21.15.32:44406) 2024-05-12T04:07:10.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.596+0000 7f7a41ac2700 1 -- 172.21.15.32:0/846206586 learned_addr learned my addr 172.21.15.32:0/846206586 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:07:10.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.596+0000 7f7a412c1700 1 --2- 172.21.15.32:0/846206586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a3c0fdfd0 0x7f7a3c0702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:07:10.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.597+0000 7f7a41ac2700 1 -- 172.21.15.32:0/846206586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a3c0fdfd0 msgr2=0x7f7a3c0702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:07:10.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.597+0000 7f7a41ac2700 1 --2- 172.21.15.32:0/846206586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a3c0fdfd0 0x7f7a3c0702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:10.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.597+0000 7f7a41ac2700 1 -- 172.21.15.32:0/846206586 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a38005040 con 0x7f7a3c0fce60 2024-05-12T04:07:10.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.597+0000 7f7a41ac2700 1 --2- 172.21.15.32:0/846206586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a3c0fce60 0x7f7a3c071de0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7a380137b0 tx=0x7f7a38009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:07:10.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.597+0000 7f7a327fc700 1 -- 172.21.15.32:0/846206586 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a3800e070 con 0x7f7a3c0fce60 2024-05-12T04:07:10.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.597+0000 7f7a327fc700 1 -- 172.21.15.32:0/846206586 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7a3801c400 con 0x7f7a3c0fce60 2024-05-12T04:07:10.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.597+0000 7f7a43d26700 1 -- 172.21.15.32:0/846206586 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a3c070c50 con 0x7f7a3c0fce60 2024-05-12T04:07:10.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.597+0000 7f7a327fc700 1 -- 172.21.15.32:0/846206586 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a3800b650 con 0x7f7a3c0fce60 2024-05-12T04:07:10.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.597+0000 7f7a43d26700 1 -- 172.21.15.32:0/846206586 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a3c115900 con 0x7f7a3c0fce60 2024-05-12T04:07:10.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.598+0000 7f7a43d26700 1 -- 172.21.15.32:0/846206586 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a3c05f5c0 con 0x7f7a3c0fce60 2024-05-12T04:07:10.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.599+0000 7f7a327fc700 1 -- 172.21.15.32:0/846206586 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f7a3801c9e0 con 0x7f7a3c0fce60 2024-05-12T04:07:10.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.599+0000 7f7a327fc700 1 --2- 172.21.15.32:0/846206586 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f7a2806cc20 0x7f7a2806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:07:10.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.599+0000 7f7a327fc700 1 -- 172.21.15.32:0/846206586 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f7a3808d560 con 0x7f7a3c0fce60 2024-05-12T04:07:10.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.599+0000 7f7a412c1700 1 --2- 172.21.15.32:0/846206586 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f7a2806cc20 0x7f7a2806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:07:10.600 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.600+0000 7f7a412c1700 1 --2- 172.21.15.32:0/846206586 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f7a2806cc20 0x7f7a2806f0d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c004fd0 tx=0x7f7a2c00b040 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:07:10.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.604+0000 7f7a327fc700 1 -- 172.21.15.32:0/846206586 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f7a38057690 con 0x7f7a3c0fce60 2024-05-12T04:07:10.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.896+0000 7f7a43d26700 1 -- 172.21.15.32:0/846206586 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f7a3c04bed0 con 0x7f7a3c0fce60 2024-05-12T04:07:10.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.897+0000 7f7a327fc700 1 -- 172.21.15.32:0/846206586 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1296 (secure 0 0 0) 0x7f7a38004490 con 0x7f7a3c0fce60 2024-05-12T04:07:10.897 INFO:teuthology.orchestra.run.smithi032.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down; Degraded data redundancy: 49/276 objects degraded (17.754%), 15 pgs degraded 2024-05-12T04:07:10.897 INFO:teuthology.orchestra.run.smithi032.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-12T04:07:10.898 INFO:teuthology.orchestra.run.smithi032.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-12T04:07:10.898 INFO:teuthology.orchestra.run.smithi032.stdout:[WRN] OSD_DOWN: 1 osds down 2024-05-12T04:07:10.898 INFO:teuthology.orchestra.run.smithi032.stdout: osd.4 (root=default,host=smithi033) is down 2024-05-12T04:07:10.898 INFO:teuthology.orchestra.run.smithi032.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 49/276 objects degraded (17.754%), 15 pgs degraded 2024-05-12T04:07:10.898 INFO:teuthology.orchestra.run.smithi032.stdout: pg 2.4 is active+undersized+degraded, acting [1,0] 2024-05-12T04:07:10.898 INFO:teuthology.orchestra.run.smithi032.stdout: pg 2.5 is active+undersized+degraded, acting [3,0] 2024-05-12T04:07:10.898 INFO:teuthology.orchestra.run.smithi032.stdout: pg 2.6 is active+undersized+degraded, acting [1,3] 2024-05-12T04:07:10.898 INFO:teuthology.orchestra.run.smithi032.stdout: pg 2.7 is active+undersized+degraded, acting [3,2] 2024-05-12T04:07:10.898 INFO:teuthology.orchestra.run.smithi032.stdout: pg 2.9 is active+undersized+degraded, acting [1,0] 2024-05-12T04:07:10.898 INFO:teuthology.orchestra.run.smithi032.stdout: pg 2.a is active+undersized+degraded, acting [1,3] 2024-05-12T04:07:10.898 INFO:teuthology.orchestra.run.smithi032.stdout: pg 2.b is active+undersized+degraded, acting [3,5] 2024-05-12T04:07:10.898 INFO:teuthology.orchestra.run.smithi032.stdout: pg 2.f is active+undersized+degraded, acting [0,5] 2024-05-12T04:07:10.898 INFO:teuthology.orchestra.run.smithi032.stdout: pg 2.13 is active+undersized+degraded, acting [0,2] 2024-05-12T04:07:10.898 INFO:teuthology.orchestra.run.smithi032.stdout: pg 2.14 is active+undersized+degraded, acting [3,5] 2024-05-12T04:07:10.899 INFO:teuthology.orchestra.run.smithi032.stdout: pg 2.18 is active+undersized+degraded, acting [5,3] 2024-05-12T04:07:10.899 INFO:teuthology.orchestra.run.smithi032.stdout: pg 2.19 is active+undersized+degraded, acting [0,2] 2024-05-12T04:07:10.899 INFO:teuthology.orchestra.run.smithi032.stdout: pg 2.1a is active+undersized+degraded, acting [3,5] 2024-05-12T04:07:10.899 INFO:teuthology.orchestra.run.smithi032.stdout: pg 2.1c is active+undersized+degraded, acting [5,2] 2024-05-12T04:07:10.899 INFO:teuthology.orchestra.run.smithi032.stdout: pg 2.1f is active+undersized+degraded, acting [0,3] 2024-05-12T04:07:10.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.899+0000 7f7a43d26700 1 -- 172.21.15.32:0/846206586 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f7a2806cc20 msgr2=0x7f7a2806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:07:10.899 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.899+0000 7f7a43d26700 1 --2- 172.21.15.32:0/846206586 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f7a2806cc20 0x7f7a2806f0d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c004fd0 tx=0x7f7a2c00b040 comp rx=0 tx=0).stop 2024-05-12T04:07:10.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.899+0000 7f7a43d26700 1 -- 172.21.15.32:0/846206586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a3c0fce60 msgr2=0x7f7a3c071de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:07:10.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.899+0000 7f7a43d26700 1 --2- 172.21.15.32:0/846206586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a3c0fce60 0x7f7a3c071de0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7a380137b0 tx=0x7f7a38009e30 comp rx=0 tx=0).stop 2024-05-12T04:07:10.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.899+0000 7f7a43d26700 1 -- 172.21.15.32:0/846206586 shutdown_connections 2024-05-12T04:07:10.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.899+0000 7f7a43d26700 1 --2- 172.21.15.32:0/846206586 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f7a2806cc20 0x7f7a2806f0d0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:10.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.899+0000 7f7a43d26700 1 --2- 172.21.15.32:0/846206586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a3c0fce60 0x7f7a3c071de0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:10.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.899+0000 7f7a43d26700 1 --2- 172.21.15.32:0/846206586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a3c0fdfd0 0x7f7a3c0702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:10.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.899+0000 7f7a43d26700 1 -- 172.21.15.32:0/846206586 >> 172.21.15.32:0/846206586 conn(0x7f7a3c0f8540 msgr2=0x7f7a3c0f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:07:10.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.899+0000 7f7a43d26700 1 -- 172.21.15.32:0/846206586 shutdown_connections 2024-05-12T04:07:10.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:10.899+0000 7f7a43d26700 1 -- 172.21.15.32:0/846206586 wait complete. 2024-05-12T04:07:10.913 INFO:journalctl@ceph.osd.4.smithi033.stdout:May 12 04:07:10 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-4[126351]: 2024-05-12T04:07:10.585+0000 7fbcff419700 -1 osd.4 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-12T04:07:10.913 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:10 smithi033 ceph-mon[107305]: pgmap v142: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 260 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 518 B/s rd, 0 op/s; 49/276 objects degraded (17.754%) 2024-05-12T04:07:10.913 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:10 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/517241129' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:07:10.914 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:10 smithi033 ceph-mon[107305]: from='osd.4 [v2:172.21.15.33:6808/1028896705,v1:172.21.15.33:6809/1028896705]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-12T04:07:10.914 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:10 smithi033 ceph-mon[107305]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-12T04:07:10.914 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:10 smithi033 ceph-mon[107305]: Health check update: Degraded data redundancy: 49/276 objects degraded (17.754%), 15 pgs degraded (PG_DEGRADED) 2024-05-12T04:07:10.914 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:10 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/1284659613' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-12T04:07:11.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:10 smithi032 ceph-mon[120392]: pgmap v142: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 260 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 518 B/s rd, 0 op/s; 49/276 objects degraded (17.754%) 2024-05-12T04:07:11.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:10 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/517241129' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:07:11.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:10 smithi032 ceph-mon[120392]: from='osd.4 [v2:172.21.15.33:6808/1028896705,v1:172.21.15.33:6809/1028896705]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-12T04:07:11.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:10 smithi032 ceph-mon[120392]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-12T04:07:11.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:10 smithi032 ceph-mon[120392]: Health check update: Degraded data redundancy: 49/276 objects degraded (17.754%), 15 pgs degraded (PG_DEGRADED) 2024-05-12T04:07:11.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:10 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/1284659613' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-12T04:07:11.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:11 smithi033 ceph-mon[107305]: from='client.34362 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:07:11.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:11 smithi033 ceph-mon[107305]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-12T04:07:11.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:11 smithi033 ceph-mon[107305]: osdmap e72: 6 total, 5 up, 6 in 2024-05-12T04:07:11.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:11 smithi033 ceph-mon[107305]: from='osd.4 [v2:172.21.15.33:6808/1028896705,v1:172.21.15.33:6809/1028896705]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-05-12T04:07:11.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:11 smithi033 ceph-mon[107305]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-05-12T04:07:11.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:11 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/846206586' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-12T04:07:11.936 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:11 smithi032 ceph-mon[120392]: from='client.34362 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:07:11.936 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:11 smithi032 ceph-mon[120392]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-12T04:07:11.936 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:11 smithi032 ceph-mon[120392]: osdmap e72: 6 total, 5 up, 6 in 2024-05-12T04:07:11.936 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:11 smithi032 ceph-mon[120392]: from='osd.4 [v2:172.21.15.33:6808/1028896705,v1:172.21.15.33:6809/1028896705]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-05-12T04:07:11.936 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:11 smithi032 ceph-mon[120392]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-05-12T04:07:11.936 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:11 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/846206586' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-12T04:07:12.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:12 smithi033 ceph-mon[107305]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-12T04:07:12.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:12 smithi033 ceph-mon[107305]: osd.4 [v2:172.21.15.33:6808/1028896705,v1:172.21.15.33:6809/1028896705] boot 2024-05-12T04:07:12.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:12 smithi033 ceph-mon[107305]: osdmap e73: 6 total, 6 up, 6 in 2024-05-12T04:07:12.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:12 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-12T04:07:12.913 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:12 smithi033 ceph-mon[107305]: pgmap v145: 65 pgs: 4 peering, 14 active+undersized+degraded, 16 active+undersized, 31 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 43/276 objects degraded (15.580%) 2024-05-12T04:07:12.913 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:12 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:07:12.913 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:12 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:07:12.913 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:12 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:07:12.913 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:12 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T04:07:12.913 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:12 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:07:12.913 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:12 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:07:12.913 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:12 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:07:12.913 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:12 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:07:12.913 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:12 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:07:12.913 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:12 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-05-12T04:07:12.913 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:12 smithi033 ceph-mon[107305]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-05-12T04:07:12.913 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:12 smithi033 ceph-mon[107305]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-05-12T04:07:12.913 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:12 smithi033 ceph-mon[107305]: osdmap e74: 6 total, 6 up, 6 in 2024-05-12T04:07:13.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:12 smithi032 ceph-mon[120392]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-12T04:07:13.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:12 smithi032 ceph-mon[120392]: osd.4 [v2:172.21.15.33:6808/1028896705,v1:172.21.15.33:6809/1028896705] boot 2024-05-12T04:07:13.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:12 smithi032 ceph-mon[120392]: osdmap e73: 6 total, 6 up, 6 in 2024-05-12T04:07:13.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:12 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-12T04:07:13.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:12 smithi032 ceph-mon[120392]: pgmap v145: 65 pgs: 4 peering, 14 active+undersized+degraded, 16 active+undersized, 31 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 43/276 objects degraded (15.580%) 2024-05-12T04:07:13.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:12 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:07:13.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:12 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:07:13.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:12 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:07:13.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:12 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T04:07:13.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:12 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:07:13.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:12 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:07:13.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:12 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:07:13.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:12 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:07:13.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:12 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:07:13.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:12 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-05-12T04:07:13.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:12 smithi032 ceph-mon[120392]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-05-12T04:07:13.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:12 smithi032 ceph-mon[120392]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-05-12T04:07:13.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:12 smithi032 ceph-mon[120392]: osdmap e74: 6 total, 6 up, 6 in 2024-05-12T04:07:15.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:14 smithi033 ceph-mon[107305]: pgmap v147: 65 pgs: 5 peering, 5 active+undersized+degraded, 7 active+undersized, 48 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 682 B/s rd, 1 op/s; 13/276 objects degraded (4.710%) 2024-05-12T04:07:15.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:14 smithi032 ceph-mon[120392]: pgmap v147: 65 pgs: 5 peering, 5 active+undersized+degraded, 7 active+undersized, 48 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 682 B/s rd, 1 op/s; 13/276 objects degraded (4.710%) 2024-05-12T04:07:16.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:15 smithi033 ceph-mon[107305]: Health check update: Degraded data redundancy: 13/276 objects degraded (4.710%), 5 pgs degraded (PG_DEGRADED) 2024-05-12T04:07:16.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:15 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:07:16.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:15 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:07:16.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:15 smithi032 ceph-mon[120392]: Health check update: Degraded data redundancy: 13/276 objects degraded (4.710%), 5 pgs degraded (PG_DEGRADED) 2024-05-12T04:07:16.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:15 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:07:16.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:15 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:07:17.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:16 smithi033 ceph-mon[107305]: pgmap v148: 65 pgs: 5 peering, 3 active+undersized+degraded, 3 active+undersized, 54 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 682 B/s rd, 1 op/s; 8/276 objects degraded (2.899%) 2024-05-12T04:07:17.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:16 smithi032 ceph-mon[120392]: pgmap v148: 65 pgs: 5 peering, 3 active+undersized+degraded, 3 active+undersized, 54 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 682 B/s rd, 1 op/s; 8/276 objects degraded (2.899%) 2024-05-12T04:07:18.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:17 smithi033 ceph-mon[107305]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 8/276 objects degraded (2.899%), 3 pgs degraded) 2024-05-12T04:07:18.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:17 smithi032 ceph-mon[120392]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 8/276 objects degraded (2.899%), 3 pgs degraded) 2024-05-12T04:07:19.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:18 smithi033 ceph-mon[107305]: pgmap v149: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:07:19.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:18 smithi032 ceph-mon[120392]: pgmap v149: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:07:21.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:20 smithi033 ceph-mon[107305]: pgmap v150: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-05-12T04:07:21.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:20 smithi032 ceph-mon[120392]: pgmap v150: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-05-12T04:07:23.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:22 smithi033 ceph-mon[107305]: pgmap v151: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 921 B/s rd, 1 op/s 2024-05-12T04:07:23.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:22 smithi032 ceph-mon[120392]: pgmap v151: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 921 B/s rd, 1 op/s 2024-05-12T04:07:25.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:24 smithi033 ceph-mon[107305]: pgmap v152: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-05-12T04:07:25.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:24 smithi032 ceph-mon[120392]: pgmap v152: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-05-12T04:07:27.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:26 smithi033 ceph-mon[107305]: pgmap v153: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:07:27.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:26 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:07:27.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:26 smithi032 ceph-mon[120392]: pgmap v153: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:07:27.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:26 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:07:28.100 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:27 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-05-12T04:07:28.100 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:27 smithi033 ceph-mon[107305]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-05-12T04:07:28.100 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:27 smithi033 ceph-mon[107305]: Upgrade: osd.5 is safe to restart 2024-05-12T04:07:28.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:27 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-05-12T04:07:28.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:27 smithi032 ceph-mon[120392]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-05-12T04:07:28.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:27 smithi032 ceph-mon[120392]: Upgrade: osd.5 is safe to restart 2024-05-12T04:07:29.756 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:29 smithi033 ceph-mon[107305]: pgmap v154: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:07:29.757 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:29 smithi033 ceph-mon[107305]: Upgrade: Updating osd.5 2024-05-12T04:07:29.757 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:29 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:07:29.757 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:29 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-12T04:07:29.757 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:29 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:07:29.757 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:29 smithi033 ceph-mon[107305]: Deploying daemon osd.5 on smithi033 2024-05-12T04:07:29.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:29 smithi032 ceph-mon[120392]: pgmap v154: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:07:29.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:29 smithi032 ceph-mon[120392]: Upgrade: Updating osd.5 2024-05-12T04:07:29.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:29 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:07:29.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:29 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-12T04:07:29.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:29 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:07:29.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:29 smithi032 ceph-mon[120392]: Deploying daemon osd.5 on smithi033 2024-05-12T04:07:31.492 INFO:journalctl@ceph.osd.5.smithi033.stdout:May 12 04:07:31 smithi033 systemd[1]: Stopping Ceph osd.5 for a3e178f8-1012-11ef-bc98-c7b262605968... 2024-05-12T04:07:31.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:31 smithi033 ceph-mon[107305]: pgmap v155: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:07:31.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:31 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:07:31.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:31 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:07:31.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:31 smithi032 ceph-mon[120392]: pgmap v155: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:07:31.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:31 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:07:31.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:31 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:07:31.912 INFO:journalctl@ceph.osd.5.smithi033.stdout:May 12 04:07:31 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-5[72931]: 2024-05-12T04:07:31.733+0000 7f70bc200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-12T04:07:31.912 INFO:journalctl@ceph.osd.5.smithi033.stdout:May 12 04:07:31 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-5[72931]: 2024-05-12T04:07:31.733+0000 7f70bc200700 -1 osd.5 74 *** Got signal Terminated *** 2024-05-12T04:07:31.912 INFO:journalctl@ceph.osd.5.smithi033.stdout:May 12 04:07:31 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-5[72931]: 2024-05-12T04:07:31.733+0000 7f70bc200700 -1 osd.5 74 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-12T04:07:32.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:32 smithi032 ceph-mon[120392]: osd.5 marked itself down and dead 2024-05-12T04:07:32.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:32 smithi033 ceph-mon[107305]: osd.5 marked itself down and dead 2024-05-12T04:07:33.425 INFO:journalctl@ceph.osd.5.smithi033.stdout:May 12 04:07:33 smithi033 bash[132042]: ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-5 2024-05-12T04:07:33.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:33 smithi032 ceph-mon[120392]: pgmap v156: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:07:33.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:33 smithi032 ceph-mon[120392]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-12T04:07:33.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:33 smithi032 ceph-mon[120392]: osdmap e75: 6 total, 5 up, 6 in 2024-05-12T04:07:33.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:33 smithi033 ceph-mon[107305]: pgmap v156: 65 pgs: 65 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:07:33.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:33 smithi033 ceph-mon[107305]: Health check failed: 1 osds down (OSD_DOWN) 2024-05-12T04:07:33.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:33 smithi033 ceph-mon[107305]: osdmap e75: 6 total, 5 up, 6 in 2024-05-12T04:07:34.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:34 smithi032 ceph-mon[120392]: osdmap e76: 6 total, 5 up, 6 in 2024-05-12T04:07:34.793 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:34 smithi033 ceph-mon[107305]: osdmap e76: 6 total, 5 up, 6 in 2024-05-12T04:07:35.412 INFO:journalctl@ceph.osd.5.smithi033.stdout:May 12 04:07:35 smithi033 systemd[1]: ceph-a3e178f8-1012-11ef-bc98-c7b262605968@osd.5.service: Succeeded. 2024-05-12T04:07:35.412 INFO:journalctl@ceph.osd.5.smithi033.stdout:May 12 04:07:35 smithi033 systemd[1]: Stopped Ceph osd.5 for a3e178f8-1012-11ef-bc98-c7b262605968. 2024-05-12T04:07:35.412 INFO:journalctl@ceph.osd.5.smithi033.stdout:May 12 04:07:35 smithi033 systemd[1]: Starting Ceph osd.5 for a3e178f8-1012-11ef-bc98-c7b262605968... 2024-05-12T04:07:35.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:35 smithi032 ceph-mon[120392]: pgmap v159: 65 pgs: 13 peering, 9 stale+active+clean, 43 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s 2024-05-12T04:07:35.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:35 smithi032 ceph-mon[120392]: Health check failed: Reduced data availability: 4 pgs peering (PG_AVAILABILITY) 2024-05-12T04:07:35.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:35 smithi033 ceph-mon[107305]: pgmap v159: 65 pgs: 13 peering, 9 stale+active+clean, 43 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s 2024-05-12T04:07:35.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:35 smithi033 ceph-mon[107305]: Health check failed: Reduced data availability: 4 pgs peering (PG_AVAILABILITY) 2024-05-12T04:07:36.412 INFO:journalctl@ceph.osd.5.smithi033.stdout:May 12 04:07:36 smithi033 bash[132316]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-12T04:07:36.412 INFO:journalctl@ceph.osd.5.smithi033.stdout:May 12 04:07:36 smithi033 bash[132316]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-05-12T04:07:36.412 INFO:journalctl@ceph.osd.5.smithi033.stdout:May 12 04:07:36 smithi033 bash[132316]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-05-12T04:07:36.412 INFO:journalctl@ceph.osd.5.smithi033.stdout:May 12 04:07:36 smithi033 bash[132316]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-05-12T04:07:36.412 INFO:journalctl@ceph.osd.5.smithi033.stdout:May 12 04:07:36 smithi033 bash[132316]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-05-12T04:07:36.412 INFO:journalctl@ceph.osd.5.smithi033.stdout:May 12 04:07:36 smithi033 bash[132316]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-12T04:07:36.412 INFO:journalctl@ceph.osd.5.smithi033.stdout:May 12 04:07:36 smithi033 bash[132316]: --> ceph-volume raw activate successful for osd ID: 5 2024-05-12T04:07:36.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:36 smithi032 ceph-mon[120392]: pgmap v160: 65 pgs: 3 active+undersized+degraded, 13 peering, 5 stale+active+clean, 4 active+undersized, 40 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 11/276 objects degraded (3.986%) 2024-05-12T04:07:36.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:36 smithi033 ceph-mon[107305]: pgmap v160: 65 pgs: 3 active+undersized+degraded, 13 peering, 5 stale+active+clean, 4 active+undersized, 40 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 11/276 objects degraded (3.986%) 2024-05-12T04:07:37.661 INFO:journalctl@ceph.osd.5.smithi033.stdout:May 12 04:07:37 smithi033 ceph-osd[132636]: -- 172.21.15.33:0/1037498487 <== mon.1 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562dbff8b1e0 con 0x562dbff5b000 2024-05-12T04:07:37.662 INFO:journalctl@ceph.osd.5.smithi033.stdout:May 12 04:07:37 smithi033 bash[132599]: 3333020b8a1528c43ece364774304bbab2a1ba7c9c8bd7976d95c18639df483e 2024-05-12T04:07:37.662 INFO:journalctl@ceph.osd.5.smithi033.stdout:May 12 04:07:37 smithi033 systemd[1]: Started Ceph osd.5 for a3e178f8-1012-11ef-bc98-c7b262605968. 2024-05-12T04:07:37.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:37 smithi033 ceph-mon[107305]: Health check failed: Degraded data redundancy: 11/276 objects degraded (3.986%), 3 pgs degraded (PG_DEGRADED) 2024-05-12T04:07:37.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:37 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:07:37.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:37 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:07:37.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:37 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:07:37.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:37 smithi032 ceph-mon[120392]: Health check failed: Degraded data redundancy: 11/276 objects degraded (3.986%), 3 pgs degraded (PG_DEGRADED) 2024-05-12T04:07:37.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:37 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:07:37.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:37 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:07:37.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:37 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:07:38.116 INFO:journalctl@ceph.osd.5.smithi033.stdout:May 12 04:07:37 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-5[132613]: 2024-05-12T04:07:37.790+0000 7f72848927c0 -1 Falling back to public interface 2024-05-12T04:07:38.786 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:38 smithi033 ceph-mon[107305]: pgmap v161: 65 pgs: 11 active+undersized+degraded, 3 peering, 14 active+undersized, 37 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 39/276 objects degraded (14.130%) 2024-05-12T04:07:38.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:38 smithi032 ceph-mon[120392]: pgmap v161: 65 pgs: 11 active+undersized+degraded, 3 peering, 14 active+undersized, 37 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 39/276 objects degraded (14.130%) 2024-05-12T04:07:40.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:39 smithi033 ceph-mon[107305]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-05-12T04:07:40.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:39 smithi032 ceph-mon[120392]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-05-12T04:07:41.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.017+0000 7f3085de0700 1 -- 172.21.15.32:0/2307467592 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3080066260 msgr2=0x7f3080101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:07:41.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.017+0000 7f3085de0700 1 --2- 172.21.15.32:0/2307467592 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3080066260 0x7f3080101e00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3070004940 tx=0x7f3070009e30 comp rx=0 tx=0).stop 2024-05-12T04:07:41.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.017+0000 7f3085de0700 1 -- 172.21.15.32:0/2307467592 shutdown_connections 2024-05-12T04:07:41.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.017+0000 7f3085de0700 1 --2- 172.21.15.32:0/2307467592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3080102340 0x7f3080104720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:41.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.017+0000 7f3085de0700 1 --2- 172.21.15.32:0/2307467592 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3080066260 0x7f3080101e00 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:41.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.017+0000 7f3085de0700 1 -- 172.21.15.32:0/2307467592 >> 172.21.15.32:0/2307467592 conn(0x7f30800f7430 msgr2=0x7f30800f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:07:41.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.017+0000 7f3085de0700 1 -- 172.21.15.32:0/2307467592 shutdown_connections 2024-05-12T04:07:41.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.017+0000 7f3085de0700 1 -- 172.21.15.32:0/2307467592 wait complete. 2024-05-12T04:07:41.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.018+0000 7f3085de0700 1 Processor -- start 2024-05-12T04:07:41.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.018+0000 7f3085de0700 1 -- start start 2024-05-12T04:07:41.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.018+0000 7f3085de0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3080066260 0x7f308010faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:07:41.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.018+0000 7f3085de0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3080102340 0x7f308010ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:07:41.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.018+0000 7f3085de0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30801105e0 con 0x7f3080066260 2024-05-12T04:07:41.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.018+0000 7f3085de0700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3080110e30 con 0x7f3080102340 2024-05-12T04:07:41.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.019+0000 7f307f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3080066260 0x7f308010faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:07:41.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.019+0000 7f307f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3080066260 0x7f308010faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50152/0 (socket says 172.21.15.32:50152) 2024-05-12T04:07:41.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.019+0000 7f307f7fe700 1 -- 172.21.15.32:0/2601347877 learned_addr learned my addr 172.21.15.32:0/2601347877 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:07:41.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.019+0000 7f307effd700 1 --2- 172.21.15.32:0/2601347877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3080102340 0x7f308010ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:07:41.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.019+0000 7f307f7fe700 1 -- 172.21.15.32:0/2601347877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3080102340 msgr2=0x7f308010ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:07:41.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.019+0000 7f307f7fe700 1 --2- 172.21.15.32:0/2601347877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3080102340 0x7f308010ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:41.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.019+0000 7f307f7fe700 1 -- 172.21.15.32:0/2601347877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3070005040 con 0x7f3080066260 2024-05-12T04:07:41.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.019+0000 7f307f7fe700 1 --2- 172.21.15.32:0/2601347877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3080066260 0x7f308010faa0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f30700137b0 tx=0x7f3070009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:07:41.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.019+0000 7f3067fff700 1 -- 172.21.15.32:0/2601347877 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3070021070 con 0x7f3080066260 2024-05-12T04:07:41.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.019+0000 7f3067fff700 1 -- 172.21.15.32:0/2601347877 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f307001c400 con 0x7f3080066260 2024-05-12T04:07:41.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.019+0000 7f3085de0700 1 -- 172.21.15.32:0/2601347877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30801110b0 con 0x7f3080066260 2024-05-12T04:07:41.022 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.019+0000 7f3067fff700 1 -- 172.21.15.32:0/2601347877 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f307000e070 con 0x7f3080066260 2024-05-12T04:07:41.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.019+0000 7f3085de0700 1 -- 172.21.15.32:0/2601347877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30801115f0 con 0x7f3080066260 2024-05-12T04:07:41.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.020+0000 7f3085de0700 1 -- 172.21.15.32:0/2601347877 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f308005f5c0 con 0x7f3080066260 2024-05-12T04:07:41.023 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.021+0000 7f3067fff700 1 -- 172.21.15.32:0/2601347877 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f307001c9c0 con 0x7f3080066260 2024-05-12T04:07:41.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.021+0000 7f3067fff700 1 --2- 172.21.15.32:0/2601347877 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f306806cc20 0x7f306806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:07:41.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.022+0000 7f3067fff700 1 -- 172.21.15.32:0/2601347877 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f307008cff0 con 0x7f3080066260 2024-05-12T04:07:41.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.022+0000 7f307effd700 1 --2- 172.21.15.32:0/2601347877 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f306806cc20 0x7f306806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:07:41.024 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.022+0000 7f307effd700 1 --2- 172.21.15.32:0/2601347877 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f306806cc20 0x7f306806f0d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3074009b80 tx=0x7f307400b040 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:07:41.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.026+0000 7f3067fff700 1 -- 172.21.15.32:0/2601347877 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f307005abb0 con 0x7f3080066260 2024-05-12T04:07:41.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.251+0000 7f3085de0700 1 -- 172.21.15.32:0/2601347877 --> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3080111a30 con 0x7f306806cc20 2024-05-12T04:07:41.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.253+0000 7f3067fff700 1 -- 172.21.15.32:0/2601347877 <== mgr.24593 v2:172.21.15.32:6800/2957111591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f3080111a30 con 0x7f306806cc20 2024-05-12T04:07:41.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.255+0000 7f3085de0700 1 -- 172.21.15.32:0/2601347877 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f306806cc20 msgr2=0x7f306806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:07:41.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.255+0000 7f3085de0700 1 --2- 172.21.15.32:0/2601347877 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f306806cc20 0x7f306806f0d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3074009b80 tx=0x7f307400b040 comp rx=0 tx=0).stop 2024-05-12T04:07:41.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.255+0000 7f3085de0700 1 -- 172.21.15.32:0/2601347877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3080066260 msgr2=0x7f308010faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:07:41.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.255+0000 7f3085de0700 1 --2- 172.21.15.32:0/2601347877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3080066260 0x7f308010faa0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f30700137b0 tx=0x7f3070009e30 comp rx=0 tx=0).stop 2024-05-12T04:07:41.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.255+0000 7f3085de0700 1 -- 172.21.15.32:0/2601347877 shutdown_connections 2024-05-12T04:07:41.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.255+0000 7f3085de0700 1 --2- 172.21.15.32:0/2601347877 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f306806cc20 0x7f306806f0d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:41.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.255+0000 7f3085de0700 1 --2- 172.21.15.32:0/2601347877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3080066260 0x7f308010faa0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:41.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.255+0000 7f3085de0700 1 --2- 172.21.15.32:0/2601347877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3080102340 0x7f308010ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:41.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.255+0000 7f3085de0700 1 -- 172.21.15.32:0/2601347877 >> 172.21.15.32:0/2601347877 conn(0x7f30800f7430 msgr2=0x7f30800f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:07:41.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.255+0000 7f3085de0700 1 -- 172.21.15.32:0/2601347877 shutdown_connections 2024-05-12T04:07:41.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.255+0000 7f3085de0700 1 -- 172.21.15.32:0/2601347877 wait complete. 2024-05-12T04:07:41.266 INFO:teuthology.orchestra.run.smithi032.stdout:true 2024-05-12T04:07:41.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.361+0000 7f3f79c1d700 1 -- 172.21.15.32:0/4243470798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f740fcf60 msgr2=0x7f3f740fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:07:41.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.361+0000 7f3f79c1d700 1 --2- 172.21.15.32:0/4243470798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f740fcf60 0x7f3f740fd3d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3f68004940 tx=0x7f3f68009e30 comp rx=0 tx=0).stop 2024-05-12T04:07:41.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.361+0000 7f3f79c1d700 1 -- 172.21.15.32:0/4243470798 shutdown_connections 2024-05-12T04:07:41.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.361+0000 7f3f79c1d700 1 --2- 172.21.15.32:0/4243470798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f740fcf60 0x7f3f740fd3d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:41.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.361+0000 7f3f79c1d700 1 --2- 172.21.15.32:0/4243470798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f740fbcb0 0x7f3f740fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:41.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.361+0000 7f3f79c1d700 1 -- 172.21.15.32:0/4243470798 >> 172.21.15.32:0/4243470798 conn(0x7f3f740f7430 msgr2=0x7f3f740f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:07:41.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.361+0000 7f3f79c1d700 1 -- 172.21.15.32:0/4243470798 shutdown_connections 2024-05-12T04:07:41.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.361+0000 7f3f79c1d700 1 -- 172.21.15.32:0/4243470798 wait complete. 2024-05-12T04:07:41.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.362+0000 7f3f79c1d700 1 Processor -- start 2024-05-12T04:07:41.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.362+0000 7f3f79c1d700 1 -- start start 2024-05-12T04:07:41.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.362+0000 7f3f79c1d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f740fbcb0 0x7f3f741105a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:07:41.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.362+0000 7f3f79c1d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f740fcf60 0x7f3f74112af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:07:41.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.362+0000 7f3f79c1d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f74066330 con 0x7f3f740fbcb0 2024-05-12T04:07:41.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.362+0000 7f3f79c1d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f74066470 con 0x7f3f740fcf60 2024-05-12T04:07:41.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.362+0000 7f3f737fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f740fbcb0 0x7f3f741105a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:07:41.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.362+0000 7f3f737fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f740fbcb0 0x7f3f741105a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50174/0 (socket says 172.21.15.32:50174) 2024-05-12T04:07:41.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.362+0000 7f3f737fe700 1 -- 172.21.15.32:0/3131717518 learned_addr learned my addr 172.21.15.32:0/3131717518 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:07:41.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.363+0000 7f3f72ffd700 1 --2- 172.21.15.32:0/3131717518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f740fcf60 0x7f3f74112af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:07:41.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.363+0000 7f3f737fe700 1 -- 172.21.15.32:0/3131717518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f740fcf60 msgr2=0x7f3f74112af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:07:41.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.363+0000 7f3f737fe700 1 --2- 172.21.15.32:0/3131717518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f740fcf60 0x7f3f74112af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:41.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.363+0000 7f3f737fe700 1 -- 172.21.15.32:0/3131717518 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f68005040 con 0x7f3f740fbcb0 2024-05-12T04:07:41.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.363+0000 7f3f72ffd700 1 --2- 172.21.15.32:0/3131717518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f740fcf60 0x7f3f74112af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-12T04:07:41.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.363+0000 7f3f737fe700 1 --2- 172.21.15.32:0/3131717518 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f740fbcb0 0x7f3f741105a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3f6400eb40 tx=0x7f3f6400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:07:41.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.363+0000 7f3f5bfff700 1 -- 172.21.15.32:0/3131717518 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f64007cc0 con 0x7f3f740fbcb0 2024-05-12T04:07:41.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.363+0000 7f3f79c1d700 1 -- 172.21.15.32:0/3131717518 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f74113030 con 0x7f3f740fbcb0 2024-05-12T04:07:41.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.363+0000 7f3f5bfff700 1 -- 172.21.15.32:0/3131717518 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3f6400f040 con 0x7f3f740fbcb0 2024-05-12T04:07:41.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.363+0000 7f3f5bfff700 1 -- 172.21.15.32:0/3131717518 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f64009570 con 0x7f3f740fbcb0 2024-05-12T04:07:41.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.363+0000 7f3f79c1d700 1 -- 172.21.15.32:0/3131717518 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f741135f0 con 0x7f3f740fbcb0 2024-05-12T04:07:41.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.364+0000 7f3f79c1d700 1 -- 172.21.15.32:0/3131717518 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f7405f5c0 con 0x7f3f740fbcb0 2024-05-12T04:07:41.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.365+0000 7f3f5bfff700 1 -- 172.21.15.32:0/3131717518 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f3f640096d0 con 0x7f3f740fbcb0 2024-05-12T04:07:41.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.365+0000 7f3f5bfff700 1 --2- 172.21.15.32:0/3131717518 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f3f5c06cc20 0x7f3f5c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:07:41.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.365+0000 7f3f5bfff700 1 -- 172.21.15.32:0/3131717518 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f3f6408bce0 con 0x7f3f740fbcb0 2024-05-12T04:07:41.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.365+0000 7f3f72ffd700 1 --2- 172.21.15.32:0/3131717518 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f3f5c06cc20 0x7f3f5c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:07:41.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.366+0000 7f3f72ffd700 1 --2- 172.21.15.32:0/3131717518 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f3f5c06cc20 0x7f3f5c06f0d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3f680137b0 tx=0x7f3f68009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:07:41.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.370+0000 7f3f5bfff700 1 -- 172.21.15.32:0/3131717518 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f3f64055da0 con 0x7f3f740fbcb0 2024-05-12T04:07:41.592 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:41 smithi033 ceph-mon[107305]: pgmap v162: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 42/276 objects degraded (15.217%) 2024-05-12T04:07:41.592 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:41 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:07:41.593 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:41 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:07:41.593 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:41 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:07:41.593 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:41 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:07:41.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.597+0000 7f3f79c1d700 1 -- 172.21.15.32:0/3131717518 --> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3f74101340 con 0x7f3f5c06cc20 2024-05-12T04:07:41.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.598+0000 7f3f5bfff700 1 -- 172.21.15.32:0/3131717518 <== mgr.24593 v2:172.21.15.32:6800/2957111591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f3f74101340 con 0x7f3f5c06cc20 2024-05-12T04:07:41.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.600+0000 7f3f79c1d700 1 -- 172.21.15.32:0/3131717518 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f3f5c06cc20 msgr2=0x7f3f5c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:07:41.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.600+0000 7f3f79c1d700 1 --2- 172.21.15.32:0/3131717518 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f3f5c06cc20 0x7f3f5c06f0d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3f680137b0 tx=0x7f3f68009e30 comp rx=0 tx=0).stop 2024-05-12T04:07:41.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.600+0000 7f3f79c1d700 1 -- 172.21.15.32:0/3131717518 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f740fbcb0 msgr2=0x7f3f741105a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:07:41.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.600+0000 7f3f79c1d700 1 --2- 172.21.15.32:0/3131717518 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f740fbcb0 0x7f3f741105a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3f6400eb40 tx=0x7f3f6400b040 comp rx=0 tx=0).stop 2024-05-12T04:07:41.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.600+0000 7f3f79c1d700 1 -- 172.21.15.32:0/3131717518 shutdown_connections 2024-05-12T04:07:41.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.600+0000 7f3f79c1d700 1 --2- 172.21.15.32:0/3131717518 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f3f5c06cc20 0x7f3f5c06f0d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:41.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.600+0000 7f3f79c1d700 1 --2- 172.21.15.32:0/3131717518 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f740fbcb0 0x7f3f741105a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:41.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.600+0000 7f3f79c1d700 1 --2- 172.21.15.32:0/3131717518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f740fcf60 0x7f3f74112af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:41.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.600+0000 7f3f79c1d700 1 -- 172.21.15.32:0/3131717518 >> 172.21.15.32:0/3131717518 conn(0x7f3f740f7430 msgr2=0x7f3f740f8b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:07:41.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.600+0000 7f3f79c1d700 1 -- 172.21.15.32:0/3131717518 shutdown_connections 2024-05-12T04:07:41.602 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.600+0000 7f3f79c1d700 1 -- 172.21.15.32:0/3131717518 wait complete. 2024-05-12T04:07:41.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.704+0000 7fda7bd65700 1 -- 172.21.15.32:0/3841791855 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda740fcff0 msgr2=0x7fda740fd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:07:41.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.704+0000 7fda7bd65700 1 --2- 172.21.15.32:0/3841791855 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda740fcff0 0x7fda740fd460 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fda64004940 tx=0x7fda64009e30 comp rx=0 tx=0).stop 2024-05-12T04:07:41.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.704+0000 7fda7bd65700 1 -- 172.21.15.32:0/3841791855 shutdown_connections 2024-05-12T04:07:41.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.704+0000 7fda7bd65700 1 --2- 172.21.15.32:0/3841791855 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda740fcff0 0x7fda740fd460 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:41.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.704+0000 7fda7bd65700 1 --2- 172.21.15.32:0/3841791855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda740fbd40 0x7fda740fc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:41.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.704+0000 7fda7bd65700 1 -- 172.21.15.32:0/3841791855 >> 172.21.15.32:0/3841791855 conn(0x7fda740f74c0 msgr2=0x7fda740f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:07:41.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.704+0000 7fda7bd65700 1 -- 172.21.15.32:0/3841791855 shutdown_connections 2024-05-12T04:07:41.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.705+0000 7fda7bd65700 1 -- 172.21.15.32:0/3841791855 wait complete. 2024-05-12T04:07:41.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.705+0000 7fda7bd65700 1 Processor -- start 2024-05-12T04:07:41.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.705+0000 7fda7bd65700 1 -- start start 2024-05-12T04:07:41.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.705+0000 7fda7bd65700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda740fbd40 0x7fda74112990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:07:41.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.705+0000 7fda7bd65700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda740fcff0 0x7fda74114ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:07:41.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.705+0000 7fda7bd65700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda74066ab0 con 0x7fda740fcff0 2024-05-12T04:07:41.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.705+0000 7fda7bd65700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda74066bf0 con 0x7fda740fbd40 2024-05-12T04:07:41.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.705+0000 7fda79300700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda740fcff0 0x7fda74114ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:07:41.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.705+0000 7fda79b01700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda740fbd40 0x7fda74112990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:07:41.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.705+0000 7fda79300700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda740fcff0 0x7fda74114ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50188/0 (socket says 172.21.15.32:50188) 2024-05-12T04:07:41.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.705+0000 7fda79300700 1 -- 172.21.15.32:0/2296625410 learned_addr learned my addr 172.21.15.32:0/2296625410 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:07:41.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.706+0000 7fda79b01700 1 -- 172.21.15.32:0/2296625410 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda740fcff0 msgr2=0x7fda74114ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:07:41.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.706+0000 7fda79b01700 1 --2- 172.21.15.32:0/2296625410 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda740fcff0 0x7fda74114ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:41.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.706+0000 7fda79b01700 1 -- 172.21.15.32:0/2296625410 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda64005040 con 0x7fda740fbd40 2024-05-12T04:07:41.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.706+0000 7fda79300700 1 --2- 172.21.15.32:0/2296625410 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda740fcff0 0x7fda74114ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-12T04:07:41.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.706+0000 7fda79b01700 1 --2- 172.21.15.32:0/2296625410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda740fbd40 0x7fda74112990 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fda70009fd0 tx=0x7fda7000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:07:41.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.706+0000 7fda6a7fc700 1 -- 172.21.15.32:0/2296625410 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda70004e90 con 0x7fda740fbd40 2024-05-12T04:07:41.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.706+0000 7fda7bd65700 1 -- 172.21.15.32:0/2296625410 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda74115420 con 0x7fda740fbd40 2024-05-12T04:07:41.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.706+0000 7fda6a7fc700 1 -- 172.21.15.32:0/2296625410 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fda7000f040 con 0x7fda740fbd40 2024-05-12T04:07:41.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.706+0000 7fda6a7fc700 1 -- 172.21.15.32:0/2296625410 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda7000c9b0 con 0x7fda740fbd40 2024-05-12T04:07:41.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.706+0000 7fda7bd65700 1 -- 172.21.15.32:0/2296625410 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda741157f0 con 0x7fda740fbd40 2024-05-12T04:07:41.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.707+0000 7fda7bd65700 1 -- 172.21.15.32:0/2296625410 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda7405f5c0 con 0x7fda740fbd40 2024-05-12T04:07:41.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.708+0000 7fda6a7fc700 1 -- 172.21.15.32:0/2296625410 <== mon.1 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fda70007cc0 con 0x7fda740fbd40 2024-05-12T04:07:41.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.708+0000 7fda6a7fc700 1 --2- 172.21.15.32:0/2296625410 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fda6006cc20 0x7fda6006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:07:41.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.708+0000 7fda6a7fc700 1 -- 172.21.15.32:0/2296625410 <== mon.1 v2:172.21.15.33:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fda7008ba90 con 0x7fda740fbd40 2024-05-12T04:07:41.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.708+0000 7fda79300700 1 --2- 172.21.15.32:0/2296625410 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fda6006cc20 0x7fda6006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:07:41.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.709+0000 7fda79300700 1 --2- 172.21.15.32:0/2296625410 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fda6006cc20 0x7fda6006f0d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fda640135e0 tx=0x7fda64009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:07:41.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.714+0000 7fda6a7fc700 1 -- 172.21.15.32:0/2296625410 <== mon.1 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fda70055af0 con 0x7fda740fbd40 2024-05-12T04:07:41.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:41 smithi032 ceph-mon[120392]: pgmap v162: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 42/276 objects degraded (15.217%) 2024-05-12T04:07:41.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:41 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:07:41.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:41 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:07:41.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:41 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:07:41.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:41 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:07:41.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.925+0000 7fda7bd65700 1 -- 172.21.15.32:0/2296625410 --> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fda741013b0 con 0x7fda6006cc20 2024-05-12T04:07:41.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.935+0000 7fda6a7fc700 1 -- 172.21.15.32:0/2296625410 <== mgr.24593 v2:172.21.15.32:6800/2957111591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fda741013b0 con 0x7fda6006cc20 2024-05-12T04:07:41.936 INFO:teuthology.orchestra.run.smithi032.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-12T04:07:41.936 INFO:teuthology.orchestra.run.smithi032.stdout:alertmanager.smithi032 smithi032 *:9093,9094 running (6m) 99s ago 15m 25.5M - 0.25.0 c8568f914cd2 a37fbab7af4c 2024-05-12T04:07:41.936 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi032 smithi032 running (3m) 99s ago 15m 6660k - 18.2.2-1272-g58249552 ddcb0a9d6fde 8c53f1fc7b06 2024-05-12T04:07:41.937 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi033 smithi033 running (3m) 1s ago 14m 6651k - 18.2.2-1272-g58249552 ddcb0a9d6fde cf4fe9ca070b 2024-05-12T04:07:41.937 INFO:teuthology.orchestra.run.smithi032.stdout:grafana.smithi032 smithi032 *:3000 running (6m) 99s ago 15m 89.7M - 9.4.7 954c08fa6188 83055e58d23c 2024-05-12T04:07:41.937 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.ialtqb smithi032 running (9m) 99s ago 9m 13.9M - 17.2.7-939-gc190bdee 284803f7d9cf 0683abf00e3e 2024-05-12T04:07:41.937 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.qpsawh smithi032 running (9m) 99s ago 9m 155M - 17.2.7-939-gc190bdee 284803f7d9cf 95a7db5c4fcd 2024-05-12T04:07:41.937 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi033.hglwml smithi033 running (9m) 1s ago 9m 14.0M - 17.2.7-939-gc190bdee 284803f7d9cf c97902b05025 2024-05-12T04:07:41.937 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi033.jgbjhk smithi033 running (9m) 1s ago 9m 102M - 17.2.7-939-gc190bdee 284803f7d9cf af97052cb97a 2024-05-12T04:07:41.937 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi032.ukatkq smithi032 *:8443,9283,8765 running (5m) 99s ago 17m 501M - 18.2.2-1272-g58249552 ddcb0a9d6fde 0d88ab9a5292 2024-05-12T04:07:41.937 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi033.pcvxwy smithi033 *:8443,9283,8765 running (5m) 1s ago 14m 450M - 18.2.2-1272-g58249552 ddcb0a9d6fde d42dcb6bd307 2024-05-12T04:07:41.937 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi032 smithi032 running (4m) 99s ago 17m 45.5M 2048M 18.2.2-1272-g58249552 ddcb0a9d6fde 81f94171c5dc 2024-05-12T04:07:41.937 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi033 smithi033 running (4m) 1s ago 14m 36.2M 2048M 18.2.2-1272-g58249552 ddcb0a9d6fde d648ba108f3e 2024-05-12T04:07:41.937 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi032 smithi032 *:9100 running (7m) 99s ago 15m 16.5M - 1.5.0 0da6a335fe13 cb75a7c0c829 2024-05-12T04:07:41.937 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi033 smithi033 *:9100 running (5m) 1s ago 14m 18.9M - 1.5.0 0da6a335fe13 7888c36c659f 2024-05-12T04:07:41.937 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0 smithi032 running (2m) 99s ago 13m 80.8M 1570M 18.2.2-1272-g58249552 ddcb0a9d6fde 07f66c4deb3b 2024-05-12T04:07:41.937 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1 smithi032 running (2m) 99s ago 13m 84.5M 1570M 18.2.2-1272-g58249552 ddcb0a9d6fde 67a87807d654 2024-05-12T04:07:41.937 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2 smithi032 running (105s) 99s ago 12m 11.9M 1570M 18.2.2-1272-g58249552 ddcb0a9d6fde 4ac48d74ee8b 2024-05-12T04:07:41.937 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3 smithi033 running (70s) 1s ago 12m 90.9M 2594M 18.2.2-1272-g58249552 ddcb0a9d6fde 11f097a455bb 2024-05-12T04:07:41.938 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4 smithi033 running (37s) 1s ago 11m 84.8M 2594M 18.2.2-1272-g58249552 ddcb0a9d6fde db6fda7d52de 2024-05-12T04:07:41.938 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5 smithi033 running (5s) 1s ago 11m 11.4M 2594M 18.2.2-1272-g58249552 ddcb0a9d6fde 3333020b8a15 2024-05-12T04:07:41.938 INFO:teuthology.orchestra.run.smithi032.stdout:prometheus.smithi032 smithi032 *:9095 running (5m) 99s ago 15m 54.2M - 2.43.0 a07b618ecd1d 46eafb25e176 2024-05-12T04:07:41.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.937+0000 7fda7bd65700 1 -- 172.21.15.32:0/2296625410 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fda6006cc20 msgr2=0x7fda6006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:07:41.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.937+0000 7fda7bd65700 1 --2- 172.21.15.32:0/2296625410 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fda6006cc20 0x7fda6006f0d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fda640135e0 tx=0x7fda64009e30 comp rx=0 tx=0).stop 2024-05-12T04:07:41.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.937+0000 7fda7bd65700 1 -- 172.21.15.32:0/2296625410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda740fbd40 msgr2=0x7fda74112990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:07:41.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.937+0000 7fda7bd65700 1 --2- 172.21.15.32:0/2296625410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda740fbd40 0x7fda74112990 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fda70009fd0 tx=0x7fda7000b040 comp rx=0 tx=0).stop 2024-05-12T04:07:41.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.937+0000 7fda7bd65700 1 -- 172.21.15.32:0/2296625410 shutdown_connections 2024-05-12T04:07:41.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.937+0000 7fda7bd65700 1 --2- 172.21.15.32:0/2296625410 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fda6006cc20 0x7fda6006f0d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:41.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.937+0000 7fda7bd65700 1 --2- 172.21.15.32:0/2296625410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda740fbd40 0x7fda74112990 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:41.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.937+0000 7fda7bd65700 1 --2- 172.21.15.32:0/2296625410 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda740fcff0 0x7fda74114ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:41.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.937+0000 7fda7bd65700 1 -- 172.21.15.32:0/2296625410 >> 172.21.15.32:0/2296625410 conn(0x7fda740f74c0 msgr2=0x7fda740f8bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:07:41.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.937+0000 7fda7bd65700 1 -- 172.21.15.32:0/2296625410 shutdown_connections 2024-05-12T04:07:41.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:41.937+0000 7fda7bd65700 1 -- 172.21.15.32:0/2296625410 wait complete. 2024-05-12T04:07:42.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.040+0000 7f761acb4700 1 -- 172.21.15.32:0/2906261020 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76140fe3e0 msgr2=0x7f76140fe7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:07:42.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.040+0000 7f761acb4700 1 --2- 172.21.15.32:0/2906261020 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76140fe3e0 0x7f76140fe7f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7608004940 tx=0x7f7608009e30 comp rx=0 tx=0).stop 2024-05-12T04:07:42.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.040+0000 7f761acb4700 1 -- 172.21.15.32:0/2906261020 shutdown_connections 2024-05-12T04:07:42.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.040+0000 7f761acb4700 1 --2- 172.21.15.32:0/2906261020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76140fed30 0x7f76140fae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:42.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.040+0000 7f761acb4700 1 --2- 172.21.15.32:0/2906261020 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76140fe3e0 0x7f76140fe7f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:42.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.040+0000 7f761acb4700 1 -- 172.21.15.32:0/2906261020 >> 172.21.15.32:0/2906261020 conn(0x7f76140f6b80 msgr2=0x7f76140f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:07:42.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.040+0000 7f761acb4700 1 -- 172.21.15.32:0/2906261020 shutdown_connections 2024-05-12T04:07:42.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.040+0000 7f761acb4700 1 -- 172.21.15.32:0/2906261020 wait complete. 2024-05-12T04:07:42.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.041+0000 7f761acb4700 1 Processor -- start 2024-05-12T04:07:42.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.041+0000 7f761acb4700 1 -- start start 2024-05-12T04:07:42.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.041+0000 7f761acb4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76140fe3e0 0x7f761410d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:07:42.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.041+0000 7f761acb4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76140fed30 0x7f761410dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:07:42.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.041+0000 7f761acb4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f761410e390 con 0x7f76140fe3e0 2024-05-12T04:07:42.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.041+0000 7f761acb4700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f761410ebe0 con 0x7f76140fed30 2024-05-12T04:07:42.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.041+0000 7f7618a50700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76140fe3e0 0x7f761410d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:07:42.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.041+0000 7f7618a50700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76140fe3e0 0x7f761410d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43488/0 (socket says 172.21.15.32:43488) 2024-05-12T04:07:42.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.041+0000 7f7618a50700 1 -- 172.21.15.32:0/1908587392 learned_addr learned my addr 172.21.15.32:0/1908587392 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:07:42.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.041+0000 7f7613fff700 1 --2- 172.21.15.32:0/1908587392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76140fed30 0x7f761410dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:07:42.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.041+0000 7f7618a50700 1 -- 172.21.15.32:0/1908587392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76140fed30 msgr2=0x7f761410dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:07:42.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.041+0000 7f7618a50700 1 --2- 172.21.15.32:0/1908587392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76140fed30 0x7f761410dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:42.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.041+0000 7f7618a50700 1 -- 172.21.15.32:0/1908587392 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7608005040 con 0x7f76140fe3e0 2024-05-12T04:07:42.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.042+0000 7f7618a50700 1 --2- 172.21.15.32:0/1908587392 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76140fe3e0 0x7f761410d850 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f76080137b0 tx=0x7f7608009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:07:42.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.042+0000 7f76117fa700 1 -- 172.21.15.32:0/1908587392 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7608021070 con 0x7f76140fe3e0 2024-05-12T04:07:42.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.042+0000 7f76117fa700 1 -- 172.21.15.32:0/1908587392 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f760801c400 con 0x7f76140fe3e0 2024-05-12T04:07:42.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.042+0000 7f761acb4700 1 -- 172.21.15.32:0/1908587392 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f761410ee60 con 0x7f76140fe3e0 2024-05-12T04:07:42.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.042+0000 7f76117fa700 1 -- 172.21.15.32:0/1908587392 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f760800e070 con 0x7f76140fe3e0 2024-05-12T04:07:42.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.042+0000 7f761acb4700 1 -- 172.21.15.32:0/1908587392 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f761410f340 con 0x7f76140fe3e0 2024-05-12T04:07:42.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.043+0000 7f761acb4700 1 -- 172.21.15.32:0/1908587392 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f761405f5c0 con 0x7f76140fe3e0 2024-05-12T04:07:42.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.043+0000 7f76117fa700 1 -- 172.21.15.32:0/1908587392 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f7608013a30 con 0x7f76140fe3e0 2024-05-12T04:07:42.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.044+0000 7f76117fa700 1 --2- 172.21.15.32:0/1908587392 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f760406cc30 0x7f760406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:07:42.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.044+0000 7f76117fa700 1 -- 172.21.15.32:0/1908587392 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f760808d120 con 0x7f76140fe3e0 2024-05-12T04:07:42.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.044+0000 7f7613fff700 1 --2- 172.21.15.32:0/1908587392 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f760406cc30 0x7f760406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:07:42.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.044+0000 7f7613fff700 1 --2- 172.21.15.32:0/1908587392 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f760406cc30 0x7f760406f0e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7600009b80 tx=0x7f760000b040 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:07:42.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.049+0000 7f76117fa700 1 -- 172.21.15.32:0/1908587392 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f760805ace0 con 0x7f76140fe3e0 2024-05-12T04:07:42.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.339+0000 7f761acb4700 1 -- 172.21.15.32:0/1908587392 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f761410f710 con 0x7f76140fe3e0 2024-05-12T04:07:42.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.339+0000 7f76117fa700 1 -- 172.21.15.32:0/1908587392 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+740 (secure 0 0 0) 0x7f760805a870 con 0x7f76140fe3e0 2024-05-12T04:07:42.341 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-05-12T04:07:42.341 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": { 2024-05-12T04:07:42.341 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 2 2024-05-12T04:07:42.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:07:42.341 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": { 2024-05-12T04:07:42.341 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 2 2024-05-12T04:07:42.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:07:42.341 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": { 2024-05-12T04:07:42.341 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 5 2024-05-12T04:07:42.341 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:07:42.342 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": { 2024-05-12T04:07:42.342 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 4 2024-05-12T04:07:42.342 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:07:42.342 INFO:teuthology.orchestra.run.smithi032.stdout: "overall": { 2024-05-12T04:07:42.342 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 4, 2024-05-12T04:07:42.342 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 9 2024-05-12T04:07:42.342 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-05-12T04:07:42.342 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-05-12T04:07:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.342+0000 7f761acb4700 1 -- 172.21.15.32:0/1908587392 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f760406cc30 msgr2=0x7f760406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:07:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.342+0000 7f761acb4700 1 --2- 172.21.15.32:0/1908587392 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f760406cc30 0x7f760406f0e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7600009b80 tx=0x7f760000b040 comp rx=0 tx=0).stop 2024-05-12T04:07:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.342+0000 7f761acb4700 1 -- 172.21.15.32:0/1908587392 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76140fe3e0 msgr2=0x7f761410d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:07:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.342+0000 7f761acb4700 1 --2- 172.21.15.32:0/1908587392 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76140fe3e0 0x7f761410d850 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f76080137b0 tx=0x7f7608009e30 comp rx=0 tx=0).stop 2024-05-12T04:07:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.342+0000 7f761acb4700 1 -- 172.21.15.32:0/1908587392 shutdown_connections 2024-05-12T04:07:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.342+0000 7f761acb4700 1 --2- 172.21.15.32:0/1908587392 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f760406cc30 0x7f760406f0e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.342+0000 7f761acb4700 1 --2- 172.21.15.32:0/1908587392 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76140fe3e0 0x7f761410d850 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:42.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.342+0000 7f761acb4700 1 --2- 172.21.15.32:0/1908587392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76140fed30 0x7f761410dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:42.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.342+0000 7f761acb4700 1 -- 172.21.15.32:0/1908587392 >> 172.21.15.32:0/1908587392 conn(0x7f76140f6b80 msgr2=0x7f76140f82b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:07:42.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.342+0000 7f761acb4700 1 -- 172.21.15.32:0/1908587392 shutdown_connections 2024-05-12T04:07:42.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.342+0000 7f761acb4700 1 -- 172.21.15.32:0/1908587392 wait complete. 2024-05-12T04:07:42.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.445+0000 7fbb69b3a700 1 -- 172.21.15.32:0/4159196911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb640fcf60 msgr2=0x7fbb640fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:07:42.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.445+0000 7fbb69b3a700 1 --2- 172.21.15.32:0/4159196911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb640fcf60 0x7fbb640fd3d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbb58004940 tx=0x7fbb58009e30 comp rx=0 tx=0).stop 2024-05-12T04:07:42.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.445+0000 7fbb69b3a700 1 -- 172.21.15.32:0/4159196911 shutdown_connections 2024-05-12T04:07:42.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.445+0000 7fbb69b3a700 1 --2- 172.21.15.32:0/4159196911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb640fcf60 0x7fbb640fd3d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:42.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.445+0000 7fbb69b3a700 1 --2- 172.21.15.32:0/4159196911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb640fbcb0 0x7fbb640fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:42.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.445+0000 7fbb69b3a700 1 -- 172.21.15.32:0/4159196911 >> 172.21.15.32:0/4159196911 conn(0x7fbb640f7430 msgr2=0x7fbb640f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:07:42.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.446+0000 7fbb69b3a700 1 -- 172.21.15.32:0/4159196911 shutdown_connections 2024-05-12T04:07:42.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.446+0000 7fbb69b3a700 1 -- 172.21.15.32:0/4159196911 wait complete. 2024-05-12T04:07:42.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.446+0000 7fbb69b3a700 1 Processor -- start 2024-05-12T04:07:42.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.446+0000 7fbb69b3a700 1 -- start start 2024-05-12T04:07:42.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.446+0000 7fbb69b3a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb640fbcb0 0x7fbb641105a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:07:42.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.447+0000 7fbb69b3a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb640fcf60 0x7fbb64112af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:07:42.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.447+0000 7fbb69b3a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb64066330 con 0x7fbb640fcf60 2024-05-12T04:07:42.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.447+0000 7fbb69b3a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb64066470 con 0x7fbb640fbcb0 2024-05-12T04:07:42.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.447+0000 7fbb62ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb640fcf60 0x7fbb64112af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:07:42.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.447+0000 7fbb637fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb640fbcb0 0x7fbb641105a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:07:42.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.447+0000 7fbb62ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb640fcf60 0x7fbb64112af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43508/0 (socket says 172.21.15.32:43508) 2024-05-12T04:07:42.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.447+0000 7fbb62ffd700 1 -- 172.21.15.32:0/4217441773 learned_addr learned my addr 172.21.15.32:0/4217441773 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:07:42.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.447+0000 7fbb62ffd700 1 -- 172.21.15.32:0/4217441773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb640fbcb0 msgr2=0x7fbb641105a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:07:42.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.447+0000 7fbb62ffd700 1 --2- 172.21.15.32:0/4217441773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb640fbcb0 0x7fbb641105a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:42.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.447+0000 7fbb62ffd700 1 -- 172.21.15.32:0/4217441773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb58005040 con 0x7fbb640fcf60 2024-05-12T04:07:42.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.447+0000 7fbb62ffd700 1 --2- 172.21.15.32:0/4217441773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb640fcf60 0x7fbb64112af0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbb5800b3b0 tx=0x7fbb58009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:07:42.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.447+0000 7fbb4bfff700 1 -- 172.21.15.32:0/4217441773 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb5800e040 con 0x7fbb640fcf60 2024-05-12T04:07:42.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.447+0000 7fbb4bfff700 1 -- 172.21.15.32:0/4217441773 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbb5800bc50 con 0x7fbb640fcf60 2024-05-12T04:07:42.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.447+0000 7fbb69b3a700 1 -- 172.21.15.32:0/4217441773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb64113030 con 0x7fbb640fcf60 2024-05-12T04:07:42.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.447+0000 7fbb4bfff700 1 -- 172.21.15.32:0/4217441773 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb5801c630 con 0x7fbb640fcf60 2024-05-12T04:07:42.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.447+0000 7fbb69b3a700 1 -- 172.21.15.32:0/4217441773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb641135f0 con 0x7fbb640fcf60 2024-05-12T04:07:42.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.448+0000 7fbb69b3a700 1 -- 172.21.15.32:0/4217441773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb6405f5c0 con 0x7fbb640fcf60 2024-05-12T04:07:42.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.449+0000 7fbb4bfff700 1 -- 172.21.15.32:0/4217441773 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fbb58013420 con 0x7fbb640fcf60 2024-05-12T04:07:42.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.449+0000 7fbb4bfff700 1 --2- 172.21.15.32:0/4217441773 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fbb4c06cc20 0x7fbb4c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:07:42.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.449+0000 7fbb4bfff700 1 -- 172.21.15.32:0/4217441773 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fbb5808cbf0 con 0x7fbb640fcf60 2024-05-12T04:07:42.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.449+0000 7fbb637fe700 1 --2- 172.21.15.32:0/4217441773 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fbb4c06cc20 0x7fbb4c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:07:42.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.450+0000 7fbb637fe700 1 --2- 172.21.15.32:0/4217441773 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fbb4c06cc20 0x7fbb4c06f0d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbb64065fb0 tx=0x7fbb5400b040 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:07:42.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.455+0000 7fbb4bfff700 1 -- 172.21.15.32:0/4217441773 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fbb58056c80 con 0x7fbb640fcf60 2024-05-12T04:07:42.646 INFO:journalctl@ceph.osd.5.smithi033.stdout:May 12 04:07:42 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-5[132613]: 2024-05-12T04:07:42.240+0000 7f72848927c0 -1 osd.5 74 log_to_monitors true 2024-05-12T04:07:42.646 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:42 smithi033 ceph-mon[107305]: from='client.34370 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:07:42.646 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:42 smithi033 ceph-mon[107305]: from='osd.5 [v2:172.21.15.33:6816/4274440090,v1:172.21.15.33:6817/4274440090]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-12T04:07:42.646 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:42 smithi033 ceph-mon[107305]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-12T04:07:42.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.701+0000 7fbb69b3a700 1 -- 172.21.15.32:0/4217441773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fbb6404bed0 con 0x7fbb640fcf60 2024-05-12T04:07:42.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.701+0000 7fbb4bfff700 1 -- 172.21.15.32:0/4217441773 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1866 (secure 0 0 0) 0x7fbb5805a2a0 con 0x7fbb640fcf60 2024-05-12T04:07:42.702 INFO:teuthology.orchestra.run.smithi032.stdout:e18 2024-05-12T04:07:42.702 INFO:teuthology.orchestra.run.smithi032.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-12T04:07:42.702 INFO:teuthology.orchestra.run.smithi032.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-12T04:07:42.702 INFO:teuthology.orchestra.run.smithi032.stdout:legacy client fscid: 1 2024-05-12T04:07:42.702 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:07:42.702 INFO:teuthology.orchestra.run.smithi032.stdout:Filesystem 'cephfs' (1) 2024-05-12T04:07:42.702 INFO:teuthology.orchestra.run.smithi032.stdout:fs_name cephfs 2024-05-12T04:07:42.702 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 18 2024-05-12T04:07:42.702 INFO:teuthology.orchestra.run.smithi032.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-12T04:07:42.702 INFO:teuthology.orchestra.run.smithi032.stdout:created 2024-05-12T03:57:48.904614+0000 2024-05-12T04:07:42.702 INFO:teuthology.orchestra.run.smithi032.stdout:modified 2024-05-12T03:58:06.821591+0000 2024-05-12T04:07:42.702 INFO:teuthology.orchestra.run.smithi032.stdout:tableserver 0 2024-05-12T04:07:42.702 INFO:teuthology.orchestra.run.smithi032.stdout:root 0 2024-05-12T04:07:42.703 INFO:teuthology.orchestra.run.smithi032.stdout:session_timeout 60 2024-05-12T04:07:42.703 INFO:teuthology.orchestra.run.smithi032.stdout:session_autoclose 300 2024-05-12T04:07:42.703 INFO:teuthology.orchestra.run.smithi032.stdout:max_file_size 1099511627776 2024-05-12T04:07:42.703 INFO:teuthology.orchestra.run.smithi032.stdout:max_xattr_size 65536 2024-05-12T04:07:42.703 INFO:teuthology.orchestra.run.smithi032.stdout:required_client_features {} 2024-05-12T04:07:42.703 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure 0 2024-05-12T04:07:42.703 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure_osd_epoch 46 2024-05-12T04:07:42.703 INFO:teuthology.orchestra.run.smithi032.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-12T04:07:42.703 INFO:teuthology.orchestra.run.smithi032.stdout:max_mds 1 2024-05-12T04:07:42.703 INFO:teuthology.orchestra.run.smithi032.stdout:in 0 2024-05-12T04:07:42.703 INFO:teuthology.orchestra.run.smithi032.stdout:up {0=14474} 2024-05-12T04:07:42.703 INFO:teuthology.orchestra.run.smithi032.stdout:failed 2024-05-12T04:07:42.703 INFO:teuthology.orchestra.run.smithi032.stdout:damaged 2024-05-12T04:07:42.703 INFO:teuthology.orchestra.run.smithi032.stdout:stopped 2024-05-12T04:07:42.704 INFO:teuthology.orchestra.run.smithi032.stdout:data_pools [3] 2024-05-12T04:07:42.704 INFO:teuthology.orchestra.run.smithi032.stdout:metadata_pool 2 2024-05-12T04:07:42.704 INFO:teuthology.orchestra.run.smithi032.stdout:inline_data enabled 2024-05-12T04:07:42.704 INFO:teuthology.orchestra.run.smithi032.stdout:balancer 2024-05-12T04:07:42.704 INFO:teuthology.orchestra.run.smithi032.stdout:bal_rank_mask -1 2024-05-12T04:07:42.704 INFO:teuthology.orchestra.run.smithi032.stdout:standby_count_wanted 1 2024-05-12T04:07:42.704 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.qpsawh{0:14474} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.32:6828/3100155296,v1:172.21.15.32:6829/3100155296] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:07:42.704 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi033.jgbjhk{0:24267} state up:standby-replay seq 3 join_fscid=1 addr [v2:172.21.15.33:6824/55417471,v1:172.21.15.33:6825/55417471] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:07:42.704 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:07:42.704 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:07:42.704 INFO:teuthology.orchestra.run.smithi032.stdout:Standby daemons: 2024-05-12T04:07:42.704 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:07:42.704 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi033.hglwml{-1:24283} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/433858238,v1:172.21.15.33:6827/433858238] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:07:42.704 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.ialtqb{-1:24287} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.32:6826/3703132329,v1:172.21.15.32:6827/3703132329] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:07:42.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.703+0000 7fbb69b3a700 1 -- 172.21.15.32:0/4217441773 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fbb4c06cc20 msgr2=0x7fbb4c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:07:42.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.703+0000 7fbb69b3a700 1 --2- 172.21.15.32:0/4217441773 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fbb4c06cc20 0x7fbb4c06f0d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbb64065fb0 tx=0x7fbb5400b040 comp rx=0 tx=0).stop 2024-05-12T04:07:42.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.703+0000 7fbb69b3a700 1 -- 172.21.15.32:0/4217441773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb640fcf60 msgr2=0x7fbb64112af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:07:42.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.703+0000 7fbb69b3a700 1 --2- 172.21.15.32:0/4217441773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb640fcf60 0x7fbb64112af0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbb5800b3b0 tx=0x7fbb58009e30 comp rx=0 tx=0).stop 2024-05-12T04:07:42.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.704+0000 7fbb69b3a700 1 -- 172.21.15.32:0/4217441773 shutdown_connections 2024-05-12T04:07:42.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.704+0000 7fbb69b3a700 1 --2- 172.21.15.32:0/4217441773 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fbb4c06cc20 0x7fbb4c06f0d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:42.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.704+0000 7fbb69b3a700 1 --2- 172.21.15.32:0/4217441773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb640fbcb0 0x7fbb641105a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:42.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.704+0000 7fbb69b3a700 1 --2- 172.21.15.32:0/4217441773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb640fcf60 0x7fbb64112af0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:42.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.704+0000 7fbb69b3a700 1 -- 172.21.15.32:0/4217441773 >> 172.21.15.32:0/4217441773 conn(0x7fbb640f7430 msgr2=0x7fbb640f8b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:07:42.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.704+0000 7fbb69b3a700 1 -- 172.21.15.32:0/4217441773 shutdown_connections 2024-05-12T04:07:42.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.704+0000 7fbb69b3a700 1 -- 172.21.15.32:0/4217441773 wait complete. 2024-05-12T04:07:42.706 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 18 2024-05-12T04:07:42.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:42 smithi032 ceph-mon[120392]: from='client.34370 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:07:42.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:42 smithi032 ceph-mon[120392]: from='osd.5 [v2:172.21.15.33:6816/4274440090,v1:172.21.15.33:6817/4274440090]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-12T04:07:42.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:42 smithi032 ceph-mon[120392]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-12T04:07:42.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.808+0000 7f5f53d61700 1 -- 172.21.15.32:0/38972869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f4c0fbcb0 msgr2=0x7f5f4c0fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:07:42.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.808+0000 7f5f53d61700 1 --2- 172.21.15.32:0/38972869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f4c0fbcb0 0x7f5f4c0fc0c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5f48004940 tx=0x7f5f48009e30 comp rx=0 tx=0).stop 2024-05-12T04:07:42.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.808+0000 7f5f53d61700 1 -- 172.21.15.32:0/38972869 shutdown_connections 2024-05-12T04:07:42.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.808+0000 7f5f53d61700 1 --2- 172.21.15.32:0/38972869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f4c0fcf60 0x7f5f4c0fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:42.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.808+0000 7f5f53d61700 1 --2- 172.21.15.32:0/38972869 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f4c0fbcb0 0x7f5f4c0fc0c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:42.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.808+0000 7f5f53d61700 1 -- 172.21.15.32:0/38972869 >> 172.21.15.32:0/38972869 conn(0x7f5f4c0f7430 msgr2=0x7f5f4c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:07:42.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.809+0000 7f5f53d61700 1 -- 172.21.15.32:0/38972869 shutdown_connections 2024-05-12T04:07:42.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.809+0000 7f5f53d61700 1 -- 172.21.15.32:0/38972869 wait complete. 2024-05-12T04:07:42.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.809+0000 7f5f53d61700 1 Processor -- start 2024-05-12T04:07:42.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.809+0000 7f5f53d61700 1 -- start start 2024-05-12T04:07:42.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.810+0000 7f5f53d61700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f4c0fbcb0 0x7f5f4c110610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:07:42.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.810+0000 7f5f53d61700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f4c0fcf60 0x7f5f4c112b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:07:42.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.810+0000 7f5f53d61700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4c0663e0 con 0x7f5f4c0fcf60 2024-05-12T04:07:42.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.810+0000 7f5f53d61700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4c066520 con 0x7f5f4c0fbcb0 2024-05-12T04:07:42.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.810+0000 7f5f512fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f4c0fcf60 0x7f5f4c112b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:07:42.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.810+0000 7f5f512fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f4c0fcf60 0x7f5f4c112b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43522/0 (socket says 172.21.15.32:43522) 2024-05-12T04:07:42.811 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.810+0000 7f5f512fc700 1 -- 172.21.15.32:0/3611115308 learned_addr learned my addr 172.21.15.32:0/3611115308 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:07:42.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.810+0000 7f5f51afd700 1 --2- 172.21.15.32:0/3611115308 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f4c0fbcb0 0x7f5f4c110610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:07:42.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.810+0000 7f5f512fc700 1 -- 172.21.15.32:0/3611115308 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f4c0fbcb0 msgr2=0x7f5f4c110610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:07:42.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.810+0000 7f5f512fc700 1 --2- 172.21.15.32:0/3611115308 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f4c0fbcb0 0x7f5f4c110610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:42.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.810+0000 7f5f512fc700 1 -- 172.21.15.32:0/3611115308 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f48005040 con 0x7f5f4c0fcf60 2024-05-12T04:07:42.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.810+0000 7f5f512fc700 1 --2- 172.21.15.32:0/3611115308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f4c0fcf60 0x7f5f4c112b60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c00b7c0 tx=0x7f5f3c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:07:42.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.810+0000 7f5f427fc700 1 -- 172.21.15.32:0/3611115308 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f3c00f6f0 con 0x7f5f4c0fcf60 2024-05-12T04:07:42.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.810+0000 7f5f53d61700 1 -- 172.21.15.32:0/3611115308 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f4c1131c0 con 0x7f5f4c0fcf60 2024-05-12T04:07:42.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.810+0000 7f5f427fc700 1 -- 172.21.15.32:0/3611115308 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5f3c00fd00 con 0x7f5f4c0fcf60 2024-05-12T04:07:42.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.810+0000 7f5f427fc700 1 -- 172.21.15.32:0/3611115308 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f3c011e90 con 0x7f5f4c0fcf60 2024-05-12T04:07:42.812 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.810+0000 7f5f53d61700 1 -- 172.21.15.32:0/3611115308 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f4c113780 con 0x7f5f4c0fcf60 2024-05-12T04:07:42.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.811+0000 7f5f53d61700 1 -- 172.21.15.32:0/3611115308 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f4c05f5c0 con 0x7f5f4c0fcf60 2024-05-12T04:07:42.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.812+0000 7f5f427fc700 1 -- 172.21.15.32:0/3611115308 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f5f3c00f850 con 0x7f5f4c0fcf60 2024-05-12T04:07:42.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.812+0000 7f5f427fc700 1 --2- 172.21.15.32:0/3611115308 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f5f3806cc30 0x7f5f3806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:07:42.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.812+0000 7f5f427fc700 1 -- 172.21.15.32:0/3611115308 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f5f3c08beb0 con 0x7f5f4c0fcf60 2024-05-12T04:07:42.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.812+0000 7f5f51afd700 1 --2- 172.21.15.32:0/3611115308 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f5f3806cc30 0x7f5f3806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:07:42.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.813+0000 7f5f51afd700 1 --2- 172.21.15.32:0/3611115308 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f5f3806cc30 0x7f5f3806f0e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5f48000c00 tx=0x7f5f48009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:07:42.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:42.817+0000 7f5f427fc700 1 -- 172.21.15.32:0/3611115308 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f5f3c055ea0 con 0x7f5f4c0fcf60 2024-05-12T04:07:43.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:43.046+0000 7f5f53d61700 1 -- 172.21.15.32:0/3611115308 --> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5f4c101320 con 0x7f5f3806cc30 2024-05-12T04:07:43.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:43.047+0000 7f5f427fc700 1 -- 172.21.15.32:0/3611115308 <== mgr.24593 v2:172.21.15.32:6800/2957111591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f5f4c101320 con 0x7f5f3806cc30 2024-05-12T04:07:43.047 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-05-12T04:07:43.048 INFO:teuthology.orchestra.run.smithi032.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:19c9734256aa43a60e203feb242f50bf8b7f58666d520536af43cc4fee96173a", 2024-05-12T04:07:43.048 INFO:teuthology.orchestra.run.smithi032.stdout: "in_progress": true, 2024-05-12T04:07:43.048 INFO:teuthology.orchestra.run.smithi032.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-12T04:07:43.048 INFO:teuthology.orchestra.run.smithi032.stdout: "services_complete": [ 2024-05-12T04:07:43.048 INFO:teuthology.orchestra.run.smithi032.stdout: "osd", 2024-05-12T04:07:43.048 INFO:teuthology.orchestra.run.smithi032.stdout: "mon", 2024-05-12T04:07:43.048 INFO:teuthology.orchestra.run.smithi032.stdout: "crash", 2024-05-12T04:07:43.048 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr" 2024-05-12T04:07:43.048 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-05-12T04:07:43.048 INFO:teuthology.orchestra.run.smithi032.stdout: "progress": "12/21 daemons upgraded", 2024-05-12T04:07:43.048 INFO:teuthology.orchestra.run.smithi032.stdout: "message": "Currently upgrading osd daemons", 2024-05-12T04:07:43.048 INFO:teuthology.orchestra.run.smithi032.stdout: "is_paused": false 2024-05-12T04:07:43.048 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-05-12T04:07:43.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:43.049+0000 7f5f53d61700 1 -- 172.21.15.32:0/3611115308 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f5f3806cc30 msgr2=0x7f5f3806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:07:43.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:43.049+0000 7f5f53d61700 1 --2- 172.21.15.32:0/3611115308 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f5f3806cc30 0x7f5f3806f0e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5f48000c00 tx=0x7f5f48009e30 comp rx=0 tx=0).stop 2024-05-12T04:07:43.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:43.049+0000 7f5f53d61700 1 -- 172.21.15.32:0/3611115308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f4c0fcf60 msgr2=0x7f5f4c112b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:07:43.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:43.049+0000 7f5f53d61700 1 --2- 172.21.15.32:0/3611115308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f4c0fcf60 0x7f5f4c112b60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c00b7c0 tx=0x7f5f3c009250 comp rx=0 tx=0).stop 2024-05-12T04:07:43.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:43.049+0000 7f5f53d61700 1 -- 172.21.15.32:0/3611115308 shutdown_connections 2024-05-12T04:07:43.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:43.049+0000 7f5f53d61700 1 --2- 172.21.15.32:0/3611115308 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f5f3806cc30 0x7f5f3806f0e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:43.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:43.049+0000 7f5f53d61700 1 --2- 172.21.15.32:0/3611115308 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f4c0fbcb0 0x7f5f4c110610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:43.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:43.049+0000 7f5f53d61700 1 --2- 172.21.15.32:0/3611115308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f4c0fcf60 0x7f5f4c112b60 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:43.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:43.049+0000 7f5f53d61700 1 -- 172.21.15.32:0/3611115308 >> 172.21.15.32:0/3611115308 conn(0x7f5f4c0f7430 msgr2=0x7f5f4c0f8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:07:43.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:43.050+0000 7f5f53d61700 1 -- 172.21.15.32:0/3611115308 shutdown_connections 2024-05-12T04:07:43.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:43.050+0000 7f5f53d61700 1 -- 172.21.15.32:0/3611115308 wait complete. 2024-05-12T04:07:43.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:43.153+0000 7f0ede347700 1 -- 172.21.15.32:0/2988631076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ed80fcf10 msgr2=0x7f0ed80fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:07:43.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:43.153+0000 7f0ede347700 1 --2- 172.21.15.32:0/2988631076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ed80fcf10 0x7f0ed80fd320 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8004940 tx=0x7f0ec8009e30 comp rx=0 tx=0).stop 2024-05-12T04:07:43.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:43.153+0000 7f0ede347700 1 -- 172.21.15.32:0/2988631076 shutdown_connections 2024-05-12T04:07:43.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:43.153+0000 7f0ede347700 1 --2- 172.21.15.32:0/2988631076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ed80fe080 0x7f0ed80fe4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:43.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:43.153+0000 7f0ede347700 1 --2- 172.21.15.32:0/2988631076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ed80fcf10 0x7f0ed80fd320 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:43.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:43.153+0000 7f0ede347700 1 -- 172.21.15.32:0/2988631076 >> 172.21.15.32:0/2988631076 conn(0x7f0ed80f8560 msgr2=0x7f0ed80fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:07:43.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:43.153+0000 7f0ede347700 1 -- 172.21.15.32:0/2988631076 shutdown_connections 2024-05-12T04:07:43.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:43.153+0000 7f0ede347700 1 -- 172.21.15.32:0/2988631076 wait complete. 2024-05-12T04:07:43.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:43.154+0000 7f0ede347700 1 Processor -- start 2024-05-12T04:07:43.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:43.154+0000 7f0ede347700 1 -- start start 2024-05-12T04:07:43.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:43.154+0000 7f0ede347700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ed80fcf10 0x7f0ed810d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:07:43.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:43.154+0000 7f0ede347700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ed80fe080 0x7f0ed810ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:07:43.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:43.154+0000 7f0ede347700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed810e3b0 con 0x7f0ed80fcf10 2024-05-12T04:07:43.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:43.154+0000 7f0ede347700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed810ec00 con 0x7f0ed80fe080 2024-05-12T04:07:43.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:43.154+0000 7f0ed7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ed80fcf10 0x7f0ed810d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:07:43.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:43.154+0000 7f0ed7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ed80fcf10 0x7f0ed810d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43546/0 (socket says 172.21.15.32:43546) 2024-05-12T04:07:43.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:43.154+0000 7f0ed7fff700 1 -- 172.21.15.32:0/3943323023 learned_addr learned my addr 172.21.15.32:0/3943323023 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:07:43.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:43.154+0000 7f0ed77fe700 1 --2- 172.21.15.32:0/3943323023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ed80fe080 0x7f0ed810ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:07:43.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:43.155+0000 7f0ed7fff700 1 -- 172.21.15.32:0/3943323023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ed80fe080 msgr2=0x7f0ed810ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:07:43.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:43.155+0000 7f0ed7fff700 1 --2- 172.21.15.32:0/3943323023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ed80fe080 0x7f0ed810ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:43.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:43.155+0000 7f0ed7fff700 1 -- 172.21.15.32:0/3943323023 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec8005040 con 0x7f0ed80fcf10 2024-05-12T04:07:43.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:43.155+0000 7f0ed7fff700 1 --2- 172.21.15.32:0/3943323023 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ed80fcf10 0x7f0ed810d870 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0ec80137b0 tx=0x7f0ec8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:07:43.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:43.155+0000 7f0ed4ff9700 1 -- 172.21.15.32:0/3943323023 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ec800e070 con 0x7f0ed80fcf10 2024-05-12T04:07:43.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:43.155+0000 7f0ed4ff9700 1 -- 172.21.15.32:0/3943323023 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0ec801c400 con 0x7f0ed80fcf10 2024-05-12T04:07:43.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:43.155+0000 7f0ed4ff9700 1 -- 172.21.15.32:0/3943323023 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ec800b650 con 0x7f0ed80fcf10 2024-05-12T04:07:43.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:43.155+0000 7f0ede347700 1 -- 172.21.15.32:0/3943323023 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ed810ee80 con 0x7f0ed80fcf10 2024-05-12T04:07:43.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:43.155+0000 7f0ede347700 1 -- 172.21.15.32:0/3943323023 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ed810f360 con 0x7f0ed80fcf10 2024-05-12T04:07:43.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:43.156+0000 7f0ede347700 1 -- 172.21.15.32:0/3943323023 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ed805f5c0 con 0x7f0ed80fcf10 2024-05-12T04:07:43.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:43.156+0000 7f0ed4ff9700 1 -- 172.21.15.32:0/3943323023 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f0ec801c9e0 con 0x7f0ed80fcf10 2024-05-12T04:07:43.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:43.157+0000 7f0ed4ff9700 1 --2- 172.21.15.32:0/3943323023 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f0ec006cc20 0x7f0ec006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:07:43.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:43.157+0000 7f0ed4ff9700 1 -- 172.21.15.32:0/3943323023 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f0ec808d560 con 0x7f0ed80fcf10 2024-05-12T04:07:43.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:43.157+0000 7f0ed77fe700 1 --2- 172.21.15.32:0/3943323023 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f0ec006cc20 0x7f0ec006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:07:43.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:43.157+0000 7f0ed77fe700 1 --2- 172.21.15.32:0/3943323023 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f0ec006cc20 0x7f0ec006f0d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc009b80 tx=0x7f0ecc00b040 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:07:43.161 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:43.161+0000 7f0ed4ff9700 1 -- 172.21.15.32:0/3943323023 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f0ec805b120 con 0x7f0ed80fcf10 2024-05-12T04:07:43.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:43.447+0000 7f0ede347700 1 -- 172.21.15.32:0/3943323023 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f0ed810f730 con 0x7f0ed80fcf10 2024-05-12T04:07:43.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:43.448+0000 7f0ed4ff9700 1 -- 172.21.15.32:0/3943323023 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1187 (secure 0 0 0) 0x7f0ec8004490 con 0x7f0ed80fcf10 2024-05-12T04:07:43.448 INFO:teuthology.orchestra.run.smithi032.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down; Degraded data redundancy: 42/276 objects degraded (15.217%), 13 pgs degraded 2024-05-12T04:07:43.448 INFO:teuthology.orchestra.run.smithi032.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-12T04:07:43.448 INFO:teuthology.orchestra.run.smithi032.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-12T04:07:43.448 INFO:teuthology.orchestra.run.smithi032.stdout:[WRN] OSD_DOWN: 1 osds down 2024-05-12T04:07:43.448 INFO:teuthology.orchestra.run.smithi032.stdout: osd.5 (root=default,host=smithi033) is down 2024-05-12T04:07:43.449 INFO:teuthology.orchestra.run.smithi032.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 42/276 objects degraded (15.217%), 13 pgs degraded 2024-05-12T04:07:43.449 INFO:teuthology.orchestra.run.smithi032.stdout: pg 2.2 is active+undersized+degraded, acting [1,0] 2024-05-12T04:07:43.449 INFO:teuthology.orchestra.run.smithi032.stdout: pg 2.3 is active+undersized+degraded, acting [2,1] 2024-05-12T04:07:43.449 INFO:teuthology.orchestra.run.smithi032.stdout: pg 2.8 is active+undersized+degraded, acting [3,0] 2024-05-12T04:07:43.449 INFO:teuthology.orchestra.run.smithi032.stdout: pg 2.b is active+undersized+degraded, acting [3,4] 2024-05-12T04:07:43.449 INFO:teuthology.orchestra.run.smithi032.stdout: pg 2.f is active+undersized+degraded, acting [4,0] 2024-05-12T04:07:43.449 INFO:teuthology.orchestra.run.smithi032.stdout: pg 2.14 is active+undersized+degraded, acting [3,4] 2024-05-12T04:07:43.449 INFO:teuthology.orchestra.run.smithi032.stdout: pg 2.16 is active+undersized+degraded, acting [3,2] 2024-05-12T04:07:43.449 INFO:teuthology.orchestra.run.smithi032.stdout: pg 2.18 is active+undersized+degraded, acting [4,3] 2024-05-12T04:07:43.449 INFO:teuthology.orchestra.run.smithi032.stdout: pg 2.1a is active+undersized+degraded, acting [3,4] 2024-05-12T04:07:43.449 INFO:teuthology.orchestra.run.smithi032.stdout: pg 2.1b is active+undersized+degraded, acting [1,0] 2024-05-12T04:07:43.449 INFO:teuthology.orchestra.run.smithi032.stdout: pg 2.1c is active+undersized+degraded, acting [4,2] 2024-05-12T04:07:43.449 INFO:teuthology.orchestra.run.smithi032.stdout: pg 2.1d is active+undersized+degraded, acting [3,0] 2024-05-12T04:07:43.449 INFO:teuthology.orchestra.run.smithi032.stdout: pg 2.1e is active+undersized+degraded, acting [2,0] 2024-05-12T04:07:43.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:43.450+0000 7f0ede347700 1 -- 172.21.15.32:0/3943323023 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f0ec006cc20 msgr2=0x7f0ec006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:07:43.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:43.450+0000 7f0ede347700 1 --2- 172.21.15.32:0/3943323023 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f0ec006cc20 0x7f0ec006f0d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc009b80 tx=0x7f0ecc00b040 comp rx=0 tx=0).stop 2024-05-12T04:07:43.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:43.450+0000 7f0ede347700 1 -- 172.21.15.32:0/3943323023 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ed80fcf10 msgr2=0x7f0ed810d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:07:43.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:43.450+0000 7f0ede347700 1 --2- 172.21.15.32:0/3943323023 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ed80fcf10 0x7f0ed810d870 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0ec80137b0 tx=0x7f0ec8009e30 comp rx=0 tx=0).stop 2024-05-12T04:07:43.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:43.450+0000 7f0ede347700 1 -- 172.21.15.32:0/3943323023 shutdown_connections 2024-05-12T04:07:43.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:43.450+0000 7f0ede347700 1 --2- 172.21.15.32:0/3943323023 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f0ec006cc20 0x7f0ec006f0d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:43.450 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:43.450+0000 7f0ede347700 1 --2- 172.21.15.32:0/3943323023 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ed80fcf10 0x7f0ed810d870 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:43.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:43.450+0000 7f0ede347700 1 --2- 172.21.15.32:0/3943323023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ed80fe080 0x7f0ed810ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:07:43.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:43.450+0000 7f0ede347700 1 -- 172.21.15.32:0/3943323023 >> 172.21.15.32:0/3943323023 conn(0x7f0ed80f8560 msgr2=0x7f0ed80f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:07:43.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:43.450+0000 7f0ede347700 1 -- 172.21.15.32:0/3943323023 shutdown_connections 2024-05-12T04:07:43.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:07:43.450+0000 7f0ede347700 1 -- 172.21.15.32:0/3943323023 wait complete. 2024-05-12T04:07:43.662 INFO:journalctl@ceph.osd.5.smithi033.stdout:May 12 04:07:43 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-5[132613]: 2024-05-12T04:07:43.249+0000 7f7271ce3700 -1 osd.5 74 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-12T04:07:43.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:43 smithi033 ceph-mon[107305]: from='client.34374 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:07:43.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:43 smithi033 ceph-mon[107305]: pgmap v163: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 547 B/s rd, 1 op/s; 42/276 objects degraded (15.217%) 2024-05-12T04:07:43.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:43 smithi033 ceph-mon[107305]: from='client.44283 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:07:43.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:43 smithi033 ceph-mon[107305]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-12T04:07:43.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:43 smithi033 ceph-mon[107305]: osdmap e77: 6 total, 5 up, 6 in 2024-05-12T04:07:43.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:43 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/1908587392' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:07:43.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:43 smithi033 ceph-mon[107305]: from='osd.5 [v2:172.21.15.33:6816/4274440090,v1:172.21.15.33:6817/4274440090]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-05-12T04:07:43.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:43 smithi033 ceph-mon[107305]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-05-12T04:07:43.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:43 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/4217441773' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-12T04:07:43.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:43 smithi033 ceph-mon[107305]: from='osd.5 ' entity='osd.5' 2024-05-12T04:07:43.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:43 smithi032 ceph-mon[120392]: from='client.34374 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:07:43.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:43 smithi032 ceph-mon[120392]: pgmap v163: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 260 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 547 B/s rd, 1 op/s; 42/276 objects degraded (15.217%) 2024-05-12T04:07:43.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:43 smithi032 ceph-mon[120392]: from='client.44283 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:07:43.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:43 smithi032 ceph-mon[120392]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-12T04:07:43.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:43 smithi032 ceph-mon[120392]: osdmap e77: 6 total, 5 up, 6 in 2024-05-12T04:07:43.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:43 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/1908587392' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:07:43.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:43 smithi032 ceph-mon[120392]: from='osd.5 [v2:172.21.15.33:6816/4274440090,v1:172.21.15.33:6817/4274440090]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-05-12T04:07:43.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:43 smithi032 ceph-mon[120392]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-05-12T04:07:43.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:43 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/4217441773' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-12T04:07:43.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:43 smithi032 ceph-mon[120392]: from='osd.5 ' entity='osd.5' 2024-05-12T04:07:44.573 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:44 smithi033 ceph-mon[107305]: from='client.34392 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:07:44.574 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:44 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/3943323023' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-12T04:07:44.574 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:44 smithi033 ceph-mon[107305]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-12T04:07:44.574 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:44 smithi033 ceph-mon[107305]: osd.5 [v2:172.21.15.33:6816/4274440090,v1:172.21.15.33:6817/4274440090] boot 2024-05-12T04:07:44.574 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:44 smithi033 ceph-mon[107305]: osdmap e78: 6 total, 6 up, 6 in 2024-05-12T04:07:44.574 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:44 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-12T04:07:44.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:44 smithi032 ceph-mon[120392]: from='client.34392 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:07:44.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:44 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/3943323023' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-12T04:07:44.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:44 smithi032 ceph-mon[120392]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-05-12T04:07:44.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:44 smithi032 ceph-mon[120392]: osd.5 [v2:172.21.15.33:6816/4274440090,v1:172.21.15.33:6817/4274440090] boot 2024-05-12T04:07:44.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:44 smithi032 ceph-mon[120392]: osdmap e78: 6 total, 6 up, 6 in 2024-05-12T04:07:44.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:44 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-12T04:07:45.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:45 smithi033 ceph-mon[107305]: pgmap v165: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 260 MiB data, 975 MiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 42/276 objects degraded (15.217%) 2024-05-12T04:07:45.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:45 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:07:45.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:45 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:07:45.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:45 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:07:45.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:45 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T04:07:45.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:45 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:07:45.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:45 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:07:45.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:45 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:07:45.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:45 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:07:45.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:45 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:07:45.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:45 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:07:45.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:45 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:07:45.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:45 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-05-12T04:07:45.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:45 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-05-12T04:07:45.664 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:45 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-05-12T04:07:45.664 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:45 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-05-12T04:07:45.664 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:45 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-05-12T04:07:45.664 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:45 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-05-12T04:07:45.664 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:45 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-05-12T04:07:45.664 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:45 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-05-12T04:07:45.664 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:45 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-05-12T04:07:45.664 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:45 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-05-12T04:07:45.664 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:45 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-05-12T04:07:45.664 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:45 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-05-12T04:07:45.664 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:45 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd require-osd-release", "release": "reef"}]: dispatch 2024-05-12T04:07:45.665 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:45 smithi033 ceph-mon[107305]: Health check update: Degraded data redundancy: 42/276 objects degraded (15.217%), 13 pgs degraded (PG_DEGRADED) 2024-05-12T04:07:45.665 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:45 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "osd require-osd-release", "release": "reef"}]': finished 2024-05-12T04:07:45.665 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:45 smithi033 ceph-mon[107305]: osdmap e79: 6 total, 6 up, 6 in 2024-05-12T04:07:45.665 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:45 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:07:45.665 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:45 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]: dispatch 2024-05-12T04:07:45.665 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:45 smithi033 ceph-mon[107305]: osdmap e80: 6 total, 6 up, 6 in 2024-05-12T04:07:45.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:45 smithi032 ceph-mon[120392]: pgmap v165: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 260 MiB data, 975 MiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 42/276 objects degraded (15.217%) 2024-05-12T04:07:45.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:45 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:07:45.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:45 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:07:45.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:45 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:07:45.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:45 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T04:07:45.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:45 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:07:45.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:45 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:07:45.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:45 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:07:45.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:45 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:07:45.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:45 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:07:45.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:45 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:07:45.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:45 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:07:45.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:45 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-05-12T04:07:45.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:45 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-05-12T04:07:45.792 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:45 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-05-12T04:07:45.792 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:45 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-05-12T04:07:45.792 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:45 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-05-12T04:07:45.792 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:45 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-05-12T04:07:45.792 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:45 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-05-12T04:07:45.792 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:45 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-05-12T04:07:45.792 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:45 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-05-12T04:07:45.792 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:45 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-05-12T04:07:45.792 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:45 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-05-12T04:07:45.792 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:45 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-05-12T04:07:45.792 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:45 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd require-osd-release", "release": "reef"}]: dispatch 2024-05-12T04:07:45.792 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:45 smithi032 ceph-mon[120392]: Health check update: Degraded data redundancy: 42/276 objects degraded (15.217%), 13 pgs degraded (PG_DEGRADED) 2024-05-12T04:07:45.792 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:45 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "osd require-osd-release", "release": "reef"}]': finished 2024-05-12T04:07:45.793 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:45 smithi032 ceph-mon[120392]: osdmap e79: 6 total, 6 up, 6 in 2024-05-12T04:07:45.793 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:45 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:07:45.793 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:45 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]: dispatch 2024-05-12T04:07:45.793 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:45 smithi032 ceph-mon[120392]: osdmap e80: 6 total, 6 up, 6 in 2024-05-12T04:07:46.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:46 smithi033 ceph-mon[107305]: Upgrade: Setting container_image for all osd 2024-05-12T04:07:46.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:46 smithi033 ceph-mon[107305]: Upgrade: Setting require_osd_release to 18 reef 2024-05-12T04:07:46.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:46 smithi033 ceph-mon[107305]: Upgrade: Disabling standby-replay for filesystem cephfs 2024-05-12T04:07:46.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:46 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:07:46.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:46 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:07:46.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:46 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:07:46.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:46 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]': finished 2024-05-12T04:07:46.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:46 smithi033 ceph-mon[107305]: fsmap cephfs:1 {0=cephfs.smithi032.qpsawh=up:active} 2 up:standby 2024-05-12T04:07:46.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:46 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:07:46.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:46 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:07:46.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:46 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T04:07:46.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:46 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:07:46.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:46 smithi032 ceph-mon[120392]: Upgrade: Setting container_image for all osd 2024-05-12T04:07:46.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:46 smithi032 ceph-mon[120392]: Upgrade: Setting require_osd_release to 18 reef 2024-05-12T04:07:46.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:46 smithi032 ceph-mon[120392]: Upgrade: Disabling standby-replay for filesystem cephfs 2024-05-12T04:07:46.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:46 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:07:46.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:46 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:07:46.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:46 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:07:46.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:46 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]': finished 2024-05-12T04:07:46.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:46 smithi032 ceph-mon[120392]: fsmap cephfs:1 {0=cephfs.smithi032.qpsawh=up:active} 2 up:standby 2024-05-12T04:07:46.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:46 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:07:46.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:46 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:07:46.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:46 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T04:07:46.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:46 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:07:47.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:47 smithi032 ceph-mon[120392]: pgmap v169: 65 pgs: 11 active+undersized+degraded, 14 active+undersized, 40 active+clean; 260 MiB data, 975 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 37/276 objects degraded (13.406%) 2024-05-12T04:07:47.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:47 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:07:47.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:47 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:07:47.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:47 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:07:47.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:47 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:07:47.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:47 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:07:47.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:47 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi032.ialtqb"]}]: dispatch 2024-05-12T04:07:47.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:47 smithi032 ceph-mon[120392]: Upgrade: It appears safe to stop mds.cephfs.smithi032.ialtqb 2024-05-12T04:07:47.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:47 smithi032 ceph-mon[120392]: mds.? [v2:172.21.15.33:6824/2944716783,v1:172.21.15.33:6825/2944716783] up:boot 2024-05-12T04:07:47.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:47 smithi032 ceph-mon[120392]: fsmap cephfs:1 {0=cephfs.smithi032.qpsawh=up:active} 3 up:standby 2024-05-12T04:07:47.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:47 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi033.jgbjhk"}]: dispatch 2024-05-12T04:07:47.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:47 smithi033 ceph-mon[107305]: pgmap v169: 65 pgs: 11 active+undersized+degraded, 14 active+undersized, 40 active+clean; 260 MiB data, 975 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 37/276 objects degraded (13.406%) 2024-05-12T04:07:47.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:47 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:07:47.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:47 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:07:47.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:47 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:07:47.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:47 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:07:47.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:47 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:07:47.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:47 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi032.ialtqb"]}]: dispatch 2024-05-12T04:07:47.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:47 smithi033 ceph-mon[107305]: Upgrade: It appears safe to stop mds.cephfs.smithi032.ialtqb 2024-05-12T04:07:47.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:47 smithi033 ceph-mon[107305]: mds.? [v2:172.21.15.33:6824/2944716783,v1:172.21.15.33:6825/2944716783] up:boot 2024-05-12T04:07:47.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:47 smithi033 ceph-mon[107305]: fsmap cephfs:1 {0=cephfs.smithi032.qpsawh=up:active} 3 up:standby 2024-05-12T04:07:47.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:47 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi033.jgbjhk"}]: dispatch 2024-05-12T04:07:49.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:49 smithi033 ceph-mon[107305]: pgmap v170: 65 pgs: 1 active+undersized+degraded, 1 active+undersized, 63 active+clean; 260 MiB data, 979 MiB used, 535 GiB / 536 GiB avail; 942 B/s rd, 1 op/s; 1/276 objects degraded (0.362%) 2024-05-12T04:07:49.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:49 smithi033 ceph-mon[107305]: Upgrade: Updating mds.cephfs.smithi032.ialtqb 2024-05-12T04:07:49.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:49 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:07:49.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:49 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi032.ialtqb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-12T04:07:49.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:49 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:07:49.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:49 smithi033 ceph-mon[107305]: Deploying daemon mds.cephfs.smithi032.ialtqb on smithi032 2024-05-12T04:07:49.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:49 smithi032 ceph-mon[120392]: pgmap v170: 65 pgs: 1 active+undersized+degraded, 1 active+undersized, 63 active+clean; 260 MiB data, 979 MiB used, 535 GiB / 536 GiB avail; 942 B/s rd, 1 op/s; 1/276 objects degraded (0.362%) 2024-05-12T04:07:49.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:49 smithi032 ceph-mon[120392]: Upgrade: Updating mds.cephfs.smithi032.ialtqb 2024-05-12T04:07:49.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:49 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:07:49.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:49 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi032.ialtqb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-12T04:07:49.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:49 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:07:49.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:49 smithi032 ceph-mon[120392]: Deploying daemon mds.cephfs.smithi032.ialtqb on smithi032 2024-05-12T04:07:51.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:51 smithi032 ceph-mon[120392]: pgmap v171: 65 pgs: 65 active+clean; 260 MiB data, 979 MiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-05-12T04:07:51.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:51 smithi032 ceph-mon[120392]: osdmap e81: 6 total, 6 up, 6 in 2024-05-12T04:07:51.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:51 smithi032 ceph-mon[120392]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 1/276 objects degraded (0.362%), 1 pg degraded) 2024-05-12T04:07:51.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:51 smithi032 ceph-mon[120392]: fsmap cephfs:1 {0=cephfs.smithi032.qpsawh=up:active} 2 up:standby 2024-05-12T04:07:51.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:51 smithi033 ceph-mon[107305]: pgmap v171: 65 pgs: 65 active+clean; 260 MiB data, 979 MiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-05-12T04:07:51.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:51 smithi033 ceph-mon[107305]: osdmap e81: 6 total, 6 up, 6 in 2024-05-12T04:07:51.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:51 smithi033 ceph-mon[107305]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 1/276 objects degraded (0.362%), 1 pg degraded) 2024-05-12T04:07:51.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:51 smithi033 ceph-mon[107305]: fsmap cephfs:1 {0=cephfs.smithi032.qpsawh=up:active} 2 up:standby 2024-05-12T04:07:52.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:52 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi032.ialtqb"}]: dispatch 2024-05-12T04:07:52.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:52 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi032.ialtqb"}]: dispatch 2024-05-12T04:07:53.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:53 smithi032 ceph-mon[120392]: pgmap v173: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-05-12T04:07:53.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:53 smithi033 ceph-mon[107305]: pgmap v173: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-05-12T04:07:55.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:55 smithi032 ceph-mon[120392]: pgmap v174: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-05-12T04:07:55.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:55 smithi033 ceph-mon[107305]: pgmap v174: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-05-12T04:07:56.613 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:56 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:07:56.613 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:56 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:07:56.613 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:56 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:07:56.613 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:56 smithi032 ceph-mon[120392]: pgmap v175: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-05-12T04:07:56.613 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:56 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:07:56.613 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:56 smithi032 ceph-mon[120392]: mds.? [v2:172.21.15.32:6826/1080358503,v1:172.21.15.32:6827/1080358503] up:boot 2024-05-12T04:07:56.613 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:56 smithi032 ceph-mon[120392]: fsmap cephfs:1 {0=cephfs.smithi032.qpsawh=up:active} 3 up:standby 2024-05-12T04:07:56.613 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:56 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi032.ialtqb"}]: dispatch 2024-05-12T04:07:56.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:56 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:07:56.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:56 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:07:56.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:56 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:07:56.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:56 smithi033 ceph-mon[107305]: pgmap v175: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-05-12T04:07:56.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:56 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:07:56.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:56 smithi033 ceph-mon[107305]: mds.? [v2:172.21.15.32:6826/1080358503,v1:172.21.15.32:6827/1080358503] up:boot 2024-05-12T04:07:56.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:56 smithi033 ceph-mon[107305]: fsmap cephfs:1 {0=cephfs.smithi032.qpsawh=up:active} 3 up:standby 2024-05-12T04:07:56.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:56 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi032.ialtqb"}]: dispatch 2024-05-12T04:07:59.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:07:58 smithi032 ceph-mon[120392]: pgmap v176: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail 2024-05-12T04:07:59.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:07:58 smithi033 ceph-mon[107305]: pgmap v176: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail 2024-05-12T04:08:01.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:00 smithi033 ceph-mon[107305]: pgmap v177: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail 2024-05-12T04:08:01.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:00 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:01.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:00 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:08:01.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:00 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:01.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:00 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:01.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:00 smithi032 ceph-mon[120392]: pgmap v177: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail 2024-05-12T04:08:01.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:00 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:01.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:00 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:08:01.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:00 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:01.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:00 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:02.718 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:02 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:02.718 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:02 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:02.718 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:02 smithi032 ceph-mon[120392]: pgmap v178: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail 2024-05-12T04:08:02.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:02 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:02.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:02 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:02.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:02 smithi033 ceph-mon[107305]: pgmap v178: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail 2024-05-12T04:08:05.128 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:04 smithi032 ceph-mon[120392]: pgmap v179: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail 2024-05-12T04:08:05.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:04 smithi033 ceph-mon[107305]: pgmap v179: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail 2024-05-12T04:08:06.537 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:06 smithi032 ceph-mon[120392]: Detected new or changed devices on smithi032 2024-05-12T04:08:06.537 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:06 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:06.537 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:06 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:06.537 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:06 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:08:06.537 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:06 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T04:08:06.537 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:06 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:06.537 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:06 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:08:06.537 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:06 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:08:06.538 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:06 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:08:06.538 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:06 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:08:06.538 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:06 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:08:06.538 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:06 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi032.qpsawh"]}]: dispatch 2024-05-12T04:08:06.538 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:06 smithi032 ceph-mon[120392]: Upgrade: It appears safe to stop mds.cephfs.smithi032.qpsawh 2024-05-12T04:08:06.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:06 smithi033 ceph-mon[107305]: Detected new or changed devices on smithi032 2024-05-12T04:08:06.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:06 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:06.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:06 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:06.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:06 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:08:06.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:06 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T04:08:06.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:06 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:06.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:06 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:08:06.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:06 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:08:06.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:06 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:08:06.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:06 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:08:06.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:06 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:08:06.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:06 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi032.qpsawh"]}]: dispatch 2024-05-12T04:08:06.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:06 smithi033 ceph-mon[107305]: Upgrade: It appears safe to stop mds.cephfs.smithi032.qpsawh 2024-05-12T04:08:07.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:07 smithi032 ceph-mon[120392]: pgmap v180: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail 2024-05-12T04:08:07.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:07 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:07.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:07 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi032.qpsawh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-12T04:08:07.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:07 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:08:07.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:07 smithi033 ceph-mon[107305]: pgmap v180: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail 2024-05-12T04:08:07.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:07 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:07.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:07 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi032.qpsawh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-12T04:08:07.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:07 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:08:08.469 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:08 smithi032 ceph-mon[120392]: Upgrade: Updating mds.cephfs.smithi032.qpsawh 2024-05-12T04:08:08.469 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:08 smithi032 ceph-mon[120392]: Deploying daemon mds.cephfs.smithi032.qpsawh on smithi032 2024-05-12T04:08:08.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:08 smithi033 ceph-mon[107305]: Upgrade: Updating mds.cephfs.smithi032.qpsawh 2024-05-12T04:08:08.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:08 smithi033 ceph-mon[107305]: Deploying daemon mds.cephfs.smithi032.qpsawh on smithi032 2024-05-12T04:08:09.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:08 smithi032 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-mon-smithi032[120369]: 2024-05-12T04:08:08.971+0000 7f474139f700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-12T04:08:09.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:09 smithi032 ceph-mon[120392]: pgmap v181: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail 2024-05-12T04:08:09.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:09 smithi032 ceph-mon[120392]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-12T04:08:09.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:09 smithi032 ceph-mon[120392]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-12T04:08:09.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:09 smithi032 ceph-mon[120392]: osdmap e82: 6 total, 6 up, 6 in 2024-05-12T04:08:09.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:09 smithi032 ceph-mon[120392]: Standby daemon mds.cephfs.smithi033.hglwml assigned to filesystem cephfs as rank 0 2024-05-12T04:08:09.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:09 smithi032 ceph-mon[120392]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-12T04:08:09.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:09 smithi032 ceph-mon[120392]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-05-12T04:08:09.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:09 smithi032 ceph-mon[120392]: fsmap cephfs:1/1 {0=cephfs.smithi033.hglwml=up:replay} 2 up:standby 2024-05-12T04:08:09.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:09 smithi033 ceph-mon[107305]: pgmap v181: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail 2024-05-12T04:08:09.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:09 smithi033 ceph-mon[107305]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-12T04:08:09.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:09 smithi033 ceph-mon[107305]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-12T04:08:09.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:09 smithi033 ceph-mon[107305]: osdmap e82: 6 total, 6 up, 6 in 2024-05-12T04:08:09.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:09 smithi033 ceph-mon[107305]: Standby daemon mds.cephfs.smithi033.hglwml assigned to filesystem cephfs as rank 0 2024-05-12T04:08:09.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:09 smithi033 ceph-mon[107305]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-12T04:08:09.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:09 smithi033 ceph-mon[107305]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-05-12T04:08:09.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:09 smithi033 ceph-mon[107305]: fsmap cephfs:1/1 {0=cephfs.smithi033.hglwml=up:replay} 2 up:standby 2024-05-12T04:08:11.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:11 smithi032 ceph-mon[120392]: pgmap v183: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail 2024-05-12T04:08:11.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:11 smithi033 ceph-mon[107305]: pgmap v183: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail 2024-05-12T04:08:12.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:12 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi032.qpsawh"}]: dispatch 2024-05-12T04:08:12.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:12 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi032.qpsawh"}]: dispatch 2024-05-12T04:08:13.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:13 smithi032 ceph-mon[120392]: pgmap v184: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 0 op/s 2024-05-12T04:08:13.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.572+0000 7f6ad1849700 1 -- 172.21.15.32:0/3992774787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6acc0fcf60 msgr2=0x7f6acc0fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:08:13.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.572+0000 7f6ad1849700 1 --2- 172.21.15.32:0/3992774787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6acc0fcf60 0x7f6acc0fd3d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6ac0004940 tx=0x7f6ac0009e30 comp rx=0 tx=0).stop 2024-05-12T04:08:13.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.572+0000 7f6ad1849700 1 -- 172.21.15.32:0/3992774787 shutdown_connections 2024-05-12T04:08:13.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.572+0000 7f6ad1849700 1 --2- 172.21.15.32:0/3992774787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6acc0fcf60 0x7f6acc0fd3d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:13.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.572+0000 7f6ad1849700 1 --2- 172.21.15.32:0/3992774787 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6acc0fbcb0 0x7f6acc0fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:13.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.572+0000 7f6ad1849700 1 -- 172.21.15.32:0/3992774787 >> 172.21.15.32:0/3992774787 conn(0x7f6acc0f7430 msgr2=0x7f6acc0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:08:13.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.573+0000 7f6ad1849700 1 -- 172.21.15.32:0/3992774787 shutdown_connections 2024-05-12T04:08:13.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.573+0000 7f6ad1849700 1 -- 172.21.15.32:0/3992774787 wait complete. 2024-05-12T04:08:13.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.573+0000 7f6ad1849700 1 Processor -- start 2024-05-12T04:08:13.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.573+0000 7f6ad1849700 1 -- start start 2024-05-12T04:08:13.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.574+0000 7f6ad1849700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6acc0fbcb0 0x7f6acc10d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:08:13.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.574+0000 7f6ad1849700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6acc0fcf60 0x7f6acc10ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:08:13.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.574+0000 7f6ad1849700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6acc10e3f0 con 0x7f6acc0fbcb0 2024-05-12T04:08:13.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.574+0000 7f6ad1849700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6acc10ec40 con 0x7f6acc0fcf60 2024-05-12T04:08:13.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.574+0000 7f6acaffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6acc0fbcb0 0x7f6acc10d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:08:13.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.574+0000 7f6aca7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6acc0fcf60 0x7f6acc10ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:08:13.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.574+0000 7f6acaffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6acc0fbcb0 0x7f6acc10d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44420/0 (socket says 172.21.15.32:44420) 2024-05-12T04:08:13.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.574+0000 7f6acaffd700 1 -- 172.21.15.32:0/2397717466 learned_addr learned my addr 172.21.15.32:0/2397717466 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:08:13.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.574+0000 7f6acaffd700 1 -- 172.21.15.32:0/2397717466 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6acc0fcf60 msgr2=0x7f6acc10ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:08:13.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.574+0000 7f6acaffd700 1 --2- 172.21.15.32:0/2397717466 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6acc0fcf60 0x7f6acc10ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:13.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.574+0000 7f6acaffd700 1 -- 172.21.15.32:0/2397717466 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ac0005040 con 0x7f6acc0fbcb0 2024-05-12T04:08:13.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.574+0000 7f6acaffd700 1 --2- 172.21.15.32:0/2397717466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6acc0fbcb0 0x7f6acc10d8b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6abc00eb40 tx=0x7f6abc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:08:13.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.575+0000 7f6ab37fe700 1 -- 172.21.15.32:0/2397717466 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6abc007cc0 con 0x7f6acc0fbcb0 2024-05-12T04:08:13.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.575+0000 7f6ad1849700 1 -- 172.21.15.32:0/2397717466 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6acc10eec0 con 0x7f6acc0fbcb0 2024-05-12T04:08:13.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.575+0000 7f6ab37fe700 1 -- 172.21.15.32:0/2397717466 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6abc00f040 con 0x7f6acc0fbcb0 2024-05-12T04:08:13.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.575+0000 7f6ab37fe700 1 -- 172.21.15.32:0/2397717466 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6abc00c990 con 0x7f6acc0fbcb0 2024-05-12T04:08:13.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.575+0000 7f6ad1849700 1 -- 172.21.15.32:0/2397717466 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6acc10f400 con 0x7f6acc0fbcb0 2024-05-12T04:08:13.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.576+0000 7f6ad1849700 1 -- 172.21.15.32:0/2397717466 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6acc101af0 con 0x7f6acc0fbcb0 2024-05-12T04:08:13.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.576+0000 7f6ab37fe700 1 -- 172.21.15.32:0/2397717466 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f6abc00cc00 con 0x7f6acc0fbcb0 2024-05-12T04:08:13.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.577+0000 7f6ab37fe700 1 --2- 172.21.15.32:0/2397717466 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f6ab406cc20 0x7f6ab406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:08:13.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.577+0000 7f6ab37fe700 1 -- 172.21.15.32:0/2397717466 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f6abc08bc60 con 0x7f6acc0fbcb0 2024-05-12T04:08:13.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.577+0000 7f6aca7fc700 1 --2- 172.21.15.32:0/2397717466 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f6ab406cc20 0x7f6ab406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:08:13.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.577+0000 7f6aca7fc700 1 --2- 172.21.15.32:0/2397717466 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f6ab406cc20 0x7f6ab406f0d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6ac0004890 tx=0x7f6ac0009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:08:13.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.582+0000 7f6ab37fe700 1 -- 172.21.15.32:0/2397717466 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f6abc059760 con 0x7f6acc0fbcb0 2024-05-12T04:08:13.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:13 smithi033 ceph-mon[107305]: pgmap v184: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 0 op/s 2024-05-12T04:08:13.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.805+0000 7f6ad1849700 1 -- 172.21.15.32:0/2397717466 --> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6acc10f620 con 0x7f6ab406cc20 2024-05-12T04:08:13.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.806+0000 7f6ab37fe700 1 -- 172.21.15.32:0/2397717466 <== mgr.24593 v2:172.21.15.32:6800/2957111591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f6acc10f620 con 0x7f6ab406cc20 2024-05-12T04:08:13.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.808+0000 7f6ad1849700 1 -- 172.21.15.32:0/2397717466 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f6ab406cc20 msgr2=0x7f6ab406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:08:13.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.808+0000 7f6ad1849700 1 --2- 172.21.15.32:0/2397717466 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f6ab406cc20 0x7f6ab406f0d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6ac0004890 tx=0x7f6ac0009e30 comp rx=0 tx=0).stop 2024-05-12T04:08:13.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.808+0000 7f6ad1849700 1 -- 172.21.15.32:0/2397717466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6acc0fbcb0 msgr2=0x7f6acc10d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:08:13.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.808+0000 7f6ad1849700 1 --2- 172.21.15.32:0/2397717466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6acc0fbcb0 0x7f6acc10d8b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6abc00eb40 tx=0x7f6abc00b040 comp rx=0 tx=0).stop 2024-05-12T04:08:13.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.808+0000 7f6ad1849700 1 -- 172.21.15.32:0/2397717466 shutdown_connections 2024-05-12T04:08:13.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.808+0000 7f6ad1849700 1 --2- 172.21.15.32:0/2397717466 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f6ab406cc20 0x7f6ab406f0d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:13.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.808+0000 7f6ad1849700 1 --2- 172.21.15.32:0/2397717466 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6acc0fbcb0 0x7f6acc10d8b0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:13.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.808+0000 7f6ad1849700 1 --2- 172.21.15.32:0/2397717466 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6acc0fcf60 0x7f6acc10ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:13.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.808+0000 7f6ad1849700 1 -- 172.21.15.32:0/2397717466 >> 172.21.15.32:0/2397717466 conn(0x7f6acc0f7430 msgr2=0x7f6acc065bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:08:13.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.808+0000 7f6ad1849700 1 -- 172.21.15.32:0/2397717466 shutdown_connections 2024-05-12T04:08:13.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.808+0000 7f6ad1849700 1 -- 172.21.15.32:0/2397717466 wait complete. 2024-05-12T04:08:13.819 INFO:teuthology.orchestra.run.smithi032.stdout:true 2024-05-12T04:08:13.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.914+0000 7f91b4145700 1 -- 172.21.15.32:0/386912762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91ac0ffee0 msgr2=0x7f91ac102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:08:13.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.914+0000 7f91b4145700 1 --2- 172.21.15.32:0/386912762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91ac0ffee0 0x7f91ac102300 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f919c004940 tx=0x7f919c009e30 comp rx=0 tx=0).stop 2024-05-12T04:08:13.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.914+0000 7f91b4145700 1 -- 172.21.15.32:0/386912762 shutdown_connections 2024-05-12T04:08:13.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.914+0000 7f91b4145700 1 --2- 172.21.15.32:0/386912762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91ac0ffee0 0x7f91ac102300 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:13.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.914+0000 7f91b4145700 1 --2- 172.21.15.32:0/386912762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91ac0662c0 0x7f91ac0ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:13.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.914+0000 7f91b4145700 1 -- 172.21.15.32:0/386912762 >> 172.21.15.32:0/386912762 conn(0x7f91ac0f7430 msgr2=0x7f91ac0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:08:13.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.915+0000 7f91b4145700 1 -- 172.21.15.32:0/386912762 shutdown_connections 2024-05-12T04:08:13.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.915+0000 7f91b4145700 1 -- 172.21.15.32:0/386912762 wait complete. 2024-05-12T04:08:13.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.915+0000 7f91b4145700 1 Processor -- start 2024-05-12T04:08:13.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.915+0000 7f91b4145700 1 -- start start 2024-05-12T04:08:13.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.915+0000 7f91b4145700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91ac0662c0 0x7f91ac0fd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:08:13.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.915+0000 7f91b4145700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91ac0ffee0 0x7f91ac0fb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:08:13.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.915+0000 7f91b4145700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91ac0fbde0 con 0x7f91ac0662c0 2024-05-12T04:08:13.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.915+0000 7f91b4145700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91ac0fbf20 con 0x7f91ac0ffee0 2024-05-12T04:08:13.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.915+0000 7f91b1ee1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91ac0662c0 0x7f91ac0fd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:08:13.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.915+0000 7f91b1ee1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91ac0662c0 0x7f91ac0fd330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44448/0 (socket says 172.21.15.32:44448) 2024-05-12T04:08:13.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.915+0000 7f91b1ee1700 1 -- 172.21.15.32:0/3562134811 learned_addr learned my addr 172.21.15.32:0/3562134811 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:08:13.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.916+0000 7f91b16e0700 1 --2- 172.21.15.32:0/3562134811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91ac0ffee0 0x7f91ac0fb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:08:13.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.916+0000 7f91b1ee1700 1 -- 172.21.15.32:0/3562134811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91ac0ffee0 msgr2=0x7f91ac0fb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:08:13.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.916+0000 7f91b1ee1700 1 --2- 172.21.15.32:0/3562134811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91ac0ffee0 0x7f91ac0fb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:13.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.916+0000 7f91b1ee1700 1 -- 172.21.15.32:0/3562134811 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f919c005040 con 0x7f91ac0662c0 2024-05-12T04:08:13.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.916+0000 7f91b1ee1700 1 --2- 172.21.15.32:0/3562134811 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91ac0662c0 0x7f91ac0fd330 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f91a800eb40 tx=0x7f91a800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:08:13.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.916+0000 7f91a27fc700 1 -- 172.21.15.32:0/3562134811 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f91a8007cc0 con 0x7f91ac0662c0 2024-05-12T04:08:13.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.916+0000 7f91b4145700 1 -- 172.21.15.32:0/3562134811 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91ac0fc1a0 con 0x7f91ac0662c0 2024-05-12T04:08:13.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.916+0000 7f91a27fc700 1 -- 172.21.15.32:0/3562134811 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f91a800f040 con 0x7f91ac0662c0 2024-05-12T04:08:13.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.916+0000 7f91a27fc700 1 -- 172.21.15.32:0/3562134811 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f91a800c8f0 con 0x7f91ac0662c0 2024-05-12T04:08:13.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.916+0000 7f91b4145700 1 -- 172.21.15.32:0/3562134811 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91ac1159a0 con 0x7f91ac0662c0 2024-05-12T04:08:13.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.917+0000 7f91b4145700 1 -- 172.21.15.32:0/3562134811 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91ac05f5c0 con 0x7f91ac0662c0 2024-05-12T04:08:13.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.918+0000 7f91a27fc700 1 -- 172.21.15.32:0/3562134811 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f91a800cb60 con 0x7f91ac0662c0 2024-05-12T04:08:13.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.918+0000 7f91a27fc700 1 --2- 172.21.15.32:0/3562134811 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f919806cc30 0x7f919806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:08:13.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.918+0000 7f91a27fc700 1 -- 172.21.15.32:0/3562134811 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f91a808bae0 con 0x7f91ac0662c0 2024-05-12T04:08:13.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.918+0000 7f91b16e0700 1 --2- 172.21.15.32:0/3562134811 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f919806cc30 0x7f919806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:08:13.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.919+0000 7f91b16e0700 1 --2- 172.21.15.32:0/3562134811 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f919806cc30 0x7f919806f0e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f919c013680 tx=0x7f919c009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:08:13.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:13.923+0000 7f91a27fc700 1 -- 172.21.15.32:0/3562134811 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f91a8055b50 con 0x7f91ac0662c0 2024-05-12T04:08:14.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.146+0000 7f91b4145700 1 -- 172.21.15.32:0/3562134811 --> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f91ac101b50 con 0x7f919806cc30 2024-05-12T04:08:14.147 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.147+0000 7f91a27fc700 1 -- 172.21.15.32:0/3562134811 <== mgr.24593 v2:172.21.15.32:6800/2957111591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f91ac101b50 con 0x7f919806cc30 2024-05-12T04:08:14.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.149+0000 7f91b4145700 1 -- 172.21.15.32:0/3562134811 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f919806cc30 msgr2=0x7f919806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:08:14.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.149+0000 7f91b4145700 1 --2- 172.21.15.32:0/3562134811 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f919806cc30 0x7f919806f0e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f919c013680 tx=0x7f919c009e30 comp rx=0 tx=0).stop 2024-05-12T04:08:14.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.149+0000 7f91b4145700 1 -- 172.21.15.32:0/3562134811 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91ac0662c0 msgr2=0x7f91ac0fd330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:08:14.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.149+0000 7f91b4145700 1 --2- 172.21.15.32:0/3562134811 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91ac0662c0 0x7f91ac0fd330 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f91a800eb40 tx=0x7f91a800b040 comp rx=0 tx=0).stop 2024-05-12T04:08:14.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.149+0000 7f91b4145700 1 -- 172.21.15.32:0/3562134811 shutdown_connections 2024-05-12T04:08:14.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.149+0000 7f91b4145700 1 --2- 172.21.15.32:0/3562134811 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f919806cc30 0x7f919806f0e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:14.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.149+0000 7f91b4145700 1 --2- 172.21.15.32:0/3562134811 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91ac0662c0 0x7f91ac0fd330 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:14.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.149+0000 7f91b4145700 1 --2- 172.21.15.32:0/3562134811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91ac0ffee0 0x7f91ac0fb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:14.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.149+0000 7f91b4145700 1 -- 172.21.15.32:0/3562134811 >> 172.21.15.32:0/3562134811 conn(0x7f91ac0f7430 msgr2=0x7f91ac0f9810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:08:14.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.149+0000 7f91b4145700 1 -- 172.21.15.32:0/3562134811 shutdown_connections 2024-05-12T04:08:14.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.149+0000 7f91b4145700 1 -- 172.21.15.32:0/3562134811 wait complete. 2024-05-12T04:08:14.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.254+0000 7f2fe2edc700 1 -- 172.21.15.32:0/2336565566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fdc0fcf60 msgr2=0x7f2fdc0fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:08:14.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.254+0000 7f2fe2edc700 1 --2- 172.21.15.32:0/2336565566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fdc0fcf60 0x7f2fdc0fd3d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8004940 tx=0x7f2fc8009e30 comp rx=0 tx=0).stop 2024-05-12T04:08:14.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.254+0000 7f2fe2edc700 1 -- 172.21.15.32:0/2336565566 shutdown_connections 2024-05-12T04:08:14.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.254+0000 7f2fe2edc700 1 --2- 172.21.15.32:0/2336565566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fdc0fcf60 0x7f2fdc0fd3d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:14.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.254+0000 7f2fe2edc700 1 --2- 172.21.15.32:0/2336565566 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fdc0fbcb0 0x7f2fdc0fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:14.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.254+0000 7f2fe2edc700 1 -- 172.21.15.32:0/2336565566 >> 172.21.15.32:0/2336565566 conn(0x7f2fdc0f7430 msgr2=0x7f2fdc0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:08:14.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.254+0000 7f2fe2edc700 1 -- 172.21.15.32:0/2336565566 shutdown_connections 2024-05-12T04:08:14.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.254+0000 7f2fe2edc700 1 -- 172.21.15.32:0/2336565566 wait complete. 2024-05-12T04:08:14.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.254+0000 7f2fe2edc700 1 Processor -- start 2024-05-12T04:08:14.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.255+0000 7f2fe2edc700 1 -- start start 2024-05-12T04:08:14.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.255+0000 7f2fe2edc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fdc0fbcb0 0x7f2fdc10d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:08:14.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.255+0000 7f2fe2edc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fdc0fcf60 0x7f2fdc10de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:08:14.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.255+0000 7f2fe2edc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fdc10e410 con 0x7f2fdc0fbcb0 2024-05-12T04:08:14.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.255+0000 7f2fe2edc700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fdc10ec10 con 0x7f2fdc0fcf60 2024-05-12T04:08:14.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.255+0000 7f2fe0c78700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fdc0fbcb0 0x7f2fdc10d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:08:14.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.255+0000 7f2fe0c78700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fdc0fbcb0 0x7f2fdc10d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44458/0 (socket says 172.21.15.32:44458) 2024-05-12T04:08:14.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.255+0000 7f2fe0c78700 1 -- 172.21.15.32:0/2216030983 learned_addr learned my addr 172.21.15.32:0/2216030983 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:08:14.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.255+0000 7f2fdbfff700 1 --2- 172.21.15.32:0/2216030983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fdc0fcf60 0x7f2fdc10de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:08:14.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.255+0000 7f2fe0c78700 1 -- 172.21.15.32:0/2216030983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fdc0fcf60 msgr2=0x7f2fdc10de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:08:14.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.255+0000 7f2fe0c78700 1 --2- 172.21.15.32:0/2216030983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fdc0fcf60 0x7f2fdc10de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:14.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.255+0000 7f2fe0c78700 1 -- 172.21.15.32:0/2216030983 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc8005040 con 0x7f2fdc0fbcb0 2024-05-12T04:08:14.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.255+0000 7f2fe0c78700 1 --2- 172.21.15.32:0/2216030983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fdc0fbcb0 0x7f2fdc10d8d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0002a80 tx=0x7f2fd00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:08:14.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.256+0000 7f2fd97fa700 1 -- 172.21.15.32:0/2216030983 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2fd0007cc0 con 0x7f2fdc0fbcb0 2024-05-12T04:08:14.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.256+0000 7f2fd97fa700 1 -- 172.21.15.32:0/2216030983 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2fd000f040 con 0x7f2fdc0fbcb0 2024-05-12T04:08:14.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.256+0000 7f2fe2edc700 1 -- 172.21.15.32:0/2216030983 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fdc10eef0 con 0x7f2fdc0fbcb0 2024-05-12T04:08:14.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.256+0000 7f2fd97fa700 1 -- 172.21.15.32:0/2216030983 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2fd000da60 con 0x7f2fdc0fbcb0 2024-05-12T04:08:14.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.256+0000 7f2fe2edc700 1 -- 172.21.15.32:0/2216030983 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fdc10f430 con 0x7f2fdc0fbcb0 2024-05-12T04:08:14.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.257+0000 7f2fe2edc700 1 -- 172.21.15.32:0/2216030983 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fdc101af0 con 0x7f2fdc0fbcb0 2024-05-12T04:08:14.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.257+0000 7f2fd97fa700 1 -- 172.21.15.32:0/2216030983 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f2fd000dbc0 con 0x7f2fdc0fbcb0 2024-05-12T04:08:14.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.258+0000 7f2fd97fa700 1 --2- 172.21.15.32:0/2216030983 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f2fcc06cc20 0x7f2fcc06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:08:14.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.258+0000 7f2fd97fa700 1 -- 172.21.15.32:0/2216030983 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f2fd008ba00 con 0x7f2fdc0fbcb0 2024-05-12T04:08:14.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.258+0000 7f2fdbfff700 1 --2- 172.21.15.32:0/2216030983 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f2fcc06cc20 0x7f2fcc06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:08:14.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.258+0000 7f2fdbfff700 1 --2- 172.21.15.32:0/2216030983 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f2fcc06cc20 0x7f2fcc06f0d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2fc800b3b0 tx=0x7f2fc8009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:08:14.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.262+0000 7f2fd97fa700 1 -- 172.21.15.32:0/2216030983 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f2fd00594d0 con 0x7f2fdc0fbcb0 2024-05-12T04:08:14.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.473+0000 7f2fe2edc700 1 -- 172.21.15.32:0/2216030983 --> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2fdc10f6a0 con 0x7f2fcc06cc20 2024-05-12T04:08:14.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.483+0000 7f2fd97fa700 1 -- 172.21.15.32:0/2216030983 <== mgr.24593 v2:172.21.15.32:6800/2957111591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f2fdc10f6a0 con 0x7f2fcc06cc20 2024-05-12T04:08:14.483 INFO:teuthology.orchestra.run.smithi032.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-12T04:08:14.483 INFO:teuthology.orchestra.run.smithi032.stdout:alertmanager.smithi032 smithi032 *:9093,9094 running (7m) 13s ago 16m 25.5M - 0.25.0 c8568f914cd2 a37fbab7af4c 2024-05-12T04:08:14.483 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi032 smithi032 running (3m) 13s ago 16m 6660k - 18.2.2-1272-g58249552 ddcb0a9d6fde 8c53f1fc7b06 2024-05-12T04:08:14.483 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi033 smithi033 running (3m) 34s ago 15m 6651k - 18.2.2-1272-g58249552 ddcb0a9d6fde cf4fe9ca070b 2024-05-12T04:08:14.483 INFO:teuthology.orchestra.run.smithi032.stdout:grafana.smithi032 smithi032 *:3000 running (6m) 13s ago 16m 89.1M - 9.4.7 954c08fa6188 83055e58d23c 2024-05-12T04:08:14.483 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.ialtqb smithi032 running (19s) 13s ago 10m 12.5M - 18.2.2-1272-g58249552 ddcb0a9d6fde 532dd9845ce7 2024-05-12T04:08:14.484 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.qpsawh smithi032 running (10m) 13s ago 10m 155M - 17.2.7-939-gc190bdee 284803f7d9cf 95a7db5c4fcd 2024-05-12T04:08:14.484 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi033.hglwml smithi033 running (10m) 34s ago 10m 14.0M - 17.2.7-939-gc190bdee 284803f7d9cf c97902b05025 2024-05-12T04:08:14.484 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi033.jgbjhk smithi033 running (10m) 34s ago 10m 102M - 17.2.7-939-gc190bdee 284803f7d9cf af97052cb97a 2024-05-12T04:08:14.484 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi032.ukatkq smithi032 *:8443,9283,8765 running (6m) 13s ago 17m 505M - 18.2.2-1272-g58249552 ddcb0a9d6fde 0d88ab9a5292 2024-05-12T04:08:14.484 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi033.pcvxwy smithi033 *:8443,9283,8765 running (5m) 34s ago 15m 450M - 18.2.2-1272-g58249552 ddcb0a9d6fde d42dcb6bd307 2024-05-12T04:08:14.484 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi032 smithi032 running (5m) 13s ago 17m 47.4M 2048M 18.2.2-1272-g58249552 ddcb0a9d6fde 81f94171c5dc 2024-05-12T04:08:14.484 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi033 smithi033 running (4m) 34s ago 15m 36.2M 2048M 18.2.2-1272-g58249552 ddcb0a9d6fde d648ba108f3e 2024-05-12T04:08:14.484 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi032 smithi032 *:9100 running (7m) 13s ago 16m 16.6M - 1.5.0 0da6a335fe13 cb75a7c0c829 2024-05-12T04:08:14.484 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi033 smithi033 *:9100 running (6m) 34s ago 15m 18.9M - 1.5.0 0da6a335fe13 7888c36c659f 2024-05-12T04:08:14.484 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0 smithi032 running (3m) 13s ago 14m 86.2M 1570M 18.2.2-1272-g58249552 ddcb0a9d6fde 07f66c4deb3b 2024-05-12T04:08:14.484 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1 smithi032 running (2m) 13s ago 13m 87.7M 1570M 18.2.2-1272-g58249552 ddcb0a9d6fde 67a87807d654 2024-05-12T04:08:14.484 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2 smithi032 running (2m) 13s ago 13m 77.8M 1570M 18.2.2-1272-g58249552 ddcb0a9d6fde 4ac48d74ee8b 2024-05-12T04:08:14.484 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3 smithi033 running (102s) 34s ago 12m 90.9M 2594M 18.2.2-1272-g58249552 ddcb0a9d6fde 11f097a455bb 2024-05-12T04:08:14.484 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4 smithi033 running (70s) 34s ago 12m 84.8M 2594M 18.2.2-1272-g58249552 ddcb0a9d6fde db6fda7d52de 2024-05-12T04:08:14.484 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5 smithi033 running (37s) 34s ago 11m 11.4M 2594M 18.2.2-1272-g58249552 ddcb0a9d6fde 3333020b8a15 2024-05-12T04:08:14.485 INFO:teuthology.orchestra.run.smithi032.stdout:prometheus.smithi032 smithi032 *:9095 running (5m) 13s ago 15m 61.3M - 2.43.0 a07b618ecd1d 46eafb25e176 2024-05-12T04:08:14.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.485+0000 7f2fe2edc700 1 -- 172.21.15.32:0/2216030983 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f2fcc06cc20 msgr2=0x7f2fcc06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:08:14.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.485+0000 7f2fe2edc700 1 --2- 172.21.15.32:0/2216030983 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f2fcc06cc20 0x7f2fcc06f0d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2fc800b3b0 tx=0x7f2fc8009e30 comp rx=0 tx=0).stop 2024-05-12T04:08:14.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.485+0000 7f2fe2edc700 1 -- 172.21.15.32:0/2216030983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fdc0fbcb0 msgr2=0x7f2fdc10d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:08:14.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.485+0000 7f2fe2edc700 1 --2- 172.21.15.32:0/2216030983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fdc0fbcb0 0x7f2fdc10d8d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0002a80 tx=0x7f2fd00046b0 comp rx=0 tx=0).stop 2024-05-12T04:08:14.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.485+0000 7f2fe2edc700 1 -- 172.21.15.32:0/2216030983 shutdown_connections 2024-05-12T04:08:14.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.485+0000 7f2fe2edc700 1 --2- 172.21.15.32:0/2216030983 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f2fcc06cc20 0x7f2fcc06f0d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:14.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.485+0000 7f2fe2edc700 1 --2- 172.21.15.32:0/2216030983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fdc0fbcb0 0x7f2fdc10d8d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:14.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.485+0000 7f2fe2edc700 1 --2- 172.21.15.32:0/2216030983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fdc0fcf60 0x7f2fdc10de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:14.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.485+0000 7f2fe2edc700 1 -- 172.21.15.32:0/2216030983 >> 172.21.15.32:0/2216030983 conn(0x7f2fdc0f7430 msgr2=0x7f2fdc0f8bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:08:14.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.485+0000 7f2fe2edc700 1 -- 172.21.15.32:0/2216030983 shutdown_connections 2024-05-12T04:08:14.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.485+0000 7f2fe2edc700 1 -- 172.21.15.32:0/2216030983 wait complete. 2024-05-12T04:08:14.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.588+0000 7f0c99f99700 1 -- 172.21.15.32:0/2569871419 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c940fcf10 msgr2=0x7f0c940fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:08:14.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.588+0000 7f0c99f99700 1 --2- 172.21.15.32:0/2569871419 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c940fcf10 0x7f0c940fd320 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0c84004940 tx=0x7f0c84009e30 comp rx=0 tx=0).stop 2024-05-12T04:08:14.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.588+0000 7f0c99f99700 1 -- 172.21.15.32:0/2569871419 shutdown_connections 2024-05-12T04:08:14.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.588+0000 7f0c99f99700 1 --2- 172.21.15.32:0/2569871419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c940fe080 0x7f0c940fe4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:14.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.588+0000 7f0c99f99700 1 --2- 172.21.15.32:0/2569871419 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c940fcf10 0x7f0c940fd320 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:14.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.588+0000 7f0c99f99700 1 -- 172.21.15.32:0/2569871419 >> 172.21.15.32:0/2569871419 conn(0x7f0c940f8560 msgr2=0x7f0c940fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:08:14.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.588+0000 7f0c99f99700 1 -- 172.21.15.32:0/2569871419 shutdown_connections 2024-05-12T04:08:14.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.588+0000 7f0c99f99700 1 -- 172.21.15.32:0/2569871419 wait complete. 2024-05-12T04:08:14.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.588+0000 7f0c99f99700 1 Processor -- start 2024-05-12T04:08:14.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.589+0000 7f0c99f99700 1 -- start start 2024-05-12T04:08:14.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.589+0000 7f0c99f99700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c940fcf10 0x7f0c9410d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:08:14.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.589+0000 7f0c99f99700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c940fe080 0x7f0c9410dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:08:14.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.589+0000 7f0c99f99700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9410e4a0 con 0x7f0c940fcf10 2024-05-12T04:08:14.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.589+0000 7f0c99f99700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9410eca0 con 0x7f0c940fe080 2024-05-12T04:08:14.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.589+0000 7f0c937fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c940fcf10 0x7f0c9410d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:08:14.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.589+0000 7f0c937fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c940fcf10 0x7f0c9410d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44480/0 (socket says 172.21.15.32:44480) 2024-05-12T04:08:14.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.589+0000 7f0c937fe700 1 -- 172.21.15.32:0/2383580352 learned_addr learned my addr 172.21.15.32:0/2383580352 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:08:14.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.589+0000 7f0c92ffd700 1 --2- 172.21.15.32:0/2383580352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c940fe080 0x7f0c9410dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:08:14.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.589+0000 7f0c937fe700 1 -- 172.21.15.32:0/2383580352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c940fe080 msgr2=0x7f0c9410dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:08:14.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.589+0000 7f0c937fe700 1 --2- 172.21.15.32:0/2383580352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c940fe080 0x7f0c9410dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:14.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.589+0000 7f0c937fe700 1 -- 172.21.15.32:0/2383580352 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c84005040 con 0x7f0c940fcf10 2024-05-12T04:08:14.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.590+0000 7f0c937fe700 1 --2- 172.21.15.32:0/2383580352 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c940fcf10 0x7f0c9410d960 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0c84009fd0 tx=0x7f0c84009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:08:14.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.590+0000 7f0c7bfff700 1 -- 172.21.15.32:0/2383580352 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c8400e040 con 0x7f0c940fcf10 2024-05-12T04:08:14.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.590+0000 7f0c7bfff700 1 -- 172.21.15.32:0/2383580352 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0c84004290 con 0x7f0c940fcf10 2024-05-12T04:08:14.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.590+0000 7f0c99f99700 1 -- 172.21.15.32:0/2383580352 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c9410ef20 con 0x7f0c940fcf10 2024-05-12T04:08:14.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.590+0000 7f0c7bfff700 1 -- 172.21.15.32:0/2383580352 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c8401c3f0 con 0x7f0c940fcf10 2024-05-12T04:08:14.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.590+0000 7f0c99f99700 1 -- 172.21.15.32:0/2383580352 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c9410f460 con 0x7f0c940fcf10 2024-05-12T04:08:14.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.591+0000 7f0c99f99700 1 -- 172.21.15.32:0/2383580352 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c9405f5c0 con 0x7f0c940fcf10 2024-05-12T04:08:14.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.591+0000 7f0c7bfff700 1 -- 172.21.15.32:0/2383580352 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f0c84013790 con 0x7f0c940fcf10 2024-05-12T04:08:14.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.592+0000 7f0c7bfff700 1 --2- 172.21.15.32:0/2383580352 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f0c7c06cc30 0x7f0c7c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:08:14.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.592+0000 7f0c7bfff700 1 -- 172.21.15.32:0/2383580352 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f0c8408d3c0 con 0x7f0c940fcf10 2024-05-12T04:08:14.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.592+0000 7f0c92ffd700 1 --2- 172.21.15.32:0/2383580352 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f0c7c06cc30 0x7f0c7c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:08:14.593 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.592+0000 7f0c92ffd700 1 --2- 172.21.15.32:0/2383580352 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f0c7c06cc30 0x7f0c7c06f0e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0c88004f40 tx=0x7f0c8800b040 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:08:14.596 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.596+0000 7f0c7bfff700 1 -- 172.21.15.32:0/2383580352 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f0c8405ae90 con 0x7f0c940fcf10 2024-05-12T04:08:14.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.880+0000 7f0c99f99700 1 -- 172.21.15.32:0/2383580352 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0c9410f830 con 0x7f0c940fcf10 2024-05-12T04:08:14.881 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.881+0000 7f0c7bfff700 1 -- 172.21.15.32:0/2383580352 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+847 (secure 0 0 0) 0x7f0c84013420 con 0x7f0c940fcf10 2024-05-12T04:08:14.881 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-05-12T04:08:14.881 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": { 2024-05-12T04:08:14.881 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 2 2024-05-12T04:08:14.881 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:08:14.882 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": { 2024-05-12T04:08:14.882 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 2 2024-05-12T04:08:14.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:08:14.882 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": { 2024-05-12T04:08:14.882 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 6 2024-05-12T04:08:14.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:08:14.882 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": { 2024-05-12T04:08:14.882 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 2, 2024-05-12T04:08:14.882 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 1 2024-05-12T04:08:14.882 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:08:14.882 INFO:teuthology.orchestra.run.smithi032.stdout: "overall": { 2024-05-12T04:08:14.883 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 2, 2024-05-12T04:08:14.883 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 11 2024-05-12T04:08:14.883 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-05-12T04:08:14.883 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-05-12T04:08:14.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.883+0000 7f0c99f99700 1 -- 172.21.15.32:0/2383580352 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f0c7c06cc30 msgr2=0x7f0c7c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:08:14.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.883+0000 7f0c99f99700 1 --2- 172.21.15.32:0/2383580352 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f0c7c06cc30 0x7f0c7c06f0e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0c88004f40 tx=0x7f0c8800b040 comp rx=0 tx=0).stop 2024-05-12T04:08:14.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.883+0000 7f0c99f99700 1 -- 172.21.15.32:0/2383580352 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c940fcf10 msgr2=0x7f0c9410d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:08:14.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.883+0000 7f0c99f99700 1 --2- 172.21.15.32:0/2383580352 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c940fcf10 0x7f0c9410d960 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0c84009fd0 tx=0x7f0c84009e30 comp rx=0 tx=0).stop 2024-05-12T04:08:14.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.883+0000 7f0c99f99700 1 -- 172.21.15.32:0/2383580352 shutdown_connections 2024-05-12T04:08:14.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.883+0000 7f0c99f99700 1 --2- 172.21.15.32:0/2383580352 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f0c7c06cc30 0x7f0c7c06f0e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:14.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.883+0000 7f0c99f99700 1 --2- 172.21.15.32:0/2383580352 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c940fcf10 0x7f0c9410d960 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:14.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.883+0000 7f0c99f99700 1 --2- 172.21.15.32:0/2383580352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c940fe080 0x7f0c9410dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:14.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.883+0000 7f0c99f99700 1 -- 172.21.15.32:0/2383580352 >> 172.21.15.32:0/2383580352 conn(0x7f0c940f8560 msgr2=0x7f0c940f9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:08:14.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.883+0000 7f0c99f99700 1 -- 172.21.15.32:0/2383580352 shutdown_connections 2024-05-12T04:08:14.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.883+0000 7f0c99f99700 1 -- 172.21.15.32:0/2383580352 wait complete. 2024-05-12T04:08:14.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.986+0000 7f79e3590700 1 -- 172.21.15.32:0/2157881430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79dc0fe420 msgr2=0x7f79dc0fe830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:08:14.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.986+0000 7f79e3590700 1 --2- 172.21.15.32:0/2157881430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79dc0fe420 0x7f79dc0fe830 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f79d0004940 tx=0x7f79d0009e30 comp rx=0 tx=0).stop 2024-05-12T04:08:14.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.986+0000 7f79e3590700 1 -- 172.21.15.32:0/2157881430 shutdown_connections 2024-05-12T04:08:14.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.986+0000 7f79e3590700 1 --2- 172.21.15.32:0/2157881430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79dc0fed70 0x7f79dc0faeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:14.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.986+0000 7f79e3590700 1 --2- 172.21.15.32:0/2157881430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79dc0fe420 0x7f79dc0fe830 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:14.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.986+0000 7f79e3590700 1 -- 172.21.15.32:0/2157881430 >> 172.21.15.32:0/2157881430 conn(0x7f79dc0f6be0 msgr2=0x7f79dc0f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:08:14.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.987+0000 7f79e3590700 1 -- 172.21.15.32:0/2157881430 shutdown_connections 2024-05-12T04:08:14.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.987+0000 7f79e3590700 1 -- 172.21.15.32:0/2157881430 wait complete. 2024-05-12T04:08:14.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.987+0000 7f79e3590700 1 Processor -- start 2024-05-12T04:08:14.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.987+0000 7f79e3590700 1 -- start start 2024-05-12T04:08:14.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.987+0000 7f79e3590700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79dc0fe420 0x7f79dc10d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:08:14.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.987+0000 7f79e3590700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79dc0fed70 0x7f79dc10dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:08:14.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.987+0000 7f79e3590700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79dc10e320 con 0x7f79dc0fed70 2024-05-12T04:08:14.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.987+0000 7f79e3590700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79dc10e490 con 0x7f79dc0fe420 2024-05-12T04:08:14.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.987+0000 7f79e0b2b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79dc0fed70 0x7f79dc10dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:08:14.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.987+0000 7f79e132c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79dc0fe420 0x7f79dc10d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:08:14.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.987+0000 7f79e0b2b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79dc0fed70 0x7f79dc10dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44502/0 (socket says 172.21.15.32:44502) 2024-05-12T04:08:14.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.987+0000 7f79e0b2b700 1 -- 172.21.15.32:0/3998980698 learned_addr learned my addr 172.21.15.32:0/3998980698 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:08:14.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.988+0000 7f79e0b2b700 1 -- 172.21.15.32:0/3998980698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79dc0fe420 msgr2=0x7f79dc10d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:08:14.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.988+0000 7f79e0b2b700 1 --2- 172.21.15.32:0/3998980698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79dc0fe420 0x7f79dc10d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:14.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.988+0000 7f79e0b2b700 1 -- 172.21.15.32:0/3998980698 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d0005040 con 0x7f79dc0fed70 2024-05-12T04:08:14.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.988+0000 7f79e132c700 1 --2- 172.21.15.32:0/3998980698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79dc0fe420 0x7f79dc10d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-12T04:08:14.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.988+0000 7f79e0b2b700 1 --2- 172.21.15.32:0/3998980698 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79dc0fed70 0x7f79dc10dd20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f79d8002a80 tx=0x7f79d80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:08:14.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.988+0000 7f79cdffb700 1 -- 172.21.15.32:0/3998980698 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f79d8007cc0 con 0x7f79dc0fed70 2024-05-12T04:08:14.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.988+0000 7f79cdffb700 1 -- 172.21.15.32:0/3998980698 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f79d800f040 con 0x7f79dc0fed70 2024-05-12T04:08:14.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.988+0000 7f79cdffb700 1 -- 172.21.15.32:0/3998980698 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f79d800da60 con 0x7f79dc0fed70 2024-05-12T04:08:14.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.988+0000 7f79e3590700 1 -- 172.21.15.32:0/3998980698 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79dc112ea0 con 0x7f79dc0fed70 2024-05-12T04:08:14.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.988+0000 7f79e3590700 1 -- 172.21.15.32:0/3998980698 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79dc113430 con 0x7f79dc0fed70 2024-05-12T04:08:14.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.989+0000 7f79e3590700 1 -- 172.21.15.32:0/3998980698 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79dc04c710 con 0x7f79dc0fed70 2024-05-12T04:08:14.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.990+0000 7f79cdffb700 1 -- 172.21.15.32:0/3998980698 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f79d800dbc0 con 0x7f79dc0fed70 2024-05-12T04:08:14.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.990+0000 7f79cdffb700 1 --2- 172.21.15.32:0/3998980698 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f79c806cc20 0x7f79c806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:08:14.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.990+0000 7f79cdffb700 1 -- 172.21.15.32:0/3998980698 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f79d808bc10 con 0x7f79dc0fed70 2024-05-12T04:08:14.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.990+0000 7f79e132c700 1 --2- 172.21.15.32:0/3998980698 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f79c806cc20 0x7f79c806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:08:14.991 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.990+0000 7f79e132c700 1 --2- 172.21.15.32:0/3998980698 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f79c806cc20 0x7f79c806f0d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f79d000b3b0 tx=0x7f79d0009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:08:14.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:14.994+0000 7f79cdffb700 1 -- 172.21.15.32:0/3998980698 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f79d80596e0 con 0x7f79dc0fed70 2024-05-12T04:08:15.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.245+0000 7f79e3590700 1 -- 172.21.15.32:0/3998980698 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f79dc105790 con 0x7f79dc0fed70 2024-05-12T04:08:15.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.245+0000 7f79cdffb700 1 -- 172.21.15.32:0/3998980698 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) v1 ==== 76+0+1667 (secure 0 0 0) 0x7f79d8059270 con 0x7f79dc0fed70 2024-05-12T04:08:15.246 INFO:teuthology.orchestra.run.smithi032.stdout:e24 2024-05-12T04:08:15.246 INFO:teuthology.orchestra.run.smithi032.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-12T04:08:15.246 INFO:teuthology.orchestra.run.smithi032.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-12T04:08:15.246 INFO:teuthology.orchestra.run.smithi032.stdout:legacy client fscid: 1 2024-05-12T04:08:15.246 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:08:15.246 INFO:teuthology.orchestra.run.smithi032.stdout:Filesystem 'cephfs' (1) 2024-05-12T04:08:15.246 INFO:teuthology.orchestra.run.smithi032.stdout:fs_name cephfs 2024-05-12T04:08:15.246 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 24 2024-05-12T04:08:15.246 INFO:teuthology.orchestra.run.smithi032.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-05-12T04:08:15.246 INFO:teuthology.orchestra.run.smithi032.stdout:created 2024-05-12T03:57:48.904614+0000 2024-05-12T04:08:15.246 INFO:teuthology.orchestra.run.smithi032.stdout:modified 2024-05-12T04:08:08.981164+0000 2024-05-12T04:08:15.246 INFO:teuthology.orchestra.run.smithi032.stdout:tableserver 0 2024-05-12T04:08:15.246 INFO:teuthology.orchestra.run.smithi032.stdout:root 0 2024-05-12T04:08:15.246 INFO:teuthology.orchestra.run.smithi032.stdout:session_timeout 60 2024-05-12T04:08:15.247 INFO:teuthology.orchestra.run.smithi032.stdout:session_autoclose 300 2024-05-12T04:08:15.247 INFO:teuthology.orchestra.run.smithi032.stdout:max_file_size 1099511627776 2024-05-12T04:08:15.247 INFO:teuthology.orchestra.run.smithi032.stdout:max_xattr_size 65536 2024-05-12T04:08:15.247 INFO:teuthology.orchestra.run.smithi032.stdout:required_client_features {} 2024-05-12T04:08:15.247 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure 0 2024-05-12T04:08:15.247 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure_osd_epoch 82 2024-05-12T04:08:15.247 INFO:teuthology.orchestra.run.smithi032.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-12T04:08:15.247 INFO:teuthology.orchestra.run.smithi032.stdout:max_mds 1 2024-05-12T04:08:15.247 INFO:teuthology.orchestra.run.smithi032.stdout:in 0 2024-05-12T04:08:15.247 INFO:teuthology.orchestra.run.smithi032.stdout:up {0=24283} 2024-05-12T04:08:15.247 INFO:teuthology.orchestra.run.smithi032.stdout:failed 2024-05-12T04:08:15.247 INFO:teuthology.orchestra.run.smithi032.stdout:damaged 2024-05-12T04:08:15.247 INFO:teuthology.orchestra.run.smithi032.stdout:stopped 2024-05-12T04:08:15.247 INFO:teuthology.orchestra.run.smithi032.stdout:data_pools [3] 2024-05-12T04:08:15.247 INFO:teuthology.orchestra.run.smithi032.stdout:metadata_pool 2 2024-05-12T04:08:15.247 INFO:teuthology.orchestra.run.smithi032.stdout:inline_data enabled 2024-05-12T04:08:15.248 INFO:teuthology.orchestra.run.smithi032.stdout:balancer 2024-05-12T04:08:15.248 INFO:teuthology.orchestra.run.smithi032.stdout:bal_rank_mask -1 2024-05-12T04:08:15.248 INFO:teuthology.orchestra.run.smithi032.stdout:standby_count_wanted 1 2024-05-12T04:08:15.248 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi033.hglwml{0:24283} state up:replay seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/433858238,v1:172.21.15.33:6827/433858238] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:08:15.248 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:08:15.248 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:08:15.248 INFO:teuthology.orchestra.run.smithi032.stdout:Standby daemons: 2024-05-12T04:08:15.248 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:08:15.248 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.ialtqb{-1:34404} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.32:6826/1080358503,v1:172.21.15.32:6827/1080358503] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:08:15.248 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi033.jgbjhk{-1:44303} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6824/2944716783,v1:172.21.15.33:6825/2944716783] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:08:15.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.247+0000 7f79e3590700 1 -- 172.21.15.32:0/3998980698 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f79c806cc20 msgr2=0x7f79c806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:08:15.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.247+0000 7f79e3590700 1 --2- 172.21.15.32:0/3998980698 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f79c806cc20 0x7f79c806f0d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f79d000b3b0 tx=0x7f79d0009e30 comp rx=0 tx=0).stop 2024-05-12T04:08:15.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.247+0000 7f79e3590700 1 -- 172.21.15.32:0/3998980698 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79dc0fed70 msgr2=0x7f79dc10dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:08:15.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.247+0000 7f79e3590700 1 --2- 172.21.15.32:0/3998980698 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79dc0fed70 0x7f79dc10dd20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f79d8002a80 tx=0x7f79d80046b0 comp rx=0 tx=0).stop 2024-05-12T04:08:15.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.247+0000 7f79e3590700 1 -- 172.21.15.32:0/3998980698 shutdown_connections 2024-05-12T04:08:15.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.247+0000 7f79e3590700 1 --2- 172.21.15.32:0/3998980698 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f79c806cc20 0x7f79c806f0d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:15.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.247+0000 7f79e3590700 1 --2- 172.21.15.32:0/3998980698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79dc0fe420 0x7f79dc10d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:15.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.247+0000 7f79e3590700 1 --2- 172.21.15.32:0/3998980698 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79dc0fed70 0x7f79dc10dd20 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:15.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.247+0000 7f79e3590700 1 -- 172.21.15.32:0/3998980698 >> 172.21.15.32:0/3998980698 conn(0x7f79dc0f6be0 msgr2=0x7f79dc0f82b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:08:15.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.248+0000 7f79e3590700 1 -- 172.21.15.32:0/3998980698 shutdown_connections 2024-05-12T04:08:15.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.248+0000 7f79e3590700 1 -- 172.21.15.32:0/3998980698 wait complete. 2024-05-12T04:08:15.250 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 24 2024-05-12T04:08:15.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.352+0000 7fa215b51700 1 -- 172.21.15.32:0/157889412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2100ffee0 msgr2=0x7fa210102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:08:15.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.352+0000 7fa215b51700 1 --2- 172.21.15.32:0/157889412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2100ffee0 0x7fa210102300 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa204004940 tx=0x7fa204009e30 comp rx=0 tx=0).stop 2024-05-12T04:08:15.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.352+0000 7fa215b51700 1 -- 172.21.15.32:0/157889412 shutdown_connections 2024-05-12T04:08:15.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.352+0000 7fa215b51700 1 --2- 172.21.15.32:0/157889412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2100ffee0 0x7fa210102300 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:15.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.352+0000 7fa215b51700 1 --2- 172.21.15.32:0/157889412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2100662c0 0x7fa2100ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:15.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.352+0000 7fa215b51700 1 -- 172.21.15.32:0/157889412 >> 172.21.15.32:0/157889412 conn(0x7fa2100f7430 msgr2=0x7fa2100f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:08:15.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.352+0000 7fa215b51700 1 -- 172.21.15.32:0/157889412 shutdown_connections 2024-05-12T04:08:15.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.352+0000 7fa215b51700 1 -- 172.21.15.32:0/157889412 wait complete. 2024-05-12T04:08:15.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.353+0000 7fa215b51700 1 Processor -- start 2024-05-12T04:08:15.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.353+0000 7fa215b51700 1 -- start start 2024-05-12T04:08:15.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.353+0000 7fa215b51700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2100662c0 0x7fa21010d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:08:15.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.353+0000 7fa215b51700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2100ffee0 0x7fa21010de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:08:15.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.353+0000 7fa215b51700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21010e430 con 0x7fa2100ffee0 2024-05-12T04:08:15.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.353+0000 7fa215b51700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21010ec80 con 0x7fa2100662c0 2024-05-12T04:08:15.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.353+0000 7fa20effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2100ffee0 0x7fa21010de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:08:15.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.353+0000 7fa20f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2100662c0 0x7fa21010d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:08:15.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.353+0000 7fa20f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2100662c0 0x7fa21010d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.32:58906/0 (socket says 172.21.15.32:58906) 2024-05-12T04:08:15.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.353+0000 7fa20f7fe700 1 -- 172.21.15.32:0/531325544 learned_addr learned my addr 172.21.15.32:0/531325544 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:08:15.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.353+0000 7fa20f7fe700 1 -- 172.21.15.32:0/531325544 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2100ffee0 msgr2=0x7fa21010de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:08:15.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.353+0000 7fa20f7fe700 1 --2- 172.21.15.32:0/531325544 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2100ffee0 0x7fa21010de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:15.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.353+0000 7fa20f7fe700 1 -- 172.21.15.32:0/531325544 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa204005040 con 0x7fa2100662c0 2024-05-12T04:08:15.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.354+0000 7fa20f7fe700 1 --2- 172.21.15.32:0/531325544 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2100662c0 0x7fa21010d8f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa20000eb40 tx=0x7fa20000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:08:15.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.354+0000 7fa1f7fff700 1 -- 172.21.15.32:0/531325544 <== mon.1 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa200007cc0 con 0x7fa2100662c0 2024-05-12T04:08:15.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.354+0000 7fa215b51700 1 -- 172.21.15.32:0/531325544 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa21010ef00 con 0x7fa2100662c0 2024-05-12T04:08:15.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.354+0000 7fa1f7fff700 1 -- 172.21.15.32:0/531325544 <== mon.1 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa20000f040 con 0x7fa2100662c0 2024-05-12T04:08:15.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.354+0000 7fa1f7fff700 1 -- 172.21.15.32:0/531325544 <== mon.1 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa20000ca30 con 0x7fa2100662c0 2024-05-12T04:08:15.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.354+0000 7fa215b51700 1 -- 172.21.15.32:0/531325544 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa21010f440 con 0x7fa2100662c0 2024-05-12T04:08:15.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.355+0000 7fa215b51700 1 -- 172.21.15.32:0/531325544 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa21005f5c0 con 0x7fa2100662c0 2024-05-12T04:08:15.356 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.355+0000 7fa1f7fff700 1 -- 172.21.15.32:0/531325544 <== mon.1 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fa20000cb90 con 0x7fa2100662c0 2024-05-12T04:08:15.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.356+0000 7fa1f7fff700 1 --2- 172.21.15.32:0/531325544 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fa1f806cc30 0x7fa1f806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:08:15.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.356+0000 7fa1f7fff700 1 -- 172.21.15.32:0/531325544 <== mon.1 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fa20008bbb0 con 0x7fa2100662c0 2024-05-12T04:08:15.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.356+0000 7fa20effd700 1 --2- 172.21.15.32:0/531325544 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fa1f806cc30 0x7fa1f806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:08:15.357 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.356+0000 7fa20effd700 1 --2- 172.21.15.32:0/531325544 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fa1f806cc30 0x7fa1f806f0e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa204013680 tx=0x7fa204009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:08:15.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.365+0000 7fa1f7fff700 1 -- 172.21.15.32:0/531325544 <== mon.1 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fa200059600 con 0x7fa2100662c0 2024-05-12T04:08:15.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:15 smithi032 ceph-mon[120392]: pgmap v185: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 15 MiB/s rd, 5 op/s 2024-05-12T04:08:15.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:15 smithi032 ceph-mon[120392]: from='client.34408 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:08:15.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:15 smithi032 ceph-mon[120392]: from='client.34412 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:08:15.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:15 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/2383580352' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:08:15.588 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.588+0000 7fa215b51700 1 -- 172.21.15.32:0/531325544 --> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa21010f7c0 con 0x7fa1f806cc30 2024-05-12T04:08:15.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.589+0000 7fa1f7fff700 1 -- 172.21.15.32:0/531325544 <== mgr.24593 v2:172.21.15.32:6800/2957111591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fa21010f7c0 con 0x7fa1f806cc30 2024-05-12T04:08:15.589 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-05-12T04:08:15.589 INFO:teuthology.orchestra.run.smithi032.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:19c9734256aa43a60e203feb242f50bf8b7f58666d520536af43cc4fee96173a", 2024-05-12T04:08:15.590 INFO:teuthology.orchestra.run.smithi032.stdout: "in_progress": true, 2024-05-12T04:08:15.590 INFO:teuthology.orchestra.run.smithi032.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-12T04:08:15.590 INFO:teuthology.orchestra.run.smithi032.stdout: "services_complete": [ 2024-05-12T04:08:15.590 INFO:teuthology.orchestra.run.smithi032.stdout: "osd", 2024-05-12T04:08:15.590 INFO:teuthology.orchestra.run.smithi032.stdout: "mon", 2024-05-12T04:08:15.590 INFO:teuthology.orchestra.run.smithi032.stdout: "crash", 2024-05-12T04:08:15.590 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr" 2024-05-12T04:08:15.590 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-05-12T04:08:15.590 INFO:teuthology.orchestra.run.smithi032.stdout: "progress": "13/21 daemons upgraded", 2024-05-12T04:08:15.590 INFO:teuthology.orchestra.run.smithi032.stdout: "message": "Currently upgrading mds daemons", 2024-05-12T04:08:15.590 INFO:teuthology.orchestra.run.smithi032.stdout: "is_paused": false 2024-05-12T04:08:15.590 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-05-12T04:08:15.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.591+0000 7fa215b51700 1 -- 172.21.15.32:0/531325544 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fa1f806cc30 msgr2=0x7fa1f806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:08:15.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.591+0000 7fa215b51700 1 --2- 172.21.15.32:0/531325544 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fa1f806cc30 0x7fa1f806f0e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa204013680 tx=0x7fa204009e30 comp rx=0 tx=0).stop 2024-05-12T04:08:15.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.591+0000 7fa215b51700 1 -- 172.21.15.32:0/531325544 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2100662c0 msgr2=0x7fa21010d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:08:15.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.591+0000 7fa215b51700 1 --2- 172.21.15.32:0/531325544 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2100662c0 0x7fa21010d8f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa20000eb40 tx=0x7fa20000b040 comp rx=0 tx=0).stop 2024-05-12T04:08:15.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.591+0000 7fa215b51700 1 -- 172.21.15.32:0/531325544 shutdown_connections 2024-05-12T04:08:15.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.591+0000 7fa215b51700 1 --2- 172.21.15.32:0/531325544 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fa1f806cc30 0x7fa1f806f0e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:15.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.591+0000 7fa215b51700 1 --2- 172.21.15.32:0/531325544 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2100662c0 0x7fa21010d8f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:15.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.591+0000 7fa215b51700 1 --2- 172.21.15.32:0/531325544 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2100ffee0 0x7fa21010de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:15.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.591+0000 7fa215b51700 1 -- 172.21.15.32:0/531325544 >> 172.21.15.32:0/531325544 conn(0x7fa2100f7430 msgr2=0x7fa2100f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:08:15.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.591+0000 7fa215b51700 1 -- 172.21.15.32:0/531325544 shutdown_connections 2024-05-12T04:08:15.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.591+0000 7fa215b51700 1 -- 172.21.15.32:0/531325544 wait complete. 2024-05-12T04:08:15.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:15 smithi033 ceph-mon[107305]: pgmap v185: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 15 MiB/s rd, 5 op/s 2024-05-12T04:08:15.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:15 smithi033 ceph-mon[107305]: from='client.34408 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:08:15.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:15 smithi033 ceph-mon[107305]: from='client.34412 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:08:15.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:15 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/2383580352' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:08:15.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.695+0000 7f6f19934700 1 -- 172.21.15.32:0/3851479567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f140fbcf0 msgr2=0x7f6f140fc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:08:15.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.695+0000 7f6f19934700 1 --2- 172.21.15.32:0/3851479567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f140fbcf0 0x7f6f140fc100 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6f04004940 tx=0x7f6f04009e30 comp rx=0 tx=0).stop 2024-05-12T04:08:15.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.695+0000 7f6f19934700 1 -- 172.21.15.32:0/3851479567 shutdown_connections 2024-05-12T04:08:15.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.695+0000 7f6f19934700 1 --2- 172.21.15.32:0/3851479567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f140fcfa0 0x7f6f140fd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:15.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.695+0000 7f6f19934700 1 --2- 172.21.15.32:0/3851479567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f140fbcf0 0x7f6f140fc100 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:15.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.695+0000 7f6f19934700 1 -- 172.21.15.32:0/3851479567 >> 172.21.15.32:0/3851479567 conn(0x7f6f140f7450 msgr2=0x7f6f140f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:08:15.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.695+0000 7f6f19934700 1 -- 172.21.15.32:0/3851479567 shutdown_connections 2024-05-12T04:08:15.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.695+0000 7f6f19934700 1 -- 172.21.15.32:0/3851479567 wait complete. 2024-05-12T04:08:15.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.696+0000 7f6f19934700 1 Processor -- start 2024-05-12T04:08:15.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.696+0000 7f6f19934700 1 -- start start 2024-05-12T04:08:15.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.696+0000 7f6f19934700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f140fbcf0 0x7f6f140ffbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:08:15.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.696+0000 7f6f19934700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f140fcfa0 0x7f6f140fe2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:08:15.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.696+0000 7f6f19934700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f141001e0 con 0x7f6f140fbcf0 2024-05-12T04:08:15.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.696+0000 7f6f19934700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f140fe820 con 0x7f6f140fcfa0 2024-05-12T04:08:15.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.696+0000 7f6f12ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f140fbcf0 0x7f6f140ffbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:08:15.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.696+0000 7f6f12ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f140fbcf0 0x7f6f140ffbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44544/0 (socket says 172.21.15.32:44544) 2024-05-12T04:08:15.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.696+0000 7f6f12ffd700 1 -- 172.21.15.32:0/275309080 learned_addr learned my addr 172.21.15.32:0/275309080 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:08:15.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.696+0000 7f6f0a7fc700 1 --2- 172.21.15.32:0/275309080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f140fcfa0 0x7f6f140fe2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:08:15.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.697+0000 7f6f12ffd700 1 -- 172.21.15.32:0/275309080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f140fcfa0 msgr2=0x7f6f140fe2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:08:15.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.697+0000 7f6f12ffd700 1 --2- 172.21.15.32:0/275309080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f140fcfa0 0x7f6f140fe2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:15.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.697+0000 7f6f12ffd700 1 -- 172.21.15.32:0/275309080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f04005040 con 0x7f6f140fbcf0 2024-05-12T04:08:15.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.697+0000 7f6f12ffd700 1 --2- 172.21.15.32:0/275309080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f140fbcf0 0x7f6f140ffbe0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6f040137b0 tx=0x7f6f04009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:08:15.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.697+0000 7f6f0bfff700 1 -- 172.21.15.32:0/275309080 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f0400e070 con 0x7f6f140fbcf0 2024-05-12T04:08:15.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.697+0000 7f6f0bfff700 1 -- 172.21.15.32:0/275309080 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6f0401c400 con 0x7f6f140fbcf0 2024-05-12T04:08:15.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.697+0000 7f6f19934700 1 -- 172.21.15.32:0/275309080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f140feaa0 con 0x7f6f140fbcf0 2024-05-12T04:08:15.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.697+0000 7f6f0bfff700 1 -- 172.21.15.32:0/275309080 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f0400b650 con 0x7f6f140fbcf0 2024-05-12T04:08:15.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.697+0000 7f6f19934700 1 -- 172.21.15.32:0/275309080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f140fef80 con 0x7f6f140fbcf0 2024-05-12T04:08:15.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.698+0000 7f6f19934700 1 -- 172.21.15.32:0/275309080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f14101b30 con 0x7f6f140fbcf0 2024-05-12T04:08:15.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.699+0000 7f6f0bfff700 1 -- 172.21.15.32:0/275309080 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f6f0401c9e0 con 0x7f6f140fbcf0 2024-05-12T04:08:15.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.699+0000 7f6f0bfff700 1 --2- 172.21.15.32:0/275309080 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f6f0006cc20 0x7f6f0006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:08:15.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.699+0000 7f6f0bfff700 1 -- 172.21.15.32:0/275309080 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f6f0408d680 con 0x7f6f140fbcf0 2024-05-12T04:08:15.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.699+0000 7f6f0a7fc700 1 --2- 172.21.15.32:0/275309080 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f6f0006cc20 0x7f6f0006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:08:15.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.699+0000 7f6f0a7fc700 1 --2- 172.21.15.32:0/275309080 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f6f0006cc20 0x7f6f0006f0d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6efc009b80 tx=0x7f6efc00b040 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:08:15.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.704+0000 7f6f0bfff700 1 -- 172.21.15.32:0/275309080 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f6f0405b150 con 0x7f6f140fbcf0 2024-05-12T04:08:15.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.993+0000 7f6f19934700 1 -- 172.21.15.32:0/275309080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f6f1404c710 con 0x7f6f140fbcf0 2024-05-12T04:08:15.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.994+0000 7f6f0bfff700 1 -- 172.21.15.32:0/275309080 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+297 (secure 0 0 0) 0x7f6f04004490 con 0x7f6f140fbcf0 2024-05-12T04:08:15.994 INFO:teuthology.orchestra.run.smithi032.stdout:HEALTH_WARN 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data 2024-05-12T04:08:15.994 INFO:teuthology.orchestra.run.smithi032.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-05-12T04:08:15.994 INFO:teuthology.orchestra.run.smithi032.stdout: fs cephfs is degraded 2024-05-12T04:08:15.994 INFO:teuthology.orchestra.run.smithi032.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-12T04:08:15.994 INFO:teuthology.orchestra.run.smithi032.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-12T04:08:15.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.996+0000 7f6f19934700 1 -- 172.21.15.32:0/275309080 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f6f0006cc20 msgr2=0x7f6f0006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:08:15.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.996+0000 7f6f19934700 1 --2- 172.21.15.32:0/275309080 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f6f0006cc20 0x7f6f0006f0d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6efc009b80 tx=0x7f6efc00b040 comp rx=0 tx=0).stop 2024-05-12T04:08:15.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.996+0000 7f6f19934700 1 -- 172.21.15.32:0/275309080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f140fbcf0 msgr2=0x7f6f140ffbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:08:15.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.996+0000 7f6f19934700 1 --2- 172.21.15.32:0/275309080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f140fbcf0 0x7f6f140ffbe0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6f040137b0 tx=0x7f6f04009e30 comp rx=0 tx=0).stop 2024-05-12T04:08:15.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.996+0000 7f6f19934700 1 -- 172.21.15.32:0/275309080 shutdown_connections 2024-05-12T04:08:15.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.996+0000 7f6f19934700 1 --2- 172.21.15.32:0/275309080 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f6f0006cc20 0x7f6f0006f0d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:15.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.996+0000 7f6f19934700 1 --2- 172.21.15.32:0/275309080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f140fbcf0 0x7f6f140ffbe0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:15.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.996+0000 7f6f19934700 1 --2- 172.21.15.32:0/275309080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f140fcfa0 0x7f6f140fe2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:15.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.996+0000 7f6f19934700 1 -- 172.21.15.32:0/275309080 >> 172.21.15.32:0/275309080 conn(0x7f6f140f7450 msgr2=0x7f6f14066880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:08:15.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.996+0000 7f6f19934700 1 -- 172.21.15.32:0/275309080 shutdown_connections 2024-05-12T04:08:15.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:15.996+0000 7f6f19934700 1 -- 172.21.15.32:0/275309080 wait complete. 2024-05-12T04:08:16.506 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:16 smithi032 ceph-mon[120392]: from='client.34416 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:08:16.507 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:16 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/3998980698' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-12T04:08:16.507 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:16 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:16.507 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:16 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:08:16.507 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:16 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/275309080' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-12T04:08:16.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:16 smithi033 ceph-mon[107305]: from='client.34416 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:08:16.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:16 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/3998980698' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-12T04:08:16.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:16 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:16.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:16 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:08:16.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:16 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/275309080' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-12T04:08:17.519 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:17 smithi032 ceph-mon[120392]: from='client.44331 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:08:17.519 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:17 smithi032 ceph-mon[120392]: pgmap v186: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 15 MiB/s rd, 5 op/s 2024-05-12T04:08:17.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:17 smithi033 ceph-mon[107305]: from='client.44331 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:08:17.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:17 smithi033 ceph-mon[107305]: pgmap v186: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 15 MiB/s rd, 5 op/s 2024-05-12T04:08:19.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:18 smithi033 ceph-mon[107305]: pgmap v187: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 23 MiB/s rd, 6 op/s 2024-05-12T04:08:19.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:19.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:19.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:08:19.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:18 smithi033 ceph-mon[107305]: mds.? [v2:172.21.15.32:6828/3603452506,v1:172.21.15.32:6829/3603452506] up:boot 2024-05-12T04:08:19.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:18 smithi033 ceph-mon[107305]: fsmap cephfs:1/1 {0=cephfs.smithi033.hglwml=up:replay} 3 up:standby 2024-05-12T04:08:19.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi032.qpsawh"}]: dispatch 2024-05-12T04:08:19.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:18 smithi032 ceph-mon[120392]: pgmap v187: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 23 MiB/s rd, 6 op/s 2024-05-12T04:08:19.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:19.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:19.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:08:19.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:18 smithi032 ceph-mon[120392]: mds.? [v2:172.21.15.32:6828/3603452506,v1:172.21.15.32:6829/3603452506] up:boot 2024-05-12T04:08:19.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:18 smithi032 ceph-mon[120392]: fsmap cephfs:1/1 {0=cephfs.smithi033.hglwml=up:replay} 3 up:standby 2024-05-12T04:08:19.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi032.qpsawh"}]: dispatch 2024-05-12T04:08:20.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:20 smithi032 ceph-mon[120392]: reconnect by client.24323 192.168.0.1:0/1200139116 after 0.000999984 2024-05-12T04:08:20.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:20 smithi032 ceph-mon[120392]: reconnect by client.14532 192.168.0.1:0/3762506211 after 0.000999984 2024-05-12T04:08:20.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:20 smithi032 ceph-mon[120392]: mds.? [v2:172.21.15.33:6826/433858238,v1:172.21.15.33:6827/433858238] up:reconnect 2024-05-12T04:08:20.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:20 smithi032 ceph-mon[120392]: fsmap cephfs:1/1 {0=cephfs.smithi033.hglwml=up:reconnect} 3 up:standby 2024-05-12T04:08:20.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:20 smithi033 ceph-mon[107305]: reconnect by client.24323 192.168.0.1:0/1200139116 after 0.000999984 2024-05-12T04:08:20.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:20 smithi033 ceph-mon[107305]: reconnect by client.14532 192.168.0.1:0/3762506211 after 0.000999984 2024-05-12T04:08:20.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:20 smithi033 ceph-mon[107305]: mds.? [v2:172.21.15.33:6826/433858238,v1:172.21.15.33:6827/433858238] up:reconnect 2024-05-12T04:08:20.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:20 smithi033 ceph-mon[107305]: fsmap cephfs:1/1 {0=cephfs.smithi033.hglwml=up:reconnect} 3 up:standby 2024-05-12T04:08:21.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:21 smithi032 ceph-mon[120392]: pgmap v188: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 23 MiB/s rd, 7 op/s 2024-05-12T04:08:21.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:21 smithi032 ceph-mon[120392]: mds.? [v2:172.21.15.33:6826/433858238,v1:172.21.15.33:6827/433858238] up:rejoin 2024-05-12T04:08:21.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:21 smithi032 ceph-mon[120392]: fsmap cephfs:1/1 {0=cephfs.smithi033.hglwml=up:rejoin} 3 up:standby 2024-05-12T04:08:21.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:21 smithi032 ceph-mon[120392]: daemon mds.cephfs.smithi033.hglwml is now active in filesystem cephfs as rank 0 2024-05-12T04:08:21.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:21 smithi033 ceph-mon[107305]: pgmap v188: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 23 MiB/s rd, 7 op/s 2024-05-12T04:08:21.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:21 smithi033 ceph-mon[107305]: mds.? [v2:172.21.15.33:6826/433858238,v1:172.21.15.33:6827/433858238] up:rejoin 2024-05-12T04:08:21.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:21 smithi033 ceph-mon[107305]: fsmap cephfs:1/1 {0=cephfs.smithi033.hglwml=up:rejoin} 3 up:standby 2024-05-12T04:08:21.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:21 smithi033 ceph-mon[107305]: daemon mds.cephfs.smithi033.hglwml is now active in filesystem cephfs as rank 0 2024-05-12T04:08:22.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:22 smithi032 ceph-mon[120392]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-12T04:08:22.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:22 smithi032 ceph-mon[120392]: mds.? [v2:172.21.15.33:6826/433858238,v1:172.21.15.33:6827/433858238] up:active 2024-05-12T04:08:22.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:22 smithi032 ceph-mon[120392]: fsmap cephfs:1 {0=cephfs.smithi033.hglwml=up:active} 3 up:standby 2024-05-12T04:08:22.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:22 smithi033 ceph-mon[107305]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-12T04:08:22.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:22 smithi033 ceph-mon[107305]: mds.? [v2:172.21.15.33:6826/433858238,v1:172.21.15.33:6827/433858238] up:active 2024-05-12T04:08:22.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:22 smithi033 ceph-mon[107305]: fsmap cephfs:1 {0=cephfs.smithi033.hglwml=up:active} 3 up:standby 2024-05-12T04:08:23.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:23 smithi032 ceph-mon[120392]: pgmap v189: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 85 B/s wr, 6 op/s 2024-05-12T04:08:23.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:23 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:23.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:23 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:23.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:23 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:23.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:23 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:23.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:23 smithi033 ceph-mon[107305]: pgmap v189: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 85 B/s wr, 6 op/s 2024-05-12T04:08:23.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:23 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:23.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:23 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:23.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:23 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:23.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:23 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:25.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:25 smithi032 ceph-mon[120392]: pgmap v190: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 511 B/s wr, 8 op/s 2024-05-12T04:08:25.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:25 smithi033 ceph-mon[107305]: pgmap v190: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 511 B/s wr, 8 op/s 2024-05-12T04:08:27.547 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:27 smithi033 ceph-mon[107305]: pgmap v191: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 8.7 MiB/s rd, 511 B/s wr, 5 op/s 2024-05-12T04:08:27.548 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:27 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:27.548 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:27 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:27.548 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:27 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:08:27.548 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:27 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T04:08:27.548 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:27 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:27.548 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:27 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:08:27.548 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:27 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:08:27.548 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:27 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:08:27.548 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:27 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:08:27.548 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:27 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:08:27.548 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:27 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi033.jgbjhk"]}]: dispatch 2024-05-12T04:08:27.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:27 smithi032 ceph-mon[120392]: pgmap v191: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 8.7 MiB/s rd, 511 B/s wr, 5 op/s 2024-05-12T04:08:27.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:27 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:27.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:27 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:27.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:27 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:08:27.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:27 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T04:08:27.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:27 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:27.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:27 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:08:27.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:27 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:08:27.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:27 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:08:27.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:27 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:08:27.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:27 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:08:27.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:27 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi033.jgbjhk"]}]: dispatch 2024-05-12T04:08:28.574 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:28 smithi033 ceph-mon[107305]: Upgrade: It appears safe to stop mds.cephfs.smithi033.jgbjhk 2024-05-12T04:08:28.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:28 smithi032 ceph-mon[120392]: Upgrade: It appears safe to stop mds.cephfs.smithi033.jgbjhk 2024-05-12T04:08:29.360 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:29 smithi033 ceph-mon[107305]: pgmap v192: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 8.4 MiB/s rd, 4.9 KiB/s wr, 8 op/s 2024-05-12T04:08:29.360 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:29 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:29.360 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:29 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi033.jgbjhk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-12T04:08:29.360 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:29 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:08:29.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:29 smithi032 ceph-mon[120392]: pgmap v192: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 8.4 MiB/s rd, 4.9 KiB/s wr, 8 op/s 2024-05-12T04:08:29.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:29 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:29.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:29 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi033.jgbjhk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-12T04:08:29.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:29 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:08:30.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:30 smithi033 ceph-mon[107305]: Upgrade: Updating mds.cephfs.smithi033.jgbjhk 2024-05-12T04:08:30.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:30 smithi033 ceph-mon[107305]: Deploying daemon mds.cephfs.smithi033.jgbjhk on smithi033 2024-05-12T04:08:30.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:30 smithi032 ceph-mon[120392]: Upgrade: Updating mds.cephfs.smithi033.jgbjhk 2024-05-12T04:08:30.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:30 smithi032 ceph-mon[120392]: Deploying daemon mds.cephfs.smithi033.jgbjhk on smithi033 2024-05-12T04:08:31.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:31 smithi033 ceph-mon[107305]: pgmap v193: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 2.4 MiB/s rd, 4.9 KiB/s wr, 6 op/s 2024-05-12T04:08:31.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:31 smithi033 ceph-mon[107305]: osdmap e83: 6 total, 6 up, 6 in 2024-05-12T04:08:31.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:31 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:31.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:31 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:08:31.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:31 smithi032 ceph-mon[120392]: pgmap v193: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 2.4 MiB/s rd, 4.9 KiB/s wr, 6 op/s 2024-05-12T04:08:31.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:31 smithi032 ceph-mon[120392]: osdmap e83: 6 total, 6 up, 6 in 2024-05-12T04:08:31.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:31 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:31.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:31 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:08:32.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:32 smithi033 ceph-mon[107305]: fsmap cephfs:1 {0=cephfs.smithi033.hglwml=up:active} 2 up:standby 2024-05-12T04:08:32.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:32 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi033.jgbjhk"}]: dispatch 2024-05-12T04:08:32.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:32 smithi032 ceph-mon[120392]: fsmap cephfs:1 {0=cephfs.smithi033.hglwml=up:active} 2 up:standby 2024-05-12T04:08:32.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:32 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi033.jgbjhk"}]: dispatch 2024-05-12T04:08:33.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:33 smithi033 ceph-mon[107305]: pgmap v195: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 4.1 KiB/s rd, 5.8 KiB/s wr, 7 op/s 2024-05-12T04:08:33.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:33 smithi032 ceph-mon[120392]: pgmap v195: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 4.1 KiB/s rd, 5.8 KiB/s wr, 7 op/s 2024-05-12T04:08:35.648 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:35 smithi033 ceph-mon[107305]: pgmap v196: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 5.3 KiB/s wr, 3 op/s 2024-05-12T04:08:35.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:35 smithi032 ceph-mon[120392]: pgmap v196: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 5.3 KiB/s wr, 3 op/s 2024-05-12T04:08:37.240 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:36 smithi033 ceph-mon[107305]: pgmap v197: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 921 B/s rd, 5.3 KiB/s wr, 3 op/s 2024-05-12T04:08:37.241 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:36 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:37.241 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:36 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:37.241 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:36 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:08:37.241 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:36 smithi033 ceph-mon[107305]: mds.? [v2:172.21.15.33:6824/997254871,v1:172.21.15.33:6825/997254871] up:boot 2024-05-12T04:08:37.241 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:36 smithi033 ceph-mon[107305]: fsmap cephfs:1 {0=cephfs.smithi033.hglwml=up:active} 3 up:standby 2024-05-12T04:08:37.241 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:36 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi033.jgbjhk"}]: dispatch 2024-05-12T04:08:37.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:36 smithi032 ceph-mon[120392]: pgmap v197: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 921 B/s rd, 5.3 KiB/s wr, 3 op/s 2024-05-12T04:08:37.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:36 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:37.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:36 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:37.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:36 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:08:37.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:36 smithi032 ceph-mon[120392]: mds.? [v2:172.21.15.33:6824/997254871,v1:172.21.15.33:6825/997254871] up:boot 2024-05-12T04:08:37.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:36 smithi032 ceph-mon[120392]: fsmap cephfs:1 {0=cephfs.smithi033.hglwml=up:active} 3 up:standby 2024-05-12T04:08:37.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:36 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi033.jgbjhk"}]: dispatch 2024-05-12T04:08:39.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:38 smithi032 ceph-mon[120392]: pgmap v198: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 0 B/s wr, 0 op/s 2024-05-12T04:08:39.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:38 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:39.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:38 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:39.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:38 smithi033 ceph-mon[107305]: pgmap v198: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 0 B/s wr, 0 op/s 2024-05-12T04:08:39.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:38 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:39.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:38 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:40.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:40 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:40.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:40 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:40.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:40 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:40.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:40 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:41.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:41 smithi033 ceph-mon[107305]: pgmap v199: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail 2024-05-12T04:08:41.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:41 smithi032 ceph-mon[120392]: pgmap v199: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail 2024-05-12T04:08:43.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:43 smithi033 ceph-mon[107305]: pgmap v200: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail 2024-05-12T04:08:43.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:43 smithi032 ceph-mon[120392]: pgmap v200: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail 2024-05-12T04:08:45.618 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:45 smithi033 ceph-mon[107305]: pgmap v201: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail 2024-05-12T04:08:45.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:45 smithi032 ceph-mon[120392]: pgmap v201: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail 2024-05-12T04:08:46.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.116+0000 7faf6e745700 1 -- 172.21.15.32:0/2790821370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf680fbcb0 msgr2=0x7faf680fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:08:46.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.116+0000 7faf6e745700 1 --2- 172.21.15.32:0/2790821370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf680fbcb0 0x7faf680fc0c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7faf50004940 tx=0x7faf50009e30 comp rx=0 tx=0).stop 2024-05-12T04:08:46.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.116+0000 7faf6e745700 1 -- 172.21.15.32:0/2790821370 shutdown_connections 2024-05-12T04:08:46.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.116+0000 7faf6e745700 1 --2- 172.21.15.32:0/2790821370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf680fcf60 0x7faf680fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:46.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.116+0000 7faf6e745700 1 --2- 172.21.15.32:0/2790821370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf680fbcb0 0x7faf680fc0c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:46.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.116+0000 7faf6e745700 1 -- 172.21.15.32:0/2790821370 >> 172.21.15.32:0/2790821370 conn(0x7faf680f7430 msgr2=0x7faf680f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:08:46.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.116+0000 7faf6e745700 1 -- 172.21.15.32:0/2790821370 shutdown_connections 2024-05-12T04:08:46.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.116+0000 7faf6e745700 1 -- 172.21.15.32:0/2790821370 wait complete. 2024-05-12T04:08:46.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.117+0000 7faf6e745700 1 Processor -- start 2024-05-12T04:08:46.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.117+0000 7faf6e745700 1 -- start start 2024-05-12T04:08:46.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.117+0000 7faf6e745700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf680fbcb0 0x7faf68110640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:08:46.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.117+0000 7faf6e745700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf680fcf60 0x7faf68112b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:08:46.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.118+0000 7faf6e745700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf68066330 con 0x7faf680fbcb0 2024-05-12T04:08:46.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.118+0000 7faf6e745700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf68066470 con 0x7faf680fcf60 2024-05-12T04:08:46.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.118+0000 7faf67fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf680fbcb0 0x7faf68110640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:08:46.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.118+0000 7faf67fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf680fbcb0 0x7faf68110640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58346/0 (socket says 172.21.15.32:58346) 2024-05-12T04:08:46.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.118+0000 7faf67fff700 1 -- 172.21.15.32:0/2131525037 learned_addr learned my addr 172.21.15.32:0/2131525037 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:08:46.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.118+0000 7faf677fe700 1 --2- 172.21.15.32:0/2131525037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf680fcf60 0x7faf68112b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:08:46.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.118+0000 7faf67fff700 1 -- 172.21.15.32:0/2131525037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf680fcf60 msgr2=0x7faf68112b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:08:46.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.118+0000 7faf67fff700 1 --2- 172.21.15.32:0/2131525037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf680fcf60 0x7faf68112b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:46.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.118+0000 7faf67fff700 1 -- 172.21.15.32:0/2131525037 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf50005040 con 0x7faf680fbcb0 2024-05-12T04:08:46.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.118+0000 7faf67fff700 1 --2- 172.21.15.32:0/2131525037 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf680fbcb0 0x7faf68110640 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7faf5000b3b0 tx=0x7faf50009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:08:46.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.118+0000 7faf64ff9700 1 -- 172.21.15.32:0/2131525037 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf5000e040 con 0x7faf680fbcb0 2024-05-12T04:08:46.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.118+0000 7faf64ff9700 1 -- 172.21.15.32:0/2131525037 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7faf5000bd20 con 0x7faf680fbcb0 2024-05-12T04:08:46.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.118+0000 7faf6e745700 1 -- 172.21.15.32:0/2131525037 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf681130d0 con 0x7faf680fbcb0 2024-05-12T04:08:46.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.118+0000 7faf64ff9700 1 -- 172.21.15.32:0/2131525037 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf5001c630 con 0x7faf680fbcb0 2024-05-12T04:08:46.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.118+0000 7faf6e745700 1 -- 172.21.15.32:0/2131525037 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf68113690 con 0x7faf680fbcb0 2024-05-12T04:08:46.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.119+0000 7faf6e745700 1 -- 172.21.15.32:0/2131525037 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf6805f5c0 con 0x7faf680fbcb0 2024-05-12T04:08:46.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.120+0000 7faf64ff9700 1 -- 172.21.15.32:0/2131525037 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7faf50013420 con 0x7faf680fbcb0 2024-05-12T04:08:46.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.120+0000 7faf64ff9700 1 --2- 172.21.15.32:0/2131525037 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7faf5406cc20 0x7faf5406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:08:46.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.121+0000 7faf64ff9700 1 -- 172.21.15.32:0/2131525037 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7faf5008ccc0 con 0x7faf680fbcb0 2024-05-12T04:08:46.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.121+0000 7faf677fe700 1 --2- 172.21.15.32:0/2131525037 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7faf5406cc20 0x7faf5406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:08:46.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.121+0000 7faf677fe700 1 --2- 172.21.15.32:0/2131525037 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7faf5406cc20 0x7faf5406f0d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7faf58007fd0 tx=0x7faf58009040 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:08:46.125 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.125+0000 7faf64ff9700 1 -- 172.21.15.32:0/2131525037 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7faf50092020 con 0x7faf680fbcb0 2024-05-12T04:08:46.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.350+0000 7faf6e745700 1 -- 172.21.15.32:0/2131525037 --> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7faf681013d0 con 0x7faf5406cc20 2024-05-12T04:08:46.351 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.351+0000 7faf64ff9700 1 -- 172.21.15.32:0/2131525037 <== mgr.24593 v2:172.21.15.32:6800/2957111591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7faf681013d0 con 0x7faf5406cc20 2024-05-12T04:08:46.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.354+0000 7faf6e745700 1 -- 172.21.15.32:0/2131525037 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7faf5406cc20 msgr2=0x7faf5406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:08:46.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.354+0000 7faf6e745700 1 --2- 172.21.15.32:0/2131525037 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7faf5406cc20 0x7faf5406f0d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7faf58007fd0 tx=0x7faf58009040 comp rx=0 tx=0).stop 2024-05-12T04:08:46.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.354+0000 7faf6e745700 1 -- 172.21.15.32:0/2131525037 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf680fbcb0 msgr2=0x7faf68110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:08:46.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.354+0000 7faf6e745700 1 --2- 172.21.15.32:0/2131525037 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf680fbcb0 0x7faf68110640 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7faf5000b3b0 tx=0x7faf50009e30 comp rx=0 tx=0).stop 2024-05-12T04:08:46.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.354+0000 7faf6e745700 1 -- 172.21.15.32:0/2131525037 shutdown_connections 2024-05-12T04:08:46.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.354+0000 7faf6e745700 1 --2- 172.21.15.32:0/2131525037 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7faf5406cc20 0x7faf5406f0d0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:46.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.354+0000 7faf6e745700 1 --2- 172.21.15.32:0/2131525037 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf680fbcb0 0x7faf68110640 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:46.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.354+0000 7faf6e745700 1 --2- 172.21.15.32:0/2131525037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf680fcf60 0x7faf68112b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:46.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.354+0000 7faf6e745700 1 -- 172.21.15.32:0/2131525037 >> 172.21.15.32:0/2131525037 conn(0x7faf680f7430 msgr2=0x7faf680f8c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:08:46.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.354+0000 7faf6e745700 1 -- 172.21.15.32:0/2131525037 shutdown_connections 2024-05-12T04:08:46.355 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.354+0000 7faf6e745700 1 -- 172.21.15.32:0/2131525037 wait complete. 2024-05-12T04:08:46.365 INFO:teuthology.orchestra.run.smithi032.stdout:true 2024-05-12T04:08:46.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.460+0000 7fabd2518700 1 -- 172.21.15.32:0/3030381239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabcc0fbcc0 msgr2=0x7fabcc0fc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:08:46.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.460+0000 7fabd2518700 1 --2- 172.21.15.32:0/3030381239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabcc0fbcc0 0x7fabcc0fc0d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fabbc009ce0 tx=0x7fabbc004d10 comp rx=0 tx=0).stop 2024-05-12T04:08:46.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.460+0000 7fabd2518700 1 -- 172.21.15.32:0/3030381239 shutdown_connections 2024-05-12T04:08:46.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.460+0000 7fabd2518700 1 --2- 172.21.15.32:0/3030381239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabcc0fcf70 0x7fabcc0fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:46.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.460+0000 7fabd2518700 1 --2- 172.21.15.32:0/3030381239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabcc0fbcc0 0x7fabcc0fc0d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:46.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.460+0000 7fabd2518700 1 -- 172.21.15.32:0/3030381239 >> 172.21.15.32:0/3030381239 conn(0x7fabcc0f7420 msgr2=0x7fabcc0f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:08:46.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.460+0000 7fabd2518700 1 -- 172.21.15.32:0/3030381239 shutdown_connections 2024-05-12T04:08:46.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.460+0000 7fabd2518700 1 -- 172.21.15.32:0/3030381239 wait complete. 2024-05-12T04:08:46.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.461+0000 7fabd2518700 1 Processor -- start 2024-05-12T04:08:46.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.461+0000 7fabd2518700 1 -- start start 2024-05-12T04:08:46.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.461+0000 7fabd2518700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabcc0fbcc0 0x7fabcc10ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:08:46.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.461+0000 7fabd2518700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabcc0fcf70 0x7fabcc10cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:08:46.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.461+0000 7fabd2518700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabcc10d560 con 0x7fabcc0fbcc0 2024-05-12T04:08:46.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.461+0000 7fabd2518700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabcc10ddb0 con 0x7fabcc0fcf70 2024-05-12T04:08:46.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.461+0000 7fabcbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabcc0fbcc0 0x7fabcc10ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:08:46.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.461+0000 7fabcbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabcc0fbcc0 0x7fabcc10ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58352/0 (socket says 172.21.15.32:58352) 2024-05-12T04:08:46.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.461+0000 7fabcbfff700 1 -- 172.21.15.32:0/1959204676 learned_addr learned my addr 172.21.15.32:0/1959204676 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:08:46.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.461+0000 7fabcb7fe700 1 --2- 172.21.15.32:0/1959204676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabcc0fcf70 0x7fabcc10cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:08:46.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.462+0000 7fabcbfff700 1 -- 172.21.15.32:0/1959204676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabcc0fcf70 msgr2=0x7fabcc10cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:08:46.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.462+0000 7fabcbfff700 1 --2- 172.21.15.32:0/1959204676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabcc0fcf70 0x7fabcc10cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:46.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.462+0000 7fabcbfff700 1 -- 172.21.15.32:0/1959204676 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabbc009940 con 0x7fabcc0fbcc0 2024-05-12T04:08:46.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.462+0000 7fabcb7fe700 1 --2- 172.21.15.32:0/1959204676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabcc0fcf70 0x7fabcc10cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-12T04:08:46.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.462+0000 7fabcbfff700 1 --2- 172.21.15.32:0/1959204676 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabcc0fbcc0 0x7fabcc10ca20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fabbc005010 tx=0x7fabbc004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:08:46.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.462+0000 7fabc8ff9700 1 -- 172.21.15.32:0/1959204676 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fabbc00e040 con 0x7fabcc0fbcc0 2024-05-12T04:08:46.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.462+0000 7fabd2518700 1 -- 172.21.15.32:0/1959204676 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabcc10e030 con 0x7fabcc0fbcc0 2024-05-12T04:08:46.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.462+0000 7fabc8ff9700 1 -- 172.21.15.32:0/1959204676 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fabbc008ea0 con 0x7fabcc0fbcc0 2024-05-12T04:08:46.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.462+0000 7fabc8ff9700 1 -- 172.21.15.32:0/1959204676 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fabbc01c760 con 0x7fabcc0fbcc0 2024-05-12T04:08:46.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.462+0000 7fabd2518700 1 -- 172.21.15.32:0/1959204676 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabcc10e570 con 0x7fabcc0fbcc0 2024-05-12T04:08:46.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.463+0000 7fabd2518700 1 -- 172.21.15.32:0/1959204676 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabcc05f5c0 con 0x7fabcc0fbcc0 2024-05-12T04:08:46.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.463+0000 7fabc8ff9700 1 -- 172.21.15.32:0/1959204676 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fabbc012bc0 con 0x7fabcc0fbcc0 2024-05-12T04:08:46.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.464+0000 7fabc8ff9700 1 --2- 172.21.15.32:0/1959204676 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fabb806cc30 0x7fabb806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:08:46.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.464+0000 7fabc8ff9700 1 -- 172.21.15.32:0/1959204676 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fabbc08dc70 con 0x7fabcc0fbcc0 2024-05-12T04:08:46.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.464+0000 7fabcb7fe700 1 --2- 172.21.15.32:0/1959204676 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fabb806cc30 0x7fabb806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:08:46.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.464+0000 7fabcb7fe700 1 --2- 172.21.15.32:0/1959204676 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fabb806cc30 0x7fabb806f0e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fabb4009fd0 tx=0x7fabb4009ee0 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:08:46.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.468+0000 7fabc8ff9700 1 -- 172.21.15.32:0/1959204676 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fabbc05b6c0 con 0x7fabcc0fbcc0 2024-05-12T04:08:46.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:46 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:08:46.691 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.691+0000 7fabd2518700 1 -- 172.21.15.32:0/1959204676 --> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fabcc10e8f0 con 0x7fabb806cc30 2024-05-12T04:08:46.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.692+0000 7fabc8ff9700 1 -- 172.21.15.32:0/1959204676 <== mgr.24593 v2:172.21.15.32:6800/2957111591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fabcc10e8f0 con 0x7fabb806cc30 2024-05-12T04:08:46.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.694+0000 7fabd2518700 1 -- 172.21.15.32:0/1959204676 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fabb806cc30 msgr2=0x7fabb806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:08:46.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.694+0000 7fabd2518700 1 --2- 172.21.15.32:0/1959204676 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fabb806cc30 0x7fabb806f0e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fabb4009fd0 tx=0x7fabb4009ee0 comp rx=0 tx=0).stop 2024-05-12T04:08:46.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.694+0000 7fabd2518700 1 -- 172.21.15.32:0/1959204676 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabcc0fbcc0 msgr2=0x7fabcc10ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:08:46.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.694+0000 7fabd2518700 1 --2- 172.21.15.32:0/1959204676 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabcc0fbcc0 0x7fabcc10ca20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fabbc005010 tx=0x7fabbc004d10 comp rx=0 tx=0).stop 2024-05-12T04:08:46.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.694+0000 7fabd2518700 1 -- 172.21.15.32:0/1959204676 shutdown_connections 2024-05-12T04:08:46.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.694+0000 7fabd2518700 1 --2- 172.21.15.32:0/1959204676 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fabb806cc30 0x7fabb806f0e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:46.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.694+0000 7fabd2518700 1 --2- 172.21.15.32:0/1959204676 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabcc0fbcc0 0x7fabcc10ca20 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:46.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.694+0000 7fabd2518700 1 --2- 172.21.15.32:0/1959204676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabcc0fcf70 0x7fabcc10cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:46.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.694+0000 7fabd2518700 1 -- 172.21.15.32:0/1959204676 >> 172.21.15.32:0/1959204676 conn(0x7fabcc0f7420 msgr2=0x7fabcc0f8ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:08:46.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.694+0000 7fabd2518700 1 -- 172.21.15.32:0/1959204676 shutdown_connections 2024-05-12T04:08:46.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.694+0000 7fabd2518700 1 -- 172.21.15.32:0/1959204676 wait complete. 2024-05-12T04:08:46.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:46 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:08:46.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.798+0000 7f050ab1d700 1 -- 172.21.15.32:0/1098882319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0504066240 msgr2=0x7f05040ffa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:08:46.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.798+0000 7f050ab1d700 1 --2- 172.21.15.32:0/1098882319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0504066240 0x7f05040ffa20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f04f8004940 tx=0x7f04f8009e30 comp rx=0 tx=0).stop 2024-05-12T04:08:46.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.798+0000 7f050ab1d700 1 -- 172.21.15.32:0/1098882319 shutdown_connections 2024-05-12T04:08:46.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.798+0000 7f050ab1d700 1 --2- 172.21.15.32:0/1098882319 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05040fff60 0x7f0504102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:46.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.798+0000 7f050ab1d700 1 --2- 172.21.15.32:0/1098882319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0504066240 0x7f05040ffa20 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:46.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.798+0000 7f050ab1d700 1 -- 172.21.15.32:0/1098882319 >> 172.21.15.32:0/1098882319 conn(0x7f05040f7430 msgr2=0x7f05040f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:08:46.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.798+0000 7f050ab1d700 1 -- 172.21.15.32:0/1098882319 shutdown_connections 2024-05-12T04:08:46.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.798+0000 7f050ab1d700 1 -- 172.21.15.32:0/1098882319 wait complete. 2024-05-12T04:08:46.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.799+0000 7f050ab1d700 1 Processor -- start 2024-05-12T04:08:46.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.799+0000 7f050ab1d700 1 -- start start 2024-05-12T04:08:46.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.799+0000 7f050ab1d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0504066240 0x7f050410d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:08:46.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.799+0000 7f050ab1d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05040fff60 0x7f050410df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:08:46.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.799+0000 7f050ab1d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f050410e510 con 0x7f0504066240 2024-05-12T04:08:46.801 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.799+0000 7f050ab1d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f050410ed10 con 0x7f05040fff60 2024-05-12T04:08:46.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.799+0000 7f05088b9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0504066240 0x7f050410d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:08:46.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.799+0000 7f05088b9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0504066240 0x7f050410d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58370/0 (socket says 172.21.15.32:58370) 2024-05-12T04:08:46.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.799+0000 7f05088b9700 1 -- 172.21.15.32:0/362470227 learned_addr learned my addr 172.21.15.32:0/362470227 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:08:46.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.799+0000 7f0503fff700 1 --2- 172.21.15.32:0/362470227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05040fff60 0x7f050410df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:08:46.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.800+0000 7f05088b9700 1 -- 172.21.15.32:0/362470227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05040fff60 msgr2=0x7f050410df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:08:46.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.800+0000 7f05088b9700 1 --2- 172.21.15.32:0/362470227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05040fff60 0x7f050410df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:46.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.800+0000 7f05088b9700 1 -- 172.21.15.32:0/362470227 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04f8005040 con 0x7f0504066240 2024-05-12T04:08:46.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.800+0000 7f05088b9700 1 --2- 172.21.15.32:0/362470227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0504066240 0x7f050410d9d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f04f800b3b0 tx=0x7f04f8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:08:46.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.800+0000 7f05017fa700 1 -- 172.21.15.32:0/362470227 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04f800e040 con 0x7f0504066240 2024-05-12T04:08:46.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.800+0000 7f05017fa700 1 -- 172.21.15.32:0/362470227 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f04f800bc50 con 0x7f0504066240 2024-05-12T04:08:46.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.800+0000 7f05017fa700 1 -- 172.21.15.32:0/362470227 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04f801c3f0 con 0x7f0504066240 2024-05-12T04:08:46.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.800+0000 7f050ab1d700 1 -- 172.21.15.32:0/362470227 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f050410ef90 con 0x7f0504066240 2024-05-12T04:08:46.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.800+0000 7f050ab1d700 1 -- 172.21.15.32:0/362470227 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f050410f4d0 con 0x7f0504066240 2024-05-12T04:08:46.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.801+0000 7f050ab1d700 1 -- 172.21.15.32:0/362470227 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f050405f5c0 con 0x7f0504066240 2024-05-12T04:08:46.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.802+0000 7f05017fa700 1 -- 172.21.15.32:0/362470227 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f04f8013420 con 0x7f0504066240 2024-05-12T04:08:46.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.802+0000 7f05017fa700 1 --2- 172.21.15.32:0/362470227 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f04f406cc20 0x7f04f406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:08:46.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.802+0000 7f05017fa700 1 -- 172.21.15.32:0/362470227 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f04f808cb40 con 0x7f0504066240 2024-05-12T04:08:46.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.802+0000 7f0503fff700 1 --2- 172.21.15.32:0/362470227 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f04f406cc20 0x7f04f406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:08:46.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.802+0000 7f0503fff700 1 --2- 172.21.15.32:0/362470227 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f04f406cc20 0x7f04f406f0d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f04f0004f40 tx=0x7f04f000b040 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:08:46.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:46.807+0000 7f05017fa700 1 -- 172.21.15.32:0/362470227 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f04f805a6d0 con 0x7f0504066240 2024-05-12T04:08:47.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.032+0000 7f050ab1d700 1 -- 172.21.15.32:0/362470227 --> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f050410f8a0 con 0x7f04f406cc20 2024-05-12T04:08:47.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.043+0000 7f05017fa700 1 -- 172.21.15.32:0/362470227 <== mgr.24593 v2:172.21.15.32:6800/2957111591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f050410f8a0 con 0x7f04f406cc20 2024-05-12T04:08:47.043 INFO:teuthology.orchestra.run.smithi032.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-12T04:08:47.043 INFO:teuthology.orchestra.run.smithi032.stdout:alertmanager.smithi032 smithi032 *:9093,9094 running (7m) 24s ago 16m 25.5M - 0.25.0 c8568f914cd2 a37fbab7af4c 2024-05-12T04:08:47.043 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi032 smithi032 running (4m) 24s ago 17m 6660k - 18.2.2-1272-g58249552 ddcb0a9d6fde 8c53f1fc7b06 2024-05-12T04:08:47.043 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi033 smithi033 running (4m) 8s ago 15m 6651k - 18.2.2-1272-g58249552 ddcb0a9d6fde cf4fe9ca070b 2024-05-12T04:08:47.043 INFO:teuthology.orchestra.run.smithi032.stdout:grafana.smithi032 smithi032 *:3000 running (7m) 24s ago 16m 90.5M - 9.4.7 954c08fa6188 83055e58d23c 2024-05-12T04:08:47.043 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.ialtqb smithi032 running (51s) 24s ago 10m 13.8M - 18.2.2-1272-g58249552 ddcb0a9d6fde 532dd9845ce7 2024-05-12T04:08:47.044 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.qpsawh smithi032 running (29s) 24s ago 10m 13.6M - 18.2.2-1272-g58249552 ddcb0a9d6fde 68e735fd79ee 2024-05-12T04:08:47.044 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi033.hglwml smithi033 running (10m) 8s ago 10m 87.1M - 17.2.7-939-gc190bdee 284803f7d9cf c97902b05025 2024-05-12T04:08:47.044 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi033.jgbjhk smithi033 running (11s) 8s ago 10m 12.6M - 18.2.2-1272-g58249552 ddcb0a9d6fde b738dbbbec0f 2024-05-12T04:08:47.044 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi032.ukatkq smithi032 *:8443,9283,8765 running (6m) 24s ago 18m 506M - 18.2.2-1272-g58249552 ddcb0a9d6fde 0d88ab9a5292 2024-05-12T04:08:47.044 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi033.pcvxwy smithi033 *:8443,9283,8765 running (6m) 8s ago 15m 450M - 18.2.2-1272-g58249552 ddcb0a9d6fde d42dcb6bd307 2024-05-12T04:08:47.044 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi032 smithi032 running (5m) 24s ago 18m 48.6M 2048M 18.2.2-1272-g58249552 ddcb0a9d6fde 81f94171c5dc 2024-05-12T04:08:47.044 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi033 smithi033 running (5m) 8s ago 15m 39.1M 2048M 18.2.2-1272-g58249552 ddcb0a9d6fde d648ba108f3e 2024-05-12T04:08:47.044 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi032 smithi032 *:9100 running (8m) 24s ago 16m 16.6M - 1.5.0 0da6a335fe13 cb75a7c0c829 2024-05-12T04:08:47.044 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi033 smithi033 *:9100 running (6m) 8s ago 15m 18.5M - 1.5.0 0da6a335fe13 7888c36c659f 2024-05-12T04:08:47.044 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0 smithi032 running (4m) 24s ago 14m 90.3M 1570M 18.2.2-1272-g58249552 ddcb0a9d6fde 07f66c4deb3b 2024-05-12T04:08:47.044 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1 smithi032 running (3m) 24s ago 14m 96.5M 1570M 18.2.2-1272-g58249552 ddcb0a9d6fde 67a87807d654 2024-05-12T04:08:47.044 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2 smithi032 running (2m) 24s ago 13m 85.2M 1570M 18.2.2-1272-g58249552 ddcb0a9d6fde 4ac48d74ee8b 2024-05-12T04:08:47.044 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3 smithi033 running (2m) 8s ago 13m 100M 2594M 18.2.2-1272-g58249552 ddcb0a9d6fde 11f097a455bb 2024-05-12T04:08:47.044 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4 smithi033 running (102s) 8s ago 12m 88.9M 2594M 18.2.2-1272-g58249552 ddcb0a9d6fde db6fda7d52de 2024-05-12T04:08:47.044 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5 smithi033 running (70s) 8s ago 12m 91.5M 2594M 18.2.2-1272-g58249552 ddcb0a9d6fde 3333020b8a15 2024-05-12T04:08:47.045 INFO:teuthology.orchestra.run.smithi032.stdout:prometheus.smithi032 smithi032 *:9095 running (6m) 24s ago 16m 61.3M - 2.43.0 a07b618ecd1d 46eafb25e176 2024-05-12T04:08:47.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.045+0000 7f050ab1d700 1 -- 172.21.15.32:0/362470227 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f04f406cc20 msgr2=0x7f04f406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:08:47.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.045+0000 7f050ab1d700 1 --2- 172.21.15.32:0/362470227 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f04f406cc20 0x7f04f406f0d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f04f0004f40 tx=0x7f04f000b040 comp rx=0 tx=0).stop 2024-05-12T04:08:47.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.045+0000 7f050ab1d700 1 -- 172.21.15.32:0/362470227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0504066240 msgr2=0x7f050410d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:08:47.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.045+0000 7f050ab1d700 1 --2- 172.21.15.32:0/362470227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0504066240 0x7f050410d9d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f04f800b3b0 tx=0x7f04f8009e30 comp rx=0 tx=0).stop 2024-05-12T04:08:47.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.045+0000 7f050ab1d700 1 -- 172.21.15.32:0/362470227 shutdown_connections 2024-05-12T04:08:47.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.045+0000 7f050ab1d700 1 --2- 172.21.15.32:0/362470227 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f04f406cc20 0x7f04f406f0d0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:47.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.045+0000 7f050ab1d700 1 --2- 172.21.15.32:0/362470227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0504066240 0x7f050410d9d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:47.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.045+0000 7f050ab1d700 1 --2- 172.21.15.32:0/362470227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05040fff60 0x7f050410df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:47.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.045+0000 7f050ab1d700 1 -- 172.21.15.32:0/362470227 >> 172.21.15.32:0/362470227 conn(0x7f05040f7430 msgr2=0x7f05040f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:08:47.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.045+0000 7f050ab1d700 1 -- 172.21.15.32:0/362470227 shutdown_connections 2024-05-12T04:08:47.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.045+0000 7f050ab1d700 1 -- 172.21.15.32:0/362470227 wait complete. 2024-05-12T04:08:47.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.151+0000 7fcb67c39700 1 -- 172.21.15.32:0/3180238217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb600fdfd0 msgr2=0x7fcb600fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:08:47.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.151+0000 7fcb67c39700 1 --2- 172.21.15.32:0/3180238217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb600fdfd0 0x7fcb600fe440 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcb50004940 tx=0x7fcb50009e30 comp rx=0 tx=0).stop 2024-05-12T04:08:47.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.151+0000 7fcb67c39700 1 -- 172.21.15.32:0/3180238217 shutdown_connections 2024-05-12T04:08:47.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.151+0000 7fcb67c39700 1 --2- 172.21.15.32:0/3180238217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb600fdfd0 0x7fcb600fe440 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:47.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.151+0000 7fcb67c39700 1 --2- 172.21.15.32:0/3180238217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb600fce60 0x7fcb600fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:47.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.151+0000 7fcb67c39700 1 -- 172.21.15.32:0/3180238217 >> 172.21.15.32:0/3180238217 conn(0x7fcb600f8540 msgr2=0x7fcb600fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:08:47.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.152+0000 7fcb67c39700 1 -- 172.21.15.32:0/3180238217 shutdown_connections 2024-05-12T04:08:47.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.152+0000 7fcb67c39700 1 -- 172.21.15.32:0/3180238217 wait complete. 2024-05-12T04:08:47.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.152+0000 7fcb67c39700 1 Processor -- start 2024-05-12T04:08:47.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.152+0000 7fcb67c39700 1 -- start start 2024-05-12T04:08:47.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.152+0000 7fcb67c39700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb600fce60 0x7fcb6010fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:08:47.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.152+0000 7fcb67c39700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb600fdfd0 0x7fcb6010ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:08:47.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.152+0000 7fcb67c39700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb601105c0 con 0x7fcb600fce60 2024-05-12T04:08:47.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.152+0000 7fcb67c39700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb60110e10 con 0x7fcb600fdfd0 2024-05-12T04:08:47.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.152+0000 7fcb659d5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb600fce60 0x7fcb6010fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:08:47.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.153+0000 7fcb659d5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb600fce60 0x7fcb6010fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58400/0 (socket says 172.21.15.32:58400) 2024-05-12T04:08:47.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.153+0000 7fcb659d5700 1 -- 172.21.15.32:0/2366596471 learned_addr learned my addr 172.21.15.32:0/2366596471 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:08:47.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.153+0000 7fcb651d4700 1 --2- 172.21.15.32:0/2366596471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb600fdfd0 0x7fcb6010ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:08:47.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.153+0000 7fcb659d5700 1 -- 172.21.15.32:0/2366596471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb600fdfd0 msgr2=0x7fcb6010ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:08:47.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.153+0000 7fcb659d5700 1 --2- 172.21.15.32:0/2366596471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb600fdfd0 0x7fcb6010ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:47.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.153+0000 7fcb659d5700 1 -- 172.21.15.32:0/2366596471 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb50005040 con 0x7fcb600fce60 2024-05-12T04:08:47.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.153+0000 7fcb659d5700 1 --2- 172.21.15.32:0/2366596471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb600fce60 0x7fcb6010fa80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c00eb40 tx=0x7fcb5c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:08:47.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.153+0000 7fcb567fc700 1 -- 172.21.15.32:0/2366596471 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb5c007cc0 con 0x7fcb600fce60 2024-05-12T04:08:47.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.153+0000 7fcb567fc700 1 -- 172.21.15.32:0/2366596471 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcb5c00f040 con 0x7fcb600fce60 2024-05-12T04:08:47.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.153+0000 7fcb67c39700 1 -- 172.21.15.32:0/2366596471 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb60111090 con 0x7fcb600fce60 2024-05-12T04:08:47.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.153+0000 7fcb567fc700 1 -- 172.21.15.32:0/2366596471 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb5c00c990 con 0x7fcb600fce60 2024-05-12T04:08:47.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.153+0000 7fcb67c39700 1 -- 172.21.15.32:0/2366596471 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb601115d0 con 0x7fcb600fce60 2024-05-12T04:08:47.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.154+0000 7fcb67c39700 1 -- 172.21.15.32:0/2366596471 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb60101960 con 0x7fcb600fce60 2024-05-12T04:08:47.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.155+0000 7fcb567fc700 1 -- 172.21.15.32:0/2366596471 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fcb5c00caf0 con 0x7fcb600fce60 2024-05-12T04:08:47.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.155+0000 7fcb567fc700 1 --2- 172.21.15.32:0/2366596471 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fcb4c06cc20 0x7fcb4c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:08:47.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.155+0000 7fcb567fc700 1 -- 172.21.15.32:0/2366596471 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fcb5c08bb90 con 0x7fcb600fce60 2024-05-12T04:08:47.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.155+0000 7fcb651d4700 1 --2- 172.21.15.32:0/2366596471 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fcb4c06cc20 0x7fcb4c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:08:47.156 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.156+0000 7fcb651d4700 1 --2- 172.21.15.32:0/2366596471 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fcb4c06cc20 0x7fcb4c06f0d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fcb50004890 tx=0x7fcb50009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:08:47.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.160+0000 7fcb567fc700 1 -- 172.21.15.32:0/2366596471 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fcb5c059710 con 0x7fcb600fce60 2024-05-12T04:08:47.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.448+0000 7fcb67c39700 1 -- 172.21.15.32:0/2366596471 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fcb601058a0 con 0x7fcb600fce60 2024-05-12T04:08:47.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.448+0000 7fcb567fc700 1 -- 172.21.15.32:0/2366596471 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+847 (secure 0 0 0) 0x7fcb5c0592a0 con 0x7fcb600fce60 2024-05-12T04:08:47.449 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-05-12T04:08:47.449 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": { 2024-05-12T04:08:47.449 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 2 2024-05-12T04:08:47.449 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:08:47.449 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": { 2024-05-12T04:08:47.449 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 2 2024-05-12T04:08:47.449 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:08:47.449 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": { 2024-05-12T04:08:47.449 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 6 2024-05-12T04:08:47.449 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:08:47.449 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": { 2024-05-12T04:08:47.449 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 1, 2024-05-12T04:08:47.450 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 3 2024-05-12T04:08:47.450 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:08:47.450 INFO:teuthology.orchestra.run.smithi032.stdout: "overall": { 2024-05-12T04:08:47.450 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 17.2.7-939-gc190bdee (c190bdeec0cb2c5a9760aa085c602e6994249982) quincy (stable)": 1, 2024-05-12T04:08:47.450 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 13 2024-05-12T04:08:47.450 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-05-12T04:08:47.450 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-05-12T04:08:47.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.450+0000 7fcb67c39700 1 -- 172.21.15.32:0/2366596471 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fcb4c06cc20 msgr2=0x7fcb4c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:08:47.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.450+0000 7fcb67c39700 1 --2- 172.21.15.32:0/2366596471 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fcb4c06cc20 0x7fcb4c06f0d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fcb50004890 tx=0x7fcb50009e30 comp rx=0 tx=0).stop 2024-05-12T04:08:47.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.450+0000 7fcb67c39700 1 -- 172.21.15.32:0/2366596471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb600fce60 msgr2=0x7fcb6010fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:08:47.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.450+0000 7fcb67c39700 1 --2- 172.21.15.32:0/2366596471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb600fce60 0x7fcb6010fa80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c00eb40 tx=0x7fcb5c00b040 comp rx=0 tx=0).stop 2024-05-12T04:08:47.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.450+0000 7fcb67c39700 1 -- 172.21.15.32:0/2366596471 shutdown_connections 2024-05-12T04:08:47.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.450+0000 7fcb67c39700 1 --2- 172.21.15.32:0/2366596471 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fcb4c06cc20 0x7fcb4c06f0d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:47.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.450+0000 7fcb67c39700 1 --2- 172.21.15.32:0/2366596471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb600fce60 0x7fcb6010fa80 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:47.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.450+0000 7fcb67c39700 1 --2- 172.21.15.32:0/2366596471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb600fdfd0 0x7fcb6010ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:47.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.450+0000 7fcb67c39700 1 -- 172.21.15.32:0/2366596471 >> 172.21.15.32:0/2366596471 conn(0x7fcb600f8540 msgr2=0x7fcb600fa050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:08:47.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.451+0000 7fcb67c39700 1 -- 172.21.15.32:0/2366596471 shutdown_connections 2024-05-12T04:08:47.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.451+0000 7fcb67c39700 1 -- 172.21.15.32:0/2366596471 wait complete. 2024-05-12T04:08:47.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.554+0000 7fb3d38d6700 1 -- 172.21.15.32:0/2524413597 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3cc0fdfd0 msgr2=0x7fb3cc0fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:08:47.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.554+0000 7fb3d38d6700 1 --2- 172.21.15.32:0/2524413597 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3cc0fdfd0 0x7fb3cc0fe440 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0004940 tx=0x7fb3c0009e30 comp rx=0 tx=0).stop 2024-05-12T04:08:47.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.554+0000 7fb3d38d6700 1 -- 172.21.15.32:0/2524413597 shutdown_connections 2024-05-12T04:08:47.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.554+0000 7fb3d38d6700 1 --2- 172.21.15.32:0/2524413597 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3cc0fdfd0 0x7fb3cc0fe440 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:47.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.554+0000 7fb3d38d6700 1 --2- 172.21.15.32:0/2524413597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3cc0fce60 0x7fb3cc0fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:47.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.554+0000 7fb3d38d6700 1 -- 172.21.15.32:0/2524413597 >> 172.21.15.32:0/2524413597 conn(0x7fb3cc0f8560 msgr2=0x7fb3cc0fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:08:47.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.554+0000 7fb3d38d6700 1 -- 172.21.15.32:0/2524413597 shutdown_connections 2024-05-12T04:08:47.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.554+0000 7fb3d38d6700 1 -- 172.21.15.32:0/2524413597 wait complete. 2024-05-12T04:08:47.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.554+0000 7fb3d38d6700 1 Processor -- start 2024-05-12T04:08:47.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.554+0000 7fb3d38d6700 1 -- start start 2024-05-12T04:08:47.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.555+0000 7fb3d38d6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3cc0fce60 0x7fb3cc10d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:08:47.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.555+0000 7fb3d38d6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3cc0fdfd0 0x7fb3cc10de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:08:47.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.555+0000 7fb3d38d6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3cc10e460 con 0x7fb3cc0fce60 2024-05-12T04:08:47.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.555+0000 7fb3d38d6700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3cc10ecb0 con 0x7fb3cc0fdfd0 2024-05-12T04:08:47.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.555+0000 7fb3d1672700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3cc0fce60 0x7fb3cc10d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:08:47.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.555+0000 7fb3d1672700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3cc0fce60 0x7fb3cc10d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58430/0 (socket says 172.21.15.32:58430) 2024-05-12T04:08:47.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.555+0000 7fb3d1672700 1 -- 172.21.15.32:0/6548165 learned_addr learned my addr 172.21.15.32:0/6548165 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:08:47.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.555+0000 7fb3d0e71700 1 --2- 172.21.15.32:0/6548165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3cc0fdfd0 0x7fb3cc10de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:08:47.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.555+0000 7fb3d1672700 1 -- 172.21.15.32:0/6548165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3cc0fdfd0 msgr2=0x7fb3cc10de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:08:47.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.555+0000 7fb3d1672700 1 --2- 172.21.15.32:0/6548165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3cc0fdfd0 0x7fb3cc10de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:47.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.555+0000 7fb3d1672700 1 -- 172.21.15.32:0/6548165 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3c0005040 con 0x7fb3cc0fce60 2024-05-12T04:08:47.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.555+0000 7fb3d1672700 1 --2- 172.21.15.32:0/6548165 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3cc0fce60 0x7fb3cc10d920 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb3c8002a50 tx=0x7fb3c80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:08:47.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.555+0000 7fb3bdffb700 1 -- 172.21.15.32:0/6548165 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb3c8007cc0 con 0x7fb3cc0fce60 2024-05-12T04:08:47.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.555+0000 7fb3bdffb700 1 -- 172.21.15.32:0/6548165 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb3c800f040 con 0x7fb3cc0fce60 2024-05-12T04:08:47.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.555+0000 7fb3d38d6700 1 -- 172.21.15.32:0/6548165 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3cc10ef90 con 0x7fb3cc0fce60 2024-05-12T04:08:47.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.555+0000 7fb3bdffb700 1 -- 172.21.15.32:0/6548165 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb3c800d9c0 con 0x7fb3cc0fce60 2024-05-12T04:08:47.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.556+0000 7fb3d38d6700 1 -- 172.21.15.32:0/6548165 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3cc10f4d0 con 0x7fb3cc0fce60 2024-05-12T04:08:47.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.556+0000 7fb3d38d6700 1 -- 172.21.15.32:0/6548165 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3cc05f5c0 con 0x7fb3cc0fce60 2024-05-12T04:08:47.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.557+0000 7fb3bdffb700 1 -- 172.21.15.32:0/6548165 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fb3c800db20 con 0x7fb3cc0fce60 2024-05-12T04:08:47.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.557+0000 7fb3bdffb700 1 --2- 172.21.15.32:0/6548165 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fb3b806cc20 0x7fb3b806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:08:47.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.558+0000 7fb3d0e71700 1 --2- 172.21.15.32:0/6548165 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fb3b806cc20 0x7fb3b806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:08:47.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.558+0000 7fb3bdffb700 1 -- 172.21.15.32:0/6548165 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fb3c808bcb0 con 0x7fb3cc0fce60 2024-05-12T04:08:47.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.558+0000 7fb3d0e71700 1 --2- 172.21.15.32:0/6548165 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fb3b806cc20 0x7fb3b806f0d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb3c000b900 tx=0x7fb3c0009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:08:47.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.562+0000 7fb3bdffb700 1 -- 172.21.15.32:0/6548165 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fb3c8059780 con 0x7fb3cc0fce60 2024-05-12T04:08:47.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:47 smithi033 ceph-mon[107305]: pgmap v202: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail 2024-05-12T04:08:47.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:47 smithi033 ceph-mon[107305]: from='client.34440 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:08:47.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:47 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:47.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:47 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:47.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:47 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:08:47.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:47 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T04:08:47.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:47 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:47.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:47 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:08:47.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:47 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:08:47.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:47 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:08:47.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:47 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:08:47.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:47 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:08:47.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:47 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi033.hglwml"]}]: dispatch 2024-05-12T04:08:47.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:47 smithi032 ceph-mon[120392]: pgmap v202: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail 2024-05-12T04:08:47.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:47 smithi032 ceph-mon[120392]: from='client.34440 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:08:47.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:47 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:47.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:47 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:47.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:47 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:08:47.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:47 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T04:08:47.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:47 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:47.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:47 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:08:47.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:47 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:08:47.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:47 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:08:47.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:47 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:08:47.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:47 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:08:47.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:47 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi033.hglwml"]}]: dispatch 2024-05-12T04:08:47.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.813+0000 7fb3d38d6700 1 -- 172.21.15.32:0/6548165 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb3cc10f8a0 con 0x7fb3cc0fce60 2024-05-12T04:08:47.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.813+0000 7fb3bdffb700 1 -- 172.21.15.32:0/6548165 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1843 (secure 0 0 0) 0x7fb3c8059310 con 0x7fb3cc0fce60 2024-05-12T04:08:47.813 INFO:teuthology.orchestra.run.smithi032.stdout:e30 2024-05-12T04:08:47.813 INFO:teuthology.orchestra.run.smithi032.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-12T04:08:47.814 INFO:teuthology.orchestra.run.smithi032.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-12T04:08:47.814 INFO:teuthology.orchestra.run.smithi032.stdout:legacy client fscid: 1 2024-05-12T04:08:47.814 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:08:47.814 INFO:teuthology.orchestra.run.smithi032.stdout:Filesystem 'cephfs' (1) 2024-05-12T04:08:47.814 INFO:teuthology.orchestra.run.smithi032.stdout:fs_name cephfs 2024-05-12T04:08:47.814 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 28 2024-05-12T04:08:47.814 INFO:teuthology.orchestra.run.smithi032.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-05-12T04:08:47.814 INFO:teuthology.orchestra.run.smithi032.stdout:created 2024-05-12T03:57:48.904614+0000 2024-05-12T04:08:47.814 INFO:teuthology.orchestra.run.smithi032.stdout:modified 2024-05-12T04:08:21.227973+0000 2024-05-12T04:08:47.814 INFO:teuthology.orchestra.run.smithi032.stdout:tableserver 0 2024-05-12T04:08:47.814 INFO:teuthology.orchestra.run.smithi032.stdout:root 0 2024-05-12T04:08:47.814 INFO:teuthology.orchestra.run.smithi032.stdout:session_timeout 60 2024-05-12T04:08:47.814 INFO:teuthology.orchestra.run.smithi032.stdout:session_autoclose 300 2024-05-12T04:08:47.815 INFO:teuthology.orchestra.run.smithi032.stdout:max_file_size 1099511627776 2024-05-12T04:08:47.815 INFO:teuthology.orchestra.run.smithi032.stdout:max_xattr_size 65536 2024-05-12T04:08:47.815 INFO:teuthology.orchestra.run.smithi032.stdout:required_client_features {} 2024-05-12T04:08:47.815 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure 0 2024-05-12T04:08:47.815 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure_osd_epoch 82 2024-05-12T04:08:47.815 INFO:teuthology.orchestra.run.smithi032.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-12T04:08:47.815 INFO:teuthology.orchestra.run.smithi032.stdout:max_mds 1 2024-05-12T04:08:47.815 INFO:teuthology.orchestra.run.smithi032.stdout:in 0 2024-05-12T04:08:47.815 INFO:teuthology.orchestra.run.smithi032.stdout:up {0=24283} 2024-05-12T04:08:47.815 INFO:teuthology.orchestra.run.smithi032.stdout:failed 2024-05-12T04:08:47.815 INFO:teuthology.orchestra.run.smithi032.stdout:damaged 2024-05-12T04:08:47.815 INFO:teuthology.orchestra.run.smithi032.stdout:stopped 2024-05-12T04:08:47.815 INFO:teuthology.orchestra.run.smithi032.stdout:data_pools [3] 2024-05-12T04:08:47.815 INFO:teuthology.orchestra.run.smithi032.stdout:metadata_pool 2 2024-05-12T04:08:47.815 INFO:teuthology.orchestra.run.smithi032.stdout:inline_data enabled 2024-05-12T04:08:47.816 INFO:teuthology.orchestra.run.smithi032.stdout:balancer 2024-05-12T04:08:47.816 INFO:teuthology.orchestra.run.smithi032.stdout:bal_rank_mask -1 2024-05-12T04:08:47.816 INFO:teuthology.orchestra.run.smithi032.stdout:standby_count_wanted 1 2024-05-12T04:08:47.816 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi033.hglwml{0:24283} state up:active seq 157 join_fscid=1 addr [v2:172.21.15.33:6826/433858238,v1:172.21.15.33:6827/433858238] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:08:47.816 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:08:47.816 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:08:47.816 INFO:teuthology.orchestra.run.smithi032.stdout:Standby daemons: 2024-05-12T04:08:47.816 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:08:47.816 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.ialtqb{-1:34404} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.32:6826/1080358503,v1:172.21.15.32:6827/1080358503] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:08:47.816 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.qpsawh{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.32:6828/3603452506,v1:172.21.15.32:6829/3603452506] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:08:47.816 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi033.jgbjhk{-1:44343} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6824/997254871,v1:172.21.15.33:6825/997254871] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:08:47.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.815+0000 7fb3d38d6700 1 -- 172.21.15.32:0/6548165 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fb3b806cc20 msgr2=0x7fb3b806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:08:47.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.815+0000 7fb3d38d6700 1 --2- 172.21.15.32:0/6548165 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fb3b806cc20 0x7fb3b806f0d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb3c000b900 tx=0x7fb3c0009e30 comp rx=0 tx=0).stop 2024-05-12T04:08:47.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.815+0000 7fb3d38d6700 1 -- 172.21.15.32:0/6548165 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3cc0fce60 msgr2=0x7fb3cc10d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:08:47.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.815+0000 7fb3d38d6700 1 --2- 172.21.15.32:0/6548165 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3cc0fce60 0x7fb3cc10d920 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb3c8002a50 tx=0x7fb3c80046b0 comp rx=0 tx=0).stop 2024-05-12T04:08:47.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.815+0000 7fb3d38d6700 1 -- 172.21.15.32:0/6548165 shutdown_connections 2024-05-12T04:08:47.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.815+0000 7fb3d38d6700 1 --2- 172.21.15.32:0/6548165 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fb3b806cc20 0x7fb3b806f0d0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:47.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.815+0000 7fb3d38d6700 1 --2- 172.21.15.32:0/6548165 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3cc0fce60 0x7fb3cc10d920 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:47.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.815+0000 7fb3d38d6700 1 --2- 172.21.15.32:0/6548165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3cc0fdfd0 0x7fb3cc10de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:47.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.815+0000 7fb3d38d6700 1 -- 172.21.15.32:0/6548165 >> 172.21.15.32:0/6548165 conn(0x7fb3cc0f8560 msgr2=0x7fb3cc0f9d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:08:47.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.815+0000 7fb3d38d6700 1 -- 172.21.15.32:0/6548165 shutdown_connections 2024-05-12T04:08:47.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.815+0000 7fb3d38d6700 1 -- 172.21.15.32:0/6548165 wait complete. 2024-05-12T04:08:47.818 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 30 2024-05-12T04:08:47.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.920+0000 7fde450ff700 1 -- 172.21.15.32:0/2541659049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde400fbcb0 msgr2=0x7fde400fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:08:47.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.920+0000 7fde450ff700 1 --2- 172.21.15.32:0/2541659049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde400fbcb0 0x7fde400fc0c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fde30004940 tx=0x7fde30009e30 comp rx=0 tx=0).stop 2024-05-12T04:08:47.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.920+0000 7fde450ff700 1 -- 172.21.15.32:0/2541659049 shutdown_connections 2024-05-12T04:08:47.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.920+0000 7fde450ff700 1 --2- 172.21.15.32:0/2541659049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde400fcf60 0x7fde400fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:47.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.920+0000 7fde450ff700 1 --2- 172.21.15.32:0/2541659049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde400fbcb0 0x7fde400fc0c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:47.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.920+0000 7fde450ff700 1 -- 172.21.15.32:0/2541659049 >> 172.21.15.32:0/2541659049 conn(0x7fde400f7430 msgr2=0x7fde400f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:08:47.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.920+0000 7fde450ff700 1 -- 172.21.15.32:0/2541659049 shutdown_connections 2024-05-12T04:08:47.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.920+0000 7fde450ff700 1 -- 172.21.15.32:0/2541659049 wait complete. 2024-05-12T04:08:47.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.921+0000 7fde450ff700 1 Processor -- start 2024-05-12T04:08:47.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.921+0000 7fde450ff700 1 -- start start 2024-05-12T04:08:47.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.921+0000 7fde450ff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde400fbcb0 0x7fde401105b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:08:47.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.921+0000 7fde450ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde400fcf60 0x7fde40112b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:08:47.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.921+0000 7fde450ff700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde40066330 con 0x7fde400fbcb0 2024-05-12T04:08:47.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.921+0000 7fde450ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde40066470 con 0x7fde400fcf60 2024-05-12T04:08:47.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.921+0000 7fde3ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde400fbcb0 0x7fde401105b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:08:47.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.921+0000 7fde3ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde400fbcb0 0x7fde401105b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58454/0 (socket says 172.21.15.32:58454) 2024-05-12T04:08:47.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.921+0000 7fde3ed9d700 1 -- 172.21.15.32:0/81249239 learned_addr learned my addr 172.21.15.32:0/81249239 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:08:47.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.921+0000 7fde3e59c700 1 --2- 172.21.15.32:0/81249239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde400fcf60 0x7fde40112b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:08:47.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.922+0000 7fde3ed9d700 1 -- 172.21.15.32:0/81249239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde400fcf60 msgr2=0x7fde40112b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:08:47.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.922+0000 7fde3ed9d700 1 --2- 172.21.15.32:0/81249239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde400fcf60 0x7fde40112b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:47.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.922+0000 7fde3ed9d700 1 -- 172.21.15.32:0/81249239 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde30005040 con 0x7fde400fbcb0 2024-05-12T04:08:47.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.922+0000 7fde3ed9d700 1 --2- 172.21.15.32:0/81249239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde400fbcb0 0x7fde401105b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fde300137b0 tx=0x7fde30009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:08:47.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.922+0000 7fde377fe700 1 -- 172.21.15.32:0/81249239 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde3000e070 con 0x7fde400fbcb0 2024-05-12T04:08:47.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.922+0000 7fde450ff700 1 -- 172.21.15.32:0/81249239 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde40113040 con 0x7fde400fbcb0 2024-05-12T04:08:47.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.922+0000 7fde377fe700 1 -- 172.21.15.32:0/81249239 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fde3001c400 con 0x7fde400fbcb0 2024-05-12T04:08:47.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.922+0000 7fde377fe700 1 -- 172.21.15.32:0/81249239 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde3000b650 con 0x7fde400fbcb0 2024-05-12T04:08:47.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.922+0000 7fde450ff700 1 -- 172.21.15.32:0/81249239 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde401135a0 con 0x7fde400fbcb0 2024-05-12T04:08:47.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.923+0000 7fde450ff700 1 -- 172.21.15.32:0/81249239 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde4005f5c0 con 0x7fde400fbcb0 2024-05-12T04:08:47.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.924+0000 7fde377fe700 1 -- 172.21.15.32:0/81249239 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fde3001c9e0 con 0x7fde400fbcb0 2024-05-12T04:08:47.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.924+0000 7fde377fe700 1 --2- 172.21.15.32:0/81249239 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fde2c06cc20 0x7fde2c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:08:47.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.924+0000 7fde377fe700 1 -- 172.21.15.32:0/81249239 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fde3008d680 con 0x7fde400fbcb0 2024-05-12T04:08:47.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.924+0000 7fde3e59c700 1 --2- 172.21.15.32:0/81249239 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fde2c06cc20 0x7fde2c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:08:47.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.924+0000 7fde3e59c700 1 --2- 172.21.15.32:0/81249239 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fde2c06cc20 0x7fde2c06f0d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fde28009b80 tx=0x7fde2800b040 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:08:47.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:47.928+0000 7fde377fe700 1 -- 172.21.15.32:0/81249239 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fde300576c0 con 0x7fde400fbcb0 2024-05-12T04:08:48.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:48.151+0000 7fde450ff700 1 -- 172.21.15.32:0/81249239 --> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fde40101320 con 0x7fde2c06cc20 2024-05-12T04:08:48.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:48.152+0000 7fde377fe700 1 -- 172.21.15.32:0/81249239 <== mgr.24593 v2:172.21.15.32:6800/2957111591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fde40101320 con 0x7fde2c06cc20 2024-05-12T04:08:48.153 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-05-12T04:08:48.153 INFO:teuthology.orchestra.run.smithi032.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:19c9734256aa43a60e203feb242f50bf8b7f58666d520536af43cc4fee96173a", 2024-05-12T04:08:48.153 INFO:teuthology.orchestra.run.smithi032.stdout: "in_progress": true, 2024-05-12T04:08:48.153 INFO:teuthology.orchestra.run.smithi032.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-12T04:08:48.153 INFO:teuthology.orchestra.run.smithi032.stdout: "services_complete": [ 2024-05-12T04:08:48.153 INFO:teuthology.orchestra.run.smithi032.stdout: "osd", 2024-05-12T04:08:48.153 INFO:teuthology.orchestra.run.smithi032.stdout: "mon", 2024-05-12T04:08:48.153 INFO:teuthology.orchestra.run.smithi032.stdout: "crash", 2024-05-12T04:08:48.153 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr" 2024-05-12T04:08:48.153 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-05-12T04:08:48.153 INFO:teuthology.orchestra.run.smithi032.stdout: "progress": "15/21 daemons upgraded", 2024-05-12T04:08:48.153 INFO:teuthology.orchestra.run.smithi032.stdout: "message": "Currently upgrading mds daemons", 2024-05-12T04:08:48.154 INFO:teuthology.orchestra.run.smithi032.stdout: "is_paused": false 2024-05-12T04:08:48.154 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-05-12T04:08:48.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:48.154+0000 7fde450ff700 1 -- 172.21.15.32:0/81249239 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fde2c06cc20 msgr2=0x7fde2c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:08:48.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:48.154+0000 7fde450ff700 1 --2- 172.21.15.32:0/81249239 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fde2c06cc20 0x7fde2c06f0d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fde28009b80 tx=0x7fde2800b040 comp rx=0 tx=0).stop 2024-05-12T04:08:48.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:48.154+0000 7fde450ff700 1 -- 172.21.15.32:0/81249239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde400fbcb0 msgr2=0x7fde401105b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:08:48.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:48.154+0000 7fde450ff700 1 --2- 172.21.15.32:0/81249239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde400fbcb0 0x7fde401105b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fde300137b0 tx=0x7fde30009e30 comp rx=0 tx=0).stop 2024-05-12T04:08:48.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:48.154+0000 7fde450ff700 1 -- 172.21.15.32:0/81249239 shutdown_connections 2024-05-12T04:08:48.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:48.154+0000 7fde450ff700 1 --2- 172.21.15.32:0/81249239 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fde2c06cc20 0x7fde2c06f0d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:48.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:48.154+0000 7fde450ff700 1 --2- 172.21.15.32:0/81249239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde400fbcb0 0x7fde401105b0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:48.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:48.154+0000 7fde450ff700 1 --2- 172.21.15.32:0/81249239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde400fcf60 0x7fde40112b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:48.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:48.154+0000 7fde450ff700 1 -- 172.21.15.32:0/81249239 >> 172.21.15.32:0/81249239 conn(0x7fde400f7430 msgr2=0x7fde400f8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:08:48.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:48.154+0000 7fde450ff700 1 -- 172.21.15.32:0/81249239 shutdown_connections 2024-05-12T04:08:48.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:48.154+0000 7fde450ff700 1 -- 172.21.15.32:0/81249239 wait complete. 2024-05-12T04:08:48.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:48.258+0000 7f765374c700 1 -- 172.21.15.32:0/1277636226 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f764c0662c0 msgr2=0x7f764c0ff9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:08:48.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:48.258+0000 7f765374c700 1 --2- 172.21.15.32:0/1277636226 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f764c0662c0 0x7f764c0ff9a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7648004940 tx=0x7f7648009e30 comp rx=0 tx=0).stop 2024-05-12T04:08:48.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:48.258+0000 7f765374c700 1 -- 172.21.15.32:0/1277636226 shutdown_connections 2024-05-12T04:08:48.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:48.258+0000 7f765374c700 1 --2- 172.21.15.32:0/1277636226 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f764c0ffee0 0x7f764c102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:48.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:48.258+0000 7f765374c700 1 --2- 172.21.15.32:0/1277636226 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f764c0662c0 0x7f764c0ff9a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:48.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:48.258+0000 7f765374c700 1 -- 172.21.15.32:0/1277636226 >> 172.21.15.32:0/1277636226 conn(0x7f764c0f7430 msgr2=0x7f764c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:08:48.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:48.258+0000 7f765374c700 1 -- 172.21.15.32:0/1277636226 shutdown_connections 2024-05-12T04:08:48.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:48.258+0000 7f765374c700 1 -- 172.21.15.32:0/1277636226 wait complete. 2024-05-12T04:08:48.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:48.258+0000 7f765374c700 1 Processor -- start 2024-05-12T04:08:48.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:48.258+0000 7f765374c700 1 -- start start 2024-05-12T04:08:48.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:48.259+0000 7f765374c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f764c0662c0 0x7f764c10d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:08:48.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:48.259+0000 7f765374c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f764c0ffee0 0x7f764c10ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:08:48.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:48.259+0000 7f765374c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764c10e4d0 con 0x7f764c0662c0 2024-05-12T04:08:48.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:48.259+0000 7f765374c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764c10ecd0 con 0x7f764c0ffee0 2024-05-12T04:08:48.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:48.259+0000 7f76514e8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f764c0662c0 0x7f764c10d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:08:48.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:48.259+0000 7f76514e8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f764c0662c0 0x7f764c10d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58470/0 (socket says 172.21.15.32:58470) 2024-05-12T04:08:48.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:48.259+0000 7f76514e8700 1 -- 172.21.15.32:0/3995815218 learned_addr learned my addr 172.21.15.32:0/3995815218 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:08:48.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:48.259+0000 7f7650ce7700 1 --2- 172.21.15.32:0/3995815218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f764c0ffee0 0x7f764c10ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:08:48.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:48.259+0000 7f76514e8700 1 -- 172.21.15.32:0/3995815218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f764c0ffee0 msgr2=0x7f764c10ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:08:48.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:48.259+0000 7f76514e8700 1 --2- 172.21.15.32:0/3995815218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f764c0ffee0 0x7f764c10ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:48.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:48.259+0000 7f76514e8700 1 -- 172.21.15.32:0/3995815218 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7648005040 con 0x7f764c0662c0 2024-05-12T04:08:48.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:48.259+0000 7f76514e8700 1 --2- 172.21.15.32:0/3995815218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f764c0662c0 0x7f764c10d990 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f764800b900 tx=0x7f7648009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:08:48.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:48.259+0000 7f7641ffb700 1 -- 172.21.15.32:0/3995815218 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f764800e040 con 0x7f764c0662c0 2024-05-12T04:08:48.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:48.259+0000 7f7641ffb700 1 -- 172.21.15.32:0/3995815218 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7648004290 con 0x7f764c0662c0 2024-05-12T04:08:48.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:48.259+0000 7f765374c700 1 -- 172.21.15.32:0/3995815218 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f764c10ef50 con 0x7f764c0662c0 2024-05-12T04:08:48.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:48.259+0000 7f7641ffb700 1 -- 172.21.15.32:0/3995815218 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f764801c3f0 con 0x7f764c0662c0 2024-05-12T04:08:48.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:48.260+0000 7f765374c700 1 -- 172.21.15.32:0/3995815218 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f764c10f490 con 0x7f764c0662c0 2024-05-12T04:08:48.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:48.261+0000 7f765374c700 1 -- 172.21.15.32:0/3995815218 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f764c05f5c0 con 0x7f764c0662c0 2024-05-12T04:08:48.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:48.261+0000 7f7641ffb700 1 -- 172.21.15.32:0/3995815218 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f7648013420 con 0x7f764c0662c0 2024-05-12T04:08:48.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:48.262+0000 7f7641ffb700 1 --2- 172.21.15.32:0/3995815218 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f763806cc30 0x7f763806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:08:48.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:48.262+0000 7f7641ffb700 1 -- 172.21.15.32:0/3995815218 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f764808d3b0 con 0x7f764c0662c0 2024-05-12T04:08:48.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:48.262+0000 7f7650ce7700 1 --2- 172.21.15.32:0/3995815218 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f763806cc30 0x7f763806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:08:48.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:48.262+0000 7f7650ce7700 1 --2- 172.21.15.32:0/3995815218 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f763806cc30 0x7f763806f0e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f763c004f40 tx=0x7f763c00b040 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:08:48.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:48.266+0000 7f7641ffb700 1 -- 172.21.15.32:0/3995815218 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f764805ae00 con 0x7f764c0662c0 2024-05-12T04:08:48.542 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:48 smithi033 ceph-mon[107305]: from='client.34444 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:08:48.542 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:48 smithi033 ceph-mon[107305]: Detected new or changed devices on smithi033 2024-05-12T04:08:48.542 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:48 smithi033 ceph-mon[107305]: from='client.34448 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:08:48.542 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:48 smithi033 ceph-mon[107305]: Upgrade: It appears safe to stop mds.cephfs.smithi033.hglwml 2024-05-12T04:08:48.542 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:48 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/2366596471' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:08:48.542 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:48 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/6548165' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-12T04:08:48.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:48.554+0000 7f765374c700 1 -- 172.21.15.32:0/3995815218 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f764c10f810 con 0x7f764c0662c0 2024-05-12T04:08:48.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:48.555+0000 7f7641ffb700 1 -- 172.21.15.32:0/3995815218 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f764805a990 con 0x7f764c0662c0 2024-05-12T04:08:48.555 INFO:teuthology.orchestra.run.smithi032.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-12T04:08:48.555 INFO:teuthology.orchestra.run.smithi032.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-12T04:08:48.555 INFO:teuthology.orchestra.run.smithi032.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-12T04:08:48.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:48.557+0000 7f765374c700 1 -- 172.21.15.32:0/3995815218 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f763806cc30 msgr2=0x7f763806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:08:48.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:48.557+0000 7f765374c700 1 --2- 172.21.15.32:0/3995815218 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f763806cc30 0x7f763806f0e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f763c004f40 tx=0x7f763c00b040 comp rx=0 tx=0).stop 2024-05-12T04:08:48.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:48.557+0000 7f765374c700 1 -- 172.21.15.32:0/3995815218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f764c0662c0 msgr2=0x7f764c10d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:08:48.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:48.557+0000 7f765374c700 1 --2- 172.21.15.32:0/3995815218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f764c0662c0 0x7f764c10d990 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f764800b900 tx=0x7f7648009e30 comp rx=0 tx=0).stop 2024-05-12T04:08:48.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:48.557+0000 7f765374c700 1 -- 172.21.15.32:0/3995815218 shutdown_connections 2024-05-12T04:08:48.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:48.557+0000 7f765374c700 1 --2- 172.21.15.32:0/3995815218 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f763806cc30 0x7f763806f0e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:48.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:48.557+0000 7f765374c700 1 --2- 172.21.15.32:0/3995815218 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f764c0662c0 0x7f764c10d990 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:48.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:48.557+0000 7f765374c700 1 --2- 172.21.15.32:0/3995815218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f764c0ffee0 0x7f764c10ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:08:48.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:48.557+0000 7f765374c700 1 -- 172.21.15.32:0/3995815218 >> 172.21.15.32:0/3995815218 conn(0x7f764c0f7430 msgr2=0x7f764c0f9810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:08:48.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:48.557+0000 7f765374c700 1 -- 172.21.15.32:0/3995815218 shutdown_connections 2024-05-12T04:08:48.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:08:48.557+0000 7f765374c700 1 -- 172.21.15.32:0/3995815218 wait complete. 2024-05-12T04:08:48.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:48 smithi032 ceph-mon[120392]: from='client.34444 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:08:48.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:48 smithi032 ceph-mon[120392]: Detected new or changed devices on smithi033 2024-05-12T04:08:48.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:48 smithi032 ceph-mon[120392]: from='client.34448 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:08:48.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:48 smithi032 ceph-mon[120392]: Upgrade: It appears safe to stop mds.cephfs.smithi033.hglwml 2024-05-12T04:08:48.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:48 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/2366596471' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:08:48.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:48 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/6548165' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-12T04:08:49.643 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:49 smithi033 ceph-mon[107305]: pgmap v203: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail 2024-05-12T04:08:49.643 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:49 smithi033 ceph-mon[107305]: from='client.34460 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:08:49.643 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:49 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/3995815218' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-12T04:08:49.643 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:49 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:49.643 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:49 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi033.hglwml", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-12T04:08:49.643 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:49 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:08:49.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:49 smithi032 ceph-mon[120392]: pgmap v203: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail 2024-05-12T04:08:49.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:49 smithi032 ceph-mon[120392]: from='client.34460 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:08:49.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:49 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/3995815218' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-12T04:08:49.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:49 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:49.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:49 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi033.hglwml", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-12T04:08:49.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:49 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:08:50.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:50 smithi033 ceph-mon[107305]: Upgrade: Updating mds.cephfs.smithi033.hglwml 2024-05-12T04:08:50.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:50 smithi033 ceph-mon[107305]: Deploying daemon mds.cephfs.smithi033.hglwml on smithi033 2024-05-12T04:08:50.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:50 smithi032 ceph-mon[120392]: Upgrade: Updating mds.cephfs.smithi033.hglwml 2024-05-12T04:08:50.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:50 smithi032 ceph-mon[120392]: Deploying daemon mds.cephfs.smithi033.hglwml on smithi033 2024-05-12T04:08:50.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:50 smithi032 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-mon-smithi032[120369]: 2024-05-12T04:08:50.380+0000 7f474139f700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-12T04:08:51.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:51 smithi033 ceph-mon[107305]: pgmap v204: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail 2024-05-12T04:08:51.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:51 smithi033 ceph-mon[107305]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-12T04:08:51.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:51 smithi033 ceph-mon[107305]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-12T04:08:51.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:51 smithi033 ceph-mon[107305]: osdmap e84: 6 total, 6 up, 6 in 2024-05-12T04:08:51.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:51 smithi033 ceph-mon[107305]: Standby daemon mds.cephfs.smithi032.ialtqb assigned to filesystem cephfs as rank 0 2024-05-12T04:08:51.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:51 smithi033 ceph-mon[107305]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-12T04:08:51.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:51 smithi033 ceph-mon[107305]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-05-12T04:08:51.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:51 smithi033 ceph-mon[107305]: fsmap cephfs:1/1 {0=cephfs.smithi032.ialtqb=up:replay} 2 up:standby 2024-05-12T04:08:51.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:51 smithi032 ceph-mon[120392]: pgmap v204: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail 2024-05-12T04:08:51.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:51 smithi032 ceph-mon[120392]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-12T04:08:51.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:51 smithi032 ceph-mon[120392]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-12T04:08:51.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:51 smithi032 ceph-mon[120392]: osdmap e84: 6 total, 6 up, 6 in 2024-05-12T04:08:51.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:51 smithi032 ceph-mon[120392]: Standby daemon mds.cephfs.smithi032.ialtqb assigned to filesystem cephfs as rank 0 2024-05-12T04:08:51.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:51 smithi032 ceph-mon[120392]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-12T04:08:51.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:51 smithi032 ceph-mon[120392]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-05-12T04:08:51.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:51 smithi032 ceph-mon[120392]: fsmap cephfs:1/1 {0=cephfs.smithi032.ialtqb=up:replay} 2 up:standby 2024-05-12T04:08:52.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:52 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi033.hglwml"}]: dispatch 2024-05-12T04:08:52.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:52 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi033.hglwml"}]: dispatch 2024-05-12T04:08:53.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:53 smithi033 ceph-mon[107305]: pgmap v206: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 0 op/s 2024-05-12T04:08:53.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:53 smithi032 ceph-mon[120392]: pgmap v206: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 0 op/s 2024-05-12T04:08:55.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:55 smithi033 ceph-mon[107305]: pgmap v207: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 11 MiB/s rd, 4 op/s 2024-05-12T04:08:55.789 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:55 smithi032 ceph-mon[120392]: pgmap v207: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 11 MiB/s rd, 4 op/s 2024-05-12T04:08:57.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:57 smithi033 ceph-mon[107305]: pgmap v208: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 11 MiB/s rd, 4 op/s 2024-05-12T04:08:57.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:57 smithi032 ceph-mon[120392]: pgmap v208: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 11 MiB/s rd, 4 op/s 2024-05-12T04:08:59.005 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:59 smithi033 ceph-mon[107305]: pgmap v209: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 6 op/s 2024-05-12T04:08:59.005 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:59 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:59.005 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:59 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:59.005 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:59 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:08:59.005 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:59 smithi033 ceph-mon[107305]: mds.? [v2:172.21.15.33:6826/1983593176,v1:172.21.15.33:6827/1983593176] up:boot 2024-05-12T04:08:59.005 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:59 smithi033 ceph-mon[107305]: fsmap cephfs:1/1 {0=cephfs.smithi032.ialtqb=up:replay} 3 up:standby 2024-05-12T04:08:59.006 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:08:59 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi033.hglwml"}]: dispatch 2024-05-12T04:08:59.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:59 smithi032 ceph-mon[120392]: pgmap v209: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 6 op/s 2024-05-12T04:08:59.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:59 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:59.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:59 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:08:59.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:59 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:08:59.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:59 smithi032 ceph-mon[120392]: mds.? [v2:172.21.15.33:6826/1983593176,v1:172.21.15.33:6827/1983593176] up:boot 2024-05-12T04:08:59.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:59 smithi032 ceph-mon[120392]: fsmap cephfs:1/1 {0=cephfs.smithi032.ialtqb=up:replay} 3 up:standby 2024-05-12T04:08:59.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:08:59 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi033.hglwml"}]: dispatch 2024-05-12T04:09:01.271 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:01 smithi033 ceph-mon[107305]: pgmap v210: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 24 MiB/s rd, 7 op/s 2024-05-12T04:09:01.271 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:01 smithi033 ceph-mon[107305]: reconnect by client.14532 192.168.0.1:0/3762506211 after 0.00099998 2024-05-12T04:09:01.272 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:01 smithi033 ceph-mon[107305]: reconnect by client.24323 192.168.0.1:0/1200139116 after 0.00199996 2024-05-12T04:09:01.272 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:01 smithi033 ceph-mon[107305]: mds.? [v2:172.21.15.32:6826/1080358503,v1:172.21.15.32:6827/1080358503] up:reconnect 2024-05-12T04:09:01.272 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:01 smithi033 ceph-mon[107305]: fsmap cephfs:1/1 {0=cephfs.smithi032.ialtqb=up:reconnect} 3 up:standby 2024-05-12T04:09:01.272 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:01 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:01.272 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:01 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:01.272 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:01 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:01.272 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:01 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:01.272 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:01 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:01.272 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:01 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:09:01.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:01 smithi032 ceph-mon[120392]: pgmap v210: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 24 MiB/s rd, 7 op/s 2024-05-12T04:09:01.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:01 smithi032 ceph-mon[120392]: reconnect by client.14532 192.168.0.1:0/3762506211 after 0.00099998 2024-05-12T04:09:01.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:01 smithi032 ceph-mon[120392]: reconnect by client.24323 192.168.0.1:0/1200139116 after 0.00199996 2024-05-12T04:09:01.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:01 smithi032 ceph-mon[120392]: mds.? [v2:172.21.15.32:6826/1080358503,v1:172.21.15.32:6827/1080358503] up:reconnect 2024-05-12T04:09:01.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:01 smithi032 ceph-mon[120392]: fsmap cephfs:1/1 {0=cephfs.smithi032.ialtqb=up:reconnect} 3 up:standby 2024-05-12T04:09:01.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:01 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:01.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:01 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:01.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:01 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:01.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:01 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:01.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:01 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:01.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:01 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:09:02.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:02 smithi033 ceph-mon[107305]: mds.? [v2:172.21.15.32:6826/1080358503,v1:172.21.15.32:6827/1080358503] up:rejoin 2024-05-12T04:09:02.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:02 smithi033 ceph-mon[107305]: fsmap cephfs:1/1 {0=cephfs.smithi032.ialtqb=up:rejoin} 3 up:standby 2024-05-12T04:09:02.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:02 smithi033 ceph-mon[107305]: daemon mds.cephfs.smithi032.ialtqb is now active in filesystem cephfs as rank 0 2024-05-12T04:09:02.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:02 smithi032 ceph-mon[120392]: mds.? [v2:172.21.15.32:6826/1080358503,v1:172.21.15.32:6827/1080358503] up:rejoin 2024-05-12T04:09:02.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:02 smithi032 ceph-mon[120392]: fsmap cephfs:1/1 {0=cephfs.smithi032.ialtqb=up:rejoin} 3 up:standby 2024-05-12T04:09:02.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:02 smithi032 ceph-mon[120392]: daemon mds.cephfs.smithi032.ialtqb is now active in filesystem cephfs as rank 0 2024-05-12T04:09:03.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:03 smithi032 ceph-mon[120392]: pgmap v211: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 6 op/s 2024-05-12T04:09:03.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:03 smithi032 ceph-mon[120392]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-12T04:09:03.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:03 smithi032 ceph-mon[120392]: mds.? [v2:172.21.15.32:6826/1080358503,v1:172.21.15.32:6827/1080358503] up:active 2024-05-12T04:09:03.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:03 smithi032 ceph-mon[120392]: fsmap cephfs:1 {0=cephfs.smithi032.ialtqb=up:active} 3 up:standby 2024-05-12T04:09:03.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:03 smithi033 ceph-mon[107305]: pgmap v211: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 6 op/s 2024-05-12T04:09:03.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:03 smithi033 ceph-mon[107305]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-12T04:09:03.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:03 smithi033 ceph-mon[107305]: mds.? [v2:172.21.15.32:6826/1080358503,v1:172.21.15.32:6827/1080358503] up:active 2024-05-12T04:09:03.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:03 smithi033 ceph-mon[107305]: fsmap cephfs:1 {0=cephfs.smithi032.ialtqb=up:active} 3 up:standby 2024-05-12T04:09:05.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:05 smithi033 ceph-mon[107305]: pgmap v212: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 85 B/s wr, 8 op/s 2024-05-12T04:09:05.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:05 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:05.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:05 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:05.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:05 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:09:05.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:05 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T04:09:05.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:05 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:05.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:05 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:09:05.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:05 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:05.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:05 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:05.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:05 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:05.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:05 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:05.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:05 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:05.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:05 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:05.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:05 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi032.ialtqb"}]: dispatch 2024-05-12T04:09:05.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:05 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi032.ialtqb"}]': finished 2024-05-12T04:09:05.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:05 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi032.qpsawh"}]: dispatch 2024-05-12T04:09:05.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:05 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi032.qpsawh"}]': finished 2024-05-12T04:09:05.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:05 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi033.hglwml"}]: dispatch 2024-05-12T04:09:05.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:05 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi033.hglwml"}]': finished 2024-05-12T04:09:05.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:05 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi033.jgbjhk"}]: dispatch 2024-05-12T04:09:05.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:05 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi033.jgbjhk"}]': finished 2024-05-12T04:09:05.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:05 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]: dispatch 2024-05-12T04:09:05.664 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:05 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]': finished 2024-05-12T04:09:05.664 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:05 smithi033 ceph-mon[107305]: fsmap cephfs:1 {0=cephfs.smithi032.ialtqb=up:active} 1 up:standby-replay 2 up:standby 2024-05-12T04:09:05.664 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:05 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:05.664 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:05 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:05.664 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:05 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:05.664 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:05 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:05.664 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:05 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:05.664 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:05 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:05.664 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:05 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:05.664 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:05 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:05.664 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:05 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:05.664 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:05 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:05.664 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:05 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:05.664 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:05 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:05.664 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:05 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:05.665 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:05 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:05.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:05 smithi032 ceph-mon[120392]: pgmap v212: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 85 B/s wr, 8 op/s 2024-05-12T04:09:05.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:05 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:05.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:05 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:05.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:05 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:09:05.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:05 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T04:09:05.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:05 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:05.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:05 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:09:05.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:05 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:05.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:05 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:05.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:05 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:05.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:05 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:05.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:05 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:05.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:05 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:05.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:05 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi032.ialtqb"}]: dispatch 2024-05-12T04:09:05.792 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:05 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi032.ialtqb"}]': finished 2024-05-12T04:09:05.792 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:05 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi032.qpsawh"}]: dispatch 2024-05-12T04:09:05.792 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:05 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi032.qpsawh"}]': finished 2024-05-12T04:09:05.792 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:05 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi033.hglwml"}]: dispatch 2024-05-12T04:09:05.792 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:05 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi033.hglwml"}]': finished 2024-05-12T04:09:05.792 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:05 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi033.jgbjhk"}]: dispatch 2024-05-12T04:09:05.792 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:05 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi033.jgbjhk"}]': finished 2024-05-12T04:09:05.792 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:05 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]: dispatch 2024-05-12T04:09:05.792 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:05 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]': finished 2024-05-12T04:09:05.792 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:05 smithi032 ceph-mon[120392]: fsmap cephfs:1 {0=cephfs.smithi032.ialtqb=up:active} 1 up:standby-replay 2 up:standby 2024-05-12T04:09:05.792 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:05 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:05.792 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:05 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:05.792 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:05 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:05.792 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:05 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:05.792 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:05 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:05.793 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:05 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:05.793 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:05 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:05.793 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:05 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:05.793 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:05 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:05.793 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:05 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:05.793 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:05 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:05.793 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:05 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:05.793 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:05 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:05.793 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:05 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:06.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:06 smithi033 ceph-mon[107305]: Upgrade: Setting container_image for all mds 2024-05-12T04:09:06.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:06 smithi033 ceph-mon[107305]: Upgrade: Enabling allow_standby_replay on filesystem cephfs 2024-05-12T04:09:06.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:06 smithi033 ceph-mon[107305]: Upgrade: Setting container_image for all rgw 2024-05-12T04:09:06.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:06 smithi033 ceph-mon[107305]: Upgrade: Setting container_image for all rbd-mirror 2024-05-12T04:09:06.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:06 smithi033 ceph-mon[107305]: Upgrade: Setting container_image for all ceph-exporter 2024-05-12T04:09:06.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:06 smithi033 ceph-mon[107305]: Upgrade: Setting container_image for all iscsi 2024-05-12T04:09:06.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:06 smithi033 ceph-mon[107305]: Upgrade: Setting container_image for all nfs 2024-05-12T04:09:06.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:06 smithi033 ceph-mon[107305]: Upgrade: Setting container_image for all nvmeof 2024-05-12T04:09:06.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:06 smithi032 ceph-mon[120392]: Upgrade: Setting container_image for all mds 2024-05-12T04:09:06.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:06 smithi032 ceph-mon[120392]: Upgrade: Enabling allow_standby_replay on filesystem cephfs 2024-05-12T04:09:06.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:06 smithi032 ceph-mon[120392]: Upgrade: Setting container_image for all rgw 2024-05-12T04:09:06.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:06 smithi032 ceph-mon[120392]: Upgrade: Setting container_image for all rbd-mirror 2024-05-12T04:09:06.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:06 smithi032 ceph-mon[120392]: Upgrade: Setting container_image for all ceph-exporter 2024-05-12T04:09:06.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:06 smithi032 ceph-mon[120392]: Upgrade: Setting container_image for all iscsi 2024-05-12T04:09:06.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:06 smithi032 ceph-mon[120392]: Upgrade: Setting container_image for all nfs 2024-05-12T04:09:06.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:06 smithi032 ceph-mon[120392]: Upgrade: Setting container_image for all nvmeof 2024-05-12T04:09:07.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:07 smithi033 ceph-mon[107305]: pgmap v213: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 13 MiB/s rd, 85 B/s wr, 5 op/s 2024-05-12T04:09:07.778 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:07 smithi032 ceph-mon[120392]: pgmap v213: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 13 MiB/s rd, 85 B/s wr, 5 op/s 2024-05-12T04:09:09.530 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:09 smithi032 ceph-mon[120392]: pgmap v214: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-05-12T04:09:09.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:09 smithi033 ceph-mon[107305]: pgmap v214: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-05-12T04:09:10.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:10 smithi033 ceph-mon[107305]: Upgrade: Updating node-exporter.smithi032 (1/2) 2024-05-12T04:09:10.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:10 smithi033 ceph-mon[107305]: Deploying daemon node-exporter.smithi032 on smithi032 2024-05-12T04:09:10.740 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:10 smithi032 ceph-mon[120392]: Upgrade: Updating node-exporter.smithi032 (1/2) 2024-05-12T04:09:10.740 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:10 smithi032 ceph-mon[120392]: Deploying daemon node-exporter.smithi032 on smithi032 2024-05-12T04:09:11.655 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:11 smithi032 ceph-mon[120392]: pgmap v215: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 14 MiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-05-12T04:09:11.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:11 smithi033 ceph-mon[107305]: pgmap v215: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 14 MiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-05-12T04:09:13.017 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:12 smithi033 ceph-mon[107305]: pgmap v216: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 11 MiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-05-12T04:09:13.017 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:12 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:13.017 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:12 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:13.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:12 smithi032 ceph-mon[120392]: pgmap v216: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 11 MiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-05-12T04:09:13.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:12 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:13.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:12 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:14.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:13 smithi033 ceph-mon[107305]: Upgrade: Updating node-exporter.smithi033 (2/2) 2024-05-12T04:09:14.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:13 smithi033 ceph-mon[107305]: Deploying daemon node-exporter.smithi033 on smithi033 2024-05-12T04:09:14.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:13 smithi032 ceph-mon[120392]: Upgrade: Updating node-exporter.smithi033 (2/2) 2024-05-12T04:09:14.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:13 smithi032 ceph-mon[120392]: Deploying daemon node-exporter.smithi033 on smithi033 2024-05-12T04:09:15.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:14 smithi033 ceph-mon[107305]: pgmap v217: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 4.2 KiB/s wr, 11 op/s 2024-05-12T04:09:15.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:14 smithi032 ceph-mon[120392]: pgmap v217: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 4.2 KiB/s wr, 11 op/s 2024-05-12T04:09:16.880 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:16 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:16.881 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:16 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:16.881 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:16 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:09:16.881 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:16 smithi033 ceph-mon[107305]: pgmap v218: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-05-12T04:09:16.881 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:16 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:16.881 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:16 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:09:17.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:16 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:17.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:16 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:17.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:16 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:09:17.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:16 smithi032 ceph-mon[120392]: pgmap v218: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-05-12T04:09:17.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:16 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:17.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:16 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:09:18.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:18.669+0000 7f9f97e81700 1 -- 172.21.15.32:0/3367937775 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f90066ed0 msgr2=0x7f9f900ff9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:09:18.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:18.669+0000 7f9f97e81700 1 --2- 172.21.15.32:0/3367937775 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f90066ed0 0x7f9f900ff9a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c004940 tx=0x7f9f8c009e30 comp rx=0 tx=0).stop 2024-05-12T04:09:18.672 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:18.669+0000 7f9f97e81700 1 -- 172.21.15.32:0/3367937775 shutdown_connections 2024-05-12T04:09:18.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:18.670+0000 7f9f97e81700 1 --2- 172.21.15.32:0/3367937775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f900ffee0 0x7f9f90102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:18.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:18.670+0000 7f9f97e81700 1 --2- 172.21.15.32:0/3367937775 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f90066ed0 0x7f9f900ff9a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:18.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:18.670+0000 7f9f97e81700 1 -- 172.21.15.32:0/3367937775 >> 172.21.15.32:0/3367937775 conn(0x7f9f900f7430 msgr2=0x7f9f900f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:09:18.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:18.670+0000 7f9f97e81700 1 -- 172.21.15.32:0/3367937775 shutdown_connections 2024-05-12T04:09:18.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:18.670+0000 7f9f97e81700 1 -- 172.21.15.32:0/3367937775 wait complete. 2024-05-12T04:09:18.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:18.670+0000 7f9f97e81700 1 Processor -- start 2024-05-12T04:09:18.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:18.670+0000 7f9f97e81700 1 -- start start 2024-05-12T04:09:18.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:18.671+0000 7f9f97e81700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f90066ed0 0x7f9f9010d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:09:18.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:18.671+0000 7f9f97e81700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f900ffee0 0x7f9f9010dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:09:18.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:18.671+0000 7f9f97e81700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9010e350 con 0x7f9f90066ed0 2024-05-12T04:09:18.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:18.671+0000 7f9f97e81700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9010e4c0 con 0x7f9f900ffee0 2024-05-12T04:09:18.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:18.671+0000 7f9f95c1d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f90066ed0 0x7f9f9010d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:09:18.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:18.671+0000 7f9f9541c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f900ffee0 0x7f9f9010dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:09:18.673 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:18.671+0000 7f9f95c1d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f90066ed0 0x7f9f9010d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55796/0 (socket says 172.21.15.32:55796) 2024-05-12T04:09:18.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:18.671+0000 7f9f95c1d700 1 -- 172.21.15.32:0/2218831252 learned_addr learned my addr 172.21.15.32:0/2218831252 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:09:18.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:18.671+0000 7f9f95c1d700 1 -- 172.21.15.32:0/2218831252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f900ffee0 msgr2=0x7f9f9010dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:09:18.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:18.671+0000 7f9f95c1d700 1 --2- 172.21.15.32:0/2218831252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f900ffee0 0x7f9f9010dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:18.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:18.671+0000 7f9f95c1d700 1 -- 172.21.15.32:0/2218831252 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f8c005040 con 0x7f9f90066ed0 2024-05-12T04:09:18.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:18.671+0000 7f9f9541c700 1 --2- 172.21.15.32:0/2218831252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f900ffee0 0x7f9f9010dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-12T04:09:18.674 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:18.671+0000 7f9f95c1d700 1 --2- 172.21.15.32:0/2218831252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f90066ed0 0x7f9f9010d810 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c000c00 tx=0x7f9f8c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:09:18.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:18.671+0000 7f9f867fc700 1 -- 172.21.15.32:0/2218831252 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f8c00e070 con 0x7f9f90066ed0 2024-05-12T04:09:18.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:18.671+0000 7f9f867fc700 1 -- 172.21.15.32:0/2218831252 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9f8c0093b0 con 0x7f9f90066ed0 2024-05-12T04:09:18.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:18.671+0000 7f9f97e81700 1 -- 172.21.15.32:0/2218831252 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f90112ec0 con 0x7f9f90066ed0 2024-05-12T04:09:18.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:18.671+0000 7f9f867fc700 1 -- 172.21.15.32:0/2218831252 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f8c01cd70 con 0x7f9f90066ed0 2024-05-12T04:09:18.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:18.671+0000 7f9f97e81700 1 -- 172.21.15.32:0/2218831252 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f901133a0 con 0x7f9f90066ed0 2024-05-12T04:09:18.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:18.672+0000 7f9f97e81700 1 -- 172.21.15.32:0/2218831252 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f9004c710 con 0x7f9f90066ed0 2024-05-12T04:09:18.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:18.673+0000 7f9f867fc700 1 -- 172.21.15.32:0/2218831252 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f9f8c01c600 con 0x7f9f90066ed0 2024-05-12T04:09:18.675 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:18.674+0000 7f9f867fc700 1 --2- 172.21.15.32:0/2218831252 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f9f7c06cc20 0x7f9f7c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:09:18.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:18.674+0000 7f9f867fc700 1 -- 172.21.15.32:0/2218831252 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f9f8c08e2c0 con 0x7f9f90066ed0 2024-05-12T04:09:18.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:18.674+0000 7f9f9541c700 1 --2- 172.21.15.32:0/2218831252 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f9f7c06cc20 0x7f9f7c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:09:18.676 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:18.675+0000 7f9f9541c700 1 --2- 172.21.15.32:0/2218831252 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f9f7c06cc20 0x7f9f7c06f0d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9f80009b80 tx=0x7f9f8000b040 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:09:18.679 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:18.678+0000 7f9f867fc700 1 -- 172.21.15.32:0/2218831252 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f9f8c05be30 con 0x7f9f90066ed0 2024-05-12T04:09:18.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:18.982+0000 7f9f97e81700 1 -- 172.21.15.32:0/2218831252 --> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9f900f8f20 con 0x7f9f7c06cc20 2024-05-12T04:09:18.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:18.983+0000 7f9f867fc700 1 -- 172.21.15.32:0/2218831252 <== mgr.24593 v2:172.21.15.32:6800/2957111591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7f9f900f8f20 con 0x7f9f7c06cc20 2024-05-12T04:09:18.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:18.986+0000 7f9f97e81700 1 -- 172.21.15.32:0/2218831252 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f9f7c06cc20 msgr2=0x7f9f7c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:09:18.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:18.986+0000 7f9f97e81700 1 --2- 172.21.15.32:0/2218831252 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f9f7c06cc20 0x7f9f7c06f0d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9f80009b80 tx=0x7f9f8000b040 comp rx=0 tx=0).stop 2024-05-12T04:09:18.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:18.986+0000 7f9f97e81700 1 -- 172.21.15.32:0/2218831252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f90066ed0 msgr2=0x7f9f9010d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:09:18.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:18.986+0000 7f9f97e81700 1 --2- 172.21.15.32:0/2218831252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f90066ed0 0x7f9f9010d810 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c000c00 tx=0x7f9f8c009e30 comp rx=0 tx=0).stop 2024-05-12T04:09:18.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:18.986+0000 7f9f97e81700 1 -- 172.21.15.32:0/2218831252 shutdown_connections 2024-05-12T04:09:18.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:18.986+0000 7f9f97e81700 1 --2- 172.21.15.32:0/2218831252 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f9f7c06cc20 0x7f9f7c06f0d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:18.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:18.986+0000 7f9f97e81700 1 --2- 172.21.15.32:0/2218831252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f90066ed0 0x7f9f9010d810 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:18.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:18.986+0000 7f9f97e81700 1 --2- 172.21.15.32:0/2218831252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f900ffee0 0x7f9f9010dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:18.987 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:18.986+0000 7f9f97e81700 1 -- 172.21.15.32:0/2218831252 >> 172.21.15.32:0/2218831252 conn(0x7f9f900f7430 msgr2=0x7f9f900f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:09:18.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:18.986+0000 7f9f97e81700 1 -- 172.21.15.32:0/2218831252 shutdown_connections 2024-05-12T04:09:18.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:18.986+0000 7f9f97e81700 1 -- 172.21.15.32:0/2218831252 wait complete. 2024-05-12T04:09:18.997 INFO:teuthology.orchestra.run.smithi032.stdout:true 2024-05-12T04:09:19.035 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:19.035 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:19.036 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:18 smithi033 ceph-mon[107305]: pgmap v219: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-05-12T04:09:19.036 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:19.036 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:19.039 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:19.039 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:19.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:18 smithi032 ceph-mon[120392]: pgmap v219: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-05-12T04:09:19.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:19.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:19.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.090+0000 7f42aea76700 1 -- 172.21.15.32:0/2567995513 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42a80fbd40 msgr2=0x7f42a80fc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:09:19.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.090+0000 7f42aea76700 1 --2- 172.21.15.32:0/2567995513 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42a80fbd40 0x7f42a80fc150 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f429c004940 tx=0x7f429c009e30 comp rx=0 tx=0).stop 2024-05-12T04:09:19.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.091+0000 7f42aea76700 1 -- 172.21.15.32:0/2567995513 shutdown_connections 2024-05-12T04:09:19.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.091+0000 7f42aea76700 1 --2- 172.21.15.32:0/2567995513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42a80fcff0 0x7f42a80fd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:19.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.091+0000 7f42aea76700 1 --2- 172.21.15.32:0/2567995513 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42a80fbd40 0x7f42a80fc150 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:19.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.091+0000 7f42aea76700 1 -- 172.21.15.32:0/2567995513 >> 172.21.15.32:0/2567995513 conn(0x7f42a80f74c0 msgr2=0x7f42a80f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:09:19.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.091+0000 7f42aea76700 1 -- 172.21.15.32:0/2567995513 shutdown_connections 2024-05-12T04:09:19.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.091+0000 7f42aea76700 1 -- 172.21.15.32:0/2567995513 wait complete. 2024-05-12T04:09:19.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.091+0000 7f42aea76700 1 Processor -- start 2024-05-12T04:09:19.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.091+0000 7f42aea76700 1 -- start start 2024-05-12T04:09:19.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.092+0000 7f42aea76700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42a80fbd40 0x7f42a81105f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:09:19.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.092+0000 7f42aea76700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42a80fcff0 0x7f42a8112b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:09:19.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.092+0000 7f42aea76700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a8066be0 con 0x7f42a80fbd40 2024-05-12T04:09:19.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.092+0000 7f42aea76700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a8066d20 con 0x7f42a80fcff0 2024-05-12T04:09:19.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.092+0000 7f42ac812700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42a80fbd40 0x7f42a81105f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:09:19.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.092+0000 7f42ac812700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42a80fbd40 0x7f42a81105f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55818/0 (socket says 172.21.15.32:55818) 2024-05-12T04:09:19.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.092+0000 7f42ac812700 1 -- 172.21.15.32:0/3043607386 learned_addr learned my addr 172.21.15.32:0/3043607386 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:09:19.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.092+0000 7f42a7fff700 1 --2- 172.21.15.32:0/3043607386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42a80fcff0 0x7f42a8112b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:09:19.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.092+0000 7f42ac812700 1 -- 172.21.15.32:0/3043607386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42a80fcff0 msgr2=0x7f42a8112b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:09:19.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.092+0000 7f42ac812700 1 --2- 172.21.15.32:0/3043607386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42a80fcff0 0x7f42a8112b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:19.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.092+0000 7f42ac812700 1 -- 172.21.15.32:0/3043607386 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f429c005040 con 0x7f42a80fbd40 2024-05-12T04:09:19.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.092+0000 7f42ac812700 1 --2- 172.21.15.32:0/3043607386 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42a80fbd40 0x7f42a81105f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f429c00b3b0 tx=0x7f429c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:09:19.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.093+0000 7f42a57fa700 1 -- 172.21.15.32:0/3043607386 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f429c00e040 con 0x7f42a80fbd40 2024-05-12T04:09:19.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.093+0000 7f42a57fa700 1 -- 172.21.15.32:0/3043607386 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f429c00bc50 con 0x7f42a80fbd40 2024-05-12T04:09:19.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.093+0000 7f42a57fa700 1 -- 172.21.15.32:0/3043607386 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f429c01c630 con 0x7f42a80fbd40 2024-05-12T04:09:19.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.093+0000 7f42aea76700 1 -- 172.21.15.32:0/3043607386 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42a81130e0 con 0x7f42a80fbd40 2024-05-12T04:09:19.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.093+0000 7f42aea76700 1 -- 172.21.15.32:0/3043607386 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42a8113620 con 0x7f42a80fbd40 2024-05-12T04:09:19.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.094+0000 7f42aea76700 1 -- 172.21.15.32:0/3043607386 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42a805f5c0 con 0x7f42a80fbd40 2024-05-12T04:09:19.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.094+0000 7f42a57fa700 1 -- 172.21.15.32:0/3043607386 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f429c013420 con 0x7f42a80fbd40 2024-05-12T04:09:19.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.094+0000 7f42a57fa700 1 --2- 172.21.15.32:0/3043607386 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f429806cc20 0x7f429806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:09:19.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.095+0000 7f42a57fa700 1 -- 172.21.15.32:0/3043607386 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f429c08cc80 con 0x7f42a80fbd40 2024-05-12T04:09:19.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.095+0000 7f42a7fff700 1 --2- 172.21.15.32:0/3043607386 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f429806cc20 0x7f429806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:09:19.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.095+0000 7f42a7fff700 1 --2- 172.21.15.32:0/3043607386 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f429806cc20 0x7f429806f0d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4294004f40 tx=0x7f429400b040 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:09:19.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.099+0000 7f42a57fa700 1 -- 172.21.15.32:0/3043607386 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f429c056c60 con 0x7f42a80fbd40 2024-05-12T04:09:19.343 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.342+0000 7f42aea76700 1 -- 172.21.15.32:0/3043607386 --> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f42a81013b0 con 0x7f429806cc20 2024-05-12T04:09:19.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.343+0000 7f42a57fa700 1 -- 172.21.15.32:0/3043607386 <== mgr.24593 v2:172.21.15.32:6800/2957111591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7f42a81013b0 con 0x7f429806cc20 2024-05-12T04:09:19.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.345+0000 7f42aea76700 1 -- 172.21.15.32:0/3043607386 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f429806cc20 msgr2=0x7f429806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:09:19.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.345+0000 7f42aea76700 1 --2- 172.21.15.32:0/3043607386 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f429806cc20 0x7f429806f0d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4294004f40 tx=0x7f429400b040 comp rx=0 tx=0).stop 2024-05-12T04:09:19.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.345+0000 7f42aea76700 1 -- 172.21.15.32:0/3043607386 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42a80fbd40 msgr2=0x7f42a81105f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:09:19.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.345+0000 7f42aea76700 1 --2- 172.21.15.32:0/3043607386 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42a80fbd40 0x7f42a81105f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f429c00b3b0 tx=0x7f429c009e30 comp rx=0 tx=0).stop 2024-05-12T04:09:19.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.345+0000 7f42aea76700 1 -- 172.21.15.32:0/3043607386 shutdown_connections 2024-05-12T04:09:19.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.345+0000 7f42aea76700 1 --2- 172.21.15.32:0/3043607386 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f429806cc20 0x7f429806f0d0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:19.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.345+0000 7f42aea76700 1 --2- 172.21.15.32:0/3043607386 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42a80fbd40 0x7f42a81105f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:19.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.345+0000 7f42aea76700 1 --2- 172.21.15.32:0/3043607386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42a80fcff0 0x7f42a8112b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:19.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.345+0000 7f42aea76700 1 -- 172.21.15.32:0/3043607386 >> 172.21.15.32:0/3043607386 conn(0x7f42a80f74c0 msgr2=0x7f42a80f8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:09:19.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.345+0000 7f42aea76700 1 -- 172.21.15.32:0/3043607386 shutdown_connections 2024-05-12T04:09:19.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.346+0000 7f42aea76700 1 -- 172.21.15.32:0/3043607386 wait complete. 2024-05-12T04:09:19.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.467+0000 7f6c83cd2700 1 -- 172.21.15.32:0/3811829219 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c7c0fd9c0 msgr2=0x7f6c7c0ffda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:09:19.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.467+0000 7f6c83cd2700 1 --2- 172.21.15.32:0/3811829219 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c7c0fd9c0 0x7f6c7c0ffda0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6c78004940 tx=0x7f6c78009e30 comp rx=0 tx=0).stop 2024-05-12T04:09:19.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.467+0000 7f6c83cd2700 1 -- 172.21.15.32:0/3811829219 shutdown_connections 2024-05-12T04:09:19.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.467+0000 7f6c83cd2700 1 --2- 172.21.15.32:0/3811829219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c7c1002e0 0x7f6c7c1026c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:19.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.467+0000 7f6c83cd2700 1 --2- 172.21.15.32:0/3811829219 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c7c0fd9c0 0x7f6c7c0ffda0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:19.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.467+0000 7f6c83cd2700 1 -- 172.21.15.32:0/3811829219 >> 172.21.15.32:0/3811829219 conn(0x7f6c7c0f7440 msgr2=0x7f6c7c0f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:09:19.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.467+0000 7f6c83cd2700 1 -- 172.21.15.32:0/3811829219 shutdown_connections 2024-05-12T04:09:19.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.467+0000 7f6c83cd2700 1 -- 172.21.15.32:0/3811829219 wait complete. 2024-05-12T04:09:19.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.468+0000 7f6c83cd2700 1 Processor -- start 2024-05-12T04:09:19.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.468+0000 7f6c83cd2700 1 -- start start 2024-05-12T04:09:19.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.468+0000 7f6c83cd2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c7c0fd9c0 0x7f6c7c10d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:09:19.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.468+0000 7f6c83cd2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c7c1002e0 0x7f6c7c10de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:09:19.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.468+0000 7f6c83cd2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7c10e450 con 0x7f6c7c0fd9c0 2024-05-12T04:09:19.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.468+0000 7f6c83cd2700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7c10eca0 con 0x7f6c7c1002e0 2024-05-12T04:09:19.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.468+0000 7f6c81a6e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c7c0fd9c0 0x7f6c7c10d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:09:19.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.468+0000 7f6c81a6e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c7c0fd9c0 0x7f6c7c10d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55828/0 (socket says 172.21.15.32:55828) 2024-05-12T04:09:19.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.468+0000 7f6c8126d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c7c1002e0 0x7f6c7c10de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:09:19.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.468+0000 7f6c81a6e700 1 -- 172.21.15.32:0/1377191884 learned_addr learned my addr 172.21.15.32:0/1377191884 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:09:19.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.468+0000 7f6c81a6e700 1 -- 172.21.15.32:0/1377191884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c7c1002e0 msgr2=0x7f6c7c10de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:09:19.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.468+0000 7f6c81a6e700 1 --2- 172.21.15.32:0/1377191884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c7c1002e0 0x7f6c7c10de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:19.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.468+0000 7f6c81a6e700 1 -- 172.21.15.32:0/1377191884 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c78005040 con 0x7f6c7c0fd9c0 2024-05-12T04:09:19.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.468+0000 7f6c8126d700 1 --2- 172.21.15.32:0/1377191884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c7c1002e0 0x7f6c7c10de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-12T04:09:19.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.468+0000 7f6c81a6e700 1 --2- 172.21.15.32:0/1377191884 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c7c0fd9c0 0x7f6c7c10d910 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6c78000c00 tx=0x7f6c78009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:09:19.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.469+0000 7f6c727fc700 1 -- 172.21.15.32:0/1377191884 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c7800e070 con 0x7f6c7c0fd9c0 2024-05-12T04:09:19.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.469+0000 7f6c727fc700 1 -- 172.21.15.32:0/1377191884 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6c780093b0 con 0x7f6c7c0fd9c0 2024-05-12T04:09:19.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.469+0000 7f6c83cd2700 1 -- 172.21.15.32:0/1377191884 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c7c10ef20 con 0x7f6c7c0fd9c0 2024-05-12T04:09:19.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.469+0000 7f6c727fc700 1 -- 172.21.15.32:0/1377191884 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c7801cd70 con 0x7f6c7c0fd9c0 2024-05-12T04:09:19.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.469+0000 7f6c83cd2700 1 -- 172.21.15.32:0/1377191884 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c7c10f400 con 0x7f6c7c0fd9c0 2024-05-12T04:09:19.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.470+0000 7f6c83cd2700 1 -- 172.21.15.32:0/1377191884 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c7c05f5c0 con 0x7f6c7c0fd9c0 2024-05-12T04:09:19.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.470+0000 7f6c727fc700 1 -- 172.21.15.32:0/1377191884 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f6c7801c600 con 0x7f6c7c0fd9c0 2024-05-12T04:09:19.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.471+0000 7f6c727fc700 1 --2- 172.21.15.32:0/1377191884 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f6c6806cc30 0x7f6c6806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:09:19.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.471+0000 7f6c727fc700 1 -- 172.21.15.32:0/1377191884 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f6c7808d560 con 0x7f6c7c0fd9c0 2024-05-12T04:09:19.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.471+0000 7f6c8126d700 1 --2- 172.21.15.32:0/1377191884 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f6c6806cc30 0x7f6c6806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:09:19.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.472+0000 7f6c8126d700 1 --2- 172.21.15.32:0/1377191884 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f6c6806cc30 0x7f6c6806f0e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c004fd0 tx=0x7f6c6c00b040 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:09:19.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.476+0000 7f6c727fc700 1 -- 172.21.15.32:0/1377191884 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f6c7805b070 con 0x7f6c7c0fd9c0 2024-05-12T04:09:19.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.691+0000 7f6c83cd2700 1 -- 172.21.15.32:0/1377191884 --> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6c7c10f7e0 con 0x7f6c6806cc30 2024-05-12T04:09:19.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.701+0000 7f6c727fc700 1 -- 172.21.15.32:0/1377191884 <== mgr.24593 v2:172.21.15.32:6800/2957111591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f6c7c10f7e0 con 0x7f6c6806cc30 2024-05-12T04:09:19.734 INFO:teuthology.orchestra.run.smithi032.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-12T04:09:19.735 INFO:teuthology.orchestra.run.smithi032.stdout:alertmanager.smithi032 smithi032 *:9093,9094 running (8m) 57s ago 17m 25.5M - 0.25.0 c8568f914cd2 a37fbab7af4c 2024-05-12T04:09:19.735 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi032 smithi032 running (5m) 57s ago 17m 6660k - 18.2.2-1272-g58249552 ddcb0a9d6fde 8c53f1fc7b06 2024-05-12T04:09:19.735 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi033 smithi033 running (4m) 2s ago 16m 6651k - 18.2.2-1272-g58249552 ddcb0a9d6fde cf4fe9ca070b 2024-05-12T04:09:19.735 INFO:teuthology.orchestra.run.smithi032.stdout:grafana.smithi032 smithi032 *:3000 running (7m) 57s ago 17m 90.5M - 9.4.7 954c08fa6188 83055e58d23c 2024-05-12T04:09:19.735 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.ialtqb smithi032 running (84s) 57s ago 11m 13.8M - 18.2.2-1272-g58249552 ddcb0a9d6fde 532dd9845ce7 2024-05-12T04:09:19.735 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.qpsawh smithi032 running (62s) 57s ago 11m 13.6M - 18.2.2-1272-g58249552 ddcb0a9d6fde 68e735fd79ee 2024-05-12T04:09:19.735 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi033.hglwml smithi033 running (22s) 2s ago 11m 14.9M - 18.2.2-1272-g58249552 ddcb0a9d6fde 7240e684b225 2024-05-12T04:09:19.735 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi033.jgbjhk smithi033 running (44s) 2s ago 11m 14.1M - 18.2.2-1272-g58249552 ddcb0a9d6fde b738dbbbec0f 2024-05-12T04:09:19.735 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi032.ukatkq smithi032 *:8443,9283,8765 running (7m) 57s ago 18m 506M - 18.2.2-1272-g58249552 ddcb0a9d6fde 0d88ab9a5292 2024-05-12T04:09:19.735 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi033.pcvxwy smithi033 *:8443,9283,8765 running (6m) 2s ago 16m 451M - 18.2.2-1272-g58249552 ddcb0a9d6fde d42dcb6bd307 2024-05-12T04:09:19.735 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi032 smithi032 running (6m) 57s ago 19m 48.6M 2048M 18.2.2-1272-g58249552 ddcb0a9d6fde 81f94171c5dc 2024-05-12T04:09:19.735 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi033 smithi033 running (5m) 2s ago 16m 42.9M 2048M 18.2.2-1272-g58249552 ddcb0a9d6fde d648ba108f3e 2024-05-12T04:09:19.735 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi032 smithi032 *:9100 starting - - - - 2024-05-12T04:09:19.735 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi033 smithi033 *:9100 running (4s) 2s ago 16m 3632k - 1.5.0 0da6a335fe13 b75dd225ce4b 2024-05-12T04:09:19.736 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0 smithi032 running (4m) 57s ago 15m 90.3M 1570M 18.2.2-1272-g58249552 ddcb0a9d6fde 07f66c4deb3b 2024-05-12T04:09:19.736 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1 smithi032 running (3m) 57s ago 14m 96.5M 1570M 18.2.2-1272-g58249552 ddcb0a9d6fde 67a87807d654 2024-05-12T04:09:19.736 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2 smithi032 running (3m) 57s ago 14m 85.2M 1570M 18.2.2-1272-g58249552 ddcb0a9d6fde 4ac48d74ee8b 2024-05-12T04:09:19.736 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3 smithi033 running (2m) 2s ago 14m 103M 2594M 18.2.2-1272-g58249552 ddcb0a9d6fde 11f097a455bb 2024-05-12T04:09:19.736 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4 smithi033 running (2m) 2s ago 13m 91.5M 2594M 18.2.2-1272-g58249552 ddcb0a9d6fde db6fda7d52de 2024-05-12T04:09:19.736 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5 smithi033 running (102s) 2s ago 13m 92.7M 2594M 18.2.2-1272-g58249552 ddcb0a9d6fde 3333020b8a15 2024-05-12T04:09:19.736 INFO:teuthology.orchestra.run.smithi032.stdout:prometheus.smithi032 smithi032 *:9095 running (6m) 57s ago 16m 61.3M - 2.43.0 a07b618ecd1d 46eafb25e176 2024-05-12T04:09:19.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.704+0000 7f6c83cd2700 1 -- 172.21.15.32:0/1377191884 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f6c6806cc30 msgr2=0x7f6c6806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:09:19.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.704+0000 7f6c83cd2700 1 --2- 172.21.15.32:0/1377191884 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f6c6806cc30 0x7f6c6806f0e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c004fd0 tx=0x7f6c6c00b040 comp rx=0 tx=0).stop 2024-05-12T04:09:19.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.704+0000 7f6c83cd2700 1 -- 172.21.15.32:0/1377191884 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c7c0fd9c0 msgr2=0x7f6c7c10d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:09:19.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.704+0000 7f6c83cd2700 1 --2- 172.21.15.32:0/1377191884 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c7c0fd9c0 0x7f6c7c10d910 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6c78000c00 tx=0x7f6c78009e30 comp rx=0 tx=0).stop 2024-05-12T04:09:19.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.704+0000 7f6c83cd2700 1 -- 172.21.15.32:0/1377191884 shutdown_connections 2024-05-12T04:09:19.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.704+0000 7f6c83cd2700 1 --2- 172.21.15.32:0/1377191884 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f6c6806cc30 0x7f6c6806f0e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:19.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.704+0000 7f6c83cd2700 1 --2- 172.21.15.32:0/1377191884 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c7c0fd9c0 0x7f6c7c10d910 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:19.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.704+0000 7f6c83cd2700 1 --2- 172.21.15.32:0/1377191884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c7c1002e0 0x7f6c7c10de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:19.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.704+0000 7f6c83cd2700 1 -- 172.21.15.32:0/1377191884 >> 172.21.15.32:0/1377191884 conn(0x7f6c7c0f7440 msgr2=0x7f6c7c0f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:09:19.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.704+0000 7f6c83cd2700 1 -- 172.21.15.32:0/1377191884 shutdown_connections 2024-05-12T04:09:19.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.704+0000 7f6c83cd2700 1 -- 172.21.15.32:0/1377191884 wait complete. 2024-05-12T04:09:19.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:19 smithi032 ceph-mon[120392]: from='client.34472 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:09:19.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:19 smithi032 ceph-mon[120392]: from='client.34476 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:09:19.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.807+0000 7fd208e80700 1 -- 172.21.15.32:0/533248206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2040fdfd0 msgr2=0x7fd2040fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:09:19.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.807+0000 7fd208e80700 1 --2- 172.21.15.32:0/533248206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2040fdfd0 0x7fd2040fe440 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec004940 tx=0x7fd1ec009e30 comp rx=0 tx=0).stop 2024-05-12T04:09:19.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.807+0000 7fd208e80700 1 -- 172.21.15.32:0/533248206 shutdown_connections 2024-05-12T04:09:19.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.807+0000 7fd208e80700 1 --2- 172.21.15.32:0/533248206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2040fdfd0 0x7fd2040fe440 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:19.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.807+0000 7fd208e80700 1 --2- 172.21.15.32:0/533248206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2040fce60 0x7fd2040fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:19.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.807+0000 7fd208e80700 1 -- 172.21.15.32:0/533248206 >> 172.21.15.32:0/533248206 conn(0x7fd2040f8560 msgr2=0x7fd2040fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:09:19.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.807+0000 7fd208e80700 1 -- 172.21.15.32:0/533248206 shutdown_connections 2024-05-12T04:09:19.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.807+0000 7fd208e80700 1 -- 172.21.15.32:0/533248206 wait complete. 2024-05-12T04:09:19.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.808+0000 7fd208e80700 1 Processor -- start 2024-05-12T04:09:19.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.808+0000 7fd208e80700 1 -- start start 2024-05-12T04:09:19.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.808+0000 7fd208e80700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2040fce60 0x7fd20410d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:09:19.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.808+0000 7fd208e80700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2040fdfd0 0x7fd20410de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:09:19.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.808+0000 7fd208e80700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd20410e460 con 0x7fd2040fdfd0 2024-05-12T04:09:19.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.808+0000 7fd208e80700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd20410ecb0 con 0x7fd2040fce60 2024-05-12T04:09:19.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.808+0000 7fd201d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2040fdfd0 0x7fd20410de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:09:19.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.808+0000 7fd201d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2040fdfd0 0x7fd20410de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55846/0 (socket says 172.21.15.32:55846) 2024-05-12T04:09:19.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.809+0000 7fd201d9b700 1 -- 172.21.15.32:0/4184372388 learned_addr learned my addr 172.21.15.32:0/4184372388 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:09:19.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.809+0000 7fd20259c700 1 --2- 172.21.15.32:0/4184372388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2040fce60 0x7fd20410d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:09:19.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.809+0000 7fd201d9b700 1 -- 172.21.15.32:0/4184372388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2040fce60 msgr2=0x7fd20410d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:09:19.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.809+0000 7fd201d9b700 1 --2- 172.21.15.32:0/4184372388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2040fce60 0x7fd20410d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:19.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.809+0000 7fd201d9b700 1 -- 172.21.15.32:0/4184372388 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1ec005040 con 0x7fd2040fdfd0 2024-05-12T04:09:19.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.809+0000 7fd201d9b700 1 --2- 172.21.15.32:0/4184372388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2040fdfd0 0x7fd20410de60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec009fd0 tx=0x7fd1ec009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:09:19.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.809+0000 7fd1faffd700 1 -- 172.21.15.32:0/4184372388 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd1ec00e040 con 0x7fd2040fdfd0 2024-05-12T04:09:19.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.809+0000 7fd208e80700 1 -- 172.21.15.32:0/4184372388 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd20410ef30 con 0x7fd2040fdfd0 2024-05-12T04:09:19.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.809+0000 7fd1faffd700 1 -- 172.21.15.32:0/4184372388 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd1ec004290 con 0x7fd2040fdfd0 2024-05-12T04:09:19.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.809+0000 7fd1faffd700 1 -- 172.21.15.32:0/4184372388 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd1ec01c3f0 con 0x7fd2040fdfd0 2024-05-12T04:09:19.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.809+0000 7fd208e80700 1 -- 172.21.15.32:0/4184372388 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd20410f470 con 0x7fd2040fdfd0 2024-05-12T04:09:19.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.810+0000 7fd208e80700 1 -- 172.21.15.32:0/4184372388 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd20405f5c0 con 0x7fd2040fdfd0 2024-05-12T04:09:19.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.811+0000 7fd1faffd700 1 -- 172.21.15.32:0/4184372388 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fd1ec013790 con 0x7fd2040fdfd0 2024-05-12T04:09:19.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.811+0000 7fd1faffd700 1 --2- 172.21.15.32:0/4184372388 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fd1f006cc30 0x7fd1f006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:09:19.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.811+0000 7fd1faffd700 1 -- 172.21.15.32:0/4184372388 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fd1ec08d4b0 con 0x7fd2040fdfd0 2024-05-12T04:09:19.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.811+0000 7fd20259c700 1 --2- 172.21.15.32:0/4184372388 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fd1f006cc30 0x7fd1f006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:09:19.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.812+0000 7fd20259c700 1 --2- 172.21.15.32:0/4184372388 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fd1f006cc30 0x7fd1f006f0e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4004f40 tx=0x7fd1f400b040 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:09:19.818 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:19.816+0000 7fd1faffd700 1 -- 172.21.15.32:0/4184372388 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fd1ec05af20 con 0x7fd2040fdfd0 2024-05-12T04:09:19.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:19 smithi033 ceph-mon[107305]: from='client.34472 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:09:19.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:19 smithi033 ceph-mon[107305]: from='client.34476 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:09:20.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.126+0000 7fd208e80700 1 -- 172.21.15.32:0/4184372388 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd20404bed0 con 0x7fd2040fdfd0 2024-05-12T04:09:20.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.126+0000 7fd1faffd700 1 -- 172.21.15.32:0/4184372388 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fd1ec013420 con 0x7fd2040fdfd0 2024-05-12T04:09:20.131 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-05-12T04:09:20.131 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": { 2024-05-12T04:09:20.132 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 2 2024-05-12T04:09:20.132 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:09:20.132 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": { 2024-05-12T04:09:20.132 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 2 2024-05-12T04:09:20.132 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:09:20.132 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": { 2024-05-12T04:09:20.132 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 6 2024-05-12T04:09:20.132 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:09:20.132 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": { 2024-05-12T04:09:20.132 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 4 2024-05-12T04:09:20.132 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:09:20.132 INFO:teuthology.orchestra.run.smithi032.stdout: "overall": { 2024-05-12T04:09:20.132 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 14 2024-05-12T04:09:20.132 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-05-12T04:09:20.132 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-05-12T04:09:20.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.129+0000 7fd208e80700 1 -- 172.21.15.32:0/4184372388 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fd1f006cc30 msgr2=0x7fd1f006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:09:20.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.129+0000 7fd208e80700 1 --2- 172.21.15.32:0/4184372388 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fd1f006cc30 0x7fd1f006f0e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4004f40 tx=0x7fd1f400b040 comp rx=0 tx=0).stop 2024-05-12T04:09:20.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.129+0000 7fd208e80700 1 -- 172.21.15.32:0/4184372388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2040fdfd0 msgr2=0x7fd20410de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:09:20.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.129+0000 7fd208e80700 1 --2- 172.21.15.32:0/4184372388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2040fdfd0 0x7fd20410de60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec009fd0 tx=0x7fd1ec009e30 comp rx=0 tx=0).stop 2024-05-12T04:09:20.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.129+0000 7fd208e80700 1 -- 172.21.15.32:0/4184372388 shutdown_connections 2024-05-12T04:09:20.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.129+0000 7fd208e80700 1 --2- 172.21.15.32:0/4184372388 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fd1f006cc30 0x7fd1f006f0e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:20.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.129+0000 7fd208e80700 1 --2- 172.21.15.32:0/4184372388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2040fce60 0x7fd20410d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:20.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.129+0000 7fd208e80700 1 --2- 172.21.15.32:0/4184372388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2040fdfd0 0x7fd20410de60 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:20.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.129+0000 7fd208e80700 1 -- 172.21.15.32:0/4184372388 >> 172.21.15.32:0/4184372388 conn(0x7fd2040f8560 msgr2=0x7fd2040f9d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:09:20.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.129+0000 7fd208e80700 1 -- 172.21.15.32:0/4184372388 shutdown_connections 2024-05-12T04:09:20.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.129+0000 7fd208e80700 1 -- 172.21.15.32:0/4184372388 wait complete. 2024-05-12T04:09:20.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.238+0000 7f8892883700 1 -- 172.21.15.32:0/951503065 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f888c102350 msgr2=0x7f888c104730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:09:20.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.238+0000 7f8892883700 1 --2- 172.21.15.32:0/951503065 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f888c102350 0x7f888c104730 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8874004940 tx=0x7f8874009e30 comp rx=0 tx=0).stop 2024-05-12T04:09:20.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.238+0000 7f8892883700 1 -- 172.21.15.32:0/951503065 shutdown_connections 2024-05-12T04:09:20.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.238+0000 7f8892883700 1 --2- 172.21.15.32:0/951503065 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f888c102350 0x7f888c104730 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:20.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.238+0000 7f8892883700 1 --2- 172.21.15.32:0/951503065 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f888c066af0 0x7f888c101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:20.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.238+0000 7f8892883700 1 -- 172.21.15.32:0/951503065 >> 172.21.15.32:0/951503065 conn(0x7f888c0f7420 msgr2=0x7f888c0f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:09:20.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.238+0000 7f8892883700 1 -- 172.21.15.32:0/951503065 shutdown_connections 2024-05-12T04:09:20.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.238+0000 7f8892883700 1 -- 172.21.15.32:0/951503065 wait complete. 2024-05-12T04:09:20.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.239+0000 7f8892883700 1 Processor -- start 2024-05-12T04:09:20.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.239+0000 7f8892883700 1 -- start start 2024-05-12T04:09:20.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.239+0000 7f8892883700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f888c066af0 0x7f888c10fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:09:20.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.239+0000 7f8892883700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f888c102350 0x7f888c110060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:09:20.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.239+0000 7f8892883700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f888c110660 con 0x7f888c066af0 2024-05-12T04:09:20.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.239+0000 7f8892883700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f888c110e40 con 0x7f888c102350 2024-05-12T04:09:20.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.239+0000 7f888bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f888c066af0 0x7f888c10fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:09:20.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.239+0000 7f888bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f888c066af0 0x7f888c10fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55862/0 (socket says 172.21.15.32:55862) 2024-05-12T04:09:20.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.239+0000 7f888bfff700 1 -- 172.21.15.32:0/3050311829 learned_addr learned my addr 172.21.15.32:0/3050311829 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:09:20.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.239+0000 7f8883fff700 1 --2- 172.21.15.32:0/3050311829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f888c102350 0x7f888c110060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:09:20.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.239+0000 7f888bfff700 1 -- 172.21.15.32:0/3050311829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f888c102350 msgr2=0x7f888c110060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:09:20.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.239+0000 7f888bfff700 1 --2- 172.21.15.32:0/3050311829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f888c102350 0x7f888c110060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:20.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.239+0000 7f888bfff700 1 -- 172.21.15.32:0/3050311829 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8874005040 con 0x7f888c066af0 2024-05-12T04:09:20.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.239+0000 7f888bfff700 1 --2- 172.21.15.32:0/3050311829 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f888c066af0 0x7f888c10fb20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f887c002a80 tx=0x7f887c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:09:20.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.240+0000 7f88897fa700 1 -- 172.21.15.32:0/3050311829 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f887c007cc0 con 0x7f888c066af0 2024-05-12T04:09:20.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.240+0000 7f88897fa700 1 -- 172.21.15.32:0/3050311829 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f887c00f040 con 0x7f888c066af0 2024-05-12T04:09:20.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.240+0000 7f8892883700 1 -- 172.21.15.32:0/3050311829 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f888c111120 con 0x7f888c066af0 2024-05-12T04:09:20.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.240+0000 7f88897fa700 1 -- 172.21.15.32:0/3050311829 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f887c00da20 con 0x7f888c066af0 2024-05-12T04:09:20.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.240+0000 7f8892883700 1 -- 172.21.15.32:0/3050311829 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f888c111660 con 0x7f888c066af0 2024-05-12T04:09:20.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.242+0000 7f88897fa700 1 -- 172.21.15.32:0/3050311829 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f887c00db80 con 0x7f888c066af0 2024-05-12T04:09:20.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.242+0000 7f8892883700 1 -- 172.21.15.32:0/3050311829 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f888c05f5c0 con 0x7f888c066af0 2024-05-12T04:09:20.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.242+0000 7f88897fa700 1 --2- 172.21.15.32:0/3050311829 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f887806cc30 0x7f887806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:09:20.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.242+0000 7f88897fa700 1 -- 172.21.15.32:0/3050311829 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f887c08b630 con 0x7f888c066af0 2024-05-12T04:09:20.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.242+0000 7f8883fff700 1 --2- 172.21.15.32:0/3050311829 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f887806cc30 0x7f887806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:09:20.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.243+0000 7f8883fff700 1 --2- 172.21.15.32:0/3050311829 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f887806cc30 0x7f887806f0e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f887400b3b0 tx=0x7f8874009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:09:20.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.247+0000 7f88897fa700 1 -- 172.21.15.32:0/3050311829 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f887c059010 con 0x7f888c066af0 2024-05-12T04:09:20.510 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.509+0000 7f8892883700 1 -- 172.21.15.32:0/3050311829 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f888c107960 con 0x7f888c066af0 2024-05-12T04:09:20.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.509+0000 7f88897fa700 1 -- 172.21.15.32:0/3050311829 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 37 v37) v1 ==== 76+0+1871 (secure 0 0 0) 0x7f887c058ba0 con 0x7f888c066af0 2024-05-12T04:09:20.516 INFO:teuthology.orchestra.run.smithi032.stdout:e37 2024-05-12T04:09:20.516 INFO:teuthology.orchestra.run.smithi032.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-12T04:09:20.516 INFO:teuthology.orchestra.run.smithi032.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-12T04:09:20.516 INFO:teuthology.orchestra.run.smithi032.stdout:legacy client fscid: 1 2024-05-12T04:09:20.516 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:09:20.516 INFO:teuthology.orchestra.run.smithi032.stdout:Filesystem 'cephfs' (1) 2024-05-12T04:09:20.516 INFO:teuthology.orchestra.run.smithi032.stdout:fs_name cephfs 2024-05-12T04:09:20.516 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 37 2024-05-12T04:09:20.516 INFO:teuthology.orchestra.run.smithi032.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-12T04:09:20.517 INFO:teuthology.orchestra.run.smithi032.stdout:created 2024-05-12T03:57:48.904614+0000 2024-05-12T04:09:20.517 INFO:teuthology.orchestra.run.smithi032.stdout:modified 2024-05-12T04:09:05.133553+0000 2024-05-12T04:09:20.517 INFO:teuthology.orchestra.run.smithi032.stdout:tableserver 0 2024-05-12T04:09:20.517 INFO:teuthology.orchestra.run.smithi032.stdout:root 0 2024-05-12T04:09:20.517 INFO:teuthology.orchestra.run.smithi032.stdout:session_timeout 60 2024-05-12T04:09:20.517 INFO:teuthology.orchestra.run.smithi032.stdout:session_autoclose 300 2024-05-12T04:09:20.517 INFO:teuthology.orchestra.run.smithi032.stdout:max_file_size 1099511627776 2024-05-12T04:09:20.517 INFO:teuthology.orchestra.run.smithi032.stdout:max_xattr_size 65536 2024-05-12T04:09:20.517 INFO:teuthology.orchestra.run.smithi032.stdout:required_client_features {} 2024-05-12T04:09:20.517 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure 0 2024-05-12T04:09:20.517 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure_osd_epoch 84 2024-05-12T04:09:20.517 INFO:teuthology.orchestra.run.smithi032.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-12T04:09:20.517 INFO:teuthology.orchestra.run.smithi032.stdout:max_mds 1 2024-05-12T04:09:20.517 INFO:teuthology.orchestra.run.smithi032.stdout:in 0 2024-05-12T04:09:20.517 INFO:teuthology.orchestra.run.smithi032.stdout:up {0=34404} 2024-05-12T04:09:20.517 INFO:teuthology.orchestra.run.smithi032.stdout:failed 2024-05-12T04:09:20.518 INFO:teuthology.orchestra.run.smithi032.stdout:damaged 2024-05-12T04:09:20.518 INFO:teuthology.orchestra.run.smithi032.stdout:stopped 2024-05-12T04:09:20.518 INFO:teuthology.orchestra.run.smithi032.stdout:data_pools [3] 2024-05-12T04:09:20.518 INFO:teuthology.orchestra.run.smithi032.stdout:metadata_pool 2 2024-05-12T04:09:20.518 INFO:teuthology.orchestra.run.smithi032.stdout:inline_data enabled 2024-05-12T04:09:20.518 INFO:teuthology.orchestra.run.smithi032.stdout:balancer 2024-05-12T04:09:20.518 INFO:teuthology.orchestra.run.smithi032.stdout:bal_rank_mask -1 2024-05-12T04:09:20.518 INFO:teuthology.orchestra.run.smithi032.stdout:standby_count_wanted 1 2024-05-12T04:09:20.518 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.ialtqb{0:34404} state up:active seq 20 join_fscid=1 addr [v2:172.21.15.32:6826/1080358503,v1:172.21.15.32:6827/1080358503] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:09:20.518 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.qpsawh{0:34436} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.32:6828/3603452506,v1:172.21.15.32:6829/3603452506] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:09:20.518 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:09:20.518 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:09:20.518 INFO:teuthology.orchestra.run.smithi032.stdout:Standby daemons: 2024-05-12T04:09:20.518 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:09:20.519 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi033.jgbjhk{-1:44343} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6824/997254871,v1:172.21.15.33:6825/997254871] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:09:20.519 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi033.hglwml{-1:44373} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/1983593176,v1:172.21.15.33:6827/1983593176] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:09:20.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.511+0000 7f8892883700 1 -- 172.21.15.32:0/3050311829 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f887806cc30 msgr2=0x7f887806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:09:20.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.511+0000 7f8892883700 1 --2- 172.21.15.32:0/3050311829 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f887806cc30 0x7f887806f0e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f887400b3b0 tx=0x7f8874009e30 comp rx=0 tx=0).stop 2024-05-12T04:09:20.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.511+0000 7f8892883700 1 -- 172.21.15.32:0/3050311829 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f888c066af0 msgr2=0x7f888c10fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:09:20.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.511+0000 7f8892883700 1 --2- 172.21.15.32:0/3050311829 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f888c066af0 0x7f888c10fb20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f887c002a80 tx=0x7f887c004600 comp rx=0 tx=0).stop 2024-05-12T04:09:20.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.512+0000 7f8892883700 1 -- 172.21.15.32:0/3050311829 shutdown_connections 2024-05-12T04:09:20.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.512+0000 7f8892883700 1 --2- 172.21.15.32:0/3050311829 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f887806cc30 0x7f887806f0e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:20.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.512+0000 7f8892883700 1 --2- 172.21.15.32:0/3050311829 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f888c066af0 0x7f888c10fb20 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:20.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.512+0000 7f8892883700 1 --2- 172.21.15.32:0/3050311829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f888c102350 0x7f888c110060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:20.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.512+0000 7f8892883700 1 -- 172.21.15.32:0/3050311829 >> 172.21.15.32:0/3050311829 conn(0x7f888c0f7420 msgr2=0x7f888c0f7840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:09:20.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.512+0000 7f8892883700 1 -- 172.21.15.32:0/3050311829 shutdown_connections 2024-05-12T04:09:20.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.512+0000 7f8892883700 1 -- 172.21.15.32:0/3050311829 wait complete. 2024-05-12T04:09:20.520 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 37 2024-05-12T04:09:20.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.622+0000 7f0b79fe5700 1 -- 172.21.15.32:0/858178488 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b74066240 msgr2=0x7f0b740ffa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:09:20.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.622+0000 7f0b79fe5700 1 --2- 172.21.15.32:0/858178488 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b74066240 0x7f0b740ffa20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0b64004940 tx=0x7f0b64009e30 comp rx=0 tx=0).stop 2024-05-12T04:09:20.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.622+0000 7f0b79fe5700 1 -- 172.21.15.32:0/858178488 shutdown_connections 2024-05-12T04:09:20.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.622+0000 7f0b79fe5700 1 --2- 172.21.15.32:0/858178488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b740fff60 0x7f0b74102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:20.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.622+0000 7f0b79fe5700 1 --2- 172.21.15.32:0/858178488 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b74066240 0x7f0b740ffa20 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:20.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.622+0000 7f0b79fe5700 1 -- 172.21.15.32:0/858178488 >> 172.21.15.32:0/858178488 conn(0x7f0b740f7430 msgr2=0x7f0b740f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:09:20.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.622+0000 7f0b79fe5700 1 -- 172.21.15.32:0/858178488 shutdown_connections 2024-05-12T04:09:20.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.622+0000 7f0b79fe5700 1 -- 172.21.15.32:0/858178488 wait complete. 2024-05-12T04:09:20.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.622+0000 7f0b79fe5700 1 Processor -- start 2024-05-12T04:09:20.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.622+0000 7f0b79fe5700 1 -- start start 2024-05-12T04:09:20.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.623+0000 7f0b79fe5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b74066240 0x7f0b7410d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:09:20.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.623+0000 7f0b79fe5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b740fff60 0x7f0b7410de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:09:20.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.623+0000 7f0b79fe5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7410e430 con 0x7f0b74066240 2024-05-12T04:09:20.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.623+0000 7f0b79fe5700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7410ec80 con 0x7f0b740fff60 2024-05-12T04:09:20.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.623+0000 7f0b737fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b74066240 0x7f0b7410d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:09:20.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.623+0000 7f0b72ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b740fff60 0x7f0b7410de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:09:20.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.623+0000 7f0b737fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b74066240 0x7f0b7410d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55894/0 (socket says 172.21.15.32:55894) 2024-05-12T04:09:20.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.623+0000 7f0b72ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b740fff60 0x7f0b7410de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.32:53032/0 (socket says 172.21.15.32:53032) 2024-05-12T04:09:20.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.623+0000 7f0b737fe700 1 -- 172.21.15.32:0/269806751 learned_addr learned my addr 172.21.15.32:0/269806751 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:09:20.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.623+0000 7f0b737fe700 1 -- 172.21.15.32:0/269806751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b740fff60 msgr2=0x7f0b7410de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:09:20.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.623+0000 7f0b737fe700 1 --2- 172.21.15.32:0/269806751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b740fff60 0x7f0b7410de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:20.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.623+0000 7f0b737fe700 1 -- 172.21.15.32:0/269806751 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b64005040 con 0x7f0b74066240 2024-05-12T04:09:20.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.623+0000 7f0b737fe700 1 --2- 172.21.15.32:0/269806751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b74066240 0x7f0b7410d8f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0b6400b900 tx=0x7f0b64009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:09:20.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.623+0000 7f0b5bfff700 1 -- 172.21.15.32:0/269806751 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b6400e040 con 0x7f0b74066240 2024-05-12T04:09:20.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.623+0000 7f0b79fe5700 1 -- 172.21.15.32:0/269806751 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b7410ef00 con 0x7f0b74066240 2024-05-12T04:09:20.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.623+0000 7f0b5bfff700 1 -- 172.21.15.32:0/269806751 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0b64004290 con 0x7f0b74066240 2024-05-12T04:09:20.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.623+0000 7f0b5bfff700 1 -- 172.21.15.32:0/269806751 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b6401c3f0 con 0x7f0b74066240 2024-05-12T04:09:20.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.623+0000 7f0b79fe5700 1 -- 172.21.15.32:0/269806751 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b7410f440 con 0x7f0b74066240 2024-05-12T04:09:20.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.625+0000 7f0b79fe5700 1 -- 172.21.15.32:0/269806751 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b7405f5c0 con 0x7f0b74066240 2024-05-12T04:09:20.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.625+0000 7f0b5bfff700 1 -- 172.21.15.32:0/269806751 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f0b64013420 con 0x7f0b74066240 2024-05-12T04:09:20.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.625+0000 7f0b5bfff700 1 --2- 172.21.15.32:0/269806751 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f0b5c06cc30 0x7f0b5c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:09:20.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.626+0000 7f0b5bfff700 1 -- 172.21.15.32:0/269806751 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f0b6408d390 con 0x7f0b74066240 2024-05-12T04:09:20.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.626+0000 7f0b72ffd700 1 --2- 172.21.15.32:0/269806751 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f0b5c06cc30 0x7f0b5c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:09:20.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.626+0000 7f0b72ffd700 1 --2- 172.21.15.32:0/269806751 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f0b5c06cc30 0x7f0b5c06f0e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0b68009fd0 tx=0x7f0b6800b040 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:09:20.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.630+0000 7f0b5bfff700 1 -- 172.21.15.32:0/269806751 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f0b6405ae00 con 0x7f0b74066240 2024-05-12T04:09:20.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.860+0000 7f0b79fe5700 1 -- 172.21.15.32:0/269806751 --> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0b7410f7c0 con 0x7f0b5c06cc30 2024-05-12T04:09:20.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.861+0000 7f0b5bfff700 1 -- 172.21.15.32:0/269806751 <== mgr.24593 v2:172.21.15.32:6800/2957111591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7f0b7410f7c0 con 0x7f0b5c06cc30 2024-05-12T04:09:20.863 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-05-12T04:09:20.863 INFO:teuthology.orchestra.run.smithi032.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:19c9734256aa43a60e203feb242f50bf8b7f58666d520536af43cc4fee96173a", 2024-05-12T04:09:20.863 INFO:teuthology.orchestra.run.smithi032.stdout: "in_progress": true, 2024-05-12T04:09:20.863 INFO:teuthology.orchestra.run.smithi032.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-12T04:09:20.863 INFO:teuthology.orchestra.run.smithi032.stdout: "services_complete": [ 2024-05-12T04:09:20.863 INFO:teuthology.orchestra.run.smithi032.stdout: "mds", 2024-05-12T04:09:20.863 INFO:teuthology.orchestra.run.smithi032.stdout: "osd", 2024-05-12T04:09:20.863 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr", 2024-05-12T04:09:20.863 INFO:teuthology.orchestra.run.smithi032.stdout: "mon", 2024-05-12T04:09:20.863 INFO:teuthology.orchestra.run.smithi032.stdout: "crash" 2024-05-12T04:09:20.863 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-05-12T04:09:20.864 INFO:teuthology.orchestra.run.smithi032.stdout: "progress": "16/21 daemons upgraded", 2024-05-12T04:09:20.864 INFO:teuthology.orchestra.run.smithi032.stdout: "message": "Currently upgrading node-exporter daemons", 2024-05-12T04:09:20.864 INFO:teuthology.orchestra.run.smithi032.stdout: "is_paused": false 2024-05-12T04:09:20.864 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-05-12T04:09:20.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.863+0000 7f0b79fe5700 1 -- 172.21.15.32:0/269806751 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f0b5c06cc30 msgr2=0x7f0b5c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:09:20.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.863+0000 7f0b79fe5700 1 --2- 172.21.15.32:0/269806751 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f0b5c06cc30 0x7f0b5c06f0e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0b68009fd0 tx=0x7f0b6800b040 comp rx=0 tx=0).stop 2024-05-12T04:09:20.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.863+0000 7f0b79fe5700 1 -- 172.21.15.32:0/269806751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b74066240 msgr2=0x7f0b7410d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:09:20.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.863+0000 7f0b79fe5700 1 --2- 172.21.15.32:0/269806751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b74066240 0x7f0b7410d8f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0b6400b900 tx=0x7f0b64009e30 comp rx=0 tx=0).stop 2024-05-12T04:09:20.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.864+0000 7f0b79fe5700 1 -- 172.21.15.32:0/269806751 shutdown_connections 2024-05-12T04:09:20.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.864+0000 7f0b79fe5700 1 --2- 172.21.15.32:0/269806751 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f0b5c06cc30 0x7f0b5c06f0e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:20.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.864+0000 7f0b79fe5700 1 --2- 172.21.15.32:0/269806751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b74066240 0x7f0b7410d8f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:20.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.864+0000 7f0b79fe5700 1 --2- 172.21.15.32:0/269806751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b740fff60 0x7f0b7410de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:20.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.864+0000 7f0b79fe5700 1 -- 172.21.15.32:0/269806751 >> 172.21.15.32:0/269806751 conn(0x7f0b740f7430 msgr2=0x7f0b740f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:09:20.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.864+0000 7f0b79fe5700 1 -- 172.21.15.32:0/269806751 shutdown_connections 2024-05-12T04:09:20.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.864+0000 7f0b79fe5700 1 -- 172.21.15.32:0/269806751 wait complete. 2024-05-12T04:09:20.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.974+0000 7f02c8683700 1 -- 172.21.15.32:0/3061301302 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02c0066ad0 msgr2=0x7f02c0101eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:09:20.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.974+0000 7f02c8683700 1 --2- 172.21.15.32:0/3061301302 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02c0066ad0 0x7f02c0101eb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f02bc004940 tx=0x7f02bc009e30 comp rx=0 tx=0).stop 2024-05-12T04:09:20.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.975+0000 7f02c8683700 1 -- 172.21.15.32:0/3061301302 shutdown_connections 2024-05-12T04:09:20.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.975+0000 7f02c8683700 1 --2- 172.21.15.32:0/3061301302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02c01023f0 0x7f02c01047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:20.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.975+0000 7f02c8683700 1 --2- 172.21.15.32:0/3061301302 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02c0066ad0 0x7f02c0101eb0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:20.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.975+0000 7f02c8683700 1 -- 172.21.15.32:0/3061301302 >> 172.21.15.32:0/3061301302 conn(0x7f02c00f74c0 msgr2=0x7f02c00f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:09:20.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.975+0000 7f02c8683700 1 -- 172.21.15.32:0/3061301302 shutdown_connections 2024-05-12T04:09:20.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.975+0000 7f02c8683700 1 -- 172.21.15.32:0/3061301302 wait complete. 2024-05-12T04:09:20.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.975+0000 7f02c8683700 1 Processor -- start 2024-05-12T04:09:20.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.975+0000 7f02c8683700 1 -- start start 2024-05-12T04:09:20.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.976+0000 7f02c8683700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02c0066ad0 0x7f02c010fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:09:20.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.976+0000 7f02c8683700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02c01023f0 0x7f02c0110090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:09:20.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.976+0000 7f02c8683700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c0110690 con 0x7f02c0066ad0 2024-05-12T04:09:20.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.976+0000 7f02c8683700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c0110ee0 con 0x7f02c01023f0 2024-05-12T04:09:20.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.976+0000 7f02c641f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02c0066ad0 0x7f02c010fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:09:20.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.976+0000 7f02c641f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02c0066ad0 0x7f02c010fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55906/0 (socket says 172.21.15.32:55906) 2024-05-12T04:09:20.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.976+0000 7f02c641f700 1 -- 172.21.15.32:0/1168444919 learned_addr learned my addr 172.21.15.32:0/1168444919 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:09:20.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.976+0000 7f02c5c1e700 1 --2- 172.21.15.32:0/1168444919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02c01023f0 0x7f02c0110090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:09:20.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.976+0000 7f02c641f700 1 -- 172.21.15.32:0/1168444919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02c01023f0 msgr2=0x7f02c0110090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:09:20.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.976+0000 7f02c641f700 1 --2- 172.21.15.32:0/1168444919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02c01023f0 0x7f02c0110090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:20.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.976+0000 7f02c641f700 1 -- 172.21.15.32:0/1168444919 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02bc005040 con 0x7f02c0066ad0 2024-05-12T04:09:20.983 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.976+0000 7f02c641f700 1 --2- 172.21.15.32:0/1168444919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02c0066ad0 0x7f02c010fb50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f02bc0043d0 tx=0x7f02bc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:09:20.984 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.977+0000 7f02b6ffd700 1 -- 172.21.15.32:0/1168444919 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02bc00e070 con 0x7f02c0066ad0 2024-05-12T04:09:20.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.977+0000 7f02b6ffd700 1 -- 172.21.15.32:0/1168444919 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f02bc01c400 con 0x7f02c0066ad0 2024-05-12T04:09:20.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.977+0000 7f02c8683700 1 -- 172.21.15.32:0/1168444919 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02c0111160 con 0x7f02c0066ad0 2024-05-12T04:09:20.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.977+0000 7f02b6ffd700 1 -- 172.21.15.32:0/1168444919 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02bc012600 con 0x7f02c0066ad0 2024-05-12T04:09:20.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.977+0000 7f02c8683700 1 -- 172.21.15.32:0/1168444919 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02c0111640 con 0x7f02c0066ad0 2024-05-12T04:09:20.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.978+0000 7f02c8683700 1 -- 172.21.15.32:0/1168444919 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02c005f5c0 con 0x7f02c0066ad0 2024-05-12T04:09:21.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.978+0000 7f02b6ffd700 1 -- 172.21.15.32:0/1168444919 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f02bc01c9e0 con 0x7f02c0066ad0 2024-05-12T04:09:21.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.979+0000 7f02b6ffd700 1 --2- 172.21.15.32:0/1168444919 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f02ac06cc20 0x7f02ac06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:09:21.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.979+0000 7f02b6ffd700 1 -- 172.21.15.32:0/1168444919 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f02bc08d690 con 0x7f02c0066ad0 2024-05-12T04:09:21.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.979+0000 7f02c5c1e700 1 --2- 172.21.15.32:0/1168444919 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f02ac06cc20 0x7f02ac06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:09:21.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.979+0000 7f02c5c1e700 1 --2- 172.21.15.32:0/1168444919 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f02ac06cc20 0x7f02ac06f0d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f02b0009b80 tx=0x7f02b000b040 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:09:21.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:20.983+0000 7f02b6ffd700 1 -- 172.21.15.32:0/1168444919 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f02bc05b100 con 0x7f02c0066ad0 2024-05-12T04:09:21.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:20 smithi032 ceph-mon[120392]: from='client.34480 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:09:21.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:20 smithi032 ceph-mon[120392]: pgmap v220: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 14 MiB/s rd, 0 B/s wr, 3 op/s 2024-05-12T04:09:21.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:20 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/4184372388' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:21.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:20 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:21.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:20 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:21.041 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:20 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/3050311829' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-12T04:09:21.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:20 smithi033 ceph-mon[107305]: from='client.34480 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:09:21.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:20 smithi033 ceph-mon[107305]: pgmap v220: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 14 MiB/s rd, 0 B/s wr, 3 op/s 2024-05-12T04:09:21.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:20 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/4184372388' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:21.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:20 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:21.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:20 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:21.163 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:20 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/3050311829' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-12T04:09:21.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:21.283+0000 7f02c8683700 1 -- 172.21.15.32:0/1168444919 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f02c01119c0 con 0x7f02c0066ad0 2024-05-12T04:09:21.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:21.284+0000 7f02b6ffd700 1 -- 172.21.15.32:0/1168444919 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f02bc05ac90 con 0x7f02c0066ad0 2024-05-12T04:09:21.286 INFO:teuthology.orchestra.run.smithi032.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-12T04:09:21.286 INFO:teuthology.orchestra.run.smithi032.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-12T04:09:21.286 INFO:teuthology.orchestra.run.smithi032.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-12T04:09:21.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:21.286+0000 7f02c8683700 1 -- 172.21.15.32:0/1168444919 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f02ac06cc20 msgr2=0x7f02ac06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:09:21.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:21.286+0000 7f02c8683700 1 --2- 172.21.15.32:0/1168444919 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f02ac06cc20 0x7f02ac06f0d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f02b0009b80 tx=0x7f02b000b040 comp rx=0 tx=0).stop 2024-05-12T04:09:21.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:21.286+0000 7f02c8683700 1 -- 172.21.15.32:0/1168444919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02c0066ad0 msgr2=0x7f02c010fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:09:21.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:21.286+0000 7f02c8683700 1 --2- 172.21.15.32:0/1168444919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02c0066ad0 0x7f02c010fb50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f02bc0043d0 tx=0x7f02bc009e30 comp rx=0 tx=0).stop 2024-05-12T04:09:21.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:21.286+0000 7f02c8683700 1 -- 172.21.15.32:0/1168444919 shutdown_connections 2024-05-12T04:09:21.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:21.286+0000 7f02c8683700 1 --2- 172.21.15.32:0/1168444919 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f02ac06cc20 0x7f02ac06f0d0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:21.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:21.286+0000 7f02c8683700 1 --2- 172.21.15.32:0/1168444919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02c0066ad0 0x7f02c010fb50 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:21.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:21.286+0000 7f02c8683700 1 --2- 172.21.15.32:0/1168444919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02c01023f0 0x7f02c0110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:21.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:21.286+0000 7f02c8683700 1 -- 172.21.15.32:0/1168444919 >> 172.21.15.32:0/1168444919 conn(0x7f02c00f74c0 msgr2=0x7f02c00f7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:09:21.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:21.286+0000 7f02c8683700 1 -- 172.21.15.32:0/1168444919 shutdown_connections 2024-05-12T04:09:21.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:21.286+0000 7f02c8683700 1 -- 172.21.15.32:0/1168444919 wait complete. 2024-05-12T04:09:22.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:21 smithi032 ceph-mon[120392]: from='client.34492 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:09:22.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:21 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:22.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:21 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:22.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:21 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/1168444919' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-12T04:09:22.153 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:21 smithi033 ceph-mon[107305]: from='client.34492 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:09:22.153 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:21 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:22.153 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:21 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:22.153 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:21 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/1168444919' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-12T04:09:23.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:23 smithi032 ceph-mon[120392]: pgmap v221: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 12 MiB/s rd, 3 op/s 2024-05-12T04:09:23.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:23 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:23.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:23 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:23.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:23 smithi033 ceph-mon[107305]: pgmap v221: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 12 MiB/s rd, 3 op/s 2024-05-12T04:09:23.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:23 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:23.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:23 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:25.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:25 smithi032 ceph-mon[120392]: pgmap v222: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 12 MiB/s rd, 4 op/s 2024-05-12T04:09:25.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:25 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:25.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:25 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:25.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:25 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:09:25.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:25 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T04:09:25.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:25 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:25.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:25 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:09:25.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:25 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:25.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:25 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:25.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:25 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:25.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:25 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:25.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:25 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:25.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:25 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:25.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:25 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:25.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:25 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:25.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:25 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:25.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:25 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:25.542 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:25 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:25.542 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:25 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:25.542 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:25 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:25.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:25 smithi033 ceph-mon[107305]: pgmap v222: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 12 MiB/s rd, 4 op/s 2024-05-12T04:09:25.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:25 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:25.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:25 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:25.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:25 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:09:25.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:25 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T04:09:25.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:25 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:25.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:25 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:09:25.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:25 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:25.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:25 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:25.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:25 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:25.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:25 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:25.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:25 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:25.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:25 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:25.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:25 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:25.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:25 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:25.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:25 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:25.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:25 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:25.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:25 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:25.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:25 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:25.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:25 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:27.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:27 smithi032 ceph-mon[120392]: pgmap v223: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.8 MiB/s rd, 1 op/s 2024-05-12T04:09:27.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:27 smithi033 ceph-mon[107305]: pgmap v223: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.8 MiB/s rd, 1 op/s 2024-05-12T04:09:28.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:28 smithi032 ceph-mon[120392]: Upgrade: Updating prometheus.smithi032 2024-05-12T04:09:28.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:28 smithi032 ceph-mon[120392]: Deploying daemon prometheus.smithi032 on smithi032 2024-05-12T04:09:28.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:28 smithi033 ceph-mon[107305]: Upgrade: Updating prometheus.smithi032 2024-05-12T04:09:28.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:28 smithi033 ceph-mon[107305]: Deploying daemon prometheus.smithi032 on smithi032 2024-05-12T04:09:29.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:29 smithi032 ceph-mon[120392]: pgmap v224: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 2 op/s 2024-05-12T04:09:29.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:29 smithi033 ceph-mon[107305]: pgmap v224: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 2 op/s 2024-05-12T04:09:31.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:31 smithi032 ceph-mon[120392]: pgmap v225: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 4.0 KiB/s rd, 1 op/s 2024-05-12T04:09:31.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:31 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:09:31.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:31 smithi033 ceph-mon[107305]: pgmap v225: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 4.0 KiB/s rd, 1 op/s 2024-05-12T04:09:31.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:31 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:09:33.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:33 smithi032 ceph-mon[120392]: pgmap v226: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 4.0 KiB/s rd, 2 op/s 2024-05-12T04:09:33.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:33 smithi033 ceph-mon[107305]: pgmap v226: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 4.0 KiB/s rd, 2 op/s 2024-05-12T04:09:35.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:35 smithi032 ceph-mon[120392]: pgmap v227: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:09:35.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:35 smithi033 ceph-mon[107305]: pgmap v227: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:09:37.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:37 smithi032 ceph-mon[120392]: pgmap v228: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:09:37.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:37 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:37.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:37 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:37.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:37 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:09:37.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:37 smithi033 ceph-mon[107305]: pgmap v228: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:09:37.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:37 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:37.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:37 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:37.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:37 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:09:39.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:39 smithi032 ceph-mon[120392]: pgmap v229: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:09:39.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:39 smithi033 ceph-mon[107305]: pgmap v229: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:09:41.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:41 smithi032 ceph-mon[120392]: pgmap v230: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:09:41.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:41 smithi033 ceph-mon[107305]: pgmap v230: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:09:42.600 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:42 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:42.601 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:42 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:42.601 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:42 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:42.601 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:42 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:42.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:42 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:42.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:42 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:42.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:42 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:42.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:42 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:43.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:43 smithi032 ceph-mon[120392]: pgmap v231: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:09:43.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:43 smithi033 ceph-mon[107305]: pgmap v231: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:09:45.491 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:45 smithi032 ceph-mon[120392]: pgmap v232: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:09:45.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:45 smithi033 ceph-mon[107305]: pgmap v232: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:09:46.661 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:46 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:09:46.661 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:46 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:46.661 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:46 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:46.661 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:46 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:09:46.661 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:46 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T04:09:46.662 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:46 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:46.662 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:46 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-12T04:09:46.662 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:46 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:09:46.662 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:46 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:46.662 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:46 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:46.662 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:46 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:46.662 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:46 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:46.662 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:46 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:46.662 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:46 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:46.662 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:46 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:46.662 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:46 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:46.662 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:46 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:46.662 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:46 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:46.662 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:46 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:46.662 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:46 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:46.663 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:46 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:46.663 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:46 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:46.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:46 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:09:46.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:46 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:46.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:46 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:46.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:46 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:09:46.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:46 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T04:09:46.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:46 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:46.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:46 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-12T04:09:46.664 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:46 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:09:46.664 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:46 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:46.664 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:46 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:46.664 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:46 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:46.664 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:46 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:46.664 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:46 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:46.664 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:46 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:46.664 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:46 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:46.664 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:46 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:46.664 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:46 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:46.664 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:46 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:46.664 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:46 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:46.664 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:46 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:46.664 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:46 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:46.664 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:46 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:47.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:47 smithi033 ceph-mon[107305]: pgmap v233: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:09:47.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:47 smithi033 ceph-mon[107305]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-12T04:09:47.738 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:47 smithi032 ceph-mon[120392]: pgmap v233: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:09:47.738 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:47 smithi032 ceph-mon[120392]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-05-12T04:09:49.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:49 smithi033 ceph-mon[107305]: pgmap v234: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:09:49.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:49 smithi033 ceph-mon[107305]: Upgrade: Updating alertmanager.smithi032 2024-05-12T04:09:49.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:49 smithi033 ceph-mon[107305]: Deploying daemon alertmanager.smithi032 on smithi032 2024-05-12T04:09:49.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:49 smithi032 ceph-mon[120392]: pgmap v234: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:09:49.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:49 smithi032 ceph-mon[120392]: Upgrade: Updating alertmanager.smithi032 2024-05-12T04:09:49.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:49 smithi032 ceph-mon[120392]: Deploying daemon alertmanager.smithi032 on smithi032 2024-05-12T04:09:51.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.396+0000 7f9d123a0700 1 -- 172.21.15.32:0/415673787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d0c0fc8e0 msgr2=0x7f9d0c0fcd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:09:51.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.396+0000 7f9d123a0700 1 --2- 172.21.15.32:0/415673787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d0c0fc8e0 0x7f9d0c0fcd50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9d00004940 tx=0x7f9d00009e30 comp rx=0 tx=0).stop 2024-05-12T04:09:51.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.396+0000 7f9d123a0700 1 -- 172.21.15.32:0/415673787 shutdown_connections 2024-05-12T04:09:51.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.396+0000 7f9d123a0700 1 --2- 172.21.15.32:0/415673787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d0c0fc8e0 0x7f9d0c0fcd50 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:51.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.396+0000 7f9d123a0700 1 --2- 172.21.15.32:0/415673787 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d0c0fe2f0 0x7f9d0c0fe700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:51.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.396+0000 7f9d123a0700 1 -- 172.21.15.32:0/415673787 >> 172.21.15.32:0/415673787 conn(0x7f9d0c0f83b0 msgr2=0x7f9d0c0fa7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:09:51.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.396+0000 7f9d123a0700 1 -- 172.21.15.32:0/415673787 shutdown_connections 2024-05-12T04:09:51.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.396+0000 7f9d123a0700 1 -- 172.21.15.32:0/415673787 wait complete. 2024-05-12T04:09:51.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.396+0000 7f9d123a0700 1 Processor -- start 2024-05-12T04:09:51.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.396+0000 7f9d123a0700 1 -- start start 2024-05-12T04:09:51.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.397+0000 7f9d123a0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d0c0fc8e0 0x7f9d0c10d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:09:51.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.397+0000 7f9d123a0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d0c0fe2f0 0x7f9d0c10dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:09:51.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.397+0000 7f9d123a0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c10e2e0 con 0x7f9d0c0fc8e0 2024-05-12T04:09:51.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.397+0000 7f9d123a0700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c10eae0 con 0x7f9d0c0fe2f0 2024-05-12T04:09:51.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.397+0000 7f9d1139e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d0c0fc8e0 0x7f9d0c10d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:09:51.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.397+0000 7f9d1139e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d0c0fc8e0 0x7f9d0c10d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52096/0 (socket says 172.21.15.32:52096) 2024-05-12T04:09:51.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.397+0000 7f9d1139e700 1 -- 172.21.15.32:0/2669223327 learned_addr learned my addr 172.21.15.32:0/2669223327 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:09:51.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.397+0000 7f9d10b9d700 1 --2- 172.21.15.32:0/2669223327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d0c0fe2f0 0x7f9d0c10dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:09:51.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.397+0000 7f9d1139e700 1 -- 172.21.15.32:0/2669223327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d0c0fe2f0 msgr2=0x7f9d0c10dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:09:51.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.397+0000 7f9d1139e700 1 --2- 172.21.15.32:0/2669223327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d0c0fe2f0 0x7f9d0c10dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:51.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.397+0000 7f9d1139e700 1 -- 172.21.15.32:0/2669223327 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d00005040 con 0x7f9d0c0fc8e0 2024-05-12T04:09:51.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.397+0000 7f9d10b9d700 1 --2- 172.21.15.32:0/2669223327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d0c0fe2f0 0x7f9d0c10dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-12T04:09:51.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.397+0000 7f9d1139e700 1 --2- 172.21.15.32:0/2669223327 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d0c0fc8e0 0x7f9d0c10d7a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9d0800c810 tx=0x7f9d08004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:09:51.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.397+0000 7f9cfdffb700 1 -- 172.21.15.32:0/2669223327 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d08012810 con 0x7f9d0c0fc8e0 2024-05-12T04:09:51.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.397+0000 7f9d123a0700 1 -- 172.21.15.32:0/2669223327 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d0c10edc0 con 0x7f9d0c0fc8e0 2024-05-12T04:09:51.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.397+0000 7f9cfdffb700 1 -- 172.21.15.32:0/2669223327 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9d08012e20 con 0x7f9d0c0fc8e0 2024-05-12T04:09:51.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.398+0000 7f9cfdffb700 1 -- 172.21.15.32:0/2669223327 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d0800a370 con 0x7f9d0c0fc8e0 2024-05-12T04:09:51.400 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.398+0000 7f9d123a0700 1 -- 172.21.15.32:0/2669223327 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d0c10f300 con 0x7f9d0c0fc8e0 2024-05-12T04:09:51.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.399+0000 7f9d123a0700 1 -- 172.21.15.32:0/2669223327 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d0c05f5c0 con 0x7f9d0c0fc8e0 2024-05-12T04:09:51.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.399+0000 7f9cfdffb700 1 -- 172.21.15.32:0/2669223327 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f9d08011780 con 0x7f9d0c0fc8e0 2024-05-12T04:09:51.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.399+0000 7f9cfdffb700 1 --2- 172.21.15.32:0/2669223327 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f9cf806cc30 0x7f9cf806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:09:51.401 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.400+0000 7f9cfdffb700 1 -- 172.21.15.32:0/2669223327 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f9d0808c530 con 0x7f9d0c0fc8e0 2024-05-12T04:09:51.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.400+0000 7f9d10b9d700 1 --2- 172.21.15.32:0/2669223327 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f9cf806cc30 0x7f9cf806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:09:51.402 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.400+0000 7f9d10b9d700 1 --2- 172.21.15.32:0/2669223327 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f9cf806cc30 0x7f9cf806f0e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9d00004f90 tx=0x7f9d00009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:09:51.405 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.405+0000 7f9cfdffb700 1 -- 172.21.15.32:0/2669223327 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f9d08059f20 con 0x7f9d0c0fc8e0 2024-05-12T04:09:51.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.630+0000 7f9d123a0700 1 -- 172.21.15.32:0/2669223327 --> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9d0c10f690 con 0x7f9cf806cc30 2024-05-12T04:09:51.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.632+0000 7f9cfdffb700 1 -- 172.21.15.32:0/2669223327 <== mgr.24593 v2:172.21.15.32:6800/2957111591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+440 (secure 0 0 0) 0x7f9d0c10f690 con 0x7f9cf806cc30 2024-05-12T04:09:51.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.634+0000 7f9d123a0700 1 -- 172.21.15.32:0/2669223327 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f9cf806cc30 msgr2=0x7f9cf806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:09:51.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.634+0000 7f9d123a0700 1 --2- 172.21.15.32:0/2669223327 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f9cf806cc30 0x7f9cf806f0e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9d00004f90 tx=0x7f9d00009e30 comp rx=0 tx=0).stop 2024-05-12T04:09:51.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.634+0000 7f9d123a0700 1 -- 172.21.15.32:0/2669223327 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d0c0fc8e0 msgr2=0x7f9d0c10d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:09:51.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.634+0000 7f9d123a0700 1 --2- 172.21.15.32:0/2669223327 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d0c0fc8e0 0x7f9d0c10d7a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9d0800c810 tx=0x7f9d08004470 comp rx=0 tx=0).stop 2024-05-12T04:09:51.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.634+0000 7f9d123a0700 1 -- 172.21.15.32:0/2669223327 shutdown_connections 2024-05-12T04:09:51.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.634+0000 7f9d123a0700 1 --2- 172.21.15.32:0/2669223327 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f9cf806cc30 0x7f9cf806f0e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:51.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.634+0000 7f9d123a0700 1 --2- 172.21.15.32:0/2669223327 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d0c0fc8e0 0x7f9d0c10d7a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:51.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.634+0000 7f9d123a0700 1 --2- 172.21.15.32:0/2669223327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d0c0fe2f0 0x7f9d0c10dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:51.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.634+0000 7f9d123a0700 1 -- 172.21.15.32:0/2669223327 >> 172.21.15.32:0/2669223327 conn(0x7f9d0c0f83b0 msgr2=0x7f9d0c0f9b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:09:51.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.634+0000 7f9d123a0700 1 -- 172.21.15.32:0/2669223327 shutdown_connections 2024-05-12T04:09:51.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.634+0000 7f9d123a0700 1 -- 172.21.15.32:0/2669223327 wait complete. 2024-05-12T04:09:51.644 INFO:teuthology.orchestra.run.smithi032.stdout:true 2024-05-12T04:09:51.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:51 smithi033 ceph-mon[107305]: pgmap v235: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:09:51.697 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:51 smithi032 ceph-mon[120392]: pgmap v235: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:09:51.742 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.741+0000 7f67efe47700 1 -- 172.21.15.32:0/930941798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67e80fd9c0 msgr2=0x7f67e80ffda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:09:51.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.741+0000 7f67efe47700 1 --2- 172.21.15.32:0/930941798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67e80fd9c0 0x7f67e80ffda0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f67e4004940 tx=0x7f67e4009e30 comp rx=0 tx=0).stop 2024-05-12T04:09:51.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.741+0000 7f67efe47700 1 -- 172.21.15.32:0/930941798 shutdown_connections 2024-05-12T04:09:51.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.741+0000 7f67efe47700 1 --2- 172.21.15.32:0/930941798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67e81002e0 0x7f67e81026c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:51.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.741+0000 7f67efe47700 1 --2- 172.21.15.32:0/930941798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67e80fd9c0 0x7f67e80ffda0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:51.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.741+0000 7f67efe47700 1 -- 172.21.15.32:0/930941798 >> 172.21.15.32:0/930941798 conn(0x7f67e80f7440 msgr2=0x7f67e80f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:09:51.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.741+0000 7f67efe47700 1 -- 172.21.15.32:0/930941798 shutdown_connections 2024-05-12T04:09:51.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.741+0000 7f67efe47700 1 -- 172.21.15.32:0/930941798 wait complete. 2024-05-12T04:09:51.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.741+0000 7f67efe47700 1 Processor -- start 2024-05-12T04:09:51.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.741+0000 7f67efe47700 1 -- start start 2024-05-12T04:09:51.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.741+0000 7f67efe47700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67e80fd9c0 0x7f67e810d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:09:51.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.741+0000 7f67efe47700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67e81002e0 0x7f67e810dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:09:51.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.741+0000 7f67efe47700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e810e4e0 con 0x7f67e80fd9c0 2024-05-12T04:09:51.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.741+0000 7f67efe47700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e810ece0 con 0x7f67e81002e0 2024-05-12T04:09:51.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.742+0000 7f67edbe3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67e80fd9c0 0x7f67e810d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:09:51.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.742+0000 7f67edbe3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67e80fd9c0 0x7f67e810d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52114/0 (socket says 172.21.15.32:52114) 2024-05-12T04:09:51.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.742+0000 7f67edbe3700 1 -- 172.21.15.32:0/748816818 learned_addr learned my addr 172.21.15.32:0/748816818 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:09:51.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.742+0000 7f67ed3e2700 1 --2- 172.21.15.32:0/748816818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67e81002e0 0x7f67e810dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:09:51.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.742+0000 7f67edbe3700 1 -- 172.21.15.32:0/748816818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67e81002e0 msgr2=0x7f67e810dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:09:51.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.742+0000 7f67edbe3700 1 --2- 172.21.15.32:0/748816818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67e81002e0 0x7f67e810dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:51.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.742+0000 7f67edbe3700 1 -- 172.21.15.32:0/748816818 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e4005040 con 0x7f67e80fd9c0 2024-05-12T04:09:51.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.742+0000 7f67edbe3700 1 --2- 172.21.15.32:0/748816818 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67e80fd9c0 0x7f67e810d9a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f67e400b3b0 tx=0x7f67e4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:09:51.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.742+0000 7f67de7fc700 1 -- 172.21.15.32:0/748816818 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67e400e040 con 0x7f67e80fd9c0 2024-05-12T04:09:51.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.742+0000 7f67de7fc700 1 -- 172.21.15.32:0/748816818 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f67e400bc50 con 0x7f67e80fd9c0 2024-05-12T04:09:51.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.742+0000 7f67efe47700 1 -- 172.21.15.32:0/748816818 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67e810ef60 con 0x7f67e80fd9c0 2024-05-12T04:09:51.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.742+0000 7f67de7fc700 1 -- 172.21.15.32:0/748816818 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67e401c3f0 con 0x7f67e80fd9c0 2024-05-12T04:09:51.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.742+0000 7f67efe47700 1 -- 172.21.15.32:0/748816818 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67e810f4a0 con 0x7f67e80fd9c0 2024-05-12T04:09:51.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.744+0000 7f67efe47700 1 -- 172.21.15.32:0/748816818 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67e805f5c0 con 0x7f67e80fd9c0 2024-05-12T04:09:51.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.744+0000 7f67de7fc700 1 -- 172.21.15.32:0/748816818 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f67e4013420 con 0x7f67e80fd9c0 2024-05-12T04:09:51.745 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.744+0000 7f67de7fc700 1 --2- 172.21.15.32:0/748816818 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f67d406cc20 0x7f67d406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:09:51.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.744+0000 7f67de7fc700 1 -- 172.21.15.32:0/748816818 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f67e408cba0 con 0x7f67e80fd9c0 2024-05-12T04:09:51.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.744+0000 7f67ed3e2700 1 --2- 172.21.15.32:0/748816818 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f67d406cc20 0x7f67d406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:09:51.746 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.745+0000 7f67ed3e2700 1 --2- 172.21.15.32:0/748816818 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f67d406cc20 0x7f67d406f0d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f67d80041d0 tx=0x7f67d80043c0 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:09:51.749 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.749+0000 7f67de7fc700 1 -- 172.21.15.32:0/748816818 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f67e405a6b0 con 0x7f67e80fd9c0 2024-05-12T04:09:51.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.977+0000 7f67efe47700 1 -- 172.21.15.32:0/748816818 --> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f67e810f870 con 0x7f67d406cc20 2024-05-12T04:09:51.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.978+0000 7f67de7fc700 1 -- 172.21.15.32:0/748816818 <== mgr.24593 v2:172.21.15.32:6800/2957111591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+440 (secure 0 0 0) 0x7f67e810f870 con 0x7f67d406cc20 2024-05-12T04:09:51.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.980+0000 7f67efe47700 1 -- 172.21.15.32:0/748816818 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f67d406cc20 msgr2=0x7f67d406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:09:51.980 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.980+0000 7f67efe47700 1 --2- 172.21.15.32:0/748816818 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f67d406cc20 0x7f67d406f0d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f67d80041d0 tx=0x7f67d80043c0 comp rx=0 tx=0).stop 2024-05-12T04:09:51.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.980+0000 7f67efe47700 1 -- 172.21.15.32:0/748816818 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67e80fd9c0 msgr2=0x7f67e810d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:09:51.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.980+0000 7f67efe47700 1 --2- 172.21.15.32:0/748816818 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67e80fd9c0 0x7f67e810d9a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f67e400b3b0 tx=0x7f67e4009e30 comp rx=0 tx=0).stop 2024-05-12T04:09:51.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.980+0000 7f67efe47700 1 -- 172.21.15.32:0/748816818 shutdown_connections 2024-05-12T04:09:51.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.980+0000 7f67efe47700 1 --2- 172.21.15.32:0/748816818 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f67d406cc20 0x7f67d406f0d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:51.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.980+0000 7f67efe47700 1 --2- 172.21.15.32:0/748816818 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67e80fd9c0 0x7f67e810d9a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:51.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.980+0000 7f67efe47700 1 --2- 172.21.15.32:0/748816818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67e81002e0 0x7f67e810dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:51.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.980+0000 7f67efe47700 1 -- 172.21.15.32:0/748816818 >> 172.21.15.32:0/748816818 conn(0x7f67e80f7440 msgr2=0x7f67e80f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:09:51.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.980+0000 7f67efe47700 1 -- 172.21.15.32:0/748816818 shutdown_connections 2024-05-12T04:09:51.981 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:51.980+0000 7f67efe47700 1 -- 172.21.15.32:0/748816818 wait complete. 2024-05-12T04:09:52.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.083+0000 7fdb50f6f700 1 -- 172.21.15.32:0/2911931813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb4c0fcf60 msgr2=0x7fdb4c0fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:09:52.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.083+0000 7fdb50f6f700 1 --2- 172.21.15.32:0/2911931813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb4c0fcf60 0x7fdb4c0fd3d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fdb40004940 tx=0x7fdb40009e30 comp rx=0 tx=0).stop 2024-05-12T04:09:52.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.083+0000 7fdb50f6f700 1 -- 172.21.15.32:0/2911931813 shutdown_connections 2024-05-12T04:09:52.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.083+0000 7fdb50f6f700 1 --2- 172.21.15.32:0/2911931813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb4c0fcf60 0x7fdb4c0fd3d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:52.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.083+0000 7fdb50f6f700 1 --2- 172.21.15.32:0/2911931813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb4c0fbcb0 0x7fdb4c0fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:52.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.083+0000 7fdb50f6f700 1 -- 172.21.15.32:0/2911931813 >> 172.21.15.32:0/2911931813 conn(0x7fdb4c0f7430 msgr2=0x7fdb4c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:09:52.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.084+0000 7fdb50f6f700 1 -- 172.21.15.32:0/2911931813 shutdown_connections 2024-05-12T04:09:52.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.084+0000 7fdb50f6f700 1 -- 172.21.15.32:0/2911931813 wait complete. 2024-05-12T04:09:52.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.084+0000 7fdb50f6f700 1 Processor -- start 2024-05-12T04:09:52.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.084+0000 7fdb50f6f700 1 -- start start 2024-05-12T04:09:52.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.084+0000 7fdb50f6f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb4c0fbcb0 0x7fdb4c1105d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:09:52.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.084+0000 7fdb50f6f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb4c0fcf60 0x7fdb4c112b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:09:52.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.084+0000 7fdb50f6f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4c066390 con 0x7fdb4c0fbcb0 2024-05-12T04:09:52.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.084+0000 7fdb50f6f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4c0664d0 con 0x7fdb4c0fcf60 2024-05-12T04:09:52.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.084+0000 7fdb4a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb4c0fbcb0 0x7fdb4c1105d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:09:52.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.084+0000 7fdb4a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb4c0fbcb0 0x7fdb4c1105d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45458/0 (socket says 172.21.15.32:45458) 2024-05-12T04:09:52.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.084+0000 7fdb4a59c700 1 -- 172.21.15.32:0/2562716229 learned_addr learned my addr 172.21.15.32:0/2562716229 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:09:52.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.084+0000 7fdb49d9b700 1 --2- 172.21.15.32:0/2562716229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb4c0fcf60 0x7fdb4c112b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:09:52.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.085+0000 7fdb4a59c700 1 -- 172.21.15.32:0/2562716229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb4c0fcf60 msgr2=0x7fdb4c112b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:09:52.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.085+0000 7fdb4a59c700 1 --2- 172.21.15.32:0/2562716229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb4c0fcf60 0x7fdb4c112b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:52.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.085+0000 7fdb4a59c700 1 -- 172.21.15.32:0/2562716229 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb40005040 con 0x7fdb4c0fbcb0 2024-05-12T04:09:52.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.085+0000 7fdb4a59c700 1 --2- 172.21.15.32:0/2562716229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb4c0fbcb0 0x7fdb4c1105d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c00c830 tx=0x7fdb3c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:09:52.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.085+0000 7fdb3affd700 1 -- 172.21.15.32:0/2562716229 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb3c0106f0 con 0x7fdb4c0fbcb0 2024-05-12T04:09:52.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.085+0000 7fdb3affd700 1 -- 172.21.15.32:0/2562716229 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdb3c010d00 con 0x7fdb4c0fbcb0 2024-05-12T04:09:52.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.085+0000 7fdb3affd700 1 -- 172.21.15.32:0/2562716229 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb3c012e90 con 0x7fdb4c0fbcb0 2024-05-12T04:09:52.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.085+0000 7fdb50f6f700 1 -- 172.21.15.32:0/2562716229 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb4c113120 con 0x7fdb4c0fbcb0 2024-05-12T04:09:52.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.085+0000 7fdb50f6f700 1 -- 172.21.15.32:0/2562716229 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb4c1136e0 con 0x7fdb4c0fbcb0 2024-05-12T04:09:52.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.086+0000 7fdb50f6f700 1 -- 172.21.15.32:0/2562716229 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb4c05f5c0 con 0x7fdb4c0fbcb0 2024-05-12T04:09:52.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.087+0000 7fdb3affd700 1 -- 172.21.15.32:0/2562716229 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fdb3c010850 con 0x7fdb4c0fbcb0 2024-05-12T04:09:52.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.087+0000 7fdb3affd700 1 --2- 172.21.15.32:0/2562716229 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fdb3406cc30 0x7fdb3406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:09:52.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.087+0000 7fdb3affd700 1 -- 172.21.15.32:0/2562716229 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fdb3c08c000 con 0x7fdb4c0fbcb0 2024-05-12T04:09:52.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.087+0000 7fdb49d9b700 1 --2- 172.21.15.32:0/2562716229 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fdb3406cc30 0x7fdb3406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:09:52.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.088+0000 7fdb49d9b700 1 --2- 172.21.15.32:0/2562716229 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fdb3406cc30 0x7fdb3406f0e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdb40004f90 tx=0x7fdb40009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:09:52.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.092+0000 7fdb3affd700 1 -- 172.21.15.32:0/2562716229 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fdb3c056080 con 0x7fdb4c0fbcb0 2024-05-12T04:09:52.326 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.325+0000 7fdb50f6f700 1 -- 172.21.15.32:0/2562716229 --> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fdb4c1011a0 con 0x7fdb3406cc30 2024-05-12T04:09:52.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.337+0000 7fdb3affd700 1 -- 172.21.15.32:0/2562716229 <== mgr.24593 v2:172.21.15.32:6800/2957111591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fdb4c1011a0 con 0x7fdb3406cc30 2024-05-12T04:09:52.338 INFO:teuthology.orchestra.run.smithi032.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-12T04:09:52.338 INFO:teuthology.orchestra.run.smithi032.stdout:alertmanager.smithi032 smithi032 *:9093,9094 running (8m) 10s ago 17m 25.5M - 0.25.0 c8568f914cd2 a37fbab7af4c 2024-05-12T04:09:52.338 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi032 smithi032 running (5m) 10s ago 18m 6660k - 18.2.2-1272-g58249552 ddcb0a9d6fde 8c53f1fc7b06 2024-05-12T04:09:52.338 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi033 smithi033 running (5m) 34s ago 17m 6651k - 18.2.2-1272-g58249552 ddcb0a9d6fde cf4fe9ca070b 2024-05-12T04:09:52.338 INFO:teuthology.orchestra.run.smithi032.stdout:grafana.smithi032 smithi032 *:3000 running (8m) 10s ago 17m 89.7M - 9.4.7 954c08fa6188 83055e58d23c 2024-05-12T04:09:52.338 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.ialtqb smithi032 running (117s) 10s ago 12m 82.9M - 18.2.2-1272-g58249552 ddcb0a9d6fde 532dd9845ce7 2024-05-12T04:09:52.338 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.qpsawh smithi032 running (95s) 10s ago 11m 86.5M - 18.2.2-1272-g58249552 ddcb0a9d6fde 68e735fd79ee 2024-05-12T04:09:52.338 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi033.hglwml smithi033 running (54s) 34s ago 11m 14.9M - 18.2.2-1272-g58249552 ddcb0a9d6fde 7240e684b225 2024-05-12T04:09:52.338 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi033.jgbjhk smithi033 running (76s) 34s ago 11m 14.1M - 18.2.2-1272-g58249552 ddcb0a9d6fde b738dbbbec0f 2024-05-12T04:09:52.338 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi032.ukatkq smithi032 *:8443,9283,8765 running (7m) 10s ago 19m 511M - 18.2.2-1272-g58249552 ddcb0a9d6fde 0d88ab9a5292 2024-05-12T04:09:52.338 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi033.pcvxwy smithi033 *:8443,9283,8765 running (7m) 34s ago 16m 451M - 18.2.2-1272-g58249552 ddcb0a9d6fde d42dcb6bd307 2024-05-12T04:09:52.338 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi032 smithi032 running (6m) 10s ago 19m 53.7M 2048M 18.2.2-1272-g58249552 ddcb0a9d6fde 81f94171c5dc 2024-05-12T04:09:52.339 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi033 smithi033 running (6m) 34s ago 16m 42.9M 2048M 18.2.2-1272-g58249552 ddcb0a9d6fde d648ba108f3e 2024-05-12T04:09:52.339 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi032 smithi032 *:9100 running (41s) 10s ago 18m 8623k - 1.5.0 0da6a335fe13 fcb3ea4af15b 2024-05-12T04:09:52.339 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi033 smithi033 *:9100 running (37s) 34s ago 16m 3632k - 1.5.0 0da6a335fe13 b75dd225ce4b 2024-05-12T04:09:52.339 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0 smithi032 running (5m) 10s ago 16m 92.8M 1570M 18.2.2-1272-g58249552 ddcb0a9d6fde 07f66c4deb3b 2024-05-12T04:09:52.339 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1 smithi032 running (4m) 10s ago 15m 100M 1570M 18.2.2-1272-g58249552 ddcb0a9d6fde 67a87807d654 2024-05-12T04:09:52.339 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2 smithi032 running (3m) 10s ago 15m 88.4M 1570M 18.2.2-1272-g58249552 ddcb0a9d6fde 4ac48d74ee8b 2024-05-12T04:09:52.339 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3 smithi033 running (3m) 34s ago 14m 103M 2594M 18.2.2-1272-g58249552 ddcb0a9d6fde 11f097a455bb 2024-05-12T04:09:52.339 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4 smithi033 running (2m) 34s ago 14m 91.5M 2594M 18.2.2-1272-g58249552 ddcb0a9d6fde db6fda7d52de 2024-05-12T04:09:52.339 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5 smithi033 running (2m) 34s ago 13m 92.7M 2594M 18.2.2-1272-g58249552 ddcb0a9d6fde 3333020b8a15 2024-05-12T04:09:52.339 INFO:teuthology.orchestra.run.smithi032.stdout:prometheus.smithi032 smithi032 *:9095 running (16s) 10s ago 17m 41.2M - 2.43.0 a07b618ecd1d fe69bbb63353 2024-05-12T04:09:52.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.340+0000 7fdb50f6f700 1 -- 172.21.15.32:0/2562716229 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fdb3406cc30 msgr2=0x7fdb3406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:09:52.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.340+0000 7fdb50f6f700 1 --2- 172.21.15.32:0/2562716229 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fdb3406cc30 0x7fdb3406f0e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdb40004f90 tx=0x7fdb40009e30 comp rx=0 tx=0).stop 2024-05-12T04:09:52.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.340+0000 7fdb50f6f700 1 -- 172.21.15.32:0/2562716229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb4c0fbcb0 msgr2=0x7fdb4c1105d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:09:52.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.340+0000 7fdb50f6f700 1 --2- 172.21.15.32:0/2562716229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb4c0fbcb0 0x7fdb4c1105d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c00c830 tx=0x7fdb3c004470 comp rx=0 tx=0).stop 2024-05-12T04:09:52.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.340+0000 7fdb50f6f700 1 -- 172.21.15.32:0/2562716229 shutdown_connections 2024-05-12T04:09:52.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.340+0000 7fdb50f6f700 1 --2- 172.21.15.32:0/2562716229 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fdb3406cc30 0x7fdb3406f0e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:52.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.340+0000 7fdb50f6f700 1 --2- 172.21.15.32:0/2562716229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb4c0fbcb0 0x7fdb4c1105d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:52.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.340+0000 7fdb50f6f700 1 --2- 172.21.15.32:0/2562716229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb4c0fcf60 0x7fdb4c112b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:52.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.340+0000 7fdb50f6f700 1 -- 172.21.15.32:0/2562716229 >> 172.21.15.32:0/2562716229 conn(0x7fdb4c0f7430 msgr2=0x7fdb4c0f8f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:09:52.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.340+0000 7fdb50f6f700 1 -- 172.21.15.32:0/2562716229 shutdown_connections 2024-05-12T04:09:52.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.340+0000 7fdb50f6f700 1 -- 172.21.15.32:0/2562716229 wait complete. 2024-05-12T04:09:52.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.453+0000 7f3eb1d33700 1 -- 172.21.15.32:0/917919256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3eac070830 msgr2=0x7f3eac070ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:09:52.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.453+0000 7f3eb1d33700 1 --2- 172.21.15.32:0/917919256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3eac070830 0x7f3eac070ca0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3ea0004940 tx=0x7f3ea0009e30 comp rx=0 tx=0).stop 2024-05-12T04:09:52.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.453+0000 7f3eb1d33700 1 -- 172.21.15.32:0/917919256 shutdown_connections 2024-05-12T04:09:52.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.453+0000 7f3eb1d33700 1 --2- 172.21.15.32:0/917919256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3eac070830 0x7f3eac070ca0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:52.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.453+0000 7f3eb1d33700 1 --2- 172.21.15.32:0/917919256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3eac071de0 0x7f3eac0702f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:52.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.454+0000 7f3eb1d33700 1 -- 172.21.15.32:0/917919256 >> 172.21.15.32:0/917919256 conn(0x7f3eac0f8590 msgr2=0x7f3eac0fa9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:09:52.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.454+0000 7f3eb1d33700 1 -- 172.21.15.32:0/917919256 shutdown_connections 2024-05-12T04:09:52.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.454+0000 7f3eb1d33700 1 -- 172.21.15.32:0/917919256 wait complete. 2024-05-12T04:09:52.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.454+0000 7f3eb1d33700 1 Processor -- start 2024-05-12T04:09:52.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.454+0000 7f3eb1d33700 1 -- start start 2024-05-12T04:09:52.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.455+0000 7f3eb1d33700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3eac070830 0x7f3eac10fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:09:52.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.455+0000 7f3eb1d33700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3eac071de0 0x7f3eac110060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:09:52.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.455+0000 7f3eb1d33700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eac110660 con 0x7f3eac071de0 2024-05-12T04:09:52.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.455+0000 7f3eb1d33700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eac110eb0 con 0x7f3eac070830 2024-05-12T04:09:52.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.455+0000 7f3eaaffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3eac071de0 0x7f3eac110060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:09:52.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.455+0000 7f3eaaffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3eac071de0 0x7f3eac110060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45480/0 (socket says 172.21.15.32:45480) 2024-05-12T04:09:52.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.455+0000 7f3eaaffd700 1 -- 172.21.15.32:0/764581490 learned_addr learned my addr 172.21.15.32:0/764581490 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:09:52.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.455+0000 7f3eab7fe700 1 --2- 172.21.15.32:0/764581490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3eac070830 0x7f3eac10fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:09:52.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.455+0000 7f3eaaffd700 1 -- 172.21.15.32:0/764581490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3eac070830 msgr2=0x7f3eac10fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:09:52.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.455+0000 7f3eaaffd700 1 --2- 172.21.15.32:0/764581490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3eac070830 0x7f3eac10fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:52.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.455+0000 7f3eaaffd700 1 -- 172.21.15.32:0/764581490 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ea0005040 con 0x7f3eac071de0 2024-05-12T04:09:52.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.455+0000 7f3eaaffd700 1 --2- 172.21.15.32:0/764581490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3eac071de0 0x7f3eac110060 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3ea000b3b0 tx=0x7f3ea0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:09:52.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.455+0000 7f3e93fff700 1 -- 172.21.15.32:0/764581490 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ea000e040 con 0x7f3eac071de0 2024-05-12T04:09:52.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.455+0000 7f3eb1d33700 1 -- 172.21.15.32:0/764581490 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3eac111130 con 0x7f3eac071de0 2024-05-12T04:09:52.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.456+0000 7f3eb1d33700 1 -- 172.21.15.32:0/764581490 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eac111670 con 0x7f3eac071de0 2024-05-12T04:09:52.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.456+0000 7f3e93fff700 1 -- 172.21.15.32:0/764581490 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3ea0013420 con 0x7f3eac071de0 2024-05-12T04:09:52.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.456+0000 7f3e93fff700 1 -- 172.21.15.32:0/764581490 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ea001c630 con 0x7f3eac071de0 2024-05-12T04:09:52.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.456+0000 7f3eb1d33700 1 -- 172.21.15.32:0/764581490 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e980052e0 con 0x7f3eac071de0 2024-05-12T04:09:52.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.457+0000 7f3e93fff700 1 -- 172.21.15.32:0/764581490 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f3ea000bb80 con 0x7f3eac071de0 2024-05-12T04:09:52.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.457+0000 7f3e93fff700 1 --2- 172.21.15.32:0/764581490 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f3e9406cc20 0x7f3e9406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:09:52.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.457+0000 7f3e93fff700 1 -- 172.21.15.32:0/764581490 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f3ea008ca40 con 0x7f3eac071de0 2024-05-12T04:09:52.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.458+0000 7f3eab7fe700 1 --2- 172.21.15.32:0/764581490 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f3e9406cc20 0x7f3e9406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:09:52.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.458+0000 7f3eab7fe700 1 --2- 172.21.15.32:0/764581490 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f3e9406cc20 0x7f3e9406f0d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3eac071b20 tx=0x7f3e9c00b040 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:09:52.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.462+0000 7f3e93fff700 1 -- 172.21.15.32:0/764581490 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f3ea005a560 con 0x7f3eac071de0 2024-05-12T04:09:52.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.779+0000 7f3eb1d33700 1 -- 172.21.15.32:0/764581490 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3e98005da0 con 0x7f3eac071de0 2024-05-12T04:09:52.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.780+0000 7f3e93fff700 1 -- 172.21.15.32:0/764581490 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f3ea0021290 con 0x7f3eac071de0 2024-05-12T04:09:52.780 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-05-12T04:09:52.781 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": { 2024-05-12T04:09:52.781 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 2 2024-05-12T04:09:52.781 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:09:52.781 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": { 2024-05-12T04:09:52.781 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 2 2024-05-12T04:09:52.781 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:09:52.781 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": { 2024-05-12T04:09:52.781 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 6 2024-05-12T04:09:52.781 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:09:52.781 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": { 2024-05-12T04:09:52.781 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 4 2024-05-12T04:09:52.781 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:09:52.781 INFO:teuthology.orchestra.run.smithi032.stdout: "overall": { 2024-05-12T04:09:52.781 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 14 2024-05-12T04:09:52.781 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-05-12T04:09:52.782 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-05-12T04:09:52.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.782+0000 7f3eb1d33700 1 -- 172.21.15.32:0/764581490 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f3e9406cc20 msgr2=0x7f3e9406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:09:52.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.782+0000 7f3eb1d33700 1 --2- 172.21.15.32:0/764581490 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f3e9406cc20 0x7f3e9406f0d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3eac071b20 tx=0x7f3e9c00b040 comp rx=0 tx=0).stop 2024-05-12T04:09:52.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.782+0000 7f3eb1d33700 1 -- 172.21.15.32:0/764581490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3eac071de0 msgr2=0x7f3eac110060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:09:52.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.782+0000 7f3eb1d33700 1 --2- 172.21.15.32:0/764581490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3eac071de0 0x7f3eac110060 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3ea000b3b0 tx=0x7f3ea0009e30 comp rx=0 tx=0).stop 2024-05-12T04:09:52.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.782+0000 7f3eb1d33700 1 -- 172.21.15.32:0/764581490 shutdown_connections 2024-05-12T04:09:52.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.782+0000 7f3eb1d33700 1 --2- 172.21.15.32:0/764581490 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f3e9406cc20 0x7f3e9406f0d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:52.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.782+0000 7f3eb1d33700 1 --2- 172.21.15.32:0/764581490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3eac070830 0x7f3eac10fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:52.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.782+0000 7f3eb1d33700 1 --2- 172.21.15.32:0/764581490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3eac071de0 0x7f3eac110060 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:52.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.782+0000 7f3eb1d33700 1 -- 172.21.15.32:0/764581490 >> 172.21.15.32:0/764581490 conn(0x7f3eac0f8590 msgr2=0x7f3eac0f9d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:09:52.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.782+0000 7f3eb1d33700 1 -- 172.21.15.32:0/764581490 shutdown_connections 2024-05-12T04:09:52.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.783+0000 7f3eb1d33700 1 -- 172.21.15.32:0/764581490 wait complete. 2024-05-12T04:09:52.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.888+0000 7f95fa2f6700 1 -- 172.21.15.32:0/3345476272 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95f40fbaf0 msgr2=0x7f95f40fe350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:09:52.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.888+0000 7f95fa2f6700 1 --2- 172.21.15.32:0/3345476272 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95f40fbaf0 0x7f95f40fe350 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f95e8004940 tx=0x7f95e8009e30 comp rx=0 tx=0).stop 2024-05-12T04:09:52.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.888+0000 7f95fa2f6700 1 -- 172.21.15.32:0/3345476272 shutdown_connections 2024-05-12T04:09:52.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.888+0000 7f95fa2f6700 1 --2- 172.21.15.32:0/3345476272 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95f40fbaf0 0x7f95f40fe350 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:52.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.888+0000 7f95fa2f6700 1 --2- 172.21.15.32:0/3345476272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95f40fb1a0 0x7f95f40fb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:52.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.888+0000 7f95fa2f6700 1 -- 172.21.15.32:0/3345476272 >> 172.21.15.32:0/3345476272 conn(0x7f95f40f6b80 msgr2=0x7f95f40f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:09:52.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.888+0000 7f95fa2f6700 1 -- 172.21.15.32:0/3345476272 shutdown_connections 2024-05-12T04:09:52.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.888+0000 7f95fa2f6700 1 -- 172.21.15.32:0/3345476272 wait complete. 2024-05-12T04:09:52.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.889+0000 7f95fa2f6700 1 Processor -- start 2024-05-12T04:09:52.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.889+0000 7f95fa2f6700 1 -- start start 2024-05-12T04:09:52.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.889+0000 7f95fa2f6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95f40fb1a0 0x7f95f410db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:09:52.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.889+0000 7f95fa2f6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95f40fbaf0 0x7f95f410e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:09:52.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.889+0000 7f95fa2f6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95f410e680 con 0x7f95f40fb1a0 2024-05-12T04:09:52.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.889+0000 7f95fa2f6700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95f410eed0 con 0x7f95f40fbaf0 2024-05-12T04:09:52.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.889+0000 7f95f3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95f40fb1a0 0x7f95f410db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:09:52.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.889+0000 7f95f3fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95f40fb1a0 0x7f95f410db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45490/0 (socket says 172.21.15.32:45490) 2024-05-12T04:09:52.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.889+0000 7f95f3fff700 1 -- 172.21.15.32:0/190567862 learned_addr learned my addr 172.21.15.32:0/190567862 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:09:52.891 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.890+0000 7f95f37fe700 1 --2- 172.21.15.32:0/190567862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95f40fbaf0 0x7f95f410e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:09:52.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.890+0000 7f95f3fff700 1 -- 172.21.15.32:0/190567862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95f40fbaf0 msgr2=0x7f95f410e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:09:52.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.890+0000 7f95f3fff700 1 --2- 172.21.15.32:0/190567862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95f40fbaf0 0x7f95f410e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:52.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.890+0000 7f95f3fff700 1 -- 172.21.15.32:0/190567862 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e8005040 con 0x7f95f40fb1a0 2024-05-12T04:09:52.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.890+0000 7f95f37fe700 1 --2- 172.21.15.32:0/190567862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95f40fbaf0 0x7f95f410e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-12T04:09:52.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.890+0000 7f95f3fff700 1 --2- 172.21.15.32:0/190567862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95f40fb1a0 0x7f95f410db40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f95e4002a50 tx=0x7f95e40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:09:52.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.890+0000 7f95f0ff9700 1 -- 172.21.15.32:0/190567862 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95e4007cc0 con 0x7f95f40fb1a0 2024-05-12T04:09:52.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.890+0000 7f95fa2f6700 1 -- 172.21.15.32:0/190567862 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95f410f1b0 con 0x7f95f40fb1a0 2024-05-12T04:09:52.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.890+0000 7f95f0ff9700 1 -- 172.21.15.32:0/190567862 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f95e400f040 con 0x7f95f40fb1a0 2024-05-12T04:09:52.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.890+0000 7f95f0ff9700 1 -- 172.21.15.32:0/190567862 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95e4009540 con 0x7f95f40fb1a0 2024-05-12T04:09:52.892 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.890+0000 7f95fa2f6700 1 -- 172.21.15.32:0/190567862 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95f410f6f0 con 0x7f95f40fb1a0 2024-05-12T04:09:52.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.891+0000 7f95fa2f6700 1 -- 172.21.15.32:0/190567862 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95f405f5c0 con 0x7f95f40fb1a0 2024-05-12T04:09:52.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.892+0000 7f95f0ff9700 1 -- 172.21.15.32:0/190567862 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f95e40096a0 con 0x7f95f40fb1a0 2024-05-12T04:09:52.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.892+0000 7f95f0ff9700 1 --2- 172.21.15.32:0/190567862 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f95dc06cc20 0x7f95dc06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:09:52.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.892+0000 7f95f0ff9700 1 -- 172.21.15.32:0/190567862 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f95e408bed0 con 0x7f95f40fb1a0 2024-05-12T04:09:52.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.892+0000 7f95f37fe700 1 --2- 172.21.15.32:0/190567862 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f95dc06cc20 0x7f95dc06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:09:52.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.892+0000 7f95f37fe700 1 --2- 172.21.15.32:0/190567862 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f95dc06cc20 0x7f95dc06f0d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f95e800b900 tx=0x7f95e8009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:09:52.897 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:52.896+0000 7f95f0ff9700 1 -- 172.21.15.32:0/190567862 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f95e40598c0 con 0x7f95f40fb1a0 2024-05-12T04:09:53.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.149+0000 7f95fa2f6700 1 -- 172.21.15.32:0/190567862 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f95f4105a90 con 0x7f95f40fb1a0 2024-05-12T04:09:53.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.150+0000 7f95f0ff9700 1 -- 172.21.15.32:0/190567862 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 37 v37) v1 ==== 76+0+1871 (secure 0 0 0) 0x7f95e4059450 con 0x7f95f40fb1a0 2024-05-12T04:09:53.150 INFO:teuthology.orchestra.run.smithi032.stdout:e37 2024-05-12T04:09:53.150 INFO:teuthology.orchestra.run.smithi032.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-12T04:09:53.151 INFO:teuthology.orchestra.run.smithi032.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-12T04:09:53.151 INFO:teuthology.orchestra.run.smithi032.stdout:legacy client fscid: 1 2024-05-12T04:09:53.151 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:09:53.151 INFO:teuthology.orchestra.run.smithi032.stdout:Filesystem 'cephfs' (1) 2024-05-12T04:09:53.151 INFO:teuthology.orchestra.run.smithi032.stdout:fs_name cephfs 2024-05-12T04:09:53.151 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 37 2024-05-12T04:09:53.151 INFO:teuthology.orchestra.run.smithi032.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-12T04:09:53.151 INFO:teuthology.orchestra.run.smithi032.stdout:created 2024-05-12T03:57:48.904614+0000 2024-05-12T04:09:53.151 INFO:teuthology.orchestra.run.smithi032.stdout:modified 2024-05-12T04:09:05.133553+0000 2024-05-12T04:09:53.151 INFO:teuthology.orchestra.run.smithi032.stdout:tableserver 0 2024-05-12T04:09:53.151 INFO:teuthology.orchestra.run.smithi032.stdout:root 0 2024-05-12T04:09:53.151 INFO:teuthology.orchestra.run.smithi032.stdout:session_timeout 60 2024-05-12T04:09:53.151 INFO:teuthology.orchestra.run.smithi032.stdout:session_autoclose 300 2024-05-12T04:09:53.151 INFO:teuthology.orchestra.run.smithi032.stdout:max_file_size 1099511627776 2024-05-12T04:09:53.151 INFO:teuthology.orchestra.run.smithi032.stdout:max_xattr_size 65536 2024-05-12T04:09:53.152 INFO:teuthology.orchestra.run.smithi032.stdout:required_client_features {} 2024-05-12T04:09:53.152 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure 0 2024-05-12T04:09:53.152 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure_osd_epoch 84 2024-05-12T04:09:53.152 INFO:teuthology.orchestra.run.smithi032.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-12T04:09:53.152 INFO:teuthology.orchestra.run.smithi032.stdout:max_mds 1 2024-05-12T04:09:53.152 INFO:teuthology.orchestra.run.smithi032.stdout:in 0 2024-05-12T04:09:53.152 INFO:teuthology.orchestra.run.smithi032.stdout:up {0=34404} 2024-05-12T04:09:53.152 INFO:teuthology.orchestra.run.smithi032.stdout:failed 2024-05-12T04:09:53.152 INFO:teuthology.orchestra.run.smithi032.stdout:damaged 2024-05-12T04:09:53.152 INFO:teuthology.orchestra.run.smithi032.stdout:stopped 2024-05-12T04:09:53.152 INFO:teuthology.orchestra.run.smithi032.stdout:data_pools [3] 2024-05-12T04:09:53.152 INFO:teuthology.orchestra.run.smithi032.stdout:metadata_pool 2 2024-05-12T04:09:53.152 INFO:teuthology.orchestra.run.smithi032.stdout:inline_data enabled 2024-05-12T04:09:53.152 INFO:teuthology.orchestra.run.smithi032.stdout:balancer 2024-05-12T04:09:53.153 INFO:teuthology.orchestra.run.smithi032.stdout:bal_rank_mask -1 2024-05-12T04:09:53.153 INFO:teuthology.orchestra.run.smithi032.stdout:standby_count_wanted 1 2024-05-12T04:09:53.153 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.ialtqb{0:34404} state up:active seq 20 join_fscid=1 addr [v2:172.21.15.32:6826/1080358503,v1:172.21.15.32:6827/1080358503] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:09:53.153 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.qpsawh{0:34436} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.32:6828/3603452506,v1:172.21.15.32:6829/3603452506] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:09:53.153 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:09:53.153 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:09:53.153 INFO:teuthology.orchestra.run.smithi032.stdout:Standby daemons: 2024-05-12T04:09:53.153 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:09:53.153 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi033.jgbjhk{-1:44343} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6824/997254871,v1:172.21.15.33:6825/997254871] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:09:53.153 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi033.hglwml{-1:44373} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/1983593176,v1:172.21.15.33:6827/1983593176] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:09:53.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.152+0000 7f95fa2f6700 1 -- 172.21.15.32:0/190567862 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f95dc06cc20 msgr2=0x7f95dc06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:09:53.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.152+0000 7f95fa2f6700 1 --2- 172.21.15.32:0/190567862 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f95dc06cc20 0x7f95dc06f0d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f95e800b900 tx=0x7f95e8009e30 comp rx=0 tx=0).stop 2024-05-12T04:09:53.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.152+0000 7f95fa2f6700 1 -- 172.21.15.32:0/190567862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95f40fb1a0 msgr2=0x7f95f410db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:09:53.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.152+0000 7f95fa2f6700 1 --2- 172.21.15.32:0/190567862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95f40fb1a0 0x7f95f410db40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f95e4002a50 tx=0x7f95e40046b0 comp rx=0 tx=0).stop 2024-05-12T04:09:53.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.152+0000 7f95fa2f6700 1 -- 172.21.15.32:0/190567862 shutdown_connections 2024-05-12T04:09:53.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.152+0000 7f95fa2f6700 1 --2- 172.21.15.32:0/190567862 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f95dc06cc20 0x7f95dc06f0d0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:53.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.152+0000 7f95fa2f6700 1 --2- 172.21.15.32:0/190567862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95f40fb1a0 0x7f95f410db40 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:53.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.152+0000 7f95fa2f6700 1 --2- 172.21.15.32:0/190567862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95f40fbaf0 0x7f95f410e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:53.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.152+0000 7f95fa2f6700 1 -- 172.21.15.32:0/190567862 >> 172.21.15.32:0/190567862 conn(0x7f95f40f6b80 msgr2=0x7f95f40f8340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:09:53.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.152+0000 7f95fa2f6700 1 -- 172.21.15.32:0/190567862 shutdown_connections 2024-05-12T04:09:53.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.152+0000 7f95fa2f6700 1 -- 172.21.15.32:0/190567862 wait complete. 2024-05-12T04:09:53.155 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 37 2024-05-12T04:09:53.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.264+0000 7f0f01e9e700 1 -- 172.21.15.32:0/3220539760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0efc0ffee0 msgr2=0x7f0efc102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:09:53.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.264+0000 7f0f01e9e700 1 --2- 172.21.15.32:0/3220539760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0efc0ffee0 0x7f0efc102300 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0004940 tx=0x7f0ef0009e30 comp rx=0 tx=0).stop 2024-05-12T04:09:53.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.264+0000 7f0f01e9e700 1 -- 172.21.15.32:0/3220539760 shutdown_connections 2024-05-12T04:09:53.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.264+0000 7f0f01e9e700 1 --2- 172.21.15.32:0/3220539760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0efc0ffee0 0x7f0efc102300 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:53.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.264+0000 7f0f01e9e700 1 --2- 172.21.15.32:0/3220539760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0efc0662c0 0x7f0efc0ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:53.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.264+0000 7f0f01e9e700 1 -- 172.21.15.32:0/3220539760 >> 172.21.15.32:0/3220539760 conn(0x7f0efc0f7430 msgr2=0x7f0efc0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:09:53.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.264+0000 7f0f01e9e700 1 -- 172.21.15.32:0/3220539760 shutdown_connections 2024-05-12T04:09:53.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.264+0000 7f0f01e9e700 1 -- 172.21.15.32:0/3220539760 wait complete. 2024-05-12T04:09:53.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.264+0000 7f0f01e9e700 1 Processor -- start 2024-05-12T04:09:53.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.265+0000 7f0f01e9e700 1 -- start start 2024-05-12T04:09:53.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.265+0000 7f0f01e9e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0efc0662c0 0x7f0efc10d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:09:53.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.265+0000 7f0f01e9e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0efc0ffee0 0x7f0efc10deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:09:53.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.265+0000 7f0f01e9e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0efc10e4b0 con 0x7f0efc0ffee0 2024-05-12T04:09:53.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.265+0000 7f0f01e9e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0efc10ec60 con 0x7f0efc0662c0 2024-05-12T04:09:53.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.265+0000 7f0efaffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0efc0ffee0 0x7f0efc10deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:09:53.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.265+0000 7f0efb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0efc0662c0 0x7f0efc10d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:09:53.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.265+0000 7f0efaffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0efc0ffee0 0x7f0efc10deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45510/0 (socket says 172.21.15.32:45510) 2024-05-12T04:09:53.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.265+0000 7f0efaffd700 1 -- 172.21.15.32:0/1975130747 learned_addr learned my addr 172.21.15.32:0/1975130747 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:09:53.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.265+0000 7f0efaffd700 1 -- 172.21.15.32:0/1975130747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0efc0662c0 msgr2=0x7f0efc10d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:09:53.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.265+0000 7f0efaffd700 1 --2- 172.21.15.32:0/1975130747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0efc0662c0 0x7f0efc10d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:53.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.265+0000 7f0efaffd700 1 -- 172.21.15.32:0/1975130747 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ef0005040 con 0x7f0efc0ffee0 2024-05-12T04:09:53.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.266+0000 7f0efaffd700 1 --2- 172.21.15.32:0/1975130747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0efc0ffee0 0x7f0efc10deb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0ef00137b0 tx=0x7f0ef0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:09:53.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.266+0000 7f0ee3fff700 1 -- 172.21.15.32:0/1975130747 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ef000e070 con 0x7f0efc0ffee0 2024-05-12T04:09:53.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.266+0000 7f0f01e9e700 1 -- 172.21.15.32:0/1975130747 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0efc10eee0 con 0x7f0efc0ffee0 2024-05-12T04:09:53.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.266+0000 7f0ee3fff700 1 -- 172.21.15.32:0/1975130747 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0ef001c400 con 0x7f0efc0ffee0 2024-05-12T04:09:53.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.266+0000 7f0ee3fff700 1 -- 172.21.15.32:0/1975130747 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ef000b650 con 0x7f0efc0ffee0 2024-05-12T04:09:53.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.266+0000 7f0f01e9e700 1 -- 172.21.15.32:0/1975130747 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0efc10f3c0 con 0x7f0efc0ffee0 2024-05-12T04:09:53.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.267+0000 7f0f01e9e700 1 -- 172.21.15.32:0/1975130747 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0efc05f5c0 con 0x7f0efc0ffee0 2024-05-12T04:09:53.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.267+0000 7f0ee3fff700 1 -- 172.21.15.32:0/1975130747 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f0ef001c9e0 con 0x7f0efc0ffee0 2024-05-12T04:09:53.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.268+0000 7f0ee3fff700 1 --2- 172.21.15.32:0/1975130747 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f0ee406cc20 0x7f0ee406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:09:53.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.268+0000 7f0ee3fff700 1 -- 172.21.15.32:0/1975130747 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f0ef008d6e0 con 0x7f0efc0ffee0 2024-05-12T04:09:53.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.268+0000 7f0efb7fe700 1 --2- 172.21.15.32:0/1975130747 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f0ee406cc20 0x7f0ee406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:09:53.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.268+0000 7f0efb7fe700 1 --2- 172.21.15.32:0/1975130747 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f0ee406cc20 0x7f0ee406f0d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0eec009b80 tx=0x7f0eec00b040 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:09:53.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.272+0000 7f0ee3fff700 1 -- 172.21.15.32:0/1975130747 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f0ef005b150 con 0x7f0efc0ffee0 2024-05-12T04:09:53.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.524+0000 7f0f01e9e700 1 -- 172.21.15.32:0/1975130747 --> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0efc10f710 con 0x7f0ee406cc20 2024-05-12T04:09:53.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.526+0000 7f0ee3fff700 1 -- 172.21.15.32:0/1975130747 <== mgr.24593 v2:172.21.15.32:6800/2957111591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+440 (secure 0 0 0) 0x7f0efc10f710 con 0x7f0ee406cc20 2024-05-12T04:09:53.527 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-05-12T04:09:53.527 INFO:teuthology.orchestra.run.smithi032.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:19c9734256aa43a60e203feb242f50bf8b7f58666d520536af43cc4fee96173a", 2024-05-12T04:09:53.527 INFO:teuthology.orchestra.run.smithi032.stdout: "in_progress": true, 2024-05-12T04:09:53.527 INFO:teuthology.orchestra.run.smithi032.stdout: "which": "Upgrading all daemon types on all hosts", 2024-05-12T04:09:53.527 INFO:teuthology.orchestra.run.smithi032.stdout: "services_complete": [ 2024-05-12T04:09:53.527 INFO:teuthology.orchestra.run.smithi032.stdout: "mds", 2024-05-12T04:09:53.528 INFO:teuthology.orchestra.run.smithi032.stdout: "osd", 2024-05-12T04:09:53.528 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr", 2024-05-12T04:09:53.528 INFO:teuthology.orchestra.run.smithi032.stdout: "mon", 2024-05-12T04:09:53.528 INFO:teuthology.orchestra.run.smithi032.stdout: "crash" 2024-05-12T04:09:53.528 INFO:teuthology.orchestra.run.smithi032.stdout: ], 2024-05-12T04:09:53.528 INFO:teuthology.orchestra.run.smithi032.stdout: "progress": "16/21 daemons upgraded", 2024-05-12T04:09:53.528 INFO:teuthology.orchestra.run.smithi032.stdout: "message": "Currently upgrading alertmanager daemons", 2024-05-12T04:09:53.528 INFO:teuthology.orchestra.run.smithi032.stdout: "is_paused": false 2024-05-12T04:09:53.528 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-05-12T04:09:53.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.528+0000 7f0f01e9e700 1 -- 172.21.15.32:0/1975130747 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f0ee406cc20 msgr2=0x7f0ee406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:09:53.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.528+0000 7f0f01e9e700 1 --2- 172.21.15.32:0/1975130747 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f0ee406cc20 0x7f0ee406f0d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0eec009b80 tx=0x7f0eec00b040 comp rx=0 tx=0).stop 2024-05-12T04:09:53.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.528+0000 7f0f01e9e700 1 -- 172.21.15.32:0/1975130747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0efc0ffee0 msgr2=0x7f0efc10deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:09:53.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.528+0000 7f0f01e9e700 1 --2- 172.21.15.32:0/1975130747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0efc0ffee0 0x7f0efc10deb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0ef00137b0 tx=0x7f0ef0009e30 comp rx=0 tx=0).stop 2024-05-12T04:09:53.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.528+0000 7f0f01e9e700 1 -- 172.21.15.32:0/1975130747 shutdown_connections 2024-05-12T04:09:53.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.528+0000 7f0f01e9e700 1 --2- 172.21.15.32:0/1975130747 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f0ee406cc20 0x7f0ee406f0d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:53.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.528+0000 7f0f01e9e700 1 --2- 172.21.15.32:0/1975130747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0efc0662c0 0x7f0efc10d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:53.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.528+0000 7f0f01e9e700 1 --2- 172.21.15.32:0/1975130747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0efc0ffee0 0x7f0efc10deb0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:53.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.528+0000 7f0f01e9e700 1 -- 172.21.15.32:0/1975130747 >> 172.21.15.32:0/1975130747 conn(0x7f0efc0f7430 msgr2=0x7f0efc0f9810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:09:53.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.528+0000 7f0f01e9e700 1 -- 172.21.15.32:0/1975130747 shutdown_connections 2024-05-12T04:09:53.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.528+0000 7f0f01e9e700 1 -- 172.21.15.32:0/1975130747 wait complete. 2024-05-12T04:09:53.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:53 smithi032 ceph-mon[120392]: from='client.34500 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:09:53.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:53 smithi032 ceph-mon[120392]: pgmap v236: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:09:53.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:53 smithi032 ceph-mon[120392]: from='client.34504 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:09:53.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:53 smithi032 ceph-mon[120392]: from='client.34508 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:09:53.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:53 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/764581490' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:53.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:53 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/190567862' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-12T04:09:53.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:53 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:53.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:53 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:53.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.638+0000 7fe2d8f46700 1 -- 172.21.15.32:0/2785299853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2d40fbb30 msgr2=0x7fe2d4102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:09:53.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.638+0000 7fe2d8f46700 1 --2- 172.21.15.32:0/2785299853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2d40fbb30 0x7fe2d4102340 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe2c8004940 tx=0x7fe2c8009e30 comp rx=0 tx=0).stop 2024-05-12T04:09:53.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.638+0000 7fe2d8f46700 1 -- 172.21.15.32:0/2785299853 shutdown_connections 2024-05-12T04:09:53.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.638+0000 7fe2d8f46700 1 --2- 172.21.15.32:0/2785299853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2d40fbb30 0x7fe2d4102340 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:53.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.638+0000 7fe2d8f46700 1 --2- 172.21.15.32:0/2785299853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2d40fb1c0 0x7fe2d40fb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:53.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.638+0000 7fe2d8f46700 1 -- 172.21.15.32:0/2785299853 >> 172.21.15.32:0/2785299853 conn(0x7fe2d40f6b80 msgr2=0x7fe2d40f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:09:53.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.638+0000 7fe2d8f46700 1 -- 172.21.15.32:0/2785299853 shutdown_connections 2024-05-12T04:09:53.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.638+0000 7fe2d8f46700 1 -- 172.21.15.32:0/2785299853 wait complete. 2024-05-12T04:09:53.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.639+0000 7fe2d8f46700 1 Processor -- start 2024-05-12T04:09:53.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.639+0000 7fe2d8f46700 1 -- start start 2024-05-12T04:09:53.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.639+0000 7fe2d8f46700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2d40fb1c0 0x7fe2d410d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:09:53.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.639+0000 7fe2d8f46700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2d40fbb30 0x7fe2d410dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:09:53.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.639+0000 7fe2d8f46700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d410e4c0 con 0x7fe2d40fbb30 2024-05-12T04:09:53.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.639+0000 7fe2d8f46700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d410ecc0 con 0x7fe2d40fb1c0 2024-05-12T04:09:53.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.639+0000 7fe2d1d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2d40fbb30 0x7fe2d410dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:09:53.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.639+0000 7fe2d1d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2d40fbb30 0x7fe2d410dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45534/0 (socket says 172.21.15.32:45534) 2024-05-12T04:09:53.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.639+0000 7fe2d1d9b700 1 -- 172.21.15.32:0/1221977726 learned_addr learned my addr 172.21.15.32:0/1221977726 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:09:53.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.639+0000 7fe2d259c700 1 --2- 172.21.15.32:0/1221977726 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2d40fb1c0 0x7fe2d410d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:09:53.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.640+0000 7fe2d1d9b700 1 -- 172.21.15.32:0/1221977726 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2d40fb1c0 msgr2=0x7fe2d410d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:09:53.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.640+0000 7fe2d1d9b700 1 --2- 172.21.15.32:0/1221977726 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2d40fb1c0 0x7fe2d410d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:53.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.640+0000 7fe2d1d9b700 1 -- 172.21.15.32:0/1221977726 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2c8005040 con 0x7fe2d40fbb30 2024-05-12T04:09:53.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.640+0000 7fe2d1d9b700 1 --2- 172.21.15.32:0/1221977726 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2d40fbb30 0x7fe2d410dec0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe2c8009fd0 tx=0x7fe2c8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:09:53.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.640+0000 7fe2c2ffd700 1 -- 172.21.15.32:0/1221977726 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe2c800e040 con 0x7fe2d40fbb30 2024-05-12T04:09:53.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.640+0000 7fe2c2ffd700 1 -- 172.21.15.32:0/1221977726 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe2c8004290 con 0x7fe2d40fbb30 2024-05-12T04:09:53.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.640+0000 7fe2d8f46700 1 -- 172.21.15.32:0/1221977726 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2d410ef40 con 0x7fe2d40fbb30 2024-05-12T04:09:53.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.640+0000 7fe2c2ffd700 1 -- 172.21.15.32:0/1221977726 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe2c801c3f0 con 0x7fe2d40fbb30 2024-05-12T04:09:53.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.640+0000 7fe2d8f46700 1 -- 172.21.15.32:0/1221977726 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2d410f480 con 0x7fe2d40fbb30 2024-05-12T04:09:53.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.641+0000 7fe2d8f46700 1 -- 172.21.15.32:0/1221977726 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2d405f5c0 con 0x7fe2d40fbb30 2024-05-12T04:09:53.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.642+0000 7fe2c2ffd700 1 -- 172.21.15.32:0/1221977726 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fe2c8013790 con 0x7fe2d40fbb30 2024-05-12T04:09:53.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.643+0000 7fe2c2ffd700 1 --2- 172.21.15.32:0/1221977726 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fe2bc06cc30 0x7fe2bc06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:09:53.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.643+0000 7fe2c2ffd700 1 -- 172.21.15.32:0/1221977726 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fe2c808d4b0 con 0x7fe2d40fbb30 2024-05-12T04:09:53.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.643+0000 7fe2d259c700 1 --2- 172.21.15.32:0/1221977726 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fe2bc06cc30 0x7fe2bc06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:09:53.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.643+0000 7fe2d259c700 1 --2- 172.21.15.32:0/1221977726 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fe2bc06cc30 0x7fe2bc06f0e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe2d40fe2b0 tx=0x7fe2c4009040 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:09:53.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.647+0000 7fe2c2ffd700 1 -- 172.21.15.32:0/1221977726 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fe2c805af20 con 0x7fe2d40fbb30 2024-05-12T04:09:53.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:53 smithi033 ceph-mon[107305]: from='client.34500 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:09:53.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:53 smithi033 ceph-mon[107305]: pgmap v236: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:09:53.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:53 smithi033 ceph-mon[107305]: from='client.34504 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:09:53.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:53 smithi033 ceph-mon[107305]: from='client.34508 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:09:53.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:53 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/764581490' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:09:53.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:53 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/190567862' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-12T04:09:53.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:53 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:53.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:53 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:53.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.954+0000 7fe2d8f46700 1 -- 172.21.15.32:0/1221977726 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe2d410f740 con 0x7fe2d40fbb30 2024-05-12T04:09:53.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.954+0000 7fe2c2ffd700 1 -- 172.21.15.32:0/1221977726 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fe2c8013420 con 0x7fe2d40fbb30 2024-05-12T04:09:53.955 INFO:teuthology.orchestra.run.smithi032.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-12T04:09:53.955 INFO:teuthology.orchestra.run.smithi032.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-12T04:09:53.955 INFO:teuthology.orchestra.run.smithi032.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-12T04:09:53.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.956+0000 7fe2d8f46700 1 -- 172.21.15.32:0/1221977726 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fe2bc06cc30 msgr2=0x7fe2bc06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:09:53.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.956+0000 7fe2d8f46700 1 --2- 172.21.15.32:0/1221977726 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fe2bc06cc30 0x7fe2bc06f0e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe2d40fe2b0 tx=0x7fe2c4009040 comp rx=0 tx=0).stop 2024-05-12T04:09:53.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.956+0000 7fe2d8f46700 1 -- 172.21.15.32:0/1221977726 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2d40fbb30 msgr2=0x7fe2d410dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:09:53.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.956+0000 7fe2d8f46700 1 --2- 172.21.15.32:0/1221977726 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2d40fbb30 0x7fe2d410dec0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe2c8009fd0 tx=0x7fe2c8009e30 comp rx=0 tx=0).stop 2024-05-12T04:09:53.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.956+0000 7fe2d8f46700 1 -- 172.21.15.32:0/1221977726 shutdown_connections 2024-05-12T04:09:53.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.956+0000 7fe2d8f46700 1 --2- 172.21.15.32:0/1221977726 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fe2bc06cc30 0x7fe2bc06f0e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:53.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.956+0000 7fe2d8f46700 1 --2- 172.21.15.32:0/1221977726 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2d40fb1c0 0x7fe2d410d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:53.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.956+0000 7fe2d8f46700 1 --2- 172.21.15.32:0/1221977726 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2d40fbb30 0x7fe2d410dec0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:09:53.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.956+0000 7fe2d8f46700 1 -- 172.21.15.32:0/1221977726 >> 172.21.15.32:0/1221977726 conn(0x7fe2d40f6b80 msgr2=0x7fe2d41004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:09:53.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.956+0000 7fe2d8f46700 1 -- 172.21.15.32:0/1221977726 shutdown_connections 2024-05-12T04:09:53.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:09:53.956+0000 7fe2d8f46700 1 -- 172.21.15.32:0/1221977726 wait complete. 2024-05-12T04:09:54.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:54 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:09:54.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:54 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/1221977726' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-12T04:09:54.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:54 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:09:54.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:54 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/1221977726' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-12T04:09:55.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:55 smithi032 ceph-mon[120392]: from='client.34520 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:09:55.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:55 smithi032 ceph-mon[120392]: pgmap v237: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:09:55.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:55 smithi033 ceph-mon[107305]: from='client.34520 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:09:55.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:55 smithi033 ceph-mon[107305]: pgmap v237: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:09:57.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:57 smithi033 ceph-mon[107305]: pgmap v238: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:09:57.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:57 smithi032 ceph-mon[120392]: pgmap v238: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:09:59.031 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:58 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:59.031 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:58 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:59.031 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:58 smithi032 ceph-mon[120392]: pgmap v239: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:09:59.031 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:58 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:59.031 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:09:58 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:59.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:58 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:59.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:58 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:59.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:58 smithi033 ceph-mon[107305]: pgmap v239: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:09:59.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:58 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:09:59.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:09:58 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:10:01.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:00 smithi033 ceph-mon[107305]: pgmap v240: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:10:01.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:00 smithi033 ceph-mon[107305]: overall HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-12T04:10:01.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:00 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:10:01.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:00 smithi032 ceph-mon[120392]: pgmap v240: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:10:01.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:00 smithi032 ceph-mon[120392]: overall HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-12T04:10:01.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:00 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:10:03.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:03 smithi032 ceph-mon[120392]: pgmap v241: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:10:03.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:03 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:10:03.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:03 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:10:03.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:03 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:10:03.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:03 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T04:10:03.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:03 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:10:03.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:03 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-12T04:10:03.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:03 smithi032 ceph-mon[120392]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-12T04:10:03.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:03 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:10:03.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:03 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:03.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:03 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:03.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:03 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:03.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:03 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:03.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:03 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:03.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:03 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:03.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:03 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:03.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:03 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:03.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:03 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:03.542 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:03 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:03.542 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:03 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:03.542 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:03 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:03.542 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:03 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:03.542 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:03 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:03.542 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:03 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:03.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:03 smithi033 ceph-mon[107305]: pgmap v241: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:10:03.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:03 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:10:03.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:03 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:10:03.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:03 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:10:03.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:03 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T04:10:03.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:03 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:10:03.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:03 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-12T04:10:03.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:03 smithi033 ceph-mon[107305]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-05-12T04:10:03.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:03 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:10:03.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:03 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:03.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:03 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:03.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:03 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:03.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:03 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:03.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:03 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:03.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:03 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:03.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:03 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:03.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:03 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:03.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:03 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:03.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:03 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:03.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:03 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:03.664 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:03 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:03.664 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:03 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:03.664 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:03 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:03.664 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:03 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:05.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:05 smithi032 ceph-mon[120392]: pgmap v242: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:10:05.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:05 smithi032 ceph-mon[120392]: Upgrade: Updating grafana.smithi032 2024-05-12T04:10:05.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:05 smithi032 ceph-mon[120392]: Deploying daemon grafana.smithi032 on smithi032 2024-05-12T04:10:05.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:05 smithi033 ceph-mon[107305]: pgmap v242: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:10:05.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:05 smithi033 ceph-mon[107305]: Upgrade: Updating grafana.smithi032 2024-05-12T04:10:05.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:05 smithi033 ceph-mon[107305]: Deploying daemon grafana.smithi032 on smithi032 2024-05-12T04:10:07.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:07 smithi032 ceph-mon[120392]: pgmap v243: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:10:07.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:07 smithi033 ceph-mon[107305]: pgmap v243: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:10:09.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:09 smithi032 ceph-mon[120392]: pgmap v244: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:10:09.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:09 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:10:09.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:09 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:10:09.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:09 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:10:09.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:09 smithi033 ceph-mon[107305]: pgmap v244: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:10:09.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:09 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:10:09.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:09 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:10:09.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:09 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:10:11.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:11 smithi032 ceph-mon[120392]: pgmap v245: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:10:11.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:11 smithi033 ceph-mon[107305]: pgmap v245: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:10:13.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:13 smithi032 ceph-mon[120392]: pgmap v246: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:10:13.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:13 smithi033 ceph-mon[107305]: pgmap v246: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:10:14.800 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:14 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:10:14.800 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:14 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:10:14.800 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:14 smithi032 ceph-mon[120392]: pgmap v247: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:10:14.800 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:14 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:10:14.800 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:14 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:10:14.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:14 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:10:14.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:14 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:10:14.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:14 smithi033 ceph-mon[107305]: pgmap v247: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:10:14.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:14 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:10:14.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:14 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:10:16.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:15 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:10:16.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:15 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:10:17.129 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:16 smithi032 ceph-mon[120392]: pgmap v248: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:10:17.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:16 smithi033 ceph-mon[107305]: pgmap v248: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:10:19.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: pgmap v249: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:10:19.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:10:19.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:10:19.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:10:19.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T04:10:19.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:10:19.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-12T04:10:19.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-12T04:10:19.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:10:19.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:19.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:19.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:19.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:19.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:19.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:19.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:19.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:19.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:19.292 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:19.292 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:19.292 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:19.292 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:19.292 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:19.292 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:19.292 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:19.292 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:19.292 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:19.292 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: Upgrade: Finalizing container_image settings 2024-05-12T04:10:19.292 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:10:19.292 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-05-12T04:10:19.292 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-05-12T04:10:19.292 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-12T04:10:19.292 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-05-12T04:10:19.293 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-05-12T04:10:19.293 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-05-12T04:10:19.293 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-05-12T04:10:19.293 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-05-12T04:10:19.293 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-05-12T04:10:19.293 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-05-12T04:10:19.293 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-05-12T04:10:19.293 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-05-12T04:10:19.293 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-05-12T04:10:19.293 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-05-12T04:10:19.293 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-12T04:10:19.293 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-05-12T04:10:19.293 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-05-12T04:10:19.293 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-05-12T04:10:19.293 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-05-12T04:10:19.294 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-05-12T04:10:19.294 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-05-12T04:10:19.294 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-05-12T04:10:19.294 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-05-12T04:10:19.294 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-12T04:10:19.294 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-12T04:10:19.294 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-12T04:10:19.294 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-12T04:10:19.294 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-12T04:10:19.294 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-12T04:10:19.294 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "mon_mds_skip_sanity", "who": "mon"}]: dispatch 2024-05-12T04:10:19.294 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "config rm", "name": "mon_mds_skip_sanity", "who": "mon"}]': finished 2024-05-12T04:10:19.294 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: Upgrade: Complete! 2024-05-12T04:10:19.294 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-05-12T04:10:19.295 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-05-12T04:10:19.295 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:10:19.295 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:10:19.295 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T04:10:19.295 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:10:19.295 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:10:19.295 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:10:19.295 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T04:10:19.295 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:18 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:10:19.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: pgmap v249: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:10:19.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:10:19.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:10:19.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:10:19.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T04:10:19.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:10:19.413 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-12T04:10:19.413 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-05-12T04:10:19.413 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:10:19.413 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:19.413 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:19.413 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:19.413 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:19.413 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:19.413 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:19.413 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:19.413 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:19.413 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:19.413 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:19.413 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:19.413 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:19.414 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:19.414 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:19.414 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:19.414 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:19.414 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:19.414 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:19.414 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: Upgrade: Finalizing container_image settings 2024-05-12T04:10:19.414 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:10:19.414 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-05-12T04:10:19.414 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-05-12T04:10:19.414 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-12T04:10:19.414 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-05-12T04:10:19.414 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-05-12T04:10:19.414 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-05-12T04:10:19.415 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-05-12T04:10:19.415 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-05-12T04:10:19.415 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-05-12T04:10:19.415 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-05-12T04:10:19.415 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-05-12T04:10:19.415 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-05-12T04:10:19.415 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-05-12T04:10:19.415 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-05-12T04:10:19.415 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-12T04:10:19.415 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-05-12T04:10:19.415 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-05-12T04:10:19.415 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-05-12T04:10:19.416 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-05-12T04:10:19.416 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-05-12T04:10:19.416 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-05-12T04:10:19.416 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-05-12T04:10:19.416 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-05-12T04:10:19.416 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-12T04:10:19.416 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-12T04:10:19.416 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-12T04:10:19.416 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-12T04:10:19.416 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-12T04:10:19.416 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-05-12T04:10:19.416 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config rm", "name": "mon_mds_skip_sanity", "who": "mon"}]: dispatch 2024-05-12T04:10:19.417 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd='[{"prefix": "config rm", "name": "mon_mds_skip_sanity", "who": "mon"}]': finished 2024-05-12T04:10:19.417 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: Upgrade: Complete! 2024-05-12T04:10:19.417 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-05-12T04:10:19.417 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-05-12T04:10:19.417 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:10:19.417 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:10:19.417 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T04:10:19.417 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:10:19.417 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:10:19.417 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:10:19.417 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T04:10:19.417 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:18 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:10:21.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:20 smithi032 ceph-mon[120392]: pgmap v250: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:10:21.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:20 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:10:21.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:20 smithi033 ceph-mon[107305]: pgmap v250: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:10:21.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:20 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:10:23.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:22 smithi032 ceph-mon[120392]: pgmap v251: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:10:23.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:22 smithi033 ceph-mon[107305]: pgmap v251: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:10:24.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:24.076+0000 7f6e3c36d700 1 -- 172.21.15.32:0/1972323580 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e340662c0 msgr2=0x7f6e340ff9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:10:24.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:24.076+0000 7f6e3c36d700 1 --2- 172.21.15.32:0/1972323580 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e340662c0 0x7f6e340ff9a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6e30004940 tx=0x7f6e30009e30 comp rx=0 tx=0).stop 2024-05-12T04:10:24.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:24.076+0000 7f6e3c36d700 1 -- 172.21.15.32:0/1972323580 shutdown_connections 2024-05-12T04:10:24.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:24.076+0000 7f6e3c36d700 1 --2- 172.21.15.32:0/1972323580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e340ffee0 0x7f6e34102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:10:24.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:24.076+0000 7f6e3c36d700 1 --2- 172.21.15.32:0/1972323580 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e340662c0 0x7f6e340ff9a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:10:24.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:24.076+0000 7f6e3c36d700 1 -- 172.21.15.32:0/1972323580 >> 172.21.15.32:0/1972323580 conn(0x7f6e340f7430 msgr2=0x7f6e340f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:10:24.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:24.076+0000 7f6e3c36d700 1 -- 172.21.15.32:0/1972323580 shutdown_connections 2024-05-12T04:10:24.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:24.076+0000 7f6e3c36d700 1 -- 172.21.15.32:0/1972323580 wait complete. 2024-05-12T04:10:24.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:24.077+0000 7f6e3c36d700 1 Processor -- start 2024-05-12T04:10:24.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:24.077+0000 7f6e3c36d700 1 -- start start 2024-05-12T04:10:24.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:24.077+0000 7f6e3c36d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e340662c0 0x7f6e3410d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:10:24.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:24.077+0000 7f6e3c36d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e340ffee0 0x7f6e3410ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:10:24.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:24.077+0000 7f6e3c36d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3410e3f0 con 0x7f6e340662c0 2024-05-12T04:10:24.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:24.077+0000 7f6e3c36d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3410ec40 con 0x7f6e340ffee0 2024-05-12T04:10:24.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:24.077+0000 7f6e3a109700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e340662c0 0x7f6e3410d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:10:24.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:24.077+0000 7f6e3a109700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e340662c0 0x7f6e3410d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60976/0 (socket says 172.21.15.32:60976) 2024-05-12T04:10:24.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:24.077+0000 7f6e3a109700 1 -- 172.21.15.32:0/17117545 learned_addr learned my addr 172.21.15.32:0/17117545 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:10:24.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:24.077+0000 7f6e39908700 1 --2- 172.21.15.32:0/17117545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e340ffee0 0x7f6e3410ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:10:24.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:24.077+0000 7f6e3a109700 1 -- 172.21.15.32:0/17117545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e340ffee0 msgr2=0x7f6e3410ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:10:24.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:24.077+0000 7f6e3a109700 1 --2- 172.21.15.32:0/17117545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e340ffee0 0x7f6e3410ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:10:24.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:24.077+0000 7f6e3a109700 1 -- 172.21.15.32:0/17117545 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e30005040 con 0x7f6e340662c0 2024-05-12T04:10:24.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:24.078+0000 7f6e3a109700 1 --2- 172.21.15.32:0/17117545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e340662c0 0x7f6e3410d8b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6e30000c00 tx=0x7f6e30009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:10:24.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:24.078+0000 7f6e2affd700 1 -- 172.21.15.32:0/17117545 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e3000e070 con 0x7f6e340662c0 2024-05-12T04:10:24.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:24.078+0000 7f6e2affd700 1 -- 172.21.15.32:0/17117545 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6e3001c400 con 0x7f6e340662c0 2024-05-12T04:10:24.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:24.078+0000 7f6e3c36d700 1 -- 172.21.15.32:0/17117545 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e3410eec0 con 0x7f6e340662c0 2024-05-12T04:10:24.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:24.078+0000 7f6e2affd700 1 -- 172.21.15.32:0/17117545 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e3000b650 con 0x7f6e340662c0 2024-05-12T04:10:24.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:24.078+0000 7f6e3c36d700 1 -- 172.21.15.32:0/17117545 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e3410f3a0 con 0x7f6e340662c0 2024-05-12T04:10:24.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:24.079+0000 7f6e3c36d700 1 -- 172.21.15.32:0/17117545 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e3405f5c0 con 0x7f6e340662c0 2024-05-12T04:10:24.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:24.079+0000 7f6e2affd700 1 -- 172.21.15.32:0/17117545 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f6e3001c570 con 0x7f6e340662c0 2024-05-12T04:10:24.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:24.080+0000 7f6e2affd700 1 --2- 172.21.15.32:0/17117545 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f6e2006cc30 0x7f6e2006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:10:24.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:24.080+0000 7f6e2affd700 1 -- 172.21.15.32:0/17117545 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f6e3008d770 con 0x7f6e340662c0 2024-05-12T04:10:24.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:24.080+0000 7f6e39908700 1 --2- 172.21.15.32:0/17117545 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f6e2006cc30 0x7f6e2006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:10:24.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:24.081+0000 7f6e39908700 1 --2- 172.21.15.32:0/17117545 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f6e2006cc30 0x7f6e2006f0e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6e24004600 tx=0x7f6e2400b040 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:10:24.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:24.085+0000 7f6e2affd700 1 -- 172.21.15.32:0/17117545 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f6e3005b1e0 con 0x7f6e340662c0 2024-05-12T04:10:24.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:24.306+0000 7f6e3c36d700 1 -- 172.21.15.32:0/17117545 --> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6e3410f770 con 0x7f6e2006cc30 2024-05-12T04:10:24.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:24.307+0000 7f6e2affd700 1 -- 172.21.15.32:0/17117545 <== mgr.24593 v2:172.21.15.32:6800/2957111591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f6e3410f770 con 0x7f6e2006cc30 2024-05-12T04:10:24.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:24.310+0000 7f6e3c36d700 1 -- 172.21.15.32:0/17117545 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f6e2006cc30 msgr2=0x7f6e2006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:10:24.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:24.310+0000 7f6e3c36d700 1 --2- 172.21.15.32:0/17117545 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f6e2006cc30 0x7f6e2006f0e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6e24004600 tx=0x7f6e2400b040 comp rx=0 tx=0).stop 2024-05-12T04:10:24.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:24.310+0000 7f6e3c36d700 1 -- 172.21.15.32:0/17117545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e340662c0 msgr2=0x7f6e3410d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:10:24.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:24.310+0000 7f6e3c36d700 1 --2- 172.21.15.32:0/17117545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e340662c0 0x7f6e3410d8b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6e30000c00 tx=0x7f6e30009e30 comp rx=0 tx=0).stop 2024-05-12T04:10:24.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:24.310+0000 7f6e3c36d700 1 -- 172.21.15.32:0/17117545 shutdown_connections 2024-05-12T04:10:24.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:24.310+0000 7f6e3c36d700 1 --2- 172.21.15.32:0/17117545 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f6e2006cc30 0x7f6e2006f0e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:10:24.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:24.310+0000 7f6e3c36d700 1 --2- 172.21.15.32:0/17117545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e340662c0 0x7f6e3410d8b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:10:24.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:24.310+0000 7f6e3c36d700 1 --2- 172.21.15.32:0/17117545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e340ffee0 0x7f6e3410ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:10:24.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:24.310+0000 7f6e3c36d700 1 -- 172.21.15.32:0/17117545 >> 172.21.15.32:0/17117545 conn(0x7f6e340f7430 msgr2=0x7f6e340f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:10:24.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:24.310+0000 7f6e3c36d700 1 -- 172.21.15.32:0/17117545 shutdown_connections 2024-05-12T04:10:24.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:24.310+0000 7f6e3c36d700 1 -- 172.21.15.32:0/17117545 wait complete. 2024-05-12T04:10:25.038 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -e sha1=58249552a22f437b8cb49ed6b1068e1158bed605 -- bash -c 'ceph orch ps' 2024-05-12T04:10:25.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:24 smithi032 ceph-mon[120392]: pgmap v252: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:10:25.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:24 smithi032 ceph-mon[120392]: from='client.34528 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:10:25.373 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T04:10:25.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:24 smithi033 ceph-mon[107305]: pgmap v252: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:10:25.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:24 smithi033 ceph-mon[107305]: from='client.34528 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:10:27.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:27.132+0000 7f28d2e10700 1 -- 172.21.15.32:0/2656718254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28cc103780 msgr2=0x7f28cc0fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:10:27.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:27.132+0000 7f28d2e10700 1 --2- 172.21.15.32:0/2656718254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28cc103780 0x7f28cc0fcdf0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f28b8004940 tx=0x7f28b8009e30 comp rx=0 tx=0).stop 2024-05-12T04:10:27.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:27.132+0000 7f28d2e10700 1 -- 172.21.15.32:0/2656718254 shutdown_connections 2024-05-12T04:10:27.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:27.132+0000 7f28d2e10700 1 --2- 172.21.15.32:0/2656718254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28cc103780 0x7f28cc0fcdf0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:10:27.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:27.132+0000 7f28d2e10700 1 --2- 172.21.15.32:0/2656718254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28cc102de0 0x7f28cc1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:10:27.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:27.132+0000 7f28d2e10700 1 -- 172.21.15.32:0/2656718254 >> 172.21.15.32:0/2656718254 conn(0x7f28cc0f89b0 msgr2=0x7f28cc0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:10:27.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:27.132+0000 7f28d2e10700 1 -- 172.21.15.32:0/2656718254 shutdown_connections 2024-05-12T04:10:27.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:27.132+0000 7f28d2e10700 1 -- 172.21.15.32:0/2656718254 wait complete. 2024-05-12T04:10:27.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:27.132+0000 7f28d2e10700 1 Processor -- start 2024-05-12T04:10:27.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:27.132+0000 7f28d2e10700 1 -- start start 2024-05-12T04:10:27.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:27.133+0000 7f28d2e10700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28cc102de0 0x7f28cc113ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:10:27.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:27.133+0000 7f28d2e10700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28cc103780 0x7f28cc114000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:10:27.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:27.133+0000 7f28d2e10700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28cc114650 con 0x7f28cc103780 2024-05-12T04:10:27.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:27.133+0000 7f28d2e10700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28cc10da40 con 0x7f28cc102de0 2024-05-12T04:10:27.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:27.133+0000 7f28cbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28cc103780 0x7f28cc114000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:10:27.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:27.133+0000 7f28cbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28cc103780 0x7f28cc114000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60994/0 (socket says 172.21.15.32:60994) 2024-05-12T04:10:27.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:27.133+0000 7f28cbfff700 1 -- 172.21.15.32:0/3782715550 learned_addr learned my addr 172.21.15.32:0/3782715550 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:10:27.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:27.133+0000 7f28d0bac700 1 --2- 172.21.15.32:0/3782715550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28cc102de0 0x7f28cc113ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:10:27.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:27.133+0000 7f28cbfff700 1 -- 172.21.15.32:0/3782715550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28cc102de0 msgr2=0x7f28cc113ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:10:27.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:27.133+0000 7f28cbfff700 1 --2- 172.21.15.32:0/3782715550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28cc102de0 0x7f28cc113ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:10:27.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:27.133+0000 7f28cbfff700 1 -- 172.21.15.32:0/3782715550 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28b8005040 con 0x7f28cc103780 2024-05-12T04:10:27.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:27.133+0000 7f28d0bac700 1 --2- 172.21.15.32:0/3782715550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28cc102de0 0x7f28cc113ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-12T04:10:27.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:27.133+0000 7f28cbfff700 1 --2- 172.21.15.32:0/3782715550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28cc103780 0x7f28cc114000 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f28b8009390 tx=0x7f28b8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:10:27.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:27.134+0000 7f28c97fa700 1 -- 172.21.15.32:0/3782715550 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28b800e040 con 0x7f28cc103780 2024-05-12T04:10:27.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:27.134+0000 7f28d2e10700 1 -- 172.21.15.32:0/3782715550 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28cc10dcc0 con 0x7f28cc103780 2024-05-12T04:10:27.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:27.134+0000 7f28c97fa700 1 -- 172.21.15.32:0/3782715550 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f28b8013420 con 0x7f28cc103780 2024-05-12T04:10:27.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:27.134+0000 7f28c97fa700 1 -- 172.21.15.32:0/3782715550 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28b801c630 con 0x7f28cc103780 2024-05-12T04:10:27.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:27.134+0000 7f28d2e10700 1 -- 172.21.15.32:0/3782715550 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28cc10e280 con 0x7f28cc103780 2024-05-12T04:10:27.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:27.135+0000 7f28d2e10700 1 -- 172.21.15.32:0/3782715550 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28cc0fe530 con 0x7f28cc103780 2024-05-12T04:10:27.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:27.135+0000 7f28c97fa700 1 -- 172.21.15.32:0/3782715550 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f28b80139e0 con 0x7f28cc103780 2024-05-12T04:10:27.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:27.136+0000 7f28c97fa700 1 --2- 172.21.15.32:0/3782715550 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f28bc06cc30 0x7f28bc06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:10:27.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:27.136+0000 7f28c97fa700 1 -- 172.21.15.32:0/3782715550 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f28b808d350 con 0x7f28cc103780 2024-05-12T04:10:27.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:27.136+0000 7f28d0bac700 1 --2- 172.21.15.32:0/3782715550 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f28bc06cc30 0x7f28bc06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:10:27.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:27.136+0000 7f28d0bac700 1 --2- 172.21.15.32:0/3782715550 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f28bc06cc30 0x7f28bc06f0e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f28c00041d0 tx=0x7f28c0009040 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:10:27.175 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:27.140+0000 7f28c97fa700 1 -- 172.21.15.32:0/3782715550 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f28b805adc0 con 0x7f28cc103780 2024-05-12T04:10:27.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:27.351+0000 7f28d2e10700 1 -- 172.21.15.32:0/3782715550 --> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f28cc109d80 con 0x7f28bc06cc30 2024-05-12T04:10:27.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:27.361+0000 7f28c97fa700 1 -- 172.21.15.32:0/3782715550 <== mgr.24593 v2:172.21.15.32:6800/2957111591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f28cc109d80 con 0x7f28bc06cc30 2024-05-12T04:10:27.362 INFO:teuthology.orchestra.run.smithi032.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-12T04:10:27.362 INFO:teuthology.orchestra.run.smithi032.stdout:alertmanager.smithi032 smithi032 *:9093,9094 running (34s) 13s ago 18m 16.8M - 0.25.0 c8568f914cd2 a00759bd6b0c 2024-05-12T04:10:27.362 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi032 smithi032 running (6m) 13s ago 18m 6660k - 18.2.2-1272-g58249552 ddcb0a9d6fde 8c53f1fc7b06 2024-05-12T04:10:27.362 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi033 smithi033 running (6m) 69s ago 17m 6651k - 18.2.2-1272-g58249552 ddcb0a9d6fde cf4fe9ca070b 2024-05-12T04:10:27.362 INFO:teuthology.orchestra.run.smithi032.stdout:grafana.smithi032 smithi032 *:3000 running (18s) 13s ago 18m 46.0M - 9.4.7 954c08fa6188 6511c2d41be5 2024-05-12T04:10:27.362 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.ialtqb smithi032 running (2m) 13s ago 12m 83.0M - 18.2.2-1272-g58249552 ddcb0a9d6fde 532dd9845ce7 2024-05-12T04:10:27.362 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.qpsawh smithi032 running (2m) 13s ago 12m 86.5M - 18.2.2-1272-g58249552 ddcb0a9d6fde 68e735fd79ee 2024-05-12T04:10:27.362 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi033.hglwml smithi033 running (89s) 69s ago 12m 14.9M - 18.2.2-1272-g58249552 ddcb0a9d6fde 7240e684b225 2024-05-12T04:10:27.362 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi033.jgbjhk smithi033 running (111s) 69s ago 12m 14.1M - 18.2.2-1272-g58249552 ddcb0a9d6fde b738dbbbec0f 2024-05-12T04:10:27.363 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi032.ukatkq smithi032 *:8443,9283,8765 running (8m) 13s ago 19m 511M - 18.2.2-1272-g58249552 ddcb0a9d6fde 0d88ab9a5292 2024-05-12T04:10:27.363 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi033.pcvxwy smithi033 *:8443,9283,8765 running (7m) 69s ago 17m 451M - 18.2.2-1272-g58249552 ddcb0a9d6fde d42dcb6bd307 2024-05-12T04:10:27.363 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi032 smithi032 running (7m) 13s ago 20m 54.5M 2048M 18.2.2-1272-g58249552 ddcb0a9d6fde 81f94171c5dc 2024-05-12T04:10:27.363 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi033 smithi033 running (6m) 69s ago 17m 42.9M 2048M 18.2.2-1272-g58249552 ddcb0a9d6fde d648ba108f3e 2024-05-12T04:10:27.363 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi032 smithi032 *:9100 running (76s) 13s ago 18m 9135k - 1.5.0 0da6a335fe13 fcb3ea4af15b 2024-05-12T04:10:27.363 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi033 smithi033 *:9100 running (72s) 69s ago 17m 3632k - 1.5.0 0da6a335fe13 b75dd225ce4b 2024-05-12T04:10:27.363 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0 smithi032 running (5m) 13s ago 16m 93.0M 1570M 18.2.2-1272-g58249552 ddcb0a9d6fde 07f66c4deb3b 2024-05-12T04:10:27.363 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1 smithi032 running (5m) 13s ago 16m 100M 1570M 18.2.2-1272-g58249552 ddcb0a9d6fde 67a87807d654 2024-05-12T04:10:27.363 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2 smithi032 running (4m) 13s ago 15m 88.4M 1570M 18.2.2-1272-g58249552 ddcb0a9d6fde 4ac48d74ee8b 2024-05-12T04:10:27.363 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3 smithi033 running (3m) 69s ago 15m 103M 2594M 18.2.2-1272-g58249552 ddcb0a9d6fde 11f097a455bb 2024-05-12T04:10:27.363 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4 smithi033 running (3m) 69s ago 14m 91.5M 2594M 18.2.2-1272-g58249552 ddcb0a9d6fde db6fda7d52de 2024-05-12T04:10:27.363 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5 smithi033 running (2m) 69s ago 14m 92.7M 2594M 18.2.2-1272-g58249552 ddcb0a9d6fde 3333020b8a15 2024-05-12T04:10:27.363 INFO:teuthology.orchestra.run.smithi032.stdout:prometheus.smithi032 smithi032 *:9095 running (51s) 13s ago 17m 44.4M - 2.43.0 a07b618ecd1d fe69bbb63353 2024-05-12T04:10:27.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:27.363+0000 7f28d2e10700 1 -- 172.21.15.32:0/3782715550 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f28bc06cc30 msgr2=0x7f28bc06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:10:27.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:27.363+0000 7f28d2e10700 1 --2- 172.21.15.32:0/3782715550 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f28bc06cc30 0x7f28bc06f0e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f28c00041d0 tx=0x7f28c0009040 comp rx=0 tx=0).stop 2024-05-12T04:10:27.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:27.363+0000 7f28d2e10700 1 -- 172.21.15.32:0/3782715550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28cc103780 msgr2=0x7f28cc114000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:10:27.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:27.363+0000 7f28d2e10700 1 --2- 172.21.15.32:0/3782715550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28cc103780 0x7f28cc114000 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f28b8009390 tx=0x7f28b8009e30 comp rx=0 tx=0).stop 2024-05-12T04:10:27.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:27.364+0000 7f28d2e10700 1 -- 172.21.15.32:0/3782715550 shutdown_connections 2024-05-12T04:10:27.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:27.364+0000 7f28d2e10700 1 --2- 172.21.15.32:0/3782715550 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f28bc06cc30 0x7f28bc06f0e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:10:27.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:27.364+0000 7f28d2e10700 1 --2- 172.21.15.32:0/3782715550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28cc102de0 0x7f28cc113ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:10:27.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:27.364+0000 7f28d2e10700 1 --2- 172.21.15.32:0/3782715550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28cc103780 0x7f28cc114000 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:10:27.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:27.364+0000 7f28d2e10700 1 -- 172.21.15.32:0/3782715550 >> 172.21.15.32:0/3782715550 conn(0x7f28cc0f89b0 msgr2=0x7f28cc0f9a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:10:27.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:27.364+0000 7f28d2e10700 1 -- 172.21.15.32:0/3782715550 shutdown_connections 2024-05-12T04:10:27.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:27.364+0000 7f28d2e10700 1 -- 172.21.15.32:0/3782715550 wait complete. 2024-05-12T04:10:27.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:27 smithi033 ceph-mon[107305]: pgmap v253: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:10:27.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:27 smithi032 ceph-mon[120392]: pgmap v253: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:10:28.095 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -e sha1=58249552a22f437b8cb49ed6b1068e1158bed605 -- bash -c 'ceph orch upgrade status' 2024-05-12T04:10:28.380 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T04:10:28.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:28 smithi033 ceph-mon[107305]: from='client.34532 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:10:28.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:28 smithi032 ceph-mon[120392]: from='client.34532 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:10:29.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:29 smithi033 ceph-mon[107305]: pgmap v254: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:10:29.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:29 smithi032 ceph-mon[120392]: pgmap v254: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:10:30.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:30.327+0000 7f70502b8700 1 -- 172.21.15.32:0/4093536813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7048102e50 msgr2=0x7f7048103220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:10:30.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:30.327+0000 7f70502b8700 1 --2- 172.21.15.32:0/4093536813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7048102e50 0x7f7048103220 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7044004940 tx=0x7f7044009e30 comp rx=0 tx=0).stop 2024-05-12T04:10:30.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:30.327+0000 7f70502b8700 1 -- 172.21.15.32:0/4093536813 shutdown_connections 2024-05-12T04:10:30.327 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:30.327+0000 7f70502b8700 1 --2- 172.21.15.32:0/4093536813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7048065640 0x7f7048065ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:10:30.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:30.327+0000 7f70502b8700 1 --2- 172.21.15.32:0/4093536813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7048102e50 0x7f7048103220 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:10:30.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:30.327+0000 7f70502b8700 1 -- 172.21.15.32:0/4093536813 >> 172.21.15.32:0/4093536813 conn(0x7f7048071d00 msgr2=0x7f7048072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:10:30.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:30.327+0000 7f70502b8700 1 -- 172.21.15.32:0/4093536813 shutdown_connections 2024-05-12T04:10:30.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:30.327+0000 7f70502b8700 1 -- 172.21.15.32:0/4093536813 wait complete. 2024-05-12T04:10:30.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:30.327+0000 7f70502b8700 1 Processor -- start 2024-05-12T04:10:30.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:30.327+0000 7f70502b8700 1 -- start start 2024-05-12T04:10:30.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:30.328+0000 7f70502b8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7048065640 0x7f7048113af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:10:30.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:30.328+0000 7f70502b8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7048102e50 0x7f7048114030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:10:30.330 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:30.328+0000 7f70502b8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7048114680 con 0x7f7048065640 2024-05-12T04:10:30.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:30.328+0000 7f70502b8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f704810da70 con 0x7f7048102e50 2024-05-12T04:10:30.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:30.328+0000 7f704e054700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7048065640 0x7f7048113af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:10:30.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:30.328+0000 7f704e054700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7048065640 0x7f7048113af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:32778/0 (socket says 172.21.15.32:32778) 2024-05-12T04:10:30.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:30.328+0000 7f704e054700 1 -- 172.21.15.32:0/365103683 learned_addr learned my addr 172.21.15.32:0/365103683 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:10:30.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:30.328+0000 7f704d853700 1 --2- 172.21.15.32:0/365103683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7048102e50 0x7f7048114030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:10:30.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:30.328+0000 7f704e054700 1 -- 172.21.15.32:0/365103683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7048102e50 msgr2=0x7f7048114030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:10:30.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:30.328+0000 7f704e054700 1 --2- 172.21.15.32:0/365103683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7048102e50 0x7f7048114030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:10:30.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:30.328+0000 7f704e054700 1 -- 172.21.15.32:0/365103683 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7044005040 con 0x7f7048065640 2024-05-12T04:10:30.331 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:30.328+0000 7f704e054700 1 --2- 172.21.15.32:0/365103683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7048065640 0x7f7048113af0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7044009360 tx=0x7f7044009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:10:30.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:30.328+0000 7f703effd700 1 -- 172.21.15.32:0/365103683 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f704400e040 con 0x7f7048065640 2024-05-12T04:10:30.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:30.328+0000 7f703effd700 1 -- 172.21.15.32:0/365103683 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7044013420 con 0x7f7048065640 2024-05-12T04:10:30.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:30.328+0000 7f70502b8700 1 -- 172.21.15.32:0/365103683 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f704810dcf0 con 0x7f7048065640 2024-05-12T04:10:30.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:30.328+0000 7f703effd700 1 -- 172.21.15.32:0/365103683 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f704401c5d0 con 0x7f7048065640 2024-05-12T04:10:30.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:30.328+0000 7f70502b8700 1 -- 172.21.15.32:0/365103683 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f704810e2b0 con 0x7f7048065640 2024-05-12T04:10:30.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:30.329+0000 7f70502b8700 1 -- 172.21.15.32:0/365103683 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70480fe540 con 0x7f7048065640 2024-05-12T04:10:30.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:30.330+0000 7f703effd700 1 -- 172.21.15.32:0/365103683 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f7044013590 con 0x7f7048065640 2024-05-12T04:10:30.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:30.330+0000 7f703effd700 1 --2- 172.21.15.32:0/365103683 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f703406cc30 0x7f703406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:10:30.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:30.331+0000 7f704d853700 1 --2- 172.21.15.32:0/365103683 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f703406cc30 0x7f703406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:10:30.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:30.331+0000 7f703effd700 1 -- 172.21.15.32:0/365103683 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f704408d1e0 con 0x7f7048065640 2024-05-12T04:10:30.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:30.331+0000 7f704d853700 1 --2- 172.21.15.32:0/365103683 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f703406cc30 0x7f703406f0e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f704810ec30 tx=0x7f703800b040 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:10:30.335 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:30.335+0000 7f703effd700 1 -- 172.21.15.32:0/365103683 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f704405ac50 con 0x7f7048065640 2024-05-12T04:10:30.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:30.558+0000 7f70502b8700 1 -- 172.21.15.32:0/365103683 --> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7048109e00 con 0x7f703406cc30 2024-05-12T04:10:30.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:30.559+0000 7f703effd700 1 -- 172.21.15.32:0/365103683 <== mgr.24593 v2:172.21.15.32:6800/2957111591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f7048109e00 con 0x7f703406cc30 2024-05-12T04:10:30.560 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-05-12T04:10:30.560 INFO:teuthology.orchestra.run.smithi032.stdout: "target_image": null, 2024-05-12T04:10:30.560 INFO:teuthology.orchestra.run.smithi032.stdout: "in_progress": false, 2024-05-12T04:10:30.560 INFO:teuthology.orchestra.run.smithi032.stdout: "which": "", 2024-05-12T04:10:30.560 INFO:teuthology.orchestra.run.smithi032.stdout: "services_complete": [], 2024-05-12T04:10:30.560 INFO:teuthology.orchestra.run.smithi032.stdout: "progress": null, 2024-05-12T04:10:30.560 INFO:teuthology.orchestra.run.smithi032.stdout: "message": "", 2024-05-12T04:10:30.560 INFO:teuthology.orchestra.run.smithi032.stdout: "is_paused": false 2024-05-12T04:10:30.560 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-05-12T04:10:30.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:30.561+0000 7f70502b8700 1 -- 172.21.15.32:0/365103683 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f703406cc30 msgr2=0x7f703406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:10:30.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:30.561+0000 7f70502b8700 1 --2- 172.21.15.32:0/365103683 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f703406cc30 0x7f703406f0e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f704810ec30 tx=0x7f703800b040 comp rx=0 tx=0).stop 2024-05-12T04:10:30.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:30.561+0000 7f70502b8700 1 -- 172.21.15.32:0/365103683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7048065640 msgr2=0x7f7048113af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:10:30.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:30.561+0000 7f70502b8700 1 --2- 172.21.15.32:0/365103683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7048065640 0x7f7048113af0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7044009360 tx=0x7f7044009e30 comp rx=0 tx=0).stop 2024-05-12T04:10:30.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:30.561+0000 7f70502b8700 1 -- 172.21.15.32:0/365103683 shutdown_connections 2024-05-12T04:10:30.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:30.561+0000 7f70502b8700 1 --2- 172.21.15.32:0/365103683 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f703406cc30 0x7f703406f0e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:10:30.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:30.561+0000 7f70502b8700 1 --2- 172.21.15.32:0/365103683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7048065640 0x7f7048113af0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:10:30.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:30.561+0000 7f70502b8700 1 --2- 172.21.15.32:0/365103683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7048102e50 0x7f7048114030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:10:30.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:30.561+0000 7f70502b8700 1 -- 172.21.15.32:0/365103683 >> 172.21.15.32:0/365103683 conn(0x7f7048071d00 msgr2=0x7f7048105b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:10:30.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:30.561+0000 7f70502b8700 1 -- 172.21.15.32:0/365103683 shutdown_connections 2024-05-12T04:10:30.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:30.561+0000 7f70502b8700 1 -- 172.21.15.32:0/365103683 wait complete. 2024-05-12T04:10:31.240 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -e sha1=58249552a22f437b8cb49ed6b1068e1158bed605 -- bash -c 'ceph health detail' 2024-05-12T04:10:31.521 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T04:10:31.539 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:31 smithi032 ceph-mon[120392]: pgmap v255: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:10:31.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:31 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:10:31.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:31 smithi033 ceph-mon[107305]: pgmap v255: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:10:31.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:31 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:10:32.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:32 smithi032 ceph-mon[120392]: from='client.34536 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:10:32.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:32 smithi033 ceph-mon[107305]: from='client.34536 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:10:33.289 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:33 smithi032 ceph-mon[120392]: pgmap v256: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:10:33.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:33.315+0000 7fa124441700 1 -- 172.21.15.32:0/4205981098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa11c101d90 msgr2=0x7fa11c102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:10:33.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:33.315+0000 7fa124441700 1 --2- 172.21.15.32:0/4205981098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa11c101d90 0x7fa11c102160 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa118004940 tx=0x7fa118009e30 comp rx=0 tx=0).stop 2024-05-12T04:10:33.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:33.315+0000 7fa124441700 1 -- 172.21.15.32:0/4205981098 shutdown_connections 2024-05-12T04:10:33.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:33.315+0000 7fa124441700 1 --2- 172.21.15.32:0/4205981098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa11c0fbd40 0x7fa11c0fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:10:33.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:33.315+0000 7fa124441700 1 --2- 172.21.15.32:0/4205981098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa11c101d90 0x7fa11c102160 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:10:33.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:33.315+0000 7fa124441700 1 -- 172.21.15.32:0/4205981098 >> 172.21.15.32:0/4205981098 conn(0x7fa11c071d00 msgr2=0x7fa11c072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:10:33.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:33.316+0000 7fa124441700 1 -- 172.21.15.32:0/4205981098 shutdown_connections 2024-05-12T04:10:33.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:33.316+0000 7fa124441700 1 -- 172.21.15.32:0/4205981098 wait complete. 2024-05-12T04:10:33.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:33.316+0000 7fa124441700 1 Processor -- start 2024-05-12T04:10:33.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:33.316+0000 7fa124441700 1 -- start start 2024-05-12T04:10:33.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:33.316+0000 7fa124441700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa11c0fbd40 0x7fa11c113a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:10:33.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:33.316+0000 7fa124441700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa11c101d90 0x7fa11c113f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:10:33.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:33.316+0000 7fa124441700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c1145c0 con 0x7fa11c0fbd40 2024-05-12T04:10:33.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:33.316+0000 7fa124441700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c10d9b0 con 0x7fa11c101d90 2024-05-12T04:10:33.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:33.316+0000 7fa1221dd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa11c0fbd40 0x7fa11c113a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:10:33.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:33.316+0000 7fa1221dd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa11c0fbd40 0x7fa11c113a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57468/0 (socket says 172.21.15.32:57468) 2024-05-12T04:10:33.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:33.316+0000 7fa1221dd700 1 -- 172.21.15.32:0/3459219207 learned_addr learned my addr 172.21.15.32:0/3459219207 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:10:33.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:33.317+0000 7fa1219dc700 1 --2- 172.21.15.32:0/3459219207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa11c101d90 0x7fa11c113f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:10:33.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:33.317+0000 7fa1221dd700 1 -- 172.21.15.32:0/3459219207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa11c101d90 msgr2=0x7fa11c113f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:10:33.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:33.317+0000 7fa1221dd700 1 --2- 172.21.15.32:0/3459219207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa11c101d90 0x7fa11c113f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:10:33.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:33.317+0000 7fa1221dd700 1 -- 172.21.15.32:0/3459219207 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa118005040 con 0x7fa11c0fbd40 2024-05-12T04:10:33.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:33.317+0000 7fa1221dd700 1 --2- 172.21.15.32:0/3459219207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa11c0fbd40 0x7fa11c113a30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa118000c00 tx=0x7fa118009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:10:33.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:33.317+0000 7fa10effd700 1 -- 172.21.15.32:0/3459219207 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa118021070 con 0x7fa11c0fbd40 2024-05-12T04:10:33.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:33.317+0000 7fa10effd700 1 -- 172.21.15.32:0/3459219207 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa1180093b0 con 0x7fa11c0fbd40 2024-05-12T04:10:33.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:33.317+0000 7fa124441700 1 -- 172.21.15.32:0/3459219207 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa11c10dc30 con 0x7fa11c0fbd40 2024-05-12T04:10:33.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:33.317+0000 7fa10effd700 1 -- 172.21.15.32:0/3459219207 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa11800e070 con 0x7fa11c0fbd40 2024-05-12T04:10:33.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:33.317+0000 7fa124441700 1 -- 172.21.15.32:0/3459219207 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa11c10e1f0 con 0x7fa11c0fbd40 2024-05-12T04:10:33.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:33.318+0000 7fa124441700 1 -- 172.21.15.32:0/3459219207 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa11c0fd430 con 0x7fa11c0fbd40 2024-05-12T04:10:33.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:33.319+0000 7fa10effd700 1 -- 172.21.15.32:0/3459219207 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fa11801c500 con 0x7fa11c0fbd40 2024-05-12T04:10:33.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:33.319+0000 7fa10effd700 1 --2- 172.21.15.32:0/3459219207 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fa10806cc20 0x7fa10806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:10:33.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:33.319+0000 7fa10effd700 1 -- 172.21.15.32:0/3459219207 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fa11808d0f0 con 0x7fa11c0fbd40 2024-05-12T04:10:33.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:33.319+0000 7fa1219dc700 1 --2- 172.21.15.32:0/3459219207 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fa10806cc20 0x7fa10806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:10:33.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:33.320+0000 7fa1219dc700 1 --2- 172.21.15.32:0/3459219207 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fa10806cc20 0x7fa10806f0d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa11c10eb70 tx=0x7fa11000b040 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:10:33.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:33.324+0000 7fa10effd700 1 -- 172.21.15.32:0/3459219207 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fa11805ac00 con 0x7fa11c0fbd40 2024-05-12T04:10:33.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:33.612+0000 7fa124441700 1 -- 172.21.15.32:0/3459219207 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa11c10f0b0 con 0x7fa11c0fbd40 2024-05-12T04:10:33.613 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:33.613+0000 7fa10effd700 1 -- 172.21.15.32:0/3459219207 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fa11805a790 con 0x7fa11c0fbd40 2024-05-12T04:10:33.613 INFO:teuthology.orchestra.run.smithi032.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-05-12T04:10:33.613 INFO:teuthology.orchestra.run.smithi032.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-05-12T04:10:33.613 INFO:teuthology.orchestra.run.smithi032.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-05-12T04:10:33.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:33.615+0000 7fa124441700 1 -- 172.21.15.32:0/3459219207 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fa10806cc20 msgr2=0x7fa10806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:10:33.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:33.615+0000 7fa124441700 1 --2- 172.21.15.32:0/3459219207 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fa10806cc20 0x7fa10806f0d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa11c10eb70 tx=0x7fa11000b040 comp rx=0 tx=0).stop 2024-05-12T04:10:33.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:33.615+0000 7fa124441700 1 -- 172.21.15.32:0/3459219207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa11c0fbd40 msgr2=0x7fa11c113a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:10:33.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:33.615+0000 7fa124441700 1 --2- 172.21.15.32:0/3459219207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa11c0fbd40 0x7fa11c113a30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa118000c00 tx=0x7fa118009e30 comp rx=0 tx=0).stop 2024-05-12T04:10:33.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:33.615+0000 7fa124441700 1 -- 172.21.15.32:0/3459219207 shutdown_connections 2024-05-12T04:10:33.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:33.615+0000 7fa124441700 1 --2- 172.21.15.32:0/3459219207 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fa10806cc20 0x7fa10806f0d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:10:33.615 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:33.615+0000 7fa124441700 1 --2- 172.21.15.32:0/3459219207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa11c0fbd40 0x7fa11c113a30 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:10:33.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:33.615+0000 7fa124441700 1 --2- 172.21.15.32:0/3459219207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa11c101d90 0x7fa11c113f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:10:33.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:33.615+0000 7fa124441700 1 -- 172.21.15.32:0/3459219207 >> 172.21.15.32:0/3459219207 conn(0x7fa11c071d00 msgr2=0x7fa11c0fa0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:10:33.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:33.615+0000 7fa124441700 1 -- 172.21.15.32:0/3459219207 shutdown_connections 2024-05-12T04:10:33.616 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:33.615+0000 7fa124441700 1 -- 172.21.15.32:0/3459219207 wait complete. 2024-05-12T04:10:33.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:33 smithi033 ceph-mon[107305]: pgmap v256: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:10:34.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:34 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/3459219207' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-12T04:10:34.376 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -e sha1=58249552a22f437b8cb49ed6b1068e1158bed605 -- bash -c 'ceph versions' 2024-05-12T04:10:34.660 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T04:10:34.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:34 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/3459219207' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-05-12T04:10:35.479 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:35 smithi032 ceph-mon[120392]: pgmap v257: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:10:35.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:35 smithi033 ceph-mon[107305]: pgmap v257: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:10:37.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:37 smithi032 ceph-mon[120392]: pgmap v258: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:10:37.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:37 smithi033 ceph-mon[107305]: pgmap v258: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:10:37.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:37.928+0000 7f0059df6700 1 -- 172.21.15.32:0/892130812 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00540fc4d0 msgr2=0x7f0054109230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:10:37.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:37.928+0000 7f0059df6700 1 --2- 172.21.15.32:0/892130812 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00540fc4d0 0x7f0054109230 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0048004940 tx=0x7f0048009e30 comp rx=0 tx=0).stop 2024-05-12T04:10:37.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:37.929+0000 7f0059df6700 1 -- 172.21.15.32:0/892130812 shutdown_connections 2024-05-12T04:10:37.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:37.929+0000 7f0059df6700 1 --2- 172.21.15.32:0/892130812 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00540fc4d0 0x7f0054109230 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:10:37.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:37.929+0000 7f0059df6700 1 --2- 172.21.15.32:0/892130812 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00540fbbc0 0x7f00540fbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:10:37.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:37.929+0000 7f0059df6700 1 -- 172.21.15.32:0/892130812 >> 172.21.15.32:0/892130812 conn(0x7f00540f78d0 msgr2=0x7f00540f9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:10:37.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:37.929+0000 7f0059df6700 1 -- 172.21.15.32:0/892130812 shutdown_connections 2024-05-12T04:10:37.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:37.929+0000 7f0059df6700 1 -- 172.21.15.32:0/892130812 wait complete. 2024-05-12T04:10:37.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:37.929+0000 7f0059df6700 1 Processor -- start 2024-05-12T04:10:37.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:37.929+0000 7f0059df6700 1 -- start start 2024-05-12T04:10:37.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:37.930+0000 7f0059df6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00540fbbc0 0x7f0054115bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:10:37.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:37.930+0000 7f0059df6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00540fc4d0 0x7f00541160f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:10:37.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:37.930+0000 7f0059df6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0054116740 con 0x7f00540fbbc0 2024-05-12T04:10:37.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:37.930+0000 7f0059df6700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f005410fb50 con 0x7f00540fc4d0 2024-05-12T04:10:37.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:37.930+0000 7f00537fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00540fbbc0 0x7f0054115bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:10:37.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:37.930+0000 7f00537fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00540fbbc0 0x7f0054115bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57478/0 (socket says 172.21.15.32:57478) 2024-05-12T04:10:37.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:37.930+0000 7f00537fe700 1 -- 172.21.15.32:0/1902179381 learned_addr learned my addr 172.21.15.32:0/1902179381 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:10:37.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:37.930+0000 7f0052ffd700 1 --2- 172.21.15.32:0/1902179381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00540fc4d0 0x7f00541160f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:10:37.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:37.930+0000 7f00537fe700 1 -- 172.21.15.32:0/1902179381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00540fc4d0 msgr2=0x7f00541160f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:10:37.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:37.930+0000 7f00537fe700 1 --2- 172.21.15.32:0/1902179381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00540fc4d0 0x7f00541160f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:10:37.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:37.930+0000 7f00537fe700 1 -- 172.21.15.32:0/1902179381 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0048005040 con 0x7f00540fbbc0 2024-05-12T04:10:37.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:37.930+0000 7f00537fe700 1 --2- 172.21.15.32:0/1902179381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00540fbbc0 0x7f0054115bb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0044002a50 tx=0x7f00440046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:10:37.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:37.930+0000 7f003bfff700 1 -- 172.21.15.32:0/1902179381 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0044007cc0 con 0x7f00540fbbc0 2024-05-12T04:10:37.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:37.930+0000 7f0059df6700 1 -- 172.21.15.32:0/1902179381 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f005410fe30 con 0x7f00540fbbc0 2024-05-12T04:10:37.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:37.930+0000 7f003bfff700 1 -- 172.21.15.32:0/1902179381 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f004400f040 con 0x7f00540fbbc0 2024-05-12T04:10:37.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:37.930+0000 7f003bfff700 1 -- 172.21.15.32:0/1902179381 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f004400d9c0 con 0x7f00540fbbc0 2024-05-12T04:10:37.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:37.931+0000 7f0059df6700 1 -- 172.21.15.32:0/1902179381 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00541103f0 con 0x7f00540fbbc0 2024-05-12T04:10:37.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:37.931+0000 7f0059df6700 1 -- 172.21.15.32:0/1902179381 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00540666f0 con 0x7f00540fbbc0 2024-05-12T04:10:37.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:37.932+0000 7f003bfff700 1 -- 172.21.15.32:0/1902179381 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f004400db20 con 0x7f00540fbbc0 2024-05-12T04:10:37.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:37.932+0000 7f003bfff700 1 --2- 172.21.15.32:0/1902179381 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f003c06cc20 0x7f003c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:10:37.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:37.933+0000 7f003bfff700 1 -- 172.21.15.32:0/1902179381 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f004408be70 con 0x7f00540fbbc0 2024-05-12T04:10:37.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:37.933+0000 7f0052ffd700 1 --2- 172.21.15.32:0/1902179381 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f003c06cc20 0x7f003c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:10:37.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:37.933+0000 7f0052ffd700 1 --2- 172.21.15.32:0/1902179381 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f003c06cc20 0x7f003c06f0d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0054111450 tx=0x7f0048009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:10:37.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:37.937+0000 7f003bfff700 1 -- 172.21.15.32:0/1902179381 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f0044055d30 con 0x7f00540fbbc0 2024-05-12T04:10:38.226 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:38.226+0000 7f0059df6700 1 -- 172.21.15.32:0/1902179381 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f005405f5c0 con 0x7f00540fbbc0 2024-05-12T04:10:38.227 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:38.226+0000 7f003bfff700 1 -- 172.21.15.32:0/1902179381 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f0044059350 con 0x7f00540fbbc0 2024-05-12T04:10:38.227 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-05-12T04:10:38.227 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": { 2024-05-12T04:10:38.227 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 2 2024-05-12T04:10:38.227 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:10:38.227 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": { 2024-05-12T04:10:38.227 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 2 2024-05-12T04:10:38.227 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:10:38.227 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": { 2024-05-12T04:10:38.227 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 6 2024-05-12T04:10:38.227 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:10:38.228 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": { 2024-05-12T04:10:38.228 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 4 2024-05-12T04:10:38.228 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:10:38.228 INFO:teuthology.orchestra.run.smithi032.stdout: "overall": { 2024-05-12T04:10:38.228 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 14 2024-05-12T04:10:38.228 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-05-12T04:10:38.228 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-05-12T04:10:38.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:38.228+0000 7f0059df6700 1 -- 172.21.15.32:0/1902179381 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f003c06cc20 msgr2=0x7f003c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:10:38.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:38.228+0000 7f0059df6700 1 --2- 172.21.15.32:0/1902179381 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f003c06cc20 0x7f003c06f0d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0054111450 tx=0x7f0048009e30 comp rx=0 tx=0).stop 2024-05-12T04:10:38.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:38.228+0000 7f0059df6700 1 -- 172.21.15.32:0/1902179381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00540fbbc0 msgr2=0x7f0054115bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:10:38.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:38.228+0000 7f0059df6700 1 --2- 172.21.15.32:0/1902179381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00540fbbc0 0x7f0054115bb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0044002a50 tx=0x7f00440046b0 comp rx=0 tx=0).stop 2024-05-12T04:10:38.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:38.228+0000 7f0059df6700 1 -- 172.21.15.32:0/1902179381 shutdown_connections 2024-05-12T04:10:38.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:38.228+0000 7f0059df6700 1 --2- 172.21.15.32:0/1902179381 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f003c06cc20 0x7f003c06f0d0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:10:38.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:38.228+0000 7f0059df6700 1 --2- 172.21.15.32:0/1902179381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00540fbbc0 0x7f0054115bb0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:10:38.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:38.228+0000 7f0059df6700 1 --2- 172.21.15.32:0/1902179381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00540fc4d0 0x7f00541160f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:10:38.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:38.228+0000 7f0059df6700 1 -- 172.21.15.32:0/1902179381 >> 172.21.15.32:0/1902179381 conn(0x7f00540f78d0 msgr2=0x7f00540f7fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:10:38.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:38.229+0000 7f0059df6700 1 -- 172.21.15.32:0/1902179381 shutdown_connections 2024-05-12T04:10:38.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:10:38.229+0000 7f0059df6700 1 -- 172.21.15.32:0/1902179381 wait complete. 2024-05-12T04:10:38.816 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -e sha1=58249552a22f437b8cb49ed6b1068e1158bed605 -- bash -c 'echo "wait for servicemap items w/ changing names to refresh"' 2024-05-12T04:10:39.095 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T04:10:39.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:39 smithi032 ceph-mon[120392]: pgmap v259: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:10:39.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:39 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/1902179381' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:39.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:39 smithi033 ceph-mon[107305]: pgmap v259: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:10:39.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:39 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/1902179381' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:10:40.916 INFO:teuthology.orchestra.run.smithi032.stdout:wait for servicemap items w/ changing names to refresh 2024-05-12T04:10:41.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:41 smithi032 ceph-mon[120392]: pgmap v260: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:10:41.575 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -e sha1=58249552a22f437b8cb49ed6b1068e1158bed605 -- bash -c 'sleep 60' 2024-05-12T04:10:41.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:41 smithi033 ceph-mon[107305]: pgmap v260: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:10:41.862 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T04:10:43.465 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:43 smithi032 ceph-mon[120392]: pgmap v261: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:10:43.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:43 smithi033 ceph-mon[107305]: pgmap v261: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:10:45.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:45 smithi032 ceph-mon[120392]: pgmap v262: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:10:45.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:45 smithi033 ceph-mon[107305]: pgmap v262: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:10:46.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:46 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:10:46.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:46 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:10:47.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:47 smithi032 ceph-mon[120392]: pgmap v263: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:10:47.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:47 smithi033 ceph-mon[107305]: pgmap v263: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:10:49.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:49 smithi032 ceph-mon[120392]: pgmap v264: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:10:49.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:49 smithi033 ceph-mon[107305]: pgmap v264: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:10:51.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:51 smithi032 ceph-mon[120392]: pgmap v265: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:10:51.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:51 smithi033 ceph-mon[107305]: pgmap v265: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:10:53.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:53 smithi032 ceph-mon[120392]: pgmap v266: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:10:53.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:53 smithi033 ceph-mon[107305]: pgmap v266: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:10:55.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:55 smithi032 ceph-mon[120392]: pgmap v267: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:10:55.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:55 smithi033 ceph-mon[107305]: pgmap v267: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:10:57.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:57 smithi032 ceph-mon[120392]: pgmap v268: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:10:57.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:57 smithi033 ceph-mon[107305]: pgmap v268: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:10:59.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:10:59 smithi032 ceph-mon[120392]: pgmap v269: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:10:59.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:10:59 smithi033 ceph-mon[107305]: pgmap v269: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:11:01.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:11:01 smithi032 ceph-mon[120392]: pgmap v270: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:11:01.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:11:01 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:11:01.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:11:01 smithi033 ceph-mon[107305]: pgmap v270: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:11:01.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:11:01 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:11:03.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:11:03 smithi032 ceph-mon[120392]: pgmap v271: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:11:03.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:11:03 smithi033 ceph-mon[107305]: pgmap v271: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:11:05.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:11:05 smithi032 ceph-mon[120392]: pgmap v272: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:11:05.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:11:05 smithi033 ceph-mon[107305]: pgmap v272: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:11:07.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:11:07 smithi032 ceph-mon[120392]: pgmap v273: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:11:07.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:11:07 smithi033 ceph-mon[107305]: pgmap v273: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:11:09.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:11:09 smithi032 ceph-mon[120392]: pgmap v274: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:11:09.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:11:09 smithi033 ceph-mon[107305]: pgmap v274: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:11:11.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:11:11 smithi032 ceph-mon[120392]: pgmap v275: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:11:11.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:11:11 smithi033 ceph-mon[107305]: pgmap v275: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:11:13.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:11:13 smithi032 ceph-mon[120392]: pgmap v276: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:11:13.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:11:13 smithi033 ceph-mon[107305]: pgmap v276: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:11:15.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:11:15 smithi032 ceph-mon[120392]: pgmap v277: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:11:15.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:11:15 smithi033 ceph-mon[107305]: pgmap v277: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:11:16.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:11:16 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:11:16.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:11:16 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:11:17.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:11:17 smithi032 ceph-mon[120392]: pgmap v278: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:11:17.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:11:17 smithi033 ceph-mon[107305]: pgmap v278: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:11:19.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:11:19 smithi032 ceph-mon[120392]: pgmap v279: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:11:19.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:11:19 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:11:19.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:11:19 smithi033 ceph-mon[107305]: pgmap v279: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:11:19.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:11:19 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:11:20.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:11:20 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:11:20.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:11:20 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T04:11:20.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:11:20 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:11:20.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:11:20 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:11:20.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:11:20 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T04:11:20.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:11:20 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:11:21.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:11:21 smithi033 ceph-mon[107305]: pgmap v280: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:11:21.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:11:21 smithi032 ceph-mon[120392]: pgmap v280: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:11:23.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:11:23 smithi033 ceph-mon[107305]: pgmap v281: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:11:23.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:11:23 smithi032 ceph-mon[120392]: pgmap v281: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:11:25.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:11:25 smithi033 ceph-mon[107305]: pgmap v282: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:11:25.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:11:25 smithi032 ceph-mon[120392]: pgmap v282: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:11:27.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:11:27 smithi033 ceph-mon[107305]: pgmap v283: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:11:27.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:11:27 smithi032 ceph-mon[120392]: pgmap v283: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:11:29.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:11:29 smithi033 ceph-mon[107305]: pgmap v284: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:11:29.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:11:29 smithi032 ceph-mon[120392]: pgmap v284: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:11:31.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:11:31 smithi033 ceph-mon[107305]: pgmap v285: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:11:31.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:11:31 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:11:31.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:11:31 smithi032 ceph-mon[120392]: pgmap v285: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:11:31.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:11:31 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:11:33.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:11:33 smithi033 ceph-mon[107305]: pgmap v286: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:11:33.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:11:33 smithi032 ceph-mon[120392]: pgmap v286: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:11:35.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:11:35 smithi033 ceph-mon[107305]: pgmap v287: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:11:35.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:11:35 smithi032 ceph-mon[120392]: pgmap v287: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:11:37.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:11:37 smithi033 ceph-mon[107305]: pgmap v288: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:11:37.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:11:37 smithi032 ceph-mon[120392]: pgmap v288: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:11:39.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:11:39 smithi033 ceph-mon[107305]: pgmap v289: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:11:39.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:11:39 smithi032 ceph-mon[120392]: pgmap v289: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:11:41.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:11:41 smithi033 ceph-mon[107305]: pgmap v290: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:11:41.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:11:41 smithi032 ceph-mon[120392]: pgmap v290: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:11:43.653 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:11:43 smithi032 ceph-mon[120392]: pgmap v291: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:11:43.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:11:43 smithi033 ceph-mon[107305]: pgmap v291: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:11:44.532 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -e sha1=58249552a22f437b8cb49ed6b1068e1158bed605 -- bash -c 'ceph orch ps' 2024-05-12T04:11:44.819 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T04:11:45.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:11:45 smithi033 ceph-mon[107305]: pgmap v292: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:11:45.690 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:11:45 smithi032 ceph-mon[120392]: pgmap v292: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:11:46.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:46.771+0000 7f2bdf628700 1 -- 172.21.15.32:0/3227168235 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bd8101ce0 msgr2=0x7f2bd81020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:11:46.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:46.771+0000 7f2bdf628700 1 --2- 172.21.15.32:0/3227168235 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bd8101ce0 0x7f2bd81020b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc004940 tx=0x7f2bcc009e30 comp rx=0 tx=0).stop 2024-05-12T04:11:46.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:46.771+0000 7f2bdf628700 1 -- 172.21.15.32:0/3227168235 shutdown_connections 2024-05-12T04:11:46.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:46.771+0000 7f2bdf628700 1 --2- 172.21.15.32:0/3227168235 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bd8102680 0x7f2bd80fbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:11:46.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:46.771+0000 7f2bdf628700 1 --2- 172.21.15.32:0/3227168235 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bd8101ce0 0x7f2bd81020b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:11:46.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:46.771+0000 7f2bdf628700 1 -- 172.21.15.32:0/3227168235 >> 172.21.15.32:0/3227168235 conn(0x7f2bd80f7850 msgr2=0x7f2bd80f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:11:46.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:46.771+0000 7f2bdf628700 1 -- 172.21.15.32:0/3227168235 shutdown_connections 2024-05-12T04:11:46.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:46.771+0000 7f2bdf628700 1 -- 172.21.15.32:0/3227168235 wait complete. 2024-05-12T04:11:46.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:46.772+0000 7f2bdf628700 1 Processor -- start 2024-05-12T04:11:46.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:46.772+0000 7f2bdf628700 1 -- start start 2024-05-12T04:11:46.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:46.772+0000 7f2bdf628700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bd8101ce0 0x7f2bd810ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:11:46.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:46.772+0000 7f2bdf628700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bd8102680 0x7f2bd810ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:11:46.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:46.772+0000 7f2bdf628700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd810f5a0 con 0x7f2bd8102680 2024-05-12T04:11:46.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:46.772+0000 7f2bdf628700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd810f6e0 con 0x7f2bd8101ce0 2024-05-12T04:11:46.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:46.772+0000 7f2bdcbc3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bd8102680 0x7f2bd810ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:11:46.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:46.772+0000 7f2bdcbc3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bd8102680 0x7f2bd810ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52848/0 (socket says 172.21.15.32:52848) 2024-05-12T04:11:46.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:46.772+0000 7f2bdcbc3700 1 -- 172.21.15.32:0/3652827411 learned_addr learned my addr 172.21.15.32:0/3652827411 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:11:46.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:46.773+0000 7f2bdd3c4700 1 --2- 172.21.15.32:0/3652827411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bd8101ce0 0x7f2bd810ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:11:46.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:46.773+0000 7f2bdcbc3700 1 -- 172.21.15.32:0/3652827411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bd8101ce0 msgr2=0x7f2bd810ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:11:46.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:46.773+0000 7f2bdcbc3700 1 --2- 172.21.15.32:0/3652827411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bd8101ce0 0x7f2bd810ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:11:46.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:46.773+0000 7f2bdcbc3700 1 -- 172.21.15.32:0/3652827411 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bcc005040 con 0x7f2bd8102680 2024-05-12T04:11:46.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:46.773+0000 7f2bdcbc3700 1 --2- 172.21.15.32:0/3652827411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bd8102680 0x7f2bd810ef50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2bd400c800 tx=0x7f2bd4004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:11:46.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:46.773+0000 7f2bc9ffb700 1 -- 172.21.15.32:0/3652827411 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2bd4011a80 con 0x7f2bd8102680 2024-05-12T04:11:46.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:46.773+0000 7f2bc9ffb700 1 -- 172.21.15.32:0/3652827411 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2bd4011be0 con 0x7f2bd8102680 2024-05-12T04:11:46.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:46.773+0000 7f2bdf628700 1 -- 172.21.15.32:0/3652827411 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bd810c800 con 0x7f2bd8102680 2024-05-12T04:11:46.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:46.773+0000 7f2bc9ffb700 1 -- 172.21.15.32:0/3652827411 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2bd4019610 con 0x7f2bd8102680 2024-05-12T04:11:46.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:46.773+0000 7f2bdf628700 1 -- 172.21.15.32:0/3652827411 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bd810cd60 con 0x7f2bd8102680 2024-05-12T04:11:46.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:46.774+0000 7f2bdf628700 1 -- 172.21.15.32:0/3652827411 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bd80fd430 con 0x7f2bd8102680 2024-05-12T04:11:46.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:46.775+0000 7f2bc9ffb700 1 -- 172.21.15.32:0/3652827411 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f2bd4017070 con 0x7f2bd8102680 2024-05-12T04:11:46.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:46.775+0000 7f2bc9ffb700 1 --2- 172.21.15.32:0/3652827411 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f2bc406cc20 0x7f2bc406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:11:46.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:46.775+0000 7f2bc9ffb700 1 -- 172.21.15.32:0/3652827411 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f2bd408ff90 con 0x7f2bd8102680 2024-05-12T04:11:46.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:46.775+0000 7f2bdd3c4700 1 --2- 172.21.15.32:0/3652827411 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f2bc406cc20 0x7f2bc406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:11:46.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:46.776+0000 7f2bdd3c4700 1 --2- 172.21.15.32:0/3652827411 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f2bc406cc20 0x7f2bc406f0d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc004f90 tx=0x7f2bcc009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:11:46.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:46.780+0000 7f2bc9ffb700 1 -- 172.21.15.32:0/3652827411 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f2bd405a010 con 0x7f2bd8102680 2024-05-12T04:11:46.789 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:11:46 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:11:46.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:11:46 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:11:46.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:46.991+0000 7f2bdf628700 1 -- 172.21.15.32:0/3652827411 --> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2bd804bed0 con 0x7f2bc406cc20 2024-05-12T04:11:47.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:47.001+0000 7f2bc9ffb700 1 -- 172.21.15.32:0/3652827411 <== mgr.24593 v2:172.21.15.32:6800/2957111591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f2bd804bed0 con 0x7f2bc406cc20 2024-05-12T04:11:47.002 INFO:teuthology.orchestra.run.smithi032.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-12T04:11:47.002 INFO:teuthology.orchestra.run.smithi032.stdout:alertmanager.smithi032 smithi032 *:9093,9094 running (114s) 93s ago 19m 16.8M - 0.25.0 c8568f914cd2 a00759bd6b0c 2024-05-12T04:11:47.002 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi032 smithi032 running (7m) 93s ago 20m 6660k - 18.2.2-1272-g58249552 ddcb0a9d6fde 8c53f1fc7b06 2024-05-12T04:11:47.002 INFO:teuthology.orchestra.run.smithi032.stdout:crash.smithi033 smithi033 running (7m) 2m ago 18m 6651k - 18.2.2-1272-g58249552 ddcb0a9d6fde cf4fe9ca070b 2024-05-12T04:11:47.002 INFO:teuthology.orchestra.run.smithi032.stdout:grafana.smithi032 smithi032 *:3000 running (98s) 93s ago 19m 46.0M - 9.4.7 954c08fa6188 6511c2d41be5 2024-05-12T04:11:47.003 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.ialtqb smithi032 running (3m) 93s ago 13m 83.0M - 18.2.2-1272-g58249552 ddcb0a9d6fde 532dd9845ce7 2024-05-12T04:11:47.003 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi032.qpsawh smithi032 running (3m) 93s ago 13m 86.5M - 18.2.2-1272-g58249552 ddcb0a9d6fde 68e735fd79ee 2024-05-12T04:11:47.003 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi033.hglwml smithi033 running (2m) 2m ago 13m 14.9M - 18.2.2-1272-g58249552 ddcb0a9d6fde 7240e684b225 2024-05-12T04:11:47.003 INFO:teuthology.orchestra.run.smithi032.stdout:mds.cephfs.smithi033.jgbjhk smithi033 running (3m) 2m ago 13m 14.1M - 18.2.2-1272-g58249552 ddcb0a9d6fde b738dbbbec0f 2024-05-12T04:11:47.003 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi032.ukatkq smithi032 *:8443,9283,8765 running (9m) 93s ago 21m 511M - 18.2.2-1272-g58249552 ddcb0a9d6fde 0d88ab9a5292 2024-05-12T04:11:47.003 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.smithi033.pcvxwy smithi033 *:8443,9283,8765 running (9m) 2m ago 18m 451M - 18.2.2-1272-g58249552 ddcb0a9d6fde d42dcb6bd307 2024-05-12T04:11:47.003 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi032 smithi032 running (8m) 93s ago 21m 54.5M 2048M 18.2.2-1272-g58249552 ddcb0a9d6fde 81f94171c5dc 2024-05-12T04:11:47.003 INFO:teuthology.orchestra.run.smithi032.stdout:mon.smithi033 smithi033 running (8m) 2m ago 18m 42.9M 2048M 18.2.2-1272-g58249552 ddcb0a9d6fde d648ba108f3e 2024-05-12T04:11:47.003 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi032 smithi032 *:9100 running (2m) 93s ago 19m 9135k - 1.5.0 0da6a335fe13 fcb3ea4af15b 2024-05-12T04:11:47.003 INFO:teuthology.orchestra.run.smithi032.stdout:node-exporter.smithi033 smithi033 *:9100 running (2m) 2m ago 18m 3632k - 1.5.0 0da6a335fe13 b75dd225ce4b 2024-05-12T04:11:47.003 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0 smithi032 running (7m) 93s ago 17m 93.0M 1570M 18.2.2-1272-g58249552 ddcb0a9d6fde 07f66c4deb3b 2024-05-12T04:11:47.003 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1 smithi032 running (6m) 93s ago 17m 100M 1570M 18.2.2-1272-g58249552 ddcb0a9d6fde 67a87807d654 2024-05-12T04:11:47.003 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2 smithi032 running (5m) 93s ago 16m 88.4M 1570M 18.2.2-1272-g58249552 ddcb0a9d6fde 4ac48d74ee8b 2024-05-12T04:11:47.003 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3 smithi033 running (5m) 2m ago 16m 103M 2594M 18.2.2-1272-g58249552 ddcb0a9d6fde 11f097a455bb 2024-05-12T04:11:47.003 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4 smithi033 running (4m) 2m ago 15m 91.5M 2594M 18.2.2-1272-g58249552 ddcb0a9d6fde db6fda7d52de 2024-05-12T04:11:47.004 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5 smithi033 running (4m) 2m ago 15m 92.7M 2594M 18.2.2-1272-g58249552 ddcb0a9d6fde 3333020b8a15 2024-05-12T04:11:47.004 INFO:teuthology.orchestra.run.smithi032.stdout:prometheus.smithi032 smithi032 *:9095 running (2m) 93s ago 19m 44.4M - 2.43.0 a07b618ecd1d fe69bbb63353 2024-05-12T04:11:47.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:47.003+0000 7f2bdf628700 1 -- 172.21.15.32:0/3652827411 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f2bc406cc20 msgr2=0x7f2bc406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:11:47.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:47.003+0000 7f2bdf628700 1 --2- 172.21.15.32:0/3652827411 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f2bc406cc20 0x7f2bc406f0d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc004f90 tx=0x7f2bcc009e30 comp rx=0 tx=0).stop 2024-05-12T04:11:47.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:47.003+0000 7f2bdf628700 1 -- 172.21.15.32:0/3652827411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bd8102680 msgr2=0x7f2bd810ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:11:47.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:47.003+0000 7f2bdf628700 1 --2- 172.21.15.32:0/3652827411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bd8102680 0x7f2bd810ef50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2bd400c800 tx=0x7f2bd4004470 comp rx=0 tx=0).stop 2024-05-12T04:11:47.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:47.003+0000 7f2bdf628700 1 -- 172.21.15.32:0/3652827411 shutdown_connections 2024-05-12T04:11:47.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:47.003+0000 7f2bdf628700 1 --2- 172.21.15.32:0/3652827411 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f2bc406cc20 0x7f2bc406f0d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:11:47.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:47.003+0000 7f2bdf628700 1 --2- 172.21.15.32:0/3652827411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bd8101ce0 0x7f2bd810ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:11:47.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:47.003+0000 7f2bdf628700 1 --2- 172.21.15.32:0/3652827411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2bd8102680 0x7f2bd810ef50 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:11:47.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:47.003+0000 7f2bdf628700 1 -- 172.21.15.32:0/3652827411 >> 172.21.15.32:0/3652827411 conn(0x7f2bd80f7850 msgr2=0x7f2bd80f8940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:11:47.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:47.003+0000 7f2bdf628700 1 -- 172.21.15.32:0/3652827411 shutdown_connections 2024-05-12T04:11:47.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:47.003+0000 7f2bdf628700 1 -- 172.21.15.32:0/3652827411 wait complete. 2024-05-12T04:11:47.743 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:11:47 smithi032 ceph-mon[120392]: pgmap v293: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:11:47.766 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -e sha1=58249552a22f437b8cb49ed6b1068e1158bed605 -- bash -c 'ceph versions' 2024-05-12T04:11:47.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:11:47 smithi033 ceph-mon[107305]: pgmap v293: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:11:48.052 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T04:11:48.576 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:11:48 smithi032 ceph-mon[120392]: from='client.34548 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:11:48.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:11:48 smithi033 ceph-mon[107305]: from='client.34548 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-12T04:11:49.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:11:49 smithi032 ceph-mon[120392]: pgmap v294: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:11:49.900 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:49.897+0000 7f570fb36700 1 -- 172.21.15.32:0/1639266569 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5708102670 msgr2=0x7f57080fb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:11:49.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:49.897+0000 7f570fb36700 1 --2- 172.21.15.32:0/1639266569 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5708102670 0x7f57080fb600 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f56f8004940 tx=0x7f56f8009e30 comp rx=0 tx=0).stop 2024-05-12T04:11:49.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:49.897+0000 7f570fb36700 1 -- 172.21.15.32:0/1639266569 shutdown_connections 2024-05-12T04:11:49.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:49.897+0000 7f570fb36700 1 --2- 172.21.15.32:0/1639266569 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5708102670 0x7f57080fb600 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:11:49.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:49.897+0000 7f570fb36700 1 --2- 172.21.15.32:0/1639266569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5708101cd0 0x7f57081020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:11:49.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:49.897+0000 7f570fb36700 1 -- 172.21.15.32:0/1639266569 >> 172.21.15.32:0/1639266569 conn(0x7f5708071460 msgr2=0x7f5708071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:11:49.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:49.898+0000 7f570fb36700 1 -- 172.21.15.32:0/1639266569 shutdown_connections 2024-05-12T04:11:49.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:49.898+0000 7f570fb36700 1 -- 172.21.15.32:0/1639266569 wait complete. 2024-05-12T04:11:49.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:49.898+0000 7f570fb36700 1 Processor -- start 2024-05-12T04:11:49.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:49.898+0000 7f570fb36700 1 -- start start 2024-05-12T04:11:49.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:49.898+0000 7f570fb36700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5708101cd0 0x7f57080fc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:11:49.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:49.898+0000 7f570fb36700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5708102670 0x7f57080fcd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:11:49.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:49.898+0000 7f570fb36700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57080fd2b0 con 0x7f5708101cd0 2024-05-12T04:11:49.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:49.898+0000 7f570fb36700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57080fd3f0 con 0x7f5708102670 2024-05-12T04:11:49.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:49.898+0000 7f570d8d2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5708101cd0 0x7f57080fc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:11:49.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:49.898+0000 7f570d8d2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5708101cd0 0x7f57080fc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52866/0 (socket says 172.21.15.32:52866) 2024-05-12T04:11:49.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:49.898+0000 7f570d8d2700 1 -- 172.21.15.32:0/1120086720 learned_addr learned my addr 172.21.15.32:0/1120086720 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:11:49.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:49.899+0000 7f570d0d1700 1 --2- 172.21.15.32:0/1120086720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5708102670 0x7f57080fcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:11:49.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:49.899+0000 7f570d8d2700 1 -- 172.21.15.32:0/1120086720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5708102670 msgr2=0x7f57080fcd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:11:49.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:49.899+0000 7f570d8d2700 1 --2- 172.21.15.32:0/1120086720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5708102670 0x7f57080fcd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:11:49.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:49.899+0000 7f570d8d2700 1 -- 172.21.15.32:0/1120086720 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f8005040 con 0x7f5708101cd0 2024-05-12T04:11:49.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:49.899+0000 7f570d0d1700 1 --2- 172.21.15.32:0/1120086720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5708102670 0x7f57080fcd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-12T04:11:49.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:49.899+0000 7f570d8d2700 1 --2- 172.21.15.32:0/1120086720 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5708101cd0 0x7f57080fc7e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5704002a50 tx=0x7f57040046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:11:49.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:49.899+0000 7f56fe7fc700 1 -- 172.21.15.32:0/1120086720 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5704007cc0 con 0x7f5708101cd0 2024-05-12T04:11:49.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:49.899+0000 7f56fe7fc700 1 -- 172.21.15.32:0/1120086720 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f570400f040 con 0x7f5708101cd0 2024-05-12T04:11:49.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:49.899+0000 7f570fb36700 1 -- 172.21.15.32:0/1120086720 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5708119db0 con 0x7f5708101cd0 2024-05-12T04:11:49.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:49.899+0000 7f56fe7fc700 1 -- 172.21.15.32:0/1120086720 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5704009540 con 0x7f5708101cd0 2024-05-12T04:11:49.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:49.899+0000 7f570fb36700 1 -- 172.21.15.32:0/1120086720 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f570811a230 con 0x7f5708101cd0 2024-05-12T04:11:49.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:49.900+0000 7f570fb36700 1 -- 172.21.15.32:0/1120086720 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5708065200 con 0x7f5708101cd0 2024-05-12T04:11:49.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:49.901+0000 7f56fe7fc700 1 -- 172.21.15.32:0/1120086720 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f57040096a0 con 0x7f5708101cd0 2024-05-12T04:11:49.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:49.901+0000 7f56fe7fc700 1 --2- 172.21.15.32:0/1120086720 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f56f406cc30 0x7f56f406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:11:49.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:49.901+0000 7f56fe7fc700 1 -- 172.21.15.32:0/1120086720 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f570408bdf0 con 0x7f5708101cd0 2024-05-12T04:11:49.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:49.901+0000 7f570d0d1700 1 --2- 172.21.15.32:0/1120086720 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f56f406cc30 0x7f56f406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:11:49.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:49.902+0000 7f570d0d1700 1 --2- 172.21.15.32:0/1120086720 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f56f406cc30 0x7f56f406f0e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f56f800b3b0 tx=0x7f56f8009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:11:49.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:49.906+0000 7f56fe7fc700 1 -- 172.21.15.32:0/1120086720 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f5704055dd0 con 0x7f5708101cd0 2024-05-12T04:11:49.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:11:49 smithi033 ceph-mon[107305]: pgmap v294: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:11:50.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:50.196+0000 7f570fb36700 1 -- 172.21.15.32:0/1120086720 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f57080fded0 con 0x7f5708101cd0 2024-05-12T04:11:50.198 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:50.196+0000 7f56fe7fc700 1 -- 172.21.15.32:0/1120086720 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f57040593f0 con 0x7f5708101cd0 2024-05-12T04:11:50.198 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-05-12T04:11:50.199 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": { 2024-05-12T04:11:50.199 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 2 2024-05-12T04:11:50.199 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:11:50.199 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": { 2024-05-12T04:11:50.199 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 2 2024-05-12T04:11:50.200 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:11:50.200 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": { 2024-05-12T04:11:50.200 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 6 2024-05-12T04:11:50.200 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:11:50.200 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": { 2024-05-12T04:11:50.201 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 4 2024-05-12T04:11:50.201 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2024-05-12T04:11:50.201 INFO:teuthology.orchestra.run.smithi032.stdout: "overall": { 2024-05-12T04:11:50.201 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)": 14 2024-05-12T04:11:50.201 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-05-12T04:11:50.201 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-05-12T04:11:50.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:50.198+0000 7f570fb36700 1 -- 172.21.15.32:0/1120086720 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f56f406cc30 msgr2=0x7f56f406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:11:50.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:50.198+0000 7f570fb36700 1 --2- 172.21.15.32:0/1120086720 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f56f406cc30 0x7f56f406f0e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f56f800b3b0 tx=0x7f56f8009e30 comp rx=0 tx=0).stop 2024-05-12T04:11:50.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:50.198+0000 7f570fb36700 1 -- 172.21.15.32:0/1120086720 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5708101cd0 msgr2=0x7f57080fc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:11:50.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:50.198+0000 7f570fb36700 1 --2- 172.21.15.32:0/1120086720 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5708101cd0 0x7f57080fc7e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5704002a50 tx=0x7f57040046b0 comp rx=0 tx=0).stop 2024-05-12T04:11:50.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:50.198+0000 7f570fb36700 1 -- 172.21.15.32:0/1120086720 shutdown_connections 2024-05-12T04:11:50.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:50.198+0000 7f570fb36700 1 --2- 172.21.15.32:0/1120086720 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f56f406cc30 0x7f56f406f0e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:11:50.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:50.198+0000 7f570fb36700 1 --2- 172.21.15.32:0/1120086720 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5708101cd0 0x7f57080fc7e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:11:50.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:50.198+0000 7f570fb36700 1 --2- 172.21.15.32:0/1120086720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5708102670 0x7f57080fcd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:11:50.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:50.198+0000 7f570fb36700 1 -- 172.21.15.32:0/1120086720 >> 172.21.15.32:0/1120086720 conn(0x7f5708071460 msgr2=0x7f57081009d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:11:50.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:50.199+0000 7f570fb36700 1 -- 172.21.15.32:0/1120086720 shutdown_connections 2024-05-12T04:11:50.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:50.199+0000 7f570fb36700 1 -- 172.21.15.32:0/1120086720 wait complete. 2024-05-12T04:11:50.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:11:50 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/1120086720' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:11:50.911 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -e sha1=58249552a22f437b8cb49ed6b1068e1158bed605 -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-05-12T04:11:50.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:11:50 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/1120086720' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:11:51.198 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T04:11:51.738 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:11:51 smithi032 ceph-mon[120392]: pgmap v295: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:11:51.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:11:51 smithi033 ceph-mon[107305]: pgmap v295: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:11:53.048 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:53.046+0000 7f08a099d700 1 -- 172.21.15.32:0/4063271675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f089c102de0 msgr2=0x7f089c1031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:11:53.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:53.046+0000 7f08a099d700 1 --2- 172.21.15.32:0/4063271675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f089c102de0 0x7f089c1031b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f088c004940 tx=0x7f088c009e30 comp rx=0 tx=0).stop 2024-05-12T04:11:53.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:53.046+0000 7f08a099d700 1 -- 172.21.15.32:0/4063271675 shutdown_connections 2024-05-12T04:11:53.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:53.046+0000 7f08a099d700 1 --2- 172.21.15.32:0/4063271675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f089c103780 0x7f089c0fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:11:53.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:53.046+0000 7f08a099d700 1 --2- 172.21.15.32:0/4063271675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f089c102de0 0x7f089c1031b0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:11:53.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:53.046+0000 7f08a099d700 1 -- 172.21.15.32:0/4063271675 >> 172.21.15.32:0/4063271675 conn(0x7f089c0f8950 msgr2=0x7f089c0fad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:11:53.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:53.046+0000 7f08a099d700 1 -- 172.21.15.32:0/4063271675 shutdown_connections 2024-05-12T04:11:53.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:53.046+0000 7f08a099d700 1 -- 172.21.15.32:0/4063271675 wait complete. 2024-05-12T04:11:53.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:53.046+0000 7f08a099d700 1 Processor -- start 2024-05-12T04:11:53.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:53.046+0000 7f08a099d700 1 -- start start 2024-05-12T04:11:53.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:53.047+0000 7f08a099d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f089c102de0 0x7f089c115cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:11:53.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:53.047+0000 7f08a099d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f089c103780 0x7f089c1161f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:11:53.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:53.047+0000 7f08a099d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f089c116840 con 0x7f089c103780 2024-05-12T04:11:53.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:53.047+0000 7f08a099d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f089c10fc10 con 0x7f089c102de0 2024-05-12T04:11:53.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:53.047+0000 7f0899d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f089c103780 0x7f089c1161f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:11:53.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:53.047+0000 7f0899d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f089c103780 0x7f089c1161f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60360/0 (socket says 172.21.15.32:60360) 2024-05-12T04:11:53.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:53.047+0000 7f0899d9b700 1 -- 172.21.15.32:0/3321224737 learned_addr learned my addr 172.21.15.32:0/3321224737 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:11:53.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:53.047+0000 7f089a59c700 1 --2- 172.21.15.32:0/3321224737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f089c102de0 0x7f089c115cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:11:53.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:53.047+0000 7f0899d9b700 1 -- 172.21.15.32:0/3321224737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f089c102de0 msgr2=0x7f089c115cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:11:53.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:53.047+0000 7f0899d9b700 1 --2- 172.21.15.32:0/3321224737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f089c102de0 0x7f089c115cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:11:53.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:53.047+0000 7f0899d9b700 1 -- 172.21.15.32:0/3321224737 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f088c005040 con 0x7f089c103780 2024-05-12T04:11:53.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:53.047+0000 7f0899d9b700 1 --2- 172.21.15.32:0/3321224737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f089c103780 0x7f089c1161f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0890002a50 tx=0x7f0890004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:11:53.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:53.047+0000 7f088affd700 1 -- 172.21.15.32:0/3321224737 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0890007cc0 con 0x7f089c103780 2024-05-12T04:11:53.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:53.047+0000 7f088affd700 1 -- 172.21.15.32:0/3321224737 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f089000f040 con 0x7f089c103780 2024-05-12T04:11:53.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:53.047+0000 7f08a099d700 1 -- 172.21.15.32:0/3321224737 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f089c10fef0 con 0x7f089c103780 2024-05-12T04:11:53.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:53.047+0000 7f088affd700 1 -- 172.21.15.32:0/3321224737 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f089000d990 con 0x7f089c103780 2024-05-12T04:11:53.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:53.047+0000 7f08a099d700 1 -- 172.21.15.32:0/3321224737 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f089c1104b0 con 0x7f089c103780 2024-05-12T04:11:53.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:53.048+0000 7f08a099d700 1 -- 172.21.15.32:0/3321224737 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f089c0fe530 con 0x7f089c103780 2024-05-12T04:11:53.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:53.049+0000 7f088affd700 1 -- 172.21.15.32:0/3321224737 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f0890009710 con 0x7f089c103780 2024-05-12T04:11:53.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:53.049+0000 7f088affd700 1 --2- 172.21.15.32:0/3321224737 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f088406cc20 0x7f088406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:11:53.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:53.049+0000 7f088affd700 1 -- 172.21.15.32:0/3321224737 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f089008bec0 con 0x7f089c103780 2024-05-12T04:11:53.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:53.049+0000 7f089a59c700 1 --2- 172.21.15.32:0/3321224737 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f088406cc20 0x7f088406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:11:53.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:53.050+0000 7f089a59c700 1 --2- 172.21.15.32:0/3321224737 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f088406cc20 0x7f088406f0d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f088c004f90 tx=0x7f088c009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:11:53.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:53.054+0000 7f088affd700 1 -- 172.21.15.32:0/3321224737 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f0890059620 con 0x7f089c103780 2024-05-12T04:11:53.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:53.348+0000 7f08a099d700 1 -- 172.21.15.32:0/3321224737 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f089c04bed0 con 0x7f089c103780 2024-05-12T04:11:53.350 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:53.349+0000 7f088affd700 1 -- 172.21.15.32:0/3321224737 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f0890059440 con 0x7f089c103780 2024-05-12T04:11:53.352 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:53.351+0000 7f08a099d700 1 -- 172.21.15.32:0/3321224737 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f088406cc20 msgr2=0x7f088406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:11:53.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:53.351+0000 7f08a099d700 1 --2- 172.21.15.32:0/3321224737 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f088406cc20 0x7f088406f0d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f088c004f90 tx=0x7f088c009e30 comp rx=0 tx=0).stop 2024-05-12T04:11:53.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:53.351+0000 7f08a099d700 1 -- 172.21.15.32:0/3321224737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f089c103780 msgr2=0x7f089c1161f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:11:53.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:53.351+0000 7f08a099d700 1 --2- 172.21.15.32:0/3321224737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f089c103780 0x7f089c1161f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0890002a50 tx=0x7f0890004600 comp rx=0 tx=0).stop 2024-05-12T04:11:53.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:53.351+0000 7f08a099d700 1 -- 172.21.15.32:0/3321224737 shutdown_connections 2024-05-12T04:11:53.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:53.351+0000 7f08a099d700 1 --2- 172.21.15.32:0/3321224737 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f088406cc20 0x7f088406f0d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:11:53.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:53.351+0000 7f08a099d700 1 --2- 172.21.15.32:0/3321224737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f089c102de0 0x7f089c115cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:11:53.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:53.351+0000 7f08a099d700 1 --2- 172.21.15.32:0/3321224737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f089c103780 0x7f089c1161f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:11:53.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:53.351+0000 7f08a099d700 1 -- 172.21.15.32:0/3321224737 >> 172.21.15.32:0/3321224737 conn(0x7f089c0f8950 msgr2=0x7f089c070620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:11:53.353 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:53.351+0000 7f08a099d700 1 -- 172.21.15.32:0/3321224737 shutdown_connections 2024-05-12T04:11:53.354 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:53.351+0000 7f08a099d700 1 -- 172.21.15.32:0/3321224737 wait complete. 2024-05-12T04:11:53.362 INFO:teuthology.orchestra.run.smithi032.stdout:true 2024-05-12T04:11:53.685 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:11:53 smithi032 ceph-mon[120392]: pgmap v296: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:11:53.685 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:11:53 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/3321224737' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:11:53.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:11:53 smithi033 ceph-mon[107305]: pgmap v296: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:11:53.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:11:53 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/3321224737' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:11:54.124 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -e sha1=58249552a22f437b8cb49ed6b1068e1158bed605 -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-05-12T04:11:54.411 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T04:11:55.691 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:11:55 smithi032 ceph-mon[120392]: pgmap v297: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:11:55.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:11:55 smithi033 ceph-mon[107305]: pgmap v297: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:11:56.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:56.262+0000 7f4832b53700 1 -- 172.21.15.32:0/1855462325 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f482c102680 msgr2=0x7f482c0fbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:11:56.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:56.262+0000 7f4832b53700 1 --2- 172.21.15.32:0/1855462325 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f482c102680 0x7f482c0fbcf0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4818004940 tx=0x7f4818009e30 comp rx=0 tx=0).stop 2024-05-12T04:11:56.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:56.262+0000 7f4832b53700 1 -- 172.21.15.32:0/1855462325 shutdown_connections 2024-05-12T04:11:56.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:56.262+0000 7f4832b53700 1 --2- 172.21.15.32:0/1855462325 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f482c102680 0x7f482c0fbcf0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:11:56.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:56.262+0000 7f4832b53700 1 --2- 172.21.15.32:0/1855462325 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f482c101ce0 0x7f482c1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:11:56.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:56.262+0000 7f4832b53700 1 -- 172.21.15.32:0/1855462325 >> 172.21.15.32:0/1855462325 conn(0x7f482c0f7850 msgr2=0x7f482c0f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:11:56.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:56.262+0000 7f4832b53700 1 -- 172.21.15.32:0/1855462325 shutdown_connections 2024-05-12T04:11:56.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:56.262+0000 7f4832b53700 1 -- 172.21.15.32:0/1855462325 wait complete. 2024-05-12T04:11:56.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:56.263+0000 7f4832b53700 1 Processor -- start 2024-05-12T04:11:56.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:56.263+0000 7f4832b53700 1 -- start start 2024-05-12T04:11:56.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:56.263+0000 7f4832b53700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f482c101ce0 0x7f482c102b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:11:56.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:56.263+0000 7f4832b53700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f482c102680 0x7f482c103080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:11:56.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:56.263+0000 7f4832b53700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f482c103610 con 0x7f482c102680 2024-05-12T04:11:56.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:56.263+0000 7f4832b53700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f482c103750 con 0x7f482c101ce0 2024-05-12T04:11:56.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:56.263+0000 7f482bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f482c102680 0x7f482c103080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:11:56.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:56.263+0000 7f482bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f482c102680 0x7f482c103080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60382/0 (socket says 172.21.15.32:60382) 2024-05-12T04:11:56.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:56.263+0000 7f482bfff700 1 -- 172.21.15.32:0/1364842881 learned_addr learned my addr 172.21.15.32:0/1364842881 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:11:56.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:56.264+0000 7f48308ef700 1 --2- 172.21.15.32:0/1364842881 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f482c101ce0 0x7f482c102b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:11:56.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:56.264+0000 7f482bfff700 1 -- 172.21.15.32:0/1364842881 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f482c101ce0 msgr2=0x7f482c102b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:11:56.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:56.264+0000 7f482bfff700 1 --2- 172.21.15.32:0/1364842881 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f482c101ce0 0x7f482c102b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:11:56.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:56.264+0000 7f482bfff700 1 -- 172.21.15.32:0/1364842881 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4818005040 con 0x7f482c102680 2024-05-12T04:11:56.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:56.264+0000 7f48308ef700 1 --2- 172.21.15.32:0/1364842881 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f482c101ce0 0x7f482c102b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-12T04:11:56.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:56.264+0000 7f482bfff700 1 --2- 172.21.15.32:0/1364842881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f482c102680 0x7f482c103080 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4818009390 tx=0x7f4818009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:11:56.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:56.264+0000 7f48297fa700 1 -- 172.21.15.32:0/1364842881 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f481800e040 con 0x7f482c102680 2024-05-12T04:11:56.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:56.264+0000 7f48297fa700 1 -- 172.21.15.32:0/1364842881 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4818013420 con 0x7f482c102680 2024-05-12T04:11:56.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:56.264+0000 7f48297fa700 1 -- 172.21.15.32:0/1364842881 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f481801c630 con 0x7f482c102680 2024-05-12T04:11:56.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:56.264+0000 7f4832b53700 1 -- 172.21.15.32:0/1364842881 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f482c1039a0 con 0x7f482c102680 2024-05-12T04:11:56.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:56.264+0000 7f4832b53700 1 -- 172.21.15.32:0/1364842881 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f482c11a190 con 0x7f482c102680 2024-05-12T04:11:56.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:56.265+0000 7f4832b53700 1 -- 172.21.15.32:0/1364842881 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f482c0fd430 con 0x7f482c102680 2024-05-12T04:11:56.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:56.265+0000 7f48297fa700 1 -- 172.21.15.32:0/1364842881 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f48180139e0 con 0x7f482c102680 2024-05-12T04:11:56.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:56.266+0000 7f48297fa700 1 --2- 172.21.15.32:0/1364842881 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f481c06cc30 0x7f481c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:11:56.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:56.266+0000 7f48297fa700 1 -- 172.21.15.32:0/1364842881 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f481808d290 con 0x7f482c102680 2024-05-12T04:11:56.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:56.266+0000 7f48308ef700 1 --2- 172.21.15.32:0/1364842881 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f481c06cc30 0x7f481c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:11:56.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:56.266+0000 7f48308ef700 1 --2- 172.21.15.32:0/1364842881 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f481c06cc30 0x7f481c06f0e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4820009fd0 tx=0x7f482000b040 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:11:56.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:56.271+0000 7f48297fa700 1 -- 172.21.15.32:0/1364842881 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f4818057310 con 0x7f482c102680 2024-05-12T04:11:56.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:56.563+0000 7f4832b53700 1 -- 172.21.15.32:0/1364842881 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f482c108cb0 con 0x7f482c102680 2024-05-12T04:11:56.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:56.563+0000 7f48297fa700 1 -- 172.21.15.32:0/1364842881 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f481805a930 con 0x7f482c102680 2024-05-12T04:11:56.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:56.566+0000 7f4832b53700 1 -- 172.21.15.32:0/1364842881 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f481c06cc30 msgr2=0x7f481c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:11:56.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:56.566+0000 7f4832b53700 1 --2- 172.21.15.32:0/1364842881 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f481c06cc30 0x7f481c06f0e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4820009fd0 tx=0x7f482000b040 comp rx=0 tx=0).stop 2024-05-12T04:11:56.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:56.566+0000 7f4832b53700 1 -- 172.21.15.32:0/1364842881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f482c102680 msgr2=0x7f482c103080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:11:56.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:56.566+0000 7f4832b53700 1 --2- 172.21.15.32:0/1364842881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f482c102680 0x7f482c103080 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4818009390 tx=0x7f4818009e30 comp rx=0 tx=0).stop 2024-05-12T04:11:56.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:56.566+0000 7f4832b53700 1 -- 172.21.15.32:0/1364842881 shutdown_connections 2024-05-12T04:11:56.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:56.566+0000 7f4832b53700 1 --2- 172.21.15.32:0/1364842881 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f481c06cc30 0x7f481c06f0e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:11:56.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:56.566+0000 7f4832b53700 1 --2- 172.21.15.32:0/1364842881 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f482c101ce0 0x7f482c102b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:11:56.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:56.566+0000 7f4832b53700 1 --2- 172.21.15.32:0/1364842881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f482c102680 0x7f482c103080 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:11:56.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:56.566+0000 7f4832b53700 1 -- 172.21.15.32:0/1364842881 >> 172.21.15.32:0/1364842881 conn(0x7f482c0f7850 msgr2=0x7f482c0f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:11:56.567 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:56.566+0000 7f4832b53700 1 -- 172.21.15.32:0/1364842881 shutdown_connections 2024-05-12T04:11:56.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:56.566+0000 7f4832b53700 1 -- 172.21.15.32:0/1364842881 wait complete. 2024-05-12T04:11:56.576 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph version 18.2.2-1272-g58249552 (58249552a22f437b8cb49ed6b1068e1158bed605) reef (stable)" 2024-05-12T04:11:57.279 DEBUG:teuthology.parallel:result is None 2024-05-12T04:11:57.279 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-12T04:11:57.291 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi032.front.sepia.ceph.com 2024-05-12T04:11:57.291 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a3e178f8-1012-11ef-bc98-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-12T04:11:57.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:11:57 smithi032 ceph-mon[120392]: pgmap v298: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:11:57.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:11:57 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/1364842881' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:11:57.576 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/mon.smithi032/config 2024-05-12T04:11:57.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:11:57 smithi033 ceph-mon[107305]: pgmap v298: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:11:57.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:11:57 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/1364842881' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-05-12T04:11:59.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:59.477+0000 7f326b3ee700 1 -- 172.21.15.32:0/1914524284 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32640702a0 msgr2=0x7f3264070670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:11:59.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:59.477+0000 7f326b3ee700 1 --2- 172.21.15.32:0/1914524284 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32640702a0 0x7f3264070670 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3258004970 tx=0x7f3258009e30 comp rx=0 tx=0).stop 2024-05-12T04:11:59.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:59.477+0000 7f326b3ee700 1 -- 172.21.15.32:0/1914524284 shutdown_connections 2024-05-12T04:11:59.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:59.477+0000 7f326b3ee700 1 --2- 172.21.15.32:0/1914524284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3264070bb0 0x7f3264109160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:11:59.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:59.477+0000 7f326b3ee700 1 --2- 172.21.15.32:0/1914524284 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32640702a0 0x7f3264070670 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:11:59.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:59.477+0000 7f326b3ee700 1 -- 172.21.15.32:0/1914524284 >> 172.21.15.32:0/1914524284 conn(0x7f32640f8980 msgr2=0x7f32640fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:11:59.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:59.477+0000 7f326b3ee700 1 -- 172.21.15.32:0/1914524284 shutdown_connections 2024-05-12T04:11:59.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:59.477+0000 7f326b3ee700 1 -- 172.21.15.32:0/1914524284 wait complete. 2024-05-12T04:11:59.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:59.478+0000 7f326b3ee700 1 Processor -- start 2024-05-12T04:11:59.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:59.478+0000 7f326b3ee700 1 -- start start 2024-05-12T04:11:59.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:59.478+0000 7f326b3ee700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32640702a0 0x7f3264115c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:11:59.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:59.478+0000 7f326b3ee700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3264070bb0 0x7f32641161b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:11:59.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:59.478+0000 7f326b3ee700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3264116800 con 0x7f3264070bb0 2024-05-12T04:11:59.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:59.478+0000 7f326b3ee700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326410fbd0 con 0x7f32640702a0 2024-05-12T04:11:59.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:59.478+0000 7f3268989700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3264070bb0 0x7f32641161b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:11:59.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:59.478+0000 7f3268989700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3264070bb0 0x7f32641161b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60394/0 (socket says 172.21.15.32:60394) 2024-05-12T04:11:59.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:59.478+0000 7f3268989700 1 -- 172.21.15.32:0/3254445369 learned_addr learned my addr 172.21.15.32:0/3254445369 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:11:59.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:59.478+0000 7f326918a700 1 --2- 172.21.15.32:0/3254445369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32640702a0 0x7f3264115c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:11:59.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:59.479+0000 7f3268989700 1 -- 172.21.15.32:0/3254445369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32640702a0 msgr2=0x7f3264115c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:11:59.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:59.479+0000 7f3268989700 1 --2- 172.21.15.32:0/3254445369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32640702a0 0x7f3264115c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:11:59.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:59.479+0000 7f3268989700 1 -- 172.21.15.32:0/3254445369 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3258005040 con 0x7f3264070bb0 2024-05-12T04:11:59.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:59.479+0000 7f326918a700 1 --2- 172.21.15.32:0/3254445369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32640702a0 0x7f3264115c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-12T04:11:59.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:59.479+0000 7f3268989700 1 --2- 172.21.15.32:0/3254445369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3264070bb0 0x7f32641161b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f326000c8c0 tx=0x7f32600044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:11:59.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:59.479+0000 7f3255ffb700 1 -- 172.21.15.32:0/3254445369 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f326000a980 con 0x7f3264070bb0 2024-05-12T04:11:59.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:59.479+0000 7f3255ffb700 1 -- 172.21.15.32:0/3254445369 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f326000ce60 con 0x7f3264070bb0 2024-05-12T04:11:59.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:59.479+0000 7f326b3ee700 1 -- 172.21.15.32:0/3254445369 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f326410feb0 con 0x7f3264070bb0 2024-05-12T04:11:59.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:59.479+0000 7f3255ffb700 1 -- 172.21.15.32:0/3254445369 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32600113a0 con 0x7f3264070bb0 2024-05-12T04:11:59.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:59.479+0000 7f326b3ee700 1 -- 172.21.15.32:0/3254445369 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32641103f0 con 0x7f3264070bb0 2024-05-12T04:11:59.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:59.480+0000 7f326b3ee700 1 -- 172.21.15.32:0/3254445369 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32640719e0 con 0x7f3264070bb0 2024-05-12T04:11:59.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:59.480+0000 7f3255ffb700 1 -- 172.21.15.32:0/3254445369 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f3260012400 con 0x7f3264070bb0 2024-05-12T04:11:59.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:59.481+0000 7f3255ffb700 1 --2- 172.21.15.32:0/3254445369 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f325006cc20 0x7f325006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:11:59.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:59.481+0000 7f3255ffb700 1 -- 172.21.15.32:0/3254445369 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f326008c0c0 con 0x7f3264070bb0 2024-05-12T04:11:59.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:59.481+0000 7f326918a700 1 --2- 172.21.15.32:0/3254445369 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f325006cc20 0x7f325006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:11:59.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:59.481+0000 7f326918a700 1 --2- 172.21.15.32:0/3254445369 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f325006cc20 0x7f325006f0d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3258004f50 tx=0x7f3258009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:11:59.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:59.485+0000 7f3255ffb700 1 -- 172.21.15.32:0/3254445369 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f3260059bd0 con 0x7f3264070bb0 2024-05-12T04:11:59.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:59.732+0000 7f326b3ee700 1 -- 172.21.15.32:0/3254445369 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f326405f5c0 con 0x7f3264070bb0 2024-05-12T04:11:59.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:59.732+0000 7f3255ffb700 1 -- 172.21.15.32:0/3254445369 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 37 v37) v1 ==== 76+0+1871 (secure 0 0 0) 0x7f3260059760 con 0x7f3264070bb0 2024-05-12T04:11:59.733 INFO:teuthology.orchestra.run.smithi032.stdout:e37 2024-05-12T04:11:59.733 INFO:teuthology.orchestra.run.smithi032.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-12T04:11:59.733 INFO:teuthology.orchestra.run.smithi032.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-12T04:11:59.733 INFO:teuthology.orchestra.run.smithi032.stdout:legacy client fscid: 1 2024-05-12T04:11:59.733 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:11:59.733 INFO:teuthology.orchestra.run.smithi032.stdout:Filesystem 'cephfs' (1) 2024-05-12T04:11:59.733 INFO:teuthology.orchestra.run.smithi032.stdout:fs_name cephfs 2024-05-12T04:11:59.733 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 37 2024-05-12T04:11:59.733 INFO:teuthology.orchestra.run.smithi032.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-12T04:11:59.734 INFO:teuthology.orchestra.run.smithi032.stdout:created 2024-05-12T03:57:48.904614+0000 2024-05-12T04:11:59.734 INFO:teuthology.orchestra.run.smithi032.stdout:modified 2024-05-12T04:09:05.133553+0000 2024-05-12T04:11:59.734 INFO:teuthology.orchestra.run.smithi032.stdout:tableserver 0 2024-05-12T04:11:59.734 INFO:teuthology.orchestra.run.smithi032.stdout:root 0 2024-05-12T04:11:59.734 INFO:teuthology.orchestra.run.smithi032.stdout:session_timeout 60 2024-05-12T04:11:59.734 INFO:teuthology.orchestra.run.smithi032.stdout:session_autoclose 300 2024-05-12T04:11:59.734 INFO:teuthology.orchestra.run.smithi032.stdout:max_file_size 1099511627776 2024-05-12T04:11:59.734 INFO:teuthology.orchestra.run.smithi032.stdout:max_xattr_size 65536 2024-05-12T04:11:59.734 INFO:teuthology.orchestra.run.smithi032.stdout:required_client_features {} 2024-05-12T04:11:59.734 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure 0 2024-05-12T04:11:59.734 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure_osd_epoch 84 2024-05-12T04:11:59.734 INFO:teuthology.orchestra.run.smithi032.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-12T04:11:59.734 INFO:teuthology.orchestra.run.smithi032.stdout:max_mds 1 2024-05-12T04:11:59.734 INFO:teuthology.orchestra.run.smithi032.stdout:in 0 2024-05-12T04:11:59.735 INFO:teuthology.orchestra.run.smithi032.stdout:up {0=34404} 2024-05-12T04:11:59.735 INFO:teuthology.orchestra.run.smithi032.stdout:failed 2024-05-12T04:11:59.735 INFO:teuthology.orchestra.run.smithi032.stdout:damaged 2024-05-12T04:11:59.735 INFO:teuthology.orchestra.run.smithi032.stdout:stopped 2024-05-12T04:11:59.735 INFO:teuthology.orchestra.run.smithi032.stdout:data_pools [3] 2024-05-12T04:11:59.735 INFO:teuthology.orchestra.run.smithi032.stdout:metadata_pool 2 2024-05-12T04:11:59.735 INFO:teuthology.orchestra.run.smithi032.stdout:inline_data enabled 2024-05-12T04:11:59.735 INFO:teuthology.orchestra.run.smithi032.stdout:balancer 2024-05-12T04:11:59.735 INFO:teuthology.orchestra.run.smithi032.stdout:bal_rank_mask -1 2024-05-12T04:11:59.735 INFO:teuthology.orchestra.run.smithi032.stdout:standby_count_wanted 1 2024-05-12T04:11:59.735 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.ialtqb{0:34404} state up:active seq 20 join_fscid=1 addr [v2:172.21.15.32:6826/1080358503,v1:172.21.15.32:6827/1080358503] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:11:59.735 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi032.qpsawh{0:34436} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.32:6828/3603452506,v1:172.21.15.32:6829/3603452506] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:11:59.735 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:11:59.735 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:11:59.735 INFO:teuthology.orchestra.run.smithi032.stdout:Standby daemons: 2024-05-12T04:11:59.736 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:11:59.736 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi033.jgbjhk{-1:44343} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6824/997254871,v1:172.21.15.33:6825/997254871] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:11:59.736 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.cephfs.smithi033.hglwml{-1:44373} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/1983593176,v1:172.21.15.33:6827/1983593176] compat {c=[1],r=[1],i=[7ff]}] 2024-05-12T04:11:59.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:59.735+0000 7f326b3ee700 1 -- 172.21.15.32:0/3254445369 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f325006cc20 msgr2=0x7f325006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:11:59.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:59.735+0000 7f326b3ee700 1 --2- 172.21.15.32:0/3254445369 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f325006cc20 0x7f325006f0d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3258004f50 tx=0x7f3258009e30 comp rx=0 tx=0).stop 2024-05-12T04:11:59.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:59.735+0000 7f326b3ee700 1 -- 172.21.15.32:0/3254445369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3264070bb0 msgr2=0x7f32641161b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:11:59.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:59.735+0000 7f326b3ee700 1 --2- 172.21.15.32:0/3254445369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3264070bb0 0x7f32641161b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f326000c8c0 tx=0x7f32600044c0 comp rx=0 tx=0).stop 2024-05-12T04:11:59.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:59.735+0000 7f326b3ee700 1 -- 172.21.15.32:0/3254445369 shutdown_connections 2024-05-12T04:11:59.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:59.735+0000 7f326b3ee700 1 --2- 172.21.15.32:0/3254445369 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f325006cc20 0x7f325006f0d0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:11:59.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:59.735+0000 7f326b3ee700 1 --2- 172.21.15.32:0/3254445369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32640702a0 0x7f3264115c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:11:59.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:59.735+0000 7f326b3ee700 1 --2- 172.21.15.32:0/3254445369 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3264070bb0 0x7f32641161b0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:11:59.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:59.735+0000 7f326b3ee700 1 -- 172.21.15.32:0/3254445369 >> 172.21.15.32:0/3254445369 conn(0x7f32640f8980 msgr2=0x7f3264105a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:11:59.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:59.735+0000 7f326b3ee700 1 -- 172.21.15.32:0/3254445369 shutdown_connections 2024-05-12T04:11:59.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:11:59.735+0000 7f326b3ee700 1 -- 172.21.15.32:0/3254445369 wait complete. 2024-05-12T04:11:59.737 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 37 2024-05-12T04:11:59.754 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:11:59 smithi032 ceph-mon[120392]: pgmap v299: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:11:59.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:11:59 smithi033 ceph-mon[107305]: pgmap v299: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:12:00.503 INFO:teuthology.run_tasks:Running task fs.post_upgrade_checks... 2024-05-12T04:12:00.514 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-12T04:12:00.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:00.647+0000 7f9b56911700 1 -- 172.21.15.32:0/3998979001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b500fe6e0 msgr2=0x7f9b50106db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:00.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:00.647+0000 7f9b56911700 1 --2- 172.21.15.32:0/3998979001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b500fe6e0 0x7f9b50106db0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9b44004940 tx=0x7f9b44009e30 comp rx=0 tx=0).stop 2024-05-12T04:12:00.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:00.647+0000 7f9b56911700 1 -- 172.21.15.32:0/3998979001 shutdown_connections 2024-05-12T04:12:00.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:00.647+0000 7f9b56911700 1 --2- 172.21.15.32:0/3998979001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b500fe6e0 0x7f9b50106db0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:00.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:00.647+0000 7f9b56911700 1 --2- 172.21.15.32:0/3998979001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b500fdd40 0x7f9b500fe110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:00.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:00.647+0000 7f9b56911700 1 -- 172.21.15.32:0/3998979001 >> 172.21.15.32:0/3998979001 conn(0x7f9b50071bb0 msgr2=0x7f9b50071fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:00.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:00.647+0000 7f9b56911700 1 -- 172.21.15.32:0/3998979001 shutdown_connections 2024-05-12T04:12:00.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:00.647+0000 7f9b56911700 1 -- 172.21.15.32:0/3998979001 wait complete. 2024-05-12T04:12:00.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:00.648+0000 7f9b56911700 1 Processor -- start 2024-05-12T04:12:00.649 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:00.648+0000 7f9b56911700 1 -- start start 2024-05-12T04:12:00.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:00.648+0000 7f9b56911700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b500fdd40 0x7f9b500fbc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:00.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:00.648+0000 7f9b56911700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b500fe6e0 0x7f9b500fc140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:00.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:00.648+0000 7f9b56911700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b500fdb50 con 0x7f9b500fdd40 2024-05-12T04:12:00.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:00.648+0000 7f9b56911700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b500fc680 con 0x7f9b500fe6e0 2024-05-12T04:12:00.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:00.648+0000 7f9b4ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b500fdd40 0x7f9b500fbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:00.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:00.648+0000 7f9b4ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b500fdd40 0x7f9b500fbc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60424/0 (socket says 172.21.15.32:60424) 2024-05-12T04:12:00.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:00.648+0000 7f9b4ffff700 1 -- 172.21.15.32:0/2507610804 learned_addr learned my addr 172.21.15.32:0/2507610804 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:12:00.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:00.649+0000 7f9b4f7fe700 1 --2- 172.21.15.32:0/2507610804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b500fe6e0 0x7f9b500fc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:00.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:00.649+0000 7f9b4ffff700 1 -- 172.21.15.32:0/2507610804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b500fe6e0 msgr2=0x7f9b500fc140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:00.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:00.649+0000 7f9b4ffff700 1 --2- 172.21.15.32:0/2507610804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b500fe6e0 0x7f9b500fc140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:00.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:00.649+0000 7f9b4ffff700 1 -- 172.21.15.32:0/2507610804 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b44005040 con 0x7f9b500fdd40 2024-05-12T04:12:00.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:00.649+0000 7f9b4ffff700 1 --2- 172.21.15.32:0/2507610804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b500fdd40 0x7f9b500fbc00 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9b40002a50 tx=0x7f9b400046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:00.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:00.649+0000 7f9b4cff9700 1 -- 172.21.15.32:0/2507610804 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b40007cc0 con 0x7f9b500fdd40 2024-05-12T04:12:00.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:00.649+0000 7f9b4cff9700 1 -- 172.21.15.32:0/2507610804 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9b4000f040 con 0x7f9b500fdd40 2024-05-12T04:12:00.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:00.649+0000 7f9b56911700 1 -- 172.21.15.32:0/2507610804 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b500fc960 con 0x7f9b500fdd40 2024-05-12T04:12:00.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:00.649+0000 7f9b4cff9700 1 -- 172.21.15.32:0/2507610804 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b4000d9c0 con 0x7f9b500fdd40 2024-05-12T04:12:00.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:00.649+0000 7f9b56911700 1 -- 172.21.15.32:0/2507610804 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b50119e70 con 0x7f9b500fdd40 2024-05-12T04:12:00.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:00.650+0000 7f9b56911700 1 -- 172.21.15.32:0/2507610804 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b500655a0 con 0x7f9b500fdd40 2024-05-12T04:12:00.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:00.651+0000 7f9b4cff9700 1 -- 172.21.15.32:0/2507610804 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f9b4000db20 con 0x7f9b500fdd40 2024-05-12T04:12:00.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:00.651+0000 7f9b4cff9700 1 --2- 172.21.15.32:0/2507610804 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f9b3806cc20 0x7f9b3806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:00.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:00.651+0000 7f9b4cff9700 1 -- 172.21.15.32:0/2507610804 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f9b4008bcf0 con 0x7f9b500fdd40 2024-05-12T04:12:00.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:00.651+0000 7f9b4f7fe700 1 --2- 172.21.15.32:0/2507610804 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f9b3806cc20 0x7f9b3806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:00.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:00.652+0000 7f9b4f7fe700 1 --2- 172.21.15.32:0/2507610804 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f9b3806cc20 0x7f9b3806f0d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9b4400b7e0 tx=0x7f9b44009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:00.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:00.656+0000 7f9b4cff9700 1 -- 172.21.15.32:0/2507610804 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f9b40055cd0 con 0x7f9b500fdd40 2024-05-12T04:12:00.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:00 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/3254445369' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-12T04:12:00.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:00.861+0000 7f9b56911700 1 -- 172.21.15.32:0/2507610804 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9b5011a0b0 con 0x7f9b500fdd40 2024-05-12T04:12:00.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:00.862+0000 7f9b4cff9700 1 -- 172.21.15.32:0/2507610804 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16628 (secure 0 0 0) 0x7f9b400592f0 con 0x7f9b500fdd40 2024-05-12T04:12:00.862 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:12:00.863 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":84,"fsid":"a3e178f8-1012-11ef-bc98-c7b262605968","created":"2024-05-12T03:50:18.356038+0000","modified":"2024-05-12T04:08:50.380422+0000","last_up_change":"2024-05-12T04:07:44.234383+0000","last_in_change":"2024-05-12T03:56:03.301729+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-12T03:54:58.296702+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-05-12T03:57:47.465323+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-05-12T03:57:48.324643+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fc534129-ef70-487d-b8d1-f98fc170c1d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":49,"up_from":53,"up_thru":78,"down_at":50,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1676344197},{"type":"v1","addr":"172.21.15.32:6803","nonce":1676344197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1676344197},{"type":"v1","addr":"172.21.15.32:6805","nonce":1676344197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1676344197},{"type":"v1","addr":"172.21.15.32:6809","nonce":1676344197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1676344197},{"type":"v1","addr":"172.21.15.32:6807","nonce":1676344197}]},"public_addr":"172.21.15.32:6803/1676344197","cluster_addr":"172.21.15.32:6805/1676344197","heartbeat_back_addr":"172.21.15.32:6809/1676344197","heartbeat_front_addr":"172.21.15.32:6807/1676344197","state":["exists","up"]},{"osd":1,"uuid":"f8c35b6a-5afc-447b-b318-d56097de28d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":54,"up_from":58,"up_thru":78,"down_at":55,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":954580115},{"type":"v1","addr":"172.21.15.32:6811","nonce":954580115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":954580115},{"type":"v1","addr":"172.21.15.32:6813","nonce":954580115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":954580115},{"type":"v1","addr":"172.21.15.32:6817","nonce":954580115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":954580115},{"type":"v1","addr":"172.21.15.32:6815","nonce":954580115}]},"public_addr":"172.21.15.32:6811/954580115","cluster_addr":"172.21.15.32:6813/954580115","heartbeat_back_addr":"172.21.15.32:6817/954580115","heartbeat_front_addr":"172.21.15.32:6815/954580115","state":["exists","up"]},{"osd":2,"uuid":"4ed4b55d-cceb-494b-bdff-fef99d6c8cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":59,"up_from":63,"up_thru":78,"down_at":60,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":793261683},{"type":"v1","addr":"172.21.15.32:6819","nonce":793261683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":793261683},{"type":"v1","addr":"172.21.15.32:6821","nonce":793261683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":793261683},{"type":"v1","addr":"172.21.15.32:6825","nonce":793261683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":793261683},{"type":"v1","addr":"172.21.15.32:6823","nonce":793261683}]},"public_addr":"172.21.15.32:6819/793261683","cluster_addr":"172.21.15.32:6821/793261683","heartbeat_back_addr":"172.21.15.32:6825/793261683","heartbeat_front_addr":"172.21.15.32:6823/793261683","state":["exists","up"]},{"osd":3,"uuid":"ef191267-c0a1-4be7-9e8b-761700af87a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":64,"up_from":68,"up_thru":78,"down_at":65,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":3436738456},{"type":"v1","addr":"172.21.15.33:6801","nonce":3436738456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":3436738456},{"type":"v1","addr":"172.21.15.33:6803","nonce":3436738456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":3436738456},{"type":"v1","addr":"172.21.15.33:6807","nonce":3436738456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":3436738456},{"type":"v1","addr":"172.21.15.33:6805","nonce":3436738456}]},"public_addr":"172.21.15.33:6801/3436738456","cluster_addr":"172.21.15.33:6803/3436738456","heartbeat_back_addr":"172.21.15.33:6807/3436738456","heartbeat_front_addr":"172.21.15.33:6805/3436738456","state":["exists","up"]},{"osd":4,"uuid":"fe45f168-076a-4888-895d-bb710bd0da73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":69,"up_from":73,"up_thru":78,"down_at":70,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1028896705},{"type":"v1","addr":"172.21.15.33:6809","nonce":1028896705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1028896705},{"type":"v1","addr":"172.21.15.33:6811","nonce":1028896705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1028896705},{"type":"v1","addr":"172.21.15.33:6815","nonce":1028896705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1028896705},{"type":"v1","addr":"172.21.15.33:6813","nonce":1028896705}]},"public_addr":"172.21.15.33:6809/1028896705","cluster_addr":"172.21.15.33:6811/1028896705","heartbeat_back_addr":"172.21.15.33:6815/1028896705","heartbeat_front_addr":"172.21.15.33:6813/1028896705","state":["exists","up"]},{"osd":5,"uuid":"f7a0e14e-a866-4163-a7df-f8a43b860bc0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":74,"up_from":78,"up_thru":78,"down_at":75,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":4274440090},{"type":"v1","addr":"172.21.15.33:6817","nonce":4274440090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":4274440090},{"type":"v1","addr":"172.21.15.33:6819","nonce":4274440090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":4274440090},{"type":"v1","addr":"172.21.15.33:6823","nonce":4274440090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":4274440090},{"type":"v1","addr":"172.21.15.33:6821","nonce":4274440090}]},"public_addr":"172.21.15.33:6817/4274440090","cluster_addr":"172.21.15.33:6819/4274440090","heartbeat_back_addr":"172.21.15.33:6823/4274440090","heartbeat_front_addr":"172.21.15.33:6821/4274440090","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-12T03:53:56.134194+0000","dead_epoch":49},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-12T03:54:25.231975+0000","dead_epoch":54},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-12T03:54:54.991052+0000","dead_epoch":59},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-12T03:55:22.680656+0000","dead_epoch":64},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-12T03:55:52.446678+0000","dead_epoch":69},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-12T03:56:22.128957+0000","dead_epoch":74}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6829/3100155296":"2024-05-13T04:08:08.971776+0000","172.21.15.32:6828/3100155296":"2024-05-13T04:08:08.971776+0000","172.21.15.32:0/912074455":"2024-05-13T04:03:15.687820+0000","172.21.15.32:6800/1121526649":"2024-05-13T03:50:49.378828+0000","172.21.15.32:0/3343483246":"2024-05-13T04:00:09.761962+0000","172.21.15.32:6827/1907818042":"2024-05-13T03:58:02.805639+0000","172.21.15.32:0/4293156271":"2024-05-13T03:51:25.269581+0000","172.21.15.32:0/2900983644":"2024-05-13T03:52:39.772717+0000","172.21.15.32:0/2236672133":"2024-05-13T04:00:09.761962+0000","172.21.15.32:6800/2967201790":"2024-05-13T03:51:25.269581+0000","172.21.15.33:6826/433858238":"2024-05-13T04:08:50.380388+0000","172.21.15.32:6826/1907818042":"2024-05-13T03:58:02.805639+0000","172.21.15.32:6801/1121526649":"2024-05-13T03:50:49.378828+0000","172.21.15.32:0/2098237559":"2024-05-13T03:50:49.378828+0000","172.21.15.32:6800/4089797100":"2024-05-13T04:00:09.761962+0000","172.21.15.32:0/1463926533":"2024-05-13T03:50:49.378828+0000","172.21.15.32:0/1834879835":"2024-05-13T04:00:09.761962+0000","172.21.15.32:0/165225041":"2024-05-13T03:51:25.269581+0000","172.21.15.32:0/625670655":"2024-05-13T03:50:49.378828+0000","172.21.15.33:6827/433858238":"2024-05-13T04:08:50.380388+0000","172.21.15.32:0/3185268518":"2024-05-13T03:52:39.772717+0000","172.21.15.33:6829/1598726962":"2024-05-13T04:02:10.132488+0000","172.21.15.32:0/1073406471":"2024-05-13T03:52:39.772717+0000","172.21.15.32:6801/4089797100":"2024-05-13T04:00:09.761962+0000","172.21.15.32:0/1356801576":"2024-05-13T03:51:25.269581+0000","172.21.15.32:0/993128492":"2024-05-13T04:00:09.761962+0000","172.21.15.32:0/2523500808":"2024-05-13T04:03:15.687820+0000","172.21.15.32:6801/2967201790":"2024-05-13T03:51:25.269581+0000","172.21.15.32:6800/3729453095":"2024-05-13T04:03:15.687820+0000","172.21.15.33:6828/1598726962":"2024-05-13T04:02:10.132488+0000","172.21.15.32:0/2757703284":"2024-05-13T04:03:15.687820+0000","172.21.15.33:0/2636219823":"2024-05-13T04:02:10.132488+0000","172.21.15.33:0/2129212605":"2024-05-13T04:02:10.132488+0000","172.21.15.33:0/619425673":"2024-05-13T04:02:10.132488+0000","172.21.15.33:0/3980223680":"2024-05-13T04:02:10.132488+0000","172.21.15.32:6800/1988399583":"2024-05-13T03:52:39.772717+0000","172.21.15.32:0/213713458":"2024-05-13T04:03:15.687820+0000","172.21.15.32:6801/1988399583":"2024-05-13T03:52:39.772717+0000","172.21.15.32:6801/3729453095":"2024-05-13T04:03:15.687820+0000","172.21.15.32:0/3906624315":"2024-05-13T04:03:15.687820+0000","172.21.15.32:0/3077610654":"2024-05-13T04:03:15.687820+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-12T04:12:00.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:00.864+0000 7f9b56911700 1 -- 172.21.15.32:0/2507610804 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f9b3806cc20 msgr2=0x7f9b3806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:00.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:00.864+0000 7f9b56911700 1 --2- 172.21.15.32:0/2507610804 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f9b3806cc20 0x7f9b3806f0d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9b4400b7e0 tx=0x7f9b44009e30 comp rx=0 tx=0).stop 2024-05-12T04:12:00.864 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:00.864+0000 7f9b56911700 1 -- 172.21.15.32:0/2507610804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b500fdd40 msgr2=0x7f9b500fbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:00.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:00.864+0000 7f9b56911700 1 --2- 172.21.15.32:0/2507610804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b500fdd40 0x7f9b500fbc00 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9b40002a50 tx=0x7f9b400046b0 comp rx=0 tx=0).stop 2024-05-12T04:12:00.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:00.864+0000 7f9b56911700 1 -- 172.21.15.32:0/2507610804 shutdown_connections 2024-05-12T04:12:00.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:00.864+0000 7f9b56911700 1 --2- 172.21.15.32:0/2507610804 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f9b3806cc20 0x7f9b3806f0d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:00.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:00.864+0000 7f9b56911700 1 --2- 172.21.15.32:0/2507610804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b500fdd40 0x7f9b500fbc00 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:00.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:00.864+0000 7f9b56911700 1 --2- 172.21.15.32:0/2507610804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b500fe6e0 0x7f9b500fc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:00.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:00.864+0000 7f9b56911700 1 -- 172.21.15.32:0/2507610804 >> 172.21.15.32:0/2507610804 conn(0x7f9b50071bb0 msgr2=0x7f9b500fa620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:00.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:00.864+0000 7f9b56911700 1 -- 172.21.15.32:0/2507610804 shutdown_connections 2024-05-12T04:12:00.865 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:00.864+0000 7f9b56911700 1 -- 172.21.15.32:0/2507610804 wait complete. 2024-05-12T04:12:00.877 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-12T03:54:58.296702+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-05-12T03:57:47.465323+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-05-12T03:57:48.324643+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-12T04:12:00.878 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-12T04:12:00.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:00 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/3254445369' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-12T04:12:01.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.002+0000 7f6e1d3b4700 1 -- 172.21.15.32:0/2155549384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e18102560 msgr2=0x7f6e180fbbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:01.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.002+0000 7f6e1d3b4700 1 --2- 172.21.15.32:0/2155549384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e18102560 0x7f6e180fbbd0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c004940 tx=0x7f6e0c009e30 comp rx=0 tx=0).stop 2024-05-12T04:12:01.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.002+0000 7f6e1d3b4700 1 -- 172.21.15.32:0/2155549384 shutdown_connections 2024-05-12T04:12:01.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.002+0000 7f6e1d3b4700 1 --2- 172.21.15.32:0/2155549384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e18102560 0x7f6e180fbbd0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:01.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.002+0000 7f6e1d3b4700 1 --2- 172.21.15.32:0/2155549384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e18101bc0 0x7f6e18101f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:01.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.002+0000 7f6e1d3b4700 1 -- 172.21.15.32:0/2155549384 >> 172.21.15.32:0/2155549384 conn(0x7f6e180f7750 msgr2=0x7f6e180f9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:01.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.003+0000 7f6e1d3b4700 1 -- 172.21.15.32:0/2155549384 shutdown_connections 2024-05-12T04:12:01.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.003+0000 7f6e1d3b4700 1 -- 172.21.15.32:0/2155549384 wait complete. 2024-05-12T04:12:01.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.003+0000 7f6e1d3b4700 1 Processor -- start 2024-05-12T04:12:01.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.003+0000 7f6e1d3b4700 1 -- start start 2024-05-12T04:12:01.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.003+0000 7f6e1d3b4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e18101bc0 0x7f6e18115970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:01.005 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.003+0000 7f6e1d3b4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e18102560 0x7f6e18115eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:01.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.003+0000 7f6e1d3b4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e18116500 con 0x7f6e18101bc0 2024-05-12T04:12:01.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.003+0000 7f6e1d3b4700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1810fb00 con 0x7f6e18102560 2024-05-12T04:12:01.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.004+0000 7f6e16ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e18101bc0 0x7f6e18115970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:01.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.004+0000 7f6e16ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e18101bc0 0x7f6e18115970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60456/0 (socket says 172.21.15.32:60456) 2024-05-12T04:12:01.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.004+0000 7f6e16ffd700 1 -- 172.21.15.32:0/3206121925 learned_addr learned my addr 172.21.15.32:0/3206121925 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:12:01.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.004+0000 7f6e167fc700 1 --2- 172.21.15.32:0/3206121925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e18102560 0x7f6e18115eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:01.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.004+0000 7f6e16ffd700 1 -- 172.21.15.32:0/3206121925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e18102560 msgr2=0x7f6e18115eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:01.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.004+0000 7f6e16ffd700 1 --2- 172.21.15.32:0/3206121925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e18102560 0x7f6e18115eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:01.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.004+0000 7f6e16ffd700 1 -- 172.21.15.32:0/3206121925 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e0c005040 con 0x7f6e18101bc0 2024-05-12T04:12:01.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.004+0000 7f6e167fc700 1 --2- 172.21.15.32:0/3206121925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e18102560 0x7f6e18115eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-12T04:12:01.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.004+0000 7f6e16ffd700 1 --2- 172.21.15.32:0/3206121925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e18101bc0 0x7f6e18115970 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6e0800c830 tx=0x7f6e08004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:01.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.004+0000 7f6dff7fe700 1 -- 172.21.15.32:0/3206121925 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e0800a760 con 0x7f6e18101bc0 2024-05-12T04:12:01.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.004+0000 7f6dff7fe700 1 -- 172.21.15.32:0/3206121925 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6e0800ad70 con 0x7f6e18101bc0 2024-05-12T04:12:01.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.004+0000 7f6e1d3b4700 1 -- 172.21.15.32:0/3206121925 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e1810fde0 con 0x7f6e18101bc0 2024-05-12T04:12:01.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.004+0000 7f6dff7fe700 1 -- 172.21.15.32:0/3206121925 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e080113a0 con 0x7f6e18101bc0 2024-05-12T04:12:01.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.004+0000 7f6e1d3b4700 1 -- 172.21.15.32:0/3206121925 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e18110370 con 0x7f6e18101bc0 2024-05-12T04:12:01.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.005+0000 7f6e1d3b4700 1 -- 172.21.15.32:0/3206121925 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e180fd310 con 0x7f6e18101bc0 2024-05-12T04:12:01.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.006+0000 7f6dff7fe700 1 -- 172.21.15.32:0/3206121925 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f6e08012400 con 0x7f6e18101bc0 2024-05-12T04:12:01.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.006+0000 7f6dff7fe700 1 --2- 172.21.15.32:0/3206121925 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f6e0006cc20 0x7f6e0006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:01.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.006+0000 7f6dff7fe700 1 -- 172.21.15.32:0/3206121925 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f6e0808c110 con 0x7f6e18101bc0 2024-05-12T04:12:01.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.006+0000 7f6e167fc700 1 --2- 172.21.15.32:0/3206121925 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f6e0006cc20 0x7f6e0006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:01.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.007+0000 7f6e167fc700 1 --2- 172.21.15.32:0/3206121925 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f6e0006cc20 0x7f6e0006f0d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6e18110de0 tx=0x7f6e0c009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:01.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.011+0000 7f6dff7fe700 1 -- 172.21.15.32:0/3206121925 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f6e08059b80 con 0x7f6e18101bc0 2024-05-12T04:12:01.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.214+0000 7f6e1d3b4700 1 -- 172.21.15.32:0/3206121925 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6e18108b50 con 0x7f6e18101bc0 2024-05-12T04:12:01.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.215+0000 7f6dff7fe700 1 -- 172.21.15.32:0/3206121925 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6e08059710 con 0x7f6e18101bc0 2024-05-12T04:12:01.215 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-05-12T04:12:01.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.217+0000 7f6e1d3b4700 1 -- 172.21.15.32:0/3206121925 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f6e0006cc20 msgr2=0x7f6e0006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:01.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.217+0000 7f6e1d3b4700 1 --2- 172.21.15.32:0/3206121925 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f6e0006cc20 0x7f6e0006f0d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6e18110de0 tx=0x7f6e0c009e30 comp rx=0 tx=0).stop 2024-05-12T04:12:01.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.217+0000 7f6e1d3b4700 1 -- 172.21.15.32:0/3206121925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e18101bc0 msgr2=0x7f6e18115970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:01.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.217+0000 7f6e1d3b4700 1 --2- 172.21.15.32:0/3206121925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e18101bc0 0x7f6e18115970 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6e0800c830 tx=0x7f6e08004470 comp rx=0 tx=0).stop 2024-05-12T04:12:01.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.217+0000 7f6e1d3b4700 1 -- 172.21.15.32:0/3206121925 shutdown_connections 2024-05-12T04:12:01.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.217+0000 7f6e1d3b4700 1 --2- 172.21.15.32:0/3206121925 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f6e0006cc20 0x7f6e0006f0d0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:01.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.217+0000 7f6e1d3b4700 1 --2- 172.21.15.32:0/3206121925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e18101bc0 0x7f6e18115970 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:01.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.217+0000 7f6e1d3b4700 1 --2- 172.21.15.32:0/3206121925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e18102560 0x7f6e18115eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:01.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.217+0000 7f6e1d3b4700 1 -- 172.21.15.32:0/3206121925 >> 172.21.15.32:0/3206121925 conn(0x7f6e180f7750 msgr2=0x7f6e180f8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:01.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.217+0000 7f6e1d3b4700 1 -- 172.21.15.32:0/3206121925 shutdown_connections 2024-05-12T04:12:01.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.217+0000 7f6e1d3b4700 1 -- 172.21.15.32:0/3206121925 wait complete. 2024-05-12T04:12:01.230 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-05-12T04:12:01.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.355+0000 7f572900f700 1 -- 172.21.15.32:0/1629492163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5724102cd0 msgr2=0x7f57241030a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:01.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.355+0000 7f572900f700 1 --2- 172.21.15.32:0/1629492163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5724102cd0 0x7f57241030a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5714004940 tx=0x7f5714009e30 comp rx=0 tx=0).stop 2024-05-12T04:12:01.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.355+0000 7f572900f700 1 -- 172.21.15.32:0/1629492163 shutdown_connections 2024-05-12T04:12:01.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.355+0000 7f572900f700 1 --2- 172.21.15.32:0/1629492163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5724103670 0x7f57240fb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:01.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.355+0000 7f572900f700 1 --2- 172.21.15.32:0/1629492163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5724102cd0 0x7f57241030a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:01.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.355+0000 7f572900f700 1 -- 172.21.15.32:0/1629492163 >> 172.21.15.32:0/1629492163 conn(0x7f5724072230 msgr2=0x7f5724072630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:01.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.355+0000 7f572900f700 1 -- 172.21.15.32:0/1629492163 shutdown_connections 2024-05-12T04:12:01.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.356+0000 7f572900f700 1 -- 172.21.15.32:0/1629492163 wait complete. 2024-05-12T04:12:01.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.356+0000 7f572900f700 1 Processor -- start 2024-05-12T04:12:01.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.356+0000 7f572900f700 1 -- start start 2024-05-12T04:12:01.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.357+0000 7f572900f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5724102cd0 0x7f57241137f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:01.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.357+0000 7f572900f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5724103670 0x7f5724113d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:01.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.357+0000 7f572900f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5724114380 con 0x7f5724102cd0 2024-05-12T04:12:01.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.357+0000 7f572900f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f572410d980 con 0x7f5724103670 2024-05-12T04:12:01.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.357+0000 7f5722d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5724102cd0 0x7f57241137f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:01.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.357+0000 7f5722d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5724102cd0 0x7f57241137f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60470/0 (socket says 172.21.15.32:60470) 2024-05-12T04:12:01.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.357+0000 7f5722d9d700 1 -- 172.21.15.32:0/2800887053 learned_addr learned my addr 172.21.15.32:0/2800887053 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:12:01.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.357+0000 7f572259c700 1 --2- 172.21.15.32:0/2800887053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5724103670 0x7f5724113d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:01.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.357+0000 7f5722d9d700 1 -- 172.21.15.32:0/2800887053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5724103670 msgr2=0x7f5724113d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:01.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.357+0000 7f5722d9d700 1 --2- 172.21.15.32:0/2800887053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5724103670 0x7f5724113d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:01.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.357+0000 7f5722d9d700 1 -- 172.21.15.32:0/2800887053 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5714005040 con 0x7f5724102cd0 2024-05-12T04:12:01.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.357+0000 7f572259c700 1 --2- 172.21.15.32:0/2800887053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5724103670 0x7f5724113d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-12T04:12:01.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.357+0000 7f5722d9d700 1 --2- 172.21.15.32:0/2800887053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5724102cd0 0x7f57241137f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5714009390 tx=0x7f5714009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:01.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.357+0000 7f570b7fe700 1 -- 172.21.15.32:0/2800887053 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f571400e040 con 0x7f5724102cd0 2024-05-12T04:12:01.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.357+0000 7f572900f700 1 -- 172.21.15.32:0/2800887053 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f572410dc00 con 0x7f5724102cd0 2024-05-12T04:12:01.365 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.357+0000 7f570b7fe700 1 -- 172.21.15.32:0/2800887053 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5714013420 con 0x7f5724102cd0 2024-05-12T04:12:01.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.358+0000 7f570b7fe700 1 -- 172.21.15.32:0/2800887053 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f571401c630 con 0x7f5724102cd0 2024-05-12T04:12:01.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.358+0000 7f572900f700 1 -- 172.21.15.32:0/2800887053 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f572410e140 con 0x7f5724102cd0 2024-05-12T04:12:01.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.359+0000 7f572900f700 1 -- 172.21.15.32:0/2800887053 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57240fe590 con 0x7f5724102cd0 2024-05-12T04:12:01.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.359+0000 7f570b7fe700 1 -- 172.21.15.32:0/2800887053 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f5714013a00 con 0x7f5724102cd0 2024-05-12T04:12:01.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.360+0000 7f570b7fe700 1 --2- 172.21.15.32:0/2800887053 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f570c06cc20 0x7f570c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:01.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.360+0000 7f570b7fe700 1 -- 172.21.15.32:0/2800887053 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f571408d380 con 0x7f5724102cd0 2024-05-12T04:12:01.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.360+0000 7f572259c700 1 --2- 172.21.15.32:0/2800887053 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f570c06cc20 0x7f570c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:01.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.360+0000 7f572259c700 1 --2- 172.21.15.32:0/2800887053 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f570c06cc20 0x7f570c06f0d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5718007fd0 tx=0x7f5718009040 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:01.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.364+0000 7f570b7fe700 1 -- 172.21.15.32:0/2800887053 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f571405adf0 con 0x7f5724102cd0 2024-05-12T04:12:01.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:01 smithi032 ceph-mon[120392]: pgmap v300: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:12:01.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:01 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:12:01.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:01 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/2507610804' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-12T04:12:01.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:01 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/3206121925' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-12T04:12:01.568 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.568+0000 7f572900f700 1 -- 172.21.15.32:0/2800887053 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f5724109cc0 con 0x7f5724102cd0 2024-05-12T04:12:01.569 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.569+0000 7f570b7fe700 1 -- 172.21.15.32:0/2800887053 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v84) v1 ==== 107+0+11 (secure 0 0 0) 0x7f571405a980 con 0x7f5724102cd0 2024-05-12T04:12:01.569 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-05-12T04:12:01.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.570+0000 7f572900f700 1 -- 172.21.15.32:0/2800887053 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f570c06cc20 msgr2=0x7f570c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:01.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.570+0000 7f572900f700 1 --2- 172.21.15.32:0/2800887053 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f570c06cc20 0x7f570c06f0d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5718007fd0 tx=0x7f5718009040 comp rx=0 tx=0).stop 2024-05-12T04:12:01.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.571+0000 7f572900f700 1 -- 172.21.15.32:0/2800887053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5724102cd0 msgr2=0x7f57241137f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:01.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.571+0000 7f572900f700 1 --2- 172.21.15.32:0/2800887053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5724102cd0 0x7f57241137f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5714009390 tx=0x7f5714009e30 comp rx=0 tx=0).stop 2024-05-12T04:12:01.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.571+0000 7f572900f700 1 -- 172.21.15.32:0/2800887053 shutdown_connections 2024-05-12T04:12:01.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.571+0000 7f572900f700 1 --2- 172.21.15.32:0/2800887053 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f570c06cc20 0x7f570c06f0d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:01.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.571+0000 7f572900f700 1 --2- 172.21.15.32:0/2800887053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5724102cd0 0x7f57241137f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:01.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.571+0000 7f572900f700 1 --2- 172.21.15.32:0/2800887053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5724103670 0x7f5724113d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:01.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.571+0000 7f572900f700 1 -- 172.21.15.32:0/2800887053 >> 172.21.15.32:0/2800887053 conn(0x7f5724072230 msgr2=0x7f57240f9e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:01.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.571+0000 7f572900f700 1 -- 172.21.15.32:0/2800887053 shutdown_connections 2024-05-12T04:12:01.572 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.571+0000 7f572900f700 1 -- 172.21.15.32:0/2800887053 wait complete. 2024-05-12T04:12:01.583 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-05-12T04:12:01.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.709+0000 7f72fcb14700 1 -- 172.21.15.32:0/520327802 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72f8102540 msgr2=0x7f72f80fbbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:01.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.709+0000 7f72fcb14700 1 --2- 172.21.15.32:0/520327802 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72f8102540 0x7f72f80fbbb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f72e8004940 tx=0x7f72e8009e30 comp rx=0 tx=0).stop 2024-05-12T04:12:01.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.709+0000 7f72fcb14700 1 -- 172.21.15.32:0/520327802 shutdown_connections 2024-05-12T04:12:01.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.709+0000 7f72fcb14700 1 --2- 172.21.15.32:0/520327802 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72f8102540 0x7f72f80fbbb0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:01.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.709+0000 7f72fcb14700 1 --2- 172.21.15.32:0/520327802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72f8101ba0 0x7f72f8101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:01.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.709+0000 7f72fcb14700 1 -- 172.21.15.32:0/520327802 >> 172.21.15.32:0/520327802 conn(0x7f72f8072ad0 msgr2=0x7f72f8072ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:01.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.709+0000 7f72fcb14700 1 -- 172.21.15.32:0/520327802 shutdown_connections 2024-05-12T04:12:01.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.709+0000 7f72fcb14700 1 -- 172.21.15.32:0/520327802 wait complete. 2024-05-12T04:12:01.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.709+0000 7f72fcb14700 1 Processor -- start 2024-05-12T04:12:01.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.709+0000 7f72fcb14700 1 -- start start 2024-05-12T04:12:01.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.710+0000 7f72fcb14700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72f8101ba0 0x7f72f81126a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:01.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.710+0000 7f72fcb14700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72f8102540 0x7f72f8112be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:01.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.710+0000 7f72fcb14700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f8113230 con 0x7f72f8102540 2024-05-12T04:12:01.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.710+0000 7f72fcb14700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f810c7f0 con 0x7f72f8101ba0 2024-05-12T04:12:01.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.710+0000 7f72f5d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72f8102540 0x7f72f8112be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:01.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.710+0000 7f72f5d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72f8102540 0x7f72f8112be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60478/0 (socket says 172.21.15.32:60478) 2024-05-12T04:12:01.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.710+0000 7f72f5d9b700 1 -- 172.21.15.32:0/4077575263 learned_addr learned my addr 172.21.15.32:0/4077575263 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:12:01.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.710+0000 7f72f659c700 1 --2- 172.21.15.32:0/4077575263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72f8101ba0 0x7f72f81126a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:01.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.710+0000 7f72f5d9b700 1 -- 172.21.15.32:0/4077575263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72f8101ba0 msgr2=0x7f72f81126a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:01.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.710+0000 7f72f5d9b700 1 --2- 172.21.15.32:0/4077575263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72f8101ba0 0x7f72f81126a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:01.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.710+0000 7f72f5d9b700 1 -- 172.21.15.32:0/4077575263 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72e8005040 con 0x7f72f8102540 2024-05-12T04:12:01.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.710+0000 7f72f5d9b700 1 --2- 172.21.15.32:0/4077575263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72f8102540 0x7f72f8112be0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f72e8009380 tx=0x7f72e8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:01.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.710+0000 7f72e6ffd700 1 -- 172.21.15.32:0/4077575263 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72e800e040 con 0x7f72f8102540 2024-05-12T04:12:01.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.710+0000 7f72e6ffd700 1 -- 172.21.15.32:0/4077575263 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f72e8013420 con 0x7f72f8102540 2024-05-12T04:12:01.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.710+0000 7f72fcb14700 1 -- 172.21.15.32:0/4077575263 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72f810ca70 con 0x7f72f8102540 2024-05-12T04:12:01.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.710+0000 7f72e6ffd700 1 -- 172.21.15.32:0/4077575263 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72e801c630 con 0x7f72f8102540 2024-05-12T04:12:01.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.710+0000 7f72fcb14700 1 -- 172.21.15.32:0/4077575263 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72f810cfb0 con 0x7f72f8102540 2024-05-12T04:12:01.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.711+0000 7f72fcb14700 1 -- 172.21.15.32:0/4077575263 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72f8112330 con 0x7f72f8102540 2024-05-12T04:12:01.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.712+0000 7f72e6ffd700 1 -- 172.21.15.32:0/4077575263 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f72e80139b0 con 0x7f72f8102540 2024-05-12T04:12:01.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.712+0000 7f72e6ffd700 1 --2- 172.21.15.32:0/4077575263 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f72e006cc20 0x7f72e006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:01.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.713+0000 7f72e6ffd700 1 -- 172.21.15.32:0/4077575263 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f72e808d210 con 0x7f72f8102540 2024-05-12T04:12:01.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.713+0000 7f72f659c700 1 --2- 172.21.15.32:0/4077575263 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f72e006cc20 0x7f72e006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:01.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.713+0000 7f72f659c700 1 --2- 172.21.15.32:0/4077575263 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f72e006cc20 0x7f72e006f0d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f72ec004f40 tx=0x7f72ec00b040 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:01.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.717+0000 7f72e6ffd700 1 -- 172.21.15.32:0/4077575263 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f72e805ac80 con 0x7f72f8102540 2024-05-12T04:12:01.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:01 smithi033 ceph-mon[107305]: pgmap v300: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:12:01.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:01 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:12:01.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:01 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/2507610804' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-12T04:12:01.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:01 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/3206121925' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-12T04:12:01.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.922+0000 7f72fcb14700 1 -- 172.21.15.32:0/4077575263 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f72f80fd2f0 con 0x7f72f8102540 2024-05-12T04:12:01.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.923+0000 7f72e6ffd700 1 -- 172.21.15.32:0/4077575263 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v84) v1 ==== 107+0+11 (secure 0 0 0) 0x7f72e805a810 con 0x7f72f8102540 2024-05-12T04:12:01.923 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-05-12T04:12:01.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.925+0000 7f72fcb14700 1 -- 172.21.15.32:0/4077575263 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f72e006cc20 msgr2=0x7f72e006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:01.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.925+0000 7f72fcb14700 1 --2- 172.21.15.32:0/4077575263 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f72e006cc20 0x7f72e006f0d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f72ec004f40 tx=0x7f72ec00b040 comp rx=0 tx=0).stop 2024-05-12T04:12:01.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.925+0000 7f72fcb14700 1 -- 172.21.15.32:0/4077575263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72f8102540 msgr2=0x7f72f8112be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:01.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.925+0000 7f72fcb14700 1 --2- 172.21.15.32:0/4077575263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72f8102540 0x7f72f8112be0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f72e8009380 tx=0x7f72e8009e30 comp rx=0 tx=0).stop 2024-05-12T04:12:01.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.925+0000 7f72fcb14700 1 -- 172.21.15.32:0/4077575263 shutdown_connections 2024-05-12T04:12:01.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.925+0000 7f72fcb14700 1 --2- 172.21.15.32:0/4077575263 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f72e006cc20 0x7f72e006f0d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:01.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.925+0000 7f72fcb14700 1 --2- 172.21.15.32:0/4077575263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72f8101ba0 0x7f72f81126a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:01.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.925+0000 7f72fcb14700 1 --2- 172.21.15.32:0/4077575263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72f8102540 0x7f72f8112be0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:01.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.925+0000 7f72fcb14700 1 -- 172.21.15.32:0/4077575263 >> 172.21.15.32:0/4077575263 conn(0x7f72f8072ad0 msgr2=0x7f72f8104940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:01.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.925+0000 7f72fcb14700 1 -- 172.21.15.32:0/4077575263 shutdown_connections 2024-05-12T04:12:01.926 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:01.925+0000 7f72fcb14700 1 -- 172.21.15.32:0/4077575263 wait complete. 2024-05-12T04:12:01.938 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-12T04:12:02.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.059+0000 7fc9eecd8700 1 -- 172.21.15.32:0/1315405434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9e8102ce0 msgr2=0x7fc9e81030b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:02.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.059+0000 7fc9eecd8700 1 --2- 172.21.15.32:0/1315405434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9e8102ce0 0x7fc9e81030b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc004940 tx=0x7fc9dc009e30 comp rx=0 tx=0).stop 2024-05-12T04:12:02.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.060+0000 7fc9eecd8700 1 -- 172.21.15.32:0/1315405434 shutdown_connections 2024-05-12T04:12:02.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.060+0000 7fc9eecd8700 1 --2- 172.21.15.32:0/1315405434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9e8103680 0x7fc9e80fccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:02.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.060+0000 7fc9eecd8700 1 --2- 172.21.15.32:0/1315405434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9e8102ce0 0x7fc9e81030b0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:02.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.060+0000 7fc9eecd8700 1 -- 172.21.15.32:0/1315405434 >> 172.21.15.32:0/1315405434 conn(0x7fc9e80f8810 msgr2=0x7fc9e80fac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:02.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.060+0000 7fc9eecd8700 1 -- 172.21.15.32:0/1315405434 shutdown_connections 2024-05-12T04:12:02.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.060+0000 7fc9eecd8700 1 -- 172.21.15.32:0/1315405434 wait complete. 2024-05-12T04:12:02.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.060+0000 7fc9eecd8700 1 Processor -- start 2024-05-12T04:12:02.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.060+0000 7fc9eecd8700 1 -- start start 2024-05-12T04:12:02.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.060+0000 7fc9eecd8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9e8102ce0 0x7fc9e81137b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:02.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.060+0000 7fc9eecd8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9e8103680 0x7fc9e8113cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:02.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.060+0000 7fc9eecd8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e81143d0 con 0x7fc9e8103680 2024-05-12T04:12:02.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.061+0000 7fc9eecd8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e810d940 con 0x7fc9e8102ce0 2024-05-12T04:12:02.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.061+0000 7fc9e7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9e8103680 0x7fc9e8113cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:02.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.061+0000 7fc9e7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9e8103680 0x7fc9e8113cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39156/0 (socket says 172.21.15.32:39156) 2024-05-12T04:12:02.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.061+0000 7fc9e7fff700 1 -- 172.21.15.32:0/4191454992 learned_addr learned my addr 172.21.15.32:0/4191454992 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:12:02.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.061+0000 7fc9eca74700 1 --2- 172.21.15.32:0/4191454992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9e8102ce0 0x7fc9e81137b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:02.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.061+0000 7fc9e7fff700 1 -- 172.21.15.32:0/4191454992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9e8102ce0 msgr2=0x7fc9e81137b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:02.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.061+0000 7fc9e7fff700 1 --2- 172.21.15.32:0/4191454992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9e8102ce0 0x7fc9e81137b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:02.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.061+0000 7fc9e7fff700 1 -- 172.21.15.32:0/4191454992 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9dc005040 con 0x7fc9e8103680 2024-05-12T04:12:02.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.061+0000 7fc9eca74700 1 --2- 172.21.15.32:0/4191454992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9e8102ce0 0x7fc9e81137b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-12T04:12:02.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.061+0000 7fc9e7fff700 1 --2- 172.21.15.32:0/4191454992 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9e8103680 0x7fc9e8113cf0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc9d400b790 tx=0x7fc9d4009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:02.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.061+0000 7fc9e57fa700 1 -- 172.21.15.32:0/4191454992 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc9d400f6f0 con 0x7fc9e8103680 2024-05-12T04:12:02.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.061+0000 7fc9e57fa700 1 -- 172.21.15.32:0/4191454992 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc9d400fd00 con 0x7fc9e8103680 2024-05-12T04:12:02.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.061+0000 7fc9eecd8700 1 -- 172.21.15.32:0/4191454992 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9e810dc20 con 0x7fc9e8103680 2024-05-12T04:12:02.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.061+0000 7fc9e57fa700 1 -- 172.21.15.32:0/4191454992 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc9d40103a0 con 0x7fc9e8103680 2024-05-12T04:12:02.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.061+0000 7fc9eecd8700 1 -- 172.21.15.32:0/4191454992 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9e810e1e0 con 0x7fc9e8103680 2024-05-12T04:12:02.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.062+0000 7fc9eecd8700 1 -- 172.21.15.32:0/4191454992 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9e80fe430 con 0x7fc9e8103680 2024-05-12T04:12:02.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.063+0000 7fc9e57fa700 1 -- 172.21.15.32:0/4191454992 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fc9d400f850 con 0x7fc9e8103680 2024-05-12T04:12:02.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.063+0000 7fc9e57fa700 1 --2- 172.21.15.32:0/4191454992 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fc9d806cc20 0x7fc9d806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:02.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.063+0000 7fc9e57fa700 1 -- 172.21.15.32:0/4191454992 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fc9d408bf50 con 0x7fc9e8103680 2024-05-12T04:12:02.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.063+0000 7fc9eca74700 1 --2- 172.21.15.32:0/4191454992 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fc9d806cc20 0x7fc9d806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:02.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.064+0000 7fc9eca74700 1 --2- 172.21.15.32:0/4191454992 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fc9d806cc20 0x7fc9d806f0d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc004560 tx=0x7fc9dc009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:02.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.068+0000 7fc9e57fa700 1 -- 172.21.15.32:0/4191454992 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fc9d40bfc50 con 0x7fc9e8103680 2024-05-12T04:12:02.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.320+0000 7fc9eecd8700 1 -- 172.21.15.32:0/4191454992 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc9e80640c0 con 0x7fc9e8103680 2024-05-12T04:12:02.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.320+0000 7fc9e57fa700 1 -- 172.21.15.32:0/4191454992 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+4831 (secure 0 0 0) 0x7fc9d4059940 con 0x7fc9e8103680 2024-05-12T04:12:02.321 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:12:02.321 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44343,"name":"cephfs.smithi033.jgbjhk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6825/997254871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":997254871},{"type":"v1","addr":"172.21.15.33:6825","nonce":997254871}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":44373,"name":"cephfs.smithi033.hglwml","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6827/1983593176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1983593176},{"type":"v1","addr":"172.21.15.33:6827","nonce":1983593176}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":37,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-12T03:57:48.904614+0000","modified":"2024-05-12T04:09:05.133553+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":34404},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34404":{"gid":34404,"name":"cephfs.smithi032.ialtqb","rank":0,"incarnation":32,"state":"up:active","state_seq":20,"addr":"172.21.15.32:6827/1080358503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":1080358503},{"type":"v1","addr":"172.21.15.32:6827","nonce":1080358503}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_34436":{"gid":34436,"name":"cephfs.smithi032.qpsawh","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.32:6829/3603452506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":3603452506},{"type":"v1","addr":"172.21.15.32:6829","nonce":3603452506}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-05-12T04:12:02.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.323+0000 7fc9eecd8700 1 -- 172.21.15.32:0/4191454992 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fc9d806cc20 msgr2=0x7fc9d806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:02.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.323+0000 7fc9eecd8700 1 --2- 172.21.15.32:0/4191454992 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fc9d806cc20 0x7fc9d806f0d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc004560 tx=0x7fc9dc009e30 comp rx=0 tx=0).stop 2024-05-12T04:12:02.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.323+0000 7fc9eecd8700 1 -- 172.21.15.32:0/4191454992 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9e8103680 msgr2=0x7fc9e8113cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:02.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.323+0000 7fc9eecd8700 1 --2- 172.21.15.32:0/4191454992 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9e8103680 0x7fc9e8113cf0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc9d400b790 tx=0x7fc9d4009220 comp rx=0 tx=0).stop 2024-05-12T04:12:02.323 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.323+0000 7fc9eecd8700 1 -- 172.21.15.32:0/4191454992 shutdown_connections 2024-05-12T04:12:02.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.323+0000 7fc9eecd8700 1 --2- 172.21.15.32:0/4191454992 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fc9d806cc20 0x7fc9d806f0d0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:02.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.323+0000 7fc9eecd8700 1 --2- 172.21.15.32:0/4191454992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9e8102ce0 0x7fc9e81137b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:02.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.323+0000 7fc9eecd8700 1 --2- 172.21.15.32:0/4191454992 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9e8103680 0x7fc9e8113cf0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:02.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.323+0000 7fc9eecd8700 1 -- 172.21.15.32:0/4191454992 >> 172.21.15.32:0/4191454992 conn(0x7fc9e80f8810 msgr2=0x7fc9e80f97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:02.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.323+0000 7fc9eecd8700 1 -- 172.21.15.32:0/4191454992 shutdown_connections 2024-05-12T04:12:02.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.323+0000 7fc9eecd8700 1 -- 172.21.15.32:0/4191454992 wait complete. 2024-05-12T04:12:02.324 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 37 2024-05-12T04:12:02.335 DEBUG:tasks.fs:checking fs fscid=1,name=cephfs state = {'epoch': 18, 'max_mds': 1, 'flags': 50} 2024-05-12T04:12:02.336 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 19 2024-05-12T04:12:02.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.458+0000 7f873df78700 1 -- 172.21.15.32:0/3989472832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87380fbed0 msgr2=0x7f87381018e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:02.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.458+0000 7f873df78700 1 --2- 172.21.15.32:0/3989472832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87380fbed0 0x7f87381018e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f872c004940 tx=0x7f872c009e30 comp rx=0 tx=0).stop 2024-05-12T04:12:02.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.458+0000 7f873df78700 1 -- 172.21.15.32:0/3989472832 shutdown_connections 2024-05-12T04:12:02.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.458+0000 7f873df78700 1 --2- 172.21.15.32:0/3989472832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87380fbed0 0x7f87381018e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:02.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.458+0000 7f873df78700 1 --2- 172.21.15.32:0/3989472832 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87380fb530 0x7f87380fb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:02.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.458+0000 7f873df78700 1 -- 172.21.15.32:0/3989472832 >> 172.21.15.32:0/3989472832 conn(0x7f8738072240 msgr2=0x7f8738072640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:02.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.458+0000 7f873df78700 1 -- 172.21.15.32:0/3989472832 shutdown_connections 2024-05-12T04:12:02.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.458+0000 7f873df78700 1 -- 172.21.15.32:0/3989472832 wait complete. 2024-05-12T04:12:02.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.459+0000 7f873df78700 1 Processor -- start 2024-05-12T04:12:02.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.459+0000 7f873df78700 1 -- start start 2024-05-12T04:12:02.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.459+0000 7f873df78700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87380fb530 0x7f8738113880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:02.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.459+0000 7f873df78700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87380fbed0 0x7f8738113dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:02.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.459+0000 7f873df78700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8738114410 con 0x7f87380fbed0 2024-05-12T04:12:02.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.459+0000 7f873df78700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873810da10 con 0x7f87380fb530 2024-05-12T04:12:02.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.459+0000 7f8736ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87380fbed0 0x7f8738113dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:02.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.459+0000 7f8736ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87380fbed0 0x7f8738113dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39162/0 (socket says 172.21.15.32:39162) 2024-05-12T04:12:02.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.459+0000 7f8736ffd700 1 -- 172.21.15.32:0/1265448510 learned_addr learned my addr 172.21.15.32:0/1265448510 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:12:02.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.460+0000 7f87377fe700 1 --2- 172.21.15.32:0/1265448510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87380fb530 0x7f8738113880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:02.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.460+0000 7f8736ffd700 1 -- 172.21.15.32:0/1265448510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87380fb530 msgr2=0x7f8738113880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:02.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.460+0000 7f8736ffd700 1 --2- 172.21.15.32:0/1265448510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87380fb530 0x7f8738113880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:02.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.460+0000 7f8736ffd700 1 -- 172.21.15.32:0/1265448510 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f872c005040 con 0x7f87380fbed0 2024-05-12T04:12:02.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.460+0000 7f8736ffd700 1 --2- 172.21.15.32:0/1265448510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87380fbed0 0x7f8738113dc0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f872c000c00 tx=0x7f872c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:02.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.460+0000 7f871ffff700 1 -- 172.21.15.32:0/1265448510 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f872c00e040 con 0x7f87380fbed0 2024-05-12T04:12:02.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.460+0000 7f871ffff700 1 -- 172.21.15.32:0/1265448510 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f872c013420 con 0x7f87380fbed0 2024-05-12T04:12:02.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.460+0000 7f873df78700 1 -- 172.21.15.32:0/1265448510 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f873810dc90 con 0x7f87380fbed0 2024-05-12T04:12:02.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.460+0000 7f871ffff700 1 -- 172.21.15.32:0/1265448510 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f872c01c630 con 0x7f87380fbed0 2024-05-12T04:12:02.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.460+0000 7f873df78700 1 -- 172.21.15.32:0/1265448510 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f873810e1d0 con 0x7f87380fbed0 2024-05-12T04:12:02.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.461+0000 7f873df78700 1 -- 172.21.15.32:0/1265448510 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87380fe600 con 0x7f87380fbed0 2024-05-12T04:12:02.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.462+0000 7f871ffff700 1 -- 172.21.15.32:0/1265448510 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f872c013590 con 0x7f87380fbed0 2024-05-12T04:12:02.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.462+0000 7f871ffff700 1 --2- 172.21.15.32:0/1265448510 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f872006cc20 0x7f872006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:02.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.462+0000 7f871ffff700 1 -- 172.21.15.32:0/1265448510 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f872c08d1a0 con 0x7f87380fbed0 2024-05-12T04:12:02.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.462+0000 7f87377fe700 1 --2- 172.21.15.32:0/1265448510 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f872006cc20 0x7f872006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:02.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.463+0000 7f87377fe700 1 --2- 172.21.15.32:0/1265448510 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f872006cc20 0x7f872006f0d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8728004f40 tx=0x7f872800b040 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:02.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.467+0000 7f871ffff700 1 -- 172.21.15.32:0/1265448510 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f872c05ac10 con 0x7f87380fbed0 2024-05-12T04:12:02.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.715+0000 7f873df78700 1 -- 172.21.15.32:0/1265448510 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 19, "format": "json"} v 0) v1 -- 0x7f8738109c70 con 0x7f87380fbed0 2024-05-12T04:12:02.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.716+0000 7f871ffff700 1 -- 172.21.15.32:0/1265448510 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 19, "format": "json"}]=0 dumped fsmap epoch 19 v37) v1 ==== 107+0+4044 (secure 0 0 0) 0x7f872c05a7a0 con 0x7f87380fbed0 2024-05-12T04:12:02.716 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:12:02.717 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24283,"name":"cephfs.smithi033.hglwml","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6827/433858238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":433858238},{"type":"v1","addr":"172.21.15.33:6827","nonce":433858238}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":24287,"name":"cephfs.smithi032.ialtqb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6827/3703132329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3703132329},{"type":"v1","addr":"172.21.15.32:6827","nonce":3703132329}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-12T03:57:48.904614+0000","modified":"2024-05-12T04:07:46.260597+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14474},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14474":{"gid":14474,"name":"cephfs.smithi032.qpsawh","rank":0,"incarnation":13,"state":"up:active","state_seq":5,"addr":"172.21.15.32:6829/3100155296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":3100155296},{"type":"v1","addr":"172.21.15.32:6829","nonce":3100155296}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-05-12T04:12:02.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.718+0000 7f873df78700 1 -- 172.21.15.32:0/1265448510 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f872006cc20 msgr2=0x7f872006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:02.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.718+0000 7f873df78700 1 --2- 172.21.15.32:0/1265448510 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f872006cc20 0x7f872006f0d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8728004f40 tx=0x7f872800b040 comp rx=0 tx=0).stop 2024-05-12T04:12:02.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.718+0000 7f873df78700 1 -- 172.21.15.32:0/1265448510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87380fbed0 msgr2=0x7f8738113dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:02.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.718+0000 7f873df78700 1 --2- 172.21.15.32:0/1265448510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87380fbed0 0x7f8738113dc0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f872c000c00 tx=0x7f872c009e30 comp rx=0 tx=0).stop 2024-05-12T04:12:02.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.718+0000 7f873df78700 1 -- 172.21.15.32:0/1265448510 shutdown_connections 2024-05-12T04:12:02.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.718+0000 7f873df78700 1 --2- 172.21.15.32:0/1265448510 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f872006cc20 0x7f872006f0d0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:02.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.718+0000 7f873df78700 1 --2- 172.21.15.32:0/1265448510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87380fb530 0x7f8738113880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:02.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.718+0000 7f873df78700 1 --2- 172.21.15.32:0/1265448510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87380fbed0 0x7f8738113dc0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:02.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.718+0000 7f873df78700 1 -- 172.21.15.32:0/1265448510 >> 172.21.15.32:0/1265448510 conn(0x7f8738072240 msgr2=0x7f87380f8ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:02.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.718+0000 7f873df78700 1 -- 172.21.15.32:0/1265448510 shutdown_connections 2024-05-12T04:12:02.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.718+0000 7f873df78700 1 -- 172.21.15.32:0/1265448510 wait complete. 2024-05-12T04:12:02.720 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 19 2024-05-12T04:12:02.729 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:02 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/2800887053' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-05-12T04:12:02.730 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:02 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/4077575263' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-05-12T04:12:02.730 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:02 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/4191454992' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-12T04:12:02.731 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 19 2024-05-12T04:12:02.731 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 20 2024-05-12T04:12:02.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.853+0000 7fb8adfb3700 1 -- 172.21.15.32:0/3784965916 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8a80fdce0 msgr2=0x7fb8a80fe0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:02.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.853+0000 7fb8adfb3700 1 --2- 172.21.15.32:0/3784965916 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8a80fdce0 0x7fb8a80fe0b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb898004940 tx=0x7fb898009e30 comp rx=0 tx=0).stop 2024-05-12T04:12:02.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.853+0000 7fb8adfb3700 1 -- 172.21.15.32:0/3784965916 shutdown_connections 2024-05-12T04:12:02.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.853+0000 7fb8adfb3700 1 --2- 172.21.15.32:0/3784965916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8a80fe680 0x7fb8a8106cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:02.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.853+0000 7fb8adfb3700 1 --2- 172.21.15.32:0/3784965916 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8a80fdce0 0x7fb8a80fe0b0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:02.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.853+0000 7fb8adfb3700 1 -- 172.21.15.32:0/3784965916 >> 172.21.15.32:0/3784965916 conn(0x7fb8a80f76f0 msgr2=0x7fb8a80f9b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:02.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.853+0000 7fb8adfb3700 1 -- 172.21.15.32:0/3784965916 shutdown_connections 2024-05-12T04:12:02.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.853+0000 7fb8adfb3700 1 -- 172.21.15.32:0/3784965916 wait complete. 2024-05-12T04:12:02.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.854+0000 7fb8adfb3700 1 Processor -- start 2024-05-12T04:12:02.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.854+0000 7fb8adfb3700 1 -- start start 2024-05-12T04:12:02.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.854+0000 7fb8adfb3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8a80fdce0 0x7fb8a8113790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:02.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.854+0000 7fb8adfb3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8a80fe680 0x7fb8a8113cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:02.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.854+0000 7fb8adfb3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a8114320 con 0x7fb8a80fe680 2024-05-12T04:12:02.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.854+0000 7fb8adfb3700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a810d920 con 0x7fb8a80fdce0 2024-05-12T04:12:02.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.854+0000 7fb8a6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8a80fe680 0x7fb8a8113cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:02.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.855+0000 7fb8a6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8a80fe680 0x7fb8a8113cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39182/0 (socket says 172.21.15.32:39182) 2024-05-12T04:12:02.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.855+0000 7fb8a6ffd700 1 -- 172.21.15.32:0/2934995792 learned_addr learned my addr 172.21.15.32:0/2934995792 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:12:02.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.855+0000 7fb8a77fe700 1 --2- 172.21.15.32:0/2934995792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8a80fdce0 0x7fb8a8113790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:02.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.855+0000 7fb8a6ffd700 1 -- 172.21.15.32:0/2934995792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8a80fdce0 msgr2=0x7fb8a8113790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:02.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.855+0000 7fb8a6ffd700 1 --2- 172.21.15.32:0/2934995792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8a80fdce0 0x7fb8a8113790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:02.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.855+0000 7fb8a6ffd700 1 -- 172.21.15.32:0/2934995792 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb898005040 con 0x7fb8a80fe680 2024-05-12T04:12:02.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.855+0000 7fb8a6ffd700 1 --2- 172.21.15.32:0/2934995792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8a80fe680 0x7fb8a8113cd0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb89c002a50 tx=0x7fb89c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:02.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.855+0000 7fb88ffff700 1 -- 172.21.15.32:0/2934995792 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb89c007cc0 con 0x7fb8a80fe680 2024-05-12T04:12:02.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.855+0000 7fb8adfb3700 1 -- 172.21.15.32:0/2934995792 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8a810dc00 con 0x7fb8a80fe680 2024-05-12T04:12:02.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.855+0000 7fb88ffff700 1 -- 172.21.15.32:0/2934995792 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb89c00f040 con 0x7fb8a80fe680 2024-05-12T04:12:02.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.855+0000 7fb88ffff700 1 -- 172.21.15.32:0/2934995792 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb89c00d9c0 con 0x7fb8a80fe680 2024-05-12T04:12:02.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.855+0000 7fb8adfb3700 1 -- 172.21.15.32:0/2934995792 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8a810e140 con 0x7fb8a80fe680 2024-05-12T04:12:02.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.856+0000 7fb8adfb3700 1 -- 172.21.15.32:0/2934995792 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8a80664b0 con 0x7fb8a80fe680 2024-05-12T04:12:02.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.857+0000 7fb88ffff700 1 -- 172.21.15.32:0/2934995792 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fb89c00db60 con 0x7fb8a80fe680 2024-05-12T04:12:02.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.857+0000 7fb88ffff700 1 --2- 172.21.15.32:0/2934995792 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fb89006cc20 0x7fb89006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:02.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.857+0000 7fb88ffff700 1 -- 172.21.15.32:0/2934995792 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fb89c08b750 con 0x7fb8a80fe680 2024-05-12T04:12:02.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.857+0000 7fb8a77fe700 1 --2- 172.21.15.32:0/2934995792 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fb89006cc20 0x7fb89006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:02.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.858+0000 7fb8a77fe700 1 --2- 172.21.15.32:0/2934995792 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fb89006cc20 0x7fb89006f0d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb898009fd0 tx=0x7fb898009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:02.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:02.862+0000 7fb88ffff700 1 -- 172.21.15.32:0/2934995792 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fb89c059140 con 0x7fb8a80fe680 2024-05-12T04:12:02.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:02 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/2800887053' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-05-12T04:12:02.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:02 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/4077575263' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-05-12T04:12:02.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:02 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/4191454992' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-12T04:12:03.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.112+0000 7fb8adfb3700 1 -- 172.21.15.32:0/2934995792 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 20, "format": "json"} v 0) v1 -- 0x7fb8a80640c0 con 0x7fb8a80fe680 2024-05-12T04:12:03.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.113+0000 7fb88ffff700 1 -- 172.21.15.32:0/2934995792 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 20, "format": "json"}]=0 dumped fsmap epoch 20 v37) v1 ==== 107+0+4823 (secure 0 0 0) 0x7fb89c058cd0 con 0x7fb8a80fe680 2024-05-12T04:12:03.113 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:12:03.113 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24283,"name":"cephfs.smithi033.hglwml","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6827/433858238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":433858238},{"type":"v1","addr":"172.21.15.33:6827","nonce":433858238}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":24287,"name":"cephfs.smithi032.ialtqb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6827/3703132329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":3703132329},{"type":"v1","addr":"172.21.15.32:6827","nonce":3703132329}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":44303,"name":"cephfs.smithi033.jgbjhk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6825/2944716783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":2944716783},{"type":"v1","addr":"172.21.15.33:6825","nonce":2944716783}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-12T03:57:48.904614+0000","modified":"2024-05-12T04:07:46.260597+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14474},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14474":{"gid":14474,"name":"cephfs.smithi032.qpsawh","rank":0,"incarnation":13,"state":"up:active","state_seq":5,"addr":"172.21.15.32:6829/3100155296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":3100155296},{"type":"v1","addr":"172.21.15.32:6829","nonce":3100155296}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-05-12T04:12:03.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.115+0000 7fb8adfb3700 1 -- 172.21.15.32:0/2934995792 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fb89006cc20 msgr2=0x7fb89006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:03.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.115+0000 7fb8adfb3700 1 --2- 172.21.15.32:0/2934995792 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fb89006cc20 0x7fb89006f0d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb898009fd0 tx=0x7fb898009e30 comp rx=0 tx=0).stop 2024-05-12T04:12:03.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.115+0000 7fb8adfb3700 1 -- 172.21.15.32:0/2934995792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8a80fe680 msgr2=0x7fb8a8113cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:03.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.115+0000 7fb8adfb3700 1 --2- 172.21.15.32:0/2934995792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8a80fe680 0x7fb8a8113cd0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb89c002a50 tx=0x7fb89c0046b0 comp rx=0 tx=0).stop 2024-05-12T04:12:03.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.115+0000 7fb8adfb3700 1 -- 172.21.15.32:0/2934995792 shutdown_connections 2024-05-12T04:12:03.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.115+0000 7fb8adfb3700 1 --2- 172.21.15.32:0/2934995792 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fb89006cc20 0x7fb89006f0d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:03.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.115+0000 7fb8adfb3700 1 --2- 172.21.15.32:0/2934995792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8a80fdce0 0x7fb8a8113790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:03.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.115+0000 7fb8adfb3700 1 --2- 172.21.15.32:0/2934995792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8a80fe680 0x7fb8a8113cd0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:03.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.115+0000 7fb8adfb3700 1 -- 172.21.15.32:0/2934995792 >> 172.21.15.32:0/2934995792 conn(0x7fb8a80f76f0 msgr2=0x7fb8a80f8d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:03.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.115+0000 7fb8adfb3700 1 -- 172.21.15.32:0/2934995792 shutdown_connections 2024-05-12T04:12:03.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.115+0000 7fb8adfb3700 1 -- 172.21.15.32:0/2934995792 wait complete. 2024-05-12T04:12:03.117 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 20 2024-05-12T04:12:03.128 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 20 2024-05-12T04:12:03.128 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 21 2024-05-12T04:12:03.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.250+0000 7f476bb98700 1 -- 172.21.15.32:0/90162607 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4764102580 msgr2=0x7f47640fbbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:03.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.250+0000 7f476bb98700 1 --2- 172.21.15.32:0/90162607 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4764102580 0x7f47640fbbf0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4754004940 tx=0x7f4754009e30 comp rx=0 tx=0).stop 2024-05-12T04:12:03.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.251+0000 7f476bb98700 1 -- 172.21.15.32:0/90162607 shutdown_connections 2024-05-12T04:12:03.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.251+0000 7f476bb98700 1 --2- 172.21.15.32:0/90162607 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4764102580 0x7f47640fbbf0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:03.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.251+0000 7f476bb98700 1 --2- 172.21.15.32:0/90162607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4764101be0 0x7f4764101fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:03.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.251+0000 7f476bb98700 1 -- 172.21.15.32:0/90162607 >> 172.21.15.32:0/90162607 conn(0x7f47640f7730 msgr2=0x7f47640f9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:03.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.251+0000 7f476bb98700 1 -- 172.21.15.32:0/90162607 shutdown_connections 2024-05-12T04:12:03.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.251+0000 7f476bb98700 1 -- 172.21.15.32:0/90162607 wait complete. 2024-05-12T04:12:03.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.251+0000 7f476bb98700 1 Processor -- start 2024-05-12T04:12:03.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.251+0000 7f476bb98700 1 -- start start 2024-05-12T04:12:03.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.252+0000 7f476bb98700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4764101be0 0x7f4764112660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:03.254 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.252+0000 7f476bb98700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4764102580 0x7f4764112be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:03.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.252+0000 7f476bb98700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4764113230 con 0x7f4764101be0 2024-05-12T04:12:03.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.252+0000 7f476bb98700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476410c7f0 con 0x7f4764102580 2024-05-12T04:12:03.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.252+0000 7f4769934700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4764101be0 0x7f4764112660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:03.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.252+0000 7f4769934700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4764101be0 0x7f4764112660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39208/0 (socket says 172.21.15.32:39208) 2024-05-12T04:12:03.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.252+0000 7f4769934700 1 -- 172.21.15.32:0/264774058 learned_addr learned my addr 172.21.15.32:0/264774058 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:12:03.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.252+0000 7f4769133700 1 --2- 172.21.15.32:0/264774058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4764102580 0x7f4764112be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:03.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.252+0000 7f4769934700 1 -- 172.21.15.32:0/264774058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4764102580 msgr2=0x7f4764112be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:03.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.252+0000 7f4769934700 1 --2- 172.21.15.32:0/264774058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4764102580 0x7f4764112be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:03.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.252+0000 7f4769934700 1 -- 172.21.15.32:0/264774058 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4754005040 con 0x7f4764101be0 2024-05-12T04:12:03.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.252+0000 7f4769133700 1 --2- 172.21.15.32:0/264774058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4764102580 0x7f4764112be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-12T04:12:03.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.252+0000 7f4769934700 1 --2- 172.21.15.32:0/264774058 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4764101be0 0x7f4764112660 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4760002a50 tx=0x7f47600046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:03.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.252+0000 7f475a7fc700 1 -- 172.21.15.32:0/264774058 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4760007cc0 con 0x7f4764101be0 2024-05-12T04:12:03.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.252+0000 7f475a7fc700 1 -- 172.21.15.32:0/264774058 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f476000f040 con 0x7f4764101be0 2024-05-12T04:12:03.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.252+0000 7f476bb98700 1 -- 172.21.15.32:0/264774058 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f476410ca80 con 0x7f4764101be0 2024-05-12T04:12:03.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.252+0000 7f475a7fc700 1 -- 172.21.15.32:0/264774058 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4760009540 con 0x7f4764101be0 2024-05-12T04:12:03.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.252+0000 7f476bb98700 1 -- 172.21.15.32:0/264774058 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f476410cfc0 con 0x7f4764101be0 2024-05-12T04:12:03.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.253+0000 7f476bb98700 1 -- 172.21.15.32:0/264774058 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47640fd330 con 0x7f4764101be0 2024-05-12T04:12:03.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.254+0000 7f475a7fc700 1 -- 172.21.15.32:0/264774058 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f47600096a0 con 0x7f4764101be0 2024-05-12T04:12:03.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.254+0000 7f475a7fc700 1 --2- 172.21.15.32:0/264774058 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f475006cc70 0x7f475006f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:03.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.254+0000 7f475a7fc700 1 -- 172.21.15.32:0/264774058 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f476008bf70 con 0x7f4764101be0 2024-05-12T04:12:03.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.254+0000 7f4769133700 1 --2- 172.21.15.32:0/264774058 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f475006cc70 0x7f475006f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:03.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.255+0000 7f4769133700 1 --2- 172.21.15.32:0/264774058 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f475006cc70 0x7f475006f120 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f475400b3b0 tx=0x7f4754009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:03.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.259+0000 7f475a7fc700 1 -- 172.21.15.32:0/264774058 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f47600598c0 con 0x7f4764101be0 2024-05-12T04:12:03.511 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.511+0000 7f476bb98700 1 -- 172.21.15.32:0/264774058 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 21, "format": "json"} v 0) v1 -- 0x7f476410d1e0 con 0x7f4764101be0 2024-05-12T04:12:03.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.512+0000 7f475a7fc700 1 -- 172.21.15.32:0/264774058 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 21, "format": "json"}]=0 dumped fsmap epoch 21 v37) v1 ==== 107+0+4044 (secure 0 0 0) 0x7f4760059450 con 0x7f4764101be0 2024-05-12T04:12:03.512 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:12:03.512 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24283,"name":"cephfs.smithi033.hglwml","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6827/433858238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":433858238},{"type":"v1","addr":"172.21.15.33:6827","nonce":433858238}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":44303,"name":"cephfs.smithi033.jgbjhk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6825/2944716783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":2944716783},{"type":"v1","addr":"172.21.15.33:6825","nonce":2944716783}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-12T03:57:48.904614+0000","modified":"2024-05-12T04:07:46.260597+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14474},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14474":{"gid":14474,"name":"cephfs.smithi032.qpsawh","rank":0,"incarnation":13,"state":"up:active","state_seq":5,"addr":"172.21.15.32:6829/3100155296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":3100155296},{"type":"v1","addr":"172.21.15.32:6829","nonce":3100155296}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-05-12T04:12:03.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.514+0000 7f476bb98700 1 -- 172.21.15.32:0/264774058 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f475006cc70 msgr2=0x7f475006f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:03.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.514+0000 7f476bb98700 1 --2- 172.21.15.32:0/264774058 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f475006cc70 0x7f475006f120 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f475400b3b0 tx=0x7f4754009e30 comp rx=0 tx=0).stop 2024-05-12T04:12:03.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.514+0000 7f476bb98700 1 -- 172.21.15.32:0/264774058 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4764101be0 msgr2=0x7f4764112660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:03.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.514+0000 7f476bb98700 1 --2- 172.21.15.32:0/264774058 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4764101be0 0x7f4764112660 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4760002a50 tx=0x7f47600046b0 comp rx=0 tx=0).stop 2024-05-12T04:12:03.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.514+0000 7f476bb98700 1 -- 172.21.15.32:0/264774058 shutdown_connections 2024-05-12T04:12:03.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.514+0000 7f476bb98700 1 --2- 172.21.15.32:0/264774058 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f475006cc70 0x7f475006f120 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:03.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.514+0000 7f476bb98700 1 --2- 172.21.15.32:0/264774058 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4764101be0 0x7f4764112660 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:03.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.514+0000 7f476bb98700 1 --2- 172.21.15.32:0/264774058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4764102580 0x7f4764112be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:03.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.514+0000 7f476bb98700 1 -- 172.21.15.32:0/264774058 >> 172.21.15.32:0/264774058 conn(0x7f47640f7730 msgr2=0x7f47640f8780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:03.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.514+0000 7f476bb98700 1 -- 172.21.15.32:0/264774058 shutdown_connections 2024-05-12T04:12:03.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.514+0000 7f476bb98700 1 -- 172.21.15.32:0/264774058 wait complete. 2024-05-12T04:12:03.516 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 21 2024-05-12T04:12:03.527 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 21 2024-05-12T04:12:03.527 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 22 2024-05-12T04:12:03.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.650+0000 7fc959a0a700 1 -- 172.21.15.32:0/738960134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9540fdd20 msgr2=0x7fc9540fe0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:03.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.650+0000 7fc959a0a700 1 --2- 172.21.15.32:0/738960134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9540fdd20 0x7fc9540fe0f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc948004970 tx=0x7fc948009e30 comp rx=0 tx=0).stop 2024-05-12T04:12:03.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.650+0000 7fc959a0a700 1 -- 172.21.15.32:0/738960134 shutdown_connections 2024-05-12T04:12:03.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.650+0000 7fc959a0a700 1 --2- 172.21.15.32:0/738960134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9540fe6c0 0x7fc954106d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:03.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.650+0000 7fc959a0a700 1 --2- 172.21.15.32:0/738960134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9540fdd20 0x7fc9540fe0f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:03.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.650+0000 7fc959a0a700 1 -- 172.21.15.32:0/738960134 >> 172.21.15.32:0/738960134 conn(0x7fc9540f7730 msgr2=0x7fc9540f9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:03.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.650+0000 7fc959a0a700 1 -- 172.21.15.32:0/738960134 shutdown_connections 2024-05-12T04:12:03.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.650+0000 7fc959a0a700 1 -- 172.21.15.32:0/738960134 wait complete. 2024-05-12T04:12:03.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.651+0000 7fc959a0a700 1 Processor -- start 2024-05-12T04:12:03.651 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.651+0000 7fc959a0a700 1 -- start start 2024-05-12T04:12:03.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.651+0000 7fc959a0a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9540fdd20 0x7fc9540fbb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:03.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.651+0000 7fc959a0a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9540fe6c0 0x7fc9540fc090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:03.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.651+0000 7fc959a0a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9540fdaf0 con 0x7fc9540fe6c0 2024-05-12T04:12:03.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.651+0000 7fc959a0a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9540fc5d0 con 0x7fc9540fdd20 2024-05-12T04:12:03.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.651+0000 7fc9527fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9540fe6c0 0x7fc9540fc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:03.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.651+0000 7fc9527fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9540fe6c0 0x7fc9540fc090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39226/0 (socket says 172.21.15.32:39226) 2024-05-12T04:12:03.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.651+0000 7fc9527fc700 1 -- 172.21.15.32:0/826243501 learned_addr learned my addr 172.21.15.32:0/826243501 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:12:03.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.651+0000 7fc952ffd700 1 --2- 172.21.15.32:0/826243501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9540fdd20 0x7fc9540fbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:03.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.651+0000 7fc9527fc700 1 -- 172.21.15.32:0/826243501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9540fdd20 msgr2=0x7fc9540fbb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:03.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.651+0000 7fc9527fc700 1 --2- 172.21.15.32:0/826243501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9540fdd20 0x7fc9540fbb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:03.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.651+0000 7fc9527fc700 1 -- 172.21.15.32:0/826243501 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc948005040 con 0x7fc9540fe6c0 2024-05-12T04:12:03.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.651+0000 7fc9527fc700 1 --2- 172.21.15.32:0/826243501 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9540fe6c0 0x7fc9540fc090 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc94400c830 tx=0x7fc944004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:03.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.652+0000 7fc93b7fe700 1 -- 172.21.15.32:0/826243501 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc9440106f0 con 0x7fc9540fe6c0 2024-05-12T04:12:03.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.652+0000 7fc93b7fe700 1 -- 172.21.15.32:0/826243501 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc944010d00 con 0x7fc9540fe6c0 2024-05-12T04:12:03.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.652+0000 7fc959a0a700 1 -- 172.21.15.32:0/826243501 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9540fc880 con 0x7fc9540fe6c0 2024-05-12T04:12:03.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.652+0000 7fc93b7fe700 1 -- 172.21.15.32:0/826243501 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc944012e90 con 0x7fc9540fe6c0 2024-05-12T04:12:03.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.652+0000 7fc959a0a700 1 -- 172.21.15.32:0/826243501 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc954119ec0 con 0x7fc9540fe6c0 2024-05-12T04:12:03.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.653+0000 7fc959a0a700 1 -- 172.21.15.32:0/826243501 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc954066450 con 0x7fc9540fe6c0 2024-05-12T04:12:03.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.653+0000 7fc93b7fe700 1 -- 172.21.15.32:0/826243501 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fc944011480 con 0x7fc9540fe6c0 2024-05-12T04:12:03.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.653+0000 7fc93b7fe700 1 --2- 172.21.15.32:0/826243501 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fc93c06cc20 0x7fc93c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:03.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.654+0000 7fc952ffd700 1 --2- 172.21.15.32:0/826243501 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fc93c06cc20 0x7fc93c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:03.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.654+0000 7fc93b7fe700 1 -- 172.21.15.32:0/826243501 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fc94408c0e0 con 0x7fc9540fe6c0 2024-05-12T04:12:03.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.654+0000 7fc952ffd700 1 --2- 172.21.15.32:0/826243501 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fc93c06cc20 0x7fc93c06f0d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc948004f50 tx=0x7fc948009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:03.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.658+0000 7fc93b7fe700 1 -- 172.21.15.32:0/826243501 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fc944056130 con 0x7fc9540fe6c0 2024-05-12T04:12:03.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:03 smithi032 ceph-mon[120392]: pgmap v301: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:12:03.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:03 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/1265448510' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-05-12T04:12:03.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:03 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/2934995792' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-05-12T04:12:03.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:03 smithi033 ceph-mon[107305]: pgmap v301: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:12:03.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:03 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/1265448510' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-05-12T04:12:03.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:03 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/2934995792' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-05-12T04:12:03.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.918+0000 7fc959a0a700 1 -- 172.21.15.32:0/826243501 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 22, "format": "json"} v 0) v1 -- 0x7fc9540640c0 con 0x7fc9540fe6c0 2024-05-12T04:12:03.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.919+0000 7fc93b7fe700 1 -- 172.21.15.32:0/826243501 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 22, "format": "json"}]=0 dumped fsmap epoch 22 v37) v1 ==== 107+0+4823 (secure 0 0 0) 0x7fc944059750 con 0x7fc9540fe6c0 2024-05-12T04:12:03.919 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:12:03.919 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24283,"name":"cephfs.smithi033.hglwml","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6827/433858238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":433858238},{"type":"v1","addr":"172.21.15.33:6827","nonce":433858238}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34404,"name":"cephfs.smithi032.ialtqb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6827/1080358503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":1080358503},{"type":"v1","addr":"172.21.15.32:6827","nonce":1080358503}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":44303,"name":"cephfs.smithi033.jgbjhk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6825/2944716783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":2944716783},{"type":"v1","addr":"172.21.15.33:6825","nonce":2944716783}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-12T03:57:48.904614+0000","modified":"2024-05-12T04:07:46.260597+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14474},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14474":{"gid":14474,"name":"cephfs.smithi032.qpsawh","rank":0,"incarnation":13,"state":"up:active","state_seq":5,"addr":"172.21.15.32:6829/3100155296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":3100155296},{"type":"v1","addr":"172.21.15.32:6829","nonce":3100155296}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-05-12T04:12:03.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.921+0000 7fc959a0a700 1 -- 172.21.15.32:0/826243501 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fc93c06cc20 msgr2=0x7fc93c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:03.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.921+0000 7fc959a0a700 1 --2- 172.21.15.32:0/826243501 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fc93c06cc20 0x7fc93c06f0d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc948004f50 tx=0x7fc948009e30 comp rx=0 tx=0).stop 2024-05-12T04:12:03.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.921+0000 7fc959a0a700 1 -- 172.21.15.32:0/826243501 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9540fe6c0 msgr2=0x7fc9540fc090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:03.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.921+0000 7fc959a0a700 1 --2- 172.21.15.32:0/826243501 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9540fe6c0 0x7fc9540fc090 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc94400c830 tx=0x7fc944004470 comp rx=0 tx=0).stop 2024-05-12T04:12:03.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.921+0000 7fc959a0a700 1 -- 172.21.15.32:0/826243501 shutdown_connections 2024-05-12T04:12:03.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.921+0000 7fc959a0a700 1 --2- 172.21.15.32:0/826243501 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fc93c06cc20 0x7fc93c06f0d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:03.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.921+0000 7fc959a0a700 1 --2- 172.21.15.32:0/826243501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9540fdd20 0x7fc9540fbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:03.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.921+0000 7fc959a0a700 1 --2- 172.21.15.32:0/826243501 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9540fe6c0 0x7fc9540fc090 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:03.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.921+0000 7fc959a0a700 1 -- 172.21.15.32:0/826243501 >> 172.21.15.32:0/826243501 conn(0x7fc9540f7730 msgr2=0x7fc9540f8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:03.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.922+0000 7fc959a0a700 1 -- 172.21.15.32:0/826243501 shutdown_connections 2024-05-12T04:12:03.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:03.922+0000 7fc959a0a700 1 -- 172.21.15.32:0/826243501 wait complete. 2024-05-12T04:12:03.923 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 22 2024-05-12T04:12:03.934 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 22 2024-05-12T04:12:03.934 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 23 2024-05-12T04:12:04.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.057+0000 7f079a10c700 1 -- 172.21.15.32:0/4283891173 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0794101c30 msgr2=0x7f0794102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:04.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.057+0000 7f079a10c700 1 --2- 172.21.15.32:0/4283891173 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0794101c30 0x7f0794102000 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0784004940 tx=0x7f0784009e30 comp rx=0 tx=0).stop 2024-05-12T04:12:04.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.057+0000 7f079a10c700 1 -- 172.21.15.32:0/4283891173 shutdown_connections 2024-05-12T04:12:04.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.057+0000 7f079a10c700 1 --2- 172.21.15.32:0/4283891173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07941025d0 0x7f07940fbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:04.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.057+0000 7f079a10c700 1 --2- 172.21.15.32:0/4283891173 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0794101c30 0x7f0794102000 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:04.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.057+0000 7f079a10c700 1 -- 172.21.15.32:0/4283891173 >> 172.21.15.32:0/4283891173 conn(0x7f0794072b00 msgr2=0x7f0794072f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:04.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.057+0000 7f079a10c700 1 -- 172.21.15.32:0/4283891173 shutdown_connections 2024-05-12T04:12:04.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.057+0000 7f079a10c700 1 -- 172.21.15.32:0/4283891173 wait complete. 2024-05-12T04:12:04.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.058+0000 7f079a10c700 1 Processor -- start 2024-05-12T04:12:04.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.058+0000 7f079a10c700 1 -- start start 2024-05-12T04:12:04.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.058+0000 7f079a10c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0794101c30 0x7f0794113770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:04.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.058+0000 7f079a10c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07941025d0 0x7f0794113cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:04.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.058+0000 7f079a10c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0794114300 con 0x7f07941025d0 2024-05-12T04:12:04.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.058+0000 7f079a10c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f079410d950 con 0x7f0794101c30 2024-05-12T04:12:04.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.058+0000 7f0792ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07941025d0 0x7f0794113cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:04.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.059+0000 7f0792ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07941025d0 0x7f0794113cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39242/0 (socket says 172.21.15.32:39242) 2024-05-12T04:12:04.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.059+0000 7f0792ffd700 1 -- 172.21.15.32:0/3771642411 learned_addr learned my addr 172.21.15.32:0/3771642411 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:12:04.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.059+0000 7f07937fe700 1 --2- 172.21.15.32:0/3771642411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0794101c30 0x7f0794113770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:04.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.059+0000 7f0792ffd700 1 -- 172.21.15.32:0/3771642411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0794101c30 msgr2=0x7f0794113770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:04.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.059+0000 7f0792ffd700 1 --2- 172.21.15.32:0/3771642411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0794101c30 0x7f0794113770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:04.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.059+0000 7f0792ffd700 1 -- 172.21.15.32:0/3771642411 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0784005040 con 0x7f07941025d0 2024-05-12T04:12:04.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.059+0000 7f0792ffd700 1 --2- 172.21.15.32:0/3771642411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07941025d0 0x7f0794113cb0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0788002a50 tx=0x7f0788004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:04.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.059+0000 7f077bfff700 1 -- 172.21.15.32:0/3771642411 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0788007cc0 con 0x7f07941025d0 2024-05-12T04:12:04.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.059+0000 7f079a10c700 1 -- 172.21.15.32:0/3771642411 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f079410dc30 con 0x7f07941025d0 2024-05-12T04:12:04.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.059+0000 7f077bfff700 1 -- 172.21.15.32:0/3771642411 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f078800f040 con 0x7f07941025d0 2024-05-12T04:12:04.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.059+0000 7f077bfff700 1 -- 172.21.15.32:0/3771642411 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f078800d920 con 0x7f07941025d0 2024-05-12T04:12:04.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.059+0000 7f079a10c700 1 -- 172.21.15.32:0/3771642411 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f079410e1f0 con 0x7f07941025d0 2024-05-12T04:12:04.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.060+0000 7f079a10c700 1 -- 172.21.15.32:0/3771642411 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07940fd380 con 0x7f07941025d0 2024-05-12T04:12:04.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.061+0000 7f077bfff700 1 -- 172.21.15.32:0/3771642411 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f0788009070 con 0x7f07941025d0 2024-05-12T04:12:04.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.061+0000 7f077bfff700 1 --2- 172.21.15.32:0/3771642411 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f077c06cc20 0x7f077c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:04.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.061+0000 7f077bfff700 1 -- 172.21.15.32:0/3771642411 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f078808be70 con 0x7f07941025d0 2024-05-12T04:12:04.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.061+0000 7f07937fe700 1 --2- 172.21.15.32:0/3771642411 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f077c06cc20 0x7f077c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:04.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.062+0000 7f07937fe700 1 --2- 172.21.15.32:0/3771642411 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f077c06cc20 0x7f077c06f0d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0784004f90 tx=0x7f0784009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:04.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.066+0000 7f077bfff700 1 -- 172.21.15.32:0/3771642411 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f0788059860 con 0x7f07941025d0 2024-05-12T04:12:04.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.321+0000 7f079a10c700 1 -- 172.21.15.32:0/3771642411 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 23, "format": "json"} v 0) v1 -- 0x7f0794108be0 con 0x7f07941025d0 2024-05-12T04:12:04.322 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.322+0000 7f077bfff700 1 -- 172.21.15.32:0/3771642411 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 23, "format": "json"}]=0 dumped fsmap epoch 23 v37) v1 ==== 107+0+4034 (secure 0 0 0) 0x7f07880593f0 con 0x7f07941025d0 2024-05-12T04:12:04.322 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:12:04.322 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24283,"name":"cephfs.smithi033.hglwml","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6827/433858238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":433858238},{"type":"v1","addr":"172.21.15.33:6827","nonce":433858238}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34404,"name":"cephfs.smithi032.ialtqb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6827/1080358503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":1080358503},{"type":"v1","addr":"172.21.15.32:6827","nonce":1080358503}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":44303,"name":"cephfs.smithi033.jgbjhk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6825/2944716783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":2944716783},{"type":"v1","addr":"172.21.15.33:6825","nonce":2944716783}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-12T03:57:48.904614+0000","modified":"2024-05-12T04:08:08.972297+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-05-12T04:12:04.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.324+0000 7f079a10c700 1 -- 172.21.15.32:0/3771642411 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f077c06cc20 msgr2=0x7f077c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:04.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.324+0000 7f079a10c700 1 --2- 172.21.15.32:0/3771642411 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f077c06cc20 0x7f077c06f0d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0784004f90 tx=0x7f0784009e30 comp rx=0 tx=0).stop 2024-05-12T04:12:04.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.324+0000 7f079a10c700 1 -- 172.21.15.32:0/3771642411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07941025d0 msgr2=0x7f0794113cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:04.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.324+0000 7f079a10c700 1 --2- 172.21.15.32:0/3771642411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07941025d0 0x7f0794113cb0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0788002a50 tx=0x7f0788004600 comp rx=0 tx=0).stop 2024-05-12T04:12:04.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.324+0000 7f079a10c700 1 -- 172.21.15.32:0/3771642411 shutdown_connections 2024-05-12T04:12:04.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.324+0000 7f079a10c700 1 --2- 172.21.15.32:0/3771642411 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f077c06cc20 0x7f077c06f0d0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:04.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.324+0000 7f079a10c700 1 --2- 172.21.15.32:0/3771642411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0794101c30 0x7f0794113770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:04.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.324+0000 7f079a10c700 1 --2- 172.21.15.32:0/3771642411 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07941025d0 0x7f0794113cb0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:04.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.324+0000 7f079a10c700 1 -- 172.21.15.32:0/3771642411 >> 172.21.15.32:0/3771642411 conn(0x7f0794072b00 msgr2=0x7f0794065180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:04.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.324+0000 7f079a10c700 1 -- 172.21.15.32:0/3771642411 shutdown_connections 2024-05-12T04:12:04.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.324+0000 7f079a10c700 1 -- 172.21.15.32:0/3771642411 wait complete. 2024-05-12T04:12:04.325 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 23 2024-05-12T04:12:04.336 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 23 2024-05-12T04:12:04.337 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 24 2024-05-12T04:12:04.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.462+0000 7ff6c1daa700 1 -- 172.21.15.32:0/928737216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6bc101bb0 msgr2=0x7ff6bc101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:04.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.462+0000 7ff6c1daa700 1 --2- 172.21.15.32:0/928737216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6bc101bb0 0x7ff6bc101f80 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac004940 tx=0x7ff6ac009e30 comp rx=0 tx=0).stop 2024-05-12T04:12:04.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.462+0000 7ff6c1daa700 1 -- 172.21.15.32:0/928737216 shutdown_connections 2024-05-12T04:12:04.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.462+0000 7ff6c1daa700 1 --2- 172.21.15.32:0/928737216 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6bc102550 0x7ff6bc0fbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:04.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.462+0000 7ff6c1daa700 1 --2- 172.21.15.32:0/928737216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6bc101bb0 0x7ff6bc101f80 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:04.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.462+0000 7ff6c1daa700 1 -- 172.21.15.32:0/928737216 >> 172.21.15.32:0/928737216 conn(0x7ff6bc0f76e0 msgr2=0x7ff6bc0f9af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:04.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.462+0000 7ff6c1daa700 1 -- 172.21.15.32:0/928737216 shutdown_connections 2024-05-12T04:12:04.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.462+0000 7ff6c1daa700 1 -- 172.21.15.32:0/928737216 wait complete. 2024-05-12T04:12:04.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.462+0000 7ff6c1daa700 1 Processor -- start 2024-05-12T04:12:04.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.463+0000 7ff6c1daa700 1 -- start start 2024-05-12T04:12:04.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.463+0000 7ff6c1daa700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6bc101bb0 0x7ff6bc10e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:04.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.463+0000 7ff6c1daa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6bc102550 0x7ff6bc10eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:04.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.463+0000 7ff6c1daa700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6bc10f3e0 con 0x7ff6bc102550 2024-05-12T04:12:04.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.463+0000 7ff6c1daa700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6bc10f520 con 0x7ff6bc101bb0 2024-05-12T04:12:04.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.463+0000 7ff6baffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6bc102550 0x7ff6bc10eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:04.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.463+0000 7ff6baffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6bc102550 0x7ff6bc10eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39260/0 (socket says 172.21.15.32:39260) 2024-05-12T04:12:04.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.463+0000 7ff6baffd700 1 -- 172.21.15.32:0/1614314269 learned_addr learned my addr 172.21.15.32:0/1614314269 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:12:04.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.463+0000 7ff6bb7fe700 1 --2- 172.21.15.32:0/1614314269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6bc101bb0 0x7ff6bc10e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:04.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.463+0000 7ff6baffd700 1 -- 172.21.15.32:0/1614314269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6bc101bb0 msgr2=0x7ff6bc10e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:04.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.463+0000 7ff6baffd700 1 --2- 172.21.15.32:0/1614314269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6bc101bb0 0x7ff6bc10e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:04.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.463+0000 7ff6baffd700 1 -- 172.21.15.32:0/1614314269 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6ac005040 con 0x7ff6bc102550 2024-05-12T04:12:04.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.464+0000 7ff6baffd700 1 --2- 172.21.15.32:0/1614314269 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6bc102550 0x7ff6bc10eea0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff6b0002a80 tx=0x7ff6b0004630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:04.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.464+0000 7ff6a3fff700 1 -- 172.21.15.32:0/1614314269 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6b0004e80 con 0x7ff6bc102550 2024-05-12T04:12:04.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.464+0000 7ff6a3fff700 1 -- 172.21.15.32:0/1614314269 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff6b000f040 con 0x7ff6bc102550 2024-05-12T04:12:04.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.464+0000 7ff6c1daa700 1 -- 172.21.15.32:0/1614314269 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6bc1179d0 con 0x7ff6bc102550 2024-05-12T04:12:04.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.464+0000 7ff6a3fff700 1 -- 172.21.15.32:0/1614314269 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6b000da00 con 0x7ff6bc102550 2024-05-12T04:12:04.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.464+0000 7ff6c1daa700 1 -- 172.21.15.32:0/1614314269 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6bc117da0 con 0x7ff6bc102550 2024-05-12T04:12:04.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.465+0000 7ff6c1daa700 1 -- 172.21.15.32:0/1614314269 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6bc0fd300 con 0x7ff6bc102550 2024-05-12T04:12:04.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.466+0000 7ff6a3fff700 1 -- 172.21.15.32:0/1614314269 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7ff6b0007cc0 con 0x7ff6bc102550 2024-05-12T04:12:04.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.466+0000 7ff6a3fff700 1 --2- 172.21.15.32:0/1614314269 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7ff6a406cc20 0x7ff6a406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:04.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.466+0000 7ff6a3fff700 1 -- 172.21.15.32:0/1614314269 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7ff6b008b720 con 0x7ff6bc102550 2024-05-12T04:12:04.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.466+0000 7ff6bb7fe700 1 --2- 172.21.15.32:0/1614314269 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7ff6a406cc20 0x7ff6a406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:04.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.467+0000 7ff6bb7fe700 1 --2- 172.21.15.32:0/1614314269 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7ff6a406cc20 0x7ff6a406f0d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac00b840 tx=0x7ff6ac009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:04.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.471+0000 7ff6a3fff700 1 -- 172.21.15.32:0/1614314269 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7ff6b0055680 con 0x7ff6bc102550 2024-05-12T04:12:04.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.717+0000 7ff6c1daa700 1 -- 172.21.15.32:0/1614314269 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 24, "format": "json"} v 0) v1 -- 0x7ff6bc0640c0 con 0x7ff6bc102550 2024-05-12T04:12:04.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.718+0000 7ff6a3fff700 1 -- 172.21.15.32:0/1614314269 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 24, "format": "json"}]=0 dumped fsmap epoch 24 v37) v1 ==== 107+0+4045 (secure 0 0 0) 0x7ff6b0058ca0 con 0x7ff6bc102550 2024-05-12T04:12:04.718 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:12:04.718 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34404,"name":"cephfs.smithi032.ialtqb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6827/1080358503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":1080358503},{"type":"v1","addr":"172.21.15.32:6827","nonce":1080358503}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":44303,"name":"cephfs.smithi033.jgbjhk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6825/2944716783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":2944716783},{"type":"v1","addr":"172.21.15.33:6825","nonce":2944716783}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-12T03:57:48.904614+0000","modified":"2024-05-12T04:08:08.981164+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24283},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24283":{"gid":24283,"name":"cephfs.smithi033.hglwml","rank":0,"incarnation":24,"state":"up:replay","state_seq":1,"addr":"172.21.15.33:6827/433858238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":433858238},{"type":"v1","addr":"172.21.15.33:6827","nonce":433858238}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-05-12T04:12:04.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.720+0000 7ff6c1daa700 1 -- 172.21.15.32:0/1614314269 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7ff6a406cc20 msgr2=0x7ff6a406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:04.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.720+0000 7ff6c1daa700 1 --2- 172.21.15.32:0/1614314269 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7ff6a406cc20 0x7ff6a406f0d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac00b840 tx=0x7ff6ac009e30 comp rx=0 tx=0).stop 2024-05-12T04:12:04.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.720+0000 7ff6c1daa700 1 -- 172.21.15.32:0/1614314269 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6bc102550 msgr2=0x7ff6bc10eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:04.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.720+0000 7ff6c1daa700 1 --2- 172.21.15.32:0/1614314269 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6bc102550 0x7ff6bc10eea0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff6b0002a80 tx=0x7ff6b0004630 comp rx=0 tx=0).stop 2024-05-12T04:12:04.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.720+0000 7ff6c1daa700 1 -- 172.21.15.32:0/1614314269 shutdown_connections 2024-05-12T04:12:04.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.720+0000 7ff6c1daa700 1 --2- 172.21.15.32:0/1614314269 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7ff6a406cc20 0x7ff6a406f0d0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:04.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.720+0000 7ff6c1daa700 1 --2- 172.21.15.32:0/1614314269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6bc101bb0 0x7ff6bc10e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:04.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.720+0000 7ff6c1daa700 1 --2- 172.21.15.32:0/1614314269 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6bc102550 0x7ff6bc10eea0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:04.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.720+0000 7ff6c1daa700 1 -- 172.21.15.32:0/1614314269 >> 172.21.15.32:0/1614314269 conn(0x7ff6bc0f76e0 msgr2=0x7ff6bc0f8780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:04.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.720+0000 7ff6c1daa700 1 -- 172.21.15.32:0/1614314269 shutdown_connections 2024-05-12T04:12:04.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.720+0000 7ff6c1daa700 1 -- 172.21.15.32:0/1614314269 wait complete. 2024-05-12T04:12:04.722 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 24 2024-05-12T04:12:04.731 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:04 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/264774058' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-05-12T04:12:04.731 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:04 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/826243501' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-05-12T04:12:04.732 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:04 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/3771642411' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-05-12T04:12:04.732 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 24 2024-05-12T04:12:04.733 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 25 2024-05-12T04:12:04.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.855+0000 7f02dddf9700 1 -- 172.21.15.32:0/3948758818 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02d8102550 msgr2=0x7f02d80fbbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:04.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.855+0000 7f02dddf9700 1 --2- 172.21.15.32:0/3948758818 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02d8102550 0x7f02d80fbbc0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f02c0004940 tx=0x7f02c0009e30 comp rx=0 tx=0).stop 2024-05-12T04:12:04.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.855+0000 7f02dddf9700 1 -- 172.21.15.32:0/3948758818 shutdown_connections 2024-05-12T04:12:04.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.855+0000 7f02dddf9700 1 --2- 172.21.15.32:0/3948758818 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02d8102550 0x7f02d80fbbc0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:04.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.855+0000 7f02dddf9700 1 --2- 172.21.15.32:0/3948758818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02d8101bb0 0x7f02d8101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:04.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.855+0000 7f02dddf9700 1 -- 172.21.15.32:0/3948758818 >> 172.21.15.32:0/3948758818 conn(0x7f02d80f7700 msgr2=0x7f02d80f9b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:04.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.855+0000 7f02dddf9700 1 -- 172.21.15.32:0/3948758818 shutdown_connections 2024-05-12T04:12:04.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.855+0000 7f02dddf9700 1 -- 172.21.15.32:0/3948758818 wait complete. 2024-05-12T04:12:04.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.856+0000 7f02dddf9700 1 Processor -- start 2024-05-12T04:12:04.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.856+0000 7f02dddf9700 1 -- start start 2024-05-12T04:12:04.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.856+0000 7f02dddf9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02d8101bb0 0x7f02d810e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:04.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.856+0000 7f02dddf9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02d8102550 0x7f02d810eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:04.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.856+0000 7f02dddf9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d810f3e0 con 0x7f02d8101bb0 2024-05-12T04:12:04.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.856+0000 7f02dddf9700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d810f520 con 0x7f02d8102550 2024-05-12T04:12:04.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.856+0000 7f02d77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02d8101bb0 0x7f02d810e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:04.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.856+0000 7f02d77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02d8101bb0 0x7f02d810e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39274/0 (socket says 172.21.15.32:39274) 2024-05-12T04:12:04.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.856+0000 7f02d77fe700 1 -- 172.21.15.32:0/1982014212 learned_addr learned my addr 172.21.15.32:0/1982014212 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:12:04.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.857+0000 7f02d6ffd700 1 --2- 172.21.15.32:0/1982014212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02d8102550 0x7f02d810eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:04.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.857+0000 7f02d77fe700 1 -- 172.21.15.32:0/1982014212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02d8102550 msgr2=0x7f02d810eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:04.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.857+0000 7f02d77fe700 1 --2- 172.21.15.32:0/1982014212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02d8102550 0x7f02d810eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:04.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.857+0000 7f02d77fe700 1 -- 172.21.15.32:0/1982014212 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02c0005040 con 0x7f02d8101bb0 2024-05-12T04:12:04.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.857+0000 7f02d77fe700 1 --2- 172.21.15.32:0/1982014212 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02d8101bb0 0x7f02d810e960 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f02c800c830 tx=0x7f02c8004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:04.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.857+0000 7f02cffff700 1 -- 172.21.15.32:0/1982014212 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02c800a760 con 0x7f02d8101bb0 2024-05-12T04:12:04.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.857+0000 7f02cffff700 1 -- 172.21.15.32:0/1982014212 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f02c800ad70 con 0x7f02d8101bb0 2024-05-12T04:12:04.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.857+0000 7f02dddf9700 1 -- 172.21.15.32:0/1982014212 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02d81179d0 con 0x7f02d8101bb0 2024-05-12T04:12:04.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.857+0000 7f02cffff700 1 -- 172.21.15.32:0/1982014212 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02c80113a0 con 0x7f02d8101bb0 2024-05-12T04:12:04.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.857+0000 7f02dddf9700 1 -- 172.21.15.32:0/1982014212 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02d8117da0 con 0x7f02d8101bb0 2024-05-12T04:12:04.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.858+0000 7f02dddf9700 1 -- 172.21.15.32:0/1982014212 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02d80fd300 con 0x7f02d8101bb0 2024-05-12T04:12:04.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.858+0000 7f02cffff700 1 -- 172.21.15.32:0/1982014212 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f02c8012400 con 0x7f02d8101bb0 2024-05-12T04:12:04.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.859+0000 7f02cffff700 1 --2- 172.21.15.32:0/1982014212 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f02c406cc20 0x7f02c406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:04.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.859+0000 7f02cffff700 1 -- 172.21.15.32:0/1982014212 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f02c808c170 con 0x7f02d8101bb0 2024-05-12T04:12:04.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.859+0000 7f02d6ffd700 1 --2- 172.21.15.32:0/1982014212 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f02c406cc20 0x7f02c406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:04.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.859+0000 7f02d6ffd700 1 --2- 172.21.15.32:0/1982014212 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f02c406cc20 0x7f02c406f0d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f02c000b3b0 tx=0x7f02c0009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:04.863 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:04.863+0000 7f02cffff700 1 -- 172.21.15.32:0/1982014212 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f02c80560f0 con 0x7f02d8101bb0 2024-05-12T04:12:04.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:04 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/264774058' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-05-12T04:12:04.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:04 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/826243501' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-05-12T04:12:04.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:04 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/3771642411' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-05-12T04:12:05.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.116+0000 7f02dddf9700 1 -- 172.21.15.32:0/1982014212 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 25, "format": "json"} v 0) v1 -- 0x7f02d80640c0 con 0x7f02d8101bb0 2024-05-12T04:12:05.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.117+0000 7f02cffff700 1 -- 172.21.15.32:0/1982014212 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 25, "format": "json"}]=0 dumped fsmap epoch 25 v37) v1 ==== 107+0+4824 (secure 0 0 0) 0x7f02c8059710 con 0x7f02d8101bb0 2024-05-12T04:12:05.117 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:12:05.117 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34404,"name":"cephfs.smithi032.ialtqb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6827/1080358503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":1080358503},{"type":"v1","addr":"172.21.15.32:6827","nonce":1080358503}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":34436,"name":"cephfs.smithi032.qpsawh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6829/3603452506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":3603452506},{"type":"v1","addr":"172.21.15.32:6829","nonce":3603452506}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":44303,"name":"cephfs.smithi033.jgbjhk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6825/2944716783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":2944716783},{"type":"v1","addr":"172.21.15.33:6825","nonce":2944716783}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-12T03:57:48.904614+0000","modified":"2024-05-12T04:08:08.981164+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24283},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24283":{"gid":24283,"name":"cephfs.smithi033.hglwml","rank":0,"incarnation":24,"state":"up:replay","state_seq":1,"addr":"172.21.15.33:6827/433858238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":433858238},{"type":"v1","addr":"172.21.15.33:6827","nonce":433858238}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-05-12T04:12:05.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.119+0000 7f02dddf9700 1 -- 172.21.15.32:0/1982014212 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f02c406cc20 msgr2=0x7f02c406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:05.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.119+0000 7f02dddf9700 1 --2- 172.21.15.32:0/1982014212 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f02c406cc20 0x7f02c406f0d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f02c000b3b0 tx=0x7f02c0009e30 comp rx=0 tx=0).stop 2024-05-12T04:12:05.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.119+0000 7f02dddf9700 1 -- 172.21.15.32:0/1982014212 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02d8101bb0 msgr2=0x7f02d810e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:05.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.119+0000 7f02dddf9700 1 --2- 172.21.15.32:0/1982014212 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02d8101bb0 0x7f02d810e960 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f02c800c830 tx=0x7f02c8004470 comp rx=0 tx=0).stop 2024-05-12T04:12:05.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.119+0000 7f02dddf9700 1 -- 172.21.15.32:0/1982014212 shutdown_connections 2024-05-12T04:12:05.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.119+0000 7f02dddf9700 1 --2- 172.21.15.32:0/1982014212 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f02c406cc20 0x7f02c406f0d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:05.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.119+0000 7f02dddf9700 1 --2- 172.21.15.32:0/1982014212 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02d8101bb0 0x7f02d810e960 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:05.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.119+0000 7f02dddf9700 1 --2- 172.21.15.32:0/1982014212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02d8102550 0x7f02d810eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:05.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.119+0000 7f02dddf9700 1 -- 172.21.15.32:0/1982014212 >> 172.21.15.32:0/1982014212 conn(0x7f02d80f7700 msgr2=0x7f02d80f87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:05.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.119+0000 7f02dddf9700 1 -- 172.21.15.32:0/1982014212 shutdown_connections 2024-05-12T04:12:05.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.119+0000 7f02dddf9700 1 -- 172.21.15.32:0/1982014212 wait complete. 2024-05-12T04:12:05.121 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 25 2024-05-12T04:12:05.132 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 25 2024-05-12T04:12:05.133 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 26 2024-05-12T04:12:05.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.255+0000 7fb58b8af700 1 -- 172.21.15.32:0/4249846993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5840fe6e0 msgr2=0x7fb584106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:05.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.255+0000 7fb58b8af700 1 --2- 172.21.15.32:0/4249846993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5840fe6e0 0x7fb584106d20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb574004940 tx=0x7fb574009e30 comp rx=0 tx=0).stop 2024-05-12T04:12:05.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.256+0000 7fb58b8af700 1 -- 172.21.15.32:0/4249846993 shutdown_connections 2024-05-12T04:12:05.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.256+0000 7fb58b8af700 1 --2- 172.21.15.32:0/4249846993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5840fe6e0 0x7fb584106d20 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:05.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.256+0000 7fb58b8af700 1 --2- 172.21.15.32:0/4249846993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5840fdd40 0x7fb5840fe110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:05.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.256+0000 7fb58b8af700 1 -- 172.21.15.32:0/4249846993 >> 172.21.15.32:0/4249846993 conn(0x7fb5840f7730 msgr2=0x7fb5840f9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:05.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.256+0000 7fb58b8af700 1 -- 172.21.15.32:0/4249846993 shutdown_connections 2024-05-12T04:12:05.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.256+0000 7fb58b8af700 1 -- 172.21.15.32:0/4249846993 wait complete. 2024-05-12T04:12:05.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.256+0000 7fb58b8af700 1 Processor -- start 2024-05-12T04:12:05.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.256+0000 7fb58b8af700 1 -- start start 2024-05-12T04:12:05.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.257+0000 7fb58b8af700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5840fdd40 0x7fb5841136b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:05.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.257+0000 7fb58b8af700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5840fe6e0 0x7fb584113bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:05.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.257+0000 7fb58b8af700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb584114240 con 0x7fb5840fdd40 2024-05-12T04:12:05.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.257+0000 7fb58b8af700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb58410d860 con 0x7fb5840fe6e0 2024-05-12T04:12:05.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.257+0000 7fb58964b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5840fdd40 0x7fb5841136b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:05.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.257+0000 7fb58964b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5840fdd40 0x7fb5841136b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39284/0 (socket says 172.21.15.32:39284) 2024-05-12T04:12:05.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.257+0000 7fb58964b700 1 -- 172.21.15.32:0/720852205 learned_addr learned my addr 172.21.15.32:0/720852205 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:12:05.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.257+0000 7fb58964b700 1 -- 172.21.15.32:0/720852205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5840fe6e0 msgr2=0x7fb584113bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:05.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.257+0000 7fb588e4a700 1 --2- 172.21.15.32:0/720852205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5840fe6e0 0x7fb584113bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:05.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.257+0000 7fb58964b700 1 --2- 172.21.15.32:0/720852205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5840fe6e0 0x7fb584113bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:05.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.257+0000 7fb58964b700 1 -- 172.21.15.32:0/720852205 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb574005040 con 0x7fb5840fdd40 2024-05-12T04:12:05.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.257+0000 7fb58964b700 1 --2- 172.21.15.32:0/720852205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5840fdd40 0x7fb5841136b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb580002a50 tx=0x7fb5800046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:05.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.257+0000 7fb579ffb700 1 -- 172.21.15.32:0/720852205 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb580007cc0 con 0x7fb5840fdd40 2024-05-12T04:12:05.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.257+0000 7fb58b8af700 1 -- 172.21.15.32:0/720852205 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb58410db40 con 0x7fb5840fdd40 2024-05-12T04:12:05.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.257+0000 7fb579ffb700 1 -- 172.21.15.32:0/720852205 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb58000f040 con 0x7fb5840fdd40 2024-05-12T04:12:05.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.257+0000 7fb579ffb700 1 -- 172.21.15.32:0/720852205 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb580009540 con 0x7fb5840fdd40 2024-05-12T04:12:05.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.258+0000 7fb58b8af700 1 -- 172.21.15.32:0/720852205 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb58410e080 con 0x7fb5840fdd40 2024-05-12T04:12:05.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.258+0000 7fb58b8af700 1 -- 172.21.15.32:0/720852205 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb584066430 con 0x7fb5840fdd40 2024-05-12T04:12:05.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.259+0000 7fb579ffb700 1 -- 172.21.15.32:0/720852205 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fb5800096a0 con 0x7fb5840fdd40 2024-05-12T04:12:05.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.259+0000 7fb579ffb700 1 --2- 172.21.15.32:0/720852205 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fb57006cc20 0x7fb57006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:05.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.260+0000 7fb579ffb700 1 -- 172.21.15.32:0/720852205 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fb58008bed0 con 0x7fb5840fdd40 2024-05-12T04:12:05.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.260+0000 7fb588e4a700 1 --2- 172.21.15.32:0/720852205 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fb57006cc20 0x7fb57006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:05.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.260+0000 7fb588e4a700 1 --2- 172.21.15.32:0/720852205 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fb57006cc20 0x7fb57006f0d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb58410ef90 tx=0x7fb574009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:05.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.264+0000 7fb579ffb700 1 -- 172.21.15.32:0/720852205 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fb5800598c0 con 0x7fb5840fdd40 2024-05-12T04:12:05.514 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.514+0000 7fb58b8af700 1 -- 172.21.15.32:0/720852205 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 26, "format": "json"} v 0) v1 -- 0x7fb58410ed40 con 0x7fb5840fdd40 2024-05-12T04:12:05.515 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.515+0000 7fb579ffb700 1 -- 172.21.15.32:0/720852205 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 26, "format": "json"}]=0 dumped fsmap epoch 26 v37) v1 ==== 107+0+4829 (secure 0 0 0) 0x7fb580059450 con 0x7fb5840fdd40 2024-05-12T04:12:05.515 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:12:05.515 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34404,"name":"cephfs.smithi032.ialtqb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6827/1080358503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":1080358503},{"type":"v1","addr":"172.21.15.32:6827","nonce":1080358503}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":34436,"name":"cephfs.smithi032.qpsawh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6829/3603452506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":3603452506},{"type":"v1","addr":"172.21.15.32:6829","nonce":3603452506}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":44303,"name":"cephfs.smithi033.jgbjhk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6825/2944716783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":2944716783},{"type":"v1","addr":"172.21.15.33:6825","nonce":2944716783}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-12T03:57:48.904614+0000","modified":"2024-05-12T04:08:19.221372+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24283},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24283":{"gid":24283,"name":"cephfs.smithi033.hglwml","rank":0,"incarnation":24,"state":"up:reconnect","state_seq":155,"addr":"172.21.15.33:6827/433858238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":433858238},{"type":"v1","addr":"172.21.15.33:6827","nonce":433858238}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-05-12T04:12:05.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.517+0000 7fb58b8af700 1 -- 172.21.15.32:0/720852205 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fb57006cc20 msgr2=0x7fb57006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:05.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.517+0000 7fb58b8af700 1 --2- 172.21.15.32:0/720852205 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fb57006cc20 0x7fb57006f0d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb58410ef90 tx=0x7fb574009e30 comp rx=0 tx=0).stop 2024-05-12T04:12:05.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.517+0000 7fb58b8af700 1 -- 172.21.15.32:0/720852205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5840fdd40 msgr2=0x7fb5841136b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:05.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.517+0000 7fb58b8af700 1 --2- 172.21.15.32:0/720852205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5840fdd40 0x7fb5841136b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb580002a50 tx=0x7fb5800046b0 comp rx=0 tx=0).stop 2024-05-12T04:12:05.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.518+0000 7fb58b8af700 1 -- 172.21.15.32:0/720852205 shutdown_connections 2024-05-12T04:12:05.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.518+0000 7fb58b8af700 1 --2- 172.21.15.32:0/720852205 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fb57006cc20 0x7fb57006f0d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:05.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.518+0000 7fb58b8af700 1 --2- 172.21.15.32:0/720852205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5840fdd40 0x7fb5841136b0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:05.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.518+0000 7fb58b8af700 1 --2- 172.21.15.32:0/720852205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5840fe6e0 0x7fb584113bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:05.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.518+0000 7fb58b8af700 1 -- 172.21.15.32:0/720852205 >> 172.21.15.32:0/720852205 conn(0x7fb5840f7730 msgr2=0x7fb5840f8dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:05.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.518+0000 7fb58b8af700 1 -- 172.21.15.32:0/720852205 shutdown_connections 2024-05-12T04:12:05.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.518+0000 7fb58b8af700 1 -- 172.21.15.32:0/720852205 wait complete. 2024-05-12T04:12:05.519 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 26 2024-05-12T04:12:05.530 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 26 2024-05-12T04:12:05.530 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 27 2024-05-12T04:12:05.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.653+0000 7f98d2fab700 1 -- 172.21.15.32:0/3179134478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98cc064e40 msgr2=0x7f98cc0652b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:05.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.653+0000 7f98d2fab700 1 --2- 172.21.15.32:0/3179134478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98cc064e40 0x7f98cc0652b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f98b8004940 tx=0x7f98b8009e30 comp rx=0 tx=0).stop 2024-05-12T04:12:05.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.653+0000 7f98d2fab700 1 -- 172.21.15.32:0/3179134478 shutdown_connections 2024-05-12T04:12:05.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.653+0000 7f98d2fab700 1 --2- 172.21.15.32:0/3179134478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98cc064e40 0x7f98cc0652b0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:05.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.653+0000 7f98d2fab700 1 --2- 172.21.15.32:0/3179134478 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98cc102d80 0x7f98cc103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:05.655 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.653+0000 7f98d2fab700 1 -- 172.21.15.32:0/3179134478 >> 172.21.15.32:0/3179134478 conn(0x7f98cc072b00 msgr2=0x7f98cc072f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:05.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.653+0000 7f98d2fab700 1 -- 172.21.15.32:0/3179134478 shutdown_connections 2024-05-12T04:12:05.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.653+0000 7f98d2fab700 1 -- 172.21.15.32:0/3179134478 wait complete. 2024-05-12T04:12:05.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.653+0000 7f98d2fab700 1 Processor -- start 2024-05-12T04:12:05.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.653+0000 7f98d2fab700 1 -- start start 2024-05-12T04:12:05.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.654+0000 7f98d2fab700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98cc064e40 0x7f98cc0fbc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:05.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.654+0000 7f98d2fab700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98cc102d80 0x7f98cc0fc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:05.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.654+0000 7f98d2fab700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc0fdbc0 con 0x7f98cc102d80 2024-05-12T04:12:05.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.654+0000 7f98d2fab700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc0fc6f0 con 0x7f98cc064e40 2024-05-12T04:12:05.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.654+0000 7f98cbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98cc102d80 0x7f98cc0fc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:05.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.654+0000 7f98cbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98cc102d80 0x7f98cc0fc1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39292/0 (socket says 172.21.15.32:39292) 2024-05-12T04:12:05.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.654+0000 7f98cbfff700 1 -- 172.21.15.32:0/561119271 learned_addr learned my addr 172.21.15.32:0/561119271 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:12:05.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.654+0000 7f98d0d47700 1 --2- 172.21.15.32:0/561119271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98cc064e40 0x7f98cc0fbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:05.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.654+0000 7f98cbfff700 1 -- 172.21.15.32:0/561119271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98cc064e40 msgr2=0x7f98cc0fbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:05.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.654+0000 7f98cbfff700 1 --2- 172.21.15.32:0/561119271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98cc064e40 0x7f98cc0fbc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:05.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.654+0000 7f98cbfff700 1 -- 172.21.15.32:0/561119271 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b8005040 con 0x7f98cc102d80 2024-05-12T04:12:05.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.654+0000 7f98cbfff700 1 --2- 172.21.15.32:0/561119271 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98cc102d80 0x7f98cc0fc1b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f98b8009360 tx=0x7f98b8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:05.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.654+0000 7f98c97fa700 1 -- 172.21.15.32:0/561119271 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98b800e040 con 0x7f98cc102d80 2024-05-12T04:12:05.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.655+0000 7f98d2fab700 1 -- 172.21.15.32:0/561119271 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98cc0fc970 con 0x7f98cc102d80 2024-05-12T04:12:05.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.655+0000 7f98c97fa700 1 -- 172.21.15.32:0/561119271 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f98b8013420 con 0x7f98cc102d80 2024-05-12T04:12:05.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.655+0000 7f98c97fa700 1 -- 172.21.15.32:0/561119271 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98b801c5f0 con 0x7f98cc102d80 2024-05-12T04:12:05.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.655+0000 7f98d2fab700 1 -- 172.21.15.32:0/561119271 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98cc119e90 con 0x7f98cc102d80 2024-05-12T04:12:05.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.655+0000 7f98d2fab700 1 -- 172.21.15.32:0/561119271 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98cc100050 con 0x7f98cc102d80 2024-05-12T04:12:05.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.656+0000 7f98c97fa700 1 -- 172.21.15.32:0/561119271 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f98b8013590 con 0x7f98cc102d80 2024-05-12T04:12:05.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.656+0000 7f98c97fa700 1 --2- 172.21.15.32:0/561119271 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f98bc06cc20 0x7f98bc06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:05.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.657+0000 7f98c97fa700 1 -- 172.21.15.32:0/561119271 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f98b808d330 con 0x7f98cc102d80 2024-05-12T04:12:05.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.657+0000 7f98d0d47700 1 --2- 172.21.15.32:0/561119271 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f98bc06cc20 0x7f98bc06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:05.658 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.657+0000 7f98d0d47700 1 --2- 172.21.15.32:0/561119271 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f98bc06cc20 0x7f98bc06f0d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f98c0004d60 tx=0x7f98c000b040 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:05.661 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.661+0000 7f98c97fa700 1 -- 172.21.15.32:0/561119271 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f98b8057310 con 0x7f98cc102d80 2024-05-12T04:12:05.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:05 smithi032 ceph-mon[120392]: pgmap v302: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:12:05.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:05 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/1614314269' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-05-12T04:12:05.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:05 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/1982014212' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-05-12T04:12:05.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.909+0000 7f98d2fab700 1 -- 172.21.15.32:0/561119271 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 27, "format": "json"} v 0) v1 -- 0x7f98cc0fe490 con 0x7f98cc102d80 2024-05-12T04:12:05.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.910+0000 7f98c97fa700 1 -- 172.21.15.32:0/561119271 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 27, "format": "json"}]=0 dumped fsmap epoch 27 v37) v1 ==== 107+0+4826 (secure 0 0 0) 0x7f98b805a930 con 0x7f98cc102d80 2024-05-12T04:12:05.910 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:12:05.910 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34404,"name":"cephfs.smithi032.ialtqb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6827/1080358503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":1080358503},{"type":"v1","addr":"172.21.15.32:6827","nonce":1080358503}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":34436,"name":"cephfs.smithi032.qpsawh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6829/3603452506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":3603452506},{"type":"v1","addr":"172.21.15.32:6829","nonce":3603452506}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":44303,"name":"cephfs.smithi033.jgbjhk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6825/2944716783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":2944716783},{"type":"v1","addr":"172.21.15.33:6825","nonce":2944716783}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-12T03:57:48.904614+0000","modified":"2024-05-12T04:08:20.222998+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24283},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24283":{"gid":24283,"name":"cephfs.smithi033.hglwml","rank":0,"incarnation":24,"state":"up:rejoin","state_seq":156,"addr":"172.21.15.33:6827/433858238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":433858238},{"type":"v1","addr":"172.21.15.33:6827","nonce":433858238}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-05-12T04:12:05.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:05 smithi033 ceph-mon[107305]: pgmap v302: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:12:05.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:05 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/1614314269' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-05-12T04:12:05.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:05 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/1982014212' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-05-12T04:12:05.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.912+0000 7f98d2fab700 1 -- 172.21.15.32:0/561119271 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f98bc06cc20 msgr2=0x7f98bc06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:05.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.912+0000 7f98d2fab700 1 --2- 172.21.15.32:0/561119271 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f98bc06cc20 0x7f98bc06f0d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f98c0004d60 tx=0x7f98c000b040 comp rx=0 tx=0).stop 2024-05-12T04:12:05.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.912+0000 7f98d2fab700 1 -- 172.21.15.32:0/561119271 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98cc102d80 msgr2=0x7f98cc0fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:05.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.912+0000 7f98d2fab700 1 --2- 172.21.15.32:0/561119271 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98cc102d80 0x7f98cc0fc1b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f98b8009360 tx=0x7f98b8009e30 comp rx=0 tx=0).stop 2024-05-12T04:12:05.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.912+0000 7f98d2fab700 1 -- 172.21.15.32:0/561119271 shutdown_connections 2024-05-12T04:12:05.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.912+0000 7f98d2fab700 1 --2- 172.21.15.32:0/561119271 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f98bc06cc20 0x7f98bc06f0d0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:05.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.912+0000 7f98d2fab700 1 --2- 172.21.15.32:0/561119271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98cc064e40 0x7f98cc0fbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:05.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.912+0000 7f98d2fab700 1 --2- 172.21.15.32:0/561119271 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98cc102d80 0x7f98cc0fc1b0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:05.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.912+0000 7f98d2fab700 1 -- 172.21.15.32:0/561119271 >> 172.21.15.32:0/561119271 conn(0x7f98cc072b00 msgr2=0x7f98cc107290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:05.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.912+0000 7f98d2fab700 1 -- 172.21.15.32:0/561119271 shutdown_connections 2024-05-12T04:12:05.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:05.912+0000 7f98d2fab700 1 -- 172.21.15.32:0/561119271 wait complete. 2024-05-12T04:12:05.914 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 27 2024-05-12T04:12:05.925 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 27 2024-05-12T04:12:05.925 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 28 2024-05-12T04:12:06.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.047+0000 7f60d96fa700 1 -- 172.21.15.32:0/3170355113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60d4101370 msgr2=0x7f60d4101740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:06.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.047+0000 7f60d96fa700 1 --2- 172.21.15.32:0/3170355113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60d4101370 0x7f60d4101740 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f60c4004a50 tx=0x7f60c400af40 comp rx=0 tx=0).stop 2024-05-12T04:12:06.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.047+0000 7f60d96fa700 1 -- 172.21.15.32:0/3170355113 shutdown_connections 2024-05-12T04:12:06.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.047+0000 7f60d96fa700 1 --2- 172.21.15.32:0/3170355113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60d4101d10 0x7f60d40fb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:06.049 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.047+0000 7f60d96fa700 1 --2- 172.21.15.32:0/3170355113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60d4101370 0x7f60d4101740 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:06.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.047+0000 7f60d96fa700 1 -- 172.21.15.32:0/3170355113 >> 172.21.15.32:0/3170355113 conn(0x7f60d40721d0 msgr2=0x7f60d40725d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:06.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.047+0000 7f60d96fa700 1 -- 172.21.15.32:0/3170355113 shutdown_connections 2024-05-12T04:12:06.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.047+0000 7f60d96fa700 1 -- 172.21.15.32:0/3170355113 wait complete. 2024-05-12T04:12:06.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.048+0000 7f60d96fa700 1 Processor -- start 2024-05-12T04:12:06.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.048+0000 7f60d96fa700 1 -- start start 2024-05-12T04:12:06.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.048+0000 7f60d96fa700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60d4101370 0x7f60d41021d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:06.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.048+0000 7f60d96fa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60d4101d10 0x7f60d4102710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:06.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.048+0000 7f60d96fa700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d4104170 con 0x7f60d4101d10 2024-05-12T04:12:06.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.048+0000 7f60d96fa700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d4102c50 con 0x7f60d4101370 2024-05-12T04:12:06.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.048+0000 7f60d27fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60d4101d10 0x7f60d4102710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:06.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.048+0000 7f60d27fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60d4101d10 0x7f60d4102710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39308/0 (socket says 172.21.15.32:39308) 2024-05-12T04:12:06.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.048+0000 7f60d27fc700 1 -- 172.21.15.32:0/637708460 learned_addr learned my addr 172.21.15.32:0/637708460 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:12:06.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.048+0000 7f60d2ffd700 1 --2- 172.21.15.32:0/637708460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60d4101370 0x7f60d41021d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:06.050 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.049+0000 7f60d27fc700 1 -- 172.21.15.32:0/637708460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60d4101370 msgr2=0x7f60d41021d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:06.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.049+0000 7f60d27fc700 1 --2- 172.21.15.32:0/637708460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60d4101370 0x7f60d41021d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:06.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.049+0000 7f60d27fc700 1 -- 172.21.15.32:0/637708460 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c4005040 con 0x7f60d4101d10 2024-05-12T04:12:06.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.049+0000 7f60d27fc700 1 --2- 172.21.15.32:0/637708460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60d4101d10 0x7f60d4102710 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f60bc002a50 tx=0x7f60bc004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:06.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.049+0000 7f60cb7fe700 1 -- 172.21.15.32:0/637708460 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60bc007cc0 con 0x7f60d4101d10 2024-05-12T04:12:06.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.049+0000 7f60cb7fe700 1 -- 172.21.15.32:0/637708460 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f60bc00f040 con 0x7f60d4101d10 2024-05-12T04:12:06.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.049+0000 7f60d96fa700 1 -- 172.21.15.32:0/637708460 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60d4102f00 con 0x7f60d4101d10 2024-05-12T04:12:06.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.049+0000 7f60cb7fe700 1 -- 172.21.15.32:0/637708460 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60bc00d920 con 0x7f60d4101d10 2024-05-12T04:12:06.051 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.049+0000 7f60d96fa700 1 -- 172.21.15.32:0/637708460 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60d4119e50 con 0x7f60d4101d10 2024-05-12T04:12:06.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.050+0000 7f60d96fa700 1 -- 172.21.15.32:0/637708460 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60d4100690 con 0x7f60d4101d10 2024-05-12T04:12:06.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.051+0000 7f60cb7fe700 1 -- 172.21.15.32:0/637708460 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f60bc009680 con 0x7f60d4101d10 2024-05-12T04:12:06.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.051+0000 7f60cb7fe700 1 --2- 172.21.15.32:0/637708460 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f60c006cc20 0x7f60c006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:06.052 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.051+0000 7f60cb7fe700 1 -- 172.21.15.32:0/637708460 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f60bc08be40 con 0x7f60d4101d10 2024-05-12T04:12:06.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.051+0000 7f60d2ffd700 1 --2- 172.21.15.32:0/637708460 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f60c006cc20 0x7f60c006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:06.053 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.051+0000 7f60d2ffd700 1 --2- 172.21.15.32:0/637708460 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f60c006cc20 0x7f60c006f0d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f60c4008d30 tx=0x7f60c400af40 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:06.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.055+0000 7f60cb7fe700 1 -- 172.21.15.32:0/637708460 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f60bc055da0 con 0x7f60d4101d10 2024-05-12T04:12:06.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.309+0000 7f60d96fa700 1 -- 172.21.15.32:0/637708460 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 28, "format": "json"} v 0) v1 -- 0x7f60d40fcac0 con 0x7f60d4101d10 2024-05-12T04:12:06.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.310+0000 7f60cb7fe700 1 -- 172.21.15.32:0/637708460 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 28, "format": "json"}]=0 dumped fsmap epoch 28 v37) v1 ==== 107+0+4826 (secure 0 0 0) 0x7f60bc0593c0 con 0x7f60d4101d10 2024-05-12T04:12:06.310 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:12:06.310 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34404,"name":"cephfs.smithi032.ialtqb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6827/1080358503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":1080358503},{"type":"v1","addr":"172.21.15.32:6827","nonce":1080358503}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":34436,"name":"cephfs.smithi032.qpsawh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6829/3603452506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":3603452506},{"type":"v1","addr":"172.21.15.32:6829","nonce":3603452506}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":44303,"name":"cephfs.smithi033.jgbjhk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6825/2944716783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":2944716783},{"type":"v1","addr":"172.21.15.33:6825","nonce":2944716783}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-12T03:57:48.904614+0000","modified":"2024-05-12T04:08:21.227973+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24283},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24283":{"gid":24283,"name":"cephfs.smithi033.hglwml","rank":0,"incarnation":24,"state":"up:active","state_seq":157,"addr":"172.21.15.33:6827/433858238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":433858238},{"type":"v1","addr":"172.21.15.33:6827","nonce":433858238}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-05-12T04:12:06.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.312+0000 7f60d96fa700 1 -- 172.21.15.32:0/637708460 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f60c006cc20 msgr2=0x7f60c006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:06.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.312+0000 7f60d96fa700 1 --2- 172.21.15.32:0/637708460 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f60c006cc20 0x7f60c006f0d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f60c4008d30 tx=0x7f60c400af40 comp rx=0 tx=0).stop 2024-05-12T04:12:06.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.312+0000 7f60d96fa700 1 -- 172.21.15.32:0/637708460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60d4101d10 msgr2=0x7f60d4102710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:06.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.312+0000 7f60d96fa700 1 --2- 172.21.15.32:0/637708460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60d4101d10 0x7f60d4102710 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f60bc002a50 tx=0x7f60bc004600 comp rx=0 tx=0).stop 2024-05-12T04:12:06.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.312+0000 7f60d96fa700 1 -- 172.21.15.32:0/637708460 shutdown_connections 2024-05-12T04:12:06.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.312+0000 7f60d96fa700 1 --2- 172.21.15.32:0/637708460 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f60c006cc20 0x7f60c006f0d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:06.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.312+0000 7f60d96fa700 1 --2- 172.21.15.32:0/637708460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60d4101370 0x7f60d41021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:06.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.312+0000 7f60d96fa700 1 --2- 172.21.15.32:0/637708460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f60d4101d10 0x7f60d4102710 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:06.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.312+0000 7f60d96fa700 1 -- 172.21.15.32:0/637708460 >> 172.21.15.32:0/637708460 conn(0x7f60d40721d0 msgr2=0x7f60d4104310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:06.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.312+0000 7f60d96fa700 1 -- 172.21.15.32:0/637708460 shutdown_connections 2024-05-12T04:12:06.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.312+0000 7f60d96fa700 1 -- 172.21.15.32:0/637708460 wait complete. 2024-05-12T04:12:06.314 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 28 2024-05-12T04:12:06.325 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 28 2024-05-12T04:12:06.325 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 29 2024-05-12T04:12:06.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.448+0000 7f8a21933700 1 -- 172.21.15.32:0/1407844916 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a1c102e30 msgr2=0x7f8a1c103200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:06.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.448+0000 7f8a21933700 1 --2- 172.21.15.32:0/1407844916 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a1c102e30 0x7f8a1c103200 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c004940 tx=0x7f8a0c009e30 comp rx=0 tx=0).stop 2024-05-12T04:12:06.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.448+0000 7f8a21933700 1 -- 172.21.15.32:0/1407844916 shutdown_connections 2024-05-12T04:12:06.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.448+0000 7f8a21933700 1 --2- 172.21.15.32:0/1407844916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a1c0fb270 0x7f8a1c0fb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:06.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.448+0000 7f8a21933700 1 --2- 172.21.15.32:0/1407844916 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a1c102e30 0x7f8a1c103200 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:06.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.448+0000 7f8a21933700 1 -- 172.21.15.32:0/1407844916 >> 172.21.15.32:0/1407844916 conn(0x7f8a1c0721d0 msgr2=0x7f8a1c0725d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:06.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.448+0000 7f8a21933700 1 -- 172.21.15.32:0/1407844916 shutdown_connections 2024-05-12T04:12:06.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.448+0000 7f8a21933700 1 -- 172.21.15.32:0/1407844916 wait complete. 2024-05-12T04:12:06.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.449+0000 7f8a21933700 1 Processor -- start 2024-05-12T04:12:06.451 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.449+0000 7f8a21933700 1 -- start start 2024-05-12T04:12:06.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.449+0000 7f8a21933700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a1c0fb270 0x7f8a1c1136c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:06.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.449+0000 7f8a21933700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a1c102e30 0x7f8a1c113c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:06.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.449+0000 7f8a21933700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1c114250 con 0x7f8a1c0fb270 2024-05-12T04:12:06.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.449+0000 7f8a21933700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1c10d870 con 0x7f8a1c102e30 2024-05-12T04:12:06.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.449+0000 7f8a1affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a1c0fb270 0x7f8a1c1136c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:06.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.449+0000 7f8a1affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a1c0fb270 0x7f8a1c1136c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39312/0 (socket says 172.21.15.32:39312) 2024-05-12T04:12:06.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.449+0000 7f8a1affd700 1 -- 172.21.15.32:0/1715337504 learned_addr learned my addr 172.21.15.32:0/1715337504 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:12:06.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.449+0000 7f8a13fff700 1 --2- 172.21.15.32:0/1715337504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a1c102e30 0x7f8a1c113c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:06.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.450+0000 7f8a1affd700 1 -- 172.21.15.32:0/1715337504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a1c102e30 msgr2=0x7f8a1c113c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:06.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.450+0000 7f8a1affd700 1 --2- 172.21.15.32:0/1715337504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a1c102e30 0x7f8a1c113c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:06.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.450+0000 7f8a1affd700 1 -- 172.21.15.32:0/1715337504 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a0c005040 con 0x7f8a1c0fb270 2024-05-12T04:12:06.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.450+0000 7f8a13fff700 1 --2- 172.21.15.32:0/1715337504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a1c102e30 0x7f8a1c113c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-12T04:12:06.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.450+0000 7f8a1affd700 1 --2- 172.21.15.32:0/1715337504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a1c0fb270 0x7f8a1c1136c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c009150 tx=0x7f8a0c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:06.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.450+0000 7f8a137fe700 1 -- 172.21.15.32:0/1715337504 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a0c00e040 con 0x7f8a1c0fb270 2024-05-12T04:12:06.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.450+0000 7f8a137fe700 1 -- 172.21.15.32:0/1715337504 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8a0c013420 con 0x7f8a1c0fb270 2024-05-12T04:12:06.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.450+0000 7f8a21933700 1 -- 172.21.15.32:0/1715337504 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a1c10daf0 con 0x7f8a1c0fb270 2024-05-12T04:12:06.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.450+0000 7f8a137fe700 1 -- 172.21.15.32:0/1715337504 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a0c01c630 con 0x7f8a1c0fb270 2024-05-12T04:12:06.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.450+0000 7f8a21933700 1 -- 172.21.15.32:0/1715337504 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a1c10e0b0 con 0x7f8a1c0fb270 2024-05-12T04:12:06.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.451+0000 7f8a21933700 1 -- 172.21.15.32:0/1715337504 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a1c105270 con 0x7f8a1c0fb270 2024-05-12T04:12:06.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.452+0000 7f8a137fe700 1 -- 172.21.15.32:0/1715337504 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f8a0c013990 con 0x7f8a1c0fb270 2024-05-12T04:12:06.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.452+0000 7f8a137fe700 1 --2- 172.21.15.32:0/1715337504 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f8a0806cc20 0x7f8a0806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:06.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.452+0000 7f8a137fe700 1 -- 172.21.15.32:0/1715337504 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f8a0c08de40 con 0x7f8a1c0fb270 2024-05-12T04:12:06.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.452+0000 7f8a13fff700 1 --2- 172.21.15.32:0/1715337504 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f8a0806cc20 0x7f8a0806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:06.454 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.453+0000 7f8a13fff700 1 --2- 172.21.15.32:0/1715337504 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f8a0806cc20 0x7f8a0806f0d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8a04007fd0 tx=0x7f8a04009040 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:06.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.456+0000 7f8a137fe700 1 -- 172.21.15.32:0/1715337504 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f8a0c05b890 con 0x7f8a1c0fb270 2024-05-12T04:12:06.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.707+0000 7f8a21933700 1 -- 172.21.15.32:0/1715337504 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 29, "format": "json"} v 0) v1 -- 0x7f8a1c109db0 con 0x7f8a1c0fb270 2024-05-12T04:12:06.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.707+0000 7f8a137fe700 1 -- 172.21.15.32:0/1715337504 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 29, "format": "json"}]=0 dumped fsmap epoch 29 v37) v1 ==== 107+0+4047 (secure 0 0 0) 0x7f8a0c05b420 con 0x7f8a1c0fb270 2024-05-12T04:12:06.708 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:12:06.708 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34404,"name":"cephfs.smithi032.ialtqb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6827/1080358503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":1080358503},{"type":"v1","addr":"172.21.15.32:6827","nonce":1080358503}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":34436,"name":"cephfs.smithi032.qpsawh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6829/3603452506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":3603452506},{"type":"v1","addr":"172.21.15.32:6829","nonce":3603452506}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-12T03:57:48.904614+0000","modified":"2024-05-12T04:08:21.227973+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24283},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24283":{"gid":24283,"name":"cephfs.smithi033.hglwml","rank":0,"incarnation":24,"state":"up:active","state_seq":157,"addr":"172.21.15.33:6827/433858238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":433858238},{"type":"v1","addr":"172.21.15.33:6827","nonce":433858238}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-05-12T04:12:06.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.710+0000 7f8a21933700 1 -- 172.21.15.32:0/1715337504 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f8a0806cc20 msgr2=0x7f8a0806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:06.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.710+0000 7f8a21933700 1 --2- 172.21.15.32:0/1715337504 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f8a0806cc20 0x7f8a0806f0d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8a04007fd0 tx=0x7f8a04009040 comp rx=0 tx=0).stop 2024-05-12T04:12:06.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.710+0000 7f8a21933700 1 -- 172.21.15.32:0/1715337504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a1c0fb270 msgr2=0x7f8a1c1136c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:06.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.710+0000 7f8a21933700 1 --2- 172.21.15.32:0/1715337504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a1c0fb270 0x7f8a1c1136c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c009150 tx=0x7f8a0c009e30 comp rx=0 tx=0).stop 2024-05-12T04:12:06.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.710+0000 7f8a21933700 1 -- 172.21.15.32:0/1715337504 shutdown_connections 2024-05-12T04:12:06.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.710+0000 7f8a21933700 1 --2- 172.21.15.32:0/1715337504 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f8a0806cc20 0x7f8a0806f0d0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:06.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.710+0000 7f8a21933700 1 --2- 172.21.15.32:0/1715337504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a1c0fb270 0x7f8a1c1136c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:06.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.710+0000 7f8a21933700 1 --2- 172.21.15.32:0/1715337504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a1c102e30 0x7f8a1c113c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:06.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.710+0000 7f8a21933700 1 -- 172.21.15.32:0/1715337504 >> 172.21.15.32:0/1715337504 conn(0x7f8a1c0721d0 msgr2=0x7f8a1c0f9520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:06.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.710+0000 7f8a21933700 1 -- 172.21.15.32:0/1715337504 shutdown_connections 2024-05-12T04:12:06.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.710+0000 7f8a21933700 1 -- 172.21.15.32:0/1715337504 wait complete. 2024-05-12T04:12:06.712 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 29 2024-05-12T04:12:06.721 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:06 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/720852205' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-05-12T04:12:06.721 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:06 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/561119271' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-05-12T04:12:06.722 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:06 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/637708460' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-05-12T04:12:06.723 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 29 2024-05-12T04:12:06.723 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 30 2024-05-12T04:12:06.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.845+0000 7fa20b7d5700 1 -- 172.21.15.32:0/1079841835 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa204064e40 msgr2=0x7fa2040652b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:06.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.845+0000 7fa20b7d5700 1 --2- 172.21.15.32:0/1079841835 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa204064e40 0x7fa2040652b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4004940 tx=0x7fa1f4009e30 comp rx=0 tx=0).stop 2024-05-12T04:12:06.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.845+0000 7fa20b7d5700 1 -- 172.21.15.32:0/1079841835 shutdown_connections 2024-05-12T04:12:06.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.845+0000 7fa20b7d5700 1 --2- 172.21.15.32:0/1079841835 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa204064e40 0x7fa2040652b0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:06.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.845+0000 7fa20b7d5700 1 --2- 172.21.15.32:0/1079841835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa204102d80 0x7fa204103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:06.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.845+0000 7fa20b7d5700 1 -- 172.21.15.32:0/1079841835 >> 172.21.15.32:0/1079841835 conn(0x7fa204072b00 msgr2=0x7fa204072f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:06.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.845+0000 7fa20b7d5700 1 -- 172.21.15.32:0/1079841835 shutdown_connections 2024-05-12T04:12:06.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.845+0000 7fa20b7d5700 1 -- 172.21.15.32:0/1079841835 wait complete. 2024-05-12T04:12:06.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.846+0000 7fa20b7d5700 1 Processor -- start 2024-05-12T04:12:06.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.846+0000 7fa20b7d5700 1 -- start start 2024-05-12T04:12:06.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.846+0000 7fa20b7d5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa204064e40 0x7fa2040fbc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:06.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.846+0000 7fa20b7d5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa204102d80 0x7fa2040fc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:06.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.846+0000 7fa20b7d5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2040fdbc0 con 0x7fa204102d80 2024-05-12T04:12:06.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.846+0000 7fa20b7d5700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2040fc6f0 con 0x7fa204064e40 2024-05-12T04:12:06.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.846+0000 7fa208d70700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa204102d80 0x7fa2040fc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:06.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.846+0000 7fa208d70700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa204102d80 0x7fa2040fc1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39342/0 (socket says 172.21.15.32:39342) 2024-05-12T04:12:06.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.846+0000 7fa208d70700 1 -- 172.21.15.32:0/2455948767 learned_addr learned my addr 172.21.15.32:0/2455948767 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:12:06.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.846+0000 7fa209571700 1 --2- 172.21.15.32:0/2455948767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa204064e40 0x7fa2040fbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:06.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.846+0000 7fa208d70700 1 -- 172.21.15.32:0/2455948767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa204064e40 msgr2=0x7fa2040fbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:06.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.846+0000 7fa208d70700 1 --2- 172.21.15.32:0/2455948767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa204064e40 0x7fa2040fbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:06.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.846+0000 7fa208d70700 1 -- 172.21.15.32:0/2455948767 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f4005040 con 0x7fa204102d80 2024-05-12T04:12:06.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.847+0000 7fa208d70700 1 --2- 172.21.15.32:0/2455948767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa204102d80 0x7fa2040fc1b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa1f40040b0 tx=0x7fa1f4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:06.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.847+0000 7fa1f9ffb700 1 -- 172.21.15.32:0/2455948767 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa1f4021070 con 0x7fa204102d80 2024-05-12T04:12:06.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.847+0000 7fa1f9ffb700 1 -- 172.21.15.32:0/2455948767 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa1f4013de0 con 0x7fa204102d80 2024-05-12T04:12:06.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.847+0000 7fa20b7d5700 1 -- 172.21.15.32:0/2455948767 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2040fc970 con 0x7fa204102d80 2024-05-12T04:12:06.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.847+0000 7fa1f9ffb700 1 -- 172.21.15.32:0/2455948767 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa1f400e070 con 0x7fa204102d80 2024-05-12T04:12:06.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.847+0000 7fa20b7d5700 1 -- 172.21.15.32:0/2455948767 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa204119df0 con 0x7fa204102d80 2024-05-12T04:12:06.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.848+0000 7fa20b7d5700 1 -- 172.21.15.32:0/2455948767 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa204100050 con 0x7fa204102d80 2024-05-12T04:12:06.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.848+0000 7fa1f9ffb700 1 -- 172.21.15.32:0/2455948767 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fa1f4013930 con 0x7fa204102d80 2024-05-12T04:12:06.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.849+0000 7fa1f9ffb700 1 --2- 172.21.15.32:0/2455948767 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fa1f006cc20 0x7fa1f006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:06.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.849+0000 7fa1f9ffb700 1 -- 172.21.15.32:0/2455948767 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fa1f408d0a0 con 0x7fa204102d80 2024-05-12T04:12:06.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.849+0000 7fa209571700 1 --2- 172.21.15.32:0/2455948767 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fa1f006cc20 0x7fa1f006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:06.850 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.849+0000 7fa209571700 1 --2- 172.21.15.32:0/2455948767 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fa1f006cc20 0x7fa1f006f0d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa200009b80 tx=0x7fa20000b040 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:06.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:06.854+0000 7fa1f9ffb700 1 -- 172.21.15.32:0/2455948767 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fa1f4057120 con 0x7fa204102d80 2024-05-12T04:12:06.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:06 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/720852205' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-05-12T04:12:06.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:06 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/561119271' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-05-12T04:12:06.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:06 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/637708460' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-05-12T04:12:07.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.101+0000 7fa20b7d5700 1 -- 172.21.15.32:0/2455948767 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 30, "format": "json"} v 0) v1 -- 0x7fa2040fe490 con 0x7fa204102d80 2024-05-12T04:12:07.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.102+0000 7fa1f9ffb700 1 -- 172.21.15.32:0/2455948767 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 30, "format": "json"}]=0 dumped fsmap epoch 30 v37) v1 ==== 107+0+4823 (secure 0 0 0) 0x7fa1f405a740 con 0x7fa204102d80 2024-05-12T04:12:07.102 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:12:07.102 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34404,"name":"cephfs.smithi032.ialtqb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6827/1080358503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":1080358503},{"type":"v1","addr":"172.21.15.32:6827","nonce":1080358503}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":34436,"name":"cephfs.smithi032.qpsawh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6829/3603452506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":3603452506},{"type":"v1","addr":"172.21.15.32:6829","nonce":3603452506}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":44343,"name":"cephfs.smithi033.jgbjhk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6825/997254871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":997254871},{"type":"v1","addr":"172.21.15.33:6825","nonce":997254871}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-12T03:57:48.904614+0000","modified":"2024-05-12T04:08:21.227973+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24283},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24283":{"gid":24283,"name":"cephfs.smithi033.hglwml","rank":0,"incarnation":24,"state":"up:active","state_seq":157,"addr":"172.21.15.33:6827/433858238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":433858238},{"type":"v1","addr":"172.21.15.33:6827","nonce":433858238}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-05-12T04:12:07.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.104+0000 7fa20b7d5700 1 -- 172.21.15.32:0/2455948767 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fa1f006cc20 msgr2=0x7fa1f006f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:07.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.104+0000 7fa20b7d5700 1 --2- 172.21.15.32:0/2455948767 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fa1f006cc20 0x7fa1f006f0d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa200009b80 tx=0x7fa20000b040 comp rx=0 tx=0).stop 2024-05-12T04:12:07.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.104+0000 7fa20b7d5700 1 -- 172.21.15.32:0/2455948767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa204102d80 msgr2=0x7fa2040fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:07.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.104+0000 7fa20b7d5700 1 --2- 172.21.15.32:0/2455948767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa204102d80 0x7fa2040fc1b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa1f40040b0 tx=0x7fa1f4009e30 comp rx=0 tx=0).stop 2024-05-12T04:12:07.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.104+0000 7fa20b7d5700 1 -- 172.21.15.32:0/2455948767 shutdown_connections 2024-05-12T04:12:07.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.104+0000 7fa20b7d5700 1 --2- 172.21.15.32:0/2455948767 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fa1f006cc20 0x7fa1f006f0d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:07.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.104+0000 7fa20b7d5700 1 --2- 172.21.15.32:0/2455948767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa204064e40 0x7fa2040fbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:07.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.104+0000 7fa20b7d5700 1 --2- 172.21.15.32:0/2455948767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa204102d80 0x7fa2040fc1b0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:07.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.104+0000 7fa20b7d5700 1 -- 172.21.15.32:0/2455948767 >> 172.21.15.32:0/2455948767 conn(0x7fa204072b00 msgr2=0x7fa2040fad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:07.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.104+0000 7fa20b7d5700 1 -- 172.21.15.32:0/2455948767 shutdown_connections 2024-05-12T04:12:07.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.104+0000 7fa20b7d5700 1 -- 172.21.15.32:0/2455948767 wait complete. 2024-05-12T04:12:07.106 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 30 2024-05-12T04:12:07.117 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 30 2024-05-12T04:12:07.117 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 31 2024-05-12T04:12:07.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.242+0000 7fcc04435700 1 -- 172.21.15.32:0/1289700643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbfc064e40 msgr2=0x7fcbfc0652b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:07.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.242+0000 7fcc04435700 1 --2- 172.21.15.32:0/1289700643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbfc064e40 0x7fcbfc0652b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fcbec004940 tx=0x7fcbec009e30 comp rx=0 tx=0).stop 2024-05-12T04:12:07.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.243+0000 7fcc04435700 1 -- 172.21.15.32:0/1289700643 shutdown_connections 2024-05-12T04:12:07.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.243+0000 7fcc04435700 1 --2- 172.21.15.32:0/1289700643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbfc064e40 0x7fcbfc0652b0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:07.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.243+0000 7fcc04435700 1 --2- 172.21.15.32:0/1289700643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbfc102d80 0x7fcbfc103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:07.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.243+0000 7fcc04435700 1 -- 172.21.15.32:0/1289700643 >> 172.21.15.32:0/1289700643 conn(0x7fcbfc072b00 msgr2=0x7fcbfc072f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:07.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.243+0000 7fcc04435700 1 -- 172.21.15.32:0/1289700643 shutdown_connections 2024-05-12T04:12:07.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.243+0000 7fcc04435700 1 -- 172.21.15.32:0/1289700643 wait complete. 2024-05-12T04:12:07.245 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.243+0000 7fcc04435700 1 Processor -- start 2024-05-12T04:12:07.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.243+0000 7fcc04435700 1 -- start start 2024-05-12T04:12:07.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.243+0000 7fcc04435700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbfc064e40 0x7fcbfc0fbc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:07.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.244+0000 7fcc04435700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbfc102d80 0x7fcbfc0fc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:07.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.244+0000 7fcc04435700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbfc0fdbc0 con 0x7fcbfc064e40 2024-05-12T04:12:07.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.244+0000 7fcc04435700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbfc0fc6f0 con 0x7fcbfc102d80 2024-05-12T04:12:07.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.244+0000 7fcc021d1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbfc064e40 0x7fcbfc0fbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:07.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.244+0000 7fcc021d1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbfc064e40 0x7fcbfc0fbc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39372/0 (socket says 172.21.15.32:39372) 2024-05-12T04:12:07.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.244+0000 7fcc021d1700 1 -- 172.21.15.32:0/763483252 learned_addr learned my addr 172.21.15.32:0/763483252 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:12:07.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.244+0000 7fcc019d0700 1 --2- 172.21.15.32:0/763483252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbfc102d80 0x7fcbfc0fc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:07.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.244+0000 7fcc021d1700 1 -- 172.21.15.32:0/763483252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbfc102d80 msgr2=0x7fcbfc0fc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:07.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.244+0000 7fcc021d1700 1 --2- 172.21.15.32:0/763483252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbfc102d80 0x7fcbfc0fc1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:07.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.244+0000 7fcc021d1700 1 -- 172.21.15.32:0/763483252 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbec005040 con 0x7fcbfc064e40 2024-05-12T04:12:07.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.244+0000 7fcc019d0700 1 --2- 172.21.15.32:0/763483252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbfc102d80 0x7fcbfc0fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-12T04:12:07.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.244+0000 7fcc021d1700 1 --2- 172.21.15.32:0/763483252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbfc064e40 0x7fcbfc0fbc70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fcbf800c830 tx=0x7fcbf8004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:07.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.244+0000 7fcbf2ffd700 1 -- 172.21.15.32:0/763483252 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcbf800a760 con 0x7fcbfc064e40 2024-05-12T04:12:07.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.244+0000 7fcbf2ffd700 1 -- 172.21.15.32:0/763483252 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcbf800ad70 con 0x7fcbfc064e40 2024-05-12T04:12:07.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.244+0000 7fcc04435700 1 -- 172.21.15.32:0/763483252 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbfc0fc9d0 con 0x7fcbfc064e40 2024-05-12T04:12:07.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.244+0000 7fcbf2ffd700 1 -- 172.21.15.32:0/763483252 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcbf80113a0 con 0x7fcbfc064e40 2024-05-12T04:12:07.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.244+0000 7fcc04435700 1 -- 172.21.15.32:0/763483252 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbfc119ef0 con 0x7fcbfc064e40 2024-05-12T04:12:07.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.245+0000 7fcc04435700 1 -- 172.21.15.32:0/763483252 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbfc100050 con 0x7fcbfc064e40 2024-05-12T04:12:07.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.246+0000 7fcbf2ffd700 1 -- 172.21.15.32:0/763483252 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fcbf8012400 con 0x7fcbfc064e40 2024-05-12T04:12:07.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.246+0000 7fcbf2ffd700 1 --2- 172.21.15.32:0/763483252 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fcbe806cc20 0x7fcbe806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:07.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.246+0000 7fcbf2ffd700 1 -- 172.21.15.32:0/763483252 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fcbf808c110 con 0x7fcbfc064e40 2024-05-12T04:12:07.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.246+0000 7fcc019d0700 1 --2- 172.21.15.32:0/763483252 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fcbe806cc20 0x7fcbe806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:07.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.247+0000 7fcc019d0700 1 --2- 172.21.15.32:0/763483252 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fcbe806cc20 0x7fcbe806f0d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc0fd3a0 tx=0x7fcbec009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:07.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.251+0000 7fcbf2ffd700 1 -- 172.21.15.32:0/763483252 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fcbf80560f0 con 0x7fcbfc064e40 2024-05-12T04:12:07.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.500+0000 7fcc04435700 1 -- 172.21.15.32:0/763483252 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 31, "format": "json"} v 0) v1 -- 0x7fcbfc0fe490 con 0x7fcbfc064e40 2024-05-12T04:12:07.501 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.501+0000 7fcbf2ffd700 1 -- 172.21.15.32:0/763483252 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 31, "format": "json"}]=0 dumped fsmap epoch 31 v37) v1 ==== 107+0+4034 (secure 0 0 0) 0x7fcbf8059710 con 0x7fcbfc064e40 2024-05-12T04:12:07.501 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:12:07.502 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34404,"name":"cephfs.smithi032.ialtqb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6827/1080358503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":1080358503},{"type":"v1","addr":"172.21.15.32:6827","nonce":1080358503}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":34436,"name":"cephfs.smithi032.qpsawh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6829/3603452506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":3603452506},{"type":"v1","addr":"172.21.15.32:6829","nonce":3603452506}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":44343,"name":"cephfs.smithi033.jgbjhk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6825/997254871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":997254871},{"type":"v1","addr":"172.21.15.33:6825","nonce":997254871}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-12T03:57:48.904614+0000","modified":"2024-05-12T04:08:50.380837+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-05-12T04:12:07.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.503+0000 7fcc04435700 1 -- 172.21.15.32:0/763483252 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fcbe806cc20 msgr2=0x7fcbe806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:07.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.503+0000 7fcc04435700 1 --2- 172.21.15.32:0/763483252 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fcbe806cc20 0x7fcbe806f0d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc0fd3a0 tx=0x7fcbec009e30 comp rx=0 tx=0).stop 2024-05-12T04:12:07.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.503+0000 7fcc04435700 1 -- 172.21.15.32:0/763483252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbfc064e40 msgr2=0x7fcbfc0fbc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:07.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.503+0000 7fcc04435700 1 --2- 172.21.15.32:0/763483252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbfc064e40 0x7fcbfc0fbc70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fcbf800c830 tx=0x7fcbf8004470 comp rx=0 tx=0).stop 2024-05-12T04:12:07.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.503+0000 7fcc04435700 1 -- 172.21.15.32:0/763483252 shutdown_connections 2024-05-12T04:12:07.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.503+0000 7fcc04435700 1 --2- 172.21.15.32:0/763483252 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fcbe806cc20 0x7fcbe806f0d0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:07.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.503+0000 7fcc04435700 1 --2- 172.21.15.32:0/763483252 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbfc064e40 0x7fcbfc0fbc70 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:07.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.503+0000 7fcc04435700 1 --2- 172.21.15.32:0/763483252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbfc102d80 0x7fcbfc0fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:07.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.503+0000 7fcc04435700 1 -- 172.21.15.32:0/763483252 >> 172.21.15.32:0/763483252 conn(0x7fcbfc072b00 msgr2=0x7fcbfc107290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:07.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.504+0000 7fcc04435700 1 -- 172.21.15.32:0/763483252 shutdown_connections 2024-05-12T04:12:07.505 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.504+0000 7fcc04435700 1 -- 172.21.15.32:0/763483252 wait complete. 2024-05-12T04:12:07.505 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 31 2024-05-12T04:12:07.516 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 31 2024-05-12T04:12:07.516 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 32 2024-05-12T04:12:07.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.639+0000 7f97b3532700 1 -- 172.21.15.32:0/919095434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97ac101c30 msgr2=0x7f97ac102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:07.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.639+0000 7f97b3532700 1 --2- 172.21.15.32:0/919095434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97ac101c30 0x7f97ac102000 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f97a8004940 tx=0x7f97a8009e30 comp rx=0 tx=0).stop 2024-05-12T04:12:07.640 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.639+0000 7f97b3532700 1 -- 172.21.15.32:0/919095434 shutdown_connections 2024-05-12T04:12:07.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.639+0000 7f97b3532700 1 --2- 172.21.15.32:0/919095434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97ac1025d0 0x7f97ac0fbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:07.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.639+0000 7f97b3532700 1 --2- 172.21.15.32:0/919095434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97ac101c30 0x7f97ac102000 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:07.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.639+0000 7f97b3532700 1 -- 172.21.15.32:0/919095434 >> 172.21.15.32:0/919095434 conn(0x7f97ac072b00 msgr2=0x7f97ac072f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:07.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.640+0000 7f97b3532700 1 -- 172.21.15.32:0/919095434 shutdown_connections 2024-05-12T04:12:07.641 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.640+0000 7f97b3532700 1 -- 172.21.15.32:0/919095434 wait complete. 2024-05-12T04:12:07.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.640+0000 7f97b3532700 1 Processor -- start 2024-05-12T04:12:07.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.640+0000 7f97b3532700 1 -- start start 2024-05-12T04:12:07.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.641+0000 7f97b3532700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97ac101c30 0x7f97ac112680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:07.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.641+0000 7f97b3532700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97ac1025d0 0x7f97ac112c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:07.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.641+0000 7f97b3532700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac113250 con 0x7f97ac1025d0 2024-05-12T04:12:07.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.641+0000 7f97b3532700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac10c810 con 0x7f97ac101c30 2024-05-12T04:12:07.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.641+0000 7f97b0acd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97ac1025d0 0x7f97ac112c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:07.643 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.641+0000 7f97b0acd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97ac1025d0 0x7f97ac112c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39396/0 (socket says 172.21.15.32:39396) 2024-05-12T04:12:07.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.641+0000 7f97b0acd700 1 -- 172.21.15.32:0/2103461688 learned_addr learned my addr 172.21.15.32:0/2103461688 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:12:07.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.641+0000 7f97b12ce700 1 --2- 172.21.15.32:0/2103461688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97ac101c30 0x7f97ac112680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:07.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.641+0000 7f97b0acd700 1 -- 172.21.15.32:0/2103461688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97ac101c30 msgr2=0x7f97ac112680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:07.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.641+0000 7f97b0acd700 1 --2- 172.21.15.32:0/2103461688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97ac101c30 0x7f97ac112680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:07.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.641+0000 7f97b0acd700 1 -- 172.21.15.32:0/2103461688 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a8005040 con 0x7f97ac1025d0 2024-05-12T04:12:07.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.641+0000 7f97b0acd700 1 --2- 172.21.15.32:0/2103461688 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97ac1025d0 0x7f97ac112c00 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f979c002a80 tx=0x7f979c004630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:07.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.641+0000 7f97a1ffb700 1 -- 172.21.15.32:0/2103461688 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f979c004e80 con 0x7f97ac1025d0 2024-05-12T04:12:07.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.641+0000 7f97a1ffb700 1 -- 172.21.15.32:0/2103461688 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f979c00f040 con 0x7f97ac1025d0 2024-05-12T04:12:07.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.641+0000 7f97b3532700 1 -- 172.21.15.32:0/2103461688 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97ac10caa0 con 0x7f97ac1025d0 2024-05-12T04:12:07.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.641+0000 7f97a1ffb700 1 -- 172.21.15.32:0/2103461688 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f979c00da00 con 0x7f97ac1025d0 2024-05-12T04:12:07.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.642+0000 7f97b3532700 1 -- 172.21.15.32:0/2103461688 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97ac10cfe0 con 0x7f97ac1025d0 2024-05-12T04:12:07.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.642+0000 7f97b3532700 1 -- 172.21.15.32:0/2103461688 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97ac0fef40 con 0x7f97ac1025d0 2024-05-12T04:12:07.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.643+0000 7f97a1ffb700 1 -- 172.21.15.32:0/2103461688 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f979c007cc0 con 0x7f97ac1025d0 2024-05-12T04:12:07.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.643+0000 7f97a1ffb700 1 --2- 172.21.15.32:0/2103461688 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f979806cc20 0x7f979806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:07.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.643+0000 7f97a1ffb700 1 -- 172.21.15.32:0/2103461688 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f979c08b720 con 0x7f97ac1025d0 2024-05-12T04:12:07.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.644+0000 7f97b12ce700 1 --2- 172.21.15.32:0/2103461688 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f979806cc20 0x7f979806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:07.645 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.644+0000 7f97b12ce700 1 --2- 172.21.15.32:0/2103461688 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f979806cc20 0x7f979806f0d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f97a8004890 tx=0x7f97a8009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:07.648 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.648+0000 7f97a1ffb700 1 -- 172.21.15.32:0/2103461688 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f979c059110 con 0x7f97ac1025d0 2024-05-12T04:12:07.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:07 smithi032 ceph-mon[120392]: pgmap v303: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:12:07.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:07 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/1715337504' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-05-12T04:12:07.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:07 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/2455948767' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-05-12T04:12:07.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.897+0000 7f97b3532700 1 -- 172.21.15.32:0/2103461688 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 32, "format": "json"} v 0) v1 -- 0x7f97ac0fd380 con 0x7f97ac1025d0 2024-05-12T04:12:07.898 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.898+0000 7f97a1ffb700 1 -- 172.21.15.32:0/2103461688 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 32, "format": "json"}]=0 dumped fsmap epoch 32 v37) v1 ==== 107+0+4045 (secure 0 0 0) 0x7f979c058ca0 con 0x7f97ac1025d0 2024-05-12T04:12:07.899 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:12:07.899 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34436,"name":"cephfs.smithi032.qpsawh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6829/3603452506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":3603452506},{"type":"v1","addr":"172.21.15.32:6829","nonce":3603452506}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":44343,"name":"cephfs.smithi033.jgbjhk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6825/997254871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":997254871},{"type":"v1","addr":"172.21.15.33:6825","nonce":997254871}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-12T03:57:48.904614+0000","modified":"2024-05-12T04:08:50.395294+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":34404},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34404":{"gid":34404,"name":"cephfs.smithi032.ialtqb","rank":0,"incarnation":32,"state":"up:replay","state_seq":1,"addr":"172.21.15.32:6827/1080358503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":1080358503},{"type":"v1","addr":"172.21.15.32:6827","nonce":1080358503}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-05-12T04:12:07.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.900+0000 7f97b3532700 1 -- 172.21.15.32:0/2103461688 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f979806cc20 msgr2=0x7f979806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:07.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.900+0000 7f97b3532700 1 --2- 172.21.15.32:0/2103461688 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f979806cc20 0x7f979806f0d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f97a8004890 tx=0x7f97a8009e30 comp rx=0 tx=0).stop 2024-05-12T04:12:07.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.900+0000 7f97b3532700 1 -- 172.21.15.32:0/2103461688 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97ac1025d0 msgr2=0x7f97ac112c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:07.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.900+0000 7f97b3532700 1 --2- 172.21.15.32:0/2103461688 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97ac1025d0 0x7f97ac112c00 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f979c002a80 tx=0x7f979c004630 comp rx=0 tx=0).stop 2024-05-12T04:12:07.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.900+0000 7f97b3532700 1 -- 172.21.15.32:0/2103461688 shutdown_connections 2024-05-12T04:12:07.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.900+0000 7f97b3532700 1 --2- 172.21.15.32:0/2103461688 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f979806cc20 0x7f979806f0d0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:07.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.900+0000 7f97b3532700 1 --2- 172.21.15.32:0/2103461688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97ac101c30 0x7f97ac112680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:07.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.900+0000 7f97b3532700 1 --2- 172.21.15.32:0/2103461688 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97ac1025d0 0x7f97ac112c00 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:07.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.900+0000 7f97b3532700 1 -- 172.21.15.32:0/2103461688 >> 172.21.15.32:0/2103461688 conn(0x7f97ac072b00 msgr2=0x7f97ac106220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:07.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.900+0000 7f97b3532700 1 -- 172.21.15.32:0/2103461688 shutdown_connections 2024-05-12T04:12:07.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:07.900+0000 7f97b3532700 1 -- 172.21.15.32:0/2103461688 wait complete. 2024-05-12T04:12:07.902 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 32 2024-05-12T04:12:07.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:07 smithi033 ceph-mon[107305]: pgmap v303: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:12:07.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:07 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/1715337504' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-05-12T04:12:07.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:07 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/2455948767' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-05-12T04:12:07.913 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 32 2024-05-12T04:12:07.913 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 33 2024-05-12T04:12:08.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.035+0000 7f995844e700 1 -- 172.21.15.32:0/4162691514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99501025d0 msgr2=0x7f99500fbc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:08.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.036+0000 7f995844e700 1 --2- 172.21.15.32:0/4162691514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99501025d0 0x7f99500fbc40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f994c004940 tx=0x7f994c009e30 comp rx=0 tx=0).stop 2024-05-12T04:12:08.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.036+0000 7f995844e700 1 -- 172.21.15.32:0/4162691514 shutdown_connections 2024-05-12T04:12:08.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.036+0000 7f995844e700 1 --2- 172.21.15.32:0/4162691514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99501025d0 0x7f99500fbc40 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:08.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.036+0000 7f995844e700 1 --2- 172.21.15.32:0/4162691514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9950101c30 0x7f9950102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:08.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.036+0000 7f995844e700 1 -- 172.21.15.32:0/4162691514 >> 172.21.15.32:0/4162691514 conn(0x7f9950072b00 msgr2=0x7f9950072f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:08.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.036+0000 7f995844e700 1 -- 172.21.15.32:0/4162691514 shutdown_connections 2024-05-12T04:12:08.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.036+0000 7f995844e700 1 -- 172.21.15.32:0/4162691514 wait complete. 2024-05-12T04:12:08.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.036+0000 7f995844e700 1 Processor -- start 2024-05-12T04:12:08.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.036+0000 7f995844e700 1 -- start start 2024-05-12T04:12:08.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.036+0000 7f995844e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9950101c30 0x7f9950112640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:08.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.036+0000 7f995844e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99501025d0 0x7f9950112bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:08.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.037+0000 7f995844e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9950113210 con 0x7f9950101c30 2024-05-12T04:12:08.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.037+0000 7f995844e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995010c7d0 con 0x7f99501025d0 2024-05-12T04:12:08.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.037+0000 7f99561ea700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9950101c30 0x7f9950112640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:08.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.037+0000 7f99561ea700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9950101c30 0x7f9950112640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39414/0 (socket says 172.21.15.32:39414) 2024-05-12T04:12:08.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.037+0000 7f99561ea700 1 -- 172.21.15.32:0/1218131769 learned_addr learned my addr 172.21.15.32:0/1218131769 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:12:08.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.037+0000 7f99559e9700 1 --2- 172.21.15.32:0/1218131769 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99501025d0 0x7f9950112bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:08.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.037+0000 7f99561ea700 1 -- 172.21.15.32:0/1218131769 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99501025d0 msgr2=0x7f9950112bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:08.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.037+0000 7f99561ea700 1 --2- 172.21.15.32:0/1218131769 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99501025d0 0x7f9950112bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:08.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.037+0000 7f99561ea700 1 -- 172.21.15.32:0/1218131769 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f994c005040 con 0x7f9950101c30 2024-05-12T04:12:08.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.037+0000 7f99561ea700 1 --2- 172.21.15.32:0/1218131769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9950101c30 0x7f9950112640 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9940002a50 tx=0x7f99400046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:08.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.037+0000 7f9946ffd700 1 -- 172.21.15.32:0/1218131769 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9940007cc0 con 0x7f9950101c30 2024-05-12T04:12:08.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.037+0000 7f9946ffd700 1 -- 172.21.15.32:0/1218131769 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f994000f040 con 0x7f9950101c30 2024-05-12T04:12:08.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.037+0000 7f995844e700 1 -- 172.21.15.32:0/1218131769 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f995010ca60 con 0x7f9950101c30 2024-05-12T04:12:08.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.037+0000 7f9946ffd700 1 -- 172.21.15.32:0/1218131769 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f994000d9c0 con 0x7f9950101c30 2024-05-12T04:12:08.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.037+0000 7f995844e700 1 -- 172.21.15.32:0/1218131769 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f995010cfa0 con 0x7f9950101c30 2024-05-12T04:12:08.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.038+0000 7f995844e700 1 -- 172.21.15.32:0/1218131769 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99500fef40 con 0x7f9950101c30 2024-05-12T04:12:08.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.039+0000 7f9946ffd700 1 -- 172.21.15.32:0/1218131769 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f994000db20 con 0x7f9950101c30 2024-05-12T04:12:08.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.039+0000 7f9946ffd700 1 --2- 172.21.15.32:0/1218131769 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f993c06cc20 0x7f993c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:08.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.039+0000 7f9946ffd700 1 -- 172.21.15.32:0/1218131769 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f994008be70 con 0x7f9950101c30 2024-05-12T04:12:08.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.039+0000 7f99559e9700 1 --2- 172.21.15.32:0/1218131769 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f993c06cc20 0x7f993c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:08.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.040+0000 7f99559e9700 1 --2- 172.21.15.32:0/1218131769 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f993c06cc20 0x7f993c06f0d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f995010dee0 tx=0x7f994c009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:08.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.044+0000 7f9946ffd700 1 -- 172.21.15.32:0/1218131769 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f99400597c0 con 0x7f9950101c30 2024-05-12T04:12:08.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.293+0000 7f995844e700 1 -- 172.21.15.32:0/1218131769 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 33, "format": "json"} v 0) v1 -- 0x7f995010dc90 con 0x7f9950101c30 2024-05-12T04:12:08.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.294+0000 7f9946ffd700 1 -- 172.21.15.32:0/1218131769 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 33, "format": "json"}]=0 dumped fsmap epoch 33 v37) v1 ==== 107+0+4824 (secure 0 0 0) 0x7f9940059350 con 0x7f9950101c30 2024-05-12T04:12:08.295 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:12:08.295 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34436,"name":"cephfs.smithi032.qpsawh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6829/3603452506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":3603452506},{"type":"v1","addr":"172.21.15.32:6829","nonce":3603452506}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":44343,"name":"cephfs.smithi033.jgbjhk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6825/997254871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":997254871},{"type":"v1","addr":"172.21.15.33:6825","nonce":997254871}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":44373,"name":"cephfs.smithi033.hglwml","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6827/1983593176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1983593176},{"type":"v1","addr":"172.21.15.33:6827","nonce":1983593176}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-12T03:57:48.904614+0000","modified":"2024-05-12T04:08:50.395294+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":34404},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34404":{"gid":34404,"name":"cephfs.smithi032.ialtqb","rank":0,"incarnation":32,"state":"up:replay","state_seq":1,"addr":"172.21.15.32:6827/1080358503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":1080358503},{"type":"v1","addr":"172.21.15.32:6827","nonce":1080358503}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-05-12T04:12:08.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.296+0000 7f995844e700 1 -- 172.21.15.32:0/1218131769 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f993c06cc20 msgr2=0x7f993c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:08.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.296+0000 7f995844e700 1 --2- 172.21.15.32:0/1218131769 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f993c06cc20 0x7f993c06f0d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f995010dee0 tx=0x7f994c009e30 comp rx=0 tx=0).stop 2024-05-12T04:12:08.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.296+0000 7f995844e700 1 -- 172.21.15.32:0/1218131769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9950101c30 msgr2=0x7f9950112640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:08.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.296+0000 7f995844e700 1 --2- 172.21.15.32:0/1218131769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9950101c30 0x7f9950112640 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9940002a50 tx=0x7f99400046b0 comp rx=0 tx=0).stop 2024-05-12T04:12:08.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.296+0000 7f995844e700 1 -- 172.21.15.32:0/1218131769 shutdown_connections 2024-05-12T04:12:08.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.296+0000 7f995844e700 1 --2- 172.21.15.32:0/1218131769 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f993c06cc20 0x7f993c06f0d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:08.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.296+0000 7f995844e700 1 --2- 172.21.15.32:0/1218131769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9950101c30 0x7f9950112640 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:08.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.296+0000 7f995844e700 1 --2- 172.21.15.32:0/1218131769 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99501025d0 0x7f9950112bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:08.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.296+0000 7f995844e700 1 -- 172.21.15.32:0/1218131769 >> 172.21.15.32:0/1218131769 conn(0x7f9950072b00 msgr2=0x7f99501061f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:08.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.296+0000 7f995844e700 1 -- 172.21.15.32:0/1218131769 shutdown_connections 2024-05-12T04:12:08.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.296+0000 7f995844e700 1 -- 172.21.15.32:0/1218131769 wait complete. 2024-05-12T04:12:08.298 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 33 2024-05-12T04:12:08.309 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 33 2024-05-12T04:12:08.309 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 34 2024-05-12T04:12:08.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.431+0000 7f7557d5f700 1 -- 172.21.15.32:0/4271000364 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7550102d80 msgr2=0x7f7550103150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:08.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.431+0000 7f7557d5f700 1 --2- 172.21.15.32:0/4271000364 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7550102d80 0x7f7550103150 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f754c004940 tx=0x7f754c009e30 comp rx=0 tx=0).stop 2024-05-12T04:12:08.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.431+0000 7f7557d5f700 1 -- 172.21.15.32:0/4271000364 shutdown_connections 2024-05-12T04:12:08.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.431+0000 7f7557d5f700 1 --2- 172.21.15.32:0/4271000364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7550064e40 0x7f75500652b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:08.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.431+0000 7f7557d5f700 1 --2- 172.21.15.32:0/4271000364 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7550102d80 0x7f7550103150 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:08.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.431+0000 7f7557d5f700 1 -- 172.21.15.32:0/4271000364 >> 172.21.15.32:0/4271000364 conn(0x7f7550072b00 msgr2=0x7f7550072f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:08.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.431+0000 7f7557d5f700 1 -- 172.21.15.32:0/4271000364 shutdown_connections 2024-05-12T04:12:08.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.431+0000 7f7557d5f700 1 -- 172.21.15.32:0/4271000364 wait complete. 2024-05-12T04:12:08.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.431+0000 7f7557d5f700 1 Processor -- start 2024-05-12T04:12:08.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.431+0000 7f7557d5f700 1 -- start start 2024-05-12T04:12:08.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.432+0000 7f7557d5f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7550064e40 0x7f75501136a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:08.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.432+0000 7f7557d5f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7550102d80 0x7f7550113be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:08.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.432+0000 7f7557d5f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7550114230 con 0x7f7550102d80 2024-05-12T04:12:08.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.432+0000 7f7557d5f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755010d850 con 0x7f7550064e40 2024-05-12T04:12:08.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.432+0000 7f75552fa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7550102d80 0x7f7550113be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:08.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.432+0000 7f75552fa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7550102d80 0x7f7550113be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39438/0 (socket says 172.21.15.32:39438) 2024-05-12T04:12:08.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.432+0000 7f75552fa700 1 -- 172.21.15.32:0/3964441511 learned_addr learned my addr 172.21.15.32:0/3964441511 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:12:08.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.432+0000 7f7555afb700 1 --2- 172.21.15.32:0/3964441511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7550064e40 0x7f75501136a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:08.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.432+0000 7f75552fa700 1 -- 172.21.15.32:0/3964441511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7550064e40 msgr2=0x7f75501136a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:08.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.432+0000 7f75552fa700 1 --2- 172.21.15.32:0/3964441511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7550064e40 0x7f75501136a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:08.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.432+0000 7f75552fa700 1 -- 172.21.15.32:0/3964441511 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f754c005040 con 0x7f7550102d80 2024-05-12T04:12:08.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.432+0000 7f75552fa700 1 --2- 172.21.15.32:0/3964441511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7550102d80 0x7f7550113be0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f754400c8c0 tx=0x7f75440044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:08.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.432+0000 7f75427fc700 1 -- 172.21.15.32:0/3964441511 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f754400a760 con 0x7f7550102d80 2024-05-12T04:12:08.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.432+0000 7f75427fc700 1 -- 172.21.15.32:0/3964441511 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f754400ad70 con 0x7f7550102d80 2024-05-12T04:12:08.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.432+0000 7f7557d5f700 1 -- 172.21.15.32:0/3964441511 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f755010db30 con 0x7f7550102d80 2024-05-12T04:12:08.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.432+0000 7f75427fc700 1 -- 172.21.15.32:0/3964441511 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f75440113a0 con 0x7f7550102d80 2024-05-12T04:12:08.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.432+0000 7f7557d5f700 1 -- 172.21.15.32:0/3964441511 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f755010e0f0 con 0x7f7550102d80 2024-05-12T04:12:08.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.433+0000 7f7557d5f700 1 -- 172.21.15.32:0/3964441511 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7550100050 con 0x7f7550102d80 2024-05-12T04:12:08.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.434+0000 7f75427fc700 1 -- 172.21.15.32:0/3964441511 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f7544012400 con 0x7f7550102d80 2024-05-12T04:12:08.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.434+0000 7f75427fc700 1 --2- 172.21.15.32:0/3964441511 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f753c06cc20 0x7f753c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:08.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.434+0000 7f75427fc700 1 -- 172.21.15.32:0/3964441511 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f754408c140 con 0x7f7550102d80 2024-05-12T04:12:08.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.434+0000 7f7555afb700 1 --2- 172.21.15.32:0/3964441511 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f753c06cc20 0x7f753c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:08.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.435+0000 7f7555afb700 1 --2- 172.21.15.32:0/3964441511 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f753c06cc20 0x7f753c06f0d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f754c004f90 tx=0x7f754c009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:08.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.439+0000 7f75427fc700 1 -- 172.21.15.32:0/3964441511 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f7544059bb0 con 0x7f7550102d80 2024-05-12T04:12:08.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.694+0000 7f7557d5f700 1 -- 172.21.15.32:0/3964441511 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 34, "format": "json"} v 0) v1 -- 0x7f75500fe490 con 0x7f7550102d80 2024-05-12T04:12:08.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.695+0000 7f75427fc700 1 -- 172.21.15.32:0/3964441511 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 34, "format": "json"}]=0 dumped fsmap epoch 34 v37) v1 ==== 107+0+4828 (secure 0 0 0) 0x7f7544059740 con 0x7f7550102d80 2024-05-12T04:12:08.695 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:12:08.695 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34436,"name":"cephfs.smithi032.qpsawh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6829/3603452506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":3603452506},{"type":"v1","addr":"172.21.15.32:6829","nonce":3603452506}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":44343,"name":"cephfs.smithi033.jgbjhk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6825/997254871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":997254871},{"type":"v1","addr":"172.21.15.33:6825","nonce":997254871}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":44373,"name":"cephfs.smithi033.hglwml","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6827/1983593176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1983593176},{"type":"v1","addr":"172.21.15.33:6827","nonce":1983593176}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-12T03:57:48.904614+0000","modified":"2024-05-12T04:09:00.004994+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":34404},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34404":{"gid":34404,"name":"cephfs.smithi032.ialtqb","rank":0,"incarnation":32,"state":"up:reconnect","state_seq":18,"addr":"172.21.15.32:6827/1080358503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":1080358503},{"type":"v1","addr":"172.21.15.32:6827","nonce":1080358503}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-05-12T04:12:08.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.697+0000 7f7557d5f700 1 -- 172.21.15.32:0/3964441511 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f753c06cc20 msgr2=0x7f753c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:08.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.697+0000 7f7557d5f700 1 --2- 172.21.15.32:0/3964441511 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f753c06cc20 0x7f753c06f0d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f754c004f90 tx=0x7f754c009e30 comp rx=0 tx=0).stop 2024-05-12T04:12:08.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.697+0000 7f7557d5f700 1 -- 172.21.15.32:0/3964441511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7550102d80 msgr2=0x7f7550113be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:08.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.697+0000 7f7557d5f700 1 --2- 172.21.15.32:0/3964441511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7550102d80 0x7f7550113be0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f754400c8c0 tx=0x7f75440044c0 comp rx=0 tx=0).stop 2024-05-12T04:12:08.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.697+0000 7f7557d5f700 1 -- 172.21.15.32:0/3964441511 shutdown_connections 2024-05-12T04:12:08.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.697+0000 7f7557d5f700 1 --2- 172.21.15.32:0/3964441511 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f753c06cc20 0x7f753c06f0d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:08.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.697+0000 7f7557d5f700 1 --2- 172.21.15.32:0/3964441511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7550064e40 0x7f75501136a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:08.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.697+0000 7f7557d5f700 1 --2- 172.21.15.32:0/3964441511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7550102d80 0x7f7550113be0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:08.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.697+0000 7f7557d5f700 1 -- 172.21.15.32:0/3964441511 >> 172.21.15.32:0/3964441511 conn(0x7f7550072b00 msgr2=0x7f75501072f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:08.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.697+0000 7f7557d5f700 1 -- 172.21.15.32:0/3964441511 shutdown_connections 2024-05-12T04:12:08.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.697+0000 7f7557d5f700 1 -- 172.21.15.32:0/3964441511 wait complete. 2024-05-12T04:12:08.699 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 34 2024-05-12T04:12:08.708 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:08 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/763483252' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-05-12T04:12:08.709 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:08 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/2103461688' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-05-12T04:12:08.709 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:08 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/1218131769' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-05-12T04:12:08.710 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 34 2024-05-12T04:12:08.710 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 35 2024-05-12T04:12:08.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.833+0000 7f8b6540b700 1 -- 172.21.15.32:0/1556886445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b60102cf0 msgr2=0x7f8b601030c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:08.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.833+0000 7f8b6540b700 1 --2- 172.21.15.32:0/1556886445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b60102cf0 0x7f8b601030c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8b50004940 tx=0x7f8b50009e30 comp rx=0 tx=0).stop 2024-05-12T04:12:08.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.833+0000 7f8b6540b700 1 -- 172.21.15.32:0/1556886445 shutdown_connections 2024-05-12T04:12:08.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.833+0000 7f8b6540b700 1 --2- 172.21.15.32:0/1556886445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b60103690 0x7f8b600fcd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:08.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.833+0000 7f8b6540b700 1 --2- 172.21.15.32:0/1556886445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b60102cf0 0x7f8b601030c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:08.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.833+0000 7f8b6540b700 1 -- 172.21.15.32:0/1556886445 >> 172.21.15.32:0/1556886445 conn(0x7f8b600f8800 msgr2=0x7f8b600fac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:08.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.833+0000 7f8b6540b700 1 -- 172.21.15.32:0/1556886445 shutdown_connections 2024-05-12T04:12:08.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.833+0000 7f8b6540b700 1 -- 172.21.15.32:0/1556886445 wait complete. 2024-05-12T04:12:08.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.834+0000 7f8b6540b700 1 Processor -- start 2024-05-12T04:12:08.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.834+0000 7f8b6540b700 1 -- start start 2024-05-12T04:12:08.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.834+0000 7f8b6540b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b60102cf0 0x7f8b60115990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:08.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.834+0000 7f8b6540b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b60103690 0x7f8b60115ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:08.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.834+0000 7f8b6540b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b60116520 con 0x7f8b60102cf0 2024-05-12T04:12:08.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.834+0000 7f8b6540b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6010fb20 con 0x7f8b60103690 2024-05-12T04:12:08.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.834+0000 7f8b5effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b60102cf0 0x7f8b60115990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:08.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.834+0000 7f8b5effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b60102cf0 0x7f8b60115990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39454/0 (socket says 172.21.15.32:39454) 2024-05-12T04:12:08.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.834+0000 7f8b5effd700 1 -- 172.21.15.32:0/2924170387 learned_addr learned my addr 172.21.15.32:0/2924170387 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:12:08.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.834+0000 7f8b5e7fc700 1 --2- 172.21.15.32:0/2924170387 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b60103690 0x7f8b60115ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:08.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.834+0000 7f8b5effd700 1 -- 172.21.15.32:0/2924170387 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b60103690 msgr2=0x7f8b60115ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:08.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.834+0000 7f8b5effd700 1 --2- 172.21.15.32:0/2924170387 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b60103690 0x7f8b60115ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:08.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.834+0000 7f8b5effd700 1 -- 172.21.15.32:0/2924170387 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b50005040 con 0x7f8b60102cf0 2024-05-12T04:12:08.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.834+0000 7f8b5e7fc700 1 --2- 172.21.15.32:0/2924170387 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b60103690 0x7f8b60115ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-12T04:12:08.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.834+0000 7f8b5effd700 1 --2- 172.21.15.32:0/2924170387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b60102cf0 0x7f8b60115990 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8b50009390 tx=0x7f8b50009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:08.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.835+0000 7f8b477fe700 1 -- 172.21.15.32:0/2924170387 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b5000e040 con 0x7f8b60102cf0 2024-05-12T04:12:08.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.835+0000 7f8b6540b700 1 -- 172.21.15.32:0/2924170387 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b6010fda0 con 0x7f8b60102cf0 2024-05-12T04:12:08.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.835+0000 7f8b477fe700 1 -- 172.21.15.32:0/2924170387 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8b50013420 con 0x7f8b60102cf0 2024-05-12T04:12:08.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.835+0000 7f8b477fe700 1 -- 172.21.15.32:0/2924170387 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b5001c630 con 0x7f8b60102cf0 2024-05-12T04:12:08.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.835+0000 7f8b6540b700 1 -- 172.21.15.32:0/2924170387 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b60110360 con 0x7f8b60102cf0 2024-05-12T04:12:08.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.836+0000 7f8b6540b700 1 -- 172.21.15.32:0/2924170387 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b600fe440 con 0x7f8b60102cf0 2024-05-12T04:12:08.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.836+0000 7f8b477fe700 1 -- 172.21.15.32:0/2924170387 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f8b500139e0 con 0x7f8b60102cf0 2024-05-12T04:12:08.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.837+0000 7f8b477fe700 1 --2- 172.21.15.32:0/2924170387 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f8b4806cc20 0x7f8b4806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:08.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.837+0000 7f8b477fe700 1 -- 172.21.15.32:0/2924170387 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f8b5008d310 con 0x7f8b60102cf0 2024-05-12T04:12:08.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.837+0000 7f8b5e7fc700 1 --2- 172.21.15.32:0/2924170387 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f8b4806cc20 0x7f8b4806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:08.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.837+0000 7f8b5e7fc700 1 --2- 172.21.15.32:0/2924170387 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f8b4806cc20 0x7f8b4806f0d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8b54007fd0 tx=0x7f8b54009040 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:08.841 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:08.841+0000 7f8b477fe700 1 -- 172.21.15.32:0/2924170387 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f8b5005ae30 con 0x7f8b60102cf0 2024-05-12T04:12:08.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:08 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/763483252' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-05-12T04:12:08.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:08 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/2103461688' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-05-12T04:12:08.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:08 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/1218131769' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-05-12T04:12:09.090 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.090+0000 7f8b6540b700 1 -- 172.21.15.32:0/2924170387 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 35, "format": "json"} v 0) v1 -- 0x7f8b60109d00 con 0x7f8b60102cf0 2024-05-12T04:12:09.091 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.091+0000 7f8b477fe700 1 -- 172.21.15.32:0/2924170387 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 35, "format": "json"}]=0 dumped fsmap epoch 35 v37) v1 ==== 107+0+4825 (secure 0 0 0) 0x7f8b5005a9c0 con 0x7f8b60102cf0 2024-05-12T04:12:09.091 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:12:09.091 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34436,"name":"cephfs.smithi032.qpsawh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6829/3603452506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":3603452506},{"type":"v1","addr":"172.21.15.32:6829","nonce":3603452506}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":44343,"name":"cephfs.smithi033.jgbjhk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6825/997254871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":997254871},{"type":"v1","addr":"172.21.15.33:6825","nonce":997254871}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":44373,"name":"cephfs.smithi033.hglwml","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6827/1983593176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1983593176},{"type":"v1","addr":"172.21.15.33:6827","nonce":1983593176}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-12T03:57:48.904614+0000","modified":"2024-05-12T04:09:01.008863+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":34404},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34404":{"gid":34404,"name":"cephfs.smithi032.ialtqb","rank":0,"incarnation":32,"state":"up:rejoin","state_seq":19,"addr":"172.21.15.32:6827/1080358503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":1080358503},{"type":"v1","addr":"172.21.15.32:6827","nonce":1080358503}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-05-12T04:12:09.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.093+0000 7f8b6540b700 1 -- 172.21.15.32:0/2924170387 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f8b4806cc20 msgr2=0x7f8b4806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:09.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.093+0000 7f8b6540b700 1 --2- 172.21.15.32:0/2924170387 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f8b4806cc20 0x7f8b4806f0d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8b54007fd0 tx=0x7f8b54009040 comp rx=0 tx=0).stop 2024-05-12T04:12:09.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.093+0000 7f8b6540b700 1 -- 172.21.15.32:0/2924170387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b60102cf0 msgr2=0x7f8b60115990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:09.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.093+0000 7f8b6540b700 1 --2- 172.21.15.32:0/2924170387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b60102cf0 0x7f8b60115990 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8b50009390 tx=0x7f8b50009e30 comp rx=0 tx=0).stop 2024-05-12T04:12:09.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.093+0000 7f8b6540b700 1 -- 172.21.15.32:0/2924170387 shutdown_connections 2024-05-12T04:12:09.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.093+0000 7f8b6540b700 1 --2- 172.21.15.32:0/2924170387 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f8b4806cc20 0x7f8b4806f0d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:09.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.093+0000 7f8b6540b700 1 --2- 172.21.15.32:0/2924170387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b60102cf0 0x7f8b60115990 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:09.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.093+0000 7f8b6540b700 1 --2- 172.21.15.32:0/2924170387 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b60103690 0x7f8b60115ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:09.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.093+0000 7f8b6540b700 1 -- 172.21.15.32:0/2924170387 >> 172.21.15.32:0/2924170387 conn(0x7f8b600f8800 msgr2=0x7f8b600f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:09.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.093+0000 7f8b6540b700 1 -- 172.21.15.32:0/2924170387 shutdown_connections 2024-05-12T04:12:09.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.093+0000 7f8b6540b700 1 -- 172.21.15.32:0/2924170387 wait complete. 2024-05-12T04:12:09.095 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 35 2024-05-12T04:12:09.105 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 35 2024-05-12T04:12:09.106 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 36 2024-05-12T04:12:09.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.228+0000 7f6d84e9c700 1 -- 172.21.15.32:0/1059409304 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d80101c30 msgr2=0x7f6d80102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:09.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.228+0000 7f6d84e9c700 1 --2- 172.21.15.32:0/1059409304 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d80101c30 0x7f6d80102000 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6d70004940 tx=0x7f6d70009e30 comp rx=0 tx=0).stop 2024-05-12T04:12:09.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.229+0000 7f6d84e9c700 1 -- 172.21.15.32:0/1059409304 shutdown_connections 2024-05-12T04:12:09.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.229+0000 7f6d84e9c700 1 --2- 172.21.15.32:0/1059409304 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d801025d0 0x7f6d800fbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:09.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.229+0000 7f6d84e9c700 1 --2- 172.21.15.32:0/1059409304 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d80101c30 0x7f6d80102000 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:09.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.229+0000 7f6d84e9c700 1 -- 172.21.15.32:0/1059409304 >> 172.21.15.32:0/1059409304 conn(0x7f6d80072b00 msgr2=0x7f6d80072f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:09.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.229+0000 7f6d84e9c700 1 -- 172.21.15.32:0/1059409304 shutdown_connections 2024-05-12T04:12:09.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.229+0000 7f6d84e9c700 1 -- 172.21.15.32:0/1059409304 wait complete. 2024-05-12T04:12:09.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.229+0000 7f6d84e9c700 1 Processor -- start 2024-05-12T04:12:09.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.229+0000 7f6d84e9c700 1 -- start start 2024-05-12T04:12:09.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.229+0000 7f6d84e9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d80101c30 0x7f6d8010e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:09.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.229+0000 7f6d84e9c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d801025d0 0x7f6d8010ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:09.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.229+0000 7f6d84e9c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8010f370 con 0x7f6d801025d0 2024-05-12T04:12:09.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.229+0000 7f6d84e9c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8010f4b0 con 0x7f6d80101c30 2024-05-12T04:12:09.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.229+0000 7f6d7dd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d801025d0 0x7f6d8010ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:09.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.230+0000 7f6d7dd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d801025d0 0x7f6d8010ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39464/0 (socket says 172.21.15.32:39464) 2024-05-12T04:12:09.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.230+0000 7f6d7dd9b700 1 -- 172.21.15.32:0/2091918066 learned_addr learned my addr 172.21.15.32:0/2091918066 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:12:09.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.230+0000 7f6d7e59c700 1 --2- 172.21.15.32:0/2091918066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d80101c30 0x7f6d8010e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:09.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.230+0000 7f6d7dd9b700 1 -- 172.21.15.32:0/2091918066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d80101c30 msgr2=0x7f6d8010e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:09.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.230+0000 7f6d7dd9b700 1 --2- 172.21.15.32:0/2091918066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d80101c30 0x7f6d8010e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:09.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.230+0000 7f6d7dd9b700 1 -- 172.21.15.32:0/2091918066 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d70005040 con 0x7f6d801025d0 2024-05-12T04:12:09.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.230+0000 7f6d7dd9b700 1 --2- 172.21.15.32:0/2091918066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d801025d0 0x7f6d8010ee30 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6d74002a50 tx=0x7f6d74004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:09.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.230+0000 7f6d6effd700 1 -- 172.21.15.32:0/2091918066 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d74007cc0 con 0x7f6d801025d0 2024-05-12T04:12:09.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.230+0000 7f6d84e9c700 1 -- 172.21.15.32:0/2091918066 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d80117960 con 0x7f6d801025d0 2024-05-12T04:12:09.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.230+0000 7f6d6effd700 1 -- 172.21.15.32:0/2091918066 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6d7400f040 con 0x7f6d801025d0 2024-05-12T04:12:09.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.230+0000 7f6d6effd700 1 -- 172.21.15.32:0/2091918066 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d7400d920 con 0x7f6d801025d0 2024-05-12T04:12:09.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.230+0000 7f6d84e9c700 1 -- 172.21.15.32:0/2091918066 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d80117d30 con 0x7f6d801025d0 2024-05-12T04:12:09.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.231+0000 7f6d84e9c700 1 -- 172.21.15.32:0/2091918066 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d800fef40 con 0x7f6d801025d0 2024-05-12T04:12:09.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.232+0000 7f6d6effd700 1 -- 172.21.15.32:0/2091918066 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f6d74009680 con 0x7f6d801025d0 2024-05-12T04:12:09.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.232+0000 7f6d6effd700 1 --2- 172.21.15.32:0/2091918066 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f6d6806cc20 0x7f6d6806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:09.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.232+0000 7f6d6effd700 1 -- 172.21.15.32:0/2091918066 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f6d7408be40 con 0x7f6d801025d0 2024-05-12T04:12:09.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.232+0000 7f6d7e59c700 1 --2- 172.21.15.32:0/2091918066 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f6d6806cc20 0x7f6d6806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:09.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.232+0000 7f6d7e59c700 1 --2- 172.21.15.32:0/2091918066 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f6d6806cc20 0x7f6d6806f0d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6d7000b840 tx=0x7f6d70009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:09.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.236+0000 7f6d6effd700 1 -- 172.21.15.32:0/2091918066 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f6d74055da0 con 0x7f6d801025d0 2024-05-12T04:12:09.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.487+0000 7f6d84e9c700 1 -- 172.21.15.32:0/2091918066 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 36, "format": "json"} v 0) v1 -- 0x7f6d80065360 con 0x7f6d801025d0 2024-05-12T04:12:09.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.487+0000 7f6d6effd700 1 -- 172.21.15.32:0/2091918066 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 36, "format": "json"}]=0 dumped fsmap epoch 36 v37) v1 ==== 107+0+4825 (secure 0 0 0) 0x7f6d740593c0 con 0x7f6d801025d0 2024-05-12T04:12:09.488 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:12:09.488 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34436,"name":"cephfs.smithi032.qpsawh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6829/3603452506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":3603452506},{"type":"v1","addr":"172.21.15.32:6829","nonce":3603452506}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":44343,"name":"cephfs.smithi033.jgbjhk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6825/997254871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":997254871},{"type":"v1","addr":"172.21.15.33:6825","nonce":997254871}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":44373,"name":"cephfs.smithi033.hglwml","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6827/1983593176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1983593176},{"type":"v1","addr":"172.21.15.33:6827","nonce":1983593176}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-12T03:57:48.904614+0000","modified":"2024-05-12T04:09:02.017947+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":34404},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34404":{"gid":34404,"name":"cephfs.smithi032.ialtqb","rank":0,"incarnation":32,"state":"up:active","state_seq":20,"addr":"172.21.15.32:6827/1080358503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":1080358503},{"type":"v1","addr":"172.21.15.32:6827","nonce":1080358503}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-05-12T04:12:09.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.490+0000 7f6d84e9c700 1 -- 172.21.15.32:0/2091918066 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f6d6806cc20 msgr2=0x7f6d6806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:09.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.490+0000 7f6d84e9c700 1 --2- 172.21.15.32:0/2091918066 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f6d6806cc20 0x7f6d6806f0d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6d7000b840 tx=0x7f6d70009e30 comp rx=0 tx=0).stop 2024-05-12T04:12:09.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.490+0000 7f6d84e9c700 1 -- 172.21.15.32:0/2091918066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d801025d0 msgr2=0x7f6d8010ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:09.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.490+0000 7f6d84e9c700 1 --2- 172.21.15.32:0/2091918066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d801025d0 0x7f6d8010ee30 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6d74002a50 tx=0x7f6d74004600 comp rx=0 tx=0).stop 2024-05-12T04:12:09.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.490+0000 7f6d84e9c700 1 -- 172.21.15.32:0/2091918066 shutdown_connections 2024-05-12T04:12:09.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.490+0000 7f6d84e9c700 1 --2- 172.21.15.32:0/2091918066 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f6d6806cc20 0x7f6d6806f0d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:09.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.490+0000 7f6d84e9c700 1 --2- 172.21.15.32:0/2091918066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d80101c30 0x7f6d8010e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:09.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.490+0000 7f6d84e9c700 1 --2- 172.21.15.32:0/2091918066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d801025d0 0x7f6d8010ee30 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:09.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.490+0000 7f6d84e9c700 1 -- 172.21.15.32:0/2091918066 >> 172.21.15.32:0/2091918066 conn(0x7f6d80072b00 msgr2=0x7f6d80106220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:09.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.490+0000 7f6d84e9c700 1 -- 172.21.15.32:0/2091918066 shutdown_connections 2024-05-12T04:12:09.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.490+0000 7f6d84e9c700 1 -- 172.21.15.32:0/2091918066 wait complete. 2024-05-12T04:12:09.491 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 36 2024-05-12T04:12:09.502 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 36 2024-05-12T04:12:09.503 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-05-12T04:12:09.515 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-05-12T04:12:09.516 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-05-12T04:12:09.516 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-12T04:12:09.536 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-05-12T04:12:09.536 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-12T04:12:09.596 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-12T04:12:09.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.757+0000 7f6ce8749700 1 -- 172.21.15.32:0/2997841279 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ce0101360 msgr2=0x7f6ce0101730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:09.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.757+0000 7f6ce8749700 1 --2- 172.21.15.32:0/2997841279 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ce0101360 0x7f6ce0101730 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0004940 tx=0x7f6cd0009e30 comp rx=0 tx=0).stop 2024-05-12T04:12:09.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.757+0000 7f6ce8749700 1 -- 172.21.15.32:0/2997841279 shutdown_connections 2024-05-12T04:12:09.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.757+0000 7f6ce8749700 1 --2- 172.21.15.32:0/2997841279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ce0101d00 0x7f6ce00fb370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:09.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.757+0000 7f6ce8749700 1 --2- 172.21.15.32:0/2997841279 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ce0101360 0x7f6ce0101730 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:09.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.757+0000 7f6ce8749700 1 -- 172.21.15.32:0/2997841279 >> 172.21.15.32:0/2997841279 conn(0x7f6ce0071310 msgr2=0x7f6ce0071710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:09.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.757+0000 7f6ce8749700 1 -- 172.21.15.32:0/2997841279 shutdown_connections 2024-05-12T04:12:09.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.757+0000 7f6ce8749700 1 -- 172.21.15.32:0/2997841279 wait complete. 2024-05-12T04:12:09.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.757+0000 7f6ce8749700 1 Processor -- start 2024-05-12T04:12:09.760 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.758+0000 7f6ce8749700 1 -- start start 2024-05-12T04:12:09.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.758+0000 7f6ce8749700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ce0101360 0x7f6ce01136b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:09.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.758+0000 7f6ce8749700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ce0101d00 0x7f6ce0113bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:09.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.758+0000 7f6ce8749700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce0114240 con 0x7f6ce0101d00 2024-05-12T04:12:09.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.758+0000 7f6ce8749700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce010d860 con 0x7f6ce0101360 2024-05-12T04:12:09.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.758+0000 7f6ce5ce4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ce0101d00 0x7f6ce0113bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:09.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.758+0000 7f6ce5ce4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ce0101d00 0x7f6ce0113bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39474/0 (socket says 172.21.15.32:39474) 2024-05-12T04:12:09.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.758+0000 7f6ce5ce4700 1 -- 172.21.15.32:0/3615553650 learned_addr learned my addr 172.21.15.32:0/3615553650 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:12:09.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.758+0000 7f6ce64e5700 1 --2- 172.21.15.32:0/3615553650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ce0101360 0x7f6ce01136b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:09.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.758+0000 7f6ce5ce4700 1 -- 172.21.15.32:0/3615553650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ce0101360 msgr2=0x7f6ce01136b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:09.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.758+0000 7f6ce5ce4700 1 --2- 172.21.15.32:0/3615553650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ce0101360 0x7f6ce01136b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:09.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.758+0000 7f6ce5ce4700 1 -- 172.21.15.32:0/3615553650 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cd0005040 con 0x7f6ce0101d00 2024-05-12T04:12:09.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.758+0000 7f6ce5ce4700 1 --2- 172.21.15.32:0/3615553650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ce0101d00 0x7f6ce0113bf0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc00eb40 tx=0x7f6cdc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:09.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.758+0000 7f6cd6ffd700 1 -- 172.21.15.32:0/3615553650 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cdc007cc0 con 0x7f6ce0101d00 2024-05-12T04:12:09.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.759+0000 7f6ce8749700 1 -- 172.21.15.32:0/3615553650 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ce010dae0 con 0x7f6ce0101d00 2024-05-12T04:12:09.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.759+0000 7f6cd6ffd700 1 -- 172.21.15.32:0/3615553650 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6cdc00f040 con 0x7f6ce0101d00 2024-05-12T04:12:09.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.759+0000 7f6cd6ffd700 1 -- 172.21.15.32:0/3615553650 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cdc00c8f0 con 0x7f6ce0101d00 2024-05-12T04:12:09.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.759+0000 7f6ce8749700 1 -- 172.21.15.32:0/3615553650 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ce010e0a0 con 0x7f6ce0101d00 2024-05-12T04:12:09.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.759+0000 7f6ce8749700 1 -- 172.21.15.32:0/3615553650 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ce00fcab0 con 0x7f6ce0101d00 2024-05-12T04:12:09.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.760+0000 7f6cd6ffd700 1 -- 172.21.15.32:0/3615553650 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f6cdc00cb60 con 0x7f6ce0101d00 2024-05-12T04:12:09.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.760+0000 7f6cd6ffd700 1 --2- 172.21.15.32:0/3615553650 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f6ccc06cc20 0x7f6ccc06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:09.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.760+0000 7f6cd6ffd700 1 -- 172.21.15.32:0/3615553650 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f6cdc08bbe0 con 0x7f6ce0101d00 2024-05-12T04:12:09.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.761+0000 7f6ce64e5700 1 --2- 172.21.15.32:0/3615553650 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f6ccc06cc20 0x7f6ccc06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:09.763 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.761+0000 7f6ce64e5700 1 --2- 172.21.15.32:0/3615553650 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f6ccc06cc20 0x7f6ccc06f0d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6cd000b0d0 tx=0x7f6cd0009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:09.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.765+0000 7f6cd6ffd700 1 -- 172.21.15.32:0/3615553650 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f6cdc0595e0 con 0x7f6ce0101d00 2024-05-12T04:12:09.789 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:09 smithi032 ceph-mon[120392]: pgmap v304: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:12:09.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:09 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/3964441511' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-05-12T04:12:09.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:09 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/2924170387' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-05-12T04:12:09.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:09 smithi033 ceph-mon[107305]: pgmap v304: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:12:09.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:09 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/3964441511' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-05-12T04:12:09.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:09 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/2924170387' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-05-12T04:12:09.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.970+0000 7f6ce8749700 1 -- 172.21.15.32:0/3615553650 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6ce00640c0 con 0x7f6ce0101d00 2024-05-12T04:12:09.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.971+0000 7f6cd6ffd700 1 -- 172.21.15.32:0/3615553650 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16628 (secure 0 0 0) 0x7f6cdc059170 con 0x7f6ce0101d00 2024-05-12T04:12:09.972 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:12:09.972 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":84,"fsid":"a3e178f8-1012-11ef-bc98-c7b262605968","created":"2024-05-12T03:50:18.356038+0000","modified":"2024-05-12T04:08:50.380422+0000","last_up_change":"2024-05-12T04:07:44.234383+0000","last_in_change":"2024-05-12T03:56:03.301729+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-12T03:54:58.296702+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-05-12T03:57:47.465323+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-05-12T03:57:48.324643+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fc534129-ef70-487d-b8d1-f98fc170c1d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":49,"up_from":53,"up_thru":78,"down_at":50,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1676344197},{"type":"v1","addr":"172.21.15.32:6803","nonce":1676344197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1676344197},{"type":"v1","addr":"172.21.15.32:6805","nonce":1676344197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1676344197},{"type":"v1","addr":"172.21.15.32:6809","nonce":1676344197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1676344197},{"type":"v1","addr":"172.21.15.32:6807","nonce":1676344197}]},"public_addr":"172.21.15.32:6803/1676344197","cluster_addr":"172.21.15.32:6805/1676344197","heartbeat_back_addr":"172.21.15.32:6809/1676344197","heartbeat_front_addr":"172.21.15.32:6807/1676344197","state":["exists","up"]},{"osd":1,"uuid":"f8c35b6a-5afc-447b-b318-d56097de28d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":54,"up_from":58,"up_thru":78,"down_at":55,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":954580115},{"type":"v1","addr":"172.21.15.32:6811","nonce":954580115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":954580115},{"type":"v1","addr":"172.21.15.32:6813","nonce":954580115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":954580115},{"type":"v1","addr":"172.21.15.32:6817","nonce":954580115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":954580115},{"type":"v1","addr":"172.21.15.32:6815","nonce":954580115}]},"public_addr":"172.21.15.32:6811/954580115","cluster_addr":"172.21.15.32:6813/954580115","heartbeat_back_addr":"172.21.15.32:6817/954580115","heartbeat_front_addr":"172.21.15.32:6815/954580115","state":["exists","up"]},{"osd":2,"uuid":"4ed4b55d-cceb-494b-bdff-fef99d6c8cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":59,"up_from":63,"up_thru":78,"down_at":60,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":793261683},{"type":"v1","addr":"172.21.15.32:6819","nonce":793261683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":793261683},{"type":"v1","addr":"172.21.15.32:6821","nonce":793261683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":793261683},{"type":"v1","addr":"172.21.15.32:6825","nonce":793261683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":793261683},{"type":"v1","addr":"172.21.15.32:6823","nonce":793261683}]},"public_addr":"172.21.15.32:6819/793261683","cluster_addr":"172.21.15.32:6821/793261683","heartbeat_back_addr":"172.21.15.32:6825/793261683","heartbeat_front_addr":"172.21.15.32:6823/793261683","state":["exists","up"]},{"osd":3,"uuid":"ef191267-c0a1-4be7-9e8b-761700af87a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":64,"up_from":68,"up_thru":78,"down_at":65,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":3436738456},{"type":"v1","addr":"172.21.15.33:6801","nonce":3436738456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":3436738456},{"type":"v1","addr":"172.21.15.33:6803","nonce":3436738456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":3436738456},{"type":"v1","addr":"172.21.15.33:6807","nonce":3436738456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":3436738456},{"type":"v1","addr":"172.21.15.33:6805","nonce":3436738456}]},"public_addr":"172.21.15.33:6801/3436738456","cluster_addr":"172.21.15.33:6803/3436738456","heartbeat_back_addr":"172.21.15.33:6807/3436738456","heartbeat_front_addr":"172.21.15.33:6805/3436738456","state":["exists","up"]},{"osd":4,"uuid":"fe45f168-076a-4888-895d-bb710bd0da73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":69,"up_from":73,"up_thru":78,"down_at":70,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1028896705},{"type":"v1","addr":"172.21.15.33:6809","nonce":1028896705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1028896705},{"type":"v1","addr":"172.21.15.33:6811","nonce":1028896705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1028896705},{"type":"v1","addr":"172.21.15.33:6815","nonce":1028896705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1028896705},{"type":"v1","addr":"172.21.15.33:6813","nonce":1028896705}]},"public_addr":"172.21.15.33:6809/1028896705","cluster_addr":"172.21.15.33:6811/1028896705","heartbeat_back_addr":"172.21.15.33:6815/1028896705","heartbeat_front_addr":"172.21.15.33:6813/1028896705","state":["exists","up"]},{"osd":5,"uuid":"f7a0e14e-a866-4163-a7df-f8a43b860bc0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":74,"up_from":78,"up_thru":78,"down_at":75,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":4274440090},{"type":"v1","addr":"172.21.15.33:6817","nonce":4274440090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":4274440090},{"type":"v1","addr":"172.21.15.33:6819","nonce":4274440090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":4274440090},{"type":"v1","addr":"172.21.15.33:6823","nonce":4274440090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":4274440090},{"type":"v1","addr":"172.21.15.33:6821","nonce":4274440090}]},"public_addr":"172.21.15.33:6817/4274440090","cluster_addr":"172.21.15.33:6819/4274440090","heartbeat_back_addr":"172.21.15.33:6823/4274440090","heartbeat_front_addr":"172.21.15.33:6821/4274440090","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-12T03:53:56.134194+0000","dead_epoch":49},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-12T03:54:25.231975+0000","dead_epoch":54},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-12T03:54:54.991052+0000","dead_epoch":59},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-12T03:55:22.680656+0000","dead_epoch":64},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-12T03:55:52.446678+0000","dead_epoch":69},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-12T03:56:22.128957+0000","dead_epoch":74}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6829/3100155296":"2024-05-13T04:08:08.971776+0000","172.21.15.32:6828/3100155296":"2024-05-13T04:08:08.971776+0000","172.21.15.32:0/912074455":"2024-05-13T04:03:15.687820+0000","172.21.15.32:6800/1121526649":"2024-05-13T03:50:49.378828+0000","172.21.15.32:0/3343483246":"2024-05-13T04:00:09.761962+0000","172.21.15.32:6827/1907818042":"2024-05-13T03:58:02.805639+0000","172.21.15.32:0/4293156271":"2024-05-13T03:51:25.269581+0000","172.21.15.32:0/2900983644":"2024-05-13T03:52:39.772717+0000","172.21.15.32:0/2236672133":"2024-05-13T04:00:09.761962+0000","172.21.15.32:6800/2967201790":"2024-05-13T03:51:25.269581+0000","172.21.15.33:6826/433858238":"2024-05-13T04:08:50.380388+0000","172.21.15.32:6826/1907818042":"2024-05-13T03:58:02.805639+0000","172.21.15.32:6801/1121526649":"2024-05-13T03:50:49.378828+0000","172.21.15.32:0/2098237559":"2024-05-13T03:50:49.378828+0000","172.21.15.32:6800/4089797100":"2024-05-13T04:00:09.761962+0000","172.21.15.32:0/1463926533":"2024-05-13T03:50:49.378828+0000","172.21.15.32:0/1834879835":"2024-05-13T04:00:09.761962+0000","172.21.15.32:0/165225041":"2024-05-13T03:51:25.269581+0000","172.21.15.32:0/625670655":"2024-05-13T03:50:49.378828+0000","172.21.15.33:6827/433858238":"2024-05-13T04:08:50.380388+0000","172.21.15.32:0/3185268518":"2024-05-13T03:52:39.772717+0000","172.21.15.33:6829/1598726962":"2024-05-13T04:02:10.132488+0000","172.21.15.32:0/1073406471":"2024-05-13T03:52:39.772717+0000","172.21.15.32:6801/4089797100":"2024-05-13T04:00:09.761962+0000","172.21.15.32:0/1356801576":"2024-05-13T03:51:25.269581+0000","172.21.15.32:0/993128492":"2024-05-13T04:00:09.761962+0000","172.21.15.32:0/2523500808":"2024-05-13T04:03:15.687820+0000","172.21.15.32:6801/2967201790":"2024-05-13T03:51:25.269581+0000","172.21.15.32:6800/3729453095":"2024-05-13T04:03:15.687820+0000","172.21.15.33:6828/1598726962":"2024-05-13T04:02:10.132488+0000","172.21.15.32:0/2757703284":"2024-05-13T04:03:15.687820+0000","172.21.15.33:0/2636219823":"2024-05-13T04:02:10.132488+0000","172.21.15.33:0/2129212605":"2024-05-13T04:02:10.132488+0000","172.21.15.33:0/619425673":"2024-05-13T04:02:10.132488+0000","172.21.15.33:0/3980223680":"2024-05-13T04:02:10.132488+0000","172.21.15.32:6800/1988399583":"2024-05-13T03:52:39.772717+0000","172.21.15.32:0/213713458":"2024-05-13T04:03:15.687820+0000","172.21.15.32:6801/1988399583":"2024-05-13T03:52:39.772717+0000","172.21.15.32:6801/3729453095":"2024-05-13T04:03:15.687820+0000","172.21.15.32:0/3906624315":"2024-05-13T04:03:15.687820+0000","172.21.15.32:0/3077610654":"2024-05-13T04:03:15.687820+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-12T04:12:09.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.973+0000 7f6ce8749700 1 -- 172.21.15.32:0/3615553650 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f6ccc06cc20 msgr2=0x7f6ccc06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:09.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.973+0000 7f6ce8749700 1 --2- 172.21.15.32:0/3615553650 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f6ccc06cc20 0x7f6ccc06f0d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6cd000b0d0 tx=0x7f6cd0009e30 comp rx=0 tx=0).stop 2024-05-12T04:12:09.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.973+0000 7f6ce8749700 1 -- 172.21.15.32:0/3615553650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ce0101d00 msgr2=0x7f6ce0113bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:09.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.973+0000 7f6ce8749700 1 --2- 172.21.15.32:0/3615553650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ce0101d00 0x7f6ce0113bf0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc00eb40 tx=0x7f6cdc00b040 comp rx=0 tx=0).stop 2024-05-12T04:12:09.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.973+0000 7f6ce8749700 1 -- 172.21.15.32:0/3615553650 shutdown_connections 2024-05-12T04:12:09.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.973+0000 7f6ce8749700 1 --2- 172.21.15.32:0/3615553650 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f6ccc06cc20 0x7f6ccc06f0d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:09.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.973+0000 7f6ce8749700 1 --2- 172.21.15.32:0/3615553650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ce0101360 0x7f6ce01136b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:09.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.973+0000 7f6ce8749700 1 --2- 172.21.15.32:0/3615553650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ce0101d00 0x7f6ce0113bf0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:09.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.973+0000 7f6ce8749700 1 -- 172.21.15.32:0/3615553650 >> 172.21.15.32:0/3615553650 conn(0x7f6ce0071310 msgr2=0x7f6ce00f96b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:09.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.973+0000 7f6ce8749700 1 -- 172.21.15.32:0/3615553650 shutdown_connections 2024-05-12T04:12:09.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:09.973+0000 7f6ce8749700 1 -- 172.21.15.32:0/3615553650 wait complete. 2024-05-12T04:12:09.986 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-12T03:54:58.296702+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-05-12T03:57:47.465323+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-05-12T03:57:48.324643+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-12T04:12:09.986 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-12T04:12:10.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.111+0000 7f0628e2b700 1 -- 172.21.15.32:0/933363718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0624102540 msgr2=0x7f06240fbbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:10.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.111+0000 7f0628e2b700 1 --2- 172.21.15.32:0/933363718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0624102540 0x7f06240fbbb0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0614004940 tx=0x7f0614009e30 comp rx=0 tx=0).stop 2024-05-12T04:12:10.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.111+0000 7f0628e2b700 1 -- 172.21.15.32:0/933363718 shutdown_connections 2024-05-12T04:12:10.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.111+0000 7f0628e2b700 1 --2- 172.21.15.32:0/933363718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0624102540 0x7f06240fbbb0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:10.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.111+0000 7f0628e2b700 1 --2- 172.21.15.32:0/933363718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0624101ba0 0x7f0624101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:10.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.111+0000 7f0628e2b700 1 -- 172.21.15.32:0/933363718 >> 172.21.15.32:0/933363718 conn(0x7f0624072ad0 msgr2=0x7f0624072ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:10.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.111+0000 7f0628e2b700 1 -- 172.21.15.32:0/933363718 shutdown_connections 2024-05-12T04:12:10.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.111+0000 7f0628e2b700 1 -- 172.21.15.32:0/933363718 wait complete. 2024-05-12T04:12:10.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.112+0000 7f0628e2b700 1 Processor -- start 2024-05-12T04:12:10.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.112+0000 7f0628e2b700 1 -- start start 2024-05-12T04:12:10.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.112+0000 7f0628e2b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0624101ba0 0x7f06241126a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:10.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.112+0000 7f0628e2b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0624102540 0x7f0624112be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:10.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.112+0000 7f0628e2b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0624113230 con 0x7f0624102540 2024-05-12T04:12:10.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.112+0000 7f0628e2b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062410c7f0 con 0x7f0624101ba0 2024-05-12T04:12:10.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.112+0000 7f0621d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0624102540 0x7f0624112be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:10.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.112+0000 7f0621d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0624102540 0x7f0624112be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39488/0 (socket says 172.21.15.32:39488) 2024-05-12T04:12:10.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.112+0000 7f0621d9b700 1 -- 172.21.15.32:0/1283611471 learned_addr learned my addr 172.21.15.32:0/1283611471 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:12:10.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.112+0000 7f062259c700 1 --2- 172.21.15.32:0/1283611471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0624101ba0 0x7f06241126a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:10.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.113+0000 7f0621d9b700 1 -- 172.21.15.32:0/1283611471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0624101ba0 msgr2=0x7f06241126a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:10.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.113+0000 7f0621d9b700 1 --2- 172.21.15.32:0/1283611471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0624101ba0 0x7f06241126a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:10.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.113+0000 7f0621d9b700 1 -- 172.21.15.32:0/1283611471 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0614005040 con 0x7f0624102540 2024-05-12T04:12:10.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.113+0000 7f062259c700 1 --2- 172.21.15.32:0/1283611471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0624101ba0 0x7f06241126a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-12T04:12:10.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.113+0000 7f0621d9b700 1 --2- 172.21.15.32:0/1283611471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0624102540 0x7f0624112be0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0614009380 tx=0x7f0614009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:10.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.113+0000 7f0612ffd700 1 -- 172.21.15.32:0/1283611471 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f061400e040 con 0x7f0624102540 2024-05-12T04:12:10.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.113+0000 7f0612ffd700 1 -- 172.21.15.32:0/1283611471 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0614013420 con 0x7f0624102540 2024-05-12T04:12:10.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.113+0000 7f0628e2b700 1 -- 172.21.15.32:0/1283611471 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f062410ca70 con 0x7f0624102540 2024-05-12T04:12:10.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.113+0000 7f0612ffd700 1 -- 172.21.15.32:0/1283611471 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f061401c630 con 0x7f0624102540 2024-05-12T04:12:10.117 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.113+0000 7f0628e2b700 1 -- 172.21.15.32:0/1283611471 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f062410cfb0 con 0x7f0624102540 2024-05-12T04:12:10.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.114+0000 7f0628e2b700 1 -- 172.21.15.32:0/1283611471 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0624112330 con 0x7f0624102540 2024-05-12T04:12:10.118 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.115+0000 7f0612ffd700 1 -- 172.21.15.32:0/1283611471 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f06140139e0 con 0x7f0624102540 2024-05-12T04:12:10.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.115+0000 7f0612ffd700 1 --2- 172.21.15.32:0/1283611471 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f060c06cc20 0x7f060c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:10.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.116+0000 7f0612ffd700 1 -- 172.21.15.32:0/1283611471 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f061408ddc0 con 0x7f0624102540 2024-05-12T04:12:10.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.116+0000 7f062259c700 1 --2- 172.21.15.32:0/1283611471 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f060c06cc20 0x7f060c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:10.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.116+0000 7f062259c700 1 --2- 172.21.15.32:0/1283611471 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f060c06cc20 0x7f060c06f0d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f06180041d0 tx=0x7f0618009040 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:10.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.120+0000 7f0612ffd700 1 -- 172.21.15.32:0/1283611471 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f061405b810 con 0x7f0624102540 2024-05-12T04:12:10.332 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.332+0000 7f0628e2b700 1 -- 172.21.15.32:0/1283611471 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0624108b30 con 0x7f0624102540 2024-05-12T04:12:10.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.332+0000 7f0612ffd700 1 -- 172.21.15.32:0/1283611471 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f061405b3a0 con 0x7f0624102540 2024-05-12T04:12:10.333 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-05-12T04:12:10.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.335+0000 7f0628e2b700 1 -- 172.21.15.32:0/1283611471 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f060c06cc20 msgr2=0x7f060c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:10.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.335+0000 7f0628e2b700 1 --2- 172.21.15.32:0/1283611471 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f060c06cc20 0x7f060c06f0d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f06180041d0 tx=0x7f0618009040 comp rx=0 tx=0).stop 2024-05-12T04:12:10.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.335+0000 7f0628e2b700 1 -- 172.21.15.32:0/1283611471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0624102540 msgr2=0x7f0624112be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:10.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.335+0000 7f0628e2b700 1 --2- 172.21.15.32:0/1283611471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0624102540 0x7f0624112be0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0614009380 tx=0x7f0614009e30 comp rx=0 tx=0).stop 2024-05-12T04:12:10.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.335+0000 7f0628e2b700 1 -- 172.21.15.32:0/1283611471 shutdown_connections 2024-05-12T04:12:10.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.335+0000 7f0628e2b700 1 --2- 172.21.15.32:0/1283611471 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f060c06cc20 0x7f060c06f0d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:10.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.335+0000 7f0628e2b700 1 --2- 172.21.15.32:0/1283611471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0624101ba0 0x7f06241126a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:10.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.335+0000 7f0628e2b700 1 --2- 172.21.15.32:0/1283611471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0624102540 0x7f0624112be0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:10.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.335+0000 7f0628e2b700 1 -- 172.21.15.32:0/1283611471 >> 172.21.15.32:0/1283611471 conn(0x7f0624072ad0 msgr2=0x7f0624104940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:10.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.335+0000 7f0628e2b700 1 -- 172.21.15.32:0/1283611471 shutdown_connections 2024-05-12T04:12:10.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.335+0000 7f0628e2b700 1 -- 172.21.15.32:0/1283611471 wait complete. 2024-05-12T04:12:10.347 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-05-12T04:12:10.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.472+0000 7fd1b8e7f700 1 -- 172.21.15.32:0/306925526 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1b4064e40 msgr2=0x7fd1b40652b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:10.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.472+0000 7fd1b8e7f700 1 --2- 172.21.15.32:0/306925526 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1b4064e40 0x7fd1b40652b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8004940 tx=0x7fd1a8009e30 comp rx=0 tx=0).stop 2024-05-12T04:12:10.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.472+0000 7fd1b8e7f700 1 -- 172.21.15.32:0/306925526 shutdown_connections 2024-05-12T04:12:10.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.472+0000 7fd1b8e7f700 1 --2- 172.21.15.32:0/306925526 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1b4064e40 0x7fd1b40652b0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:10.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.472+0000 7fd1b8e7f700 1 --2- 172.21.15.32:0/306925526 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1b4102cf0 0x7fd1b41030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:10.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.472+0000 7fd1b8e7f700 1 -- 172.21.15.32:0/306925526 >> 172.21.15.32:0/306925526 conn(0x7fd1b4072ad0 msgr2=0x7fd1b4072ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:10.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.472+0000 7fd1b8e7f700 1 -- 172.21.15.32:0/306925526 shutdown_connections 2024-05-12T04:12:10.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.472+0000 7fd1b8e7f700 1 -- 172.21.15.32:0/306925526 wait complete. 2024-05-12T04:12:10.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.473+0000 7fd1b8e7f700 1 Processor -- start 2024-05-12T04:12:10.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.473+0000 7fd1b8e7f700 1 -- start start 2024-05-12T04:12:10.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.473+0000 7fd1b8e7f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1b4064e40 0x7fd1b41136c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:10.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.473+0000 7fd1b8e7f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1b4102cf0 0x7fd1b4113c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:10.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.473+0000 7fd1b8e7f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b4114290 con 0x7fd1b4102cf0 2024-05-12T04:12:10.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.473+0000 7fd1b8e7f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b410d850 con 0x7fd1b4064e40 2024-05-12T04:12:10.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.473+0000 7fd1b1d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1b4102cf0 0x7fd1b4113c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:10.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.473+0000 7fd1b1d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1b4102cf0 0x7fd1b4113c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39506/0 (socket says 172.21.15.32:39506) 2024-05-12T04:12:10.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.473+0000 7fd1b1d9b700 1 -- 172.21.15.32:0/874147887 learned_addr learned my addr 172.21.15.32:0/874147887 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:12:10.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.473+0000 7fd1b259c700 1 --2- 172.21.15.32:0/874147887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1b4064e40 0x7fd1b41136c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:10.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.474+0000 7fd1b1d9b700 1 -- 172.21.15.32:0/874147887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1b4064e40 msgr2=0x7fd1b41136c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:10.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.474+0000 7fd1b1d9b700 1 --2- 172.21.15.32:0/874147887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1b4064e40 0x7fd1b41136c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:10.475 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.474+0000 7fd1b1d9b700 1 -- 172.21.15.32:0/874147887 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1a8005040 con 0x7fd1b4102cf0 2024-05-12T04:12:10.476 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.474+0000 7fd1b1d9b700 1 --2- 172.21.15.32:0/874147887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1b4102cf0 0x7fd1b4113c40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8000c00 tx=0x7fd1a8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:10.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.474+0000 7fd1a2ffd700 1 -- 172.21.15.32:0/874147887 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd1a8021070 con 0x7fd1b4102cf0 2024-05-12T04:12:10.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.474+0000 7fd1a2ffd700 1 -- 172.21.15.32:0/874147887 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd1a80093b0 con 0x7fd1b4102cf0 2024-05-12T04:12:10.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.474+0000 7fd1b8e7f700 1 -- 172.21.15.32:0/874147887 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1b410da80 con 0x7fd1b4102cf0 2024-05-12T04:12:10.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.475+0000 7fd1a2ffd700 1 -- 172.21.15.32:0/874147887 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd1a800e070 con 0x7fd1b4102cf0 2024-05-12T04:12:10.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.475+0000 7fd1b8e7f700 1 -- 172.21.15.32:0/874147887 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1b410dfc0 con 0x7fd1b4102cf0 2024-05-12T04:12:10.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.475+0000 7fd1b8e7f700 1 -- 172.21.15.32:0/874147887 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1b40fe400 con 0x7fd1b4102cf0 2024-05-12T04:12:10.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.476+0000 7fd1a2ffd700 1 -- 172.21.15.32:0/874147887 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fd1a801c490 con 0x7fd1b4102cf0 2024-05-12T04:12:10.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.476+0000 7fd1a2ffd700 1 --2- 172.21.15.32:0/874147887 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fd19c06cc20 0x7fd19c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:10.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.476+0000 7fd1a2ffd700 1 -- 172.21.15.32:0/874147887 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fd1a808d060 con 0x7fd1b4102cf0 2024-05-12T04:12:10.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.476+0000 7fd1b259c700 1 --2- 172.21.15.32:0/874147887 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fd19c06cc20 0x7fd19c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:10.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.477+0000 7fd1b259c700 1 --2- 172.21.15.32:0/874147887 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fd19c06cc20 0x7fd19c06f0d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4073080 tx=0x7fd1a400b040 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:10.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.481+0000 7fd1a2ffd700 1 -- 172.21.15.32:0/874147887 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fd1a805ab70 con 0x7fd1b4102cf0 2024-05-12T04:12:10.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.682+0000 7fd1b8e7f700 1 -- 172.21.15.32:0/874147887 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7fd1b404c710 con 0x7fd1b4102cf0 2024-05-12T04:12:10.682 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.682+0000 7fd1a2ffd700 1 -- 172.21.15.32:0/874147887 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v84) v1 ==== 107+0+11 (secure 0 0 0) 0x7fd1a805a700 con 0x7fd1b4102cf0 2024-05-12T04:12:10.685 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-05-12T04:12:10.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.684+0000 7fd1b8e7f700 1 -- 172.21.15.32:0/874147887 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fd19c06cc20 msgr2=0x7fd19c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:10.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.684+0000 7fd1b8e7f700 1 --2- 172.21.15.32:0/874147887 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fd19c06cc20 0x7fd19c06f0d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4073080 tx=0x7fd1a400b040 comp rx=0 tx=0).stop 2024-05-12T04:12:10.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.684+0000 7fd1b8e7f700 1 -- 172.21.15.32:0/874147887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1b4102cf0 msgr2=0x7fd1b4113c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:10.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.684+0000 7fd1b8e7f700 1 --2- 172.21.15.32:0/874147887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1b4102cf0 0x7fd1b4113c40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8000c00 tx=0x7fd1a8009e30 comp rx=0 tx=0).stop 2024-05-12T04:12:10.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.684+0000 7fd1b8e7f700 1 -- 172.21.15.32:0/874147887 shutdown_connections 2024-05-12T04:12:10.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.684+0000 7fd1b8e7f700 1 --2- 172.21.15.32:0/874147887 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fd19c06cc20 0x7fd19c06f0d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:10.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.684+0000 7fd1b8e7f700 1 --2- 172.21.15.32:0/874147887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1b4064e40 0x7fd1b41136c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:10.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.684+0000 7fd1b8e7f700 1 --2- 172.21.15.32:0/874147887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1b4102cf0 0x7fd1b4113c40 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:10.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.685+0000 7fd1b8e7f700 1 -- 172.21.15.32:0/874147887 >> 172.21.15.32:0/874147887 conn(0x7fd1b4072ad0 msgr2=0x7fd1b4100870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:10.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.685+0000 7fd1b8e7f700 1 -- 172.21.15.32:0/874147887 shutdown_connections 2024-05-12T04:12:10.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.685+0000 7fd1b8e7f700 1 -- 172.21.15.32:0/874147887 wait complete. 2024-05-12T04:12:10.696 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:10 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/2091918066' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-05-12T04:12:10.696 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:10 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/3615553650' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-12T04:12:10.696 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:10 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/1283611471' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-12T04:12:10.697 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-05-12T04:12:10.824 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.823+0000 7f42d6142700 1 -- 172.21.15.32:0/3997964519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42d0064e40 msgr2=0x7f42d0065210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:10.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.823+0000 7f42d6142700 1 --2- 172.21.15.32:0/3997964519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42d0064e40 0x7f42d0065210 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f42c4004940 tx=0x7f42c4009e30 comp rx=0 tx=0).stop 2024-05-12T04:12:10.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.823+0000 7f42d6142700 1 -- 172.21.15.32:0/3997964519 shutdown_connections 2024-05-12T04:12:10.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.823+0000 7f42d6142700 1 --2- 172.21.15.32:0/3997964519 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42d0065750 0x7f42d0109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:10.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.823+0000 7f42d6142700 1 --2- 172.21.15.32:0/3997964519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42d0064e40 0x7f42d0065210 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:10.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.823+0000 7f42d6142700 1 -- 172.21.15.32:0/3997964519 >> 172.21.15.32:0/3997964519 conn(0x7f42d0072ad0 msgr2=0x7f42d0072ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:10.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.823+0000 7f42d6142700 1 -- 172.21.15.32:0/3997964519 shutdown_connections 2024-05-12T04:12:10.825 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.823+0000 7f42d6142700 1 -- 172.21.15.32:0/3997964519 wait complete. 2024-05-12T04:12:10.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.823+0000 7f42d6142700 1 Processor -- start 2024-05-12T04:12:10.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.823+0000 7f42d6142700 1 -- start start 2024-05-12T04:12:10.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.824+0000 7f42d6142700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42d0064e40 0x7f42d0115900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:10.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.824+0000 7f42d6142700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42d0065750 0x7f42d0115e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:10.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.824+0000 7f42d6142700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d0116490 con 0x7f42d0064e40 2024-05-12T04:12:10.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.824+0000 7f42d6142700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d010fae0 con 0x7f42d0065750 2024-05-12T04:12:10.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.824+0000 7f42cf7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42d0064e40 0x7f42d0115900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:10.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.824+0000 7f42cf7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42d0064e40 0x7f42d0115900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39516/0 (socket says 172.21.15.32:39516) 2024-05-12T04:12:10.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.824+0000 7f42cf7fe700 1 -- 172.21.15.32:0/3083877088 learned_addr learned my addr 172.21.15.32:0/3083877088 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:12:10.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.824+0000 7f42ceffd700 1 --2- 172.21.15.32:0/3083877088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42d0065750 0x7f42d0115e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:10.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.824+0000 7f42cf7fe700 1 -- 172.21.15.32:0/3083877088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42d0065750 msgr2=0x7f42d0115e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:10.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.824+0000 7f42cf7fe700 1 --2- 172.21.15.32:0/3083877088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42d0065750 0x7f42d0115e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:10.826 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.824+0000 7f42cf7fe700 1 -- 172.21.15.32:0/3083877088 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42c4005040 con 0x7f42d0064e40 2024-05-12T04:12:10.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.824+0000 7f42cf7fe700 1 --2- 172.21.15.32:0/3083877088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42d0064e40 0x7f42d0115900 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f42c4000c00 tx=0x7f42c4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:10.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.824+0000 7f42b7fff700 1 -- 172.21.15.32:0/3083877088 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f42c4021070 con 0x7f42d0064e40 2024-05-12T04:12:10.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.824+0000 7f42b7fff700 1 -- 172.21.15.32:0/3083877088 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f42c4013e00 con 0x7f42d0064e40 2024-05-12T04:12:10.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.824+0000 7f42d6142700 1 -- 172.21.15.32:0/3083877088 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42d010fd60 con 0x7f42d0064e40 2024-05-12T04:12:10.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.824+0000 7f42b7fff700 1 -- 172.21.15.32:0/3083877088 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f42c400e070 con 0x7f42d0064e40 2024-05-12T04:12:10.827 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.824+0000 7f42d6142700 1 -- 172.21.15.32:0/3083877088 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42d0110320 con 0x7f42d0064e40 2024-05-12T04:12:10.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.825+0000 7f42d6142700 1 -- 172.21.15.32:0/3083877088 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42d01068b0 con 0x7f42d0064e40 2024-05-12T04:12:10.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.826+0000 7f42b7fff700 1 -- 172.21.15.32:0/3083877088 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f42c401c690 con 0x7f42d0064e40 2024-05-12T04:12:10.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.826+0000 7f42b7fff700 1 --2- 172.21.15.32:0/3083877088 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f42b806cc20 0x7f42b806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:10.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.826+0000 7f42b7fff700 1 -- 172.21.15.32:0/3083877088 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f42c408d1c0 con 0x7f42d0064e40 2024-05-12T04:12:10.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.826+0000 7f42ceffd700 1 --2- 172.21.15.32:0/3083877088 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f42b806cc20 0x7f42b806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:10.828 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.827+0000 7f42ceffd700 1 --2- 172.21.15.32:0/3083877088 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f42b806cc20 0x7f42b806f0d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f42c0004600 tx=0x7f42c000b040 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:10.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:10.831+0000 7f42b7fff700 1 -- 172.21.15.32:0/3083877088 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f42c405ac30 con 0x7f42d0064e40 2024-05-12T04:12:10.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:10 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/2091918066' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-05-12T04:12:10.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:10 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/3615553650' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-12T04:12:10.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:10 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/1283611471' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-12T04:12:11.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.035+0000 7f42d6142700 1 -- 172.21.15.32:0/3083877088 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f42d0110f50 con 0x7f42d0064e40 2024-05-12T04:12:11.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.036+0000 7f42b7fff700 1 -- 172.21.15.32:0/3083877088 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v84) v1 ==== 107+0+11 (secure 0 0 0) 0x7f42c405a7c0 con 0x7f42d0064e40 2024-05-12T04:12:11.036 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-05-12T04:12:11.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.038+0000 7f42d6142700 1 -- 172.21.15.32:0/3083877088 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f42b806cc20 msgr2=0x7f42b806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:11.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.038+0000 7f42d6142700 1 --2- 172.21.15.32:0/3083877088 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f42b806cc20 0x7f42b806f0d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f42c0004600 tx=0x7f42c000b040 comp rx=0 tx=0).stop 2024-05-12T04:12:11.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.038+0000 7f42d6142700 1 -- 172.21.15.32:0/3083877088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42d0064e40 msgr2=0x7f42d0115900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:11.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.038+0000 7f42d6142700 1 --2- 172.21.15.32:0/3083877088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42d0064e40 0x7f42d0115900 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f42c4000c00 tx=0x7f42c4009e30 comp rx=0 tx=0).stop 2024-05-12T04:12:11.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.038+0000 7f42d6142700 1 -- 172.21.15.32:0/3083877088 shutdown_connections 2024-05-12T04:12:11.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.038+0000 7f42d6142700 1 --2- 172.21.15.32:0/3083877088 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f42b806cc20 0x7f42b806f0d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:11.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.038+0000 7f42d6142700 1 --2- 172.21.15.32:0/3083877088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42d0064e40 0x7f42d0115900 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:11.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.038+0000 7f42d6142700 1 --2- 172.21.15.32:0/3083877088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42d0065750 0x7f42d0115e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:11.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.038+0000 7f42d6142700 1 -- 172.21.15.32:0/3083877088 >> 172.21.15.32:0/3083877088 conn(0x7f42d0072ad0 msgr2=0x7f42d00f96d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:11.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.038+0000 7f42d6142700 1 -- 172.21.15.32:0/3083877088 shutdown_connections 2024-05-12T04:12:11.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.038+0000 7f42d6142700 1 -- 172.21.15.32:0/3083877088 wait complete. 2024-05-12T04:12:11.051 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-12T04:12:11.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.174+0000 7ff27d64d700 1 -- 172.21.15.32:0/3439211314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff278101d10 msgr2=0x7ff2781020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:11.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.174+0000 7ff27d64d700 1 --2- 172.21.15.32:0/3439211314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff278101d10 0x7ff2781020e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff268004940 tx=0x7ff268009e30 comp rx=0 tx=0).stop 2024-05-12T04:12:11.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.174+0000 7ff27d64d700 1 -- 172.21.15.32:0/3439211314 shutdown_connections 2024-05-12T04:12:11.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.174+0000 7ff27d64d700 1 --2- 172.21.15.32:0/3439211314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2780fc3f0 0x7ff2780fc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:11.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.174+0000 7ff27d64d700 1 --2- 172.21.15.32:0/3439211314 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff278101d10 0x7ff2781020e0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:11.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.174+0000 7ff27d64d700 1 -- 172.21.15.32:0/3439211314 >> 172.21.15.32:0/3439211314 conn(0x7ff2780722a0 msgr2=0x7ff2780726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:11.176 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.174+0000 7ff27d64d700 1 -- 172.21.15.32:0/3439211314 shutdown_connections 2024-05-12T04:12:11.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.174+0000 7ff27d64d700 1 -- 172.21.15.32:0/3439211314 wait complete. 2024-05-12T04:12:11.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.174+0000 7ff27d64d700 1 Processor -- start 2024-05-12T04:12:11.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.175+0000 7ff27d64d700 1 -- start start 2024-05-12T04:12:11.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.175+0000 7ff27d64d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2780fc3f0 0x7ff27810e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:11.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.175+0000 7ff27d64d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff278101d10 0x7ff27810edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:11.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.175+0000 7ff27d64d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27810f330 con 0x7ff278101d10 2024-05-12T04:12:11.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.175+0000 7ff27d64d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27810f470 con 0x7ff2780fc3f0 2024-05-12T04:12:11.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.175+0000 7ff2767fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff278101d10 0x7ff27810edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:11.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.175+0000 7ff2767fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff278101d10 0x7ff27810edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39534/0 (socket says 172.21.15.32:39534) 2024-05-12T04:12:11.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.175+0000 7ff2767fc700 1 -- 172.21.15.32:0/4175802848 learned_addr learned my addr 172.21.15.32:0/4175802848 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:12:11.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.175+0000 7ff276ffd700 1 --2- 172.21.15.32:0/4175802848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2780fc3f0 0x7ff27810e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:11.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.175+0000 7ff2767fc700 1 -- 172.21.15.32:0/4175802848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2780fc3f0 msgr2=0x7ff27810e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:11.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.175+0000 7ff2767fc700 1 --2- 172.21.15.32:0/4175802848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2780fc3f0 0x7ff27810e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:11.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.175+0000 7ff2767fc700 1 -- 172.21.15.32:0/4175802848 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff268005040 con 0x7ff278101d10 2024-05-12T04:12:11.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.175+0000 7ff2767fc700 1 --2- 172.21.15.32:0/4175802848 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff278101d10 0x7ff27810edf0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff26000c830 tx=0x7ff260004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:11.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.175+0000 7ff26f7fe700 1 -- 172.21.15.32:0/4175802848 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2600106f0 con 0x7ff278101d10 2024-05-12T04:12:11.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.176+0000 7ff27d64d700 1 -- 172.21.15.32:0/4175802848 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff278117920 con 0x7ff278101d10 2024-05-12T04:12:11.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.176+0000 7ff26f7fe700 1 -- 172.21.15.32:0/4175802848 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff260010d00 con 0x7ff278101d10 2024-05-12T04:12:11.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.176+0000 7ff26f7fe700 1 -- 172.21.15.32:0/4175802848 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff260012e90 con 0x7ff278101d10 2024-05-12T04:12:11.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.176+0000 7ff27d64d700 1 -- 172.21.15.32:0/4175802848 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff278117cf0 con 0x7ff278101d10 2024-05-12T04:12:11.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.177+0000 7ff27d64d700 1 -- 172.21.15.32:0/4175802848 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2780fd400 con 0x7ff278101d10 2024-05-12T04:12:11.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.177+0000 7ff26f7fe700 1 -- 172.21.15.32:0/4175802848 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7ff260010850 con 0x7ff278101d10 2024-05-12T04:12:11.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.178+0000 7ff26f7fe700 1 --2- 172.21.15.32:0/4175802848 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7ff26406cc20 0x7ff26406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:11.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.178+0000 7ff26f7fe700 1 -- 172.21.15.32:0/4175802848 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7ff26008c070 con 0x7ff278101d10 2024-05-12T04:12:11.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.178+0000 7ff276ffd700 1 --2- 172.21.15.32:0/4175802848 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7ff26406cc20 0x7ff26406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:11.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.178+0000 7ff276ffd700 1 --2- 172.21.15.32:0/4175802848 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7ff26406cc20 0x7ff26406f0d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff268004560 tx=0x7ff268009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:11.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.182+0000 7ff26f7fe700 1 -- 172.21.15.32:0/4175802848 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7ff260056050 con 0x7ff278101d10 2024-05-12T04:12:11.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.404+0000 7ff27d64d700 1 -- 172.21.15.32:0/4175802848 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff27810f630 con 0x7ff278101d10 2024-05-12T04:12:11.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.404+0000 7ff26f7fe700 1 -- 172.21.15.32:0/4175802848 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 41 entries v84) v1 ==== 81+0+2424 (secure 0 0 0) 0x7ff260059670 con 0x7ff278101d10 2024-05-12T04:12:11.405 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6829/3100155296 2024-05-13T04:08:08.971776+0000 2024-05-12T04:12:11.405 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6828/3100155296 2024-05-13T04:08:08.971776+0000 2024-05-12T04:12:11.405 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/912074455 2024-05-13T04:03:15.687820+0000 2024-05-12T04:12:11.405 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6800/1121526649 2024-05-13T03:50:49.378828+0000 2024-05-12T04:12:11.405 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/3343483246 2024-05-13T04:00:09.761962+0000 2024-05-12T04:12:11.405 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6827/1907818042 2024-05-13T03:58:02.805639+0000 2024-05-12T04:12:11.405 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/4293156271 2024-05-13T03:51:25.269581+0000 2024-05-12T04:12:11.405 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2900983644 2024-05-13T03:52:39.772717+0000 2024-05-12T04:12:11.405 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2236672133 2024-05-13T04:00:09.761962+0000 2024-05-12T04:12:11.405 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6800/2967201790 2024-05-13T03:51:25.269581+0000 2024-05-12T04:12:11.405 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.33:6826/433858238 2024-05-13T04:08:50.380388+0000 2024-05-12T04:12:11.405 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6826/1907818042 2024-05-13T03:58:02.805639+0000 2024-05-12T04:12:11.405 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6801/1121526649 2024-05-13T03:50:49.378828+0000 2024-05-12T04:12:11.406 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2098237559 2024-05-13T03:50:49.378828+0000 2024-05-12T04:12:11.406 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6800/4089797100 2024-05-13T04:00:09.761962+0000 2024-05-12T04:12:11.406 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/1463926533 2024-05-13T03:50:49.378828+0000 2024-05-12T04:12:11.406 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/1834879835 2024-05-13T04:00:09.761962+0000 2024-05-12T04:12:11.406 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/165225041 2024-05-13T03:51:25.269581+0000 2024-05-12T04:12:11.406 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/625670655 2024-05-13T03:50:49.378828+0000 2024-05-12T04:12:11.406 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.33:6827/433858238 2024-05-13T04:08:50.380388+0000 2024-05-12T04:12:11.406 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/3185268518 2024-05-13T03:52:39.772717+0000 2024-05-12T04:12:11.406 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.33:6829/1598726962 2024-05-13T04:02:10.132488+0000 2024-05-12T04:12:11.406 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/1073406471 2024-05-13T03:52:39.772717+0000 2024-05-12T04:12:11.406 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6801/4089797100 2024-05-13T04:00:09.761962+0000 2024-05-12T04:12:11.406 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/1356801576 2024-05-13T03:51:25.269581+0000 2024-05-12T04:12:11.406 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/993128492 2024-05-13T04:00:09.761962+0000 2024-05-12T04:12:11.407 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2523500808 2024-05-13T04:03:15.687820+0000 2024-05-12T04:12:11.407 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6801/2967201790 2024-05-13T03:51:25.269581+0000 2024-05-12T04:12:11.407 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6800/3729453095 2024-05-13T04:03:15.687820+0000 2024-05-12T04:12:11.407 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.33:6828/1598726962 2024-05-13T04:02:10.132488+0000 2024-05-12T04:12:11.407 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2757703284 2024-05-13T04:03:15.687820+0000 2024-05-12T04:12:11.407 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.33:0/2636219823 2024-05-13T04:02:10.132488+0000 2024-05-12T04:12:11.407 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.33:0/2129212605 2024-05-13T04:02:10.132488+0000 2024-05-12T04:12:11.407 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.33:0/619425673 2024-05-13T04:02:10.132488+0000 2024-05-12T04:12:11.407 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.33:0/3980223680 2024-05-13T04:02:10.132488+0000 2024-05-12T04:12:11.407 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6800/1988399583 2024-05-13T03:52:39.772717+0000 2024-05-12T04:12:11.407 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/213713458 2024-05-13T04:03:15.687820+0000 2024-05-12T04:12:11.407 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6801/1988399583 2024-05-13T03:52:39.772717+0000 2024-05-12T04:12:11.407 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6801/3729453095 2024-05-13T04:03:15.687820+0000 2024-05-12T04:12:11.407 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/3906624315 2024-05-13T04:03:15.687820+0000 2024-05-12T04:12:11.408 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/3077610654 2024-05-13T04:03:15.687820+0000 2024-05-12T04:12:11.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.406+0000 7ff27d64d700 1 -- 172.21.15.32:0/4175802848 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7ff26406cc20 msgr2=0x7ff26406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:11.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.406+0000 7ff27d64d700 1 --2- 172.21.15.32:0/4175802848 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7ff26406cc20 0x7ff26406f0d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff268004560 tx=0x7ff268009e30 comp rx=0 tx=0).stop 2024-05-12T04:12:11.408 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.406+0000 7ff27d64d700 1 -- 172.21.15.32:0/4175802848 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff278101d10 msgr2=0x7ff27810edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:11.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.406+0000 7ff27d64d700 1 --2- 172.21.15.32:0/4175802848 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff278101d10 0x7ff27810edf0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff26000c830 tx=0x7ff260004470 comp rx=0 tx=0).stop 2024-05-12T04:12:11.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.406+0000 7ff27d64d700 1 -- 172.21.15.32:0/4175802848 shutdown_connections 2024-05-12T04:12:11.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.406+0000 7ff27d64d700 1 --2- 172.21.15.32:0/4175802848 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7ff26406cc20 0x7ff26406f0d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:11.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.406+0000 7ff27d64d700 1 --2- 172.21.15.32:0/4175802848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2780fc3f0 0x7ff27810e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:11.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.406+0000 7ff27d64d700 1 --2- 172.21.15.32:0/4175802848 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff278101d10 0x7ff27810edf0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:11.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.406+0000 7ff27d64d700 1 -- 172.21.15.32:0/4175802848 >> 172.21.15.32:0/4175802848 conn(0x7ff2780722a0 msgr2=0x7ff2780f9530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:11.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.406+0000 7ff27d64d700 1 -- 172.21.15.32:0/4175802848 shutdown_connections 2024-05-12T04:12:11.409 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.406+0000 7ff27d64d700 1 -- 172.21.15.32:0/4175802848 wait complete. 2024-05-12T04:12:11.409 INFO:teuthology.orchestra.run.smithi032.stderr:listed 41 entries 2024-05-12T04:12:11.419 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-05-12T04:12:11.419 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-12T04:12:11.439 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-12T04:12:11.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.601+0000 7fd6b2652700 1 -- 172.21.15.32:0/2060949961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6ac101bf0 msgr2=0x7fd6ac101fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:11.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.601+0000 7fd6b2652700 1 --2- 172.21.15.32:0/2060949961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6ac101bf0 0x7fd6ac101fc0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd69c004940 tx=0x7fd69c009e30 comp rx=0 tx=0).stop 2024-05-12T04:12:11.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.601+0000 7fd6b2652700 1 -- 172.21.15.32:0/2060949961 shutdown_connections 2024-05-12T04:12:11.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.601+0000 7fd6b2652700 1 --2- 172.21.15.32:0/2060949961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6ac102590 0x7fd6ac0fbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:11.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.601+0000 7fd6b2652700 1 --2- 172.21.15.32:0/2060949961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6ac101bf0 0x7fd6ac101fc0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:11.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.601+0000 7fd6b2652700 1 -- 172.21.15.32:0/2060949961 >> 172.21.15.32:0/2060949961 conn(0x7fd6ac071bb0 msgr2=0x7fd6ac071fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:11.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.601+0000 7fd6b2652700 1 -- 172.21.15.32:0/2060949961 shutdown_connections 2024-05-12T04:12:11.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.601+0000 7fd6b2652700 1 -- 172.21.15.32:0/2060949961 wait complete. 2024-05-12T04:12:11.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.602+0000 7fd6b2652700 1 Processor -- start 2024-05-12T04:12:11.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.602+0000 7fd6b2652700 1 -- start start 2024-05-12T04:12:11.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.602+0000 7fd6b2652700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6ac101bf0 0x7fd6ac113710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:11.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.602+0000 7fd6b2652700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6ac102590 0x7fd6ac113c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:11.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.602+0000 7fd6b2652700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac1142a0 con 0x7fd6ac102590 2024-05-12T04:12:11.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.602+0000 7fd6b2652700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac10d8c0 con 0x7fd6ac101bf0 2024-05-12T04:12:11.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.602+0000 7fd6ab7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6ac102590 0x7fd6ac113c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:11.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.602+0000 7fd6ab7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6ac102590 0x7fd6ac113c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39544/0 (socket says 172.21.15.32:39544) 2024-05-12T04:12:11.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.602+0000 7fd6ab7fe700 1 -- 172.21.15.32:0/3086615264 learned_addr learned my addr 172.21.15.32:0/3086615264 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:12:11.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.602+0000 7fd6abfff700 1 --2- 172.21.15.32:0/3086615264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6ac101bf0 0x7fd6ac113710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:11.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.603+0000 7fd6ab7fe700 1 -- 172.21.15.32:0/3086615264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6ac101bf0 msgr2=0x7fd6ac113710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:11.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.603+0000 7fd6ab7fe700 1 --2- 172.21.15.32:0/3086615264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6ac101bf0 0x7fd6ac113710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:11.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.603+0000 7fd6ab7fe700 1 -- 172.21.15.32:0/3086615264 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd69c005040 con 0x7fd6ac102590 2024-05-12T04:12:11.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.603+0000 7fd6ab7fe700 1 --2- 172.21.15.32:0/3086615264 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6ac102590 0x7fd6ac113c50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd6a000eb40 tx=0x7fd6a000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:11.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.603+0000 7fd6a8ff9700 1 -- 172.21.15.32:0/3086615264 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6a0007cc0 con 0x7fd6ac102590 2024-05-12T04:12:11.605 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.603+0000 7fd6a8ff9700 1 -- 172.21.15.32:0/3086615264 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd6a000f040 con 0x7fd6ac102590 2024-05-12T04:12:11.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.603+0000 7fd6b2652700 1 -- 172.21.15.32:0/3086615264 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6ac10db40 con 0x7fd6ac102590 2024-05-12T04:12:11.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.603+0000 7fd6a8ff9700 1 -- 172.21.15.32:0/3086615264 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6a000ca30 con 0x7fd6ac102590 2024-05-12T04:12:11.606 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.603+0000 7fd6b2652700 1 -- 172.21.15.32:0/3086615264 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6ac10e080 con 0x7fd6ac102590 2024-05-12T04:12:11.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.604+0000 7fd6b2652700 1 -- 172.21.15.32:0/3086615264 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6ac0fd340 con 0x7fd6ac102590 2024-05-12T04:12:11.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.605+0000 7fd6a8ff9700 1 -- 172.21.15.32:0/3086615264 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fd6a000cb90 con 0x7fd6ac102590 2024-05-12T04:12:11.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.605+0000 7fd6a8ff9700 1 --2- 172.21.15.32:0/3086615264 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fd69406cc20 0x7fd69406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:11.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.605+0000 7fd6a8ff9700 1 -- 172.21.15.32:0/3086615264 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fd6a008bb90 con 0x7fd6ac102590 2024-05-12T04:12:11.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.605+0000 7fd6abfff700 1 --2- 172.21.15.32:0/3086615264 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fd69406cc20 0x7fd69406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:11.607 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.606+0000 7fd6abfff700 1 --2- 172.21.15.32:0/3086615264 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fd69406cc20 0x7fd69406f0d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd69c00b0d0 tx=0x7fd69c009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:11.610 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.610+0000 7fd6a8ff9700 1 -- 172.21.15.32:0/3086615264 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fd6a0059600 con 0x7fd6ac102590 2024-05-12T04:12:11.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:11 smithi032 ceph-mon[120392]: pgmap v305: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:12:11.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:11 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/874147887' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-05-12T04:12:11.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:11 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/3083877088' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-05-12T04:12:11.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:11 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/4175802848' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-12T04:12:11.813 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.812+0000 7fd6b2652700 1 -- 172.21.15.32:0/3086615264 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd6ac108bc0 con 0x7fd6ac102590 2024-05-12T04:12:11.814 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.814+0000 7fd6a8ff9700 1 -- 172.21.15.32:0/3086615264 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16628 (secure 0 0 0) 0x7fd6a0059190 con 0x7fd6ac102590 2024-05-12T04:12:11.814 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-12T04:12:11.814 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":84,"fsid":"a3e178f8-1012-11ef-bc98-c7b262605968","created":"2024-05-12T03:50:18.356038+0000","modified":"2024-05-12T04:08:50.380422+0000","last_up_change":"2024-05-12T04:07:44.234383+0000","last_in_change":"2024-05-12T03:56:03.301729+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-12T03:54:58.296702+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-05-12T03:57:47.465323+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-05-12T03:57:48.324643+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fc534129-ef70-487d-b8d1-f98fc170c1d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":49,"up_from":53,"up_thru":78,"down_at":50,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":1676344197},{"type":"v1","addr":"172.21.15.32:6803","nonce":1676344197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":1676344197},{"type":"v1","addr":"172.21.15.32:6805","nonce":1676344197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":1676344197},{"type":"v1","addr":"172.21.15.32:6809","nonce":1676344197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":1676344197},{"type":"v1","addr":"172.21.15.32:6807","nonce":1676344197}]},"public_addr":"172.21.15.32:6803/1676344197","cluster_addr":"172.21.15.32:6805/1676344197","heartbeat_back_addr":"172.21.15.32:6809/1676344197","heartbeat_front_addr":"172.21.15.32:6807/1676344197","state":["exists","up"]},{"osd":1,"uuid":"f8c35b6a-5afc-447b-b318-d56097de28d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":54,"up_from":58,"up_thru":78,"down_at":55,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":954580115},{"type":"v1","addr":"172.21.15.32:6811","nonce":954580115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":954580115},{"type":"v1","addr":"172.21.15.32:6813","nonce":954580115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":954580115},{"type":"v1","addr":"172.21.15.32:6817","nonce":954580115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":954580115},{"type":"v1","addr":"172.21.15.32:6815","nonce":954580115}]},"public_addr":"172.21.15.32:6811/954580115","cluster_addr":"172.21.15.32:6813/954580115","heartbeat_back_addr":"172.21.15.32:6817/954580115","heartbeat_front_addr":"172.21.15.32:6815/954580115","state":["exists","up"]},{"osd":2,"uuid":"4ed4b55d-cceb-494b-bdff-fef99d6c8cf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":59,"up_from":63,"up_thru":78,"down_at":60,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":793261683},{"type":"v1","addr":"172.21.15.32:6819","nonce":793261683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":793261683},{"type":"v1","addr":"172.21.15.32:6821","nonce":793261683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":793261683},{"type":"v1","addr":"172.21.15.32:6825","nonce":793261683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":793261683},{"type":"v1","addr":"172.21.15.32:6823","nonce":793261683}]},"public_addr":"172.21.15.32:6819/793261683","cluster_addr":"172.21.15.32:6821/793261683","heartbeat_back_addr":"172.21.15.32:6825/793261683","heartbeat_front_addr":"172.21.15.32:6823/793261683","state":["exists","up"]},{"osd":3,"uuid":"ef191267-c0a1-4be7-9e8b-761700af87a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":64,"up_from":68,"up_thru":78,"down_at":65,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":3436738456},{"type":"v1","addr":"172.21.15.33:6801","nonce":3436738456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":3436738456},{"type":"v1","addr":"172.21.15.33:6803","nonce":3436738456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":3436738456},{"type":"v1","addr":"172.21.15.33:6807","nonce":3436738456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":3436738456},{"type":"v1","addr":"172.21.15.33:6805","nonce":3436738456}]},"public_addr":"172.21.15.33:6801/3436738456","cluster_addr":"172.21.15.33:6803/3436738456","heartbeat_back_addr":"172.21.15.33:6807/3436738456","heartbeat_front_addr":"172.21.15.33:6805/3436738456","state":["exists","up"]},{"osd":4,"uuid":"fe45f168-076a-4888-895d-bb710bd0da73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":69,"up_from":73,"up_thru":78,"down_at":70,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1028896705},{"type":"v1","addr":"172.21.15.33:6809","nonce":1028896705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1028896705},{"type":"v1","addr":"172.21.15.33:6811","nonce":1028896705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1028896705},{"type":"v1","addr":"172.21.15.33:6815","nonce":1028896705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1028896705},{"type":"v1","addr":"172.21.15.33:6813","nonce":1028896705}]},"public_addr":"172.21.15.33:6809/1028896705","cluster_addr":"172.21.15.33:6811/1028896705","heartbeat_back_addr":"172.21.15.33:6815/1028896705","heartbeat_front_addr":"172.21.15.33:6813/1028896705","state":["exists","up"]},{"osd":5,"uuid":"f7a0e14e-a866-4163-a7df-f8a43b860bc0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":74,"up_from":78,"up_thru":78,"down_at":75,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":4274440090},{"type":"v1","addr":"172.21.15.33:6817","nonce":4274440090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":4274440090},{"type":"v1","addr":"172.21.15.33:6819","nonce":4274440090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":4274440090},{"type":"v1","addr":"172.21.15.33:6823","nonce":4274440090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":4274440090},{"type":"v1","addr":"172.21.15.33:6821","nonce":4274440090}]},"public_addr":"172.21.15.33:6817/4274440090","cluster_addr":"172.21.15.33:6819/4274440090","heartbeat_back_addr":"172.21.15.33:6823/4274440090","heartbeat_front_addr":"172.21.15.33:6821/4274440090","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-12T03:53:56.134194+0000","dead_epoch":49},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-12T03:54:25.231975+0000","dead_epoch":54},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-12T03:54:54.991052+0000","dead_epoch":59},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-12T03:55:22.680656+0000","dead_epoch":64},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-12T03:55:52.446678+0000","dead_epoch":69},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-12T03:56:22.128957+0000","dead_epoch":74}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6829/3100155296":"2024-05-13T04:08:08.971776+0000","172.21.15.32:6828/3100155296":"2024-05-13T04:08:08.971776+0000","172.21.15.32:0/912074455":"2024-05-13T04:03:15.687820+0000","172.21.15.32:6800/1121526649":"2024-05-13T03:50:49.378828+0000","172.21.15.32:0/3343483246":"2024-05-13T04:00:09.761962+0000","172.21.15.32:6827/1907818042":"2024-05-13T03:58:02.805639+0000","172.21.15.32:0/4293156271":"2024-05-13T03:51:25.269581+0000","172.21.15.32:0/2900983644":"2024-05-13T03:52:39.772717+0000","172.21.15.32:0/2236672133":"2024-05-13T04:00:09.761962+0000","172.21.15.32:6800/2967201790":"2024-05-13T03:51:25.269581+0000","172.21.15.33:6826/433858238":"2024-05-13T04:08:50.380388+0000","172.21.15.32:6826/1907818042":"2024-05-13T03:58:02.805639+0000","172.21.15.32:6801/1121526649":"2024-05-13T03:50:49.378828+0000","172.21.15.32:0/2098237559":"2024-05-13T03:50:49.378828+0000","172.21.15.32:6800/4089797100":"2024-05-13T04:00:09.761962+0000","172.21.15.32:0/1463926533":"2024-05-13T03:50:49.378828+0000","172.21.15.32:0/1834879835":"2024-05-13T04:00:09.761962+0000","172.21.15.32:0/165225041":"2024-05-13T03:51:25.269581+0000","172.21.15.32:0/625670655":"2024-05-13T03:50:49.378828+0000","172.21.15.33:6827/433858238":"2024-05-13T04:08:50.380388+0000","172.21.15.32:0/3185268518":"2024-05-13T03:52:39.772717+0000","172.21.15.33:6829/1598726962":"2024-05-13T04:02:10.132488+0000","172.21.15.32:0/1073406471":"2024-05-13T03:52:39.772717+0000","172.21.15.32:6801/4089797100":"2024-05-13T04:00:09.761962+0000","172.21.15.32:0/1356801576":"2024-05-13T03:51:25.269581+0000","172.21.15.32:0/993128492":"2024-05-13T04:00:09.761962+0000","172.21.15.32:0/2523500808":"2024-05-13T04:03:15.687820+0000","172.21.15.32:6801/2967201790":"2024-05-13T03:51:25.269581+0000","172.21.15.32:6800/3729453095":"2024-05-13T04:03:15.687820+0000","172.21.15.33:6828/1598726962":"2024-05-13T04:02:10.132488+0000","172.21.15.32:0/2757703284":"2024-05-13T04:03:15.687820+0000","172.21.15.33:0/2636219823":"2024-05-13T04:02:10.132488+0000","172.21.15.33:0/2129212605":"2024-05-13T04:02:10.132488+0000","172.21.15.33:0/619425673":"2024-05-13T04:02:10.132488+0000","172.21.15.33:0/3980223680":"2024-05-13T04:02:10.132488+0000","172.21.15.32:6800/1988399583":"2024-05-13T03:52:39.772717+0000","172.21.15.32:0/213713458":"2024-05-13T04:03:15.687820+0000","172.21.15.32:6801/1988399583":"2024-05-13T03:52:39.772717+0000","172.21.15.32:6801/3729453095":"2024-05-13T04:03:15.687820+0000","172.21.15.32:0/3906624315":"2024-05-13T04:03:15.687820+0000","172.21.15.32:0/3077610654":"2024-05-13T04:03:15.687820+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-12T04:12:11.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.816+0000 7fd6b2652700 1 -- 172.21.15.32:0/3086615264 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fd69406cc20 msgr2=0x7fd69406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:11.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.816+0000 7fd6b2652700 1 --2- 172.21.15.32:0/3086615264 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fd69406cc20 0x7fd69406f0d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd69c00b0d0 tx=0x7fd69c009e30 comp rx=0 tx=0).stop 2024-05-12T04:12:11.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.816+0000 7fd6b2652700 1 -- 172.21.15.32:0/3086615264 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6ac102590 msgr2=0x7fd6ac113c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:11.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.816+0000 7fd6b2652700 1 --2- 172.21.15.32:0/3086615264 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6ac102590 0x7fd6ac113c50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd6a000eb40 tx=0x7fd6a000b040 comp rx=0 tx=0).stop 2024-05-12T04:12:11.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.816+0000 7fd6b2652700 1 -- 172.21.15.32:0/3086615264 shutdown_connections 2024-05-12T04:12:11.816 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.816+0000 7fd6b2652700 1 --2- 172.21.15.32:0/3086615264 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fd69406cc20 0x7fd69406f0d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:11.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.816+0000 7fd6b2652700 1 --2- 172.21.15.32:0/3086615264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6ac101bf0 0x7fd6ac113710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:11.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.816+0000 7fd6b2652700 1 --2- 172.21.15.32:0/3086615264 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6ac102590 0x7fd6ac113c50 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:11.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.816+0000 7fd6b2652700 1 -- 172.21.15.32:0/3086615264 >> 172.21.15.32:0/3086615264 conn(0x7fd6ac071bb0 msgr2=0x7fd6ac064dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:11.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.816+0000 7fd6b2652700 1 -- 172.21.15.32:0/3086615264 shutdown_connections 2024-05-12T04:12:11.817 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.816+0000 7fd6b2652700 1 -- 172.21.15.32:0/3086615264 wait complete. 2024-05-12T04:12:11.828 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-12T03:54:58.296702+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-05-12T03:57:47.465323+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-05-12T03:57:48.324643+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-12T04:12:11.829 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-12T04:12:11.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:11 smithi033 ceph-mon[107305]: pgmap v305: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:12:11.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:11 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/874147887' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-05-12T04:12:11.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:11 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/3083877088' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-05-12T04:12:11.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:11 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/4175802848' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-12T04:12:11.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.953+0000 7facd03d7700 1 -- 172.21.15.32:0/586044260 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7facc8102550 msgr2=0x7facc80fbbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:11.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.953+0000 7facd03d7700 1 --2- 172.21.15.32:0/586044260 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7facc8102550 0x7facc80fbbc0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7facbc004940 tx=0x7facbc009e30 comp rx=0 tx=0).stop 2024-05-12T04:12:11.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.954+0000 7facd03d7700 1 -- 172.21.15.32:0/586044260 shutdown_connections 2024-05-12T04:12:11.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.954+0000 7facd03d7700 1 --2- 172.21.15.32:0/586044260 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7facc8102550 0x7facc80fbbc0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:11.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.954+0000 7facd03d7700 1 --2- 172.21.15.32:0/586044260 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7facc8101bb0 0x7facc8101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:11.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.954+0000 7facd03d7700 1 -- 172.21.15.32:0/586044260 >> 172.21.15.32:0/586044260 conn(0x7facc80f7740 msgr2=0x7facc80f9b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:11.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.954+0000 7facd03d7700 1 -- 172.21.15.32:0/586044260 shutdown_connections 2024-05-12T04:12:11.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.954+0000 7facd03d7700 1 -- 172.21.15.32:0/586044260 wait complete. 2024-05-12T04:12:11.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.954+0000 7facd03d7700 1 Processor -- start 2024-05-12T04:12:11.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.954+0000 7facd03d7700 1 -- start start 2024-05-12T04:12:11.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.954+0000 7facd03d7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7facc8101bb0 0x7facc8113760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:11.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.954+0000 7facd03d7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7facc8102550 0x7facc8113ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:11.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.954+0000 7facd03d7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc81142f0 con 0x7facc8101bb0 2024-05-12T04:12:11.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.954+0000 7facd03d7700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc810d8f0 con 0x7facc8102550 2024-05-12T04:12:11.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.954+0000 7facce173700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7facc8101bb0 0x7facc8113760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:11.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.955+0000 7facce173700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7facc8101bb0 0x7facc8113760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54620/0 (socket says 172.21.15.32:54620) 2024-05-12T04:12:11.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.955+0000 7facce173700 1 -- 172.21.15.32:0/2452951391 learned_addr learned my addr 172.21.15.32:0/2452951391 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:12:11.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.955+0000 7faccd972700 1 --2- 172.21.15.32:0/2452951391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7facc8102550 0x7facc8113ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:11.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.955+0000 7facce173700 1 -- 172.21.15.32:0/2452951391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7facc8102550 msgr2=0x7facc8113ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:11.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.955+0000 7facce173700 1 --2- 172.21.15.32:0/2452951391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7facc8102550 0x7facc8113ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:11.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.955+0000 7facce173700 1 -- 172.21.15.32:0/2452951391 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facbc005040 con 0x7facc8101bb0 2024-05-12T04:12:11.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.955+0000 7facce173700 1 --2- 172.21.15.32:0/2452951391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7facc8101bb0 0x7facc8113760 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7facc400c8c0 tx=0x7facc40044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:11.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.955+0000 7facbaffd700 1 -- 172.21.15.32:0/2452951391 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7facc400a760 con 0x7facc8101bb0 2024-05-12T04:12:11.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.955+0000 7facbaffd700 1 -- 172.21.15.32:0/2452951391 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7facc400ad70 con 0x7facc8101bb0 2024-05-12T04:12:11.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.955+0000 7facd03d7700 1 -- 172.21.15.32:0/2452951391 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facc810dbd0 con 0x7facc8101bb0 2024-05-12T04:12:11.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.955+0000 7facbaffd700 1 -- 172.21.15.32:0/2452951391 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7facc40113a0 con 0x7facc8101bb0 2024-05-12T04:12:11.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.955+0000 7facd03d7700 1 -- 172.21.15.32:0/2452951391 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facc810e160 con 0x7facc8101bb0 2024-05-12T04:12:11.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.956+0000 7facd03d7700 1 -- 172.21.15.32:0/2452951391 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facc80fd300 con 0x7facc8101bb0 2024-05-12T04:12:11.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.957+0000 7facbaffd700 1 -- 172.21.15.32:0/2452951391 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7facc4012400 con 0x7facc8101bb0 2024-05-12T04:12:11.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.957+0000 7facbaffd700 1 --2- 172.21.15.32:0/2452951391 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7facb406cc20 0x7facb406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:11.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.957+0000 7faccd972700 1 --2- 172.21.15.32:0/2452951391 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7facb406cc20 0x7facb406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:11.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.957+0000 7facbaffd700 1 -- 172.21.15.32:0/2452951391 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7facc408c010 con 0x7facc8101bb0 2024-05-12T04:12:11.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.958+0000 7faccd972700 1 --2- 172.21.15.32:0/2452951391 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7facb406cc20 0x7facb406f0d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7facc810ebd0 tx=0x7facbc009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:11.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:11.962+0000 7facbaffd700 1 -- 172.21.15.32:0/2452951391 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7facc4059b20 con 0x7facc8101bb0 2024-05-12T04:12:12.163 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.163+0000 7facd03d7700 1 -- 172.21.15.32:0/2452951391 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7facc8108b40 con 0x7facc8101bb0 2024-05-12T04:12:12.164 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.163+0000 7facbaffd700 1 -- 172.21.15.32:0/2452951391 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7facc40596b0 con 0x7facc8101bb0 2024-05-12T04:12:12.164 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-05-12T04:12:12.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.166+0000 7facd03d7700 1 -- 172.21.15.32:0/2452951391 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7facb406cc20 msgr2=0x7facb406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:12.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.166+0000 7facd03d7700 1 --2- 172.21.15.32:0/2452951391 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7facb406cc20 0x7facb406f0d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7facc810ebd0 tx=0x7facbc009e30 comp rx=0 tx=0).stop 2024-05-12T04:12:12.166 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.166+0000 7facd03d7700 1 -- 172.21.15.32:0/2452951391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7facc8101bb0 msgr2=0x7facc8113760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:12.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.166+0000 7facd03d7700 1 --2- 172.21.15.32:0/2452951391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7facc8101bb0 0x7facc8113760 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7facc400c8c0 tx=0x7facc40044c0 comp rx=0 tx=0).stop 2024-05-12T04:12:12.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.166+0000 7facd03d7700 1 -- 172.21.15.32:0/2452951391 shutdown_connections 2024-05-12T04:12:12.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.166+0000 7facd03d7700 1 --2- 172.21.15.32:0/2452951391 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7facb406cc20 0x7facb406f0d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:12.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.166+0000 7facd03d7700 1 --2- 172.21.15.32:0/2452951391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7facc8101bb0 0x7facc8113760 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:12.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.166+0000 7facd03d7700 1 --2- 172.21.15.32:0/2452951391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7facc8102550 0x7facc8113ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:12.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.166+0000 7facd03d7700 1 -- 172.21.15.32:0/2452951391 >> 172.21.15.32:0/2452951391 conn(0x7facc80f7740 msgr2=0x7facc80f8840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:12.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.166+0000 7facd03d7700 1 -- 172.21.15.32:0/2452951391 shutdown_connections 2024-05-12T04:12:12.167 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.166+0000 7facd03d7700 1 -- 172.21.15.32:0/2452951391 wait complete. 2024-05-12T04:12:12.178 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-05-12T04:12:12.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.303+0000 7f3198749700 1 -- 172.21.15.32:0/3884107979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3190101be0 msgr2=0x7f3190101fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:12.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.303+0000 7f3198749700 1 --2- 172.21.15.32:0/3884107979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3190101be0 0x7f3190101fb0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f318c004940 tx=0x7f318c009e30 comp rx=0 tx=0).stop 2024-05-12T04:12:12.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.304+0000 7f3198749700 1 -- 172.21.15.32:0/3884107979 shutdown_connections 2024-05-12T04:12:12.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.304+0000 7f3198749700 1 --2- 172.21.15.32:0/3884107979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31900fc380 0x7f31900fc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:12.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.304+0000 7f3198749700 1 --2- 172.21.15.32:0/3884107979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3190101be0 0x7f3190101fb0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:12.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.304+0000 7f3198749700 1 -- 172.21.15.32:0/3884107979 >> 172.21.15.32:0/3884107979 conn(0x7f3190072230 msgr2=0x7f3190072630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:12.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.304+0000 7f3198749700 1 -- 172.21.15.32:0/3884107979 shutdown_connections 2024-05-12T04:12:12.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.304+0000 7f3198749700 1 -- 172.21.15.32:0/3884107979 wait complete. 2024-05-12T04:12:12.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.304+0000 7f3198749700 1 Processor -- start 2024-05-12T04:12:12.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.304+0000 7f3198749700 1 -- start start 2024-05-12T04:12:12.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.304+0000 7f3198749700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31900fc380 0x7f3190113740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:12.306 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.304+0000 7f3198749700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3190101be0 0x7f3190113c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:12.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.304+0000 7f3198749700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31901142d0 con 0x7f31900fc380 2024-05-12T04:12:12.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.304+0000 7f3198749700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f319010d8d0 con 0x7f3190101be0 2024-05-12T04:12:12.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.304+0000 7f31964e5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31900fc380 0x7f3190113740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:12.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.304+0000 7f31964e5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31900fc380 0x7f3190113740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54648/0 (socket says 172.21.15.32:54648) 2024-05-12T04:12:12.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.304+0000 7f31964e5700 1 -- 172.21.15.32:0/3914667693 learned_addr learned my addr 172.21.15.32:0/3914667693 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:12:12.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.305+0000 7f3195ce4700 1 --2- 172.21.15.32:0/3914667693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3190101be0 0x7f3190113c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:12.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.305+0000 7f31964e5700 1 -- 172.21.15.32:0/3914667693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3190101be0 msgr2=0x7f3190113c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:12.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.305+0000 7f31964e5700 1 --2- 172.21.15.32:0/3914667693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3190101be0 0x7f3190113c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:12.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.305+0000 7f31964e5700 1 -- 172.21.15.32:0/3914667693 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f318c005040 con 0x7f31900fc380 2024-05-12T04:12:12.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.305+0000 7f3195ce4700 1 --2- 172.21.15.32:0/3914667693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3190101be0 0x7f3190113c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-12T04:12:12.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.305+0000 7f31964e5700 1 --2- 172.21.15.32:0/3914667693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31900fc380 0x7f3190113740 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f318c009390 tx=0x7f318c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:12.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.305+0000 7f3186ffd700 1 -- 172.21.15.32:0/3914667693 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f318c00e040 con 0x7f31900fc380 2024-05-12T04:12:12.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.305+0000 7f3186ffd700 1 -- 172.21.15.32:0/3914667693 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f318c013420 con 0x7f31900fc380 2024-05-12T04:12:12.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.305+0000 7f3198749700 1 -- 172.21.15.32:0/3914667693 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f319010db50 con 0x7f31900fc380 2024-05-12T04:12:12.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.305+0000 7f3186ffd700 1 -- 172.21.15.32:0/3914667693 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f318c01c630 con 0x7f31900fc380 2024-05-12T04:12:12.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.305+0000 7f3198749700 1 -- 172.21.15.32:0/3914667693 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f319010e110 con 0x7f31900fc380 2024-05-12T04:12:12.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.306+0000 7f3198749700 1 -- 172.21.15.32:0/3914667693 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3190104fd0 con 0x7f31900fc380 2024-05-12T04:12:12.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.307+0000 7f3186ffd700 1 -- 172.21.15.32:0/3914667693 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f318c0139e0 con 0x7f31900fc380 2024-05-12T04:12:12.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.307+0000 7f3186ffd700 1 --2- 172.21.15.32:0/3914667693 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f317c06cc20 0x7f317c06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:12.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.307+0000 7f3186ffd700 1 -- 172.21.15.32:0/3914667693 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f318c08d220 con 0x7f31900fc380 2024-05-12T04:12:12.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.307+0000 7f3195ce4700 1 --2- 172.21.15.32:0/3914667693 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f317c06cc20 0x7f317c06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:12.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.307+0000 7f3195ce4700 1 --2- 172.21.15.32:0/3914667693 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f317c06cc20 0x7f317c06f0d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3180009fd0 tx=0x7f318000b040 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:12.311 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.311+0000 7f3186ffd700 1 -- 172.21.15.32:0/3914667693 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7f318c05ade0 con 0x7f31900fc380 2024-05-12T04:12:12.520 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.520+0000 7f3198749700 1 -- 172.21.15.32:0/3914667693 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f319010ed90 con 0x7f31900fc380 2024-05-12T04:12:12.521 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.520+0000 7f3186ffd700 1 -- 172.21.15.32:0/3914667693 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v84) v1 ==== 107+0+11 (secure 0 0 0) 0x7f318c05a970 con 0x7f31900fc380 2024-05-12T04:12:12.521 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-05-12T04:12:12.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.523+0000 7f3198749700 1 -- 172.21.15.32:0/3914667693 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f317c06cc20 msgr2=0x7f317c06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:12.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.523+0000 7f3198749700 1 --2- 172.21.15.32:0/3914667693 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f317c06cc20 0x7f317c06f0d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3180009fd0 tx=0x7f318000b040 comp rx=0 tx=0).stop 2024-05-12T04:12:12.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.523+0000 7f3198749700 1 -- 172.21.15.32:0/3914667693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31900fc380 msgr2=0x7f3190113740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:12.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.523+0000 7f3198749700 1 --2- 172.21.15.32:0/3914667693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31900fc380 0x7f3190113740 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f318c009390 tx=0x7f318c009e30 comp rx=0 tx=0).stop 2024-05-12T04:12:12.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.523+0000 7f3198749700 1 -- 172.21.15.32:0/3914667693 shutdown_connections 2024-05-12T04:12:12.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.523+0000 7f3198749700 1 --2- 172.21.15.32:0/3914667693 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7f317c06cc20 0x7f317c06f0d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:12.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.523+0000 7f3198749700 1 --2- 172.21.15.32:0/3914667693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f31900fc380 0x7f3190113740 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:12.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.523+0000 7f3198749700 1 --2- 172.21.15.32:0/3914667693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3190101be0 0x7f3190113c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:12.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.523+0000 7f3198749700 1 -- 172.21.15.32:0/3914667693 >> 172.21.15.32:0/3914667693 conn(0x7f3190072230 msgr2=0x7f31900f9830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:12.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.523+0000 7f3198749700 1 -- 172.21.15.32:0/3914667693 shutdown_connections 2024-05-12T04:12:12.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.523+0000 7f3198749700 1 -- 172.21.15.32:0/3914667693 wait complete. 2024-05-12T04:12:12.537 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-05-12T04:12:12.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.662+0000 7fd38050e700 1 -- 172.21.15.32:0/1501700227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd378102cc0 msgr2=0x7fd378103090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:12.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.662+0000 7fd38050e700 1 --2- 172.21.15.32:0/1501700227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd378102cc0 0x7fd378103090 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd368004940 tx=0x7fd368009e30 comp rx=0 tx=0).stop 2024-05-12T04:12:12.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.662+0000 7fd38050e700 1 -- 172.21.15.32:0/1501700227 shutdown_connections 2024-05-12T04:12:12.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.662+0000 7fd38050e700 1 --2- 172.21.15.32:0/1501700227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd378103660 0x7fd3780fccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:12.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.662+0000 7fd38050e700 1 --2- 172.21.15.32:0/1501700227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd378102cc0 0x7fd378103090 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:12.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.662+0000 7fd38050e700 1 -- 172.21.15.32:0/1501700227 >> 172.21.15.32:0/1501700227 conn(0x7fd3780f8810 msgr2=0x7fd3780fac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:12.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.662+0000 7fd38050e700 1 -- 172.21.15.32:0/1501700227 shutdown_connections 2024-05-12T04:12:12.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.662+0000 7fd38050e700 1 -- 172.21.15.32:0/1501700227 wait complete. 2024-05-12T04:12:12.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.662+0000 7fd38050e700 1 Processor -- start 2024-05-12T04:12:12.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.662+0000 7fd38050e700 1 -- start start 2024-05-12T04:12:12.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.663+0000 7fd38050e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd378102cc0 0x7fd378113760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:12.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.663+0000 7fd38050e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd378103660 0x7fd378113ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:12.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.663+0000 7fd38050e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3781142f0 con 0x7fd378102cc0 2024-05-12T04:12:12.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.663+0000 7fd38050e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd37810d8f0 con 0x7fd378103660 2024-05-12T04:12:12.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.663+0000 7fd37e2aa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd378102cc0 0x7fd378113760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:12.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.663+0000 7fd37e2aa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd378102cc0 0x7fd378113760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54672/0 (socket says 172.21.15.32:54672) 2024-05-12T04:12:12.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.663+0000 7fd37e2aa700 1 -- 172.21.15.32:0/2232997202 learned_addr learned my addr 172.21.15.32:0/2232997202 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:12:12.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.663+0000 7fd37daa9700 1 --2- 172.21.15.32:0/2232997202 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd378103660 0x7fd378113ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:12.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.663+0000 7fd37e2aa700 1 -- 172.21.15.32:0/2232997202 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd378103660 msgr2=0x7fd378113ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:12.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.663+0000 7fd37e2aa700 1 --2- 172.21.15.32:0/2232997202 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd378103660 0x7fd378113ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:12.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.663+0000 7fd37e2aa700 1 -- 172.21.15.32:0/2232997202 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd368005040 con 0x7fd378102cc0 2024-05-12T04:12:12.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.663+0000 7fd37e2aa700 1 --2- 172.21.15.32:0/2232997202 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd378102cc0 0x7fd378113760 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd36800b3b0 tx=0x7fd368009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:12.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.663+0000 7fd36effd700 1 -- 172.21.15.32:0/2232997202 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd36800e040 con 0x7fd378102cc0 2024-05-12T04:12:12.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.663+0000 7fd36effd700 1 -- 172.21.15.32:0/2232997202 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd368013420 con 0x7fd378102cc0 2024-05-12T04:12:12.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.663+0000 7fd38050e700 1 -- 172.21.15.32:0/2232997202 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd37810db70 con 0x7fd378102cc0 2024-05-12T04:12:12.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.663+0000 7fd36effd700 1 -- 172.21.15.32:0/2232997202 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd36801c630 con 0x7fd378102cc0 2024-05-12T04:12:12.666 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.663+0000 7fd38050e700 1 -- 172.21.15.32:0/2232997202 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd37810e130 con 0x7fd378102cc0 2024-05-12T04:12:12.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.664+0000 7fd38050e700 1 -- 172.21.15.32:0/2232997202 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3780fe410 con 0x7fd378102cc0 2024-05-12T04:12:12.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.665+0000 7fd36effd700 1 -- 172.21.15.32:0/2232997202 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fd368013970 con 0x7fd378102cc0 2024-05-12T04:12:12.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.665+0000 7fd36effd700 1 --2- 172.21.15.32:0/2232997202 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fd36406cc20 0x7fd36406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:12.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.665+0000 7fd36effd700 1 -- 172.21.15.32:0/2232997202 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fd36808d200 con 0x7fd378102cc0 2024-05-12T04:12:12.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.665+0000 7fd37daa9700 1 --2- 172.21.15.32:0/2232997202 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fd36406cc20 0x7fd36406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:12.667 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.666+0000 7fd37daa9700 1 --2- 172.21.15.32:0/2232997202 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fd36406cc20 0x7fd36406f0d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd374007fd0 tx=0x7fd374009040 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:12.670 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.670+0000 7fd36effd700 1 -- 172.21.15.32:0/2232997202 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7fd36805ad10 con 0x7fd378102cc0 2024-05-12T04:12:12.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:12 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/3086615264' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-12T04:12:12.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:12 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/2452951391' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-12T04:12:12.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.872+0000 7fd38050e700 1 -- 172.21.15.32:0/2232997202 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fd378109cc0 con 0x7fd378102cc0 2024-05-12T04:12:12.873 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.873+0000 7fd36effd700 1 -- 172.21.15.32:0/2232997202 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v84) v1 ==== 107+0+11 (secure 0 0 0) 0x7fd36805a8a0 con 0x7fd378102cc0 2024-05-12T04:12:12.873 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2024-05-12T04:12:12.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.875+0000 7fd38050e700 1 -- 172.21.15.32:0/2232997202 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fd36406cc20 msgr2=0x7fd36406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:12.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.875+0000 7fd38050e700 1 --2- 172.21.15.32:0/2232997202 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fd36406cc20 0x7fd36406f0d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd374007fd0 tx=0x7fd374009040 comp rx=0 tx=0).stop 2024-05-12T04:12:12.875 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.875+0000 7fd38050e700 1 -- 172.21.15.32:0/2232997202 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd378102cc0 msgr2=0x7fd378113760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:12.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.875+0000 7fd38050e700 1 --2- 172.21.15.32:0/2232997202 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd378102cc0 0x7fd378113760 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd36800b3b0 tx=0x7fd368009e30 comp rx=0 tx=0).stop 2024-05-12T04:12:12.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.875+0000 7fd38050e700 1 -- 172.21.15.32:0/2232997202 shutdown_connections 2024-05-12T04:12:12.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.875+0000 7fd38050e700 1 --2- 172.21.15.32:0/2232997202 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7fd36406cc20 0x7fd36406f0d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:12.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.875+0000 7fd38050e700 1 --2- 172.21.15.32:0/2232997202 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd378102cc0 0x7fd378113760 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:12.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.875+0000 7fd38050e700 1 --2- 172.21.15.32:0/2232997202 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd378103660 0x7fd378113ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:12.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.875+0000 7fd38050e700 1 -- 172.21.15.32:0/2232997202 >> 172.21.15.32:0/2232997202 conn(0x7fd3780f8810 msgr2=0x7fd3780f99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:12.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.875+0000 7fd38050e700 1 -- 172.21.15.32:0/2232997202 shutdown_connections 2024-05-12T04:12:12.876 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:12.875+0000 7fd38050e700 1 -- 172.21.15.32:0/2232997202 wait complete. 2024-05-12T04:12:12.888 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-12T04:12:12.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:12 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/3086615264' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-12T04:12:12.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:12 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/2452951391' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-12T04:12:13.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:13.010+0000 7ff0d6218700 1 -- 172.21.15.32:0/2868065308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0d0102580 msgr2=0x7ff0d00fbbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:13.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:13.010+0000 7ff0d6218700 1 --2- 172.21.15.32:0/2868065308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0d0102580 0x7ff0d00fbbf0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff0c0004940 tx=0x7ff0c0009e30 comp rx=0 tx=0).stop 2024-05-12T04:12:13.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:13.010+0000 7ff0d6218700 1 -- 172.21.15.32:0/2868065308 shutdown_connections 2024-05-12T04:12:13.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:13.010+0000 7ff0d6218700 1 --2- 172.21.15.32:0/2868065308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0d0102580 0x7ff0d00fbbf0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:13.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:13.010+0000 7ff0d6218700 1 --2- 172.21.15.32:0/2868065308 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0d0101be0 0x7ff0d0101fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:13.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:13.010+0000 7ff0d6218700 1 -- 172.21.15.32:0/2868065308 >> 172.21.15.32:0/2868065308 conn(0x7ff0d00f76f0 msgr2=0x7ff0d00f9b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:13.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:13.010+0000 7ff0d6218700 1 -- 172.21.15.32:0/2868065308 shutdown_connections 2024-05-12T04:12:13.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:13.010+0000 7ff0d6218700 1 -- 172.21.15.32:0/2868065308 wait complete. 2024-05-12T04:12:13.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:13.010+0000 7ff0d6218700 1 Processor -- start 2024-05-12T04:12:13.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:13.011+0000 7ff0d6218700 1 -- start start 2024-05-12T04:12:13.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:13.011+0000 7ff0d6218700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0d0101be0 0x7ff0d0113760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:13.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:13.011+0000 7ff0d6218700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0d0102580 0x7ff0d0113ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:13.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:13.011+0000 7ff0d6218700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d01142f0 con 0x7ff0d0101be0 2024-05-12T04:12:13.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:13.011+0000 7ff0d6218700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d010d8f0 con 0x7ff0d0102580 2024-05-12T04:12:13.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:13.011+0000 7ff0cf7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0d0101be0 0x7ff0d0113760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:13.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:13.011+0000 7ff0cf7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0d0101be0 0x7ff0d0113760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54694/0 (socket says 172.21.15.32:54694) 2024-05-12T04:12:13.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:13.011+0000 7ff0cf7fe700 1 -- 172.21.15.32:0/3010988789 learned_addr learned my addr 172.21.15.32:0/3010988789 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-12T04:12:13.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:13.011+0000 7ff0ceffd700 1 --2- 172.21.15.32:0/3010988789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0d0102580 0x7ff0d0113ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:13.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:13.011+0000 7ff0cf7fe700 1 -- 172.21.15.32:0/3010988789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0d0102580 msgr2=0x7ff0d0113ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:13.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:13.011+0000 7ff0cf7fe700 1 --2- 172.21.15.32:0/3010988789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0d0102580 0x7ff0d0113ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:13.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:13.011+0000 7ff0cf7fe700 1 -- 172.21.15.32:0/3010988789 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c0005040 con 0x7ff0d0101be0 2024-05-12T04:12:13.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:13.012+0000 7ff0cf7fe700 1 --2- 172.21.15.32:0/3010988789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0d0101be0 0x7ff0d0113760 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4002a50 tx=0x7ff0c40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:13.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:13.012+0000 7ff0b7fff700 1 -- 172.21.15.32:0/3010988789 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0c4007cc0 con 0x7ff0d0101be0 2024-05-12T04:12:13.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:13.012+0000 7ff0d6218700 1 -- 172.21.15.32:0/3010988789 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0d010dbd0 con 0x7ff0d0101be0 2024-05-12T04:12:13.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:13.012+0000 7ff0b7fff700 1 -- 172.21.15.32:0/3010988789 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff0c400f040 con 0x7ff0d0101be0 2024-05-12T04:12:13.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:13.012+0000 7ff0b7fff700 1 -- 172.21.15.32:0/3010988789 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0c400d9c0 con 0x7ff0d0101be0 2024-05-12T04:12:13.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:13.012+0000 7ff0d6218700 1 -- 172.21.15.32:0/3010988789 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0d010e160 con 0x7ff0d0101be0 2024-05-12T04:12:13.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:13.013+0000 7ff0d6218700 1 -- 172.21.15.32:0/3010988789 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0d00fd330 con 0x7ff0d0101be0 2024-05-12T04:12:13.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:13.013+0000 7ff0b7fff700 1 -- 172.21.15.32:0/3010988789 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 40) v1 ==== 90850+0+0 (secure 0 0 0) 0x7ff0c400db20 con 0x7ff0d0101be0 2024-05-12T04:12:13.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:13.014+0000 7ff0b7fff700 1 --2- 172.21.15.32:0/3010988789 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7ff0b806cc20 0x7ff0b806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-12T04:12:13.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:13.014+0000 7ff0b7fff700 1 -- 172.21.15.32:0/3010988789 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7ff0c408bdd0 con 0x7ff0d0101be0 2024-05-12T04:12:13.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:13.014+0000 7ff0ceffd700 1 --2- 172.21.15.32:0/3010988789 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7ff0b806cc20 0x7ff0b806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-12T04:12:13.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:13.014+0000 7ff0ceffd700 1 --2- 172.21.15.32:0/3010988789 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7ff0b806cc20 0x7ff0b806f0d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff0d010ebd0 tx=0x7ff0c0009e30 comp rx=0 tx=0).ready entity=mgr.24593 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-12T04:12:13.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:13.018+0000 7ff0b7fff700 1 -- 172.21.15.32:0/3010988789 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183351 (secure 0 0 0) 0x7ff0c40597c0 con 0x7ff0d0101be0 2024-05-12T04:12:13.243 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:13.243+0000 7ff0d6218700 1 -- 172.21.15.32:0/3010988789 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff0d0108b70 con 0x7ff0d0101be0 2024-05-12T04:12:13.244 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:13.244+0000 7ff0b7fff700 1 -- 172.21.15.32:0/3010988789 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 41 entries v84) v1 ==== 81+0+2424 (secure 0 0 0) 0x7ff0c4059350 con 0x7ff0d0101be0 2024-05-12T04:12:13.244 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6829/3100155296 2024-05-13T04:08:08.971776+0000 2024-05-12T04:12:13.244 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6828/3100155296 2024-05-13T04:08:08.971776+0000 2024-05-12T04:12:13.245 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/912074455 2024-05-13T04:03:15.687820+0000 2024-05-12T04:12:13.245 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6800/1121526649 2024-05-13T03:50:49.378828+0000 2024-05-12T04:12:13.245 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/3343483246 2024-05-13T04:00:09.761962+0000 2024-05-12T04:12:13.245 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6827/1907818042 2024-05-13T03:58:02.805639+0000 2024-05-12T04:12:13.245 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/4293156271 2024-05-13T03:51:25.269581+0000 2024-05-12T04:12:13.245 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2900983644 2024-05-13T03:52:39.772717+0000 2024-05-12T04:12:13.245 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2236672133 2024-05-13T04:00:09.761962+0000 2024-05-12T04:12:13.245 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6800/2967201790 2024-05-13T03:51:25.269581+0000 2024-05-12T04:12:13.245 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.33:6826/433858238 2024-05-13T04:08:50.380388+0000 2024-05-12T04:12:13.245 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6826/1907818042 2024-05-13T03:58:02.805639+0000 2024-05-12T04:12:13.245 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6801/1121526649 2024-05-13T03:50:49.378828+0000 2024-05-12T04:12:13.245 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2098237559 2024-05-13T03:50:49.378828+0000 2024-05-12T04:12:13.245 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6800/4089797100 2024-05-13T04:00:09.761962+0000 2024-05-12T04:12:13.245 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/1463926533 2024-05-13T03:50:49.378828+0000 2024-05-12T04:12:13.246 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/1834879835 2024-05-13T04:00:09.761962+0000 2024-05-12T04:12:13.246 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/165225041 2024-05-13T03:51:25.269581+0000 2024-05-12T04:12:13.246 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/625670655 2024-05-13T03:50:49.378828+0000 2024-05-12T04:12:13.246 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.33:6827/433858238 2024-05-13T04:08:50.380388+0000 2024-05-12T04:12:13.246 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/3185268518 2024-05-13T03:52:39.772717+0000 2024-05-12T04:12:13.246 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.33:6829/1598726962 2024-05-13T04:02:10.132488+0000 2024-05-12T04:12:13.246 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/1073406471 2024-05-13T03:52:39.772717+0000 2024-05-12T04:12:13.246 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6801/4089797100 2024-05-13T04:00:09.761962+0000 2024-05-12T04:12:13.246 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/1356801576 2024-05-13T03:51:25.269581+0000 2024-05-12T04:12:13.247 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/993128492 2024-05-13T04:00:09.761962+0000 2024-05-12T04:12:13.247 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2523500808 2024-05-13T04:03:15.687820+0000 2024-05-12T04:12:13.247 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6801/2967201790 2024-05-13T03:51:25.269581+0000 2024-05-12T04:12:13.247 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6800/3729453095 2024-05-13T04:03:15.687820+0000 2024-05-12T04:12:13.247 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.33:6828/1598726962 2024-05-13T04:02:10.132488+0000 2024-05-12T04:12:13.247 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2757703284 2024-05-13T04:03:15.687820+0000 2024-05-12T04:12:13.247 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.33:0/2636219823 2024-05-13T04:02:10.132488+0000 2024-05-12T04:12:13.247 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.33:0/2129212605 2024-05-13T04:02:10.132488+0000 2024-05-12T04:12:13.247 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.33:0/619425673 2024-05-13T04:02:10.132488+0000 2024-05-12T04:12:13.247 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.33:0/3980223680 2024-05-13T04:02:10.132488+0000 2024-05-12T04:12:13.247 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6800/1988399583 2024-05-13T03:52:39.772717+0000 2024-05-12T04:12:13.247 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/213713458 2024-05-13T04:03:15.687820+0000 2024-05-12T04:12:13.247 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6801/1988399583 2024-05-13T03:52:39.772717+0000 2024-05-12T04:12:13.247 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6801/3729453095 2024-05-13T04:03:15.687820+0000 2024-05-12T04:12:13.248 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/3906624315 2024-05-13T04:03:15.687820+0000 2024-05-12T04:12:13.248 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/3077610654 2024-05-13T04:03:15.687820+0000 2024-05-12T04:12:13.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:13.246+0000 7ff0d6218700 1 -- 172.21.15.32:0/3010988789 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7ff0b806cc20 msgr2=0x7ff0b806f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:13.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:13.246+0000 7ff0d6218700 1 --2- 172.21.15.32:0/3010988789 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7ff0b806cc20 0x7ff0b806f0d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff0d010ebd0 tx=0x7ff0c0009e30 comp rx=0 tx=0).stop 2024-05-12T04:12:13.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:13.246+0000 7ff0d6218700 1 -- 172.21.15.32:0/3010988789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0d0101be0 msgr2=0x7ff0d0113760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-12T04:12:13.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:13.246+0000 7ff0d6218700 1 --2- 172.21.15.32:0/3010988789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0d0101be0 0x7ff0d0113760 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4002a50 tx=0x7ff0c40046b0 comp rx=0 tx=0).stop 2024-05-12T04:12:13.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:13.246+0000 7ff0d6218700 1 -- 172.21.15.32:0/3010988789 shutdown_connections 2024-05-12T04:12:13.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:13.246+0000 7ff0d6218700 1 --2- 172.21.15.32:0/3010988789 >> [v2:172.21.15.32:6800/2957111591,v1:172.21.15.32:6801/2957111591] conn(0x7ff0b806cc20 0x7ff0b806f0d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:13.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:13.246+0000 7ff0d6218700 1 --2- 172.21.15.32:0/3010988789 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0d0101be0 0x7ff0d0113760 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:13.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:13.246+0000 7ff0d6218700 1 --2- 172.21.15.32:0/3010988789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0d0102580 0x7ff0d0113ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-12T04:12:13.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:13.246+0000 7ff0d6218700 1 -- 172.21.15.32:0/3010988789 >> 172.21.15.32:0/3010988789 conn(0x7ff0d00f76f0 msgr2=0x7ff0d00f8ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-12T04:12:13.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:13.246+0000 7ff0d6218700 1 -- 172.21.15.32:0/3010988789 shutdown_connections 2024-05-12T04:12:13.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-12T04:12:13.246+0000 7ff0d6218700 1 -- 172.21.15.32:0/3010988789 wait complete. 2024-05-12T04:12:13.250 INFO:teuthology.orchestra.run.smithi032.stderr:listed 41 entries 2024-05-12T04:12:13.259 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi032.front.sepia.ceph.com... 2024-05-12T04:12:13.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-12T04:12:13.259 DEBUG:teuthology.orchestra.run.smithi032:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-12T04:12:13.295 INFO:teuthology.orchestra.run:waiting for 300 2024-05-12T04:12:13.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:13 smithi032 ceph-mon[120392]: pgmap v306: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:12:13.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:13 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/3914667693' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-05-12T04:12:13.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:13 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/2232997202' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-05-12T04:12:13.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:13 smithi032 ceph-mon[120392]: from='client.? 172.21.15.32:0/3010988789' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-12T04:12:13.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:13 smithi033 ceph-mon[107305]: pgmap v306: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:12:13.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:13 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/3914667693' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-05-12T04:12:13.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:13 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/2232997202' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-05-12T04:12:13.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:13 smithi033 ceph-mon[107305]: from='client.? 172.21.15.32:0/3010988789' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-12T04:12:14.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:14 smithi032 ceph-mon[120392]: pgmap v307: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:12:14.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:14 smithi033 ceph-mon[107305]: pgmap v307: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:12:16.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:15 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:12:16.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:15 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:12:17.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:16 smithi033 ceph-mon[107305]: pgmap v308: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:12:17.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:16 smithi032 ceph-mon[120392]: pgmap v308: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:12:19.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:18 smithi033 ceph-mon[107305]: pgmap v309: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:12:19.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:18 smithi032 ceph-mon[120392]: pgmap v309: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:12:20.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:19 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:12:20.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:19 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:12:20.162 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:19 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:12:20.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:19 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:12:20.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:19 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:12:20.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:19 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:12:21.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:21 smithi032 ceph-mon[120392]: pgmap v310: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:12:21.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:21 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:12:21.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:21 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T04:12:21.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:21 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:12:21.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:21 smithi033 ceph-mon[107305]: pgmap v310: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:12:21.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:21 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:12:21.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:21 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T04:12:21.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:21 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:12:23.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:23 smithi032 ceph-mon[120392]: pgmap v311: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:12:23.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:23 smithi033 ceph-mon[107305]: pgmap v311: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:12:25.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:25 smithi032 ceph-mon[120392]: pgmap v312: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:12:25.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:25 smithi033 ceph-mon[107305]: pgmap v312: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:12:27.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:27 smithi032 ceph-mon[120392]: pgmap v313: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:12:27.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:27 smithi033 ceph-mon[107305]: pgmap v313: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:12:29.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:29 smithi032 ceph-mon[120392]: pgmap v314: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:12:29.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:29 smithi033 ceph-mon[107305]: pgmap v314: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:12:31.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:31 smithi032 ceph-mon[120392]: pgmap v315: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:12:31.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:31 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:12:31.665 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:31 smithi033 ceph-mon[107305]: pgmap v315: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:12:31.665 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:31 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:12:33.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:33 smithi032 ceph-mon[120392]: pgmap v316: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:12:33.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:33 smithi033 ceph-mon[107305]: pgmap v316: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:12:35.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:35 smithi032 ceph-mon[120392]: pgmap v317: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:12:35.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:35 smithi033 ceph-mon[107305]: pgmap v317: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:12:37.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:37 smithi032 ceph-mon[120392]: pgmap v318: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:12:37.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:37 smithi033 ceph-mon[107305]: pgmap v318: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:12:39.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:39 smithi032 ceph-mon[120392]: pgmap v319: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:12:39.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:39 smithi033 ceph-mon[107305]: pgmap v319: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:12:41.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:41 smithi032 ceph-mon[120392]: pgmap v320: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:12:41.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:41 smithi033 ceph-mon[107305]: pgmap v320: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:12:43.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:43 smithi032 ceph-mon[120392]: pgmap v321: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:12:43.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:43 smithi033 ceph-mon[107305]: pgmap v321: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:12:45.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:45 smithi032 ceph-mon[120392]: pgmap v322: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:12:45.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:45 smithi033 ceph-mon[107305]: pgmap v322: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:12:46.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:46 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:12:46.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:46 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:12:47.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:47 smithi032 ceph-mon[120392]: pgmap v323: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:12:47.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:47 smithi033 ceph-mon[107305]: pgmap v323: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:12:49.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:49 smithi032 ceph-mon[120392]: pgmap v324: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:12:49.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:49 smithi033 ceph-mon[107305]: pgmap v324: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:12:51.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:51 smithi032 ceph-mon[120392]: pgmap v325: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:12:51.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:51 smithi033 ceph-mon[107305]: pgmap v325: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:12:53.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:53 smithi032 ceph-mon[120392]: pgmap v326: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:12:53.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:53 smithi033 ceph-mon[107305]: pgmap v326: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:12:55.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:55 smithi032 ceph-mon[120392]: pgmap v327: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:12:55.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:55 smithi033 ceph-mon[107305]: pgmap v327: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:12:57.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:57 smithi032 ceph-mon[120392]: pgmap v328: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:12:57.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:57 smithi033 ceph-mon[107305]: pgmap v328: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:12:59.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:12:59 smithi032 ceph-mon[120392]: pgmap v329: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:12:59.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:12:59 smithi033 ceph-mon[107305]: pgmap v329: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:13:01.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:13:01 smithi032 ceph-mon[120392]: pgmap v330: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:13:01.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:13:01 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:13:01.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:13:01 smithi033 ceph-mon[107305]: pgmap v330: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:13:01.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:13:01 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:13:03.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:13:03 smithi032 ceph-mon[120392]: pgmap v331: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:13:03.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:13:03 smithi033 ceph-mon[107305]: pgmap v331: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:13:05.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:13:05 smithi032 ceph-mon[120392]: pgmap v332: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:13:05.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:13:05 smithi033 ceph-mon[107305]: pgmap v332: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:13:07.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:13:07 smithi032 ceph-mon[120392]: pgmap v333: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:13:07.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:13:07 smithi033 ceph-mon[107305]: pgmap v333: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:13:09.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:13:09 smithi032 ceph-mon[120392]: pgmap v334: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:13:09.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:13:09 smithi033 ceph-mon[107305]: pgmap v334: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:13:11.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:13:11 smithi032 ceph-mon[120392]: pgmap v335: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:13:11.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:13:11 smithi033 ceph-mon[107305]: pgmap v335: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:13:13.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:13:13 smithi032 ceph-mon[120392]: pgmap v336: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:13:13.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:13:13 smithi033 ceph-mon[107305]: pgmap v336: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:13:15.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:13:15 smithi032 ceph-mon[120392]: pgmap v337: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:13:15.666 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:13:15 smithi033 ceph-mon[107305]: pgmap v337: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:13:16.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:13:16 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:13:16.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:13:16 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:13:17.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:13:17 smithi032 ceph-mon[120392]: pgmap v338: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:13:17.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:13:17 smithi033 ceph-mon[107305]: pgmap v338: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:13:19.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:13:19 smithi032 ceph-mon[120392]: pgmap v339: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:13:19.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:13:19 smithi033 ceph-mon[107305]: pgmap v339: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:13:21.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:13:21 smithi032 ceph-mon[120392]: pgmap v340: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:13:21.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:13:21 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:13:21.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:13:21 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:13:21.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:13:21 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:13:21.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:13:21 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:13:21.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:13:21 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T04:13:21.541 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:13:21 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:13:21.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:13:21 smithi033 ceph-mon[107305]: pgmap v340: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:13:21.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:13:21 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:13:21.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:13:21 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:13:21.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:13:21 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:13:21.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:13:21 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:13:21.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:13:21 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T04:13:21.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:13:21 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:13:23.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:13:23 smithi032 ceph-mon[120392]: pgmap v341: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:13:23.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:13:23 smithi033 ceph-mon[107305]: pgmap v341: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:13:25.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:13:25 smithi032 ceph-mon[120392]: pgmap v342: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:13:25.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:13:25 smithi033 ceph-mon[107305]: pgmap v342: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:13:27.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:13:27 smithi032 ceph-mon[120392]: pgmap v343: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:13:27.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:13:27 smithi033 ceph-mon[107305]: pgmap v343: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:13:29.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:13:29 smithi032 ceph-mon[120392]: pgmap v344: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:13:29.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:13:29 smithi033 ceph-mon[107305]: pgmap v344: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:13:31.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:13:31 smithi032 ceph-mon[120392]: pgmap v345: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:13:31.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:13:31 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:13:31.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:13:31 smithi033 ceph-mon[107305]: pgmap v345: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:13:31.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:13:31 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:13:33.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:13:33 smithi032 ceph-mon[120392]: pgmap v346: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:13:33.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:13:33 smithi033 ceph-mon[107305]: pgmap v346: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:13:35.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:13:35 smithi032 ceph-mon[120392]: pgmap v347: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:13:35.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:13:35 smithi033 ceph-mon[107305]: pgmap v347: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:13:37.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:13:37 smithi032 ceph-mon[120392]: pgmap v348: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:13:37.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:13:37 smithi033 ceph-mon[107305]: pgmap v348: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:13:39.666 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:13:39 smithi033 ceph-mon[107305]: pgmap v349: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:13:39.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:13:39 smithi032 ceph-mon[120392]: pgmap v349: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:13:41.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:13:41 smithi033 ceph-mon[107305]: pgmap v350: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:13:41.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:13:41 smithi032 ceph-mon[120392]: pgmap v350: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:13:43.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:13:43 smithi033 ceph-mon[107305]: pgmap v351: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:13:43.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:13:43 smithi032 ceph-mon[120392]: pgmap v351: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:13:45.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:13:45 smithi033 ceph-mon[107305]: pgmap v352: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:13:45.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:13:45 smithi032 ceph-mon[120392]: pgmap v352: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:13:46.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:13:46 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:13:46.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:13:46 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:13:47.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:13:47 smithi033 ceph-mon[107305]: pgmap v353: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:13:47.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:13:47 smithi032 ceph-mon[120392]: pgmap v353: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:13:49.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:13:49 smithi033 ceph-mon[107305]: pgmap v354: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:13:49.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:13:49 smithi032 ceph-mon[120392]: pgmap v354: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:13:51.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:13:51 smithi033 ceph-mon[107305]: pgmap v355: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:13:51.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:13:51 smithi032 ceph-mon[120392]: pgmap v355: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:13:53.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:13:53 smithi033 ceph-mon[107305]: pgmap v356: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:13:53.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:13:53 smithi032 ceph-mon[120392]: pgmap v356: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:13:55.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:13:55 smithi033 ceph-mon[107305]: pgmap v357: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:13:55.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:13:55 smithi032 ceph-mon[120392]: pgmap v357: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:13:57.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:13:57 smithi033 ceph-mon[107305]: pgmap v358: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:13:57.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:13:57 smithi032 ceph-mon[120392]: pgmap v358: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:13:59.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:13:59 smithi033 ceph-mon[107305]: pgmap v359: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:13:59.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:13:59 smithi032 ceph-mon[120392]: pgmap v359: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:14:01.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:14:01 smithi033 ceph-mon[107305]: pgmap v360: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:14:01.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:14:01 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:14:01.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:14:01 smithi032 ceph-mon[120392]: pgmap v360: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:14:01.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:14:01 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:14:03.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:14:03 smithi033 ceph-mon[107305]: pgmap v361: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:14:03.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:14:03 smithi032 ceph-mon[120392]: pgmap v361: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:14:05.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:14:05 smithi033 ceph-mon[107305]: pgmap v362: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:14:05.791 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:14:05 smithi032 ceph-mon[120392]: pgmap v362: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:14:07.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:14:07 smithi033 ceph-mon[107305]: pgmap v363: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:14:07.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:14:07 smithi032 ceph-mon[120392]: pgmap v363: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:14:09.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:14:09 smithi033 ceph-mon[107305]: pgmap v364: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:14:09.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:14:09 smithi032 ceph-mon[120392]: pgmap v364: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:14:11.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:14:11 smithi033 ceph-mon[107305]: pgmap v365: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:14:11.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:14:11 smithi032 ceph-mon[120392]: pgmap v365: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:14:13.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:14:13 smithi033 ceph-mon[107305]: pgmap v366: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:14:13.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:14:13 smithi032 ceph-mon[120392]: pgmap v366: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:14:15.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:14:15 smithi033 ceph-mon[107305]: pgmap v367: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:14:15.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:14:15 smithi032 ceph-mon[120392]: pgmap v367: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:14:16.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:14:16 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:14:16.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:14:16 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:14:17.655 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:14:17 smithi032 ceph-mon[120392]: pgmap v368: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:14:17.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:14:17 smithi033 ceph-mon[107305]: pgmap v368: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:14:19.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:14:19 smithi033 ceph-mon[107305]: pgmap v369: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:14:19.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:14:19 smithi032 ceph-mon[120392]: pgmap v369: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:14:21.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:14:21 smithi033 ceph-mon[107305]: pgmap v370: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:14:21.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:14:21 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:14:21.789 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:14:21 smithi032 ceph-mon[120392]: pgmap v370: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:14:21.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:14:21 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:14:22.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:14:22 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:14:22.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:14:22 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T04:14:22.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:14:22 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:14:22.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:14:22 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:14:22.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:14:22 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T04:14:22.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:14:22 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:14:23.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:14:23 smithi033 ceph-mon[107305]: pgmap v371: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:14:23.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:14:23 smithi032 ceph-mon[120392]: pgmap v371: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:14:25.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:14:25 smithi033 ceph-mon[107305]: pgmap v372: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:14:25.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:14:25 smithi032 ceph-mon[120392]: pgmap v372: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:14:27.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:14:27 smithi033 ceph-mon[107305]: pgmap v373: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:14:27.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:14:27 smithi032 ceph-mon[120392]: pgmap v373: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:14:29.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:14:29 smithi033 ceph-mon[107305]: pgmap v374: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:14:29.789 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:14:29 smithi032 ceph-mon[120392]: pgmap v374: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:14:31.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:14:31 smithi033 ceph-mon[107305]: pgmap v375: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:14:31.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:14:31 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:14:31.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:14:31 smithi032 ceph-mon[120392]: pgmap v375: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:14:31.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:14:31 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:14:33.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:14:33 smithi033 ceph-mon[107305]: pgmap v376: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:14:33.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:14:33 smithi032 ceph-mon[120392]: pgmap v376: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:14:35.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:14:35 smithi033 ceph-mon[107305]: pgmap v377: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:14:35.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:14:35 smithi032 ceph-mon[120392]: pgmap v377: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:14:37.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:14:37 smithi033 ceph-mon[107305]: pgmap v378: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:14:37.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:14:37 smithi032 ceph-mon[120392]: pgmap v378: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:14:39.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:14:39 smithi033 ceph-mon[107305]: pgmap v379: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:14:39.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:14:39 smithi032 ceph-mon[120392]: pgmap v379: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:14:41.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:14:41 smithi033 ceph-mon[107305]: pgmap v380: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:14:41.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:14:41 smithi032 ceph-mon[120392]: pgmap v380: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:14:43.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:14:43 smithi033 ceph-mon[107305]: pgmap v381: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:14:43.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:14:43 smithi032 ceph-mon[120392]: pgmap v381: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:14:45.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:14:45 smithi033 ceph-mon[107305]: pgmap v382: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:14:45.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:14:45 smithi032 ceph-mon[120392]: pgmap v382: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:14:46.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:14:46 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:14:46.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:14:46 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:14:47.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:14:47 smithi033 ceph-mon[107305]: pgmap v383: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:14:47.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:14:47 smithi032 ceph-mon[120392]: pgmap v383: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:14:49.662 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:14:49 smithi033 ceph-mon[107305]: pgmap v384: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:14:49.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:14:49 smithi032 ceph-mon[120392]: pgmap v384: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:14:51.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:14:51 smithi032 ceph-mon[120392]: pgmap v385: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:14:51.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:14:51 smithi033 ceph-mon[107305]: pgmap v385: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:14:53.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:14:53 smithi032 ceph-mon[120392]: pgmap v386: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:14:53.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:14:53 smithi033 ceph-mon[107305]: pgmap v386: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:14:55.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:14:55 smithi032 ceph-mon[120392]: pgmap v387: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:14:55.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:14:55 smithi033 ceph-mon[107305]: pgmap v387: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:14:57.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:14:57 smithi032 ceph-mon[120392]: pgmap v388: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:14:57.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:14:57 smithi033 ceph-mon[107305]: pgmap v388: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:14:59.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:14:59 smithi032 ceph-mon[120392]: pgmap v389: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:14:59.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:14:59 smithi033 ceph-mon[107305]: pgmap v389: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:15:01.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:15:01 smithi032 ceph-mon[120392]: pgmap v390: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:15:01.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:15:01 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:15:01.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:15:01 smithi033 ceph-mon[107305]: pgmap v390: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:15:01.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:15:01 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:15:03.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:15:03 smithi032 ceph-mon[120392]: pgmap v391: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:15:03.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:15:03 smithi033 ceph-mon[107305]: pgmap v391: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:15:05.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:15:05 smithi032 ceph-mon[120392]: pgmap v392: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:15:05.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:15:05 smithi033 ceph-mon[107305]: pgmap v392: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:15:07.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:15:07 smithi032 ceph-mon[120392]: pgmap v393: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:15:07.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:15:07 smithi033 ceph-mon[107305]: pgmap v393: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:15:09.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:15:09 smithi032 ceph-mon[120392]: pgmap v394: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:15:09.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:15:09 smithi033 ceph-mon[107305]: pgmap v394: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:15:11.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:15:11 smithi032 ceph-mon[120392]: pgmap v395: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:15:11.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:15:11 smithi033 ceph-mon[107305]: pgmap v395: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:15:13.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:15:13 smithi032 ceph-mon[120392]: pgmap v396: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:15:13.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:15:13 smithi033 ceph-mon[107305]: pgmap v396: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:15:15.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:15:15 smithi032 ceph-mon[120392]: pgmap v397: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:15:15.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:15:15 smithi033 ceph-mon[107305]: pgmap v397: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:15:16.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:15:16 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:15:16.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:15:16 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:15:17.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:15:17 smithi032 ceph-mon[120392]: pgmap v398: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:15:17.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:15:17 smithi033 ceph-mon[107305]: pgmap v398: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:15:19.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:15:19 smithi032 ceph-mon[120392]: pgmap v399: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:15:19.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:15:19 smithi033 ceph-mon[107305]: pgmap v399: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:15:21.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:15:21 smithi032 ceph-mon[120392]: pgmap v400: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:15:21.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:15:21 smithi033 ceph-mon[107305]: pgmap v400: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:15:22.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:15:22 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:15:22.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:15:22 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:15:22.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:15:22 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T04:15:22.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:15:22 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:15:22.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:15:22 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:15:22.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:15:22 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:15:22.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:15:22 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T04:15:22.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:15:22 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:15:23.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:15:23 smithi032 ceph-mon[120392]: pgmap v401: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:15:23.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:15:23 smithi033 ceph-mon[107305]: pgmap v401: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:15:25.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:15:25 smithi032 ceph-mon[120392]: pgmap v402: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:15:25.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:15:25 smithi033 ceph-mon[107305]: pgmap v402: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:15:27.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:15:27 smithi032 ceph-mon[120392]: pgmap v403: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:15:27.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:15:27 smithi033 ceph-mon[107305]: pgmap v403: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:15:29.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:15:29 smithi032 ceph-mon[120392]: pgmap v404: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:15:29.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:15:29 smithi033 ceph-mon[107305]: pgmap v404: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:15:31.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:15:31 smithi032 ceph-mon[120392]: pgmap v405: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:15:31.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:15:31 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:15:31.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:15:31 smithi033 ceph-mon[107305]: pgmap v405: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:15:31.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:15:31 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:15:33.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:15:33 smithi032 ceph-mon[120392]: pgmap v406: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:15:33.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:15:33 smithi033 ceph-mon[107305]: pgmap v406: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:15:35.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:15:35 smithi032 ceph-mon[120392]: pgmap v407: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:15:35.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:15:35 smithi033 ceph-mon[107305]: pgmap v407: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:15:37.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:15:37 smithi032 ceph-mon[120392]: pgmap v408: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:15:37.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:15:37 smithi033 ceph-mon[107305]: pgmap v408: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:15:39.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:15:39 smithi032 ceph-mon[120392]: pgmap v409: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:15:39.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:15:39 smithi033 ceph-mon[107305]: pgmap v409: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:15:41.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:15:41 smithi032 ceph-mon[120392]: pgmap v410: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:15:41.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:15:41 smithi033 ceph-mon[107305]: pgmap v410: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:15:43.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:15:43 smithi032 ceph-mon[120392]: pgmap v411: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:15:43.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:15:43 smithi033 ceph-mon[107305]: pgmap v411: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:15:45.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:15:45 smithi032 ceph-mon[120392]: pgmap v412: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:15:45.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:15:45 smithi033 ceph-mon[107305]: pgmap v412: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:15:46.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:15:46 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:15:46.912 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:15:46 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:15:47.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:15:47 smithi032 ceph-mon[120392]: pgmap v413: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:15:47.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:15:47 smithi033 ceph-mon[107305]: pgmap v413: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:15:49.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:15:49 smithi032 ceph-mon[120392]: pgmap v414: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:15:49.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:15:49 smithi033 ceph-mon[107305]: pgmap v414: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:15:50.911 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:15:50 smithi033 ceph-mon[107305]: pgmap v415: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:15:51.040 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:15:50 smithi032 ceph-mon[120392]: pgmap v415: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:15:53.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:15:52 smithi032 ceph-mon[120392]: pgmap v416: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:15:53.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:15:52 smithi033 ceph-mon[107305]: pgmap v416: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:15:55.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:15:54 smithi032 ceph-mon[120392]: pgmap v417: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:15:55.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:15:54 smithi033 ceph-mon[107305]: pgmap v417: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:15:57.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:15:56 smithi032 ceph-mon[120392]: pgmap v418: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:15:57.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:15:56 smithi033 ceph-mon[107305]: pgmap v418: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:15:59.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:15:58 smithi032 ceph-mon[120392]: pgmap v419: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:15:59.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:15:58 smithi033 ceph-mon[107305]: pgmap v419: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:16:01.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:16:00 smithi032 ceph-mon[120392]: pgmap v420: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:16:01.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:16:00 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:16:01.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:16:00 smithi033 ceph-mon[107305]: pgmap v420: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:16:01.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:16:00 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:16:03.291 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:16:02 smithi032 ceph-mon[120392]: pgmap v421: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:16:03.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:16:02 smithi033 ceph-mon[107305]: pgmap v421: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:16:05.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:16:04 smithi032 ceph-mon[120392]: pgmap v422: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:16:05.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:16:04 smithi033 ceph-mon[107305]: pgmap v422: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:16:07.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:16:06 smithi032 ceph-mon[120392]: pgmap v423: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:16:07.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:16:06 smithi033 ceph-mon[107305]: pgmap v423: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:16:09.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:16:08 smithi032 ceph-mon[120392]: pgmap v424: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:16:09.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:16:08 smithi033 ceph-mon[107305]: pgmap v424: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:16:11.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:16:10 smithi032 ceph-mon[120392]: pgmap v425: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:16:11.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:16:10 smithi033 ceph-mon[107305]: pgmap v425: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:16:13.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:16:12 smithi032 ceph-mon[120392]: pgmap v426: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:16:13.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:16:12 smithi033 ceph-mon[107305]: pgmap v426: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:16:15.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:16:14 smithi032 ceph-mon[120392]: pgmap v427: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:16:15.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:16:14 smithi033 ceph-mon[107305]: pgmap v427: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:16:16.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:16:15 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:16:16.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:16:15 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:16:17.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:16:16 smithi032 ceph-mon[120392]: pgmap v428: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:16:17.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:16:16 smithi033 ceph-mon[107305]: pgmap v428: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:16:19.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:16:18 smithi032 ceph-mon[120392]: pgmap v429: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:16:19.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:16:18 smithi033 ceph-mon[107305]: pgmap v429: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:16:21.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:16:20 smithi032 ceph-mon[120392]: pgmap v430: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-05-12T04:16:21.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:16:20 smithi033 ceph-mon[107305]: pgmap v430: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-05-12T04:16:23.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:16:22 smithi032 ceph-mon[120392]: pgmap v431: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-05-12T04:16:23.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:16:22 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:16:23.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:16:22 smithi033 ceph-mon[107305]: pgmap v431: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-05-12T04:16:23.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:16:22 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-12T04:16:24.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:16:23 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:16:24.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:16:23 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T04:16:24.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:16:23 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:16:24.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:16:23 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-12T04:16:24.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:16:23 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-12T04:16:24.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:16:23 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' 2024-05-12T04:16:25.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:16:25 smithi032 ceph-mon[120392]: pgmap v432: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-05-12T04:16:25.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:16:25 smithi033 ceph-mon[107305]: pgmap v432: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-05-12T04:16:27.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:16:27 smithi032 ceph-mon[120392]: pgmap v433: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-05-12T04:16:27.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:16:27 smithi033 ceph-mon[107305]: pgmap v433: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-05-12T04:16:29.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:16:29 smithi032 ceph-mon[120392]: pgmap v434: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:16:29.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:16:29 smithi033 ceph-mon[107305]: pgmap v434: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:16:31.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:16:31 smithi032 ceph-mon[120392]: pgmap v435: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:16:31.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:16:31 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:16:31.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:16:31 smithi033 ceph-mon[107305]: pgmap v435: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:16:31.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:16:31 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:16:33.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:16:33 smithi032 ceph-mon[120392]: pgmap v436: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:16:33.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:16:33 smithi033 ceph-mon[107305]: pgmap v436: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:16:35.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:16:35 smithi032 ceph-mon[120392]: pgmap v437: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:16:35.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:16:35 smithi033 ceph-mon[107305]: pgmap v437: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:16:37.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:16:37 smithi032 ceph-mon[120392]: pgmap v438: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:16:37.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:16:37 smithi033 ceph-mon[107305]: pgmap v438: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:16:39.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:16:39 smithi032 ceph-mon[120392]: pgmap v439: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:16:39.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:16:39 smithi033 ceph-mon[107305]: pgmap v439: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:16:41.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:16:41 smithi032 ceph-mon[120392]: pgmap v440: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:16:41.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:16:41 smithi033 ceph-mon[107305]: pgmap v440: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:16:43.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:16:43 smithi032 ceph-mon[120392]: pgmap v441: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:16:43.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:16:43 smithi033 ceph-mon[107305]: pgmap v441: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:16:45.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:16:45 smithi032 ceph-mon[120392]: pgmap v442: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:16:45.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:16:45 smithi033 ceph-mon[107305]: pgmap v442: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:16:46.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:16:46 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:16:46.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:16:46 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:16:47.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:16:47 smithi032 ceph-mon[120392]: pgmap v443: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:16:47.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:16:47 smithi033 ceph-mon[107305]: pgmap v443: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:16:49.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:16:49 smithi032 ceph-mon[120392]: pgmap v444: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:16:49.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:16:49 smithi033 ceph-mon[107305]: pgmap v444: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:16:51.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:16:51 smithi033 ceph-mon[107305]: pgmap v445: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:16:51.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:16:51 smithi032 ceph-mon[120392]: pgmap v445: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:16:53.398 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:16:53 smithi032 ceph-mon[120392]: pgmap v446: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:16:53.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:16:53 smithi033 ceph-mon[107305]: pgmap v446: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:16:55.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:16:55 smithi033 ceph-mon[107305]: pgmap v447: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:16:55.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:16:55 smithi032 ceph-mon[120392]: pgmap v447: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:16:57.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:16:57 smithi033 ceph-mon[107305]: pgmap v448: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:16:57.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:16:57 smithi032 ceph-mon[120392]: pgmap v448: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:16:59.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:16:59 smithi033 ceph-mon[107305]: pgmap v449: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:16:59.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:16:59 smithi032 ceph-mon[120392]: pgmap v449: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:17:01.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:17:01 smithi033 ceph-mon[107305]: pgmap v450: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:17:01.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:17:01 smithi033 ceph-mon[107305]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:17:01.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:17:01 smithi032 ceph-mon[120392]: pgmap v450: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:17:01.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:17:01 smithi032 ceph-mon[120392]: from='mgr.24593 172.21.15.32:0/309032898' entity='mgr.smithi032.ukatkq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-05-12T04:17:03.398 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:17:03 smithi032 ceph-mon[120392]: pgmap v451: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:17:03.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:17:03 smithi033 ceph-mon[107305]: pgmap v451: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-05-12T04:17:05.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:17:05 smithi033 ceph-mon[107305]: pgmap v452: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:17:05.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:17:05 smithi032 ceph-mon[120392]: pgmap v452: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:17:07.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:17:07 smithi033 ceph-mon[107305]: pgmap v453: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:17:07.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:17:07 smithi032 ceph-mon[120392]: pgmap v453: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:17:09.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:17:09 smithi033 ceph-mon[107305]: pgmap v454: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:17:09.542 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:17:09 smithi032 ceph-mon[120392]: pgmap v454: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-12T04:17:11.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:17:11 smithi033 ceph-mon[107305]: pgmap v455: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:17:11.540 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:17:11 smithi032 ceph-mon[120392]: pgmap v455: 65 pgs: 65 active+clean; 260 MiB data, 980 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-05-12T04:17:12.368 ERROR:tasks.cephfs.fuse_mount:process failed to terminate after unmount. This probably indicates a bug within ceph-fuse. 2024-05-12T04:17:12.369 ERROR:teuthology.run_tasks:Manager failed: ceph-fuse Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_fccfb453906af8847786c471919d347112ea0360/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_fccfb453906af8847786c471919d347112ea0360/qa/tasks/cephfs/fuse_mount.py", line 400, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-05-12T04:17:12.370 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-12T04:17:12.382 INFO:tasks.cephadm:Teardown begin 2024-05-12T04:17:12.383 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph_fccfb453906af8847786c471919d347112ea0360/qa/tasks/cephadm.py", line 1750, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_fccfb453906af8847786c471919d347112ea0360/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_fccfb453906af8847786c471919d347112ea0360/qa/tasks/cephfs/fuse_mount.py", line 400, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-05-12T04:17:12.384 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-12T04:17:12.422 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-12T04:17:12.461 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-12T04:17:12.461 DEBUG:teuthology.orchestra.run.smithi032:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-12T04:17:12.484 DEBUG:teuthology.orchestra.run.smithi033:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-12T04:17:12.523 INFO:tasks.cephadm:Stopping all daemons... 2024-05-12T04:17:12.523 INFO:tasks.cephadm.mon.smithi032:Stopping mon.smithi032... 2024-05-12T04:17:12.523 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ceph-a3e178f8-1012-11ef-bc98-c7b262605968@mon.smithi032 2024-05-12T04:17:12.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:17:12 smithi032 systemd[1]: Stopping Ceph mon.smithi032 for a3e178f8-1012-11ef-bc98-c7b262605968... 2024-05-12T04:17:13.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:17:12 smithi032 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-mon-smithi032[120369]: 2024-05-12T04:17:12.814+0000 7f4746baa700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi032 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-12T04:17:13.290 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:17:12 smithi032 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-mon-smithi032[120369]: 2024-05-12T04:17:12.814+0000 7f4746baa700 -1 mon.smithi032@0(leader) e3 *** Got Signal Terminated *** 2024-05-12T04:17:13.790 INFO:journalctl@ceph.mon.smithi032.smithi032.stdout:May 12 04:17:13 smithi032 bash[191356]: ceph-a3e178f8-1012-11ef-bc98-c7b262605968-mon-smithi032 2024-05-12T04:17:13.902 DEBUG:teuthology.orchestra.run.smithi032:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a3e178f8-1012-11ef-bc98-c7b262605968@mon.smithi032.service' 2024-05-12T04:17:13.969 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-12T04:17:13.969 INFO:tasks.cephadm.mon.smithi032:Stopped mon.smithi032 2024-05-12T04:17:13.969 INFO:tasks.cephadm.mon.smithi033:Stopping mon.smithi033... 2024-05-12T04:17:13.970 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-a3e178f8-1012-11ef-bc98-c7b262605968@mon.smithi033 2024-05-12T04:17:14.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:17:14 smithi033 systemd[1]: Stopping Ceph mon.smithi033 for a3e178f8-1012-11ef-bc98-c7b262605968... 2024-05-12T04:17:14.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:17:14 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-mon-smithi033[107282]: 2024-05-12T04:17:14.248+0000 7f50e2a7a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi033 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-12T04:17:14.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:17:14 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-mon-smithi033[107282]: 2024-05-12T04:17:14.248+0000 7f50e2a7a700 -1 mon.smithi033@1(peon) e3 *** Got Signal Terminated *** 2024-05-12T04:17:15.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:May 12 04:17:14 smithi033 bash[154823]: ceph-a3e178f8-1012-11ef-bc98-c7b262605968-mon-smithi033 2024-05-12T04:17:15.293 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a3e178f8-1012-11ef-bc98-c7b262605968@mon.smithi033.service' 2024-05-12T04:17:15.340 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-12T04:17:15.340 INFO:tasks.cephadm.mon.smithi033:Stopped mon.smithi033 2024-05-12T04:17:15.341 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-12T04:17:15.341 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ceph-a3e178f8-1012-11ef-bc98-c7b262605968@osd.0 2024-05-12T04:17:15.645 INFO:journalctl@ceph.osd.0.smithi032.stdout:May 12 04:17:15 smithi032 systemd[1]: Stopping Ceph osd.0 for a3e178f8-1012-11ef-bc98-c7b262605968... 2024-05-12T04:17:16.040 INFO:journalctl@ceph.osd.0.smithi032.stdout:May 12 04:17:15 smithi032 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-0[131307]: 2024-05-12T04:17:15.644+0000 7f55a3d6f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-12T04:17:16.040 INFO:journalctl@ceph.osd.0.smithi032.stdout:May 12 04:17:15 smithi032 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-0[131307]: 2024-05-12T04:17:15.644+0000 7f55a3d6f700 -1 osd.0 84 *** Got signal Terminated *** 2024-05-12T04:17:16.040 INFO:journalctl@ceph.osd.0.smithi032.stdout:May 12 04:17:15 smithi032 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-0[131307]: 2024-05-12T04:17:15.644+0000 7f55a3d6f700 -1 osd.0 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-12T04:17:21.790 INFO:journalctl@ceph.osd.0.smithi032.stdout:May 12 04:17:21 smithi032 bash[191513]: ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-0 2024-05-12T04:17:23.451 DEBUG:teuthology.orchestra.run.smithi032:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a3e178f8-1012-11ef-bc98-c7b262605968@osd.0.service' 2024-05-12T04:17:23.499 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-12T04:17:23.499 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-12T04:17:23.499 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-12T04:17:23.500 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ceph-a3e178f8-1012-11ef-bc98-c7b262605968@osd.1 2024-05-12T04:17:23.790 INFO:journalctl@ceph.osd.1.smithi032.stdout:May 12 04:17:23 smithi032 systemd[1]: Stopping Ceph osd.1 for a3e178f8-1012-11ef-bc98-c7b262605968... 2024-05-12T04:17:24.290 INFO:journalctl@ceph.osd.1.smithi032.stdout:May 12 04:17:23 smithi032 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-1[138560]: 2024-05-12T04:17:23.808+0000 7feefff67700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-12T04:17:24.290 INFO:journalctl@ceph.osd.1.smithi032.stdout:May 12 04:17:23 smithi032 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-1[138560]: 2024-05-12T04:17:23.808+0000 7feefff67700 -1 osd.1 84 *** Got signal Terminated *** 2024-05-12T04:17:24.290 INFO:journalctl@ceph.osd.1.smithi032.stdout:May 12 04:17:23 smithi032 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-1[138560]: 2024-05-12T04:17:23.808+0000 7feefff67700 -1 osd.1 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-12T04:17:29.790 INFO:journalctl@ceph.osd.1.smithi032.stdout:May 12 04:17:29 smithi032 bash[191780]: ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-1 2024-05-12T04:17:31.371 DEBUG:teuthology.orchestra.run.smithi032:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a3e178f8-1012-11ef-bc98-c7b262605968@osd.1.service' 2024-05-12T04:17:31.415 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-12T04:17:31.415 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-12T04:17:31.416 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-12T04:17:31.416 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ceph-a3e178f8-1012-11ef-bc98-c7b262605968@osd.2 2024-05-12T04:17:31.728 INFO:journalctl@ceph.osd.2.smithi032.stdout:May 12 04:17:31 smithi032 systemd[1]: Stopping Ceph osd.2 for a3e178f8-1012-11ef-bc98-c7b262605968... 2024-05-12T04:17:32.041 INFO:journalctl@ceph.osd.2.smithi032.stdout:May 12 04:17:31 smithi032 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-2[145975]: 2024-05-12T04:17:31.727+0000 7f9fda6d1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-12T04:17:32.041 INFO:journalctl@ceph.osd.2.smithi032.stdout:May 12 04:17:31 smithi032 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-2[145975]: 2024-05-12T04:17:31.727+0000 7f9fda6d1700 -1 osd.2 84 *** Got signal Terminated *** 2024-05-12T04:17:32.041 INFO:journalctl@ceph.osd.2.smithi032.stdout:May 12 04:17:31 smithi032 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-2[145975]: 2024-05-12T04:17:31.727+0000 7f9fda6d1700 -1 osd.2 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-12T04:17:37.790 INFO:journalctl@ceph.osd.2.smithi032.stdout:May 12 04:17:37 smithi032 bash[192050]: ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-2 2024-05-12T04:17:39.365 DEBUG:teuthology.orchestra.run.smithi032:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a3e178f8-1012-11ef-bc98-c7b262605968@osd.2.service' 2024-05-12T04:17:39.414 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-12T04:17:39.414 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-12T04:17:39.414 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-12T04:17:39.414 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-a3e178f8-1012-11ef-bc98-c7b262605968@osd.3 2024-05-12T04:17:39.717 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 12 04:17:39 smithi033 systemd[1]: Stopping Ceph osd.3 for a3e178f8-1012-11ef-bc98-c7b262605968... 2024-05-12T04:17:40.161 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 12 04:17:39 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-3[120224]: 2024-05-12T04:17:39.716+0000 7fd29d1d9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-12T04:17:40.162 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 12 04:17:39 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-3[120224]: 2024-05-12T04:17:39.716+0000 7fd29d1d9700 -1 osd.3 84 *** Got signal Terminated *** 2024-05-12T04:17:40.162 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 12 04:17:39 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-3[120224]: 2024-05-12T04:17:39.716+0000 7fd29d1d9700 -1 osd.3 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-12T04:17:42.161 INFO:journalctl@ceph.osd.4.smithi033.stdout:May 12 04:17:41 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-4[126351]: 2024-05-12T04:17:41.661+0000 7fbd00c1c700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-12T04:17:15.903007+0000 front 2024-05-12T04:17:15.903072+0000 (oldest deadline 2024-05-12T04:17:41.202888+0000) 2024-05-12T04:17:42.911 INFO:journalctl@ceph.osd.4.smithi033.stdout:May 12 04:17:42 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-4[126351]: 2024-05-12T04:17:42.644+0000 7fbd00c1c700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-12T04:17:15.903007+0000 front 2024-05-12T04:17:15.903072+0000 (oldest deadline 2024-05-12T04:17:41.202888+0000) 2024-05-12T04:17:43.661 INFO:journalctl@ceph.osd.5.smithi033.stdout:May 12 04:17:43 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-5[132613]: 2024-05-12T04:17:43.173+0000 7f72734e6700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-12T04:17:19.621559+0000 front 2024-05-12T04:17:19.621470+0000 (oldest deadline 2024-05-12T04:17:43.121591+0000) 2024-05-12T04:17:44.162 INFO:journalctl@ceph.osd.4.smithi033.stdout:May 12 04:17:43 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-4[126351]: 2024-05-12T04:17:43.682+0000 7fbd00c1c700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-12T04:17:15.903007+0000 front 2024-05-12T04:17:15.903072+0000 (oldest deadline 2024-05-12T04:17:41.202888+0000) 2024-05-12T04:17:44.661 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 12 04:17:44 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-3[120224]: 2024-05-12T04:17:44.359+0000 7fd299fd0700 -1 osd.3 84 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-12T04:17:14.833821+0000 front 2024-05-12T04:17:14.833851+0000 (oldest deadline 2024-05-12T04:17:44.234175+0000) 2024-05-12T04:17:44.662 INFO:journalctl@ceph.osd.5.smithi033.stdout:May 12 04:17:44 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-5[132613]: 2024-05-12T04:17:44.196+0000 7f72734e6700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-12T04:17:19.621559+0000 front 2024-05-12T04:17:19.621470+0000 (oldest deadline 2024-05-12T04:17:43.121591+0000) 2024-05-12T04:17:45.162 INFO:journalctl@ceph.osd.4.smithi033.stdout:May 12 04:17:44 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-4[126351]: 2024-05-12T04:17:44.706+0000 7fbd00c1c700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-12T04:17:15.903007+0000 front 2024-05-12T04:17:15.903072+0000 (oldest deadline 2024-05-12T04:17:41.202888+0000) 2024-05-12T04:17:45.635 INFO:journalctl@ceph.osd.5.smithi033.stdout:May 12 04:17:45 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-5[132613]: 2024-05-12T04:17:45.224+0000 7f72734e6700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-12T04:17:19.621559+0000 front 2024-05-12T04:17:19.621470+0000 (oldest deadline 2024-05-12T04:17:43.121591+0000) 2024-05-12T04:17:45.912 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 12 04:17:45 smithi033 bash[154972]: ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-3 2024-05-12T04:17:45.912 INFO:journalctl@ceph.osd.4.smithi033.stdout:May 12 04:17:45 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-4[126351]: 2024-05-12T04:17:45.700+0000 7fbd00c1c700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-12T04:17:15.903007+0000 front 2024-05-12T04:17:15.903072+0000 (oldest deadline 2024-05-12T04:17:41.202888+0000) 2024-05-12T04:17:46.662 INFO:journalctl@ceph.osd.5.smithi033.stdout:May 12 04:17:46 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-5[132613]: 2024-05-12T04:17:46.272+0000 7f72734e6700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-12T04:17:19.621559+0000 front 2024-05-12T04:17:19.621470+0000 (oldest deadline 2024-05-12T04:17:43.121591+0000) 2024-05-12T04:17:47.162 INFO:journalctl@ceph.osd.4.smithi033.stdout:May 12 04:17:46 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-4[126351]: 2024-05-12T04:17:46.689+0000 7fbd00c1c700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-12T04:17:15.903007+0000 front 2024-05-12T04:17:15.903072+0000 (oldest deadline 2024-05-12T04:17:41.202888+0000) 2024-05-12T04:17:47.528 INFO:journalctl@ceph.osd.5.smithi033.stdout:May 12 04:17:47 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-5[132613]: 2024-05-12T04:17:47.311+0000 7f72734e6700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-12T04:17:19.621559+0000 front 2024-05-12T04:17:19.621470+0000 (oldest deadline 2024-05-12T04:17:43.121591+0000) 2024-05-12T04:17:47.538 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a3e178f8-1012-11ef-bc98-c7b262605968@osd.3.service' 2024-05-12T04:17:47.586 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-12T04:17:47.587 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-12T04:17:47.587 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-12T04:17:47.587 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-a3e178f8-1012-11ef-bc98-c7b262605968@osd.4 2024-05-12T04:17:47.860 INFO:journalctl@ceph.osd.4.smithi033.stdout:May 12 04:17:47 smithi033 systemd[1]: Stopping Ceph osd.4 for a3e178f8-1012-11ef-bc98-c7b262605968... 2024-05-12T04:17:47.860 INFO:journalctl@ceph.osd.4.smithi033.stdout:May 12 04:17:47 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-4[126351]: 2024-05-12T04:17:47.673+0000 7fbd00c1c700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-12T04:17:15.903007+0000 front 2024-05-12T04:17:15.903072+0000 (oldest deadline 2024-05-12T04:17:41.202888+0000) 2024-05-12T04:17:48.162 INFO:journalctl@ceph.osd.4.smithi033.stdout:May 12 04:17:47 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-4[126351]: 2024-05-12T04:17:47.859+0000 7fbd03e25700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-12T04:17:48.162 INFO:journalctl@ceph.osd.4.smithi033.stdout:May 12 04:17:47 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-4[126351]: 2024-05-12T04:17:47.859+0000 7fbd03e25700 -1 osd.4 84 *** Got signal Terminated *** 2024-05-12T04:17:48.162 INFO:journalctl@ceph.osd.4.smithi033.stdout:May 12 04:17:47 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-4[126351]: 2024-05-12T04:17:47.859+0000 7fbd03e25700 -1 osd.4 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-12T04:17:48.661 INFO:journalctl@ceph.osd.5.smithi033.stdout:May 12 04:17:48 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-5[132613]: 2024-05-12T04:17:48.357+0000 7f72734e6700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-12T04:17:19.621559+0000 front 2024-05-12T04:17:19.621470+0000 (oldest deadline 2024-05-12T04:17:43.121591+0000) 2024-05-12T04:17:49.162 INFO:journalctl@ceph.osd.4.smithi033.stdout:May 12 04:17:48 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-4[126351]: 2024-05-12T04:17:48.666+0000 7fbd00c1c700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-12T04:17:15.903007+0000 front 2024-05-12T04:17:15.903072+0000 (oldest deadline 2024-05-12T04:17:41.202888+0000) 2024-05-12T04:17:49.626 INFO:journalctl@ceph.osd.5.smithi033.stdout:May 12 04:17:49 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-5[132613]: 2024-05-12T04:17:49.327+0000 7f72734e6700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-12T04:17:19.621559+0000 front 2024-05-12T04:17:19.621470+0000 (oldest deadline 2024-05-12T04:17:43.121591+0000) 2024-05-12T04:17:49.911 INFO:journalctl@ceph.osd.4.smithi033.stdout:May 12 04:17:49 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-4[126351]: 2024-05-12T04:17:49.626+0000 7fbd00c1c700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-12T04:17:15.903007+0000 front 2024-05-12T04:17:15.903072+0000 (oldest deadline 2024-05-12T04:17:41.202888+0000) 2024-05-12T04:17:50.637 INFO:journalctl@ceph.osd.5.smithi033.stdout:May 12 04:17:50 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-5[132613]: 2024-05-12T04:17:50.285+0000 7f72734e6700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-12T04:17:19.621559+0000 front 2024-05-12T04:17:19.621470+0000 (oldest deadline 2024-05-12T04:17:43.121591+0000) 2024-05-12T04:17:50.638 INFO:journalctl@ceph.osd.5.smithi033.stdout:May 12 04:17:50 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-5[132613]: 2024-05-12T04:17:50.285+0000 7f72734e6700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.32:6814 osd.1 since back 2024-05-12T04:17:27.822768+0000 front 2024-05-12T04:17:27.822796+0000 (oldest deadline 2024-05-12T04:17:49.423116+0000) 2024-05-12T04:17:50.912 INFO:journalctl@ceph.osd.4.smithi033.stdout:May 12 04:17:50 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-4[126351]: 2024-05-12T04:17:50.637+0000 7fbd00c1c700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-12T04:17:15.903007+0000 front 2024-05-12T04:17:15.903072+0000 (oldest deadline 2024-05-12T04:17:41.202888+0000) 2024-05-12T04:17:51.656 INFO:journalctl@ceph.osd.5.smithi033.stdout:May 12 04:17:51 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-5[132613]: 2024-05-12T04:17:51.302+0000 7f72734e6700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-12T04:17:19.621559+0000 front 2024-05-12T04:17:19.621470+0000 (oldest deadline 2024-05-12T04:17:43.121591+0000) 2024-05-12T04:17:51.656 INFO:journalctl@ceph.osd.5.smithi033.stdout:May 12 04:17:51 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-5[132613]: 2024-05-12T04:17:51.302+0000 7f72734e6700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.32:6814 osd.1 since back 2024-05-12T04:17:27.822768+0000 front 2024-05-12T04:17:27.822796+0000 (oldest deadline 2024-05-12T04:17:49.423116+0000) 2024-05-12T04:17:51.911 INFO:journalctl@ceph.osd.4.smithi033.stdout:May 12 04:17:51 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-4[126351]: 2024-05-12T04:17:51.655+0000 7fbd00c1c700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-12T04:17:15.903007+0000 front 2024-05-12T04:17:15.903072+0000 (oldest deadline 2024-05-12T04:17:41.202888+0000) 2024-05-12T04:17:51.912 INFO:journalctl@ceph.osd.4.smithi033.stdout:May 12 04:17:51 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-4[126351]: 2024-05-12T04:17:51.655+0000 7fbd00c1c700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.32:6814 osd.1 since back 2024-05-12T04:17:27.604006+0000 front 2024-05-12T04:17:27.604096+0000 (oldest deadline 2024-05-12T04:17:51.103946+0000) 2024-05-12T04:17:52.607 INFO:journalctl@ceph.osd.5.smithi033.stdout:May 12 04:17:52 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-5[132613]: 2024-05-12T04:17:52.289+0000 7f72734e6700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-12T04:17:19.621559+0000 front 2024-05-12T04:17:19.621470+0000 (oldest deadline 2024-05-12T04:17:43.121591+0000) 2024-05-12T04:17:52.607 INFO:journalctl@ceph.osd.5.smithi033.stdout:May 12 04:17:52 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-5[132613]: 2024-05-12T04:17:52.289+0000 7f72734e6700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.32:6814 osd.1 since back 2024-05-12T04:17:27.822768+0000 front 2024-05-12T04:17:27.822796+0000 (oldest deadline 2024-05-12T04:17:49.423116+0000) 2024-05-12T04:17:52.912 INFO:journalctl@ceph.osd.4.smithi033.stdout:May 12 04:17:52 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-4[126351]: 2024-05-12T04:17:52.607+0000 7fbd00c1c700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-12T04:17:15.903007+0000 front 2024-05-12T04:17:15.903072+0000 (oldest deadline 2024-05-12T04:17:41.202888+0000) 2024-05-12T04:17:52.912 INFO:journalctl@ceph.osd.4.smithi033.stdout:May 12 04:17:52 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-4[126351]: 2024-05-12T04:17:52.607+0000 7fbd00c1c700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.32:6814 osd.1 since back 2024-05-12T04:17:27.604006+0000 front 2024-05-12T04:17:27.604096+0000 (oldest deadline 2024-05-12T04:17:51.103946+0000) 2024-05-12T04:17:53.521 INFO:journalctl@ceph.osd.5.smithi033.stdout:May 12 04:17:53 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-5[132613]: 2024-05-12T04:17:53.244+0000 7f72734e6700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-12T04:17:19.621559+0000 front 2024-05-12T04:17:19.621470+0000 (oldest deadline 2024-05-12T04:17:43.121591+0000) 2024-05-12T04:17:53.521 INFO:journalctl@ceph.osd.5.smithi033.stdout:May 12 04:17:53 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-5[132613]: 2024-05-12T04:17:53.244+0000 7f72734e6700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.32:6814 osd.1 since back 2024-05-12T04:17:27.822768+0000 front 2024-05-12T04:17:27.822796+0000 (oldest deadline 2024-05-12T04:17:49.423116+0000) 2024-05-12T04:17:53.911 INFO:journalctl@ceph.osd.4.smithi033.stdout:May 12 04:17:53 smithi033 bash[155251]: ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-4 2024-05-12T04:17:54.662 INFO:journalctl@ceph.osd.5.smithi033.stdout:May 12 04:17:54 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-5[132613]: 2024-05-12T04:17:54.270+0000 7f72734e6700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-12T04:17:19.621559+0000 front 2024-05-12T04:17:19.621470+0000 (oldest deadline 2024-05-12T04:17:43.121591+0000) 2024-05-12T04:17:54.662 INFO:journalctl@ceph.osd.5.smithi033.stdout:May 12 04:17:54 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-5[132613]: 2024-05-12T04:17:54.270+0000 7f72734e6700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.32:6814 osd.1 since back 2024-05-12T04:17:27.822768+0000 front 2024-05-12T04:17:27.822796+0000 (oldest deadline 2024-05-12T04:17:49.423116+0000) 2024-05-12T04:17:55.328 INFO:journalctl@ceph.osd.5.smithi033.stdout:May 12 04:17:55 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-5[132613]: 2024-05-12T04:17:55.274+0000 7f72734e6700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-12T04:17:19.621559+0000 front 2024-05-12T04:17:19.621470+0000 (oldest deadline 2024-05-12T04:17:43.121591+0000) 2024-05-12T04:17:55.329 INFO:journalctl@ceph.osd.5.smithi033.stdout:May 12 04:17:55 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-5[132613]: 2024-05-12T04:17:55.274+0000 7f72734e6700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.32:6814 osd.1 since back 2024-05-12T04:17:27.822768+0000 front 2024-05-12T04:17:27.822796+0000 (oldest deadline 2024-05-12T04:17:49.423116+0000) 2024-05-12T04:17:55.337 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a3e178f8-1012-11ef-bc98-c7b262605968@osd.4.service' 2024-05-12T04:17:55.379 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-12T04:17:55.380 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-12T04:17:55.380 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-12T04:17:55.380 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-a3e178f8-1012-11ef-bc98-c7b262605968@osd.5 2024-05-12T04:17:55.661 INFO:journalctl@ceph.osd.5.smithi033.stdout:May 12 04:17:55 smithi033 systemd[1]: Stopping Ceph osd.5 for a3e178f8-1012-11ef-bc98-c7b262605968... 2024-05-12T04:17:56.162 INFO:journalctl@ceph.osd.5.smithi033.stdout:May 12 04:17:55 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-5[132613]: 2024-05-12T04:17:55.677+0000 7f727a6f7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-12T04:17:56.162 INFO:journalctl@ceph.osd.5.smithi033.stdout:May 12 04:17:55 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-5[132613]: 2024-05-12T04:17:55.677+0000 7f727a6f7700 -1 osd.5 84 *** Got signal Terminated *** 2024-05-12T04:17:56.162 INFO:journalctl@ceph.osd.5.smithi033.stdout:May 12 04:17:55 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-5[132613]: 2024-05-12T04:17:55.677+0000 7f727a6f7700 -1 osd.5 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-12T04:17:56.662 INFO:journalctl@ceph.osd.5.smithi033.stdout:May 12 04:17:56 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-5[132613]: 2024-05-12T04:17:56.234+0000 7f72734e6700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-12T04:17:19.621559+0000 front 2024-05-12T04:17:19.621470+0000 (oldest deadline 2024-05-12T04:17:43.121591+0000) 2024-05-12T04:17:56.663 INFO:journalctl@ceph.osd.5.smithi033.stdout:May 12 04:17:56 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-5[132613]: 2024-05-12T04:17:56.234+0000 7f72734e6700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.32:6814 osd.1 since back 2024-05-12T04:17:27.822768+0000 front 2024-05-12T04:17:27.822796+0000 (oldest deadline 2024-05-12T04:17:49.423116+0000) 2024-05-12T04:17:57.661 INFO:journalctl@ceph.osd.5.smithi033.stdout:May 12 04:17:57 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-5[132613]: 2024-05-12T04:17:57.188+0000 7f72734e6700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-12T04:17:19.621559+0000 front 2024-05-12T04:17:19.621470+0000 (oldest deadline 2024-05-12T04:17:43.121591+0000) 2024-05-12T04:17:57.662 INFO:journalctl@ceph.osd.5.smithi033.stdout:May 12 04:17:57 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-5[132613]: 2024-05-12T04:17:57.188+0000 7f72734e6700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.32:6814 osd.1 since back 2024-05-12T04:17:27.822768+0000 front 2024-05-12T04:17:27.822796+0000 (oldest deadline 2024-05-12T04:17:49.423116+0000) 2024-05-12T04:17:58.661 INFO:journalctl@ceph.osd.5.smithi033.stdout:May 12 04:17:58 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-5[132613]: 2024-05-12T04:17:58.217+0000 7f72734e6700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-12T04:17:19.621559+0000 front 2024-05-12T04:17:19.621470+0000 (oldest deadline 2024-05-12T04:17:43.121591+0000) 2024-05-12T04:17:58.662 INFO:journalctl@ceph.osd.5.smithi033.stdout:May 12 04:17:58 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-5[132613]: 2024-05-12T04:17:58.217+0000 7f72734e6700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.32:6814 osd.1 since back 2024-05-12T04:17:27.822768+0000 front 2024-05-12T04:17:27.822796+0000 (oldest deadline 2024-05-12T04:17:49.423116+0000) 2024-05-12T04:17:59.662 INFO:journalctl@ceph.osd.5.smithi033.stdout:May 12 04:17:59 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-5[132613]: 2024-05-12T04:17:59.176+0000 7f72734e6700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-12T04:17:19.621559+0000 front 2024-05-12T04:17:19.621470+0000 (oldest deadline 2024-05-12T04:17:43.121591+0000) 2024-05-12T04:17:59.662 INFO:journalctl@ceph.osd.5.smithi033.stdout:May 12 04:17:59 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-5[132613]: 2024-05-12T04:17:59.176+0000 7f72734e6700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.32:6814 osd.1 since back 2024-05-12T04:17:27.822768+0000 front 2024-05-12T04:17:27.822796+0000 (oldest deadline 2024-05-12T04:17:49.423116+0000) 2024-05-12T04:17:59.662 INFO:journalctl@ceph.osd.5.smithi033.stdout:May 12 04:17:59 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-5[132613]: 2024-05-12T04:17:59.176+0000 7f72734e6700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.32:6822 osd.2 since back 2024-05-12T04:17:35.224117+0000 front 2024-05-12T04:17:35.224203+0000 (oldest deadline 2024-05-12T04:17:58.724122+0000) 2024-05-12T04:18:00.662 INFO:journalctl@ceph.osd.5.smithi033.stdout:May 12 04:18:00 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-5[132613]: 2024-05-12T04:18:00.216+0000 7f72734e6700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-12T04:17:19.621559+0000 front 2024-05-12T04:17:19.621470+0000 (oldest deadline 2024-05-12T04:17:43.121591+0000) 2024-05-12T04:18:00.662 INFO:journalctl@ceph.osd.5.smithi033.stdout:May 12 04:18:00 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-5[132613]: 2024-05-12T04:18:00.216+0000 7f72734e6700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.32:6814 osd.1 since back 2024-05-12T04:17:27.822768+0000 front 2024-05-12T04:17:27.822796+0000 (oldest deadline 2024-05-12T04:17:49.423116+0000) 2024-05-12T04:18:00.662 INFO:journalctl@ceph.osd.5.smithi033.stdout:May 12 04:18:00 smithi033 ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-5[132613]: 2024-05-12T04:18:00.216+0000 7f72734e6700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.32:6822 osd.2 since back 2024-05-12T04:17:35.224117+0000 front 2024-05-12T04:17:35.224203+0000 (oldest deadline 2024-05-12T04:17:58.724122+0000) 2024-05-12T04:18:01.662 INFO:journalctl@ceph.osd.5.smithi033.stdout:May 12 04:18:01 smithi033 bash[155516]: ceph-a3e178f8-1012-11ef-bc98-c7b262605968-osd-5 2024-05-12T04:18:03.098 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a3e178f8-1012-11ef-bc98-c7b262605968@osd.5.service' 2024-05-12T04:18:03.127 INFO:journalctl@ceph.osd.5.smithi033.stdout:May 12 04:18:03 smithi033 systemd[1]: ceph-a3e178f8-1012-11ef-bc98-c7b262605968@osd.5.service: Succeeded. 2024-05-12T04:18:03.127 INFO:journalctl@ceph.osd.5.smithi033.stdout:May 12 04:18:03 smithi033 systemd[1]: Stopped Ceph osd.5 for a3e178f8-1012-11ef-bc98-c7b262605968. 2024-05-12T04:18:03.142 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-12T04:18:03.142 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-12T04:18:03.142 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a3e178f8-1012-11ef-bc98-c7b262605968 --force --keep-logs 2024-05-12T04:18:05.896 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi032.stderr:ceph-fuse[100208]: fuse finished with error 0 and tester_r 0 2024-05-12T04:18:23.145 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a3e178f8-1012-11ef-bc98-c7b262605968 --force --keep-logs 2024-05-12T04:18:35.006 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-12T04:18:35.045 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-12T04:18:35.077 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-12T04:18:35.079 DEBUG:teuthology.misc:Transferring archived files from smithi032:/var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-05-01_22:24:03-fs-reef-distro-default-smithi/7685017/remote/smithi032/crash 2024-05-12T04:18:35.080 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar c -f - -C /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/crash -- . 2024-05-12T04:18:35.115 INFO:teuthology.orchestra.run.smithi032.stderr:tar: /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/crash: Cannot open: No such file or directory 2024-05-12T04:18:35.116 INFO:teuthology.orchestra.run.smithi032.stderr:tar: Error is not recoverable: exiting now 2024-05-12T04:18:35.117 DEBUG:teuthology.misc:Transferring archived files from smithi033:/var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-05-01_22:24:03-fs-reef-distro-default-smithi/7685017/remote/smithi033/crash 2024-05-12T04:18:35.118 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar c -f - -C /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/crash -- . 2024-05-12T04:18:35.145 INFO:teuthology.orchestra.run.smithi033.stderr:tar: /var/lib/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/crash: Cannot open: No such file or directory 2024-05-12T04:18:35.145 INFO:teuthology.orchestra.run.smithi033.stderr:tar: Error is not recoverable: exiting now 2024-05-12T04:18:35.146 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-12T04:18:35.146 DEBUG:teuthology.orchestra.run.smithi032:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-05-12T04:18:35.196 INFO:teuthology.orchestra.run.smithi032.stderr:grep: /var/log/ceph/a3e178f8-1012-11ef-bc98-c7b262605968/ceph.log: No such file or directory 2024-05-12T04:18:35.196 INFO:tasks.cephadm:Compressing logs... 2024-05-12T04:18:35.196 DEBUG:teuthology.orchestra.run.smithi032:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-12T04:18:35.239 DEBUG:teuthology.orchestra.run.smithi033:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-12T04:18:35.265 INFO:teuthology.orchestra.run.smithi032.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-12T04:18:35.266 INFO:teuthology.orchestra.run.smithi033.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-12T04:18:53.252 INFO:teuthology.orchestra.run.smithi033.stderr:gzip: /var/log/ceph/ceph-client.1.log: file size changed while zipping 2024-05-12T04:21:16.388 INFO:tasks.cephadm:Archiving logs... 2024-05-12T04:21:16.389 DEBUG:teuthology.misc:Transferring archived files from smithi032:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-05-01_22:24:03-fs-reef-distro-default-smithi/7685017/remote/smithi032/log 2024-05-12T04:21:16.390 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-12T04:21:34.745 DEBUG:teuthology.misc:Transferring archived files from smithi033:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-05-01_22:24:03-fs-reef-distro-default-smithi/7685017/remote/smithi033/log 2024-05-12T04:21:34.746 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-12T04:21:40.173 INFO:tasks.cephadm:Removing cluster... 2024-05-12T04:21:40.174 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a3e178f8-1012-11ef-bc98-c7b262605968 --force 2024-05-12T04:21:40.779 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a3e178f8-1012-11ef-bc98-c7b262605968 --force 2024-05-12T04:21:41.279 INFO:tasks.cephadm:Removing cephadm ... 2024-05-12T04:21:41.280 DEBUG:teuthology.orchestra.run.smithi032:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-12T04:21:41.300 DEBUG:teuthology.orchestra.run.smithi033:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-12T04:21:41.320 INFO:tasks.cephadm:Teardown complete 2024-05-12T04:21:41.320 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-12T04:21:41.337 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_fccfb453906af8847786c471919d347112ea0360/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_fccfb453906af8847786c471919d347112ea0360/qa/tasks/cephfs/fuse_mount.py", line 400, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-05-12T04:21:41.338 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-12T04:21:41.338 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-12T04:21:41.342 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-12T04:21:41.439 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-12T04:21:41.454 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-12T04:21:41.454 DEBUG:teuthology.orchestra.run.smithi032:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-12T04:21:41.456 DEBUG:teuthology.orchestra.run.smithi033:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-12T04:21:41.473 INFO:teuthology.orchestra.run.smithi032.stderr:bash: ntpq: command not found 2024-05-12T04:21:41.474 INFO:teuthology.orchestra.run.smithi033.stderr:bash: ntpq: command not found 2024-05-12T04:21:41.478 INFO:teuthology.orchestra.run.smithi032.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-12T04:21:41.478 INFO:teuthology.orchestra.run.smithi032.stdout:=============================================================================== 2024-05-12T04:21:41.478 INFO:teuthology.orchestra.run.smithi032.stdout:^* hv01.front.sepia.ceph.com 3 6 377 8 +918us[ +905us] +/- 56ms 2024-05-12T04:21:41.478 INFO:teuthology.orchestra.run.smithi032.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 9 -825us[ -839us] +/- 80ms 2024-05-12T04:21:41.478 INFO:teuthology.orchestra.run.smithi032.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 8 -460us[ -460us] +/- 66ms 2024-05-12T04:21:41.478 INFO:teuthology.orchestra.run.smithi032.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-12T04:21:41.480 INFO:teuthology.orchestra.run.smithi033.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-12T04:21:41.480 INFO:teuthology.orchestra.run.smithi033.stdout:=============================================================================== 2024-05-12T04:21:41.480 INFO:teuthology.orchestra.run.smithi033.stdout:^* hv01.front.sepia.ceph.com 3 6 377 7 +915us[ +916us] +/- 56ms 2024-05-12T04:21:41.480 INFO:teuthology.orchestra.run.smithi033.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 9 -816us[ -815us] +/- 80ms 2024-05-12T04:21:41.480 INFO:teuthology.orchestra.run.smithi033.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 11 -457us[ -456us] +/- 66ms 2024-05-12T04:21:41.480 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-12T04:21:41.480 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-12T04:21:41.492 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-12T04:21:41.493 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-12T04:21:41.526 DEBUG:teuthology.orchestra.run.smithi032:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-12T04:21:41.605 DEBUG:teuthology.orchestra.run.smithi033:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-12T04:21:41.681 DEBUG:teuthology.orchestra.run.smithi032:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-12T04:21:41.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-12T04:21:41.714 DEBUG:teuthology.orchestra.run.smithi033:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-12T04:21:41.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-12T04:21:41.746 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-12T04:21:41.855 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-12T04:21:41.929 INFO:teuthology.task.internal:Duration was 2527.595239 seconds 2024-05-12T04:21:41.930 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-12T04:21:41.973 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-12T04:21:41.973 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-12T04:21:41.982 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-12T04:21:42.029 INFO:teuthology.orchestra.run.smithi032.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-12T04:21:42.031 INFO:teuthology.orchestra.run.smithi033.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-12T04:21:42.424 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-12T04:21:42.424 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi032.front.sepia.ceph.com 2024-05-12T04:21:42.425 DEBUG:teuthology.orchestra.run.smithi032:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-12T04:21:42.452 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi033.front.sepia.ceph.com 2024-05-12T04:21:42.453 DEBUG:teuthology.orchestra.run.smithi033:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-12T04:21:42.477 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-12T04:21:42.477 DEBUG:teuthology.orchestra.run.smithi032:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-12T04:21:42.495 DEBUG:teuthology.orchestra.run.smithi033:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-12T04:21:42.575 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-12T04:21:42.575 DEBUG:teuthology.orchestra.run.smithi032:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-12T04:21:42.578 DEBUG:teuthology.orchestra.run.smithi033:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-12T04:21:42.706 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-12T04:21:42.918 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-12T04:21:42.918 DEBUG:teuthology.orchestra.run.smithi032:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-12T04:21:42.947 DEBUG:teuthology.orchestra.run.smithi033:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-12T04:21:42.976 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-12T04:21:43.013 DEBUG:teuthology.orchestra.run.smithi032:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-12T04:21:43.016 DEBUG:teuthology.orchestra.run.smithi033:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-12T04:21:43.041 INFO:teuthology.orchestra.run.smithi032.stdout:kernel.core_pattern = core 2024-05-12T04:21:43.046 INFO:teuthology.orchestra.run.smithi033.stdout:kernel.core_pattern = core 2024-05-12T04:21:43.065 DEBUG:teuthology.orchestra.run.smithi032:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-12T04:21:43.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-12T04:21:43.121 DEBUG:teuthology.orchestra.run.smithi033:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-12T04:21:43.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-12T04:21:43.139 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-12T04:21:43.152 INFO:teuthology.task.internal:Transferring archived files... 2024-05-12T04:21:43.153 DEBUG:teuthology.misc:Transferring archived files from smithi032:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-05-01_22:24:03-fs-reef-distro-default-smithi/7685017/remote/smithi032 2024-05-12T04:21:43.154 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-12T04:21:43.208 DEBUG:teuthology.misc:Transferring archived files from smithi033:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-05-01_22:24:03-fs-reef-distro-default-smithi/7685017/remote/smithi033 2024-05-12T04:21:43.208 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-12T04:21:43.252 INFO:teuthology.task.internal:Removing archive directory... 2024-05-12T04:21:43.252 DEBUG:teuthology.orchestra.run.smithi032:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-12T04:21:43.254 DEBUG:teuthology.orchestra.run.smithi033:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-12T04:21:43.302 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-12T04:21:43.319 INFO:teuthology.task.internal:Not uploading archives. 2024-05-12T04:21:43.319 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-12T04:21:43.354 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-12T04:21:43.355 DEBUG:teuthology.orchestra.run.smithi032:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-12T04:21:43.357 DEBUG:teuthology.orchestra.run.smithi033:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-12T04:21:43.373 INFO:teuthology.orchestra.run.smithi032.stdout: 525357 4 drwxr-xr-x 3 ubuntu ubuntu 4096 May 12 04:21 /home/ubuntu/cephtest 2024-05-12T04:21:43.373 INFO:teuthology.orchestra.run.smithi032.stdout: 527555 4 d--------- 2 ubuntu ubuntu 4096 May 12 03:58 /home/ubuntu/cephtest/mnt.0 2024-05-12T04:21:43.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-12T04:21:43.374 INFO:teuthology.orchestra.run.smithi032.stderr:find: ‘/home/ubuntu/cephtest/mnt.0’: Permission denied 2024-05-12T04:21:43.374 INFO:teuthology.orchestra.run.smithi032.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-05-12T04:21:43.375 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_fccfb453906af8847786c471919d347112ea0360/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_fccfb453906af8847786c471919d347112ea0360/qa/tasks/cephfs/fuse_mount.py", line 400, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi032 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-05-12T04:21:43.375 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-12T04:21:43.395 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-12T04:21:43.438 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-05-12T04:21:43.438 INFO:teuthology.nuke:Checking targets against current locks 2024-05-12T04:21:43.465 DEBUG:teuthology.nuke:shortname: smithi032 2024-05-12T04:21:43.465 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-12T04:21:43.488 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi032.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-05-01_22:24:03-fs-reef-distro-default-smithi/7685017', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-12 03:31:48.055764', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM6+E1ZOfltaZUZhW75ipR9UF/xLND7F/dzURXMhLIg1efwt/pRqs9P4GPtOvzZ/Yz00BHfZ6Yqbhv+gU8oiDLY='} 2024-05-12T04:21:43.492 DEBUG:teuthology.nuke:shortname: smithi033 2024-05-12T04:21:43.492 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-12T04:21:43.509 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi033.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-05-01_22:24:03-fs-reef-distro-default-smithi/7685017', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-12 03:31:48.053853', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLXj6EiPmflYQsMVB04Ib6euDtM77z2Z1HSEetzuh7uSO7ixFy8H90H1oj6fJRGaXnj4+bn5Rt0OJ5WW6HK9RNQ='} 2024-05-12T04:21:43.539 INFO:teuthology.orchestra.console.smithi032:Power off 2024-05-12T04:21:43.539 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-12T04:21:43.562 DEBUG:teuthology.orchestra.console.smithi032:power off output: Chassis Power Control: Down/Off 2024-05-12T04:21:43.563 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-12T04:21:43.578 INFO:teuthology.orchestra.console.smithi033:Power off 2024-05-12T04:21:43.579 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-12T04:21:43.588 DEBUG:teuthology.orchestra.console.smithi032:check power output: Chassis Power is on 2024-05-12T04:21:43.604 DEBUG:teuthology.orchestra.console.smithi033:power off output: Chassis Power Control: Down/Off 2024-05-12T04:21:43.604 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-12T04:21:43.646 DEBUG:teuthology.orchestra.console.smithi033:check power output: Chassis Power is on 2024-05-12T04:21:47.589 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-12T04:21:47.647 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-12T04:21:47.700 DEBUG:teuthology.orchestra.console.smithi032:check power output: Chassis Power is on 2024-05-12T04:21:47.757 DEBUG:teuthology.orchestra.console.smithi033:check power output: Chassis Power is on 2024-05-12T04:21:51.702 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-12T04:21:51.758 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-12T04:21:51.812 DEBUG:teuthology.orchestra.console.smithi032:check power output: Chassis Power is off 2024-05-12T04:21:51.868 DEBUG:teuthology.orchestra.console.smithi033:check power output: Chassis Power is on 2024-05-12T04:21:51.913 INFO:teuthology.orchestra.console.smithi032:Power off completed 2024-05-12T04:21:55.870 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-12T04:21:55.982 DEBUG:teuthology.orchestra.console.smithi033:check power output: Chassis Power is off 2024-05-12T04:21:56.083 INFO:teuthology.orchestra.console.smithi033:Power off completed 2024-05-12T04:21:56.187 INFO:teuthology.run:Summary data: description: fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} duration: 2527.595239162445 failure_reason: reached maximum tries (51) after waiting for 300 seconds flavor: default owner: scheduled_teuthology@teuthology status: fail success: false 2024-05-12T04:21:56.187 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-12T04:21:56.284 INFO:teuthology.run:FAIL